Description
Whenever decode() or encode() is called, we are to increase respectively decodeQueueSize or encodeQueueSize and enqueue a control message while immediately processing the control message.
If the codec is saturated, then we are to abort.
However "Codec Saturation" is defined as:
**_Codec Saturation
The state of an underlying codec implementation where the number of active decoding or encoding requests has reached an implementation specific maximum such that it is temporarily unable to accept more work. The maximum may be any value greater than 1, including infinity (no maximum). While saturated, additional calls to decode() or encode() will be buffered in the control message queue, and will increment the respective decodeQueuSize and encodeQueueSize attributes. The codec implementation will become unsaturated after making sufficient progress on the current workload._**
If a UA choose for a maximum a value of infinity; a side effect is that decodeQueueSize and encodeQueueSize will only ever be observed with a value of 0.
WebCodecs' demo player https://w3c.github.io/webcodecs/samples/audio-video-player/audio_video_player.html uses the value of decodeQueueSize as a rate limiter on how quickly it needs to decode ahead of time to fill its ring buffer for audio.
If the UA chose to have an unlimited queue and decodeQueueSize be always 0, then such rate limitation will be ineffective.
In the demo above, this causes the ring buffer to become full almost instantly and decoded audio frames are all dropped, and playback never progress.
While it's obviously an issue in this reference player itself, it will be used by others as an example (I already know of a few parties that based their work on it)
Chrome uses a maximum queue size of between 1 and 8 for video (Default: https://source.chromium.org/chromium/chromium/src/+/main:media/base/video_decoder.cc;drc=0e516eb08aa56163bfe649cebbba13da1fcf5f3c;bpv=1;bpt=1;l=30 max: https://source.chromium.org/chromium/chromium/src/+/main:media/gpu/chromeos/oop_video_decoder.h;drc=0e516eb08aa56163bfe649cebbba13da1fcf5f3c;bpv=1;bpt=1;l=207) and just 1 for audio (https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/modules/webcodecs/audio_decoder.cc;drc=0e516eb08aa56163bfe649cebbba13da1fcf5f3c;bpv=1;bpt=1;l=132)
Firefox uses a maximum queue size of 1 (https://searchfox.org/mozilla-central/source/dom/media/webcodecs/DecoderTemplate.cpp#564-567) and won't enqueue a new decode/encode if one is pending.
And so the demo video player works fine on Firefox and Chrome; but it wouldn't had they not constricted the limit to such a low value.
It may be beneficial to provide guidelines of best practices and amend the demo player to cater for such condition.