grpc streaming example crashes or hangs due to deadline seconds

[V3SalC]

Using example, DEADLINE_SECS = 8 * 60 * 60 would cause the code to stop responding after about 60 seconds (which i think is the streaming limit) and would not gracefully exit or provide any errors (have to kill python). So to work around that, i changed it to DEADLINE_SECS = 60 which has been working up until today where now there is an error like below after about 20 seconds:
error {
code: 3
message: "Client GRPC deadline too short. Should be at least: 3 * audio-duration + 5 seconds. Current deadline is: 59.8 s. Required at least: 59.9 s"
}

[jerjou]

So the general idea is that the usage limits specify how much audio it will transcribe, but the server needs some more time than that to actually perform the transcription (including retries, if necessary).

You could think of it as - the DEADLINE_SECS isn't really a measure of the length of audio you want transcribed. It's more low-level than that - it's just how long to keep the connection open to the server (regardless of which API you're using). And if you close the connection to the server before the server has a chance to respond with the transcription, that isn't great either.

That said, I agree this isn't ideal. Perhaps the Right Way to do it is to measure the audio time on the client, cut it off at 60 seconds, and wait for the transcription result with is_final==True... what do you think?

[V3SalC]

Hi jerjou. Thanks for the response. I agree that would be a better approach. Would that fix the issue where it is closing the connection prematurely, because given 60 seconds, it should not give the error it did 20 seconds later. Could this also be an issue with the service?

[bprasana]

Do we know if this issue is getting fixed anytime soon with solution @jerjou mentioned above. It would help a lot.

[jerjou]

Happy to review any pull requests y'all might want to send our way ;-)
Otherwise - this is sample code! :-) Y'all should feel free to change it to suit your needs.
That said, I do agree that it's probably a general enough issue to be included in the canonical sample, though, so it's on my todo list.. I will update this bug if I get to it before I see a pull request come by.

Oh, and to answer your question, @V3SalC - if you increase your DEADLINE_SECS to be 60*3 + 5, it should listen for the full 60 seconds (and then return that error if you keep sending data after that). What's happening is - the server is verifying that your client will stick around long enough to get the transcription response (including time for retries - thus the multiplier). You should think of DEADLINE_SECS less as "how much audio I'm going to send", and more as "how long to keep the connection open, waiting for a response".

[jerjou]

Okay - I have a draft (or two) of a solution to this. Check out the minute_workaround branch. It has two additional files:

• transcribe_streaming_minute.py which listens normally until WRAP_IT_UP_SECS seconds before the minute mark, then starts looking for the END_OF_SPEECH marker. Once it hits it, it looks for a is_final transcription, prints it, and starts a new stream.
• transcribe_streaming_utterance.py - after implementing the above, I realized that I was basically just looking for utterances, so I dashed this version out quickly. I think there's still some bugs, but it should be simpler than the above..

LMKWYT. There's still the problem where if you're speaking while the new stream is being constructed, it might miss some words, but it's a start at least.

[jerjou]

Updated to minimize the amount of audio lost between streams. It seems to work pretty seamlessly for me, though I have trouble coming up with things to say to fill in the silence -_-; Maybe I should read from a novel or something for my testing...

Anyway, any feedback would be appreciated. I'll probably leave the branch to simmer over the new year, and if I don't hear any opinions, I'll pick one of the versions and merge it into master alongside the existing sample.

@jonparrott @puneith FYI

[jerjou]

Added a ring buffer to capture some overlapping audio between streams, just for funsies. ;-) Should be fairly robust now.

[burnzzz]

Where did the ring-buffer solution end up? Cannot find the mentioned branch. Is it merged in the master branch?

[jerjou]

It seems like it got lost in the transition from the grpc auto-generated clients, to the nicer google-cloud-python client lib.

I just wrote a New and Improved (!!) version here. I had it subclass from the existing transcribe_streaming_mic.py in hopes of making it more obvious how we're working around the 1-minute limit.

Also, for No Extra Cost (!!), I added a class to simulate streaming from a wav/raw file. Seems a bunch of folks are already doing it on their own, so figured I'd add it to the sample.

PTAL, and let me know if folks have Opinons about it :-) I'll open a PR for it once I get around to writing tests...