Deepgram Latency Reduction - Javascript/Typescript Developer
Budget: ₹1,500 – ₹12,500 INR
This repo contains everything needed to create a simple calling application that uses gpt to send voice replies. https://github.com/twilio-labs/call-gpt
A simple first project, that shouldn't take too long is reducing the latency on the calls, bettering the silence detection when calling the agent. This should enable the user to interrupt the agent when the agent is talking, and the agent should stop talking and re-respond. Right now, if you set this repo up, at certain times it will wait too long even after you stop talking before it will process the speech that you said because it hears background noise on the phone.
Deepgram has the capability to fix both of these issues, this might just be tweaking with the parameters in recording-service.js, stream-service.js, and transcription-service.js until the latency issues are fixed. Deepgram enables endpointing (docs here: https://developers.deepgram.com/docs/endpointing) and advanced end-of-speech detecting https://developers.deepgram.com/docs/understanding-end-of-speech-detection
If this goes well and you're able to solve these two issues and make a snappier version of this agent in this repo, we will have a lot more projects related to this, that will lead to longer term work if you're interested.
Acceptance Criteria
An upgraded version of the repo linked above, with better silence detection specified above: 1) allows the user to interrupt the agent where the agent stops talking and starts listening again and 2) the agent doesn't wait several seconds at times before responding because it hears (very quiet) background noise
A simple first project, that shouldn't take too long is reducing the latency on the calls, bettering the silence detection when calling the agent. This should enable the user to interrupt the agent when the agent is talking, and the agent should stop talking and re-respond. Right now, if you set this repo up, at certain times it will wait too long even after you stop talking before it will process the speech that you said because it hears background noise on the phone.
Deepgram has the capability to fix both of these issues, this might just be tweaking with the parameters in recording-service.js, stream-service.js, and transcription-service.js until the latency issues are fixed. Deepgram enables endpointing (docs here: https://developers.deepgram.com/docs/endpointing) and advanced end-of-speech detecting https://developers.deepgram.com/docs/understanding-end-of-speech-detection
If this goes well and you're able to solve these two issues and make a snappier version of this agent in this repo, we will have a lot more projects related to this, that will lead to longer term work if you're interested.
Acceptance Criteria
An upgraded version of the repo linked above, with better silence detection specified above: 1) allows the user to interrupt the agent where the agent stops talking and starts listening again and 2) the agent doesn't wait several seconds at times before responding because it hears (very quiet) background noise