Chrome Extension for Teams Call Captions
Budget: $30 – $250 USD
Your task is to create a chrome extension for Microsoft Teams calls to extract the caption and send it to a server. Note that I have built parts of it but it is buggy, so you can use my approach as a reference (I will provide my source code). Note: Not Teams Desktop app, but Teams on Google Chrome (Mac and Windows).
The extension must:
1. Be active on the target website (teams.microsoft.com and teams.live.com)
2. Automatically launch closed captions
3. Send the closed captions via websockets to my server (API information will be provided)
4. Memory efficient, clean, robust and not affect performance.
I will now describe the parts in more detail.
Part 1 - Be active on target website
- Automatically detect when in meeting state, waiting for people to join, or people joined
- Right now my approach seems to work, could be improved and checked for performance
Part 2 - Automatically launch closed captions
- I would like you to do it via direct injection, but so far whatever i had tried results in CSP errors. I know this is possible because another plugin has successfully done it, via rtc injection. I can share their source code if it is helpful.
- Another approach is via keyboard shortcuts, I am currently using this but i want this to be a backup option. the main approach must be the direct toggle injection, which is way more reliable.
Implement a check to ensure that caption box is enabled and present, if not retry to open.
Part 3 - Send the closed captions via websockets
- For the purpose of this project, i have set up 2 basic test websites.
Teams Simulator - https://teamsview.netlify.app/ can be used to dummy send the messages via websockets. Transcript Viewer - https://transcript-view.netlify.app/ will show you if the messages are successfully received.
You can check your work if its successful, the captions will be displayed at Transcript View.
I will provide API Documentation for the websockets, as well as the source code for teamsview and transcript view are on github.
- You need to detect the captions via chrome UI, and send them to the server. Note that the caption is streaming, and you will need to capture it correctly (as sometimes the caption formats itself) and then send it.
- In my current code sometimes the messages overwrite itself and are not detected correctly. something for you to test / fix.
Part 4 - Performance
- Be careful on performance / memory leaks. with a teams call of 20 people, resource management is important.
Structure
There should be a popup, where the user would implement their username / redirect as authentication for the websocket / check for cookie. Will share more details later.
There should be a transcript window that clearly shows the captions are being tracked and sent to the server. Implement a reconnect feature, stop, start, pause.
You DO NOT need to:
- build a webapp to receive the captions
- build the server to handle the websockets
Please say Apple in your proposal so I know you have read this.
I will provide my current source code when we chat and walk you through it.
Thanks
The extension must:
1. Be active on the target website (teams.microsoft.com and teams.live.com)
2. Automatically launch closed captions
3. Send the closed captions via websockets to my server (API information will be provided)
4. Memory efficient, clean, robust and not affect performance.
I will now describe the parts in more detail.
Part 1 - Be active on target website
- Automatically detect when in meeting state, waiting for people to join, or people joined
- Right now my approach seems to work, could be improved and checked for performance
Part 2 - Automatically launch closed captions
- I would like you to do it via direct injection, but so far whatever i had tried results in CSP errors. I know this is possible because another plugin has successfully done it, via rtc injection. I can share their source code if it is helpful.
- Another approach is via keyboard shortcuts, I am currently using this but i want this to be a backup option. the main approach must be the direct toggle injection, which is way more reliable.
Implement a check to ensure that caption box is enabled and present, if not retry to open.
Part 3 - Send the closed captions via websockets
- For the purpose of this project, i have set up 2 basic test websites.
Teams Simulator - https://teamsview.netlify.app/ can be used to dummy send the messages via websockets. Transcript Viewer - https://transcript-view.netlify.app/ will show you if the messages are successfully received.
You can check your work if its successful, the captions will be displayed at Transcript View.
I will provide API Documentation for the websockets, as well as the source code for teamsview and transcript view are on github.
- You need to detect the captions via chrome UI, and send them to the server. Note that the caption is streaming, and you will need to capture it correctly (as sometimes the caption formats itself) and then send it.
- In my current code sometimes the messages overwrite itself and are not detected correctly. something for you to test / fix.
Part 4 - Performance
- Be careful on performance / memory leaks. with a teams call of 20 people, resource management is important.
Structure
There should be a popup, where the user would implement their username / redirect as authentication for the websocket / check for cookie. Will share more details later.
There should be a transcript window that clearly shows the captions are being tracked and sent to the server. Implement a reconnect feature, stop, start, pause.
You DO NOT need to:
- build a webapp to receive the captions
- build the server to handle the websockets
Please say Apple in your proposal so I know you have read this.
I will provide my current source code when we chat and walk you through it.
Thanks