Solution to feed GPT model with big data on Python
Budget: €250 – €750 EUR
I have an application that analyzes audio chats between people. On the basis of the conversation, you can get ratings of trustworthiness, charisma, grammar, and so on. Evaluations of both the user of the app and the people he is talking to. At a basic level, if data analysis is done only within a single audio (5-120 min) - everything works well, GPT will give the right answer with the right question. But now the challenge is how to combine all transcripts from audio together and already give analysis on the basis of all meetings with a particular interlocutor.
For example, the current flagship GPT - 4 accepts no more than 128k tokens, which is a minimum of $1.28 per request. 128k tokens is only about 10 hours of pure speech. The question is, what if a user records his conversations for a month to a year, how do I feed such a volume into the model for sentiment analysis? In theory, I know that there are methods of summarisation, selective analysis in conjunction with vector databases + you can always feed queries in separate chunks of 128k tokens, which will be even more expensive. But this is just general advice. I need a person with practical experience in this field.
Full task https://docs.google.com/document/d/1evX_jqgqrkJg2YLn-aK01Fc2PrM_Y2MfZXegdpito5o/edit
For example, the current flagship GPT - 4 accepts no more than 128k tokens, which is a minimum of $1.28 per request. 128k tokens is only about 10 hours of pure speech. The question is, what if a user records his conversations for a month to a year, how do I feed such a volume into the model for sentiment analysis? In theory, I know that there are methods of summarisation, selective analysis in conjunction with vector databases + you can always feed queries in separate chunks of 128k tokens, which will be even more expensive. But this is just general advice. I need a person with practical experience in this field.
Full task https://docs.google.com/document/d/1evX_jqgqrkJg2YLn-aK01Fc2PrM_Y2MfZXegdpito5o/edit