Evaluation of a Korean Dataset -- 4
Budget: $250 – $750 USD
I need an experienced data scientist or AI specialist to assist with evaluating a Korean dataset using OpenAI API calls.
my goal is to set up a good testing template I can repeat
Key Aspects:
- Inference Speed: Achieving a balance of optimal inference speed is vital for this project. The desired inference speed for the evaluation stands at a moderate level when considering 2000 users.
- Total Processing Accuracy: I'm looking to ensure the total processing accuracy of this evaluation is high.
- TPA (Total Processing Accuracy): Aspects of TPA are crucial for this project; we aim for a balanced importance between total processing accuracy and inference speed. (<1 /s is imperative)
- Implement this on a shared googleColab or Jupyternotebook for on-demand use of the application
Ideal Skills and Experience:
- Proficiency in Korean language and understanding of language nuances.
- Previous experience with data evaluation using OpenAI API calls.
- Proven track record in achieving optimal inference speeds for data processing.
- Demonstrated ability to balance and optimize processing accuracy alongside inference speed.
- Strong communication skills to provide regular updates on the project's progress.
refer to this documentation;
https://python.langchain.com/docs/langsmith/walkthrough/
request for access to the example document where all the testing data and instructions should be maintained and updated;
https://docs.google.com/document/d/16GFgABbbnH7RXgL56SXNu41xRspstgtVcy0TQxmwd-A/edit?usp=drive_link
API example;
https://api-engtoprod.meta-wedit.com/api-docs#/USER%20API/UsersController_perpectConversation
deliverables;
The test procedure as described and per our discussion to the format of the sample google doc.
Set of scripts for JMeter configs, python for accuracy and inference test. (hopefully Colab)
A working PC implementation to run the test, on a machine and VM (dockerized) to replicate the same testing environment repeatedly.
A full documentation of test procedure, steps, screenshots, reference links, of running tests, and setups.
my goal is to set up a good testing template I can repeat
Key Aspects:
- Inference Speed: Achieving a balance of optimal inference speed is vital for this project. The desired inference speed for the evaluation stands at a moderate level when considering 2000 users.
- Total Processing Accuracy: I'm looking to ensure the total processing accuracy of this evaluation is high.
- TPA (Total Processing Accuracy): Aspects of TPA are crucial for this project; we aim for a balanced importance between total processing accuracy and inference speed. (<1 /s is imperative)
- Implement this on a shared googleColab or Jupyternotebook for on-demand use of the application
Ideal Skills and Experience:
- Proficiency in Korean language and understanding of language nuances.
- Previous experience with data evaluation using OpenAI API calls.
- Proven track record in achieving optimal inference speeds for data processing.
- Demonstrated ability to balance and optimize processing accuracy alongside inference speed.
- Strong communication skills to provide regular updates on the project's progress.
refer to this documentation;
https://python.langchain.com/docs/langsmith/walkthrough/
request for access to the example document where all the testing data and instructions should be maintained and updated;
https://docs.google.com/document/d/16GFgABbbnH7RXgL56SXNu41xRspstgtVcy0TQxmwd-A/edit?usp=drive_link
API example;
https://api-engtoprod.meta-wedit.com/api-docs#/USER%20API/UsersController_perpectConversation
deliverables;
The test procedure as described and per our discussion to the format of the sample google doc.
Set of scripts for JMeter configs, python for accuracy and inference test. (hopefully Colab)
A working PC implementation to run the test, on a machine and VM (dockerized) to replicate the same testing environment repeatedly.
A full documentation of test procedure, steps, screenshots, reference links, of running tests, and setups.