Improve scalability and concurrency of an ML app
Budget: ₹600 – ₹1,500 INR
More details:
I have deploy ML inference code on uvicorn FastAPI server. When I run request sequentially or send 2-3 concurrent request then I get very low response time but when I increase to higher concurrency it increases exponentially. Want to debug this problem and fix it. I wont be able to provide you code to run on your system. I will share my screen and can any experiment as you suggest
Which programming language is the ML app currently written in? Python
What type of machine learning model does the app utilize? FastAPI
Does the app require real-time predictions or batch processing? Real-time predictions
I have deploy ML inference code on uvicorn FastAPI server. When I run request sequentially or send 2-3 concurrent request then I get very low response time but when I increase to higher concurrency it increases exponentially. Want to debug this problem and fix it. I wont be able to provide you code to run on your system. I will share my screen and can any experiment as you suggest
Which programming language is the ML app currently written in? Python
What type of machine learning model does the app utilize? FastAPI
Does the app require real-time predictions or batch processing? Real-time predictions