training and autoscaling deployment SageMaker AWS python ML
Budget: $30 – $250 CAD
python SageMaker training and deployment for simple given ML model : main challenge is autoscaling
you have 7 days to do
1. I do not provide you my AWS account . you develop all on your AWS account
2. Teach me ML project end to end development on SageMaker AWS with autoscaling in production
3. Any python package installed by pip can be used for this ML project (not limited to scikit or deep learning) .
such as
from lightning.classification import CDClassifier
from gensim.parsing.preprocessing import preprocess_documents
4. Build with Automatic scaling a customized model training for python code ML model provided by me or you can use your modle code if I like it
5. Build with Automatic scaling a customized inference
6. Automatic scaling, load balancing to handle 1234 requests/sec, requests done by python from any computer on web
7. many SageMaker AWS instances Distributed training with Hyperparameters search in parallel calculations
8. many SageMaker AWS instances Distributed Preprocessing in parallel calculations
9. many SageMaker AWS instances Distributed Inference with autoscaling in parallel calculations
10. requests implemented by you developed python code for predictions may be done from any computer on web up to 1234 requests per second , everybody with credentials using python to web access can create requests (not ssh or any remote connection to AWS web site)
11. no GUI design, but all done by python SDK - only python code used
12. ML starter code and data will be provided or you can use your ML model code if I like it
13. Provide 2 solutions with Docker and without Docker
14 use 1 to 5 low cost AWS instances. Scaling time 1 second for both scaling up and scaling down
for example for scaling up
Or requests per second criteria
less then 100 requests per second use 1 instance
more than 100 but less then 200 requests per second use 2 instance
more than 200 but less then 300 requests per second use 3 instance
etc
Or percentage load criteria
when computing utilization more than 60% add one more instance
Or suggest your way
15 when new data added to s3 backet , ML model will be automatically retrained
16 make sure no one request got lost: if request got lost then resend it many times again and again to AWS till request is processed . If after 15 times request is not processed any way , record this request to black list log file
YOU PROVIDE PYTHON CODE AND
VIDEO of yours or our online meeting
WITH EXPLANATIONS STEP BY STEP HOW TO DO
**** PROJECT ACCEPTANCE : I CAN DO BY MYSELF on my AWS account ****
Testing : I use your python code which hosted on my AWS cloud project created by me using your instructions
example: I provide to John, Peter, Ana, Paul.. etc, only one the same web link and only python code and credentials (only people with credentials can make request )for :
1) request implementation for test data
2) receiving response and comparing with prediction calculated locally
3) Requests are sent by each person in parallel: 1234 requests per second combined for all of them (not each one 1243), for requests generation you can use data from train and test data set. Make sure there are no dropped requests ( requests without response )
4) Requests consists of data sample and unique Request ID. Machine learning model code receives data and Request ID. Then returns prediction and the same Request ID concatenated with date time of processing. Response: id and prediction and date time
Many requests are sent asynchronously in batches or continuously. When predictions are received: unique ID is used to match prediction and request and date time stamp to estimate delay time
( example :concatenate to id time of predictions for example ID = abc34bnm at time 20240211164320 , then together with data send to aws additional variable abc34bnm_20240211164320
returned receipt at time 20240211164324 is abc34bnm_20240211164320_20240211164324 )
ID to confirm it is the same request what was sent for prediction and time stamps enable to see delay
just ideas
https://github.com/wideflat/aws-sagemaker-hptuning-insurance-claims-model/tree/main
https://github.com/aws-samples/amazon-sagemaker-pipeline-deploy-manage-100x-models-python-cdk/issues/4
https://youtu.be/8JFlJlB9laU?t=4552
Elastic Load Balancing | AWS Cloud Certification | iNeuron
https://github.com/ChandraLingam/AmazonSageMakerCourse/blob/master/IntegrationExamples/invoke_api_gateway.ipynb
https://towardsdatascience.com/bring-your-own-container-with-amazon-sagemaker-37211d8412f4
https://github.com/maxbrenner-ai/aws-custom-docker-model-deploy-front-end
https://towardsdatascience.com/deploying-a-custom-docker-model-with-sagemaker-to-a-serverless-front-end-with-s3-8ee07edc24e6
https://github.com/datadrivers/effective-guide-mlops/tree/main
FYI fast inference https://aws.amazon.com/blogs/machine-learning/optimize-aws-inferentia-utilization-with-fastapi-and-pytorch-models-on-amazon-ec2-inf1-inf2-instances/
you have 7 days to do
1. I do not provide you my AWS account . you develop all on your AWS account
2. Teach me ML project end to end development on SageMaker AWS with autoscaling in production
3. Any python package installed by pip can be used for this ML project (not limited to scikit or deep learning) .
such as
from lightning.classification import CDClassifier
from gensim.parsing.preprocessing import preprocess_documents
4. Build with Automatic scaling a customized model training for python code ML model provided by me or you can use your modle code if I like it
5. Build with Automatic scaling a customized inference
6. Automatic scaling, load balancing to handle 1234 requests/sec, requests done by python from any computer on web
7. many SageMaker AWS instances Distributed training with Hyperparameters search in parallel calculations
8. many SageMaker AWS instances Distributed Preprocessing in parallel calculations
9. many SageMaker AWS instances Distributed Inference with autoscaling in parallel calculations
10. requests implemented by you developed python code for predictions may be done from any computer on web up to 1234 requests per second , everybody with credentials using python to web access can create requests (not ssh or any remote connection to AWS web site)
11. no GUI design, but all done by python SDK - only python code used
12. ML starter code and data will be provided or you can use your ML model code if I like it
13. Provide 2 solutions with Docker and without Docker
14 use 1 to 5 low cost AWS instances. Scaling time 1 second for both scaling up and scaling down
for example for scaling up
Or requests per second criteria
less then 100 requests per second use 1 instance
more than 100 but less then 200 requests per second use 2 instance
more than 200 but less then 300 requests per second use 3 instance
etc
Or percentage load criteria
when computing utilization more than 60% add one more instance
Or suggest your way
15 when new data added to s3 backet , ML model will be automatically retrained
16 make sure no one request got lost: if request got lost then resend it many times again and again to AWS till request is processed . If after 15 times request is not processed any way , record this request to black list log file
YOU PROVIDE PYTHON CODE AND
VIDEO of yours or our online meeting
WITH EXPLANATIONS STEP BY STEP HOW TO DO
**** PROJECT ACCEPTANCE : I CAN DO BY MYSELF on my AWS account ****
Testing : I use your python code which hosted on my AWS cloud project created by me using your instructions
example: I provide to John, Peter, Ana, Paul.. etc, only one the same web link and only python code and credentials (only people with credentials can make request )for :
1) request implementation for test data
2) receiving response and comparing with prediction calculated locally
3) Requests are sent by each person in parallel: 1234 requests per second combined for all of them (not each one 1243), for requests generation you can use data from train and test data set. Make sure there are no dropped requests ( requests without response )
4) Requests consists of data sample and unique Request ID. Machine learning model code receives data and Request ID. Then returns prediction and the same Request ID concatenated with date time of processing. Response: id and prediction and date time
Many requests are sent asynchronously in batches or continuously. When predictions are received: unique ID is used to match prediction and request and date time stamp to estimate delay time
( example :concatenate to id time of predictions for example ID = abc34bnm at time 20240211164320 , then together with data send to aws additional variable abc34bnm_20240211164320
returned receipt at time 20240211164324 is abc34bnm_20240211164320_20240211164324 )
ID to confirm it is the same request what was sent for prediction and time stamps enable to see delay
just ideas
https://github.com/wideflat/aws-sagemaker-hptuning-insurance-claims-model/tree/main
https://github.com/aws-samples/amazon-sagemaker-pipeline-deploy-manage-100x-models-python-cdk/issues/4
https://youtu.be/8JFlJlB9laU?t=4552
Elastic Load Balancing | AWS Cloud Certification | iNeuron
https://github.com/ChandraLingam/AmazonSageMakerCourse/blob/master/IntegrationExamples/invoke_api_gateway.ipynb
https://towardsdatascience.com/bring-your-own-container-with-amazon-sagemaker-37211d8412f4
https://github.com/maxbrenner-ai/aws-custom-docker-model-deploy-front-end
https://towardsdatascience.com/deploying-a-custom-docker-model-with-sagemaker-to-a-serverless-front-end-with-s3-8ee07edc24e6
https://github.com/datadrivers/effective-guide-mlops/tree/main
FYI fast inference https://aws.amazon.com/blogs/machine-learning/optimize-aws-inferentia-utilization-with-fastapi-and-pytorch-models-on-amazon-ec2-inf1-inf2-instances/