Cloud Engineer Needed
Budget: $250 – $750 CAD
We have a C code which compiles successfully on many Linux platforms and runs on the local cluster producing numerous files as output. The code is a scientific code for image analysis and it is supposed to run on 100,000 separate images when successful testing is done.
1. Your job is to translate the current pipeline for the local cluster for one on AWS so that multiple runs can be initiated by project owner from outside of the cloud by visiting or sending a signal to a specific internet address.
2. Your job will also entail the task of reading all the output files and summarizing them in one single table which can be downloaded by the project owner for analysis.
3. Your job will also entail giving us an estimate of the costs of operation on the cloud and ways to minimize it. Each run can take anywhere between 1 hour and 5 hours and 100,000 runs might become too costly too soon.
4. The libraries used in the C-code are _not_standard machine learning libraries. You will have to containerize the binary and modify the bash scripts that manage the current pipeline in our local cluster to enable a cloud operation.
5. You should communicate what you do to the project owner so that when you finish the job he can continue using your scripts and modify them if necessary.
1. Your job is to translate the current pipeline for the local cluster for one on AWS so that multiple runs can be initiated by project owner from outside of the cloud by visiting or sending a signal to a specific internet address.
2. Your job will also entail the task of reading all the output files and summarizing them in one single table which can be downloaded by the project owner for analysis.
3. Your job will also entail giving us an estimate of the costs of operation on the cloud and ways to minimize it. Each run can take anywhere between 1 hour and 5 hours and 100,000 runs might become too costly too soon.
4. The libraries used in the C-code are _not_standard machine learning libraries. You will have to containerize the binary and modify the bash scripts that manage the current pipeline in our local cluster to enable a cloud operation.
5. You should communicate what you do to the project owner so that when you finish the job he can continue using your scripts and modify them if necessary.