Go Lang Script For Dump 50 Million Images
Budget: $250 – $2,000 USD
I need the script in 2 or 3 days.
I'm seeking a skilled Go Lang developer to create a script for downloading 50 million PNG files (around 500KB each). The source of these files will be an API. The challenge lies in the time constraint - all files need to be downloaded within 72 hours of starting the script. This translates to an average of 190 files per second.
Key Project Components:
- Efficiently downloading files using AWS Lambda to manage high call volumes.
- Storing each downloaded file in AWS S3.
- Implementing immediate retry logic for failed downloads.
To keep track of the script's performance, I will be using AWS CloudWatch. Ideally, you have experience with this tool and can integrate it into the script for real-time monitoring.
Your skills should include:
- Proficiency in Go Lang, with a portfolio of similar projects.
- Extensive experience with AWS services, particularly Lambda and S3.
- Familiarity with API interaction and handling large-scale data downloads.
- Knowledge of AWS CloudWatch for performance tracking.
If you have the skills and experience to tackle this project, I'd love to hear from you.
I'm seeking a skilled Go Lang developer to create a script for downloading 50 million PNG files (around 500KB each). The source of these files will be an API. The challenge lies in the time constraint - all files need to be downloaded within 72 hours of starting the script. This translates to an average of 190 files per second.
Key Project Components:
- Efficiently downloading files using AWS Lambda to manage high call volumes.
- Storing each downloaded file in AWS S3.
- Implementing immediate retry logic for failed downloads.
To keep track of the script's performance, I will be using AWS CloudWatch. Ideally, you have experience with this tool and can integrate it into the script for real-time monitoring.
Your skills should include:
- Proficiency in Go Lang, with a portfolio of similar projects.
- Extensive experience with AWS services, particularly Lambda and S3.
- Familiarity with API interaction and handling large-scale data downloads.
- Knowledge of AWS CloudWatch for performance tracking.
If you have the skills and experience to tackle this project, I'd love to hear from you.