MP3 Scrapy App Development
Budget: $30 – $250 USD
I need a scrapy app that would efficiently perform the function of downloading MP3 files from specific URLs I will provide which has Username and Password Authentication.
The app should then upload these downloaded MP3 files directly to my S3 storage and return publicly accessible URL to the MP3 file.
The scrapy app should be designed in such a way that it can input the username and password if necessary to download the MP3 file.
Ideal skills and experience:
- Proficiency in Python with a strong knowledge of Scrapy web crawling framework.
- Experience working with AWS S3 storage.
- Experience in handling potential conflicts with web scraping, like handling "robots.txt" rules, rotating IP addresses (proxies) and setting correct delays.
- Strong knowledge of web scraping ethics to avoid crashing the websites or any legal issues.
- Ability to handle potential challenges with downloading and uploading large MP3 files.
- Experience in web scraping especially with audio files would be a great advantage.
Please bid with your proposed solution, estimated timeline, and relevant experience or projects.
The app should then upload these downloaded MP3 files directly to my S3 storage and return publicly accessible URL to the MP3 file.
The scrapy app should be designed in such a way that it can input the username and password if necessary to download the MP3 file.
Ideal skills and experience:
- Proficiency in Python with a strong knowledge of Scrapy web crawling framework.
- Experience working with AWS S3 storage.
- Experience in handling potential conflicts with web scraping, like handling "robots.txt" rules, rotating IP addresses (proxies) and setting correct delays.
- Strong knowledge of web scraping ethics to avoid crashing the websites or any legal issues.
- Ability to handle potential challenges with downloading and uploading large MP3 files.
- Experience in web scraping especially with audio files would be a great advantage.
Please bid with your proposed solution, estimated timeline, and relevant experience or projects.