iHerb.com - Scraping product script - READ DESCRIPTION BEFORE BIDDING!

Job ID: 37788510

Budget: €30 – €250 EUR

NOTE: Whoever makes an offer within the first 5 minutes of posting this job, I will automatically reject the offer because it means that he has not read the description of this job.

EVENT MORE IMPORTANT NOTE: iHerb.com has bot detection system. If you don't know how to bypass it, don't bid.

-----------------------------

I need script or app for scraping products from www.iherb.com

There are sitemaps
https://hr.iherb.com/sitemaps/products-0-hr-1.xml
https://hr.iherb.com/sitemaps/products-0-hr-2.xml
https://hr.iherb.com/sitemaps/products-0-hr-3.xml
with all URL products.

Every product has ID in the end of URL.
Example: https://hr.iherb.com/pr/bluebonnet-nutrition-l-glutamine-powder-8-oz-228-g/12047

The next step is to take that ID and connect it to another URL https://catalog.app.iherb.com/product/+ID where you will get the product data in XML format.
Example: https://catalog.app.iherb.com/product/12047

The script should scrap the data and create an XLSX document and write the following data in the columns:
partNumber
displayName
weight
listPrice
description
isAvailableToPurchase
brandName
categoryDisplayName
images (read instructions for this column)

images - important! -> to scrap images, you need use URL https://cloudinary.images-iherb.com/image/upload/images/$brandCode/$partNumber/l/$imageIndices.jpg
Example: for product https://catalog.app.iherb.com/product/12047 brandCode is BLB, partNumber is BLB-00049 and imageIndices are 37,43.
Second what you need to do is partNumber.replace("-","").
Than you will have 2 URLs for image for this product:
https://cloudinary.images-iherb.com/image/upload/images/BLB/BLB00049/l/37.jpg
https://cloudinary.images-iherb.com/image/upload/images/BLB/BLB00049/l/43.jpg

Scraping time: iHerb has almost 50.000 products. Scraping time of script should be under 24h. Thats 34 products per minute.
I can provide proxy servers for this script.

Feel free to ask all questions.
Related categories: PHP XML Python Web Scraping Software Architecture