iHerb.com - Scraping product script - READ DESCRIPTION BEFORE BIDDING!
Budget: €30 – €250 EUR
NOTE: Whoever makes an offer within the first 5 minutes of posting this job, I will automatically reject the offer because it means that he has not read the description of this job.
EVENT MORE IMPORTANT NOTE: iHerb.com has bot detection system. If you don't know how to bypass it, don't bid.
-----------------------------
I need script or app for scraping products from www.iherb.com
There are sitemaps
https://hr.iherb.com/sitemaps/products-0-hr-1.xml
https://hr.iherb.com/sitemaps/products-0-hr-2.xml
https://hr.iherb.com/sitemaps/products-0-hr-3.xml
with all URL products.
Every product has ID in the end of URL.
Example: https://hr.iherb.com/pr/bluebonnet-nutrition-l-glutamine-powder-8-oz-228-g/12047
The next step is to take that ID and connect it to another URL https://catalog.app.iherb.com/product/+ID where you will get the product data in XML format.
Example: https://catalog.app.iherb.com/product/12047
The script should scrap the data and create an XLSX document and write the following data in the columns:
partNumber
displayName
weight
listPrice
description
isAvailableToPurchase
brandName
categoryDisplayName
images (read instructions for this column)
images - important! -> to scrap images, you need use URL https://cloudinary.images-iherb.com/image/upload/images/$brandCode/$partNumber/l/$imageIndices.jpg
Example: for product https://catalog.app.iherb.com/product/12047 brandCode is BLB, partNumber is BLB-00049 and imageIndices are 37,43.
Second what you need to do is partNumber.replace("-","").
Than you will have 2 URLs for image for this product:
https://cloudinary.images-iherb.com/image/upload/images/BLB/BLB00049/l/37.jpg
https://cloudinary.images-iherb.com/image/upload/images/BLB/BLB00049/l/43.jpg
Scraping time: iHerb has almost 50.000 products. Scraping time of script should be under 24h. Thats 34 products per minute.
I can provide proxy servers for this script.
Feel free to ask all questions.
EVENT MORE IMPORTANT NOTE: iHerb.com has bot detection system. If you don't know how to bypass it, don't bid.
-----------------------------
I need script or app for scraping products from www.iherb.com
There are sitemaps
https://hr.iherb.com/sitemaps/products-0-hr-1.xml
https://hr.iherb.com/sitemaps/products-0-hr-2.xml
https://hr.iherb.com/sitemaps/products-0-hr-3.xml
with all URL products.
Every product has ID in the end of URL.
Example: https://hr.iherb.com/pr/bluebonnet-nutrition-l-glutamine-powder-8-oz-228-g/12047
The next step is to take that ID and connect it to another URL https://catalog.app.iherb.com/product/+ID where you will get the product data in XML format.
Example: https://catalog.app.iherb.com/product/12047
The script should scrap the data and create an XLSX document and write the following data in the columns:
partNumber
displayName
weight
listPrice
description
isAvailableToPurchase
brandName
categoryDisplayName
images (read instructions for this column)
images - important! -> to scrap images, you need use URL https://cloudinary.images-iherb.com/image/upload/images/$brandCode/$partNumber/l/$imageIndices.jpg
Example: for product https://catalog.app.iherb.com/product/12047 brandCode is BLB, partNumber is BLB-00049 and imageIndices are 37,43.
Second what you need to do is partNumber.replace("-","").
Than you will have 2 URLs for image for this product:
https://cloudinary.images-iherb.com/image/upload/images/BLB/BLB00049/l/37.jpg
https://cloudinary.images-iherb.com/image/upload/images/BLB/BLB00049/l/43.jpg
Scraping time: iHerb has almost 50.000 products. Scraping time of script should be under 24h. Thats 34 products per minute.
I can provide proxy servers for this script.
Feel free to ask all questions.