Web scrapping / Web crawler task
Budget: $250 – $750 USD
1. web scrapping / web crawler task web pages and subpages
2. parsing and crawling amazon best sellers categories (name and subsequent sub category children) into db (mysql). the structure has to be maintained (parent category), this will be recursive until the category item has no more children (child category will be located in another page, given by the link listed on the page of the parent item)
3. parsing the products under each category (all pre-collected list of category/subcategory) (asin, name, technical details, price, stock, stars, number of reviews, product images, shipping cost info) saving to db (mysql)
4. any images linked to category (icon/banner images) and product (main image and images in the product image gallery), must be saved in a tree structure folder where the path is listed on the db row relating to the item
5. need quality parsing tool, stable, with error handling and logging
6. usage of TOR to make requests use its provided proxies in sending the request to target site
7. target site like (amazon.ae)
2. parsing and crawling amazon best sellers categories (name and subsequent sub category children) into db (mysql). the structure has to be maintained (parent category), this will be recursive until the category item has no more children (child category will be located in another page, given by the link listed on the page of the parent item)
3. parsing the products under each category (all pre-collected list of category/subcategory) (asin, name, technical details, price, stock, stars, number of reviews, product images, shipping cost info) saving to db (mysql)
4. any images linked to category (icon/banner images) and product (main image and images in the product image gallery), must be saved in a tree structure folder where the path is listed on the db row relating to the item
5. need quality parsing tool, stable, with error handling and logging
6. usage of TOR to make requests use its provided proxies in sending the request to target site
7. target site like (amazon.ae)