Need Python Web Scraper created to Extract data from Json files. Scrapy.
Budget: $250 – $750 CAD
I need a web scraper created that will continously fetch SKU/Stock Codes and some other information from json files of multiple shopify websites.
There are 2 different types of SKU formats that the webscraper is looking for. One is 6 Characters like this ----> GF4456 (the first 2 characters always letters and last 4 always numbers). And the other is like this --->TH4323-232, 9 Characters with the last 3 separated by a "-".
The SKU's are in different locations in the json file depending on the websites, so I am thinking maybe a script such as the following...
For each website in "LIST 1", run a specific process to extract the SKU,,,,,, then for each website in "LIST 2" , run a different process to extract the SKU,,,,,, then for each website in "LIST 3", run a different process to extract the SKU.
The web scraper must:
Export info to a MySQL database.
Must support Rotating Proxies.
Must have some kind of HTML UI so i can stop/start the web scraper and also where i can add new proxies
I would think using Scrapy would be best for this project, unless you have any other suggestions.
I have attached an image to this project to give an idea of different locations that the Stock Codes will be on on different sites.
There are 2 different types of SKU formats that the webscraper is looking for. One is 6 Characters like this ----> GF4456 (the first 2 characters always letters and last 4 always numbers). And the other is like this --->TH4323-232, 9 Characters with the last 3 separated by a "-".
The SKU's are in different locations in the json file depending on the websites, so I am thinking maybe a script such as the following...
For each website in "LIST 1", run a specific process to extract the SKU,,,,,, then for each website in "LIST 2" , run a different process to extract the SKU,,,,,, then for each website in "LIST 3", run a different process to extract the SKU.
The web scraper must:
Export info to a MySQL database.
Must support Rotating Proxies.
Must have some kind of HTML UI so i can stop/start the web scraper and also where i can add new proxies
I would think using Scrapy would be best for this project, unless you have any other suggestions.
I have attached an image to this project to give an idea of different locations that the Stock Codes will be on on different sites.