Web Scraping Tool using Beautiful Soup and Selenium or Similar Coding
Budget: €8 – €50 EUR
We have currently an AWS hosted dashboard web app which gives us ability to run product web scrapes using Beautiful Soup and Selenium.
The current functionality it provides using the script is:
Login into website backends with username/passwords we hold and pull product date like stock, sku, image url, descriptions etc.
Return the data on a csv file
2 version options on scrapes between sku/stock just for stock udpates and also full product information for data uploads etc.
Ability to run comparison between up to last 5 download files to identify products that removed from website or new products added.
We someone to design something with identical functionality but maybe with cheaper hosting option or created in different code so if our main scrape system goes down we have backup. Current speed of scrape takes about 10 mins for 1000 products to run.
Option to suggestions but must be dashboard runnable for our staff to use!
The current functionality it provides using the script is:
Login into website backends with username/passwords we hold and pull product date like stock, sku, image url, descriptions etc.
Return the data on a csv file
2 version options on scrapes between sku/stock just for stock udpates and also full product information for data uploads etc.
Ability to run comparison between up to last 5 download files to identify products that removed from website or new products added.
We someone to design something with identical functionality but maybe with cheaper hosting option or created in different code so if our main scrape system goes down we have backup. Current speed of scrape takes about 10 mins for 1000 products to run.
Option to suggestions but must be dashboard runnable for our staff to use!
Related categories:
Web Scraping
Software Architecture
Amazon Web Services
BeautifulSoup
Selenium Webdriver