Enhance Python Scraper with Postgres & Teable.io
Budget: $10 – $200 USD
I have an existing scraper built in python for the website www.chemistwarehouse.com.au (CWH).
It scrapes the following information:
- product description, price, images
- SEO metadata
- Additional metadata
Currently, all the scraped data is stored in a CSV file.
I am looking for a developer who has expertise in python / scraping and postgres to do the following:
[Phase / Milestone 1]
1. Convert the existing scraper (code attached - cwh.rar /site-packages removed for space) to load the data in a postgres database.
2. Scrape reviews of products from the chemistwarehouse.com.au (CWH) website and store them in the database.
3. Look up the availability of the product in specific stores and write the boolean value in the postgres database. (code attached - UpdateShopifyStock.py.zip)
[Phase / Milestone 2]
4. Once this data is scraped, I then move it to a Google Sheet where I run a script to enrich the data using ChatGPT. I want to reduce these steps or make them efficient so that it can be done within the postgress / python application.
5. Install Teable.io (it is like an airtable frontend for postgres) - to provide me with a frontend that will help me modify the prices and any other changes to the data - before pushing it to Shopify via API.
You must be proficient with Python / Postgres and good with scripting. I am looking for someone who can give me ideas to make this more efficient / fast / streamlined / automated.
This needs to be developed on a Windows Server w/ WSL or Docker Desktop (provided by me)
It scrapes the following information:
- product description, price, images
- SEO metadata
- Additional metadata
Currently, all the scraped data is stored in a CSV file.
I am looking for a developer who has expertise in python / scraping and postgres to do the following:
[Phase / Milestone 1]
1. Convert the existing scraper (code attached - cwh.rar /site-packages removed for space) to load the data in a postgres database.
2. Scrape reviews of products from the chemistwarehouse.com.au (CWH) website and store them in the database.
3. Look up the availability of the product in specific stores and write the boolean value in the postgres database. (code attached - UpdateShopifyStock.py.zip)
[Phase / Milestone 2]
4. Once this data is scraped, I then move it to a Google Sheet where I run a script to enrich the data using ChatGPT. I want to reduce these steps or make them efficient so that it can be done within the postgress / python application.
5. Install Teable.io (it is like an airtable frontend for postgres) - to provide me with a frontend that will help me modify the prices and any other changes to the data - before pushing it to Shopify via API.
You must be proficient with Python / Postgres and good with scripting. I am looking for someone who can give me ideas to make this more efficient / fast / streamlined / automated.
This needs to be developed on a Windows Server w/ WSL or Docker Desktop (provided by me)