Car Parts Data Extraction Automation -- 2
Budget: $25 – $50 USD
I purchase automobile parts worldwide and need their details—supplier contact number, location, part type, specifications, and current price—captured automatically instead of hunting through dozens of sites every day.
Here is what I want built:
• A small C helper that runs inside our existing Windows desktop environment, launches on-demand, and passes search terms or part numbers into a Python layer.
• The Python layer handles the heavy lifting: it scrapes public websites, calls any open or authenticated APIs the supplier provides, taps into our internal parts database when needed, then normalises every record.
• The final, cleaned result must be written straight into our central purchasing database; I do not need CSVs or spreadsheets, only live database entries ready for our ERP to consume.
Key functions
– Web scraping of supplier catalogues, with respect for robots.txt and pagination
– API integration where suppliers expose endpoints for stock and pricing updates
– Basic data analysis to flag unusually high prices or missing specs before they reach my screen
Tech preferences
Python (requests/BeautifulSoup or Selenium where necessary), SQLAlchemy for inserts, and C for the lightweight launcher that fits with our internal tooling. If you favour equivalent libraries, just state why.
Acceptance criteria
1. Given a sample list of ten part numbers, the program populates our test database with accurate contact, location, type, spec, and price data.
2. Running the job again a day later updates only changed fields, leaving prior history intact.
3. Average runtime per part under 5 s when pulling from both an API and a web page.
I will supply database credentials and a handful of supplier URLs or API keys once we start. Let me know your relevant projects scraping or integrating similar multi-source data, and roughly how you would structure the C ↔ Python hand-off.
Here is what I want built:
• A small C helper that runs inside our existing Windows desktop environment, launches on-demand, and passes search terms or part numbers into a Python layer.
• The Python layer handles the heavy lifting: it scrapes public websites, calls any open or authenticated APIs the supplier provides, taps into our internal parts database when needed, then normalises every record.
• The final, cleaned result must be written straight into our central purchasing database; I do not need CSVs or spreadsheets, only live database entries ready for our ERP to consume.
Key functions
– Web scraping of supplier catalogues, with respect for robots.txt and pagination
– API integration where suppliers expose endpoints for stock and pricing updates
– Basic data analysis to flag unusually high prices or missing specs before they reach my screen
Tech preferences
Python (requests/BeautifulSoup or Selenium where necessary), SQLAlchemy for inserts, and C for the lightweight launcher that fits with our internal tooling. If you favour equivalent libraries, just state why.
Acceptance criteria
1. Given a sample list of ten part numbers, the program populates our test database with accurate contact, location, type, spec, and price data.
2. Running the job again a day later updates only changed fields, leaving prior history intact.
3. Average runtime per part under 5 s when pulling from both an API and a web page.
I will supply database credentials and a handful of supplier URLs or API keys once we start. Let me know your relevant projects scraping or integrating similar multi-source data, and roughly how you would structure the C ↔ Python hand-off.