Motorcycle Parts Data Scraping Project

I need to lay the digital foundation for our motorcycle-parts store by turning the catalogues of our suppliers into a clean, reusable database.
Phase 1 focuses on the full product catalogue of MRM; Phase 2 will add Alessia and any other future sources that use a similar structure.

Core tasks
• Write reliable scraping scripts (Python + Scrapy/BeautifulSoup/Selenium—choose the stack you master) that log in if required, paginate, and pull every product.
• Extract every field we rely on: description, SKU, and the complete compatibility matrix (brand, model, year).
• Download all images, rename them by SKU, and deliver them in a single, well-organised directory tree.
• Clean and normalise the raw data, then export to a single master table structured by SKU. A CSV plus a MySQL/PostgreSQL dump is perfect.
• Document the entire workflow so that any team member can re-run the scripts later to refresh the data.

Acceptance criteria
1. 100 % of live SKUs on MRM are present in the final table.
2. Each SKU contains at least one image saved with a predictable naming rule.
3. No duplicate SKUs; text fields free of HTML artefacts.
4. Code executes from the command line with one parameter (e.g., date stamp) and finishes without manual intervention.
5. README explains setup, dependencies, and re-execution steps.

A clear, orderly approach is essential because this database will plug into future systems (ERP, e-commerce, maybe a POS later). Deliver quality now and there is plenty of work ahead as we expand to Alessia and additional suppliers.