Competitor Product Data Scraper

I need a reliable script that pulls up-to-date product information from my main competitors’ websites and drops everything neatly into a CSV file. The scraper should capture standard catalogue fields—title, price, description, SKU, stock status, image URL and category—and do so in a way that can be re-run on demand without manual tweaks each time the sites change minor layouts.

Python is my preferred environment because of its mature scraping ecosystem, so feel free to lean on libraries such as Requests, BeautifulSoup, Scrapy or even Selenium when dynamic content gets in the way. If an official or unofficial API exists for any target, please use it to minimise breakage and respect rate limits.

The final hand-off will include:
• The well-documented source code and requirements.txt so I can reproduce the environment easily.
• A short README explaining how to add new competitor URLs or tweak export fields.
• A sample run that outputs a clean CSV confirming all expected columns are present and properly encoded.

Accuracy and resilience are more important to me than sheer speed, so please build in basic error handling and polite scraping etiquette (back-offs, user-agent rotation, etc.). Let me know how you plan to approach the tougher sites, and share any previous experience scraping e-commerce data—this will help me choose quickly and get the project moving.