Competitor Product Scraping Database

I need a reliable web-scraping setup that gathers product details from Australian e-commerce, retailer, and manufacturer websites and consolidates everything into a single, Shopify-ready file I can import without extra tweaking.

The data I want captured for every matching product is: product name, price, kit configuration, shipping cost or policy, and any membership or subscription pricing the site displays. Because many stores carry the same items, the script must recognise duplicates, merge the records, and clearly show which site each data point came from.

Require packet size, images, and exact names for different varities.

Please build the scraper, run an initial full pull, and deliver:
• A clean CSV formatted exactly to Shopify’s column requirements, with the fields above populated for every unique product.
• The documented code or workflow (Python, Scrapy, BeautifulSoup, Selenium—use whatever you prefer) so I can rerun it when prices change.
• A brief read-me explaining how to update the regional site list or add new sources.

Accuracy and deduplication are critical; I will spot-check before sign-off.