Python Data Scraping Script

Create a clean, well-documented Python script that automatically gathers structured data from online sources I will specify once the project begins. The code should accept a list of URLs (or a URL pattern), pull the required fields, and export the results to CSV or JSON in a single run. Please keep the design modular so I can later adapt it for different sites or schedule it for regular execution.

Core expectations
• Use standard libraries such as requests (or httpx), BeautifulSoup or lxml for parsing, and pandas for tidy output.
• Respect robots.txt and add reasonable delays, rotating headers or proxies if a site requires it.
• Handle pagination, redirects, and common errors gracefully with clear logging.
• Include a short README explaining setup, dependencies (pip/conda), and sample command-line usage.
• Deliver a single .py file (or small package) plus the README and a sample output file that proves the script works.

I will supply the target URLs and the exact data points after awarding the project; the layout is straightforward, so no CAPTCHA or login steps are involved. Clean, readable code and a quick turnaround are more important to me than fancy tooling—stick to vanilla Python 3.x.