Bioinformatics Text Scraper Development

I need a small but reliable piece of software that automatically harvests text-based information from a set of public bioinformatic websites. The scraper must pull only the textual content I specify (no images or files) and save it in a clean, structured form I can later analyse—CSV or JSON will be fine.

Core expectations
• Navigate multiple pages on each target site, respecting robots.txt and rate limits.
• Extract the visible text inside articles, abstracts, tables and annotations while skipping ads, menus or unrelated elements.
• Offer a simple way for me to add or remove URLs and fine-tune selectors without touching the core code—ideally through a config file or command-line flags.
• Output should be UTF-8 encoded and free of HTML tags.

Technical notes
Scrapy, BeautifulSoup, Selenium or similar Python-based tools are perfectly acceptable; if you prefer another language, let me know as long as the final script runs on Windows or Linux without paid dependencies.

Deliverables
1. Fully working scraper (source code + executable/command script).
2. Sample run on at least one bioinformatic site showing correctly extracted text.
3. Brief README covering setup, usage and any environment requirements.

If you have experience working with scientific or life-science portals, please mention it; that context will help us move quickly.