Python News Keyword Scraper
Budget: $3,000 – $5,000 USD
I need a small Python utility that scans ESPN.com twice a day, looks at every headline, sub-headline, video title, and video description, and flags any of roughly 900 supplied keywords or short phrases.
Here is the workflow I have in mind: the script fetches fresh content, parses the four content types, checks each string against my keyword list, then writes matches—along with the page URL, publication time-stamp, and the exact text it matched—to a clean Excel-compatible CSV file. I will download the finished program, drop it in a local folder, and run it manually, so all external dependencies need to install with a simple requirements.txt and the command python main.py should be enough to start a new scan.
Please keep the codebase pure Python. Feel free to rely on requests/BeautifulSoup for static pages, but if any of these sites load headlines dynamically, a lightweight headless solution such as Selenium or Playwright is acceptable—just document it clearly.
Deliverables
• Fully commented Python source code and a one-line command for execution
• A separate, editable text/CSV file holding the 900 keywords
• Sample output CSV showing a successful run on each site
• A short README explaining setup, configuration, and how to add more sites or keywords
Acceptance criteria: running the script on my machine must create a CSV that lists every matching headline, sub-headline, video title, or description from the three sites within the same session, with no duplicates and accurate timestamps.
Here is the workflow I have in mind: the script fetches fresh content, parses the four content types, checks each string against my keyword list, then writes matches—along with the page URL, publication time-stamp, and the exact text it matched—to a clean Excel-compatible CSV file. I will download the finished program, drop it in a local folder, and run it manually, so all external dependencies need to install with a simple requirements.txt and the command python main.py should be enough to start a new scan.
Please keep the codebase pure Python. Feel free to rely on requests/BeautifulSoup for static pages, but if any of these sites load headlines dynamically, a lightweight headless solution such as Selenium or Playwright is acceptable—just document it clearly.
Deliverables
• Fully commented Python source code and a one-line command for execution
• A separate, editable text/CSV file holding the 900 keywords
• Sample output CSV showing a successful run on each site
• A short README explaining setup, configuration, and how to add more sites or keywords
Acceptance criteria: running the script on my machine must create a CSV that lists every matching headline, sub-headline, video title, or description from the three sites within the same session, with no duplicates and accurate timestamps.
Related categories:
Python
Web Scraping
Software Architecture
Data Mining
Data Extraction
BeautifulSoup
Selenium
Automation