Python Career Page Scraper

I have a growing list of company names, and I need a small, reliable Python script that can:
Automatically find each company’s career/jobs page where open positions are posted (pages may be built using HTML, JavaScript, or modern front-end frameworks)
Navigate through all job listings, including:
Pagination (page numbers, next/previous, etc.)
“Load more” buttons
Infinite scrolling
Ability to fetch data from multiple pages (e.g., page 3, 4, or beyond)
Apply job filters, especially location-based filtering, so that only job links for specific locations are collected
Extract only individual job posting links after filters are applied
Visit each job link and scrape complete job details, including:
Job title
Job description
Location
Employment type (if available)
Department / team
Any other relevant metadata
Additional Requirements
Script should be scalable for hundreds or thousands of companies
Handle different career page structures (custom sites, Greenhouse, Lever, Workday, etc.)
Support static and dynamic content (HTML + JavaScript-rendered pages)
Avoid duplicates and handle missing fields gracefully
Output data in a structured format (CSV / JSON / database)
Clean, well-documented Python code with logging and error handling
If possible, include basic anti-bot handling and retry mechanisms for failed pages.