Python OCR Script for PDF from Website

Project Title:
Automated PDF Downloader with OCR and Text Summarization from a Dynamic Website

Project Description:
We need a Python-based script that can:

Access a public web page containing dynamically generated links to PDF files.

Detect and download all PDFs whose filenames match a specified pattern (e.g., starting with "b").

Extract text content from each PDF file, including scanned/image-based PDFs using OCR.

Summarize or extract insights (e.g., challenges, solutions, QoR metrics) from the text using OpenAI GPT API.

Core Requirements:
Use firefox

Use EasyOCR or Tesseract to extract text from scanned PDFs.

Use PyMuPDF (fitz) or equivalent to render PDF pages to images for OCR.

Extracted text should be stored per file and previewed/logged during execution.

Implement chunking and embedding of text content using langchain + OpenAIEmbeddings.

Use FAISS or another in-memory vector database to perform semantic search.

Use GPT-4 (via API) to answer a given question across all extracted texts.

Environment & Compatibility:
The solution must work reliably in Google Colab AND a local Python 3 environment.

Must run on a standard setup with minimal manual configuration.

Avoid browser-specific or OS-specific dependencies that limit portability.

Deliverables:
Python script with clearly modularized functions.

README or inline instructions for setup and execution.

Output examples:

List of PDFs downloaded

Snippet of extracted text per file

GPT-4 answer based on text analysis

Optional Bonus:
Option to pass any URL and filename filter pattern via CLI or config

Automatically retry failed downloads or OCR attempts

Include UI elements (e.g., Streamlit) for interactive exploration (optional)