Project: Quiz Automation Tool — macOS Sonoma (Apple Silicon)

Overview

I already have a working Python quiz automation prototype that:
• Uses Apple Vision / Tesseract / PaddleOCR for OCR.
• Captures screenshots with Quartz (macOS API).
• Sends parsed text to OpenAI API for answers.
• Runs with a hotkey (/) and quits with ESC.
• Installs via setup_macos.sh and works inside a virtual environment.

The only thing that doesn’t work properly:
When there’s a full page of MCQs, the script doesn’t go question by question in order. Instead, it picks and answers them in a random order. I want it to process all MCQs sequentially, top-to-bottom, for the full visible page.

⸻

Requirements

Core Fix
• Ensure that on a page with multiple MCQs, the tool:
• Detects all visible questions and their answer choices.
• Processes them in sequence (top-to-bottom), not randomly.
• Returns or highlights the correct answer consistently.

DOM Parsing (Preferred)
• Add Selenium or Playwright to directly parse the webpage DOM.
• Extract structured text (questions + choices).
• Send each question block to the AI backend.
• Much more reliable than OCR-only.

OCR Fallback
• Keep OCR backends (Apple Vision / PaddleOCR / Tesseract) as fallback.
• When DOM parsing fails (e.g., locked exam platforms).
• Must still process all questions on the visible page, in order.

System Requirements
• Must run on macOS Sonoma (Apple Silicon).
• Uses Python virtual environment with requirements.txt.
• Needs macOS permissions: Accessibility, Screen Recording, Input Monitoring.

AI Backend
• Uses latest OpenAI SDK.
• Configurable model (gpt-4o-mini by default).
• API key stored in .env file.

Usage
• Hotkey (/) to trigger automation.
• ESC to quit.
• Should highlight/log answers clearly.

⸻

Deliverables
1. Updated quiz_automation.py (fixed MCQ ordering).
2. requirements.txt (with Selenium/Playwright + OCR deps).
3. setup_macos.sh (updated bootstrap).
4. Documentation (README + setup guide).
5. Optional: diagnostic tool for backend testing.

⸻

Skills Needed
• Python 3 (macOS development).
• Selenium / Playwright for DOM automation.
• OCR libraries (PaddleOCR, Tesseract, Apple Vision).
• Quartz/macOS APIs for screen capture.
• OpenAI SDK integration.

⸻

Budget & Terms
• The base prototype is already working — just needs this MCQ ordering fix + DOM parsing support.
• Looking for a stable and reliable solution.
• Please share examples of automation projects you’ve done before.