Local-First Automation Framework Developer
Budget: $250 – $350 CAD
Summary
1. Project Overview
We are building a local-first automation framework that captures user workflows on the desktop and translates them into autonomous browser actions. The project requires forking and integrating two major open-source repositories to create a secure, anonymized "observation-to-execution" pipeline.
The Objective: Create a bridge where a local screen-capture engine feeds data into a PII-scrubbing layer, which then stores structured "Intent Nodes" in a cloud database to be executed by an AI browser agent.
2. Technical Stack
Observation Layer: ScreenPipe (Rust / SQLite)
Execution Layer: Stagehand (TypeScript / Playwright)
Privacy Layer: Microsoft Presidio (Python / NLP)
Database/State: Convex (Real-time syncing)
Environment: Cross-platform (Windows/macOS), primarily Local-first.
3. Scope of Work and Deliverables
Deliverable 1: The Secure ScreenPipe Fork
Fork and Setup: Fork the ScreenPipe repository and implement a custom build pipeline.
On-Device PII Scrubbing: Integrate Microsoft Presidio (or a performance-equivalent SLM) directly into the capture stream.
Requirement: Text extracted via OCR or the Accessibility Tree must be scrubbed of PII (Names, IDs, SSNs, etc.) locally before it is stored or synced.
Convex Sync: Wire the scrubbed metadata and accessibility events to a Convex database via streaming mutations.
Deliverable 2: The Intent Node Generator
Implement a logic layer that converts raw screen/accessibility events into a structured Intent Node JSON.
Schema Requirements:
Goal: The high-level objective detected.
Intent: The specific next step intended.
Context: The surrounding metadata (Active App, URL, Field Labels).
Primary Action: The exact interaction (example: click Submit).
Fallback: Alternative logic if the primary action fails.
Deliverable 3: Stagehand LOCAL Integration
Configure Stagehand to run in LOCAL mode (no Browserbase cloud dependency).
Create a "Trigger" listener that watches the Convex database for new Intent Nodes and initiates a Stagehand "execute" or "act" command in a local Chromium instance.
4. Mandatory Requirements (Selection Criteria)
Systems Proficiency: Proven experience with Rust (for ScreenPipe/Tauri internals) and TypeScript (for Stagehand/Playwright).
Privacy Engineering: Experience with PII detection and NLP-based scrubbing (Presidio).
Database Architect: Deep knowledge of real-time syncing (Convex) and vector-search implementation.
Browser Automation: Expertise in Playwright/CDP (Chrome DevTools Protocol).
Local-First Mindset: Understanding of resource management (CPU/RAM) for background desktop applications.
5. Definition of Done (Success Metrics)
The project is considered complete when the following "Technical Loop" is functional:
Capture: I perform a task on a web portal (example: a mock login).
Scrub: The system captures the event, redacts my username/password locally, and sends the scrubbed metadata to Convex.
Execute: Within less than 500ms, a separate Stagehand instance detects the new Intent Node in Convex and replicates the same action in a "Headless" local browser.
Stability: The system maintains under 10 percent CPU usage on a standard 16GB RAM machine while active.
6. How to Apply
Please provide:
Links to your GitHub/Portfolio showing Rust or Playwright-based projects.
A brief description of a time you integrated a local desktop app with a cloud-syncing database.
Your estimated timeline for building the "Technical Bridge."
1. Project Overview
We are building a local-first automation framework that captures user workflows on the desktop and translates them into autonomous browser actions. The project requires forking and integrating two major open-source repositories to create a secure, anonymized "observation-to-execution" pipeline.
The Objective: Create a bridge where a local screen-capture engine feeds data into a PII-scrubbing layer, which then stores structured "Intent Nodes" in a cloud database to be executed by an AI browser agent.
2. Technical Stack
Observation Layer: ScreenPipe (Rust / SQLite)
Execution Layer: Stagehand (TypeScript / Playwright)
Privacy Layer: Microsoft Presidio (Python / NLP)
Database/State: Convex (Real-time syncing)
Environment: Cross-platform (Windows/macOS), primarily Local-first.
3. Scope of Work and Deliverables
Deliverable 1: The Secure ScreenPipe Fork
Fork and Setup: Fork the ScreenPipe repository and implement a custom build pipeline.
On-Device PII Scrubbing: Integrate Microsoft Presidio (or a performance-equivalent SLM) directly into the capture stream.
Requirement: Text extracted via OCR or the Accessibility Tree must be scrubbed of PII (Names, IDs, SSNs, etc.) locally before it is stored or synced.
Convex Sync: Wire the scrubbed metadata and accessibility events to a Convex database via streaming mutations.
Deliverable 2: The Intent Node Generator
Implement a logic layer that converts raw screen/accessibility events into a structured Intent Node JSON.
Schema Requirements:
Goal: The high-level objective detected.
Intent: The specific next step intended.
Context: The surrounding metadata (Active App, URL, Field Labels).
Primary Action: The exact interaction (example: click Submit).
Fallback: Alternative logic if the primary action fails.
Deliverable 3: Stagehand LOCAL Integration
Configure Stagehand to run in LOCAL mode (no Browserbase cloud dependency).
Create a "Trigger" listener that watches the Convex database for new Intent Nodes and initiates a Stagehand "execute" or "act" command in a local Chromium instance.
4. Mandatory Requirements (Selection Criteria)
Systems Proficiency: Proven experience with Rust (for ScreenPipe/Tauri internals) and TypeScript (for Stagehand/Playwright).
Privacy Engineering: Experience with PII detection and NLP-based scrubbing (Presidio).
Database Architect: Deep knowledge of real-time syncing (Convex) and vector-search implementation.
Browser Automation: Expertise in Playwright/CDP (Chrome DevTools Protocol).
Local-First Mindset: Understanding of resource management (CPU/RAM) for background desktop applications.
5. Definition of Done (Success Metrics)
The project is considered complete when the following "Technical Loop" is functional:
Capture: I perform a task on a web portal (example: a mock login).
Scrub: The system captures the event, redacts my username/password locally, and sends the scrubbed metadata to Convex.
Execute: Within less than 500ms, a separate Stagehand instance detects the new Intent Node in Convex and replicates the same action in a "Headless" local browser.
Stability: The system maintains under 10 percent CPU usage on a standard 16GB RAM machine while active.
6. How to Apply
Please provide:
Links to your GitHub/Portfolio showing Rust or Playwright-based projects.
A brief description of a time you integrated a local desktop app with a cloud-syncing database.
Your estimated timeline for building the "Technical Bridge."