Senior Engineer for AI & Web Scraping Project
Budget: $15 – $25 USD
Senior Full-Stack Engineer / Technical Lead – Data Ingestion, Web Scraping & AI Systems (Long-Term)
Job Description
We are a fast-scaling real estate investment and technology company building a data-driven platform focused on sourcing, analyzing, and tracking commercial real estate opportunities across the U.S.
We are looking for a senior-level engineer / technical lead who can help us own and scale our existing platform, improve reliability, and build robust data ingestion systems. This is not a short-term task — we are looking for someone who wants to grow with the project and eventually help lead a broader technical team.
This role is ideal for someone who is hands-on, systems-oriented, and proactive, not someone who needs detailed instructions for every task.
What We’re Building
A web-based platform that aggregates CRE listings, auctions, tax records, and distressed asset data
Automated web scraping and data ingestion pipelines (daily + weekly)
Structured data systems (JSON → database → UI)
Broker and contact relationship mapping
AI-assisted workflows for analysis, enrichment, and automation
We already have an existing platform and data models — this role is about making them reliable, scalable, and production-grade.
Core Responsibilities
1. Web Scraping & Data Ingestion
Design and maintain scraping pipelines for hundreds of sites (auction sites, county records, tax databases, listing platforms)
Handle inconsistent HTML, pagination, rate limits, CAPTCHAs, and site changes
Normalize scraped data into clean, structured schemas (JSON → DB)
Ensure all available fields are captured (NOI, cap rate, square footage, price, URLs, images, contacts, etc.)
Build monitoring/alerting so broken scrapers don’t silently fail
2. Platform & Performance
Improve platform speed, reliability, and scalability
Optimize backend ingestion flows and frontend performance
Ensure data consistency between ingestion, storage, and UI
Help clean up and standardize existing pipelines
3. Architecture & Ownership
Take ownership of ingestion and scraping systems end-to-end
Proactively suggest improvements, tools, and architectural changes
Help define best practices as we scale the engineering team
Communicate clearly about blockers, risks, and timelines
4. AI & Automation (Bonus but Important)
Assist with AI-powered workflows (data enrichment, classification, analysis)
Integrate AI tools where they meaningfully reduce manual work
Help evaluate models, APIs, and automation strategies
Required Experience
5+ years as a full-stack or backend-focused engineer
Strong experience with web scraping at scale
Comfortable with messy, real-world data
Experience building ingestion pipelines that don’t break quietly
Strong debugging and problem-solving skills
Ability to work independently and take ownership
Strongly Preferred
Experience scraping government, auction, or real estate sites
Familiarity with CRE data (NOI, cap rates, price/SF, etc.)
Experience with Python, Node.js, or similar
Experience with databases (Postgres, Supabase, etc.)
Experience working in fast-moving startups
Experience leading or mentoring other engineers
How We Work
We move fast
We value clear communication and accountability
We care more about working systems than fancy demos
We prefer engineers who flag issues early instead of working silently
What Success Looks Like
Scrapers run reliably without constant babysitting
Data ingests completely and correctly
The platform always has fresh, usable opportunities
Engineering reduces workload instead of creating more
We can confidently scale volume without breaking systems
Engagement Details
Long-term opportunity
Hourly or monthly retainer (open to discussion)
Immediate start for the right candidate
Opportunity to grow into a lead technical role
Job Description
We are a fast-scaling real estate investment and technology company building a data-driven platform focused on sourcing, analyzing, and tracking commercial real estate opportunities across the U.S.
We are looking for a senior-level engineer / technical lead who can help us own and scale our existing platform, improve reliability, and build robust data ingestion systems. This is not a short-term task — we are looking for someone who wants to grow with the project and eventually help lead a broader technical team.
This role is ideal for someone who is hands-on, systems-oriented, and proactive, not someone who needs detailed instructions for every task.
What We’re Building
A web-based platform that aggregates CRE listings, auctions, tax records, and distressed asset data
Automated web scraping and data ingestion pipelines (daily + weekly)
Structured data systems (JSON → database → UI)
Broker and contact relationship mapping
AI-assisted workflows for analysis, enrichment, and automation
We already have an existing platform and data models — this role is about making them reliable, scalable, and production-grade.
Core Responsibilities
1. Web Scraping & Data Ingestion
Design and maintain scraping pipelines for hundreds of sites (auction sites, county records, tax databases, listing platforms)
Handle inconsistent HTML, pagination, rate limits, CAPTCHAs, and site changes
Normalize scraped data into clean, structured schemas (JSON → DB)
Ensure all available fields are captured (NOI, cap rate, square footage, price, URLs, images, contacts, etc.)
Build monitoring/alerting so broken scrapers don’t silently fail
2. Platform & Performance
Improve platform speed, reliability, and scalability
Optimize backend ingestion flows and frontend performance
Ensure data consistency between ingestion, storage, and UI
Help clean up and standardize existing pipelines
3. Architecture & Ownership
Take ownership of ingestion and scraping systems end-to-end
Proactively suggest improvements, tools, and architectural changes
Help define best practices as we scale the engineering team
Communicate clearly about blockers, risks, and timelines
4. AI & Automation (Bonus but Important)
Assist with AI-powered workflows (data enrichment, classification, analysis)
Integrate AI tools where they meaningfully reduce manual work
Help evaluate models, APIs, and automation strategies
Required Experience
5+ years as a full-stack or backend-focused engineer
Strong experience with web scraping at scale
Comfortable with messy, real-world data
Experience building ingestion pipelines that don’t break quietly
Strong debugging and problem-solving skills
Ability to work independently and take ownership
Strongly Preferred
Experience scraping government, auction, or real estate sites
Familiarity with CRE data (NOI, cap rates, price/SF, etc.)
Experience with Python, Node.js, or similar
Experience with databases (Postgres, Supabase, etc.)
Experience working in fast-moving startups
Experience leading or mentoring other engineers
How We Work
We move fast
We value clear communication and accountability
We care more about working systems than fancy demos
We prefer engineers who flag issues early instead of working silently
What Success Looks Like
Scrapers run reliably without constant babysitting
Data ingests completely and correctly
The platform always has fresh, usable opportunities
Engineering reduces workload instead of creating more
We can confidently scale volume without breaking systems
Engagement Details
Long-term opportunity
Hourly or monthly retainer (open to discussion)
Immediate start for the right candidate
Opportunity to grow into a lead technical role