Universal Web Scraper System Creation
Budget: ₹1,500 – ₹12,500 INR
I need a Universal Web Scraper System that can extract data from ANY website.
The scraper must work on:
Car dealership websites
Classified listing websites
Ecommerce sites
Blogs / article sites
Any site that requires going into subpages
Facebook posts
Dynamic JS websites
Infinite scrolling websites
Sites with “See more”, “Load more”, or popup loading
Websites with pagination
1. Must fully load the website like a human
The system must:
Scroll until all content loads
Click “See more” buttons
Click expandable sections
Load lazy-loaded content
Load AJAX-based content
If a human can see it, the scraper must capture it.
2. Must detect and open detail pages
The scraper must automatically:
Find listing items (e.g., car cards)
Open each item’s detail page
Extract all information from that page
Return to the list or continue to next item
Complete the entire list
This logic should work for ANY site structure.
3. Extract ALL data from each item
From each detail page, the scraper must extract:
Title
Full description (including hidden/expanded content)
Specifications / attributes / key-value fields
Price (if available)
Mileage, year, model, etc. (if available)
Location (if available)
Tags, categories, labels
Any extra details visible on the page
The full URL of the page
Absolutely everything that is useful must be extracted.
4. Image requirements (VERY IMPORTANT)
The scraper must:
A) Download all images
Every item’s images must be downloaded
Saved neatly in folders (one folder per item)
B) CSV must contain the image URLs (not file paths)
The CSV/Excel output must include the actual image links from the website
NOT local file paths
If there are multiple images, they should be included according to my template (comma-separated or multiple columns)
5. Output format MUST follow my template
The final output must:
Match my CSV / Excel template exactly
Use the same columns, same order
Fill every field correctly
Include image URLs in the correct template fields
Include all extracted data in the matching template columns
I will provide the template file; the scraper must follow it strictly.
6. Work on ALL types of site behavior
The scraper must handle:
Scrolling pages
Pagination (1,2,3…)
“Load more” buttons
“See more” expanders
Popups that hide content
Hidden sections
Content that loads only after user actions
Dynamic JavaScript
The scraper must NOT stop early or miss items.
7. Facebook support
The scraper must:
Extract public Facebook page posts
Scroll to load more posts
Expand “See more”
Extract text, captions, links
Extract and download images
Include image URLs in the output
Only public, visible content required.
8. Reliability requirements
The scraper must:
Not freeze
Not stop partway
Be able to resume or continue even if one page fails
Scrape large lists
Scrape websites with complex layouts
Produce complete data
9. Easy to use
I must be able to:
Enter the website URL
Choose the template
Run the scraper
Receive a final CSV/Excel file following my template
Receive image folders with downloaded images
Without editing code or writing scripts
The scraper must work on:
Car dealership websites
Classified listing websites
Ecommerce sites
Blogs / article sites
Any site that requires going into subpages
Facebook posts
Dynamic JS websites
Infinite scrolling websites
Sites with “See more”, “Load more”, or popup loading
Websites with pagination
1. Must fully load the website like a human
The system must:
Scroll until all content loads
Click “See more” buttons
Click expandable sections
Load lazy-loaded content
Load AJAX-based content
If a human can see it, the scraper must capture it.
2. Must detect and open detail pages
The scraper must automatically:
Find listing items (e.g., car cards)
Open each item’s detail page
Extract all information from that page
Return to the list or continue to next item
Complete the entire list
This logic should work for ANY site structure.
3. Extract ALL data from each item
From each detail page, the scraper must extract:
Title
Full description (including hidden/expanded content)
Specifications / attributes / key-value fields
Price (if available)
Mileage, year, model, etc. (if available)
Location (if available)
Tags, categories, labels
Any extra details visible on the page
The full URL of the page
Absolutely everything that is useful must be extracted.
4. Image requirements (VERY IMPORTANT)
The scraper must:
A) Download all images
Every item’s images must be downloaded
Saved neatly in folders (one folder per item)
B) CSV must contain the image URLs (not file paths)
The CSV/Excel output must include the actual image links from the website
NOT local file paths
If there are multiple images, they should be included according to my template (comma-separated or multiple columns)
5. Output format MUST follow my template
The final output must:
Match my CSV / Excel template exactly
Use the same columns, same order
Fill every field correctly
Include image URLs in the correct template fields
Include all extracted data in the matching template columns
I will provide the template file; the scraper must follow it strictly.
6. Work on ALL types of site behavior
The scraper must handle:
Scrolling pages
Pagination (1,2,3…)
“Load more” buttons
“See more” expanders
Popups that hide content
Hidden sections
Content that loads only after user actions
Dynamic JavaScript
The scraper must NOT stop early or miss items.
7. Facebook support
The scraper must:
Extract public Facebook page posts
Scroll to load more posts
Expand “See more”
Extract text, captions, links
Extract and download images
Include image URLs in the output
Only public, visible content required.
8. Reliability requirements
The scraper must:
Not freeze
Not stop partway
Be able to resume or continue even if one page fails
Scrape large lists
Scrape websites with complex layouts
Produce complete data
9. Easy to use
I must be able to:
Enter the website URL
Choose the template
Run the scraper
Receive a final CSV/Excel file following my template
Receive image folders with downloaded images
Without editing code or writing scripts