Python Scraper for User Profiles
Budget: ₹1,500 – ₹12,500 INR
I want a clean, well-documented Python script that reliably collects user-profile data from one or more social-media sites. At this stage I’m interested solely in profile-level information—think names, bios, follower counts, and any other public fields the platform exposes—rather than posts, comments, or media.
The exact platform is still flexible; Facebook, LinkedIn, and Twitter are all possibilities, so your approach needs to accommodate typical anti-scraping measures (login, scrolling, dynamic content, rate limits, captchas). Feel free to propose the most efficient combination of libraries or frameworks—Selenium, Playwright, Scrapy, BeautifulSoup, or API workarounds are all welcome as long as you can justify the choice and keep dependencies minimal.
Deliverables
• A repeatable Python 3 script or small package with README
• Configurable input (list of profile URLs or search term) and output to CSV/JSON
• Clear setup instructions and any required environment variables or cookies
• Brief note on legal/ethical usage and limitations of the method
Acceptance Criteria
• Script fetches and stores the requested profile fields for at least 50 sample users without manual intervention
• Handles pagination or infinite scrolling where needed
• Logs errors gracefully and respects platform rate limits
If you have questions about target platforms or need test accounts, let me know so we can finalise the scope before you begin coding.
The exact platform is still flexible; Facebook, LinkedIn, and Twitter are all possibilities, so your approach needs to accommodate typical anti-scraping measures (login, scrolling, dynamic content, rate limits, captchas). Feel free to propose the most efficient combination of libraries or frameworks—Selenium, Playwright, Scrapy, BeautifulSoup, or API workarounds are all welcome as long as you can justify the choice and keep dependencies minimal.
Deliverables
• A repeatable Python 3 script or small package with README
• Configurable input (list of profile URLs or search term) and output to CSV/JSON
• Clear setup instructions and any required environment variables or cookies
• Brief note on legal/ethical usage and limitations of the method
Acceptance Criteria
• Script fetches and stores the requested profile fields for at least 50 sample users without manual intervention
• Handles pagination or infinite scrolling where needed
• Logs errors gracefully and respects platform rate limits
If you have questions about target platforms or need test accounts, let me know so we can finalise the scope before you begin coding.
Related categories:
JavaScript
Python
Web Scraping
Software Architecture
Scrapy
API
BeautifulSoup
Selenium