Expert on Data Scraping Automation
Budget: $250 – $750 USD
I need to collect social-media data—specifically tweets from Twitter—for an internal analytics project.
Objective
Develop a script to extract tweets (text only) from specific X/Twitter profiles without using the official API. The data contains stock-related signals critical for buy/sell decisions.
Please see the Data Extraction requirement document attached for more details
Deliverables
The script should:
- Log in to X/Twitter using provided credentials (browser-based, not API).
- Extract tweet text, username, timestamp, and source every 1 minute.
- Run automatically between 1:00 AM EST – 9:00 PM EST, Monday through Friday.
- Save all extracted tweets as a JSON file.
- Upload each JSON file automatically to AWS S3.
- Operate continuously without getting the account blocked or banned due to scraping.
Please answer below questions:
1. What is your experience in scraping data from websites like X/Twitter (without using API)?
2. What tools or libraries will you use for this project? (example: Playwright, Puppeteer, Selenium, etc.)
3. How will you make sure the X/Twitter profile doesn’t get blocked or banned while scraping?
4. Will you provide support if the script stops working or needs updates later?
5. How soon can you finish the project, and what’s your price quote?
6. Apart from the given credentials, AWS token, and VPS access — do you need anything else to run your solution?
Objective
Develop a script to extract tweets (text only) from specific X/Twitter profiles without using the official API. The data contains stock-related signals critical for buy/sell decisions.
Please see the Data Extraction requirement document attached for more details
Deliverables
The script should:
- Log in to X/Twitter using provided credentials (browser-based, not API).
- Extract tweet text, username, timestamp, and source every 1 minute.
- Run automatically between 1:00 AM EST – 9:00 PM EST, Monday through Friday.
- Save all extracted tweets as a JSON file.
- Upload each JSON file automatically to AWS S3.
- Operate continuously without getting the account blocked or banned due to scraping.
Please answer below questions:
1. What is your experience in scraping data from websites like X/Twitter (without using API)?
2. What tools or libraries will you use for this project? (example: Playwright, Puppeteer, Selenium, etc.)
3. How will you make sure the X/Twitter profile doesn’t get blocked or banned while scraping?
4. Will you provide support if the script stops working or needs updates later?
5. How soon can you finish the project, and what’s your price quote?
6. Apart from the given credentials, AWS token, and VPS access — do you need anything else to run your solution?
Related categories:
Data Processing
Web Scraping
Data Extraction
Data Analysis
Automation
Data Collection
Data Management