Web Scraping a featured image and a video from Crowdfunding website
Budget: $30 – $250 USD
Project Description:
I am looking for a freelancer who can help me with web scraping on Kickstarter. I have specific projects in mind that I would like to scrape the featured image and video from. The scraped content needs to be formatted in a specific way.
Skills and Experience:
- Strong experience in web scraping
- Proficiency in Python or a similar programming language
- Knowledge of HTML and CSS
- Attention to detail to ensure accurate scraping and formatting
- Ability to work independently and meet deadlines
Definition:
- Featured image: the image file at the front page of each project; need to scrape one for each project
- Featured video: the explanatory video on the front page of each project. Some projects do not have this.
Details:
- I have all the URLs of around 40,000 crowdfunding projects. I can provide this data.
- I also have the Python code that you can refer to. This file contains xpath information for images and videos. You can see which part you need to focus on the HTML page.
- I tried web-scraping by myself, but the website is protected by Cloudflare. I need anti-bot features and premium proxies, but I do not have those.
- The task is simple:
* For each project, download one featured image on the front page of the crowdfunding project. The title should be "project id number_img.avif." For example, if the project id number is 1234567, the file should be 1234567_img.avif. I checked that all the images are avif type.
* For each project, download one featured video (if available) at the front page of the crowdfunding project. The title should be "project id number_vid.mp4." I checked that all the videos are mp4 type.
* Overall, I need to collect all the featured image files and all available featured video files for 40,000 projects.
* My dataset includes all the information of projects about the project id number, URL, and whether the project has a video.
- This task may require several hundred GB to save the image and video files.
- This is a one-time task—no need to scrape this regularly.
For your reference, I attached the zip file.
The data file (.csv) has three columns: URL, id (project id number), and video (whether the project has a featured video or not)
In the attached Python code, you can find xpath information for featured images and videos.
Please note that I need to collect only one main image file and one video file (if available) for each project.
For example, the project id numbers of the first and second projects in the data are 558744194 and 547078236, respectively.
The first one has only the featured image on the project's main page (558744194_img.avif) and does not have the video.
The second one has both the featured image and video. So, as you see, I could get one image file for the first one. And I could get one image file (547078236_img.avif) and 1 video file (547078236_vid.mp4) for the second one.
Based on this information, could you please give me the price quote and delivery timeline?
I am looking for a freelancer who can help me with web scraping on Kickstarter. I have specific projects in mind that I would like to scrape the featured image and video from. The scraped content needs to be formatted in a specific way.
Skills and Experience:
- Strong experience in web scraping
- Proficiency in Python or a similar programming language
- Knowledge of HTML and CSS
- Attention to detail to ensure accurate scraping and formatting
- Ability to work independently and meet deadlines
Definition:
- Featured image: the image file at the front page of each project; need to scrape one for each project
- Featured video: the explanatory video on the front page of each project. Some projects do not have this.
Details:
- I have all the URLs of around 40,000 crowdfunding projects. I can provide this data.
- I also have the Python code that you can refer to. This file contains xpath information for images and videos. You can see which part you need to focus on the HTML page.
- I tried web-scraping by myself, but the website is protected by Cloudflare. I need anti-bot features and premium proxies, but I do not have those.
- The task is simple:
* For each project, download one featured image on the front page of the crowdfunding project. The title should be "project id number_img.avif." For example, if the project id number is 1234567, the file should be 1234567_img.avif. I checked that all the images are avif type.
* For each project, download one featured video (if available) at the front page of the crowdfunding project. The title should be "project id number_vid.mp4." I checked that all the videos are mp4 type.
* Overall, I need to collect all the featured image files and all available featured video files for 40,000 projects.
* My dataset includes all the information of projects about the project id number, URL, and whether the project has a video.
- This task may require several hundred GB to save the image and video files.
- This is a one-time task—no need to scrape this regularly.
For your reference, I attached the zip file.
The data file (.csv) has three columns: URL, id (project id number), and video (whether the project has a featured video or not)
In the attached Python code, you can find xpath information for featured images and videos.
Please note that I need to collect only one main image file and one video file (if available) for each project.
For example, the project id numbers of the first and second projects in the data are 558744194 and 547078236, respectively.
The first one has only the featured image on the project's main page (558744194_img.avif) and does not have the video.
The second one has both the featured image and video. So, as you see, I could get one image file for the first one. And I could get one image file (547078236_img.avif) and 1 video file (547078236_vid.mp4) for the second one.
Based on this information, could you please give me the price quote and delivery timeline?