python repo to scrap leboncoin.fr
Budget: €30 – €250 EUR
Hi,
I am looking for a Python developer to create a repository that will scrape data from [leboncoin.fr](http://leboncoin.fr/). The specific data I want to scrape includes listing details such as title, price, and location.
Here are more details :
--> Input : a name of a product, for example “playstation5” (url : https://www.leboncoin.fr/recherche?text=playstation5&page=1)
--> Output : all infos for each product contained in all searching pages. In other words, after typing “playstation5” on the search bar , I wish to fetch :
- the product name,
- price,
- location,
- post date,
- delivery type
- and category per product
for each product. No need to go in each product page, simply scraping the infos from the searching pages.
Those infos should be stored in a pandas dataframe (and if possible SQL database + CSV file in the file “data” from the repo)
--> The scraped data does not need to be integrated into other systems.
--> There are no existing Python scripts for this task, so the developer will need to start from scratch.
--> I wish to run this script twice a day for around 10 products (so fetching from 100 to 300 searching pages depending on the products for each run)
--> Able to run on VS code
--> Scraper should automatically pass all protection steps : captcha, etc. [leboncoin.fr](http://leboncoin.fr) uses Datadome!!
--> Possibility to select between : 1. using my IP 2. or a proxy
--> The script should use only free libraries (selenium, puppeteer, etc) and maintenance should have no costs. The only acceptable fees could be to create new proxies (from Bright Data for example) but I would be more than happy if you know a way to get free proxies.
--> Once the script is working, I wish to be able to get 2 extra fixes for the next 12 months in case the script crashes because of new security layers from [leboncoin.fr](http://leboncoin.fr) (new captcha, etc)
Thank you and let me know if any questions.
I am looking for a Python developer to create a repository that will scrape data from [leboncoin.fr](http://leboncoin.fr/). The specific data I want to scrape includes listing details such as title, price, and location.
Here are more details :
--> Input : a name of a product, for example “playstation5” (url : https://www.leboncoin.fr/recherche?text=playstation5&page=1)
--> Output : all infos for each product contained in all searching pages. In other words, after typing “playstation5” on the search bar , I wish to fetch :
- the product name,
- price,
- location,
- post date,
- delivery type
- and category per product
for each product. No need to go in each product page, simply scraping the infos from the searching pages.
Those infos should be stored in a pandas dataframe (and if possible SQL database + CSV file in the file “data” from the repo)
--> The scraped data does not need to be integrated into other systems.
--> There are no existing Python scripts for this task, so the developer will need to start from scratch.
--> I wish to run this script twice a day for around 10 products (so fetching from 100 to 300 searching pages depending on the products for each run)
--> Able to run on VS code
--> Scraper should automatically pass all protection steps : captcha, etc. [leboncoin.fr](http://leboncoin.fr) uses Datadome!!
--> Possibility to select between : 1. using my IP 2. or a proxy
--> The script should use only free libraries (selenium, puppeteer, etc) and maintenance should have no costs. The only acceptable fees could be to create new proxies (from Bright Data for example) but I would be more than happy if you know a way to get free proxies.
--> Once the script is working, I wish to be able to get 2 extra fixes for the next 12 months in case the script crashes because of new security layers from [leboncoin.fr](http://leboncoin.fr) (new captcha, etc)
Thank you and let me know if any questions.