Help with Scrapy Web Scraper
Budget: €8 – €30 EUR
I have a scrapy script that is supposed to scrape data on a website.
The problem is that when I scrape the products of a website that contains 52 products, unfortunately I don't get all of them saved to my file but only 24 products from the page.
However, the problem only exists when I scrape the prices, if I omit the prices then all products are scraped. Therefore the error must lie somewhere in the code that scrapes the prices.
If anyone can help, please let me know.
Two files attached
File 1: Here all products are scraped + results
File 2: Here half of the products are missing after I scrape the prices + results
The problem is that when I scrape the products of a website that contains 52 products, unfortunately I don't get all of them saved to my file but only 24 products from the page.
However, the problem only exists when I scrape the prices, if I omit the prices then all products are scraped. Therefore the error must lie somewhere in the code that scrapes the prices.
If anyone can help, please let me know.
Two files attached
File 1: Here all products are scraped + results
File 2: Here half of the products are missing after I scrape the prices + results