(NSFW) Python script to download files from a website that generates the URL based on button click
Budget: €8 – €30 EUR
Hi, I've done 2 prototypes in python that works partially, one uses requests and another uses Playwright, I will explain what's the problem with both of them, it seems that requests generates the URLs but when I add them to a download manager like jDownloader2, the first 1-2 URLs works but then the rest are invalid (404 error), I have no clue why this happens, I tried to change the download manager, the header, the cookies multiple times, but nothing works.
The one in Playwright seems to be working better it download like 20-30 files but it at the end there are always some timeout issue (caused by broken URLs sometimes) and sometimes it just crash killing everything, or it saves 0 bytes files.
The script objective is using a cookie and a txt file containing the list of items to download, the base URL is always the same just the code changes and there will be 1 x line in the txt file, the website generates the URL on button click, the URL can be used only 1 time, with requests to avoid invalidating the URL after the button press I've blocked the redirect to get an URL that once open with give the final correct URL, some URLs won't work (it seems they have server problems) and should be listed as invalid in another list if that's the case.
I can't share publicly any of the scripts I used to test as it uses a cookie connected to the account, but I will share them with the one I will pick to do the job.
The one in Playwright seems to be working better it download like 20-30 files but it at the end there are always some timeout issue (caused by broken URLs sometimes) and sometimes it just crash killing everything, or it saves 0 bytes files.
The script objective is using a cookie and a txt file containing the list of items to download, the base URL is always the same just the code changes and there will be 1 x line in the txt file, the website generates the URL on button click, the URL can be used only 1 time, with requests to avoid invalidating the URL after the button press I've blocked the redirect to get an URL that once open with give the final correct URL, some URLs won't work (it seems they have server problems) and should be listed as invalid in another list if that's the case.
I can't share publicly any of the scripts I used to test as it uses a cookie connected to the account, but I will share them with the one I will pick to do the job.