Web scraping publicly available site
Budget: $30 – $250 AUD
I'm looking for some help in creating a xl list with hyperlinks from publicly available data that is on a webpage. The webpage is only serving up 10 searches / pdf links at the time which is not very helpful for what I would like to do, which is create a list of all PDF files and their links.
A typical link is in this format below, which I have tried in xl and from doing some basic research expect is problematic for xl in being too long. It's also got some function that times out and has a credential check with AWS that returns an error message when tried i other platforms such as word etc...
https://qbcc-services-api-adjudication-decisions-production.s3.ap-southeast-2.amazonaws.com/00000001406600_1.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA54ULEG4LURTRQNJX%2F20221223%2Fap-southeast-2%2Fs3%2Faws4_request&X-Amz-Date=20221223T051031Z&X-Amz-Expires=120&X-Amz-Signature=513af3f48e24ff336c77ed63154292ad98ef7fe991c3538768b7709d8ba9cf69&X-Amz-SignedHeaders=host
The website I would like to scrape is https://my.qbcc.qld.gov.au/s/adjudication-registry
The site has a list of decisions in pdf that I would either like to make a list of with active links or alternatively download the pdfs.
A typical link is in this format below, which I have tried in xl and from doing some basic research expect is problematic for xl in being too long. It's also got some function that times out and has a credential check with AWS that returns an error message when tried i other platforms such as word etc...
https://qbcc-services-api-adjudication-decisions-production.s3.ap-southeast-2.amazonaws.com/00000001406600_1.pdf?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA54ULEG4LURTRQNJX%2F20221223%2Fap-southeast-2%2Fs3%2Faws4_request&X-Amz-Date=20221223T051031Z&X-Amz-Expires=120&X-Amz-Signature=513af3f48e24ff336c77ed63154292ad98ef7fe991c3538768b7709d8ba9cf69&X-Amz-SignedHeaders=host
The website I would like to scrape is https://my.qbcc.qld.gov.au/s/adjudication-registry
The site has a list of decisions in pdf that I would either like to make a list of with active links or alternatively download the pdfs.