C# Web Scraper + PDFs

I’m building a small C# utility that will crawl a target site, pull out every piece of on-page information I need (both the visible text and the images), and also save any PDF that the page links to. The PDFs aren’t generated on the fly—each one is simply a normal hyperlink sitting in the HTML—so the job comes down to fetching the page, parsing for the data, spotting the *.pdf anchors, and downloading those files to disk.

You are free to approach this with HtmlAgilityPack, AngleSharp, HttpClient, Selenium, or any other .NET-friendly library you’re comfortable with, as long as the final code is clean, asynchronous where it makes sense, and easy to extend. I will pass in the root URL (or a list of starting URLs) plus an output folder path; the tool should handle the rest: navigate, extract, download, and save.

Deliverables
• Visual Studio solution (targeting .NET 6 or newer)
• Source code with clear comments showing where the crawling rules live
• A short README explaining how to run, configure, and extend the scraper
• Sample output folder demonstrating:
 - a text dump (JSON or CSV) of the on-page text you captured
 - the downloaded images, saved with sensible filenames
 - the PDFs pulled from the hyperlink targets

Acceptance criteria
• All linked PDFs reachable from the start page(s) are downloaded successfully.
• On-page text and images are captured without duplication.
• No hard-coded paths, credentials, or magic numbers.

If you have questions about rate limiting, authentication, or unusual HTML structures, just let me know and I’ll clarify quickly.