Script to extract text data from hundreds of files
Budget: $30 – $250 USD
I need a script written that can:
- Open a URL address.
- Download a .gz file at that address (approx 65k in size)
- Unzip the .gz file so as to access the HTML file inside
- Open the HTML file and extract all the URL text within the HTML file (it follows a similar pattern). This is the text between <loc>...</loc> in each line of the file.
- Import this URL text into a excel spreadsheet.
- open the next .gz file and copy the steps above
- repeat (loop) until the URL text in 763 .gz files has been extracted into excel.
- combine the information in as few excel files as possible.
- I should be able to run this script on my own laptop and leave it run whenever I want. The number of files may change over time, the script should know it has downloaded and extracted the information from the last file and then stop the script.
-
- Open a URL address.
- Download a .gz file at that address (approx 65k in size)
- Unzip the .gz file so as to access the HTML file inside
- Open the HTML file and extract all the URL text within the HTML file (it follows a similar pattern). This is the text between <loc>...</loc> in each line of the file.
- Import this URL text into a excel spreadsheet.
- open the next .gz file and copy the steps above
- repeat (loop) until the URL text in 763 .gz files has been extracted into excel.
- combine the information in as few excel files as possible.
- I should be able to run this script on my own laptop and leave it run whenever I want. The number of files may change over time, the script should know it has downloaded and extracted the information from the last file and then stop the script.
-