Fix/Repair a Perl script scraping a website
Budget: $30 – $250 USD
I am looking for a freelancer to fix/repair a Perl script that scrapes a website. The current issue is that the script is no longer working, with the last successful scrape being on May 22. The website's layout or structure has not changed since the script was last working. The desired output format for the scraped data is CSV.
Ideal skills and experience for this job include:
- Expertise in Perl programming
- Experience in web scraping
- Knowledge of CSV file format
- Ability to troubleshoot and fix errors in code
- Attention to detail to ensure all desired data is retrieved
- Strong communication skills to provide updates on progress and ask any necessary questions.
Notes on Existing Code:
The code worked fine, now no results. The programmer who worked on it is not replying to any messages. He made no comments within the code. While I understand basic perl, I do not understand all of modules he is using.
I do not know if the code on the webpage has changed or if the website is blocking my IP address. I was not using a VPN. The script worked good for a few weeks. When it stopped working I installed a VPN hoping that it would fix the problem. It did not.
What do I want?
1. That as you read the code to make a short comment on what 'this loop' or some 'while or for' stmt is doing.
2. If the website is blocking the script that in the script will print to the command window a message saying something to that effect.
3. That within the code there is a flag that allows me to save the raw html code that is downloaded to a text file.
4. I have the option (within the code) to sign it using my user id and password, however I only want to use this if the script is being blocking my IP.
5. If a regex fails a message prints to the command window with the line number of the script which has the regex.
The modules being used:
use WWW::Mechanize;
use JSON::XS;
use Text::CSV_XS;
use HTML::TreeBuilder::LibXML;
use DateTime;
use DateTime::Format::Strptime;
use feature qw(say);
use IO::Handle;
use Getopt::Long;
use HTML::FormatText;
use HTML::TreeBuilder;
Thank you for you interest.
Ideal skills and experience for this job include:
- Expertise in Perl programming
- Experience in web scraping
- Knowledge of CSV file format
- Ability to troubleshoot and fix errors in code
- Attention to detail to ensure all desired data is retrieved
- Strong communication skills to provide updates on progress and ask any necessary questions.
Notes on Existing Code:
The code worked fine, now no results. The programmer who worked on it is not replying to any messages. He made no comments within the code. While I understand basic perl, I do not understand all of modules he is using.
I do not know if the code on the webpage has changed or if the website is blocking my IP address. I was not using a VPN. The script worked good for a few weeks. When it stopped working I installed a VPN hoping that it would fix the problem. It did not.
What do I want?
1. That as you read the code to make a short comment on what 'this loop' or some 'while or for' stmt is doing.
2. If the website is blocking the script that in the script will print to the command window a message saying something to that effect.
3. That within the code there is a flag that allows me to save the raw html code that is downloaded to a text file.
4. I have the option (within the code) to sign it using my user id and password, however I only want to use this if the script is being blocking my IP.
5. If a regex fails a message prints to the command window with the line number of the script which has the regex.
The modules being used:
use WWW::Mechanize;
use JSON::XS;
use Text::CSV_XS;
use HTML::TreeBuilder::LibXML;
use DateTime;
use DateTime::Format::Strptime;
use feature qw(say);
use IO::Handle;
use Getopt::Long;
use HTML::FormatText;
use HTML::TreeBuilder;
Thank you for you interest.