Need an updated web spider scraping program written for the following username DFUSER39164
Budget: $30 – $250 USD
I need a web scraper written for the URL:
https://www.mvlogistics.org/available-freight
The blue box that says "ZACH'S LOAD BOARD" will need to be clicked.
All information needed is available on the main page. The number of rows will vary. If the row is blank, skip that row.
The output should be a pipe (|) delimited file with the following column mappings:
origin_city --> data located in the "Pickup Location" column before the comma
origin_state --> data located in the "Pickup Location" column after the comma (data is the 2 letter state abbreviation after the comma)
ship_date --> data located in the "Pick Up Date" column, change to the YYYY-MM-DD format, if there is text other than a date add the current days date with the YYYY-MM-DD format
destination_city --> data located in the "Delivery Location" column before the comma
destination_state --> data located in the "Delivery Location" column after the comma (data is the 2 letter state abbvreviation after the comma)
receive_date --> data located in the "Delivery Date" column, change to the YYYY-MM-DD format, if there is text other than a date leave blank
trailer_type --> data located in the "Equipment" column
load_size --> add the text "Full"
weight --> data located in the "Per" column
length --> leave blank
width --> leave blank
height --> leave blank
trip_miles --> leave blank
pay_rate --> data located in the "Rate" column
contact_phone --> leave blank
contact_name --> data located in the "Dispatcher" column
tarp_required --> leave blank
comment --> data located in the "Type" and "Product" columns
load_number --> leave blank
commodity --> leave blank
The first line of the output should contain all of the column headers.
Any field that contains no data should be left blank.
Please do not use words like "null" or "blank" in blank columns.
Below is a sample output of the first 5 columns using sample data:
origin_city|origin_state|ship_date|destination_city|destination_state|
chicago|IL|2017-03-15|new york|NY|
kansas city|MO|2017-03-15|houston|TX|
The deliverable will be a Perl .pl file that must run on
Ubuntu Linux and must use Modern::Perl. The Perl .pl file
should be called 'DFUSER39164.pl' and the output file should be
called 'DFUSER39164.pipeSV'
It will be scheduled in cron to run unattended every 15 minutes.
Please specific what language/OS/modules you plan to use.
Also, please include the word "raccoon" in your bid so I know that
you read this description.
https://www.mvlogistics.org/available-freight
The blue box that says "ZACH'S LOAD BOARD" will need to be clicked.
All information needed is available on the main page. The number of rows will vary. If the row is blank, skip that row.
The output should be a pipe (|) delimited file with the following column mappings:
origin_city --> data located in the "Pickup Location" column before the comma
origin_state --> data located in the "Pickup Location" column after the comma (data is the 2 letter state abbreviation after the comma)
ship_date --> data located in the "Pick Up Date" column, change to the YYYY-MM-DD format, if there is text other than a date add the current days date with the YYYY-MM-DD format
destination_city --> data located in the "Delivery Location" column before the comma
destination_state --> data located in the "Delivery Location" column after the comma (data is the 2 letter state abbvreviation after the comma)
receive_date --> data located in the "Delivery Date" column, change to the YYYY-MM-DD format, if there is text other than a date leave blank
trailer_type --> data located in the "Equipment" column
load_size --> add the text "Full"
weight --> data located in the "Per" column
length --> leave blank
width --> leave blank
height --> leave blank
trip_miles --> leave blank
pay_rate --> data located in the "Rate" column
contact_phone --> leave blank
contact_name --> data located in the "Dispatcher" column
tarp_required --> leave blank
comment --> data located in the "Type" and "Product" columns
load_number --> leave blank
commodity --> leave blank
The first line of the output should contain all of the column headers.
Any field that contains no data should be left blank.
Please do not use words like "null" or "blank" in blank columns.
Below is a sample output of the first 5 columns using sample data:
origin_city|origin_state|ship_date|destination_city|destination_state|
chicago|IL|2017-03-15|new york|NY|
kansas city|MO|2017-03-15|houston|TX|
The deliverable will be a Perl .pl file that must run on
Ubuntu Linux and must use Modern::Perl. The Perl .pl file
should be called 'DFUSER39164.pl' and the output file should be
called 'DFUSER39164.pipeSV'
It will be scheduled in cron to run unattended every 15 minutes.
Please specific what language/OS/modules you plan to use.
Also, please include the word "raccoon" in your bid so I know that
you read this description.