Need a web spider scraping program written for the following username LANDSTARAE -- 2
Budget: $30 – $250 USD
I need a web scraper written for the following url:
https://www.landstaronline.com/loadspublic
Separate searches for each state and each Canadian province will need to be performed to scrape all the data needed. The searches will need to be performed in the Origin search box. Each state and province will need to be separately entered in the "State or City, State" option under Origin. Also, for each search under Limits: the Max. Results needs to be changed to 500. All searches will then need combined to one file.
States: Alabama, Alaska, Arizona, Arkansas, California, Colorado, Connecticut, Delaware, Florida, Georgia, Hawaii, Idaho, Illinois, Indiana, Iowa, Kansas, Kentucky, Louisiana, Maine, Maryland, Massachusetts, Michigan, Minnesota, Mississippi, Missouri, Montana, Nebraska, Nevada, New Hampshire, New Jersey, New Mexico, New York, North Carolina, North Dakota, Ohio, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, South Dakota, Tennessee, Texas, Utah, Vermont, Virginia, Washington, West Virginia, Wisconsin, Wyoming
Provinces: British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec
All of the information needed for EACH search will be available on the main page. The number of rows will vary.
The output should be a pipe (|) delimited file with the following column mappings:
origin_city --> data located in the "Origin Destination" column, data will be on the top line of each row located before the comma
origin_state --> data located in the "Origin Destination" column, data will be on the top line of each row located after the comma, data will be the 2 letter state/province abbreviation
ship_date --> data located in the "Pickup From/Thru Delivery From/Thru" column, data will be on the top line of each row, changed to the YYYY-MM-DD format, data listed after the date is not needed
destination_city --> data located in the "Origin Destination" column, data will be on the bottom line of each row located before the comma
destination_state --> data located in the "Origin Destination" column, data will be on the bottom line of each row locate after the comma, data will be the 2 letter state/province abbreviation
receive_date --> data located in the "Pickup From/Thru Delivery From/Thru" column, data will be on the bottom line of each row, changed to the YYYY-MM-DD format, data listed after the date is not needed
trailer_type --> data located in the "Trlr Type" column
load_size --> data located in the "Weight Type" column, data will be on the bottom line of each row
weight --> data located in the "Weight Type" column, data will be on the top line of each row
length --> Leave blank
width --> leave blank
height--> leave blank
trip_miles --> leave blank
pay_rate --> leave blank
contact_phone --> data located in the "Agency Contact #" column, data will be on the bottom line of each row
contact_name --> data located in the "Agency Contact #" column, data will be on the top line of each row
tarp_required --> leave blank
comment --> leave blank
load_number --> leave blank
commodity --> data located in the "Cmdty Code" column
The first line of the output should contain all of the column headers.
Any field that contain no data should be left blank.
Please do not use words like "null" or "blank" in blank columns.
Below is a sample output of the first 5 columns using sample data:
origin_city|origin_state|ship_date|destination_city|destination_state|
chicago|IL|2017-03-15|new york|NY|
kansas city|MO|2017-03-15|houston|TX|
The deliverable will be a Perl .pl file that must run on
Ubuntu Linux and must use Modern::Perl. The Perl .pl file
should be called 'LANDSTARAE.pl' and the output file should be
called 'LANDSTARAE.pipeSV'
It will be scheduled in cron to run unattended every 10 minutes.
Please specify what language/OS/modules you plan to use.
Also, please include the word "raccoon" in your bid so I know that
you read this description.
https://www.landstaronline.com/loadspublic
Separate searches for each state and each Canadian province will need to be performed to scrape all the data needed. The searches will need to be performed in the Origin search box. Each state and province will need to be separately entered in the "State or City, State" option under Origin. Also, for each search under Limits: the Max. Results needs to be changed to 500. All searches will then need combined to one file.
States: Alabama, Alaska, Arizona, Arkansas, California, Colorado, Connecticut, Delaware, Florida, Georgia, Hawaii, Idaho, Illinois, Indiana, Iowa, Kansas, Kentucky, Louisiana, Maine, Maryland, Massachusetts, Michigan, Minnesota, Mississippi, Missouri, Montana, Nebraska, Nevada, New Hampshire, New Jersey, New Mexico, New York, North Carolina, North Dakota, Ohio, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, South Dakota, Tennessee, Texas, Utah, Vermont, Virginia, Washington, West Virginia, Wisconsin, Wyoming
Provinces: British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec
All of the information needed for EACH search will be available on the main page. The number of rows will vary.
The output should be a pipe (|) delimited file with the following column mappings:
origin_city --> data located in the "Origin Destination" column, data will be on the top line of each row located before the comma
origin_state --> data located in the "Origin Destination" column, data will be on the top line of each row located after the comma, data will be the 2 letter state/province abbreviation
ship_date --> data located in the "Pickup From/Thru Delivery From/Thru" column, data will be on the top line of each row, changed to the YYYY-MM-DD format, data listed after the date is not needed
destination_city --> data located in the "Origin Destination" column, data will be on the bottom line of each row located before the comma
destination_state --> data located in the "Origin Destination" column, data will be on the bottom line of each row locate after the comma, data will be the 2 letter state/province abbreviation
receive_date --> data located in the "Pickup From/Thru Delivery From/Thru" column, data will be on the bottom line of each row, changed to the YYYY-MM-DD format, data listed after the date is not needed
trailer_type --> data located in the "Trlr Type" column
load_size --> data located in the "Weight Type" column, data will be on the bottom line of each row
weight --> data located in the "Weight Type" column, data will be on the top line of each row
length --> Leave blank
width --> leave blank
height--> leave blank
trip_miles --> leave blank
pay_rate --> leave blank
contact_phone --> data located in the "Agency Contact #" column, data will be on the bottom line of each row
contact_name --> data located in the "Agency Contact #" column, data will be on the top line of each row
tarp_required --> leave blank
comment --> leave blank
load_number --> leave blank
commodity --> data located in the "Cmdty Code" column
The first line of the output should contain all of the column headers.
Any field that contain no data should be left blank.
Please do not use words like "null" or "blank" in blank columns.
Below is a sample output of the first 5 columns using sample data:
origin_city|origin_state|ship_date|destination_city|destination_state|
chicago|IL|2017-03-15|new york|NY|
kansas city|MO|2017-03-15|houston|TX|
The deliverable will be a Perl .pl file that must run on
Ubuntu Linux and must use Modern::Perl. The Perl .pl file
should be called 'LANDSTARAE.pl' and the output file should be
called 'LANDSTARAE.pipeSV'
It will be scheduled in cron to run unattended every 10 minutes.
Please specify what language/OS/modules you plan to use.
Also, please include the word "raccoon" in your bid so I know that
you read this description.