Scrape the data of 1 website (GRI)
Budget: €30 – €250 EUR
The goal of this project is to deliver a python script capable of scraping the GRI website: https://database.globalreporting.org/search/. The script should be able to scrape all the details about all the orgs(15558).
>>Business requirements<<
For each org we need to have at least the following fields
-Name
-Description
-Size
-Type
-Listed status
-Sector
-Country
-Country status
-Employees
-Revenue
-GRI Community
-Stock listing code
-Contact
---Website
---Hq address
---HQ City
---Contact person
---Email
-Reports
---Downoad all the reports in a separate location
---The reports should be easily traceable back to the organisation
-Risk
-Opportunities
-Datetime of scraping
>>Technical requirements<<
-Mandatory Technical requirements
-The script should be written in python
-Last step of the scripts, should be pandas dataframe(s) dropping the data to csvs.
-Outputs must always have the same numbers of columns (columns migbt be empty)
-The script should allow for stopping and restart from lastest completed scrape (via table of content for example).
-The script should allow for the usage of proxies
-Script should be commented
>>Important technical requirements<<
-Script should used some “stealth” techniques (sleep, user-agent switches, …)
-Api scraping is preferred
-No selenium
>>Nice to have technical requirements<<
-Possibility to do parallel sraping with different proxies
>>Deliverables<<
-1(or more) Python scripts
-Csvs containing the data for the scripts scrapped (for verification)
>>Business requirements<<
For each org we need to have at least the following fields
-Name
-Description
-Size
-Type
-Listed status
-Sector
-Country
-Country status
-Employees
-Revenue
-GRI Community
-Stock listing code
-Contact
---Website
---Hq address
---HQ City
---Contact person
-Reports
---Downoad all the reports in a separate location
---The reports should be easily traceable back to the organisation
-Risk
-Opportunities
-Datetime of scraping
>>Technical requirements<<
-Mandatory Technical requirements
-The script should be written in python
-Last step of the scripts, should be pandas dataframe(s) dropping the data to csvs.
-Outputs must always have the same numbers of columns (columns migbt be empty)
-The script should allow for stopping and restart from lastest completed scrape (via table of content for example).
-The script should allow for the usage of proxies
-Script should be commented
>>Important technical requirements<<
-Script should used some “stealth” techniques (sleep, user-agent switches, …)
-Api scraping is preferred
-No selenium
>>Nice to have technical requirements<<
-Possibility to do parallel sraping with different proxies
>>Deliverables<<
-1(or more) Python scripts
-Csvs containing the data for the scripts scrapped (for verification)