Scrape tables and text from public website
Budget: $20 – $100 USD
I need python code for extracting tables and text from specific web pages. I will provide the web pages in a zip file.
For each html page, there will be multiple output files:
- For each table within the html:
o Identify name of table, use this as name of below two output files
o Table_<name>.csv : the table extracted in csv format
o Table_<name>.json : the table extracted in json format
- All text in the html page to be extracted in json format without the tables such that
o SectionName : is based on the name within table of contents
o SectionText : is based on the text without the tables, page numbers within the section
o Example : [ { “SectionName” : <name of section from table of contents>,
“SectionText” : <text within the section without tables and page numbers>}
]
o Output file name : Text.json
For the project to successfully complete, I would need following:
- Python code that generates above output files. Code must use popular html parsing libraries like BeautifulSoup etc and should be independent of the specific html file being looked at.
- All above outputs to be provided back as zip file
Examples of html pages here :
https://www.sec.gov/Archives/edgar/data/1713683/000171368320000184/zs_fy2020xproxy.htm
https://www.sec.gov/Archives/edgar/data/0000320193/000119312521001987/d767770ddef14a.htm
For the selected candidate, I will provide a larger set of these saved and zipped htmls which are to be used as inputs
For each html page, there will be multiple output files:
- For each table within the html:
o Identify name of table, use this as name of below two output files
o Table_<name>.csv : the table extracted in csv format
o Table_<name>.json : the table extracted in json format
- All text in the html page to be extracted in json format without the tables such that
o SectionName : is based on the name within table of contents
o SectionText : is based on the text without the tables, page numbers within the section
o Example : [ { “SectionName” : <name of section from table of contents>,
“SectionText” : <text within the section without tables and page numbers>}
]
o Output file name : Text.json
For the project to successfully complete, I would need following:
- Python code that generates above output files. Code must use popular html parsing libraries like BeautifulSoup etc and should be independent of the specific html file being looked at.
- All above outputs to be provided back as zip file
Examples of html pages here :
https://www.sec.gov/Archives/edgar/data/1713683/000171368320000184/zs_fy2020xproxy.htm
https://www.sec.gov/Archives/edgar/data/0000320193/000119312521001987/d767770ddef14a.htm
For the selected candidate, I will provide a larger set of these saved and zipped htmls which are to be used as inputs