Web Scrapping with Python
Budget: $250 – $750 USD
We are looking for a python developer who can develop a python module that initially contains two functions:
1)Function name: "getUrlContent"
Parameter: URL (string)
Returns dataframe: the URL's title, header, and content (text only)
Business logic: it is required the results is cleaned, processed with all HTML tags removed.
2) Function name "getUrlNetworkContent"
Parameter: URL, number of links (numlinks).
Steps:
- Call getUrlContent passing URL parameter and obtain title, header, and content.
- Identify the first numlinks URLs in the content string obtained from the previous step.
- Call the getUrlContent passing each time a different URL collected in the previous step
- Return the results in a data frame which will have 4 columns (URL, title, head, content) and numlinks+1 rows
I want the module written in lambda so it can be called from an endpoint
1)Function name: "getUrlContent"
Parameter: URL (string)
Returns dataframe: the URL's title, header, and content (text only)
Business logic: it is required the results is cleaned, processed with all HTML tags removed.
2) Function name "getUrlNetworkContent"
Parameter: URL, number of links (numlinks).
Steps:
- Call getUrlContent passing URL parameter and obtain title, header, and content.
- Identify the first numlinks URLs in the content string obtained from the previous step.
- Call the getUrlContent passing each time a different URL collected in the previous step
- Return the results in a data frame which will have 4 columns (URL, title, head, content) and numlinks+1 rows
I want the module written in lambda so it can be called from an endpoint