Python script to clean up a list of urls from duplicates
Budget: $10 – $30 CAD
Need help with the following python code.
The goal is to compare two lists of URLs and creating a third list that doesn't contain duplicates.
Existing code:
1. Getting list from an XML file:
def get_urls_of_xml(xml_url):
r = requests.get(xml_url)
xml = r.text
soup = BeautifulSoup(xml)
links_arr = []
for link in soup.findAll('loc'):
linkstr = link.getText('', True)
#print(linkstr)
links_arr.append(linkstr)
return links_arr
2. Getting list from a database (i have this part of code, I can provide if needed)
db_urls_list = []
3. This is the part that i am missing:
Compare the list taken from a database and the list downloaded from XML and create a list that doesnt have duplicates.
The goal is to compare two lists of URLs and creating a third list that doesn't contain duplicates.
Existing code:
1. Getting list from an XML file:
def get_urls_of_xml(xml_url):
r = requests.get(xml_url)
xml = r.text
soup = BeautifulSoup(xml)
links_arr = []
for link in soup.findAll('loc'):
linkstr = link.getText('', True)
#print(linkstr)
links_arr.append(linkstr)
return links_arr
2. Getting list from a database (i have this part of code, I can provide if needed)
db_urls_list = []
3. This is the part that i am missing:
Compare the list taken from a database and the list downloaded from XML and create a list that doesnt have duplicates.