Small (3) Python with PostgreSQL functions
Budget: £10 – £20 GBP
Hi I have:
- three little Postgres tables (see attached files in CSV form so the developer can easily reproduce) users , meta, and details
- One python file with some functions (also attached)
- Data file ‘updates.csv’ to operate and test
I have a python script that needs 3 tasks - should be simple for an expert dev
In python script:
get_base_data() function gets the data form the tables
Note developer has to use PostgresSQL and set up tables by himself (import csvs)
Then use the way in the python script to get the data from tables
Fixed filter words = list of fixed words to filter the data
Function is_covid() filters the data according to the fixed filter words.
>>>TASK1 (code comment in python file):
#TASK1: Need new function here:
#"def is_important"
#loops over all "rid" datapoints in table meta if "rid" starts with "covid" or "covx"
#(note all "rid" start with ["topic"-"hash"] need to compare "topic here")
#if "rid" starts with "covid" or "covx" then add "text:" elements from column
#"keyword_filter" to "fixed_filter" list
#if other topic then takes all keywords from "text:" in table meta "keyword_filter"
# and run similar true/false check as is_covid function (for example on topic stroke)
#
#so for example loop over rid it finds the stroke-hash data point that means it is not
#covid so need to look at keyword_filter extract keywords from "text" in this case "stroke" and "heart"
#then store stroke & heart in a temp list and filter in function "is_important"
#keep in mind is_covid() loops over all covid related rows so only use covid filter
#all others need to be filtered by their respective keywords - so for a rid=stroke-... take
#the keywords from keyword_filter under "text:" and use this to filter in function is_important
>>>TASK2 (again comments in the python code at the relevant position):
#TASK 2: for all "rid" check if is_trained in table meta is set to 0 or 1
#loop through each rid if is_trained==0 then get ti & ab from table "details" for each rid and
#label from table meta
#and store in list such as example:
# data = [
#{'ti': 'Reactive stroma in cancer progression.',
# 'abs': ‘How reactive stroma affects cancer progression and how carcinoma regulates reactive stroma.',
# 'label': '1',
#},
# save data list as csv with: data.to_csv('data.csv', index=False)
>>>TASK3 (again in python file as code comment):
#TASK 3:
#check table meta then
#save all "rid" in the list
#if new data is added to the table for example a new rid that is not in the list
#then check table details on the new rid and email all rows where decision="include" in table details
#take email from table users and send email via sendmail to that address topic: "new data added"
# "on rid=stroke or rid=covid"
#then in the body of the email all rows of data from details where decision="include"
That’s it!
Note, I have more small tasks and if things go well I am looking to take the developer on for additional tasks.
- three little Postgres tables (see attached files in CSV form so the developer can easily reproduce) users , meta, and details
- One python file with some functions (also attached)
- Data file ‘updates.csv’ to operate and test
I have a python script that needs 3 tasks - should be simple for an expert dev
In python script:
get_base_data() function gets the data form the tables
Note developer has to use PostgresSQL and set up tables by himself (import csvs)
Then use the way in the python script to get the data from tables
Fixed filter words = list of fixed words to filter the data
Function is_covid() filters the data according to the fixed filter words.
>>>TASK1 (code comment in python file):
#TASK1: Need new function here:
#"def is_important"
#loops over all "rid" datapoints in table meta if "rid" starts with "covid" or "covx"
#(note all "rid" start with ["topic"-"hash"] need to compare "topic here")
#if "rid" starts with "covid" or "covx" then add "text:" elements from column
#"keyword_filter" to "fixed_filter" list
#if other topic then takes all keywords from "text:" in table meta "keyword_filter"
# and run similar true/false check as is_covid function (for example on topic stroke)
#
#so for example loop over rid it finds the stroke-hash data point that means it is not
#covid so need to look at keyword_filter extract keywords from "text" in this case "stroke" and "heart"
#then store stroke & heart in a temp list and filter in function "is_important"
#keep in mind is_covid() loops over all covid related rows so only use covid filter
#all others need to be filtered by their respective keywords - so for a rid=stroke-... take
#the keywords from keyword_filter under "text:" and use this to filter in function is_important
>>>TASK2 (again comments in the python code at the relevant position):
#TASK 2: for all "rid" check if is_trained in table meta is set to 0 or 1
#loop through each rid if is_trained==0 then get ti & ab from table "details" for each rid and
#label from table meta
#and store in list such as example:
# data = [
#{'ti': 'Reactive stroma in cancer progression.',
# 'abs': ‘How reactive stroma affects cancer progression and how carcinoma regulates reactive stroma.',
# 'label': '1',
#},
# save data list as csv with: data.to_csv('data.csv', index=False)
>>>TASK3 (again in python file as code comment):
#TASK 3:
#check table meta then
#save all "rid" in the list
#if new data is added to the table for example a new rid that is not in the list
#then check table details on the new rid and email all rows where decision="include" in table details
#take email from table users and send email via sendmail to that address topic: "new data added"
# "on rid=stroke or rid=covid"
#then in the body of the email all rows of data from details where decision="include"
That’s it!
Note, I have more small tasks and if things go well I am looking to take the developer on for additional tasks.