Python code - Schedule Job in Airflow!

Job ID: 30810131

Budget: $10 – $25 USD

-- TASK GOAL --

-> Python Code attached
-> Airflow put up easy steps guide attached



Take the existing Python code attached and make necessary changes and than schedule a daily job in Airflow
* Implement DAG and Orchestration functions and schedule the job in Airflow.

** More details - Python Code **
Today, the code stores the web-scrapped files into a folder and also into a dataframe
- Keep that logic
But implement also the logic, where:
- It stores the files into a database (Postgree)
- check first if the file exists in the database
- if it doesn't , than include the new record

** If you don't have Airflow environment installed
- we advise to put the environment up in docker
- simply docker-compose up from the official build here
https://github.com/apache/airflow/blob/main/docs/apache-airflow/start/docker-compose.yaml


** You will be responsible to guide our team on the airflow job deployment **
*** Partials executions of this task wont be accepted
We are a very small team and Airflow is new to us.
We wont be able to fix or continue any implementation.
** So please don't bid if you cant execute it from beginning to the end.