Ingest on prem table to databricks from in prem audit file
Budget: $15 – $25 USD
I am looking for a freelancer to help me with ingesting an on-prem table to Databricks from an on-prem audit file. The project requirements are as follows:
Frequency of ingestion:
- The audit file needs to be ingested into Databricks on a daily basis.
Size of the audit file:
- The approximate size of the audit file is between 1GB and 10GB.
Desired output format in Databricks:
- The desired output format in Databricks is Delta Tables.
Ideal skills and experience for the job:
- Experience with ingesting data into Databricks from on-prem sources.
- Proficiency in working with Delta Tables in Databricks.
- Strong understanding of data ingestion processes and best practices.
- Familiarity with handling large file sizes for efficient data processing.
- Ability to set up automated ingestion processes for daily updates.
If you have the necessary skills and experience, please submit your proposal for this project.
Frequency of ingestion:
- The audit file needs to be ingested into Databricks on a daily basis.
Size of the audit file:
- The approximate size of the audit file is between 1GB and 10GB.
Desired output format in Databricks:
- The desired output format in Databricks is Delta Tables.
Ideal skills and experience for the job:
- Experience with ingesting data into Databricks from on-prem sources.
- Proficiency in working with Delta Tables in Databricks.
- Strong understanding of data ingestion processes and best practices.
- Familiarity with handling large file sizes for efficient data processing.
- Ability to set up automated ingestion processes for daily updates.
If you have the necessary skills and experience, please submit your proposal for this project.