Create data warehouse to store real time semi-structured data and automatically convert to CSV
Budget: $250 – $750 USD
I have around 150 MB data in text format transferred from SFTP everyday. need an expert to help me design and complete below data life circle
Ideal scenario to manage these data is to
1, Store the text files into cloud space real time, whenever data is ready, it can load into bucket directly
2, Convert text files to structured data (CSV format will be the best) to be able to use SQL query to join other tables and do more further analysis.
3, Plan a back up for all the raw data
After seeing cover letter, I will send some data file as example to get clearer picture on the expectation.
Ideal scenario to manage these data is to
1, Store the text files into cloud space real time, whenever data is ready, it can load into bucket directly
2, Convert text files to structured data (CSV format will be the best) to be able to use SQL query to join other tables and do more further analysis.
3, Plan a back up for all the raw data
After seeing cover letter, I will send some data file as example to get clearer picture on the expectation.