Big Query near realtime sync with MongoDB

Job ID: 33163426

Budget: $30 – $250 USD

We need to use MongoDB CDC and send the data (inserts/ updates/ deletes) realtime to Big Query for analytics purposes.

We need to keep big query in sync with MongoDB.

Solution should be using Terraform. Transformations should be done in Python.

A flow chart is attached on how it should work, if you have a better suggestion, we are open to it.
Related categories: Python Data Processing BigQuery Big Data Terraform