MongoDB + Elasticsearch Sync Tool

We are looking for an expert in MongoDB + ElasticSearch infrastructure. We are looking to build from the ground up, in the cloud (AWS), a very complex infra with 500M Person object, 50M Company, and dozens more challenges. The dataset will be always evoling, with hundreds of thousands of updates a day on every object etc.

Please see the screenshot that explain the technical setup wanted: [https://gyazo.com/3d1a60fbe593df1ec1d59ef68a8c7d3f](https://gyazo.com/3d1a60fbe593df1ec1d59ef68a8c7d3f)

The final goal is to have a full blown searchable database, with list feature, and respond time always staying below a clean 100ms.

The main DB will be Mongo, that needs to be optimized (shard, cluster config, index...) properly. On top of that DB will live Elastisearch, which will require a near real-time and bulletproof syncing between the two.

The server budget expenses is 5-10k$ which should be enough to sustain such a setup.

You will have to:

Comment and detail what you think about our current thought-out infra strategy (see attached)
Make large scale testing by generating hundreds of millions of fake elements
Perform load tests on the infra you built
Build a primary API to perform deep searches


All of that in the cloud on AWS - noting that if ElasticSearch is for example not available, we can use OpenSearch or other. What matters is the final results, and to try to stick as much as possible with the proposed diagram.

PLEASE: send over your thoughts about the diagram, as well as previous experience dealing with such challenges etc, and if we like your approach, we will be in touch!