Apache Spark development to sync data into elastic search from multiple table from postgreSql
Budget: $30 – $250 USD
Virtual Dataset syncing through Spark.
Spark job to written in Scala/Java which will retrieve the data from multiple tables with complex join conditions. There will be multiple stages before the final outcome and each stage can transformed and will be input to next stage for joining with other table/previous output.
Request will have all the metadata in JSON format for the stages involved in the virtual dataset and there detailed joins as well as transformation conditions.
Screenshots attached for more understanding.
Spark job to written in Scala/Java which will retrieve the data from multiple tables with complex join conditions. There will be multiple stages before the final outcome and each stage can transformed and will be input to next stage for joining with other table/previous output.
Request will have all the metadata in JSON format for the stages involved in the virtual dataset and there detailed joins as well as transformation conditions.
Screenshots attached for more understanding.
Related categories:
Business, Accounting, Human Resources & Legal
MySQL
Hadoop
Map Reduce
PostgreSQL