Need a coder
Budget: ₹12,500 – ₹37,500 INR
Build a Metadriven DBT( Data build tool ) pipeline.
Metadata table in Databricks with Values(Database name, schema, Table Names, Columns, Load time(opt),config parameters.env name etc) will be populated.(Values to be decided later)
Similarly for connection details can be populated from table if we want to profiles.yml for DBT to pick env to run.
The Python utility will read the metadata table and build the schema.yml file with source configuration with Db name,Schema,column name populated for the DBT to pick and run and populate the staging environment or Silver layer.
The DBT model Standard across mostly will be referencing the schema.yml file and variables provided by users to build on the model.
Metadata table in Databricks with Values(Database name, schema, Table Names, Columns, Load time(opt),config parameters.env name etc) will be populated.(Values to be decided later)
Similarly for connection details can be populated from table if we want to profiles.yml for DBT to pick env to run.
The Python utility will read the metadata table and build the schema.yml file with source configuration with Db name,Schema,column name populated for the DBT to pick and run and populate the staging environment or Silver layer.
The DBT model Standard across mostly will be referencing the schema.yml file and variables provided by users to build on the model.