Developing a Data Warehouse for Clinical Business Intelligence Using MIMIC4 Data
Budget: $30 – $250 USD
We are seeking an experienced data engineer to develop a data warehouse using MIMIC4 data, structured in Kimball’s star schema format. This project aims to enhance clinical business intelligence by organizing and optimizing the data for efficient querying and reporting.
Tasks:
Review Existing Data:
Analyze the MIMIC4 dataset to understand the structure and relationships between tables.
Identify key dimensions and facts relevant to clinical events and patient outcomes.
Design the Data Warehouse:
Propose a star schema design tailored to the clinical data, including necessary dimensions such as Hospitalization, Patient, Service Provider, and Concepts.
Ensure the design supports efficient querying and reporting for clinical analytics.
ETL Process Implementation:
Develop an ETL process within the stage_area environment to clean, transform, and load data from MIMIC4 into the star schema.
Create necessary tables and views to support the transformation and integration of data.
Testing and Validation:
Test the ETL process to ensure data integrity and accuracy in the warehouse.
Validate the final data warehouse against source data and project requirements.
Final Product Documentation:
Document the data warehouse design, including the schema, ETL process, and any transformations applied.
Provide a report detailing the challenges, limitations, and conclusions of the project.
Submit SQL scripts for creating the warehouse and an Excel file with aggregations relevant to clinical reporting.
Additional Requirements:
Star Schema Enhancement:
Consider adding additional dimensions or refining existing ones (e.g., Date, Junk dimensions) to optimize warehouse performance.
Source Data Integration:
Ensure compliance with the structure of the MIMIC4 dataset, with no alterations allowed in the original source environment.
Debugging and Optimization:
Utilize tools for debugging the ETL process to resolve any issues that arise during development.
Skills Required:
Strong proficiency in data engineering and SQL.
Experience with data warehousing, particularly in a healthcare context.
Solid understanding of ETL processes and schema design.
Familiarity with clinical datasets, especially MIMIC4, is a plus.
Ideal Candidate:
The ideal candidate should have a deep understanding of data warehousing principles, particularly in the context of clinical data. Experience with ETL processes and the ability to design efficient schemas for complex datasets is highly desired. Good documentation skills and the ability to explain the data transformation process clearly are also important.
Tasks:
Review Existing Data:
Analyze the MIMIC4 dataset to understand the structure and relationships between tables.
Identify key dimensions and facts relevant to clinical events and patient outcomes.
Design the Data Warehouse:
Propose a star schema design tailored to the clinical data, including necessary dimensions such as Hospitalization, Patient, Service Provider, and Concepts.
Ensure the design supports efficient querying and reporting for clinical analytics.
ETL Process Implementation:
Develop an ETL process within the stage_area environment to clean, transform, and load data from MIMIC4 into the star schema.
Create necessary tables and views to support the transformation and integration of data.
Testing and Validation:
Test the ETL process to ensure data integrity and accuracy in the warehouse.
Validate the final data warehouse against source data and project requirements.
Final Product Documentation:
Document the data warehouse design, including the schema, ETL process, and any transformations applied.
Provide a report detailing the challenges, limitations, and conclusions of the project.
Submit SQL scripts for creating the warehouse and an Excel file with aggregations relevant to clinical reporting.
Additional Requirements:
Star Schema Enhancement:
Consider adding additional dimensions or refining existing ones (e.g., Date, Junk dimensions) to optimize warehouse performance.
Source Data Integration:
Ensure compliance with the structure of the MIMIC4 dataset, with no alterations allowed in the original source environment.
Debugging and Optimization:
Utilize tools for debugging the ETL process to resolve any issues that arise during development.
Skills Required:
Strong proficiency in data engineering and SQL.
Experience with data warehousing, particularly in a healthcare context.
Solid understanding of ETL processes and schema design.
Familiarity with clinical datasets, especially MIMIC4, is a plus.
Ideal Candidate:
The ideal candidate should have a deep understanding of data warehousing principles, particularly in the context of clinical data. Experience with ETL processes and the ability to design efficient schemas for complex datasets is highly desired. Good documentation skills and the ability to explain the data transformation process clearly are also important.