Commercial Insurance Data Extraction to SQL
Budget: £20 – £250 GBP
I'm looking for a professional capable of extracting unstructured data from various commercial insurance documents. You will be working with Google Cloud to process these documents and then transferring the extracted data to SQL or a similar database.
Key Responsibilities:
- Document identification and data extraction from commercial insurance documents
- Targeted data points include: Policy details, Claims information, Customer details
- Processing via Google Cloud
- Storing extracted data in SQL or similar database
Ideal Skills:
- Proficiency in data extraction and processing
- Experience with Google Cloud, SQL
- Familiarity with commercial insurance documents
- Attention to detail and accuracy in data handling
Data Extraction Workflow for Insurance Documents
Project Goal:
Build a data extraction workflow to create a database of information from insurance documents. The workflow should handle both initial policy documents and mid-term adjustments.
Minimum Requirements:
Document AI: Proficiency in using Google Cloud's Document AI to extract data from PDF documents.
Data Storage: Experience with storing structured data in a database (e.g., Cloud SQL, BigQuery).
Data Manipulation: Familiarity with data manipulation techniques (e.g., using Python, SQL, or data transformation services) to append data to existing entries or create new rows.
Workflow Automation: Understanding of how to automate workflows using Google Cloud Functions or similar tools.
Data Validation: Ability to implement data validation checks to ensure data consistency and accuracy.
Error Handling: Experience with handling potential errors (e.g., document not found, data extraction errors) gracefully.
Workflow Steps:
Document Extraction:
Use Document AI to extract data from insurance documents (PDFs).
Identify key fields:
Policy Number
Policy Type
Insured Name
Postal Address
Business Description
Coverage Details (Sections, Limits, Exclusions)
Broker Information
Claims Information
Adjustment Reason (if applicable)
Data Storage:
Store extracted data in a structured format (e.g., CSV, JSON, or a database table).
Data Manipulation:
If the document is a "Mid Term Adjustment" or "MTA," create a new row in the database table.
Otherwise, append the extracted data to the existing row in the database table.
Additional Considerations:
Document Identification: Develop a mechanism to identify documents based on filename patterns, policy numbers, or document types.
Data Validation: Implement data validation checks to ensure data consistency and accuracy.
Error Handling: Handle potential errors gracefully.
Scalability: Design the workflow to handle large volumes of documents and data.
Google Cloud Tools:
Document AI: For document extraction.
Cloud Storage: For storing documents.
Cloud Functions: For automating the workflow.
Cloud SQL or BigQuery: For storing and analyzing the extracted data.
Deliverables:
Data Extraction Workflow: A fully functional workflow that extracts data from insurance documents and stores it in a database.
Database Schema: A well-defined database schema to store the extracted data.
Documentation: Clear documentation explaining the workflow, data schema, and any dependencies.
Note: This is a high-level overview. The specific implementation details will depend on the chosen tools and the complexity of the insurance documents.
1. New email arrives in O365 mailbox.
2. Cloud Function is triggered.
3. Cloud Function uses Microsoft Graph API to check if the email meets criteria (e.g., subject line, sender, attachment type).
4. If criteria are met, Cloud Function downloads the attachment.
5. Cloud Function uploads the attachment to Cloud Storage.
6. Document AI processor is triggered to extract data from the document in Cloud Storage.
Key Responsibilities:
- Document identification and data extraction from commercial insurance documents
- Targeted data points include: Policy details, Claims information, Customer details
- Processing via Google Cloud
- Storing extracted data in SQL or similar database
Ideal Skills:
- Proficiency in data extraction and processing
- Experience with Google Cloud, SQL
- Familiarity with commercial insurance documents
- Attention to detail and accuracy in data handling
Data Extraction Workflow for Insurance Documents
Project Goal:
Build a data extraction workflow to create a database of information from insurance documents. The workflow should handle both initial policy documents and mid-term adjustments.
Minimum Requirements:
Document AI: Proficiency in using Google Cloud's Document AI to extract data from PDF documents.
Data Storage: Experience with storing structured data in a database (e.g., Cloud SQL, BigQuery).
Data Manipulation: Familiarity with data manipulation techniques (e.g., using Python, SQL, or data transformation services) to append data to existing entries or create new rows.
Workflow Automation: Understanding of how to automate workflows using Google Cloud Functions or similar tools.
Data Validation: Ability to implement data validation checks to ensure data consistency and accuracy.
Error Handling: Experience with handling potential errors (e.g., document not found, data extraction errors) gracefully.
Workflow Steps:
Document Extraction:
Use Document AI to extract data from insurance documents (PDFs).
Identify key fields:
Policy Number
Policy Type
Insured Name
Postal Address
Business Description
Coverage Details (Sections, Limits, Exclusions)
Broker Information
Claims Information
Adjustment Reason (if applicable)
Data Storage:
Store extracted data in a structured format (e.g., CSV, JSON, or a database table).
Data Manipulation:
If the document is a "Mid Term Adjustment" or "MTA," create a new row in the database table.
Otherwise, append the extracted data to the existing row in the database table.
Additional Considerations:
Document Identification: Develop a mechanism to identify documents based on filename patterns, policy numbers, or document types.
Data Validation: Implement data validation checks to ensure data consistency and accuracy.
Error Handling: Handle potential errors gracefully.
Scalability: Design the workflow to handle large volumes of documents and data.
Google Cloud Tools:
Document AI: For document extraction.
Cloud Storage: For storing documents.
Cloud Functions: For automating the workflow.
Cloud SQL or BigQuery: For storing and analyzing the extracted data.
Deliverables:
Data Extraction Workflow: A fully functional workflow that extracts data from insurance documents and stores it in a database.
Database Schema: A well-defined database schema to store the extracted data.
Documentation: Clear documentation explaining the workflow, data schema, and any dependencies.
Note: This is a high-level overview. The specific implementation details will depend on the chosen tools and the complexity of the insurance documents.
1. New email arrives in O365 mailbox.
2. Cloud Function is triggered.
3. Cloud Function uses Microsoft Graph API to check if the email meets criteria (e.g., subject line, sender, attachment type).
4. If criteria are met, Cloud Function downloads the attachment.
5. Cloud Function uploads the attachment to Cloud Storage.
6. Document AI processor is triggered to extract data from the document in Cloud Storage.