QA for Information Extraction from Medical Transcripts
Budget: $40 – $60 USD
This project is quite open-ended: you will build a solution based on reading comprehension QA to extract specific information from medical transcriptions written by attending physicians. We are primarily interested in the following information:
• How old is the patient?
• Does the patient have any complaints?
• What is the reason for this consultation?
• What other symptoms does the patient have?
However, you are encouraged to extend this list and extract any other type of information that you find valuable. In the spirit of replicating a real-world project defined by a non-expert customer, this project provides you with significant freedom in terms of your approach: you may use BiDAF-like architectures or fine-tune any pre-trained transformer model that you wish. You may even use pre-tuned general purpose QA models like RoBERTA-QALinks to an external site. or BioBERT pre-tuned on SQuADLinks to an external site.. However, your final solution must satisfy 2 core requirements: (1) Exact match > 60% across all information types; and (2) F1 > 60% across all information types. There is another catch: our dataset does not come with annotations (i.e., ground truth)! Therefore, you will need to devise a solution for performance evaluation. You may decide to pick a small subset of the dataset and manually annotate its data points, or you may want to incorporate more elaborate solutions such as NER and dependency parsing to automatically annotate your dataset.
The dataset for this task is a scraped subset of the Medical TranscriptsLinks to an external site. repository. The scraped dataset is available on KaggleLinks to an external site., and is structured as follows:
A data frame with 4999 rows and 6 variables:
• note_id
o A unique identifier for each note
• description
o A description or chief concern
• medical_specialty
o Medical specialty of the note
• sample_name
o mtsamples.com note name
• transcription
o Transcription of note text
• keywords
o Keywords
The focus of this project is primarily on the transcription column. Here is an example of what your solution should be able to achieve for any given transcription:
• How old is the patient?
• Does the patient have any complaints?
• What is the reason for this consultation?
• What other symptoms does the patient have?
However, you are encouraged to extend this list and extract any other type of information that you find valuable. In the spirit of replicating a real-world project defined by a non-expert customer, this project provides you with significant freedom in terms of your approach: you may use BiDAF-like architectures or fine-tune any pre-trained transformer model that you wish. You may even use pre-tuned general purpose QA models like RoBERTA-QALinks to an external site. or BioBERT pre-tuned on SQuADLinks to an external site.. However, your final solution must satisfy 2 core requirements: (1) Exact match > 60% across all information types; and (2) F1 > 60% across all information types. There is another catch: our dataset does not come with annotations (i.e., ground truth)! Therefore, you will need to devise a solution for performance evaluation. You may decide to pick a small subset of the dataset and manually annotate its data points, or you may want to incorporate more elaborate solutions such as NER and dependency parsing to automatically annotate your dataset.
The dataset for this task is a scraped subset of the Medical TranscriptsLinks to an external site. repository. The scraped dataset is available on KaggleLinks to an external site., and is structured as follows:
A data frame with 4999 rows and 6 variables:
• note_id
o A unique identifier for each note
• description
o A description or chief concern
• medical_specialty
o Medical specialty of the note
• sample_name
o mtsamples.com note name
• transcription
o Transcription of note text
• keywords
o Keywords
The focus of this project is primarily on the transcription column. Here is an example of what your solution should be able to achieve for any given transcription: