Progress Note Understanding: Assessment and Plan Reasoning 

Tentative Timeline

Background

Healthcare providers generate notes in the electronic health records (EHRs) to document the daily progress and plan. The Subjective, Objective, Assessment and Plan (SOAP) format is ubiquitous and an essential component in medical training to properly document care in the daily progress notes. The Assessment and Plan sections of the progress notes are the free-text fields where healthcare providers identify patients’ problems/diseases and treatment plans. The Assessment section summarizes the patients’ active health problems or diseases for that day. The Plan section consists of multiple subsections, each addressing a specific problem or diagnosis followed by a detailed treatment plan (e.g., Respiratory failure with MRSA pneumonia: continue seven days of vancomycin, continue mechanical ventilation with lung-protective strategy, wean oxygen). Patients typically have a main diagnosis or problem with associated conditions or downstream health effects. Healthcare providers can implicitly understand what parts of a treatment plan directly target the primary concerns of a series of health problems based on their background knowledge and medical reasoning, but this information is not easily available for downstream analysis. Automatic extraction of this kind of information can potentially be useful for downstream use cases such as problem list generation.

Task Descriptions

We formulate the physicians’ reasoning process as a relation prediction task between assessment and every plan subsection. Specifically, each plan subsection will be labeled with one of the four relations indicating the nature of its association with the assessment:

1. Direct, if the plan subsection targets the primary diagnoses or problem
2. Indirect, if the plan subsection lists problems/diagnoses (e.g., organ failure) associated with the primary diagnosis/problem, including other diagnoses/problems that are not part of the main diagnosis.   
3. Neither, if the plan subsection mentions a problem/diagnosis that is not mentioned in that day’s progress note. 
4. Not Relevant, if the plan subsection does not include a problem/diagnosis

We will provide pairs of assessment section and plan subsection (A&P pair). Given each A&P pair, the goal is to predict the correct relation from the four labels. An example A&P pair is provided in the figure below:

Data

This task utilizes progress notes from MIMIC-III Dataset sampled from all the intensive care units (https://physionet.org/content/mimiciii/1.4/). A team of medical knowledge experts annotated the progress notes for this task. Sample input and output data may be found at https://git.doit.wisc.edu/YGAO/N2C2-AP-Reasoning.

The data must be obtained through N2C2. All participants for this track must agree to MIMIC’s data usage agreement.

Evaluation

We will use Macro F1 as the evaluation metric. System outputs are expected to be submitted in the exact format of the ground truth annotations provided by the organizers.

Dissemination

Participants are asked to submit a 500-word long abstract describing their methodologies. Abstracts may also have a graphical summary of the proposed architecture. The document should not exceed 2 pages (i.e., 1.5 line spacing, 12pt-font size, excluding spaces for figures). The authors of either top-performing systems or particularly novel approaches will be invited to present or demonstrate their systems at the workshop. A journal venue will be organized following the workshop.

Contact

Please join the discussion group below for announcements. Questions about the challenge can be addressed to the organizers by posting to the group (New Topic button) or sending email to the address below.
Google Group: N2C2_UW_ICU_DataSci_2022_AP_Task@googlegroups.com

References

Gao, Yanjun, Dmitriy Dligach, Timothy Miller, Samuel Tesch, Ryan Laffin, Matthew M. Churpek, and Majid Afshar. "Hierarchical Annotation for Building A Suite of Clinical Natural Language Processing Tasks: Progress Note Understanding." arXiv preprint arXiv:2204.03035 (2022).