Loading...

Using bottleneck adapters to identify cancer in clinical notes under low-resource constraints

Rohanian, O ; Sharif University of Technology | 2023

706 Viewed
  1. Type of Document: Article
  2. DOI: 10.18653/v1/2023.bionlp-1.5
  3. Publisher: Association for Computational Linguistics (ACL) , 2023
  4. Abstract:
  5. Processing information locked within clinical health records is a challenging task that remains an active area of research in biomedical NLP. In this work, we evaluate a broad set of machine learning techniques ranging from simple RNNs to specialised transformers such as BioBERT on a dataset containing clinical notes along with a set of annotations indicating whether a sample is cancer-related or not. Furthermore, we specifically employ efficient fine-tuning methods from NLP, namely, bottleneck adapters and prompt tuning, to adapt the models to our specialised task. Our evaluations suggest that fine-tuning a frozen BERT model pre-trained on natural language and with bottleneck adapters outperforms all other strategies, including full fine-tuning of the specialised BioBERT model. Based on our findings, we suggest that using bottleneck adapters in low-resource situations with limited access to labelled data or processing capacity could be a viable strategy in biomedical text mining. The code used in the experiments are going to be made available at [LINK ANONYMIZED]. © 2023 Association for Computational Linguistics
  6. Keywords:
  8. Source: Proceedings of the Annual Meeting of the Association for Computational Linguistics ; 2023 , Pages 62-78 ; 0736587X (ISSN); 978-195942985-2 (ISBN)
  9. URL: https://aclanthology.org/2023.bionlp-1.5.pdf