Loading...

NIMZ at SemEval-2024 Task 9: Evaluating Methods in Solving Brainteasers Defying Commonsense

Rahimi, Z ; Sharif University of Technology | 2024

53 Viewed
  1. Type of Document: Article
  2. Publisher: 2024
  3. Abstract:
  4. The goal and dream of the artificial intelligence field have long been the development of intelligent systems or agents that mimic human behavior and thinking. Creativity is an essential trait in humans that is closely related to lateral thinking. The remarkable advancements in Language Models have led to extensive research on question-answering and explicit and implicit reasoning involving vertical thinking. However, there is an increasing need to shift focus towards research and development of models that can think laterally. One must step outside the traditional frame of commonsense concepts in lateral thinking to conclude. Task 9 of SemEval-2024 is Brainteaser (Jiang et al., 2024), which requires lateral thinking to answer riddle-like multiple-choice questions. In our study, we assessed the performance of various models for the Brainteaser task. We achieved an overall accuracy of 75% for the Sentence Puzzle subtask and 66.7% for the Word Puzzle subtask. All the codes, along with the links to our saved models, are available on our GitHub. © 2024 Association for Computational Linguistics
  5. Keywords:
  6. Brain models ; Computational linguistics ; Intelligent systems ; Brain teasers ; Evaluating method ; Human behaviors ; Human thinking ; Language model ; Lateral thinking ; Multiple-choice questions ; Question Answering ; Research and development ; Subtask ; Systems thinking
  7. Source: SemEval 2024 - 18th International Workshop on Semantic Evaluation, Proceedings of the Workshop ; 2024 , Pages 148-154 ; 979-889176107-0 (ISBN)
  8. URL: https://aclanthology.org/2024.semeval-1.23