Resources
Abstract

We describe the sixth edition of the CheckThat! lab, part of the 2023 Conference and Labs of the Evaluation Forum (CLEF). The five previous editions of CheckThat! focused on the main tasks of the information verification pipeline: check-worthiness, verifying whether a claim was fact-checked before, supporting evidence retrieval, and claim verification. In this sixth edition, we zoom into some new problems and for the first time we offer five tasks in seven languages: Arabic, Dutch, English, German, Italian, Spanish, and Turkish. Task 1 asks to determine whether an item —text or text plus image— is check-worthy. Task 2 aims to predict whether a sentence from a news article is subjective or not. Task 3 asks to assess the political bias of the news at the article and at the media outlet level. Task 4 focuses on the factuality of reporting of news media. Finally, Task 5 looks at identifying authorities in Twitter that could help verify a given target claim. For a second year, CheckThat! was the most popular lab at CLEF-2023 in terms of team registrations: 127 teams. About one-third of them (a total of 37) actually participated.

Citation

Alberto Barrón-Cedeño, Firoj Alam, Andrea Galassi, Giovanni Da San Martino, Preslav Nakov, Tamer Elsayed, Dilshod Azizov, Tommaso Caselli, Gullal S. Cheema, Fatima Haouari, Maram Hasanain, Mücahid Kutlu, Chengkai Li, Federico Ruggeri, Julia Maria Struß, and Wajdi Zaghouani. Overview of the CLEF-2023 checkthat! lab on checkworthiness, subjectivity, political bias, factuality, and authority of news articles and their source. In Avi Arampatzis, Evangelos Kanoulas, Theodora Tsikrika, Stefanos Vrochidis, Anastasia Giachanou, Dan Li, Mohammad Aliannejadi, Michalis Vlachos, Guglielmo Faggioli, and Nicola Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction - 14th International Conference of the CLEF Association, CLEF 2023, Thessaloniki, Greece, September 18-21, 2023, Proceedings, volume 14163 of Lecture Notes in Computer Science, pages 251–275. Springer, 2023.

@inproceedings{cedeno-etal-2023-clef-overview,
  author       = {Alberto Barr{\'{o}}n{-}Cede{\~{n}}o and
                  Firoj Alam and
                  Andrea Galassi and
                  Giovanni Da San Martino and
                  Preslav Nakov and
                  Tamer Elsayed and
                  Dilshod Azizov and
                  Tommaso Caselli and
                  Gullal S. Cheema and
                  Fatima Haouari and
                  Maram Hasanain and
                  M{\"{u}}cahid Kutlu and
                  Chengkai Li and
                  Federico Ruggeri and
                  Julia Maria Stru{\ss} and
                  Wajdi Zaghouani},
  editor       = {Avi Arampatzis and
                  Evangelos Kanoulas and
                  Theodora Tsikrika and
                  Stefanos Vrochidis and
                  Anastasia Giachanou and
                  Dan Li and
                  Mohammad Aliannejadi and
                  Michalis Vlachos and
                  Guglielmo Faggioli and
                  Nicola Ferro},
  title        = {Overview of the {CLEF-2023} CheckThat! Lab on Checkworthiness, Subjectivity,
                  Political Bias, Factuality, and Authority of News Articles and Their
                  Source},
  booktitle    = {Experimental {IR} Meets Multilinguality, Multimodality, and Interaction
                  - 14th International Conference of the {CLEF} Association, {CLEF}
                  2023, Thessaloniki, Greece, September 18-21, 2023, Proceedings},
  series       = {Lecture Notes in Computer Science},
  volume       = {14163},
  pages        = {251--275},
  publisher    = {Springer},
  year         = {2023},
  url          = {https://doi.org/10.1007/978-3-031-42448-9\_20},
  doi          = {10.1007/978-3-031-42448-9\_20}
}