Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval

Overview of the CLEF-2025 CheckThat! Lab: Subjectivity, fact-checking, claim normalization, and retrieval

This paper presents the eighth edition of the CheckThat! lab, part of the 2025 Conference and Labs of the Evaluation Forum (CLEF).

September 2025 · Alberto Barrón-Cedeño, Firoj Alam, Julia Maria Struß, Preslav Nakov, Tanmoy Chakraborty, Tamer Elsayed, Piotr Przybyła, Tommaso Caselli, Giovanni Da San Martino, Fatima Haouari, Maram Hasanain, Chengkai Li, Jakub Piskorski, Federico Ruggeri, Xingyi Song, Reem Suwaileh
Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates

Overview of MM-ArgFallacy2025 on Multimodal Argumentative Fallacy Detection and Classification in Political Debates

We present an overview of the MM-ArgFallacy2025 shared task on Multimodal Argumentative Fallacy Detection and Classification in Political Debates, co-located with the 12th Workshop on Argument Mining at ACL 2025.

July 2025 · Eleonora Mancini, Federico Ruggeri, Serena Villata, Paolo Torroni
Overview of the CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness

Overview of the CLEF-2024 CheckThat! Lab: Check-Worthiness, Subjectivity, Persuasion, Roles, Authorities, and Adversarial Robustness

We describe the seventh edition of the CheckThat! lab, part of the 2024 Conference and Labs of the Evaluation Forum (CLEF).

September 2024 · Alberto Barrón-Cedeño, Firoj Alam, Julia Maria Struß, Preslav Nakov, Tanmoy Chakraborty, Tamer Elsayed, Piotr Przybyła, Tommaso Caselli, Giovanni Da San Martino, Fatima Haouari, Maram Hasanain, Chengkai Li, Jakub Piskorski, Federico Ruggeri, Xingyi Song, Reem Suwaileh
Promoting the Responsible Development of Speech Datasets for Mental Health and Neurological Disorders Research

Promoting the Responsible Development of Speech Datasets for Mental Health and Neurological Disorders Research

We chart the landscape of available speech datasets for this domain, to highlight possible pitfalls and opportunities for improvement and promote fairness and diversity.

August 2024 · Eleonora Mancini, Ana Tanevska, Andrea Galassi, Alessio Galatolo, Federico Ruggeri, Paolo Torroni
A Corpus for Sentence-Level Subjectivity Detection on English News Articles

A Corpus for Sentence-Level Subjectivity Detection on English News Articles

We develop novel annotation guidelines for sentence-level subjectivity detection, which are not limited to language-specific cues.

May 2024 · Francesco Antici, Federico Ruggeri, Andrea Galassi, Katerina Korre, Arianna Muti, Alessandra Bardi, Alice Fedotova, Alberto Barrón-Cedeño
PejorativITy: Disambiguating Pejorative Epithets to Improve Misogyny Detection in Italian Tweets

PejorativITy: Disambiguating Pejorative Epithets to Improve Misogyny Detection in Italian Tweets

We present PejorativITy, a novel corpus of 1,200 manually annotated Italian tweets for pejorative language at the word level and misogyny at the sentence level.

May 2024 · Arianna Muti, Federico Ruggeri, Cagri Toraman, Alberto Barrón-Cedeño, Samuel Algherini, Lorenzo Musetti, Silvia Ronchi, Gianmarco Saretto, Caterina Zapparoli
Disruptive situation detection on public transport through speech emotion recognition

Disruptive situation detection on public transport through speech emotion recognition

We propose to frame disruptive situation detection as a speech emotion recognition task. To validate our hypotheses, we carry out an extensive experimental study focusing on the development of a model characterized by speaker/gender independence, robustness to noise, and robustness against multiple voices.

March 2024 · Eleonora Mancini, Andrea Galassi, Federico Ruggeri, Paolo Torroni
Multimodal Fallacy Classification in Political Debates

Multimodal Fallacy Classification in Political Debates

We propose multimodal argument mining for argumentative fallacy classification in political debates.

March 2024 · Eleonora Mancini, Federico Ruggeri, Paolo Torroni
Argumentation Structure Prediction in CJEU Decisions on Fiscal State Aid

Argumentation Structure Prediction in CJEU Decisions on Fiscal State Aid

We study how propositions are combined inhigher-level structures and how the relations between propositionscan be predicted by NLP models.

September 2023 · Piera Santin, Giulia Grundler, Andrea Galassi, Federico Galli, Francesca Lagioia, Elena Palmieri, Federico Ruggeri, Giovanni Sartor, Paolo Torroni
Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source

Overview of the CLEF–2023 CheckThat! Lab on Checkworthiness, Subjectivity, Political Bias, Factuality, and Authority of News Articles and Their Source

We describe the seventh edition of the CheckThat! lab, part of the 2024 Conference and Labs of the Evaluation Forum (CLEF).

September 2023 · Alberto Barrón-Cedeño, Firoj Alam, Andrea Galassi, Giovanni Da San Martino, Preslav Nakov, Tamer Elsayed, Dilshod Azizov, Tommaso Caselli, Gullal S Cheema, Fatima Haouari, Maram Hasanain, Mucahid Kutlu, Chengkai Li, Federico Ruggeri, Julia Maria Struß, Wajdi Zaghouani
A Dataset of Argumentative Dialogues on Scientific Papers

A Dataset of Argumentative Dialogues on Scientific Papers

We introduce ArgSciChat, a dataset of 41 argumentative dialogues between scientists on 20 NLP papers.

July 2023 · Federico Ruggeri, Mohsen Mesgar, Iryna Gurevych
Detecting and explaining unfairness in consumer contracts through memory networks

Detecting and explaining unfairness in consumer contracts through memory networks

We posit that in this domain useful explanations of classifier outcomes can be provided by resorting to legal rationales.

October 2022 · Federico Ruggeri, Francesca Lagioia, Marco Lippi, Paolo Torroni
Detecting Arguments in CJEU Decisions on Fiscal State Aid

Detecting Arguments in CJEU Decisions on Fiscal State Aid

We present Demosthenes, a novel corpus for argument mining in legal documents, composed of 40 decisions of the Court of Justice of the European Union on matters of fiscal state aid.

October 2022 · Giulia Grundler, Piera Santin, Andrea Galassi, Federico Galli, Francesco Godano, Francesca Lagioia, Elena Palmieri, Federico Ruggeri, Giovanni Sartor, Paolo Torroni
Multimodal Argument Mining: A Case Study in Political Debates

Multimodal Argument Mining: A Case Study in Political Debates

We propose a study on multimodal argument mining in the domain of political debates

October 2022 · Eleonora Mancini, Federico Ruggeri, Andrea Galassi, Paolo Torroni
Argument mining as rapid screening tool of COVID-19 literature quality: Preliminary evidence

Argument mining as rapid screening tool of COVID-19 literature quality: Preliminary evidence

We develop an artificial intelligence system for the analysis of the scientific literature by leveraging on recent developments in the field of Argument Mining.

July 2022 · Gianfranco Brambilla, Antonella Rosi, Francesco Antici, Andrea Galassi, Daniele Giansanti, Fabio Magurano, Federico Ruggeri, Paolo Torroni, Evaristo Cisbani, Marco Lippi
AMICA: An Argumentative Search Engine for COVID-19 Literature

AMICA: An Argumentative Search Engine for COVID-19 Literature

AMICA is an argument mining-based search engine, specifically designed for the analysis of scientific literature related to Covid-19.

July 2022 · Marco Lippi, Francesco Antici, Gianfranco Brambilla, Evaristo Cisbani, Andrea Galassi, Daniele Giansanti, Fabio Magurano, Antonella Rosi, Federico Ruggeri, Paolo Torroni