Vera Demberg

Topic:

description

Michael Hahn

Topic: TBD

TBD

Simon Ostermann

Topic: TBD

TBD

Joseph van Genabith

Topic: TBD

TBD

Chair

Katja Konermann

I am in the Master's program Language Science and Technology and work at the German National Library.

Vice Chair

Misha Sonkin

2nd year student at LST, HiWi at project A8

Administrative Assistant

Bri Lehman

2nd year Masters LST student, focusing on psycholinguistics

Andreas Rouvalis

Couchsurfing and Sponsorship Coordinator

Daniil Gurgurov

Master’s Student in Language Science and Technology, Research Assistant at German Research Center for AI (DFKI)

Sponsorship and Social Media Coordinator

Katharina Trinley

1st year Master's student in Language Science and Technology

Beach

William LaCroix

Second year Master's student in Language Science and Technology, focusing on natural language processing and machine learning. Working as a data science research assistant with the German Research Center for Artificial Intelligence.

Helper

Marc Altmeyer

I'm a former Coli student from Saarbrücken, who is helping the TaCoS Team with organizing to provide you with a great conference experience.

Opening Session

Participants

Katja Konermann, Misha Sonkin, Bri Lehman

Room

Room 1.17, C7.4

Language Processing in Humans and Machines

Participants

Prof. Dr. Michael Hahn

Room

Room 1.17, C7.4

Slides

The field of Natural Language Processing (NLP) has displayed rapid progress, exemplified by large-scale language models such as ChatGPT, displaying striking capabilities and promising diverse applications. However, these systems remain 'black boxes', often failing in inexplicable ways, which hinders their reliability and trustworthiness in real-world applications. I will describe foundational research conducted in our lab that aims to address this challenge. First, to pinpoint limitations of existing systems, I will describe theoretical tools for analyzing the capabilities and limitations of the transformer architecture underlying modern language models. Second, in order to understand how machines could be made more human-like, I will describe new machine learning-based models of human language comprehension. I will conclude by outlining directions for future research.

Keynote: Pragmatic Processing in Humans and Large Language Models

Participants

Prof. Dr. Vera Demberg

Room

Room 1.17, C7.4

Slides

Keynote: Pragmatic Processing in Humans and Large Language Models

Participants

Dr. Simon Ostermann

Room

Room 1.17, C7.4

Slides

"Equal goes it loose"

Participants

Leonie Harter

Room

Room 2.07.1, A2.2

Slides

Auf Grund ihrer Nichtkompositionalität stellen Redewendungen (Idiome) immer noch eine Herausforderung für neuronale maschinelle Übersetzungsmodelle dar. Einige vorhergehende Arbeiten zu diesem Thema, die sich mit der Fähigkeit von Transformern zur Übersetzung von Redewendungen beschäftigen, schlagen vor, diese zu verbessern, indem parallel zum Übersetzen von der Quellsprache zur Zielsprache auch das Detektieren von Idiomen in den Quellsätzen trainiert wird. Daher teste ich in meiner Arbeit, ob dieser Multi-Task-Learning-Ansatz dem Transformer wirklich beim Übersetzen von Idiomen hilft. Dazu habe ich verschiedene Modelle entwickelt. Zwei Modelle trainieren einfach ein zusätzliches Teilmodul darauf, Idiome in den Quellsätzen zu finden. Basierend auf den Untersuchungsergebnissen von Dankers et al. zum Verhalten der Attentionmechanismen des Transformers, wenn dieser Idiome richtig übersetzt, habe ich zusätzlich vier weitere Modelle getestet, die die Ausgabe des Idiomtagging-Moduls verwenden, um die Attentionmuster entsprechend zu modifizieren. Im BLEU-Score übertreffen alle sechs Multi-Task-Learning-Modelle den Vanilla-Transformer auf meinen Testdaten, besonders die beiden Modelle, die die Cross-Attention zwischen Decoder und Encoder modifizieren. Beim Übersetzen von Idiomen übertreffen aber leider nur die beiden Modelle, die die Attentionmuster nicht modifizieren und eins der Cross-Attention-Modelle den Vanilla-Transformer. Meine Ergebnisse lassen darauf schließen, dass die Idiomtagging-Module der anderen Modelle nicht gut genug aus den Silbertrainingsdaten, die ich für das Trainieren von Idiomerkennung erstellt habe, generalisieren können und die Verbesserung dieser Fähigkeit auch zur Verbesserung ihrer Idiomübersetzungen führen würde.

Computational approaches to verbal aspect

Participants

Samuel Innes

Room

Room 2.16, A2.2

Verbal aspect is one of the most studied areas in theoretical linguistics, and yet there has been very little discussion of this topic in the NLP community to date. This talk aims to present some computational perspectives on verbal aspect and show how tools from computational linguistics can contribute to our understanding of aspect as a linguistic phenomenon. More broadly, it will explore how modern advances in NLP, such as language models, can contribute to the field of linguistics and provide a third way between corpus-based and introspection-based research methods.

Limitations of Code Benchmarks

Participants

Jan Kels

Room

Room 2.07.1, A2.2

Slides

Virtually all papers or "technical reports" for new (large) language models make claims about model performance across a variety of tasks. Coding ability is usually indicated by unit-test benchmarks like *HumanEval* or *MBPP*. Arriving at a single number to indicate a models ability to code comes with several pitfalls. We explore how common evaluations work and focus on their shortcomings. Recent literature suggest several ideas on how these evaluations can be improved beyond generic problems. Finally we propose our own approach to evaluate models for code generation, which goes beyond functional correctness and investigates model creativity. We hope to bring a greater understanding of these benchmarks by providing very illustrative examples and highlight trends.

Multilingual Sentiment Analysis: Fine-Tuning Pre-Trained Models for Mizo Sentiment Classification

Participants

Ravi Kiran Chikkala

Room

Room 2.16, A2.2

In this project I wanted to check if the current multi-lingual pretrained models XLM-RoBERTa, m-Bert and Indic Bert can be used for the Indian low-resource languages on which the pretrained-models were not trained on? back-translation technique data augmentation technique was used. The goal was to determine if including back-translated data can enhance the performance of the model and to identify which pretrained multilingual model provided better sentiment classification for Mizo. I conducted three experiments by using the four Indian languages English, Telugu,Hindi and Mizo.The data for English, Telugu, and Hindi were obtained from existing sources, while Mizo data was collected by scraping it from various YouTube videos using its API. This data was then cleaned and sentiment labels were given by native mizo speakers. Synthetic data was created by using Google Translate. In the first experiment Multi-lingual pretrained models were fine-tuned with the training data from each language without any synthetic data.In the second experiment Multi-lingual pretrained models were fine-tuned using back-translated data. Finally in the third experiment Mizo data of experiment one was concatenated with the best performing back translated data of experiment two to check if the model’s performance increases or not. Two hyperparameters, batch size and number of epochs, were considered for building the Sentiment classifiers for Mizo. Finally, the models were evaluated using two metrics: accuracy and F1 score.

Probe Ability Theory

Participants

Misha Sonkin, Katja Konermann

Room

Room 2.07.1, A2.2

What knowledge is stored in Language models? How can we extract it? In this workshop, we will introduce you to probing; a method to explore what information could implicitly be stored in Language Models such as BERT. We will talk about the applications and limitations of probing, and run some probing experiments ourselves. Bring your laptop if you want to take a crack at it yourself! No prior programming skills required.
Slides are available here

Revisiting nltk-drt: DRT extension for NLTK updated and improved

Participants

Fedor Sizov

Room

Room 2.16, A2.2

Slides

Under the supervision of Ivan Rygaev, I've improved and updated the "nltk-drt" project (Kislev, Makarov, Li 2010), focusing on extending support for Discourse Representation Theory (DRT) within NLTK for Python. I restored compatibility with Python 3.x, optimized the code, and migrated the test suite to pytest, getting 100% of tests working. Additionally, I enabled the definition of presupposition schemes via external grammars, facilitating the incorporation of new functionalities like it-clefts and factive verbs. Our improved extension is now available on GitHub, with plans to submit a high-quality pull request to integrate it into the nltk library, aiming to benefit the broader NLP open-source community.

Surprisal from Large Language Models

Participants

AriaRay Brown, Julius Steuer

Room

Room 2.07.1, A2.2

During the tutorial, students will work with a corpus created for cognitive modeling to reproduce a part of the experiments in [1]. A toolkit will be provided for visualizing hypotheses of surprisal. There will be opportunities to engage in technical code and theoretical exploration. Students are expected to be familiar with running Jupyter Notebooks and should bring a working laptop with internet connection. As background information or a refresher to the topic, we will provide 2-3 optional but interesting readings. (Enough to find out more on why, "researchers may have been aligning LLMs to not the exact model of humans but rather a superhuman chat agent with instruction tuning; our results might reflect the paradox—pursuing human preferences has resulted in creating something different from humans [2].") Background: This tutorial (originally designed and taught by Julius Steuer, LSV research group) has existed as part of the Information Theory block course offered in 2023 and 2024 to graduate students and researchers in the Language Science and Technology department. We are happy to offer the tutorial at TaCoS for those who may be interested in furthering their knowledge and skills in this area. [1] Oh, Byung-Doh, and William Schuler. "Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?," 2022. https://doi.org/10.48550/ARXIV.2212.12131. [2] Kuribayashi, Tatsuki, Yohei Oseki and Timothy Baldwin. "Psychometric Predictive Power of Large Language Models," 2023. https://doi.org/10.48550/arXiv.2311.07484.

Computer-aided assessment of speech fluency of preschool children

Participants

Valentin Kany

Room

Room 2.16, A2.2

The study introduced in my talk investigates the extent to which the scripts for the automatic assessment of various aspects of speech fluency by de Jong et al. (2021), which are widely used in the field of phonetics, are suitable for assessing the language proficiency of preschool children and how this method could be improved. As part of this study, speech data from preschool children with German as L1 as well as children with German as L2 were elicited by means of a serious game. The audio data were annotated with respect to articulation rate, pauses, and filler particles. The annotation was conducted automatically by the scripts on one hand, and manually by human raters on the other hand to evaluate the scripts' performance.
The results show a fairly high agreement between the measurement of articulation rate by the scripts and the manual measurement. Thus, the scripts could be used and integrated into language proficiency assessments for this purpose.
Further, the automatic detection of speech pauses showed a high precision value and could therefore also be utilised in language proficiency assessments. Such use would benefit from the manual annotation method for disfluent and non-disfluent pauses presented in this study.
Regarding the filler particles, the automatic classification by the introduced scripts proved to be less suitable since no high level of agreement was found with the human annotation. In order to be used in practice, this method still needs to be extended, for example by including the automatic pause detection of the scripts.

Using information theory to explain language change

Participants

Hannes Düe

Room

Room 2.16, A2.2

Whether a language uses prepositions or postpositions correlates with whether it uses Verb-Object (VO) or Object-Verb (OV) word order. Proto-Uralic is a typical OV language. Finnish, which descends from Proto-Uralic, is typologically unusual in that it generally uses VO order and postpositions. It also features bipositions, i.e. adpositions that can appear in either position. In my presentation, I will discuss historical word order change from the perspective of Dependency Length Minimization (DLM) Theory and diachronic typology. Then I will very briefly present my analysis of a Finnish dependency corpus and show that (1) only using prepositions would make Finnish more efficient according to DLM theory, and that (2) bipositions are mostly used as prepositions when it is efficient to do so. I hypothesize that Finnish favors postpositions due to specific grammaticalization processes but is undergoing change toward a more preposition-heavy system.

Learn about the RTG Neuroexplicit Models

Participants

Alexander Koller

Room

Room 2.16, A2.2

We develop novel models that combine neural and human-interpretable ("explicit") components to accurately solve tasks in natural language processing, computer vision, and action-decision making. These include neurosymbolic models, which combine neural and symbolic elements.
We are a Research Training Group: A team of up to 24 PhD students under the supervision of 13 internationally renowned professors. Each student is funded for four years and will freely choose their research topic within the wide scope of the RTG. Students participate in a qualification program designed to build a tight-knit group that can collaborate across research fields and together explore the deeper principles of effective neuroexplicit models.

Academia or Industry? A Computational Linguist's Perspective

Participants

Torsten Kai Jachmann, Yulia Zaitova, Manuel John, Akash Kumar Gautam

Room

Room 1.17, C7.4

Have you ever been worried about the future? This round table is for you! Our guests from academia and industry discuss different career options for computational linguists. Join us!

Transformers or RNNs or SSMs: Who's more sensitive?

Participants

Yana Veitsman, Yash Sarrof, Mark Rofin

Room

Room 2.07.1, A2.2

Slides

As of now, for most researchers Transformers are the building block of choice in making foundation models. However, other architectures, such as RNNs, can still do something much better. For example, they are better at learning formal languages. What does it mean for architectures that are completely based on Transformers? If this question intrigues you, then join us in our workshop, where we will delve into the question and try to gain a deeper understanding of the limits of foundation models. We will start with a hands on session introducing several formal languages, deciphering Parity, Dyck, Tomitas and others together. Then we will share our research group's recent work on the topic, i.e. "studying the struggle of Transformers in learning highly sensitive functions." In continuation, we will unpack the concept of sensitivity, its connection to formal languages, and contrast the performance of Transformers and LSTMs in this regard. Finally, we will wrap up discussing the concept of State Space Models (SSMs) as an alternative to LSTMs and Transformers as well as discuss the recent buzzy variant of Mamba and its potentially best approach to both worlds given our formal language and sensitivity framework.

"Bogue" statt "Bug": Eine typologische Analyse der französischen Ersatzwörter für Anglizismen

Participants

Paulina Hussein

Room

Room 2.16, A2.2

In Frankreich sind Schutz und „Reinheit“ der französischen Sprache ein staatliches Anliegen, mit dessen Umsetzung zahlreiche Institutionen betraut sind, darunter die Académie française, die seit Jahrhunderten als Hüterin der französischen Sprache gilt. Hauptanliegen sind nicht nur die Situierung der französischen Sprache und Varietäten, sondern auch der französische Wortschatz, der sich auf die eine oder andere Weise an die neuen Gegebenheiten einer sich verändernden Welt anpasst. Dabei stehen innersprachliche Neuerungen (Neologie) im Spannungsverhältnis mit meist anglo-amerikanischen Entlehnungen. Entwicklung, Prüfung, und Veröffentlichung der Neologismen wird vom französischen Sprachbereicherungsorgan, dem dispositif d’enrichissement de la langue française umgesetzt, dessen Hierarchie die Académie française abschließt. Diese Arbeit stellt eine typologische Analyse der veröffentlichten offiziellen Terminologie dar. Betrachtet wurden Neologismen des Sport-, Medien- und Wissenschaftsvokabulars. Als Ersatzwörter für zu vermeidende anglo-amerikanische Äquivalente lassen sich diese Termini verschiedentlich begreifen: Als innerfranzösische Neologismen, also etwa Kompositionen und Derivationen, als Übertragung oder Übersetzung des zu ersetzenden Worts, und als Anglizismus, dessen Vermeidung durch die terminologische Arbeit eigentlich angestrebt wird. Anhand dieser Kategorisierungen sollen quantitative und qualitative Analysen angestellt und Tendenzen der institutionellen Terminologie vor dem Hintergrund der Arbeitsweise und der Hauptakteure des dispositif d’enrichissement de la langue française diskutiert werden.
Die quantitative Analyse zeigt, dass bei der offiziellen Terminologie im Sportvokabular vermehrt auf französische Lexeme zurückgegriffen wird, die durch semantische Neologie eine weitere Bedeutungsdimension erhalten sollen. Anglizismen in der Medienterminologie werden zumeist mittels Kompositionen paraphrasiert; was den semantischen und stilistischen Bedürfnissen der Sprecher in den seltensten Fällen gerecht wird. In der Wissenschaft dagegen kommen gehäuft integrierte Anglizismen und Lehnübersetzungen vor. Die etwa 50% der offiziellen Neologismen, die sich als unerwünschte Anglizismen qualifizieren, deuten darauf hin, dass die Académie française zugunsten der Tendenzen des Sprachgebrauchs Abstriche bei ihren eigentlich souveränen Auswahlkriterien macht. Eine Abstandnahme vom verbissenen Kampf gegen den sich verändernden Sprachgebrauch und vordergründig gegen anglo-amerikanische Entlehnungen ist wohl nicht anzunehmen.

Tutorial Bash

Participants

Aikaterini Azoidou

Room

Room 2.16, A2.2