Publications

Cross-cultural competence in large language models (LLMs) requires understanding and adapting Culture-Specific Items (CSIs) across varying cultural contexts. However, progress in evaluating this capability remains limited by the lack of high-quality CSI-annotated corpora with parallel cross-cultural sentence pairs. We introduce XCR-Bench, a Cross(X)-Cultural Reasoning Benchmark containing 4.1k parallel sentences and 1,098 CSIs across three reasoning tasks. XCR-Bench integrates Newmark's CSI framework with Hall's Triad of Culture, enabling evaluation across levels of cultural visibility -- from observable practices to implicit social norms and values. Experiments on eight multilingual LLMs show that state-of-the-art models exhibit consistent weaknesses in identifying and adapting specific categories of CSIs, revealing a gap between surface-level recall and explicit cultural reasoning. Performance declines significantly on culturally sensitive categories and deeper cultural levels (p<0.005, 8/8 models), and adaptation quality varies systematically across target cultures and Bengali regional variants, indicating encoded regional and ethno-religious biases even within a single linguistic setting. We publicly release the corpus and code to support future research on cross-cultural NLP.

Inference-time scaling of LLM reasoning relies on a confidence score over each partial trace, yet existing estimators treat prefix quality as intrinsic to the prefix. A given prefix can succeed at very different rates and rankings under different continuation policies. Reasoning confidence is therefore not policy-invariant, and a squared-loss lower bound shows that policy-blind predictors retain residual error proportional to cross-policy disagreement. Calibration must condition on the continuation model and inference budget. To bridge this gap, we propose the Comparable Policy-Conditioned Calibrator (CPCC), built from a shared frozen-LLM prefix encoder, comparable scalar policy features under per-policy z-score normalization, and a Beta–Binomial head trained on bootstrap success counts. Across six continuation policies and three reasoning benchmarks (GSM8K, SVAMP, MATH-500), all evaluated under majority-vote aggregation, the CPCC family attains the lowest Brier on every evaluation cell, and CPCC is strongest on the held-out model families. It also avoids the held-out-family collapse of raw-hidden conditioning, lowers ECE by 13 to 26 percent on out-of-distribution math, and reduces downstream selective risk by 10 to 30 percent.

Generalized additive models (GAMs) offer high interpretability through independent univariate feature effects but underfit when interactions are present in data. GA2Ms add selected pairwise interactions, which improves accuracy, but sacrifices interpretability. We propose Conditionally Additive Local Models (CALMs), a new model class that narrows this trade-off: it stays close to GAMs in interpretability and close to GA2Ms in accuracy. CALMs allow multiple univariate shape functions per feature, each active in different regions of the input space. These regions are defined independently for each feature as simple logical conditions (thresholds) on the features it interacts with. As a result, effects remain locally additive while varying across subregions to capture interactions. We further propose a principled distillation-based training pipeline that identifies homogeneous regions with limited interactions and fits interpretable shape functions via region-aware backfitting. Experiments on diverse classification and regression tasks show that CALMs never underperform GAMs, often beat them and achieve accuracy broadly comparable to GA2Ms, while preserving the univariate auditability that GA2Ms forfeit.

Although Large Language Models (LLMs) exhibit remarkable adaptability across domains, these models often fall short in structured knowledge extraction tasks such as named entity recognition (NER). This paper explores an innovative, cost-efficient strategy to harness LLMs with modest NER capabilities for producing superior NER datasets. Our approach diverges from the basic class-conditional prompts by instructing LLMs to self-reflect on the specific domain, thereby generating domain-relevant attributes (such as category and emotions for movie reviews), which are utilized for creating attribute-rich training data. Furthermore, we preemptively generate entity terms and then develop NER context data around these entities, effectively bypassing the LLMs’ challenges with complex structures. Our experiments across both general and niche domains reveal significant performance enhancements over conventional data generation methods while being more cost-effective than existing alternatives.

This article presents the methodology and results of the joint collaboration between the AUEB NLP Group and another group from NKUArepresenting the Archimedes Unit in the 10th edition of the ImageCLEFmedical Caption evaluation campaign. Our participation focused exclusively on the Concept Detection task, which involves the automatic association of biomedical images with relevant medical concepts. In this task, our system achieved 2nd place overall with a primary F1 score of 0.5781 on the competition’s final test set. Building upon our previous work, we experimented with the same family of models, combining image encoders based on Convolutional Neural Networks (CNNs) with Feed-Forward Neural Network (FFNN) classifiers. To improve robustness and generalization, we developed several ensemble strategies that combine predictions from multiple architectures. We further incorporated conformal prediction to provide more reliable and uncertainty-aware predictions. This allowed our system to better account for prediction confidence. Overall, our approach demonstrates that carefully designed CNN-based classifiers, ensemble methods, and conformal prediction can form a competitive pipeline for biomedical image concept detection.