Projects

Studies how multimodal models break when inputs move beyond English and text. Introduced M2CQA, a culturally grounded benchmark built from images spanning 17 MENA countries with true and counterfactual statements in English, Arabic and its dialects, along with the CounterFactual Hallucination Rate (CFHR), which measures hallucination beyond raw accuracy. Also studies how model behavior shifts when the same input is spoken rather than read.

Developed English–Arabic translation models that outperformed commercial systems such as Google Translate, along with low-resource models that translate from English and Arabic into several Arabic dialects. The models were used by Fanar, QCRI's Arabic-centric generative AI platform, to generate synthetic data for pretraining and supervised fine-tuning.

Contributed to a framework for evaluating the zero- and few-shot capabilities of LLMs across NLP tasks. It offers generic dataset loaders, multiple model providers and standard metrics, and was tested with 53 datasets in 12 languages, yielding 200 ready-to-run benchmarking assets. Published at EACL 2024 (System Demonstrations).

Used LLMs to annotate latent concepts inside pretrained language models (EMNLP 2023), analyzed cross-lingual alignment in multilingual models (ACL 2024), devised synthetic-data methods to compare neuron-ranking algorithms, and extended the Latent Concepts Framework to multimodal and speech models (Interspeech 2025). Also built interactive visualization and annotation web apps to communicate findings.

Led the development of benchmarks for dialect comprehension and generation, built with machine translation plus human post-editing, and a fine-grained cultural-awareness benchmark covering the Gulf, Egypt and the Levant. Published at COLING 2025, with datasets and dialectal translation models released.