LLM decision-making evaluation
Evaluation frameworks for multi-agent LLM systems deployed in labor-market and social-media recommender pipelines.
See papers
Senior Research Scientist
Social Computing Group, Department of Informatics, University of Zurich
I evaluate and audit AI systems that make consequential decisions, from who gets hired to what we see on social media.
My work sits at the intersection of Responsible AI and AI safety: measuring how large language models and multi-agent systems behave, where they fail and whom they disadvantage, and designing evaluations that make them fairer and more reliable.
I am an AI Research Scientist with a decade of experience in AI research and three years in production software engineering. My background is mathematical (game theory, dynamical systems and multi-agent strategic interaction), and I apply it to auditing economically consequential LLM decisions, from labor markets to social-media recommender systems.
I developed inverse-optimization frameworks for inferring agents' preferences, established the first formal taxonomy of feedback loops and biases in ML systems, and designed scalable LLM evaluation and auditing frameworks, fine-tuning workflows and RAG architectures across models and platforms. I was also part of the team that won a gold medal in the 2024 Kaggle ARC Challenge, one of AI's hardest benchmarks for abstract reasoning and few-shot generalization.
Published inPNAS · Nature Communications · Scientific Reports · npj Climate Action · COLM · EMNLP · ACM RecSys · ACM EAAMO · IEEE CDC · L4DC · ECAF
Evaluation frameworks for multi-agent LLM systems deployed in labor-market and social-media recommender pipelines.
See papersCan LLMs stand in for people? Benchmarks that tell human from AI-generated social-media content at scale.
See papersFormalizing bias risks and feedback loops in AI-assisted decision-making and recommendation.
See papersGame-theoretic and control-theoretic models of strategic behaviour on social networks.
See papersResearch
Interactive
A live version of the meritocratic network formation model (Nature Communications, 2021). Each user discovers one creator per step and follows them only if they are better than everyone they already follow. Nothing else, and yet a few creators end up with almost all the followers: the expected number of followers of the i-th best creator converges to Zipf's law, N/i.
Talks
Posters
For collaborations, talk invitations or questions about my work.
nicolo.pagan@uzh.chUniversity of Zürich, Social Computing Group, Binzmühlestrasse 11, 8050 Zürich