-
This is the official repository of the paper Rethinking Fairness in LLM-Based Recommender Systems: A Survey.
-
Authors: Song-Duo Ma, Chu-Yun Chen, Bang-An Li, Pin-Yu Chen, Shau-Yung Hsu, Yun-Nung Chen (National Taiwan University, Taipei, Taiwan).
- 👀 Overview
- ⚙️ The Roles of LLMs in Recommendation
- ⚖️ A Taxonomy of Fairness in LLM4Rec
- 📊 Evaluation Resources and Protocols
- 🔧 Fairness Mitigation in LLM4Rec
- 🛡️ Cross-Cutting Trustworthy Issues
- 🍣 Open Challenges and Future Directions
Large Language Models (LLMs) are reshaping recommender systems (RecSys), moving them beyond traditional collaborative filtering and user/item IDs toward pipelines enhanced by semantic understanding, natural language generation, and reasoning. This shift, however, also introduces new fairness challenges: bias may arise not only from interaction data and exposure distributions, but also from pretrained knowledge, prompt design, generated explanations, decoding strategies, and feedback loops.
This survey organizes fairness in LLM-based recommender systems (LLM4Rec) through a two-dimensional view: bias mechanisms (where unfairness emerges) and fairness targets (which stakeholders are affected). It further connects fairness with broader trustworthy concerns — explainability, privacy, robustness, and controllability — and consolidates the evaluation landscape and mitigation strategies. To the best of our knowledge, this is the first survey specifically focused on fairness in LLM4Rec.
Depending on their position in the pipeline, LLMs can support different stages of recommendation, and fairness issues take on different forms across these roles.
| Role | Venue | Paper |
|---|---|---|
| User and Item Extractors | WWW'24 | Representation Learning with Large Language Models for Recommendation |
| User and Item Extractors | WSDM'24 | LLMRec: Large Language Models with Graph Augmentation for Recommendation |
| Re-Rankers | ECIR'24 | Large Language Models are Zero-Shot Rankers for Recommender Systems |
| Re-Rankers | COLING'25 | Enhancing Reranking for Recommendation with LLMs through User Preference Retrieval |
| Generators | RecSys'22 | Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5) |
| Generators | arXiv'22 | M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems |
| Explanation Modules | EMNLP'24 | XRec: Large Language Models for Explainable Recommendation |
| Explanation Modules | UMAP'24 | LLM-generated Explanations for Recommender Systems |
Fairness in LLM4Rec is analyzed along two dimensions. Bias mechanisms describe the sources
from which unfairness emerges; fairness targets describe the stakeholders affected by
recommendation outcomes (users, items, and both jointly). In each table below, the Target column
indicates whether the work addresses User-Side, Item-Side, or Two-Sided fairness.
Bias arising when recommendations vary across sensitive or socially salient attributes such as gender, age, nationality, religion, occupation, or race — whether explicitly provided or inferred from names, occupations, language styles, or conversational context. (Item-side social bias is rarely studied explicitly.)
Bias originating from language patterns, cultural associations, and world knowledge encoded in LLM pretraining corpora, biasing recommendations toward mainstream or culturally dominant items even without explicit demographic signals.
| Venue | Target | Paper |
|---|---|---|
| IPM'23 | User-Side | Towards Understanding and Mitigating Unintended Biases in Language Model-Driven Conversational Recommendation |
| EMNLP'24 | User-Side | A Study of Implicit Ranking Unfairness in Large Language Models |
| arXiv'26 | User-Side | Can Fairness Be Prompted? Prompt-Based Debiasing Strategies in High-Stakes Recommendations |
| arXiv'25 | User-Side | Revealing Potential Biases in LLM-Based Recommender Systems in the Cold Start Setting |
| arXiv'25 | Item-Side | BiFair: A Fairness-Aware Training Framework for LLM-Enhanced Recommender Systems via Bi-Level Optimization |
| RecSys'25 | Item-Side | LLM-RecG: A Semantic Bias-Aware Framework for Zero-Shot Sequential Recommendation |
| WWW'24 | Item-Side | Item-Side Fairness of Large Language Model-Based Recommendation System |
| arXiv'25 | Two-Sided | Investigating and Mitigating Stereotype-Aware Unfairness in LLM-Based Recommendations |
Bias stemming from popularity skews, selection effects, exposure inequalities, and imbalanced interaction histories, which LLMs may inherit as rankers, user modelers, item encoders, or generators.
Bias arising when design and inference choices — prompt formulation, candidate ordering, decoding, and feedback incorporation — systematically shape recommendation outcomes across the pipeline.
- User-Side Fairness — ensures equitable recommendation quality and utility across user groups, typically evaluated as group fairness (comparable performance across demographic groups) and individual fairness (similar users receive similar treatment).
- Item-Side Fairness — centers on the equitable allocation of visibility among items or content providers, addressing popularity bias, exposure disparity, and long-tail suppression.
- Two-Sided Fairness — balances user-side utility with item-side exposure equity; in LLM4Rec it is particularly challenging because improving personalization may unintentionally amplify exposure disparities among items or providers.
Fairness datasets are grouped into three broad types according to how the evaluation data are constructed and used.
| Dataset Type | Subcategory | Example Datasets / Sources |
|---|---|---|
| Curated Candidate-Pool | Curated Item Catalogs | IMDb, MTV, Spotify, QS World University Rankings |
| Curated Candidate-Pool | Real-World Content Pools | Twitter, BlueSky, Reddit, Review-5k, resume-score-details, Shopping Queries Dataset |
| Behavioral Interaction | Entertainment and Media | MovieLens, LastFM, Steam, Book-Crossing, Goodreads, Goodbooks-10k |
| Behavioral Interaction | E-Commerce Review | Amazon Review Datasets (Books, Movies and TV, Video Games, ...) |
| Behavioral Interaction | Service and Community | Yelp, BeerAdvocate, ZhihuRec, AliEC |
| Behavioral Interaction | High-Stakes Decision Domains | CareerBuilder, Insurance, MIND |
| Constructed Evaluation Scenario | Simulated Environments | CS-Domain Job Recommendation Data, SynthTRIPS, University-Profile Data |
| Constructed Evaluation Scenario | Prompt-Synthesized Data | Prompt-Synthesized Books / Movies / Songs Recommendation Data |
Existing protocols are organized into four families by their primary evaluation focus.
Sensitive Attribute — Modify sensitive attributes (e.g., gender, age) in prompts and measure whether outputs change. Metrics are similarity- or ranking-based (e.g., SNSR, SNSV, Jaccard@K, SERP*@K, PRAG*@K).
Preference Aligned — Evaluate whether output differences actually harm user benefit, rather than merely measuring list similarity. The main metric is benefit deviation (e.g., ∆B).
Target Specific — Domain-specific scenarios (e.g., job, academic recommendation). Fairness is evaluated through counterfactual testing, group-level parity (SP, EO, PPV_diff), and domain-specific ranking measures (DRS, GRS, U-NDCG).
Item Side — Examine whether exposure is equitably distributed across items, especially popular vs. long-tail. Metrics include Gini Index, HHI, entropy, MGU/DGU, and long-tail coverage.
Mitigation interventions are organized into four levels: input, data, model, and re-ranking.
Guides LLMs toward fair outcomes during inference without altering parameters (e.g., online prompt optimization, conformal thresholding with prompt engineering). Prompting can be brittle.
Targets historical biases in interaction logs via counterfactual data augmentation, counterfactual debiasing, and causal intervention.
Improves fairness by updating or regularizing parameters while avoiding costly full retraining (PEFT, gated adapters, fairness regularization, bi-level optimization, MoE + contrastive learning).
Modifies decoding to curtail homogeneity (D3) and applies post-hoc re-ranking as a secondary fairness filter (explainable re-rankers, IPS-based dual debiasing).
Fairness in LLM4Rec is increasingly intertwined with other dimensions of trustworthiness and should be studied as a cross-cutting system property rather than a standalone metric.
Natural language rationales can make biased behavior more visible, support popularity debiasing, and enable explainable re-ranking — but plausible explanations may also obscure biased ranking factors.
Privacy protection can reshape how personalization quality is distributed across users and items, and LLMs may infer sensitive attributes even when explicit demographics are removed.
Fair behavior should not depend on a specific prompt, profile, or interaction trajectory; minor input changes, feedback loops, and adversarial manipulation can destabilize fairness outcomes.
LLM4Rec expose multiple control points (prompts, model objectives, decoding, agentic policies), making fairness a controllable behavior rather than only an evaluative property.
-
Toward Cross-Target Fairness Analysis — Research is unevenly distributed: user-side social and attribute bias is well studied, while item-side social bias and two-sided linguistic/knowledge bias remain underexplored. Future work should investigate how fairness objectives interact across stakeholders (e.g., improving user-side personalization may amplify provider exposure disparities).
-
Toward LLM-Specific Fairness Benchmarks — Current protocols largely inherit static designs from traditional RecSys and focus on isolated sensitive-attribute perturbations. Future benchmarks should move beyond single-turn ranking and assess whether fairness remains stable under prompt variations, counterfactual user profiles, generated rationales, and feedback loops.
-
Toward Holistic Trustworthy Evaluation — Fairness should be evaluated jointly with explainability, privacy, robustness, and controllability. In particular, the link between fairness and faithfulness is underexplored: biased recommendations may be made persuasive by fluent but unfaithful explanations.
If you find our survey and this repository beneficial for your research, please kindly cite our paper.
@misc{ma2026rethinkingfairness,
title={Rethinking Fairness in LLM-Based Recommender Systems: A Survey},
author={Song-Duo Ma and Chu-Yun Chen and Bang-An Li and Pin-Yu Chen and Shau-Yung Hsu and Yun-Nung Chen},
year={2026},
eprint={XXXX.XXXXX},
archivePrefix={arXiv},
primaryClass={cs.IR},
url={https://arxiv.org/abs/XXXX.XXXXX},
}