13 Large Language Models and Recommendation: the Landscape
What this chapter assumes. The graph recommender of From Graphs to LightGCN (§3 builds on it): every user and item gets one learned vector, an identity (ID) embedding that carries nothing but which user or item it is; the score is a dot product; and training uses BPR (Bayesian Personalized Ranking), a loss that asks a clicked item to outrank an unclicked one. Also the contrastive loss InfoNCE (information noise-contrastive estimation) of Self-Supervised & Contrastive Learning (§3), built from zero in Loss Functions & Regularizers §2.4, and the softmax of the Probability primer (§3, §7; §2.1 uses it). Of a large language model (LLM) you need only this: a large model that maps text to a vector and can also write text; Representation Learning & the Transformer builds it from zero.
Not assumed: cosine similarity, the temperature of a softmax and mutual information (each is explained in §3, with a pointer to the chapter that builds it), and Recall@K and normalized discounted cumulative gain (NDCG) (§2.2 and §5 use them with a pointer to Evaluation Metrics).
This chapter is a survey, not a method. Its job is to map the field: the four roles an LLM can play in a recommender, the advantage and disadvantage of each, and then one line in depth, the LLM as a semantic second view aligned to a graph-collaborative backbone. Every statement about a paper is what its authors say, reported here and not tested here (§2 to §5 keep this). Where a claim comes from practice and not from a paper, it is marked (practice, not tested here) (§2.1, §4 and §5 keep this).
1. Why bother? The complementarity thesis
Ann loved Titanic, and the site knows which films she and thousands of others clicked, but not what any film is about. A large language model (LLM, a program trained on a vast amount of writing) has read about nearly every film, but has never seen who clicked what. This section sets the two blind spots side by side and says what the rest of the chapter does with them.
Two words first. Serving is answering a live request (Ann opens the page); the serving path is everything that runs between her request and the list she sees; latency is how long that takes, counted in milliseconds (ms, thousandths of a second). Now the two families of recommender, each strong exactly where the other is blind (the table below).
| Collaborative filtering (CF) | Large language models (LLMs) | |
|---|---|---|
| Learns from | the interaction graph (who clicked what) | a vast text corpus (the world’s writing) |
| Great at | the collaborative signal: “users like you also liked…” | semantics, world knowledge, reasoning, language |
| Blind to | meaning: LightGCN embeddings are featureless identity (ID) vectors, one per user and item with no content attached (From Graphs to LightGCN §17); a brand-new item with no clicks is invisible (cold start) | the collaborative signal: a pretrained LLM has never seen your click logs; it does not know item 4072 is co-watched with item 88 |
| Cost | cheap to serve (one dot product) | expensive to serve: a forward pass (one run of the text through the whole network; the next request means a new run) of billions of parameters (the learned numbers inside it) per request; it can also hallucinate, that is, confidently name items that are not in the catalog (for example a film title that does not exist) |
The thesis this survey uses to organize the field is that these are complementary. Collaborative filtering knows that two items go together but not why; an LLM knows why a sci-fi fan might like a particular space opera but not who actually co-watches them. Every method below is an answer to one question, how do you combine the LLM’s semantics with the collaborative signal? The methods differ mainly in what job they hand the LLM.
The one-line map. Give the LLM the whole job → LLM-as-recommender (§2.1) and generative recommendation (§2.2). Give it a supporting job feeding a classical model → LLM-as-enhancer (§2.3). Wrap it in a loop with tools and memory → agentic (§2.4). The trade-off is always the same axis: how much LLM you put on the serving path (power and cost) against how much you keep the cheap collaborative backbone, the main model that does the scoring (speed, and the signal LLMs lack).
In one line: a click model cannot see meaning and an LLM cannot see clicks, so every method below fuses them; the methods differ in how much of the serving path the LLM gets, and the price along that path runs from about $0.004 to $0.006 (values we chose for a sub-cent serve, §4.1) to about $25 per 1,000 requests for a 50-film reranker (a model that re-scores a short list another stage fetched). That figure is a cost model: 500 tokens a film, a token being a piece of text of a few letters that an LLM reads and is billed by (§2), at $1 per million tokens, built in §4.1. Ranking a whole catalog has no single price, and the generator and the agent are not priced by any source used here.
2. The four roles an LLM can play
The site’s owner wants an LLM’s help with the film site, and there are four different jobs the owner could hand it: pick the film, spell out a film’s code, write notes overnight for his old recommender, or chat with Ann. This section names the four jobs in one table that every later section follows.
Two more words the table uses. A prompt is the text handed to an LLM (the question, plus whatever it should know): “Ann watched Titanic and The Notebook; which film next?” is a prompt. A token is the piece of text a model reads or writes in one step: a short word, or part of a long one. “Film” is one token, and a long word such as “unwatchable” may be two, “un” and “watchable” (an illustrative split; real tokenizers differ).
| Role (section) | What the LLM does | Model call per request? | Serve cost (§4.1) |
|---|---|---|---|
| 2.1 Recommender | reads the user’s history as a prompt and ranks or picks items | yes, one or more | about $25 per 1,000 requests as a reranker of 50 films (a cost model, §4.1); ranking a whole catalog has no single price |
| 2.2 Generator | writes the item’s identifier, one token at a time (a Semantic ID, a short code of discrete tokens that names an item, §2.2) | yes, a compact model | no sourced price |
| 2.3a Enhancer: feature engineering | writes profiles or knowledge offline (ahead of any request) for a classical model to align with or use | no | sub-cent: the classical model’s own cost |
| 2.3b Enhancer: feature encoding | turns item or user text into vectors offline for the classical side | no, or one small embedding call | sub-cent |
| 2.4 Agent | plans, remembers and calls tools over several turns | yes, several | no sourced price |
Where the surveys put it. Two published surveys cut the field differently, and this book’s cut is a third, so a reader who meets them needs a translation.
Wu et al. (A Survey on Large Language Models for Recommendation, World Wide Web 2024) sort the field by the kind of language model: discriminative models such as BERT (Bidirectional Encoder Representations from Transformers), language models that read text and turn it into a vector, usually trained further on the task (fine-tuned) and used to produce the vectors a recommender consumes, against generative models such as GPT (Generative Pre-trained Transformer), which write text. They then subdivide by how the model is adapted: fine-tuning, prompt tuning (training for one task, such as rating prediction, through a prompt) or instruction tuning (training on many tasks written as instructions, as in the prompt–answer pairs of §2.1); the paper’s own text draws this three-way split.
Lin et al. (How Can Recommender Systems Benefit from LLMs, ACM Transactions on Information Systems 2025) use two axes: where the LLM sits in the pipeline, in five stages (feature engineering · feature encoder · scoring/ranking · user interaction · pipeline controller), and how it is adapted (tune the LLM or not, keep a conventional model or not).
The same prompt under each survey’s axes. Take Ann’s prompt of §2.1, shortened here (her history Titanic, The Notebook and La La Land, with three candidate films). Fine-tuning updates the model’s weights (the learned numbers, §1’s parameters) on many pairs of a history and the film watched next; instruction tuning does the same with the pairs written as instructions, such as “Ann watched Titanic and The Notebook; what next? → Pride & Prejudice”. Prompt tuning leaves the weights alone and trains only a short learned prompt placed before the text. For the five stages, the LLM writes a profile (“likes slow-burn romance”) in feature engineering, turns that profile into a vector in the feature encoder, rates the candidate films in scoring and ranking, chats with Ann in user interaction, and decides which of these to call in the pipeline controller.
This chapter’s four roles are our own cut, by what job the LLM does, and so by how much of it sits on the serving path. They are not either survey’s categories, and they cross Wu et al.’s: RLMRec (§2.3 below; the paper is Representation Learning with Large Language Models for Recommendation) asks a generative LLM (the paper uses OpenAI’s gpt-3.5-turbo) to write profiles, which makes it generative in their sense and an enhancer here. The enhancer’s two cells borrow Lin et al.’s stage names: KAR (Knowledge Augmented Recommendation) and RLMRec appear in that survey under feature engineering, while EasyRec (§2.3), published later, is not in it, so filing it under feature encoding is our classification. Careful: “discriminative” and “generative” mean a third thing in Self-Supervised & Contrastive Learning §6.1 (modeling \(p(y\mid x)\) against modeling how the data is produced).
The next four subsections take each role in turn.
In one line: the five cells of the roles table sort by one question, does a model run on every request: three cells say yes (recommender, generator, agent) and two say no LLM call (the enhancer’s; 2.3b may add one small embedding call), the split behind the largest cost gap of §4.1.
2.1 LLM-as-Recommender: the LLM is the ranker
Ann asks a friend who has read every film review, but never saw the site’s click logs, which of three films to watch tonight. He answers from what he knows about the films, not from what other viewers did. That friend is an LLM used as a recommender; this section scores one example by hand and says what goes wrong.
How it works. Write the user’s history into a prompt, “Ann watched Titanic, The Notebook and La La Land. Which of these will she watch next: A) Pride & Prejudice, B) Vendetta, C) Interstellar?”, and let the LLM rank or pick. Zero-shot means the model is frozen and simply asked, with no solved examples and no extra training (the Titanic prompt above is zero-shot); few-shot also keeps it frozen but pastes a handful of solved examples into the prompt first (say, two of Ann’s past picks, each with the film she took next).
Three systems show the range. TALLRec (Bao et al., RecSys 2023, the ACM Conference on Recommender Systems; its name packs its title, a Tuning framework to Align LLMs with Recommendation) fine-tunes, that is, keeps training, a Llama model (Meta’s openly released LLM family) on recommendation instructions: prompt–answer pairs such as “User liked A, B; will they like C? → Yes”. Its authors report gains in the movie and book domains with fewer than 100 training samples.
P5 (Geng et al., RecSys 2022) casts five kinds of task (rating, sequential recommendation, explanation, review, and direct recommendation) as text-to-text on a single T5 (Text-to-Text Transfer Transformer), Google’s encoder–decoder Transformer (the attention-based network of Representation Learning & the Transformer), under the “Pretrain, Personalized Prompt & Predict Paradigm”: five P’s, hence P5. LlamaRec (Yue et al., 2023) is the two-stage version: a small sequential recommender first retrieves a shortlist from the user’s history, then a fine-tuned LLM scores the shortlist in a single pass, reading off the probability its output layer gives each candidate’s label instead of writing a title.
A worked score (toy numbers, not LlamaRec’s). Take the three candidates of the prompt above, labelled A, B and C. In one pass the output layer gives each label a raw score, a logit: \(2.0\) for A, \(1.0\) for B and \(0.0\) for C. The softmax of the Probability primer (§3) turns them into probabilities: \(e^{2}=7.389\), \(e^{1}=2.718\), \(e^{0}=1\), which add to \(11.107\), so the shares are \(0.665\), \(0.245\) and \(0.090\). The list handed back is A, B, C, and one pass produced all three scores. Because they come from one reading of one prompt, they can depend on how the prompt is ordered; Hou et al., below, measure that.
- Advantage. It works from text, so a film with no clicks still has words to be judged on, and it can explain its pick in words. TALLRec’s authors also report that the tuned model generalizes across domains.
- Disadvantage. It is slow and expensive at serve time (a billion-parameter forward pass per request, priced in §4.1), it can name films that are not in the catalog, and it struggles to rank against millions of candidates, so you must pre-filter (practice, not tested here). BIGRec’s authors (§5) report that LLMs absorb popularity and collaborative statistics poorly, which is the signal a small LightGCN captures well.
Hou et al. (Large Language Models are Zero-Shot Rankers for Recommender Systems, ECIR 2024, the European Conference on Information Retrieval) report two systematic biases in zero-shot LLM rankers, position bias (performance drops when the item the user went on to choose sits among the last positions of the prompt) and popularity bias (a lean toward popular items whatever the fit). They report that specially designed prompting and bootstrapping alleviate these issues; bootstrapping ranks the same candidates several times with the order shuffled each round, then merges the rounds into one ranking (nothing to do with the statistical bootstrap of the Probability primer).
A toy case, with invented numbers and not Hou et al.’s: the film a user went on to choose is picked \(30\) times in \(100\) prompts that list it first and \(18\) times in \(100\) that list it last, and that gap is position bias. Popularity bias is a blockbuster ranked first for a user who never watches blockbusters. Bootstrapping on three candidates A, B and C: three rounds, each in a different order, rank them (A, B, C), (C, A, B) and (B, A, C); the average ranks are \(1.67\), \(2.0\) and \(2.33\), so the merged ranking is A, B, C.
The production sub-pattern: LLM-as-reranker. A model cannot rank millions of films per request, so in practice it re-scores a short list, often 50 to 200 films (practice, not tested here), that a cheap retriever or CF model already fetched. That is the ranking stage of the retrieval \(\to\) ranking funnel (Traditional Recommender Systems §7; Implementation Choices); when the stage re-scores a list that something else retrieved, it is usually called reranking, the same stage under a second name.
A cross-encoder reads each (user, item) pair together in one forward pass, so its scores cannot be precomputed (it reads “Ann watched Titanic” together with Pride & Prejudice and returns one score, say \(0.8\); the next film means a new pass; the Implementation Choices appendix, reranking section). It is widely used as a reranker (practice, not tested here), but it is usually a specialist model, not an LLM. The enhancer of §2.3 puts no LLM call on the serving path (2.3b may add one small embedding call).
In one line: the recommender role turns one prompt into scores such as \(0.665\), \(0.245\) and \(0.090\) for three films, but every request pays for a model run (about $25 per 1,000 requests for a 50-film shortlist, §4.1), and Hou et al. report that the order of the prompt can change the answer.
2.2 Generative recommendation: recommend by generating the item
The owner’s site stores each film as a number such as 4072, and the number says nothing about the film: 4072 and 4073 may have nothing in common. What if the identifier itself spelled out what the film is like? This section builds such identifiers with a box of crayons, then shows a model writing them one piece at a time.
How it works. Like §2.1, the model writes its answer token by token instead of computing a score for every item on a list; each token is chosen given the ones before it, which is called autoregressive generation (for the identifier \((\textsf{A},\textsf{p})\) built below: write A, then p given A; here the letters A to D and p to s name codewords, a different job from the A, B and C that labelled candidate films in §2.1). But where §2.1 picks from a candidate list written into the prompt, here the model writes a compact item identifier directly, so no candidate list is needed at all.
This line builds on sequential ID recommenders such as SASRec (Self-Attentive Sequential Recommendation) and BERT4Rec (BERT, spelled out above, trained on item sequences; both built in the Sequential chapter), both Transformer-based, which already predict the next item ID from a user’s history, but replaces the fixed per-item ID with a generated, content-derived code. TIGER (Transformer Index for Generative Recommenders; Rajput et al., NeurIPS 2023, the Conference on Neural Information Processing Systems) introduced it, and its paper calls the idea generative retrieval.
Four small ideas, one at a time. That content-derived code is built out of four ideas, each of which the worked example below puts a number on.
- Quantization is rounding with a fixed menu of allowed answers. Round \(4.17\) to the nearest whole number and you get \(4\): you have replaced a real number by one symbol from a small set. Quantizing a vector is the same move in two or more dimensions.
- The menu is called a codebook and each entry a codeword. Ours holds four: \(\textsf{A}=(1,0)\), \(\textsf{B}=(0,1)\), \(\textsf{C}=(-1,0)\), \(\textsf{D}=(0,-1)\). To quantize a vector, take the nearest codeword, nearest in ordinary straight-line (Euclidean) distance, and keep only its letter. Rounding a color to the nearest crayon in a fixed box is exactly this.
- One codeword is a coarse answer, so the vector and its codeword differ. That difference, vector minus chosen codeword, is the residual, and it is what the next level is asked to describe.
- Residual quantization is doing that repeatedly: quantize the residual against a second codebook, its leftover against a third, and so on. The identifier is the tuple of letters, one per level; the chosen codewords add back up to (an approximation of) the original vector, and each extra level usually shrinks the error still left over (usually, not always: the second example below shows a leftover that remains, and the third a case where it grows).
Run this on an item’s content (encode the item’s text with a text encoder, a model that turns a sentence into a vector, then quantize that embedding level by level) and the tuple of codewords you keep is the item’s Semantic ID. Train a sequence model to generate the next item’s Semantic ID, exactly as a language model generates the next word, and you have a recommender. Because similar items share codeword prefixes, the model can in principle carry what it learned about one item onto another (the idea behind the design, not tested here), and there is no candidate list to maintain.
Worked micro-example, item 1: a Semantic ID by residual quantization. Say an item’s (tiny) text embedding is \((0.7,0.2)\). Its Euclidean distances to the four level-1 codewords are \[d_{\textsf{A}}=0.36,\qquad d_{\textsf{B}}=1.06,\qquad d_{\textsf{C}}=1.71,\qquad d_{\textsf{D}}=1.39,\] so \(\textsf{A}=(1,0)\) wins and the first code is A. Subtract it: the residual is \((0.7,0.2)-(1,0)=(-0.3,0.2)\). Snap that against a second-level codebook of smaller vectors, \(\textsf{p}=(-0.3,0.2)\), \(\textsf{q}=(0.3,0.2)\), \(\textsf{r}=(-0.3,-0.2)\), \(\textsf{s}=(0.3,-0.2)\), whose distances from the residual are \(0\), \(0.60\), \(0.40\) and \(0.72\): it lands exactly on p. The Semantic ID is the tuple \((\textsf{A},\textsf{p})\). The adding-up that makes the scheme work holds: \(\textsf{A}+\textsf{p}=(1,0)+(-0.3,0.2)=(0.7,0.2)\), the embedding back again, so the error fell from \(0.36\) after one level to \(0\) after two.
Item 2, where a leftover remains. Take the embedding \((1.2,0.3)\). Its distances to the level-1 codewords are \(0.36\), \(1.39\), \(2.22\) and \(1.77\), so it also snaps to A and shares the first code with item 1. Its residual is \((0.2,0.3)\), and against the same second-level codebook the distances are \(0.51\), \(0.14\), \(0.71\) and \(0.51\): nearest is \(\textsf{q}=(0.3,0.2)\), so its Semantic ID is \((\textsf{A},\textsf{q})\). Adding back, \(\textsf{A}+\textsf{q}=(1.3,0.2)\) misses \((1.2,0.3)\) by \((-0.1,0.1)\), a distance of \(0.14\).
The error fell from \(0.36\) to \(0.14\), not to \(0\): a level can shrink what is left, and does not always remove it. The two items share the prefix A and differ in the second code, which is the idea behind letting a model that has learned the “A…” continuations of one carry them onto the other (the idea, not tested here).
Item 3, where the second level makes it worse. Take \((1.05,0)\). It snaps to A, the residual is \((0.05,0)\), and the error is \(0.05\). But the nearest second-level codewords, \(\textsf{q}\) and \(\textsf{s}\), are \(0.32\) away from that residual (\(\textsf{p}\) and \(\textsf{r}\) are \(0.40\)), so after two levels the error is \(0.32\): larger. This illustrative codebook was not fitted to the data. A codebook fitted to the residuals, as the residual-quantized variational autoencoder below does, can avoid this (practice, not tested here), but with an arbitrary codebook a level can grow the error. (The level-2 codebook is illustrative; E4 in §6 uses its own.)
How the codebook is learned. The worked example snapped to a given codebook, but where does a good codebook come from? TIGER fits one with an RQ-VAE (a residual-quantized variational autoencoder). An autoencoder is a pair of small networks playing a squeeze-and-restore game: an encoder compresses its input into a shorter code, a decoder has to rebuild the input from that code alone, and the training signal is the reconstruction error between what went in and what came back out. A toy case: the input \((0.7,0.2)\) is squeezed to the one-number code \(0.9\) and rebuilt as \((0.72,0.18)\), a reconstruction error of \(0.03\) (invented numbers). (“Variational” is historical naming inherited from that family of models; what matters here is the encoder \(\to\) quantize \(\to\) decoder loop.)
The RQ-VAE puts the quantizer in the middle: an encoder maps the item’s content embedding to a vector; that vector is quantized level by level (each level snaps the residual the level above left over, exactly as in the example); and the decoder must reconstruct the original embedding from the chosen codewords. Training the encoder, decoder and codebooks together to minimize reconstruction error forces the codewords to be informative: semantically similar items come to share high-level codewords (the shared A prefix above), which is the entire point. The codebook is not handed to you; it is fitted so that a handful of discrete symbols preserve the item’s content.
How you decode a recommendation. At serving time the user’s history becomes a sequence of Semantic IDs, and a sequence-to-sequence model (seq2seq: one network reads the input sequence, a second writes an output sequence one token at a time; given the IDs of Ann’s last two films, \((\textsf{A},\textsf{p})\) and \((\textsf{B},\textsf{p})\), it writes the next film’s ID, say \((\textsf{A},\textsf{q})\), a codeword at a time) generates the next item’s ID tuple, one codeword at a time. To get a ranked top-\(K\) list, not just a single item, you run beam search: rather than commit to the single most probable codeword at each step, keep the \(W\) most-probable partial ID prefixes (the beam), extend every one of them at the next step, score each extension by the product of its codewords’ probabilities, and again keep the best \(W\). The completed tuples, ordered by that product, are the recommendations.
Beam search in three lines (width \(W=2\), on the two codebooks above; \(W\) is the beam width, kept apart from the codeword B). Step 1: the model’s level-1 probabilities are \(\textsf{A}\,0.6\), \(\textsf{B}\,0.3\), \(\textsf{C}\,0.07\), \(\textsf{D}\,0.03\); keep the best two, \(\textsf{A}\) and \(\textsf{B}\). Step 2: extend each. After \(\textsf{A}\) the level-2 probabilities are \(\textsf{p}\,0.5\), \(\textsf{q}\,0.3\), \(\textsf{r}\,0.15\), \(\textsf{s}\,0.05\), and after \(\textsf{B}\) they are \(\textsf{p}\,0.7\), \(\textsf{q}\,0.2\), \(\textsf{r}\,0.07\), \(\textsf{s}\,0.03\). Scoring each whole prefix by the product gives \((\textsf{A},\textsf{p})=0.6\times0.5=0.30\), \((\textsf{B},\textsf{p})=0.3\times0.7=0.21\), \((\textsf{A},\textsf{q})=0.18\), \((\textsf{A},\textsf{r})=0.09\), \((\textsf{B},\textsf{q})=0.06\), \((\textsf{A},\textsf{s})=0.03\), \((\textsf{B},\textsf{r})=0.021\) and \((\textsf{B},\textsf{s})=0.009\). Step 3: keep the best two: the ranked list handed back is \((\textsf{A},\textsf{p})\), then \((\textsf{B},\textsf{p})\). That is where the ranking comes from: the same probabilities that spell the tuples also order them.
The same, with a narrower beam (\(W=1\)). Step 1 now keeps only \(\textsf{A}\), so step 2 extends only \(\textsf{A}\) and scores its four extensions, \((\textsf{A},\textsf{p})=0.30\), \((\textsf{A},\textsf{q})=0.18\), \((\textsf{A},\textsf{r})=0.09\) and \((\textsf{A},\textsf{s})=0.03\); step 3 keeps only the best, \((\textsf{A},\textsf{p})\), so the list has one entry and every candidate started with \(\textsf{A}\). The tuple \((\textsf{B},\textsf{p})=0.21\), which the wider beam ranked second, was cut at step 1 because \(\textsf{B}\)’s first code looked weaker (\(0.3\) against \(0.6\)): a narrower beam is cheaper but can lose a better whole tuple.
This needs no approximate nearest-neighbor (ANN) index over millions of items, no precomputed table of item vectors to search through (§4 builds ANN properly; Training and Serving a Recommender §7 builds it from zero), because generation replaces the retrieval step altogether, though TIGER’s authors add that beam-search decoding can cost more at inference than an ANN lookup.
Two catches follow. Distinct items can quantize to the same tuple (a collision: the embeddings \((0.7,0.2)\) and \((0.72,0.18)\) both give \((\textsf{A},\textsf{p})\)), so TIGER appends one extra disambiguating token to keep every ID unique. And a beam can spell a tuple that names no real item; TIGER reports these are rare, about \(0.1\)–\(1.6\%\) of top-10 predictions and about \(0.3\)–\(6\%\) of top-20 (its Figure 6), and handles them by enlarging the beam and filtering the invalid IDs. The more general remedy is the constrained decoding of the box below, which restricts generation to valid IDs.
“LLM” here means LLM-shaped, not LLM-sized. TIGER’s generator is not a large pretrained language model reading English: it is a compact sequence-to-sequence Transformer whose entire vocabulary is codewords, trained from scratch on Semantic-ID sequences. What is borrowed from language modeling is the idea, predicting the next token autoregressively, not the billions of parameters. That is why the roles table calls it a compact model and why Figure Figure 13.1 puts the Generator to the left of the Recommender on serve-time cost. (A system that plugs Semantic IDs into a large pretrained LLM would change that argument; this chapter prices only the compact case.)
- Advantage. One end-to-end model; no separate retrieval index; content is baked into the ID. TIGER’s authors report one cold-start experiment, on the Amazon Beauty data with 5% of items withheld from training (their Figure 5), in which they report that TIGER outperforms a nearest-neighbor baseline built on the same text embeddings in Recall@K, the share of the held-out items that appear in the top K places of the list (Evaluation Metrics §4).
- Disadvantage. Everything rides on the Semantic-ID construction (a bad quantizer would cap performance, a claim from practice, not tested here); autoregressive decoding has its own failure modes and, per TIGER’s authors, a higher inference cost than ANN; and scaling to industrial catalogs was long an open engineering problem (now partly answered, below).
Reported in production. OneRec (Deng et al., 2025, from Kuaishou, a large Chinese short-video platform) replaces the classical retrieval \(\to\) ranking cascade with one generative model, which its authors describe as the first end-to-end generative model to clearly beat the established pipeline in a real deployment. It is tuned with iterative direct preference optimization (DPO; Rafailov et al., 2023: repeatedly fine-tuning the model on pairs of preferred and rejected outputs until it favors what users engage with; in OneRec one pair is the best and the worst of 128 lists that the model itself writes for one user, ranked by a separate reward model (a second network that scores a list by how well users engaged with it) trained on logged session outcomes). Careful with the word that paper attaches to it. “Alignment” there means matching user preferences, a different job from the embedding alignment of §3.
The OneRec abstract reports a \(1.6\%\) increase in watch-time in Kuaishou’s main scene; its Table 2 gives \(+1.21\%\) for the 1-billion-parameter model without the preference step and \(+1.68\%\) with it, so the figure belongs to the whole deployed system, of which the preference step is one ingredient. The same group has since posted the OneRec Technical Report and OneRec-V2 Technical Report (both 2025); we read only their abstracts, which report an App Stay Time gain of \(0.54\%\) and \(1.24\%\) on two apps, with the system handling \(25\%\) of Kuaishou’s queries per second, for the first, and \(0.467\%\) and \(0.741\%\) for the second. Those are other deployments and other metrics, not updates to the \(1.6\%\).
Zhai et al. (Actions Speak Louder than Words, Meta, ICML 2024, the International Conference on Machine Learning) report generative recommenders of a different kind, sequential transducers (models that read a user’s sequence of actions and write the next one) that they name HSTU (Hierarchical Sequential Transduction Units), deployed on multiple surfaces of a large internet platform with billions of users, and a \(12.4\%\) metric improvement in online A/B tests (live experiments that give two versions of the system to different real users). How many other deployments exist is not something this chapter checked.
Grounding the output (reducing hallucination). A free-text LLM ranker (§2.1) writes token sequences with no guarantee they name real catalog items: it can confidently recommend a film that does not exist. Three standard fixes: constrained decoding (restrict generation to the item or ID vocabulary: the model may write only IDs of films in the catalog, so a code pair that names no film cannot be emitted), candidate-list forcing (choose only from a retrieved shortlist, the reranker pattern above, so a film not on the list cannot be picked), and post-hoc catalog filtering (drop any generated item not in the catalog, so an invented title such as “Moonlit Harbor” is deleted afterwards). Generative recommendation avoids inventing a title, because a Semantic ID is assembled from real codewords; it can still spell a tuple that names no item (TIGER: \(0.1\)–\(1.6\%\) of top-10 predictions), which is why the filter is kept.
In one line: a Semantic ID is two letters instead of an arbitrary number, for example \((\textsf{A},\textsf{p})\) for the embedding \((0.7,0.2)\) with error \(0\) and \((\textsf{A},\textsf{q})\) for \((1.2,0.3)\) with error \(0.14\); the two share the prefix A, and OneRec reports a \(1.6\%\) watch-time gain for the whole deployed system (its abstract’s figure; Table 2’s \(+1.68\%\) is the 1-billion-parameter model with the preference step).
2.3 LLM-as-Enhancer: the LLM feeds a classical model (the focus of §3)
The owner’s old recommender works well for Ann and Cara, so the owner does not want to replace it. What it lacks is what the films are about, and an LLM can read that overnight, long before anyone opens the site. This section names the two ways of handing the LLM’s output to the old recommender and what each needs.
How it works. Keep a fast classical recommender (matrix factorization, a LightGCN graph model, or even a simple tag-matching rule) as the thing that actually serves. Use the LLM ahead of time, away from any live request, to manufacture the semantic signal the classical model lacks, then fuse it in. Following Lin et al.’s where axis, the role has two cells, by where the LLM’s output enters.
Feature engineering (2.3a): the LLM writes something, the classical model consumes it.
- Knowledge augmentation, KAR (Xi et al., RecSys 2024; Knowledge Augmented Recommendation). The LLM is prompted for reasoning about a user (“likes slow-burn romance”) and factual knowledge about an item. Both answers are stored, encoded, and fed to the backbone as extra input features through a mixture-of-experts (MoE) adapter: several small specialist networks (“experts”) plus a trained gate that weights each expert’s output and sums them (three experts that output \(2\), \(4\) and \(6\), with gate weights \(0.5\), \(0.3\) and \(0.2\), give \(1+1.2+1.2=3.4\)). The LLM is never fine-tuned in KAR, but the adapter is trained jointly with the backbone, so the backbone is retrained on its widened input.
- Representation alignment, RLMRec (Ren et al., WWW 2024, the ACM Web Conference; Representation Learning with Large Language Models for Recommendation). The LLM writes a profile for each user and item, a text encoder embeds it, and a loss pulls that semantic embedding toward the collaborative embedding of a CF backbone. The authors present it as a way to let ID-based graph recommenders use the text attached to users and items, and to cope with noisy, biased implicit feedback. §3 unpacks it.
Feature encoding (2.3b): the LLM turns text into vectors. EasyRec (Ren & Huang, EMNLP 2025, the Conference on Empirical Methods in Natural Language Processing) is a language model tuned with contrastive learning so that the vectors it produces from user and item text line up with collaborative behavior; its authors report strong zero-shot results and describe it as a plug-in for CF frameworks. Two-tower LLM-embedding retrieval (§4) belongs to the same cell on the retrieval side: item text embedded ahead of time, searched at request time.
In both cells the classical model stays on the serving path. The hands-on recipe is Generating Features with Large Language Models.
- Advantage. The serving path stays cheap and strong (the LLM is not called per request), the design, its authors say, drops into existing backbones (not tested here), and the semantic signal is new information, not a perturbed copy of the clicks. Whether the semantics help most where clicks are thin has not been shown by these papers: RLMRec’s paper has no popularity breakdown (§3), and a search of KAR’s text found no cold-start experiment or long-tail analysis (it mentions new users and items when discussing efficiency, but reports no result on them).
- Disadvantage. You need a fusion mechanism, and results hinge on the profiles and the alignment: RLMRec’s authors report that shuffling the profile vectors between nodes lowers performance (Self-Supervised & Contrastive Learning §7). Also, a trained ID-embedding backbone (one learned vector per item, as in matrix factorization or LightGCN) has no vector for an item added after training until it is retrained, so a 2.3a enhancer built on one, such as RLMRec, cannot place a brand-new item; a tag-matching rule that reads the item’s tags needs no per-item vector and can. The feature-encoding cell can too, in principle, because it embeds from text on day one (by construction, not tested here).
In one line: the enhancer moves the LLM’s bill from every request to every retrain, about 8% to 25% more time per training epoch with RLMRec-Con on four of RLMRec’s six backbones and about twice the time (100% more) on the two lightest (its Table 4, profile writing not counted), and no LLM call is added to a request (2.3b may add one small embedding call).
2.4 Agentic and conversational: the LLM as a planner with memory
Cara types: something light, under 100 minutes, that my kids can watch. That is three conditions, and she may change her mind after the first answer, so no single score will do. This section shows a program that asks, looks things up and remembers, counts its steps on a toy catalog, and says what is known about it.
How it works. Wrap the LLM in a loop: it plans, keeps memory of the dialogue and of the user’s long-term preferences (often by retrieving stored notes), and calls tools, such as a retriever, the CF model or a database. A 2025 survey (Peng et al., A Survey on LLM-powered Agents for Recommender Systems, Findings of EMNLP 2025) sorts these agents into three kinds: recommender-oriented (the agent plans and makes the recommendation itself), interaction-oriented (it converses, explains and refines with the user) and simulation-oriented (it role-plays users to generate synthetic behavior, see §5). Conversational recommenders are the second kind.
A worked count (a toy catalog). Say the catalog has six films with running times \(88\), \(95\), \(102\), \(97\), \(118\) and \(91\) minutes, and two of the four films under \(100\) minutes carry a “family” tag in the database. The agent’s first step reads Cara’s message and plans; its second calls the tool with “under \(100\) minutes”, which keeps \(4\) of the \(6\) films; its third calls it again with “suitable for children”, which keeps \(2\); its fourth writes the answer. That is four model steps. If each costs about as much as one recommender call (§2.1), the request costs and takes about four times as much, before any memory is read.
- Advantage. Interactive and explainable, and in the surveys’ account able to handle open-ended, multi-constraint requests that a one-shot ranker handles badly.
- Disadvantage. Early-stage (the surveys’ account); evaluation is hard, since a conversation has no fixed ground truth; and latency and cost compound, because every step is another model call (see the count above).
In one line: an agent turns one request into a loop, here \(4\) model steps that narrow \(6\) films to \(4\) and then to \(2\), so, if each step costs one recommender call, it costs about four times as much.
3. The graph line, in depth: the LLM as a semantic second view
Ann’s clicks say which films she watched, not what they were like. What if every film also carried a short written description, and the site pulled each film’s learned list of numbers (its vector) toward it? This section builds that idea in three layers, works its loss out on four tiny vectors, and says what the RLMRec paper does and does not show.
This is the role the RLMRec line of §2.3a lives in, and it is the natural endpoint of From Graphs to LightGCN, Self-Supervised & Contrastive Learning, and The Spectral / Graph-Filter View. Read it as three layers stacked.
Layer 1: the collaborative backbone (From Graphs to LightGCN). LightGCN (He et al., 2020) gives every user and item a featureless ID embedding and refines it by propagating over the interaction graph; the score is a dot product \(\mathbf{e}_u\!\cdot\!\mathbf{e}_i\), trained with BPR, the pairwise loss that asks only that a clicked item outrank an unclicked one (Loss Functions & Regularizers §9). Strong collaborative signal, but the embeddings carry no meaning (From Graphs to LightGCN §17).
Layer 2: self-supervised learning (SSL) views (Self-Supervised & Contrastive Learning). Self-supervised learning invents a training task from the data itself, with no labels. Here the task is contrastive: build a second view of every node and pull a node’s two views together with InfoNCE (information noise-contrastive estimation, the loss of Loss Functions & Regularizers §2.4) while pushing different nodes apart.
Three methods differ in how they build the second view: SGL (self-supervised graph learning) drops random edges of the graph, SimGCL (Yu et al., 2022; simple graph contrastive learning) adds a small random nudge to the embeddings and touches no edges, and LightGCL (simple yet effective graph contrastive learning for recommendation) builds its view from a truncated singular-value decomposition of the interaction table (a compressed, low-rank summary of it). This regularizes the geometry, in the two senses Self-Supervised & Contrastive Learning §3.3 names, alignment (matched pairs close) and uniformity (everything else spread out; §3 below uses both), and is sometimes aimed at popularity bias (practice, not tested here), but the second view is still built from the same interaction data: no new outside information enters.
Layer 3: the LLM as a better second view (RLMRec). The idea: instead of a perturbed copy of the collaborative embedding, make the second view a semantic one that the LLM supplies.
The alignment objective, decoded. RLMRec maximizes the agreement between a node’s collaborative embedding \(\mathbf{e}_i\) and its semantic embedding \(\mathbf{s}_i\). The contrastive variant (RLMRec-Con) is Self-Supervised & Contrastive Learning’s InfoNCE with the two views being \((\mathbf{e}_i, \mathbf{s}_i)\); the version below carries a temperature \(\tau\) as that chapter does, whereas the paper’s Eq. 16 writes none, which is the case \(\tau=1\): \[ \mathcal{L}_{\text{align}} \;=\; \sum_i -\ln \frac{\exp\!\big(\operatorname{sim}(\mathbf{e}_i,\,g(\mathbf{s}_i))/\tau\big)} {\sum_{j}\exp\!\big(\operatorname{sim}(\mathbf{e}_i,\,g(\mathbf{s}_j))/\tau\big)} . \]
Here \(\ln\) is the natural logarithm (base \(e\), Python’s math.log). Per component: \(i\) indexes an item and \(j\) runs over the items in the same batch; \(\mathbf{e}_i\) is item \(i\)’s collaborative embedding (from LightGCN); \(\mathbf{s}_i\) is its semantic embedding (a frozen text encoder applied to the LLM-written profile of \(i\)); \(g\) is a small trained network, a multi-layer perceptron, i.e. a few ordinary neural-network layers, that maps \(\mathbf{s}_i\) into \(\mathbf{e}_i\)’s space; \(\operatorname{sim}\) is cosine similarity, the dot product of two vectors after each is scaled to length 1 (for \((1,0)\) and \((0.8,0.6)\), already of length 1, it is \(0.8\); Self-Supervised & Contrastive Learning §3); \(\tau\) is the temperature, a divisor on the similarity that sharpens the softmax when it is small (Probability primer §3/§7); for a cosine gap of \(0.2\), a weight changes by a factor \(e^{0.2}=1.2\) at \(\tau=1\) but \(e^{2}=7.4\) at \(\tau=0.1\).
Why \(g\) is not optional. Cosine similarity is defined only between two vectors of the same length, and these two are not. RLMRec’s collaborative embeddings are short (the paper uses \(32\) dimensions for every backbone), while \(\mathbf{s}_i\) comes from a general-purpose text-embedding model, is far longer, and lives in a space with nothing to do with the click graph. So RLMRec first passes \(\mathbf{s}_i\) through \(g\) (its paper writes this map \(\sigma_{\downarrow}\)) and only then takes the cosine. The text encoder and \(\mathbf{s}_i\) stay frozen; \(g\) is trained along with the embeddings, which is part of how this loss gets to move the collaborative space at all.
The numerator rewards a node’s own two views for agreeing (“this item’s collaborative identity should match what the LLM says it is”). The denominator sums over all items \(j\), its own view included, so the matched pair sits in the numerator and in the denominator, and shrinking it means pushing \(i\)’s collaborative embedding away from other items’ meanings. (Dropping the \(j=i\) term instead, as some implementations do, changes the loss value; Self-Supervised & Contrastive Learning §3.2 works through both conventions.)
These are the two forces that chapter names alignment (numerator: matched pairs close) and uniformity (denominator: everything else spread out); see it for the picture on the sphere. RLMRec’s authors argue that minimizing \(\mathcal{L}_{\text{align}}\) pulls the collaborative space toward the LLM’s semantic space and so injects world knowledge that pure interactions never carried (the paper’s claim, not tested here). The worked example below tests only the direction of the loss: a case where turning toward the own view lowers it and a case where a nearby negative wins instead.
The same loss, twice. RLMRec writes a profile for each user and each item (§2.3), and runs this loss once over users, with pairs \((\mathbf{e}_u,\,g(\mathbf{s}_u))\) and other users as the negatives, and once over items. We have written the item version; the user version is what leaves \(\mathbf{e}_u\) knowledge-injected too, which, the authors’ account runs, puts semantics on both sides of the served dot product \(\mathbf{e}_u\!\cdot\!\mathbf{e}_i\) (their claim, not tested here).
The loss on four tiny vectors. Take item \(i\)’s collaborative embedding \(\mathbf{e}_i=(1,0)\) and, after the projection, its own semantic view \(g(\mathbf{s}_i)=(0.8,\,0.6)\), with two other items’ projected semantic views as the negatives, \(g(\mathbf{s}_a)=(0.6,\,0.8)\) and \(g(\mathbf{s}_b)=(-0.6,\,0.8)\). All four are unit vectors, so each cosine is just a dot product: \(0.8\), \(0.6\) and \(-0.6\). At temperature \(\tau=0.2\) those become \[\exp(0.8/0.2)=54.60,\qquad \exp(0.6/0.2)=20.09,\qquad \exp(-0.6/0.2)=0.05,\] and the denominator is all three added together, \(74.73\). The own view’s share of that total is \(54.60/74.73=0.731\), so \[\mathcal{L}_{\text{align},\,i} \;=\; -\ln(0.731) \;=\; 0.31 .\]
Now turn \(\mathbf{e}_i\) toward its own view \((0.8,0.6)\), which is the pull the authors intend, and the loss at first goes up (the table below lists every value). When \(\mathbf{e}_i\) lands exactly on \((0.8,0.6)\) (about \(36.9^\circ\)) the three cosines are \(1.0\), \(0.96\) and \(0.0\) and the loss is \(0.60\).
The reason is the near negative \(g(\mathbf{s}_a)=(0.6,0.8)\): it lies closer to the own view than \(\mathbf{e}_i\) does, so every turn toward the own view is also a turn toward \(g(\mathbf{s}_a)\), whose similarity rises faster (from \(0.60\) to \(0.96\)) than the own view’s (from \(0.80\) to \(1.0\)). The loss falls the other way, to about \(0.22\) at \(-43^\circ\), its lowest.
The pull toward the own view and the push away from the negatives act together, and here the push wins.
Which one wins depends on where the negatives sit, so take the mirror case: keep \(\mathbf{e}_i=(1,0)\) and the own view \((0.8,0.6)\), but put the two negatives on the far side of \(\mathbf{e}_i\), at \((0.6,-0.8)\) and \((-0.6,-0.8)\). The cosines at the start are the same \(0.8\), \(0.6\) and \(-0.6\), so the loss starts at the same \(0.31\), but now every turn toward the own view is a turn away from both negatives. The loss falls at every turn toward the own view and is \(0.01\) when \(\mathbf{e}_i\) lands on \((0.8,0.6)\), where the cosines are \(1.0\), \(0.0\) and \(-0.96\).
So in the first toy the push wins, because the near negative’s cosine rises faster (\(0.60\) to \(0.96\)) than the own view’s (\(0.80\) to \(1.0\)) and its share of the denominator grows; in the mirror toy both negatives move away and the pull wins (the table below lists both toys).
| Turn of \(\mathbf{e}_i\) toward its own view | Loss, first toy | Loss, mirror toy |
|---|---|---|
| \(-60^\circ\) (away) | 0.24 | 5.61 |
| \(-43^\circ\) (away) | 0.22 | 4.07 |
| \(-20^\circ\) (away) | 0.25 | 1.67 |
| \(0^\circ\) | 0.31 | 0.31 |
| \(10^\circ\) | 0.37 | 0.11 |
| \(20^\circ\) | 0.44 | 0.04 |
| \(36.9^\circ\) (on the own view) | 0.60 | 0.01 |
Seven turns are enough to show the shape, because the first toy’s loss is a single valley in the turn (its bottom is near \(-43^\circ\), at \(0.22\)) and the mirror toy’s loss falls all the way.
What the two toys support is only this: for a small turn, the loss falls exactly when the own view’s cosine gains more than the average gain of the negatives’ cosines, where each negative counts in proportion to its share of the negatives’ total weight in the denominator (so a near negative dominates that average). Over the whole turn to the own view (a \(36.9^\circ\) turn, with the weights read at the start; the sign agrees with the small-turn rule), the first toy’s average negative gain is \(0.36\) (the near negative holds \(99.8\%\) of the negatives’ weight) against the own view’s \(0.20\), so the loss rises; in the mirror toy both negatives lose cosine, an average of \(-0.60\) against \(+0.20\), so it falls. A near negative alone does not settle the sign.
In a real batch many views are in play and all the vectors move too, so which case dominates is not something these two toys decide. (These are Self-Supervised & Contrastive Learning §3.2’s numbers with its two perturbed views swapped for the collaborative and the semantic one: same loss, new views.)
The temperature, at a second and a third value. Keep \(\mathbf{e}_i=(1,0)\) and change only \(\tau\). At \(\tau=1\) (the paper’s equation, which writes no temperature) the three weights are \(e^{0.8}=2.23\), \(e^{0.6}=1.82\) and \(e^{-0.6}=0.55\), the own share is \(0.484\) and the loss is \(0.73\). At \(\tau=0.2\) it is the \(0.31\) above. At \(\tau=0.1\) the weights are \(2981\), \(403\) and \(0.002\), the share is \(0.881\) and the loss is \(0.13\). A small \(\tau\) makes the softmax sharp, so a gap of \(0.2\) in cosine (the own view against the near negative) counts for much more; a large \(\tau\) blurs it.
RLMRec-Gen is the generative cousin: instead of contrasting, it hides the input of a random subset of nodes and asks the model to rebuild each hidden node’s semantic vector from the embedding it computes without them (Self-Supervised & Contrastive Learning §7 works both variants on one example).
Both variants are framed as maximizing the mutual information between the two views, how much knowing one tells you about the other (two unrelated coins share \(0\) bits; a coin and an exact copy of it share \(1\) bit, because the copy tells you the coin’s face). For the contrastive variant that framing is a concrete inequality: over a batch of \(N\) items, \(\ln N\) minus the batch average of the per-item loss (\(\mathcal{L}_{\text{align}}/N\), since the displayed \(\mathcal{L}_{\text{align}}\) is a sum) is a lower bound on the mutual information \(I(\mathbf{e};\mathbf{s})\), so pushing the loss down raises the guaranteed floor.
As an illustration only, with the three vectors above as a batch of \(N=3\) and the unrounded loss \(0.3139\) of the worked node standing in for the batch average, the bound is \(\ln 3 - 0.3139 = 0.7847\), which rounds to \(0.78\) (the rounded loss \(0.31\) would give \(0.79\)). One node’s loss is not a batch average, so this is a picture of the arithmetic, not a measurement. Self-Supervised & Contrastive Learning §7 carries the argument.
The full training objective is then Loss Functions & Regularizers’ recipe with one extra term, exactly the shape of Self-Supervised & Contrastive Learning’s joint loss: \[ \mathcal{L} \;=\; \underbrace{\mathcal{L}_{\text{BPR}}}_{\text{ranking}} \;+\; \lambda\,\underbrace{\mathcal{L}_{\text{align}}}_{\text{LLM semantic view}} \;+\; \underbrace{\lambda_2\lVert\Theta\rVert^2}_{\ell_2\ \text{penalty}} . \]
Decoding the three new symbols: \(\Theta\) collects every trainable parameter of the model (the identity embeddings and the projection \(g\)), and \(\lVert\Theta\rVert^2\) is the sum of their squares, the penalty that keeps them small; \(\lambda\) sets how much the alignment term counts against the ranking term; \(\lambda_2\) is the strength of the \(\ell_2\) penalty. The paper’s own objective (its Eq. 18) adds the alignment term with weight \(1\); \(\lambda\) is the generic knob of this family of methods. Watch the subscripts if you flip back: Loss Functions & Regularizers writes that penalty strength as a plain \(\lambda\), because there it is the only weight in the formula.
What the paper shows. RLMRec’s authors give their aim as letting graph recommenders that rely on IDs use the text attached to users and items, and cope with noise and bias in implicit feedback (misclicks, popularity bias). They test it on six collaborative backbones: GCCF (graph convolutional collaborative filtering) and LightGCN, plain graph models; SGL, SimGCL and DCCF (disentangled contrastive collaborative filtering), contrastive ones; and AutoCF (automated self-supervised learning for recommendation), a self-supervised masked autoencoder that picks its own augmentation. They use three datasets: Amazon-book, Yelp and Steam. Self-Supervised & Contrastive Learning §7 describes each backbone and owns the per-cutoff numbers (for example the LightGCN and Amazon-book figures) and the contrastive-against-generative comparison.
In its Table 1, the better of the two RLMRec variants beats the plain backbone in \(106\) of the \(108\) backbone-by-dataset-by-metric cells; one cell is equal, and one is lower: AutoCF on Amazon-book at normalized discounted cumulative gain (NDCG, a ranking metric that rewards relevant items placed high; built in Evaluation Metrics §7) at 5, a \(0.14\%\) decrease in the paper’s own improvement row too. Of the \(216\) cells for the two variants, \(172\) carry the paper’s significance star (\(p<0.05\) as the paper reports it; its test and its number of runs are the paper’s and not checked here).
The gains are smallest on SimGCL, DCCF and AutoCF (the mean over a backbone’s 18 cells of the best variant’s printed gain, with the one tied cell counted as \(0\) and AutoCF’s one decrease as negative, is \(2.1\%\) to \(3.4\%\), against \(5.5\%\) and \(6.0\%\) for GCCF and LightGCN); SGL sits between them at \(4.7\%\) and holds the table’s largest single gain, \(+8.1\%\) on Yelp Recall@5. For SimGCL on Yelp, Recall@5 rises \(0.64\%\) and is not starred. So the semantic view helps in nearly every cell, on top of an SSL backbone like SimGCL too, but not everywhere and not always significantly.
Sparse or long-tail items are not the paper’s stated motivation (popularity bias, which it does mention, is a different matter from items with few clicks), and as far as we could see its text has no breakdown by item popularity and no cold-item analysis, so that the gain is largest there is an expectation, not a result. Self-Supervised & Contrastive Learning §7 adds that its LightGCN gains on three datasets of different density do not line up with sparsity.
Why the enhancer is the cheapest role to serve. It keeps the cheap, strong collaborative backbone on the serving path (a dot product, no LLM call per request), uses the LLM offline to enrich, and fuses semantics into the collaborative geometry instead of bolting a slow ranker on top. That is the design space of the LLM-as-enhancer line: an LLM augmenting a graph-collaborative backbone, where the open questions are in how the semantic signal is produced and aligned (§5).
In one line: RLMRec adds one loss term to BPR, which is \(0.31\) for the worked item at \(\tau=0.2\), and serving stays one dot product, with no LLM call.
4. Choosing a role: a decision guide
The owner has the four roles but not a decision: the catalog has ten million films, and Ann will not wait more than 50 milliseconds between her tap and her list. This section gives a lookup from the one limit that matters most to a role, then spells out the option that is easiest to miss: storing every film’s vector ahead of time and searching those vectors at request time.
| If your constraint is… | …reach for | because |
|---|---|---|
| Low serving latency, huge catalog | Enhancer, 2.3a or 2.3b | the LLM runs offline and a dot product or ANN lookup serves |
| Items added after training, with no clicks yet | Enhancer 2.3b (two-tower), Recommender 2.1, or a 2.3a design that consumes tags or features (a tag-matching rule) | each reads the item’s text or tags; a model with one learned vector per item (an ID-embedding backbone, even with 2.3a alignment) has none until retrained |
| Items with few clicks at training time | Enhancer 2.3a, untested for this | untested in the papers read here (§3): an expectation that a semantic view may help where clicks are thin |
| No retrieval index wanted, content-rich items | Generator 2.2 | content is baked into the ID and no candidate set is kept; decoding can cost more than ANN (TIGER’s authors, §2.2) |
| Open-ended, multi-turn requests | Agent 2.4 | only a conversation can elicit and refine fuzzy intent |
| You already have a strong graph-CF model | Enhancer 2.3a | align an LLM semantic view to it and keep everything you have |
| Semantic retrieval at scale, content-rich | Enhancer 2.3b (two-tower retrieval) | LLM semantics on the retrieval path, with no per-query LLM call |
| Short candidate list, generous latency, a written reason wanted for each pick | Recommender 2.1 as a reranker | it reads the item text and can say why; about $25 per 1,000 requests for 50 films (§4.1’s cost model) |
The option easiest to miss: two-tower LLM-embedding retrieval. A tower is simply an encoder network. The pattern uses two of them, a user tower that maps a user’s history to a vector and an item tower that maps an item’s text to a vector, and they meet only in a dot product, never inside one model (a toy case: the user tower turns Ann’s history into \((0.9,0.1)\), the item tower turns Titanic’s text into \((0.8,0.2)\), and the dot product is \(0.74\); invented numbers). That separation is the whole trick: the item tower’s output depends on the item alone, so every item vector can be computed ahead of time with an LLM-grade text embedder and frozen into an index.
At request time you run approximate nearest-neighbor (ANN) search over those vectors, returning almost the closest ones and trading a little accuracy for a large speed-up (of 1,000 stored vectors the exact nearest may be number 17, and ANN, after checking only a few dozen, may return number 20, the second nearest; invented numbers).
In the taxonomy this is the feature-encoding cell, 2.3b. Both towers and ANN are built from zero in Training and Serving a Recommender §7, the next chapter; this is the retrieval stage of the funnel (Traditional Recommender Systems §7).
The user side has two variants, and they price differently. Run the user tower once per request: a single small embedding call, far cheaper than an LLM reranker but not free. Or refresh user vectors in periodic batches: nothing to pay at request time, but a user’s newest clicks stay unseen until the next batch. Either way there is no LLM ranking call per query, only an ANN lookup plus at most one embedding call, so it is enhancer-class on cost, yet it carries the LLM’s semantics straight into retrieval, so a brand-new item is, by construction, findable from its text alone (not tested here).
The natural full stack is candidates from two-tower retrieval plus cheap CF or heuristic sources (these run in parallel, not in series), then one merged shortlist, then an optional LLM-as-reranker on its top-\(K\) (§2.1). This is a design that practitioners report using when content matters and the catalog is large (practice, not tested here).
The through-line. Read the table by constraint, not top to bottom: it is keyed by what binds your system, so the low-LLM options appear at both ends. The roles are what form the ladder, from Enhancer (the LLM runs offline; no LLM call is on the serving path) through Generator and Recommender to Agent (an LLM call, sometimes several, per request), with more power and flexibility, and more cost and risk, at each step. The enhancer line (§3) buys the LLM’s semantics while paying its cost at training time, not per request.
This chapter’s reading of the sources, October 2026 (a summary of practice, not tested here). Keeping the LLM offline is the cheapest way to use it, and two-tower retrieval is the cheapest way to put LLM semantics on the retrieval path. Generative recommendation has published online deployments (OneRec at Kuaishou, §2.2; HSTU at Meta, of a different kind), but how many exist is not something this chapter checked. The vendor guide of §4.1 describes the LLM reranker as a late, expensive stage on a short list. Push the LLM further down the serving path only when you can pay for it.
In one line: with ten million films and a 50 ms budget, only the roles that keep a general LLM off the per-request path are shown to fit, and the generator is unpriced here; §4.1 puts the gap between those roles and an LLM reranker at three to four orders of magnitude (a factor of 1,000 to 10,000; about 4,200 on the chapter’s anchors) in cost, against a chosen sub-cent serve.
4.1 Cost and latency: the orders of magnitude (dated)
Each role puts a different amount of work between Ann’s tap and her list. This section puts a price on that work in dollars and milliseconds, and shows that one yes-or-no question, does a model run for every request, matters more than the size of any model.
Prices and millisecond figures move monthly, so reason in orders of magnitude: one order of magnitude is a factor of \(10\), so three orders is a factor of \(1{,}000\). Prices are per 1k queries, that is, per \(1{,}000\) requests, and times are in ms. A p50 latency is the median: half of the requests are faster (latencies of 10, 20 and 90 ms have a p50 of 20 ms). The cost of serving a recommender is dominated by whether an LLM runs per query, and that one fact spans roughly three to four orders of magnitude (the division is below).
Where the anchors come from. The two sub-cent rows have no sourced price: they are placeholders for “less than a cent”, the cost of a dot product or an ANN lookup (the Implementation Choices appendix reports a filtered-ANN latency we measured ourselves, under a millisecond to a few milliseconds per query on one laptop with a library that keeps the whole index in memory, and finds no published run to cite; it gives no price, so treat the sub-cent placeholders as placeholders). For the bars of Figure 13.5 we chose $0.004 for pure CF and $0.006 for the two-tower serve, once and arbitrarily, two points below a cent.
Every ratio against them is therefore arithmetic on a chosen number, not a measurement: \(25/0.006=4{,}167\), which we round to about 4,200, and \(2/0.006=333\), which we round to about 300 (the first step of the ladder below). Any serve cost between $0.001 and $0.009 would do, and it would move the first ratio between about 2,800 and 25,000, still three to four-plus orders of magnitude.
The $25 reranking price is this chapter’s own arithmetic, and the $2 price is a round figure; both are built as follows.
A reranker that scores \(50\) candidates of about \(500\) tokens each reads \(25{,}000\) tokens per request, so \(25\) million tokens per \(1{,}000\) requests, and at a round $1 per million input tokens that is $25. The $2 cross-encoder anchor is the same 25 million tokens at a round $0.08 per million (\(25\times0.08=2\)), a little above the guide’s cross-encoder prices given next.
A reranker vendor’s guide (Should You Use LLMs for Reranking?, 5 September 2025, listed in the references, read 17 October 2026) lists input prices of $0.05 to $0.80 per million tokens for the four LLMs it times (a September 2025 snapshot; the vendor has since been acquired, see its reference entry), which on the same inputs would give $1.25 to $20 per 1,000 requests, and $0.025 to $0.05 for the two cross-encoders it times (one its own, one a competitor’s), about $0.6 to $1.3. Its summary says a dedicated reranker is 10 to 30 times cheaper than a small LLM, and its price table gives multipliers from 2 to 32. Our round anchors, $25 and $2, sit at or above the guide’s dollars and inside its range of ratios, so treat them as upper-middle orders of magnitude and re-measure on your hardware and traffic.
| Pattern (row of the roles table) | Model call per query? | Serve latency (order) | Serve cost / 1k queries (order) |
|---|---|---|---|
| Enhancer 2.3a, serve path (the plain CF dot product over an ANN index) | no | single-digit ms | sub-cent |
| Enhancer 2.3b, two-tower retrieval (ANN over offline LLM vectors) | no, or one small embedding call | single-digit to tens of ms | sub-cent |
| Reranker: specialist cross-encoder on top-\(K\) (not an LLM) | a specialist model, on a short list | about 0.1 to 0.2 s (vendor p50 on 75 kB of text, two cross-encoders) | about $2 (anchor; $0.6 to $1.3 on the guide’s prices) |
| Recommender 2.1 as a reranker: a general LLM, pointwise (one call per candidate) over \(K\) | yes, \(K\) calls | about 0.7 to 2 s (vendor p50 for four LLMs on 75 kB) | about $25 (anchor; $1.25 to $20 on the guide’s prices) |
| Recommender 2.1 ranking the whole catalog | yes, the core step | hundreds of ms to seconds (our judgment, no source) | highest, drawn open-ended in Figure 13.5 |
| Generator 2.2 (compact sequence-to-sequence model with beam search) | yes, a compact model | no sourced figure | no sourced price |
| Agent 2.4 | yes, several steps | no sourced figure | no sourced price |
Read it as a ladder. The first step is the big one: a pure-CF serve makes no model call, so adding any per-query model, even a specialist cross-encoder on a top-\(K\) shortlist, lifts cost by two-plus orders of magnitude (from sub-cent to about $2) and adds about 10 to 200 times the latency (about 0.1 to 0.2 s against single-digit ms, taken here as 1 to 9 ms: \(100/9\approx11\), \(200/1=200\); our own ANN timing is lower still, which would only widen the ratio), which exceeds a 50 ms budget at these figures (vendor measurement, 75 kB per request).
That latency is the vendor’s, measured on 75 kB of text per request. The guide does not say how many documents make up the 75 kB; we read it as about 100 documents of 750 bytes, which is our assumption, not the vendor’s. The Implementation Choices appendix says only “tens of milliseconds” for a pass over 100 documents on a graphics processing unit, a claim from practice, not tested there. The two figures differ in the length of the texts, the hardware and the network path, and we have not tested which of these accounts for the gap; we keep the vendor’s published measurement.
Swapping that cross-encoder for a general LLM scoring the same list pointwise (each candidate gets its own model call) costs about \(12.5\times\) as much (from about $2 to about $25 per 1k) and turns about 0.1 to 0.2 s into about 0.7 to 2 s (the vendor’s medians in the table above). That is why production tends to keep the LLM offline (Enhancer) or confined to a short reranked list (practice, not tested here), and why “the LLM ranks the whole catalog” stays rare.
The offline price is real but bounded. RLMRec’s Table 4 gives the time of one training epoch on one graphics processing unit (GPU). With RLMRec-Con it is \(8\%\) to \(25\%\) higher on SGL, SimGCL, DCCF and AutoCF (the paper writes “around 10% to 20%” for its larger models) and \(1.9\) to \(2.2\) times on GCCF and LightGCN, not counting the profile writing. That overhead is paid at training time, at every retrain, and a production model is retrained as clicks arrive; it is never paid at serving time. What is genuinely paid once per item is the LLM profile, written when the item is added. (Model picks and exact latencies: the Implementation Choices appendix’s retrieval and reranking sections.)
In one line: a two-tower serve at a chosen $0.006 against a general-LLM reranker at about $25 per 1,000 requests is a factor of about \(4{,}200\), three to four orders of magnitude, and whether a model runs per request is what sets it.
5. Open problems: the 2024–2026 frontier
Which questions would change the owner’s choice if someone answered them tomorrow? This section lists ten that are still open, says for each what is known and from whom, and marks the ones that are this chapter’s own reading rather than a paper’s result.
Do LLMs understand the collaborative signal? An LLM knows semantics, but co-occurrence (“people who bought X bought Y”) is learned from logs, not text. BIGRec’s authors (see the BIGRec bullet below) report that more training data gives only modest gains, which they read as LLMs having limited capacity to absorb popularity and collaborative statistics. It is the central question for the enhancer line, since a semantic view only helps if it aligns with real collaborative structure.
Retrieval-augmented generation (RAG) as a bridge role. At generation time, relevant evidence (reviews, profiles, collaborative neighbors) is retrieved and inserted into the LLM’s prompt, so the model grounds its reasoning in catalog signal and not only in pretrained knowledge. This blurs the line between enhancer and agent. Gao et al. (2023) survey RAG, for language models at large and not for recommendation; this chapter names no recommendation paper for it (practice, not tested here). The same retrieval can run offline, feeding retrieved neighbors into the prompt while the LLM writes profiles for an enhancer.
Fairness and diversity of LLM rankers. Hou et al. (§2.1) report position and popularity bias in zero-shot LLM rankers. Whether such rankers also amplify demographic biases from their pretraining text, and how to audit and correct unfairness across item popularity, provider exposure (how often each seller’s or creator’s items are shown) and user groups when the ranker is a black box, is a question this chapter raises without citing a paper for it.
LLM-based user simulators. Here the LLM plays the user, generating synthetic interaction sequences, an idea proposed for stress-testing recommenders and for studying long-horizon feedback loops (practice, not tested here). RecAgent (Wang et al., 2025; first posted in 2023), a system in which LLM agents role-play users, is an early reference. How far simulator-generated data matches real logs remains an open measurement question.
Evaluation integrity. An LLM may have seen the test items during pretraining (data leakage); every test metric assumes the test items were unseen when the model was trained. Conversational and agentic systems have no fixed ground truth, and an LLM in the loop can amplify feedback loops and popularity bias. Careful evaluation (the Probability primer’s significance testing §8; the metrics of Evaluation Metrics) matters more, not less. Guard it with the protocol of Evaluation Metrics §11 plus three LLM-specific moves: time-based splits (test only on interactions after the model’s knowledge cutoff, the date after which it has seen no training text), held-out or obfuscated item IDs (so the LLM cannot recognize a title it memorized), and stating whether candidates were retrieved or sampled (picked by a real retriever, or drawn at random), since an LLM silently ranking against easy random negatives looks far better than it is.
Cost and scale. Serving LLM rankers under tight latency budgets remains hard (the numbers are in §4.1), which is a reason this chapter’s enhancer keeps its LLM offline (practice, not tested here). For generative recommendation over industrial catalogs the status has moved: OneRec (Deng et al., 2025) reports an online watch-time gain of \(1.6\%\) at Kuaishou for a whole generative system tuned with iterative DPO among other things (§2.2), and Zhai et al. (2024) report deployed generative recommenders of another kind. The status is now “open in general, reported working in the deployments their authors describe.”
Does the semantic view help the thin-data items most? This is the expectation behind many enhancer designs, and it is an expectation, not a result. RLMRec’s paper has no popularity breakdown (§3), the multimodal paper discussed below reports no cold-start analysis either, and a search of KAR’s text found none. TIGER’s authors do report one cold-start experiment (§2.2), for a generator, not an enhancer. Two cautions apply. A top-line NDCG, the normalized discounted cumulative gain of Evaluation Metrics §7, is an average over all test interactions. And an item added after training has no trained vector at all (§2.3), which is a different problem from an item with few clicks.
Multimodal recommendation, a richer enhancer branch. Items are often more than text: a fashion item is an image (visual style), a title and an ID. BM3 (Bootstrap Latent Representations for Multi-Modal Recommendation, Zhou et al., WWW 2023, the ACM Web Conference) takes the image and text features of items alongside the ID embeddings of users and items, perturbs each embedding into a dropout copy of itself (a copy with some entries randomly zeroed, such as \((0.5,-0.2,0.8)\to(0.5,0,0.8)\)) and trains without negative samples. It combines three objectives (rebuilding the user–item interactions, aligning image and text features across modalities, and a masked loss within each modality). Its authors report gains on three Amazon datasets and no cold-start analysis. Its lesson for §3’s framework is that the second view need not be text.
Instruction-tuned sequential LLM recommenders, a sub-role of §2.1. A frozen LLM ranking a candidate list differs from a fine-tuned one that reads a user’s interaction sequence and writes the next item by name, with no list at all. BIGRec (Bao et al., 2025; bi-step grounding, the paradigm of its title) tunes the model on instructions such as “given the sequence A, B, C, predict the next item”, lets it write freely, embeds the written text, and returns the catalog item with the nearest embedding, optionally reweighted by popularity or a collaborative score. That is a post-hoc cousin of §2.2’s catalog filtering, and it writes a title where generative retrieval writes a Semantic ID.
Its authors find that more tuning data helps only modestly, so when does such a model beat an aligned enhancer, and when does it lose to one?
Reasoning recommenders. After DeepSeek-R1, an openly released reasoning LLM from DeepSeek-AI (2025), reported that reinforcement learning (RL: training by trial and error against a score, the reward; for instance, each answer is scored and the model is nudged toward higher-scoring ones) can teach a model to write out its reasoning step by step before it answers (chain-of-thought, for instance “Ann liked two romances, so a third fits” before naming the film), the same idea reached recommendation. OneRec-Think (Liu et al., at the Annual Meeting of the Association for Computational Linguistics, ACL 2026) adds explicit, controllable reasoning to OneRec, trained with a reward built for recommendation and, as its authors report, validated at Kuaishou. Rank-R1 (Zhuang et al., 2026) does the analogous thing for document reranking. The open question is whether explicit reasoning earns its serving cost when the output is a ranked list and not a derivation.
In one line: ten questions are open, and for the one that decides the choice, cost, §4.1 puts a specialist reranker at about \(300\) times the cost of a sub-cent serve ($2 against the chosen $0.006); for the one many enhancer designs lean on, whether thin-data items gain most, the papers cited here give no result.
6. Exercises
Ten problems that exercise the survey end to end. Most ask you to classify a scenario into one of the four roles or reason about a trade-off, and five are small computes on new numbers, using the chapter’s method (§2.2’s quantizer, §3’s loss, §4.1’s cost ladder). Everything is solvable from this chapter alone; no figure is needed, and where a number appears it is a tiny round one. Each is tagged (compute), (extend) or (apply). Full worked solutions are in the Solutions appendix at the back of the book.
E1. (apply) — Classify five deployments into the four roles. For each, name the role (recommender §2.1, generator §2.2, enhancer §2.3, agent §2.4) and give the one-clue tell:
A frozen Llama is handed a prompt “User watched A, B, C; from this list of 50, pick the next” and returns a ranked pick.
Overnight, an LLM writes three genre tags for every film; at serve time a keyword match between those tags and the tags of the user’s last five films orders a shortlist, and no vector and no collaborative model is involved.
A single sequence model emits an item’s identifier token-by-token,
(A, p, …), with no candidate list anywhere in the system.An LLM re-ranks a retriever’s 20 films, but only when the retriever’s best score falls below a threshold, which happens on 3 requests in 20. A call costs $10 per 1,000 calls and the retriever $0.006 per 1,000 requests: what does the average 1,000 requests cost?
A tuned LLM is shown a user’s last five films and writes the next film’s title as free text, with no candidate list; a lookup then returns the catalog film whose embedded title is nearest to the text. Of 1,000 written titles, 870 match a catalog film exactly, yet the lookup returns a film for all 1,000. Name the role, and say how many of the 1,000 recommendations are nearest-neighbor guesses rather than exact matches, and what share that is.
E2. (apply) — Five systems on the trade-off axis. Each system below is described by what runs on a request. (i) A nightly job has an LLM write a taste profile per user, and requests are served from a table of stored vectors by a dot product, at $0.008 per 1,000 requests (a sub-cent price chosen for this exercise, not the $0.006 of §4.1). (ii) A compact sequence-to-sequence model, trained from scratch, writes a three-code Semantic ID with beam search; no source prices it. (iii) A general LLM reads a user’s history and a list of 50 films and picks one, at $25 per 1,000 requests. (iv) A chat assistant plans, calls a search tool, calls a catalog tool and writes the reply: four model steps per request, each as dear as (iii). (v) A two-tower retriever runs a small user network once per request over precomputed film vectors, at $0.008 per 1,000 requests.
Name the role of each system and order the five from least to most model work on the serving path, marking ties and any pair you cannot order from the numbers given.
Which pairs of systems differ in cost by more than a factor of 100, by what factor for the widest pair, and what do the pairs have in common?
E3. (compute) — The cost cliff on new numbers. A pure collaborative-filtering serve (no model call per request) costs $0.004 per 1k queries, the value Figure 13.5 draws for it. A shop builds a general-LLM reranker that scores 36 films of about 450 tokens each per request, at $0.50 per million input tokens.
What does the reranker cost per 1k queries? By what factor is it dearer than the CF serve, to two significant figures, and how many orders of magnitude is that?
At 2,500,000 queries/day, what does each path cost per day?
A specialist cross-encoder on the same list costs $1.6 per 1k. Roughly how many times cheaper is it than the LLM reranker, and how many times more expensive than the CF serve? (Round to one significant figure.)
E4. (compute) — A Semantic ID by residual quantization. Reusing §2.2’s quantize→codeword recipe: an item’s tiny text embedding is \((0.7,\,0.4)\). The level-1 codebook is §2.2’s \(\{\textsf{A}=(1,0),\ \textsf{B}=(0,1),\ \textsf{C}=(-1,0),\ \textsf{D}=(0,-1)\}\).
Snap \((0.7,0.4)\) to the nearest codeword (by Euclidean distance): which letter, and what is the residual (embedding minus chosen codeword)?
Snap that residual against the level-2 codebook \(\{\textsf{p}=(-0.3,0.4),\ \textsf{q}=(0.3,0.4),\ \textsf{r}=(-0.3,-0.4),\ \textsf{s}=(0.3,-0.4)\}\). What is the item’s Semantic ID tuple?
A second item has embedding \((0.6,0.5)\). Show it shares the same level-1 prefix, then give its full Semantic ID using the level-2 codebook of (b). Do the two items now have the same ID, and what does that do to a sequence model that must choose between them?
E5. (compute) — The alignment loss on new vectors. Use §3’s \(\mathcal{L}_{\text{align},\,i}\) with item \(i\)’s collaborative embedding \(\mathbf{e}_i=(1,0)\), its own projected semantic view \((0.6,\,0.8)\) and two other items’ projected views as the negatives, \((0.8,\,0.6)\) and \((-0.8,\,0.6)\). All are unit vectors, so each cosine is a dot product.
Compute the loss at \(\tau=0.5\) to two decimals, and the own view’s share of the denominator.
Compute it again at \(\tau=0.1\). By what factor does the near negative’s weight \(\exp(\cos/\tau)\) now exceed the own view’s, and what was that factor at \(\tau=0.5\)?
At \(\tau=0.5\), turn \(\mathbf{e}_i\) by \(20^\circ\) toward its own view. Give the three new cosines and the new loss. Did the loss rise or fall, and how does that compare with §3’s own toy case?
E6. (compute) — The offline bill of an enhancer, on a schedule. A site retrains its LightGCN every night for 50 epochs. One epoch of the plain model takes 20 s; with RLMRec-Con it takes 1.9 times as long, about the LightGCN ratio of the paper’s Table 4. The catalog gains 1,000 films a day, each described once by a hosted LLM (gpt-3.5-turbo) when it is added, and the site serves 2 million requests a day.
How many extra minutes of training does the alignment add per night, and how many extra hours per year?
How many LLM profile calls are made per day and per year, and how many per request?
The 1,000 films arrive evenly around the clock, and each night’s retrain uses the catalog as it stood at 02:00, and its vectors count as live at 02:00 (ignore the 32 minutes of training). At 18:00, how many films have no trained vector in the aligned LightGCN, and how many hours until a retrain gives them one?
E7. (apply) — A whole-catalog prompt and a long-tail claim.
A small site has 5,000 films and shoppers who will wait 2 s, so latency seems no obstacle to an LLM that reads the whole catalog on every request (set aside that a prompt this long may not fit in the model’s input window). Using §4.1’s anchors (films of about 500 tokens, $1 per million input tokens), price that design per 1,000 requests, compare it with the 50-film reranker, and name the cheaper design that is better.
A blog post says an LLM enhancer “helps long-tail items most”. Its only evidence is that Recall@20 rose from \(0.1411\) to \(0.1483\), averaged over all test users and items. Does that number support the claim? Name the table you would ask for.
E8. (compute) — Sizing a two-tower retriever. A film site has 2 million films, each embedded by an LLM-grade text encoder into a 768-dimensional vector of 4-byte numbers, for an approximate nearest-neighbor index; 10,000 films are added every day.
How much memory do the film vectors take, in gigabytes (1 GB is \(10^9\) bytes), and how much at 2 bytes per number?
What must run each day for the new films, what share of the catalog does one day’s addition make, and how many bytes of new vectors is that?
Every user vector is computed in one batch at 02:00. A user clicks a film at 14:05 and opens the next page. How old is the vector the batch can serve, in hours, and what share of a 24-hour day is that? What is the oldest a batch vector can ever be?
The encoder embeds 200 films per second on one GPU. How long do the day’s 10,000 new films take, and how long would a full re-embedding of the 2 million films take after you swap the encoder?
E9. (apply) — Which interactions may a test use? One user’s log, as (date, film): 12 Jan 2025 A; 3 Mar 2025 B; 20 Apr 2025 C; 9 Jun 2025 D; 2 Aug 2025 E; 14 Sep 2025 F. A vendor reports a “zero-shot” result for an LLM recommender whose training data ends on 31 March 2025.
With a time-based split, which of the six interactions may serve as test targets, and what share of the log is that?
A second LLM, whose training data runs to 30 September 2025, is tested on the same log. How many interactions can serve as test targets, and what should the evaluator do instead?
In one sentence, what would a high score from the second LLM on a random two-out-of-six hold-out be measuring?
E10. (extend / apply) — Pick the roles a budget allows, then defend them on cost. A small film site has 30,000 films, a 1.5 s serving budget, an already-good LightGCN model, and new titles every week, at 200,000 requests a day.
Using §4.1’s latency column, which of the four roles (§2.1 as a reranker, §2.2, §2.3, §2.4 with the four steps of its worked count) does the budget leave open on the per-request path, and which does it rule out?
What does each of the pure-CF serve ($0.004 per 1k) and the 50-film LLM reranker ($25 per 1k) cost per day, and in what ratio?
Sketch the offline vs. serve split you would run for the new titles, and name one thing you would still measure per bucket rather than trusting top-line NDCG.
7. How this wraps the modeling ladder
Six chapters of models, and the metrics that judge them, are behind you, and Ann still has no list on her screen. Where does this chapter sit among them, and what is missing before she gets one in milliseconds?
Parts III–V climbed a ladder of models and the metrics that judge them: Traditional Recommender Systems (similarity, matrix factorization) → Evaluation Metrics → Sequential & Session-Based Recommendation → From Graphs to LightGCN → Self-Supervised & Contrastive Learning → The Spectral / Graph-Filter View. This chapter is the capstone of that ladder: it places the whole collaborative line inside the broader field of large language models and recommendation, and shows that the frontier most relevant here, the LLM as a semantic second view aligned to a graph backbone, is the direct continuation of From Graphs to LightGCN and Self-Supervised & Contrastive Learning.
Featureless ID embeddings (From Graphs to LightGCN) gained self-supervised views (Self-Supervised & Contrastive Learning) and now gain a semantic one (this chapter). Now the collaborative signal and the LLM’s world knowledge sit on the same axis.
A ladder of models is not yet a recommender. Every chapter so far deferred the same machinery: where the negatives come from, how the loop is actually trained, how a top-\(K\) list is served in milliseconds. That is Part VII, Building a Recommender, whose first chapter, Training and Serving a Recommender, is next; Part VIII, Frontiers, then walks the field’s edges. The models are done; the system is not.
In one line: the ladder of six model-and-metric chapters gains one more step here, a single extra loss term (worth \(0.31\) on the worked item) with serving still one dot product, and what is missing is the training and serving machinery of Part VII.
8. Glossary
| Term | Plain meaning |
|---|---|
| LLM | Large Language Model; maps text to vectors and generates text, carrying broad world knowledge. |
| Serving path / serving | Serving is answering a live request; the serving path is everything that runs between the request and the list shown (§1). |
| Latency | How long the serving path takes, in milliseconds (ms); p50 is the median, the time half of the requests beat (§4.1). |
| Backbone | The main recommender model that does the scoring; an enhancer adds to it instead of replacing it (§1). |
| Prompt | The text handed to an LLM: the question plus whatever it should know (§2). |
| Token | The piece of text a model reads or writes in one step: a short word, or part of a long one (§2). |
| Forward pass | One run of an input through the whole network to get its output; the unit of an LLM’s serving cost (§1). |
| Order of magnitude | A factor of 10; three orders of magnitude is a factor of 1,000 (§4.1). |
| Pointwise | Scoring each candidate with its own separate model call, as against scoring a whole list at once (§4.1). |
| Collaborative signal | What can be learned from who interacted with what (From Graphs to LightGCN, Self-Supervised & Contrastive Learning, The Spectral / Graph-Filter View), independent of item content. |
| Cold start | A new user or item with (almost) no interactions, so pure CF has nothing to go on. |
| LLM-as-recommender | The LLM directly ranks or selects items from a prompt of the user’s history (§2.1). |
| LLM-as-reranker | The recommender role in its common production form: the LLM re-scores a short top-\(K\) shortlist that a cheap retriever already fetched, instead of ranking the whole catalog. Reranking is the funnel’s ranking stage under a second name (§2.1). |
| Cross-encoder | A model that reads a (user, item) pair together in one forward pass, so its score cannot be precomputed. A specialist reranker is usually one, and it is not an LLM (§2.1). |
| Zero-shot / few-shot | Zero-shot: a frozen model is simply asked, with no examples and no extra training. Few-shot: still frozen, but a handful of solved examples are pasted into the prompt (§2.1). |
| Fine-tuning / instruction tuning | Continuing to train a pretrained model on task data. Instruction tuning uses prompt–answer pairs (“User liked A, B; will they like C? → Yes”), which is how TALLRec and BIGRec are trained (§2.1, §5). |
| Hallucination | An LLM confidently producing an item that is not in the catalog; reduced by constrained decoding, candidate-list forcing or post-hoc filtering (defined in §1, fixes in §2.2). |
| Constrained decoding | Restricting what the model may generate to a valid vocabulary (here, real item identifiers), so an invalid answer cannot be produced at all (§2.2). |
| Codebook / codeword | A small fixed menu of prototype vectors (the codebook); each entry is a codeword. Quantizing a vector means replacing it by its nearest codeword’s symbol (§2.2). |
| Residual quantization | Quantize a vector, subtract the codeword, quantize what is left over against a second codebook, and so on. The chosen codewords add back up to the vector, and each level usually shrinks the error, though with an arbitrary codebook it can grow (§2.2). |
| Beam width \(W\) | How many partial candidates beam search carries forward at each step; the surviving completed tuples, ranked by probability, are the top-\(K\) (§2.2). |
| DPO (direct preference optimization) | Fine-tuning a model on pairs of preferred and rejected outputs so it favors the preferred kind; the “alignment” of OneRec’s training, not the embedding alignment of §3 (§2.2). |
| MoE (mixture of experts) | Several specialist sub-networks plus a trained gate that weights them and sums their outputs; KAR’s adapter for turning LLM knowledge into features (§2.3). |
| RAG (retrieval-augmented generation) | Retrieving relevant evidence and inserting it into the LLM’s prompt so it reasons over real data instead of pretrained memory (§5). |
| TALLRec | An LLM fine-tuned on recommendation instructions: a Tuning framework to Align LLMs with Recommendation (§2.1). |
| P5 | One text-to-text model for five families of recommendation task: “Pretrain, Personalized Prompt & Predict Paradigm”, five P’s (§2.1). |
| LlamaRec | Two-stage recommendation: a small sequential model retrieves a shortlist, a fine-tuned LLM scores it in one pass from the probabilities of the candidates’ labels (§2.1). |
| TIGER | The system that introduced Semantic-ID generative recommendation: Transformer Index for Generative Recommenders (§2.2). |
| OneRec | Kuaishou’s end-to-end generative recommender, which replaces the retrieval-and-ranking cascade with one model tuned by iterative DPO (§2.2). |
| KAR | Knowledge Augmented Recommendation: prompt an LLM for user reasoning and item facts offline, then feed them to any backbone through a mixture-of-experts adapter (§2.3). |
| EasyRec | A language model tuned with contrastive learning so that vectors from text line up with collaborative behavior: the feature-encoding cell of the enhancer (§2.3). |
| BIGRec | Bi-step grounding: an instruction-tuned LLM writes the next item by name, and the written text is matched to the nearest catalog item by embedding (§5). |
| Two-tower LLM-embedding retrieval | Embed users and items with an LLM-grade encoder offline, then retrieve by ANN over those vectors at serve time: enhancer-class cost, LLM semantics on the retrieval path (§4). |
| ANN (approximate nearest-neighbor) | Fast vector search that returns almost the closest vectors to a query, trading a little accuracy for large speed-ups; how embedding retrieval is served (Training and Serving a Recommender §7). |
| Generative recommendation | Recommend by generating an item’s Semantic ID token by token (§2.2). |
| Semantic ID | A short sequence of content-derived codewords identifying an item (a quantized text embedding) (§2.2). |
| RQ-VAE | The residual-quantized variational autoencoder that learns the Semantic-ID codebooks by reconstructing the content embedding (§2.2). |
| Beam search (decoding) | Keep the \(W\) best partial ID prefixes per step; the completed tuples, ranked, are the top-\(K\) recommendations (§2.2). |
| Autoregressive generation | Writing an output one token at a time, each token chosen given the ones before it (§2.2). |
| Logit | The raw score an output layer gives a label before the softmax turns the scores into probabilities (§2.1). |
| HSTU | Hierarchical Sequential Transduction Units: Meta’s generative recommender that reads a user’s sequence of actions and writes the next one (§2.2). |
| LLM-as-enhancer | The LLM produces semantic features or profiles offline that feed or align with a classical model (§2.3). |
| RLMRec | An enhancer that aligns an LLM-written semantic embedding with a CF embedding via contrastive or generative alignment (§3). |
| Cross-view alignment | Pulling a node’s two embeddings (collaborative \(\mathbf{e}_i\) and semantic \(\mathbf{s}_i\)) together with InfoNCE (§3). |
| BM3 | A multimodal recommender that trains without negative samples by comparing an embedding with a dropout copy of itself (§5). |
| Agentic recommender | An LLM that plans, remembers, and calls tools over multi-turn conversation (§2.4). |
9. References
- Bao, K., Zhang, J., Zhang, Y., Wang, W., Feng, F., & He, X. (2023). TALLRec: An effective and efficient tuning framework to align large language model with recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys ’23) (pp. 1007–1014). https://doi.org/10.1145/3604915.3608857 arXiv:2305.00447
- Bao, K., Zhang, J., Wang, W., Zhang, Y., Yang, Z., Luo, Y., Chen, C., Feng, F., & Tian, Q. (2025). A bi-step grounding paradigm for large language models in recommendation systems (BIGRec). ACM Transactions on Recommender Systems, 3(4), 1–27. https://doi.org/10.1145/3716393 arXiv:2308.08434 (v1, Aug 2023).
- DeepSeek-AI (Guo, D., et al.) (2025). DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature, 645, 633–638. https://doi.org/10.1038/s41586-025-09422-z arXiv:2501.12948
- Deng, J., Wang, S., Cai, K., Ren, L., Hu, Q., Ding, W., Luo, Q., & Zhou, G. (2025). OneRec: Unifying retrieve and rank with generative recommender and iterative preference alignment. arXiv:2502.18965
- Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, M., & Wang, H. (2023). Retrieval-augmented generation for large language models: A survey. arXiv:2312.10997 (a survey of retrieval-augmented generation for large language models in general, not for recommendation; §5 cites it for the RAG pattern itself).
- Geng, S., Liu, S., Fu, Z., Ge, Y., & Zhang, Y. (2022). Recommendation as language processing (RLP): A unified pretrain, personalized prompt & predict paradigm (P5). In Proceedings of the 16th ACM Conference on Recommender Systems (RecSys ’22) (pp. 299–315). https://doi.org/10.1145/3523227.3546767 arXiv:2203.13366
- He, X., Deng, K., Wang, X., Li, Y., Zhang, Y., & Wang, M. (2020). LightGCN: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’20) (pp. 639–648). https://doi.org/10.1145/3397271.3401063 arXiv:2002.02126
- Hou, Y., Zhang, J., Lin, Z., Lu, H., Xie, R., McAuley, J., & Zhao, W. X. (2024). Large language models are zero-shot rankers for recommender systems. In Proceedings of the 46th European Conference on Information Retrieval (ECIR), LNCS 14609 (pp. 364–381). https://doi.org/10.1007/978-3-031-56060-6_24 arXiv:2305.08845
- Lin, J., Dai, X., Xi, Y., Liu, W., Chen, B., Zhang, H., Liu, Y., Wu, C., Li, X., Zhu, C., Guo, H., Yu, Y., Tang, R., & Zhang, W. (2025). How can recommender systems benefit from large language models: A survey. ACM Transactions on Information Systems, 43(2), 1–47. https://doi.org/10.1145/3678004 arXiv:2306.05817 (The survey’s five “where” stages: feature engineering · feature encoder · scoring/ranking · user interaction · pipeline controller.)
- Liu, Z., Wang, S., Wang, X., Zhang, R., Deng, J., Bao, H., Zhang, J., Li, W., Zheng, P., Wu, X., Hu, Y., Hu, Q., Luo, X., Ren, L., Zhang, Z., Wang, Q., Cai, K., Wu, Y., Cheng, H., Cheng, Z., Ren, L., Wang, H., Su, Y., Tang, R., Gai, K., & Zhou, G. (2026). OneRec-Think: In-text reasoning for generative recommendation. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (ACL 2026), Volume 1: Long Papers (pp. 2664–2681). https://doi.org/10.18653/v1/2026.acl-long.123 arXiv:2510.11639
- Peng, Q., Liu, H., Huang, H., Yang, J., Yang, Q., & Shao, M. (2025). A survey on LLM-powered agents for recommender systems. In Findings of the Association for Computational Linguistics: EMNLP 2025 (pp. 11574–11583). https://doi.org/10.18653/v1/2025.findings-emnlp.620 arXiv:2502.10050
- Rafailov, R., Sharma, A., Mitchell, E., Manning, C. D., Ermon, S., & Finn, C. (2023). Direct preference optimization: Your language model is secretly a reward model. In Advances in Neural Information Processing Systems 36 (NeurIPS ’23) (pp. 53728–53741). arXiv:2305.18290
- Rajput, S., Mehta, N., Singh, A., Keshavan, R. H., Vu, T., Heldt, L., Hong, L., Tay, Y., Tran, V. Q., Samost, J., Kula, M., Chi, E. H., & Sathiamoorthy, M. (2023). Recommender systems with generative retrieval (TIGER). In Advances in Neural Information Processing Systems 36 (NeurIPS ’23) (pp. 10299–10315). arXiv:2305.05065
- Ren, X., Wei, W., Xia, L., Su, L., Cheng, S., Wang, J., Yin, D., & Huang, C. (2024). Representation learning with large language models for recommendation (RLMRec). In Proceedings of the ACM Web Conference 2024 (WWW ’24) (pp. 3464–3475). https://doi.org/10.1145/3589334.3645458 arXiv:2310.15950
- Ren, X., & Huang, C. (2025). EasyRec: Simple yet effective language models for recommendation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 17717–17732, as the ACL Anthology gives them; Crossref gives 17728–17743). https://doi.org/10.18653/v1/2025.emnlp-main.894 arXiv:2408.08821
- Wang, L., Zhang, J., Yang, H., Chen, Z., Tang, J., Zhang, Z., Chen, X., Lin, Y., Sun, H., Song, R., Zhao, W. X., Xu, J., Dou, Z., Wang, J., & Wen, J.-R. (2025). User behavior simulation with large language model-based agents (RecAgent). ACM Transactions on Information Systems, 43(2), 1–37. https://doi.org/10.1145/3708985 arXiv:2306.02552 (v1, Jun 2023).
- Wu, L., Zheng, Z., Qiu, Z., Wang, H., Gu, H., Shen, T., Qin, C., Zhu, C., Zhu, H., Liu, Q., Xiong, H., & Chen, E. (2024). A survey on large language models for recommendation. World Wide Web, 27(5), Article 60. https://doi.org/10.1007/s11280-024-01291-2 arXiv:2305.19860
- Xi, Y., Liu, W., Lin, J., Cai, X., Zhu, H., Zhu, J., Chen, B., Tang, R., Zhang, W., & Yu, Y. (2024). Towards open-world recommendation with knowledge augmentation from large language models (KAR). In Proceedings of the 18th ACM Conference on Recommender Systems (RecSys ’24) (pp. 12–22). https://doi.org/10.1145/3640457.3688104 arXiv:2306.10933
- Yu, J., Yin, H., Xia, X., Chen, T., Cui, L., & Nguyen, Q. V. H. (2022). Are graph augmentations necessary? Simple graph contrastive learning for recommendation (SimGCL). In Proceedings of the 45th International ACM SIGIR Conference (SIGIR ’22) (pp. 1294–1303). https://doi.org/10.1145/3477495.3531937 arXiv:2112.08679
- Yue, Z., Rabhi, S., de Souza Pereira Moreira, G., Wang, D., & Oldridge, E. (2023). LlamaRec: Two-stage recommendation using large language models for ranking. In PGAI@CIKM 2023. arXiv:2311.02089
- ZeroEntropy (2025). Should you use LLMs for reranking? A deep dive into pointwise, listwise, and cross-encoders. Vendor blog post, 5 September 2025. https://www.zeroentropy.dev/articles/should-you-use-llms-for-reranking-a-deep-dive-into-pointwise-listwise-and-cross-encoders (a vendor guide, so indicative rather than measured; the page, read on 17 October 2026, links a related post on the same site saying that the vendor was acquired by Notion on 24 July 2026 and that its products stay supported until 4 September 2026, a date now past, so the prices and latencies are a September 2025 snapshot of the four LLMs it timed).
- Zhai, J., Liao, L., Liu, X., Wang, Y., Li, R., Cao, X., Gao, L., Gong, Z., Gu, F., He, J., Lu, Y., & Shi, Y. (2024). Actions speak louder than words: Trillion-parameter sequential transducers for generative recommendations. In Proceedings of the 41st International Conference on Machine Learning (ICML 2024), PMLR 235, 58484–58509. arXiv:2402.17152 (the arXiv listing names a different ‘He’ among its twelve authors, Michael He, than the proceedings, Jiayuan He).
- Zhou, G., Deng, J., Zhang, J., et al. (2025). OneRec technical report. arXiv:2506.13695 (65 authors in the arXiv record; only the abstract was read).
- Zhou, G., Hu, H., Cheng, H., et al. (2025). OneRec-V2 technical report. arXiv:2508.20900 (75 authors in the arXiv record; only the abstract was read).
- Zhou, X., Zhou, H., Liu, Y., Zeng, Z., Miao, C., Wang, P., You, Y., & Jiang, F. (2023). Bootstrap latent representations for multi-modal recommendation (BM3). In Proceedings of the ACM Web Conference 2023 (WWW ’23) (pp. 845–854). https://doi.org/10.1145/3543507.3583251 arXiv:2207.05969
- Zhuang, S., Ma, X., Yao, Z., Wang, S., Koopman, B., Lin, J., & Zuccon, G. (2026). Rank-R1: Enhancing reasoning in LLM-based document rerankers via reinforcement learning. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’26) (pp. 4419–4425). https://doi.org/10.1145/3805712.3809961 arXiv:2503.06034
Reference notes. For TALLRec and EasyRec only the abstract was read; KAR’s text was searched for a cold-start or long-tail analysis, and none was found. Pages come from Crossref, except EasyRec (the ACL Anthology gives 17717–17732, Crossref gives 17728–17743, and we print the proceedings’) and Zhai et al. (PMLR). BIGRec’s Crossref record has volume, issue and pages but no article number, so none is printed.
Author lists are complete as the record gives them, except the group author DeepSeek-AI (more than a hundred names) and the two OneRec technical reports (65 and 75 names, shortened to the first three). Names used in passing (Llama, SGL, LightGCL, GCCF, DCCF, AutoCF) have their entries in the chapters that own them; BERT and GPT likewise have theirs in Representation Learning & the Transformer, and T5 is named only to explain the name of P5, so it has no entry.
Cost anchors (§4.1) are order-of-magnitude figures, not prices: \(50\) candidates \(\times\) about \(500\) tokens \(= 25{,}000\) tokens per request, \(25\) million per \(1{,}000\) requests, and at a round $1 per million input tokens that is $25; the vendor guide’s own prices would give $1.25 to $20 for LLMs and $0.6 to $1.3 for cross-encoders on the same inputs. Reranker model picks are from the Implementation Choices appendix (its ANN latency is our own measurement there, on one machine); RLMRec’s extra training time is from its Table 4 and Section 4.4.3, per epoch. Re-measure on your own stack before quoting.