Appendix A — Implementation Choices: A Practitioner’s Field Guide
How to read this appendix
A practitioner reference appendix, not a step in the conceptual ladder. Read it when you are about to build — when the question stops being “how does this work?” and becomes “which one do I actually install?” It is pointed to from Representation Learning & the Transformer (embeddings), Traditional Recommender Systems (vector search, frameworks), Evaluation Metrics (reproducible baselines), Datasets & Benchmarks (frameworks and baselines), Large Language Models and Recommendation (retrieval, reranking), Training and Serving a Recommender (vector search, serving), Click-Through Rate Prediction & Feature Interactions (§9, on trusting a leaderboard) and Generating Features with Large Language Models (the sections on embedders, vector search and language models, §2 to §4).
Assumes: no chapter in particular; the software words are in the vocabulary table of §1, and every other term is explained where it first appears (and in §8).
It is Friday afternoon and Cara has to ship a “more like this” row for Ann’s film shop by Monday. She needs a model that turns a film blurb into numbers, a place to search those numbers, and maybe a small language model to write the blurbs. Every blog post names a different winner. This appendix is for that moment.
The rest of this handbook teaches the ideas — and ideas age slowly. This appendix answers a different, faster-moving question that every practitioner hits the moment they leave the whiteboard: of the dozens of open-source models and libraries for each job, which one do I actually pick? The options are many, the marketing is loud, and the answer is usually “it depends.” This appendix makes the “it depends” concrete.
Two promises, adapted to a field guide:
- Every leaderboard number is dated and traced to a primary source — a public leaderboard, a model card (the documentation page a publisher posts alongside a released model, stating its size, license, training data and scores), or a paper; most named picks link to their model card, repository or vendor page in §9 (a few newest ones are cited by vendor announcement only). In this corner of the field, an undated claim is a wrong claim. Where we have only experience and no source, the sentence says claim from practice, not tested here. (§2 to §7 keep this promise in their captions and labels, §9 holds the sources. The Best for columns and the statements of usual practice in the tables are our reading, claims from practice, not tested here.)
- Every brief gives you a decision rule you can act on, and how to read its benchmark critically — so when the model names change (they will), you can re-decide for yourself. (The “How to choose” parts of §2 to §6 and the “In one line” parts of §2 to §7 keep this promise.)
Reading order:
- §1 how to choose anything here — a vocabulary box, then the five rules that outlast the leaderboard.
- §2 text embedding models. §3 vector search / approximate nearest-neighbor (ANN) indexes, then reranking and cost-and-scale anchors.
- §4 open large language models (LLMs) & small language models (SLMs), then the serving runtime. §5 recommender-system (RecSys) frameworks & libraries.
- §6 named-entity recognition (NER) & natural-language processing (NLP) libraries (text → features). §7 speech (adjacent; brief).
- §8 glossary; §9 references (grouped, web-verified).
1. How to choose anything here — five rules that outlast the leaderboard
Bob finds two embedding models on a leaderboard, a public ranking of scores. One scores a point higher but needs a far bigger computer chip, the kind that does thousands of multiplications at once (a graphics processing unit, GPU); the other is under a license his company’s lawyer has never heard of. Neither fact is on the leaderboard. This section gives Bob the questions to ask first, in an order that does not change when the model names do. It starts with the words the later sections use, then dates the snapshot, then gives the five rules.
Words you will meet
This appendix is about software, so it leans on a handful of words the rest of the handbook never needed. One plain line each, all used from here on:
| Word | What it means |
|---|---|
| library | a package of ready-written code you download and call from your own program |
| framework | a bigger library that also fixes the shape of the program (data loading, training loop, evaluation); you fill in the parts |
| repository (“repo”), commit, release | the folder of code a project publishes; one saved change to it; a numbered, packaged version of it |
| branch, tag, README | a parallel copy of the code that someone is working on; a name pinned to one version; the front page of a repository, saying what the project is and how to use it |
| open source | the code is published under a license that lets anyone use, change and redistribute it — as recognized by the Open Source Initiative (OSI), the body that approves such licenses |
| weights, checkpoint, model card | the learned numbers of a trained model, downloadable as a file; one saved copy of them (a model often has several, e.g. before and after a later round of training); the documentation page published beside them (size, license, training data, scores) |
| benchmark, leaderboard | a fixed public test; the public ranking of scores on it |
| Hugging Face (HF), Hub, Space | the website and company where most open models are published; its Hub holds each model’s page and its metadata fields (license, size, and whether it is gated, meaning download only after accepting terms); a Space is a small web app hosted there, such as a leaderboard |
| application programming interface (API) model | one you cannot download: your program sends each request over the internet to the maker’s servers and pays per use — the opposite of running the weights yourself |
| parameters, “7B” | the learned weights of a network (Neural Networks & Back-propagation), counted with M = million and B = billion: a 7B model has 7,000,000,000 of them, which sets how much memory it needs and roughly how slow each answer is |
| quantization | storing each number in fewer bits: a coordinate normally kept as a 32-bit decimal (float32) is rounded to one of 256 levels (int8, 1 byte, \(4\times\) smaller) or to its sign alone (binary, 1 bit, \(32\times\) smaller), at a small accuracy cost |
| JavaScript Object Notation (JSON) | a plain-text format for records, e.g. {"brand": "Acme", "material": "leather", "fit": "slim"} |
| central processing unit (CPU), graphics processing unit (GPU) | the computer’s general-purpose processor; the chip that does thousands of multiplications at once, which is what a neural network needs |
| random-access memory (RAM), video RAM (VRAM), solid-state drive (SSD) | the computer’s fast working memory; the GPU’s own memory, which must hold the whole model; disk storage — far larger and cheaper than RAM, and slower |
| latency, throughput | how long one request takes (e.g. 0.2 milliseconds, ms); how many requests are answered per second when many arrive together (§3) |
| context (context window) | the longest input a model reads at once, counted in tokens (word pieces): “8K context” \(\approx\) 8,000 tokens \(\approx\) 6,000 English words; anything longer is cut off |
⟹ Dated snapshot — verify before you rely on a name. This guide was compiled and web-verified on 2026-06-21. The §4 license column was re-read on 2026-09-17, the license cells of every other table against the project’s own license file, README or model card on 2026-09-25, and the dated facts listed at the end of §8 on 2026-10-17 and again on 2026-10-31 (the revision paragraphs there say what each pass re-read). It is the fastest-moving material in the handbook: leaderboards change often and the top open large language model (LLM) is typically a few months old.
The model names below are a snapshot; the decision rules in the five rules that follow, and the way to read each benchmark, are the durable part. Our own rule of thumb is that a model name here has a half-life of about three months (a claim from practice, not tested here). The §9 leaderboard URLs are the live source; re-check them before committing a name to production. Where a figure was contested across sources, this appendix says so rather than inventing false precision.
Before any specific brief, the meta-method. These five rules recur in every category below; internalize them and you can re-evaluate any option, even ones released after this appendix.
The benchmark is a prior, not a verdict — validate on your data. Every leaderboard here is general-domain, and some have been shown to be gameable (§4 gives an arena case, §9 a paper on MTEB): MTEB (the Massive Text Embedding Benchmark) ranks embedding models, ann-benchmarks (approximate-nearest-neighbor benchmarks) ranks vector-search indexes (§3), LMArena ranks chat LLMs by human votes, and CoNLL (the shared-task data sets of the Conference on Computational Natural Language Learning) scores entity extraction (§6). A model trained on benchmark-similar data can top the chart yet generalize worse out-of-domain than a lower-ranked one (§9 lists a 2025 paper on this gap).
Use the leaderboard to shortlist 2–3 finalists, then rank them on a tiny in-domain eval you build yourself (worked just after the five rules). Leaderboard rank does not reliably transfer to your domain — use it to shortlist, never to decide.
License is a hard gate — check it first, and check the version. “Open weights” (the trained model file is downloadable) \(\ne\) “open source” (an OSI-approved license, as the vocabulary table above defines it) \(\ne\) “open data” (the training data is published too). Four families cover almost everything you will meet — they are tabulated right after this list. Read the weights license (it can differ from the repo’s code license), and re-check per version — Gemma (Google’s family of open-weights language models) switched from a custom license to Apache-2.0 at version 4; assuming “Gemma = Apache” for an older version is a compliance bug.
Pick the smallest thing that clears your metric. In a recommender the model is almost always a feature, not the product — an embedding, a generated tag, a candidate list. Suppose a 7B embedder scores one benchmark point higher than a 560M one, and the 560M one already passes your offline bar. Take the 560M one: the 7B one has \(7000/560 = 12.5\) times the parameters, hence roughly that much arithmetic per text, to buy a point you may not see on your data. Likewise a 4B LLM can beat an 8B one that doubles the cost (a claim from practice, not tested here). Shrink further with quantization (fewer bits per number, in the vocabulary table above) and Matryoshka (truncatable embeddings, §2). Start small; scale only when a measured bottleneck forces it.
Maintenance is a feature. An abandoned library is a liability — no bug fixes, no security patches, breakage the next time a package it depends on changes. Check, on GitHub (the site where most open-source projects publish their code), the date of the last real commit to the main branch (the project’s primary line of code), when the last numbered release shipped, and whether reported problems (“issues”) get answered — before adopting. Ignore GitHub’s “updated” and “pushed” timestamps: they move when anyone pushes a tag or a side branch (both defined in the vocabulary table above), even if nobody touched the main branch. A clever dead project loses to a boring maintained one.
This is a snapshot; the rule is the asset. Re-derive, don’t memorize. When a new model tops the board next quarter, run it through rules 1–4 — your data, its license, its size, its maintenance — and decide for yourself. The brief is perishable; the method is not.
Rule 1’s in-domain eval, worked. Take 100 searches your own shoppers really typed, each paired with the film they ended up choosing, and score each finalist by how often that film is in its top 10. A hundred is a small sample: if a model’s true hit rate is 0.5, the score from 100 searches wobbles (its standard error) by about \(\sqrt{0.25/100} = 0.05\), five points. So score the finalists on the same searches and treat a gap below about ten points as a tie, or use more searches. A gap between two scores wobbles more than one score, about seven points if the scores were independent. Ten points is then only about 1.4 such wobbles, and fourteen points (two wobbles) is the firmer call. Scoring both on the same searches shrinks the gap’s wobble by an unmeasured amount, so ten points is a workable tie bar, not a safe one.
Rule 2’s four families, once, so that every “License” column below decodes:
| Family | Examples | What it lets you do |
|---|---|---|
| permissive | Apache-2.0, MIT, BSD (the Berkeley Software Distribution licenses) | use it, change it, sell it — just keep the copyright notice |
| copyleft | GPL (GNU General Public License), AGPL (its Affero variant) | when you distribute it, you must publish your own source under the same terms — AGPL even when you merely serve the modified version over a network |
| non-commercial | CC-BY-NC, a Creative Commons license: BY means credit the author, NC means no commercial use, and a further SA (share-alike) means derivatives keep the same license | study it and publish about it, but use it in nothing commercial |
| custom | Llama’s community license, Google’s pre-Gemma-4 terms | read the conditions — these are not OSI-approved open source |
If you must pick today — the default stack. The rules above outlive any name, but here is where they land for this handbook’s jobs, as of this snapshot. Start here; let rules 1–4 (your data, its license, its size, its maintenance) move you off it only when a measured need forces the change.
| Job | Default pick | What it is |
|---|---|---|
| Text embedding (§2) | multilingual-e5 or BGE-M3 | small, permissively licensed encoders; BGE-M3 is from the Beijing Academy of Artificial Intelligence and returns dense, sparse and multi-vector forms (three ways to represent a text, explained in §2); a newer MIT-licensed encoder of the same size, harrier-oss-v1-0.6b, scores higher on the live board; why it is not the default is in §2 |
| Vector search (§3) | HNSW or pgvector | HNSW (Hierarchical Navigable Small World) is a search index that links each vector to its near neighbors (Training and Serving a Recommender §7); pgvector is an add-on that gives PostgreSQL (a widely used open-source database) vector search |
| Feature generation (§4) | an Apache-2.0 or MIT LLM of about 0.5–9B parameters, served with vLLM | vLLM is an open-source program that runs an LLM on a GPU and answers many requests at once |
| Baselines (§5) | Cornac (Apache-2.0); RecBole for paper baselines only | two research libraries of ready-made recommender models; RecBole’s README restricts use to academic purposes (§5) |
| Text to features (§6) | spaCy or GLiNER | spaCy is a fast natural-language-processing (NLP) library; GLiNER (Generalist model for Named-Entity Recognition, NER) finds any entity type you name, with no training |
In one line: ask first what your own data says, then the license, then the size, then whether anyone still maintains it; a leaderboard only shortlists.
2. Text embedding models
Cara has 40,000 film blurbs. A shopper types “a slow, sad space film” and the right answer, Interstellar, never uses the word “sad”. An embedding model is the tool that makes the two meet: it turns each text into a list of numbers so that texts with similar meaning land near each other.
This is the engine of semantic retrieval and of every LLM feature pipeline in recsys (recommender systems — the field’s own shorthand, used throughout this appendix): text (item descriptions, user profiles, reviews) \(\rightarrow\) a vector you can compare by cosine (Linear Algebra for Machine Learning §4) and serve through a vector index (§3). It is the same kind of object as the embeddings the graph chapters propagate — a \(d\)-dimensional vector — but not the same vector: a graph embedding \(\mathbf{e}_i\) is learned from clicks, while a text embedding \(\mathbf{s}_i\) is computed from words by a frozen encoder. They live in different spaces until something aligns them, which is exactly the job Large Language Models and Recommendation §3 gives RLMRec. Do not expect one to substitute for the other.
How to choose. Default to a small, permissive workhorse — multilingual-e5-large-instruct (560M, MIT) or BGE-M3 (MIT, 8K-token context, gives dense + sparse + multi-vector from one model). A newer encoder of about the same size, Microsoft’s harrier-oss-v1-0.6b (MIT, released 2026-03-27 by the leaderboard’s metadata; the Hub repository dates from 2026-03-30), scores higher on the live board (the dated reading below: 69.01 against 63.22 for e5 on the mean, and 70.75 against 57.11 on the retrieval slice). We still name e5 and BGE-M3 first because they are older, more widely deployed and better supported by tools (a claim from practice, not tested here); on the numbers harrier-0.6b is the stronger small model, so put it among your finalists.
Step up to a 7–8B LLM-based embedder (Qwen3-Embedding, Apache-2.0) only when retrieval quality is the measured bottleneck and you can pay the latency and the VRAM. There is no single winner, only a size/latency/license/domain trade-off, so rank your finalists on your own corpus.
Those three words, dense, sparse and multi-vector, name three ways to represent a text. Dense = one ordinary embedding vector, what this section is mostly about: 1,024 numbers for a blurb, say. Sparse = a keyword-weight vector as long as the vocabulary and almost all zeros, like the TF-IDF (term frequency–inverse document frequency) vectors of Traditional Recommender Systems §2: with a six-word vocabulary (film, sad, slow, space, jazz, ship) the blurb “slow, sad space film” is \((1.2,\ 0.8,\ 0.9,\ 1.5,\ 0,\ 0)\), four non-zero weights (illustrative numbers).
Multi-vector = one vector per token, compared token by token — more precise, many times the storage: four tokens of 1,024 numbers each is \(4 \times 1024 \times 4 = 16{,}384\) bytes, four times the \(4{,}096\) bytes of one dense vector. Mixing dense and sparse scores is what “hybrid” retrieval means (§3). In Cara’s case the dense vector is what lets “a slow, sad space film” find Interstellar even though the blurb never says “sad”; the sparse vector would find it only through words the two texts share, such as “space”.
Treat the hosted APIs (Google’s Gemini embeddings, Voyage, Cohere, OpenAI) as a quality ceiling to benchmark against, then self-host if economics or data governance demand it. Use Matryoshka truncation to fit the vector to your index and latency budget.
Matryoshka, worked. Matryoshka Representation Learning (MRL) (Kusupati et al., 2022, §9a) is named after Russian nesting dolls: the model is trained so that the first 64, 256 or 512 numbers of its vector are each a usable embedding on their own. Chop an ordinary embedding the same way and you throw away whatever the model happened to put in the missing coordinates, which is why the training matters. The arithmetic: a 1024-dim float32 vector costs \(1024 \times 4 = 4{,}096\) bytes, so a million items need \(4.10\) gigabytes (GB). Keep only the first 256 numbers — \(1{,}024\) bytes, \(1.02\) GB, four times less — and re-normalize each truncated vector to length 1, so that a plain inner-product index (§3) returns cosines.
What it costs, for one model: Google’s documentation for
gemini-embedding-001lists MTEB scores by output size, \(68.17\) at 1,536 dimensions, \(67.55\) at 512 and \(66.19\) at 256. A \(3\times\) cut costs \(68.17 - 67.55 = 0.62\) points and a \(6\times\) cut costs \(68.17 - 66.19 = 1.98\) points (read 2026-10-17). Those are one model’s numbers; measure your own. This is why the state-of-play bullet below recommends trying truncation early (a claim from practice, not tested here: no other lever, such as a smaller model or quantization, is measured in this appendix).
| Model | Maker | Dim · MRL | License | Best for |
|---|---|---|---|---|
| Qwen3-Embedding (0.6 / 4 / 8B) | Alibaba | \(\le 4096,\) MRL | Apache-2.0 | Ranked first on the MTEB multilingual leaderboard on 2025-06-05 (model card); 100+ langs; 32K context; 0.6B is a strong small option |
| multilingual-e5-large-instruct | Microsoft | 1024 | MIT | The default multilingual workhorse — cheap, permissive, short text |
| BGE-M3 | BAAI (Beijing Academy of Artificial Intelligence) | 1024 | MIT | One model for dense + sparse + multi-vector (hybrid); 8K context |
| harrier-oss-v1 (0.27 / 0.6 / 27B) | Microsoft | 640 / 1024 / 5376 | MIT | First on the MTEB multilingual v2 board read 2026-10-17 (27B: \(74.27\)); the 0.6B size scores \(69.01\) |
| EmbeddingGemma-300M | 768, MRL | Gemma terms | On-device (edge) use; the smallest “good” model | |
| EmbeddingGemma 2 (740M) | 768, MRL | Apache-2.0 (Hub metadata) | Text, image, video and audio in one 768-dim space, 8K context; posted 2026-09-14 | |
| Stella-1.5B (en v5) | NovaSearch | 512–8192, MRL | MIT | Rich truncation ladder for storage/latency tuning (“1024d \(\approx 8192\)d”) |
| Nomic-embed-text-v2 | Nomic | 768, MRL | Apache-2.0 | Fully open (weights + code + data); audit/reproducibility priority |
| Hosted APIs (not open) | Google, Voyage, Cohere, OpenAI | MRL-style (Google); fixed menu (Cohere) | proprietary | Google lists gemini-embedding-001 (text) and gemini-embedding-2 (text, image, video, audio, PDF; 8,192-token inputs), both trained with MRL for 128–3072 dimensions; Cohere lists embed-v5.0-pro and -fast (text and images, 128K-token inputs, 256–2048 dimensions) beside embed-v4.0 (256–1536) |
Reading the benchmark — the MTEB family. The Massive Text Embedding Benchmark (MTEB) (Muennighoff et al., 2023, §9a) and its 250+-language successor MMTEB (Massive Multilingual Text Embedding Benchmark; Enevoldsen et al., 2025, §9a) are the standard. Both score each model on several kinds of task — retrieval (find the right document for a query), classification, clustering, reranking, sentence similarity — and headline the average of those task scores on a 0–100 scale. Only the retrieval column measures what a recommender’s candidate stage actually does.
A dated reading (re-check, the board moves): the Qwen3-Embedding-8B card says the model ranked first on the MTEB multilingual leaderboard with a mean task score of \(70.58\) out of 100 as of 2025-06-05. We re-read the live MTEB(Multilingual, v2) board on 2026-10-17 through its public backend. Microsoft’s harrier-oss-v1-27b (MIT, 27B parameters, released 2026-03-27 by the leaderboard’s metadata) is first with a mean of \(74.27\), ahead of Tencent’s 12B-parameter embedder, second at a mean of \(72.32\). Qwen3-Embedding-8B still scores \(70.58\), yet the board’s rank column lists it fourth, so rank is not the same as mean order. Ordered by mean alone it is third; NVIDIA’s llama-embed-nemotron-8b, which the rank column lists third, scores \(69.46\). The 2025 card claim is therefore stale as a statement about today; look up today’s order yourself (§9).
Three habits keep you honest. (1) Sort by the Retrieval sub-score, not the headline average. For recsys and for retrieval-augmented generation (RAG: fetch relevant documents, then let an LLM answer from them), retrieval is the column that predicts your quality (a claim from practice, not tested here), and a model can top the headline average yet lose on the retrieval slice. (2) Treat a lead of 1–2 points (out of that same 100) as a tie (a rule of thumb from practice, not tested here) — overfitting to benchmark-like data is documented, and the same model carries different scores across sources. (3) Prefer models that disclose training data, and check the zero-shot share — the fraction of MTEB’s tasks the model was not trained on. A high score on tasks it trained on proves little.
2026 state of play.
- LLM-based embedders won the leaderboard — harrier-oss-v1-27b (built on a Gemma 3 text model), Qwen3-Embedding-8B, and GTE-Qwen2-7B (Alibaba’s General Text Embeddings model, built on its Qwen2 LLM; 7B, Apache-2.0 on its model card; rank 23 in the rank column of the board read 2026-10-17, with a mean of \(62.51\), below e5-large-instruct’s \(63.22\) — rank is not mean order) — but the top ones bring 7–27B latency and VRAM, a real serving cost in a recommender. Small LLM-derived models exist for tight budgets (EmbeddingGemma-300M, Qwen3-Embedding-0.6B); it is the size, not the lineage, that costs you.
- Small models are usually the right recsys call (claim from practice, not tested here). A 560M
e5model usually gets most of a larger model’s quality at a fraction of the cost; for an embedding used as a feature, that is the default. The live board (read 2026-10-17) puts numbers on “most”: the mean task score is \(63.22\) for multilingual-e5-large-instruct, \(64.34\) for Qwen3-Embedding-0.6B and \(69.01\) for harrier-oss-v1-0.6b, against \(74.27\) for the 27B model; on the retrieval slice the same models score \(57.11\), \(64.65\), \(70.75\) and \(78.27\). Against the 27B model, e5 keeps \(63.22/74.27 \approx 85\%\) of the mean score but only \(57.11/78.27 \approx 73\%\) of the retrieval score, so “most” is weakest on the slice that matters. - Matryoshka (MRL) is now table stakes — embed once at full dim, store/index at a shorter prefix (e.g. Stella’s \(8192 \rightarrow 1024\) in the table above) to cut ANN (approximate nearest-neighbor, §3) memory and latency; the box above shows a loss of under one point for a \(3\times\) cut on one model. Pair with int8/binary quantization for further shrink. A lever worth trying early (a claim from practice, not tested here; the box measures only truncation, for one model).
- License traps: Apache/MIT are clean (Qwen3, e5, BGE-M3, Stella, Nomic — each verified against its model card, 2026-09-25; harrier-oss-v1 is MIT, read 2026-10-17). Gemma-licensed models (EmbeddingGemma-300M) are gated and not OSI-approved, and EmbeddingGemma-300M is a separate release from Gemma 4, so it does not inherit that family’s Apache-2.0 (§4), while its successor EmbeddingGemma 2 shows Apache-2.0 and no gate in its Hub metadata (read 2026-10-17): check the model, never the brand. NVIDIA’s Nemotron-3.5-Embed (Hub pages created 2026-10-07) carries the license its card names
openmdw-1.1, which is not on the Open Source Initiative’s license list as we read it on 2026-10-17, and the Jina v5 embedding models are CC-BY-NC-4.0. Jina-embeddings-v4’s card says it was first posted under CC-BY-NC-4.0 “due to an error” and that the correct license is the Qwen Research License (read 2026-10-17); Jina’s multimodal model below is CC-BY-NC-4.0. Read before you ship. - Fine-tune only when your domain is genuinely off-distribution (specialist catalogs, non-English item text, behavior-driven similarity that text alone can’t capture) and you have labeled / implicit-feedback pairs. In recsys the bigger win is usually combining a decent off-the-shelf text embedding with collaborative signal, not chasing MTEB points.
Multimodal embeddings (image + text items)
When items are images as well as text (products, films, listings), embed both into one shared space, so a text query can retrieve an image and an image can retrieve text. As of mid-2026, two long-standing open choices are:
- SigLIP 2, Google’s Sigmoid Loss for Language–Image Pre-training model; its ViT-SO400M variant (a Vision Transformer (ViT): the Transformer of Representation Learning & the Transformer applied to image patches; its vision tower has about 400 million parameters, and the whole checkpoint with its text tower has 1.14 billion) is Apache-2.0. The model card reports its evaluation results, taken from the SigLIP 2 paper (Tschannen et al., 2025, §9a); compare there rather than on our word.
- JinaCLIP-v2, Jina AI’s version of CLIP (Contrastive Language–Image Pre-training: image and caption vectors are pulled together exactly as InfoNCE, the contrastive loss of Self-Supervised & Contrastive Learning, pulls positive pairs together). It adds multilingual text and Matryoshka dimensions (truncate to 64-dim to shrink storage), but its weights ship under CC-BY-NC-4.0, so it is not commercial-safe (model card, checked 2026-09-25 and again 2026-10-17).
A third open option is EmbeddingGemma 2 (Apache-2.0; text, images, video and audio in one 768-dim space, per its card, read 2026-10-17). Managed options include Cohere Embed (embed-v5.0-pro and -fast, text and images in one model, per Cohere’s documentation) and Voyage’s multimodal models: the voyage-multimodal-3 model now sits under “older models” in Voyage’s documentation, which recommends voyage-multimodal-3.5 (read 2026-10-17; both reached only through Voyage’s API as far as its documentation shows).
Decision rule: pick one model that maps text and images into the same space (so a single cosine works across modalities), and use Matryoshka or quantization to cap the storage of a large catalog — don’t bolt a separate image model onto a separate text model, because their spaces won’t align.
In one line: start from a small permissive embedder, truncate it with Matryoshka to fit your index, and let your own 100-query test, not the leaderboard average, pick the winner (a gap under about ten points on 100 queries is a tie, and under fourteen is shaky, Rule 1).
3. Vector search / approximate nearest neighbor (ANN)
The shop has 10 million films in all (Cara’s 40,000 blurbs are only the first batch), and a shopper has just tapped “more like this”. Comparing the tapped film with every other one would take too long, so the shop must find a few hundred close films in a few milliseconds. That is the candidate-generation stage of the recommendation funnel (Training and Serving a Recommender §8): cut millions of items to a few hundred fast, so the expensive ranking model scores only those. This section is about how you serve embeddings at scale.
What an ANN index is. Exact search would compare the query vector with every item — millions of dot products per request. An approximate nearest-neighbor (ANN) index organizes the vectors in advance — as a graph of near neighbors, or as clusters — so a query touches only a few thousand of them. The price is that it occasionally misses a true nearest neighbor; hence approximate. Training and Serving a Recommender §7 builds one by hand; this section is about which implementation to install.
How to choose. Start with the simplest thing that fits in RAM. If your catalog is \(\le 1\)–10 M vectors with light filtering, an embedded library — hnswlib or FAISS (Facebook AI Similarity Search) — is enough (a claim from practice, not tested here); you own one index file and run no extra system. For Ann’s 10 million films at 768 numbers each the vectors take \(10^{7}\times 768\times 4 = 30.7\) GB, which fits in the RAM of one ordinary server (the index graph adds some overhead on top), so a library can hold them. Graduate to a vector database the moment you need any of: rich metadata filtering fused with ANN (“in-stock AND in-region AND not-already-seen”), live inserts/updates/deletes, sharding/replication, or hybrid dense+sparse retrieval.
The sparse half of hybrid retrieval is keyword scoring by BM25 (“Best Matching 25”, the standard refinement of the TF-IDF weighting of Traditional Recommender Systems §2: it rewards query words that are rare in the catalog, with diminishing returns for repeats and a correction for long documents). The distinction under all of this: a library runs inside your own program and keeps the index in one file; a vector database is a separate, always-on server that stores the vectors with their metadata, accepts changes while running, can split the data across machines (sharding) and keep spare copies for safety (replication). The shop’s 30.7 GB needs only the library; the moment the shop must hide sold-out films and add new ones all day without rebuilding the file, it needs a database.
If your data already lives in PostgreSQL (“Postgres”, a widely used open-source database queried in the Structured Query Language, SQL) and you are \(\le 10\) M items, the pgvector add-on is often the answer (a claim from practice, not tested here) — it lets that same database do ANN search and filter the results with ordinary SQL conditions, so there is no second system to keep in sync. One caution on filters, spelled out under Filtered ANN below: pgvector applies a filter after it scans the index.
The “Core algorithm(s)” column below is a menu of index types, so here is the menu, one line each. HNSW (the neighbor-graph index of Training and Serving a Recommender §7) is the in-RAM default. IVF (inverted file) groups the vectors into clusters and searches only the few nearest clusters; IVFFlat (inverted file, flat) is IVF with the vectors kept uncompressed; PQ (product quantization; Jégou et al., 2011) compresses each vector to a few bytes of codebook indices — a cousin of the codebook trick behind the Semantic IDs of Large Language Models and Recommendation.
DiskANN (a disk-resident ANN index, built by the Vamana algorithm; Jayaram Subramanya et al., 2019) lives on an SSD instead of in RAM; CAGRA (NVIDIA’s graph index for GPUs; Ootomo et al., 2024) trades single-query latency for batched throughput; ACORN (Patel et al., 2024) and “filterable-HNSW” are variants of HNSW that respect a metadata filter during the graph walk (all in §9a).
When does each win? Stay on HNSW while the vectors fit in RAM. Move to IVF with PQ when they no longer fit and you can accept a small loss of recall (the share of the true nearest neighbors still returned; these pick rules are a claim from practice, not tested here). Use DiskANN when one machine’s SSD must hold billions of vectors, and CAGRA only when you own an NVIDIA GPU and care about batches, not single queries. Reach for ACORN-style filtering when most queries carry a metadata condition. A first project needs none of these but HNSW.
| Tool (library or database?) | Maker | Core algorithm(s) | License | Best for |
|---|---|---|---|---|
| FAISS (library) | Meta | IVF, IVF-PQ, HNSW, optional GPU support | MIT | The Swiss-army library: a wide index menu, optional GPU support, billion-scale on a big box (the last is a claim from practice, not tested here) |
| hnswlib (library) | nmslib | HNSW only | Apache-2.0 | The reference HNSW — one tiny, long-tested dependency when HNSW is all you need |
| Qdrant (database) | Qdrant | HNSW + filterable-HNSW | Apache-2.0 | A good fit for recsys: filterable HNSW (it applies the filter during the graph search) and a query API that takes positive and negative example items |
| Milvus (database) | Zilliz | HNSW / IVF / DiskANN / +GPU | Apache-2.0 | Built for scale: its overview page says it supported billion-scale vectors in 2022 and tens of billions in 2023; GPU ANN (NVIDIA’s CAGRA) inside a database rather than a library |
| pgvector (Postgres add-on) | open / Postgres | HNSW + IVFFlat | PostgreSQL (permissive) | Already on Postgres + \(\le 10\) M + SQL conditions — no new system; filters apply after the index scan (iterative scans since 0.8.0 re-scan when too few rows pass) |
| Weaviate (database) | Weaviate | HNSW (+ACORN, hybrid) | BSD-3 core; parts under the Weaviate license | Strong hybrid (dense+BM25); its current documentation lists a Weaviate Cloud-only Query Agent, and we found no built-in recommender there (read 2026-10-17) |
| LanceDB (embedded database) | LanceDB | IVF and IVF-HNSW, each with optional compressed (quantized) storage (its indexing page, read 2026-10-17) | Apache-2.0 | Runs inside your program with no server; its README claims billion-scale search in milliseconds (a vendor claim, not tested here) |
| Pinecone (managed service) | Pinecone | proprietary | proprietary | The “buy, don’t build” option: a managed service with nothing to run, at the cost of price and lock-in; read its service-level agreement (SLA, a contractual uptime and speed guarantee) yourself |
Also know: DiskANN (on-disk Vamana — billion-scale on one SSD machine when RAM is the wall), ScaNN (Scalable Nearest Neighbors, Google’s library; its README names Linux, with Python 3.9 to 3.13, as the supported platform, read 2026-10-17), and GPU CAGRA (listed by Milvus; it is aimed at batched throughput and index build speed, not single-query latency).
Latency, measured here: we found no published run that gives a median latency for filtered search at about a million vectors for Qdrant, pgvector and Milvus together. Qdrant’s benchmark page (read 2026-10-17) shows charts for 1 M to 10 M vectors without readable latency numbers, and its filtered-search results date from February 2023; the README of Zilliz’s vector-database benchmark, named below (read 2026-10-17), lists no figures. Both are run by a vendor, Qdrant and Zilliz (the maker of Milvus), so we quote no per-database millisecond figure.
What we can give is our own run (2026-10-17, Apple M2 Pro, in-RAM libraries, not databases). Its settings, all of them: \(10^6\) synthetic 128-dimensional vectors, each a random cluster centre (1,000 centres, standard normal) plus noise of 0.5 per coordinate, then scaled to length 1 and compared by inner product (random seed 0); each vector carries a random category from 0 to 9, so 10 % pass the filter; top 10; HNSW with \(M = 16\) links per node and efConstruction \(= 200\) (the number of candidates kept while building the graph), as hnswlib’s index and as faiss’s IndexHNSWFlat (its plain HNSW graph over uncompressed vectors); index builds on 10 threads; Python 3.14.5, faiss 1.15.1, numpy 2.5.4 (the hnswlib version was not recorded). The code is not distributed with this book.
The 1,000 queries are drawn the same way. Each is timed once, one at a time on one thread, so the median (p50, half the timings are faster) and the 95th percentile (p95, 19 in 20 are faster) are taken over those 1,000 timings.
With the filter applied inside the graph walk (hnswlib), recall@10 (the share of the true 10 nearest neighbors that the search returned) was \(1.000\) at a p50 of \(0.84\) ms and a p95 of \(1.32\) ms (ef=100, where ef is the search-breadth parameter, the number of candidates the walk keeps); the unfiltered query took \(0.20\) ms (p50).
The filtered query already reached recall \(1.000\) at ef=50 (\(0.55\) ms), and its cost climbed steeply with breadth: \(2.90\) ms at ef=200 and \(6.18\) ms at ef=400, against \(0.25\) and \(0.33\) ms unfiltered. Filtering after the search (faiss, ten times oversampling) ran at \(0.17\) ms at ef=100, but recall was only \(0.88\) and plateaued there as ef grew (\(0.85\), \(0.88\), \(0.88\), \(0.88\) at ef 50 to 400).
The cause is arithmetic, not search effort: fetching \(10 \times 10 = 100\) candidates when 10 % pass leaves \(X\) survivors, where each of the 100 candidates passes with probability \(0.1\) (so \(X\) averages 10 and is sometimes below 10). The share of the ten slots that get filled is \(\min(X,10)/10\), counting at most ten survivors: \(0.7\) when \(X=7\), \(1.0\) when \(X=10\) or \(13\). Averaged over every possible \(X\), that expected share is \(E[\min(X,10)]/10 = 0.881\), which no larger ef can change (the same arithmetic as the 1 % example below).
Read the figures with these caveats. The data are easy (clustered, recall saturates). The filter is a random category unrelated to the vectors, whereas real filters (region, stock) correlate with the embeddings. The hnswlib filter is a Python function called on each candidate, so its time includes interpreter overhead that a native filter would not. hnswlib and faiss are different libraries, so compare their recall, not their times. The machine is one laptop, and a database adds network and storage. So take it as the order of magnitude of the filter’s cost at one setting (about four times the unfiltered query at ef=100, and far more at ef=400), and re-measure on your vectors and filters.
Our reading, a claim from practice, not tested here: for a given index type, recall and throughput depend more on its parameters (such as ef) than on which engine hosts it. For throughput numbers rather than latency, read them live off the two dashboards below — they move with every release, so this appendix teaches how to read the chart instead of freezing one row of it.
Reading the benchmark. ann-benchmarks.com (libraries) and VectorDBBench (databases, production-scale) both report the one chart that matters: recall (fraction of true neighbors found) vs queries per second — the recall/throughput Pareto frontier. Keep the two time words apart: latency is how long one query takes; throughput (queries per second, QPS) is how many are answered per second when many arrive together. A batched GPU index can win the second and lose the first.
Why a curve and not a point? Every index has a knob that trades accuracy for speed — for HNSW, how many graph neighbors it explores per query. Each setting of that knob gives one (recall, QPS) pair; sweeping the knob traces a curve, and the Pareto frontier is the set of best-possible points on it: from a point on the frontier you cannot gain recall without losing speed, or the reverse. To compare two indexes, draw a vertical line at the recall you actually need and take the higher curve there.
Three cautions follow: a method that dominates at 90% recall can lose at 99% (always read QPS at a fixed recall); a fast-query index may take much longer to build (a claim from practice, not tested here; decisive for frequently-rebuilt catalogs); and GPU numbers usually assume large query batches, a different operating point from single-query latency.
The first caution is the one beginners miss, so it is worth a picture. On the recall/QPS frontier, two indexes can cross: the one you would pick from a single headline number is the wrong one once you fix the recall you actually need.
2026 state of play.
- HNSW is the safe default; on-disk (DiskANN) and GPU (CAGRA) extend the extremes (billion-scale on one box; batched throughput).
- Filtered ANN is the real reason to pick a database over a library. Suppose only \(1\%\) of items pass the filter (in stock, in your region). Search-then-filter: fetch the 100 nearest, then drop the failures — on average \(100 \times 0.01 = 1\) survives, so returning 10 means fetching about \(10/0.01 = 1{,}000\). Filter-then-search on a graph index: deleting the other \(99\%\) first also deletes most of the graph’s links, so the search walk gets stranded among the survivors and misses good matches (the mechanism described in Qdrant’s article, cited in the next bullet; not measured here).
- Databases built for filtering avoid both by filtering during the graph traversal, keeping the links but returning only items that pass: Qdrant’s article on filterable HNSW describes applying the filter to the nodes of the graph during the greedy search, and Weaviate’s documentation describes its ACORN strategy, which ignores non-matching objects in its distance calculations and evaluates the neighborhood of candidates with a multi-hop approach, and is the default filter strategy since version 1.34 (both read 2026-10-17; the Qdrant article is dated 2019-11-24 and describes the original design).
- pgvector is the exception. Its README says that with approximate indexes “filtering is applied after the index is scanned”, so a filter that few rows pass can leave fewer results than you asked for; since version 0.8.0 you can enable iterative scans, which keep scanning the index until enough rows pass (for HNSW, up to
hnsw.max_scan_tuplesentries, 20,000 by default), a repair of search-then-filter rather than a filtered traversal (README, 2026-10-17). If filtered retrieval is core — and in recsys it usually is — this capability, not raw QPS, should drive the choice. - Hybrid (dense + sparse/BM25) retrieval is now table stakes — one call blends semantic, lexical (matching the exact words) and metadata signals. The two ranked lists are usually merged by reciprocal-rank fusion (Cormack et al., 2009, §9a): each item scores \(\sum 1/(60 + \text{its rank in that list})\). An item ranked 1st by the embedding and 10th by BM25 scores \(1/61 + 1/70 = 0.0307\); one ranked 3rd in both scores \(2/63 = 0.0317\) and wins — steady agreement beats a single top spot.
- The “normalize \(\rightarrow\) inner-product = cosine” trick (often gotten wrong). FAISS has no native cosine metric: L2-normalize your vectors, then use inner product — for unit vectors the inner product is the cosine. Forget the normalization and you are silently not doing cosine search — a common, quiet recall bug. (Check with Linear Algebra for Machine Learning §4 example: for \(\mathbf{u}=(3,4)\) and \(\mathbf{v}=(4,3)\) the raw inner product is \(24\), while the cosine — and the inner product of the normalized vectors — is \(24/(5\cdot 5)=0.96\).) The trap applies whenever you choose an inner-product metric, always in FAISS; the databases also offer a cosine metric that normalizes for you.
- License watch: cores are mostly Apache/MIT/BSD, but VectorChord (a pgvector successor) is dual-licensed under AGPL or the Elastic License v2 (ELv2) (its license file and README, read 2026-10-17), and across all the databases the managed cloud and enterprise add-ons are separate commercial products, not covered by the open-source engine’s license.
- Legacy: Annoy, Spotify’s older ANN library, has had no release since v1.17.3 (2023-06-14) and its last commit was 2025-10-29 (
gh api, GitHub’s command-line tool, 2026-10-17). Voyager, Spotify’s newer library, describes itself as “like Annoy, but with much higher recall” and uses HNSW (its README, same date). For new work start from HNSW, not Annoy. - Maintenance snapshot (latest release, then last commit on the main branch;
gh apiand the Python Package Index, read 2026-10-17). Rule 4 says to re-run this check on the day you adopt.- FAISS: v1.15.1 (2026-09-16); 2026-10-15.
- hnswlib: v0.9.0 (2026-03-28); 2026-09-14, its README announcing 0.10.0rc2 as a release candidate, not a final release.
- Qdrant: v1.19.2 (2026-10-05); 2026-10-05.
- Milvus: v3.0.3 (2026-10-14); 2026-10-16.
- Weaviate: v1.40.1 (2026-10-13); 2026-10-16.
- LanceDB: v0.40.0 (2026-10-07); 2026-10-16.
- pgvector: 0.8.7 (2026-10-01 per its changelog); same-day last commit, no GitHub release object.
- VectorChord: 1.1.1 (2026-02-28); 2026-04-23.
- ScaNN: 1.4.2 on the Python Package Index (2025-08-29); its last commit was not read.
Reranking — the cross-encoder stage after retrieval
Ann’s search returned 100 films for “a slow, sad space film”, but the best one sits at rank 37. A heavier model that reads the query and each film together can fix the order, if you only let it read those 100 and not all ten million. Whether it lifts that film from rank 37 to the top is exactly what the “lift on your own evaluation set” test below measures.
Retrieval (above) returns a cheap shortlist of \(N\) candidates (\(N \approx 50\)–\(200\)); a reranker then re-scores those \(N\) with a heavier model that reads the (query, item) pair together — a cross-encoder — and reorders them. This is the ranking stage that Training and Serving a Recommender §8 names: search and LLM writing call the same step reranking because a first ranked list already exists, but it is one stage under two names, and only the final cut shown to the user is the top-\(K\). The retrieve-then-rank funnel is near-universal in modern search and increasingly standard in recsys (rerank a shortlist retrieved by collaborative filtering, CF).
Decision rule: rerank only the shortlist (\(N = 50\)–\(200\)), never the whole catalog. In raw arithmetic the gap is enormous: a cross-encoder reads the query and the item as one text, roughly 2 operations per parameter per token, so a 0.6B-parameter reranker reading 200 tokens does about \(2 \times 0.6\times10^{9} \times 200 = 2.4\times10^{11}\) operations, against \(2 \times 768 = 1{,}536\) for one 768-dimensional dot product, a ratio near \(1.6\times10^{8}\). In wall-clock time on a GPU the gap is far smaller, partly because a GPU runs the pair in parallel; a cross-encoder then costs roughly \(100\)–\(1000\times\) a dot product per pair (a claim from practice, not tested here, and the figure below prints it as such). Either way the bill is \(N \times\) one forward pass.
Current picks (mid-2026; licenses checked on each model card, 2026-09-25): open + Apache-2.0 — BGE-reranker-v2-m3, Qwen3-Reranker (0.6 / 4 / 8 B, 100+ languages, 32 k context) and mxbai-rerank-v2 (base and large; the large card gives 1.5 billion parameters, and the base one 0.5 billion).
Open weights but non-commercial: Jina-reranker-v3 (0.6B, listwise, meaning it scores the whole candidate list in one pass, CC-BY-NC-4.0), whose card reports \({\approx}\,0.62\) NDCG@10 (normalized discounted cumulative gain over the top 10) on the public BEIR (Benchmarking Information Retrieval) zero-shot suite, a collection of retrieval test sets a model has never trained on. Leaderboards print NDCG@10 \(\times 100\), as \(61.94\), though Evaluation Metrics §7 defines it between 0 and 1. That card says “Prefer jina-reranker-v3.5 for new projects” (read 2026-10-17), so v3 is no longer the maker’s own recommendation. A managed option is Cohere Rerank (the rerank-v4.0-pro and rerank-v4.0-fast models in Cohere’s documentation, read 2026-10-17; the page, read again on 2026-10-31, also lists newer rerank-v5.0-pro and -fast, with no deprecation note for v4.0).
Choose by: lift on your own evaluation set — how much reranking improves your metric (say Recall@10) over the retrieval order alone (Evaluation Metrics §4 defines Recall@\(K\)); rerankers transfer imperfectly across domains — then end-to-end latency at your \(N\), license, and language coverage.
The whole point of the funnel is to spend the expensive model on a tiny shortlist. The figure makes the economics literal: a cheap per-item operation cuts the catalog to a shortlist, then a costly per-pair operation reorders only those, and a final cheap sort picks the few that are shown.
| reranker (mid-2026) | license | size | languages | a score from its own card |
|---|---|---|---|---|
| Jina-reranker-v3 | CC-BY-NC-4.0 (non-commercial) | 0.6B, listwise | multilingual | BEIR 61.94 NDCG@10 |
| Qwen3-Reranker | Apache-2.0 | 0.6 / 4 / 8B | 100+ | MTEB retrieval (MTEB-R) 65.80 / 69.76 / 69.02 |
| BGE-reranker-v2-m3 | Apache-2.0 | 0.6B | multilingual | none as text on the card |
| mxbai-rerank-v2 | Apache-2.0 | base (0.5B); large (1.5B on its card) | 100+ | BEIR average 55.57 (base), 57.49 (large) |
Cohere Rerank (rerank-v4.0-pro, -fast; v5.0 now also listed) |
managed API | not published here | see Cohere’s documentation | none read |
Cost & scale anchors (back-of-envelope)
A few numbers worth carrying, so a design discussion stays grounded:
- Vector storage. Each coordinate costs what its precision costs (the quantization row of the §1 vocabulary table). A \(d\)-dim float32 vector (32 bits, 4 bytes per coordinate) is \(4d\) bytes, so 1 M \(\times\) 768-dim \(= 3{,}072\) bytes each \({\approx}\,3\) GB in all; int8 (one byte per coordinate, \(d\) bytes) is \({\approx}\,0.77\) GB; binary (one bit per coordinate, \(d/8\) bytes) is \({\approx}\,0.1\) GB — with a small recall hit you can recover by reranking, above. Choose precision by catalog size \(\times\) recall budget.
- Retrieve vs. rank (claims from practice, not tested here). A raw in-memory ANN index lookup is sub-millisecond; through a database with filters and a network hop, expect more than the \(0.20\) ms (unfiltered) or \(0.84\) ms (filtered) of our in-RAM laptop run at
ef=100in the latency measurement above, because a network hop and storage come on top (Training and Serving a Recommender §8 budgets \({\approx}\,5\) ms for the retrieval step). A cross-encoder ranking pass over \(N{=}100\) is tens of milliseconds on a GPU — budget it per request, not per item. - Embedding compute (claims from practice, not tested here). Open encoders run on your own GPU/CPU; managed APIs bill per token/image, on the order of cents per 1 k items. At scale, self-hosting an open encoder usually wins on cost; an API wins on time-to-ship. Read the vendor’s price page for today’s rate.
- The rule that dominates all of these. Before committing, validate the finalists on a small in-domain eval set — 50–200 labeled query→relevant-item pairs, scored with Recall@\(K\) (at about 100 pairs, treat a gap under about ten points as a tie; Rule 1 shows why): a cheaper model that wins on your data beats a pricier leaderboard champion (Rule 1, §1).
In one line: use a library until you need filters, live updates or hybrid search, then a database; read speed at a fixed recall, and rerank only a shortlist.
4. Open LLMs & small language models (SLMs)
Cara needs a two-line profile for each of her 40,000 films by Monday, and she does not want to pay a chat service for 40,000 answers or send her catalog to someone else’s servers. Which model can she run herself, and which licenses let her ship the result?
For recsys, the LLM is rarely a chatbot — it is an enhancer (generating item/user text features, profiles, synthetic tags), a reranker (listwise scoring of a candidate set), or a text encoder for new items (Large Language Models and Recommendation). That changes what “best” means: you want terse, faithful, schema-valid output — exactly in the required format, e.g. carrying the fields a JSON record must have — with no hallucinations and at high throughput; not arena charm.
How to choose. Pick on three axes in this order: license \(\rightarrow\) size/serving budget \(\rightarrow\) task. For the bread-and-butter — cheap, high-throughput batch feature generation — default to a small Apache-2.0/MIT model in the 0.5–9B tier run locally, with vLLM (an open-source GPU serving program, first met in §1’s default-stack table) for GPU throughput or llama.cpp for CPU and edge. llama.cpp is a lightweight program that runs LLMs on ordinary processors and laptops, and it reads GGUF (its own compressed model-file format).
At this size the model is cheap per item (a claim from practice, not tested here) and “write a 50-word profile from these attributes” needs no frontier model (one of the handful of most capable models available, usually closed and paid). Step up to a mid/large reasoning model — one trained to write out its intermediate steps (chain-of-thought, or “thinking tokens”) before answering, which is more accurate on hard ranking but bills you for every extra token — only where quality dominates cost (listwise reranking, hard cold-start). Treat license as a hard gate, and don’t pick from arena Elo (the chat-arena rating, explained under “Reading the benchmark” below) — validate finalists on your offline metric (NDCG and Recall lift, Evaluation Metrics §7 and §4; profile-faithfulness).
A mixture-of-experts (MoE) model holds many parallel sub-networks, the “experts”, and for each token a small router picks only a few of them. All of them must sit in memory, but each token pays only for the few it uses, so inference is cheaper than the total size suggests. “6B active” means the parameters one token uses; Mistral Large 3, for example, holds 675B parameters and uses 41B per token. A dense model, the opposite, puts every parameter to work on every token. “Effective” parameters count only the compute path of a small on-device model, not the extra lookup tables it stores. A perception encoder is the part that turns an image into numbers the language model can read.
Reading the table below in one line each: Apache-2.0 and MIT mean commercial use is allowed with no negotiation. “Conditional” means commercial use is allowed only while you stay under a stated limit (such as a revenue cap) or after you ask the maker. Remarks A to D after the table spell out the conditional cases.
| Family | Maker | Open sizes (tier) | License — commercial? | Best for (recsys) |
|---|---|---|---|---|
| Qwen3.5 / 3.6 / 3.8 | Alibaba | Qwen3.5: 0.8 / 2 / 4 / 9 / 27B dense and 35B, 122B, 397B MoE; Qwen3.6: 27B and 35B-A3B (35B in total, 3B active); Qwen3.8: 27B dense, Flash-Next (125B with 6B active; remark D), and a 2.4T-parameter model | Apache-2.0 — yes for the Qwen3.5, 3.6 and Qwen3.8-27B cards; conditional for Qwen3.8-Flash-Next and the 2.4T model, whose custom licenses grant sell and host rights but require a separate license for a Model-as-a-Service (a business that sells access to the model) or work-assistant business (remark A below the table; license files, read 2026-10-17) | Default all-rounder; small tiers for batch feature-gen, MoE for reranking; the version trap again: the license follows the checkpoint |
| Gemma 4 | two small on-device models (2.3 and 4.5B “effective” parameters), a 12B, a 26B mixture-of-experts with 4B active, and a 31B | Apache-2.0 — yes (NEW at v4; \(\le 3\) was custom!) | High-quality small profile gen; multimodal item features | |
| Mistral (Small 4, Large 3, Medium 3.5; earlier 7B and Small 3.x) | Mistral AI | Small 4 is 119B in total with 6.5B active; Large 3 is 675B in total with 41B active per token; Medium 3.5 is a dense 128B | Apache-2.0 — yes for Small 4 (its card) and Large 3 (Mistral’s 2025-12-02 announcement); conditional for Medium 3.5, a Modified MIT with a revenue cap (remark C below the table; license file, 2026-10-17) | Permissive workhorses; based in the European Union (EU), no usage caps on the Apache tier. Large 4 was announced on 2026-10-06 as a preview API with weights promised for the end of the month; none was on Mistral’s Hugging Face page on 2026-10-17, so it is not open yet |
| Ministral (2024-10: 3B, 8B) | Mistral AI | 3 / 8B | No — the 8B is under the Mistral Commercial and Research licenses, the 3B under the Commercial license only (Mistral’s announcement) | Edge / on-device tier, and the Rule-2 version trap: the same name one generation later — Ministral 3 (3B / 8B / 14B, announced 2025-12-02) — is Apache-2.0, so the license follows the release, never the brand |
| Phi-4 | Microsoft | 14B; Phi-4-mini is 3.8B | MIT — yes (both cards) | Small reasoning-grade reranking on a budget |
| DeepSeek (V3/V4/V4.1) | DeepSeek | large MoE: V4-Pro-0813 has about 1.65T parameters in its weight files, V4.1-Flash (2026-09-10) about 0.76T in its weight files (remark D below the table) | MIT from DeepSeek-V3-0324 through V4.1-Flash (cards read 2026-10-17); the original V3 weights are not MIT (its card: MIT code, weights under the DeepSeek Model License, commercial use allowed) | Large-tier quality reranking / hard reasoning; a version trap, like Ministral |
| Llama 3.2 / 4 | Meta | 1–3B edge; large MoE | Llama community license — conditional | No new Llama weights since 2025-04 on Meta’s meta-llama Hugging Face page; the 1–3B Llama 3.2 models are now matched in size by Apache-2.0 small tiers (Qwen3.5 0.8–9B and the two small Gemma 4 models; our reading, a claim from practice, not tested here), so choose Llama only for an existing dependency; read the attribution, EU and monthly-active-user (MAU) clauses |
| Muse (Meta) | Meta | Glimmer: 30B dense with a perception encoder; Spark Open: 816B in total with 42B active (Meta’s developer pages; the card is gated, and its weight files total about 822B) | Glimmer: Apache-2.0 — yes. Spark Open: conditional — a gated (download only after accepting terms) Muse Model License that grants sell and distribute rights, with an attribution duty, a usage policy and a revenue-gated agreement for a Model-as-a-Service business (remark B below the table; license file, read 2026-10-17) | Glimmer is the open-weight agent model that replaced Llama as Meta’s small open entry; Spark Open is a large model whose license repeats Llama’s attribution duty and adds a revenue gate |
| SmolLM3; OLMo 3 / 3.1 | Hugging Face; Ai2 | SmolLM3 is one 3B model; OLMo 3 is 7B and 32B, Instruct and Think variants (3.1 refreshes the 32B) | Apache-2.0 — yes (cards); open recipes | Reproducible, auditable baselines; provenance matters. OLMo 3’s card says it is pretrained on the Dolma 3 dataset, post-trained on the Dolci datasets, and releases code and checkpoints (logs “coming soon”); SmolLM3’s card says its pretraining data mixture is public and later-stage datasets were still to be uploaded |
Remark A, the two Qwen licenses. Qwen3.8-Flash-Next is under the Qwen Community License 1.0, and the 2.4T model under the Qwen3.8-Max License. Both require a separate license from Qwen before commercial use by a licensee that runs a “Model as a Service or AI Work Assistant business” (the latter an independent artificial-intelligence (AI) coding or office-productivity product). The Max file applies this only when aggregate revenue exceeds $50 M in any twelve months; Flash-Next applies it at any size. Internal use that exposes nothing to third parties is exempt. Both also require the model name on products above 100 M monthly users or $20 M monthly revenue.
Remark B, Muse Spark Open. The licensee must display “Built with Muse Spark Open”. A licensee whose aggregate gross revenue exceeds $20 M in twelve months and that offers the model as a service to third parties must notify Meta and sign a written agreement. Internal use and powering the licensee’s own product’s features are exempt.
Remark C, Mistral Medium 3.5. Its Modified MIT license withdraws all rights if the licensee company’s (or its employer’s) global consolidated revenue for the preceding month exceeds $20 M. Below that limit the model is commercial.
Remark D, sizes that include a lookup table. DeepSeek’s V4.1-Flash card describes a 552B backbone plus a 196B lookup table of learned vectors, which adds memory but little compute; the card’s safetensors count (safetensors is a weight-file format) is 763B, so the two figures are not simply additive. Qwen3.8-Flash-Next likewise carries a 51B embedding table of word-sequence vectors; we did not check whether its “125B” figure includes it.
Reading the benchmark. LMArena shows a person two anonymous chatbot answers side by side and asks which is better; the votes become Elo ratings — the chess rating system, where beating a strong opponent raises your score more than beating a weak one. For two players rated 1,000 and 1,200 the system expects the lower one to win \(1/(1+10^{200/400}) = 0.24\) of the time; if it wins, it gains \(32 \times (1 - 0.24) = 24.3\) points (with the common step size 32), against 16 for beating an equal.
So the ratings measure chat preference — verbosity and tone, the opposite of what a terse batch profiler needs; use its style-controlled view (the arena’s ranking adjusted for the style of the answers), and recall that arenas can be gamed: a chat-tuned “experimental” Llama 4 variant was listed second on the raw leaderboard in April 2025, and LMArena then said style and tone were an important factor, as its style-control ranking showed (The Register, 2025-04-08, §9).
MMLU (Massive Multitask Language Understanding, 57 subjects of multiple-choice exams) is saturated — the top models all score near its ceiling, so it no longer separates them (claim from practice, not tested here); its harder successor MMLU-Pro and GPQA (graduate-level, “Google-proof” Q&A — science questions) resist contamination, the leaking of test questions into training data, rather better (claim from practice, not tested here). A small case: a model that really answers 60 % of a 100-question test but saw 10 of its questions in training and memorized them scores \(0.6 \times 90 + 10 = 64\) points, four more than its skill.
The load-bearing caveat for 2026: “benchmaxxing” (our word for it, borrowed from practitioners) means training on or near benchmark questions, so a high score can be memorization, not reasoning (the case above: 64 reported against a true 60). Trust only agreement across three eval types (a static academic test, a style-controlled arena, an agentic suite, meaning a test where the model must use tools over many steps) — and then your own task eval.
Hugging Face announced on 2025-03-13 that its Open LLM Leaderboard was “officially retiring” (its Space, the web app hosting it, still ran on 2026-10-17); practitioners now triangulate on Artificial Analysis (an independent site that benchmarks models) and Epoch AI (a research group that tracks progress in artificial intelligence), whose Epoch Capabilities Index (ECI) — a single score combining many benchmarks — puts open weights ~4 months behind the closed frontier since January 2026, an ~8-point gap on that index (Epoch AI, 2026-05-29; its October 2025 analysis, over a different window, found about 3 months).
2026 state of play.
- The open-weights frontier is Chinese-led (as of 2026-10) — Qwen, DeepSeek, GLM (General Language Model, from Zhipu / Z.ai), Kimi (Moonshot AI) and MiniMax are all families of open-weight LLMs, now only partly MIT/Apache. Google (Gemma 4), Microsoft (Phi-4), Meta (Muse Glimmer), Ai2 (OLMo 3), Hugging Face (SmolLM3) and Mistral (its Apache-2.0 models; Medium 3.5 is revenue-gated, see the table) are the main Western entrants (our judgment, a claim from practice, not tested here).
- “Mostly” has become “read the file”. The newest flagships ship under custom licenses with a revenue gate: Kimi K3 (2026-06-13, about 2.8T parameters) requires a separate agreement with Moonshot AI from a model-as-a-service business above $20 M revenue over twelve months, and shows “Kimi K3” in a product above 100 million monthly active users or $20 M monthly revenue; GLM-5.3 (2026-08-25, about 753B) asks a model-as-a-service business above $10 billion revenue to pass a Z.ai security review; MiniMax-M3 (2026-06-02) allows non-commercial use freely, but commercial use must display “Built with MiniMax M3” and send MiniMax a one-time notice, or, if the products or services earn more than $20 M a year, obtain MiniMax’s prior written authorization (license files, read 2026-10-17).
- The same families still have a clean tier. GLM-5.3-Flash (320B in total, 18B active) is MIT, as are DeepSeek V4.1-Flash and GLM-5.2. Kimi K2.6 (2026-04-14) was still a Modified MIT License whose one modification obliges the same display of “Kimi K2.6” — the kind of clause flagged for Llama below.
- Llama is no longer the default open choice (as of 2026-10). Llama 4 Maverick: a technology news report of 2025-04-11 (linked in §9) put the shipped version below several months-old models from OpenAI, Anthropic and Google on LMArena and quoted a user’s post placing it 32nd, after Meta had submitted an unreleased “experimental” variant tuned for human preference — which, per The Register’s report of 2025-04-08, led LMArena to update its leaderboard policies. Llama 3.2-1B/3B still runs on the edge but has no advantage over the Apache-2.0 small tiers (see the table).
- Meta then ran both tracks (our reading). It succeeded Llama with Muse Spark (2026-04-08), which it did not release as downloadable weights: Meta’s announcement describes an API in private preview for select partners and says Meta hopes “to open-source future versions” (by 2026-07-09 the Meta Model API was in public preview, per Meta’s post of that date). In August 2026 it released Muse Glimmer, a 30B open-weight LLM under Apache-2.0 (Meta’s developer post of 2026-08-12 and the model card). Meta’s Muse Spark 1.1 post (2026-07-09) says nothing about open weights. Press reports of 2026-08 and 2026-09 (two outlets, linked in §9) had anticipated the weights, citing an X post by Mark Zuckerberg that we could not open; Meta’s developer pages now supersede them.
- Muse Spark Open is not Apache. A gated Muse Spark Open now exists on Hugging Face, with a Muse Model License dated 2026-10-15 (card metadata, read 2026-10-17; the repository was created 2026-09-15): a Llama-like attribution duty plus a revenue gate, so it is not Apache. Meta’s developer pages (dev.meta.ai/models/muse-spark-open and /docs/muse-spark-open, read 2026-10-17) describe it as derived from a Muse Spark model released in August (a later release than the April one; the version is not named there), 816B in total and 42B active, under the Muse Model License; we found no dated announcement on those pages, and Meta’s reasons for the release are not stated there. The Llama-to-Muse succession is the single fastest-to-flip verdict in this appendix — re-check before relying on it.
- The small-model surge is the recsys story. Each generation pushes capability down the size curve — the measured “densing law” puts the best open models’ capability per parameter on a doubling cycle of roughly 3.5 months (Xiao et al., Nature Machine Intelligence 7:1823–1833, 2025). Take that premise at face value and a year buys \(2^{12/3.5} \approx 10.8\times\), so a 4–9B model today lands roughly where a ten-times-larger one — 40–100B — sat a year ago. Treat the ratio as a trend, not a constant, and verify on your own task. You can profile millions of items locally for near-zero marginal cost (a claim from practice, not tested here) — exactly the enhancer use case.
- “Thinking” modes are now standard and toggleable. Turn reasoning on for listwise reranking / hard cold-start; off for high-throughput batch generation (thinking tokens are pure cost there).
- The emerging recsys recipe (a claim from practice, not tested here): use a large reasoning model to write high-quality outputs for a few thousand items — a gold set, the trusted reference answers everything else is measured against — then distill: fine-tune a small Apache/MIT model (say 4–9B) to reproduce those outputs, so it inherits most of the quality at a fraction of the serving cost, with a clean commercial license. One license check the recipe needs and the hype usually skips: the teacher’s terms must allow training other models on its outputs. Many API terms forbid exactly that.
- License traps (the most error-prone area): name the family and the version, never the brand — truly unencumbered (Apache-2.0 or MIT) = Qwen3 through Qwen3.6 and Qwen3.8-27B, Mistral 7B / Small 3.x / Small 4 / the Mistral 3 family, DeepSeek from V3-0324 on (not the first V3 release), GLM-4.5 through GLM-5.2 and GLM-5.3-Flash, OLMo 3, SmolLM3, Phi-4, Muse Glimmer, Gemma 4. Not unencumbered: Qwen3.8-Flash-Next and the 2.4T model, Mistral Medium 3.5, GLM-5.3, Kimi K2.6 and K3, MiniMax-M3, Muse Spark Open. Gemma \(\le 3\) used a custom, non-OSI license — check the version.
- Llama is a custom community license (an attribution duty — products must say “Built with Llama” — a clause requiring Meta’s separate permission for companies above 700 million monthly active users, and, in Meta’s acceptable-use policy rather than the license page, an EU restriction on its multimodal models) — fine for many, a dealbreaker for some; never call it “open source.” Open weights \(\ne\) open data — OLMo 3 (Dolma 3 and Dolci datasets, code, checkpoints; logs were “coming soon” on its card) and SmolLM3 (public pretraining mixture, some later-stage datasets still to come, per its card) are the open-recipe families here, if reproducibility or provenance is a requirement.
Serving the model — the runtime is a separate choice from the weights
Picking the weights is half the decision; how you run them is the other half, and it is the lever that actually moves throughput and cost. A runtime is the program that loads the weights and generates answers. The three tricks named in the table concern the KV-cache (a store of the key and value vectors of the tokens already processed; Representation Learning & the Transformer §5): vLLM’s PagedAttention (Kwon et al., 2023, §9a) keeps each request’s cache in small fixed pages, so many requests fit on one GPU without reserving a worst-case block each; continuous batching is about scheduling, not the cache itself: it slots a new request in the moment another finishes instead of waiting for the whole batch; and SGLang (Structured Generation Language; Zheng et al., 2024, §9a) reuses the cache of an identical prompt beginning across requests, which is its prefix caching (RadixAttention).
Match the runtime to the workload, not the model:
| Workload | Pick (as of 2026-10-17) | Why |
|---|---|---|
| Multi-user production LLM, GPU | vLLM | PagedAttention + continuous batching keep the GPU saturated; the de-facto default |
| Shared-prefix load (RAG, agents, batch with one big system prompt) | SGLang | Prefix-caching (RadixAttention) gives a real throughput edge when prompts overlap |
| Single-user / local / desktop | Ollama or llama.cpp (GGUF) | One-command local serving; runs on a plain processor or an Apple-silicon laptop GPU (through Apple’s Metal interface); the simplest path, not for many-user traffic |
| Embedding serving (§2) | TEI (Text-Embeddings-Inference) or vLLM | Purpose-built embedding server — token-batched, serves E5 / BGE / GTE / Qwen3 / Gemma encoders |
Two dated notes that change the old advice: Hugging Face’s TGI is in maintenance mode (its README notice is undated; the repository was archived on 2026-03-21) and the notice names vLLM, SGLang and llama.cpp as alternatives — don’t start new work on it; and for the batch enhancer job that dominates recsys (profile a million items overnight, latency-insensitive), offline batched vLLM is usually the cheapest path of all (a claim from practice, not tested here), since you can run the GPU at \(100\%\) with no request-latency budget to protect. Cara’s answer, as of this snapshot: an Apache-2.0 or MIT model of 0.5–9B parameters, run offline in one batch with vLLM.
In one line: pick the license first, then the smallest model that passes your own test, then the runtime that fits the workload, and write the date beside every verdict.
5. RecSys frameworks & libraries
Bob has to show that his new recommender beats an existing method, LightGCN (From Graphs to LightGCN), and a skeptical colleague will ask whether he ran that comparison method, his baseline, himself. Writing every model from scratch would take a month; this section is about the libraries that let him do it in a day. One gate first: several of these are research tools whose terms stop at the laboratory door, so read the License column before you ship anything built on one.
What a recsys framework gives you. It is a library that bundles the whole experiment — loading and splitting a dataset, dozens of ready-made models, the training loop, and the evaluation metrics — so that comparing two recommenders means editing a settings file rather than writing each piece yourself. That bundle is called a model zoo when the emphasis is on how many models ship with it: with a zoo of 94 models, comparing Bob’s model with four baselines means writing four model names in a settings file, not four implementations.
How to choose. For a reproducible paper baseline — LightGCN and NGCF (Neural Graph Collaborative Filtering; From Graphs to LightGCN), BPR (Bayesian Personalized Ranking; Loss Functions & Regularizers), SASRec (Self-Attentive Sequential Recommendation; Sequential & Session-Based Recommendation) on one pipeline — use RecBole, the broadest model zoo, which has all four out of the box. Its license is the reason it is named here for baselines and not for a product: the license file is MIT, but the project’s own README adds that “all data and code in this project can only be used for academic purposes” (README, checked 2026-09-25), which plain MIT does not say.
If your contribution is graph or self-supervised, prefer SSLRec (Slowing, by the table below), which implements SGL (self-supervised graph learning), SimGCL and NCL (neighborhood-enriched contrastive learning) — the graph contrastive methods of Self-Supervised & Contrastive Learning — natively alongside LightGCN; RecBole-GNN does too, but the table below marks it Stale, so use it only to reproduce a paper. If it is an evaluation/fairness claim, run it through Cornac or Elliot, which bake in hyperparameter search and significance tests — though of the two only Cornac is still actively maintained (the table below dates both).
For production at scale use TorchRec (Meta’s recommender library for PyTorch, built around sharded embedding tables; TensorFlow Recommenders is the other large-scale framework with recent commits in the table below); for a fast classical implicit-feedback model in a product, use the implicit library — a fast Python package for alternating least squares (ALS) and BPR on implicit feedback, whose name is unhelpfully the same as the kind of data it eats.
For LLM-for-RecSys there is no mature framework yet — only paper repos (the code published alongside individual research papers) and awesome-lists (curated collections of links, kept on GitHub).
Rule 4 asks for dates, so the table gives them, taken from each project’s default branch with GitHub’s command-line tool (gh api) on 2026-10-17 (GitLab, a code-hosting site like GitHub, for RecPack). Our own labelling rule, a claim from practice, not tested here (so are the Best for cells): Active if the last commit is under 6 months old, Slowing from 6 to 24 months, Stale beyond 24 months.
gh api and the GitLab API on 2026-10-17 and checked again on 2026-10-31, as the revision notes at the end of §8 record) — RecBole for the broadest model zoo, Cornac for the most current comparative experiments, SSLRec for self-supervised-learning contributions, TorchRec for production at scale. The status words follow the labelling rule above, applied to the last commit on the default branch. Two cells show why Rule 4 says to ignore GitHub’s own timestamps: its “pushed” time for Elliot is 2026-10-08 and for LightFM 2024-07-24, though their last default-branch commits are the dates shown. Read the first row as Rule 2 in miniature: an MIT file is not the whole license when the README adds a use restriction on top of it.
| Framework | Maker | Best for | Last commit · last release (2026-10-17); the status and the months count the age of the last commit | License |
|---|---|---|---|---|
| RecBole | RUCAIBox (the AI Box lab at Renmin University of China) | Broadest research baselines (94 models, per its README) | 2025-02-24 · v1.2.1, 2025-02-23: Slowing, 20 months | MIT file; README restricts use to academic purposes |
| Cornac | Preferred.AI | Comparative experiments + multimodal | 2026-09-14 · v3.0.1, 2026-09-14: Active | Apache-2.0 |
| Elliot | Politecnico di Bari | Reproducibility: hyper-parameter optimization (HPO — an automatic search over settings such as learning rate and embedding size) + significance tests | 2025-12-14 · v0.3.1, 2021-07-08: Slowing, 10 months | Apache-2.0 |
| SSLRec | HKUDS (the Data Intelligence Lab at the University of Hong Kong) | Self-supervised + graph CF | 2025-03-21 · no release: Slowing, 19 months | Apache-2.0 |
| RecBole-GNN | RUCAIBox | Graph CF on the RecBole pipeline | 2023-10-30 · v0.1.0, 2022-04-07: Stale, 36 months | MIT |
| ReChorus | Tsinghua | Sequential / session + click-through-rate (CTR) prediction | 2026-02-11 · no release: Slowing, 8 months | MIT |
| TorchRec | Meta | Billion-param embeddings, DLRM (Deep Learning Recommendation Model, Meta’s reference click-predictor), sharding | 2026-10-16 · v1.9.2-rc3 (a release candidate), 2026-09-30: Active | BSD-3 |
implicit |
B. Frederickson | Fast implicit-feedback CF (ALS/BPR) | 2026-05-08 · v0.7.3, 2026-05-08: Active, 5 months | MIT |
| torch-rechub | Datawhale (a Chinese open-learning community) | Lightweight PyTorch framework for recommendation models (repository description) | 2026-10-16 · v0.9.0, 2026-09-14: Active | MIT |
| TensorFlow Recommenders | Library for building recommender models in TensorFlow (repository description) | 2026-10-06 · v0.7.7, 2026-01-23: Active | Apache-2.0 | |
| Recommenders (the Microsoft Recommenders of Datasets and Benchmarks; the repository moved from microsoft to recommenders-team) | recommenders-team | “Best practices on recommendation systems” (repository description) | 2026-10-16 · 1.2.1, 2024-12-24: Active | MIT |
| Avoid for new work (LightFM, Spotlight, RecPack) or check first (Merlin, Transformers4Rec) | — | Do not start new work on these (read the dates) | LightFM 2023-04-30; Spotlight 2020-02-09; RecPack 2023-12-11: all Stale by the commit rule. Merlin and Transformers4Rec are not Stale by that rule (commits as recent as 2026-07-21 and 2026-08-23); they are listed for having no release since 2024-06-14 and 2024-01-11, so check their recent commits before relying on them | various (RecPack is AGPL) |
The reproducible-baseline angle (this is where Evaluation Metrics §11 bites). A framework fixes the code, not the protocol — two RecBole users still get non-comparable numbers if they choose different splits, filters, samplers, or cutoffs. A recurring finding in the reproducibility literature is that copied baseline numbers — quoting another paper’s figure instead of re-running it under your own protocol — are a major source of false “progress.” (Datasets & Benchmarks §5 tells the reproducibility critique behind this checklist, and the Anelli et al. and Petrov & Macdonald studies in item 3 below are the evidence on re-running published recommenders.) Five pitfalls, three of them documented in the literature:
Split protocol and the leakage it allows — a random split (hold out a random fraction of the interactions) leaks the future even within a user; leave-one-out (LOO) — per user, hold out the chronologically last interaction (a user with 5 films keeps the first 4 for training and tests on the 5th) — is leak-free within a user but still leaks across users, because one user’s test item can predate another user’s training data (our reading of the same point). Ji et al. (ACM Transactions on Information Systems, TOIS, 2023) trace the leakage to not observing the global timeline, where a single cut-off timestamp separates train from test for everybody, and report that it makes the models’ relative orders “unpredictable” in their leave-last-one-out experiments.
Sampled vs full ranking — ranking against ~100 sampled negatives is inconsistent with full-catalog ranking and can reverse model orderings (Krichene & Rendle, KDD 2020).
Under-tuned baselines — properly tuned simple methods beat most “neural” gains. Ferrari Dacrema et al. (2019) could reproduce only 7 of 18 recent neural recommenders and found 6 of those 7 often beaten by comparably simple heuristics, such as neighbor-based methods. Rendle et al. (2020) showed that a well-tuned dot-product matrix factorization (MF, Traditional Recommender Systems §5) beats NeuMF (its neural successor).
Two later re-running studies: Anelli et al. (2023) reproduce six graph recommenders on three standard datasets, compare them with traditional collaborative-filtering methods, and add two further datasets to see how dataset characteristics affect accuracy; and Petrov & Macdonald (2022) re-run BERT4Rec and find the proposed model under-trained — it only reproduces its published numbers when trained up to \(30\times\) longer than its default configuration, so the trap cuts the other way too. An untuned baseline flatters the new model; an untuned new model buries it.
Negative sampling (train and eval) silently changes what BPR/LightGCN learn.
Metric/cutoff/\(k\)-core inconsistencies shift rankings. (\(k\)-core filtering = repeatedly dropping users and items with fewer than \(k\) interactions until none are left — the common 10-core, say: a user with 9 interactions is dropped, and an item left with 9 after that drop goes too. It shrinks the data and, in practice, usually makes every method look better, a claim from practice, not tested here.) Items 4 and 5 are claims from practice, not tested here, and carry no citation.
One-line rule: a recsys accuracy number is only meaningful as a tuple — (model, dataset, \(k\)-core filter, split + seed, train sampler, eval candidate set, metric + cutoff, tuning budget). Re-run every baseline yourself under one fixed protocol.
2026 state of play. TorchRec (2026-10-16), torch-rechub (2026-10-16) and Recommenders (2026-10-16) all have recent commits, and TensorFlow Recommenders (2026-10-06) is the other large-scale framework in the table; the table dates every row. NVIDIA (the maker of the GPUs almost all of this runs on) has shipped no release of Merlin or Transformers4Rec since 2024, so don’t build new systems on them without checking what its recent commits are. LightFM, Spotlight and RecPack are stale by the table’s rule — a high star count (GitHub’s bookmark counter) does not mean maintenance; always check the last commit, not the “updated” date.
The LLM-for-recsys layer is paper repos (e.g. RLMRec, built on SSLRec; LLaRA, a research model that feeds item embeddings into an LLM for sequential recommendation) plus awesome-lists — citable references, not infrastructure; and generative retrieval with the Transformer Index for Generative Recommenders (TIGER) we know of only third-party reimplementations (we have not searched for an official one). License note: most are permissive, but RecPack is AGPL (license file, checked 2026-09-25), and some popular LLM-recsys awesome-lists carry no license (= all rights reserved — not safely reusable).
In one line: use a framework for the code, not for the protocol: re-run every baseline yourself under one fixed tuple, and check the last commit, not the star count.
6. NER & NLP libraries — turning text into features
Cara’s film blurbs are free text, but her ranker wants columns: who directed it, what sub-genre, what mood. Typing 40,000 rows by hand is out. A recommender often needs structured features from item/user text — entities, tags, attributes ({director, sub-genre, mood} from a film blurb; {brand, material, fit} from product copy) — for content-based, hybrid, or LLM recommenders, and this section is about the tools that pull them out.
What the job is, in one box. Named-entity recognition (NER) finds the names of things in a piece of text and labels each one with a type. Take the blurb
“Christopher Nolan’s time-bending thriller starring Cillian Murphy”
and a tool should return
[Christopher Nolan:
PERSON], [Cillian Murphy:PERSON], [thriller:GENRE].Each labeled stretch of words is a span; the list of types you want (
PERSON,GENRE,BRAND, whatever your catalog needs) is your schema. A supervised tool learns one fixed schema from thousands of labeled examples; a zero-shot tool is handed the type names at run time and needs no training data at all. Everything in §6 is a way of producing those spans, and every score below measures how well a tool reproduces spans a human wrote down first.
Scoring spans, worked. Tools are compared by span-level F1 — the same harmonic mean of precision and recall as the F1@K of Evaluation Metrics §4, counted over spans instead of ranked items. A span counts as correct only if both its words and its type match the human answer. Suppose the gold spans for the blurb above are {Christopher Nolan:
PERSON, Cillian Murphy:PERSON, thriller:GENRE} and a tool predicts {Christopher Nolan:PERSON, Cillian Murphy:ORG(organization — the wrong type), thriller:GENRE}. Two of its three predictions are right, so precision \(= 2/3\); it found two of the three gold spans, so recall \(= 2/3\); and \[\text{F1} \;=\; \frac{2 \cdot (2/3)(2/3)}{(2/3)+(2/3)} \;=\; \frac{2}{3} \;=\; 0.667 .\]Every F1 in this section is printed the way leaderboards print it, multiplied by 100 — so that \(0.667\) appears as \(66.7\); a score of \({\approx}\,90\) on OntoNotes (an 18-type, multi-genre entity benchmark, used throughout §6) means \(0.90\); and “10 F1 points” means \(0.10\). Evaluation Metrics keeps F1 between 0 and 1; this section follows the leaderboards instead, and says so once here.
How to choose. Default to a fast supervised pipeline when your schema is fixed and high-throughput, and reach for zero-shot/LLM extraction when your schema is arbitrary or training-data-free. Tagging a large catalog against a stable entity set, with a one-time fine-tune affordable? spaCy with its lg pipeline (the fast one in spaCy’s own table) wins clearly on cost-per-item; Stanza (a multilingual library) is slower, at 878 words/s about 11 times slower than spaCy lg. The table’s speeds are one benchmark’s words per second on Reddit comments, so measure throughput on your own catalog before budgeting, and mind which spaCy pipeline a quoted score belongs to (below). Need arbitrary attributes you can’t pre-train? GLiNER or NuNER-Zero (fast span extraction at inference, no training) or a template-driven extractor (NuExtract) / general LLM under structured-output decoding for nested, typed, normalized fields.
Every F1 below names its benchmark — a number without one is meaningless (the same model can move \(>10\) F1 across suites). All are supervised in-domain unless marked zero-shot; CoNLL (\(4\) types) and OntoNotes (\(18\) types) are different tests and do not compare directly.
| Tool | Type | F1 (named suite) · speed | Custom schema? | License |
|---|---|---|---|---|
spaCy (lg) |
Supervised pipeline | \({\approx}\,85.5\) OntoNotes (reported on the development set); fastest — \({\approx}\,10{,}000\) words/s on CPU (whole pipeline) | yes (easy training) | MIT |
spaCy (trf) |
Transformer pipeline (trf = its transformer-based variant) |
\({\approx}\,89.8\) OntoNotes (reported on the development set); \({\approx}\,684\) words/s on CPU (whole pipeline), GPU preferred | yes (easy training) | MIT |
| Stanza | Supervised neural | Multilingual (claim from practice, not tested here); \({\approx}\,88.8\) OntoNotes and \(92.1\) CoNLL in spaCy’s table; \({\approx}\,878\) words/s on CPU (whole pipeline, spaCy’s own run), between the two spaCy pipelines | yes (needs GPU) | Apache-2.0 |
| Flair | Supervised (contextual) | \({\approx}\,89.7\) OntoNotes and \(93.1\) CoNLL (accuracy table); \({\approx}\,323\) words/s on CPU, the slowest there (speed table); two separate facts, explained below the table | yes | MIT |
| HF Transformers | Fine-tuned BERT (Bidirectional Encoder Representations from Transformers, the text encoder of Representation Learning & the Transformer §5) or DeBERTa (a BERT variant with a better attention scheme) | \({\approx}\,93\)–\(94\) CoNLL ceiling (claim from practice, not tested here), GPU | yes (needs labels) | Apache-2.0 (per-model varies) |
| GLiNER | Zero-shot spans | \({\approx}\,48\) on the paper’s 20-dataset NER benchmark (Table 2) / \({\approx}\,61\) on its 7-dataset out-of-domain (OOD) benchmark (Table 1), fast | yes — any types, no training | Apache-2.0 for the library and the v2.1 checkpoints; the v1 weights behind the paper’s scores are CC-BY-NC — use v2.1 |
| NuExtract | Template LLM → JSON | extractive, multimodal (template-bound) | yes — JSON template = schema | MIT (v1.x, v2.0); Apache-2.0 (NuExtract 3, 4B, 2026-04) |
| LLM + structured output | LLM + constrained decoding | \({\approx}\,37\) zero-shot on the 20-dataset benchmark (the chat-LLM baseline of a 2023 paper, 36.5, Table 2; not a measurement of constrained decoding), slowest/costliest | yes — maximal flexibility | per-provider / open source |
The Flair row. The row joins two facts from two tables on spaCy’s page. Its accuracy table gives Flair \({\approx}\,89.7\) on OntoNotes and \(93.1\) on CoNLL, citing Akbik et al. (2018, §9a), with no pipeline named; its speed table separately times Flair’s pos-fast and ner-fast pipeline at \({\approx}\,323\) words/s on CPU, the slowest there. The page does not say the two rows are the same model. The \({\approx}\,94\) CoNLL (94.09) on Flair’s own README belongs to a larger model (ner-english-large).
Reading the benchmark. CoNLL-2003 — an English news data set from a 2003 shared task, with 4 entity types — is saturated (\({\approx}\,93\)–\(94\) F1 for the supervised top, so it no longer separates good tools) and a contamination risk (old and widely mirrored on the web, so LLM zero-shot scores on it are probably optimistic: a claim from practice, not tested here).
OntoNotes 5.0 (18 types, multi-genre) is harder and more honest — prefer it, and expect a lower number when you do: the table above gives four OntoNotes figures, all from spaCy’s own page: \({\approx}\,85.5\) for spaCy’s lg pipeline (the page lists no sm or md score), \({\approx}\,89.8\) for its trf, and, for the other two libraries, \({\approx}\,88.8\) for Stanza and \({\approx}\,89.7\) for Flair (the page’s accuracy table names no pipeline for that row). Within spaCy, which figure you can budget for depends on the pipeline you run, not on the library’s name. (The \({\approx}\,93\)–\(94\) above is CoNLL-2003, a different, easier suite.)
The supervised-vs-zero-shot gap is large but suite-dependent: on the original GLiNER paper’s 20-dataset NER benchmark (its Table 2), GLiNER-L (0.3B parameters) averages \({\approx}\,48\) F1 (47.8) and the paper’s chat-LLM baseline \({\approx}\,37\) (36.5), so GLiNER beats the LLM zero-shot, while on the paper’s 7-dataset out-of-domain benchmark — its Table 1: five CrossNER (cross-domain NER) domains, whose entity types come from specialist fields, plus two sets from the Massachusetts Institute of Technology — both score higher (GLiNER \({\approx}\,61\), the chat baseline 47.5).
That gap is irrelevant when no labeled set exists for your schema — the common recsys case. And any quoted “zero-shot average” is meaningless without naming the suite (the same model differs by \({\approx}\,13\) F1 across the two suites: \(60.9 - 47.8 = 13.1\)). Build a small in-domain gold set for your schema; public F1 is a loose prior, never a substitute.
2026 state of play.
- Classic supervised NER still wins on throughput/cost for fixed-schema catalog tagging (spaCy, last commit 2026-09-30; Stanza, 2026-10-01; Flair is Slowing by §5’s rule, last commit 2025-06-12; dates from
gh api, 2026-10-17). - Zero-shot NER (GLiNER, NuNER-Zero) is a good fit for the “new entity type, no training data” problem — arbitrary types at inference, small and fast, and in the GLiNER paper’s 2023 comparison it beats a general chat LLM asked the same question cold (a newer chat LLM may do better; not tested here); useful, not at supervised parity. The GLiNER library is active (v0.2.29, 2026-09-08; last commit 2026-10-05), and its authors’ successor GLiNER2 (EMNLP 2025 demo track; Apache-2.0 library and checkpoints; v2.1.0, 2026-10-09) is one schema-driven encoder family that also does text classification, structured extraction and relation extraction; its scores were not re-read here (dates from
gh apiand the model cards, 2026-10-17). - Structured-output extraction is the right mechanism — not free-form “return JSON” but schema-constrained decoding: at each step of the decode loop (Representation Learning & the Transformer §5) the runtime forbids any next token that would break the required format, so the output is always well-formed (after an opening brace, say, only a quote or a closing brace is allowed). Libraries: Outlines, Instructor, XGrammar; hosted, OpenAI’s
json_schemaoption. Caveat: this guarantees the shape, not the truth — the JSON is valid, the values may be invented. Check each value against a fixed list of allowed values (an enum, short for enumeration) before it becomes a feature. - Practical hybrid: use GLiNER or an LLM to label thousands of examples automatically — silver labels, cheap but imperfect, as against a hand-checked gold set — then distil a small, fast supervised pipeline by training it on them for the high-volume production path: flexibility upfront, throughput in production.
- License: the recommended stack is permissive (spaCy and Flair MIT; Stanza, HF Transformers and the GLiNER library Apache-2.0 — each read off the project’s own license file, 2026-09-25). Two traps: the GLiNER v1 checkpoints that produced the paper’s numbers are CC-BY-NC, so quote v1 but ship v2.1; and some per-model weights under the HF pipeline carry their own terms. (NuExtract is not such a case: on 2026-10-17 every NuExtract checkpoint that declares a license on its maker’s Hugging Face hub is MIT or Apache-2.0; six compressed (quantized) copies of the Apache-2.0 NuExtract 3 for Apple chips declare none.) Rule 2 again — the library’s license is not the weights’ license.
In one line: fixed schema and high volume, use a supervised pipeline; arbitrary schema and no labels, use a zero-shot tool and check every value; and name the benchmark suite beside every F1.
7. Speech — speech-to-text and text-to-speech (adjacent)
Ann’s shop also sells 500 podcast episodes, and none has a transcript, so there is no text to embed and nothing for a search box to find. Or a customer wants to say “play the new jazz album” out loud. Both need speech models.
Off the core recsys path: this section is brief. Two directions, two acronyms: speech-to-text (STT), also called automatic speech recognition (ASR) — the same thing under two names — turns audio into words; text-to-speech (TTS) turns words into audio. You need them in only two cases: (a) a voice interface in front of recommendations, or (b) your items are audio (podcasts, audiobooks, voice notes) and you need STT to turn them into transcripts that then feed the normal text-feature pipeline (§2, §6). In case (b), STT is usually all you need; TTS only if you also speak results back.
| Direction | Default pick | Strong alternatives | License notes |
|---|---|---|---|
| STT / ASR | Whisper large-v3(-turbo), a speech-recognition model, via faster-whisper (99 langs, robust, fast runtime: claim from practice, not tested here) | NVIDIA Canary-Qwen-2.5B; Parakeet-TDT (a Token-and-Duration Transducer model; English only in v2); Qwen3-ASR-1.7B (Apache-2.0); Cohere Transcribe (Apache-2.0, 2B); Moonshine (an on-device toolkit; its README says it is MIT-licensed by default). Their scores follow the table. | Whisper Apache/MIT; Parakeet/Canary CC-BY-4.0 — all commercial-OK |
| TTS | Kokoro-82M (Apache, tiny, good: claim from practice, not tested here) or Piper (MIT — but see below) | Orpheus (expressive; card says Apache, weights fine-tuned from Llama); ElevenLabs (a hosted text-to-speech API, per its documentation) | Caution: F5-TTS, Fish-Speech, Coqui-XTTS are non-commercial |
The one trap to flag: several popular “open” TTS models are non-commercial — F5-TTS (CC-BY-NC-4.0), Fish-Speech (CC-BY-NC-SA-4.0), Coqui-XTTS (the Coqui Public Model License). Note that the code repository is under the Mozilla Public License (MPL) version 2.0, so the repo badge tells you nothing about the weights. The original repository has had no push since 2024-08-16, because Coqui wound down; a community fork, idiap/coqui-ai-TTS, is active (last push 2026-10-16, code under MPL-2.0 too; both dates are push dates as reported by the host, so Rule 4’s caution applies and they show activity only roughly; its weights license was not read, so check the weights, not the repo). Read the weights license, not the repo’s code license.
Two more cases, read on 2026-09-25, both a version trap rather than a maker trap: Orpheus’s model card says Apache-2.0, but it is fine-tuned from Llama-3.2-3B, so treat it as carrying Llama’s conditions (§4); and Piper’s original repository is MIT but was archived on 2025-10-06, while its maintained successor (piper1-gpl; last commit 2026-10-16, v1.8.0 of 2026-09-04) is GPL-3.0 — copyleft, not permissive.
Commercial-safe open TTS is therefore a short and version-specific list: Kokoro, Parler-TTS (Apache-2.0 on its mini-v1 card), Piper at its MIT release, and Orpheus if Llama’s terms suit you.
WER counts mistakes against the correct transcript: \[\text{WER} = \frac{\text{substituted} + \text{deleted} + \text{inserted words}}{\text{words in the reference}}.\] “play the new jazz album” heard as “play a new jazz” is one substitution plus one deletion out of five words, \(2/5 = 40\%\).
STT is commoditized on clean English speech but not solved: on the Hugging Face Open ASR Leaderboard (edition of 2026-10-15, read 2026-10-17 from its results file; nine public English sets) Canary-Qwen-2.5B averages 5.38% WER (our unweighted mean; the leaderboard’s own average also folds in private sets, so it can differ) — 1.23% on LibriSpeech test-clean (read-aloud audiobooks), but still 7.91% on AMI (a corpus of recorded multi-speaker meeting audio, in the leaderboard’s cleaned version). The earlier edition (results as of 2026-03-26, eight sets, also what Canary-Qwen’s model card quotes) gave it 5.63%; the leaderboard changed its sets and its text normalization, so scores from the two editions do not compare.
Canary-Qwen is not the lowest open model on either edition: on the new one, six open-license entries scored on all nine sets have a lower mean (the best, Qwen3-ASR-1.7B, at 4.94%), and on the March one an open Cohere model scored 5.42%. Whisper large-v3 averages 6.66% and its turbo variant 7.17% on the same basis. Parakeet-TDT in its v2 version averages 5.45% but runs about \(7\times\) as fast as Canary-Qwen (6,148 against Canary-Qwen’s 882 seconds of audio transcribed per second of compute in the results file read on 2026-10-17; the leaderboard’s own table for the same edition, read on 2026-10-31, shows 5,988 against 862, the same ratio of about \(7\times\)), and Cohere Transcribe averages 5.21%. So the default pick above is a choice for language coverage and runtime, not for the lowest English WER.
So \(5.38\%\) is about one word in 19 wrong, and \(7.91\%\) on meeting audio is one in 13 — \(6.4\times\) worse than the \(1.23\%\) on clean audiobooks, on the same model. Pick for your acoustic conditions; the real engineering choice there is usually the runtime (faster-whisper / WhisperX, two faster re-implementations of Whisper), not the architecture (a claim from practice, not tested here).
In one line: for audio items you need speech-to-text, read the weights license of any text-to-speech model before shipping it, and judge a speech model on audio like yours, not on audiobooks.
8. Glossary
| Term | Plain meaning |
|---|---|
| Open weights / open source / open data | Released weights only / released under an OSI-approved license (§1 vocabulary table) / training data published too. They are different and often confused. |
| License families / OSI | Permissive (Apache-2.0, MIT, BSD) / copyleft (GPL, AGPL) / non-commercial (CC-BY-NC) / custom (Llama, Gemma \(\le 3\)). Only the first two are open source as the Open Source Initiative (OSI) defines it. §1 rule 2. |
| Model card | The documentation page a publisher posts beside a released model: size, license, training data, scores. The primary source for every license claim in this appendix. |
| API (application programming interface) model | One you cannot download: your program sends each request to the maker’s servers over the internet and pays per use. |
| Parameters, “7B” | The learned weights of a network, counted with M = million and B = billion; a 7B model has 7,000,000,000 of them. Sets memory and roughly sets speed. |
| Context (context window) | The longest input a model reads at once, in tokens: “8K context” \(\approx\) 8,000 tokens (word pieces). Longer text is cut off. |
| CPU / GPU / RAM / VRAM | The general-purpose processor / the chip that does thousands of multiplications at once / the computer’s fast working memory / the GPU’s own memory, which must hold the whole model. |
| NER (named-entity recognition) | Finding the names of things in text and labeling each span with a type (PERSON, BRAND, GENRE …). The list of types you want is your schema. §6. |
| JSON (JavaScript Object Notation) | A plain-text format for records, e.g. {"brand": "Acme", "fit": "slim"} — how an extractor returns structured fields. |
| BM25 / hybrid retrieval | “Best Matching 25”, the standard keyword-scoring refinement of TF-IDF / blending its (sparse) scores with embedding (dense) scores in one query. |
| Reciprocal-rank fusion | How two ranked lists are merged: each item scores \(\sum 1/(60+\text{rank})\) over the lists it appears in, so steady agreement beats one top spot. §3. |
| Pareto frontier | The set of best-possible (recall, speed) points an index can reach by turning its accuracy/speed knob; off the frontier you can do better on both, on it you trade one for the other. |
| Elo | The chess rating system, reused for chatbot arenas: beating a strong opponent raises your rating more than beating a weak one. |
| Fine-tune / distil / gold & silver labels | Train a released model further on your data / train a small model to copy a big one’s outputs / hand-checked reference answers vs cheap machine-written ones. |
| STT / ASR / TTS / WER | Speech-to-text (= automatic speech recognition) / text-to-speech / word error rate — (substituted + deleted + inserted words) / words in the correct transcript. §7. |
| Matryoshka (MRL) | Truncatable embeddings — a shorter prefix of the vector still works, so you index at a smaller dim to save memory/latency. |
| Quantization | Storing weights/vectors in fewer bits (4/8-bit, int8/binary) to shrink memory and speed inference, at a small quality cost. |
| ANN | Approximate nearest-neighbor search — fast, slightly-inexact vector retrieval; the candidate-generation engine. |
| HNSW / IVF-PQ / DiskANN / CAGRA | ANN index types: graph (in-RAM default) / inverted-list + compression (memory-thrifty) / on-disk (billion-scale) / GPU graph (batched throughput). |
| Recall vs QPS | The ANN trade-off: fraction of true neighbors found vs queries-per-second. Always compare at a fixed recall. |
| SLM | Small language model (\(\approx 0.5\)–9B) — cheap, high-throughput; the recsys feature-generation workhorse. |
| MoE (mixture-of-experts) | A model that holds many parameters but activates only a few per token — compute per token is below what the total size suggests, but every expert must still sit in memory. |
| Enhancer / reranker (LLM role) | Recsys LLM roles: generate item/user text features / re-order a candidate list — not chat. |
| MTEB / ann-benchmarks / LMArena / CoNLL | The standard benchmarks and leaderboards for embeddings / vector indexes / chat-LLMs / NER. Priors, not verdicts. |
| Benchmaxxing | Training on (or near) benchmark data so a high score can reflect memorization, not ability — why one leaderboard number is untrustworthy. |
| Structured output | Forcing an LLM to emit schema-valid JSON via constrained decoding — valid shape, not guaranteed-correct values. |
| Zero-shot NER | Extracting arbitrary entity types with no task-specific training (GLiNER, NuNER-Zero). |
| Cross-encoder | A model that reads the (query, item) pair together in one pass, so its scores cannot be precomputed — accurate, and affordable only over a short list (§3). |
Revision notes: how fresh each part is
Read this only to learn how old a particular cell is. Each table caption in sections 2 to 7 gives the date its cells were last read. The spaCy speed and accuracy figures of §6 were read on 2026-09-25.
What was re-read. On 2026-09-17 the license column of the §4 table, from each maker’s own announcement. On 2026-09-25 the license cell of every table in §2, §3, §5, §6 and §7, from the project’s license file, model card or announcement.
On 2026-10-17 the license cells of the tables in §2, §3, §4 and §7, and the facts, dates and labels listed next: last-commit and release dates in §3 and §5 (with
gh api), the live MTEB (Massive Text Embedding Benchmark) board in §2, the Open ASR Leaderboard results file of 2026-10-15 in §7, and the references in §9. On 2026-10-31 the dated claims of this appendix were checked again against their sources: the MTEB board values, the Open ASR Leaderboard values, the model-card sizes and licenses, Cohere’s embedding and rerank pages, and the last-commit and release dates of the frameworks (withgh api), which corrected three dates in §5 and §7.
How to read the labels. The latency figures in §3 are our own measurement of 2026-10-17, with every setting printed there. Claims that no source supports carry the label “claim from practice, not tested here”.
What was not re-read on 2026-10-17. The live ann-benchmarks board and its entry count, the TIGER claims, the CoNLL ceiling for the Hugging Face pipelines and the multilingual claim for Stanza (labeled as from practice), the Open TTS Leaderboard contents, the GLiNER2 scores, Pinecone’s service-level wording, the 2025-12-02 Mistral announcement date, ElevenLabs’ documentation, the Weaviate hybrid-search page, the Nemotron-3.5 release date, the Artificial Analysis and live LMArena boards, and the Stanza, Flair and RecPack license files (last read 2026-09-25). Rows keep their original dates unless the caption says otherwise.
9. References & resources
A curated directory — tools, leaderboards, and primary sources, current as of mid-2026; re-check before relying on a name.
This is the handbook’s most perishable appendix: leaderboards change often and the top open LLM is typically a few months old. Re-check live leaderboards before relying on a specific ranking.
(a) Papers
- Akbik, A., Blythe, D., & Vollgraf, R. (2018). Contextual string embeddings for sequence labeling. In Proceedings of the 27th International Conference on Computational Linguistics (COLING 2018), 1638–1649. https://aclanthology.org/C18-1139/
- Anelli, V. W., Malitesta, D., Pomo, C., Bellogín, A., Di Sciascio, E., & Di Noia, T. (2023). Challenging the myth of graph collaborative filtering: A reasoned and reproducibility-driven analysis. In Proceedings of the 17th ACM Conference on Recommender Systems (RecSys ’23), 350–361. https://doi.org/10.1145/3604915.3609489 arXiv:2308.00404
- Cormack, G. V., Clarke, C. L. A., & Buettcher, S. (2009). Reciprocal rank fusion outperforms Condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’09), 758–759. https://doi.org/10.1145/1571941.1572114
- Enevoldsen, K., et al. (2025). MMTEB: Massive multilingual text embedding benchmark. In Proceedings of the 13th International Conference on Learning Representations (ICLR 2025). arXiv:2502.13595
- Ferrari Dacrema, M., Cremonesi, P., & Jannach, D. (2019). Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In Proceedings of the 13th ACM Conference on Recommender Systems (RecSys ’19), 101–109. https://doi.org/10.1145/3298689.3347058 arXiv:1907.06902
- Jayaram Subramanya, S., et al. (2019). DiskANN: Fast accurate billion-point nearest neighbor search on a single node. In Advances in Neural Information Processing Systems 32 (NeurIPS 2019). https://proceedings.neurips.cc/paper_files/paper/2019/hash/09853c7fb1d3f8ee67a61b6bf4a7f8e6-Abstract.html (the proceedings page lists five authors; only the first is given here)
- Jégou, H., Douze, M., & Schmid, C. (2011). Product quantization for nearest neighbor search. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33(1), 117–128. https://doi.org/10.1109/TPAMI.2010.57
- Ji, Y., Sun, A., Zhang, J., & Li, C. (2023). A critical study on data leakage in recommender system offline evaluation. ACM Transactions on Information Systems, 41(3), 75:1–75:27. https://doi.org/10.1145/3569930 arXiv:2010.11060
- Krichene, W., & Rendle, S. (2020). On sampled metrics for item recommendation. In Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD ’20), 1748–1757. https://doi.org/10.1145/3394486.3403226
- Kusupati, A., Bhatt, G., Rege, A., Wallingford, M., Sinha, A., Ramanujan, V., Howard-Snyder, W., Chen, K., Kakade, S., Jain, P., & Farhadi, A. (2022). Matryoshka representation learning. In Advances in Neural Information Processing Systems 35 (NeurIPS 2022), 30233–30249. https://doi.org/10.52202/068431-2192 arXiv:2205.13147
- Kwon, W., Li, Z., Zhuang, S., Sheng, Y., Zheng, L., Yu, C. H., Gonzalez, J., Zhang, H., & Stoica, I. (2023). Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP ’23), 611–626. https://doi.org/10.1145/3600006.3613165 arXiv:2309.06180
- Muennighoff, N., Tazi, N., Magne, L., & Reimers, N. (2023). MTEB: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics (EACL 2023), 2014–2037. https://doi.org/10.18653/v1/2023.eacl-main.148 arXiv:2210.07316
- Ootomo, H., Naruse, A., Nolet, C., Wang, R., Feher, T., & Wang, Y. (2024). CAGRA: Highly parallel graph construction and approximate nearest neighbor search for GPUs. In 2024 IEEE 40th International Conference on Data Engineering (ICDE), 4236–4247. https://doi.org/10.1109/ICDE60146.2024.00323 arXiv:2308.15136
- Patel, L., Kraft, P., Guestrin, C., & Zaharia, M. (2024). ACORN: Performant and predicate-agnostic search over vector embeddings and structured data. Proceedings of the ACM on Management of Data, 2(3), 1–27. https://doi.org/10.1145/3654923 arXiv:2403.04871
- Petrov, A., & Macdonald, C. (2022). A systematic review and replicability study of BERT4Rec for sequential recommendation. In Proceedings of the 16th ACM Conference on Recommender Systems (RecSys ’22), 436–447. https://doi.org/10.1145/3523227.3548487 arXiv:2207.07483
- Rendle, S., Krichene, W., Zhang, L., & Anderson, J. (2020). Neural collaborative filtering vs. matrix factorization revisited. In Proceedings of the 14th ACM Conference on Recommender Systems (RecSys ’20), 240–248. https://doi.org/10.1145/3383313.3412488 arXiv:2005.09683
- Tschannen, M., Gritsenko, A., Wang, X., Naeem, M. F., Alabdulmohsin, I., Parthasarathy, N., Evans, T., Beyer, L., Xia, Y., Mustafa, B., Hénaff, O., Harmsen, J., Steiner, A., & Zhai, X. (2025). SigLIP 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv:2502.14786 (preprint)
- Zaratiana, U., Tomeh, N., Holat, P., & Charnois, T. (2024). GLiNER: Generalist model for named entity recognition using bidirectional transformer. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers) (NAACL 2024), 5364–5376. https://doi.org/10.18653/v1/2024.naacl-long.300 arXiv:2311.08526
- Zaratiana, U., Pasternak, G., Boyd, O., Hurn-Maloney, G., & Lewis, A. (2025). GLiNER2: Schema-driven multi-task learning for structured information extraction. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations (EMNLP 2025), 130–140. https://doi.org/10.18653/v1/2025.emnlp-demos.10
- Zheng, L., Yin, L., Xie, Z., Sun, C., Huang, J., Yu, C., Cao, S., Kozyrakis, C., Stoica, I., Gonzalez, J., Barrett, C., & Sheng, Y. (2024). SGLang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 62557–62583. https://doi.org/10.52202/079017-2000 arXiv:2312.07104
(b) Tools, libraries & leaderboards
Embeddings (§2).
- MMTEB leaderboard — https://huggingface.co/spaces/mteb/leaderboard; MTEB repo — https://github.com/embeddings-benchmark/mteb
- “Maintaining MTEB” (contamination / leaderboard-reality gap), Chung et al., 2025 (arXiv preprint; a poster at the CODEML workshop at ICML 2025) — https://arxiv.org/abs/2506.21182
- Model cards: Qwen3-Embedding — https://huggingface.co/Qwen/Qwen3-Embedding-8B; multilingual-e5-large-instruct — https://huggingface.co/intfloat/multilingual-e5-large-instruct; BGE-M3 — https://huggingface.co/BAAI/bge-m3; EmbeddingGemma — https://huggingface.co/google/embeddinggemma-300m; EmbeddingGemma 2 — https://huggingface.co/google/embeddinggemma-2; harrier-oss-v1 — https://huggingface.co/microsoft/harrier-oss-v1-27b; Nemotron-3.5-Embed — https://huggingface.co/nvidia/Nemotron-3.5-Embed-8B-BF16; Nomic-embed-v2 — https://huggingface.co/nomic-ai/nomic-embed-text-v2-moe; Stella-1.5B-en-v5 — https://huggingface.co/NovaSearch/stella_en_1.5B_v5; GTE-Qwen2-7B-instruct — https://huggingface.co/Alibaba-NLP/gte-Qwen2-7B-instruct; Jina-embeddings-v4 (license note) — https://huggingface.co/jinaai/jina-embeddings-v4
- Hosted embedding APIs (documentation, read 2026-10-17): Google Gemini embeddings (MTEB by output size) — https://ai.google.dev/gemini-api/docs/embeddings; Cohere Embed (the v5 models) — https://docs.cohere.com/docs/cohere-embed; Voyage multimodal embeddings — https://docs.voyageai.com/docs/multimodal-embeddings; OpenAI embeddings — https://developers.openai.com/api/docs/guides/embeddings
- Multimodal model cards: SigLIP 2 (ViT-SO400M) — https://huggingface.co/google/siglip2-so400m-patch14-384; JinaCLIP-v2 (CC-BY-NC-4.0) — https://huggingface.co/jinaai/jina-clip-v2
- Reranker model cards (§3): Jina-reranker-v3 (CC-BY-NC-4.0; BEIR 61.94 NDCG@10; the card prefers v3.5) — https://huggingface.co/jinaai/jina-reranker-v3; Cohere Rerank (v4.0 models) — https://docs.cohere.com/docs/rerank; BGE-reranker-v2-m3 — https://huggingface.co/BAAI/bge-reranker-v2-m3; Qwen3-Reranker — https://huggingface.co/Qwen/Qwen3-Reranker-4B; mxbai-rerank-v2 — https://huggingface.co/mixedbread-ai/mxbai-rerank-large-v2
Vector search (§3).
- ann-benchmarks — https://ann-benchmarks.com/; VectorDBBench (Zilliz) — https://github.com/zilliztech/VectorDBBench; VIBE (modern-embedding ANN benchmark), Jääsaari et al., Journal of Data-centric Machine Learning Research (2026, as the arXiv record’s comments field states; arXiv v3 of 2026-08-23, read 2026-10-17) — https://arxiv.org/abs/2505.17810
- FAISS — https://github.com/facebookresearch/faiss (cosine = normalize + inner-product: https://github.com/facebookresearch/faiss/wiki/MetricType-and-distances); hnswlib — https://github.com/nmslib/hnswlib; Qdrant — https://qdrant.tech/documentation/; Milvus — https://milvus.io/docs; pgvector (filtering and iterative scans, README) — https://github.com/pgvector/pgvector; Qdrant filterable HNSW — https://qdrant.tech/articles/filterable-hnsw/; Weaviate filtering and ACORN — https://docs.weaviate.io/weaviate/concepts/filtering; Milvus overview — https://milvus.io/docs/overview.md; ScaNN — https://github.com/google-research/google-research/tree/master/scann; DiskANN — https://github.com/microsoft/DiskANN
- The rest of the §3 table, for the license column: Weaviate — https://github.com/weaviate/weaviate; LanceDB — https://github.com/lancedb/lancedb; Pinecone — https://www.pinecone.io/; VectorChord (AGPL / ELv2) — https://github.com/supervc-stack/VectorChord (formerly
tensorchord/VectorChord, which redirects there); Annoy and Voyager, Spotify’s newer HNSW-based library — https://github.com/spotify/annoy, https://github.com/spotify/voyager
LLMs / SLMs (§4).
- Leaderboards: LMArena — https://lmarena.ai/; Artificial Analysis — https://artificialanalysis.ai/leaderboards/models; Epoch AI (open-vs-closed lag, 2026-05-29) — https://epoch.ai/data-insights/open-closed-eci-gap
- Licenses (the trap), by maker:
- Google and Meta: Gemma 4 → Apache-2.0 — https://blog.google/innovation-and-ai/technology/developers-tools/gemma-4/; Llama community license — https://www.llama.com/llama4/license/, and its acceptable-use policy, which holds the EU multimodal clause — https://dev.meta.ai/llama/llama4/use-policy/
- Mistral: Ministral 3B/8B, 2024-10 (Mistral Research / Commercial license) — https://mistral.ai/news/ministraux; Ministral 3 and Mistral Large 3, 2025-12-02 (Apache-2.0; Large 3 = 41B active of 675B total) — https://mistral.ai/news/mistral-3/; Mistral Medium 3.5 (Modified MIT) and Mistral Small 4 (Apache-2.0) — https://huggingface.co/mistralai/Mistral-Medium-3.5-128B, https://huggingface.co/mistralai/Mistral-Small-4-119B-2603
- Moonshot (Kimi): Kimi K2 Modified MIT License (the 2026-09-25 reading) — https://huggingface.co/moonshotai/Kimi-K2-Instruct/blob/main/LICENSE; Kimi K3 and its LICENSE — https://huggingface.co/moonshotai/Kimi-K3, https://huggingface.co/moonshotai/Kimi-K3/raw/main/LICENSE; Kimi K2.6 — https://huggingface.co/moonshotai/Kimi-K2.6
- Z.ai, MiniMax and Qwen: GLM-5.3 and its LICENSE — https://huggingface.co/zai-org/GLM-5.3, https://huggingface.co/zai-org/GLM-5.3/raw/main/LICENSE; MiniMax-M3 LICENSE — https://huggingface.co/MiniMaxAI/MiniMax-M3/raw/main/LICENSE; Qwen3.8-Flash-Next and Qwen3.8-2.4T-A95B LICENSE files — https://huggingface.co/Qwen/Qwen3.8-Flash-Next/raw/main/LICENSE, https://huggingface.co/Qwen/Qwen3.8-2.4T-A95B/raw/main/LICENSE
- DeepSeek: DeepSeek-V4.1-Flash — https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
- Families: Qwen — https://qwenlm.github.io/; Mistral — https://mistral.ai/news/; DeepSeek — https://huggingface.co/deepseek-ai (original V3 card, weights under the DeepSeek Model License — https://huggingface.co/deepseek-ai/DeepSeek-V3; MIT from https://huggingface.co/deepseek-ai/DeepSeek-V3-0324); Phi-4 — https://huggingface.co/microsoft/phi-4, Phi-4-mini — https://huggingface.co/microsoft/Phi-4-mini-instruct; SmolLM3 — https://huggingface.co/HuggingFaceTB/SmolLM3-3B (blog: https://huggingface.co/blog/smollm3); Mistral Large 4 (announcement of 2026-10-06) — https://mistral.ai/news/mistral-large-4; Olmo 3.1 — https://huggingface.co/allenai/Olmo-3.1-32B-Instruct
- Model-size trend: Xiao et al., Densing law of LLMs, Nature Machine Intelligence 7, 1823–1833 (2025) — https://doi.org/10.1038/s42256-025-01137-0
- Serving runtimes: vLLM — https://github.com/vllm-project/vllm; SGLang — https://github.com/sgl-project/sglang; Ollama — https://github.com/ollama/ollama; llama.cpp — https://github.com/ggml-org/llama.cpp; TGI (maintenance mode, notice undated; archived 2026-03-21) — https://github.com/huggingface/text-generation-inference; TEI (embeddings) — https://github.com/huggingface/text-embeddings-inference
- 2026 Meta open/closed swing and Llama 4’s reception: Muse Spark announcement (Meta, 2026-04-08) — https://about.fb.com/news/2026/04/introducing-muse-spark-meta-superintelligence-labs/; Muse Glimmer, 30B, Apache-2.0 (2026-08-10; the post says nothing about Muse Spark 1.2 weights) — https://research.meta.ai/blog/introducing-muse-glimmer-open-agentic-model, and Meta’s developer post of 2026-08-12 — https://dev.meta.ai/resources/blog/build-with-muse-glimmer/; Muse Spark 1.1 and the Meta Model API (2026-07-09) — https://ai.meta.com/blog/introducing-muse-spark-meta-model-api/; Muse Spark Open (Meta’s developer pages) — https://dev.meta.ai/models/muse-spark-open/, https://dev.meta.ai/docs/muse-spark-open, and its gated card — https://huggingface.co/meta-models/Muse-Spark-Open-816B-A42B; press on the Spark weights promise: OfficeChai — https://officechai.com/ai/mark-zuckerberg-says-meta-will-release-weights-of-muse-spark-1-2-announces-30b-muse-glimmer-model-for-local-use/, The Register (2026-09-02) — https://www.theregister.com/a/5294093; LMArena style-control incident — https://www.theregister.com/2025/04/08/meta_llama4_cheating/, and the 32nd place — https://techcrunch.com/2025/04/11/metas-vanilla-maverick-ai-model-ranks-below-rivals-on-a-popular-chat-benchmark/; Open LLM Leaderboard retirement notice (2025-03-13) — https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard/discussions/1135
RecSys frameworks (§5).
- last-commit dates in §5 from
gh apion 2026-10-17; RecBole — https://github.com/RUCAIBox/RecBole; RecBole-GNN — https://github.com/RUCAIBox/RecBole-GNN; Cornac — https://github.com/PreferredAI/cornac; Elliot — https://github.com/sisinflab/elliot; SSLRec — https://github.com/HKUDS/SSLRec; ReChorus — https://github.com/THUwangcy/ReChorus; TorchRec — https://github.com/meta-pytorch/torchrec; implicit — https://github.com/benfred/implicit; torch-rechub — https://github.com/datawhalechina/torch-rechub; TensorFlow Recommenders — https://github.com/tensorflow/recommenders; Recommenders (recommenders-team) — https://github.com/recommenders-team/recommenders; RecPack (AGPL-3.0) — https://gitlab.com/recpack-maintainers/recpack; the “avoid” row: LightFM — https://github.com/lyst/lightfm, Spotlight — https://github.com/maciejkula/spotlight, Merlin — https://github.com/NVIDIA-Merlin/Merlin, Transformers4Rec — https://github.com/NVIDIA-Merlin/Transformers4Rec
NER / NLP (§6).
- spaCy — https://spacy.io/; its own accuracy-and-speed table, the source of the two spaCy rows in §6 (
en_core_web_trf\(89.8\) F on OntoNotes NER at \(684\) words/s on CPU;en_core_web_lg\(85.5\) at \(10{,}014\) words/s) — https://spacy.io/usage/facts-figures; Stanza — https://stanfordnlp.github.io/stanza/; Flair (CoNLL-2003 F1 \({\approx}\,94.1\)) — https://github.com/flairNLP/flair; GLiNER paper — https://arxiv.org/abs/2311.08526, https://aclanthology.org/2024.naacl-long.300.pdf; GLiNER library and checkpoints (Apache-2.0 library and v2.1; CC-BY-NC v1) — https://github.com/urchade/GLiNER, https://huggingface.co/urchade/gliner_large-v2.1; GLiNER2 (Apache-2.0; v2.1.0, 2026-10-09) — https://github.com/fastino-ai/GLiNER2, https://huggingface.co/fastino/gliner2-large-v1; NuExtract — https://huggingface.co/numind/NuExtract-2.0-8B (MIT), and its successor NuExtract 3 (4B, Apache-2.0) — https://huggingface.co/numind/NuExtract3; NuNER-Zero — https://huggingface.co/numind/NuNER_Zero; CrossNER (the specialist-domain benchmark) — https://arxiv.org/abs/2012.04373; OpenAI Structured Outputs — https://openai.com/index/introducing-structured-outputs-in-the-api/; Outlines — https://github.com/dottxt-ai/outlines; Instructor — https://github.com/567-labs/instructor; XGrammar — https://github.com/mlc-ai/xgrammar
Speech (§7).
- Open ASR Leaderboard (English WER, edition of 2026-10-15, read from its results file 2026-10-17; the repository’s own CSV is the older 2026-03-26 edition the model cards quote) — https://github.com/huggingface/open_asr_leaderboard, https://huggingface.co/spaces/hf-audio/open_asr_leaderboard; its paper, Srivastav et al., arXiv:2510.06961 (preprint; a search found no published version) — https://arxiv.org/abs/2510.06961; Open TTS Leaderboard (a Hugging Face Space, last modified 2026-10-09; its results were not read) — https://huggingface.co/spaces/hf-audio/open_tts_leaderboard; Qwen3-ASR — https://huggingface.co/Qwen/Qwen3-ASR-1.7B; Cohere Transcribe — https://huggingface.co/CohereLabs/cohere-transcribe-03-2026; Whisper large-v3 — https://huggingface.co/openai/whisper-large-v3; faster-whisper — https://github.com/SYSTRAN/faster-whisper; WhisperX — https://github.com/m-bain/whisperX; NVIDIA Parakeet — https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2; NVIDIA Canary-Qwen-2.5B — https://huggingface.co/nvidia/canary-qwen-2.5b; Moonshine — https://github.com/moonshine-ai/moonshine; ElevenLabs — https://elevenlabs.io/docs/overview; Kokoro-82M — https://huggingface.co/hexgrad/Kokoro-82M; Piper (MIT, archived 2025-10-06) — https://github.com/rhasspy/piper, and its maintained GPL-3.0 successor — https://github.com/OHF-Voice/piper1-gpl; Parler-TTS — https://huggingface.co/parler-tts/parler-tts-mini-v1; Orpheus (Apache card, Llama-3.2-3B base) — https://huggingface.co/canopylabs/orpheus-3b-0.1-ft; the non-commercial three — F5-TTS https://huggingface.co/SWivid/F5-TTS, Fish-Speech https://huggingface.co/fishaudio/fish-speech-1.5, Coqui-XTTS-v2 https://huggingface.co/coqui/XTTS-v2
Online sources verified 2026-06-21; the §4 open-weights entries re-verified 2026-08-10; the §4 license cells re-verified 2026-09-17; on 2026-09-25 the license cell of every table in §2, §3, §5, §6 and §7 was read off the project’s own license file, README, model card or announcement (see the revision notes at the end of §8), along with the spaCy speed/accuracy figures in §6 and the cross-encoder claims in §3; and on 2026-10-17, and again on 2026-10-31, the dated facts listed at the end of §8 were re-read from the primary sources listed above.