VLM-Rec: A Multimodal Recommendation Cascade and a Stage-Matched Audit of Generative Retrieval

1Introduction

Production recommenders are cascades. A cheap retriever narrows a large catalog to a few hundred candidates, heavier models re-order them, and business rules shape the final slate. Public work concentrates on the ranking model, yet every stage of a cascade reads the same item representation, and that representation decides what the system can do for items with little interaction history. VLM-Rec puts multimodal item understanding at the center of a complete cascade and measures, stage by stage, what each component contributes.

The second half of the report takes up a question this kind of system raises directly. Generative retrieval with semantic IDs, in the style of TIGER [1], replaces the nearest-neighbour index with a decoder that writes the next item's codes. Our own TIGER-style model reaches half of the two-tower's Recall@10 with no index at all (Section 4.5). A team deciding whether to migrate needs to know which stage gave up the other half, and an end-to-end comparison cannot say, because it changes the tokenizer, the encoder, the training objective and the decoder together. The stage-level measurement is available, and Section 5 takes it.

The report makes four contributions.

1. A complete, configuration-driven cascade: two-tower retrieval over FAISS with a co-visitation channel, a distilled pre-ranker, a DIN + DCN-v2 + MMoE ranker, MMR and DPP post-processing, and a FastAPI service that answers in 16 ms p99 on CPU and scales from 25.6K to 208K items by editing one file (Section 3, Section 4.7).
2. Controlled measurements of what multimodal content is worth at each stage, including VLM-generated item profiles and RQ-VAE semantic IDs, replicated on a second category (Section 4).
3. A diagnosis of cascade inconsistency that separates two mechanisms, label poisoning by held-out positives and a missing cross-stage score, and repairs both (Section 4.3).
4. A catalog-wide lazy-exact top-K instrument and a stage-matched loss ledger for the dense-to-generative retrieval gap on three catalogs, with hierarchical bootstrap confidence intervals on every entry (Section 5).

2Data and item representation

2.1Catalogs and splits

We use Amazon Reviews 2023 [2]. Video_Games is the development catalog: its listings pair distinctive box art with rich text, and its size permits fast iteration on one GPU. Beauty_and_Personal_Care is the scale catalog, eight times larger in items and 7.7 times sparser. The loss-ledger study adds the Amazon-2014 Beauty split used by TIGER, whose user, item and interaction counts our preprocessing reproduces exactly. Every split is temporal leave-last-out: each user's last interaction is the test target and the second-to-last is the validation target. Table 1 summarizes the catalogs.

Throughout the system results, the cold slice is the set of test users whose target item falls in the bottom quartile of training popularity. It isolates the long tail, where collaborative signal is weakest and content has to carry the prediction.

2.2Content features

Each item is described by three frozen embeddings, computed once and aligned by item index, so row i of every matrix refers to the same product. Listing text (title, description and feature bullets) is encoded with all-MiniLM-L6-v2 [3] into 384 dimensions. The main product image is encoded with CLIP ViT-B/32 [4] into 512 dimensions, with a zero vector and a flag for the few items without one. A structured profile written by a vision-language model is encoded into a further 384 dimensions. Concatenated, these form a 1,281-dimensional content vector that every stage of the cascade shares.

Keeping the encoders frozen is deliberate. It makes every downstream ablation cheap, and it attributes a gain to the information an encoder supplies rather than to a fine-tuning budget.

2.3VLM item profiles

Qwen2.5-VL-7B [5] reads each item's image and title and writes a JSON profile with a fixed schema: refined category, sub-genre, visual style, key attributes, target audience, tone, quality cues and a one-line summary. A deterministic template turns the profile into text, which passes through the same MiniLM encoder as the raw listing, so the profile block differs from the text block only in what the VLM said. Generation is batched, checkpointed in shards with atomic writes, and resumable after interruption. The full Video_Games catalog took 243 minutes on one RTX 4090 and produced valid JSON for 100% of items; the Beauty catalog took 16.8 hours with Qwen2.5-VL-3B at 99.76% validity.

Throughput and validity both depend on details of batched generation. Decoder-only models must be padded on the left: with the tokenizer's default right padding, every row but the longest in a batch continues after pad tokens, and validity fell to 3 of 24 rows. Batch size must also be re-tuned per model size. The 3B model at the 7B model's batch size left the GPU 61% utilized; a batch of 24 brought utilization to 100% and throughput up 2.5 times.

3System

3.1Retrieval

The retriever is a two-tower model. The item tower maps the content vector through a two-layer MLP (1,281 → 256 → 128) and normalizes the output. The user tower applies an MLP of the same shape to the mean content vector of the user's training history. Neither tower reads a user ID, so one model scores a logged user and a brand-new session alike. Training uses an in-batch sampled softmax with temperature τ and the log-frequency correction of Yi et al. [6], which removes the penalty that popular items otherwise receive for appearing more often as in-batch negatives:

where s is cosine similarity, B is the batch and q_j is the sampling frequency of item j. For ablation the tower has two further modes: hybrid adds learned item-ID, price-bucket and category embeddings to the content input, and ID-only replaces content with learned user and item vectors. FAISS [7] serves the item embeddings from a flat, HNSW [8] or IVF index. A second candidate source counts item-to-item co-visitation in training histories, damped by popularity. Retrieval passes 200 candidates downstream.

3.2Ranking

The ranker is where the modeling budget goes. A DIN [9] target-attention block weights the user's behaviour sequence by its relevance to the candidate, with the candidate masked out of its own history so a positive cannot leak through it. A DCN-v2 [10] cross network models explicit feature interactions. An MMoE [11] layer shares experts between two task heads, P(click) and P(satisfaction), where satisfaction means a rating of four or more. Structured features (price, category leaf, item ID) enter as quantile buckets with learned embeddings. Two inputs are switched by configuration: the retrieval score u·i of the candidate (Section 4.3) and semantic-ID code embeddings (Section 3.4). Each checkpoint ships with a sidecar metadata file that records which inputs it uses, and the serving path builds the model from that file.

3.3Pre-ranking and post-processing

Running the full ranker over 200 candidates spends most of its work on items that never reach the user. A pre-ranker with the ranker's architecture minus the DIN and cross blocks is distilled from the ranker's click logits and cuts 200 candidates to 50; its top 50 retains 79% of the full ranker's top 10. Post-processing multiplies the two task scores, P(click)·P(sat), blends in the retrieval score with a weight α tuned on validation, then trades relevance for diversity with maximal marginal relevance [12] or greedy MAP inference for a determinantal point process [13] over item content embeddings, and finally applies a per-category cap.

3.4Semantic IDs

An RQ-VAE [14] quantizes each content vector into three levels of 256 codes, so items with similar content share code prefixes and a new item receives a meaningful identifier from its content alone. Residual quantizers are prone to codebook collapse, so training initializes codebooks by k-means on first-batch residuals, updates them by exponential moving average, re-seeds dead codes, and fails any run whose per-level utilization drops below 60%. The codes serve three purposes: they replace the learned item ID in the two-tower, they enter the ranker as features, and they form the vocabulary of a TIGER-style generative retriever. That retriever is a small decoder-only transformer that reads the user's history as code tokens and generates the next item's codes by beam search constrained to a trie of real items, so every output is a valid item and no ANN index is involved.

3.5Serving and operations

At startup a registry loads every artifact into memory on CPU: both towers, the FAISS index, the pre-ranker, the ranker and the item feature tables. Each request then runs the four stages in sequence and records the latency of each. A FastAPI application exposes the cascade at GET /recommend for logged users and at POST /recommend/session for an ad-hoc basket of items; because no stage reads a user ID, a new session receives a personalized feed with no retraining. An explain mode returns the candidates and scores after every stage. The ranker exports to ONNX with its feature tables baked in and matches PyTorch to a maximum absolute difference of 2.4×10⁻⁶.

Around the service sit a Docker Compose deployment with a Streamlit demo, Prometheus metrics with per-stage latency histograms, MLflow run tracking, and continuous integration that runs ruff and a pure-logic test suite on every push. Every stage is one CLI command driven by a single YAML file, and command-line overrides are logged with each run.

4Results: the cascade

All results in this section are on the test split under temporal leave-last-out, with model variants chosen on the validation split. Unless stated otherwise the catalog is Video_Games and the item content is the fused 1,281-d vector.

4.1Retrieval

Content beats identity. The content tower reaches Recall@100 of 0.214 against 0.129 for the ID-only tower, a 66% gain that grows to 81% on the cold slice, and it does so without per-user or per-item embedding tables. The ID-only tower fits its training data more tightly and generalizes worse, the expected behaviour of free per-item vectors on sparse data. Adding ID, price and category embeddings to the content tower (hybrid) leaves overall recall where it was and lowers cold-slice recall from 0.044 to 0.041, so structured-feature work belongs in the ranker.

The sources complement each other. Co-visitation is the weakest source overall and the strongest on the cold slice (0.065), while popularity retrieves nothing there; together these numbers make the case for blending a learned retriever with a co-visitation channel. An HNSW index recovers 81% of the exact top 100 at its default search depth.

4.2Ranking

Content is the dominant ranking input as well. Removing it lowers GAUC from 0.858 to 0.761 and drops cold-slice GAUC to 0.347, below the 0.5 of a random ranker: without content, a rarely seen item gives the model nothing to separate it from noise. DIN and the DCN-v2 cross network each add 0.006 GAUC. The MMoE head leaves click GAUC unchanged (0.856 single-task against 0.858) and adds the satisfaction objective without cost to click quality.

4.3Cascade consistency

Re-ranking retrieved candidates exposes a consistency problem that offline ranking metrics do not show. Table 4 isolates its mechanisms by measuring the NDCG@10 of each ranker's order over the retriever's top 200. A ranker trained on random negatives scores 0.858 GAUC yet orders these candidates at 0.081, because random negatives are far easier than the candidates retrieval surfaces; this is sample-selection bias between stages. Mining negatives from the candidate set exposes a second mechanism, label poisoning. The candidate file masked only training items, and because retrieval works, a user's held-out positive is usually among the candidates, so it was drawn as a negative and labelled click = 0 (0.051). Rejection-resampling every negative that collides with the row positive or with the user's validation and test items repairs the pool (0.060).

The third mechanism is a missing cross-stage signal. Without the retrieval score the ranker can only replace the retriever's opinion, never refine it. Adding the raw two-tower score u·i as an input, a quantity defined for every user-item pair and identical to the FAISS score at serving time, lifts cascade NDCG@10 to 0.075, 48% above the poisoned pool. How the feature is built matters. A binary was-retrieved flag looks equivalent but would encode the label in reverse: training positives are masked out of the candidate file, so every positive would carry flag 0 and most hard negatives flag 1. The same residual bias appears when hard negatives are concentrated in the top 50, the slate most correlated with candidate membership, which drops cascade NDCG@10 to 0.032. Mixing easy and hard negatives, as reported for embedding-based retrieval at Facebook [15], is the stable regime.

Two properties of the evaluation govern how these numbers are used. The candidates were selected by the retriever's own similarity, which gives the retriever's order a home-field advantage on this metric, and validation-tuned fusion settles at α between 0.75 and 1.0. The served path therefore ranks by the fused score, which on validation is never worse than retrieval order because α = 1 is in the grid, and the comparison free of the advantage is an online one. The winning variant also depends on the feature set: the text+image run selects the score-feature ranker, the fused-feature run selects the clean hard-negative ranker (0.063, +23% over its poisoned baseline), and Beauty selects the listwise score-feature ranker. Selection is therefore automated on validation, and the chosen checkpoint carries its architecture in sidecar metadata, so serving follows every change without code edits.

4.4VLM item profiles

The full combination gives the best retrieval (Recall@100 0.2135, +1.5% over text and image) and is the production feature set. Two comparisons show what the profile carries. On its own, the 385-d profile block nearly matches the 513-d image block (0.188 against 0.192), so one short text embedding of the profile captures most of what the image embedding provides. Paired with the image at equal width, the profile matches the raw listing text (0.2126 against 0.2104), so it can stand in for the listing. Video_Games listings are text-rich, and much of what the profile says is already in the raw text; on the sparser Beauty catalog the lift doubles to +3.1% (0.0859 against 0.0833). Profiles leave the cold slice unchanged on both catalogs, and the next subsection shows that the identifier pathway is what moves it.

4.5Semantic IDs and generative retrieval

The RQ-VAE trains in about 16 seconds on the fused vectors and uses its codebooks fully: 100% utilization on all three levels and perplexities of 208, 249 and 252 out of 256. Of the 25,612 items, 9.6% share a full code tuple with another item, and the generative retriever resolves these collisions by within-group popularity.

Semantic IDs act exactly where learned IDs are weakest. In place of the learned item vector they lift cold-slice Recall@100 by 46% (0.0241 → 0.0353), because a new item's codes are already trained by every item that shares them, and the gain replicates on Beauty at +47%. Warm items still prefer a free per-item vector, so overall recall favors learned IDs (0.129 against 0.084); the two representations divide the catalog by popularity. As ranker features the codes add 0.003 GAUC, and 0.008 on the cold slice, even on top of full content. Appending codes to content in the retriever leaves recall unchanged, since codes derived from content carry nothing the content lacks; their role is a compact, content-grounded identifier.

The TIGER-style generative retriever reaches Recall@10 of 0.031 on 10,000 sampled users, half of the two-tower's 0.063, with one million parameters, a 769-token vocabulary, 48 seconds of training and no ANN index. That gap is the subject of Section 5, which measures where a generative stack loses recall relative to a dense one.

4.6Diversity and latency

Post-processing exposes a smooth relevance-diversity frontier (Table 7). Lowering the MMR weight λ, or switching to DPP, buys intra-list diversity with relevance. DPP gives the most diverse lists, 22% above pure relevance at 42% lower NDCG. Where to sit on this frontier is a product decision, and the API accepts the diversity method per request.

On CPU the cascade answers in 11 ms at the median and 16 ms at p99 (Table 8), well inside a 100 ms budget. The pre-ranker is the most expensive stage because it scores all 200 candidates, while the heavy ranker sees only 50. With the fused 1,281-d features the cascade measures 12 ms p50 and 16.4 ms p99 over a 300-request load test, and the ONNX Runtime path returns item-identical results.

4.7Replication at scale

The Beauty run reuses every line of code; one configuration file points the pipeline at a category-scoped data directory. At 729,576 users and 207,649 items, the 7.4 GB image download takes 25.7 minutes with 32 workers, CLIP encoding takes 4 minutes with DataLoader workers, and the serving registry loads in 60 seconds into 11 GB of memory. With an HNSW index the cascade serves at 8.6 ms p50 and 10.1 ms p99. Table 9 lists the comparisons repeated on both catalogs; absolute recall is lower on Beauty because it is 7.7 times sparser, so the within-catalog contrasts carry the signal.

5Where does generative retrieval lose recall?

A generative retriever differs from a dense retriever in several places at once. Item content is quantized into discrete codes, the encoder and the construction of training examples change, the scoring head and its objective change, and candidates come from a trie-constrained beam instead of a nearest-neighbour lookup. End-to-end comparisons vary all of these together, so the contribution of each stage is not isolated, and the decoder tends to take the blame because it is the easiest stage to point at. This section measures each stage on its own.

5.1Catalog-wide lazy-exact top-K ranking

Let X be the item catalog. A tokenizer assigns each item x a code tuple (c₁, …, c_L), and given a user history h an autoregressive (AR) model scores the item by

Every term is non-positive, so the score of a code prefix upper-bounds the score of every item beneath it. A best-first expansion over the code tree, with a frontier ordered by prefix score, can stop as soon as the K-th complete item outscores the best remaining prefix, and it then returns exactly the top K items with no beam width and no approximation, while touching a small fraction of a finite catalog. We compute it in a raw variant (R4a) and in a trie-renormalized variant (R4b) that renormalizes each level over the codes that continue to a real item. The scorer agrees with brute-force enumeration on toy catalogs in unit tests. With it, generative retrieval with approximate decoding removed becomes an observable system, which separates what the objective costs from what the decoder costs.

5.2A stage-matched ladder

The ladder walks from a dense retriever to a deployed generative one, and each step changes one stage. R1 is a dense two-tower over frozen content. R2 retrains the same tower on RQ-VAE-reconstructed content, isolating what quantization costs before any discrete interface appears. R3bag replaces the reconstruction with pooled code embeddings. R3s is a sequential contrastive model. R4a is the exact AR system of Section 5.1, R4b adds trie renormalization, and R5 is the deployed trie-constrained beam (width 10, filled to the cutoff with admissible items). The R3bag → R3s step changes the encoder family and the construction of training examples together, and we report it as one joint entry.

The matched pair. The R3s → R4a step carries the identification burden, so the two systems are matched rather than merely comparable. They share the same encoder architecture and initialization, consume byte-identical chronological next-item examples in the same seeded order, match total parameters within 0.02% (999,168 contrastive against 999,041 autoregressive), and receive identical validation-selected training budgets. Only the scoring head and its objective differ, and we call the resulting quantity the composite head/objective entry.

Protocol. Every rung is evaluated under one protocol: a shared candidate universe per user, a global score → popularity → index tie rule, and beams filled so that every user receives at least min(K, 200) admissible items, with zero shortages across the nine main ladders (three catalogs, three seeds each). Every entry carries a hierarchical seed × user bootstrap confidence interval: seeds are resampled, user rows are resampled jointly across the drawn seeds, and both sources of uncertainty enter the interval. Equivalence uses two one-sided tests (TOST) at a margin of 10% of the local gap. Designs, margins and the stability rule of Section 5.6 were registered before launch. Item content is the fused vector on the 2023 catalogs and text-only on Amazon-2014.

5.3Decoding is secondary but nonzero

Trie renormalization passes the equivalence test on all three catalogs, at 2.5%, 2.1% and 1.4% of the respective gaps, and it keeps passing at K = 50 and K = 100. Cutoff-matched beam search accounts for 19.2% of the Video_Games gap at R@10, 8.9% on Beauty and 5.3% on Amazon-2014, and its cutoff-specific share falls as the cutoff grows, to 2.7%, 1.2% and 0.4% at K = 100. The stage that attracts most of the decoding effort in the generative-retrieval literature is therefore not where most of the recall goes. It is not free either: a fifth of the Video_Games gap at the cutoff a product would deploy is large enough to tune for.

5.4Quantization dominates and responds to a controlled dose

Quantization (R1 → R2) is the largest gross entry on every catalog: 91.9% of the gap on Video_Games, 58.2% on Beauty and 80.5% on Amazon-2014. To test whether that share reflects the tokenizer rather than where the ladder starts, we intervene on it. Holding the code structure at three levels of 256 codes, we train the tokenizer to three operating points and re-run the ladder at each. Reconstruction error falls monotonically, and the quantization entry falls with it, from 0.0331 to 0.0240 to 0.0206 R@10 on Video_Games, with separated endpoint confidence intervals. On Video_Games the discrete codes also recover part of what reconstruction lost: the R2 → R3bag entry is −28.6%, so pooled codes retrieve better than the reconstructed vectors decoded from them. That recovery shrinks as the tokenizer improves (−0.0167 → −0.0097 → −0.0064).

5.5The matched head/objective entry depends on capacity

At the matched one-million-parameter point, the composite head/objective entry is positive on all three catalogs, with intervals excluding zero: 56.9% of the gap on Video_Games, 24.0% on Beauty and 52.3% on Amazon-2014. This is what a team comparing lightweight stacks at this scale would measure, and it describes an operating point rather than a constant. We retrain both heads at each decoder width under a frozen recipe, with learning rate and temperature fixed from the d128 grid and with identical examples and validation selection. On Video_Games the paired entry declines monotonically and changes sign (Figure 3): +0.01725 at d64 (0.4M parameters), +0.01275 at d128 (1.0M, the main ladder point), +0.00645 at d256 (3.6M) and −0.00807 [−0.01133, −0.00502] at d512 (13.4M). The sign change is bracketed between the tested d256 and d512 points, and at d512 the AR head beats its parameter-matched contrastive twin in every seed.

Both heads move. The AR head improves steadily with width, while the contrastive head peaks at d256 and falls back. The AR head's disadvantage at small width is a property of the tested regime, not of the objective as such.

Where the entry sits. At R@10 the head/objective loss concentrates in the upper popularity deciles and peaks in the hottest decile on all three catalogs (hot-versus-rest interaction of +0.062, +0.040 and +0.057, with intervals excluding zero), whereas the quantization loss spreads across the upper popularity half. One candidate explanation is that the AR head mis-calibrates item popularity, so we ran ten training-time logit-adjustment runs on Video_Games with marginal and conditional-prefix priors at |τ| ≤ 0.5. None produced a statistically resolved R@10 improvement, and every run at |τ| = 0.5 lowered R@10, by 0.0025 to 0.0077. Within this family of priors, the entry is not a popularity-calibration artifact.

5.6Attribution depends on the reference rung

Every contrast above is measured directly. A statement of the form “the AR side accounts for X% of the gap” needs one more choice: a reference rung to measure the AR side from. Three rungs are admissible on this ladder (R3s, R3bag and R2), and Figure 4 shows the AR-side share under each. On Video_Games the share runs from +56.9% under R3s to +15.0% under R3bag and −13.6% under R2, a span of 70.5 percentage points [56.3, 88.3]; none of 2,000 bootstrap replicates falls within the pre-registered 15-point stability rule. The same measured systems thus support both “the AR side costs 57% of the gap” and “the AR side recovers 14% of it”. Amazon-2014 spans 39.5 points, with 3 of 2,000 replicates within the rule. Beauty spans 9.2 points and satisfies the rule on its point estimate, with bootstrap support of 1,749 of 2,000.

The cause is the shape of the ladder. It contains measured recoveries (codes versus reconstruction at −28.6% and the joint encoder-and-examples step at −41.9% on Video_Games), so admissible paths accumulate different amounts before they reach the AR side. The same behaviour appears one level down. Frozen-encoder head swaps split the head/objective entry into a representation part and a scoring part, and reversing the order of the swap reallocates the split on both catalogs that have the reciprocal arm: on Video_Games the forward path is representation-heavy (0.0092 against 0.0036), while the reciprocal path assigns 0.0181 to the head. We therefore report reference-labeled contrasts with confidence intervals, together with the span across admissible references as a reference-sensitivity diagnostic. The span costs nothing beyond quantities a ledger already computes, and it tells a reader whether a headline attribution would survive a different, equally defensible reference.

5.7Implications and scope

For practice. For a team weighing a migration, the ledger reorders the work queue. The largest entries are the tokenizer and the matched head/objective transition, while cutoff-matched decoding stays secondary at every cutoff measured, so tokenizer training and the discrete interface repay effort sooner than wider beams. Because the stack ordering reverses with decoder width on the catalog and recipe tested, contrastive and autoregressive stacks should be compared at matched capacity before an ordering is assumed. And because attribution depends on the reference, a reported attribution should name its reference rung and give the span across the alternatives.

Scope. Both ladder endpoints are lightweight by construction, matched at about one million parameters; this is what makes a stage-matched ledger possible, and it is what the entries describe. The capacity crossover is demonstrated on Video_Games under a frozen recipe at this data scale, and it establishes that the ordering is not invariant rather than a law every catalog follows. The ledger reports reference-labeled contrasts and is not a unique causal decomposition.

6Engineering notes

Several failure modes surfaced while the system was built, and each is now guarded in code or configuration. Table 11 lists those with the widest relevance beyond this project; the repository's pitfall log records the rest with root causes.

7Takeaways

1. Item representation sets the ceiling of every stage. Content adds 66% to retrieval recall and 0.097 to ranking GAUC, and without it the ranker falls below chance on the cold slice.
2. Content-derived signals divide the catalog. VLM profiles capture most of the multimodal signal in a short embedding and their lift grows with sparsity (+1.5% to +3.1%); semantic IDs carry the cold slice, +46% and +47% on the two catalogs.
3. Cascade consistency is a data problem before it is a model problem. A clean negative pool and a continuous cross-stage score recover up to 48% of cascade NDCG@10, and a feature has to be judged by how it is built at training time, not by what it means at serving time.
4. Model choices should be selected, not fixed. Three runs produced three different winning cascade variants, and validation selection with sidecar metadata kept serving consistent through each change.
5. In the generative stack, the largest gross entries sit upstream of the decoder. Quantization is the largest entry on every catalog, trie renormalization is equivalence-negligible, and beam search is secondary but nonzero.
6. The ordering of contrastive and autoregressive heads depends on capacity, and gap attribution depends on the reference rung. Compare at matched capacity, and report the reference with the span.

VLM-Rec measures each component of a recommendation cascade instead of assuming its value, and the same system serves as a controlled laboratory for a question the field is asking now. Finite catalogs make exact top-K ranking of a generative retriever computable; with that instrument the dense-to-generative gap becomes a ledger of measured stage contrasts, and the ledger points practitioners to the tokenizer and to the head/objective transition, measured at matched capacity.

AReproduction

Every stage is one make target or CLI command, configured by configs/config.yaml; configs/scale.yaml is the Beauty profile.

uv sync --all-extras    # every stage, incl. FAISS and MLflow
make data               # download, 5-core, split, embeddings
make retrieval          # two-tower variants + retrieval ablation
make ranking            # DIN + DCN-v2 + MMoE + ranking ablation
make rerank             # consistency grid, distillation, diversity
make vlm                # Qwen2.5-VL profiles + five-way ablation
make sid                # RQ-VAE semantic IDs + SID-vs-ID ablation
make serve              # API on :8000, Streamlit demo on :8501
uv run vlmrec <stage> --config configs/scale.yaml    # Beauty

System experiments ran on a single RTX 4090 (24 GB) under WSL2, and serving latencies are CPU-only. The loss-ledger study used about 118 GPU-hours across three catalogs, three seeds and the intervention arms. Numbers in Sections 2 to 4 come from the metrics files these commands write, and docs/RESULTS.md records the provenance of each table.

References

1. S. Rajput et al. Recommender systems with generative retrieval. NeurIPS, 2023.
2. Y. Hou et al. Bridging language and items for retrieval and recommendation. arXiv preprint, 2024.
3. N. Reimers and I. Gurevych. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. EMNLP, 2019.
4. A. Radford et al. Learning transferable visual models from natural language supervision. ICML, 2021.
5. S. Bai et al. Qwen2.5-VL technical report. arXiv preprint, 2025.
6. X. Yi et al. Sampling-bias-corrected neural modeling for large corpus item recommendations. RecSys, 2019.
7. J. Johnson, M. Douze and H. Jégou. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 2019.
8. Y. Malkov and D. Yashunin. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE TPAMI, 2018.
9. G. Zhou et al. Deep interest network for click-through rate prediction. KDD, 2018.
10. R. Wang et al. DCN V2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. WWW, 2021.
11. J. Ma et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. KDD, 2018.
12. J. Carbonell and J. Goldstein. The use of MMR, diversity-based reranking for reordering documents and producing summaries. SIGIR, 1998.
13. L. Chen, G. Zhang and H. Zhou. Fast greedy MAP inference for determinantal point process to improve recommendation diversity. NeurIPS, 2018.
14. D. Lee et al. Autoregressive image generation using residual quantization. CVPR, 2022.
15. J.-T. Huang et al. Embedding-based retrieval in Facebook search. KDD, 2020.