生成式推薦上生產要專屬處理,但這層目前長在通用 LLM 推論引擎上,沒有另成一個領域
生成式推薦(GR)上生產需要通用 LLM serving 預設路徑以外的專屬處理:大 beam 寬、短輸出、受項目集約束的解碼,在 vLLM 類框架的預設 beam search 上跑不滿硬體(京東自比 vLLM-Ascend 算力嚴重閒置、Spotify 生產上用 vLLM 撞到 CPU 瓶頸而改用低階加速器);但這層專屬處理目前長在通用 LLM 推論引擎上或裡面——京東 xGR 建在自家通用推論框架 xLLM 上,Spotify 選擇把所需功能併進 TensorRT LLM——而不是分化成脫離 LLM serving 的獨立工程領域(RecoGEM 另建專屬系統、底座未公開,是目前唯一可能的反例)。對推論框架與硬體廠商,GR 是要靠在引擎內補上寬 beam 與項目約束解碼來爭取的推薦流量,不是另起爐灶的新基礎設施市場。
這不是預測，所以沒有排定哪一天對答案。
什麼會推翻它
各條行尾若有日期，那是相反的證據要在那天前出現才算數的期限，不是對答案的日子。
具名公司(RecoGEM 作者所屬公司以外)在官方論文或工程部落格公開、明寫已上線生產的 GR serving 系統中,自建獨立推論引擎、不建在任何通用 LLM 推論引擎(vLLM、SGLang、TensorRT LLM、xLLM 這類)上的家數 — ≥2 家 — 達到即推翻『專屬處理長在通用引擎上或裡面』的半句,改判 GR serving 正分化成獨立技術棧 — 2027-06-30
非 GR 系統作者、非推論框架廠商的第三方公開 benchmark:未改核心的 vLLM 或 SGLang 預設 beam search,在 beam 寬 ≥128、每請求輸出 ≤4 個 token 的生成式推薦負載上,吞吐相對 GR 專用路徑的比值 — ≥0.8 — 達到即推翻『通用框架預設路徑跑不滿硬體、需要專屬處理』的半句 — 2027-06-30
Netflix 或其他具名公司公開以未改核心的 vLLM 或 SGLang、不另加專屬 serving 層,在線上生產(附 A/B 結果或流量數字)服務 semantic-ID 生成式推薦的案例數 — ≥1 家 — 出現即削弱『GR 上生產需要專屬處理』的半句 — 2027-05-31
寫下判斷時的出處
https://arxiv.org/html/2512.11529v3
「These characteristics suggest that naively adopting an LLM inference system is inadequate for GR, and a comprehensive reconstruction of the underlying pipeline is required.」
「Since vLLM-Ascend is not well optimized, its redundant memory accesses under large-width beam search lead to severe underutilization of compute units.」
https://engineering.atspotify.com/2026/10/advancing-generative-recommendation-inference-our-collaboration-with-nvidia-on-tensorrt-llm
「Specifically, vLLM’s beam search implementation operates outside the core GPU engine, which introduces CPU-side bottlenecks and additional GPU work due to the recomputation of partially filled KV blocks.」
「For our January launch, these limitations meant we couldn’t fully utilize high-end GPUs, so we opted for lower-tier accelerators to manage costs while keeping the system at a more optimal load point.」
「We kicked off our collaboration with NVIDIA with a clear goal: unify the inference features that we require (efficient beam search, scoring, and user soft tokens) into a single engine.」
「The efforts mentioned above have landed in recent versions of TensorRT LLM, and the following benchmarks show the performance edge that the implementation is delivering.」
https://developer.nvidia.com/blog/how-generative-recommenders-are-redefining-recsys-at-scale/
「They are powerful general frameworks, but do not naturally expose the core abstractions needed by Semantic ID-GR: shared request-level context KV, short per-beam decode KV, beam-path tracking, dynamic beam width, and item-constrained generation.」
https://arxiv.org/html/2512.22386v2
「To meet strict Service Level Objectives (SLOs), we develop xGR [44], a dedicated serving system built upon the high-performance xLLM framework [38].」
「We deployed OxygenREC in JD.com’s core recommendation scenarios.」
https://arxiv.org/html/2510.14686v2
「We have deployed xLLM in production to support a range of core business scenarios at JD.com, covering areas including LLM, Multimodal Large Language Model (MLLM), and generative recommendation.」
RecoGEM: A Serving System for Production-Scale Generative Recommendation · Xiangrui Yang, Yunfeng Zhao, Jiawei Yang et al. — arXiv
「RecoGEM has been deployed for over nine months across our production lines, serving applications with hundreds of millions of users and supporting more than ten GR model families.」