BlockLock: Reproducible LLM Inference with Fixed Batch Geometry
arXiv:2611.06977v1 Announce Type: new Abstract: Reproducible LLM inference requires that a request produces the same output regardless of the other requests in the batch. Existing solutions pay for this in different ways: batch-invariant kernels constrain every operator, and isolating prefill with fixed-shape decode ({our CoRun-style baseline}) gives up shared prefill compute and pads decode. We present BlockLock, which keeps shared batched prefill and pins only the numerical path of the protected request: it fixes the geometry that the request sees, with {two modes: BlockLock-Rows uses fixed-
阅读原文 · arXiv 人工智能本文由 SoccerDream AI 自动聚合索引，内容版权归原作者与 arXiv 人工智能 所有。
本站仅作技术动态聚合，不构成任何形式的建议。
本站仅作技术动态聚合，不构成任何形式的建议。