Sambanova Systems | Software Engineer (Senior) | Coding | Onsite

""" Coding Interview Question: Implement a KV Cache Registry Background: In large language models (LLMs), a KV cache (Key Value cache) stores intermediate representations to speed up autoregressive generation. Given a sequence of token IDs as a prompt , the KV cache stores computed values such that each cache[i] corresponds to prompt[:i+1] , meaning it depends on all tokens from the start up to index i . Problem Stat...