Spotify is personalizing open-weight LLMs without full fine-tuning by combining three key components: foundational user embeddings from streaming history, 'Semantic IDs' that tokenize its 100M+ item catalog, and a 'soft tokenization' layer that projects a user's embedding directly into the LLM's context. This allows the model to autoregressively generate the next song or podcast as the next token in a sequence.
This summary explores Arpita Vats's insights on how Large Language Models (LLMs) are revolutionizing recommender systems. It contrasts the traditional feature-engineering-heavy approach with the contextual understanding of LLMs, which shifts the focus to prompt engineering. Key challenges like inference latency and cost are discussed, along with practical solutions such as lightweight models, knowledge distillation, and hybrid architectures. The conversation also touches on advanced applications like sequential recommendation and the future potential of agentic AI.