Prompt to Pipeline: Building with Google's Gen Media Stack — Paige & Guillaume, Google DeepMind

Paige Bailey, from the Google DeepMind team, opened by recounting her career journey, deeply rooted in the open-source scientific computing stack (NumPy, SciPy, TensorFlow). She shared a key insight on industry trends: many tools and frameworks that arise to solve temporary model limitations are eventually absorbed into the models themselves.

“Usually if you see everybody sprinting to do the same thing, that’s a great indication that it’s the wrong thing… or a thing that eventually the model will have that capability to do.”

She cited examples like the rush to build vector databases to overcome small context windows, the creation of language-specific fine-tunes, and the proliferation of agent frameworks—all capabilities that are increasingly becoming native to foundation models like Gemini.

AI Studio and Gemini 1.5 Capabilities

Paige demonstrated a series of powerful features within AI Studio, emphasizing speed, cost-effectiveness, and ease of use.

• Cost-Effective Video Analysis: Using Gemini 1.5 Flash, she analyzed a 5-minute (300-second) YouTube video of dinosaurs. The model processed the video frame-by-frame (at 1 FPS) and, with Google Search grounding enabled, generated a markdown table of all the dinosaurs present, complete with timestamps and a fun fact for each. The total cost for analyzing the 30,900-token input was less than a dollar, showcasing the model’s efficiency.

• Code Execution for Vision Tasks: To demonstrate the built-in Code Execution tool, Paige provided an image of LEGO bricks and prompted the model to “draw bounding boxes around all of the green Lego bricks using Python.” Gemini 1.5 Flash quickly generated and executed the necessary Python code within a sandboxed environment to identify the bricks and display the resulting image with bounding boxes, costing only a few pennies.

• Full-Stack App Generation with AI Studio Build: In a powerful demonstration of low-code development, Paige used the Build feature to create a full-stack web application from a single, detailed prompt.

  • The Prompt: Create an app where a user can log in with their Google account, upload an image of their bookshelf, and have the app identify the books, use Google Search to find details (author, description, genre), and save the information to a database associated with their account.
  • The Result: AI Studio generated a complete application named “Shelf Scan,” automatically setting up Firebase for Google authentication and Firestore for the database. When an initial run failed due to insufficient permissions in the generated Firestore rules, Paige simply described the error to the model, which correctly identified the issue, located the firestore.rules file, and rewrote the rules to fix the bug. The final, working app allowed users to log in, upload a photo, and see their books cataloged in a persistent, user-specific library.

• Genie 3: Interactive World Generation: Paige showcased Genie 3, a “world model” that generates interactive, playable 2D worlds from text descriptions. Unlike game engines that create 3D assets, Genie generates the experience pixel-by-pixel, frame-by-frame. From the prompt “a pink sparkly squirrel with a pirate hat on Regent’s Canal with dolphins,” Genie generated a playable environment where the user could control the squirrel with keyboard inputs.

Orchestrating a Generative Media Pipeline

Guillaume “Guom” Vernade demonstrated how to orchestrate multiple generative media models to bring a public domain book, The Wind in the Willows, to life.

1. Content Ingestion & Prompt Generation: Gemini was used to read the entire book and generate structured prompts for subsequent tasks, acting as the central orchestrator for the project.
2. Character & Scene Illustration: Using the prompts from Gemini, Nano Banana 2 generated character portraits and key scenes for each chapter. To ensure visual consistency, the generated character portraits were fed back into the context for subsequent scene generations.
3. Video Animation: VO 3.1 Light took the static images from Nano Banana and animated them into short video clips, bringing moments like a picnic by the river to life.
4. Musical Scoring: LIA 3 was prompted with simple mood descriptions (e.g., “jaunty adventurous rhythmic with fiddle”) to generate unique, fitting instrumental music for each chapter.
5. Multi-Voice Audio Narration: In a particularly clever use of prompting, Guom used the text-to-speech model to create a dialogue with multiple distinct characters from a single base voice. By embedding style instructions directly into the script (e.g., Character (fast-paced, British accent): "Hello!"), he made the model alter its delivery for each line, creating a rich audio scene.

Gemma 4: Powerful On-Device and Open Models

Ian Valentine concluded the session by showcasing the capabilities of Gemma 4, Google’s latest family of open-weight models, emphasizing their ability to run complex agentic workflows entirely on-device without cloud API calls.

• Local Multi-Agent System: Running the Gemma 4 26B model locally on a Mac via LM Studio, Ian launched an orchestrator agent that spawned 10 parallel sub-agents. Each sub-agent was tasked with generating a unique SVG image. The entire process, from task distribution to final SVG compilation, happened on the local machine.

• On-Device Code Generation and Debugging: In the most compelling demo, Ian used the local Gemma 4 26B model with Open Code, a coding environment.

  1. Implementation: He provided a text file containing a high-level specification for a game called “Nebula Drift.” The model read the spec and wrote the necessary HTML, CSS, and JavaScript files to create the game.
  2. Debugging: The first version of the game had a bug and wouldn’t start. Ian copied the JavaScript console error message and fed it back to the model.
  3. Self-Correction: Gemma 4 read the error, re-examined the code it had written, identified the mistake, and successfully rewrote the file to fix the bug. The corrected game then ran perfectly. This entire build-debug-fix loop was performed locally with no external API calls.

• Multimodal Understanding: Ian also demonstrated Gemma 4’s ability to take a screenshot of a webpage and generate the corresponding HTML and CSS to recreate its structure and style, showcasing its powerful vision capabilities.

• Tags:
• Code Generation
• Feature
• World Models
• On-Device AI
• Gemma 4
• AI Studio
• Gemini 1.5
• Generative Media