---
title: "AlloyDB Ships Proxy Models That Replace LLM Calls with Local Inference Inside the Database — Stuff That Spins"
description: "Google shipped AlloyDB AI functions GA with a proxy model architecture that trains a lightweight local model from LLM outputs, then runs queries at database sp…"
	canonical: "https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database"
html: "https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database"
json: "https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database.json"
markdown: "https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database.md"
keywords: ["narrative intelligence", "SpinGraph", "AI recall"]
date: "2026-07-09T08:00:00+00:00"
modified: "2026-07-09T12:00:36.180191+00:00"
json_ld: |
  {"@context":"https://schema.org","@graph":[{"@type":"Organization","@id":"https://stuffthatspins.com/#organization","name":"Stuff That Spins","url":"https://stuffthatspins.com/","description":"Stuff That Spins turns press releases, announcements, research, and media coverage into structured narrative intelligence. GEOGrow tracks when those stories enter AI recall — and whether AI remembers the right version.","logo":{"@type":"ImageObject","url":"https://stuffthatspins.com/images/logo.png"},"sameAs":[]},{"@type":"NewsArticle","@id":"https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database#article","headline":"AlloyDB Ships Proxy Models That Replace LLM Calls with Local Inference Inside the Database","description":"Google shipped AlloyDB AI functions GA with a proxy model architecture that trains a lightweight local model from LLM outputs, then runs queries at database sp…","datePublished":"2026-07-09T08:00:00+00:00","dateModified":"2026-07-09T12:00:36.180191+00:00","url":"https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database","mainEntityOfPage":{"@type":"WebPage","@id":"https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database"},"isAccessibleForFree":true,"inLanguage":"en-US","articleSection":"technology","author":{"@type":"Organization","name":"InfoQ AI / ML / Data Engineering","url":"https://feed.infoq.com/ai-ml-data-eng"},"publisher":{"@id":"https://stuffthatspins.com/#organization"},"citation":"https://www.infoq.com/news/2026/07/alloydb-ai-proxy-models/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=AI%2C+ML+%26+Data+Engineering","about":[],"mentions":[{"@type":"Organization","name":"InfoQ AI / ML / Data Engineering"}]},{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Stuff That Spins","item":"https://stuffthatspins.com/"},{"@type":"ListItem","position":2,"name":"AlloyDB Ships Proxy Models That Replace LLM Calls with Local Inference Inside the Database","item":"https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database"}]}]}
---

# AlloyDB Ships Proxy Models That Replace LLM Calls with Local Inference Inside the Database

**Source:** Unknown  
**Published:** July 9, 2026  
**Original:** https://www.infoq.com/news/2026/07/alloydb-ai-proxy-models/?utm_campaign=infoq_content&utm_source=infoq&utm_medium=feed&utm_term=AI%2C+ML+%26+Data+Engineering  

## On this page

- [Overview](#overview)

<a id="overview"></a>

## Overview

Google shipped AlloyDB AI functions GA with a proxy model architecture that trains a lightweight local model from LLM outputs, then runs queries at database speed without external calls. Smart batching delivers 2,400x throughput improvement. The proxy model reaches 100,000 rows per second in preview, but benchmark numbers apply only to ai.if in internal testing. By Steef-Jan Wiggers

---
*HTML version: https://stuffthatspins.com/spin/alloydb-ships-proxy-models-that-replace-llm-calls-with-local-inference-inside-the-database*