Zillow, a leader in real estate technology, faces a unique challenge: it lacks a cohesive conversation thread with its customers. Users transition from a phone screen to loan officers and real estate agents, often across months or even years, trusting that the process will unfold smoothly. Unfortunately, a single chatbot cannot manage this intricate journey effectively.
During VB Transform 2026, Zillow’s Senior Vice President of Engineering, Toby Roberts, and Arvind Jain, Co-Founder and CEO of Glean, shed light on their innovative AI architecture designed to maintain context throughout the customer journey. They emphasized that preserving contextual understanding, rather than merely processing raw data, presents significant challenges. With Zillow influencing nearly 80% of U.S. real estate transactions annually, the company has harnessed AI technology long before the rise of ChatGPT.
“We quickly realized that we needed a persistent context layer to accompany our customers and professionals wherever they are,” states Roberts.
Data Management: A Foundation for Success
Roberts explains that Zillow’s AI progression mirrors where most companies begin: with data. “We initiated a comprehensive effort to establish a robust data foundation,” he said. This involved creating a governance structure using a data mesh approach, ensuring clear data lineage, alongside permissions linked directly to the data.
While these foundational tasks were straightforward, the real challenge lay in creating a system that could remember customer interactions along their journey, regardless of the platform they used next.
“At any point in your journey, this context layer must remain active to support your needs,” Roberts asserts. Instead of relying on a standalone chat interface, Zillow opted to manage this context layer internally. The decision stemmed from analyzing the transaction’s overall structure rather than a single conversation point.
Zillow’s Custom Architecture and Glean’s Role
Rather than funneling customers through a single model API, Zillow developed its own harness, leveraging its 20-year legacy in machine learning seen in products like Zestimate. The focus was on creating smaller, task-oriented models that are specifically fine-tuned, rather than centering around one generalized model.
On the technical side, this harness seamlessly integrates with Glean. As Roberts indicated, Zillow currently operates thousands of Glean agents, managing repetitive tasks effectively and executing tens of thousands of operations across the organization. Jain highlighted Glean’s premise: instead of having multiple departments like finance, legal, and marketing develop separate connections to the same system, integration can be streamlined through the Glean MCP gateway.
This centralization not only boosts efficiency but also acts as a cost lever. Jain identified two significant mechanisms: model routing, which directs most tasks to smaller, cheaper models instead of defaulting to the primary model; and precomputed context, which significantly cuts down token consumption by assembling context ahead of time.
“Claude can be sluggish, with the initial context assembly taking an extensive amount of time,” says Jain. By relaying requests through Glean, businesses can potentially halve their token usage.
Implications of Zillow and Glean’s Approach for Businesses
This insightful session offered practical guidance for companies eager to develop agent AI tailored to their unique systems in terms of data management, costs, and permissions.
Create measurement baselines before your AI initiatives. Roberts emphasized that Zillow’s capacity to demonstrate a 40% increase in shipping codes resulting from AI implemented stemmed from the DORA metrics baseline established years prior to deployment.
Centralize context management to avoid duplicative efforts. Jain advocated for Glean’s platform, highlighting that many organizations overlook the hidden costs associated with redundant integration projects across different departments.
Relying solely on permission inheritance may not suffice for regulated data. Despite having a context platform designed for permissions, Zillow has enforced stringent rules and compliance checks for sensitive data categories, rather than depending solely on the automated architecture.
View context as a cost lever, not just an added feature. Jain pointed to model routing and precomputed context as strategic methods for reducing AI expenditures while minimizing wasteful token consumption without expanding new functionalities.
“Models alone do not suffice to bring AI automation to your enterprise,” Jain concludes. “You must incorporate corporate context alongside them.”
Source: venturebeat.com