Context
In 2023, Zillow decided to find out what LLMs could do for the hardest part of home buying: the money. As a senior designer on the Dream and Search team, I was asked to lead design for AskZ, a finance-focused chatbot to help people with financial questions on the way to a home, and the company's first LLM product. I was the design lead and the only designer, partnering with three PMs, three engineering teams, a researcher, and a content strategist, with AI, Search, Mortgage, and Legal leadership all at the table. It shipped on web and mobile web, with webview integrations for iOS and Android.
Real estate is an unforgiving domain for generative AI. A hallucinated restaurant recommendation is a bad dinner; a hallucinated answer about mortgages or property taxes can distort a six-figure decision. The quality bar had to be set accordingly, and leadership set it high.
How I worked
This was a new technology for the whole organization, so I embedded directly with engineering rather than working downstream of them, following their investigation of building on our own LLM versus a GPT integration, and what each meant for guardrails. I needed to understand what the model could actually do, because those properties are the design material. I also built empathy the unglamorous way: by becoming a heavy LLM user myself (for work, for personal questions, even recipes) until I had instincts for where these systems delight and where they quietly fail.
Before any interface work, I ran a collaboration session to align the team on ethics: we combined industry-leading AI values with Zillow's core values and our design principles, landing on four commitments: fairness and integrity, inclusiveness, accountability, and transparency, with fair housing compliance woven through all of them. Those principles weren't a poster; they became acceptance criteria for the design decisions that followed. And I brought the content strategist in from day one, because in a product where the interface is mostly language, content strategy is half the design team.
Two directions, three weeks
Leadership gave us a deadline of just a few weeks for design direction, so I presented two: a traditional chatbot pattern, and an emerging AI-native UI. We chose the latter, stripped to what one Microsoft designer memorably called "deceptively simple": a clean, brand-aligned conversational interface where responses stay clear and concise, cutting cognitive load while keeping warmth. Simple to look at; very hard to keep simple.
The transparency principles showed up as concrete patterns: the product clearly disclosed that you were talking to an AI, acknowledged that answers could be wrong with avenues to verify, named its sources explicitly, and was upfront about data use. In 2023 these weren't yet table stakes; we assembled them from first principles and early industry practice.
Scope as a safety decision
The first real design decision wasn't a screen. It was the boundary. We deliberately launched finance-only. In a domain where a wrong answer can distort a six-figure decision, a narrow product that's reliable beats a broad one that's impressive, and every capability we declined to build was a failure mode we didn't have to design for.
The beta then taught us something important about boundaries: users don't read them. Many testers asked general home-buying questions anyway, which meant we either had to expand the knowledge or make the scope unmistakable. That finding directly shaped the scope-teaching patterns, like recommended next questions that quietly demonstrated what the product was good at, and informed the roadmap toward the broader home buying and selling scope that came next.
Fair housing: where AI safety got real
Real estate has a failure mode most AI products don't: the law. An LLM asked the wrong question about neighborhoods can walk straight into illegal steering: influencing where people look based on characteristics protected under the Fair Housing Act. And the stakes aren't abstract: Zillow's own research found a majority of Americans report experiencing housing discrimination in their lifetime, with far higher rates among LGBTQ+, Black, and Hispanic respondents. We spent a great deal of the project weaving fair housing guardrails into the experience, and the team developed a Fair Housing Classifier to detect noncompliant language, which Zillow later open-sourced.
My role sat around the classifier, not inside it. I was in the room as VPs and directors worked through why these guardrails had to exist, and I could contribute meaningfully because I'd been a real estate agent. Fair housing law wasn't a learning curve for me; it was a professional obligation I'd personally carried. Few designers in AI have operated under the regulations their product now has to respect.
From there, I contributed in two concrete ways. First, I helped train and test the model by deliberately asking it inappropriate questions, probing for the steering, bias, and boundary-pushing a real user (or a malicious one) would eventually try. I later learned the industry calls this red-teaming; at the time I just knew which questions an agent would get sued for answering. Second, when the classifier flagged a violation, the product still needed to respond: declining without shaming, explaining without a legal lecture. I designed those states and connected my content strategist to the legal and fair-housing parties needed to formulate the actual response rules, so the refusal language was correct, humane, and owned by the right people.
Designing the failure states
An AI product is defined by its worst answer, not its best, so we treated the unhappy paths as core product surface:
- Unknown answers. When the model didn't know, it said so, and when the question warranted a human, we linked directly to customer service rather than letting the model improvise.
- Policy violations. Distinct, carefully written states for questions the product wouldn't answer, designed to preserve dignity and offer a path forward.
- Time-outs and lost connections. Mundane failures, designed with the same care, because to the user, a hung response and a wrong response erode the same trust budget.
- Grounding every answer. Responses signaled their sources with links to Zillow's own articles and listings, so people could verify rather than take the model's word for it.
Closing the loop
Trust also needs an exhaust system. We built thumbs up/down on responses, with a reporting flow behind the thumbs-down so problems came back to us with enough detail to act on. And recommended next questions, related to what the person just asked, did double duty: they kept the conversation moving and quietly taught people what the product was good at, keeping usage inside the scope where we could be reliable.
Alongside our engineering partners and SLT executives, I participated directly in testing and training the model, from adversarial probing to evaluating everyday outputs and feeding failure patterns back into both the model and the interface. Design critique and model evaluation turned out to be the same muscle.
How we tested: dogfood, then the real world
Our primary launch goal was risk mitigation: get a high-quality experience in front of the broader public, not a flashy one. So we tested in rings. First, an internal MVP: we recruited across Zillow, had colleagues use the test environment, leave feedback on responses, and complete a survey, with our researcher synthesizing results. We learned a lot and shipped quick fixes before any external user saw the product.
Then a structured public beta: 10 days, 75 external users with the beta activated on their real Zillow accounts, measured on overall experience, relevancy, perceived latency, ease of use, value, and likelihood to use, followed by a six-person focus group to go deeper than the numbers.
The beta gave us two clear improvement paths: the scope confusion above, and the finding that responses were too text-heavy, pointing toward embedded objects like data visualizations and calculators as the way to make answers genuinely useful rather than merely correct. Testers loved what worked: 24/7 availability and fast answers to simple questions. That signal is ultimately what unlocked the investment that followed.
Outcome
The beta launched smoothly, which for a first LLM product is itself evidence the failure-state work paid off. The results led leadership to greenlight a full-service Zillow chatbot, since evolved into an AI copilot with its own dedicated design team of four, and a full-knowledge beta reaching thousands of users. AskZ became the direct foundation for Zillow's AI Search experience today. The fair housing guardrail work grew beyond our product entirely: out of the combined effort, the engineers on my team created the Fair Housing Classifier, which Zillow open-sourced so the rest of the industry could build on it.