03Enterprise AI Answers PlatformShipped
How do you enable knowledge owners to clean up issues in thousands of Q&A entries?
Customers' AI-powered knowledge bases had become cluttered with duplicates and conflicting answers, degrading answer quality. I led definition, design and launch of an MVP to address the problem that drove organic adoption and unsolicited customer praise.
Role: Principal Product DesignerDuration: 6 monthsTeam: Product Manager, Lead Engineer, Customer Success
Adoption & praise
Organic
Problem
The product's AI questionnaire answering capabilities depended on consistent, clean, and accurate documents and saved answers, but customers' knowledge base entries had become extremely cluttered and stale over time. Duplicate and conflicting answers were degrading answer quality, and users lacked the bandwidth to deal with them.
Key insight
Knowledge base owners are quite motivated to identify conflicts and clear them out when highlighted, but only if it can be done efficiently.
I began by analyzing other knowledge base tools and the varied approaches they had taken to surface duplicates and conflicts. I scoured help articles, demo videos, and blog posts to capture how others had approached these workflows.
In parallel, I coordinated with the customer success team, joining calls to better understand how customers were managing their content. Some managed offline, some managed within the UI. Some were excited and ready for automation, but most were hesitant about letting go of quality control.
Synthesis
I synthesized my findings into simple visualizations and walked the extended team through the context. Customers without an offline process for managing content were less tolerant of automation, so we decided to focus on surfacing suggestions for approval.
Concept exploration
I spent a day mocking up 5–10 concept wireframes to illustrate the range of approaches I could see for effectively surfacing the issues we identified. One concept framed it as an 'integrity check' tool with its own workflow, navigable sections organized by different types of issues.
A more automated approach
One concept involved jumping users straight to suggestions, where issues and bulk fixes were automated in a single click. When reviewing with target customers, however, we found that they weren't ready to trust automation; they needed to manually review and revise any changes for now.
question we faced
Should the tool automate fixes or surface suggestions for manual review?
Options we considered
Automated bulk fixes
Faster resolution for users, but customers weren't ready to trust automation with the quality of their knowledge base.
Suggestions for approval
More manual, but matches the level of control customers needed to feel confident adopting the tool.
What we chose and why
We focused on surfacing grouped suggestions for manual review. Customers could scan related entries, understand the differences, identify which to keep, and edit inline, giving them control, while dramatically reducing the time required.
What we learned
Automation that erodes trust doesn't save time, it adds time and stress. Meeting customers at their current comfort level made adoption faster than any shortcut would have.
Finding the right MVP
Customers preferred a concept that grouped related entries and surfaced them in a list for review, with the ability to dive into each one. Users could scan all related entries, understand the differences, identify which to keep, and edit them inline. While I had envisioned more advanced AI-powered semantic understanding, the engineering team wanted to start with a simpler semantic similarity tool for the first launch, one only able to detect similar and duplicate questions. Fortunately, we discovered with customers that this would still be a significant value add. They were excited by the prospect that these entries with similar questions could help them find conflicting answers, and jumped at the chance for an easy way to quickly compare.
I introduced a new card component into the design system to more elegantly display these groups, cards listed in order of severity.
I worked closely with the lead engineer to implement the feature, collaborating to refine interactions and UI together until it felt sufficiently intuitive, smooth, and clear.
We launched with minimal promotion and found users discovering it organically, fully adopting it into their process.
Adoption and what came next
Not only was usage trending up quickly, we also soon began to receive unsolicited praise from customers through the support team. Customers were mentioning it proactively in check-ins — often enthusiastically. Having lost our PM before the project started, I stepped up to organize and synthesize all analytics and feedback, and propose a roadmap for future iterations. After reviewing with the VP of Product, we were able to get a v2 prioritized.
Outcomes
Customer behavior
Adopted
Customer feedback
Unsolicited praise
Competitive advantage
Differentiated
AI Answer quality
Improved
Reflection
- Validating the concept with skeptical customers before building saved us from shipping automation that would have gone unused, leaving AI quality degraded.
- Scoping to a simpler similarity tool for the MVP felt like a compromise, but customers confirmed it would still be a significant value add. Sometimes the simpler version is exactly right.
- Losing the PM before the project started led to a broader role; synthesizing analytics, collecting feedback, and proposing (and eventually shipping) the v2 roadmap.
Next case study
Consumer Accounting Platform
No more accounting mistakes
Multi-quarter overhaul that reduced accounting errors by ~25% and shaped how QuickBooks features were built for years.