Konovo: AI-Assisted Question Coding

TURNING Manual Analysis into QUICK AI-Assisted Research

Role: Director of Product Design
Timeline: December 2024 – May 2025
Designed With: Jie Liu (Product Designer)
Teams: Product, Engineering, Product Management, Research Operations
Responsibilities: Product Strategy, UX Design Direction, User Research, Usability Testing, Interaction Design, Stakeholder Alignment, Design Leadership
Impact: Reduced the time and effort required to analyze open-ended survey responses by introducing AI-assisted categorization workflows that accelerated qualitative research while preserving researcher control and customization.

Overview

Open-ended survey questions often produce deep qualitative insights, but analyzing hundreds of open ended text responses has traditionally been an expensive and time-consuming manual process. Researchers must review every response, group similar themes, create categories, and continually refine those categories as new responses arrive. While AI presented an opportunity to automate much of this work, researchers still need full transparency and control over the final output.

As Director of Product Design, I led the UX strategy and design direction for an AI-assisted Open-Ended Question Coding experience in collaboration with Product Designer Jie Liu, whom I managed throughout the project. Together we worked closely with Product, Engineering, and research stakeholders to design an AI-powered workflow that dramatically accelerated the team’s analysis while preserving user confidence through intuitive review, editing, and customization tools. The result established a practical framework for incorporating AI into a complex professional workflow without removing users from the decision-making process.

The Problem

Designing this experience required much more than adding AI-generated categories to survey results. Researchers needed to see how the AI reached its conclusions, quickly identify responses that required adjustment, and efficiently refine categories without starting over. The workflow also needed to support surveys in multiple states of progress, accommodate both newly generated and previously categorized data, and integrate naturally into an existing reporting platform used by experienced researchers.

The editing experience presented an equally complex design challenge. Users needed the flexibility to adjust individual responses while also performing bulk updates across a multitude of records. We explored multiple interaction models before arriving at a solution that balanced speed, discoverability, and precision. At the same time, we modernized the interface with improved accessibility, WCAG-compliant color contrast, clearer hover and interaction states, and scalable visual patterns.

Usability Testing

Because this feature introduced an entirely new AI-assisted workflow, validating the interaction model before development was a critical part of the design process. Jie Liu and I planned and facilitated a series of moderated usability studies to evaluate three distinct prototype concepts with both internal stakeholders and experienced researchers. Participants completed realistic tasks such as creating and editing categories, modifying AI-generated results, performing bulk edits, recategorizing responses, and filtering large datasets while thinking aloud throughout the session.

Rather than measuring whether users could complete tasks alone, we focused on understanding how researchers naturally approached qualitative coding and where AI could accelerate their workflow without reducing transparency or control. We observed how participants expected to refine AI-generated categories, when they preferred editing responses individually versus in bulk, and how they interpreted complex interactions in the experience.

The research uncovered several consistent themes that directly shaped the final design. Participants favored starting with AI-generated categories before refining individual responses and using bulk editing later in the workflow for larger-scale adjustments. We also identified opportunities to improve discoverability by simplifying bulk editing, expanding clickable target areas, clarifying sorting and filtering controls, strengthening category editing interactions, and making system actions more visually apparent. Researchers also requested richer analytical context, including category counts, percentages, summary tables, and visualizations to better understand the distribution of coded responses.

Following the research, we synthesized findings into a prioritized backlog of usability improvements and created an updated design that incorporated the highest-impact changes. These iterations included clearer editing workflows, enhanced accessibility and hover states, improved WCAG-compliant color contrast, stronger visual hierarchy, summary reporting views, and refined interactions for both novice and experienced users. The resulting experience aligned much more closely with researchers' existing mental models while significantly improving efficiency and confidence in AI-assisted qualitative analysis.

Research & Design Process

The project was highly iterative, involving multiple rounds of concept exploration, stakeholder reviews, and usability testing. We evaluated several information architectures for presenting AI-generated coding results, experimented with different dashboard and editing layouts, and tested how researchers interacted with AI-generated recommendations. These sessions validated core workflows while uncovering opportunities to simplify navigation, improve editing efficiency, and strengthen user trust in the automated categorization process.

The final experience combines AI-assisted categorization with intuitive human review. Researchers can generate categories automatically, inspect AI-generated results, edit individual responses or entire groups through dedicated single and bulk editing modes, recategorize data as additional responses arrive, and organize results using sorting and filtering tools. Throughout the design process, we intentionally positioned AI as an assistant rather than a replacement for researcher expertise, allowing users to quickly reach an accurate starting point before applying their own judgment where needed.

Impact

The finished experience transformed one of the most labor-intensive steps in qualitative market research into a streamlined AI-assisted workflow. By dramatically reducing the amount of manual categorization required while preserving complete user control, researchers could spend less time organizing data and more time interpreting insights.

Beyond improving productivity, the project established design patterns for future AI-assisted workflows across the platform. It demonstrated how thoughtful interaction design, usability testing, accessibility improvements, and human-centered AI principles can work together to create experiences that increase efficiency without sacrificing transparency, confidence, or user control.