AI Agents

Beyond Buttons and Prompts: How We Let AI Agents Build Their Own UI

How Auraa Semantic Flow renders human-in-the-loop AI agent interactions identically in web and chat from one JSON descriptor, no new frontend code per agent capability.

How We Let AI Agents Build Their Own UI with Semantic Flows
6:29

Buttons_Prompts

 
 

Imagine an AI agent that has just analyzed 47 source tables from your R&D database. It profiled query activity, grouped the tables into three ingestion frequency tiers, proposed 28 quality rules with varying confidence levels, and generated complete DataDuct ingestion specifications.

Now it needs to tell you what it found and get your approval before running anything.

How does it do that?

The honest answer, for most AI platforms today, is: badly.

The interaction problem no one talks about

The conversation about AI in data engineering almost always focuses on what agents can do. There is a second problem that gets far less attention: what happens when an agent needs a human to review something before it acts?

Agents have two options today. Free-text prompts are expressive but unstructured. Buttons are structured but rigid. Neither works for complex data engineering decisions.

When an agent proposes quality rules, you need to see which rules and how confident the agent is about each one. When it groups tables into ingestion tiers, you need to reassign tables before approving. That is not a prompt. That is not a button. We built something else.

Auraa Semantic Flow

A Semantic Flow is a JSON-serializable structure that completely describes a human-facing interaction: what to display, what the user can do, where to send their input, and when the interaction expires.

Any backend service creates a Semantic Flow. A generic rendering engine displays it, identically, in the web UI or in a conversational interface. The backend service never writes frontend code. 

The frontend is a rendering engine, not a feature host. New agent capabilities ship without new UI code.

What agents can express

Auraa Semantic Flows provide 14 domain-specific section types. Four of them show the range:

grouped_table lets an agent propose categorized data with drag-to-reassign between groups, which fits frequency tier assignment exactly.

checkbox_group with confidence makes AI uncertainty visible. Items above a threshold are pre-checked; items below are unchecked with an explanation. The human sees what the agent proposes and how certain it is.

progress with built-in polling handles async execution automatically. No custom polling infrastructure, no separate frontend state management.

confirmation surfaces context-specific warnings before anything runs:

This will create 47 ingestion specs and start pipeline execution. Estimated processing time: 15 minutes.

The business impact

The first production consumer was the pipeline configuration workflow. Before Semantic Flows, delivering that interaction would have required a dedicated React page, custom state management, and drag-and-drop components, an estimated 2 to 3 days of frontend work.

With Semantic Flows, the backend engineer wrote an adapter function that transforms the agent's PipelinePlan into three AuraaSemanticFlow instances. The frontend rendered them using existing components. Effort: about 2 hours.

 

Workflow Old approach With Semantic Flows
Pipeline configuration 2–3 days, dedicated React page ~2 hours, one adapter
Grant template approval New frontend build Under 2 hours, no frontend changes

The second consumer was grant template approval: policy preview, conflict detection, scope confirmation. No frontend changes. One adapter function, written in under two hours.

Dual rendering: one descriptor, two surfaces

The same Semantic Flow renders as an interactive web page and in a conversational interface. The web renderer maps section types to lazy-loaded React components. The text renderer maps the same sections to ASCII tables and numbered action options.

One backend service. One descriptor. Two rendering surfaces. No divergence between the web UI and the chat interface.

 

See Semantic Flows in action

Watch how Auraa agents propose, explain, and get approval for data engineering decisions, on the web and in chat from a single descriptor.

 

Similar posts

Get notified on new marketing insights

Be the first to know about new B2B SaaS Marketing insights to build or refine your marketing function with the tools and knowledge of today’s industry.