Extend Assist with home management help, starting with documentation

[nielsrowinbik]

Problem statement

Home Assistant's Assist feature is built for home control: turning lights on, adjusting the thermostat, running a script. But managing a home, configuring automations, diagnosing problems, understanding what a setting does, is a different kind of task, and Assist offers no help with it today. Home maintainers who need guidance are on their own, searching docs, browsing forums, or turning to external AI tools that know nothing about their setup.

This is the first step in a broader effort to bring AI-powered management help into Home Assistant. For the full vision, see the AI in Home Assistant vision document (TODO: Add link). This opportunity establishes the split between "control" and "management" as a product concept, and validates it with a first concrete capability: documentation access.

Community signals

• Multiple custom integrations exist that add AI-assisted home management to Home Assistant, indicating strong demand for this kind of capability. These projects take two different approaches: external tooling via an MCP server that accesses HA through an API (most notably ha-mcp, with 2600 stars on GitHub), and internal tooling built into HA itself (such as the community-built ha_claw). This opportunity assumes an internal tooling pattern; the choice between these two approaches is an open architectural question listed in Risks.
• This demand is confirmed by the numerous posts on Reddit, like this one, and articles on tech news outlets, like this one on The Verge, all explaining how powerful being able to use AI with Home Assistant can be.

While none of these ask for what we've proposed in this opportunity specifically, they clearly signal the enormous amount of potential of our overall vision that this opportunity contributes to.

Scope & Boundaries

In scope

• A "home management" mode toggle within the existing Assist dialog that switches between control and management capabilities
• HA documentation retrieval as the first management capability: the assistant can look up and explain documentation in response to maintainer questions
• The toggle and mode distinction as a reusable foundation for future management capabilities

Not in scope

• Context awareness (knowing what the user is looking at when they open Assist) — that is a separate opportunity
• Any authoring or editing actions such as creating automations or modifying dashboards
• Changes to the Assist dialog surface or layout
• Expanding or modifying the existing home control capabilities of Assist

Foreseen solution

A way, possibly a toggle, possibly auto-inferred, for the user to switch the Assist dialog into "management mode." In this mode, the assistant has access to HA documentation and can answer questions about how things work, what settings mean, and how to approach a task. The control pipeline remains unchanged and is still accessible via the same dialog. This capability will only be available to Home Assistant users with admin access.

In the video below, we show a quick prototype on what the Assist dialog may look like once this opportunity is done. Note how there is no toggle. It also provides a sneak peak on extended capabilities and how we envision we will handle permissions.

CleanShot.2026-05-06.at.12.38.46.mp4

Risks & open questions

• AI back-end: Users can bring their own AI provider, but is HA Cloud's own offering sufficient for a good out-of-the-box documentation experience?
• Documentation retrieval approach: RAG over hosted docs vs. live retrieval from docs.home-assistant.io have different tradeoffs in latency, accuracy, and infrastructure. This decision should be made early.
• Surface choice: This opportunity commits to Assist as the surface for both control and management. Assist is a text input with optional voice, not a voice-only product, and evolving its scope is the right call over introducing a parallel surface. This is a strategic product decision that should be reviewed explicitly before the opportunity is approved.
• Privacy: Even read-only management queries may include home configuration details depending on future scope. Establish the privacy principle now, before context is added.
• Tooling location: Where will the documentation (and future) tooling live? Externally, such as in an MCP server that access HA via an API (this is ha-mcp's approach), or internally, like how our own mcp_server integration and the community-built ha_claw operate?.

Appetite

Medium — 1 cycle. The toggle concept is lightweight, and documentation retrieval is a well-understood problem. The main investment is the back-end integration and the mode distinction in the UI.

Execution issues

No response

Decision log

Date	Decision	Outcome

[jlpouffier]

AS discussed already @nielsrowinbik
I think the main decision boils down to where this documentation too, but also all future tools will live.

I see two routes

• External: An MCP server that accesses Home Assistant via some API, like this https://github.com/homeassistant-ai/ha-mcp
• Internal: Just like our intents are. Just like this https://github.com/ha-china/ha_claw

As you can see the community is already building projects going these two routes. We just need to drive a decision now so that we can iterate on the tools creation later on.

This opportunity is just the first one, so a decision is needed

[frenck]

Putting my view here so it lands publicly. I have raised this in our internal direction conversations and it belongs on the issue too.

The direction is right. A management-oriented capability for maintainers, admin-only, documentation-aware to start. That fits where the broader AI work has been heading, and I agree with the framing of splitting "control" from "management" as a product concept.

The surface is wrong IMHO. Folding management mode into Assist couples a new product (AI-assisted home management) to the identity of an existing one (voice control). Users will expect voice-control behavior from the management mode, and management-mode behavior from voice control. We end up holding two products inside one surface for the long run, and the coupling will be hard to undo later. This is a strategic call, not a UX-testing call. The "toggle discoverability" risk in the body treats it as the latter.

There is also a fit problem the body does not call out. The Assist dialog as it stands today is too narrow for the kind of interaction this opportunity is asking it to host. Looking something up in documentation, getting an explanation, or asking how to approach a task is multi-turn and reads-heavy. The current dialog was sized for "turn on the kitchen light", not for "explain what this option does and what the tradeoffs are". The scope explicitly excludes changes to the surface or layout, which means we are dropping a wider use case into a UI that was sized for a narrower one.

The "AI in Home Assistant vision document (TODO: Add link)" reference is still a TODO in the body. That document is what this opportunity is supposed to hang off of. Without the link, the betting table cannot see the framework this is a piece of.

On the core itself. Even the documentation capability looks safer than it is, because it inherits two architectural calls this opportunity is making by implication.

The first is the surface call described above (lives in Assist, or in a new sidekick).

The second is the tooling-layer call that @jlpouffier just flagged: do AI tools live external to HA via an MCP server (the ha-mcp pattern that this opportunity cites as a demand signal), or internal to HA as native plumbing (the intent-style pattern, with ha_claw as a community example). These are different patterns with different consequences for portability, latency, security, and the integration surface.

The opportunity picks the internal-Assist surface and (implicitly) an internal-tooling pattern. Both are valid choices. Neither is named as a choice in the opportunity. The ha-mcp citation under Community signals actually points at the opposite tooling pattern from the one this opportunity is built on, which makes the signal-architecture mismatch worth calling out.

What I would suggest:

• Figure out our broader direction first: the chat-based sidekick shape this opportunity assumes, versus the sprinkled-across-the-UI shape that Make AI features in Home Assistant easy to discover and set up and disable #53 assumes. Both opportunities sit downstream of that decision.
• Resolve the surface call (Assist vs sidekick with its own identity) and the tooling call (external MCP vs internal intents) explicitly, before this opportunity is approved. They are different decisions that this opportunity bundles into one default.
• Decouple the documentation capability from these assumptions. Specify it as a capability that can attach to whichever direction, surface, and tooling layer we land on.
• Cross-link Make AI features in Home Assistant easy to discover and set up and disable #53 as the sibling on the direction axis. Cross-link @jlpouffier's comment above as the sibling on the tooling axis.
• Fill in the link to the vision document, or strip the reference if it is not ready yet.
• Acknowledge all three architectural questions in Risks. Right now the surface question is implicit in the toggle-discoverability bullet, and the direction and tooling questions are not surfaced at all.

Open to being wrong on any of these if there is a path that works that I am not seeing.

../Frenck

                       

Blogging my personal ramblings at frenck.dev

[Hedda]

Towards the broader direction, could the new Open Floor Protocol (OFP) that the Open Voice Interoperability Initiative is now promoting potentially align and be a strategic fit for extending Assist in the long-run?

• https://voiceinteroperability.ai/

As I understand it is specifically designed so different assistants can discover each other, hand off tasks, and collaborate. The initiative basically describes OFP as enabling assistants to locate other assistants with specific skills and delegate requests between systems.

• https://github.com/orgs/home-assistant/discussions/3832

OFP is made for interoperability and delegation between assistants/services, which could become relevant if and when extending Assist will evolve further toward broader workflows with multi-agent coordination. It may be worth considering whether future Assist architecture and/or HA Cloud's own offering could benefit from compatibility with OFP concepts rather than creating entirely custom assistant-to-assistant orchestration.

At least it would be one possible way forward to introduce a standard path and avoid custom assistant-to-assistant communications.

Using their standard would also connect the Open Home Foundation the with other broader open assistant interoperability efforts.

[nielsrowinbik]

Replying to both @jlpouffier and @frenck at once because the points overlap.

On the vision document

Fair, the TODO link is a gap. I've drafted a document in which I make a decision and build the case for it, but I haven't managed to finish it yet. I intended to do it much sooner after the roadmapping session, but life got in the way. I will pick this back up and finish it by the first half of next week, update the initiative above (and any others on the same topic), and ping you.

Having this document available will also clear up the confusion caused between this proposal and #53. These appear to be making different assumptions on how to surface AI features. They don't, but that's an issue primarily with the wording of #53. I will address this in a reply to your comment there @frenck.

On the tooling-layer call

@jlpouffier is right that this needs to be resolved before the opportunity is approved, and @frenck is right that the ha-mcp citation in Community signals actually points at the opposite pattern from the one the proposal assumes. I shouldn't have left that implicit. I've added an explicit item to risks and open questions, and we should resolve it separately before this moves forward.

On using Assist as the UX surface

@frenck writes that "users will expect voice-control behavior from the management mode, and management-mode behavior from voice control." That assumes Assist is fundamentally a voice product. It may have been marketed as such, but it doesn't have to remain like that. Assist, within the context of the Home Assistant UI, is a text input with optional voice. That's the same shape as ChatGPT, Claude, and every other AI chat interface.

Even if Assist is currently architecturally a voice product, it makes more sense to brand the proposed extension as the next evolution of Assist rather than introducing an entirely new concept next to it: splitting control and management into two separate surfaces would mean two chat interfaces, both accepting text and voice, with no meaningful distinction in how the user reaches them. That's the worse UX outcome in my opinion. I believe evolving what Assist means is the right call here.

On the dialog being "too narrow": the scope explicitly excludes layout changes for this opportunity, but not permanently. Starting with documentation retrieval in the existing dialog is a deliberate choice to keep the scope of this opportunity limited, not an architectural commitment to keeping it small forever. In fact, an opportunity already exists to address this problem: #158. I'll make this link explicit in the opportunity description.

[marcinbauer-ohf]

To the discussion about reusing the assist dialog, I’d like to add that our primary argument for adopting this approach is based on the industry’s UX pattern for interacting with chat or text. In this pattern, there’s a single place for all types of text input from the user, regardless of the output. This design minimizes cognitive load on users even before they decide whether they want to control their home or create something. By asking users to click to access a chat interface located elsewhere (but visually similar) with a different name, just to change their desired action, we introduce unnecessary friction.

The common touchpoint for users interacting with text is a text field. Looking at the history of development of LLM products like ChatGPT, Claude, and others, we can observe that they initially started with a single text field capable of capturing user input and generating a limited text output. Over time, their output capabilities have expanded, and most LLM products now share the same input mechanism. However they offer different modes of operation (as seen with Claude) or allow users to invoke various tools to pinpoint the desired output. I believe we are at the same point with the "Assist" dialog, and we can certainly re-use it and expand it. That said there are design decisions that can make the separation of the "modes" more apparent.