Skip to main content
Now on the Comfy Registry. Install ComfyUI Agent Panel (comfyui-agent-panel) from ComfyUI-Manager, or install from git for the latest build (see Setup below).
comfyui-mcp-panel puts an autonomous agent in ComfyUI’s sidebar. Ask it for an image, a workflow, or a change — it works against your ComfyUI and replies right there. Pick a provider — Claude, ChatGPT, Gemini, or Ollama (local) — and the matching agent runs in the background: subscriptions with no API key, local models with no account at all (and the Ollama backend reaches any hosted OpenAI-compatible endpoint too). See Backends for the matrix and the LLM Arena for how each model class actually performs.
No API keys, no per-token billing on the subscription providers — and fully free/offline on local models. The agent authenticates with your on-disk login (or, for Ollama, just talks to the local daemon). One orchestrator on one loopback bridge port (ws://127.0.0.1:9199) serves every provider; each panel tab picks its provider in the handshake. The bridge is loopback-only, and the panel executes a fixed allowlist of graph commands (no arbitrary JavaScript).
New here: Backends / providers explains the picker, the provider-neutral AgentBackend port, and the capability matrix (what’s identical across Claude and ChatGPT, and the few things that differ).

​
Setup

  1. Install the pack — search comfyui-agent-panel in ComfyUI-Manager, or from git:
    Pick Latest, not Nightly, in Manager’s version dropdown. Despite the name, Manager’s Nightly is not built nightly: it clones the repo once at install time and then never tracks the branch again. It freezes you on whatever commit happened to be main that day, while Latest keeps pace with every release — so Nightly is routinely older than Latest, and silently drifts further behind the longer you leave it. It reports no update available, because from its point of view there isn’t one.To check where you actually are, compare the SHA under Node Pack Info → Version against the repo’s commit history. To escape, either select Latest (x.y.z) in that same dropdown, or — if you want to stay on git — git pull in custom_nodes/comfyui-mcp-panel, which fast-forwards cleanly.
  2. Sign in to the provider you want once, so the background agent can use your subscription:
  3. Start the orchestrator on your machine and leave it running — see Starting the panel orchestrator:
  4. Restart ComfyUI, open the Agent tab, pick a provider (Claude / ChatGPT chips), and click Connect. The panel links to the orchestrator’s loopback bridge. Type a request and the agent responds. Disconnect unlinks it; the orchestrator keeps running until you stop it.
No claude mcp add, no API key. The panel is a pure-frontend extension and cannot spawn the orchestrator itself, so it is always a process you start — the panel auto-connects to a bridge that is already running. Prerequisites are Node.js/npx on your PATH and the provider login above. Change the bridge port with COMFYUI_MCP_BRIDGE_PORT.
Driving a remote ComfyUI (a cloud GPU pod, another box on your LAN)? The setup above still applies, but run the orchestrator on your own machine, not the remote box: npx -y comfyui-mcp@latest connect <remote-url> handles the rest, including a secure tunnel back to the pod’s HTTPS page. Then click Connect in the panel. See Cloud deployment for the full walkthrough.
Provider onboarding. At Connect the panel checks each provider’s readiness (its CLI on your PATH + a login on disk; the macOS Keychain is handled). An onboarding card appears only when neither provider is signed in. If your saved pick isn’t usable the panel auto-switches to a ready provider (your saved preference is kept), and a not-ready provider’s row offers a “set up” action that seeds the one-time claude / codex login step. See Backends → readiness & onboarding.

​
What it can do

The agent (Claude or ChatGPT) loads comfyui-mcp’s model skills (IDEOGRAM, WAN, LTX, Qwen, and more), so it already knows the models you run — Claude natively, and ChatGPT via the same knowledge exposed as MCP tools (knowledge parity). It can generate images, video and audio, inspect and manage your ComfyUI, and reason about your setup — then reply in the panel chat. It can also load a whole workflow or installer pack in one shot (panel_load_workflow pack:<name>) and is cost-aware: bundled packs are local-GPU / free, and for an ad-hoc graph the agent checks the runtime (list_packs with action:"check_runtime") and asks before spending paid API credits. See Skills, Packs & Runtime Cost.

​
Driving the live graph

The autonomous agent works your live ComfyUI through a fixed allowlist of panel_* commands — no arbitrary JavaScript. Every graph mutation goes through LiteGraph’s change tracking, so each is undoable with Ctrl+Z. The same panel_* surface is exposed identically to both backends (in-process for Claude, over a loopback HTTP MCP for ChatGPT/Codex), so parity is automatic.

​
Read

​
Edit the graph (undoable)

​
Subgraphs

​
Spatial layout

The agent can see node geometry — panel_query_graph detail rows return each node’s pos/size, plus subgraph input/output rails, groups, and per-node color/collapsed — and arrange the canvas with a matching set of writes, then screenshot the result to judge its own layout. A workflow-layout skill ties these into a dependency-layered, overlap-free auto-layout whose headline rule is always leave inputs and outputs exposed so you can jump straight in.

​
Workflow tabs

​
Load a workflow in one shot

​
Knowledge & cost awareness

The agent discovers bundled expertise and checks runtime cost before spending credits (same tools on both backends — see Skills, Packs & Runtime Cost):

​
Run & view

​
Custom nodes (built-in ComfyUI Manager)

​
MCP & session

​
Working with the user

Every tool accepts an optional tab_id — each browser tab holds its own connection, and routing defaults to the only tab, or the tab the user last typed in.

​
Rewind & rollback

Past messages aren’t frozen. Hover any message and the ✎ edit button opens a rollback modal: roll back the code (revert the graph to that turn’s snapshot), the conversation (fork the session back to that point), or both, then resend an edited message from there. Graph reverts use per-turn snapshots, so undoing a turn restores exactly the graph it started from. Two shortcuts cover the common cases:
  • /revert — undo the graph edits from the last turn.
  • Double-Esc — a quick last-turn rewind: revert the graph and recall the message into the composer to edit and resend.
Code (graph) rollback works on both providers — it lives in the orchestrator via per-turn snapshots. Conversation rollback (forking the chat back to a past turn) is currently Claude-only; the ChatGPT/Codex backend resumes whole threads only, so the panel gates that scope off for it. See the capability matrix.

​
Pending-message tray

Type while the agent is busy and your message doesn’t get lost in the chat — it waits in a fixed Pending tray docked above the downloads tray, out of the chat flow. Each pending message has edit, send-now, and delete buttons, and a drag handle (≡, on the left) to reorder how the agent flushes them. Send-now interrupts the current turn to steer it immediately. When a pending message is dequeued it materializes at the bottom of the chat, so the transcript reads in the exact order the agent (Claude or ChatGPT) processed it.

​
Destructive-op confirmation

Irreversible actions ask first. panel_clear (wipe every node) and panel_restart_comfyui pop a yes / no card and only act on yes — so the agent can’t silently nuke your graph or bounce ComfyUI.

​
Reconnect durability

A wedged orchestrator no longer strands the panel. If a previous orchestrator is still holding the bridge port, clicking Connect reclaims that zombie instead of failing — the panel reconnects rather than leaving you stuck.

​
Composer attachments

Attach, drag-and-drop, or paste into the composer. Alongside images, the composer now accepts video, workflow .json, and text files, so you can hand the agent a reference clip, a workflow to adapt, or a notes file directly.

​
Rich media in agent replies

When a run’s media is fed back to the agent, the output isn’t just an image block — it carries metadata the agent can reason about: each output’s path (subfolder-relative), file size, pixel dimensions, and asset-set grouping (“output K of N from this run” with its sibling filenames, or “single output”), plus render duration and completion time. Video storyboards add the format and the real frame count / fps when the payload carries them. So the agent names the actual saved result accurately and can talk about size, dimensions, and how many files a run produced.

​
Copy & wrap code blocks

Rendered fenced code blocks get a hover Copy button and a persisted global line-wrap toggle (off by default — long lines scroll horizontally until you turn it on). Inline code gets its own Copy. Both are styled to match the panel.

​
Render-stall warning

The orchestrator runs a passive watchdog over your ComfyUI queue: a render whose node/progress stops advancing is flagged as stalled and the agent is told (so it stops blindly stacking jobs behind a wedged one). The threshold is the Render stall warning (seconds) setting under Settings → Comfy MCP Agent → General (default 180s, range 15–3600). It’s sent on Connect and pushed live — changing it applies without a reconnect. See Configuration → COMFYUI_MCP_STALL_S.

​
Background-tab reliability

Streamed replies now render even when the ComfyUI tab is in the background. Previously the reply typewriter ran on requestAnimationFrame, which the browser pauses in a hidden tab — so switching away during a long multi-stage run left the bubble empty with a stuck streaming cursor, looking like the agent was “stuck thinking” even though the turn had finished. The reply now finalizes synchronously when the tab is hidden, and a visibilitychange handler flushes any pending reply on hide and resumes the typewriter on return. Foreground animation is unchanged.

​
RunPod cloud control

A toolbar host pill reads 🟢 Local · your rig or **🔵 RunPod · <pod> · GPU · /hr∗∗andopensa∗∗RunPodcontrolpanel∗∗:alivestatuscard(GPU/VRAM/uptime//hr** and opens a **RunPod control panel**: a live status card (GPU / VRAM / uptime / ·hr / ComfyUI URL / idle auto-stop countdown), a pod dropdown that lists your pods by name, Connect / Start / Stop / Use Local, and a confirm-armed Deploy. Set RUNPOD_API_KEY once in the API-Keys card and you can deploy, monitor, switch local⇄pod, and stop a cloud GPU without touching the RunPod console — the host pill always tells you where the next render runs. See Cloud deployment and the blog Run ComfyUI on a rented cloud GPU.

​
CivitAI browser

The Civitai toolbar button opens a full CivitAI browser — images, videos, checkpoints, LoRAs and workflows with search, filters, and a fullscreen viewer. Pick a result to share it with the agent, download it to your machine, or save an embedded workflow to the canvas. The story: CivitAI inside ComfyUI.

​
See also