Using ComfyUI with OpenClaw

Prerequisites

• ✓A running ComfyUI instance reachable at a URL like http://127.0.0.1:8188 or a Comfy Cloud account.
• ✓Existing ComfyUI workflow JSON files for image, video, and/or music generation (for example ./workflows/flux-api.json).
• ✓An OpenClaw project where you can edit the models.providers and agents.defaults configuration.
• ✓A COMFY_API_KEY or COMFY_CLOUD_API_KEY if you plan to use Comfy Cloud instead of a local ComfyUI server.

Steps

1. 1

   Configure the comfy provider for local ComfyUI

   Start by registering the comfy provider in your OpenClaw models config so it can talk to your local ComfyUI server. You point each capability (image, video, music) at a specific workflow JSON and define which nodes handle prompts and outputs.

   jsonCopy

   {
     models: {
       providers: {
         comfy: {
           mode: "local",
           baseUrl: "http://127.0.0.1:8188",
           image: {
             workflowPath: "./workflows/flux-api.json",
             promptNodeId: "6",
             outputNodeId: "9",
           },
           video: {
             workflowPath: "./workflows/video-api.json",
             promptNodeId: "12",
             outputNodeId: "21",
           },
           music: {
             workflowPath: "./workflows/music-api.json",
             promptNodeId: "3",
             outputNodeId: "18",
           },
         },
       },
     },
   }

2. 2

   Set comfy as the default image and video generation model

   Next, tell your agents to use the comfy/workflow model when they call the shared image and video generation tools. This keeps your prompts simple while routing all image and video work through your ComfyUI workflows.

   jsonCopy

   {
     agents: {
       defaults: {
         imageGenerationModel: {
           primary: "comfy/workflow",
         },
       },
     },
   }
   

3. 3

   Configure reference-image editing for ComfyUI image workflows

   If you want image editing with an uploaded reference image, add input image node settings to the comfy image section. This tells OpenClaw which node in your ComfyUI graph receives the uploaded image and how to name that input.

   jsonCopy

   {
     models: {
       providers: {
         comfy: {
           image: {
             workflowPath: "./workflows/edit-api.json",
             promptNodeId: "6",
             inputImageNodeId: "7",
             inputImageInputName: "image",
             outputNodeId: "9",
           },
         },
       },
     },
   }
   

4. 4

   Enable comfy for video generation workflows

   To use ComfyUI for text-to-video or image-to-video, set the default videoGenerationModel to comfy/workflow. OpenClaw then routes video_generate calls through your configured ComfyUI video graph.

   jsonCopy

   {
     agents: {
       defaults: {
         videoGenerationModel: {
           primary: "comfy/workflow",
         },
       },
     },
   }
   

5. 5

   Wire up music or audio workflows with the music_generate tool

   For music or audio generation, you use the shared music_generate tool backed by your comfy music workflow. Once the music section is configured, you can trigger audio generation directly from prompts.

   textCopy

   /tool music_generate prompt="Warm ambient synth loop with soft tape texture"

6. 6

   Switch comfy to Comfy Cloud mode with API keys

   When you want OpenClaw to call Comfy Cloud instead of a local server, change the comfy mode to cloud and provide an API key. apiKey config.

   textCopy

   Use `mode: "cloud"` plus one of:
   *   `COMFY_API_KEY`
   *   `COMFY_CLOUD_API_KEY`
   *   `models.providers.comfy.apiKey`
   

7. 7

   Run live tests for the comfy plugin

   After configuration, run the bundled live tests to verify that OpenClaw can reach your ComfyUI workflows. The test suite only exercises image, video, or music cases that you have configured, which helps you catch miswired node IDs early.

   bashCopy

   OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts

Configuration

Troubleshooting

{
  models: {
    providers: {
      comfy: {
        image: {
          workflowPath: "./workflows/flux-api.json",
          promptNodeId: "6",
          outputNodeId: "9",
        },
      },
    },
  },
}

// Add pollIntervalMs and timeoutMs under models.providers.comfy.video

{
  models: {
    providers: {
      comfy: {
        music: {
          workflowPath: "./workflows/music-api.json",
          promptNodeId: "3",
          outputNodeId: "18",
        },
      },
    },
  },
}

OPENCLAW_LIVE_TEST=1 COMFY_LIVE_TEST=1 pnpm test:live -- extensions/comfy/comfy.live.test.ts

Frequently asked questions