SDK for ComfyUI
npm install @saintno/comfyui-sdk

!CI

A robust and meticulously crafted TypeScript SDK ๐ for seamless interaction with the ComfyUI API. This SDK significantly simplifies the complexities of building, executing, and managing ComfyUI workflows, all while providing real-time updates and supporting multiple instances. ๐ผ๏ธ
- ๐ Key Features ๐
- ๐ฆ Installation ๐ฆ
- ๐ Getting Started ๐
- ๐ฌ Basic Usage
- ๐ Managing Multiple Instances with ComfyPool
- ๐ Authentication
- ๐ API Reference ๐
- ComfyApi
- CallWrapper
- PromptBuilder
- ComfyPool
- ๐๏ธ Enums
- ๐๏ธ Types
- ๐งฉ Features
- ๐ Examples
- ๐ค Contributing
- ๐ License
- ๐ TypeScript Powered: Enjoy a fully typed codebase, ensuring enhanced development, maintainability, and type safety. ๐ก๏ธ
- ๐๏ธ Workflow Builder: Construct and manipulate intricate ComfyUI workflows effortlessly using a fluent, intuitive builder pattern. ๐งฉ
- ๐คน Multi-Instance Management: Handle a pool of ComfyUI instances with ease, employing flexible queueing strategies for optimal resource utilization. ๐
- โก Real-Time Updates: Subscribe to WebSocket events for live progress tracking, image previews, and error notifications. ๐
- ๐ง Custom WebSocket Support: Supply your own WebSocket implementation for greater flexibility in different environments, with robust reconnection handling and fallback options. ๐
- ๐ Authentication Flexibility: Supports Basic Auth, Bearer Token, and Custom Authentication Headers, catering to diverse security requirements. ๐
- ๐ Extension Support: Seamlessly integrate with ComfyUI Manager and leverage system monitoring through the ComfyUI-Crystools extension. ๐ ๏ธ
- ๐ Flexible Node Bypassing: Strategically bypass specific nodes in your workflows during generation, enabling advanced customization. โญ๏ธ
- ๐ Comprehensive Examples: Includes practical examples for Text-to-Image (T2I), Image-to-Image (I2I), and complex multi-node workflows. ๐
- ๐จ Robust Error Handling: Provides detailed error messages to facilitate debugging and graceful handling of API failures. ๐
- ๐ Automatic Changelog: Automatically generates a changelog with each release, utilizing auto-changelog for transparent version tracking. ๐
``bash`
bun add @saintno/comfyui-sdk
or
`bash`
npm i @saintno/comfyui-sdk
Here's a simplified example to quickly get you started:
`typescript
import { ComfyApi, CallWrapper, PromptBuilder, TSamplerName, TSchedulerName, seed } from "@saintno/comfyui-sdk";
import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";
const api = new ComfyApi("http://localhost:8189").init();
const workflow = new PromptBuilder(
ExampleTxt2ImgWorkflow,
["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],
["images"]
)
.setInputNode("checkpoint", "4.inputs.ckpt_name")
.setInputNode("seed", "3.inputs.seed")
.setInputNode("batch", "5.inputs.batch_size")
.setInputNode("negative", "7.inputs.text")
.setInputNode("positive", "6.inputs.text")
.setInputNode("cfg", "3.inputs.cfg")
.setInputNode("sampler", "3.inputs.sampler_name")
.setInputNode("sheduler", "3.inputs.scheduler")
.setInputNode("step", "3.inputs.steps")
.setInputNode("width", "5.inputs.width")
.setInputNode("height", "5.inputs.height")
.setOutputNode("images", "9")
.input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)
.input("seed", seed())
.input("step", 6)
.input("cfg", 1)
.input
.input
.input("width", 1024)
.input("height", 1024)
.input("batch", 1)
.input("positive", "A picture of cute dog on the street");
new CallWrapper(api, workflow)
.onFinished((data) => console.log(data.images?.images.map((img: any) => api.getPathImage(img))))
.run();
`
#### ๐ Breakdown
- Import essential components from the SDK.
- Create and initialize the ComfyApi instance.PromptBuilder
- Use to define the workflow structure and set input nodes.CallWrapper
- Set specific input values, including the checkpoint path, seed, and prompt.
- Execute the workflow and log the generated image URLs using the .
`typescript
import {
ComfyApi,
CallWrapper,
ComfyPool,
EQueueMode,
PromptBuilder,
seed,
TSamplerName,
TSchedulerName
} from "@saintno/comfyui-sdk";
import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";
const ApiPool = new ComfyPool(
[new ComfyApi("http://localhost:8188"), new ComfyApi("http://localhost:8189")],
EQueueMode.PICK_ZERO
)
.on("init", () => console.log("Pool in initializing"))
.on("add_job", (ev) => console.log("Job added at index", ev.detail.jobIdx, "weight:", ev.detail.weight))
.on("added", (ev) => console.log("Client added", ev.detail.clientIdx));
const generateFn = async (api: ComfyApi, clientIdx?: number) => {
const workflow = new PromptBuilder(
ExampleTxt2ImgWorkflow,
["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],
["images"]
)
.setInputNode("checkpoint", "4.inputs.ckpt_name")
.setInputNode("seed", "3.inputs.seed")
.setInputNode("batch", "5.inputs.batch_size")
.setInputNode("negative", "7.inputs.text")
.setInputNode("positive", "6.inputs.text")
.setInputNode("step", "3.inputs.steps")
.setInputNode("width", "5.inputs.width")
.setInputNode("height", "5.inputs.height")
.setInputNode("cfg", "3.inputs.cfg")
.setInputNode("sampler", "3.inputs.sampler_name")
.setInputNode("scheduler", "3.inputs.scheduler")
.setOutputNode("images", "9")
.input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)
.input("seed", seed())
.input("step", 6)
.input("width", 512)
.input("height", 512)
.input("batch", 2)
.input("cfg", 1)
.input
.input
.input("positive", "A close up picture of cute Cat")
.input("negative", "text, blurry, bad picture, nsfw");
return new Promise
new CallWrapper(api, workflow)
.onFinished((data) => {
const url = data.images?.images.map((img: any) => api.getPathImage(img));
resolve(url as string[]);
})
.run();
});
};
const jobA = ApiPool.batch(Array(5).fill(generateFn), 10).then((res) => {
console.log("Batch A done");
return res.flat();
});
const jobB = ApiPool.batch(Array(5).fill(generateFn), 0).then((res) => {
console.log("Batch B done");
return res.flat();
});
console.log(await Promise.all([jobA, jobB]).then((res) => res.flat()));
`
#### ๐ Breakdown
- Create a ComfyPool with multiple ComfyApi instances.generateFn
- Set up event listeners for pool initialization, job additions, and client connections.
- Define an async function () that creates a workflow, sets its inputs, and executes it with a CallWrapper.ApiPool.batch
- Use to run multiple jobs and wait for all batches to complete.
`typescript
import { ComfyApi, BasicCredentials, BearerTokenCredentials, CustomCredentials } from "@saintno/comfyui-sdk";
// Basic Authentication
const basicAuth = new ComfyApi("http://localhost:8189", "node-id", {
credentials: { type: "basic", username: "username", password: "password" } as BasicCredentials
}).init();
// Bearer Token Authentication
const bearerAuth = new ComfyApi("http://localhost:8189", "node-id", {
credentials: { type: "bearer_token", token: "your_bearer_token" } as BearerTokenCredentials
}).init();
// Custom Header Authentication
const customAuth = new ComfyApi("http://localhost:8189", "node-id", {
credentials: { type: "custom", headers: { "X-Custom-Header": "your_custom_header" } } as CustomCredentials
}).init();
`
#### ๐ Breakdown
- Import the necessary types from the SDK.
- Create ComfyApi instances using the corresponding credential types: BasicCredentials, BearerTokenCredentials, and CustomCredentials..
`typescript
import { ComfyApi, WebSocketInterface } from "@saintno/comfyui-sdk";
import CustomWebSocket from "your-custom-websocket-library";
// Create a ComfyApi instance with a custom WebSocket implementation
const api = new ComfyApi("http://localhost:8189", "node-id", {
credentials: { type: "basic", username: "username", password: "password" },
customWebSocketImpl: CustomWebSocket as WebSocketInterface
}).init();
`
#### ๐ Breakdown
- Import the necessary types and your custom WebSocket implementation.
- Pass the custom WebSocket implementation to the ComfyApi constructor using the customWebSocketImpl option.
- The SDK will use your implementation instead of the default one, allowing for greater flexibility in environments where the standard WebSocket implementation might not be available or suitable.
#### ๐ Benefits
- Environment Flexibility: Run in environments where the standard WebSocket might not be available or optimal.
- Enhanced Stability: The SDK includes robust reconnection logic with exponential backoff and jitter to handle connection issues gracefully.
- Fallback Mechanism: Automatically falls back to HTTP polling if WebSocket connections fail, ensuring your application remains functional.
- Custom Protocol Support: Implement custom protocols or security features through your WebSocket implementation.
#### ๐๏ธ Constructor
`typescript`
constructor(host: string, clientId: string, opts?: { forceWs?: boolean, wsTimeout?: number, credentials?: BasicCredentials | BearerTokenCredentials | CustomCredentials; })
- host: The base URL of your ComfyUI server.clientId
- : A unique ID for WebSocket communication (optional). Defaults to a generated ID.opts
- : Optional settings:forceWs
- : Boolean to force WebSocket usage.wsTimeout
- : Timeout for WebSocket connections (milliseconds).credentials
- : Optional authentication credentials.
#### โ๏ธ Methods
- init(maxTries?: number, delayTime?: number): Initializes the client and establishes connection.on
- : Attach an event listener.off
- : Detach an event listener.removeAllListeners()
- : Detach all event listeners.fetchApi(route: string, options?: FetchOptions)
- : Fetch data from the API endpoint.pollStatus(timeout?: number)
- : Polls the ComfyUI server status.queuePrompt(number: number | null, workflow: object)
- : Queues a prompt for processing.appendPrompt(workflow: object)
- : Adds a prompt to the workflow queue.getQueue()
- : Retrieves the current state of the queue.getHistories(maxItems?: number)
- : Retrieves the prompt execution history.getHistory(promptId: string)
- : Retrieves a specific history entry by ID.getSystemStats()
- : Retrieves system and device statistics.getExtensions()
- : Retrieves a list of installed extensions.getEmbeddings()
- : Retrieves a list of available embeddings.getCheckpoints()
- : Retrieves a list of available checkpoints.getLoras()
- : Retrieves a list of available Loras.getSamplerInfo()
- : Retrieves sampler and scheduler information.getNodeDefs(nodeName?: string)
- : Retrieves node object definitions.getUserConfig()
- : Get user configuration data.createUser(username: string)
- : Create new user.getSettings()
- : Get all setting values for the current user.getSetting(id: string)
- : Get a specific setting for the current user.storeSettings(settings: Record
- : Store setting for the current user.storeSetting(id: string, value: unknown)
- : Store a specific setting for the current user.uploadImage(file: Buffer | Blob, fileName: string, config?: { override?: boolean; subfolder?: string })
- : Uploads an image file.uploadMask(file: Buffer | Blob, originalRef: ImageInfo)
- : Uploads a mask file.freeMemory(unloadModels: boolean, freeMemory: boolean)
- : Frees memory by unloading models.getPathImage(imageInfo: ImageInfo)
- : Returns the path to an image.getImage(imageInfo: ImageInfo)
- : Returns the blob data of image.getUserData(file: string)
- : Get a user data file.storeUserData(file: string, data: unknown, options?: RequestInit & { overwrite?: boolean, stringify?: boolean, throwOnError?: boolean })
- : Store a user data file.deleteUserData(file: string)
- : Delete a user data file.moveUserData(source: string, dest: string, options?: RequestInit & { overwrite?: boolean })
- : Move a user data file.listUserData(dir: string, recurse?: boolean, split?: boolean)
- : List a user data file.interrupt()
- : Interrupts the execution of the running prompt.reconnectWs(opened?: boolean)
- : Reconnects to the WebSocket server.
#### ๐๏ธ Constructor
`typescript`
constructor(client: ComfyApi, workflow: PromptBuilder)
- client: An instance of the ComfyApi client.workflow
- : An instance of PromptBuilder defining the workflow.
#### โ๏ธ Methods
- onPreview(fn: (ev: Blob, promptId?: string) => void): Set callback for preview events.onPending(fn: (promptId?: string) => void)
- : Set callback when job is queued.onStart(fn: (promptId?: string) => void)
- : Set callback when the job is started.onOutput(fn: (key: keyof PromptBuilder["mapOutputKeys"], data: any, promptId?: string) => void)
- : Sets a callback for when an output node is executed.onFinished(fn: (data: Record
- : Set callback when the job is finished.onFailed(fn: (err: Error, promptId?: string) => void)
- : Set callback when the job failed.onProgress(fn: (info: NodeProgress, promptId?: string) => void)
- : Set callback for progress updates.run()
- : Executes the workflow.
#### ๐๏ธ Constructor
`typescript`
constructor(prompt: T, inputKeys: I[], outputKeys: O[])
- prompt: The initial workflow data object.inputKeys
- : An array of input node keys.outputKeys
- : An array of output node keys.
#### โ๏ธ Methods
- clone(): Creates a new PromptBuilder instance with the same configuration.bypass(node: keyof T | (keyof T)[]): PromptBuilder
- : Marks node(s) to be bypassed at generation.reinstate(node: keyof T | (keyof T)[]): PromptBuilder
- : Unmarks node(s) from bypass at generation.setInputNode(input: I, key: DeepKeys
- : Sets input node path for a key.setRawInputNode(input: I, key: string | string[])
- : Sets raw input node path for a key.appendInputNode(input: I, key: DeepKeys
- : Appends a node to the input node path.appendRawInputNode(input: I, key: string | string[])
- : Appends a node to the raw input node path.setOutputNode(output: O, key: DeepKeys
- : Sets output node path for a key.setRawOutputNode(output: O, key: string)
- : Sets raw output node path for a key.input
- : Sets an input value.inputRaw
- : Sets a raw input value with dynamic key.get workflow
- : Retrieves the workflow object.get caller
- : Retrieves current PromptBuilder object.
#### ๐๏ธ Constructor
`typescript`
constructor(clients: ComfyApi[], mode: EQueueMode = EQueueMode.PICK_ZERO)
- clients: Array of ComfyApi instances.mode
- : The queue mode using EQueueMode enum values.
#### โ๏ธ Methods
- on: Attach an event listener.off
- : Detach an event listener.addClient(client: ComfyApi)
- : Adds a new client to the pool.removeClient(client: ComfyApi)
- : Removes a client from the pool.removeClientByIndex(index: number)
- : Removes a client by index.changeMode(mode: EQueueMode)
- : Changes the queue mode.pick(idx?: number)
- : Picks a client by index.pickById(id: string)
- : Picks a client by ID.run
- : Run a job with priority on an available client.batch
- : Run multiple jobs concurrently.
- EQueueMode:PICK_ZERO
- : Selects the client with zero remaining queue.PICK_LOWEST
- : Selects the client with the lowest remaining queue.PICK_ROUTINE
- : Selects clients in a round-robin manner.
- OSType:POSIX
- : For Unix-like systems.NT
- : For Windows systems.JAVA
- : For Java virtual machine.TSamplerName
- : A union type of all available sampler names.TSchedulerName
- : A union type of all available scheduler names.
- ManagerFeature: Provides methods to manage ComfyUI Manager Extension.
`typescript
const api = new ComfyApi("http://localhost:8189").init();
await api.waitForReady();
if (api.ext.manager.isSupported) {
await api.ext.manager.getExtensionList().then(console.log);
// Check api.ext.manager for more methods
}
`
- MonitoringFeature: Provides methods to monitor system resources using ComfyUI-Crystools Extension.
`typescript
const api = new ComfyApi("http://localhost:8189").init();
await api.waitForReady();
if (api.ext.monitor.isSupported) {
// For subscribing to system monitor events
api.ext.monitor.on("system_monitor", (ev) => {
console.log(ev.detail);
});
// For getting current monitor data
console.log(api.ext.monitor.monitorData);
}
`
> Note: Features require respective extensions (ComfyUI-Manager and ComfyUI-Crystools) to be installed.
The examples directory contains practical demonstrations of SDK usage:
- example-i2i.ts: Demonstrates image-to-image generation.example-pool.ts
- : Demonstrates how to manage multiple ComfyUI instances using ComfyPool.example-pool-basic-auth.ts
- : Demonstrates how to use ComfyPool with HTTP Basic Authentication.example-t2i.ts
- : Demonstrates text-to-image generation.example-t2i-upscaled.ts
- : Demonstrates text-to-image generation with upscaling.example-img2img-workflow.json
- : Example workflow for image-to-image.example-txt2img-workflow.json
- : Example workflow for text-to-image.example-txt2img-upscaled-workflow.json`: Example workflow for text-to-image with upscaling.
-
Contributions are always welcome! Feel free to submit pull requests or create issues for bug reports and feature enhancements. ๐
This project is licensed under the MIT License - see the LICENSE file for more details. ๐