✨ ComfyUI SDK ✨

!CI

A robust and meticulously crafted TypeScript SDK 🚀 for seamless interaction with the ComfyUI API. This SDK significantly simplifies the complexities of building, executing, and managing ComfyUI workflows, all while providing real-time updates and supporting multiple instances. 🖼️

🧭 Table of Contents

- 🌟 Key Features 🌟
- 📦 Installation 📦
- 🚀 Getting Started 🚀
- 🎬 Basic Usage
- 🔄 Managing Multiple Instances with ComfyPool
- 🔑 Authentication
- 📚 API Reference 📚
- ComfyApi
- CallWrapper
- PromptBuilder
- ComfyPool
- 🗂️ Enums
- 🗄️ Types
- 🧩 Features
- 📂 Examples
- 🤝 Contributing
- 📜 License

🌟 Key Features 🌟

- 💎 TypeScript Powered: Enjoy a fully typed codebase, ensuring enhanced development, maintainability, and type safety. 🛡️
- 🏗️ Workflow Builder: Construct and manipulate intricate ComfyUI workflows effortlessly using a fluent, intuitive builder pattern. 🧩
- 🤹 Multi-Instance Management: Handle a pool of ComfyUI instances with ease, employing flexible queueing strategies for optimal resource utilization. 🌐
- ⚡ Real-Time Updates: Subscribe to WebSocket events for live progress tracking, image previews, and error notifications. 🔔
- 🔧 Custom WebSocket Support: Supply your own WebSocket implementation for greater flexibility in different environments, with robust reconnection handling and fallback options. 🔄
- 🔑 Authentication Flexibility: Supports Basic Auth, Bearer Token, and Custom Authentication Headers, catering to diverse security requirements. 🔒
- 🔌 Extension Support: Seamlessly integrate with ComfyUI Manager and leverage system monitoring through the ComfyUI-Crystools extension. 🛠️
- 🔀 Flexible Node Bypassing: Strategically bypass specific nodes in your workflows during generation, enabling advanced customization. ⏭️
- 📚 Comprehensive Examples: Includes practical examples for Text-to-Image (T2I), Image-to-Image (I2I), and complex multi-node workflows. 📝
- 🚨 Robust Error Handling: Provides detailed error messages to facilitate debugging and graceful handling of API failures. 🐛
- 📝 Automatic Changelog: Automatically generates a changelog with each release, utilizing auto-changelog for transparent version tracking. 🔄

📦 Installation 📦

``bash
bun add @saintno/comfyui-sdk
`

or

`bash
npm i @saintno/comfyui-sdk
`

🚀 Getting Started 🚀

$3

Here's a simplified example to quickly get you started:

`typescript
import { ComfyApi, CallWrapper, PromptBuilder, TSamplerName, TSchedulerName, seed } from "@saintno/comfyui-sdk";
import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";

const api = new ComfyApi("http://localhost:8189").init();
const workflow = new PromptBuilder(
ExampleTxt2ImgWorkflow,
["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],
["images"]
)
.setInputNode("checkpoint", "4.inputs.ckpt_name")
.setInputNode("seed", "3.inputs.seed")
.setInputNode("batch", "5.inputs.batch_size")
.setInputNode("negative", "7.inputs.text")
.setInputNode("positive", "6.inputs.text")
.setInputNode("cfg", "3.inputs.cfg")
.setInputNode("sampler", "3.inputs.sampler_name")
.setInputNode("sheduler", "3.inputs.scheduler")
.setInputNode("step", "3.inputs.steps")
.setInputNode("width", "5.inputs.width")
.setInputNode("height", "5.inputs.height")
.setOutputNode("images", "9")
.input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)
.input("seed", seed())
.input("step", 6)
.input("cfg", 1)
.input("sampler", "dpmpp_2m_sde_gpu")
.input("sheduler", "sgm_uniform")
.input("width", 1024)
.input("height", 1024)
.input("batch", 1)
.input("positive", "A picture of cute dog on the street");

new CallWrapper(api, workflow)
.onFinished((data) => console.log(data.images?.images.map((img: any) => api.getPathImage(img))))
.run();
`

#### 🔍 Breakdown

- Import essential components from the SDK.
- Create and initialize the ComfyApi instance.
- Use PromptBuilder to define the workflow structure and set input nodes.
- Set specific input values, including the checkpoint path, seed, and prompt.
- Execute the workflow and log the generated image URLs using the CallWrapper.

$3

`typescript
import {
ComfyApi,
CallWrapper,
ComfyPool,
EQueueMode,
PromptBuilder,
seed,
TSamplerName,
TSchedulerName
} from "@saintno/comfyui-sdk";
import ExampleTxt2ImgWorkflow from "./example-txt2img-workflow.json";

const ApiPool = new ComfyPool(
[new ComfyApi("http://localhost:8188"), new ComfyApi("http://localhost:8189")],
EQueueMode.PICK_ZERO
)
.on("init", () => console.log("Pool in initializing"))
.on("add_job", (ev) => console.log("Job added at index", ev.detail.jobIdx, "weight:", ev.detail.weight))
.on("added", (ev) => console.log("Client added", ev.detail.clientIdx));

const generateFn = async (api: ComfyApi, clientIdx?: number) => {
const workflow = new PromptBuilder(
ExampleTxt2ImgWorkflow,
["positive", "negative", "checkpoint", "seed", "batch", "step", "cfg", "sampler", "sheduler", "width", "height"],
["images"]
)
.setInputNode("checkpoint", "4.inputs.ckpt_name")
.setInputNode("seed", "3.inputs.seed")
.setInputNode("batch", "5.inputs.batch_size")
.setInputNode("negative", "7.inputs.text")
.setInputNode("positive", "6.inputs.text")
.setInputNode("step", "3.inputs.steps")
.setInputNode("width", "5.inputs.width")
.setInputNode("height", "5.inputs.height")
.setInputNode("cfg", "3.inputs.cfg")
.setInputNode("sampler", "3.inputs.sampler_name")
.setInputNode("scheduler", "3.inputs.scheduler")
.setOutputNode("images", "9")
.input("checkpoint", "SDXL/realvisxlV40_v40LightningBakedvae.safetensors", api.osType)
.input("seed", seed())
.input("step", 6)
.input("width", 512)
.input("height", 512)
.input("batch", 2)
.input("cfg", 1)
.input("sampler", "dpmpp_2m_sde_gpu")
.input("scheduler", "sgm_uniform")
.input("positive", "A close up picture of cute Cat")
.input("negative", "text, blurry, bad picture, nsfw");

return new Promise((resolve) => {
new CallWrapper(api, workflow)
.onFinished((data) => {
const url = data.images?.images.map((img: any) => api.getPathImage(img));
resolve(url as string[]);
})
.run();
});
};

const jobA = ApiPool.batch(Array(5).fill(generateFn), 10).then((res) => {
console.log("Batch A done");
return res.flat();
});

const jobB = ApiPool.batch(Array(5).fill(generateFn), 0).then((res) => {
console.log("Batch B done");
return res.flat();
});

console.log(await Promise.all([jobA, jobB]).then((res) => res.flat()));
`

#### 🔍 Breakdown

- Create a ComfyPool with multiple ComfyApi instances.
- Set up event listeners for pool initialization, job additions, and client connections.
- Define an async function (generateFn) that creates a workflow, sets its inputs, and executes it with a CallWrapper.
- Use ApiPool.batch to run multiple jobs and wait for all batches to complete.

$3

`typescript
import { ComfyApi, BasicCredentials, BearerTokenCredentials, CustomCredentials } from "@saintno/comfyui-sdk";

// Basic Authentication
const basicAuth = new ComfyApi("http://localhost:8189", "node-id", {
credentials: { type: "basic", username: "username", password: "password" } as BasicCredentials
}).init();

// Bearer Token Authentication
const bearerAuth = new ComfyApi("http://localhost:8189", "node-id", {
credentials: { type: "bearer_token", token: "your_bearer_token" } as BearerTokenCredentials
}).init();

// Custom Header Authentication
const customAuth = new ComfyApi("http://localhost:8189", "node-id", {
credentials: { type: "custom", headers: { "X-Custom-Header": "your_custom_header" } } as CustomCredentials
}).init();
`

#### 🔍 Breakdown

- Import the necessary types from the SDK.
- Create ComfyApi instances using the corresponding credential types: BasicCredentials, BearerTokenCredentials, and CustomCredentials..

$3

`typescript
import { ComfyApi, WebSocketInterface } from "@saintno/comfyui-sdk";
import CustomWebSocket from "your-custom-websocket-library";

// Create a ComfyApi instance with a custom WebSocket implementation
const api = new ComfyApi("http://localhost:8189", "node-id", {
credentials: { type: "basic", username: "username", password: "password" },
customWebSocketImpl: CustomWebSocket as WebSocketInterface
}).init();
`

#### 🔍 Breakdown

- Import the necessary types and your custom WebSocket implementation.
- Pass the custom WebSocket implementation to the ComfyApi constructor using the customWebSocketImpl option.
- The SDK will use your implementation instead of the default one, allowing for greater flexibility in environments where the standard WebSocket implementation might not be available or suitable.

#### 📝 Benefits

- Environment Flexibility: Run in environments where the standard WebSocket might not be available or optimal.
- Enhanced Stability: The SDK includes robust reconnection logic with exponential backoff and jitter to handle connection issues gracefully.
- Fallback Mechanism: Automatically falls back to HTTP polling if WebSocket connections fail, ensuring your application remains functional.
- Custom Protocol Support: Implement custom protocols or security features through your WebSocket implementation.

📚 API Reference 📚

$3

#### 🏗️ Constructor

`typescript
constructor(host: string, clientId: string, opts?: { forceWs?: boolean, wsTimeout?: number, credentials?: BasicCredentials | BearerTokenCredentials | CustomCredentials; })
`

- host: The base URL of your ComfyUI server.
- clientId: A unique ID for WebSocket communication (optional). Defaults to a generated ID.
- opts: Optional settings:
- forceWs: Boolean to force WebSocket usage.
- wsTimeout: Timeout for WebSocket connections (milliseconds).
- credentials: Optional authentication credentials.

#### ⚙️ Methods

- init(maxTries?: number, delayTime?: number): Initializes the client and establishes connection.
- on(type: K, callback: (event: TComfyAPIEventMap[K]) => void, options?: AddEventListenerOptions | boolean): Attach an event listener.
- off(type: K, callback: (event: TComfyAPIEventMap[K]) => void, options?: EventListenerOptions | boolean): Detach an event listener.
- removeAllListeners(): Detach all event listeners.
- fetchApi(route: string, options?: FetchOptions): Fetch data from the API endpoint.
- pollStatus(timeout?: number): Polls the ComfyUI server status.
- queuePrompt(number: number | null, workflow: object): Queues a prompt for processing.
- appendPrompt(workflow: object): Adds a prompt to the workflow queue.
- getQueue(): Retrieves the current state of the queue.
- getHistories(maxItems?: number): Retrieves the prompt execution history.
- getHistory(promptId: string): Retrieves a specific history entry by ID.
- getSystemStats(): Retrieves system and device statistics.
- getExtensions(): Retrieves a list of installed extensions.
- getEmbeddings(): Retrieves a list of available embeddings.
- getCheckpoints(): Retrieves a list of available checkpoints.
- getLoras(): Retrieves a list of available Loras.
- getSamplerInfo(): Retrieves sampler and scheduler information.
- getNodeDefs(nodeName?: string): Retrieves node object definitions.
- getUserConfig(): Get user configuration data.
- createUser(username: string): Create new user.
- getSettings(): Get all setting values for the current user.
- getSetting(id: string): Get a specific setting for the current user.
- storeSettings(settings: Record): Store setting for the current user.
- storeSetting(id: string, value: unknown): Store a specific setting for the current user.
- uploadImage(file: Buffer | Blob, fileName: string, config?: { override?: boolean; subfolder?: string }): Uploads an image file.
- uploadMask(file: Buffer | Blob, originalRef: ImageInfo): Uploads a mask file.
- freeMemory(unloadModels: boolean, freeMemory: boolean): Frees memory by unloading models.
- getPathImage(imageInfo: ImageInfo): Returns the path to an image.
- getImage(imageInfo: ImageInfo): Returns the blob data of image.
- getUserData(file: string): Get a user data file.
- storeUserData(file: string, data: unknown, options?: RequestInit & { overwrite?: boolean, stringify?: boolean, throwOnError?: boolean }): Store a user data file.
- deleteUserData(file: string): Delete a user data file.
- moveUserData(source: string, dest: string, options?: RequestInit & { overwrite?: boolean }): Move a user data file.
- listUserData(dir: string, recurse?: boolean, split?: boolean): List a user data file.
- interrupt(): Interrupts the execution of the running prompt.
- reconnectWs(opened?: boolean): Reconnects to the WebSocket server.

$3

#### 🏗️ Constructor

`typescript
constructor(client: ComfyApi, workflow: PromptBuilder)
`

- client: An instance of the ComfyApi client.
- workflow: An instance of PromptBuilder defining the workflow.

#### ⚙️ Methods

- onPreview(fn: (ev: Blob, promptId?: string) => void): Set callback for preview events.
- onPending(fn: (promptId?: string) => void): Set callback when job is queued.
- onStart(fn: (promptId?: string) => void): Set callback when the job is started.
- onOutput(fn: (key: keyof PromptBuilder["mapOutputKeys"], data: any, promptId?: string) => void): Sets a callback for when an output node is executed.
- onFinished(fn: (data: Record["mapOutputKeys"], any>, promptId?: string) => void): Set callback when the job is finished.
- onFailed(fn: (err: Error, promptId?: string) => void): Set callback when the job failed.
- onProgress(fn: (info: NodeProgress, promptId?: string) => void): Set callback for progress updates.
- run(): Executes the workflow.

$3

#### 🏗️ Constructor

`typescript
constructor(prompt: T, inputKeys: I[], outputKeys: O[])
`

- prompt: The initial workflow data object.
- inputKeys: An array of input node keys.
- outputKeys: An array of output node keys.

#### ⚙️ Methods

- clone(): Creates a new PromptBuilder instance with the same configuration.
- bypass(node: keyof T | (keyof T)[]): PromptBuilder: Marks node(s) to be bypassed at generation.
- reinstate(node: keyof T | (keyof T)[]): PromptBuilder: Unmarks node(s) from bypass at generation.
- setInputNode(input: I, key: DeepKeys | Array>): Sets input node path for a key.
- setRawInputNode(input: I, key: string | string[]): Sets raw input node path for a key.
- appendInputNode(input: I, key: DeepKeys | Array>): Appends a node to the input node path.
- appendRawInputNode(input: I, key: string | string[]): Appends a node to the raw input node path.
- setOutputNode(output: O, key: DeepKeys): Sets output node path for a key.
- setRawOutputNode(output: O, key: string): Sets raw output node path for a key.
- input(key: I, value: V, encodeOs?: OSType): Sets an input value.
- inputRaw(key: string, value: V, encodeOs?: OSType): Sets a raw input value with dynamic key.
- get workflow: Retrieves the workflow object.
- get caller: Retrieves current PromptBuilder object.

$3

#### 🏗️ Constructor

`typescript
constructor(clients: ComfyApi[], mode: EQueueMode = EQueueMode.PICK_ZERO)
`

- clients: Array of ComfyApi instances.
- mode: The queue mode using EQueueMode enum values.

#### ⚙️ Methods

- on(type: K, callback: (event: TComfyPoolEventMap[K]) => void, options?: AddEventListenerOptions | boolean): Attach an event listener.
- off(type: K, callback: (event: TComfyPoolEventMap[K]) => void, options?: EventListenerOptions | boolean): Detach an event listener.
- addClient(client: ComfyApi): Adds a new client to the pool.
- removeClient(client: ComfyApi): Removes a client from the pool.
- removeClientByIndex(index: number): Removes a client by index.
- changeMode(mode: EQueueMode): Changes the queue mode.
- pick(idx?: number): Picks a client by index.
- pickById(id: string): Picks a client by ID.
- run(job: (client: ComfyApi, clientIdx?: number) => Promise, weight?: number, clientFilter?: { includeIds?: string[]; excludeIds?: string[] }): Run a job with priority on an available client.
- batch(jobs: Array<(client: ComfyApi, clientIdx?: number) => Promise>, weight?: number, clientFilter?: { includeIds?: string[]; excludeIds?: string[] }): Run multiple jobs concurrently.

$3

- EQueueMode:
- PICK_ZERO: Selects the client with zero remaining queue.
- PICK_LOWEST: Selects the client with the lowest remaining queue.
- PICK_ROUTINE: Selects clients in a round-robin manner.

$3

- OSType:
- POSIX: For Unix-like systems.
- NT: For Windows systems.
- JAVA: For Java virtual machine.
- TSamplerName: A union type of all available sampler names.
- TSchedulerName: A union type of all available scheduler names.

$3

- ManagerFeature: Provides methods to manage ComfyUI Manager Extension.

`typescript
const api = new ComfyApi("http://localhost:8189").init();
await api.waitForReady();

if (api.ext.manager.isSupported) {
await api.ext.manager.getExtensionList().then(console.log);
// Check api.ext.manager for more methods
}
`

- MonitoringFeature: Provides methods to monitor system resources using ComfyUI-Crystools Extension.

`typescript
const api = new ComfyApi("http://localhost:8189").init();
await api.waitForReady();

if (api.ext.monitor.isSupported) {
// For subscribing to system monitor events
api.ext.monitor.on("system_monitor", (ev) => {
console.log(ev.detail);
});

// For getting current monitor data
console.log(api.ext.monitor.monitorData);
}
`

> Note: Features require respective extensions (ComfyUI-Manager and ComfyUI-Crystools) to be installed.

📂 Examples

The examples directory contains practical demonstrations of SDK usage:

- example-i2i.ts: Demonstrates image-to-image generation.
- example-pool.ts: Demonstrates how to manage multiple ComfyUI instances using ComfyPool.
- example-pool-basic-auth.ts: Demonstrates how to use ComfyPool with HTTP Basic Authentication.
- example-t2i.ts: Demonstrates text-to-image generation.
- example-t2i-upscaled.ts: Demonstrates text-to-image generation with upscaling.
- example-img2img-workflow.json: Example workflow for image-to-image.
- example-txt2img-workflow.json: Example workflow for text-to-image.
- example-txt2img-upscaled-workflow.json`: Example workflow for text-to-image with upscaling.

🤝 Contributing

Contributions are always welcome! Feel free to submit pull requests or create issues for bug reports and feature enhancements. 🙏

📜 License

This project is licensed under the MIT License - see the LICENSE file for more details. 📄