!@thi.ng/pixel

!npm downloads

> [!NOTE]
> This is one of 214 standalone projects, maintained as part
> of the @thi.ng/umbrella monorepo
> and anti-framework.
>
> 🚀 Please help me to work full-time on these projects by sponsoring me on
> GitHub. Thank you! ❤️

- About
- Integer pixel formats
- Indexed, palette-based pixel formats
- Floating point pixel formats
- Filtered image sampling and resizing
- Filters
- Wrap mode
- Status
- Support packages
- Related packages
- Installation
- Dependencies
- Usage examples
- API
- Authors
- License

About

Typedarray integer & float pixel buffers w/ customizable formats, blitting, drawing, convolution.

> [!IMPORTANT]
> In July 2024 this package was restructured and split-up to extract some
> features into smaller more focused packages:
>
> - @thi.ng/pixel-convolve
> - @thi.ng/pixel-dominant-colors

- Buffer creation from HTML image elements or canvas w/ opt resize & format
conversion (browser only)
- 12 packed integer and 6 floating point preset formats (see table below)
- Palette-based indexed pixel formats
- Buffer-to-buffer blitting w/ automatic format conversion
- Buffer-to-canvas blitting (incl. offscreen canvas support)
- Buffer-to-buffer blending w/ Porter-Duff
operators
- Pre/post-multiply alpha
- Region / sub-image extraction
- Single-channel manipulation / extraction / replacement / conversion
- Accessors for normalized channel value
- Image sampling & filtered resizing
- Filters: nearest neighbor, bilinear, bicubic
- Wrap behaviors: clamp, wrap, repeat
- Invert image
- XY coordinate-based pixel & channel-only accessors (w/ optional bounds checking)
- Declarative custom pixel formats with optimized code generation
- HTML canvas creation & ImageData utilities

$3

All integer formats use the canvas native ABGR 32bit format as common
intermediate for conversions. During conversion to ABGR, channels with sizes
smaller than 8 bits will be scaled appropriately to ensure an as full-range and
as linear as possible mapping. E.g. a 4 bit channel will be scaled by 255 / 15 =
17.

Format specs can freely control channel layout within current limits:

- Channel sizes: 1 - 32 bits.
- Storage: 8, 16 or 32 bits per pixel

Custom formats can be defined via
defIntFormat().

| Format ID | Bits per pixel | Description |
|----------------|-------------------|------------------------------------------------------|
| ALPHA8 | 8 | 8 bit channel (alpha only) |
| GRAY8 | 8 | 8 bit single channel (grayscale conv) |
| GRAY_ALPHA8 | 16 | 8 bit single channel (grayscale conv), 8 bit alpha |
| GRAY16 | 16 | 16 bit single channel (grayscale conv) |
| GRAY_ALPHA16 | 32 | 16 bit single channel (grayscale conv), 16 bit alpha |
| ARGB4444 | 16 | 4 channels @ 4 bits each |
| ARGB1555 | 16 | 5 bits each for RGB, 1 bit alpha |
| RGB565 | 16 | 5 bits red, 6 bits green, 5 bits blue |
| RGB888 | 32 (24 effective) | 3 channels @ 8 bits each |
| ARGB8888 | 32 | 4 channels @ 8 bits each |
| BGR888 | 32 (24 effective) | 3 channels @ 8 bits each |
| ABGR8888 | 32 | 4 channels @ 8 bits each |

- ALPHA8 is mapped from/to ABGR alpha channel
- GRAY8/16, GRAY_ALPHA8/16 compute grayscale/luminance when
converting from ABGR and in return produce grayscale ABGR
- In all built-in formats supporting it, the alpha channel always
occupies the most-significant bits (up to format size)

$3

Instead of storing colors directly for each pixel, palette-based formats are
supported which only store a color index per pixel (e.g. as is done for GIF
and/or indexed PNG formats). These formats can be created via the
defIndexed() family of
functions.

$3

Strided floating point format presets for use with
floatBuffer().
New formats can be defined via
defFloatFormat().

| Format ID | Channel count | Description |
|--------------------|---------------|--------------------------------------|
| FLOAT_GRAY | 1 | Single channel / grayscale |
| FLOAT_GRAY_ALPHA | 2 | Grayscale and alpha channel |
| FLOAT_GRAY_RANGE | 1 | Grayscale (user defined value range) |
| FLOAT_NORMAL | 3 | Normal map (signed values) |
| FLOAT_RGB | 3 | Red, Green, Blue |
| FLOAT_RGBA | 4 | Red, Green, Blue, Alpha |

- All color channels are unclamped (but can be clamped via buf.clamp()). For
conversion to packed int formats assumed to contain normalized data (i.e.
[0..1] interval, with exception of FLOAT_NORMAL which uses [-1..1] range)
- Conversion between float formats is currently unsupported

$3

Available (and optimized) for both integer & floating point formats, image
samplers can be created with the following filters & wrap modes:

#### Filters

- "nearest" - nearest neighbor
- "linear" - bilinear interpolation
- "cubic" - bicubic interpolation

#### Wrap mode

- "clamp" - outside values return 0
- "wrap" - infinite tiling
- "repeat" - edge pixels are repeated

``ts
import { intBuffer, defSampler, ABGR8888 } from "@thi.ng/pixel";

const src = intBuffer(4, 4, ABGR8888);

// fill w/ random colors
src.forEach((_,i) => 0xff000000 | Math.random() * 0xffffff);

// create bilinear sampler w/ repeated edge pixels
const sampler = defSampler(src, "linear", "repeat");

// sample at fractional positions (even outside image)
sampler(-1.1, 0.5).toString(16)
// 'ff79643a'

// resize image to 1024x256 using bicubic sampling
const img = src.resize(1024, 256, "cubic");
`

| Filter | |
|-------------|------------------------------------------------------------------------------------------------------------------------------------------|
| "nearest" | !resized image w/ nearest neighbor sampling |
| "linear" | !resized image w/ bilinear sampling |
| "cubic" | !resized image w/ bicubic sampling |

Status

STABLE - used in production

Search or submit any issues for this package

Support packages

- @thi.ng/pixel-analysis - Image color & feature analysis utilities
- @thi.ng/pixel-convolve - Extensible bitmap image convolution, kernel presets, normal map & image pyramid generation
- @thi.ng/pixel-dither - Extensible image dithering w/ various algorithm presets
- @thi.ng/pixel-dominant-colors - k-means based dominant color extraction from images/pixel buffers
- @thi.ng/pixel-flow - Naive, lightweight CPU-based dense optical flow implementation
- @thi.ng/pixel-io-geotiff - GeoTIFF reader support for @thi.ng/pixel
- @thi.ng/pixel-io-netpbm - Multi-format NetPBM reader & writer support for @thi.ng/pixel
- @thi.ng/pixel-io-pfm - Portable FloatMap image format support for @thi.ng/pixel
- @thi.ng/pixel-io-tiff - Basic TIFF image format support for @thi.ng/pixel

Related packages

- @thi.ng/color - Array-based color types, CSS parsing, conversions, transformations, declarative theme generation, gradients, presets
- @thi.ng/porter-duff - Porter-Duff operators for packed ints & float-array alpha compositing
- @thi.ng/rasterize - Headless 2D shape drawing, filling & rasterization for arbitrary targets/purposes (no canvas required)
- @thi.ng/shader-ast - DSL to define shader code in TypeScript and cross-compile to GLSL, JS and other targets
- @thi.ng/webgl - WebGL & GLSL abstraction layer

Installation

`bash
yarn add @thi.ng/pixel
`

ESM import:

`ts
import * as pix from "@thi.ng/pixel";
`

Browser ESM import:

`html

`

JSDelivr documentation

For Node.js REPL:

`js
const pix = await import("@thi.ng/pixel");
`

Package sizes (brotli'd, pre-treeshake): ESM: 7.53 KB

Dependencies

- @thi.ng/api
- @thi.ng/canvas
- @thi.ng/checks
- @thi.ng/errors
- @thi.ng/math
- @thi.ng/porter-duff

Note: @thi.ng/api is in _most_ cases a type-only import (not used at runtime)

Usage examples

27 projects in this repo's
/examples
directory are using this package:

| Screenshot | Description | Live demo | Source |
|:--------------------------------------------------------------------------------------------------------------------------|:---------------------------------------------------------------------------------------------|:---------------------------------------------------------|:--------------------------------------------------------------------------------------|
| | Interactive image processing (adaptive threshold) | Demo | Source |
| | ASCII art raymarching with thi.ng/shader-ast & thi.ng/text-canvas | Demo | Source |
| | Interactive & reactive image blurhash generator | Demo | Source |
| | Color palette generation via dominant color extraction from uploaded images | Demo | Source |
| | 2.5D hidden line visualization of digital elevation files (DEM) | Demo | Source |
| | Barnsley fern IFS fractal renderer | Demo | Source |
| | Optical flow analysis of web cam or video inputs | Demo | Source |
| | Pixel buffer manipulations | Demo | Source |
| | Matrix-based image color adjustments | Demo | Source |
| | Showcase of various dithering algorithms | Demo | Source |
| | Randomized 4-point 2D color gradient image generator | Demo | Source |
| | Image dithering and remapping using indexed palettes | Demo | Source |
| | Normal map creation/conversion basics | Demo | Source |
| | Interactive pixel sorting tool using thi.ng/color & thi.ng/pixel | Demo | Source |
| | RGB waveform image analysis | Demo | Source |
| | Image-based Poisson-disk sampling | Demo | Source |
| | Port-Duff image compositing / alpha blending | Demo | Source |
| | Steering behavior drawing with alpha-blended shapes | Demo | Source |
| | Basic usage of the declarative rdom-forms generator | Demo | Source |
| | Responsive image gallery with tag-based Jaccard similarity ranking | Demo | Source |
| | 2D scenegraph & image map based geometry manipulation | Demo | Source |
| | WebGL & Canvas2D textured tunnel shader | Demo | Source |
| | Fork-join worker-based raymarch renderer (JS/CPU only) | Demo | Source |
| | Textmode image warping w/ 16bit color output | Demo | Source |
| | Multi-layer vectorization & dithering of bitmap images | Demo | Source |
| | Visual comparison of biased vs. unbiased normal vectors projected on the surface of a sphere | Demo | Source |
| | Minimal multi-pass / GPGPU example | Demo | Source |

API

Generated API docs

`ts
import * as pix from "@thi.ng/pixel";
import { SRC_OVER_I } from "@thi.ng/porter-duff";
import { pixelCanvas2d } from "@thi.ng/canvas";

import IMG from "../assets/haystack.jpg";
import LOGO from "../assets/logo-64.png";

const [img, logo] = await Promise.all([IMG, LOGO].map((x) => imageFromURL(x)));

// init 16bit int RGB pixel buffer from image (resized to 256x256)
const buf = intBufferFromImage(img, RGB565, 256, 256);

// create grayscale buffer for logo and use Porter-Duff operator to
// composite with main image. Since the logo has transparency, we
// need to premultiply alpha first...
intBufferFromImage(logo, GRAY_ALPHA8).premultiply().blend(SRC_OVER_I, buf, {
dx: 10,
dy: 10,
});

// extract sub-image
// (method returns undefined if result region is < 1 pixel)
const region = buf.getRegion(32, 96, 128, 64)!;
// copy region back at new position
region.blit(buf, { dx: 96, dy: 32 });

// or alternatively blit buf into itself
// buf.blit(buf, { dx: 96, dy: 32, sx: 32, sy: 96, w: 128, h: 64 });

// create html canvas
// (returns obj of canvas & 2d context)
const { canvas } = pixelCanvas2d(buf.width, buf.height * 3, document.body);

// write pixel buffer to canvas
buf.blitCanvas(canvas);

// manipulate single color channel
const id = 0;
const ch = buf.getChannel(id).invert();
for (let y = 0; y < ch.height; y += 2) {
for (let x = (y >> 1) & 1; x < ch.width; x += 2) {
ch.setAt(x, y, 0xff);
}
}
// replace original channel
buf.setChannel(id, ch);
// write pixel buffer to new position
buf.blitCanvas(canvas, { y: buf.height });
// create & write grayscale version
buf.as(GRAY8).blitCanvas(canvas, { y: buf.height * 2 });
`

Authors

- Karsten Schmidt (Main author)
- Askar Yusupov

If this project contributes to an academic publication, please cite it as:

`bibtex
@misc{thing-pixel,
title = "@thi.ng/pixel",
author = "Karsten Schmidt and others",
note = "https://thi.ng/pixel",
year = 2019
}
``

License

© 2019 - 2026 Karsten Schmidt // Apache License 2.0