A TypeScript library for uploading files to IPFS and managing pinning operations with support for multiple upload protocols, custom storage backends, and flexible configuration.
npm install @lumeweb/pinnerA TypeScript library for uploading files to IPFS and managing pinning operations with support for multiple upload protocols, custom storage backends, and flexible configuration.
- Multiple Upload Methods: TUS resumable uploads, XHR uploads, and direct CAR file uploads
- Directory Support: Upload multiple files as a directory to IPFS
- Pin Management: Add, list, remove, and check status of pinned content
- Custom Blockstore: Flexible storage backend using unstorage (IndexedDB, filesystem, Redis, etc.)
- Adapters: Built-in Pinata adapter with extensible adapter pattern
- Encoders: Support for CSV, JSON, text, base64, and URL encoding
- Progress Tracking: Real-time upload progress and operation polling
- Cross-Platform: Works in both browser and Node.js environments
- TypeScript: Fully typed with comprehensive type definitions
``bash`
pnpm add @lumeweb/pinner
`typescript
import { Pinner } from "@lumeweb/pinner";
// Initialize with JWT token
const pinner = new Pinner({
jwt: "your-jwt-token",
endpoint: "https://ipfs.pinner.xyz",
gateway: "https://dweb.link"
});
// Upload a file
const file = new File(["Hello, IPFS!"], "hello.txt", { type: "text/plain" });
const operation = await pinner.upload(file);
// Wait for completion
const result = await operation.result;
console.log("CID:", result.cid);
console.log("URL:", result.url);
// List pins
const pins = await pinner.listPins();
console.log("Pinned content:", pins);
`
The Pinner class accepts a PinnerConfig object:
`typescript
interface PinnerConfig {
// Required
jwt: string;
// Optional
endpoint?: string; // Default: "https://ipfs.pinner.xyz"
gateway?: string; // Default: "https://dweb.link"
allowedFileTypes?: string[]; // MIME types allowed for upload
fetch?: typeof fetch; // Custom fetch implementation
datastore?: Datastore; // Custom datastore for Helia
storage?: Storage; // Custom unstorage instance
datastoreName?: string; // Base name for storage (default: "pinner-helia-data")
}
`
`typescript
// Upload with default options
const operation = await pinner.upload(file);
// Upload with custom options
const operation = await pinner.upload(file, {
metadata: { name: "My File" },
timeout: 30000
});
// Wait for result
const result = await operation.result;
`
Convenience method for simple use cases:
`typescript`
const result = await pinner.uploadAndWait(file);
console.log("CID:", result.cid);
`typescript
const files = [
new File(["content1"], "file1.txt"),
new File(["content2"], "file2.txt")
];
const operation = await pinner.uploadDirectory(files);
const result = await operation.result;
`
Upload pre-generated CAR files without preprocessing:
`typescript`
const carFile = new File([carData], "content.car", { type: "application/vnd.ipld.car" });
const operation = await pinner.uploadCar(carFile);
const result = await operation.result;
Use the builder API for more control:
`typescript
// Upload JSON
const operation = await pinner.upload.json({ foo: "bar" });
// Upload text
const operation = await pinner.upload.text("Hello, world!");
// Upload CSV
const operation = await pinner.upload.csv([
{ name: "Alice", age: 30 },
{ name: "Bob", age: 25 }
]);
`
Upload operations return an UploadOperation with progress tracking:
`typescript
const operation = await pinner.upload(file);
// Listen to progress events
operation.on("progress", (progress) => {
console.log(Progress: ${progress.progress}%);Speed: ${progress.speed} bytes/sec
console.log();
});
// Listen to completion
operation.on("complete", (result) => {
console.log("Upload complete:", result.cid);
});
// Listen to errors
operation.on("error", (error) => {
console.error("Upload failed:", error);
});
// Or await the result directly
const result = await operation.result;
`
`typescript
import { CID } from "multiformats/cid";
// Pin existing content
const cid = CID.parse("Qm...");
await pinner.pinByHash(cid);
// With options
await pinner.pinByHash(cid, {
name: "My Pin",
metadata: { key: "value" }
});
`
`typescript
// List all pins
const pins = await pinner.listPins();
// List with filters
const pins = await pinner.listPins({
status: "pinned",
limit: 10,
offset: 0
});
`
`typescript`
const pin = await pinner.getPinStatus(cid);
console.log("Pin status:", pin.status);
console.log("Created:", pin.created);
`typescript`
const isPinned = await pinner.isPinned(cid);
if (isPinned) {
console.log("Content is pinned");
}
`typescript`
await pinner.setPinMetadata(cid, {
name: "Updated Name",
description: "Updated description"
});
`typescript`
await pinner.unpin(cid);
Wait for operations to complete with custom polling options:
`typescript
// Poll with default settings
const result = await pinner.waitForOperation(operationId);
// Poll with custom options
const result = await pinner.waitForOperation(operationId, {
interval: 1000, // Check every 1 second
timeout: 60000, // Timeout after 60 seconds
settledStates: ["completed", "failed"]
});
`
Configure a custom storage backend using unstorage:
`typescript
import { Pinner, createBlockstore, setDriverFactory } from "@lumeweb/pinner";
import { createStorage } from "unstorage";
import redisDriver from "unstorage/drivers/redis";
// Create Redis storage
const storage = createStorage({
driver: redisDriver({
host: "localhost",
port: 6379,
base: "pinner:"
})
});
// Initialize with custom storage
const pinner = new Pinner({
jwt: "your-token",
storage
});
`
`typescript
import { createBlockstore } from "@lumeweb/pinner";
// Auto-configure (browser: IndexedDB, Node.js: filesystem)
const blockstore = createBlockstore();
// Custom driver
const blockstore = createBlockstore({
driver: localStorageDriver({ base: "my-app:" })
});
// Pre-configured storage
const blockstore = createBlockstore({
storage: myStorageInstance
});
// Disable auto-configuration (uses memory)
const blockstore = createBlockstore({
autoConfigure: false
});
`
See blockstore/README.md for detailed blockstore documentation.
The Pinata adapters provide Pinata SDK API compatibility for the Pinner client, allowing applications written for the Pinata SDK to work with Lume's IPFS pinning infrastructure with minimal code changes.
Attribution: These adapters include TypeScript type definitions and API interfaces adapted from the Pinata SDK for compatibility purposes. The original Pinata SDK is available at:
- Pinata SDK 2.x: https://github.com/PinataCloud/pinata/commit/cdc0c06116aaadaf7c4b287a2673cd23b6ba1125
- Pinata SDK 1.x: https://github.com/PinataCloud/pinata/commit/c141177ff3036e46fa7b95fcc68c159b58817836
The adapters provide Pinata SDK API compatibility but route all operations through Lume's IPFS pinning infrastructure. They do NOT use Pinata's servers or services.
#### Available Adapters
- V2 Adapter (pinataAdapter): Compatible with Pinata SDK 2.x API (recommended, latest)pinataLegacyAdapter
- Legacy Adapter (): Compatible with Pinata SDK 1.x API
See adapters/README.md for comprehensive documentation including migration guides, feature support tables, and detailed examples.
#### Setup
`typescript
import { Pinner, pinataAdapter, pinataLegacyAdapter } from "@lumeweb/pinner";
// Initialize Pinner
const pinner = new Pinner({
jwt: "your-jwt-token",
endpoint: "https://your-pinning-service-endpoint.com"
});
// Create Pinata V2 adapter (recommended)
const pinata = pinataAdapter(pinner);
// Or use the legacy adapter
const pinataLegacy = pinataLegacyAdapter(pinner);
`
#### Upload Methods
The Pinata adapter provides multiple upload methods with a fluent builder pattern:
##### Upload a File
`typescript
// Simple file upload
const result = await pinata.upload.public.file(file).execute();
console.log("CID:", result.IpfsHash);
console.log("Size:", result.PinSize);
// Upload with metadata
const result = await pinata.upload.public.file(file)
.name("My File")
.keyvalues({ key: "value" })
.execute();
`
##### Upload Multiple Files (Directory)
`typescript
const files = [
new File(["content1"], "file1.txt"),
new File(["content2"], "file2.txt")
];
const result = await pinata.upload.public.fileArray(files)
.name("My Directory")
.keyvalues({ type: "directory" })
.execute();
`
##### Upload JSON Data
`typescript
const data = { foo: "bar", number: 42 };
const result = await pinata.upload.public.json(data)
.name("data.json")
.keyvalues({ format: "json" })
.execute();
`
##### Upload Base64 String
`typescript
const base64String = "SGVsbG8sIHdvcmxkIQ==";
const result = await pinata.upload.public.base64(base64String)
.name("base64-file.txt")
.execute();
`
##### Upload from URL
`typescript`
const result = await pinata.upload.public.url("https://example.com/data.json")
.name("downloaded-file.json")
.execute();
##### Pin by CID
`typescript
// Pin existing content
await pinata.upload.public.cid("Qm...").execute();
// Pin with metadata
await pinata.upload.public.cid("Qm...")
.name("Existing Content")
.keyvalues({ source: "external" })
.execute();
`
#### Pin Management
##### Pin by Hash
`typescript`
await pinata.pinByHash("Qm...", {
name: "My Pin",
keyvalues: { key: "value" }
});
##### Unpin Content
`typescript`
await pinata.unpin("Qm...");
##### Get Pin Status
`typescript`
const pin = await pinata.getPinStatus("Qm...");
console.log("Pin ID:", pin.id);
console.log("IPFS Hash:", pin.ipfsPinHash);
console.log("Size:", pin.size);
console.log("Date Pinned:", pin.datePinned);
console.log("Metadata:", pin.metadata);
##### Check if Pinned
`typescript`
const isPinned = await pinata.isPinned("Qm...");
if (isPinned) {
console.log("Content is pinned");
}
##### Update Pin Metadata
`typescript`
await pinata.setPinMetadata("Qm...", {
name: "Updated Name",
key: "value"
});
#### Files Management
##### List Files
`typescript
// List all files
const files = await pinata.files.public.list().execute();
// List with pagination
const files = await pinata.files.public.list()
.limit(10)
.pageToken("next-page-token")
.execute();
files.forEach(file => {
console.log("ID:", file.id);
console.log("CID:", file.cid);
console.log("Size:", file.size);
console.log("Name:", file.name);
console.log("Created:", file.createdAt);
});
`
##### Get File by ID
`typescript`
const file = await pinata.files.public.get("Qm...");
console.log("File details:", file);
#### Pinata SDK Compatibility
The adapter follows the Pinata SDK's API conventions, making migration straightforward:
Pinata SDK:
`typescript
import pinataSDK from '@pinata/sdk';
const pinata = pinataSDK('apiKey', 'apiSecret');
const result = await pinata.pinFileToIPFS(file, {
pinataMetadata: { name: 'My File' },
pinataOptions: { cidVersion: 1 }
});
`
Pinner with Pinata Adapter:
`typescript
import { Pinner, pinataAdapter } from '@lumeweb/pinner';
const pinner = new Pinner({ jwt: 'your-jwt-token' });
const pinata = pinataAdapter(pinner);
const result = await pinata.upload.public.file(file)
.name('My File')
.execute();
`
#### Result Format
Upload operations return a PinataUploadResult:
`typescript`
interface PinataUploadResult {
IpfsHash: string; // IPFS CID
PinSize: number; // Size in bytes
Timestamp: string; // ISO timestamp
isDuplicate: boolean; // Whether content was already pinned
}
#### Error Handling
The adapter provides specific errors:
`typescript
import { PinataAdapterError } from "@lumeweb/pinner/adapters/pinata";
try {
const result = await pinata.upload.file(file).execute();
} catch (error) {
if (error instanceof PinataAdapterError) {
switch (error.code) {
case "UPLOAD_FAILED":
console.error("Upload failed:", error.message);
break;
case "EMPTY_FILE_ARRAY":
console.error("Cannot upload empty file array");
break;
case "INVALID_CID":
console.error("Invalid CID:", error.message);
break;
}
}
}
`
The library provides several encoders for different data formats:
`typescript
// JSON encoder
const operation = await pinner.upload.json({ data: "value" });
// CSV encoder
const operation = await pinner.upload.csv([
{ column1: "value1", column2: "value2" }
]);
// Text encoder
const operation = await pinner.upload.text("Plain text content");
// Base64 encoder
const operation = await pinner.upload.base64("SGVsbG8sIHdvcmxkIQ==");
// URL encoder
const operation = await pinner.upload.url("https://example.com/data");
`
The library provides specific error types:
`typescript
import {
PinnerError,
ConfigurationError,
AuthenticationError,
UploadError,
NetworkError,
ValidationError,
EmptyFileError,
TimeoutError,
PinError,
NotFoundError,
RateLimitError
} from "@lumeweb/pinner";
try {
const result = await pinner.uploadAndWait(file);
} catch (error) {
if (error instanceof AuthenticationError) {
console.error("Authentication failed:", error.message);
} else if (error instanceof NetworkError) {
console.error("Network error:", error.message);
} else if (error instanceof ValidationError) {
console.error("Validation error:", error.message);
} else {
console.error("Unknown error:", error);
}
}
`
`typescript
import { isRetryable, isAuthenticationError } from "@lumeweb/pinner";
if (isRetryable(error)) {
// Retry the operation
}
if (isAuthenticationError(error)) {
// Re-authenticate
}
`
`typescript
import {
streamToBlob,
calculateStreamSize,
asyncGeneratorToReadableStream,
readableStreamToAsyncIterable
} from "@lumeweb/pinner";
// Convert stream to blob
const blob = await streamToBlob(stream);
// Calculate stream size
const size = await calculateStreamSize(stream);
// Convert async generator to readable stream
const readableStream = asyncGeneratorToReadableStream(asyncGenerator);
// Convert readable stream to async iterable
const asyncIterable = readableStreamToAsyncIterable(readableStream);
`
The library includes comprehensive tests:
`bashRun all tests
pnpm test
$3
- Unit Tests: Test individual components with mocks
- Integration Tests: Test real-world scenarios without mocks
- Browser Tests: Run tests in Chromium using Playwright
- Node Tests: Run tests in Node.js environment
Build
`bash
Build the library
pnpm buildType checking
pnpm lint
`The library outputs:
- ESM:
dist/esm/
- CJS: dist/cjs/
- Type definitions: dist/esm/*.d.tsPackage Exports
`typescript
// Main exports
import { Pinner } from "@lumeweb/pinner";// Upload types and utilities
import {
UploadManager,
UploadResult,
UploadOptions,
UploadProgress
} from "@lumeweb/pinner";
// Pin types
import type {
RemotePins,
RemotePin,
RemoteAddOptions,
RemoteLsOptions
} from "@lumeweb/pinner";
// Blockstore
import {
createBlockstore,
createDatastore,
setDriverFactory
} from "@lumeweb/pinner";
// Adapters
import { pinataAdapter, pinataLegacyAdapter } from "@lumeweb/pinner";
// Blockstore module
import { UnstorageBlockstore } from "@lumeweb/pinner/blockstore";
``MIT
Contributions are welcome! Please ensure all tests pass and follow the existing code style.