Convert files to content-addressed archives (.car) and back
npm install ipfs-car> Convert files to content-addressable archives (.car) and back
ipfs-car is a library and CLI tool to pack & unpack files from Content Addressable aRchives (CAR) file. A thin wrapper over @ipld/car and unix-fs.
Content-addressable archives store data as blocks (a sequence of bytes) each prefixed with the Content ID (CID) derived from the hash of the data; typically in a file with a .car extension.
Use ipfs-car to pack your files into a .car; a portable, verifiable, IPFS compatible archive.
``sh`
$ ipfs-car pack path/to/files --output my-files.car
or unpack files from a .car, and verify that every block matches it's CID
`sh`
$ ipfs-car unpack my-files.car --output path/to/write/to
Fetch and locally verify files from a IPFS gateway over http
`sh`
curl "https://ipfs.io/ipfs/bafybeidd2gyhagleh47qeg77xqndy2qy3yzn4vkxmk775bg2t5lpuy7pcu?format=car" | ipfs-car unpack -o images
`shinstall it as a dependency
$ npm i ipfs-car
Usage
Pack files into a .car
sh
write a content addressed archive to stdout.
$ ipfs-car pack path/to/file/or/dir
note: CAR data streamed to stdout will not have roots set in CAR header!
specify the car file name.
$ ipfs-car pack path/to/files --output path/to/write/a.carby default, ipfs-car will wrap files in an IPFS directory.
use --no-wrap to avoid this.
$ ipfs-car pack path/to/file --no-wrap --output path/to/write/a.car
`Unpack files from a .car
sh
unpack files to a specific path.
$ ipfs-car unpack path/to/my.car --output /path/to/unpack/files/tounpack a specific root.
$ ipfs-car unpack path/to/my.car --root unpack files from a .car on stdin.
$ cat path/to/my.car | ipfs-car unpack
`Show the files and directories in a .car
sh
show the files and directories.
$ ipfs-car ls path/to/my.carshow the files and directories, their CIDs and byte sizes.
$ ipfs-car ls path/to/my.car --verbose
`Show the root CIDs in a .car
sh
show the CID roots found in the CAR header.
$ ipfs-car roots path/to/my.carshow the CID roots found implicitly from the blocks in the file.
$ ipfs-car roots --implicit path/to/my.car
`Show the block CIDs in a .car
sh
show the CIDs for all the blocks.
$ ipfs-car blocks path/to/my.car
`Get other information about a CAR
sh
generate CID for a CAR.
$ ipfs-car hash path/to/my.car
`API
To pack files into content-addressable archives, you can use the following:
-
a factory function for creating a ReadableStream that encodes a single file into DAG Blocks.
- createDirectoryEncoderStream a factory function for creating a ReadableStream for encoding a directory of files into DAG Blocks.
- CAREncoderStream a TransformStream sub-class that you can write Blocks to and read Uint8Array CAR file data from.To unpack content-addressable archives to files, you should use
@ipld/car and ipfs-unixfs-exporter modules.$3
#### Basic single file pack
`js
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'const file = new Blob(['Hello ipfs-car!'])
const carStream = createFileEncoderStream(file).pipeThrough(new CAREncoderStream())
// carStream.pipeTo(somewhereWritable)
`#### Directory pack to file system in Node.js
js
import { Writable } from 'stream'
import { createDirectoryEncoderStream, CAREncoderStream } from 'ipfs-car'
import { filesFromPaths } from 'files-from-path'const files = await filesFromPaths(process.argv.slice(2))
await createDirectoryEncoderStream(files)
.pipeThrough(new CAREncoderStream())
.pipeTo(Writable.toWeb(process.stdout))
`Usage:
.#### Obtaining the root CID
The root CID is the final block generated by the file/directory encoder stream. Use a transform stream to record the CID of the last block generated:
`js
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'const file = new Blob(['Hello ipfs-car!'])
let rootCID
await createFileEncoderStream(file)
.pipeThrough(new TransformStream({
transform (block, controller) {
rootCID = block.cid
controller.enqueue(block)
}
}))
.pipeThrough(new CAREncoderStream())
.pipeTo(new WritableStream())
console.log(rootCID.toString())
`#### Adding root CIDs to the CAR header
If you need root CIDs in the CAR header, there are two approaches you can use:
1. Buffer all the DAG blocks, then encode with known root:
js
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'const file = new Blob(['Hello ipfs-car!'])
const blocks = []
// buffer the output
await createFileEncoderStream(file)
.pipeTo(new WritableStream({ write: b => blocks.push(b) }))
const rootCID = blocks.at(-1).cid
const blockStream = new ReadableStream({
pull (controller) {
if (blocks.length) {
controller.enqueue(blocks.shift())
} else {
controller.close()
}
}
})
await blockStream
.pipeThrough(new CAREncoderStream([rootCID])) // pass root to CAR encoder
.pipeTo(new WritableStream())
`2. Write to disk with placeholder CID, then update after DAG is completely generated (Note: Node.js only):
js
import fs from 'fs'
import { Writable } from 'stream'
import { CarWriter } from '@ipld/car/writer'
import { CID } from 'multiformats/cid'
import { createFileEncoderStream, CAREncoderStream } from 'ipfs-car'// Root CID written in CAR file header before it is updated with the real root CID.
const placeholderCID = CID.parse('bafybeigdyrzt5sfp7udm7hu76uh7y26nf3efuylqabf3oclgtqy55fbzdi')
const file = new Blob(['Hello ipfs-car!'])
let rootCID
await createFileEncoderStream(file)
.pipeThrough(new TransformStream({
transform (block, controller) {
rootCID = block.cid
controller.enqueue(block)
}
}))
.pipeThrough(new CAREncoderStream(placeholderCID))
.pipeTo(Writable.toWeb(fs.createWriteStream('path/to/my.car')))
// update roots in CAR header
const fd = await fs.promises.open(opts.output, 'r+')
await CarWriter.updateRootsInFile(fd, [rootCID])
await fd.close()
`#### Unpacking files from a CAR
This functionality is not provided by this library, but is easy to do with
and ipfs-unixfs-exporter modules:`js
import { CarIndexedReader } from '@ipld/car/indexed-reader'
import { recursive as exporter } from 'ipfs-unixfs-exporter'const reader = await CarIndexedReader.fromFile('path/to/my.car')
const roots = await reader.getRoots()
const entries = exporter(roots[0], {
async get (cid) {
const block = await reader.get(cid)
return block.bytes
}
})
for await (const entry of entries) {
if (entry.type === 'file' || entry.type === 'raw') {
console.log('file', entry.path, entry.content)
} else if (entry.type === 'directory') {
console.log('directory', entry.path)
}
}
``Feel free to join in. All welcome. Open an issue!
Dual-licensed under MIT + Apache 2.0