NEARFS Uploader

A package to upload files to NEARFS via near-api-js (Node.js/browser) or web4 (browser).

NEARFS is a distributed file system compatible with IPFS that uses the NEAR blockchain as a backend. It allows you to store and retrieve files using the NEAR blockchain infrastructure.

This package is built on top of the following projects:
- near-api-js: NEAR JavaScript API
- web4: Unstoppable websites onchain
- nearfs: NEAR File System

Installation

``bash
npm install nearfs-upload
`

Usage

$3

The package provides a command-line tool for easy uploads. After installation, you can use it directly:

`bash
npx nearfs-upload [options]
`

The CLI automatically detects the type of upload based on the file extension - you can upload individual files, directories, or CAR files (.car extension) using the same command.

Options:
- -h, --help: Show help message
- -n, --network: NEAR network (default: testnet)
- -a, --account-id: NEAR account ID
- -k, --private-key: NEAR account private key
- --gateway-url: Custom IPFS gateway URL for non-mainnet/testnet networks
- --node-url: Custom NEAR RPC node URL

Credentials can be provided in three ways:
1. Command line arguments
2. Environment variables (NEAR_ACCOUNT_ID, NEAR_PRIVATE_KEY)
3. near-cli credentials (~/.near-credentials/{network}/{accountId}.json)

Examples:

`bash


Upload a directory

nearfs-upload ./my-files --account-id example.testnet --private-key "ed25519:..."

Upload a CAR file

nearfs-upload ./my-file.car --account-id example.testnet

Use environment variables

NEAR_ACCOUNT_ID=example.testnet NEAR_PRIVATE_KEY=ed25519:... nearfs-upload ./my-files

Custom network configuration

nearfs-upload ./my-files --network custom --gateway-url https://ipfs.custom.example.com
nearfs-upload ./my-files --node-url https://my-custom-near-node.com
`

$3

`javascript
import { uploadFiles } from 'nearfs-upload';
import { connect, keyStores, transactions } from 'near-api-js';

async function main() {
// Set up NEAR connection
const keyStore = new keyStores.InMemoryKeyStore();
const nearConnection = await connect({
networkId: 'testnet',
keyStore,
nodeUrl: 'https://rpc.testnet.near.org',
});

// Access NEAR account
const accountId = 'your-account.testnet';
const account = await nearConnection.account(accountId);

const signAndSendTransaction = async (blockDataArray) => {
return await account.signAndSendTransaction({
receiverId: accountId,
actions: blockDataArray.map(data =>
transactions.functionCall('fs_store', data, '30000000000000', '0')
),
});
};

// Prepare files for upload
const files = [
{ name: 'file1.txt', content: Buffer.from('Hello, world!') },
{ name: 'folder/file2.txt', content: Buffer.from('Nested file') },
];

// Upload files
const rootCid = await uploadFiles(files, {
signAndSendTransaction,
log: console.log,
statusCallback: ({ currentBlocks, totalBlocks }) => {
console.log(Progress: ${currentBlocks}/${totalBlocks} blocks uploaded);
},
});

console.log('Upload complete. Root CID:', rootCid);
}

main().catch(console.error);
`

$3

To upload a pre-made CAR file, use the uploadCAR function:

`javascript
import { uploadCAR } from 'nearfs-upload';
import fs from 'fs';

async function uploadCarFile(signAndSendTransaction) {
const carBuffer = fs.readFileSync('your-file.car');

await uploadCAR(carBuffer, {
signAndSendTransaction,
log: console.log,
});

console.log('CAR file upload complete');
}
`

$3

After uploading files, you can access them through the following public gateways:

- https://ipfs.web4.near.page - Access data stored on NEAR mainnet.
- https://ipfs.web4.testnet.page - Access data stored on NEAR testnet.

These gateways provide IPFS-compatible access to the files stored in NEARFS.

$3

NEARFS supports accessing content via subdomains, allowing you to serve full websites via the NEARFS gateway with isolated security contexts. You can access uploaded content using URLs like:

- http://.ipfs.web4.near.page/
- http://.ipfs.web4.near.page/:path

This provides a more intuitive way to share and access IPFS content through the NEARFS gateway.

API

$3

Uploads multiple files to NEARFS.

- files: An array of file objects with name and content properties.
- options: An object containing upload options.

Returns a Promise that resolves to the root CID of the uploaded files.

$3

Uploads a CAR file to NEARFS.

- carBuffer: A Buffer containing the CAR file data.
- options: An object containing upload options.

Returns a Promise that resolves when the upload is complete.

$3

- signAndSendTransaction: A function that implements the upload logic (required).
- log: A function for logging (default: console.log).
- statusCallback: A function called with upload progress (default: no-op).
- timeout: Timeout for checking if a block is already uploaded (default: 2500ms).
- retryCount: Number of retries for checking if a block is already uploaded (default: 3).
- gatewayUrl: URL of the IPFS gateway (default: 'https://ipfs.web4.near.page').

Testing

To run the tests:

`bash
npm test
``

License

MIT