@blocklet/crawler

@blocklet/crawler is a package crawler for Blocklet.

Package Structure

The package is composed of both frontend and backend components. The backend code can be found in the middlewares folder.

Development

$3

``


You can use npm / yarn


pnpm add @blocklet/crawler
`

$3

To install the required dependencies, run the following command:

`
pnpm i
`

$3

To build the packages, execute the following command:

`
pnpm build
`

$3

For building, watching changes, and running the development server, use the following command:

`
pnpm run dev
`

Backend Example

`javascript
const { initSEOMiddleware, initCronCrawlBlocklet } = require('@blocklet/crawler/middlewares');

// init cron job to crawl blocklet that generate SEO pages
initCronCrawlBlocklet({
time: '0 0 /12 ',
});

const router = express.Router();
// init SEO middleware, when request come, we will return the SEO static page
router.use(initSEOMiddleware());
``

License

This package is licensed under the MIT license.