@idio/idio

!Node.js CI

@idio/idio contains Koa's fork called Goa — web server compiled with _Closure Compiler_ so that its source code is optimised and contains only 1 external dependency (mime-db). Idio adds essential middleware to Goa for session, static files, CORS and compression and includes the router. As the project grows, more middleware will be added and optimised.

This is a production-ready server that puts all components together for the ease of use, while providing great developer experience using JSDoc annotations for auto-completions. _Idio_ is not a framework, but a library that enables idiomatic usage and compilation of the server and middleware.

``console
idio~:$ \
yarn add @idio/idio
npm install @idio/idio
`





Example Apps

There are some example apps that you can look at.

1. File Upload: a front-end + back-end application for uploading photos. Demo requires GitHub authorisation without any scope permissions to enable session middleware showcase.
1. Akashic.Page: a service for managing email and web-push subscriptions, with JS widgets and Mongo database connection.





Table Of Contents

- Example Apps
- Table Of Contents
- API
- async idio(middlewareConfig=, config=): !Idio
* MiddlewareConfig
* Config
* Idio
* ConfiguredMiddleware
- Middleware
* Static
* Session
* CORS
* Compression
* File Upload
* Front End
- Additional Middleware
- Custom Middleware
- Router Set-up
- SSR
- NeoLuddite.Dev
* NeoLudditeOptions
- WebSocket
- Copyright & License








API

The package is available by importing its default function and named components:

`js
import idio, { Keygrip, Router } from '@idio/idio'
`





async idio(
  middlewareConfig=: !MiddlewareConfig,
  config=: !Config,
): !Idio


Start the server. Sets the proxy property to true when the NODE_ENV is equal to _production_.

- middlewareConfig server.">!MiddlewareConfig (optional): The middleware configuration for the idio server.
- config !Config (optional): The server configuration object.

The app can be stopped with an async .destroy method implemented on it that closes all connections.


There are multiple items for middleware configuration:

__MiddlewareConfig extends FnMiddlewareConfig__: Middleware configuration for the idio server.


| Name | Type | Description |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| static | (!StaticOptions \| !Array<!StaticOptions>) | _Static_ middleware options. |
| compress | (boolean \| !CompressOptions) | _Compression_ middleware options. |
| session | !SessionOptions | _Session_ middleware options. |
| cors | !CorsOptions | _CORS_ middleware options. |
| form | !FormDataOptions | _Form Data_ middleware options for receiving file uploads and form submissions. |
| frontend | !FrontEndOptions | _Front End_ middleware allows to serve source code from node_modules and transpile JSX. |
| neoluddite | !NeoLudditeOptions | Records the usage of middleware to compensate their developers' intellectual work. |
| csrfCheck | !CsrfCheckOptions | Enables the check for the presence of session with csrf property, and whether it matches the token from either ctx.request.body or ctx.query. |
| github | (!GitHubOptions \| !Array<!GitHubOptions>) | Sets up a route for GitHub OAuth authentication. The returned middleware will be installed on the app automatically so it doesn't need to be passed to the router. |
| jsonErrors | (boolean \| !JSONErrorsOptions \| !Array<!JSONErrorsOptions>) | Tries all downstream middleware, and if an error was caught, serves a JSON response with error and stack properties (only if exposeStack is set to true). Client errors with status code _4xx_ (or that start with !) will have full message, but server errors with status code _5xx_ will only be served as { error: 'internal server error '} and the app will emit an error via app.emit('error') so that it's logged. |
| jsonBody | (boolean \| !JSONBodyOptions) | Allows to parse incoming JSON request and store the result in ctx.request.body. Throws 400 when the request cannot be parsed. |
| logarithm | !LogarithmOptions | Options to record hits in _ElasticSearch_. |

The types for starting the server include the address, port and router configuration.

__Config__: Server configuration object.


| Name | Type | Description | Default |
| ------ | ----------------------------------------------------------------------------------------------------------------------------------------- | -------------------------------------- | --------- |
| port | number | The port on which to start the server. | 5000 |
| host | string | The host on which to listen. | 0.0.0.0 |
| router | !_goa.RouterConfig | The configuration for the router. | - |

After the app is started, it can be accessed from the return type.

__Idio__: The return type of the idio.


| Name | Type | Description |
| --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| __url*__ | string | The URL on which the server was started, such as http://localhost:5000. |
| __server*__ | !http.Server | The server instance. |
| __app*__ | !Application | The Goa application instance (with additional .destroy method). |
| __middleware*__ | !ConfiguredMiddleware | An object with configured middleware functions, which can be installed manually using app.use, or router.use. The context will be a standard Goa context with certain properties set by bundled middleware such as .session. |
| __router*__ | !Router | The router instance. |

All middleware can be accessed from the middleware property, so that it can be installed on individual basis on specific routes, if it's not used app-wise.

ConfiguredMiddleware extends MiddlewareObject: Idio-specific properties of the middleware object.

| Name | Type | Description |
| ---------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------- |
| form | !_multipart.FormData | An instance of the form data class that can be used to create middleware. |
| session | method.">!Middleware | The session middleware to be installed on individual routes. |
| frontend | method.">!Middleware | The frontend middleware. |
| csrfCheck | method.">!Middleware | Configured CSRF check middleware. |
| jsonErrors | ( method.">!Middleware \| !Array< method.">!Middleware>) | Middleware to server errors as JSON. |

The example below starts a simple server with session and custom middleware, which is installed (used) automatically because it's defined as a function.








Source

Output

`js const { url, app, middleware: { session, form }, router, } = await idio({ // Developers' payment scheme neoluddite.dev neoluddite: { env: process.env.NODE_ENV, key: '0799b7f0-d2c7-4903-a531-00c8092c2911', app: 'idio.example', }, // Idio's bundled middleware. session: { algorithm: 'sha512', keys: ['hello', 'world'], prefix: 'example-', }, static: { use: true, root: 'upload', }, form: { config: { dest: 'upload', }, }, // Any middleware function to be use app-wise. async middleware(ctx, next) { console.log('//', ctx.method, ctx.path) await next() }, }) app.use(router.routes()) router.get('/', session, (ctx) => { ctx.body = 'hello world' }) router.post('/upload', session, async (ctx, next) => { if (!ctx.session.user) { ctx.status = 403 ctx.body = 'you must sign in to upload' return } await next() }, form.single('/upload'), (ctx) => { // db.create({ // user: ctx.session.id, // file: ctx.req.file.path, // }) ctx.body = 'Thanks for the upload. Link: ' + ${url}/${ctx.file.filename} }) `

` http://localhost:5000 // GET / hello world `





Middleware

Idio's advantage is that is has the essential middleware, that was compiled together with the server, so that the packages are reused and memory footprint is low.





$3

🗂 Explore Static Middleware Configuration

Used to serve static files, such as stylesheets, images, videos, html and everything else. Will perform mime-type lookup to serve the correct content-type in the returned header.









Static source	The Output
`js const { url, app } = await idio({ static: { root: 'example', use: true, }, ` `js // or multiple locations static: [{ root: ['example'], use: true, }, { root: ['wiki'], use: true, }], }, { port: null }) `	`css /* http://localhost:57537/app.css / body { font-size: larger; } `
Show Response Headers `http Content-Length: 29 Last-Modified: Thu, 18 Jul 2019 14:34:31 GMT Cache-Control: max-age=0 Content-Type: text/css; charset=utf-8 Date: Thu, 05 Mar 2020 13:30:57 GMT Connection: close ` `http Content-Length: 114 Last-Modified: Sat, 28 Dec 2019 18:07:31 GMT Cache-Control: max-age=0 Content-Type: image/svg+xml Date: Thu, 05 Mar 2020 13:30:59 GMT Connection: close `





$3

👳‍♂️Explore Session Middleware Configuration

Allows to store data in the .session property of the context. The session is serialised and placed in cookies. When the request contains the cookie, the session will be restored and validated (if signed) against the key.












Session Config
`js const { url, app } = await idio({ session: { use: true, keys: ['hello', 'world'], algorithm: 'sha512' }, async middleware(ctx, next) { if (ctx.session.user) ctx.body = 'welcome back ' + ctx.session.user else { ctx.session.user = 'u' + (Math.random() * 1000).toFixed(1) ctx.body = 'hello new user' } await next() }, }) `
The session data is encrypted with base64 and signed by default, unless the .signed option is set to false. Signing means that the signature will contain the hash which will be validated server-side, to ensure that the session data was not modified by the client. The default algorithm for signing is sha1, but it can be easily changed to a more secure sha512.
`js // GET / "hello new user" / set-cookie / [ { name: 'koa:sess', value: 'eyJ1c2VyIjoidTg2LjciLCJfZXhwaXJlIjoxNTgzNTAxNDU5ODQzLCJfbWF4QWdlIjo4NjQwMDAwMH0=', path: '/', expires: 'Fri, 06 Mar 2020 13:30:59 GMT', httponly: true }, { name: 'koa:sess.sig', value: '5hRueSOyLuhp6nZvOi4TcziXNiADlaIhE6fJHruR-I8cGtEVDYCgNe9t3LS0SyV-SEN1kPa8ZwIz-a91GWPw-A', path: '/', expires: 'Fri, 06 Mar 2020 13:30:59 GMT', httponly: true } ] // GET / "welcome back u86.7" `





$3

👮‍♀️Explore CORS Middleware Configuration

To enable dynamic communication between clients and the server via JavaScript requests from the browser, the server must respond with Access-Control-Allow-Origin header that sets the appropriate allowed _Origin_. This middleware is easy to use on production and development environments.








CORS source

The Output

`js const { NODE_ENV } = process.env const { url, app } = await idio({ async example(ctx, next) { console.log('//', ctx.method, ctx.path, 'from', ctx.get('Origin')) ctx.body = 'hello world' await next() }, cors: { use: true, origin: NODE_ENV == 'production' ? 'http://prod.com' : null, allowMethods: ['GET', 'POST'], }, }) `

`js // GET / from https://3rd.party { vary: 'Origin', 'access-control-allow-origin': 'http://prod.com', date: 'Thu, 05 Mar 2020 13:31:01 GMT', connection: 'close' } // GET / from http://prod.com { vary: 'Origin', 'access-control-allow-origin': 'http://prod.com', date: 'Thu, 05 Mar 2020 13:31:01 GMT', connection: 'close' } // OPTIONS / from http://prod.com { vary: 'Origin', 'access-control-allow-origin': 'http://prod.com', 'access-control-allow-methods': 'GET,POST', date: 'Thu, 05 Mar 2020 13:31:01 GMT', connection: 'close' } `





$3


🗜Explore Compression Middleware Configuration

When the body of the response is non-empty, it can be compressed using gzip algorithm. This allows to save data transmitted over the network. The default threshold is 1024 bytes, since below that the benefits of compression are lost as the compressed response might end up being even larger.








Compression source	The Output
`js const { url, app } = await idio({ async serve(ctx, next) { console.log('//', ctx.method, ctx.path) ctx.body = packageJson await next() }, compress: { use: true, }, }) `	`js // GET / { 'content-type': 'application/json; charset=utf-8', vary: 'Accept-Encoding', 'content-encoding': 'gzip', date: 'Thu, 05 Mar 2020 13:31:01 GMT', connection: 'close', 'transfer-encoding': 'chunked' } `

$3

🖼Explore Form Data Middleware Configuration

Browser will submit forms and send files using multipart/form-data type of request. It will put all fields of the form together and stream them to the server, sending pairs of keys/values as well as files when they were attached. The _Form Data_ middleware is the Multer middleware specifically rewritten for Koa that can handle file uploads.








File Upload source	The Output
`js const { url, app, router, middleware: { form, } } = await idio({ form: { dest: 'example/upload', }, }) app.use(router.routes()) router.post('/example', form.single('bio'), (ctx) => { delete ctx.file.stream ctx.body = { file: ctx.file, body: ctx.request.body } } ) `	`js { file: { fieldname: 'bio', originalname: 'bio.txt', encoding: '7bit', mimetype: 'application/octet-stream', destination: 'example/upload', filename: '106e5', path: 'example/upload/106e5', size: 29 }, body: { hello: 'world' } } `





$3


🌐Explore Front End Middleware Configuration

Web applications are always full stack and involve both back-end together with front-end. Whereas all previously described middleware was for the server only, the front-end middleware facilitates browser development, as it allows to serve source code from the node_modules directory and transpile JSX. Modern browsers support modules, but JavaScript needs to be patched to rename imports like
`js
// was
import X from 'package-name'
// becomes
import X from '/node_modules/package-name/src/index.mjs'
`
This is achieved by resolving the module field from package.json of served packages (with fallback to the main field, but in that case require statements will not work).














Configuration	JSX Component
`js const { url, app } = await idio({ frontend: { use: true, directory: 'example/frontend', }, }) `	`jsx import { render, Component } from 'preact' class MyComp extends Component { render() { return ( Hello World! ) } } render(MyComp, document.body) `
Using the simple configuration from above, and a JSX file, the browser will receive the following patched source code. The middleware will also look for requests that start with the /node_modules path, and serve them also. The pragma (import { h } from 'preact') is also added automatically, but it can be configured.
`js import { h } from '/node_modules/preact/dist/preact.mjs' import { render, Component } from '/node_modules/preact/dist/preact.mjs' class MyComp extends Component { render() { return (h('div',{className:"example"}, Hello World! )) } } render(MyComp, document.body) `

The idea here is to provide a basic mechanism to serve front-end JavaScript code, without inventing any module systems, adapting to _CommonJS_, or transpiling old features. We simply want to execute our modern code and browsers are more than capable to do that, without us having to run complex build systems on the development code. Our simple JSX parser is not rocket science either and works perfectly well without building ASTs (but check for minor limitations in Wiki).





Additional Middleware

There are some small bits of middleware that can be used in server as well, but which are not essential to its functioning. They are listed in 📖 Wiki.

- csrfCheck: Ensures that the csrf token from session matches one in the request.
- jsonErrors: Allows to serve errors as _JSON_, which is useful for APIs.
- jsonBody: Parses requests with the application/json content type into ctx.request.body.
- logarithm: Record hits in _ElasticSearch_.
- github: Sets up _GitHub_ OAuth routes.





Custom Middleware

When required to add any other middleware in the application not included in the _Idio_ bundle, it can be done in several ways.

1. Passing the middleware function as part of the server.">_MiddlewareConfig_. It will be automatically installed to be used by the _Application_. All middleware will be installed in order it is found in the _MiddlewareConfig_.
`js
import idio from '@idio/idio'

const APIServer = async (port) => {
const { url } = await idio({
// 1. Add logging middleware.
async log(ctx, next) {
await next()
console.log(' --> API: %s %s %s', ctx.method, ctx.url, ctx.status)
},
// 2. Add always used error middleware.
async error(ctx, next) {
try {
await next()
} catch (err) {
ctx.status = 403
ctx.body = err.message
}
},
// 3. Add validation middleware.
async validateKey(ctx, next) {
if (ctx.query.key !== 'app-secret')
throw new Error('Wrong API key.')
ctx.body = 'ok'
await next()
},
}, { port })
return url
}

export default APIServer
`
`
Started API server at: http://localhost:5005
--> API: GET / 403
--> API: GET /?key=app-secret 200
`
2. Passing a configuration object as part of the _MiddlewareConfig_ that includes the middlewareConstructor property which will receive the reference to the app. Other properties such as conf and use will be used in the same way as when setting up bundled middleware: setting use to true will result in the middleware being used for every request, and the config will be passed to the constructor.
`js
import rqt from 'rqt'
import idio from '@idio/idio'
import APIServer from './api-server'

const ProxyServer = async (port) => {
// 1. Start the API server.
const API = await APIServer(5001)
console.log('API server started at %s', API)

// 2. Start a proxy server to the API.
const { url } = await idio({
async log(ctx, next) {
await next()
console.log(' --> Proxy: %s %s %s', ctx.method, ctx.url, ctx.status)
},
api: {
use: true,
async middlewareConstructor(app, config) {
// e.g., read from a virtual network
app.context.SECRET = await Promise.resolve('app-secret')

/* @type {import('@typedefs/goa').Middleware} /
const fn = async (ctx, next) => {
const { path } = ctx
const res = await rqt(${config.API}${path}?key=${ctx.SECRET})
ctx.body = res
await next()
}
return fn
},
config: {
API,
},
},
}, { port })
return url
}
`
`
API server started at http://localhost:5001
Proxy started at http://localhost:5002
--> API: GET /?key=app-secret 200
--> Proxy: GET / 200
`





Router Set-up

After the _Application_ and _Router_ instances are obtained after starting the server as the app and router properties of the returned object, the router can be configured to respond to custom paths. This can be done by assigning configured middleware from the map and standalone middleware, and calling the use method on the _Application_ instance.

`js
import { collect } from 'catchment'
import idio from '@idio/idio'

const Server = async () => {
const {
url, router, app, middleware: { pre, post, bodyparser },
} = await idio({
// 1. Configure middlewares via middlewareConstructor without installing them.
pre: {
middlewareConstructor() {
return async function(ctx, next) {
console.log(' <-- %s %s',
ctx.request.method,
ctx.request.path,
)
await next()
}
},
},
post: {
middlewareConstructor() {
return async function(ctx, next) {
console.log(' --> %s %s %s',
ctx.request.method,
ctx.request.path,
ctx.response.status,
)
await next()
}
},
},
bodyparser: {
middlewareConstructor() {
return async (ctx, next) => {
let body = await collect(ctx.req)
if (ctx.is('application/json')) {
body = JSON.parse(body)
}
ctx.request.body = body
await next()
}
},
},
}, { port: 5003 })

// 2. Setup router with the bodyparser and path-specific middleware.
router.post('/example',
pre,
bodyparser,
async (ctx, next) => {
ctx.body = {
ok: true,
request: ctx.request.body,
}
await next()
},
post,
)
app.use(router.routes())
return url
}
`







Logging	Response
` Page available at: http://localhost:5003 <-- POST /example --> POST /example 200 `	`js // server response: { ok: true, request: { hello: 'world' } } `

Also checkout the _Router_ package that allows to automatically initialise routes from a given directory, and watch for changes in them during development. This means you don't have to refresh the server manually after a change to a route.

`js
const w = await initRoutes(router, 'routes', {
middleware,
})
if (process.env.NODE_ENV == 'prod') watchRoutes(w)
`





SSR

_Idio_ supports Server-Side rendering of JSX components (same restrictions apply as for front-end). You can easily mark up your back-end pages using full-scale HTML, or basic placeholders in which you can then render your front-end app.

`jsx
import idio, { render } from '@idio/idio'

const { url, app, router } = await idio()
router.get('/', (ctx) => {
ctx.body = render(




Hello World!

, {
addDoctype: true,
pretty: true,
})
})
app.use(router.routes())
`

`html



Hello World!

`





NeoLuddite.Dev

This web server integrates with NeoLuddite: the package monetary reward scheme. It's currently in beta, and this section will be relevant when it's open to the public.

Every time you invoke certain functionality in a package somebody has written (e.g., koa-static for static files, koa-session for creation of session), via _Idio_, your usage will be counted and your balance in _Ludds_ on the neoluddite server will be transferred to the software engineer as a reward for his/her intellectual work. Contact license@neoluddite.dev for any requests.

`js
const { url, app,
middleware: { session, form },
router,
} = await idio({
// Developers' payment scheme neoluddite.dev
neoluddite: {
env: process.env.NODE_ENV,
key: '0799b7f0-d2c7-4903-a541-10d8092c2911',
app: 'idio.example',
},
// ...
}
`

The usage will be billed for apps running in production mode, therefore the env variable is needed. Setting the app has no effect but allows to break down statistics by web application on the portal. See the license section for more info.

__NeoLudditeOptions__: Options for the neoluddite.dev client.


| Name | Type | Description | Default |
| -------- | --------------- | -------------------------------------------------------------------------------------------------------- | ------------------------ |
| __key*__ | string | The API key received from the app. | - |
| env | string | The environment (e.g., dev/staging). The production env must be indicated as prod which is billed. | - |
| host | string | The hostname of the server. | https://neoluddite.dev |
| app | string | The name of the application. | - |





WebSocket

We've implemented a library to upgrade requests into _WebSocket_ connections. You can read more at the actual package page. _Idio_ simply exports this method via its API. You need to configure it yourself.

`js
import idio, { websocket } from '@idio/idio'

const { url, app, server } = await idio()
// clients stores current connections against ID
const clients = websocket(server, {
onConnect(clientId) {
// the value is a function to send messages
clientsclientId
},
})
`





Copyright & License

GNU Affero General Public License v3.0

Affero GPL means that you're not allowed to use this web server on the web unless you release the source code for your application. This is a restrictive license which has the purpose of defending Open Source work and its creators.

To be able to use the server, just set up a monthly payment on Open Collective for any amount of your choice.

All original work on middleware and _Koa_ are under MIT license. See Goa Page for the list of packages and modules used in compilation of the Goa server, and the package.json` file for dependencies of this project (todo: create wiki page w/ licenses table).

alt="Art Deco">	© Art Deco™ for Idio 2020