commoner

Commoner 
---

Commoner makes it easy to write scripts that flexibly and efficiently
transpile any dialect of JavaScript into a directory structure of
Node-compatible CommonJS module files.

This task is made possible by

1. a declarative syntax for defining how module source code should be
found and processed,
2. the use of promises to manage an
asynchronous build pipeline, and
3. never rebuilding modules that have already been built.

The output files can be required seamlessly by Node, or served by any
static file server, or bundled together using a tool such as
Browserify,
WrapUp, or
Stitch for delivery to a web
browser.

If you pass the --relativize option, Commoner also takes care to rewrite
all require calls to use relative module
identifiers,
so that the output files can be installed into any subdirectory of a
larger project, and external tools do not have to give special treatment
to top-level modules (or even know which modules are top-level and which
are nested).

Commoner was derived from an earlier, more opinionated tool called
Brigade that provided additional
support for packaging modules together into multiple non-overlapping
bundles. Commoner grew out of the realization that many tools already
exist for bundling CommonJS modules, but that fewer tools focus on getting
to that point.

Installation
---

From NPM:

npm install commoner

From GitHub:

cd path/to/node_modules
git clone git://github.com/reactjs/commoner.git
cd commoner
npm install .

Usage
---

Here's the output of bin/commonize --help:
``
Usage: commonize [options]

[ [ ...]]

Options:

-h, --help output usage information
-V, --version output the version number
-c, --config [file] JSON configuration file (no file means STDIN)
-w, --watch Continually rebuild
-x, --extension File extension to assume when resolving module identifiers
--relativize Rewrite all module identifiers to be relative
--follow-requires Scan modules for required dependencies
--cache-dir Alternate directory to use for disk cache
--no-cache-dir Disable the disk cache
--source-charset Charset of source (default: utf8)
--output-charset Charset of output (default: utf8)
`

In a single sentence: the commonize command finds modules with the given
module identifiers in the source directory and places a processed copy of
each module into the output directory, along with processed copies of all
required modules.

If you do not provide any module identifiers, commonize will process all
files that it can find under the source directory that have the preferred
file extension (.js by default). If your source files have a file
extension other than .js, use the -x or --extension option to
specify it. For example, --extension coffee to find .coffee files.

Output
---

Commoner prints various status messages to STDERR, so that you can see
what it's doing, or figure out why it's not doing what you thought it
would do.

The only information it prints to STDOUT is a JSON array of module
identifiers, which includes the identifiers passed on the command line and
all their dependencies. This array contains no duplicates.

Internally, each module that Commoner generates has a hash computed from
the module's identifier, source code, and processing steps. Since this
hash can be computed before processing takes place, Commoner is able to
avoid processing a module if it has ever previously processed the same
module in the same way.

If you dig into the
code,
you'll find that Commoner maintains a cache directory (by default,
~/.commoner/module-cache/) containing files with names like
9ffc5c853aac07bc106da1dc1b2486903ca688bf.js. When Commoner is about to
process a module, it checks its hash against the file names in this
directory. If no match is found, processing procedes and the resulting
file is written to the cache directory with a new hash. If the appropriate
hash file is already present in the cache directory, however, Commoner
merely creates a hard link between the hash file and a file with a more
meaningful name in the output directory.

When you pass the --watch flag to bin/commonize, Commoner avoids
exiting after the first build and instead watches for changes to
previously read files, printing a new JSON array of module identifiers to
STDOUT each time rebuilding finishes. Thanks to the caching of processed
modules, the time taken to rebuild is roughly proportional to the number
of modified files.

Customization
---

The bin/commonize script is actually quite simple, and you can write
similar scripts yourself. Let's have a look:
`js
#!/usr/bin/env node

require("commoner").resolve(function(id) {
var context = this;

return context.getProvidedP().then(function(idToPath) {
// If a module declares its own identifier using @providesModule
// then that identifier will be a key in the idToPath object.
if (idToPath.hasOwnProperty(id))
return context.readFileP(idToPath[id]);
});

}, function(id) {
// Otherwise assume the identifier maps directly to a filesystem path.
// The readModuleP method simply appends the preferred file extension
// (usually .js) to the given module identifier and opens that file.
return this.readModuleP(id);
});
`
The scriptable interface of the commoner module abstracts away many of
the annoyances of writing a command-line script. In particular, you don't
have to do any parsing of command-line arguments, and you don't have to
worry about installing any dependencies other than commoner in your
$NODE_PATH.

What you are responsible for, at a minimum, is telling Commoner how to
find the source of a module given a module identifier, and you do this by
passing callback functions to require("commoner").resolve. The script
above uses two strategies that will be tried in sequence: first, it calls
the helper function this.getProvidedP to retrieve an object mapping
identifiers to file paths (more about this below); and, if that doesn't
work, it falls back to interpreting the identifier as a path relative to
the source directory.

Now, you might not care about this.getProvidedP. It's really just a
proof of concept that Commoner can support modules that declare their own
identifiers using the // @providesModule syntax, and I
included it by default because it doesn't make a difference unless you
decide to use @providesModule. If you don't like it, you could write an
even simpler script:
`js
#!/usr/bin/env node

require("commoner").resolve(function(id) {
return this.readModuleP(id);
});
`
The point is, it's entirely up to you to define how module identifiers are
interpreted. In fact, the source you return doesn't even have to be valid
JavaScript. It could be CoffeeScript, or
LESS, or whatever language you prefer to write by
hand. Commoner doesn't care what your source code looks like, because
Commoner allows you to define arbitrary build steps to turn that source
code into plain old CommonJS.

Let's consider the example of using LESS to write dynamic CSS
modules. First, let's apply what we already know to give special meaning
to .less files:
`js
#!/usr/bin/env node

require("commoner").resolve(function(id) {
if (isLess(id))
return this.readFileP(id);
}, function(id) {
return this.readModuleP(id);
});

function isLess(id) {
return /\.less$/i.test(id);
}
`
All this really accomplishes is to avoid appending the .js file
extension to identifiers that already have the .less extension.

Now we need to make sure the contents of .less files somehow get
transformed into plain old CommonJS, and for that we need
require("commoner").process:
`js
require("commoner").resolve(function(id) {
if (isLess(id))
return this.readFileP(id);
}, function(id) {
return this.readModuleP(id);
}).process(function(id, source) {
if (isLess(id))
return compileLessToJs(source);
return source;
});
`
How should compileLessToJs be implemented? At a high level, I propose
that we generate a CommonJS module that will append a new