Run YUI tests for node packages using npm test with support for saving test results, coverage and more. Supports YUITest as well as YUI3 style tests.
npm install ytestrunnerytestrunner
===========
Run YUI tests for node packages using "npm test" with support for saving test results, coverage and more. Supports YUITest as well as YUI3 style tests.
To use this module, declare it as a devDependency along with the YUI module that you are using for tests (yui for yui3 style tests or yuitest for standalone yui tests).
Usage
-----
* Write some code (this module assumes, by default, that code will found at the source root and under lib/)
* Write tests using a supported format (this module assumes, by default, that tests will be found test/)
* Modify your package.json as follows
For tests that are implemented using yuitest:
{
"devDependencies": {
"ytestrunner": "*",
"yuitest": "*"
},
"scripts": {
"test": "ytestrunner
}
}
For tests in YUI3 module format that are implemented using the Y.Test object from yui use:
{
"devDependencies": {
"ytestrunner": "*",
"yui": "*"
},
"scripts": {
"test": "ytestrunner --yui3
}
}
* To run your tests, just type:
$ npm test
in the folder that has your package.json file
The ytestrunner script will run all tests and store results and coverage, if requested.
Command line options
---------------------
Usage: ytestrunner
Basic options
--root
Tests and results
--include
(default: [test//.js, tests//.js])
--exclude
(default: [ empty ])
--save-results : save results to disk
--results-format
--results-file
(default: ./results)
Instrumentation and coverage
--coverage, -c : enable instrumentation and coverage stats
--fast-cover : skip source coverage if a newer instrumented file found from previous run
--cov-include
(default: [.js, lib//.js])
--cov-exclude
(default: [/., /node_modules/*])
--save-coverage : save JSON coverage information to disk
--coverage-file
(default: ./coverage/test-coverage)
--coverage-report-format
In addition, the environment variable ytestopts can be set with override options. These take precedence over the options passed to command line.
Using this module effectively
-----------------------------
The module is designed so that it can provide test results and coverage information without having to write any files under your source tree. Saving results and coverage information to disk must be asked for explicitly.
This allows you to have a simple command in your package.json (e.g. ytestrunner or ytestrunner -c for default coverage) and yet have an automated build set the ytestopts environment variable to control writing of results and coverage in supported formats.
Use of ytestopts is also useful during development. If you want to just run one test, say, foo.test.js and instrument only one file, say, foo.js you could set ytestopts thus:
$ export ytestopts="--include=test/foo.test.js --cov-include=lib/foo.js"
$ npm test
This strategy requires no changes to package.json and yet will only instrument your source under test and run only one test file.
Also, keep in mind that boolean options can be negated. So, if your test script says ytestrunner -c and you are not interested in coverage, you could set ytestopts to --no-coverage to disable coverage.
Examples
--------
See the strawman examples in the directories yuitest-sample and yui3-test-sample. Run npm install and npm test in either of those directories.
A word on code coverage
-----------------------
Code coverage is implemented as follows:
* Create an instrumented version of all JS files matching the coverage filters somewhere under /tmp. This is done using the yuitest-coverage java module.
* Hook the module loader to serve the instrumented content at require time when the original file is requested. This step is skipped for a file if the coverage module reports an error when instrumenting it.
* Collect stats from the global coverage object created by the instrumented code at the end of all tests
* Save coverage information and write reports if requested using the yuitest-coverage-report java module
Debugging test execution
------------------------
The -v option is your friend.
Caveats and known issues
------------------------
This module must not be globally installed. Even though it uses a YUI test library as part of its runtime execution, it does not* declare dependencies on these. The caller is responsible for declaring this module as well as the test library used as devDependencies. This strategy ensures that this module uses the same instance of the testing library that your tests use.
For the same reason, this module may not* be npm-linked into your package either, since linked modules cannot "see" other dependencies under your node_modules/ folder.
* A modern java is required for instrumentation and coverage reports.
Since instrumentation is done by transparently substituting the instrumented file for the original at require time, stack traces will refer to original file but the line numbers in the instrumented file and will not* make sense when code coverage is turned on
License
-------
ytestrunner is licensed under the MIT License.
Third-party libraries
---------------------
The following third-party libraries are used by this module:
* async: https://github.com/caolan/async
* colors: https://github.com/Marak/colors.js
* fileset: https://github.com/mklabs/node-fileset
* mkdirp: https://github.com/substack/node-mkdirp
* nopt: https://github.com/isaacs/nopt
* resolve: https://github.com/substack/node-resolve
* nodeunit: https://github.com/caolan/nodeunit (dev dependency for unit tests of this module)
and enables running tests supported by these libraries:
* YUITest: https://github.com/yui/yuitest
* YUI: https://github.com/yui/
Credits
-------
* mfncooper - for ideas on module loader hooks to support code coverage
* baechul - for ideas on standard test running using Y.Test.Runner
* nzakas - test loading strategy for YUITest tests heavily influenced by command line support in that module
* davglass - for review of this module