I have been playing around with a number of open source projects pertaining to testing different aspects of a web based application. Over the past few days i have been playing with PhantomJS, Mocha, and Chai.
What is PhantomJS?
PhantomJS is a full stack headless web browser based on Webkit. That means it uses the same browser engine as many of the top browsers including Chrome and Safari.
ZombieJS was the other option that I considered. The difference is that Zombie works with JSDOM a javascript implementation of the DOM.
I opted to use PhantomJS because Zombie is not a particularly stable product (in my opinion), and having tested both the 1.4.1 version and the alpha 2.0.0 version I encountered a number of issues. The biggest problem for me was that it did not work very well with my complex shimming of externally loaded Javascript files, nor did it play nicely with ReactJS.
The other obvious consideration is that I believe tests should be run in as real an environment as possible.
PhantomJS is very good - it is easy to install and setup, but the documentation is a little sparse.
What is Mocha?
Mocha is a javascript testing suite that can be used with node OR in the browser. I want to use it in the browser, my browser being a PhantomJS browser instance.
Mocha allows you to hook in at various points to make sure for example that the necessary setup is complete before running your tests. It also has a really nice way of dealing with code which executes asynchronously. It is actively developed and has a good community around it.
before(function(done) {
//run asynchronous setup
//tell mocha when you are done
done();
});
//test code
What is Chai?
Chai is an assertion library. It essentially provides methods that can be used to assert that what you get is what you expect. Mocha and Chai work extremely well together.
I want to use Chain because it is extremely readable (BDD constructs) and is extremely well documented.
expect(resultCount).to.be.above(0);
Combining the three
This is where my adventure got a little tougher.
I want to essentially load a webpage (the page under test), execute a number of commands, and then assert that they did what I expected.
It is simple enough* to create a PhantomJS browser instance and load a page but how does one then load both Mocha and Chai and manipulate the page in a testable way?
Whereas when using node you can simply require() dependencies, because we are using phantomJS from the command line we cannot.
There is a phantomJS runner available called mocha-phantomjs however I found it to be somewhat constraining. You can call a file which contains the code you want to test and the libraries you want to use to test.. which are then run. I can see this being useful for unit testing, but I want to test an already bult page without needing to adapt it for testing. It essentially takes control of the browser (PhantomJS) piece of the puzzle which in may case is unsuitable.
My approach
PhantomJS has a webpage module that has an injectJs() method. I chose to utilize this to inject my test code (and all its requirements) into my page under test. What this means is that I can utilize jQuery (which is already loaded on my page) to manipulate the DOM and access the elements, properties, and values that i want to test.
PhantomJS also provides a method on the client side, callPhantom(). This allows you to callback to the Phantom instance where it triggers the callback that you setup on page.onCallback().
As such my approach is to:
- Run a PhantomJS browser instance and load the page I want to test
- Inject my tests
- Run my tests using Mocha and Chai
- Pass the formatted response back to PhantomJS
- Output the results on the command line.
Execution
Given the above, my execution is as follows:
var page = require("webpage").create();
var args = require('system').args;
//pass in the name of the file that contains your tests
var testFile = args[1];
//pass in the url you are testing
var pageAddress = args[2];
if (typeof testFile === 'undefined') {
console.error("Did not specify a test file");
phantom.exit();
}
page.open(pageAddress, function(status) {
if (status !== 'success') {
console.error("Failed to open", page.frameUrl);
phantom.exit();
}
//Inject mocha and chai page.injectJs("../node_modules/mocha/mocha.js");
page.injectJs("../node_modules/chai/chai.js");
//inject your test reporter
page.injectJs("mocha/reporter.js");
//inject your tests
page.injectJs("mocha/" + testFile);
page.evaluate(function() {
window.mocha.run();
});
});
page.onCallback = function(data) {
data.message && console.log(data.message);
data.exit && phantom.exit();
};
page.onConsoleMessage = function(msg, lineNum, sourceId) {
console.log('CONSOLE: ' + msg + ' (from line #' + lineNum + ' in "' + sourceId + '")');
};
The only bit of the above code that i have yet to explain is reporters. Mocha provides a number of reporters for formatting your test results. Because of the nature of this setup you cannot simply use Mocha's reporters - you have to build your own. This is one benefit of mocha-phantomjs (see above) in that the author has successfully ported over the reporters for you to use.
My basic implementation of a reporter is as follows:
(function() {
var color = Mocha.reporters.Base.color;
function log() {
var args = Array.apply(null, arguments);
if (window.callPhantom) {
window.callPhantom({ message: args.join(" ") });
} else {
console.log( args.join(" ") );
}
}
var Reporter = function(runner){
Mocha.reporters.Base.call(this, runner);
var out = [];
var stats = { suites: 0, tests: 0, passes: 0, pending: 0, failures: 0 }
runner.on('start', function() {
stats.start = new Date;
out.push([ "Testing", window.location.href, "\n"]);
});
runner.on('suite', function(suite) {
stats.suites++;
out.push([suite.title, "\n"]);
});
runner.on('test', function(suite) {
stats.tests++;
});
runner.on("pass", function(test) {
stats.passes++;
if ('fast' == test.speed) {
out.push([ color('checkmark', ' ✓ '), test.title, "\n" ]);
} else {
out.push([
color('checkmark', ' ✓ '),
test.title,
color(test.speed, test.duration + "ms"),
'\n'
]);
}
});
runner.on('fail', function(test, err) {
stats.failures++;
out.push([ color('fail', ' × '), color('fail', test.title), ":\n ", err ,"\n"]);
});
runner.on("end", function() {
out.push(['ending']);
stats.end = new Date;
stats.duration = new Date - stats.start;
out.push([stats.tests, "tests ran in", stats.duration, "ms"]);
out.push([ color('checkmark', stats.passes), "passed and", color('fail', stats.failures), "failed"]);
while (out.length) {
log.apply(null, out.shift());
}
if (window.callPhantom) {
window.callPhantom({ exit: true });
}
});
};
mocha.setup({
ui: 'bdd',
ignoreLeaks: true,
reporter: Reporter
});
}());
Issues
When I was playing with ZombieJS, my usage of React caused a number of issues. In my mind this was understandable - given how React works with the virtual DOM etc, I kind of figured that a javascript DOM implementation may have problems with it.
There was however an issue using React with PhantomJS. This is outlined in detail here - you just need to polyfill the bind method. This occurs because PhantomJS is using an old version of WebKit. PhantomJS 2.0 will be coming at some point, and this will resolve this issue. This update (when it comes) may change callPhantom() (discussed above) as the documentation outlines that it is an experimental API.
Soo..
Hopefully you find the above helpful. I'd be interested to hear peoples thoughts on this approach as well as any suggestions people may have for improvements.
