By Mark Levison


2008-11-19 02:35:49 8 Comments

I've looked into and considered many JavaScript unit tests and testing tools, but have been unable to find a suitable option to remain fully TDD compliant. So, is there a JavaScript unit test tool that is fully TDD compliant?

13 comments

@gregers 2009-03-25 08:44:55

Karma or Protractor

Karma is a JavaScript test-runner built with Node.js and meant for unit testing.

The Protractor is for end-to-end testing and uses Selenium Web Driver to drive tests.

Both have been made by the Angular team. You can use any assertion-library you want with either.

Screencast: Karma Getting started

related:

pros:

  • Uses node.js, so compatible with Win/OS X/Linux
  • Run tests from a browser or headless with PhantomJS
  • Run on multiple clients at once
  • Option to launch, capture, and automatically shut down browsers
  • Option to run server/clients on development computer or separately
  • Run tests from a command line (can be integrated into ant/maven)
  • Write tests xUnit or BDD style
  • Supports multiple JavaScript test frameworks
  • Auto-run tests on save
  • Proxies requests cross-domain
  • Possible to customize:
    • Extend it to wrap other test-frameworks (Jasmine, Mocha, QUnit built-in)
    • Your own assertions/refutes
    • Reporters
    • Browser Launchers
  • Plugin for WebStorm
  • Supported by Netbeans IDE

Cons:

mocha.js

I'm totally unqualified to comment on mocha.js's features, strengths, and weaknesses, but it was just recommended to me by someone I trust in the JS community.

List of features, as reported by its website:

  • browser support
  • simple async support, including promises
  • test coverage reporting
  • string diff support
  • javascript # API for running tests
  • proper exit status for CI support etc
  • auto-detects and disables coloring for non-ttys
  • maps uncaught exceptions to the correct test case
  • async test timeout support
  • test-specific timeouts
  • growl notification support
  • reports test durations
  • highlights slow tests
  • file watcher support
  • global variable leak detection
  • optionally run tests that match a regexp
  • auto-exit to prevent "hanging" with an active loop
  • easily meta-generate suites & test-cases
  • mocha.opts file support
  • clickable suite titles to filter test execution
  • node debugger support
  • detects multiple calls to done()
  • use any assertion library you want
  • extensible reporting, bundled with 9+ reporters
  • extensible test DSLs or "interfaces"
  • before, after, before each, after each hook
  • arbitrary transpiler support (coffee-script etc)
  • TextMate bundle

yolpo

yolpo

This no longer exists, redirects to sequential.js instead

Yolpo is a tool to visualize the execution of javascript. Javascript API developers are encouraged to write their use cases to show and tell their API. Such use cases forms the basis of regression tests.

AVA

AVA logo

Futuristic test runner with built-in support for ES2015. Even though JavaScript is single-threaded, IO in Node.js can happen in parallel due to its async nature. AVA takes advantage of this and runs your tests concurrently, which is especially beneficial for IO heavy tests. In addition, test files are run in parallel as separate processes, giving you even better performance and an isolated environment for each test file.

  • Minimal and fast
  • Simple test syntax
  • Runs tests concurrently
  • Enforces writing atomic tests
  • No implicit globals
  • Isolated environment for each test file
  • Write your tests in ES2015
  • Promise support
  • Generator function support
  • Async function support
  • Observable support
  • Enhanced asserts
  • Optional TAP o utput
  • Clean stack traces

Buster.js

A JavaScript test-runner built with Node.js. Very modular and flexible. It comes with its own assertion library, but you can add your own if you like. The assertions library is decoupled, so you can also use it with other test-runners. Instead of using assert(!...) or expect(...).not..., it uses refute(...) which is a nice twist imho.

A browser JavaScript testing toolkit. It does browser testing with browser automation (think JsTestDriver), QUnit style static HTML page testing, testing in headless browsers (PhantomJS, jsdom, ...), and more. Take a look at the overview!

A Node.js testing toolkit. You get the same test case library, assertion library, etc. This is also great for hybrid browser and Node.js code. Write your test case with Buster.JS and run it both in Node.js and in a real browser.

Screencast: Buster.js Getting started (2:45)

pros:

  • Uses node.js, so compatible with Win/OS X/Linux
  • Run tests from a browser or headless with PhantomJS (soon)
  • Run on multiple clients at once
  • Supports NodeJS testing
  • Don't need to run server/clients on development computer (no need for IE)
  • Run tests from a command line (can be integrated into ant/maven)
  • Write tests xUnit or BDD style
  • Supports multiple JavaScript test frameworks
  • Defer tests instead of commenting them out
  • SinonJS built-in
  • Auto-run tests on save
  • Proxies requests cross-domain
  • Possible to customize:
    • Extend it to wrap other test-frameworks (JsTestDriver built in)
    • Your own assertions/refutes
    • Reporters (xUnit XML, traditional dots, specification, tap, TeamCity and more built-in)
    • Customize/replace the HTML that is used to run the browser-tests
  • TextMate and Emacs integration

Cons:

  • Stil in beta so can be buggy
  • No plugin for Eclipse/IntelliJ (yet)
  • Doesn't group results by os/browser/version like TestSwarm *. It does, however, print out the browser name and version in the test results.
  • No history of previous test results like TestSwarm *
  • Doesn't fully work on windows as of May 2014

* TestSwarm is also a Continuous Integration server, while you need a separate CI server for Buster.js. It does, however, output xUnit XML reports, so it should be easy to integrate with Hudson, Bamboo or other CI servers.

TestSwarm

https://github.com/jquery/testswarm

TestSwarm is officially no longer under active development as stated on their GitHub webpage. They recommend Karma, browserstack-runner, or Intern.

Jasmine

Jasmine

This is a behavior-driven framework (as stated in quote below) that might interest developers familiar with Ruby or Ruby on Rails. The syntax is based on RSpec that are used for testing in Rails projects.

Jasmine specs can be run from an html page (in qUnit fashion) or from a test runner (as Karma).

Jasmine is a behavior-driven development framework for testing your JavaScript code. It does not depend on any other JavaScript frameworks. It does not require a DOM.

If you have experience with this testing framework, please contribute with more info :)

Project home: http://jasmine.github.io/

QUnit

QUnit focuses on testing JavaScript in the browser while providing as much convenience to the developer as possible. Blurb from the site:

QUnit is a powerful, easy-to-use JavaScript unit test suite. It's used by the jQuery, jQuery UI, and jQuery Mobile projects and is capable of testing any generic JavaScript code

QUnit shares some history with TestSwarm (above):

QUnit was originally developed by John Resig as part of jQuery. In 2008 it got its own home, name and API documentation, allowing others to use it for their unit testing as well. At the time it still depended on jQuery. A rewrite in 2009 fixed that, now QUnit runs completely standalone. QUnit's assertion methods follow the CommonJS Unit Testing specification, which was to some degree influenced by QUnit.

Project home: http://qunitjs.com/

Sinon

Another great tool is sinon.js by Christian Johansen, the author of Test-Driven JavaScript Development. Best described by himself:

Standalone test spies, stubs and mocks for JavaScript. No dependencies works with any unit testing framework.

Intern

The Intern Web site provides a direct feature comparison to the other testing frameworks on this list. It offers more features out of the box than any other JavaScript-based testing system.

JEST

A new but yet very powerful testing framework. It allows snapshot based testing as well this increases the testing speed and creates a new dynamic in terms of testing

Check out one of their talks: https://www.youtube.com/watch?v=cAKYQpTC7MA

Better yet: Getting Started

@user181548 2011-03-28 18:00:20

@gregers: no, it was down when I wrote that comment and it is still down now. downforeveryoneorjustme.com/http://testswarm.com

@gregers 2011-09-01 08:30:15

@broofa Thanks for the heads up, but that's the TestSwarm instance for jQuery. The GitHub project is more useful for those who want to set up TestSwarm for their own project.

@Merlyn Morgan-Graham 2011-12-16 02:43:14

Jasmine image is broken.

@jerseyboy 2012-01-15 06:19:45

Jasmine can work headless using V8, but you can also use it interactively. While the DOM is not necessary with respect to Jasmine, your codebase might access DOM. With discipline it is possible to eliminate, guard with conditions, or provide mocks for parts of the code that access the DOM and run tests completely apart from HTML fixtures. You can also get command-line support and fixtures using add-ons.

@gregers 2012-06-14 11:58:07

@rehevkor5: Selenium is for integration testing, while the tools here are for unit testing. typemock.com/unit-tests-integration-tests

@Lèse majesté 2012-07-02 10:35:45

@gregers: I thought Selenium was for browser automation/test automation. It's certainly useful for integration testing, but there's a use case for unit testing as well.

@Jules 2012-10-15 07:58:26

Jasmine: - behavior-driven development framework - It does not require a DOM - Uses HTML Unit for headless browser runner - HTML Units only supports Firefox and IE

@davidjnelson 2013-02-16 19:54:44

I just released an open source toolkit which makes using jstestdriver easy. It is a composition of many open source tools which gives you a working requirejs backbone app out of the box. It provides single commands to run: dev web server, jasmine single browser test runner, jasmine js-test-driver multi browser test runner, and concatenization/minification for JavaScript and CSS. It also outputs an unminified version of your app for production debugging, precompiles your handlebar templates, and supports i18n. No setup is required. It just works. github.com/davidjnelson/agilejs

@Shannon 2013-06-26 18:11:33

@gregers What I was getting at is that they both allow you to write once, run everywhere (on a variety of browsers).

@Camilo Martin 2013-07-12 21:45:40

By the way, the book Test-Driven JavaScript Development was written before Sinon (I guess) as it contains no info on it, and instead uses JsTestDriver throughout the book.

@Rudolf Olah 2014-01-19 19:41:03

Almost every single test runner relies on a browser. Wtf, doesn't anyone ever run unit tests only on the server-side????

@Sam Backus 2014-04-22 21:21:38

I'm currently using teaspoon on an ember project. It's great. github.com/modeset/teaspoon

@Dale Anderson 2014-05-18 01:44:15

test-studio is an npm package that provides a powerful, web based front end for unit testing. It supports things like executing individual or groups of tests and stepping node-inspector into individual tests. It doesn't yet support browser tests, but that is coming. Read more about it at npmjs.org/package/test-studio and danderson00.com/2014/05/….

@hellboy 2015-03-17 12:40:25

and what about cucumber.js ?

@cregox 2015-03-31 23:30:46

Wouldn't it be better to split / divide each alternative in different answers? It might invalidate the current votes on this one, but I think it'd make the most sense.

@Raisen 2015-12-08 08:07:18

Any unit testing framework that is using ECMAScript 6 features?

@gregers 2015-12-08 08:13:40

@Raisen You can plug ES 2015 into most of them with Babel, but AVA by Sindre Sorhus has it built in.

@kolen 2011-10-05 10:59:54

google-js-test:

JavaScript testing framework released by Google: https://github.com/google/gjstest

  • Extremely fast test startup and execution time, without having to run a browser.
  • Clean, readable output in the case of both passing and failing tests.
  • A browser-based test runner that can simply be refreshed whenever JS is changed.
  • Style and semantics that resemble Google Test for C++.
  • A built-in mocking framework that requires minimal boilerplate code (e.g. no $tearDown or $verifyAll) with style and semantics based on the Google C++ Mocking Framework.

There are currently no binaries for Windows

@vsync 2015-11-07 22:58:40

It seems to have almost zero interest on Github, also it requires unix-bases OS, and I'm a huge windows fan, I don't leave my house without kissing my windows machine goodbye.

@Tauren 2012-02-08 21:52:15

BusterJS

There is also BusterJS from Christian Johansen, the author of Test Driven Javascript Development and the Sinon framework. From the site:

Buster.JS is a new JavaScript testing framework. It does browser testing by automating test runs in actual browsers (think JsTestDriver), as well as Node.js testing.

@Matthew Manela 2011-06-24 18:44:32

Chutzpah - A JavaScript Test Runner

I created an open source project called Chutzpah which is a test runner for JavaScript unit tests. Chutzpah enables you to run JavaScript unit tests from the command line and from inside of Visual Studio. It also supports running in the TeamCity continuous integration server.

@GarethOwen 2012-03-07 16:00:32

I just started using Chutzpah to run Jasmine tests inside visual studio - it's nicely integrated: right click in the test file and chose 'run js tests' or 'run JS tests in browser'. I run the same jasmine tests using JSTestDriver. I prefer Chutzpah because I specify which files I depend upon being loaded at the top of the test file. For JSTestDriver I need a separate config file.

@Tom Stickel 2011-12-08 20:33:29

We are now using Qunit with Pavlov and JSTestDriver all together. This approach works well for us.

QUnit

Pavlov, source

jsTestDriver, source

@vsync 2015-11-07 23:01:23

Would you care to explain what is the role in each of these in the whole testing process and how they connect with each other?

@Tom Stickel 2015-11-08 07:15:05

Sorry it has been a long time and many contract jobs ago to recall the details on this.

@alex.c 2012-01-10 21:57:35

We added JUnit integration to our Java to Javascript code generator ST-JS (http://st-js.org). The framework generates to corresponding Javascript for both the tested code and the unit tests and sends the code to different browsers.

There is no need for a separate server as the unit test runner opens the needed http port (and closes it once the tests finished). The framework manipulates the Java stacktrace so that the failed asserts are correctly displayed by the JUnit Eclipse plugin. Here is a simple example with jQuery and Mockjax:

@RunWith(STJSTestDriverRunner.class)
@HTMLFixture("<div id='fortune'></div>")

@Scripts({ "classpath://jquery.js",
       "classpath://jquery.mockjax.js", "classpath://json2.js" })
public class MockjaxExampleTest {
  @Test
  public void myTest() {
    $.ajaxSetup($map("async", false));
    $.mockjax(new MockjaxOptions() {
      {
        url = "/restful/fortune";
        responseText = new Fortune() {
          {
            status = "success";
            fortune = "Are you a turtle?";
          }
        };
      }
    });

    $.getJSON("/restful/fortune", null, new Callback3<Fortune, String, JQueryXHR>() {
      @Override
      public void $invoke(Fortune response, String p2, JQueryXHR p3) {
        if (response.status.equals("success")) {
          $("#fortune").html("Your fortune is: " + response.fortune);
        } else {
          $("#fortune").html("Things do not look good, no fortune was told");
        }

      }
    });
    assertEquals("Your fortune is: Are you a turtle?", $("#fortune").html());
  }

  private static class Fortune {
    public String status;
    public String fortune;
  }
}

@Matthew Taylor 2009-01-22 14:53:14

YUI has a testing framework as well. This video from Yahoo! Theater is a nice introduction, although there are a lot of basics about TDD up front.

This framework is generic and can be run against any JavaScript or JS library.

@Peter V. Mørch 2014-10-24 07:04:57

@Eugene Lazutkin 2008-11-19 03:32:01

Take a look at the Dojo Object Harness (DOH) unit test framework which is pretty much framework independent harness for JavaScript unit testing and doesn't have any Dojo dependencies. There is a very good description of it at Unit testing Web 2.0 applications using the Dojo Objective Harness.

If you want to automate the UI testing (a sore point of many developers) — check out doh.robot (temporary down. update: other link http://dojotoolkit.org/reference-guide/util/dohrobot.html ) and dijit.robotx (temporary down). The latter is designed for an acceptance testing. Update:

Referenced articles explain how to use them, how to emulate a user interacting with your UI using mouse and/or keyboard, and how to record a testing session, so you can "play" it later automatically.

@Mark Levison 2008-11-19 15:06:10

Thanks for the suggestion of Dojo Object Harness, I never would have found it. I appreciate the other suggestions - but one step at a time.

@Rakesh Pai 2008-11-19 17:20:21

I've actually used this in a previous project, and found it invaluable. But then again, I can't compare - haven't used any other TDD framework.

@Eugene Lazutkin 2011-03-25 18:01:17

Thanks for reporting dead links. I updated one of them, and will replace links to robots docs as soon as I they are up on a new web site.

@Aram Kocharyan 2014-01-01 08:08:22

One thing I don't like about DOH is that line numbers aren't reported when assertions fail. Commenting them out manually and re-running the test works.

@user64141 2014-11-20 22:15:04

Dojo is replacing DOH with TheIntern testing framework. TheIntern is very powerful and has substantial improvements. sitepen.com/blog/2014/02/18/…

@Fredrik 2008-11-19 10:16:08

MochiKit has a testing framework called SimpleTest that seems to have caught on. Here's a blog post from the original author.

@Mark Levison 2008-11-19 15:05:40

SimpleTest - another great option. I'm glad I'm not doing the investigative work here ;-)

@Pete TerMaat 2009-06-01 18:07:42

The JavaScript section of the Wikipedia entry, List of Unit Testing Frameworks, provides a list of available choices. It indicates whether they work client-side, server-side, or both.

@Aaron Digulla 2008-12-01 10:33:33

You should have a look at env.js. See my blog for an example how to write unit tests with env.js.

@Andreas Ecker 2008-11-28 19:54:34

You might also be interested in the unit testing framework that is part of qooxdoo, an open source RIA framework similar to Dojo, ExtJS, etc. but with quite a comprehensive tool chain.

Try the online version of the testrunner. Hint: hit the gray arrow at the top left (should be made more obvious). It's a "play" button that runs the selected tests.

To find out more about the JS classes that let you define your unit tests, see the online API viewer.

For automated UI testing (based on Selenium RC), check out the Simulator project.

@Jeffrey Fredrick 2008-11-19 06:11:18

You have "runs on actual browser" as a pro, but in my experience that is a con because it is slow. But what makes it valuable is the lack of sufficient JS emulation from the non-browser alternatives. It could be that if your JS is complex enough that only an in browser test will suffice, but there are a couple more options to consider:

HtmlUnit: "It has fairly good JavaScript support (which is constantly improving) and is able to work even with quite complex AJAX libraries, simulating either Firefox or Internet Explorer depending on the configuration you want to use." If its emulation is good enough for your use then it will be much faster than driving a browser.

But maybe HtmlUnit has good enough JS support but you don't like Java? Then maybe:

Celerity: Watir API running on JRuby backed by HtmlUnit.

or similarly

Schnell: another JRuby wrapper of HtmlUnit.

Of course if HtmlUnit isn't good enough and you have to drive a browser then you might consider Watir to drive your JS.

Related Questions

Sponsored Content

39 Answered Questions

[SOLVED] For-each over an array in JavaScript?

93 Answered Questions

[SOLVED] How do I remove a particular element from an array in JavaScript?

  • 2011-04-23 22:17:18
  • Walker
  • 6263415 View
  • 7822 Score
  • 93 Answer
  • Tags:   javascript arrays

42 Answered Questions

[SOLVED] How do I remove a property from a JavaScript object?

49 Answered Questions

[SOLVED] How do I test for an empty JavaScript object?

52 Answered Questions

27 Answered Questions

[SOLVED] What does "use strict" do in JavaScript, and what is the reasoning behind it?

58 Answered Questions

[SOLVED] How do I include a JavaScript file in another JavaScript file?

86 Answered Questions

[SOLVED] How do JavaScript closures work?

4 Answered Questions

Sponsored Content