Wallaby.js Blog

Code Coverage Report: the Wallaby Way

2016-09-06T00:00:00+00:00

Many of you have used several different code coverage solutions and reports. In this blog post we will explore some of the essential features of a code coverage report solution and introduce the way we have implemented them in wallaby.js app.

File tree

First, a good JavaScript code coverage report should display an annotated project source code tree. Folders is a natural way to organise your project code into modules or subsystems, and sometimes one may only be interested in focusing on a specific part of the project when working with code coverage metrics.

Not many code coverage solutions bother to provide the tree, and it is sad, because the project file tree with node annotations is a great way to explore the code coverage for individual modules.

Here’s an example from JetBrains IDEs:

Sortable table

Second, a good JavaScript code coverage report should provide an option to sort source files by coverage value, also by a number of total/covered branches/lines/functions, whatever criteria of the code coverage is used. It helps to quickly identify what files need more coverage.

For example, let’s take a look at the Istanbul code coverage report:

We are able to sort all folders, including any nested ones, by coverage, and also open an individual folder and sort its files.

JetBrains IDEs provide a separate view, where you can sort by coverage in any selected folder, but for any nested subfolders you’ll only get an aggregated coverage and have to open it separately to see its files. It’s just like a file tree, but sortable on each individual level.

The issue here in both mentioned ways to present the data is that you can only sort files located in an opened folder, and for the files in its subfolders you can only see/sort by the aggregated values. So it’s not possible to easily display a few least covered files in a whole project or a specific module folder with a few subfolders.

The Wallaby Way

In the new wallaby.js app coverage report we have decided to address the above mentioned issues and to efficiently combine both annotated project file tree and a selected folder/module sortable list of files.

The idea is very simple - you select a folder, and you get the full sortable table of all files in the folder and its subfolders (with a relative path where required).

This way you can both browse the tree structure, see the aggregated coverage values, select a folder/module, and easily find the least covered files no matter how deep your folder structure is.

You can read more about Wallaby.js App and its other features in our introductory blog post.

That’s it for the post, thanks for reading! Follow us on Twitter to avoid missing new wallaby.js app features announcements.

Introducing Wallaby.js App: realtime bird's eye view of your project's tests connected to your editor

2016-07-19T00:00:00+00:00

When I think about tactics vs. strategy, one of my favourite video games always comes to mind - Heroes of Might and Magic (of course version III). It may sound like a strange opening line for a software development tool announcement, but bear with me.

It is recognised by many JavaScript developers that wallaby.js, as a plugin for your code editor, is the great tactical level tool. It increases your productivity by instantly providing the test results right where you need them just as you type your code. And like in a middle of the Heroes game battle you don’t care about you resources and castles and concentrate on your troops actions, when writing your code for a specific feature you don’t care much about how many tests you have and what’s the overall code coverage level.

However, when the battle is over and the feature’s tests are passing (and you maybe even levelled up), it’s often good to take a step back and get a bird’s eye view of the project. Since the very first release of wallaby.js, we have been getting requests to provide more strategic level features, such as ~~Castle View and World Map~~ Test Explorer or Code Coverage Reports.

To implement the requests and create a foundation for future strategic level features, some time ago we have started to work on something we call wallaby.js app and today I am happy to announce the release of wallaby.js app beta.

Now, instead of providing a long installation guide, what if I told you that you can just open a URL in your browser to start using the app right now? It’s as simple as that: wallabyjs.com/app.

Note that you need to have wallaby.js configured and running in your editor (core v1.0.257 or later) to use the most of the app.

For those of you who feel uncomfortable seeing the external URL, it’s worth mentioning that we just use it to host the app, which makes a connection to the locally running wallaby core. Wallaby.js doesn’t upload any of your project’s source code to the cloud. If you like, you may easily check it in your browser’s Dev Tool Network panel. We’ll also be providing an npm package to install wallaby.js app locally and fully run from the localhost.

Ok, let’s already check out what the app can do. In the first version we are releasing the Tests View that allows you to quickly get tests-centric overview and analytics for your project.

On the left hand side you can see the tree of your project tests. You may navigate it with the keyboard using arrow keys, plus Cmd/Ctrl (Shift) to expand a node or the whole tree. For each tree node you may see the execution time (aggregated for files and test suites) and an indicator if some of the tests are slow. A test is considered as slow if its execution time is greater than a configurable value. With mocha framework, the value can also be overriden by any describe or it using the this.slow function.

Before describing the rest, I’d like to mention that, despite being a strategic level tool, wallaby.js app is not a static report: almost everything gets updated automatically in realtime when you change your code, add or remove files, increase code coverage, etc.

For any file or test displayed anywhere in the app, you may open its location in your editor by clicking the editor icon (or the stack entry/file link), or pressing Cmd/Ctrl + Enter while in the tests tree.

However, sometimes you may not want to leave the app to view the code, for example because you are not planning to change it right now. In this case you may use the eye icon or press Enter for the selected tree node to open the source right inside the wallaby.js app.

As you can see, the displayed file view provides a similar experience that you have in wallaby.js plugin for your editor - you may see the coverage indicators, uncovered code regions and inline error/log messages.

While being able to view some file or a test under your mouse is great, sometimes you may want to open any file of your project. To do this, you may just use the magnifier icon next to the project name in the app header, or press Shift twice, enter the file name and view it.

While in the tests tree, you may select a test or a file to see its details. The details include errors or console.log messages logged while executing the test. If you select a test file, you’ll see the errors and log messages for all tests in the file.

The details of the selected test or test file also display the tree of the source files that are covered by the test. Each file in the tree contains its current code coverage value. If the value is too low, it is highlighted in yellow.

The overall project code coverage and other project stats are also displayed in the top right corner.

The Tests View by no means is a final version, let us know what you’d like to be added or changed, your feedback is very welcome. In the upcoming weeks, we are planning to add the Coverage View to provide file tree centric view of project code coverage, so stay tuned. You are also welcome to join the discussion and share your ideas about what else you’d like to see in the app.

That’s it for the post, thanks for reading! Follow us on Twitter to avoid missing new wallaby.js app features announcements.

Wallaby.js - Sublime Text integration release, Electron and AVA, Angular 2

2016-04-26T00:00:00+00:00

Almost 12 months ago we have started to sell first commercial licenses for wallaby.js, so on the 7th of May 2016 we are turning one year as a product. Like every business, we have had some ups and downs, but overall it’s been a great a year for us. We keep growing steadily, adding new features and improving the tool’s quality.

Some exciting and ambitious plans are prepared for the next year, some of them are very close for their first preview. But enough about the future plans, let’s see what we have already implemented in the last couple of months, and that you can use today.

One thing I’d like to mention before jumping to the new features, is our new License Upgrade and Renewal Calculator. Using the form you may quickly upgrade your existing license to ‘Freedom’ or renew your existing license. All you need to do is just to paste your existing license key(s) or an email with the keys, and the form will display pricing options tailored for your case.

Sublime Text integration

Many of you have been eagerly waiting so long for the integration. We have been brushing up the first preview version released earlier this year and finally Sublime Text integration beta program has ended earlier this month, so licenses are available for purchasing.

If you haven’t done it already, check out our tutorial and also make sure to upgrade to the latest version of the package in Sublime Text.

Electron runner support

Being able to run your tests in a headless browser, such as PhantomJs, is awesome for many reasons. Still, sometimes you may want to run your unit tests in a real browser, and not to just occasionally verify that your test suite passes in that browser, but in a continuous testing tool like wallaby.js as well.

For example, you may want to use the latest Chrome/V8 features, possibly without Babel, or use some features that Babel doesn’t support. Or you may want to TDD your way through some of your app features that are using Web Audio API (that PhantomJs doesn’t support). Or you just got tired of using various polyfills in your test environment.

Even though wallaby.js supports the latest PhantomJs builds, up until recently PhantomJs was the only option to run your unit tests in browser environment with wallaby. Now we also have added Electron test runner support.

It’s very simple to use: you just install the electron-prebuilt NPM package, specify kind: 'electron' in your env setting in wallaby config. That’s it, now you are ready to run your tests in the latest Chromium/V8 environment without any opened browsers windows and other related distractions.

AVA support

If you are doing node.js testing, then you have probably heard of, and maybe even are using, AVA testing framework and test runner. As a testing framework, AVA is minimal, has a simple test syntax and supports a few nice ways to write async tests (such as returning a promise or an observable, generator or async functions). As a test runner, AVA does a few cool things that wallaby.js brings to other testing frameworks, such as running tests files in separate processes.

We have started prototyping AVA support in wallaby.js a while ago and a few things changed in AVA since then. Now when the runner looks pretty stable, we are happy to announce that we have added its experimental support in wallaby.js.

Wallaby.js provides AVA tests many super-powers of a continuous test runner, including (but not limited to) reporting realtime code coverage and execution results right into your editor, like JetBrains IDEs (such as WebStorm), Visual Studio, Atom, VS Code and Sublime Text.

AVA itself is pretty fast, but wallaby is making AVA even faster by not only executing affected test files, but individual affected tests. So if you are changing an AVA test or some code covered just by that test, wallaby will only execute that test, not all tests in the test file.

Configuring AVA in wallaby.js is literally one line in addition to the rest of your wallaby.js configuration file:

module.exports = wallaby => ({
    files: [...],
    tests: [...],
    ...
    testFramework: 'ava'
});

If you are using Babel and would like to enable power-assert for AVA, you may configure it as follows:

module.exports = wallaby => ({
  files: [...],
  tests: [...],
  ...
  testFramework: 'ava',

  compilers: {
    '**/*.js': wallaby.compilers.babel({
      presets: ['es2015'],
      plugins: [
        require('babel-plugin-espower/create')(
         require('babel-core'), {
          embedAst: true,
          patterns:require('ava/lib/enhance-assert').PATTERNS
         })
      ]
    })
  }
});

Actually, you may use power-assert with any testing framework that wallaby.js supports. Just need to add babel-plugin-espower to your babel plugins list.

One more thing that is worth noting, is that unlike AVA, that forks a separate process per a test file, wallaby.js uses a pool of processes with a configurable capacity and optional process recycling to execute tests. We believe that wallaby.js approach is generally faster, and using a process per file may negatively affect the performance, especially if you have more than just few test files. Anyway, if you like, you may make wallaby.js to run tests as AVA does by just configuring wallaby to use a large number of processes and to recycle worker processes.

Angular 2 support

Angular 2 release date is approaching, and the framework can be used in many different ways: with TypeScript or ES6, System.js or Webpack/Browserify, JSPM or not. No matter what’s your preferred stack, we’d like to make it as easy as possible for you to test your ng2 apps with wallaby.js.

To help you configuring wallaby.js, we have prepared (and are keeping updated) a few sample projects using Angular 2 with wallaby.js and various combinations of technologies that you may use with it.

Miscellaneous

Don’t get upset if you haven’t found anything interesting for you in the announced features. We are doing a few releases every week, so you are still regularly getting improvements that affect your work with the tool. You may also check out the more detailed list of changes, below are some notable changes and fixes.

Function.prototype.bind polyfill for PhantomJs is now shipped with wallaby.js core, so if you are using some packages as polyfills, you may get rid of them. Note that those polyfills may still be required in you are running PhantomJs tests in Karma.
TypeScript 1.8 (and 1.9 dev) support.
Ongoing support for Webpack 2 beta changes. Also some significant performance improvements for larger Webpack projects (make sure to update wallaby-webpack package to latest).
Wallaby.js tests indicator for the Distraction Free mode (and Presentation mode) in JetBrains IDEs.
Support for the latest versions of Jest and Jasmine.

That’s it for the post, thanks for reading!

Wallaby.js - VS Code integration release and Sublime Text integration beta

2016-01-27T00:00:00+00:00

New Year celebrations are over, it’s time to open your editors and start crunching some code. And, of course, tests for the code. Unless you’d like it to become an unmaintainable mess pretty quickly, or just love the Page Refresh Driven development.

New editors

VS Code integration

First of all, I am happy to announce that today we are officially releasing wallaby.js VS Code integration. The beta program has been running for a couple of months and the integration have been tested by more than a thousand of our beta users, so we are pretty confident to proceed and start selling commercial licenses for it. Make sure to update the extension to the latest version to be able to enter your license key.

If you have tried wallaby.js for VS code, enjoyed it, and now just need to find a good way to convince your management to purchase licenses for the tool - just send your boss the link to this article by Scott Hanselman. Scott doesn’t need any introduction, especially if you come from the .NET world.

By the way, if you are a .NET developer or a company mostly using Microsoft development stack, then it’s definitely worth considering getting “Freedom” licenses to use wallaby.js in VS Code as well as in old good Visual Studio (and a few other editors that we support).

As usually, we are providing a free license key to keep using wallaby.js in VS Code for a couple more weeks, while you are making the decision whether to purchase the tool.

Sublime Text integration beta

The time has finally come to add +1000 intelligence to JavaScript code editing in one of the fastest text editors out there. Please welcome wallaby.js for Sublime Text editor!

We are planning to run the beta program for about 2 months, so you should have plenty of time to try the tool, even accomplish a project or two with it while it’s free.

You may get started by reading the tutorial and, as usually, ask questions and raise issues in our public repository. Your feedback is very welcome!

Notable updates of the last few weeks

Last but not least, I’d like to share a few notes about what we have added/improved recently:

Started simplifying configuration for TypeScript/CoffeeScript/Babel. Now you don’t have to pass babel or typescript instance in your configuration options. If wallaby finds the module locally installed, then it will use it. We also have some big plans to further simplify wallaby.js configuration process, your opinion is welcome.
Improved assertion data diff in Atom, check it out.
Improved error reporting from PhantomJs, say goodbye to those pesky Parse Error without stacks.
Improved chai (error stacks) and jasmine (pending tests) support.
Added a few new sample projects, including the Angular2 + Webpack sample.
Improved symlinks resolution.

That’s it for the post, thanks for reading!

Wallaby.js - where are we at and where are we going to?

2015-12-11T00:00:00+00:00

According to the recent JavaScript Developer Survey Results, ~78% of JavaScript developers write tests and 21.7% don’t write tests. It means that we, as a company, have the great market to keep growing, but still have a lot of work ahead to make JavaScript testing productive, affordable and fun for the rest 21.7%.

Do you write tests?

So how exactly are we making the JavaScript testing world better and planning to keep making it better?

New editors

VS Code integration beta

In case you have missed it, we have recently released the first beta of wallaby.js integration for Visual Studio Code. If you are using the editor or just planning to try it out, make sure you install wallaby.js extension and enjoy continuous testing in the powerful and streamlined editor.

There’re a few things we would like to improve in wallaby.js for Visual Studio Code during the beta period and some of these things require to add new features to the editor, but the great thing is that the editor is now open source, so we can start hacking it and submitting pull requests.

Sublime Text integration ETA

We have been approaching and prototyping it for the last few months, but, unlike JavaScript/TypeScript/CoffeeScript/Java/C#, Python and its ecosystem wasn’t something that we’ve had much experience with, so extending Sublime Text was going slower than we’d like it to go.

To speed things up, we have decided to expand our team and have hired a professional Python developer. I’m happy to share that we have made some significant progress, and also happy to announce that we are planning to release the first beta version of wallaby.js for Sublime Text in the next 2 months.

Strategic results and plans

For the last few months we have been expanding both vertically by adding new features and horizontally by supporting more editors.

During just over 10 months, we have released wallaby.js 240 times. It’s 4 live updates with new features and bug fixes during every 5 days. Apart from being a great way to deliver you the wallaby awesomeness fast and easy, frequent releases and hassle-free automatic updates is also our way to embrace the ever changing chaos of the JavaScript ecosystem.

The tool had grown from supporting only JetBrains IDEs initially to support Visual Studio, Atom, VS Code, and now we are working on Sublime Text 3. Adding the support of these 4 different editors (with the fifth on the way) within the 10 months time frame was a great way to bring wallaby.js to as many JavaScript developers as possible.

While we know that there’re more editors out there, we would like to take a break from adding more editor integrations and shift our focus on growing the product vertically. Apart from working on a few already requested features, like the test explorer, code coverage reports, and adding more test runners (such as Chrome and Electron), we are also planning to start working on a couple of truly amazing and innovative new features that will boost your productivity even further, so stay tuned.

Notable updates of the last few weeks

To conclude, I’d like to share a few notes about what we have added/improved recently:

Jasmine for Node support was added.
Babel 6 support was added. All of the samples were update to include comments how to configure wallaby.js to use it.
TypeScript 1.7 support was added.
Webpack 2 (beta) support was added to the latest wallaby-webpack package version.
Have added the documentation page and more samples for system.js. Make sure to have a look to see how to configure wallaby.js inline test error reporting for Angular 2, Aurelia and other system.js based projects.
We have finally added support for advanced glob patterns to JetBrains IDEs integration.

That’s it for the post, thanks for reading and Happy Holidays!

Wallaby.js for Visual Studio Code

2015-11-26T00:00:00+00:00

Wallaby.js is a continuous test runner for JavaScript. It solves the fundamental issue of being able to provide an instant feedback from your tests as you write your code, no matter how large your codebase is. Wallaby.js uses smart dependency analysis to only execute tests affected by your code changes and leverages parallel test execution.

It also displays test execution results, including code coverage, right in your code editor. No context switching, no need to switch to browser - it’s game changing. The tool pays for itself very quickly and is bringing you to a whole new productivity level.

When wallaby.js was released early this year, it was supporting only IntelliJ platform IDEs, such as WebStorm and IntelliJ IDEA, a few months ago we have added Visual Studio and Atom editor support.

Now it’s time to bring the awesomeness of wallaby to Visual Studio Code editor. The streamlined and powerful editor plus the power of wallaby.js is a great combo to deal with JavaScript and TypeScript projects of any size fast and easy.

Setting up

Wallaby.js for Visual Studio Code consists of two separately versioned parts: wallaby.js Extension and wallaby.js Core. To get started, you need to install Wallaby.js Extension from your editor (using Extensions: Install Extension command).

After the extension is installed, wallaby.js Core will be automatically downloaded and installed. Please note, that the very first time wallaby.js needs to download and install its dependencies, so it may take a minute or two. Further updates are much faster. The notification is displayed when the installation is finished.

Sample code

For this tutorial I will use this sample calculator repository. You can clone it to try locally:

git clone https://github.com/wallabyjs/calculator-sample.git

Once you have the source code locally, open the project folder in Visual Studio Code.

There you will find a simple calculator project there with wallaby.js configuration file. Wallaby configuration file is a simple JSON or JavaScript file in your project root folder with just a couple of mandatory settings. The configuration file includes a list of files and tests of your application as well as some settings for various technologies that you may be using to build your project, such as Webpack or Browserify configuration Babel/CoffeeScript/TypeScript compiler options, etc.

Wallaby supports lots of technologies and scenarios, we have collection of sample projects with configuration files for different tools, frameworks and languages, such as Webpack, Browserify, ES6 and ES7 via Babel, TypeScript and CoffeeScript, React and Angular, node.js and io.js, etc.

First steps

Selecting wallaby.js config file

Wallaby.js for Visual Studio Code needs to know what configuration file to use to run your tests. If you only have one file named wallaby.js/wallaby.conf.js or wallaby.json in your project root, wallaby will automatically select it.

However, if you have multiple configuration files, use different naming convention or have the config file not in the root folder, you may select the config file using Wallaby.js: Select Configuration File command from the editor Command Palette.

Wallaby.js Command Palette

As you probably know, you may press Ctrl/Cmd + Shift + P in Visual Studio Code to get the editor Command Palette in case if you’ve forgotten some shortcuts or just like invoking commands this way. There you may search for wallaby and use the found commands.

Also, for your convenience wallaby.js adds its own Command Palette to the editor that you may display by pressing Ctrl/Cmd + Shift + =. It’s a fast and easy way to quickly use various wallaby features.

Now let’s have some fun.

Let’s try starting wallaby.js: to do that you need to invoke wallaby Start command. You may do it by using the Ctrl/Cmd + Shift + R, R shortcut, or using wallaby Command Palette.

To stop wallaby.js, you may use the Stop command.

Wallaby.js in action

Once you have started wallaby.js, at the bottom left corner of the editor you’ll see wallaby status indicator. It’s job is pretty simple: when it displays a spinner - your tests are running, when it shows a cross - you have some failing tests, when it shows a tick - all of your tests are passing. It also shows the number of failing or passing tests, and you may also use the indicator to show ‘Wallaby.js Tests’ Output Channel by clicking it.

Now, let’s open the sample calculator project spec (test/calculatorSpec.js).

When wallaby.js is running, you can see the code coverage in opened source files (specified in wallaby configuration file). As you can see, there are various colored squares displayed for each line of your source code.

Try jumping to any of the passing tests and break it, for example try changing the 2 plus 2 expectation result to 5 in the should add numbers test, or try breaking the code covered by the test. Right after the change wallaby.js automatically runs your tests and displays the result in status bar.

The code coverage and messages are automatically updated, just as you type.

Also, try editing some console.log arguments or just console.log any object you like.

This is a great way to quickly inspect any objects without having to leave your editor, without switching to any other console. Think how cool it is - your code editor is your console with all the required context.

Coverage indicators

Let’s have a look at the editor gutter where you may see some indicators. Here is what these coverage indicators mean:

Gray square means that the source line is not covered by any of your tests.
Green square means that the source line is covered by at least one of your tests.
Yellow square means that the source line is only partially covered by some of your tests.
Red square means that the source line is the source of an error or failed expectation, or is in the stack of an error.
Pink square means that the source line is on the execution path of a failing test.

Failing tests

‘Wallaby.js Tests’ Output Channel displays all tests that are currently failing along with the error stacks and registered console.log calls. Some bits of the displayed information are hyperlinks so you can navigate to different places, for example to the exact error line, or a line where something is logged to console. To quickly focus the panel you may use the Show Failing Tests wallaby command (Ctrl/Cmd + Shift + R, T).

Commands

To help you writing/navigating your code and tests more efficiently, wallaby.js provides a few commands. You may see the full list in the wallaby.js Command Palette (Ctrl/Cmd + Shift + =).

These commands include:

Show line tests: displays the line related test data. The command allows you to quickly view all tests (both failing and passing) that cover the current line of code, navigate to test and code errors, view console messages related to some specific test or some source code line.

Jump to failing test: allows to quickly navigate to the failing test from any ‘pink’ context (the failing test execution path).
Jump to error source: allows to quickly navigate to the source of the test error from any ‘pink’ context (failing test execution path).
Toggle uncovered regions: displays/hides uncovered regions for an opened file. The command is very useful for ‘yellow’ lines to display what is exactly not covered in the line. Highlighted markers will automatically disappear when you start editing your code.

Run line tests: the command is pretty simple, it just runs a single test (if invoked from within the test) or all related tests (if invoked within some source code covered by the tests).
Run file tests: the command runs all tests within the test file it is invoked in, or all tests in all the test files that cover the source file that the command is invoked within.
Run project tests: the command runs all project tests.

Troubleshooting

If you encounter any issues with configuring or using wallaby.js and not sure why it happens, check out our troubleshooting docs section or search our public repository issues to see if someone else have been experiencing something similar.

Another useful technique is to run wallaby.js with the debug flag set to true. In this case wallaby outputs a lot of additional information about the test execution progress which may help you to understand what’s going on if something goes wrong. The output can be viewed in the Wallaby.js Console Output Channel.

More information

Follow us on twitter and subscribe to our newsletter to stay updated.
Browse our full documentation, ask questions or create issues.
Check out wallaby.js planned pricing for Visual Studio Code to get an idea how much it’ll cost after the beta period ends.
Consider getting yourself a copy of ‘Freedom’ license today to help the project and be able to use wallaby.js in any editor that we support today and will support in future.

Wallaby.js for Atom text editor

2015-08-31T00:00:00+00:00

It also displays test execution results, including code coverage, right in your code editor. No context switching, no need to switch to browser - it’s game changing. The tool pays for itself very quickly and is bringing you to a whole new productivity level.

When wallaby.js was released early this year, it was supporting only IntelliJ platform IDEs, such as WebStorm and IntelliJ IDEA, a few months ago we have added Visual Studio support. Now it’s time to bring the awesomeness of wallaby to Atom text editor. The lightweight and elegant editor plus the power of wallaby.js is a great combo to deal with JavaScript projects of any size fast and easy.

Setting up

Wallaby.js for Atom consists of three separately versioned parts: Atom Package, wallaby.js Plugin and wallaby.js Core. To get started, you need to install wallaby.js Atom package from your editor (Settings - Install).

After the package is installed, wallaby.js Plugin and wallaby.js Core will be automatically downloaded and installed. The Atom package is just a loader/updater of wallaby.js Plugin and Core, it gets updated quite infrequently and does it the same way as all other Atom packages that you may have installed.

Wallaby.js Plugin and wallaby.js Core on the other hand are updated more frequently, the update is performed automatically behind the scenes and the notification is displayed when the update is finished.

Sample code

For this tutorial I will use this sample calculator repository. You can clone it to try locally:

git clone https://github.com/wallabyjs/calculator-sample.git

Once you have the source code locally, open the project folder in Atom.

First steps

Selecting wallaby.js config file

Wallaby.js for Atom needs to know what configuration file to use to run your tests. If you only have one file named wallaby.js/wallaby.conf.js or wallaby.json in your project root, wallaby will automatically select it.

However, if you have multiple configuration files, use different naming convention or have the config file not in the root folder, you may select the config file from the Tree View with context menu or by opening the config in an editor tab and using Select as Wallaby.js Config context menu item.

Wallaby.js Command Palette

As you probably know, you may press Ctrl/Cmd + Shift + P in Atom to get the editor Command Palette in case if you’ve forgotten some shortcuts or just like invoking commands this way. There you may search for wallaby and use the found commands.

Also, for your convenience wallaby.js adds its own Command Palette to Atom that you may display by pressing Ctrl/Cmd + Shift + Space. It’s a fast and easy way to quickly use various wallaby features.

Wallaby.js Panel

One of the things you’ll be using a lot is wallaby.js Panel. It displays the name of the selected config file, failing test results as well as some test stats. You can open and collapse it by using the corresponding command (from the palette or via Ctrl/Cmd + Shift + T, T shortcut).

Let’s try starting wallaby.js: to do that you need to invoke wallaby Start command. You may do it by using the Ctrl/Cmd + Shift + T, R shortcut, or using wallaby Command Palette, or by clicking Start button on the wallaby.js Panel.

To stop wallaby.js, you may use Stop command or Stop button on the wallaby.js Panel.

Now let’s have some fun.

Wallaby.js in action

Once you have started wallaby.js, at the bottom right corner of the editor you’ll see wallaby status indicator. It’s job is pretty simple: when it displays a spinner - your tests are running, when it’s red - you have some failing tests, when it’s green - all of your tests are passing. You may also use the indicator to toggle wallaby.js Panel by clicking it.

Now, let’s open the sample calculator project spec (test/calculatorSpec.js).

The code coverage and inline messages are automatically updated, just as you type.

Also, try editing some console.log arguments or just console.log any object you like.

Coverage indicators

Let’s have a look at the editor gutter where you may see some indicators. Here is what these coverage indicators mean:

Gray square means that the source line is not covered by any of your tests.
Green square means that the source line is covered by at least one of your tests.
Yellow square means that the source line is only partially covered by some of your tests.
Red square means that the source line is the source of an error or failed expectation, or is in the stack of an error.
Pink square means that the source line is on the execution path of a failing test.

Any indicator click triggers Show line tests command which we will review later.

Failing tests

Wallaby.js Panel displays all tests that are currently failing along with the error stacks and registered console.log calls. Some bits of the displayed information are hyperlinks that you can use with your mouse or keyboard (Enter) to navigate to different places, for example to the exact error line, or a failing test, or a place where something is logged to console. To quickly focus the panel you may use the Focus Panel wallaby command (Ctrl + 1), pressing Esc will return the focus back to the editor.

Commands

To help you writing/navigating your code and tests more efficiently, wallaby.js provides a few commands. You may see the full list in the wallaby.js Command Palette (Ctrl/Cmd + Shift + Space).

These commands include:

Show line tests: displays the line related test data. The command allows you to quickly view all tests (both failing and passing) that cover the current line of code, navigate to test and code errors, view console messages related to some specific test or some source code line.

Jump to failing test: allows to quickly navigate to the failing test from any ‘pink’ context (the failing test execution path).
Jump to error source: allows to quickly navigate to the source of the test error from any ‘pink’ context (failing test execution path).
Toggle uncovered regions: displays/hides uncovered regions for an opened file. The command is very useful for ‘yellow’ lines to display what is exactly not covered in the line. Highlighted markers will automatically disappear when you start editing your code.

Run line tests: the command is pretty simple, it just runs a single test (if invoked from within the test) or all related tests (if invoked within some source code covered by the tests).
Run file tests: the command runs all tests within the test file it is invoked in, or all tests in all the test files that cover the source file that the command is invoked within.
Run project tests: the command runs all project tests.
Show last run screen shot: the command just does what it says. After every test run, wallaby.js captures a screen shot of the generated test runner page, so if you have some tests that are displaying anything - the command could be useful for you. Very powerful combo is using the command together with the Run line tests - you run a specific test, then view the screen shot for the test. Please note that you may often have some ‘tear down’ code, that clears the DOM before or after each test - wallaby.js is smart enough to capture the screenshot before the tear down code. Atom automatically updates the screen shot file when it’s changed, that allows you to have ‘live’ preview right in your editor.

Troubleshooting

More information

Follow us on twitter and subscribe to our newsletter to stay updated.
Browse our full documentation, ask questions or create issues.
Check out wallaby.js planned pricing for Atom to get an idea how much it’ll cost after the beta period ends.
Consider getting yourself a copy of ‘Freedom’ license today to help the project and be able to use wallaby.js in any editor that we support today and will support in future.

The Holy Grail of JavaScript Testing

2015-06-29T00:00:00+00:00

My last blog post was about mighty knights of JavaScript testing, today it’s turn of the Holy Grail of JavaScript Testing.

So I was watching this video the other day, and while the purpose of the video is to demonstrate how to set things up and get started, the end of it really reminded me about the pain I had to experience before I created wallaby.js.

If you watch the last minute of the video (from 3:14), you’ll surely notice that

to accomplish one simple task of writing a test, the author had to switch context several times between three different applications - terminal, code editor and browser test page;
code coverage is missing (and when present, may add the fourth application to keep switching to);
larger application will take a few seconds to build and a few more to run your tests.

Of course, there are a few solutions that may get you closer to a better workflow, such as a file watcher via gulp or grunt, karma with webpack plugin, etc. I have previously spent many hours trying to get closer to what I think the modern productive automated testing workflow should look like.

And I failed. There are just too many moving parts to stitch together - even if or when it all works, the whole thing is far from satisfying my list of requirements. It’s like trying to build something from a pile of LEGO bricks with a few non-matching and few missing bits and pieces.

Don’t get me wrong. I love small tools, when each one is doing one thing well. It’s nice to be able to choose a testing framework, a module bundler, language dialect, a transpiler, an application framework, a code editor, etc., as opposed to be forced to use a monolith from a single vendor.

But once I’m happy about each and every part, I want them to work together smoothly. When I write my code with tests, I’m doing just one thing - I am writing my code with tests. So I need a tool that would allow me to to do this one thing well.

Also, to be fair, my expectations for the modern and productive testing workflow are pretty high. I sometimes do JavaScript TDD, sometimes I am using some mixed approach where I write some of my code first, but either way:

I’m too lazy to save files to run my tests. Code changes should be enough as a trigger.
I don’t want to wait too much for my tests to finish running. Running the whole suite when a single test or file changes just doesn’t cut anymore.
I don’t like to leave my code editor to see the feedback from my tests. Hate apps/context switching. Docked tool windows or a console/terminal/browser next to my code editor - are still distracting.

For those of you who write tests, everything I have listed probably sounds like the Holy Grail of JavaScript Testing, doesn’t it? Well, it exists now. And yes, it supports everything I have mentioned, Webpack, node.js, ES6 and ES7, React JSX, TypeScript and CoffeeScript, many other technologies and their crazy combinations.

Heroes of JavaScript and Testing

2015-05-29T00:00:00+00:00

In 1995, Earth has become a JavaScript faring Federation. While colonizing new web technologies, humans have encountered an insectoid species known as “Bugs”. The bugs appear to be little more than killing machines, though there are suggestions that they were provoked by the intrusion of humans into their habitats.

I have taken a few lines from the “Starship Troopers” plot summary, but it sometimes feels like comparing to the Federation tech, we are still like medieval knights with our JavaScript weapons. Especially when killing bugs, or doing automated testing.

Hail fellow, I bid thee welcome, and ask you to join my journey to the cellars of the JavaScript testing armory.

Clubs

Our great ancestors, who stood shoulder to shoulder against the first bugs, used alert. Legendary spiked club, it crushed many simple enemies.

It’s glorified version console.log, or a morning star, is used nowadays. It may cause some confusion in enemy rear areas, but is not good enough to be used as a systematic approach for significant size battles.

Swords

Testing frameworks, or swords, are the main weapons in our merciless fights.

QUnit, or the Gladius. Quite old and oddly looking to some modern knights, it covered itself with glory in jQuery lands. Used to be, and for many remains, a very fearsome weapon.

Jasmine, or the Great sword. Large and two-handed sword in a sense that you don’t need anything else in your hands. It comes with the expectation framework and out-of-box mocking support. What else does a good knight need to kill a dozen or two of bugs with?

Mocha, or the Arming sword. Being simpler, smaller and faster, it allows to carry something else, such as an assertion library of your choice to make your bug enemies look miserable. Not only the sword can be you best friend in the land of JavaScript, but also in the Seas of Node.js.

Shields

Various application frameworks are like shields. Many knights love sitting around the Round Table of Internet and debate days and nights which shield is better. For smaller battles, you may be fine without it or with your own one. For larger battles, some good and tested by other knights shield will better protect you from certain bugs.

Siege engines

Browser automation tools and E2E testing frameworks, like Selenium, WebDriver, Protractor, etc. are the siege engines of testing. Like catapult, ballista or trebuchet, they are bulky, slow and hard to set up. Nevertheless, when up and shooting, will do a good job in destroying the enemy’s rampart.

Armour

Code editors is our armour, that’s what we spend endless battles with bugs in. Blacksmiths from different kingdoms all over the world are doing the great job to satisfy any knight’s taste.

Some armour is heavy and large but solid, some is smaller and more lightweight but may require a few patches here and there.

We have a very sublime armour from one humble blacksmith, some solid merchandise from the JetBrains guild, a couple of nice sets from the Microsoft kingdom, elegant armour from the GitHub’s forge and even ancient Dark Elvimn one.

Magic

JavaScript is the main language we, mighty knights, speak. However, many of us have learned to record our wisdom in other languages and dialects. From far away lands of C# and Ruby, TypeScript and CoffeeScript came to help us in our war. From across the Seas of Node.js, CommonJs reinforcements have arrived. Even within our homeland, the language evolves and we more often hear minstrels singing glorious tales in ES6 or ES7.

In order to make it work against our enemies, we need some magic. Opening a scroll with a good TypeScript or ES7 spell and using it together with a potion of Babel Transpilation, sorcery of Webpack and Browserify or witchcraft of require.js, we make even our strongest bug enemies shake in fear.

Squires

Squires, or testing tools - are our humble servants. They more or less know your weaponry and armour, your spells chest content. Some ascetic knights can travel and fight without them, but normally a knight has a few.

Grunt and Gulp, brothers from one mighty but peaceful clan, with a few less known relatives. The brothers are strong and brave, and can even run your tests, but they were born to build your castle, not to slay bugs.

HTML test page. Easy and responsive chap, but a bit dumb and you need to poke him to make him work. You need to tell him what to do (add your scripts and tests) and when. Page refresh driven development in all of its beauty.

JSTestDriver. Old man from the Google kingdom. Hard-working and with Java in his heart, he had spent his best years running our countless tests. Knows when to run your tests, certain armour types friendly.

Karma. JSTestDriver’s son, who has taken the best from his father. Supports some armour types, very open fellow. As his father, Karma knows when to run your tests, but has no idea what tests to run, so just runs them all. It makes him perfect for some smaller battles, but he is a bit slow for medium and especially large ones. Also, Karma can’t swim the Seas of Node.js.

Wallaby.js. Highly skilled mercenary, specially trained to continuously kill bugs and do it fast. Not only he knows when to run your tests, he also knows how to run only those that are affected by your changes. Proficient with pretty much every weapon, shield and magic type listed in the article. Knows how to mix them well, more than just friendly to certain armour types. For a few gold coins in the nearby tavern you may get yourself a great helper for a battle of any size.

Fare thee well, mighty knights! Choose your weapons wisely and let there be no mercy for the bugs.

Wallaby.js for Visual Studio

2015-05-18T00:00:00+00:00

Wallaby.js is a continuous test runner for JavaScript. It solves the fundamental issue of being able to provide an instant feedback from your tests as you write your code, no matter how large your codebase is. Wallaby.js uses dependency analysis to only execute tests affected by your code changes and leverages parallel test execution.

It also displays test execution results, including code coverage, right in your code editor. If you have used NCrunch for C#, then many of wallaby.js UI concepts will be familiar to you.

Up until recently wallaby.js only supported IntelliJ platform editors, such as WebStorm and IntelliJ IDEA. Today I am happy to announce that wallaby.js adds support for Visual Studio 2013 (Update 4; Community, Professional, Premium and Ultimate editions).

Setting up

First, you need to download wallaby.js VSIX from our website and install it in your Visual Studio. Once installed, wallaby.js may be managed as all other “Extensions and Updates”.

For this blog post I will use this sample from our public GitHub repository. You can clone it to try locally:

git clone https://github.com/wallabyjs/public

or just download the zip file with the source code.

Once you have the source code locally, open sample/visualstudio/Wallaby-Sample.sln solution.

You will find a simple calculator project there and a few wallaby configuration files for different testing frameworks. Wallaby configuration file is a simple JSON or JavaScript file in your project root folder with just a couple of mandatory settings.

Wallaby supports lots of technologies, we have collection of sample projects with configuration files for different tools, frameworks and languages, such as Webpack, Browserify, ES6 and ES7 via Babel, TypeScript and CoffeeScript, React and Angular, node.js and io.js, etc.

First steps

In this sample we will be using vanilla JavaScript. Let’s try starting wallaby.js: right click on the wallaby-jasmine-2.1.json configuration file in the Solution Explorer and select Start Wallaby.js from context menu. Wallaby will start with selected configuration file.

To stop wallaby.js, you may use Stop Wallaby.js context menu item.

The very first time you run it, wallaby.js may take a minute or two to install all the dependencies it needs, such as node.js and PhantomJs. It only does this once and then uses the saved dependencies. The dependencies are updated automatically when required.

Now let’s have some fun.

Code coverage

Once you have started wallaby.js, in the bottom right corner of the screen you’ll see wallaby status indicator. It’s job is pretty simple: when it displays a spinner - your tests are running, when it’s red - you have some failing tests, when it’s green - all of your tests are passing.

Let’s open the sample calculator Jasmine spec.

When wallaby.js is running, you can see the code coverage in the source files that it tracks (specified in the configuration file). As you can see, there are various colored squares displayed for each line of your source code.

Jump to any of the passing tests and break it, for example try changing the 2 plus 2 expectation result to 5 in the should add numbers test. Right after the change wallaby.js automatically runs your tests and displays the result in Visual Studio, right where you need to see it.

Also, try editing some console.log arguments or just console.log any object you like.

The code coverage and inline messages are automatically be updated, just as you type.

Here is what coverage indicators mean:

Gray square means that the source line is not covered by any of your tests.
Green square means that the source line is covered by at least one of your tests.
Yellow square means that the source line is only partially covered by some of your tests.
Red square means that the source line is the source of an error or failed expectation, or is in the stack of an error.
Pink square means that the source line is on the execution path of a failing test.

You may also inspect wallaby.js output in the Output tool window. In the tool window you may view output from Failing Tests tab and Console.

Failing Tests output source displays all tests that are currently failing along with the error stacks and registered console.log calls. Some bits of the displayed information are hyperlinks that you can use with your mouse (Ctrl + mouse click) or keyboard (F12) to navigate to different places, for example to the exact error line, or a failing test, or a place where something is logged to console.

Console output source displays some additional information about test execution progress. The tab can also be useful for troubleshooting wallaby.js issues.

Smart actions

To help you writing your code and tests more efficiently, wallaby.js provides a few smart actions available by clicking wallaby smart actions icon or pressing Ctrl + . on any line.

The actions include:

Show line test(s) action: displays the line related test data. This is the most frequently used action that allows you to quickly view and navigate to test and code errors, console messages related to some specific test or some source code line.

Jump to failing test action: allows to quickly navigate to the failing test from any “pink” context (the failing test execution path).
Jump to error source action: allows to quickly navigate to the source of the test error from any “pink” context (failing test execution path).
Show line uncovered regions: the action is useful for “yellow” lines to display what is exactly not covered in the line. Not covered expressions will be highlighted in red.
Show file uncovered regions: the action is similar to the previous one, it displays uncovered regions for the whole file. The red highlights for both actions can be removed by pressing Esc (or you can just start changing the file code and they will disappear).

Run line test(s): the action is pretty simple (yet so much wanted by so many people), it just runs a single test (if invoked from within the test) or all related tests (if invoked within some source code covered by the tests).
Run file test(s): the action runs all tests within the test file it is invoked in, or all tests in all the test files that cover the source file that the action is invoked within.
Show last run screen shot: the action just does what it says. After every test run, wallaby.js captures a screen shot of the generated test runner page, so if you have some tests that are displaying anything - the action could be useful for you. Very powerful combo is using the action together with the Run line test(s) - you run a specific test, then view the screen shot for the test. Please note that you may often have some “tear down” code, that clears the DOM before or after each test - wallaby.js is smart enough to capture the screenshot before the tear down code.

Fonts and colors

If some of wallaby.js default colors don’t fit your theme, it’s very easy to adjust.

More information

Follow us on twitter and subscribe to our newsletter to stay updated.
Browse our documentation, ask questions or create issues.
Check out wallaby.js pricing for Visual Studio.
Consider getting yourself a copy of “Freedom” license today to help the project and be able to use wallaby.js in any editor that we support today and will support in future.

Wallaby.js is expanding - new languages and module bundlers

2015-04-13T00:00:00+00:00

If you are new to wallaby.js, it is an intelligent test runner that solves the fundamental issue of being able to provide an instant feedback from your tests as you write your code, no matter how large your codebase is. Wallaby displays test execution results including code coverage right in your code editor. In this blog post I will cover new features that have been added during the last few weeks.

TypeScript, CoffeeScript, ES7

Up until recently wallaby.js only supported JavaScript (up to ES6 plus React JSX). You could and can of course use any other language with preprocessors and wallaby.js can run tests for you, but it wasn’t able to display code coverage for anything but JavaScript (up to ES6 plus React JSX).

Today I am happy to announce that wallaby.js adds support for TypeScript and CoffeeScript. We also now support ES7 features via Babel. Whether you have your whole codebase in one (or more) of those languages, or just prefer to write your tests using a language compiled to JavaScript - wallaby.js can display code coverage for the language of your choice.

There is nothing special to configure to start using new features, just list your files in wallaby configuration file:

{
  files: [
    'src/*.ts',
    // and/or
    'src/*.coffee'
  ],
  tests: [
    'test/*.ts'
    // and/or
    'test/*.coffee'
  ]
}

and you are ready to get real time coverage for TypeScript:

or CoffeeScript

There are sample projects with a couple of source files and wallaby.js configuration file in CoffeeScript sample and TypeScript sample repositories to get you started quickly.

If you would like to start using ES6 and ES7 features in your tests or application code, wallaby.js has got you covered.

Wallaby.js ships with a built-in Babel compiler integration. Simple configuration:

module.exports = function (wallaby) {
  return {
    files: [
      'src/*.js'
    ],

    tests: [
      'test/*Spec.js'
    ],

    compilers: {
      '**/*.js': wallaby.compilers.babel()
    }
  };
};

and you are ready to enjoy ES7 magic with all wallaby.js features, such as async/await:

And of course new features work with existing features, so you may use all combinations of supported technologies. Whatever your stack looks like, for example, CoffeeScript with node.js or io.js, TypeScript with CommonJs via Webpack or ES7 async/await in your React.js application along wth JSX - wallaby.js can provide you real-time code coverage and instant feedback from your tests.

As I have promised before, we are (and will be) providing more and more ways to extend wallaby.js the way you need. For example, new languages support is added with a new extensibility concept called ‘compilers’. With wallaby.js compilers extensibility point you can also your own custom compiler for any language.

Browserify and Webpack

Some of you are happily using CommonJs or AMD and NPM modules in browser with a solution to package code bundles/chunks for deployment. To support nice and fast code testing for such scenarios, a couple of weeks ago we have added a new extensibility concept to wallaby.js called ‘postprocessor’. With a stateful postprocessor it is possible to extend Wallaby core to leverage incremental compilation and caching features of Browserify and Webpack to keep providing an instant feedback from your tests when running them with wallaby.js.

Both extensions, Wallabify and Wallaby-Webpack, are open source, pull requests with improvements are very welcome.

In the above mentioned repositories you may find some recommendations on how to configure wallaby.js to use with Browserify and Webpack. You may also find simple examples with wallaby.js configuration Browserify sample repository and Webpack sample repository.

That’s it for the blog post, thanks for reading and hope it helps you to get started with new wallaby.js features in your project. Follow us on twitter to stay updated.

Testing React applications with wallaby.js

2015-03-11T00:00:00+00:00

Wallaby.js is a continuous test runner. It runs your tests as you type and displays various results right inside your code editor. Wallaby.js allows you to test your React applications and get an instant feedback from your tests, no matter how large your codebase is and whether you are using browser environment or node.js with jsdom. With wallaby.js, you can use any popular testing framework of your choice.

In this article I will describe how to test React application, specifically React components, using Jasmine and wallaby.js with code coverage, including code coverage inside JSX element JavaScript expressions. You can find the full code in this repository.

First, if you haven’t done it already, go ahead download and install wallaby.js. Then clone this repository from WebStorm or command line, open the project and install dependencies.

git clone https://github.com/wallabyjs/wallaby-react-component.git

npm install

Next, create wallaby configuration file for your project. It is pretty straightforward, especially if you have used something like Karma before. You can name it wallaby.js and put the following code into it:

module.exports = function (wallaby) {
  return {

    files: [
      {pattern: 'node_modules/react/dist/react-with-addons.js', instrument: false},
      'assets/**',
      'src/**'
    ],

    tests: [
      'test/**/*Spec.jsx'
    ],

    compilers: {
      '**/*.js*': wallaby.compilers.babel()
    }
  };
};

In the configuration file, we have specified the list of our application files and tests using glob patterns. We are also setting up a compiler to convert jsx files code to ES5 (so it can run in browser) using react tools.

Other configuration options and the full format of wallaby.js configuration file is described in wallaby.js documentation.

We are ready to go now - start wallaby.js run configuration using Run menu or keyboard shortcut.

When starting for the first time, wallaby.js will download and install some dependencies. Those include node.js, that is used to run wallaby.js, and PhantomJs headless browser engine to run your tests in. The installation may take a minute or so and only happens once.

After the start, wallaby.js displays its tool window at the bottom:

and its status indicator at the right bottom corner of the window.

The indicator displays a spinner when your tests are running, red icon if any of the tests is failing and green icon if all tests are passing. The tool window Failing tests tab displays the information about failing tests, such as error stacks and failed expectation details.

Once started, you don’t need to restart wallaby.js run configuration to run your tests, as it does it automatically, as you change your code. You can hide the tool window by using the Hide icon in its right top corner, as you will be getting the information about failing tests right in the context of the code that you’re editing. You can always get the tool window back by clicking wallaby.js status indicator.

Now, let’s see wallaby.js in action. This project contains a sample implementation of a composite user profile control

var UserProfile = React.createClass({
  render: function() {
    return (
      <div>
        <ProfilePicture username={this.props.username} />
        <FollowButton username={this.props.username} />
      </div>
    );
  }
});

that displays a user picture

var ProfilePicture = React.createClass({
  render: function () {
    return (
      <img src={'assets/' + this.props.username + '.jpg'}/>
    );
  }
});

and also has a “Follow” button that displays user name and changes its text when clicked.

var FollowButton = React.createClass({

  handleClick() {
    this.setState({text: 'Unfollow'});
  },

  render() {
    return (
      <p onClick={this.handleClick}
        className={this.state ? 'unfollow' : 'follow'}>
        {this.state && this.state.text
          || 'Follow ' + this.props.username}
      </p>
    );
  }
});

Now, let’s open the application spec test/avatarSpec.jsx.

'use strict';

var testUtils = React.addons.TestUtils;

describe('User Profile', () => {

  var instance;

  beforeEach(() => {
    this.container = document.createElement('div');
    document.body.appendChild(this.container);
    instance = React.render(<UserProfile username='Homer' />,
                            this.container);
  });

  afterEach(() => {
    document.body.removeChild(this.container);
  });

  describe('Follow button', () => {

    it('should display user name', () => {
      expect(instance.getDOMNode().textContent).toBe('Follow Homer');
    });

    it('should change its text to "Unfollow" when clicked', () => {
      testUtils.Simulate.click(
        testUtils.findRenderedDOMComponentWithTag(instance, 'p'));

      expect(instance.getDOMNode().textContent).toBe('Unfollow');
    });
  });

});

We are testing that the “Follow” button works as expected by rendering a user profile, checking the button text, clicking it and checking its changed text.

When wallaby.js is up and running, you should be able to see coverage indicators in the gutter of the code editor in your tests and other source files.

Gray square means that the source line is not covered by any of your tests.
Green square means that the source line is covered by at least one of your tests.
Yellow square means that the source line is only partially covered by some of your tests.
Red square means that the source line is the source of an error or failed expectation, or is in the stack of an error.
Pink square means that the source line is on the execution path of a failing test.

On any line of the source code you can hit Alt + Enter (or use the light bulb icon) to display the list wallaby.js context actions. You can read more about all actions in the documentation.

For this example, let’s mute one of the tests. The easiest way to do this is to change it test function to xit. This will make Jasmine to ignore the test.

As you can see, wallaby.js immediately executes the test and marks the test lines with gray markers, because they are not executed anymore. It also changes src/components/FollowButton.jsx file indicators.

We can also see that handleClick function is not invoked anymore, so it has a gray indicator next to it. We also see yellow indicators next to the JSX element JavaScript expression lines of code. It means that the expression code is only partially covered by our tests. To know what exactly is NOT covered, let’s hit Alt + Enter and use ‘Show file uncovered regions’ context action.

Now we clearly see what is exactly not covered inside the lines with the yellow coverage indicator. Because we have muted the test that was clicking the button, unfollow CSS class is not assigned to the button tag and because handleClick function is not invoked anymore, this.state is undefined so this.state.text is never evaluated and not set as the button text.

Wallaby.js tracks coverage for logical expressions and conditional operators, as you may see it supports JSX as well (and ES6 and many more technologies).

Using wallaby.js context actions you can quickly jump to the failing test or the error source, display tests that are covering some particular line of code, inspect console.log output of each individual test, full error stacks and expectation errors, and much more.

Let’s try to make one of our tests fail:

Immediately as we do so, wallaby.js displays the expectation error right where it occurs. The test path line indicators become pink, the source of the error line indicator is now red.

Apart from displaying errors, where and when they occur, wallaby.js also displays console.log output in a similar manner, so if you’re a fan of console.log, wallaby.js can make you very productive. Now you don’t have to leave your editor anymore to console.log all the things.

Finally, if you would like to see what is actually displayed when wallaby.js runs your tests, you may use “Show last screenshot” context action to get a realisation that your app is not just a bunch of components but is something alive.

Please note, that you have to run a test, that you’d like to see a screenshot for, by using “Run scope test(s)” action within the test first. The reason is simple - normally you have many tests and there’s also some tear down code in the afterEach function. However, when you run one specific test, wallaby.js makes a screenshot before the tear down code is executed.

That’s it for the blog post, thanks for reading and hope it helps you to get started with wallaby.js in your React project. Follow us on twitter to stay updated.

Bringing together JavaScript live editing and testing.

2015-03-05T00:00:00+00:00

8-bit Super Mario Bros was the very first video game that I have played as a kid. Later there were no less awesome Battle City, Chip ‘N Dale, Teenage Mutant Ninja Turtles, Duck Tales, Battletoads, Darkwing Duck and many more. Those happy days, when I couldn’t even imagine that I’m going to become a programmer.

Many years later, when I became a programmer and had spent several years in the industry, I was watching the “Inventing on Principle” talk by Bret Victor. I am pretty sure, many of you have seen the talk, and even know a few software development tools inspired by it or just following the similar ideas.

As many of you, I have been amazed how in the presentation Mario looking dude jumps higher as we change the velocity, or how the tree blossoms right when we change the code.

We are also amazed when in some demos we change some code on a web page, right in DevTools or some other tool, click some button and see the result without the page reload.

These ideas are great and inspiring, but then we get back to our daily job to find out about the cruel reality.

First, it gets annoying very quickly to switch between our code editor and our browser to make Mario jump.

Later, some other developer changes the game level, everything suddenly breaks, it’s 9pm and we sitting at work and manually making Mario jump again and again.

Or worse, your version of jumping Mario is actually a button in a form somewhere deep inside your web application. And to click the button that would run some ‘live’ reloaded function, you have to do several steps each time before that function gets executed.

Hot swapping and live editing does help for CSS and HTML reloading, can also help when you are debugging JavaScript and would like to quickly change some very simple stuff. But then you make a change to your JavaScript code and your live editing tool can’t catch up and the whole page reload is required. All because what seemed as a trivial change to you, is often a high wall for ‘live’ editing solutions and code hot-swappers. And they simply can’t jump over the wall, no matter how hard they try and excel in what they do. The issue is not only in technology, it’s also the approach that matters.

So, why does it look nice in all those presentations and doesn’t work for you?

The reason is simple and obvious. Your application complexity goes beyond simple demos and vanilla live editing approaches don’t scale. You don’t just have a simple setInterval/setTimeout/requestAnimationFrame callback or a single form hosted right in your index.html. Your code changes are not limited to some easily hot-swappable ones.

Moreover, you pretty much always have to set up some context before executing any code and there’s no auto-magic solution to create/restore a desired context for your hot swapped code execution. You may need to manually take like 5 different steps before freaking Mario can jump or before you can click that button again to trigger your hot-swapped code. And all that ‘live’ and ‘hot’ quickly become no more attractive than your old-school coding approaches, if not less attractive.

So, is truly live JavaScript editing with instant feedback even possible at all in real-world apps?

The answer is yes. Automated and instant feedback beyond simple prototypes is possible with recorded, runnable, relatively small/fast and isolated scenarios. We have had the name for those for a long time and the name is tests. TDD/BDD or not, Given-When-Then or Arrange-Act-Assert, test first or after, or mixed - the important thing that tests have to be written.

When you are writing and later maintaining the bulk of your application logic, it’s not the visual image of jumping Mario that helps you making your inventions. No doubt, visualising things can help to get things started, then later to get a feeling that it is all real. To get a realisation that your app is not just a combo of your Controllers, Models and Views, but something alive. Surely, you may occasionally check that your Mario looks good and jumps when you press your keyboard keys. You may even write some E2E test for it.

But you don’t need your browser to make your Mario jump again and again. You don’t need to see the sky background every time, you don’t need to press your keyboard keys manually. Not only you can be “inventing on principle”, you can automate your invention process and make it more focused and efficient.

If you need to check something about how Mario jumps, just record the fact that Mario can jump into that hole as a verifiable and fast spec/test. Make it work for you not just once or twice, but when you change the size of the hole, when you change its position, when you change Mario to Luigi, or even to Scrooge McDuck.

So for the most part of your work: get rid of what you don’t need - close your browser, put on your headphones, open your code editor and nothing else, and do the bulk of your work efficiently. Write tests and code, let right tools to run it automatically and without distraction, get an instant feedback as you type.

At the end of the day, it doesn’t matter if your running program is reloaded or not, what is running your program and how it reloads it, as long as the feedback for your code changes is close to instant.

At this point, you may say: ok, but wait a minute. Tests are good, we have a thousand or two and mostly run them overnight or very infrequently, because it’s too expensive to run them on code changes. So how does it relate to live JavaScript editing, and how do I make my tests live?

The answer is using the right tool.

First, running tests that you don’t need to run is a waste of time. If you changed one test or some code that is covered only by few tests, then why do you need to run hundreds of them? You wouldn’t want to complete all levels of Mario game every time you change princess Peach rescue scene, would you?

Second, getting your tests feedback somewhere outside your code editor, or even somewhere far from the line of the code that you are editing, is also not good enough anymore.

The tool in JavaScript land that can make your your tests and code truly live is wallaby.js. Without you having to change your testing framework or use any special APIs in your tests, wallaby runs minimal required set of tests for your changes, just as you type. It displays the feedback immediately and where you need it most.

And if you’d like to get some visual feedback, you can use wallaby feature to display a screenshot of the test result. It supports not only HTML and CSS, but also SVG and Canvas. So if your test makes Mario jump, wallaby can show you jumping Mario, just like the calculator with buttons and screen in this video.

Most importantly, wallaby.js goes way beyond shiny simple demos, it really helps in real-world applications.

Worked on reducing down the 30 second wait for 1800 angular unit tests to run in Karma. Got it down to, well, instant with @wallabyjs :D
— Andrew Shelton (@Sheltonial) March 4, 2015

We support various technologies and have a few repositories to get you started with wallaby.js quickly:

So go ahead and start live editing JavaScript today, make your code and tests truly live with wallaby.js. Follow us on twitter to stay updated.

Wallaby.js gets its first full-time developer and is looking for investors

2015-03-03T00:00:00+00:00

Wallaby.js, our smart test runner, that makes you extremely productive when you are writing your JavaScript code and tests, is getting some great traction. More and more people are starting to use wallaby.js, community based plugins are starting to emerge. We have released version one, are already covering a great part of the mainstream JavaScript technologies and frameworks, and we are working on more.

Wallaby.js goal is to teach how and help the community to write testable and tested JavaScript applications of any scale with an immediate feedback from the tests. We would like to support as many code editors as we can, to bring wallaby love to everyone, from Vim users to various IDE users. Future plans include exposing more APIs and even gradually open sourcing the core of wallaby.js or its parts once our business gets a bit stronger. We also have many other ideas on our roadmap and some not yet announced but, trust me, very exciting and ambitious plans.

The technology and the approach (parallel and especially incremental test runs) that we use is unique and highly scalable. Efficient combo of our strict and complex runtime analysis based rules, along with a lot of experience based heuristics, makes wallaby.js a very special and smart tool. Our small team is dedicated to make wallaby.js an essential tool for writing and testing JavaScript, and I am confident that we will do it sooner or later.

Creating and maintaining professional and production quality tools of such scale, working in a variety of environments, using half a dozen of programming languages - not only requires a dedicated and highly motivated team but also an adequate financing. So far, myself and my small team have been working on wallaby.js in our spare time. As of today, I have made a decision to suspend my employment contract and start working on wallaby.js full-time.

It’s a big step for me personally. I knew I would do it one day and was prepared to face it, but when the day comes - it is still very exciting and scary at the same time.

While I have a solid plan to make wallaby.js community focused and sustainable business by selling commercial licenses, I would also like to try to boost wallaby.js evolution and development by trying to attract some additional external financing/investments.

You may already be using wallaby.js in your team and finding it helpful. Maybe you just like the idea and would like to see it in more editors, supporting more technologies and languages, supporting cloud testing, and would like to see it quickly becoming a high quality and affordable instrument for you and your team. In either case, please pass the information to the people you think might be interested in helping wallaby.js to reach its shiny future faster.

That’s it for the post, thanks for reading, follow us on twitter to stay updated. Drop me a line artem.govorov@gmail.com if you have any questions or suggestions.

The story of one inner hipster, part one.

2015-02-28T00:00:00+00:00

Chapter one. Our sick industry.

About a year and a half ago, my career path took a bit unexpected turn. After spending 10 years in product teams and custom development, doing .NET consulting and web programming - everything has suddenly changed. I have caught the virus of developing ‘my own thing’.

From what I see around - it’s a pretty common ‘disease’ in our industry. One day you get bored working in the bloody enterprise software business and your inner hipster demands some revolutionary changes.

My next personal project

The virus symptoms may be different - some are seeking financial freedom and wealth, some just want to do what they want to and not what they are being told to. Your idea may be different - an indie game, next social network beyond geo-location based photo-sharing, PaaS, IDE or another software development tool.

Chapter two. My case history first record.

And I wasn’t an exception. Historically, I have always been concerned that JavaScript, while being so popular, doesn’t have enough good tools comparing to other languages that I worked with. More than enough frameworks, but not enough tools. So my inner hipster made me write a software development tool. It was JavaScript tracing tool, called spy-js.

The success of the tool was quick and even before I had a chance to digest it and make my inner hipster less hungry, spy-js was acquired by JetBrains. I have been offered to join WebStorm team to work on it full time as an independent contractor from home on the warm and sunny Gold Coast, Australia.

Life after spy-js acquisition

Chapter three. Changes and challenges.

The change was huge for me. Regular shaving and pants wearing ceremonies quickly became a forgotten past. My top notch Lenovo ThinkPad was quickly and happily replaced by MacBook Pro, Windows by OSX, Ctrl by Cmd, cmd by terminal, Visual Studio by IntelliJ IDEA, and my long lasting love C# was replaced with at first supercilious Java.

Heading to MacDonald’s in my home outfit

Anyway, not just my inner hipster, but the inner hipster of my inner hipster and his inner hipster were happy. A Russian doll of happy hipsters. Or even a fractal of happy hipsters.

Surely, big changes never come without some challenges. Not everyone can work from home. It takes discipline, concentration, motivation, dedication, and a few other words that end with “-tion”. I always knew I could do it and had some successful experience in past of working remotely. But during the first few months it was very unusual to get used to communicate with your team in realtime only within a tiny couple of hours frame allowed by our time zones.

Thankfully, the majority of the challenges are easy to overcome when your relationship with your company is based on the mutual trust, and JetBrains is one of those awesome companies where many people working from different offices and homes all around the world, doing it happily and efficiently.

Chapter four. When it all not just ends but also begins

Another pleasant challenge was the nature of my new work. I have never been feeling comfortable strictly within the formal boundaries of a software developer duties. Apart from hacking together awesome stuff, I have always had a variety of interests in software development marketing, development process management, supporting users and advocating/evangelising certain things I strongly believe in, such as some software design and testing practices.

With spy-js in WebStorm I had and totally used the great chance to do what covers the majority of my interests. I was able fully implement my ideas in the product, to write blog posts about it, to tweet about new features, to help users adopt it. I have been given a full trust and freedom to do whatever I think is good and bring it to the world with two (and upcoming third) major releases of WebStorm.

Always enjoyed growth hacking

Of course, it took more than one person to make spy-js a successful, reliable and useful tool. I’d like to use the opportunity and thank everyone in my team who has participated in the tool ideas generation, development, testing, marketing, evangelising. And also you - software developers and users of spy-js for your kind words, help and feedback.

The story could end here. Great idea, successful acquisition, nice product, joyful tears and everyone lived happily ever after. Not in my story. My inner hipster didn’t want to rest on his laurels. He woke up again and he’s hungry. But something has changed in him…

To be continued in part two.

Wallaby.js hits version 1.0

2015-02-23T00:00:00+00:00

Today we are proud to announce that wallaby.js, the smartest test runner for JavaScript, hits version one.

New version is packed with new features and significant performance improvements, not to mention a few mercilessly exterminated bugs. Here is a list of the most important changes:

Node.js and io.js testing support,
ES2015/ES6 support,
React JSX dialect support,
JavaScript file configuration,
Karma-like code preprocessors,
Require.js support.

We have also added a couple of new useful context actions, added a few caching layers to our web server and sandbox to make the tests to even faster, made a few significant performance improvements and bug fixes, migrated wallaby.js internal engine from node.js to io.js, updated a few internally used libraries (such as acorn, lodash, escodegen).

Version 1.0 and beyond

Publicly released just few weeks ago, wallaby.js has already gained a great traction amongst JavaScript developers. As we know, many of you have already been happily using the tool for your daily jobs, helped and are helping us to make it better. Our team really appreciates your help, kind words and encouragement, we are dedicated to keep making your JavaScript development and testing experience better and better.

As you also probably already know, wallaby.js updates itself automatically (with your permission) and is meant to be an ‘evergreen’ tool.

So what does exactly version 1 mean, apart from all these new features?

It basically means that we are now confident to recommend using the tool on everyday basis for your everyday work. By no means we are claiming wallaby.js totally bug-free, nor we’re freezing the existing set of features. Check out what’s on our roadmap and please keep coming up with new ideas and suggestions. So the version number is just a formality to tell you that now, when our idea became a solid reality, we care even more about you and your experience with wallaby.js.

And the great news is that the tool is still free to use. We are still working on the licensing and pricing policy and will announce it in advance, so you have enough time to make a decision about the purchase.

Enough talk, let’s jump to the new features!

Node.js and io.js

When we have just started developing wallaby.js, since we are using io.js internally, our dream was to use wallaby.js for developing wallaby.js. A few weeks ago, the dream came true, and today we excited to release node.js and io.js testing support for everyone.

From what we know (correct us, if we are wrong, in comments), wallaby.js is the first and (for now) the only specialized continuous testing tool for node.js/io.js.

In order to use wallaby.js for testing your node.js code, just specify the environment type in your configuration file, and you are ready to go. Right now we support Mocha framework, feel free to create more framework feature requests in our public repository.

module.exports = function () {
  return {
    files: [
      'lib/*.js'
    ],

    tests: [
      'test/*Spec.js'
    ],

    env: {
      // use 'node' type to use node.js or io.js
      type: 'node',

      // if runner property is not set, then wallaby.js
      // embedded node/io.js version will be used
      // runner: 'node or full path'

      params: {
        runner: '--harmony --harmony_arrow_functions',
        env: 'ENV_PARAM1=true;ENV_PARAM2=false'
      }
    }
  };
};

See more details in our documentation, check out this sample repository with configuration to quickly get started.

ES6/ES2015/Harmony, React JSX

Wallaby.js now supports ES6/ES2015 and JSX. ES6 is supported both natively for engines that can execute it and via code transpilers, such as Babel (former 6to5). You can see realtime coverage for your generators, classes, arrow functions and much more with wallaby.js. For JSX, not only wallaby.js understands its syntax, it also displays coverage inside JavaScript expressions within JSX elements.

Check out React sample repository or ES2015/ES6 sample repository with configuration to quickly get started.

JavaScript file configuration and preprocessors

Wallaby.js now supports JavaScript file configuration in addition to its existing JSON configuration, as well as Karma-like preprocessors. It means that now you can create files and tests lists dynamically or import them from your Karma config file. With preprocessors you can modify your source files content before feeding them to the test sandbox, synchronously or asynchronously.

module.exports = function () {
  return {
    files: [
      'src/*.js'
    ],

    tests: [
      'test/*Spec.js'
    ],

    preprocessors: {
      '**/*.js': file => require('babel').transform(file.content, {sourceMap: true})
    }
  };
};

See more details in our documentation.

We encourage to use existing npm packages and write simple preprocessors without too much ceremony when possible, preprocessor functions however don’t have to be inline (or use ES6), feel free to encapsulate them anywhere, reuse, wrap and adapt karma plugins or other npm packages.

Over time we are planning to expose more and more extension points, but would like to do it carefully to keep wallaby.js public API as simple and clean as possible. In a meantime, Karma to Wallaby adapter could be a nice and interesting open source project.

Require.js support

Last but not least, wallaby.js now supports require.js. The set up is similar to Karma, but easier. Please follow the instructions provided in this sample repository readme file.

That’s it for the blog post, thanks for reading and hope you enjoy using new wallaby.js features. More blog posts are coming the next few weeks, follow us on twitter to stay updated.

Testing angular.js application with wallaby.js

2015-02-16T00:00:00+00:00

In case if you haven’t seen wallaby.js yet - it is an intelligent test runner for JavaScript, that runs your tests and displays the feedback as you write your code. Wallaby.js runs your tests very fast - it uses runtime dependency analysis to only execute tests affected by your code changes and parallel test execution.

In this tutorial, I’ll demonstrate to how to set up wallaby.js for a typical angular.js project and use it in WebStorm. The same way you can also use it in PhpStorm, RubyMine, PyCharm or IntelliJ IDEA.

First, open this angular.js todo mvc sample project. You can either clone the repository:

git clone https://github.com/wallabyjs/wallaby-angular-todomvc.git

and open the project in WebStorm, or just open WebStorm and clone the repository right from the splash screen.

Second, download and install wallaby.js, if you haven’t already. To do that:

Download wallaby.js plugin from the website.
Install it by opening WebStorm settings dialog and using Install plugin from disk button on Plugins configuration page.

Next, we need to install the project dependencies. To do this, run:

npm install

using your CLI, WebStorm built-in Terminal, or just by right-clicking on the package.json file in the project root folder and selecting Run ‘npm install’ action.

Once all the dependencies have been installed, we need to create wallaby.js configuration file. All we need to do is to let wallaby.js know what source files it should track and where our tests are.

Create wallaby.js file in the project root folder and add the following content to it:

module.exports = function () {
  return {
    "files": [
      {pattern: 'node_modules/angular/angular.js', instrument: false},
      {pattern: 'node_modules/angular-route/angular-route.js', instrument: false},
      {pattern: 'node_modules/angular-mocks/angular-mocks.js', instrument: false},
      'js/**/*.js'
    ],
    "tests": [
      'test/unit/**/*.js'
    ]
  };
};

This means that wallaby.js should use angular.js from the installed node_modules, track all source files located in js folder and run all tests located in the test/unit folder.

Other configuration options and the full format of wallaby.js configuration file is described in wallaby.js documentation.

Now we need to create wallaby.js Run Configuration, that we will be using to start and stop wallaby.js. Select Run menu, Edit configurations and create new wallaby.js configuration.

Give the configuration name you like and from the Configuration file drop-down list, select the JSON file we have just created.

We are ready to go now - start the created configuration using Run menu.

After the start, wallaby.js displays its tool window at the bottom:

and its status indicator at the right bottom corner of the window.

Once started, you don’t need to restart wallaby.js run configuration to run your tests, as it does it automatically, as you change your code. You can even hide the tool window by using the Hide icon in its right top corner, as you will be getting the information about failing tests right in the context of the code that you’re editing. You can always get the tool window back by clicking wallaby.js status indicator.

So, let’s see wallaby.js in action. Open one of the application source files, for example angular.js controller todoCtrl.js.

When wallaby.js is up and running, you should be able to see coverage indicators in the gutter of the code editor.

Gray square means that the source line is not covered by any of your tests.
Green square means that the source line is covered by at least one of your tests.
Yellow square means that the source line is only partially covered by some of your tests.
Red square means that the source line is the source of an error or failed expectation, or is in the stack of an error.
Pink square means that the source line is on the execution path of a failing test.

On any line of the source code you can hit Alt + Enter (or use the light bulb icon) and display wallaby.js context actions. You can read more about all actions in the documentation. For this example, let’s see all uncovered regions of the source file that we have opened.

We can now clearly see what is exactly not covered inside the line with the yellow coverage indicator, because wallaby.js tracks coverage for logical expressions and conditional operators as well.

Now, let’s see what tests are covering the source file. To do that, on any line hit Alt + Enter and select the Show line test(s) action.

Tests window is displayed and all the tests that are covering this line of code are displayed inside. If any of the tests is failing, then full error stacks and expectation errors are displayed as well.

The window is searchable with Ctrl/Cmd + F and all the links inside it can be clicked using your mouse or used from the keyboard with Ctrl/Cmd + B shortcut. You can hide the window by hitting Esc.

While inside the window, let’s navigate to one of the covering tests by using the link next to the test full name.

The test is checking that if we change some todo item title to be an empty string, then, when saving items, the item with empty title should be removed. Now let’s change the expectation code to see how wallaby.js reacts.

Immediately as we do so, wallaby.js displays the expectation error right where it occurs. The test path line indicators become pink, the source of the error line indicator is now red.

Apart from displaying errors where and when they occur, wallaby.js also displays console.log output in a similar manner, so if you’re a fan of console.log, you are going to love what you see next:

We pass todo object to the console.log and we can immediately see the result. You can output anything and anywhere this way, to quickly inspect the objects that you’re interested in.

Some objects or errors may contain large amount of data, that is not viewable inline. To inspect the full message, you can always use Alt + Enter and invoke Show line test(s) action to see the full output in the context of some source code or a test that you are interested in.

Finally, let’s get back to the controller source file todoCtrl.js, find some uncovered function and write a test for it.

For example, let’s cover removeTodo function with a test. To do that, I will add a new test to one of the existing test files. In the test, I will get one of the pre-created todo items, pass it to the removeTodo function and ensure that the total number of the items has decreased by one.

That’s it for the blog post, thanks for reading and hope it helps you to get started with wallaby.js in your project. Follow us on twitter to stay updated.

Wallaby.js - unleash the full power of your tests

2015-01-30T00:00:00+00:00

Many JavaScript developers understand the importance of writing tests and do use them. There are a few testing frameworks and test runners available in JavaScript land. Some of them are even nicely integrated into some code editors. For example, there’s a nice integration of Karma runner in WebStorm. There are some automated testing workflows, where you can enjoy your tests’ feedback after a few seconds or so (depending on the size of your codebase).

How we use our JavaScript tests

The issue is that an average JavaScript codebase size is growing. We write and test more JavaScript code, but at the same time we want to run our tests faster, because faster feedback means better control over the growing codebase. But when we have hundreds and thousands of tests, even relatively fast ones, it becomes quite a burden to run them frequently, as we write our code. Yes, we’re still running them all as a part of our CI build, we’re inspecting generated coverage reports, we’re using various cloud and on-premises solutions to run our JavaScript tests in different browsers and on different devices. But it doesn’t help us where we’re spending the most of our time - when we are writing the bulk of our application logic.

Each good test we write adds some value, and, in JavaScript, tests are more than just recorded assumptions, But we are not using our tests full potential. They can do much better job than they are doing now. They can run on every key stroke and provide some helpful feedback, immediately highlight found issues in the code that you’re writing. But when we run them all and sequentially - we can’t have it. It’s like having a sports car and only driving it once a day and within a school zone speed limits.

Unleash the full power of your tests

In wallaby.js, I have decided to tackle these issues and build an intelligent test runner, that allows you to exploit the full power of your tests.

First, code coverage is not an option - it’s the must have. But that’s not those boring numbers of coverage percentage I am talking about. When I am writing my code and tests, I don’t care whether I have 89% or 91% percent of my code covered. I don’t want to know if it’s a statement coverage or a condition coverage, when I’m typing my function code. I just want to see coverage indicators right when and where I’m editing my code and without too much distraction.

Second, if I am editing a single test or a function that I know is only covered by a few tests, why my test runner can’t know about it and use the knowledge? I just want it to run everything that depends on my change, everything that may change its behaviour after my change - no more, no less.

Third, if my change requires a lot of tests to be executed and my test files are independent (and they should be), I want to use the full power of my machine and run tests in parallel.

Wallaby.js design is based on the simple principles listed above and is focused on improving your everyday and every minute code writing experience. Inline error/console messages and expectation failures, where and when they occur, test execution screenshots, context actions - these are just few things wallaby.js provides to make you more productive.

With wallaby.js, there’s no longer a wall between you and the full power of your tests. Go ahead, install wallaby.js and release the beast from its cage.

JavaScript backdoor debugging

2014-11-14T00:00:00+00:00

Sometimes I put “console.log” in my JavaScript code just to inspect some latest object values in some deeply nested scopes of my application. Quite often I realise that one “console.log” isn’t enough, so I add more, and so forth. Occasionally I need to modify some objects in the code that I’m tracing on the fly, right from the console, but I don’t have an access to the scope that I’d like to alter.

To address the above listed issues, I sometimes use a simple debugging technique, that I call backdoor debugging. The idea is very simple: instead of (or in addition to) putting one or more “console.log” statements into some nested function scope(s), I am creating a closure in the deepest scope. The closure is doing just one thing: evaluating whatever code is passed into it. For example:

(function () {
  var totalNumberOfDrinks = 0;
  var totalNumberOfCodeLines = 0;

  var programmer = {
    writeCode: function () {
      totalNumberOfCodeLines += Math.floor(Math.random() * 100);
    },

    drinkCoffee: function () {
      totalNumberOfDrinks += Math.floor(Math.random() * 3);
    },

    isSleepy: function () {
      backdoor(function my(p) {return eval(p);});
      return Math.random() < .5;
    }
  };

  if (programmer.isSleepy()) {
    programmer.drinkCoffee();
  }

  var codeWritingInterval = setInterval(function () {
    programmer.writeCode();
  }, 100);

})();

The code is pretty simple: IIFE initializes a couple of variables (that I don’t have an access to from the global scope) and creates a “programmer” object. The programmer drinks coffee if sleepy and then starts writing some code - sounds familiar, doesn’t it?

I have also placed the “backdoor” named “my” inside “isSleepy” function, so now I can run arbitrary code with access to the “isSleepy” function and all scopes it is nested in. Let’s open console and have some fun. For example I can query how many lines of code have the programmer wrote so far, or even forever stop the programmer from writing code (sounds terrible though).

The backdoor function is extremely simple. You can grab the source code from the backdoor.js repository. Any ideas about how to make it even more simple or extend it are welcome.

JavaScript tests cost efficiency

2014-11-10T00:00:00+00:00

Everything has its price. Adding a new feature, maintaining an existing one, writing tests. When you are writing and maintaining tests in C# or Java, you are getting recorded and executable scenarios with your assumptions for that price. What about JavaScript, are you getting exactly the same?

Let’s imagine that in the code below, I have covered the “try” block in my tests, but didn’t bother covering “catch” block as I thought it is trivial and not worth testing.

var myService = { 
  'doWork': function() { throw new Error('error'); },
  'cancelWork': function() {},
  'handleError': function() {},
  'stoр': function() {}
};

try {
  myService.doWork();
}
catch (err) {
  myService.cancelWork();

  if (e.message) {
    myService.hanldeError();
  }
  
  myService.stop();
}

How many problems did I miss? Simple code editor will find nothing. Careful reader may find one or two issues. Intelligent code editor with nice static analysis and spell checker may find all three, though there are many cases when it won’t help either as static analysis capabilities in the dynamic world of JavaScript are limited. Test will find all three errors just as a free addition to its main duty.

So in case if you are still looking for more reasons to cover your JavaScript code, here is one more - JavaScript tests are more cost efficient than in C# or Java, because for the same price you are also getting some form of a compiler for your code.

Tools vs. Frameworks

2014-11-03T00:00:00+00:00

How many times did you wish there was a good tool for some web programming/testing related task and you couldn’t find one? Now how frequently an average JavaScript developer needs a new framework with some features that are not already present anywhere else? And yet, the next OSS project that developer will start is going to be a framework not a tool.

While not necessarily a bad thing, why does it happen? One may say that good tools are hard to create. Well, so as good frameworks. Your thoughts?