Skip to main content

Incrementally adding Stylelint rules with Betterer ☀️

· 6 min read

I just released v4.0.0 of Betterer 🎉 (now with sweet new docs!) and it has a bunch of simplified APIs for writing tests. And just before I shipped it, I got an issue asking how to write a Stylelint test, so let's do it here and explain it line by line:

TL;DR;#

Here's the full test:

// stylelint.tsimport { BettererFileResolver, BettererFileTest } from '@betterer/betterer';import { promises as fs } from 'fs';import { Configuration, lint } from 'stylelint';
export function stylelint(configOverrides: Partial<Configuration> = {}) {  const resolver = new BettererFileResolver();  return new BettererFileTest(resolver, async (filePaths, fileTestResult) => {    const result = await lint({      files: [...filePaths],      configOverrides    });
    await Promise.all(      result.results.map(async (result) => {        const contents = await fs.readFile(result.source, 'utf8');        const file = fileTestResult.addFile(result.source, contents);        result.warnings.forEach((warning) => {          const { line, column, text } = warning;          file.addIssue(line - 1, column - 1, line - 1, column - 1, text, text);        });      })    );  });}

And then using the test:

// .betterer.tsimport { stylelint } from './stylelint';
export default {  'no stylelint issues': stylelint({    rules: {      'unit-no-unknown': true    }  }).include('./**/*.css')};

NTL;PR (not that long, please read 😂)#

Stylelint#

So how does it all work? Let's start with the actual Stylelint part.

Stylelint is pretty easy to set-up. You need a stylelintrc.json file with configuration:

{  "extends": "stylelint-config-standard"}

And then run it on your CSS files:

stylelint "**/*.css"

Running that does the following:

  1. searches for the stylelintrc.json configuration file
  2. reads the configuration
  3. finds the valid files
  4. runs the rules
  5. returns the results

Stylelint also has a JS API which we're going to use:

import { lint } from 'stylelint';
const result = await lint({  // ...});

We could just run the above and it will test the current state of the files with the current configuration in stylelintrc.json. And that's great ✨!

Augmenting the configuration:#

For the Betterer test we want to augment the stylelintrc.json configuration with some extra rules... and Stylelint has a really easy way to do that:

import { Configuration, lint } from 'stylelint';
function stylelint(configOverrides: Partial<Configuration> = {}) {  const result = await lint({    configOverrides  });}

Passing the list of files:#

Stylelint also allows us to pass a specific set of files to test:

import { Configuration, lint } from 'stylelint';
function stylelint(configOverrides: Partial<Configuration> = {}, files: Array<string>) {  const result = await lint({    files,    configOverrides  });}

So we could call the stylelint function like:

stylelint(  {    rules: {      'unit-no-unknown': true    }  },  './**/*.css');

And that will run the Stylelint from the stylelinerc.json file, plus the unit-no-unknown rule, on all .css files! Thats most of the tricky stuff sorted ⭐️!

Hooking into Betterer:#

This test needs to take advantage of all the snapshotting and diffing magic of Betterer, so we need to wrap it in a test. We want to be able to target individual files, so it specifically needs to be a BettererFileTest.

We first create a BettererFileResolver(), which is a little bit of magic that helps work out which file paths are relevant for the test. That is passed as the first argument to BettererFileTest. The second argument is the actual test, which will be an async function that runs the linter.

import { BettererFileResolver, BettererFileTest } from '@betterer/betterer';import { Configuration, lint } from 'stylelint';
function stylelint(configOverrides: Partial<Configuration> = {}) {  const resolver = new BettererFileResolver();  return new BettererFileTest(resolver, async (filePaths) => {    // ...  });}

Each time it runs Betterer will call that function with the relevant set of files, which we will pass along to Stylelint:

import { BettererFileResolver, BettererFileTest } from '@betterer/betterer';import { Configuration, lint } from 'stylelint';
function stylelint(configOverrides: Partial<Configuration> = {}) {  const resolver = new BettererFileResolver();  return new BettererFileTest(resolver, async (filePaths) => {    const result = await lint({      files: [...filePaths],      configOverrides    });  });}

Adding files:#

Next thing is telling Betterer about all the files with issues reported by Stylelint. To do this we can use the BettererFileTestResult object, which is the second parameter of the test function:

new BettererFileTest(resolver, async (filePaths, fileTestResult) => {  // ...});

The result object from Stylelint contains a list of results. For each item in that list, we need to read the file with Node's fs module, and then call addFile() with the file path (result.source), and the contents of the file. That returns a BettererFile object:

import { promises as fs } from 'fs';
await Promise.all(  result.results.map(async (result) => {    const contents = await fs.readFile(result.source, 'utf8');    const file = fileTestResult.addFile(result.source, contents);  }));

Adding issues:#

The last thing to do is convert from Stylelint warnings to Betterer issues. To do that we use the addIssue() function! In this case we will use the following overload:

addIssue(startLine: number, startCol: number, endLine: number, endCol: number, message: string, hash?: string):

Stylelint only gives us the line and column for the start of the issue, so we use that as both the start position and the end position. Betterer expects them to be zero-indexed so we subtract 1 from both. This also means that the VS Code extension will add a diagnostic to the whole token with the issue, which is pretty handy! We also pass the text of the issue twice, once as the message, and a second time as the hash. The hash is used by Betterer to track issues as they move around within a file. Stylelint adds specific details to the message so that makes it a good enough hash for our purposes. All up, converting an issue looks like this:

result.warnings.forEach((warning) => {  const { line, column, text } = warning;  file.addIssue(line - 1, column - 1, line - 1, column - 1, text, text);});

The whole test:#

Putting that all together and you get this:

// stylelint.tsimport { BettererFileResolver, BettererFileTest } from '@betterer/betterer';import { promises as fs } from 'fs';import { Configuration, lint } from 'stylelint';
export function stylelint(configOverrides: Partial<Configuration> = {}) {  const resolver = new BettererFileResolver();  return new BettererFileTest(resolver, async (filePaths, fileTestResult) => {    const result = await lint({      files: [...filePaths],      configOverrides    });
    await Promise.all(      result.results.map(async (result) => {        const contents = await fs.readFile(result.source, 'utf8');        const file = fileTestResult.addFile(result.source, contents);        result.warnings.forEach((warning) => {          const { line, column, text } = warning;          file.addIssue(line - 1, column - 1, line - 1, column - 1, text, text);        });      })    );  });}

And then we can use the test like this:

// .betterer.tsimport { stylelint } from './stylelint';
export default {  'no stylelint issues': stylelint({    rules: {      'unit-no-unknown': true    }  }).include('./**/*.css')};

And that's about it! The Stylelint API is the real MVP here, nice job to their team! 🔥🔥🔥

Hopefully that makes sense! I'm still pretty excited by Betterer, so hit me up on Twitter if you have thoughts/feelings/ideas 🦄

Improving accessibility with Betterer ☀️

· 4 min read

So, yesterday I announced the new release of ☀️ Betterer, thanks if you've checked it out already!

I wanted to write another post describing a different example, this time with a custom test instead of a built-in test! Let's take a look at how we can prevent accessibility regressions (and hopefully encourage improvements!) 👀

Betterer TL;DR#

Betterer is a test runner that helps make incremental improvements to your code! It is based upon Jest's snapshot testing, but with a twist...

Betterer works in two stages. The first time it runs a test, it will take a snapshot of the current state. From that point on, whenever it runs it will compare against that snapshot. It will either throw an error (if the test got worse ❌), or update the snapshot (if the test got better ✅). That's pretty much it!

Our first Betterer test#

Writing a test with Betterer involves implementing two functions! More formally, we need to implement the BettererTest type:

type BettererTest<ResultType> = {  test: () => ResultType | Promise<ResultType>;  constraint: (result: ResultType, expected: ResultType) => ResultType | Promise<ResultType>;};

So we need to write two functions:

  • test - the action that needs to happen to get a result,
  • constraint - the rule to apply to check if the result is better, worse or the same

We can implement these in their own file, or straight in the .betterer.ts file. To keep it simple, we'll do the latter:

// .betterer.tsexport default {  'improve accessibility': {    test: ...?,    constraint: ...?  }};

Writing the test#

To implement our test, we're going to use Puppeteer and Axe. Puppeteer is a tool that will run a browser and load a page. Axe is a set of accessibility audits that we can run over a web page. We're also going to use Axe Puppeteer which makes it a bit easier to use Axe with Puppeteer.

Lucky for us, we can take the example straight from the Axe Puppeteer documentation! 😍

We launch Puppeteer, get the page that it creates for us and navigate to a website. Then we execute Axe and get the results. Next, we close the page and the browser, before finally returning the number of violations! 🤓

import { AxePuppeteer } from 'axe-puppeteer';import * as puppeteer from 'puppeteer';
async function improveAccessibility() {  const browser = await puppeteer.launch();  const [page] = await browser.pages();
  await page.goto('https://phenomnomnominal.github.io/betterer');  const results = await new AxePuppeteer(page).analyze();
  await page.close();  await browser.close();
  return results.violations.length;}

That's our test sorted!

Writing the constraint#

Now what about the constraint? Since our test returns a number, we just need to compare the two results. We want our test to improve when the result is smaller, so the constraint should look something like this:

import { ConstraintResult } from '@betterer/constraint';
function constraint(result: number, expected: number): ConstraintResult {  if (current === previous) {    return ConstraintResult.same;  }  if (current < previous) {    return ConstraintResult.better;  }  return ConstraintResult.worse;}

Comparing numbers is fairly common, so we can use the pre-defined smaller or bigger constraints:

import { smaller } from '@betterer/constraint';

So I kind of lied, you can write a test with just one function! 😅

The whole thing#

Putting it all together, we have our test:

// .betterer.tsimport { smaller } from '@betterer/constraint';import { AxePuppeteer } from 'axe-puppeteer';import * as puppeteer from 'puppeteer';
export default {  'improve accessibility': {    async test() {      const browser = await puppeteer.launch();      const [page] = await browser.pages();
      await page.goto('https://phenomnomnominal.github.io/betterer');      const results = await new AxePuppeteer(page).analyze();
      await page.close();      await browser.close();
      return results.violations.length;    },    constraint: smaller  }};

How's that look? Not bad I reckon! Betterer will run the test for us and update the test snapshot whenever the results get better. That will help make sure that our audit score only goes in the right direction.

This test isn't perfect - you may noticed that it doesn't matter what the violations are, but how many there are! That's something that we could improve later! For now it will stop us introducing more audit violations, which is a good start ⭐️⭐️⭐️

We could improve this test by keeping track of the specific violations that occurred, so we can have a clearer definition of what better or worse really is, but let's leave that for another day!

That's it!#

That's all I got for now. Please let me know what you think on Twitter! 🦄

Betterer v1.0.0 ☀️

· 8 min read

I'm stoked to announce v1.0.0 of Betterer!

I've been locked down in New Zealand for the last little while, and I've used some of that time to smash out what I think is a pretty compelling v1 release of a tool that I'm really excited about!

What is Betterer?#

Betterer is a test runner that helps make incremental improvements to your code! It is based upon Jest's snapshot testing, but with a twist...

I'm sure many of us have been in situations where we've seen big changes we'd like to make, or new standards or design decisions that we'd like to encourage, but we just don't have the time to do it.

Usually one of two things happen:

  1. You start a long-lived branch that is awful to maintain and often impossible to merge. It ends up being a time sink ⏱

  2. You and your team make an agreement to make the improvement over time. It gets forgotten about and nothing gets better (in fact usually it gets worse!) 😕

I've seen this happen time and time and again! Sometimes it's introducing a new style rule to a codebase. Other times it's enabling stricter compilation, or decreasing the number of accessibility failures!

Betterer works in two stages. The first time it runs a test, it will take a snapshot of the current state. From that point on, whenever it runs it will compare against that snapshot. It will either throw an error (if the test got worse ❌), or update the snapshot (if the test got better ✅). That's pretty much it!

How does it work?#

To get started, you can run the following from the root of your project:

npx @betterer/cli init

That will give you a brand new .betterer.ts config file which looks something like this:

// .betterer.ts
export default {  // Add tests here ☀️};

From here, it's up to you to add some tests!

Let's imagine you're working with a codebase that uses Moment.js. You'd like to migrate away from it for performance reasons.

// src/subtract.jsimport * as moment from 'moment';
const now = moment();
console.log(now.subtract(4, 'years'));

Let's also imagine that you're using ESLint in this codebase. One approach to remove Moment.js might be to use the no-restricted-imports ESLint rule, add the eslint-disable-next-line comment all over the place, and cross your fingers that people don't just add more... 🤔

Betterer gives us a better option! We can create a test for that specific rule:

// .betterer.tsimport { eslintBetterer } from '@betterer/eslint';
export default {  'no import from moment': eslintBetterer('./src/**/*.js', [    'no-restricted-imports',    [      'error',      {        name: 'moment',        message: 'Please use "date-fns" instead.'      }    ]  ])};

The first time we run the test with Betterer, it will look something like this:

Terminal output showing Betterer running and indicating that the "no import from moment" test has run for the first time

Betterer has now created a snapshot of the current state, stored by default in a .betterer.results file:

// BETTERER RESULTS V1.exports[`no import from moment`] = {  timestamp: 1589459511808,  value: `{    "src/subtract.js:566118541": [      [0, 0, 33, "\'moment\' import is restricted from being used. Please use \\"date-fns\\" instead.", "4035178381"]    ]  }`};

The snapshot contains information about the current issues in the code.

The next time we run the test, it will look like this:

Terminal output showing Betterer running and indicating that the "no import from moment" test has run and stayed the same

Now, someone else on the team comes along and doesn't know about the new rule, and they add a new file that uses Moment.js:

// src/add.jsimport * as moment from 'moment';
const now = moment();
console.log(now.add(4, 'years'));

When Betterer runs on their code, they get a nice big error:

Terminal output showing Betterer running and indicating that the "no import from moment" test has run and got worse

Even though a new issue has been introduced, the .betterer.results file doesn't change!

Our teammate reads the helpful error message from ESLint and they update their code to use date-fns...

// src/add.jsimport { addYears } from 'date-fns';
console.log(addYears(Date.now(), 4));

... and once again Betterer tells them that the result is the same:

Terminal output showing Betterer running and indicating that the "no import from moment" test has run and stayed the same

Our teammate has a bit of time on their hands, so they decide to fix up our usage of Moment.js as well! 🎉

This time when they run Betterer, everything is good:

Terminal output showing Betterer running and indicating that the "no import from moment" test has run and got better

There are now no remaining issues, so this test has met its goal. Since the existing issue has been resolved, it is removed from the snapshot in the .betterer.results file. This means we can move the rule from Betterer over to the normal ESLint configuration, so we don't reintroduce the issues again.

Pretty neat eh! That's an example of the built-in @betterer/eslint test, but there are other built-in tests too. And you can of course write your own tests! Check out the documentation for more details (still a WIP 🚧)!

What's in v1.0.0?#

Everything I've mentioned so far has been working for a while! Over the last few months I've really solidified the implementation (basically a whole rewrite to be honest!):

  • Better error handling and error messages
  • Better issue comparison. It now understands file renames and issues that around within the same file
  • The ability to run tests on a single file via the JS API, with betterer.single
  • A whole bunch more tests!

But I've also added a few key features that are worthy of a 1.0.0 release! 🔥🔥🔥

Force Update (!)#

First things first, you can now run Betterer with the --update flag, and the snapshot will be updated even if it got worse! This is handy for when you need to ship something, even if it makes it temporarily worse:

betterer --update

And because this is shamelessly stolen from Jest, you can also use -u.

Watch mode (!!!)#

This one is huge! All the changes that I made were building up to this. You can now run Betterer in watch mode and get feedback as you fix up issues:

Terminal output showing Betterer running in watch mode indicating that the "no import from moment" test has run on a specific file and got better

Same rules apply here, the snapshot will update whenever the test gets better!

There's a bunch of other cool things that could happen with watch mode (gamification much?), so I'm pumped that it's working! 🤩

VS Code extension (!!!!!)#

Way to bury the lede! This is probably the coolest bit! Let's just say it was a build up.

Betterer runs entirely in its own world, so the usual ESLint or TypeScript extensions can't report the issues. But everyone loves seeing red squiggly lines in the code editor, so Betterer now has its very own VS Code Extension 🤯!

Initialise in a new project#

You can run the betterer.init command in a project! It will generate config files and update your package.json with scripts and dependencies:

VS Code screen capture output showing Betterer being initialised in a project

See all issues in a file#

And when you've got some tests setup, it will show you all the existing issues in a file, and when they were first created. And it will show you any new issues as you make them:

VS Code screen capture output showing Betterer highlighting issues in a project

I reckon that's pretty sweet!

So, what's next?#

Well, you're going to try out Betterer and open lots of issues and help me make it better(er)! If you'd like to contribute...

  • There are definitely bugs to fix
  • The test coverage is pretty good, but there aren't any E2E tests for the VS Code extension yet
  • The logging and reporting implementation could use some work
  • There is so much documentation to be written

What a time to be alive! 🤓

In case you can't tell, I'm unreasonably excited about this and I really think this will help with large and legacy codebases. Please let me know what you think on the Twitters!