Today I am going to actually write about a tool my team introduced recently but I am going to spend some time getting to know better.
Our team is producing the front end for a very complex, big data, back end. We spend a lot of time with JS and CSS and are finding that the focus on Visual Design is very high, but not always robust between different screen sizes or when different elements get added to the pages. Over the course of the last 6 months, we have spent a lot of time pushing, and re-pushing, pixels to get aligned with a photoshop design (of which we can not inspect since developers do not get photoshop licenses).
We tackled the long feedback loops with a couple of different collaboration wins, including daily screen shares/phone calls with the Visual Designer and weekly in person pairing sessions. But there was still a concern that each time the designer looked at our site, something else got pushed. Now I had a sneaking suspicion that while some of this was regression, it could not possibly all be regression, yet we had no way to compare what was asked for and what was current since a lot was done in Photoshop files and/or by voice.
Enter visual regression testing.
Sigh. I already know that I have lost a large group of you because visual regression testing can be flakey, high cost/low value, etc, etc, etc. I too am very skeptical, but this is by far the best use case I have seen and the dev team is actually who suggested it for us so it is team bought in.
So we put a card in our backlog and we spiked out some options and ended up implementing BackstopJS. I hadn’t heard of it before, but it seemed to meet our needs and was quick to get running so we figured we would give it a shot.
So far we have felt some of the pain of a relatively new tool, as well as some of the pain of visual regression testing, but we are about to put it to the test as we shift focus from one set of screens to another. With this shift, we will be refactoring a lot of SCSS files to limit duplication, and in order to move fast, we need to have confidence that the original set of pages are not changing. Sound familiar? Yup. Pretty much the textbook definition of when you need to introduce testing.
So today I am going to dig into the tool a bit more, and in particular try to tackle an issue we are seeing where the element we want to screenshot does not get found on first try and can create false negatives.