What I Learned by Being My Own Test Engineer

The company I work for shifted focus from producing features very quickly to getting our code quality very solid to allow us to add new features without regressions. This means that it's "all hands on deck" for putting necessary tests in place to catch as many regressions as possible before we switch back into building new features.

I previously thought that it might not be worthwhile to have UI developers dive into Cucumber, Gherkin, and Selenium in Java, but this experience has opened my eyes. I've learned some things I'd been wondering about how the QA side of things works, as well as somethings I didn't know I didn't know.

QA Has a Set of Steps Written Down That They Run Through to Test

When we started writing our integration tests, one of our test engineers created a set of test cases for us to code. I'd been aware that there was a separate system used by QA, but not exactly what was in it. I belatedly realized that QA had been writing these kind of steps down all along–in that system I didn't have access to. The UI team had asked before for a set of steps to run through before submitting to QA, without success. I think the QA team had no idea we didn't have access to the system or know how to use it, and they probably wondered why we weren't using their steps. Classic miscommunication.

I also noticed how clear and precise the steps in the test case were relative to the requirements we usually get. The Project Owners I've worked with are typically focused on what they want to enable the user to do and how it should look, but there are often gaps in between. I really see the value in having the kinds of steps in a Gherkin Specification and getting sign-off before you start coding. So far, we've been writing these after the fact, but maybe in the future we can bring that forward in the process.

Some Automation Problems Are Probably Quicker and Easier to Solve if the Person Automating is Also the Developer

I've long thought that CSS in JS was one of those trends that codes around a basic lack of understanding, but it's an absolute nightmare from an automation perspective. A developer on our team had chosen to use React-Select for a search box. By default, this library only attaches dynamically-generated class names and inline styles, which makes it impossible to reliably target things like, oh, all the currently-displayed options.

Probably a test engineer would have either tried to drill down to the desired element with XPath or would have asked me to add an ID to the container inside react-select (which as far as I know is impossible). The XPath solution might have worked, but would have been extremely brittle. I would have had to say no to the impossible request. And that probably would have been the end of it for being able to test that functionality.

But since I understood the context of what was needed from the front-end developer side and I was really determined from the test engineer side, I was able to figure out how to get react-select to apply something akin to semantic class names (albeit it still feels a bit hacky and fragile based on the fact you have to concatenate the classNamePrefix you provide together with another string you can find only by inspecting an instance of the component to get the other part of the class name to use).

I'm going to add an editorial comment here that there's just no polite way to say–CSS in JS is crap. But if you're going to use it and you're developing a library you want others to use, please also provide semantic class names that can be used on their own for developers who need to integrate your component into a look and feel provided by designers who don't really have detailed understanding about how you intended customization to work. They are pretty much guaranteed to specify something you didn't imagine, that's very difficult to provide without hooks to allow developers to target the innards of your component.

Also, basic things like being able to look at the source of a page and provide an anchor link to a specific spot (like I tried to do for "classNamePrefix," in the react-select docs to use in the above paragraph) become impossible. And, circling back, semantic class names will make your code much easier to automate as well. You don't have to attach any styling of your own to the sematic class names if that bothers you. But please provide the hooks, for everyone else's sake!

Unit Tests Won't Catch Everything

I formerly had the opinion that if you practiced Test-Driven-Development on the front end and your tests covered pretty much all new functionality, that integration problems would be really obvious when you ran the code. Then you could easily see them and fix them. But I found a number of subtle bugs when writing these test cases that my unit tests hadn't caught.

For example, every test case has a requirement to check the console log for errors. This caught a call that we were making to a third-party component that's not entirely built in React that happened before that component had loaded, which threw a null pointer exception. In this case, the only side-effect was a console error, but you don't want your code that's aimed at savvy developers throwing console errors. It doesn't inspire a warm fuzzy.

In another place, I found that logic that merges the results of two API calls in different formats to generate a series of other API calls was not merging those calls correctly with realistic data, rather than our limited test data that we use while developing. That's not something my unit tests are equipped to catch.

In Summary

I'm probably not going to "quit my day job" to become a full-time QA Engineer, but I feel like pulling back the curtain some has really helped me as a developer. Even aside from the fact that thinking about things in an OO, strongly-typed way for a change has given me a fresh perspective, understanding more about how the QA sauce is made has made me way more sympathetic to the issues our test engineers face. Hopefully, I'll be able to provide more QA-friendly code in the future.