Forem: Franco

Let me give you a piece of advice: don’t use AI (Copilot, GPT, etc.) for testing. Don’t get me wrong—what I mean by this is: use AI for testing in moderation, but please, don’t delegate all your testing responsibilities to these tools.

Why?

Okay, I admit it—I’m a big fan of testing. I believe that, at an enterprise level, testing = code = documentation in terms of importance. Without proper tests, it’s incredibly difficult to deploy correct features to production that meet the requirements. It’s also challenging to maintain these features in the long run.

Whether you engage in defensive testing (writing tests after the code) or offensive testing (writing tests before the code, such as using TDD), the mere act of writing tests forces you to think critically about the code. It compels you to review the main cases and edge cases, giving you an extra opportunity to ensure the code is as robust as possible before it reaches staging or production.

Looking Beyond

If we underestimate testing and treat it as a "necessary evil" while working on a ticket, we risk falling into the trap of a typical Copilot prompt: “Write a unit test for this class.” Worse yet, we might simply copy-paste the result without even reviewing it. This can lead to weak tests that pass only because of superficial assertions like “true to be true” or similar.

While this might seem like a time-saver at first, avoiding the effort of writing tests from scratch, the reality is that in the short, medium, and long term, this approach will bring us significant headaches. The reason? The test won’t be solid enough to prove that the code meets its requirements and is free of critical bugs (not all bugs—since it’s impossible to prove the absence of bugs, only their presence).

Beyond Code

Beyond the code itself, this approach can harm us as developers. We risk losing reputation and credibility regarding the quality of our work. At the same time, we lose the chance to learn valuable skills related to testing, which I find both beautiful (subjectively speaking) and crucial.

It’s Like Everything Else

What I’m saying about testing applies to many other concepts in software development. It’s essential to understand that AI is just another tool in the developer’s arsenal. When used properly, it can make a significant difference—when used properly. It cannot be used blindly for everything, just as we can’t dismiss it entirely and refuse to use it at all.

As always, the challenge for software developers is to recognize this and balance the use of tools according to the context where they’re solving a problem. After all, a software developer’s job is to solve problems strategically, using the right tools to achieve that goal.

Conclusion

This perspective is based on my experience as a software developer. Of course, it’s far from being the ultimate truth and should be taken as one more point of view. I encourage you to share your thoughts in the comments so we can spark a discussion within the community and learn even more from each other.

Happy coding! 🤓

Don't use AI for testing

Franco — Sun, 22 Dec 2024 13:37:58 +0000

Why?

Looking Beyond

Beyond Code

It’s Like Everything Else

Conclusion

Happy coding! 🤓

Stop doing component unit testing

Franco — Fri, 24 Mar 2023 20:25:13 +0000

Introduction

The first time I entered a company as a developer, I did it as a sort of full stack, more oriented to Front End than Back End. At that moment, I was assigned to those who later became my very good mentors. Their role were basically sit next to me, and teach me the company rules, the project code, and algorithmic concepts that, for my zero experience, and my until that moment, academic formation, I don't knew very well.

One of that concepts, were "testing". Although I had an idea of what it meant, and I had interacted with it in the past years, I had never done automated testing from scratch, much less involving the company's tools and languages at the time (JS, Jest, Vue).

So, little by little, with the passing of the tasks, I was learning to do this, giving as results tests of the style:

// ExampleComponent.vue

// ExampleComponent.spec.js

Note that this test is an approximate and might not work, the important thing here is that you get the point. Here I'm using Vue as a Front End library https://vuejs.org/ and jest as a tool for testing https://jestjs.io/

The problem

Yes, this is a correct unit test, which covers the script tag of the example Vue component, and, in a way, gives us some security... it's better than nothing. But, with the passing of the developments and the use of the system by the users, it became more and more clear that this type of test did not cover all the possible errors.

In fact, the test is covering the behavior of the init method and dataToShow computed, but, we can see that we aren't testing the connection between the Vue component and its store, and we aren't testing the reaction of the component's template when the state of the variables change. In conclusion, we aren't close to simulate the user's behavior in the view.

Something I have learned as I have gained experience, is that while the more close is the test to the user, and the less mocks it has, the better.

While a unit test, tells us if the code of some method runs in an expected way, an integration test tells us if a complete use case behaves in an expected way, resulting in a stronger, safer, and more reliable test.

And believe me, in my case, this type of testing has caught bugs in situations that a unit test hasn't.

The solution

So, at the end, how can we do a test like that?

Well, the key here is to think on how we can convert the manual actions that the user does in the view, in automated actions in our test.

For the example we saw before, we have the following manual actions:

Open the view
Press the init button
View the information once it is loaded

So, how can we convert the previous test to simulate this manual actions?

In this Vue concrete case, we can do it using the Vue Test Utils https://v1.test-utils.vuejs.org/ (every library/framework has its own tool for this):

Note that this approach force us to make identifiable the elements we are testing. Because of that, we refactor the example Vue component as follows:

Now, through this test, we are ensuring the correct behavior of the complete use case that we saw before, since we are simulating the actions the user would do in the view. And now, we have a more coverage (Vue script, store, Vue template) from a test that has fewer lines of code. This is great, isn't?

Some cons and their solutions

Right, from my experience, this type of test come with a couple of cons:

They are more difficult to build, because now, we have to think how can we test a complete use case, instead of think how can we test some lines of code from a method. This leads us to think in how can we do the manual actions of the user in an automated test, and, how can we test more lines of code together.
Since we are using the template to perform actions and asserts, there are troubles with the time of the render. A correct test might fail randomly, because some part of the component is not render yet for the next action or the next assert.

For the first cons, the solution is very simple, just get practice until this type of test become more natural.

For the second cons, after a lot of research, I find a very solid solution, through this method:



export const retry = (assertion, {
  interval = 20, timeout = 1000,
} = {}) => {
  return new Promise((resolve, reject) => {
    const startTime = Date.now();

    const tryAgain = () => {
      setTimeout(() => {
        try {
          resolve(assertion());
        } catch (err) {
          Date.now() - startTime > timeout ? reject(err) : tryAgain();
        }
      }, interval);
    };

    tryAgain();
  });
};

This method repeat an action in a period of time that cannot be measure for us, the humans, and allows the component to finish its render. It's like perform an await wrapper.vm.$nextTick() the necessary times until the component reach the render of the previous action, without increase the running time of the test. This way, we create a test that doesn't depend on the render time of the component. If the test is correct, then pass, if the test isn't, then doesn't pass. Simple.

Now, we refactor our test to remove the second cons:

Conclusion

I hope you enjoyed this article and, more importantly, that you learned something new. Remember, this (like everything in programming) is all about context, in some cases it's better to do unit tests, in others it's better to do integration tests, or do both, It depends on you.

Any opinion/suggestion/improvement is welcome, please comment below!

Thanks so much for reading, happy code!