Forem: Adèle Gauvrit

Kumo becomes Serverless By Theodo ☁️

Adèle Gauvrit — Fri, 04 Aug 2023 15:03:54 +0000

I am thrilled to announce a significant transformation in our brand as Kumo evolves into Serverless by Theodo. Over the years, the Kumo team has grown and shared here our learnings and achievements in the field of Serverless. Now, we are taking the next big step to enhance our platform and naming our blog after our Mother Company Theodo.

Rest assured, while our name evolves, the recipe remains unchanged. If anything, this rebranding allows us to better communicate our values and focus on providing you with even more insightful and relevant articles on serverless computing.

The whole team is incredibly grateful for your support the past 2 years. We are looking forward to embarking on this new chapter as Serverless By Theodo. As always, your feedback and suggestions are essential to us. Reach out to us with any comments or ideas you may have.

Together, let's continue our journey to better serverless computing with Serverless by Theodo!

A serverless solution to just-in-time scheduling

Adèle Gauvrit — Thu, 09 Jun 2022 13:30:51 +0000

TL; DR: We published a CDK construct to schedule events precisely.
Documentation is available on the project's github

When I worked on a serverless CMS with Guillaume, we had to find a way to schedule an event (publishing a new article)
at a precise time with serverless tools. AWS doesn't offer a built-in service to do so. We found straight-forward options but none as precise as we wished. We came up with a solution of our own. We published a CDK construct for you to add our solution to your stack without boilerplate.

Naive approaches to serverless scheduling

Scheduling a lambda to run every minute

A first approach is to use lambda schedule trigger feature. Lambdas can be triggered every down to every minute.
Every minute, you can query your database to find if any events need to be published now. If so, run the action you planned (like publishing an article), else do nothing.

A lambda can be triggered down to every minute and the trigger time has a one-minute precision guarantee. It's good enough for some use-cases but not if you are looking for a highly precise method.

Using DynamoDB time-to-live and streams

DynamoDB comes with a time-to-live feature. You can add a ttl attribute to an entry. At said time, AWS will remove the entry form your table. If you have a stream plugged to your table you can play an action at scheduled time.

However, this approach is even less precise than the first. AWS only guarantees a forty-eight-hour precision on the time-to-live date. In practice, users observe a ten-minute delay but it's not guaranteed. If, like us, you work on time-sensitive events, you will want to avoid this method.

Introducing cdk-scheduler

To match our just-in-time ambition, we leveraged the SQS delay feature. When publishing on an SQS queue you can set a delay that goes up to fifteen minutes.

Instead of starting a lambda every minute, we do it every fifteen minutes, which reduces fixed costs. This method has a one-second precision gap!

In case you're in hurry to schedule, we also added a "near-future" handler: a second lambda is plugged on a DynamoDB stream. It handles events that are to be scheduled in less than fifteen minutes after their creation and pushes them directly on the SQS with a delay. This edge case is just as precise.

Pricing

CDK-scheduler lambda calls are low. Fixed costs for lambda execution is $0.06/month. Regarding variable costs, CDK-scheduler costs $2.50 for one million scheduled events
($1.25 to put your items in DynamoDB and $1.25 to delete them, SQS is free for the first million events).

Comparison

Solution	Fixed costs (per month)	Variable costs (one million events)	Precision
CDK-schedluer	$0.05	$2.50	1 second
Scheduled lambda	$0.80	$2.50	2 minutes
DynamoDB Time to leave	$0	$2.50	1 day

What's next?

Our construct is now published. Documentation is available on our open-source repository. If you would like to try it on your project run yarn add cdk-scheduler. It fits the need we had when we developed a CMS like a glove. We're now looking for your feedback. What do you need when scheduling an event? What configuration or features would you like to see?

Cover photo by alexandru vicol on Unsplash

Write tests for Humans

Adèle Gauvrit — Tue, 08 Sep 2020 16:31:42 +0000

Let me tell you the story of the day I almost quit testing my code altogether.
For the tenth times this week, the CI failed with this cryptic error message "Received value does not match stored snapshot" and a diff that looked something like:

<div
  - className="myClassName" >
  + className="myNewClassName" >
  + <div>
    Hello world
  + </div>
</div>

It felt pointless to fix this test : why does it matter if the class-name is different or I added a div tag?
It does not keep me from introducing regressions as nobody in the team reads the snapshots updates.

I was so frustrated, I almost broke the computer, slammed the door to quit my job.
But instead, I decided to channel this energy into something more positive.
I started thinking about how the project tests got into such a state.

How come everybody agrees that we need to test our code base yet so many developers hate it?
Was I the only one being frustrated? (No I was not)

When people say they use TDD, I don’t believe them.
— emily freeman (@editingemily) August 26, 2020

Why is nobody doing something about this?

The Vicious Circle Theory or How you stopped writing test for humans

Here is my theory to make sense of this kind of situation.
I have called it "The vicious circle of testing" ™:

A Developer changes the code for a feature. A test fails, but it is not clear why.
The Developer fixes the test. They do not know why the test was here in the first place. They grow used to the idea that it is not worth their time investment to fix it.
Next time, this developer writes code, they spend less time and effort writing tests.
And back to the top...

If this resonates with you, you've been writing tests for the CI to work but not for your teammates to understand your intent.
Tests are a collaboration tool, they are meant to be read when they fail by you of your teammates.
You need to make sure you can understand them in a month when you have forgotten all about what you were supposed to do.

Tests are meant to be read by humans, not your CI.

How do you write tests for humans, you ask?
Here are my 3 tips for you :

Why: You need to know why you are testing. Because testing is a universal truth in software engineering, it is easy to forget why you need to do it on your project. Find what motivates you: it will guide you to write good tests that you want other humans to read.
What: You need to focus. I think it is better to do one thing well rather than try to do it all and do it mediocrely. Choose what you want to test, do not try to test it all.
How: Choose the best tools adapted to your goal. They must help you write quality tests that you can read and maintain

#1: Why do you write tests?

Writing great tests for humans takes time, you need to be invested in what you are doing.
You cannot be if you do not know exactly why you need to write great tests.

Do not write tests because you have to. Make sure you and your team know why you write tests.

Automatic testing is often a given when setting up quality tools for your project.
Unlike your linter, tests do not fail every day, they are a longer-term investment to code quality.
It is easy to lose sight of why you are doing it.

Before starting to set up your test environment you should clarify why you are testing.

What are you hoping to gain in the long term?
Better documentation?
Fewer regressions bugs? Developer experience tool for TDD?

Make sure you and your team know why you are doing it. Because it is so easy to fall into the vicious circle of bad testing, if you all know why you are driven to write tests, you can catch early when you are deviating from your goal. Making your goal clear will give your teammates ownership over your test and they will help you push for improvements when needed.

#2: Choose what parts need to be tested first

If we chose to use snapshots on my project, it was to go faster.
We wanted to test all of the codebase without investing much time.
By aiming to keep a 100% coverage, we started writing tests that needed to be maintained every time we added features. They could not help us catch regressions because they changed too often.
No matter how small the investment was, it was not worth it.

Do not aim for a 100% code-coverage of generic tests, aim for 100% of readable tests

You do not need to test everything.
Your set up may not be adapted to every aspect of your code.

I always write unit tests for very logical functions :

They are hard to read and easy to break. A test is a safeguard to make sure another developer will not break it.
It is cheap to set up and to write.
It is fast so it does not slow down the ci.

But for most part of the code, it is a case-by-case study.

Should I test my frontend app UX?
If I am writing a complex single-page app, UX is very likely to evolve.
Testing it may be my priority so I can prevent regressions that have a big impact on the user's appreciation of my app.
I want to write integration or end-to-end tests for this.

Should I test your frontend app UI?
If I am writing a shared-component library, I am focusing on UI and have a few logical components.
My first mission is to provide my users with reliable and consistent UI.
Testing UX logic is a bonus, it is ok to make choices and not to focus on this at first.

#3 Choose your tools right

There are a lot of great testing tools available.
Now that you know why you want to write tests and what parts of code you want to focus on, you can find the tools that fit your needs like a glove

On another project, I chose to test my frontend React app with react-testing-library.
I wanted a tool that :

would test our React app components logic and document what they did
would help new developers feel confident when starting on the project
would help us focus on these goals even if the teams changed

That is why I chose react-testing-library to write integrations tests on it.
It turned out to be a great developer experience.
I also had a personal goal of trying this new tool that had a lot of great reviews.

Find a tool that makes writing test fun, or that makes you want to do test-driven-development, you will only write tests for humans this way.

You are ready!

Now you know: if you ever find yourself thinking testing is useless or painful to maintain, you are probably in a very vicious circle of testing.
If you want to escape this, here are the steps to follow:

Remember tests are a documentation and collaboration tool, they should be made to be maintained by teams and read by humans
Make sure your team knows why you are writing tests. This will motivate you to write good tests that you can be proud of.
Do not try to do it all. Coverage is just a number, it is not a token of code quality. Focus on testing some parts well rather than all of it in a poor way.
Choose the tool that fits you and that makes the developer experience fun