Forem: Secfi

How we're discovering Secfi's engineering culture

Petro Momot — Wed, 30 Mar 2022 11:53:44 +0000

Intro

Every company and every team has a distinct mission and a culture. It doesn’t matter if it’s defined and articulated in words that each team member can recite or if it lives in the subconscious of a team. You can feel it when you enter the office or join a call, you can get a grasp of it during the interview process. Whenever I have an interview as a candidate, I enjoy asking the question about the company mission and its values. In most cases, the answers are vague and people say something like: “Yeah, we actually had a session when we defined something along these lines, but it was some time ago” and then start looking through the internal documentation or their own public website to find those.

At Secfi we have defined a list of values that we come back to quite often – each of us is encouraged to give feedback/praise to a colleague at any moment and while doing it one has to select one of the values that the feedback is falling into. Also, during the mid-year and end-of-year reviews we score each other using the same list of values. All of this works well for the whole company, because each team or department in a company has its own unique vibe and set of HOW’s or culture differentiators. They correlate with the company-wide ones, but are still unique and special to the function and people on that team.

The engineering department at Secfi is actively growing and, at the moment of writing, we have 21 colleagues. We decided that this is a great moment to discover our WHY and HOWs as a department. The reason why I’m saying “discover” and not “define” is that the culture and mission is something that’s already alive and in motion – we just need to discover it, to do a specific exercise that helps to uncover something that all of us already feel and live by.

What we wanted to achieve

The main goal was to discover our WHY and HOWs. But why?

It’s more or less easy to keep the unspoken culture alive and strong when you don’t have new people joining – everyone knows everyone, processes are well defined and followed. Things change when the team is actively growing. Since the beginning of the year we have new people joining every couple weeks. Now it’s clear that we need to have these things well defined. To go further, we need to reinforce them by some daily/weekly/monthly/quarterly activities that somebody has to think about, prepare and execute. Another important point is when you have your WHY and HOWs defined it’s much easier to interview and assess people on a cultural fit. This goes both ways – it’s easier for us as a company to assess a candidate while the candidate themselves will be able to easily find these materials on our public website. To achieve these goals we decided to organize an (almost) full day workshop for the whole team.

What happened in the workshop

The goal of the workshop session was to find out how the engineering department contributes to and what the effect is on the company and the world. In the session we reflected on successes and special moments when the team was at its best. Situations when we felt proud to be part of this team. Events that have shaped it as we are today.

A small but very important side note here: while I suggested the idea to do this workshop the following way, it would not be possible to actually do it without the relentless efforts of Sarah van der Laan. She organized the room, equipment, lunch and arranged everything with all possible stakeholders 🚀

The whole workshop consisted of several blocks of activities which we took from the Find Your Why book written by Simon Sinek, which is a great resource along with his other book “Start With Why”. Both books bring understanding to why it is important to start with why in every situation – at your work or at home, or somewhere else. The “Find Your Why” gives step by step instructions on how to do it on a personal level and on a group or tribe level.

The workshop opened with an introduction of the methodology and background followed by an icebreaker. For the icebreaker we divided all participants into groups of two and asked each colleague to share something that inspired them to join the company. This activity took just 5 minutes but set a great foundation for the further discussions and interactions.

I will not list here all the details of the workshop – I encourage you to read the book and organize one in your company or team. What I will share is that after roughly 5 hours (including breaks) we had plenty of material to define our HOWs, our culture and a solid draft of our WHY statement as a department.

The results

On top of having a draft for the mission of an engineering at Secfi we have achieved multiple goals.

Everyone felt like an active part of the tribe.

People shared stories that made them feel proud to work here. We received great positive feedback on the spot and during the week following the workshop.

Some of the testimonials:

"The workshop itself was our best to date, I don't know what I'd change"

"We need more events like this!"

"Can you share a guide or just your experience organizing it? It would be nice to have more workshops and sharing your experience might give ideas/inspiration to people"

"Everything was perfect! Thank you!"

"[I enjoyed] the collaboration, everyone was involved in the process."

"The exercise, the vibes, the fun"

Actionable next steps brainstorm

We brainstormed actionable next steps for our department and decided to pick several of them as soon as possible. There were quite a few great action points that came from the brainstorming. I would split all the ideas into three groups – 1) one-time actions that are quick to implement; 2) bigger mid-term projects that need a bit more brainstorming and ownership; 3) broader ideas that need some thorough review and thinking. Overall the list looks very exciting.

Next steps

You may have noticed that I have not shown any of our WHY or HOW statements in this article. The reason for this is that at this moment we’re working on finalizing the WHY and refining our HOWs. For this we asked several volunteers from the team to join and now the five of us are preparing this and the next steps that we’ll describe in more detail in the following article. Thank you for reading and I hope that our experience will inspire you to do the same activity in your team and or company.

Mobx -> Apollo client journey

Petro Momot — Thu, 10 Feb 2022 12:50:55 +0000

Hi, my name is Petro and I’m a senior frontend engineer at Secfi. We are helping startup employees understand their equity and assisting some of them to avoid losing their deserved reward.

Secfi is actively growing — client applications are getting bigger and maintaining one global state by hand becomes a difficult task. This article will explore our approach in resolving this.

Summary: What did the migration from MobX to Apollo client give us?

Simplifying our data layer with GQL/Apollo allowed us to reduce a lot of the state management needs and boilerplate — to the point of removing Mobx altogether.

In the end we got:

smaller amount of abstractions
most of caching is handled automatically
code generation (25k-30k lines code removed + backend and frontend always in sync)
faster loading time due to smaller amount of calls and reduced amount of data transferred

This migration was not done in isolation. There were a lot of data model, tooling and even organizational changes that also occurred in parallel. These all interconnect, work together and influence one another.

How we grew to a point that a refactor was needed

At Secfi we utilize React as the main library for our FE stack so all our other technology choices are based on React and its ecosystem. Before diving into the MobX migration it's important to explore why and how we used MobX in the first place — to provide some much needed context and history behind our decision making process.

When our app grew to the state of needing the state management library (pun intended) we explored the two most common options in the React ecosystem — Redux or MobX. We didn’t like the amount of boilerplate code that we had to write if we went the Redux path and at the same time the MobX community had come up with the MobX-State-Tree library which offered cool benefits, such as runtime type checking, data normalization and clear structure. On top of that we could (and did) set it up in a way that mimicked our backend data model and the microservices structure using the MobX-State-Tree models and actions. The benefit of this was obvious — frontend state structure was in sync with the backend — what can be better? The drawback though was in the actual details behind it.

Problems that started to arise with time

Models interdependency
Increasing complexity, plus amount of calls
Maintainability

To understand how these issues rose from our set-up back then it’s important to show a part of our business data model.

MobX-State-Tree has a great mechanism called actions in their models. These actions allow subscriptions to events on the model and facilitate performing side effects. We used it on all our models to fetch all related models in the tree. When the customer loaded the home page we needed to get all Affiliations for them, resulting in each MobX-State-Tree model of the Affiliation making calls to resolve Company, Company Assessment, Tax Info and arrays of Option Grants and Share Grants. Each of these entities had their own initializing logic to fetch all other entities that they had references to.

Of course there were checks in place to not fetch the same entity (checked by uuid) twice, but this improvement paled in comparison with the amount of REST API calls that were initiated on the page load. For reference — if the customer had indicated that they worked in 5 companies there could be 100 rest api calls initiated on the application load to populate the state with all necessary information. And while we could optimize specifically for the home page by joining all the calls into a new backend endpoint, the overfetching issue would remain on a platform level.

As you might have guessed, this was also not fun to maintain. Models were naturally utilized as a source of parts of the business logic, since they were foundational to the application. Soon enough some of our UI pieces started to be affected as well: we created a separate store for theming configuration; all models grew to have computed properties that were meant for pure UI representation. At some point we realized that the state grew into one very big and hard to maintain creature.

Apollo client to the rescue!

It was clear that the situation had to be improved but where to start? There were different solutions we could leverage to solve this problem, we went with the GraphQL in combination with React Context api — for parts that were client-specific.

Backend transformation

One action point the team decided on was to start utilizing the power of the GraphQL. In our business case the data model is represented in multiple ways by our tools, helping the user to understand their equity options and their complications by presenting them in different ways. Another great benefit was that we could hide the backend implementation and logic altogether and have one orchestration service/facade which would serve as an “API Contract” giving the team certainty in the expected inputs and outputs of each operation. This in turn gave the ability to generate types for the client apps and queries + mutation hooks to write even less code. Last but not least, having data fetched through GraphQL allowed us to retrieve only the necessary bits of the model and not the whole thing. To read a bit more about this — check out the backend article on the migration. Apollo client also gave us local cache out of the box, so here we saved on even more code, complexity and unnecessary api calls.

Frontend transformation

While slowly migrating most of the API interactions to the facade we realized that our frontend architecture is not well defined and scalable either. We had two client side applications — client facing and admin facing — that were written in quite different ways and at some point it became a real struggle to switch between projects and fix bugs. This motivated us to define one architectural standard for all frontend apps. We'll cover this process and our learnings and wins in a separate article. With every feature refactored we also moved the backend integration to the Apollo client removing the dependency on the central MobX store. One important thing worth mentioning here is — frontend applications have user interactions-driven state and the Apollo client does not cover this part. For smaller pieces of state we utilize React hooks api — useState, useEffect and useReducer. For more complex we use React Context api. There are several top level contexts that handle logic such as authentication, theme and multiple feature-specific contexts throughout the app.

How it works now

First, we define a *.graphql file in the folder where it is going to be used — eg near the container or specific hook. Example:

fragment AffiliationOverview on Affiliation {
  uuid
  country
  customer {
    uuid
  }
  company {
    uuid
    name
    logo
  }
  company_assessment {
    uuid
  }
}

query getAllAffiliationOverview($customerUuid: ID!) {
  affiliations: allAffiliations(filters: { customer: $customerUuid }) {
    totalCount
    nodes {
      ...AffiliationOverview
    }
  }
}

The reader can notice that we define uuid property inside of each entity — we've configured the Apollo client to use uuid as unique identifiers to handle automatic cache updates and linking (by default it uses id property). Fragment here is a reusable piece of the entity. If we need the same piece of the model in multiple queries in the same file — we move it to the local fragment. If it becomes common for more queries and mutations across the app — we move it to global fragments.

Second, we run the generate command — it will get all the types from the relevant backend environment.

Now we are able to import the generated hooks and types across our applications and use them as regular React hooks, while ensuring type safety and alignment with our backend.

import { useGetAllAffiliationOverviewQuery } from '@generated';

Queries are quite straightforward. Mutations, on the other hand, become more tricky, especially those that add or remove items in an array. Apollo client is not smart enough to determine how to update the cache in case of addition or removal mutations. There are two ways to do it:

simple: provide list of queries to refetch, this way the cache gets updated with the fresh response from the backend; drawback — additional backend call(s)
more complex but more efficient: update cache manually, it saves on the backend calls, but one needs to mutate the cache which might not be trivial in some cases.

Half a year ago we removed the last bits of the MobX in our apps — logic related to authenticating the user, interacting with session tokens and other profile related bits and pieces. Only the data fetching part migrated to the Apollo client implementation, the rest got its own React Provider(s) and now the whole app interacts with those pieces via hooks. That pull request alone reduced our codebase by 5k lines of code. It's not the line count that made the whole team happy that day, but the realization that now we have one way of interacting with the backend and a year+ long migration has been finished.

To reiterate, in the end of this process we got:

smaller amount of abstractions
most of caching is handled automatically
code generation, backend and frontend always in sync
faster loading time due to smaller amount of calls and reduced amount of data transferred
and last but not least — happier team maintaining all of this!

Using GraphQL to improve our microservices at Secfi

Manu Hortet — Wed, 16 Jun 2021 13:52:00 +0000

A couple of years ago, we at the engineering team at Secfi decided to go GraphQL. This helped us reduce complexity in our system, improve performance and have a more intuitive API. Let's talk about that!

The problem

Our decision to start using GraphQL was a response to some specific problems we had. So let's start by giving some context about our system, and what those problems were.

At that point in time, our frontend was made up of a couple of different clients. And as we kept adding features to them, it got clear that they had different needs, data consumption wise. Some of those needs were difficult to fulfill given our backend capacities.

Our backend was designed as a distributed system. In it, different services written in Node and Python exposed their own REST APIs. Each API allowed clients to interact with the subset of models the service was responsible for. Some of our models were related to one another, although living separately and being manipulated by different services. This created a clear pain point: joining and working with data coming from multiple services.

As we initially lacked a strategy to fix this on the backend side, logic to operate around it was written in the clients. In these cases, the client would need to call the server multiple times, collecting data from different services. When everything was fetched, it could be joined and used. This was slow and introduced too much unnecessary complexity to the frontend. We identified our first, big need: to simplify our clients.

But having a distributed system came with another drawback for us, data management wise. The APIs of the multiple services tended to be slightly different as they were written using various frameworks and languages. This also added extra complexity to the clients, which needed to be able to interact with the server in multiple ways. Moreover, it made working around the system less intuitive.

To sum up, our data strategy was causing us these problems:

The frontend was getting complex.
The frontend was getting slow.
Fetching data was not intuitive.

The solution

The problem was clearly identified now. We needed to rethink our data strategy. The solution had to come from tweaking our distributed system, not from punishing our clients any more.

One of the ways we figured could potentially improve the situation, was to start using the facade pattern. This software pattern was introduced in the Gang of Four's prominent book Design Patterns, and using it we expected to move the complexity that our data model had brought to the clients away from them. This complexity wouldn't disappear, but would be treated by a new service (the facade), focused on concentrating and minimizing it.

Now that we had decided on the facade pattern, GraphQL started to look like part of the ideal solution. This is because GraphQL is designed to get many resources in a single request, and that was exactly what we wanted to allow our clients to do. The new facade service would then be a GraphQL server, using the aggregate of all our different models as a schema.

Our new GraphQL server, acting as a facade, also dissipated the problem we had with the backend not being intuitive. This is because we could now always be served by the facade when requesting any data. There is no more need to talk to different services, in different ways. Now designing new services was more friendly, as choosing new frameworks or simply API designs was not a problem in the frontend.

Conclusion

After having used the facade pattern and GraphQL for a while, we can say we are happy with the results. The refactor was expensive, but our original problems have been mitigated:

Complexity was effectively reduced in the frontend. The developer experience is now nicer, and adding features is faster.
Performance was improved. Most data management happens now on the server side, reducing communication times.
Our API got to be more intuitive and easier to interact with. Also, we can now check the auto-generated Docs offered by Apollo Server's GraphQL Playground. Pretty cool!

What is next?

We want to keep stepping up our GraphQL game, of course! Some of the initiatives we are discussing or already working on are:

Federating our services. This aims at improving separation of concerns across our backend. You can read more about federation here.
Using DataLoader. As our product scales up, we are working to improve the platform's performance. The DataLoader library is one of the solutions we are considering.

Does this sound like fun to you? If so, we may be a good match. 🥰
Consider checking our open positions at secfi.com/careers.

4 simple steps you can take now to improve your TypeScript codebase

Manon — Wed, 17 Feb 2021 19:46:39 +0000

Because in an ideal world, readability and maintainability would be the north star of every programmer.

TL;DR

Make your TS Config as strict as possible so you can agree on naming conventions, avoid typing errors, etc
Use a linter and formatter tool so your team can focus on the real deal
Agree on a project architecture
Leverage the power of pair programming to solve complex issues

Don't try to abstract so much that your teammate (and, most likely, your future self) cannot understand your code later. When working within a team, often the most critical issue is to read and understand other people's code. Focusing on readability and maintainability will not only help you read and refactor your code in two months from now, but it will also help the whole team to be more productive, efficient, and, overall, happy!

Readable and maintainable codebases are also key when it comes to scaling your application. When starting a new project, it is better to start simple and small with strong and clear foundations rather than over-engineering your application without even knowing where your project will be in a few months from now.

But what can you do to make readability and maintainability your north star?

1. Make your TS Config as strict as possible

If you can, make your TS Config strict. You might have tons of errors, to begin with, but this is worth it as you will enjoy all the benefits of better type safety.
Here are some of the most useful rules you can enable in your TSconfig file:

"strict": the strict flag will enable a wide range of type checking, therefore, resulting in stronger guarantees of program correctness. If you turn this on, you will enable all of the strict mode family options. If you want to enable only some of the strict family (see below), it is not necessary to enable strict.

"strictNullChecks": set this rule to truewill save you a considerable amount of Javascript errors; especially the much-loved runtime error 'undefined' is not a function , frequently caused by bugs in JavaScript code. With strictNullChecksenabled, TypeScript won't allow you to use a variable that is not known to the compiler (an exception is to be made with the use of any typed variables, more on this later).

"strictBindCallApply": this rule is also super useful as it will check that the built-in methods of functions call, bindand apply are correctly invoked.

"strictFunctionTypes": if enabled, function parameters are checked more correctly. However, this rule doesn't apply to functions written in the method syntax, only for function syntax.

"forceConsistentCasingInFilesNames": enabling this rule will be useful if you are working on a project where developers use different OS because it will prevent issues related to the file casing and how the files are referenced in the code. TypeScript will thus issue an error if a program tries to include a file with a casing different from the casing on disk.

"noImplicitAny": TypeScript will issue an error whenever it would have found a type any. This rule is really strict and probably hard to implement in large codebases or codebases transitioning to TypeScript.

"noImplicitThis": TypeScript will raise an error if your code has implied any type of "this" expression.

"noImplicitReturns": if you enable this rule, TypeScript will check all code paths in a function to ensure they actually return a value.

"noUnusedParameters": enabling this will raise errors on unused parameters in functions.
For full reference, you can check the official documentation here.

2. Use a linter and formatter tool

Why is it so critical to have a TypeScript code that is readable, well-formatted, and clean? Because it will allow programmers to focus on what they do best and enjoy the most: implement business logic, improve performance, enhance the architecture of a codebase, build a UI library… you name it!
Using a linter and formatter tool (ESlint, Prettier…) will provide consistency to your codebase and make your team happier. And, we all know that a happy team is an efficient one! By keeping your TypeScript clean with linters and formatters, you are avoiding long debates among programmers regarding missing semi-colons or indentation (oh, joy!)

Your codebase will become:

easier to read and start with for new joiners
easier to review
easier to refactor
a happier place where programmers have fewer frictions

3. Agree on a project architecture

As Daniel Irvine pointed out in this article, 'almost always the biggest problem I see is interpersonal issues that stem from disagreements about project direction and structure.'

If you are working on a solo project or within a team, you will always be rewarded if you agree and follow a clear project structure. This will allow you to:

onboard new joiners more easily
navigate your project in a structured and clear way
find relevant files faster
better refactor your code as it will be easier to see the bigger picture within your project
overall, enhance your productivity and happiness with working on your project

4. Use the power of pair programming to solve complex issues

Why should you consider pairing with a teammate more often? For many good reasons!

It's fun and creates bonding within a team, which is not to be neglected as it will most likely improve the overall codebase in the long run.
It allows a faster and better spread of knowledge among teams, especially if the programmer leading the session is familiar with a part of the codebase the other teammate is not.
Along with the same idea, pair programming will allow a better transfer of skills towards junior developers as they can pick-up tips and tricks from more experienced developers, leading to faster growth.
Usually, when pair programming, you and your teammate are programming 'out loud', which ultimately leads to discover hidden items in the code and to a clearer understanding of the complexities.

Thanks for reading, and happy coding! 😀