Forem: Lakshay

From Ghostbusting Tiny Monsters to Resurrecting the Giant

Lakshay — Thu, 27 Jun 2024 10:43:07 +0000

Well, I am not really busting ghosts here although it would be fun to be a part of the original Ghostbuster series.

This one, 2 years later, is about the same side project. It is the story of haunting microservices, deployments, and my heartfelt sad decision to move back to the monolith.

Recap - A series of unfortunate events

The backend of my ECom side project was 6 node apps running Apollo GraphQL servers and another node app running Apollo gateway, to stitch all the federated schemas - or as I would quote "One ring to rule them all"!

Once upon a time I used to deploy these easily when Heroku was free. Its seamless CI/CD pipelines were smooth AF and the cold starts weren't bad either.

So what happened?!

Sauron decided to turn things upside down and Heroku went from free to paid, forcing me to move towards gathering the alliances or thinking about monoliths.

Challenges you ask?!

Managing 7 node servers - even for development locally, was tiring.
Like imagine hitting pnpm start for each server one by one, I found that tiring. Of course, I could have written a shell script but... never mind.

This was just a small thing. The bigger one was deploying this. This being just a side project, a playground for me to tinker with my ideas, I did not want to shed some bucks around.

So I started exploring the possible free solutions.

Possible solution I explored?

I did find multiple free services that were allowing me to do something similar, but there were hurdles, hurdles that got me thinking - "I shall not pass!"

All the free services are limited to a very low grade CPU which was not able to run all the 7 servers. Deploying them individually was another idea but the cold starts were bad, like reeeaaaallly bad, that many times API calls used to fail because response time was too much for our dear gateway to wait for.

Sweating on this for days, I got an idea - what if I could bind them all under one shell! Yes, the glorious concept of monorepos often allows us to ship and manage projects seamlessly.

But as Uncle Ben said - "with great powers, comes great responsibility", and with monorepos comes heavy machinery.

I used Vercel's Turborepo for this task since that is what we use at work. It did make my job easy for local development, I was even able to set up a workspace-global prisma instance which was much much easier than maintaining the schema for each service using Mongoose.

But as I mentioned above, free services don't give a strong CPU, running 1 server wasn't easy, imagine running 7 under a shell. Kaboom!

...and that's what happened. Some would start and some would not since RAM was not sufficient either.

So how will Frodo reach the fires of Mount Doom?

Since I did not want to pay, and being the only dev in a side project, all of it was just killing the fun, and the only way forward I could see was doing things the old way.

So one Saturday night at 00:00 I took out my favorite Starbucks' mug and poured me a noice freshly brewed cup of french press, and forked a new branch from main and named it project-knightfall!

First of all, if you know what is Project Knightfall, we should be friends!

I took the plunge, re-wrote a lot of it, deleted a few things, and was able to get the backend up and running in a few hours.

I deployed it on - shoutout to - render.com and even though the cold start is really annoying, but things were finally working. No calls were failing, schema operations were working fine, and post cold start, response time was also good.

Conclusion

Coding is fun, building is fun but sometimes making such small decisions can ruin multiple nights and lives.

For let's say a hobbyist it is annoying to encounter such challenges. But you definitely learn a lot, after all your mistakes are your best teachers.

It is good that we have new SSR frameworks like Remix and Next which do eliminate such issues to some extent(?) but - and I am frontend guy - building a federated GraphQL backend was super cool and I wanted to do a lot more with it.

Do you think, I should pay and get back on it? Do you have such stories to share?

Drop in the comments and say hi!

GraphQL Microservices and the magic of Apollo Federation

Lakshay — Sun, 06 Mar 2022 12:29:05 +0000

I am just a UI Engineer.

Hello from the Dev cave! 🍻
Hope you are safe and healthy.
I am a UI engineer or a Frontend developer. I am just trying to explore stuff and build cool things. I don't have enough experience or knowledge on backend.

Why I say this is because if you have worked with GraphQL and you are skilled/knowledgeable on the backend, I would love to hear your views in the comment section and anyways in which I can improve my code and application.

The Application and Tech Stack

For this post, I will be designing an application where a user can perform the following actions -

signup
login
update user profile

For these actions, I will be creating 4 services. It feels like being a part of LotR -

"One Ring to rule them all,
One Ring to find them,
One Ring to bring them all
and in the darkness bind them."

😂 😂

Auth - a service to rule them all and to handle sign up and login.
User - a service to find them all and to handle user profile operations.
Gateway - the gateway service to bind them all and create a super graph.

Gateway service

The simplest of all services. All it needs to do is compile all the sub graphs into a super graph, redirect the request to the correct resolver and if you want, you can also validate your user token in this service - but I haven't.

Let's install the dependencies.

apollo-server - we will use this to create an instance of the appolo server and start it.
@apollo-gateway - this will be used to configure our apollo gateway and also to create a connection between your federated gateway and your subgraphs.
apollo-serve-core - we use this to enable a plugin that enables the GraphQL playground.
dotenv - we use this to inject environment variables.
esm - we use this as a transpiler so that we can use ES6 stuff in our code or as they say - "babel-less, bundle-less ECMAScript module loader".
graphql - The JavaScript reference implementation for GraphQL
nodemon - a utility to keep track of your JS files and restart the server if any changes occur.
wait-on - gives a hint to your node server to wait for another service/server to run on another port before running this one.

Let's start building!

It will be pretty much straight forward. You can amend the implementation as per your requirements.

Create a new file - index.js - and import the required stuff.

Now, I will use RemoteGraphQLDataSource class utility to create a connection between my Gateway and all other services. The only thing I want is my Gateway to pass down the context along with the request.

All we need here is, to pass down my authorisation header from the gateway to the other connected sub graphs.
I am not sure, why they used a class for this, but this is how Apollo has provided it. It was a bummer but I couldn't find any other way. You can real more about it here.

Let's configure our gateway!

While initialising our ApolloGateway, we need to provide 2 options -

serviceList - an array of objects where each object represents a service i.e. a subgraph apollo server.
buildService - a function that defines how we build the connection to the subgraphs.

Let's configure our server!

While initialising the apollo server, we pass some config -

the apollo gateway that we initialised above.
we set subscriptions to false as - I am not sure about this - Apollo Federation does not support subscriptions as of now but either way, I haven't tried it yet. If you want to read how to configure subscriptions, check out this blog post here
A plugins array.
context - remember we extended the class RemoteGraphQLDataSource, to enable our connection to pass of the headers or anything sent in the request, we need to return that from here so that our class extension can access it via context.

Last thing to do is is to tell our server to listen to the port for any requests.

One last thing!

So we have setup our gateway, the connections and even the servers. But when we are running it on or local environment, I don't want to run it unless my subgraph services are running.

So here is my start script -

So what it means is wait for the http resources to become available on port 4001 and then run index.js with nodemon by using esm.

So that's it, that's your gateway!

We have our gateway setup to combine various sub graphs. Time to start building the underlying logics or the subgraphs.

The Auth service!

This service will be responsible for your user sign up and login. And the dependencies for this project are almost the same.

Also, just for your info, this is my project structure. You may create something similar or something of your own. If you have a better idea, feel free to add it in the comments.

And this is something that we be constant throughout all the services.

Create your index.js and import the dependencies.

Configure your Apollo server -

What is happening here -

I am not initialising a new schema because this service won't run independently. Since this will be triggered by a gateway, we do this

buildFederatedSchema([{ typeDefs, resolvers, tracing: true }])

next is configuring the context - I prefer to inject my DB modals in context so that way all the resolvers already have the access to it without me needing to import it explicitly.

Also, if we are running on local host, we need not check the x-api-key in headers. I often forget what my key is while playing with queries and mutations on the playground so this really helps.

Type Defs

I will create 3 types -

user - describes a user.

@key(fields: "id") means that if this is ever referenced externally i.e. from another service, then it will be done through the id field.

new user input - describes the user input required for sign up.
sign in - describes the user input required for sign in.

And this Auth service will house the following operations -

I am assuming that since you are trying Apollo federation, you already are having experience with GraphQL and hence I am not deep diving into all the resolvers for Queries and Mutations. Nothing changes there. The only resolver I will be discussing here is about resolving the references.

Once this service/sub graph is contacted/connected from another service, to resolve the User reference, we need to tell our Auth service how to do it by creating a resolver for the reference.

Since this resolver will be called out by our gateway/apollo server when it needs to resolver the reference of type User, it will always have a reference object that will have that key property which we defined in our typeDef - in this case user Id. You can use that to fetch your user from the DB.

Once you have it all setup, don't forget to start your server and also add it in the service list in your gateway serve - if you already haven't.

The User service!

This service will be responsible for your user profile related operations like get user profile or update something. And the dependencies for this project are almost the same as Auth service. We just won't need stuff like bcrypt or jsonwebtoken.

Also, remember in the gateway service, we passed down the authorisation header and x-api-key. Don't forget to consume that in the context here. If user is not authorised, that is the token is not present, you must reject the request.

Also I have created a helper to check the auth token -

Talking about the main thing here - type definitions. Once you have defined a type, you cannot define it again. If you try to define your User type here again, Apollo will throw an error because it is already getting a User type from the auth service.

So to use the User type and to do you operations, you need to extend it.

Once you have extended it, you can then add more fields to it. Also, since this type is an extension of a User type coming from an another service, we tell GraphQL that it will be referenced using the Id field by adding '@external' directive in front of it.

Rest is all the same.

You can build as many services as you want for separating the concern - the only key to understand how to define and link the sub graphs.

Try building around this and add a Trip service that will have source, dest, date, and an array of members. If you do try, feel free to reach out to me in case you get stuck.

That's all folks! That's all I know around Apollo federation - defining, extending and linking schemas or types. :D

Any feedback, feel free to comment below or hit me up. Hope I helped someone in some way.

Cheers!

GraphQL Subscriptions - magic on the frontend

Lakshay — Sat, 31 Jul 2021 21:52:32 +0000

Hello from the Dev cave! 🍻
Hope you are safe and healthy.

This one is a long due about how I implemented a notification system in one of my react projects using Apollo GraphQL subscriptions.

Install the dependencies

Amending the Apollo Client config

The thing with subscriptions is that it doesn't work on the same http protocol like queries and mutations. Subscriptions work using websockets. So we need to tweak our Apollo Client config to enable and handle this communication.

Import the dependencies -

Just like we create a http link to enable communication for queries and mutations, we will create a an instance of WebSocketLink for our subscriptions.

In the uri property, pass the endpoint for subscriptions which you have defined in your server, something like - ws://server:port/subscriptions. In the options property, we pass various config parameters ( like reconnect i.e. auto reconnect once it disconnects) and connection parameters like your auth and other headers.

Once done, we inform/configure the Apollo Client how it can differentiate when to use which link. For this, we use the "split" functionality and "getMainDefinition" utility. Apollo docs explain this extremely clearly - Although Apollo Client can use your WebSocketLink to execute all operation types, in most cases it should continue using HTTP for queries and mutations. This is because queries and mutations don't require a stateful or long-lasting connection, making HTTP more efficient and scalable if a WebSocket connection isn't already present. Split here basically helps in separating the communication channel. We use the getMainDefinition utility to check whether our operation is a query (or a mutation) or a subscription.

Once done, we feed this splitLink to our client.

Consuming subscriptions

Just like queries and mutations, we have a hook for this! The useSubscription hooks, consumes the subscription and return data, loading and error variables. It takes a defined GQL subscription as input along with connection options - like I have use shouldResubscribe here. Doing this, confirms that this is not a one time things and the application should be open to receive updates.

Give this a shot and let me know how it worked out for you.

Cheers!

GraphQL Subscriptions - the magic in the servers

Lakshay — Sat, 26 Jun 2021 22:05:39 +0000

Hello from the Dev cave! 🍻
Hope you are safe and healthy.

This one is about how I implemented a notification system in one of my projects using Apollo GraphQL subscriptions.

The example that I am going to use here is an example project that has 2 types of users - manager and associate and below are 2 scenarios.

As a manager and a logged in user,
- I want to get notified when my associate raises a request
- I want my associate - if logged in - to get notified when I raise a request for them.
As an associate and a logged in user,
- I want my manager - if logged in - to get notified when I raise a request.
- I want to get notified when my manager raises a request for me.

Let's see how I have implemented this!

Some Server side stuff

TypeDefs

Define a type Subscription along with all the subscriptions that you want.

Resolvers

Just like queries and mutations, subscriptions also have a resolver.

A couple of points here -

A resolver name should match the subscription name like newRequestAdded here. In that object, we will have a subscribe object where you define the logic for what data should be returned.
I am using withFilter method that assists me to filter out data that needs to be returned. This is optional.
First parameter that withFilter accepts is the function that you would use to register an event. Subscriptions here work on the concept of PubSub or Publish-Subscribe i.e. you define an event and register it.

How we got this pubsub instance here, I will cover that in the points below. Just the thing to consider is that each subscription has the same pubsub instance throughout the app and each subscription resolver must return an asyncIterator with the array of event names to register those events and get in the pubsub ecosystem.
Second parameter is filter functions that returns true if the criteria is met and subscription should be sent to a client. This function can be async as per your requirements. In this function, you will get
- payload which the resolved value via the mutations's resolver.
- variables i.e. the stuff you got as a subscription variable
- context if you choose to inject this while configuring your Apollo server for subscriptions.

Publishing an event

The best way to inject the pubsub instance is via context. That way you can have a single copy and avoid weird traps or situations like circular dependencies. So say I added a new request in my DB and before I return it from the resolver I use the publish method on my pubsub instance to emit this event with a payload (sounds familiar? REDUX - TADA! And that is why I used the work ecosystem above! :D).
Whatever payload you are passing, the key should be same as the subscription name - in this case newRequestAdded and newRequest is the object Mongo returned after saving the new request.

Configure Apollo Server instance.

We need to define a couple of things that will assist Apollo server to get on with this. Let's start by installing the dependency.

From this, import PubSub and create its instance.
Yeah yeah I know I should be using webpack and ES6+! Will do that some other weekend.

We need to modify our server config. The context here also gives you a connection object. What we need to do is check that if we get a connection object, you can use it for securing your app by accessing API KEY headers from this and/or just return it directly for subscription resolvers to handle.

Once that is done, we need to define the endpoint.

We define a path - or endpoint - which we will configure in our client. Just remember that this is a websocket driven function and hence our endpoint protocol changes i.e. in my case it will be - ws://localhost:5000/subscriptions.

This config object also give us 2 functions onConnect which you can use to return connection parameters i.e. the variables received AND onDisconnect.

Once it is done, give it a shot using GraphiQL. That's all the config we need on the server side, rest is all your logic.

Give this a shot and we'll talk about frontend in the next post!

Cheers!

Configuring GraphQL for backend

Lakshay — Sun, 16 May 2021 14:42:15 +0000

Hello from the Dev cave! 🍻
Hope you are safe and healthy.

This one is about sharing with you all how I configure GraphQL for my backend. It might help someone, or someone might have a better idea and would increase my knowledge.

I also did one about how I configure and consume on frontend and here is a link to that post.

For my setup, I use NodesJS & Apollo Server with MongoDB. You can find its docs and read more about Apollo here.

You need three main things to implement GraphQL -

Something that can understand the GraphQL spec, implement it and handle its requests - something like Apollo server. There are many out there, but I prefer Apollo because I find it simple. As their docs say, Apollo Server is an open-source, spec-compliant GraphQL server that's compatible with any GraphQL client, including Apollo Client. Also, I use Apollo client for my frontend.
Type Definitions - things that define your operations and the shape of data those operations will handle.
Resolvers - functions that assist to resolve (or fulfil) those operations.

Setup

Install the dependencies.

yarn add apollo-server graphql

Once installed, I suggest you to create a folder - something like graphql - in your src directory, just keep things organised. Inside that, you can create your resolvers and type defs.

What I do is, I have a single source for types i.e. typeDefs.js and one folders for resolvers. This folder has respective resolvers as per type defs like users.js, posts.js, etc and all of them being combined in an index file.

Type Definitions

Import the gql tag from apollo-server package.

const { gql } = require('apollo-server');

I use gql tag to defines types. Writing Type Defs is like defining schemas that shape your data. They define the basic structures with fields on which your GraphQL operates.

One key thing to remember about them is, GraphQL follows these schemas strictly. It doesn't care what you do or how you do, all it cares for is how you maintain the shape of the data you are receiving or returning.

Let's write 3 TypeDefs - a user schema, a query (or a request) to get that user and a mutation ( an operation that does something).

A User Schema

A Query and Mutation


    type Query{
        getUserByUserId(userId: String): User
        getAllUsers: [User]!
    }

    type Mutation{
        login(username: String!, password: String!): User!
        register(registerInput: RegisterInput): User!
    }

Also you can create custom input types to expect an argument in mutations or queries.

input RegisterInput{
        name: String!
        email: String!
        password: String!
}

The ! means that this value is required and cannot be null.

Queries and Mutation (i.e. resolvers)

You can defines your queries and mutations like general async functions. All Apollo GraphQL demands demands is that -

You should wrap these in a JS Object.
Name of each queries and mutation should be same as what you have defined in Type Definitions. Remember, GraphQL is a very strict follower.
All queries should be wrapped in a parent Object with key Query and same for mutations.
The values you return from these resolvers, should have the same shape as you have defined in the Type Defs.

You may write different types of queries/mutations in their own files like one for users and one for posts, etc. You can combine them in an index.

const usersResolvers = require('./users');

module.exports = {
    Query: { 
        ...usersResolvers.Query,
    },
    Mutation: {
        ...usersResolvers.Mutation,
    }
}

Once you have your initial template ready, you need to configure the Apollo server instance.

The Apollo server instance

I configure the Apollo server instance with 3 config options -

The Type defs
The Resolvers
Context

Think of context like a middleware which gives you access to the request object. You'll learn more about this when you'll start learning and/or implementing GraphQL.

I use this context, to check for the auth header to confirm whether the request has arrived from a valid source.

Below is the snippet.

Once this is done, you can hit your package manager yarn/npm to start and spin this server.

Let me know what you think about this and if you are experienced with GraphQL, I would love to know your suggestions on making my GQL code better.

Any feedback, feel free to comment below or hit me up. Hope I helped someone in some way.

Cheers!

I finally learnt SASS!

Lakshay — Sat, 21 Nov 2020 20:10:18 +0000

Hello.

Well, for some reason I've been putting myself off from learning SASS/SCSS even though it is widely used and in demand. But since now I am working on improving my HTML, CSS and Vanilla JS skills, so I thought why not!

I decided to get started with the challenges on Frontend mentor but first, I learnt something new.

This is a short 20 min course that I referred as reading the documentation was making me feel sleepy. But always go for the documentations first!

So here are the key take aways from the course -

1. The SASS compiler and extension

So this seems to be an awesome extension for VS Code. So can find it here. It is called "Live SASS Compiler". Looking at the number of installs, seems to be widely used.

So this extension is continuously watching your SASS files for changes, and if any changes occur, it compiles your SASS files into CSS - because browsers can only understand CSS and not SASS - and also adds appropriate prefixes for supporting various functionalities (like flex) across browsers.

2. Variables

Variables are simple to define in SASS. Variable names should always begin with a $ and then you can write something appropriate to justify that variable's usage.

$background: #ffffff;

Do note, that these are different from CSS variables. SASS variables are imperative. If you use a variable and then change its value, it won't change where you have used it previously.

2. Mixins

Don't we love functions? When we can separate a commonly used block of code and call again and again, why write the entire thing again!

Mixins are functions in SASS. They allow you to reuse styles just like functions. To define a mixin, you start with an identifier @mixin followed by the name of the mixin.

@mixin flexCenter {}

Just like functions, you can also pass different values to mixins for more flexibility. For example, consider the following mixin - flexCenter - its task is to apply a flexbox to a container and center items but flex direction can be either row or a column.

@mixin flexCenter($direction) {
    height: 100vh;
    display: flex;
    flex-direction: $direction;
    justify-content: center;
    align-items: center;
}

Once you have defined your mixin, to add them in a style, you can use @include followed by the mixin name passing on values if any.

header{
    @include flexCenter(column);
}

Also, just like in functions we can assign a default value for the arguments, we can do that here as well.

@mixin flexCenter($direction: row) {
    height: 100vh;
    display: flex;
    flex-direction: $direction;
    justify-content: center;
    align-items: center;
}

3. Nesting styles

You can nest styles easily in SASS/SCSS.

header{
    background: rgb(100, 100, 177);
    height: 100vh;

    button {
        background-color: $primaryButton;
        &:hover{
            background-color: red;
        }

        &::after{
            content: "hello";
        }
    }
}

4. Interpolation

String interpolation has always made my life easy. We can do that here also. In JavaScript, we perform interpolation using ${} but here we do it with #{}.

Following is an example from the docs.

@mixin corner-icon($name, $top-or-bottom, $left-or-right) {
  .icon-#{$name} {
    background-image: url("/icons/#{$name}.svg");
    position: absolute;
    #{$top-or-bottom}: 0;
    #{$left-or-right}: 0;
  }
}

@include corner-icon("mail", top, left);

5. Extends

Using mixins is one option to share styles. Another is called extends. Say you have a class that has some specific styles and you define another one that will be using the same styles but with some extra features or modifications.

You can either use mixins to separate the style block and just apply where ever needed or you can extend the class having that style to the new class (class and extends - kinda reminds me of inheritance 😆).

.some-container{
    //some styles
    height: 100px;
    width: 70px;
}

.another-container{
    @extend .some-container;
    width: 100px;
}

6. Operators

We also have operators our usual operators. I am not going into this as you must be aware operators from JS or any other language. But just as an overview, following is what we have-

unary operators not, +, -, and /
*, /, and %
>, >=, < and <=
== and !=
and (boolean AND)
or (boolean OR)
=

Conclusion

It was interesting in learning SASS/SCSS today. I will be playing with this a lot more. As of now I am using styled-components in my React applications but I will definitely try a React app with SCSS once I a done playing.

As I said, this was my first hands-on on SASS/SCSS. Let me know what I have understood is correct and if not, please do let me know the right concept.

Cheers! 🍺