Forem: Pablo Díaz

Next.js Full Example of Next-Iron-Session with Mongoose

Pablo Díaz — Mon, 01 Nov 2021 19:41:18 +0000

Full Example

I couldn't find a complete example of Next.js authentication with next-iron-session so I used all the little pieces I found online and created one to share.

The Code

Before talking about authentication we need to setup the model of the User. To do this we will be using mongoose for flexibility and ease of usage. You can ignore the usage of httpStatus and APIError. Here is the user.model.js inside a models folder at root level.

import mongoose from 'mongoose';
import httpStatus from 'http-status';
import APIError from '@/lib/APIError';

/**
 * User Schema
 */
const UserSchema = new mongoose.Schema({
  name: {
    type: String,
    required: true,
    unique: false,
    lowercase: false,
    trim: true,
  },
  email: {
    type: String,
    required: true,
    unique: true,
    trim: true,
  },
  password: {
    type: String,
    required: true,
  },
  updatedAt: {
    type: Date,
    default: Date.now,
  },
  createdAt: {
    type: Date,
    default: Date.now,
  },
});

/**
 * Add your
 * - pre-save hooks
 * - validations
 * - virtuals
 */

/**
 * Methods
 */
UserSchema.method({});

/**
 * Statics
 */
UserSchema.statics = {
  /**
   * Get user
   * @param {string} id - The email of user.
   * @returns {Promise<User, APIError>}
   */
  get(email) {
    return this.findOne({ email });
      .select(
        'name email'
      )
      .exec()
      .then((user) => {
        if (user) {
          return user;
        }
        const err = new APIError(
          'User id does not exist',
          httpStatus.NOT_FOUND
        );
        return Promise.reject(err);
      });
  },
  /**
   * List users in descending order of 'createdAt' timestamp.
   * @param {number} skip - Number of users to be skipped.
   * @param {number} limipt - Limit number of users to be returned.
   * @returns {Promise<User[]>}
   */
  list({ skip = 0, limit = 50 } = {}) {
    return this.find()
      .select(
        'name email createdAt updatedAt '
      )
      .sort({ createdAt: -1 })
      .skip(+skip)
      .limit(+limit)
      .exec();
  },
};

/**
 * @typedef User
 */
export default mongoose.models.User || mongoose.model('User', UserSchema);

One important detail to see here is the last line, Next.js will try to load the same file creating a duplicate instance and will give an error without the statement.

export default mongoose.models.User || mongoose.model('User', UserSchema);

Config the Database

Create a file with the mongoose connection. Also see the part were the connection is saved in local memory to re use it. Created a file lib/dbConnect.js

import mongoose from 'mongoose';

const MONGODB_URI = process.env.MONGODB_URI;

if (!MONGODB_URI) {
  throw new Error(
    'Please define the MONGODB_URI environment variable inside .env.local'
  );
}

/**
 * Global is used here to maintain a cached connection across hot reloads
 * in development. This prevents connections growing exponentially
 * during API Route usage.
 */
let cached = global.mongoose;

if (!cached) {
  cached = global.mongoose = { conn: null, promise: null };
}

async function dbConnect() {
  if (cached.conn) {
    return cached.conn;
  }

  if (!cached.promise) {
    const opts = {
      useNewUrlParser: true,
      useUnifiedTopology: true,
      bufferCommands: false,
      bufferMaxEntries: 0,
      useFindAndModify: false,
      useCreateIndex: true,
    };

    cached.promise = mongoose.connect(MONGODB_URI, opts).then((mongoose) => {
      return mongoose;
    });
  }
  cached.conn = await cached.promise;
  return cached.conn;
}

export default dbConnect;

Auth Code

For the authentication we need first to create a user! So here is the React function to accomplish it. You can ignore the components that have the actual html.

import React, { useState } from 'react';
import { useRouter } from 'next/router';
import SignUpLayout from '@/components/SignUp/SignUpLayout';
import Form from '@/components/SignUp/Form';
import fetchJson from '@/lib/fetchJson';

export default function Register() {
  const router = useRouter();
  const [errorMsg, setErrorMsg] = useState('');

  const handleSubmit = async (e) => {
    e.preventDefault();
    const email = e.currentTarget.email.value;
    const password = e.currentTarget.password.value;
    const name = e.currentTarget.name.value;

    try {
      await fetchJson('/api/auth/register', {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify({ email, password, name }),
      });
      return router.push('/index');
    } catch (error) {
      setErrorMsg(error.data.message);
    }
  };
  return (
    <SignUpLayout>
      <Form isLogin errorMessage={errorMsg} onSubmit={handleSubmit}></Form>
    </SignUpLayout>
  );
}

Now let's configure the session handler/guard. This is inside lib/session.js and this is the configuration that iron session will use to create the cookie.

// this file is a wrapper with defaults to be used in both API routes and `getServerSideProps` functions
import { withIronSession } from 'next-iron-session'

export default function withSession(handler) {
  return withIronSession(handler, {
    password: process.env.SECRET_COOKIE_PASSWORD,
    cookieName: 'cookie-name',
    cookieOptions: {
      // the next line allows to use the session in non-https environments like
      // Next.js dev mode (http://localhost:3000)
      maxAge: 60 * 60 * 24 * 30, // 30 days
      secure: process.env.NODE_ENV === 'production' ? true : false,
    },
  })
}

Register

Now we can use the Next.js api inside pages/api/auth/register.js. This code includes de usage of the User model from mongoose, the database connection, and the user creation with the hashed password.

import dbConnect from '@/lib/dbConnect';
import User from '@/models/user.model';
import bcrypt from 'bcryptjs';
import httpStatus from 'http-status';
import withSession from '@/lib/session';

export default withSession(async (req, res) => {
  const { name, email, password } = await req.body;
  try {
    if (req.method === 'POST') {
      await dbConnect();
      const userCheck = await User.findOne({ email: email.toLowerCase() });
      if (userCheck) {
        return res.status(httpStatus.BAD_REQUEST).json({ message: 'User already exists' });
      }
      // create user
      const hashPassword = await bcrypt.hash(password, 10);
      const user = await new User({
        name,
        email,
        password: hashPassword,
      });
      await user.save();
      req.session.set('user', { id: user._id, email: user.email });
      await req.session.save();
      return res.status(httpStatus.OK).end();
    }
    return res.status(httpStatus.BAD_REQUEST).end();
  } catch (error) {
    console.log(error, error.message);
    const { response: fetchResponse } = error;
    res.status(fetchResponse?.status || 500).json(error.message);
  }
});

Remember to set the user in the session request with only the required fields and not the whole user. We don't want to save the users' password in the cookie, right?

req.session.set('user', { id: user._id, email: user.email });

You can remove the cookie creation if you don't want to login automatically the user after its creation.

Login

Now for the login, first we will be creating a custom hook, so we can check if the user is logged in and redirect if it is the case. So here is the hook. This hook also uses the useSWR that the official Next.js suggests to use.

import { useEffect } from 'react'
import Router from 'next/router'
import useSWR from 'swr'

export default function useUser({
  redirectTo = false,
  redirectIfFound = false,
} = {}) {
  const { data: user, mutate: mutateUser } = useSWR('/api/user')

  useEffect(() => {
    // if no redirect needed, just return (example: already on /dashboard)
    // if user data not yet there (fetch in progress, logged in or not) then don't do anything yet
    if (!redirectTo || !user) return

    if (
      // If redirectTo is set, redirect if the user was not found.
      (redirectTo && !redirectIfFound && !user?.isLoggedIn) ||
      // If redirectIfFound is also set, redirect if the user was found
      (redirectIfFound && user?.isLoggedIn)
    ) {
      Router.push(redirectTo)
    }
  }, [user, redirectIfFound, redirectTo])

  return { user, mutateUser }
}

The hook queries the session saved in the request like this: (this is the pages/api/user.js).

import withSession from '@/lib/session';

export default withSession(async (req, res) => {
  const user = req.session.get('user');
  if (user) {
    res.json({
      isLoggedIn: true,
      ...user,
    });
  } else {
    res.json({
      isLoggedIn: false,
    });
  }
});

Now we can proceed to the React Login function.


import { useState } from "react";
import Form from "@/components/Login/Form";
import LoginLayout from "@/components/Login/LoginLayout";
import useUser from '@/lib/useUser';
import fetchJson from '@/lib/fetchJson';

// layout for page

export default function Login() {

  const { mutateUser } = useUser({
    redirectTo: '/',
    redirectIfFound: true,
  });

  const [errorMsg, setErrorMsg] = useState('');

  async function handleSubmit(e) {
    e.preventDefault();

    const body = {
      email: e.currentTarget.email.value,
      password: e.currentTarget.password.value
    };

    try {
      mutateUser(
        await fetchJson('/api/auth/login', {
          method: 'POST',
          headers: { 'Content-Type': 'application/json' },
          body: JSON.stringify(body),
        })
      );
    } catch (error) {
      console.error('An unexpected error happened:', error);
      setErrorMsg(error.data.message);
    }
  }
  return (
    <>
      <LoginLayout>
        <Form isLogin errorMessage={errorMsg} onSubmit={handleSubmit}></Form>
      </LoginLayout>
    </>
  );
}

And here is the pages/api/auth/login.js API call that uses the mongoose connection and bcrypt validation.

import withSession from '@/lib/session';
import User from '@/models/user.model';
import bcrypt from 'bcryptjs';
import httpStatus from 'http-status';
import dbConnect from '@/lib/dbConnect';

export default withSession(async (req, res) => {
  const { email, password } = await req.body;
  try {
    await dbConnect();
    // get user from db
    const user = await User.findOne({ email: email.toLowerCase() });
    if (!user) {
      // password not valid
      return res.status(httpStatus.UNAUTHORIZED).json({ message: 'User does not exist'});
    }
    // compare hashed password
    const valid = await bcrypt.compare(password, user.password);
    // if the password is a match
    if (valid === true) {
      req.session.set('user', { id: user._id, email: user.email });
      await req.session.save();
      return res.json(user);
    } else {
      // password not valid
      return res.status(httpStatus.UNAUTHORIZED).json({ message: 'Invalid Password'});
    }
  } catch (error) {
    console.log(error);
    const { response: fetchResponse } = error;
    res.status(fetchResponse?.status || 500).json(error.data);
  }
});

How analytics work in a company with 11 million active users.

Pablo Díaz — Sat, 19 Oct 2019 00:28:44 +0000

The Database

That is 11,177,895 active users. We run an eight cluster Mongo Database and we pay around $100k a year just in support. In this database, we only keep the most recent data and not transactional data so we can analyze better and faster. Faster because in a single document(JSON) we have all the information we need. It took almost a year to centralize and sanitize the data, in analytics, the data (and it's quality) is the most important part.

The Goal

So what do our stakeholders need? We are an internal department that works along with all other departments, so we adjust to the needs of each one. Our goal is to provide the best offer to the right client at the right time. To do that, we built an offer engine in Java and Google Web Toolkit (currently being migrated to Vue.js) so we can configure offers at every possible situation. This engine is a tree of rules (it also looks like a tree in the UI), each rule is configurable and each rule can evaluate in real-time and in Mongo's Map-Reduce (like Hadoop but in Mongo). That way we provide the offer for each of our stores (89) and for all digital channels.

Where is the analytics?

So far, I haven't talked about analytics and actually this is the part that takes the least of our time and effort. Why? Because making models is easy and simple. We only use two kind of models. Logistic regressions and Decision Trees. These are the simplest, effective and easier to explain to other people. For a long time, we have been using R studio to create them and we export them in a JSON structure to be evaluated in Java in real-time. Exporting models to production is hard, so we are changing from R to python. Our engine is highly configurable so we can configure what model to apply for each node of the tree.

Results

Last year we generated 30 million in incremental revenue, and this goal is increasing every year.

Ugly Parts of GraphQL with Prisma

Pablo Díaz — Sat, 25 May 2019 17:56:10 +0000

The Stack

First, this is a post of what I think are the disadvantages of graphql combined with Prisma and graphql-yoga.

First

For this model

type Company @db(name: "companies") {
  id: ID! @id
  active: Boolean! @default(value: true)
  name: String!
  createdBy: User! @relation(link: INLINE)
  createdAt: DateTime! @createdAt
  updatedAt: DateTime! @updatedAt
}

Here is how a mutation would be done.

async createCompany(parent, args, ctx, info) {
    // 1. Check if they are logged in
    if (!ctx.request.userId) {
      throw new Error('You must be logged in!');
    }
    // 2. create
    const company = await ctx.db.mutation.createCompany(
      {
        data: {
          ...args,
          active: true,
        }
      },
      info
    );
    return company;
  }

Now for adding the createdBy attribute, this should be added to data

 {
        data: {
          ...args,
          active: true,
          // notice the connect object
          createdBy: { connect: { id: ctx.request.userId } }
        }
 }

Adding the userId in the connect object feels a little bloated, but it is not a big deal, it is fine. This will add only the id in the document of company.

Now, what if we want to add the user as a whole subdocument?

Then the existing Company model cannot be extended and the current model of User should have the @embedded config.

type User @embedded {
  name: String!
  email: String! @unique
  password: String!
  resetToken: String
  resetTokenExpiry: Float
}

And here is when it gets complicated

Now we want to use the create company mutation

  {
        data: {
          ...args,
          active: true,
          user: {
              create: {
                 // here goes all the user attributes that are required
              }
          },
          createdBy: { connect: { id: ctx.request.userId } }
        }
      },

This time the user object should be wrapped around a create object, and guess what happens if the user has another subdocument?

  {
        data: {
          ...args,
          active: true,
          user: {
              create: {
                 // what if the user has a permissions subdocument ?
                 permissions: {
                     create: {
                          name: "ADMIN"
                     }
                 }
              }
          },
          createdBy: { connect: { id: ctx.request.userId } }
        }
      },

Each time a subdocument is added it has to be wrapped around a create object! That's really annoying. For complex models, this will bloat the creation of documents with a lot of create objects.

And here it is another thing, for each create object, when it is saved in the database, it will create an _id attribute for each level even if it is not specified in the model, so this is unnecessary.

Second

Reusing the first mutation defined

async createCompany(parent, args, ctx, info) {
    // 1. Check if they are logged in
    if (!ctx.request.userId) {
      throw new Error('You must be logged in!');
    }
    // 2. create
    const company = await ctx.db.mutation.createCompany(
      {
        data: {
          ...args,
          active: true,
        }
      },
      info
    );
    return company;
  }

When this is called from the frontend using the Apollo Client

This is how the mutation can be called

 // create new company
 const promise = mutate({
       mutation: COMPANY_MUTATION,
      variables: { name: inputValue.value }
 });

promise.then((item) => {
  // to access the data returned from the server.
  const name = item.data.createCompany.name;
});

So, the response is filled with an object called data and another object with the name of the method defined in the server. This also feels bloated because now you have to know the method name of the backend while accessing the data in the frontend.

End

This is list is short, but I really don't like them. What do you think?

React Native without Expo

Pablo Díaz — Mon, 11 Jun 2018 01:57:28 +0000

Intro

I took an old react native project that I started long ago to finally finish it. I created a fresh start new react native project with create-react-native-app. I found out that now that Expo is the default option when creating a default app. I made some changes, integrated NativeBase and Redux (using some boilerplate) and I wanted to test the compilation process. I found out that now Expo can compile your code in their cloud and just download the compiled file to test it on the phone. I thought this was great until I saw the file size: 25 mb 😱 and I read that for iOS is around 30 mb. I just could not believe it. After reading some docs, Expo claims that they include multiple libraries so when a new library is needed, it is already in the user's phone and only the JS code is pushed over the cloud. I can't afford it, it is too many space for what I'm doing, so I decided I needed to detach 🤷.

ExpoKit

There are two detach possibles: ExpoKit and React Native. So, I tested first the ExpoKit. To test it I copied the folder because you can't go back after detaching it. The command created an android and ios folder. I don't know ios, so I only tested with Android, so I created the debug APK and the size was 30mb 😱 even bigger than the Expo cloud compilation. So, after multiple optimizations that included the minify, shrink, separateCPUbuilds, proGuard and I deleted most of the dependencies that includes and only got to reduce it at 15 mb 😕. It took me two days to make this, I had a lot of errors, I had to upgrade to latest Gradle because of my java version, added some repositories and I felt disappointed.

React Native

I didn't give up, so I created another folder and ejected the app to pure React Native. At first, I had the same problems with the Gradle, I had to updated it, clean and build it again. The Expo parts didn't worked anymore. I had a splash screen, app icon and fonts/icons with Expo that were not compatible, so that will be more work. I left the splash screen pending and fixed the fonts/icons installing react-native-icons. After that I compiled the APK and guess what? 7.5 mb with only the proGuard option enabled 🔥. It can be reduced even more with SeparteCPUBuilds.

Conclusion

Expo has really cool (but not necessarily) API's integrated, a Mac is not needed to compile to iOS, can update JS code without downloading app from store but the drawback of the size is too big, in my opinion. I still have to explore the iOS part.

How do you document your APIs?

Pablo Díaz — Tue, 06 Mar 2018 19:35:41 +0000

Do you use a full enterprise tool? Or maybe just a word document? What's your stack?

Building a parallel scraper

Pablo Díaz — Wed, 17 Jan 2018 16:12:31 +0000

The Why

Simon Sinek always says that people first care about the why and then about the how. So here it is. A few months ago, some friends that sell products on Amazon had a business idea. They said that among Amazon sellers there is the problem that the Amazon algorithm (known as A9) can change from one day to another without previous notification and that can lower the sales significantly because the keywords configured can stop being indexed. So they have to be aware when this happens. To solve this problem the current market uses some indexing services that tells you the indexing percentage of a product, so my friends had the idea that this still was tedious and they wanted to build an automated service that will check for this everyday and only notify when the indexing percentage reaches certain configured level. So how we will do this? Scraping is part of the solution.

The How

First we have to consider that having a business model based on scraping is very dangerous so we don't want to spend a lot of time developing this side project and we don't want to rely 100% on scraping. I chose python because it is great for scraping, so I for the web app I used Django. Now for the scraping I used beautiful soup and for the parallel processing I used Celery, with RabbitMQ as the message broker and Redis to store temporary the results. So what Amazon does to prevent scraping? First, they block the IPs that make more than 1 request per second, so I had to use Storm Proxies to change the IP for every single request. That should be enough right? Well it isn't enough, Amazon recognises when a request doesn't have a cookie so when multiple requests are made, they show a captcha. So is it possible to generate a cookie? Well, Amazon signs their cookies so it is quite impossible to create one or at least by me. If a captcha is shown we use an Amazon API to get the data we need, so we don't rely one hundred percent on scraping. Scraping Amazon has been quite a challenge, the service to change IP for every single request adds a delay of almost two seconds, so it is fully necessary to make it parallel because if we don't it would be two seconds for every keyword, instead it is two seconds for maximum 250 keywords or at least that's how it should theoretically. Sometimes one keyword lasts more than a minute and that's too much time for a user, so it is necessary to add timeouts and rate limit to not overload the server.

Currently we have 243,431 keywords indexed and adding around 14k per day and remember, it is one request per keyword.

If you want to check the service here it is: Check My Keywords

Writing My First Compiler

Pablo Díaz — Wed, 05 Apr 2017 19:43:37 +0000

Start

I wrote my first compiler for a university class project and I really enjoyed it so I want to share my experience on that. I'm about to get very technical on this, so first a little introduction.

Set

As you know, a computer does not understand source code, it understands binary code. So how a we get from source code to binary code? The computer hardware, the physical form in which it performs cycles and processes with the electrical signals, takes care of converting the zeros and ones to those electric signals. So guess what? Each compiler depends on the processor on which it is compiled (unless something else is used between like the java virtual machine). There exists an upper layer for binary code that is called assembly, a low level programming language. So the goal for my compiler is to convert high level programming language for a low level programming language. Take in consideration that a compiler does not necessarily needs to make a conversion to a low level programming language, it could be a conversion to another high level programming language.

The process

A compiler is divided in two main stages: the analysis and the synthesis. The analysis is the recognition of the structure of the source program with the recollection of information (like variables) and the synthesis is the construction of the translation from the structure and the information recollected. I made my own version of the analysis just to go through that process but for my compiler I actually used a powerful parser generator to avoid the problems of my own implementation, called ANTLR using the Visitor pattern.

The phases

Lexical Analysis

Identifies the elements of the source program according to the grammar of the language analysed. This phase saves information on the symbol table and produce tokens that represent the identification of the elements.

pos = initial + rate * 60

Transforms to

<id, 1> <=> <id, 2> <+> <id, 3> <*> <60>

Syntax Analysis

Also called parsing, this uses the tokens from the previous phase to construct a grammatical structure called the syntax tree. In this phase it is possible to recognize the syntax errors like missing ;, missing brackets

Semantic Analysis

Here things start to get fun. I started from this point because I used ANTLR for the previous two phases. The things I liked in this part was that I had to decide how actually my high level language input was going to be. Should I use indent (like python) or brackets (like java) to define scopes? Should I use strong typed variables(int var) or loosely typed (var) ? Along all this questions, there is something that always has to be kept in mind, the target assembly. The target assembly will determine what you can actually do or not. For my compiler, I choose the ARM compiler of the RaspberryPi.

The semantic analysis uses the syntax tree and the symbol table to verify the semantic of the instructions, in other words, that the source program actually makes sense.

Intermediate Code Generation

In this phase, I had to start generating assembly code look alike, called the three address code. Imagine this

pos = initial + rate * 60

t1 = id3 * t1
t3 = id2 + t2
id1 = t3

Where id and t are going to be registers of the assembly code in the next phase. In theory, this phase could make the compiler independent of the target compiler because you could generate different assembly from the same intermediate code generation.

Generating the code

In this part I had to take a few things into consideration. How I will store the values? How many space will each type use? How will I save globals? These are the first question to resolve and that's why it is important to know the how the assembly works. I decided that everything would be saved in 4 bytes, no matter what type of variable it is just to make it simple, but not efficient. And where I will save this bytes? I decided, the global vars would be saved in memory and everything else in the stack. Another step to take in consideration is how and how many registers are going to be used and for that a register descriptor and an address descriptor is needed because it is necessary to know the current values and addresses of the data and when a register is available for usage.

Finally, the most difficult part for me was the activation record. This part is used to initialise the registers to use in a block of code, it includes the parameters, the return value, the temporary variables and the control link. This part is the key to make recursion work.

Based on the Dragon Book.