Forem: Mailisk

Extracting verification codes with Cypress

Uroš Štok — Sat, 03 Feb 2024 16:26:12 +0000

When it comes to automating signups, verification codes are almost always used. These codes serve as an additional layer of security, ensuring that the user signing up is genuine and has access to the email associated with the account.

Verification codes range from only digits to mixed characters; it all depends on the platform.

How do we go about parsing and extracting these codes?

There’s two ways: we either focus on the text and search based on that, or we look for html elements in the email that contain the said code. Sometimes, a bit of both is required. In this post, we’ll explore both methods and show some common examples.

If you're looking for a more general guide on how to automate the verification flow, check out Email verification with Cypress

Searching with Regex

If our verification code consists solely of digits, we can write a simple regex that searches for a digit string of a certain length. If the verification code is 6 digits, we can use the following regex:

\b(\d{6})\b

it("Should verify email", () => {
  let code;

  cy.mailiskSearchInbox(Cypress.env("MY_NAMESPACE"), {
    to_addr_prefix: "testuser",
    subject_includes: "verification code",
  }).then((response) => {
    const emails = response.data;
    const email = emails[0];
    // we search the text version of the email
    const match = email.text.match(/\b(\d{6})\b/);
    expect(match.length).toBeGreaterThan(0);
    code = match[1];
    expect(code).to.not.be.undefined;
  });

  console.log("Your code is: " + code);
  ...
});

In the above code, we used the text version of the email. This doesn't contain a lot of the HTML clutter like styles, tags and classes, making it less likely we'd match the wrong content.

But what if the code contains both digits and letters? In this case, the Regex method isn't quite enough unless there's a prefix before the code that we can use. In the above example, the text version looks like this:

Please enter this verification code to get started on CoolApp: \n> \n> 065406 \n> \n>

Let's modfiy the code so it includes both numbers and letters, making it harder to extract:

Please enter this verification code to get started on CoolApp: \n> \n> A3G56L \n> \n>

Since we know that the code is after verification code to get started on CoolApp: we can include this in our regex, modifying it like so:

verification code to get started on CoolApp:.\*([A-Z0-9]{6})

it("Should verify email", () => {
  let code;

  cy.mailiskSearchInbox(Cypress.env("MY_NAMESPACE"), {
    to_addr_prefix: "testuser",
    subject_includes: "verification code",
  }).then((response) => {
    const emails = response.data;
    const email = emails[0];
    // we search the text version of the email
    const match = email.text.match(/verification code to get started on CoolApp:.*([A-Z0-9]{6})/);
    expect(match.length).toBeGreaterThan(0);
    code = match[1];
    expect(code).to.not.be.undefined;
  });

  console.log("Your code is: " + code);
  ...
});

We constructed our regex in three parts. First the prefix which ends on CoolApp:, then any characters .* and finally the format of our code [A-Z0-9]{6}.

Of course, in this example, simply using \b[A-Z0-9]{6}\b would have been enough. However, this can fail depending on the text content and lenght of the code, which is why we used the prefix.

Searching with HTML

Using HTML is usually a surefire way to easily extract verification codes. The reason is that most codes are marked differently (e.g., bolded), making them easy for the user to see and for us to extract.

Let's use the cheerio package. This will allow us to use selectors to fetch certain parts of the HTML.

In this example the email's html contains the following:

...
<tr>
  <td class="h1 black" align="left" dir="ltr" style="...">065406</td>
</tr>
...

This is quite convenient, as in this example there is only one td element with h1 black. If there were more then we'd have to filter from higher in the hierarchy (e.g. tr, table, etc.).

it("Should verify email", () => {
  let code;

  cy.mailiskSearchInbox(Cypress.env("MY_NAMESPACE"), {
    to_addr_prefix: "testuser",
    subject_includes: "verification code",
  }).then((response) => {
    const emails = response.data;
    const email = emails[0];
    // we load the email html into cheerio
    const $ = cheerio.load(email.html);
    // we find the element using the selector
    const nodesText = $("td.h1.black").text();
    // since there's one element and it only contains the code we can extract it directly
    const code = nodesText;
    expect(code).to.not.be.undefined;
  });

  console.log("Your code is: " + code);
  ...
});

And there we have it – two ways verification codes can be parsed and extracted from emails.

Password reset with Selenium in Node

Uroš Štok — Sun, 08 Jan 2023 11:58:16 +0000

Testing password reset functionality is an important part of ensuring the security and reliability of any web application. In this tutorial, we will learn how to test password reset in Selenium with Node.js.

In order to test this functionality we'll also need some demo app which supports password reset. For ease of use the final code including the test app for this post is available on GitHub.

Selenium Setup

We'll be using the selenium-webdriver library in Node. You can add this into your project using the following command

npm install selenium-webdriver

We're not setup to use selenium quite yet though, we'll also need to install the relevant driver. In this example we'll be using chrome and install the ChromeDriver. For an easier setup let's use the chromedriver npm package which uses one of the latest ChromeDriver versions. It's installed like so

npm install chromedriver

This will make it easier for other developers to checkout our repository since they won't need to do a separate installation step for the driver, doing npm install will be enough.

Writing the test

We'll write a test which focuses on the key aspects of resetting passwords

Create a new user, this will make it easier to run the test by itself with no other dependent logic.
Login as this user, here we'll just confirm that the selected password works.
Reset the password for this user, this will be a two step process. First we'll trigger the password reset email, after it arrives we'll enter our new password and reset it.
Login with new password, finally we confirm that the new password is accepted and the password reset works as expected.

As mentioned before, we're using the app available in the repository which has both the frontend and backend already prepared.

Let's start by getting the parts not related to emails out of the way. Here are the contents of our password-reset.test.js

const { Builder, By } = require("selenium-webdriver");
const { MailiskClient } = require("mailisk");

(async function passwordResetTest() {
  let resetLink;
  const namespace = "mynamespace";
  const testEmailAddress = `test.${new Date().getTime()}@${namespace}.mailisk.net`;

  const mailisk = new MailiskClient({ apiKey: "YOUR_API_KEY" });

  // Create a new Chrome driver
  let driver = await new Builder().forBrowser("chrome").build();

  try {
    // Let's visit the signup page and create a new user
    await driver.get("http://localhost:3000/register");
    await new Promise((r) => setTimeout(r, 500));
    await driver.findElement(By.id("email")).sendKeys(testEmailAddress);
    await driver.findElement(By.id("password")).sendKeys("password");
    await driver.findElement(By.css("form")).submit();
    await new Promise((r) => setTimeout(r, 500));

    // We should have been redirected to the login page, so let's enter our credentials
    await driver.findElement(By.id("email")).sendKeys(testEmailAddress);
    await driver.findElement(By.id("password")).sendKeys("password");
    await driver.findElement(By.css("form")).submit();
    await new Promise((r) => setTimeout(r, 500));

    // Let's send a password reset email
    await driver.get("http://localhost:3000/forgot");
    await new Promise((r) => setTimeout(r, 500));
    await driver.findElement(By.id("email")).sendKeys(testEmailAddress);
    await driver.findElement(By.css("form")).submit();
    // the reset email is sent here!

    // Now we wait for the email to arrive extract the link
    // TODO: we'll need to implement this part

    // We visit the reset link and set the new password
    await driver.get(resetLink);
    await new Promise((r) => setTimeout(r, 500));
    await driver.findElement(By.id("new-password")).sendKeys("newpassword");
    await driver.findElement(By.css("form")).submit();
    await new Promise((r) => setTimeout(r, 500));

    // Let's try logging in again, but this time with the new password
    await driver.get("http://localhost:3000");
    await new Promise((r) => setTimeout(r, 500));
    await driver.findElement(By.id("email")).sendKeys(testEmailAddress);
    await driver.findElement(By.id("password")).sendKeys("newpassword");
    await driver.findElement(By.css("form")).submit();
    await new Promise((r) => setTimeout(r, 500));

    // If we successfully log in, the app will redirect us to the dashboard
    let currentUrl = await driver.getCurrentUrl();
    if (currentUrl !== "http://localhost:3000/dashboard") {
      throw new Error(`Expected url to be 'http://localhost:3000/dashboard' got '${currentUrl}'`);
    }
  } catch (error) {
    console.error(error);
  } finally {
    // Close the browser
    await driver.quit();
  }
})();

The provided code above implements most of the key steps in resetting the password. We only need some way to receive the email programmatically and get the reset link.

Receiving Email

We'll use the mailisk library to fetch the password emails. First install it with

npm install mailisk

The test sample above already contains some relevant code, such as importing the library and creating a client

const { MailiskClient } = require("mailisk");
...
const mailisk = new MailiskClient({ apiKey: "YOUR_API_KEY" });

In order to use the client, you'll also need an api key, you can find yours in the dashboard.

Mailisk works based on namespaces you can think of these as a catch-all address which we can access programmatically. For example if we have the namespace mynamespace we can find all emails sent to john@mynamespace.mailisk.net (you can replace john with any valid email address).

...
const namespace = "mynamespace";
const testEmailAddress = `test.${new Date().getTime()}@${namespace}.mailisk.net`;

We use this snippet of code to create a unique email address per test. Every time the test starts it'll create a new user using a random email address, which will look like this

test.123456789@mynamespace.mailisk.net

This means we don't have to worry about clutter due to existing emails (e.g. it returning an old email with an invalid reset link).

searchInbox

In order to read the email we'll fill out the missing code using mailisk's searchInbox function

// Now we wait for the email to arrive extract the link
const { data: emails } = await mailisk.searchInbox(namespace, {
  to_addr_prefix: testEmailAddress,
});
const email = emails[0];
resetLink = email.text.match(/.*\[(http:\/\/localhost:3000\/.*)\].*/)[1];

The searchInbox function takes a namespace and options. We use the to_addr_prefix option to filter out emails sent to other addresses (e.g. john@mynamespace.mailisk.net). In combination with the email being random this ensures that we're guaranteed to only find the password reset email.

By default the searchInbox function filters out older emails and waits for at least one email to be returned. Which is why we don't need to pool/wait manually for the results.

As a response we'll get an array of emails, but we only care about the one. We'll run some regex to filter out the link and assign it to resetLink so it can be used later in the test.

And that's it. We've got an automated test that will reset the password and try logging in again with the new password.

Receive emails with Node

Uroš Štok — Tue, 01 Nov 2022 18:02:09 +0000

You've probably found a dozen posts about sending emails with Node. But what about receiving them?

Why?

When would you want to receive emails instead of sending them? The most common use cases are for testing password resets and email flows. A common email flow is signing up, receiving a code and entering that code to signup.

How?

We'll be using the mailisk library for receiving emails. This is a free email testing service which allows us to read emails. It works by creating a virtual email server that we can send emails to.

The email addreses look like this <anything>@mynamespace.mailisk.net. Where mynamespace is the name of your namespace. An email sent to an address that ends in @mynamespace.mailisk.net can be read using the API.

Setup

Install the library with

npm install mailisk

We'll interact with the API using the client, you can create one by passing your API Key

const { MailiskClient } = require('mailisk')

const mailisk = new MailiskClient({ apiKey: 'YOUR_API_KEY' })

Receiving Email

We can use the searchInbox function to receive emails. It only needs one parameter, our namespace.

const { MailiskClient } = require('mailisk')

const mailisk = new MailiskClient({ apiKey: 'YOUR_API_KEY' })

mailisk.searchInbox('mynamespace').then(console.log)

A note here is that, by default, this will only return the latest emails - any emails received in the past 5 seconds. This can be changed by overriding the from_timestamp parameter. You can check the docs for more information.

With that in mind, let's change it so it gives us all the emails in the inbox, regardless of when they were received

mailisk.searchInbox('mynamespace', { from_timestamp: 0 }).then(console.log)

In order to show this works we'll need to send an email. You can send one using your own email provider (e.g. Gmail). Any address that ends with your namespace will work, here let's simply send it to john@mynamespace.mailisk.net

After sending the email and executing the script, this is what we get in the console

{
  total_count: 1,
  options: { limit: 10, offset: 0, from_timestamp: 0, wait: true },
  data: [
    {
      id: '1667305800547-9KJHqVLG5',
      from: { address: 'test@example.com', name: '' },
      to: [ { address: 'john@mynamespace.mailisk.net', name: '' } ],
      subject: 'This is a test',
      html: 'false',
      text: 'Something goes here',
      received_date: '2022-11-01T12:30:00.000Z',
      received_timestamp: 1667305800,
      expires_timestamp: 1667565000,
      spam_score: 0.1
    }
  ]
}

Sending emails

To make testing a bit easier we can also use mailisk to send emails. These can only be sent to mailisk namespaces though

...
const namespace = 'mynamespace'

await mailisk.sendVirtualEmail(namespace, {
  from: 'test@example.com',
  to: `john@${namespace}.mailisk.net`,
  subject: 'Testing',
  text: 'This is a test.',
})

As an example, let's send ourselves an email and then read it

const { MailiskClient } = require('mailisk')

const namespace = 'mynamespace'

// create client
const mailisk = new MailiskClient({ apiKey: 'YOUR_API_KEY' })

async function main() {
  // send email
  await mailisk.sendVirtualEmail(namespace, {
    from: 'test@example.com',
    to: `john@${namespace}.mailisk.net`,
    subject: 'Testing',
    text: 'This is a test.',
  })

  // receive latest email
  const { data: emails } = await mailisk.searchInbox(namespace, {
    to_addr_prefix: `john@${namespace}.mailisk.net`,
  })

  // print out the subjects of all the recieved emails
  for (const email of emails) {
    console.log(`Received email with subject '${email.subject}'`)
  }
}

main().then(() => console.log('Done'))

You'll notice we also used to_addr_prefix here. This let's us filter the emails we fetch by their destination address. In this example we sent an email to john@..., and therefore we only care about emails sent to john@.... If we didn't filter this we would also return other emails, such as ones sent to somethingelse@...

Closing notes

This should be enough to get you up and started with receiving emails in Node.

If you're using Cypress take a look at our post on Password reset with Cypress, which features the mailisk cypress plugin (and an example repository). Otherwise there's additional information about options for fetching emails in the documentation and API Reference

Password reset with Cypress

Uroš Štok — Thu, 06 Oct 2022 17:10:42 +0000

Every website that has user accounts with passwords has a "Forgot password?" function. Suffice to say that this is a common test case that you usually want to be covered.

If you'd like to follow along or just want to see the final result checkout the repository.

With that out of the way. First, let's take a look at how this password reset functionality usually works:

User clicks "Forgot password", they're taken to a reset password screen
User inputs the email they want to reset the password for, an email is sent
User clicks the link from the email, taking them to a new page where they enter their new password
User can now login with their new password

The hard part here is getting the password reset link from an email. Automating the rest is pretty easy with Cypress.

Cypress

We're using Cypress as the test framework. If this is your first time using Cypress, take a look at their Installation guide. The provided repository for this post already has Cypress setup.

Mailisk

For "getting the password reset link from an email" part, we'll be using Mailisk and it's Cypress plugin. This is an email testing service that offers API endpoints for reading emails.

This lets us create a virtual email server that we can send emails to. The emails look like this <anything>@mynamespace.mailisk.net. Where mynamespace is the name of the namespace.

So any email we send to these addresses can be read using the API, which will allow us to automate the email password reset. But more on that when we get to it.

cypress-mailisk setup

Since we're using Cypress we'll be using the cypress-mailisk plugin. This will give us a simple command to access the received emails.

First install the plugin using

npm install -D cypress-mailisk

After installing the package add the following in your project's cypress/support/e2e.js

import 'cypress-mailisk'

In order to be able to use the API we'll also need to add our API key to cypress.config.js. An additional thing we'll be adding is the MAILISK_NAMESPACE. Doing this will let us easily use a different namespace.

module.exports = defineConfig({
  env: {
    MAILISK_API_KEY: 'YOUR_API_KEY',
    MAILISK_NAMESPACE: 'YOUR_NAMESPACE', // we're also adding this
  },
})

Writing the test

In the app/cypress/e2e folder we'll add a password-reset.cy.js file with the following contents

describe('Test password reset', () => {
  let resetLink
  const testEmailAddress = `test.${new Date().getTime()}@${Cypress.env(
    'MAILISK_NAMESPACE'
  )}.mailisk.net`

  it('Should sign up a new user', () => {
    cy.visit('http://localhost:3000/register')
    cy.get('#email').type(testEmailAddress)
    cy.get('#password').type('password')
    cy.get('form').submit()
    // if the register was successful we should be redirected to the login screen
    cy.location('pathname').should('eq', '/')
  })

  it('Should login as user', () => {
    cy.get('#email').type(testEmailAddress)
    cy.get('#password').type('password')
    cy.get('form').submit()
    // if the login was successful we should be redirected to the dashboard screen
    cy.location('pathname').should('eq', '/dashboard')
  })

  it('Should reset password', () => {
    cy.visit("http://localhost:3000/forgot");
    cy.get('#email').type(testEmailAddress)
    cy.get('form').submit()
    // this will send an email with a reset link to the provided email address

    // TODO: we need to fill this out
  })

  it('Should visit reset link and set new password', () => {
    cy.visit(resetLink)
    cy.get('#new-password').type('newpassword')
    cy.get('form').submit()
    // if the reset was successful we should be redirected to the login screen
    cy.location('pathname').should('eq', '/')
  })

  it('Should login as user with new password', () => {
    cy.get('#email').type(testEmailAddress)
    cy.get('#password').type('newpassword')
    cy.get('form').submit()
    // if the login was successful we should be redirected to the dashboard screen
    cy.location('pathname').should('eq', '/dashboard')
  })
})

This is a basic test which:

creates a user
tries logging in
resetting the password
logging in again.

You probably noticed the following line

const testEmailAddress = `test.${new Date().getTime()}@${Cypress.env(
  'MAILISK_NAMESPACE'
)}.mailisk.net`

Since we'll be registering and resetting the password of a user, we need an email for them. This will create a string similar to this

test.123456789@mynamespace.mailisk.net

This matches the pattern Mailisk uses for emails

<anything>@mynamespace.mailisk.net

The only difference is that we added the current time to the email. We're using this as a filtering trick. If we run the same test later, we can be sure there won't be any other emails for this user, which means checking for a new email will be really easy. And since with Mailisk we have unlimited email addresses, there's no worry of using them up.

To improve on this you could change from using the timestamp to using something like a uuid which would make the email address even more unique.

Reading the email

Now let's take a look at the password reset part

it('Should reset password', () => {
  cy.visit("http://localhost:3000/forgot");
  cy.get('#email').type(testEmailAddress)
  cy.get('form').submit()
  // this will send an email with a reset link to the provided email address

  // TODO: we need to fill this out
})

After we enter the email and hit submit and email is sent to that email address. We need to read the email, extract the link and save it in resetLink so we can visit it in the next step.

Let's use the cy.mailiskSearchInbox command for this

it('Should reset password', () => {
  cy.get('#email').type(testEmailAddress)
  cy.get('form').submit()
  // this will send an email with a reset link to the provided email address

  cy.mailiskSearchInbox(Cypress.env('MAILISK_NAMESPACE'), {
    to_addr_prefix: testEmailAddress,
  }).then((response) => {
    const emails = response.data
    const email = emails[0]
    resetLink = email.text.match(/.*\[(http:\/\/localhost:3000\/.*)\].*/)[1]
    expect(resetLink).to.not.be.undefined
  })
})

Let's break down what this does:

mailiskSearchInbox

The cy.mailiskSearchInbox command takes a namespace, options and callback as it's parameters. For the namespace we're using what we defined in the env earlier.

For the options, we pass our email in to_addr_prefix. This way, only emails sent to this email address are returned. A namespace can have unlimited email addresses, so if we don't filter, it would return emails from all addresses in this namespace. Including the ones we're not interested in, like john@mynamespace.mailisk.net.

By default this command also does a few quality of life things for us:

It calls the API with the wait flag, this means the call won't timeout until at least one email is received or 5 minutes pass. We can adjust the timeout by passing timeout in the options
It uses a default from_timestamp of current timestamp - 5 seconds. This simply means that old emails will be ignored, we wouldn't want the command to return with an old email just because it's still in the inbox.

The callback

In the callback we get the data which represents all the emails found. Since in this scenario only 1 email will be sent, we filter it out. Next, we use some regex to look for this string http://localhost:3000/anythingafterthis.

And et voilà we have our link, we just save this in the global variable resetLink so it can be used in the next test (where it vists the link and resets the password).

And that's it, we've automated the password reset using Cypress and Mailisk.

If we run the E2E spec all of our tests should pass.