Forem: Tom Camp

Announcing the winners of the Ably Realtime Hackathon

Tom Camp — Thu, 16 Nov 2023 11:25:13 +0000

The Ably Realtime Experiences Hackathon winning projects announcement is here!

Participants were challenged to build either a realtime dashboard or collaboration experience using the Ably APIs, with a bonus category for projects built with React ecosystem tools.

Over 700 participants signed up and over 50 submissions were received which you can now view in the project gallery. A huge thank you to all those who submitted projects. It was great to see Ably used in such a wide range of applications - everything from a multilingual chat app to collaborative science experiments! The volume and quality of entries made our judge’s task very difficult, but we are delighted to announce the winning projects!

Collaboration experiences

This category challenged participants to demonstrate the benefits of collaborating in realtime using Ably’s capabilities (including our new Spaces product).

Judging explored the usefulness of the application, the use of Ably collaboration features, and the overall functionality.

Judged by:

Matthew O'Riordan, CEO at Ably
Faye McClenahan, Senior Marketing Manager at Ably
Finalist guest judge: Patrik Modig, Director Of Engineering at Mentimeter

“It was great to see such diversity from the Hackathon submissions, all finding innovative ways to use collaborative features to really enhance the user experience - from educational apps, to community groups and project management tools." - Faye McClenahan

And the winners are …

1st place: SharedOS
2nd place: Ionic
3rd place: Plan it

Realtime dashboards

This category was focused on using Ably to demonstrate innovative realtime information displays and dashboards.

Judges explored the usefulness, data source, and data visualization of the application.

Judged by:

Paddy Byers, CTO at Ably
Tim Buntel, CPO at Ably
Andra Dinu, Senior Marketing Manager at Ably

“It’s inspiring to see the new and inventive ways these submissions are using Ably’s products in this category. Some enhanced existing experiences, making them more compelling with realtime features, while others found totally new applications with realtime collaboration at their core. The entrants’ creativity, and benefits of realtime data were clear to see.” - Tim Buntel

And the winners are …

1st place: Mading Live
2nd place: Mosiac
3rd place: Real Time Log Monitor

React bonus category

We challenged participants to build their application using Ably's React SDK and other React ecosystem tools such as Next.js to be eligible for a challenge bonus prize.

Judged by:

Stephane Moreau, Senior Manager, SDK at Ably

“To see our newly created react hooks for channels and spaces get used a lot in the hackathon was fantastic, the quality of submissions was impressive and we're looking forward to receiving more feedback as more and more people take advantage of them" - Stephane Moreau

And the winners are …

CodeABLy and Ionic

Want to see all the hackathon submissions?

These winning submissions, and all of the entries, included in the project gallery are great examples of what you can do using Ably. We will be sharing more entries on our blog and social channels in the coming weeks - so stay tuned on X (Twitter), LinkedIn and Discord.

And if you've been inspired to build something with Ably yourself, you can sign up for free and get:

6M Monthly messages
200 Peak concurrent channels
200 Peak concurrent connections

Create a realtime closed captioning solution with Deepgram and Ably

Tom Camp — Wed, 01 Nov 2023 15:34:30 +0000

Ensuring accessibility to all types of media is incredibly important, both for enabling as many people as possible to enjoy the media, as well as supporting as many ways as possible for interacting with it.

Closed Captioning helps massively with ensuring video content is more accessible by providing a transcription of its audio. Not only does this make video content more accessible to people who are deaf or hard of hearing, but also means it can be enjoyed in situations where having audio on isn't possible.

Historically it's been challenging to provide closed captioning for live experiences, be it a live interview, a sports game with commentary, or a livestream. But Deepgram's AI tooling has changed this, allowing users to easily convert realtime streams of audio into accurate transcripts.

In this tutorial, we'll look at how you can make use of Deepgram to generate realtime closed captioning, and then use Ably to handle the distribution of these transcripts to as many clients as you need.

All the code is available on GitHub.

Overview

For this tutorial, we'll be making a React application with Vite, coded in TypeScript.

The project will consist of a simple page with two buttons, 'Start Recording' and 'Stop Recording'. We will also have a separate server, which will be responsible for generating authentication tokens, as well as interacting with Deepgram for us. When someone is recording their audio, they'll send their audio to our server to then send it to Deepgram to be interpreted.

Once the transcript has been returned to our server, it will then publish the updated scripts to Ably, which will handle distributing them to all interested clients for rendering. We'll also include identification information per client so that we can create scripts indicating who said what.

Technology

Deepgram: A speech recognition (ASR) service that converts audio into text. Deepgram's API is powerful yet simple, making it a great choice for this project.
Ably: A Serverless WebSockets solution, which makes it easy to scale up communication between potentially millions of frontend devices and backends.
Vite and React: Vite is a frontend tooling product that enables a faster development environment. Coupled with React, building interactive and performant user interfaces is a breeze.

Both Ably and Deepgram provide free access to their products. Ably provides up to 6 million messages per month in their free package, and Deepgram provides $200 worth of usage for free.

You can sign up to Deepgram, and then go to your project to create an API key.

You can get an Ably API key by signing up to Ably, creating an app, and then getting an API key from the API key section of the app.

For this tutorial you’ll need an API key for each.

Setting Up the project environment

1. Initializing the project

Create a new directory for your project and navigate into it via your terminal. Initialize a new Vite project with the following commands:

npm create vite@latest realtime-transcription -- --template react-ts
cd realtime-transcription

This will create a new Vite project with a React and TypeScript template.

2. Installing necessary packages

Next, install the necessary packages for your project:

npm install @deepgram/sdk ably dotenv

@deepgram/sdk: The Deepgram SDK for interacting with the Deepgram API.
ably: The Ably library for realtime messaging, which includes the React Hooks functionality.
dotenv: A module that loads environment variables from a .env file into process.env. This will be used by the server, as Vite already provides a way to access environment variables.

To make sure everything is working, run npm run dev, and look at the endpoint it mentions (usually http://127.0.0.1:5173/) to check everything is rendering correctly.

3. Creating a `.env.local` file

Create a .env.local file in the root of your project directory to store your Ably and Deepgram API keys. This file should look like:

VITE_ABLY_API_KEY=your-ably-api-key
VITE_DEEPGRAM_API_KEY=your-deepgram-api-key

Make sure to replace your-ably-api-key and your-deepgram-api-key with your actual API keys from Ably and Deepgram respectively.

Setting up the transcription server

With the authentication details stored, let's start with setting up a server that will handle interfacing between Deepgram and the clients. This will just be a basic Node.JS server.

1. Creating the server directory and file:

First, let’s create a directory for our server files. In the root of your project directory, create a new folder named server. Inside the server directory, create a file named server.js.

mkdir server
cd server
touch server.js

2. Importing necessary modules:

At the top of the server.js file, you import the necessary modules that you will use throughout the server logic.

import dotenv from 'dotenv';
dotenv.config({ path: `.env.local` });

import Deepgram from '@deepgram/sdk';
import Ably from 'ably';

3. Initializing Deepgram and Ably clients:

const deepgram = new Deepgram.Deepgram(process.env.VITE_DEEPGRAM_API_KEY)
const ably = new Ably.Realtime(process.env.VITE_ABLY_API_KEY);

Here, you create new instances of the Deepgram and Ably clients, passing in your API keys from the environment variables.

4. Setting Up Ably Channels:

With our Ably connection established, we can create instances of Ably Channels.

const fromClientChannel = ably.channels.get('request-channel');
const broadcastChannel = ably.channels.get('broadcast-channel');

Here we have two Ably channels; request-channel for receiving audio data from the clients, and broadcast-channel for publishing transcriptions back to the clients.

5. Subscribing to Presence events:

Ably allows for meta information on connections to be communicated via the Presence feature. For our use case here, we will want to know when a client is ready to start communicating its audio to the server, as well as when they are stopping. Additionally, we will want to know the identifying details of a client, so that it can be appropriately attributed to their audio.

To do this, we will first start by subscribing to enter and leave events from the clients.

fromClientChannel.presence.subscribe('enter', (member) => {
    console.log("New member joined: " + member.clientId);
    // Start up Deepgram session

    // Start sending audio to Deepgram

    // Listen to clients via Ably for audio messages
});

fromClientChannel.presence.subscribe('leave', (member) => { 
    console.log("Member left: " + member.clientId);
});

6. Creating a Deepgram live transcription session:

Inside the callback for the enter event, just under the Start up Deepgram session comment, you create a new Deepgram live transcription session.

const deepgramLive = deepgram.transcription.live({
    punctuate: true,
    smart_format: true,
});

As part of the configuration, we set punctuate and smart_format to true to help make the produced transcripts easier to read.

7. Setting Up Deepgram event listeners:

Next, let's add some listeners for this session. The main one we're interested in is transcriptReceived, which will be how we will receive the transcript segments generated by Deepgram. The error and close listeners are helpful for debugging and future issues you may encounter.

Add the following just below the Start sending audio to Deepgram comment.

deepgramLive.addListener("transcriptReceived", (transcription) => {
    // Publish the transcript to Ably for the clients to receive
});

deepgramLive.addListener("error", (err) => {
    console.log(err);
});

deepgramLive.addListener("close", (closeMsg) => {
    console.log("Connection closed");
});

8. Handling received transcripts:

With the above we'll be able to recieve the transcripts, but we now need to extract the transcript text, and publish it to the broadcast-channel on Ably.

Add the following just below the Publish the transcript to Ably for the clients to receive comment:

const data = JSON.parse(transcription);
if (data.channel == null) return;
const transcript = data.channel.alternatives[0].transcript;

if (transcript) {
    broadcastChannel.publish(member.clientId, transcript);
}

9. Subscribing to client messages:

Finally, we need to listen for messages from the specific client who we're transcribing. The messages will come through Ably, and we will use the name field of a message in this tutorial to hold the client ID of the publishing client, making it easy for us to filter out any messages we're not interested in.

Add the following below the Listen to clients via Ably for audio messages comment.

const queue = [];

fromClientChannel.subscribe(member.clientId, (msg) => {
    if(deepgramLive.getReadyState() === 1) {
        if (queue.length > 0) {
            queue.forEach((data) => {
                deepgramLive.send(data);
            });
            queue.length = 0;
        }
        deepgramLive.send(msg.data);
    } else {
        queue.push(msg.data);
    }
});

Here all messages are being forwarded to the Deepgram live transcription session. It's worth noting here that we are communicating using an ArrayBuffer data type. We also store messages which arrive prior to the connection to Deepgram being established in a queue, so that we can still deliver them correctly.

Running the server

As we will want the Node server to run alongside the Vite server, let's set it to run when we execute npm run dev. Inside of our package.json file, update the "dev": "vite", line to be "dev": "vite & node server/server.js",.

Assuming everything has been done correctly, if you now run npm run dev both should start running without any errors.

Setting Up the client

With the server ready to start interfacing with Deepgram, it's time to set up our Vite app.

1. Instantiating Ably React Hooks

Navigate to the src directory, and replace the contents of main.tsx with the following:

import React from 'react'
import ReactDOM from 'react-dom/client'
import App from './App.tsx'
import './index.css'
import * as Ably from 'ably/promises';
import { AblyProvider } from 'ably/react';

const client = new Ably.Realtime({ authUrl: '/api/token' });

ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
  <AblyProvider client={client}>
     <App />
  </AblyProvider>
</React.StrictMode>,
)

All we're doing here is creating an instance of the Ably client library, which will be accessible within the App.tsx file due to us wrapping it in the AblyProvider.

You may note that when instantiating the Ably client, we're making use of an authUrl rather than a key like we did in the Node.js server. This is because we want to keep the API key hidden away from clients, so as to avoid unmoderated usage of them by potentially untrusted devices and users.

The authUrl method allows for us to instead provide a Token from the defined endpoint, which will be short-lived and able to be revoked if needed.

2. Creating the Ably authentication endpoint

Let's create this endpoint within the Vite application, which can provide our clients their Ably Tokens.

Firstly, we'll need to vite-plugin-api, which makes it easy for us to create API endpoints for our clients to use within Vite to obtain tokens. Run the following:

npm install vite-plugin-api express

Next, replace the contents of vite.config.ts with:

import { defineConfig } from 'vite';
import react from '@vitejs/plugin-react';
import { pluginAPI } from "vite-plugin-api";

// https://vitejs.dev/config/
export default defineConfig({
plugins: [react(),
   pluginAPI({
      // Configuration options go here
   }),
],
});

Create a new file in /src/api/token called index.ts. In it, add the following code:

import Ably from "ably/promises";

export const GET = async (req:any, res:any) => {
const client = new Ably.Rest(import.meta.env.VITE_ABLY_API_KEY);
const clientId = Math.random().toString(36).substring(2, 15) + Math.random().toString(36).substring(2, 15);
const tokenRequestData:Ably.Types.TokenRequest = await client.auth.createTokenRequest({ clientId });

return res.json(tokenRequestData);
}

Here we are using the Ably client library to generate a TokenRequest object, which we can return to the requesting client to use to authenticate with. Usually you'd have some form of login or check prior to just giving unlimited access to a client, but for this tutorial we'll keep things simple.

We are also assigning the token a clientId, which in this case is just a random string. This is what will be used to identify the client when generating transcriptions.

Updating App.tsx

Our Vite client should now have access to Ably, which means we're ready to start assembling everything. What we need from the client is:

We need to have two buttons in our app, one to start recording and one to stop recording. We will also need a text field to put our transcripts in.
We will need to request the client for permission to use their microphone. Upon receiving permission, we will start listening for audio to be shared with our server, and thus Deepgram.
For each snippet of audio we receive, we will publish it to Ably, for our server to use.
The client will listen for any transcript updates via Ably, ready to append them to our transcript on the page.
Importing dependencies:

Start by importing the necessary dependencies at the top of your /src/App.tsx file.

import { useState } from "react";
import './App.css'
import { useAbly, useChannel } from 'ably/react';
import { Types } from 'ably';

useAbly and useChannel are Hooks for interacting with Ably.
Types: Type definitions from the ably library.

6. Defining the App Component:

Replace the App component with the following:

function App() {
  const [state, setState] = useState({ active: 'stop' });
  const [transcription, setTranscription] = useState("");

  // Get Ably channel

  // Obtain states

  // Listen for transcription updates from Ably

  async function start(_e: any) {
     setState({ active: 'start' });
     //Add microphone access and send audio to Ably
  }

  async function stop(_e: any) {
     setState({ active: 'stop' });
     // Stop recording
  }

  return (
     <div>
        <p id="realtime-title">Click start to begin recording!</p>
        <button onClick={start}
        className={state.active === 'start' ? 'active' : ''}
        >Start</button>
        <button onClick={stop}
        className={state.active === 'stop' ? 'active' : ''}
        >Stop</button>
        <p id="message">{transcription}</p>
     </div>
  );
}

export default App;

Here we are setting up the framework for what we'll be designing. Importantly, we've added two functions, start and stop, which will handle the actions of clicking the 'start' and 'stop' button respectively. The currently active button is held in the state string. We also have defined the transcription string, which will be the string which holds our responses from Deepgram via Ably.

So that we can see which button is active visually, add the following CSS to the end of the App.css file:

.active {
   color: #61dafb;
   outline: 4px auto -webkit-focus-ring-color;
}
#message {
   white-space: break-spaces;
}

If you run the project again now with npm run dev, you should see our new page.

3. Setting Up Ably:

Inside the App component, under the Get Ably request channel comment, set up Ably by calling the useAbly and useChannel hooks.

const client = useAbly();
const requestChannel = useChannel('request-channel').channel;

The request-channel will be the channel we will publish to with our audio snippets.

4. Sending Audio to Ably

With our Ably channel instantiated, we can now start populating our start function, where we will be obtaining audio from our mic and publishing it to Ably.

We want to ensure we only use the one recorder, so let's set up recorder at the top of the App function, which is what we'll use as our singular MediaRecorder:

const [recorder, setRecorder] = useState({} as MediaRecorder);
  const [state, setState] = useState({ active: 'stop' });
  const [transcription, setTranscription] = useState("");

Under the Add microphone access and send audio to Ably comment, add the following code:

navigator.mediaDevices.getUserMedia({ audio: true }).then(async (stream) => {
   const mediaRecorder = new MediaRecorder(stream);
   setRecorder(mediaRecorder);

   // Enter Ably Presence to indicate we're ready to send audio

   // Send audio to Ably
});

With this code, we are obtaining permission to access the client's microphone audio, and then set up a Recorder to use.

Next, we want to enter the Ably Presence set, so as to indicate to the server we're about to start sharing audio data with it. Under the Enter Ably Presence to indicate we're ready to send audio comment, add the following:

   await requestChannel.presence.enter();

Finally, we can set up a listener for the audio, and publish these segments of audio as ArrayBuffers.

   mediaRecorder.addEventListener('dataavailable', async (event) => {
     if (event.data.size > 0) {
       const arrayBuffer = await event.data.arrayBuffer();
       requestChannel.publish(client.auth.clientId, arrayBuffer);
     }
   });
   mediaRecorder.start(1000);

We are also setting the recorder to emit the dataavailable event once every second with mediaRecorder.start(1000).

5. Stopping recording

With our start button functional, let's add the functionality we need to stop the recording in the stop function:

async function stop(_e: any) {
   setState({ active: 'stop' });
   if (recorder.state != null) {
      recorder.stop();
      await requestChannel.presence.leave();
   }
}

Here we're just stopping our MediaRecorder, and then also leaving the Ably Presence set to indicate to the server we're done sending data.

6. Subscribing to the Ably Channel for captioning:

With the above code written, we're now able to send our audio to our server via Ably, which is being then converted to a transcript by Deepgram, and published back to the Ably channel broadcast-channel. We now need to subscribe to that channel, and add the text attached to messages on it to our transcript string.

Use the useChannel hook to subscribe to the broadcast-channel on Ably.

const [lastPersonTalking, setLastPersonTalking] = useState('');

useChannel('[?rewind=100]broadcast-channel', (message: Types.Message) => {
   // Update the transcript
});

The ?rewind=100 parameter ensures we receive the last 100 messages published on the channel.

For each message, let's now update the transcript. Whenever a new person talks, we will create a new line, and include the client ID of the new speaker. Add the following under the Update the transcript comment:

let transcript = message.data;
if (lastPersonTalking !== message.name) {
   setTranscription(prevTranscription => (prevTranscription + '\n' + message.name + ': '));
   setLastPersonTalking (message.name);
}
setTranscription(prevTranscription => (prevTranscription + transcript + ' '));

Running the completed project

With all of that done, we should now have a fully functional app! Run npm run dev once more, and you should see the words you speak being captioned whilst recording.

Conclusion

With this tutorial we've shown how we can integrate captioning into almost any project. So long as your devices have an internet connection, you can make use of Deepgram to interpret an audio stream and create the appropriate transcripts. Although in testing this you're likely only making use of one or two subscribers, with the usage of Ably we're able to potentially scale an application such as this up to millions of subscribers, with many clients all conversing with one another.

Even if you're not looking to create a project focused around captioning and transcription, the capacity of converting voice into text and distributing it can be incredibly powerful, and I hope this tutorial can act as a strong start for implementing such projects.

The code for this project is all available on GitHub.

Handling Massive Fan-out with AWS Lambdas

Tom Camp — Fri, 01 Sep 2023 08:15:39 +0000

AWS Lambda is a serverless compute service that allows developers to run their code without having to manage the underlying infrastructure. With Lambda, developers can upload their code and the service takes care of scaling, provisioning, and managing the servers required to run the code.

This means developers can focus on writing code and not worry about the underlying infrastructure. AWS Lambda is useful because it allows developers to build and deploy applications quickly and efficiently without worrying about managing servers.

It also allows for more efficient use of resources, as the service automatically scales up or down to meet the demand, which can result in cost savings. Additionally, Lambda integrates with many other AWS services, providing developers with a wide range of tools to build and deploy their applications.

In this post, we'll dig deeper into Lambdas, why they’re useful, and what role they can play in creating more interactive, realtime experiences across the web.

Lambdas in realtime experiences

Almost every user-facing application these days is expected to have a responsive design for users. A couple of examples would be:

Chat applications, where users expect to see messages and reactions as they happen.
Social media where new posts and comments should load seamlessly.
Games where player positions and actions should be represented as they happen.
Collaborative spaces such as shared text documents, where other user’s changes should be visible instantly to avoid desynchronization.

Even experiences that have historically been viewed as fairly static, say an online shop, is now expected to be more interactive. Be it an accurate representation of remaining stock, new deals, or even up-to-date comments and reviews.

Lambdas are an incredibly powerful tool in all of these scenarios. For chat for example, usually, there will be some form of manipulation required of the chat message required prior to being shared with other users. Words may need to be filtered, images inserted, formatting applied, etc. Rather than having various backend servers you need to manage and scale up and down, you can instead have a Lambda function set up which expects to receive a message, apply these transformations, and then return it.

The issue is that the message doesn’t need to just go back to the client who has sent a message - it needs to be sent to all relevant clients. In the case of chat rooms, for example, everyone else in a chat room needs to receive the message.

This is where we need some form of broker, which can handle connections, which clients should receive what messages, and so on. This broker can be responsible for receiving messages from clients, passing them to our Lambda function, and then publishing the message to the relevant clients.

Additionally, in order to connect and communicate with end-users in a way that facilitates realtime communication, we need a communication protocol that allows for bidirectional low-latency messaging between devices and backend services.

WebSockets + Lambda: The perfect match for realtime experiences

WebSockets are a protocol ideal for this scenario. Not only are they bidirectional, allowing both clients and servers to send messages as they’re ready, but they also are widely supported across modern browsers and backend stacks.

Due to Lambdas being inherently dynamic, however, there needs to be an intermediary service that is capable of handling the fan-out and fan-in of connections, much like a load balancer, and decoupling the connections from the backend Lambdas.

This sort of service, which handles the distribution of messages to many connections, will typically make use of a pattern known as the Publish/Subscribe pattern. This pattern dictates a separation of ‘publishers’ (a client sending a message), and ‘subscribers’ (clients who’ve indicated they wish to receive messages) by a broker server(s). All a client or server needs to know is where to send their message, and which clients who’ve ‘subscribed’ to messages will receive messages from the Pub/Sub broker.

With this pattern, we can ensure our end-user devices are connected to the Pub/Sub broker via WebSockets, ready to receive and send messages. Our Lambda can then be sent messages from the broker.

WebSockets + Lambda example: A chat application

As an example of why these tools together are so powerful, consider a chat application that has a room with 100,000 members in it, which is realistic for some of the larger communities on popular chat apps such as Discord. When a user publishes a message to the chat room, it must:

Be checked for inappropriate content and have media inserted where appropriate.
Be persisted for future retrieval by users, in a database service such as Aurora.
Be distributed to the 100,000 members in the chat room.

Each of these steps needs to be handled by very different tools. Step 1 requires some compute power to be available to run the appropriate checks and changes.
Step 2 requires querying a database. Lambda allows for this computation and database querying to occur scalably.

Step 3 requires a large quantity of compute to handle tracking of who is currently ‘connected’ to a room, and thus needs updates, who’s received which updates, and handling of potential connection issues, all while ensuring that data isn’t duplicated, or even worse silently never makes it. Additionally, the ability to resume a connection from clients wanting chat updates and retrieve missed data is important for avoiding unnecessary load and accessing your database.

Step 3 is the domain of Pub/Sub brokers. By handling the distribution of data it helps simplify the interactions with your own backend services immensely. It helps in providing all the guarantees and functionality you’d eventually expect and need from this communication with client devices. It will also help to remove a lot of the potential load by providing tooling to effectively act as a cache for realtime data.

Demo: Making a chat app with Lambdas and WebSockets

To demonstrate these patterns, we’ll look to create a basic chat application. We’ll use an AWS Lambda function, which will be responsible for applying a filter to messages sent to it. We will be using WebSockets for communicating between clients and a Pub/Sub broker, which will be responsible for interfacing them with Lambdas. As our Pub/Sub broker we’ll make use of Ably, a realtime messaging broker intended for handling massive scale, which will be responsible for handling the fan-in and fan-out of data over WebSockets to end users.

Creating an AWS account

The first thing you’ll need to do is make sure you have an AWS account with which you can create our Lambda instance.

If you don’t have an AWS account, create a free one here.

Lambda

With your AWS account created, go to the Lambda functions page. Click the ‘Create function’ button.

On the creation page, make sure you have ‘Author from scratch’ selected, and call the function something like ‘chatModeration’. Once that’s done, select ‘Create function’ at the bottom of the page.

You should be redirected to the overview page of your new function. In the code section on this page, put in the following code:

// We'll need the Node.js HTTPS module to post a response back to Ably
import https from "https";

// -----------------------------------------
const ablyApiKey = "INSERT_YOUR_ABLY_KEY_HERE";
// -----------------------------------------

export const handler = async (event) => {
  let ablyMsg = JSON.parse(event.body);
  const data = ablyMsg.messages[0].data;
  const BANNED_WORDS = /(potato)|(cabbage)/g;
  let filteredMsg = data.replaceAll(BANNED_WORDS, (substring, offset, string) => {
    return "*".repeat(substring.length);
  });

  const channel = "chat:to-clients";

  return await postMessage(channel, filteredMsg);
};

async function postMessage(channel, message, callback) {
  // Prepare the data to post to Ably:
  const data = JSON.stringify({
    name: "message",
    data: message
  });

  const options = {
    host: "rest.ably.io",
    port: 443,
    path: `/channels/${channel}/messages`,
    method: "POST",
    headers: {
      Authorization: `Basic ${Buffer.from(ablyApiKey).toString("base64")}`,
      "Content-Type": "application/json",
      "Content-Length": Buffer.byteLength(data)
    }
  };

  return new Promise((resolve, reject) => {
    const req = https.request(options, (res) => {
      resolve(JSON.stringify(res.statusCode));
    });

    // handle the possible errors
    req.on("error", (e) => {
      reject(e.message);
    });
    //do the request
    req.write(data);

    //finish the request
    req.end();
  });
}

This code will receive messages, apply our filter (checking for ‘potato’ and ‘cabbage’), and star out any occurrences. This new message will be passed back to a different Ably Channel for our clients to receive. It will then return this filtered message back to the caller. Select ‘Deploy’ to make this code live, and then you can run a test with the ‘Test’ button. Specify a name for our test event, such as cabbageTest, and then add a string to represent a message which will be filtered. For example, “I don’t like how cabbage tastes”, and select ‘Save’.

Now, click ‘Test’ again to run our test input. You should see the response includes the word cabbage starred out.

We also want to make an endpoint for Ably to communicate with. In the Configuration tab, select the Function URL button. Select Create Function URL, and for this demo set the Auth type to None.

Setting up the Pub/Sub broker

With our lambda ready to receive messages, we will now need to set up our Pub/Sub message broker. In this demo we’ll be using Ably to handle this for us. You’ll need to create a free account if you don’t have one already.

Once you have an account, you can go to your account dashboard, and create a new app. You can call it anything you like, preferably something easily identifiable such as ‘Lambda Chat app’.

Once you have your new app, you should go to the API Keys tab of the app. API keys are used to authenticate with Ably and this specific app. Make a note of the ‘Root’ key there. We will use this from the Lambda function to reply with our filtered messages. We will also use it to generate Ably Tokens which will be used by our clients to authenticate with Ably.

We make use of Ably Tokens for users rather than the raw API key so that it’s easier to revoke access and control permissions of each user. If we gave untrusted users access to the API key, they’d have full access to everything and we’d have limited control over limiting them.

Make a note of this API key, and make sure to also insert it into your existing Lambda code as the ‘key’ value.

With the API key obtained, next go to the ‘Integrations’ tab. This is where we can easily establish certain endpoints for messages published into our Ably Broker, such as our Lambda function.

On this page, select ‘New integration rule’, ‘Webhook’, then ‘Webhook’ again. On this setup page, paste in the URL generated for your Function URL, and set the Channel FIlter to ‘chat:from-clients.

After creating this Rule, any messages which go into the chat:from-clients channel will be sent on to our new Lambda.

Creating the client-side chat app

Now we need a chat app for our users to use. We will run this locally to keep it simple. We’ll make it an Express server, with a single page for our users to chat on.

Create a new folder for this app locally, and run npm init inside it. Call it whatever you want, and set the entry point to ‘server.js’.

Next, run npm install ably express dotenv to get both Ably and Express available.

We’re using dotenv to allow us to use a .env file to hold our API key. Create a ‘.env’ file in the base of the directory, and add API_KEY=YOUR_ABLY_API_KEY, replacing YOUR_ABLY_API_KEY with your Ably API key.

Create the server.js file at the base of your directory, and fill it with the following:

const Ably = require("ably");
const express = require("express");
const app = express();

require("dotenv").config();

// Ably
const FROM_CLIENT_CHANNEL_NAME = "chat:from-clients";
const TO_CLIENT_CHANNEL_NAME = "chat:to-clients";

const API_KEY = process.env.ABLY_API_KEY;

// Use Ably to listen and send updates to users
const realtime = new Ably.Realtime({ key: API_KEY });

/* Express */
const port = 3000;

/* Server */
app.use(express.static("public"));

// Issue token requests to clients sending a request to the /auth endpoint
app.get("/auth", async (req, res) => {
  let tokenParams = {
    capability: {},
    clientId: uuidv4()
  };
  tokenParams.capability[`${FROM_CLIENT_CHANNEL_NAME}`] = ["publish"];
  tokenParams.capability[`${TO_CLIENT_CHANNEL_NAME}`] = ["subscribe"];

  console.log("Sending signed token request:", JSON.stringify(tokenParams));

  realtime.auth.createTokenRequest(tokenParams, (err, tokenRequest) => {
    if (err) {
      res.status(500).send(`Error requesting token: ${JSON.stringify(err)}`);
    } else {
      res.setHeader("Content-Type", "application/json");
      res.send(JSON.stringify(tokenRequest));
    }
  });
});

app.listen(port, () => {
  console.log(`Listening on port ${port}`);
});

function uuidv4() {
  return (
    "comp-" +
    "xxxxxxxx-xxxx-4xxx-yxxx-xxxxxxxxxxxx".replace(/[xy]/g, function (c) {
      var r = (Math.random() * 16) | 0,
        v = c == "x" ? r : (r & 0x3) | 0x8;
      return v.toString(16);
    })
  );
}

Here, we’re creating two endpoints for our express app effectively. One will be content in a ‘public’ folder, and the other is an /authendpoint, which is used to generate Ably Tokens for our clients to use. It’s important for clients to only ever make use of Tokens, rather than an API key, so as to ensure they’re limited to only specific Channels and actions, as well as have revocable access. They’re also given a random unique ID to identify them.

As we still need our main app page for users to use for chatting, create a folder called ‘public’, and create a file index.html in it. Add the following to that file:

This makes use of an existing Ably Chat widget to make the UI elements simple. We set the channel the client is subscribed to to chat:to-clients, and the channel we are publishing to as chat:from-clients.

Running the server

Now we’ve written our code, we can run the server:

npm rum start

The server will be accessible from port 3000. If you load it up, you should find yourself able to talk between clients.

Conclusion

With Lambdas we have the perfect backend for processing data. WebSockets allow for reliable communication between clients. These in combination with Ably for handling communication with devices and the backend mean that we have a fully scalable solution for any form of problem which requires realtime interactions.

In this case we’ve demonstrated the basics of a chat application, but there’s a lot more that can be done even just for chat. Things such as typing indicators, multiple chat rooms, and media insertion are all features that are expected in realtime chat applications, which make use of the same base model. Data can be communicated through a solution such as Ably to indicate typing, our backend Lambda functions apply some form of computation on the data.

You can find all the code used in this guide on GitHub.

Cursors Everywhere: An experiment on shared cursors for every website

Tom Camp — Tue, 15 Aug 2023 15:56:45 +0000

Working with collaboration tools like Miro and VS Code Live Share has transformed my expectations of collaborative interactions. However, I found myself trying to move my cursor around websites during calls with colleagues, despite not sharing my screen. It was as if I believed they could see my cursor. I even caught myself saying, 'and when I click here...', only to realize I was sharing with nothing but the void.

I decided to do something about this with a project I'm calling 'Cursors Everywhere'. The idea is simple; a Google Chrome plugin which allows for anyone on a website to share their cursor position with everyone else on the website with the plugin.

If you find this intriguing, join me on this adventure of building a cross-browser plugin. This post provides a transparent look into the decisions made and the challenges faced, from handling diverse web layouts to dealing with different browser behaviors and performance considerations. Let's dive in!

The Making of the Plugin

The Plan

The first step was defining the requirements for this "Cursors Everywhere" concept. A Chrome Plugin seemed like a logical medium—it's shareable with my non-technical colleagues and can easily access browser details like URLs and active tabs. Consequently, JavaScript became the language of choice.

The next question was: how to share the cursor positions?

A realtime protocol like WebSockets seemed like a logical way to share real-time updates of cursor positions. Working at Ably, it was a no-brainer to use it as my WebSocket-based pub/sub broker. A pub/sub broker simplifies many aspects of projects like this, often coming with built-in features that speed up development. For instance, I wanted each browser's cursor position to be continually available to other clients on the page. To achieve this, I could use the Presence feature of Ably, which maintains a set of attributes associated with each client and its connection.

Starting with the Basics - Sharing X and Y Coordinates

I began with a Chrome Plugin and Ably, expecting the remaining work to be minimal. My job was to add a listener for the cursor, publish a cursor's position to Ably, and share it with other clients. Ably's Channels could naturally segregate data sets, so each web page could have a unique Ably Channel to hold its cursor positions. Other clients would listen to these channels using the Ably Client library, ready to render cursors in the appropriate position.

Capturing Cursor Movements

The first task was capturing cursor movements, achieved by attaching an event listener to the mousemove event in JavaScript. This event fires whenever the cursor moves, providing the X and Y coordinates of the cursor at that time. Here's a snippet of how I achieved this:

// Using Ably for realtime messaging
let channel = ably.channels.get("cursor-position");

// Sending cursor position
document.addEventListener("mousemove", function(event) {
    let x = event.clientX;
    let y = event.clientY;
    channel.publish("position", { x, y });
});

// Receiving cursor position
channel.subscribe("position", function(message) {
    let x = message.data.x;
    let y = message.data.y;
    // Display cursor at (x, y)
});

This approach worked... technically. However, it had a number of issues. Most importantly, it did not take into account different screen sizes and resolutions. The same absolute coordinates represented different positions on different screens, leading to inconsistent cursor positions across clients.

Improving the Original Approach: Relative Positioning

To address this, I modified the approach to use relative positions instead of absolute positions. I represented the cursor position as a percentage of the total width and height of the page. This provided more consistent cursor positions across different screen sizes.

Here's the updated code:

// Sending cursor position
document.addEventListener("mousemove", function(event) {
    let x = event.clientX / window.innerWidth;
    let y = event.clientY / window.innerHeight;
    channel.publish("position", { x, y });
});

// Receiving cursor position
channel.subscribe("position", function(message) {
    let x = message.data.x * window.innerWidth;
    let y = message.data.y * window.innerHeight;
    // Display cursor at (x, y)
});

The Result

Now for the moment of truth. Was it really that easy?

Unfortunately, no. The issue was that everyone's browser dimensions varied, meaning the browser positions of cursors did not map between clients. This approach could work for websites with no adjusting elements, which enforced a static size. However, for anything more complex, this would likely be more misleading than useful.

How it's done professionally: Miro

It's worth reflecting at this stage on how an app such as Miro achieves accurate representations of cursor positions across their boards. Whilst browsers are incredibly dynamic in capacity, able to completely change their view and arrangement depending on current browser dimensions, what device you're viewing it from, and so on, it increases the amount of things that can go wrong in representing the actual position of something. Miro instead has a singular canvas, to which everything is mapped. When you zoom in to a Miro board, it won't try to adjust the text to fit in the display, and it won't shuffle around elements to better fit everything on to say a mobile view. The canvas has elements upon it locked rigidly upon it in set positions.

This concept of a 'true' representation of the space allows for everything to be defined far more easily. If a button is always at the same coordinate within the canvas (as opposed to having to deal with the browser's percieved positions of elements), then as long as we communicate the position of a user's cursor on the canvas then we will always perceive the cursor as being on the button.

As lovely as it would be to bring out my magic wand and make it so that all websites can be defined so simply, it is unfortunately just not the case. With dynamically adjusting elements and views, we're going to have to introduce some more complex cursor positioning logic.

Stepping Up the Game: Smooth Transitions and Fewer Messages

The amount of messages being sent was another major issue. Every time a user moved their cursor, a new message was sent. This led to a large volume of messages, which could slow down the browser and consume a lot of bandwidth.

1: Position Interpolation and Message Batching for Bandwidth Efficiency

Position Interpolation

Here's a rough example of position interpolation from the publishing side:

let lastPosition = {};

document.addEventListener("mousemove", function(event) {
    let clientId = ably.connection.id;
    let x = event.clientX;
    let y = event.clientY;

    // Interpolate the coordinates to represent the movement to the new position
    let interpolatedPositions = interpolate(lastPosition[clientId], { x, y });

    interpolatedPositions.forEach((position) => {
          // Some time delay to ensure it's a smooth movement
        updatePosition(clientId, position);
    });

    lastPosition[clientId] = { x, y };
});

In this code, we keep track of the last cursor position (lastPosition). When the cursor moves, we calculate the interpolated positions between the last position and the new position using a helper function (interpolate). We then send these interpolated positions instead of the actual positions.

2.1: Message Batching

Even with position interpolation, clients were still sending a large number of messages. To further reduce the volume, I decided to introduce message batching. Instead of sending a message for each cursor position, I batched multiple positions together and sent them as one message.

Here's a rough example of message batching from the publishing side:

let batch = [];

document.addEventListener("mousemove", function(event) {
    let x = event.clientX;
    let y = event.clientY;

    batch = batch.concat({ x, y });

    if (batch.length >= BATCH_SIZE) {
        channel.presence.update(batch);
        batch = [];
    }
});

In this code, we maintain a batch of cursor positions (batch). When the batch reaches a certain size (BATCH_SIZE), we send the entire batch at once using channel.presence.update.

By introducing position interpolation and message batching, we significantly reduced the number of messages sent, saving bandwidth and improving the performance of our plugin. These techniques, while adding a bit of complexity, were key to making the plugin practical for real-world use.

2.2: Improving Batching Methods with the Spaces SDK

As you might notice from the above code samples, the premise of batching and interpolation is inherently simple. However, efficiently implementing these methods in practice can be quite challenging. Luckily, some of my colleagues had been working on a new SDK called 'Spaces', designed to simplify these types of functionalities.

Integrating it into the project didn't require much tinkering. It makes use of our existing Ably instance and requires us to replace our usage of the presence sets directly with the Space's abstraction of it for cursors:

const spaces = new Spaces(ably);
space = await spaces.get(url);

// Updating the cursor position
space.cursors.set(position);

// Listening to cursor position changes
space.cursors.subscribe("cursorsUpdate", handleCursorUpdate);

The Spaces SDK handles the publishing of data, automatically batching cursor updates if the number of messages per second becomes too high.

Taking Into Account the Element Under the Cursor

With the existing solution improved it was time to address the real issue: accurate cursor position representation. Depending on the browser, device, and individual website design, the same coordinates can point to different elements on different users' screens.

After some thought, it dawned on me that the goal was usually to point at some form of element on the page, be it a button, a text box, or something else. Perhaps combining element identifiers and positions would bring us a step closer to accurately representing cursor position.

Element Identification

Ideally, each element would have a unique id, allowing us to communicate said id and call it a day. Unfortunately, many websites do not assign ids to all elements. To achieve my goal, I needed a way to uniquely identify each element on a webpage. Eventually, I decided to use the CSS path of the element, a string that describes the element's location in the DOM tree. If an element has an id, we've uniquely identified it already. If not, we can attempt to uniquely identify it by this path.

function getCSSPath(element) {
    let path = [];
    while (element.nodeType === Node.ELEMENT_NODE) {
        let selector = element.nodeName.toLowerCase();
        if (element.id) {
            selector += "#" + element.id;
        } else {
            let sibling = element;
            let siblingSelectors = [];
            while (sibling !== null && sibling.nodeType === Node.ELEMENT_NODE) {
                siblingSelectors.unshift(sibling.nodeName.toLowerCase());
                sibling = sibling.previousSibling;
            }
            if (siblingSelectors.length > 0) {
                selector += ":nth-child(" + siblingSelectors.length + ")";
            }
        }
        path.unshift(selector);
        element = element.parentNode;
    }
    return path.join(" > ");
}

This function traverses up the DOM tree from the given element, building a string that uniquely identifies the element.

Including Element in Cursor Position

I then modified the mousemove event handler to get the CSS path of the current element and include it in the cursor position data:

document.addEventListener("mousemove", function(event) {
    let x = event.clientX;
    let y = event.clientY;
    let elements = document.elementsFromPoint(x, y);
    let element = elements[0];
    let elementPath = getCSSPath(element);

    // Other code...

    let cursorPosition = {
        x,
        y,
        element: elementPath
    };

    // Other code...
});

Replaying Cursor Movements

When replaying cursor movements on the client side, we used the CSS path to find the corresponding element and position the cursor relative to that element. If the element was not found (e.g., because the DOM has changed), it can fall back to positioning the cursor at the absolute X and Y coordinates.

space.cursors.subscribe("cursorsUpdate", function(cursorUpdate) {
    let position = cursorUpdate.position;
    let element = document.querySelector(position.element);
    if (element) {
        // Position cursor relative to the element...
    } else {
        // Position cursor at absolute coordinates...
    }
});

By considering the element under the cursor, we improved the accuracy of cursor sharing and made the shared browsing experience more consistent across different browsers and devices.

Still Not Quite There...

Although we now have better cursor positioning than before, we still have issues. Although we can sometimes represent the element we want to highlight with the CSS path, it's not something we can rely on. As seen in the above gif, the cursor will jump to quite differing locations as the solution swaps between element positioning and just using the x and y coordinates.

Additionally, within elements themselves, it doesn't work well if they themselves can change form. For example, a common use-case I'd expect from this plugin would be to indicate a word in a paragraph of text. Unless we have specific spans or ids on each letter, we'll just be using the element as the entire paragraph's element. This means that as a display scales up/down, the text will wrap around, and result in the specific coordinates within our paragraph element for a word or letter varying from client to client.

This isn't to mention the premise of certain elements only existing for certain clients. For example, it's not irregular to have certain elements appear only on mobile devices, and others on monitors.

Improving Websites for Cursor Tracking

With all of the above, I decided I'd need to dig deeper into what exactly I'd want to see in a website to properly allow for accurate cursor positioning being shared. Two key considerations emerged: element identification and element mapping across different views.

1. Adding Better Identification on Elements

For the shared cursor to be most useful, it needs to be able to accurately represent not just the cursor's position in terms of x and y coordinates, but also the context in which it is positioned, i.e., which element it is hovering over. To achieve this, we used the CSS path of the element.

However, not all elements on a web page have unique identifiers, and in some cases, the structure of the page could change dynamically, leading to inconsistencies in CSS paths.

To improve this, websites would need to have unique IDs or data attributes for elements that are important for having accurate cursor representation. This allows the plugin to more accurately track the cursor's position relative to these elements, improving the shared experience.

For example:

<button id="submit-button" data-cursor-id="unique-button">Click me!</button>

This level of detail could significantly enhance the shared cursor experience, providing more precise and meaningful cursor positions.

2. Mapping Elements Across Different Views

Modern websites often have different layouts for desktop and mobile views. Elements that represent the same functionality might be completely different across these views.

For example, a navigation menu could be a horizontal list of links in a desktop view, but a collapsible dropdown menu in a mobile view. This can cause issues with the shared cursor plugin, as the cursor's position over an element in one view might not map accurately to the equivalent element in a different view.

To mitigate this, a system where equivalent elements across different views are tagged with the same unique identifiers. This allows the plugin to accurately map the cursor's position across different views, ensuring a consistent shared experience regardless of the user's device or screen size.

<!-- Desktop view -->
<nav id="desktop-nav" data-cursor-id="main-nav">
  <!-- Navigation links -->
</nav>

<!-- Mobile view -->
<div id="mobile-nav" data-cursor-id="main-nav">
  <!-- Navigation links -->
</div>

With these considerations in mind, I created an incredibly simple React app, to test out these concepts and see how well the cursor positioning worked. The site I made had two view types, mobile and browser, with each element having unique identifiers. I even set it up to have each letter have its own unique identifier, which made me slightly queezy:

import React from 'react';

export default function DesktopView() {
  const lines = [
    { id: 'line-1', text: 'This is the first line of text blablablablablablablablabla.' },
    { id: 'line-2', text: 'This is the second line of text blablablablablablablablabla.' },
    { id: 'line-3', text: 'This is the third line of text blablablablablablablablabla.' }
  ];

  return (
    <div className="desktop-view">
      {lines.map((line) => (
        <p key={line.id} id={line.id}>
          {line.text.split('').map((character, index) => (
            <span key={index} id={`${line.id}_${index}`}>
              {character}
            </span>
          ))}
        </p>
      ))}
    </div>
  );
}

With that all done, I was quite happy with the results of this specially designed website. Unfortunately, making this plugin work in a general-purpose way for all existing website was still quite a while away.

Conclusion

At the end of this adventure, I'd unfortunately fallen short of designing a solution to work for accurate canvas representation for cursor position sharing across clients. However, I still had a tool now that somewhat worked across web pages, certainly for larger-scale gesturing between clients, such as to point at an image or a block of text.

I hope that sharing this experience might help others when it comes to creating their own interactive elements on their own sites. Although this can be a massive challenge to implement generically across all websites, that's different when it's your own website. When you're in control of how you lay out your website, you have the power to enforce proper identification of elements, and choose where on the site to say make use of a canvas technique that large players in the field such as Miro employ.

I'm expecting to see more and more sites implementing shared interactive elements to their sites as the technologies required for it become cheaper and easier to use. With this change, I anticipate a shift in website design to better accommodate these functionalities. Maybe one day it'll be more viable to make general-purpose tools such as this Cursors Everywhere project. For now, I'll just have to make sure I'm sharing my screen when trying to point at things.

If you're interested in the code for the project as it stands, you can find it on GitHub.

Building realtime experiences with Amazon Aurora

Tom Camp — Tue, 01 Aug 2023 15:39:50 +0000

Aurora is a managed database service from Amazon compatible with MySQL and PostgreSQL. It allows for the use of existing MySQL code, tools, and applications and can offer increased performance for certain workloads compared to MySQL and PostgreSQL.

Being an AWS product, it benefits from the range of tools and services available on AWS, with simple integrations for analytics and processing. Having all of these tools available makes building larger projects easier and quicker.

However, this largely applies to the development and interactions within AWS’s systems. One of the largest requirements for any application being developed is for it to eventually interact with users. Be it a chat app or a multiplayer game, there’s a requirement for a reliable bridge between the user and the backend services.

In order to connect and communicate with end-users in a way that facilitates realtime communication, we need a communication protocol that allows for bidirectional low-latency messaging between devices and backend services.

In this blog post, we will dig into the useful implementations of Aurora and WebSockets across different use cases, and the best architectures to ensure scalability and reliability, followed by an in-depth tutorial on building with two to make a chat app.

We have also produced a short video that showcases the key things you need to consider when developing user-facing applications with AWS and Aurora as a backend.

What scenarios need Aurora and WebSockets?

Almost every user-facing application these days are expected to have a responsive design for users. A couple of examples would be:

Chat applications, where users expect to see messages and reactions as they happen.
Social media where new posts and comments should load seamlessly.
Games where player positions and actions should be represented as they happen.
Collaborative spaces such as shared text documents, where other user’s changes should be visible instantly to avoid desynchronization.

Even experiences that have historically been viewed as fairly static, say an online shop, is expected now to be more interactive. Be it an accurate representation of stock remaining, new deals, or even comment sections and reviews being up to date.

How does Aurora help?

Whilst all the above to the user are happening ephemerally, with messages, positions, and actions all occurring and being superseded by further actions, there still needs to be a source of truth for establishing both the current state, as well as historical state. In a chat application for example, we need somewhere which holds past messages and interactions, which can be queried by users when scrolling through past conversations, or searching for certain terms.

Aurora acts as a scalable solution to this problem, allowing developers to get going with little setup required.

How do WebSockets help?

For all these examples, it’s not just the user needing to receive updates, the user also needs to be able to send updates themselves. With all of these examples, protocols such as WebSockets enable this bi-directional communication, where any device at any time is able to send a message to update others.

Architecture: Ensuring scalable realtime experiences

With Aurora and WebSockets, we have two tools that can enable both realtime communications, as well as persistence. However, we need a pattern that can enable make use of these two tools in a scalable way.

At the core, most systems will consist of many clients, all needing to communicate with one another. These interactions will likely require some form of interaction with an authoritative backend stack, be it for moderation, rate-limiting, or payload manipulation. After some form of processing step, potentially making use of persisted information in a database, the result should be communicated to all other relevant clients, and the message persisted in our database.

In principle, we can simply have our clients all connect via WebSockets to our server, and have the server handle maintaining the connections and ensuring the delivery of data to clients. As the service scales up, however, the load of maintaining connections, state of clients, on top of performing the processing and database interactions can become too much for a single server.

Much like for HTTP requests, this is the sort of problem where you’d typically want some form of load balancing, where requests coming in require are divided up by some frontend load balancer. This load balancer will also be responsible then for maintaining connections, helping reduce the unnecessary work occurring on our main servers.

WebSockets require a sustained connection however and are stateful, which means that a load balancer cannot work in the same way one would build an HTTP load balancer. Instead, we need these load balancers to maintain a shared state of clients and their connections, with whom these clients are communicating, and also information on what messages have been sent to who. This isn’t to then get started on things such as handling connectivity issues which add far more complexity.

This sort of service, which handles the distribution of messages to many connections, will typically make use of a pattern known as the Publish/Subscribe pattern. This pattern dictates a separation of ‘publishers’ (a client sending a message), and ‘subscribers’ (clients who’ve indicated they wish to receive messages) by a broker server(s). All a client or server needs to know is where to send their message, and any clients who’ve ‘subscribed’ to messages will receive messages from Pub/Sub broker.

With this pattern, we can ensure our end-user devices are connected to the Pub/Sub broker via WebSockets, ready to receive and send messages. Our backend services can also be connected to the broker, ready to process incoming messages and publish new state changes where needed with only a single connection. This separation of focus, with the Pub/Sub broker focused on handling connection states, and our backend focused on the actual processing of data allows for solutions to be scalable.

If there’s a large influx of new connections, but processing requirements are low, we can scale up the number of pub/sub brokers but keep the number of backend servers the same. If the number of connections is the same, but the requests coming in require a larger amount of processing, we can do the reverse.

Architecture example: Chat

Be checked for inappropriate content and have media inserted where appropriate
Be persisted for future retrieval by users, in a database service such as Aurora
Be distributed to the 100,000 members in the chat room
Each of these steps requires very different tools to handle. Step 1 requires some compute power to be available to run the appropriate checks and changes. Step 2 requires all the guarantees that a database such as Aurora provides.

Step 3 requires a large quantity of compute to handle tracking of who is currently ‘connected’ to a room, and thus needs updates, who’s received which updates, handling of potential connection issues, all while ensuring that data isn’t duplicated, or even worse silently never makes it. Additionally, the ability to resume a connection from clients wanting chat updates and retrieve missed data is important for avoiding unnecessary load and accessing your database.

Step 3 is the domain of realtime brokers. By handling the distribution of data, and providing all the guarantees and functionality you’d eventually expect and need from this communication with client devices, it helps simplify the interactions with your own backend services immensely, in addition to removing a lot of the potential load by providing tooling to effectively act as a cache for realtime data.

Demo: Making a chat app with Aurora and WebSockets

To demonstrate these patterns, we’ll look to create a basic chat application. We’ll use an AWS EC2 instance, which will have a server responsible for interfacing with our Aurora database. We will be using WebSockets for communicating between clients and this server. As our pub/sub broker we’ll make use of Ably, a realtime messaging broker intended for handling massive scale, which will be responsible for handling the fan-in and fan-out of data over WebSockets to end-users.

Step 1: AWS account

The first thing you’ll need to do is make sure you have an AWS account with which you can create an EC2 instance where we’ll host a server, in addition to an Aurora instance.

If you don’t have an AWS account, create a free one here.

EC2

First, navigate to the EC2 console. Here you should be able to see an option for ‘Launch instance’. If you click this, you should be navigated to the setup page for an EC2 instance.

On this setup page, fill in the following details:

Set the Name to ‘ec2-chat’.
Application and OS Images (Amazon Machine Image), should be set to Amazon Linux, with the specific image being Amazon Linux 2 AMI (the default free option).
The Instance should be t2.micro
For the Key pair (login), you can either use an existing key you have (which should be listed in the dropdown) or generate a new key from the button next to the input. If creating a new key, put in a memorable name, and keep the other settings on their defaults. This should automatically save a .pem file to your device, so make a note of where it’s saved to.
For Network settings, make sure that ‘Allow HTTPS traffic from the internet’ and ‘Allow HTTP traffic from the internet’ are both enabled.

Keep the remaining sections as-is, and select ‘Launch instance’ once you’re ready. If this is successful, you should see a message stating that it’s been successful.

We can now go look at our running instance by following the ‘View all instances’ button. Here you should see the ‘ec2-chat’ EC2 instance, which should eventually enter the ‘Running’ state.

Aurora setup

With the EC2 instance running, we’ll want to get Aurora going for it to interact with. To get started, head to the AWS RDS console. On this page, check the top-right corner to ensure that the region matches the region you created the EC2 instance in.

On this page, you should see a button to ‘Create database’. Click it to start the setup process.

On the Create database page, select 'Standard create'.

For this tutorial, we’ll be going with a MySQL-compatible Aurora. For Engine type, choose Aurora (MySQL Compatible).

As this demo is only intended for testing purposes, Under Templates select ‘Dev/Test’.

Under ‘Settings’, create your own cluster identifier, such as say ‘chat-db-cluster’, and then your own username and passwords which you’ll remember.

In the Instance configuration section, set the instance class to be ‘Burstable classes’, and then select the ‘db.t3.small’ version. This is again to prioritize reducing potential costs for this demo. If you were to create this at scale, you’d want to look to scale up your clusters.

Next, under Connectivity, we’ll want to allow for our previously created EC2 instance to communicate with our Aurora cluster. Tick the ‘Connect to an EC2 compute resource’ section, and select your ‘ec2-chat’ instance. This should set up your VPC for this interaction automatically.

Finally, there is a collapsed section at the bottom of the page called ‘Additional configuration’. Here we can define an initial database for the Aurora cluster. Set the ‘Initial database name’ to ‘messages’, and leave the remaining options as they are.

With these settings configured, select the ‘Create database’ button. You should be re-directed to the databases overview page, where you can see the cluster we just set up being created.

Once they’re done being created, select the top-level database cluster, and look at the ‘Endpoints’ section. Make a note of the endpoint with the ‘writer’ type, as this will be used by the EC2 instance to interact with the database.

Step 2: Adding the chat interface to the EC2 instance

xWith the EC2 instance and Aurora cluster running, we can now work to run our web server on the EC2 instance. To do this, we’ll need to connect to the EC2 instance.

The easiest way is to go to the EC2 instance we created in the AWS dashboard. On the ‘ec2-chat’ overview page, select the ‘Connect’ button.

From here, select ‘Connect’ again on the new page within the ‘EC2 Instance Connect’ section.

This should open up a terminal which we can interact with. As a good practice, run sudo yum update initially to fetch any security patches and fixes.

With that done, let’s install nvm on the instance by running the following:

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.3/install.sh | bash

Activate nvm using the following command:

. ~/.nvm/nvm.sh

Next, let’s install version 16 of Node.JS, which is the latest version as of writing supported by Amazon Linux 2:

nvm install 16

With Node.js installed, let’s get git:

sudo yum install git -y

To make sure this worked, you can run git — version, which should spit out the version of git you have installed.

With git installed, we can now pull the Node.js code we’ll need to make our Aurora server.

git clone https://github.com/ably-labs/ably-aurora-chat.git

Enter the folder:

cd ably-aurora-chat

This repo contains a basic node.js express server which we’ll run on our EC2 instance. It makes use of the Ably Chat WebComponent to handle subscribing to Ably and the UI side of things.

All of the express details are contained in the src/server.js file. There are three main components; setting up the express server, connecting to Ably to communicate with clients, and communicating with our Aurora instance.

Express

For our Express server, we’re providing access to our chat app page via the root of our URL (/), originating from the public folder:

/* Server */
app.use(express.static('public'));

We then also provide an endpoint, /auth, which is used to generate Ably Tokens for our clients to use. It’s important for clients to only ever make use of Tokens, rather than an API key, so as to ensure they’re limited to only specific Channels and actions, as well as have revocable access. They’re also given a random unique ID to identify them.

// Issue token requests to clients sending a request to the /auth endpoint
app.get("/auth", async (req, res) => {
    let tokenParams = {
        capability: { },
        clientId: uuidv4()
    };
    tokenParams.capability[`${FROM_CLIENT_CHANNEL_NAME}`] = ["publish"];
    tokenParams.capability[`${TO_CLIENT_CHANNEL_NAME}`] = ["subscribe"];

    console.log("Sending signed token request:", JSON.stringify(tokenParams));

    realtime.auth.createTokenRequest(tokenParams, (err, tokenRequest) => {
        if (err) {
            res.status(500).send(`Error requesting token: ${JSON.stringify(err)}`);
        } else {
            res.setHeader("Content-Type", "application/json");
            res.send(JSON.stringify(tokenRequest));
        }
    });
});

Ably

For Ably, the server is subscribed to an Ably Channel which clients will be sending their messages to. It will then apply any transformations required on the data (in this case a basic filter), store the data to Aurora, and then pass the new message to all other clients.

const BANNED_WORDS = /(potato)|(cheese)/g;
// Use Ably to listen and send updates to users, as well as save messages to Aurora
const realtime = new Ably.Realtime({ key: API_KEY });
realtime.channels.get(FROM_CLIENT_CHANNEL_NAME).subscribe(async (msg) => {
    let filteredMsg = msg.data.replaceAll(BANNED_WORDS, (substring, offset, string) => {
        console.log(substring);
        return '*'.repeat(substring.length);
    });
    console.log(filteredMsg);
    createNewMessageInAurora(filteredMsg, FROM_CLIENT_CHANNEL_NAME);
    new Ably.Rest({ key: API_KEY, clientId: msg.clientId }).channels.get(TO_CLIENT_CHANNEL_NAME).publish(msg.name, filteredMsg);
});

Aurora

For communicating with Aurora, we initialize our connection first:

let sequelize = new Sequelize.Sequelize(
    process.env.DB_NAME,
    process.env.DB_USERNAME,
    process.env.DB_PASSWORD,
    {
        dialect: 'mysql',
        dialectModule: mysql2,
        host: process.env.DB_HOST,
        logging: console.log,
        port: Number.parseInt(process.env.DB_PORT),
    }
);

We define the Message type, which is used to hold our messages, and then sync to Aurora:

const Message = sequelize.define('Message', {
    clientId: DataTypes.STRING,
    message: DataTypes.STRING,
    room: DataTypes.STRING
});

sequelize.sync();

Finally, we have a function to send messages to Aurora, which was used in the Ably section:

async function createNewMessageInAurora(message, chatRoomName) {
    const response = await Message.create({
        clientId: message.clientId,
        message: message.data,
        room: chatRoomName
    });

    console.log(response);
};

Step 3: Running the server

To run, we first need to install the required npm modules:

npm install

Next, you’ll need to add your own config details for your Aurora instance to a .env file to be used by the server. You can copy a template .env file from .env.example:

cp .env.example .env

In this .env file you’ll need to add the database host, username, and password you noted earlier when setting up Aurora.

You’ll also need an Ably API key, which is used by the server to authenticate with Ably and generate Ably Tokens for clients. If you have an Ably Account already, you can get it from one of your Ably Apps. If not, you can sign up for a free Ably Account, create an app, and then get your API key from there.

Once you’ve added your variables, you’re ready to run the code:

npm run start

The server will be accessible from port 3000. You can find the server’s URL on the EC2 instance’s page:

Note: This is not configured for HTTPS, so make sure you don’t specify https in the URL.

If you go to the URL ‘http://YOUR_URL:YOUR_PORT’, you should see the chat app.

You should find that we’re able to communicate between multiple clients through different browsers. If you type in one of the banned words, it will also be starred out. All data will be persisted to our Aurora backend, where it can be queried in the future, say if we wanted to search for a specific word that’s appeared historically in messages.

Conclusion

With EC2 and Aurora, we have the perfect backend for persisting and processing data. WebSockets allow for reliable communication between clients. These in combination with Ably for handling communication with devices mean that we have a fully scalable solution for any form of problem which requires realtime interactions.

In this case, we’ve demonstrated the basics of a chat application, but there’s a lot more that can be done even just for chat. Things such as typing indicators, multiple chat rooms, and media insertion are all features that are expected in realtime chat applications, which make use of the same base model. Data can be communicated through a solution such as Ably to indicate typing, our backend EC2 instance could make use of S3 for storing images, and we can make use of multiple Ably Channels for more chat rooms.

Other realtime use-cases also make use of the same sort of structure: sports apps need to send out notifications from their backend services to frontend devices whenever a big event occurs in a game. This will again make use of a backend such as an EC2 instance with Aurora backing up necessary information, and use a platform such as Ably to handle the scale of distributing updates to everyone.

Creating real-time experiences in e-commerce with Fauna and Ably

Tom Camp — Thu, 01 Sep 2022 07:46:38 +0000

As e-commerce has continued to mature, the level of service expected from consumers creates the need to build more reactive experiences. To meet those expectations, you’ll need a site that shows up-to-date information, with reliable displays of the latest product availability and discounts provided frictionlessly. We’ll be using Fauna and Ably to build this site.

What we’ll be making is a simple website with the following features:

Ability to browse all available items
Each item can be viewed in further detail, with descriptions, price, etc., all being viewable
Any changes to these details will be reflected in real time to the customer
Items can be purchased. There will be checks to ensure each purchase can be fulfilled, such as checking the existing stock to make sure users aren’t buying out-of-stock items
The ability for a customer to browse their previous orders

Why Fauna?

Fauna is a distributed document-relational database delivered as a global API. On top of providing a substantial free tier, it’s incredibly easy to get started. It’s highly scalable partly due to being serverless-first, so as projects expand, it helps keep things simple. Fauna delivers globally distributed ACID transactions by default and a document-relational data model, accommodating shifting access patterns and dynamic applications.

Due to allowing for relational data, querying, and event streaming, it’s perfect for us to host our e-commerce products, customers, and orders.

Why Ably?

Ably is a serverless pub/sub solution, making use of real-timerealtime protocols such as WebSockets to provide seamless communication between devices. With a strong focus on reliability, exactly-once message delivery, and message ordering consistency, Ably is a powerful match with Fauna.

Why Ably + Fauna?

Both Ably and Fauna are serverless solutions, taking away the complexities of hosting and managing your own messaging and database solutions, respectively. Both are designed to be massively scalable at what they do. With Fauna allowing for scalable and fast data querying and storage, Ably is the perfect match for interfacing and distributing events to clients. Fauna’s Event Streaming allows for real-time events to be communicated to Ably for distribution, allowing for minimum latencies in updates, utilizing the strengths of both services.

Creating our e-commerce site

The first step is to create our Fauna and Ably accounts.

Setup Fauna Account

Firstly, you will need a Fauna account to create a new Database. Populate it with the demo data by ticking the 'Use demo data' button during this process. Also, ensure the region is 'Classic (C).'

If you choose to have the database in a specific region, you'll need to state this region in the .env file at the base of this directory. This will be either eu or us as the FAUNA_REGION value.

Once you have a database, create an API key for it with admin privilege by going to the 'Security' tab in the sidebar and selecting '+ New key' in the new tab.

Keep this key somewhere as we will be using it later for interacting with Fauna.

Finally, we need to create a new Index in Fauna, which we can use to ensure we don't get customers with duplicate usernames. Within the Fauna database's Shell tab, run the following query:

CreateIndex({
  name: "users_by_username",
  permissions: { read: "public"},
  source: Collection("users"),
  terms: [{field: ["data", "username"]}],
  unique: true,
})

Setup Ably Account

Next, we need to set up an Ably account. Sign up for an Ably account if you don't have one, then go to the Ably App you intend to use for this.

Within the app, go to the API key tab, and copy the Default API key's value. Save this API key for later.

Finally, we need to create some Channel Rules, which will allow for the last message on a channel to be persisted for up to a year. This is useful for ensuring that data is always accessible to our customers. Within the Ably App, go to the 'Settings' tab and the 'Channel Rules' section. Select 'Add new Rule', set the “Namespace' to be 'app,” and tick the “Persist last message” box. Finally, click 'Create channel rule.’

Creating a server

With our Fauna and Ably accounts created, we now need to create a server, which we will use to host our website, handle authentication with Ably, and act as a means of consuming from Fauna to re-distribute to Ably.

We’ll be using Node.js for this, so ensure your system has it installed.

Once you have it, create a new folder, and run npm init to create a new Node.js project. You can use all the default values, except set the entry point as server.js.

With the project created, let’s install the npm modules we’ll use.

npm install ably faunadb express cookie-parser

The Ably and Fauna packages will allow us to interact with those systems, respectively, express.js allows us to host a server easily, and cookie-parser will make it easier for our server to check our customer’s cookies.

Now, create a file called server.js. Within this, add the following:

/* Start the Express.js web server */
const express = require('express'),
      app = express(),
      cookieParser = require('cookie-parser');

app.use(cookieParser());
app.use(express.static(__dirname + '/public'));

app.listen(process.env.PORT || 3000);

Here we’ve created our express.js server, which should, by default, run on port 3000. We’ve declared that it’ll make a folder called public available via this port. If you run npm run start at the base of your project, you should be able to try and access the server. As we’ve not currently got anything in a public folder to return, it’ll fail to get anything. Let’s rectify that by creating the skeleton for our home page.

Creating the main page

This page will be how users will browse items and buy them.

Create a new folder called public, and within it, add index.html. Add the following code to it:

<html>
<head>
  <link href='https://fonts.googleapis.com/css?family=Actor' rel='stylesheet'>
  <link rel="stylesheet" type="text/css" href="/css/index.css">
</head>

<body>
  <header>
    <a class="home" href="/">
      <h1>💸 Buy Things Inc</h1>
    </a>
    <nav>
      <a href="/orders" class="orders">Your orders</a>

      <form class="login-form" action="/login" id="panel-anonymous">
        <input type="text" name="username" placeholder="Enter your username" class="login-text">
        <input type="submit" value="Login" class="login-submit">
      </form>

      <div id="panel-logged-in">
        You are logged in. <a href="/logout">Log out</a>
      </div>
    </nav>
  </header>

  <ul id="products" class="items"></ul>

  <main class="item">
    <div id="loader" class="loader" style="display: none"></div>
    <p id="intro-section">Select a product from the sidebar to see details</p>
    <div id="item-details" class="item-details" style="display: none">
      <img id="item-image" src="http://placekitten.com/600/400" class="item-img">
      <div class="item-info">
        <h2 id="title" class="item-title"></h2>
        <h3 id="description" class="item-description"></h3>
        <p class="item-price">£<span id="price"></span></p>
        <button onclick="order()" id="buy-button" class="buy">Buy now</button>
        <p class="item-stock">Stock: <span id="quantity"></span></p>
        <p class="item-stock" id="login-button-hint">Login to add items to your cart</p>
      </div>
    </div>
  </main>
  <footer>
    <p>Made with <a href="https://www.ably.com">Ably</a> and <a href="https://www.fauna.com">Fauna</a></p>
  </footer>
</body>
</html>

This page will be bland, so let’s create a css file for it. In the public folder, add another folder called css, and in that, add index.css. In that file, add the contents of this css file from GitHub.

With that done, we have a basic website made. However, we’re lacking any products to display at this stage. Within our Fauna database, we have all the products created for us automatically as part of the test data we instantiated it with, so we need to get that data available to our clients.

Adding Ably to the webpage

To do this, we’ll set up the clients to use Ably, and consume from Ably Channels. We’ll need the clients to know what products exist and specific product details when a customer tries to view a product.

To accomplish this, we can have one main channel called products, which will contain the most recent list of product IDs and names, and then sub-channels in the pattern of product:1, product:2, etc. This will mean any client will only ever need to be subscribed to two channels at any time, the overall list of products and the currently viewed product.

As we’ve set up Ably to have the last message in a channel persisted within the app namespace, we’ll want all channels we use to exist within that namespace. This means we just need to prepend all the channel names with app, so app:products and app:product:1.

We’ll need to include the Ably library on the page to use Ably. In the head, add the following line:

<script src="https://cdn.ably.io/lib/ably.min-1.js" type="text/javascript"></script>

Next, add the following code just below the body:

<script type="text/javascript">
  let productId;

  // Connect to Ably
  const realtime = new Ably.Realtime({ authUrl: '/auth' });
  // Instantiate the products channel, specifying rewind=1 so that we will as part of our subscription get the last state on the channel
  const productsChannel = realtime.channels.get('app:products', { params: { rewind: '1' } });

  let products;
  productsChannel.subscribe((msg) => {
    const productsContainer = document.getElementById('products');
    productsContainer.innerHTML = '';
    products = msg.data;
    for (let productID in products) {
      let activeText = '';
      if (productID == productId) activeText = 'active';
      let item = document.createElement("li");
      item.innerHTML = `<span id="products-${productID}" class="${activeText}" onclick="loadProduct('${productID}')">${products[productID]}</span>`;
      productsContainer.appendChild(item);
    }
  });

  let loggedIn = document.cookie.indexOf('username') >= 0;
  document.getElementById('panel-anonymous').setAttribute('style', "display: " + (loggedIn ? 'none' : 'flex'));
  document.getElementById('panel-logged-in').setAttribute('style', "display: " + (loggedIn ? 'inline' : 'none'));
  document.getElementById('buy-button').setAttribute('style', "display: " + (loggedIn ? 'flex' : 'none'));
  document.getElementById('login-button-hint').setAttribute('style', "display: " + (loggedIn ? 'none' : 'block'));
</script>

Here we instantiate Ably, specifying that we intend to authenticate via a url (more on this in a moment). We then declare a new Ably Channel object, which will connect to the channel app:products. We also specify in its parameters that it should use rewind=1, a feature of Ably that allows for the last message on the channel to be sent to a subscription seamlessly with any new messages and updates that may come through.

With the channel declared, we then subscribe to any new messages which will be published on it. We take those messages and extract the data we want to display in the sidebar.

Authentication with Ably

If you were to load the page at this stage, you should see the error Ably: Auth.requestToken(): token request signing call returned error. This is as we’ve not implemented an authentication method for Ably yet.

For Ably, two main authentication methods are available: API keys and Tokens. API keys are the keys you can find on your app page on the Ably site.

They will exist until you choose to delete them and are useful for trusted devices such as a server for accessing Ably. They’re not great for non-trusted clients like customers; however, for that reason, there’s no way to control who’s using them.

For non-trusted clients, tokens are ideal. These are short-lived, can be revoked easily, and generally, you’d have a service selectively generate them with limited permissions.

Here, for the customer’s browser, we’ll be using tokens. Within the Ably declaration, you’ll see we used authUrl: ‘/auth’. This states that the client library will attempt to get a token from that url (so localhost:3000/auth in our case). As we’ve yet to define an auth endpoint for our server, it’s coming back with nothing.

Let’s create this endpoint now. Firstly, we’ll need to define an Ably API key for our server to use to generate tokens. Create a .env file in the project's base, and add ABLY_API_KEY=your api key, replacing the ‘your api key’ bit with the API key you got during the Setup Ably Account section.

Next, let’s start using Ably on the server. Add the following to the top of the server.js file:

require('dotenv').config();
const Ably = require("ably");
const realtime = new Ably.Realtime(process.env.ABLY_API_KEY);

With Ably declared, let’s use it to create an auth endpoint, which will return a token. We’ll want there to be two states of users, logged-in users who can make orders and anonymous users who just want to browse. We can create different tokens to only provide the needed permissions for both cases. Add the following to the bottom of the server.js file:

app.get('/auth', function (req, res) {
  var tokenParams;
  if (req.cookies.username) {
    tokenParams = {
      'capability': {
        'app:product:*': ['subscribe'],
        'app:products': ['subscribe'],
        'app:submit_order': ['publish'],
       },
      'clientId': req.cookies.username
    };
  } else {
    /* Issue a token request with only subscribe privileges for products */
    tokenParams = {
      'capability': { 'app:product:*': ['subscribe'], 'app:products': ['subscribe'] }
    };
  }

  console.log("Sending signed token request:", JSON.stringify(tokenParams));
  realtime.auth.createTokenRequest(tokenParams, function(err, tokenRequest) {
    if (err) {
      res.status(500).send('Error requesting token: ' + JSON.stringify(err));
    } else {
      res.setHeader('Content-Type', 'application/json');
      res.send(JSON.stringify(tokenRequest));
    }
  });
});

Here we’re just using the customer’s username, stored in a cookie, to identify them. This is not a secure way to authenticate a user by any means, but for this tutorial with suffice. If you were ever to create a project like this for real, you would want to implement a robust authentication system to validate who a user is.

Now, if you run everything again with npm run start, you should be able to load up localhost:3000 and not see any errors in the console! However, our sidebar is still empty as we’re yet to put any data into Ably.

We can send some data via a curl to check everything is working. Within a terminal, send the following message, replacing API_KEY with your Ably API key:

curl -X POST https://rest.ably.io/channels/app%3Aproducts/messages -u "API_KEY" -H "Content-Type: application/json" --data '{ "name": "products", "data": {"101": "Pineapple" } }'

You should see this instantly reflected in the sidebar with the product ‘Pineapple’ appearing.

With that done, let’s now let users log in and out so they can be given either an anonymous user token or a logged-in user token. Add the following functions to the server.js file:

app.get('/login', async function (req, res) {
  if (req.query['username']) {
    res.cookie('username', req.query['username']);
    res.redirect('back');
  } else {
    res.status(500).send('Username is required to login');
  }
});

/* Clear the cookie when the user logs outs */
app.get('/logout', function (req, res) {
  res.clearCookie('username');
  res.redirect('/');
});

If you run the server again, and try to login in the top right with a username, you should hopefully see from the server logs Sending signed token request: {"capability":{"app:product:*":["subscribe"],"app:products":["subscribe"],"app:submit_order":["publish"]},"clientId":"Tom C"}.

Getting the initial product state from Fauna

Now that we have our products page connected to Ably, it’s time to get the data within Ably properly matched up to the data in Fauna. We’ll need to make use of two mechanics to do this:

We will need to do an initial check of what products exist with a query
We will need to check for the state of these objects and check for any changes to them over time

To get started on this, let’s create a folder in the base of our project called fauna, and add a file called faunaHandler.js. Here, let’s set up our Ably and Fauna objects and create an Ably Channel object for the products channel we’re using. We will also need to add a new environment variable to our .env file, FAUNADB_API_KEY. Set this to the Fauna API key you got during the Setup Fauna Account section.

If you’ve set up your Fauna database in a specific region, you will need to create a FAUNA_REGION environment variable, set to eu or us, respectively.

const faunadb = require('faunadb');
const Ably = require('ably');

const ablyClient = new Ably.Realtime(process.env.ABLY_API_KEY);
const productsChannel = ablyClient.channels.get('app:products');

/* Setup Fauna client */
const q = faunadb.query;
let domainRegion = "";
if (process.env.FAUNA_REGION) domainRegion = `${process.env.FAUNA_REGION}.`;
client = new faunadb.Client({
    secret: process.env.FAUNADB_API_KEY,
    domain: `db.${domainRegion}fauna.com`,  
    scheme: 'https',
});

With these made, we can create our query to Fauna for all the Products. As we’ll be wanting both the product IDs and names, we’ll need to query for all products, and then get all their values:

/* Product functions */
let allProductsQuery = q.Map(
    q.Paginate(q.Documents(q.Collection('products'))),
    q.Lambda(x => q.Get(x))
);

With the query made, let’s execute it, and publish the output into Ably:

const allProducts = {};
client.query(allProductsQuery)
.then((products) => {
    for (const product of products.data) {
        const id = product.ref.value.id;
        allProducts[id] = product.data.name;
    ablyClient.channels.get(`app:product:${id}`).publish('update', product.data);
    }
    productsChannel.publish('products', allProducts);
})
.catch((err) => console.error(
    'Error: [%s] %s',
    err.name,
    err.message
));

This code should allow us to initially populate the Ably app:products channel with an object of product IDs and their names. It will also post the specific product’s details into a channel for that product, matching the pattern app:product:${productID}. Let’s require our faunaHandler file in the server.js file and then run everything again. Add the following to the top of the server.js file, just below the require(‘dotenv’).require() line:

require('./fauna/faunaHandler.js');

Start the server with npm run start, and go to localhost:3000. You should see the products from your Fauna database in the sidebar!

Realtime product updates

With the above code, we can get the initial state of our products. However, we’re not currently able to see when something changes to the products, such as a new one being added or an existing product being removed. We could query Fauna every X seconds to see if there have been any changes compared to the previous state and then publish an updated object to Ably if there have been any changes.

We can avoid unnecessarily polling when there have been no changes and avoid delays in changes being reflected by clients by using Fauna’s Event Streaming. This allows our server to be sent updates on specific Product and Collection changes whenever they occur.

We can use the Set Streaming option to see whenever a document in a collection or index is created or removed. We could stream the Collection of Product documents, however, we’d only receive creation and removal events, which means we wouldn’t see when a specific Product changes, say in quantity.

Instead, we can create an Index that will check for changes in our Product timestamps. This will mean the Index will emit a delete event when the ‘old’ document is deleted and then a ‘create’ event when the document is created again with the new timestamp. This will allow us to receive updates whenever our products are updated.

To create this Index, go to your Fauna Dashboard, and within the database you made for this project, go to the ‘Shell’ tab on the left side of the screen. Within the shell, input the following:

CreateIndex({
  name: "products_by_ts",
  source: Collection("products"),
  values: [{ field: "ts" }]
})

Run the command, and a new Index will have been created called products_by_ts. This may take a few minutes to populate but should eventually be visible within the Indexes tab.

Now we’ve created this Index, we can try listening to it for changes. Add the following to your faunaHandler.js file:

function listenForProductChanges () {
    const ref = q.Match(q.Index("products_by_ts"));
    let stream = client.stream(ref)
    .on('set', (set) => {
        let productId = set.document.ref.value.id;
        if (set.action == 'add') {
            client.query(
                q.Get(q.Ref(q.Collection('products'), `${productId}`))
            )
            .then((product) => {
                allProducts[productId] = product.data.name;
                productsChannel.publish('products', allProducts);
                ablyClient.channels.get(`app:product:${productId}`).publish('update', product.data);
            });
        } else {
            delete allProducts[productId];
            productsChannel.publish('products', allProducts);
        }
    })
    .on('error', (error) => {
        console.log('Error:', error);
        stream.close();
        setTimeout(() => { listenForProductChanges() }, 1000);
    })
    .start();
    return stream;
}
listenForProductChanges();

With this, we have a stream that listens for any changes, be it adding, removing, or changing Products. When a change occurs, we get an update as a set event and can react accordingly.

We can try this out at this stage to see if changes in Fauna are reflected in the browser while the server is running. Run npm run start, and open up localhost:3000. Within the Fauna Dashboard for the database, try deleting or adding a new Product to the collection, and it should be updated within the browser immediately. The same should for updating an existing Product’s name.

Displaying the product details in the browser

At this stage, when the server is run, each product should now have an event stream feeding into its own Ably channel, for example, app:product:201. We now need to handle this from the HTML page to subscribe to these channels when we select a product in the sidebar and render the results on the page.

Within the index.html file, add the following code to the bottom of the <script> we’ve been making:

  let productChannel;
  // Load a new product to display from Ably, and subscribe to any changes
  function loadProduct(productID) {
    document.getElementById("intro-section").style.display = "none";
    document.getElementById("loader").style.display = "flex";
    document.getElementById("item-details").style.display = "none";
    if (productChannel != null) {
      productChannel.detach();
    }
    productId = productID;

    // Update the url to the new item, so if we refresh the page this item loads again
    if (products) {
      window.history.pushState('page2', products[productID], '?product=' + productID);
      const activeSidebarItem = document.getElementsByClassName('active');
      for (let i=0; i < activeSidebarItem.length; i++) {
        activeSidebarItem[i].className = activeSidebarItem[0].className.replace("active", "");
      }
      const newActiveItem = document.getElementById(`products-${productID}`);
      newActiveItem.className += "active";
    }

    productChannel = realtime.channels.get(`app:product:${productID}`, { params: { rewind: '1' } });
    productChannel.subscribe((msg) => {
      document.getElementById("loader").style.display = "none";
      document.getElementById("item-details").style.display = "flex";
      document.getElementById("title").textContent = msg.data.name;
      document.getElementById("description").textContent = msg.data.description;
      document.getElementById("quantity").textContent = msg.data.quantity;
      document.getElementById("price").textContent = msg.data.price;
      if (msg.data.image) {
        document.getElementById("item-image").src = msg.data.image;
      } else {
        document.getElementById("item-image").src = 'http://placekitten.com/600/400';
      }
    });
  }

  // If there is a param for the product to load in the URL, load it
  const urlParams = new URLSearchParams(window.location.search);
  if (urlParams.get('product')) {
    let productId = urlParams.get('product');
    loadProduct(productId);
  }

Here we’ve added the addProduct function, which is called when an item in the sidebar is clicked on. This’ll connect us to the product’s channel, and the last message on the channel will be used to render the product. It’ll remain subscribed to the product’s channel, meaning it’ll keep updating the UI with any changes to the product.

To make the page a bit more user-friendly for sharing specific products with others, we are changing a product parameter in the URL, and if there’s a product parameter when loading the page, it’ll load up the product straight away.

Finally, an image field can be included in your products stored in Fauna, which isn’t set by default. This should be a URL to an image that is hosted somewhere. A cute cat picture will be shown instead if the image field isn’t set for a product.

Purchasing items

Users can now see what products are available and view specific product details, all able to update in real-time. However, logged-in users are not yet able to buy the items they’re viewing.

Within your Fauna database, assuming you initialized it with test data, there should be a Function called submit_order. This function will take an order from a customer and checks that each item they want has enough stock remaining to allow for the purchase to happen. It will subtract the order from each item’s stock and create a new order collection if there is enough stock.

We can make a call to using the Fauna client library, but we will want to make sure only a client who is logged in as a specific customer can make orders for themself. To do this, we will need to:

Create a Customer document in Fauna for each account that signs in
Identify what users are requesting to buy products, to then request submit_order

Customer link to logged-in users

Let’s first link up Customers in Fauna with logged-in users. In the Setup Fauna Account section, we already made it so that Customer documents cannot have duplicate username fields, so we can trust that in creating new Customers, we won’t be able to have customers with duplicate usernames.

Due to this, we can create a new customer document with the user’s username when a client tries to place an order, and if it says a customer already exists with that username, we can use their ID in our request to make an order. Add the following to the end of the faunaHandler.js file:

/* Customer functions */
async function createOrFindCustomer(username) {
    /* We're hard-coding the address and other details, but for an actual
     solution we'd get these details from the user */
    let newCustomerObject = {
        "username": username,
        "address": {
            "street": "0 Fake Street",
            "city": "Washington",
            "state": "DC",
            "zipCode": "00000"
        },
        "telephone": "208-346-0715",
        "creditCard": {
            "network": "Visa",
            "number": "000000000000"
        }
    };
    // Try to create a new customer
    return await new Promise(r => {
        createP = client.query(
            q.Create(q.Collection('customers'), { data: newCustomerObject })
        ).then(function(response) {
            r(response.ref.value.id);
        })
        .catch(async () => {
            // If an existing user with this username exists, throws an error due to
            // uniqueness requirement defined within Fauna. We can find it with our username.
            let customerId = await getCustomerIdByUsername(username);
            r(customerId);
        });
    });
}

async function getCustomerIdByUsername(username) {
    // Uniqueness requirement should mean there's only 1 user with the username
    return await new Promise(r=> {
        client.query(
            q.Get(q.Match(q.Index('customers_by_username'), username))
        ).then(function(response) {
            r(response.ref.value.id);
        }).catch(function(err) {
            console.log(err);
        });
    });
}

In this, we’re making of the Fauna library to try and create a new Customer using a passed-in username. If that fails, due to having a customer already with this username, we instead find that customer by the username and return their ID.

Now, we can use this to convert usernames to user IDs. When we create Ably Tokens for clients in the Authentication with Ably section, we assign them a clientId, which can be used to identify users. As this is assigned by the server to them, we can trust that users with this attached to them in Ably are those users.

We can make use of this clientId to thus identify which user is making an order request and thus attach their customerId to the order we submit to Fauna. Let’s make it so that this clientId is the customerId. First, let’s export the username function so our server can use it. Add the following to the very bottom of the faunaHandler.js file:

module.exports = createOrFindCustomer;

Next, in the server.js file, replace the currently existing require(‘./fauna/faunaHandler.js); line with:

const createOrFindCustomer = require('./fauna/faunaHandler.js');

Where the clientId is assigned in the auth token generation in the server.js file, instead set it to be the following:

app.get('/auth', async function (req, res) {
  let tokenParams;
  const customerId = await createOrFindCustomer(req.query['username']);
  if (req.cookies.username) {
    tokenParams = {
      'capability': {
        'app:product:*': ['subscribe'],
        'app:products': ['subscribe'],
        'app:submit_order': ['publish'],
       },
      'clientId': customerId
    };
  } else {
    /* Issue a token request with only subscribe privileges for products */
    tokenParams = {
      'capability': { 'app:product:*': ['subscribe'], 'app:products': ['subscribe'] }
    };
  }
…

With each Ably client having their Customer ID attached to their connection, we can now listen to any messages they may try to send for orders. In the faunaHandler.js file, add the following to the bottom, just above the module.exports… we added:

const ordersChannel = ablyClient.channels.get('app:submit_order');

/* Listen for new order requests from clients */
ordersChannel.subscribe((msg) => {
    submitOrder(msg.clientId, msg.data);
});

const publicOrderConversion = (({ customer, cart, status, creationDate, deliveryAddress }) =>
        ({
            customer, cart, status, creationDate, deliveryAddress
        }));

async function submitOrder(customerId, orders) {
    client.query(
      q.Call('submit_order', customerId, orders)
    )
    .then((ret) => {
        const publicOrder = publicOrderConversion(ret.data);

        let userId = ret.data.customer.value.id;
        let orderId = ret.ref.value.id;
        ablyClient.channels.get(`app:order:${userId}:${orderId}`).publish('order', publicOrder);

        if (!allOrders[customerId]) allOrders[customerId] = [];
        allOrders[customerId].push(orderId);
        ablyClient.channels.get(`app:orders:${customerId}`).publish(`order`, allOrders[customerId]);
    })
    .catch((err) => {
        console.log(err);
        console.error(
      'Error: [%s] %s: %s',
      err.name,
      err.message,
      err.errors()[0].description,
    )}
    );
}

Note: In this, we’ve also added a publish to an orders channel in Ably, which will be mentioned in more detail in the 'Viewing orders' section.

With this, we’re subscribed to the order details sent by clients through the app:submit_order Ably Channel. The data of the order will be sent to the submitOrder function, which takes the user’s client ID (which is their customer ID), and submits the order to Fauna.

We now just need to make it so that the ‘Buy now’ button on our index.html file functions. Add the following to the index.html file’s script:

  const orderChannel = realtime.channels.get('app:submit_order');

  function order() {
    // customer Id, order
    orderChannel.publish("order", [
      {
        "productId": productId,
        "quantity": 1
      }
    ]);
  }

The ‘Buy now’ button should now be fully functional! Start up the server again with npm run start, load up localhost:3000, and try going to a product and buying it. If it works you should see the stock go down for the item in the browser, and get a console log in your terminal with the details of the order placed!

Viewing orders

Now that customers can place orders, it makes sense for us to add the ability for customers to view their orders. We should be able to do this using the same mechanisms as before:
We can have a sidebar with a list of a customer’s orders
The customer will be able to click on any order in the sidebar to view its details

The main difference between what we did for products and this is that the orders should only be accessible to the specific user who placed them. To ensure this, we can break down orders in Ably Channels that match up to specific customerIDs.

For example, the customer ‘101’ should have the list of all their orders placed for the sidebar in app:orders:101. Likewise, the details of each order will be in a channel of the structure (for order ID 20) app:order:101:20. When creating Ably Tokens for users, we can only provide them access to channels that match the pattern app:orders:101 and app:order:101:*, ensuring they only get access to their orders.

Client order viewing functionality

Let’s add these permissions to the existing Ably Token generator in the server.js file. Change the token part of the auth endpoint to be the following:

  let tokenParams;
  if (req.cookies.username) {
    const customerId = await createOrFindCustomer(req.cookies['username']);
    const ordersPattern = `app:orders:${customerId}`;
    const orderPattern = `app:order:${customerId}:*`;
    tokenParams = {
      'capability': {
        'app:product:*': ['subscribe'],
        'app:products': ['subscribe'],
        'app:submit_order': ['publish'],
       },
      'clientId': customerId
    };
    tokenParams.capability[orderPattern] = ['subscribe'];
    tokenParams.capability[ordersPattern] = ['subscribe'];
  } else {
    /* Issue a token request with only subscribe privileges for products */
    tokenParams = {
      'capability': { 'app:product:*': ['subscribe'], 'app:products': ['subscribe'] }
    };
  }
…

Now that customers, once signed in, will have access to the order channels matching their customer ID, we need to use it from the customer side. Let’s add new endpoint customers can use to view their orders, /orders. Add the following to the server.js file:

app.get('/orders', (req, res) => {
  res.sendFile(__dirname + '/public/orders.html');
});

Next, in the public folder, add a file called orders.html. In it, add the following code:

<html>

<head>
  <script src="https://cdn.ably.io/lib/ably.min-1.js" type="text/javascript"></script>
  <link href='https://fonts.googleapis.com/css?family=Actor' rel='stylesheet'>
  <link rel="stylesheet" type="text/css" href="/css/index.css">
</head>

<body>
  <header>
    <a class="home" href="/">
      <h1>💸 Buy Things Inc</h1>
    </a>
    <nav>
      <a class="orders" href="/orders">Your orders</a>

      <form class="login-form" action="/login" id="panel-anonymous">
        <input type="text" name="username" placeholder="Enter your username" class="login-text">
        <input type="submit" value="Login" class="login-submit">
      </form>

      <div id="panel-logged-in">
        You are logged in. <a href="/logout">Log out</a>
      </div>
    </nav>
  </header>

  <ul id="orders" class="items"></ul>

  <main class="item">
    <div id="loader" class="loader" style="display: none"></div>
    <p id="intro-section">Select an order from the sidebar to see details</p>
    <div id="item-details" class="item-details" style="display: none">
      <div class="item-info">
        <h2 class="item-title">Order <span id="title"></span></h2>
        <h3>Time of purchase: <span id="date"></span></h3>
        <h3>Delivery Address</h3>
        <p id="delivery-address"></p>
        <h3>Ordered items</h3>
        <ul id="cart"></ul>
        <p>Total price: £<span id="total-price"></span></p>
      </div>
    </div>
  </main>
  <footer>
    <p>Made with <a href="https://www.ably.com">Ably</a> and <a href="https://www.fauna.com">Fauna</a></p>
  </footer>
</body>

<script type="text/javascript">
  let orderId;
  let orderChannel;
  let orders;
  let customerId;

  const realtime = new Ably.Realtime({ authUrl: '/auth' });
  realtime.connection.on('connected', () => {
    customerId = realtime.auth.clientId;
    console.log(`app:orders:${customerId}`);
    const ordersChannel = realtime.channels.get(`app:orders:${customerId}`, { params: { rewind: '1' } });
    ordersChannel.subscribe((msg) => {
      const ordersContainer = document.getElementById('orders');
      ordersContainer.innerHTML = '';
      orders = msg.data;
      for (let orderID of orders) {
        let activeText = '';
        if (orderId == orderID) activeText = 'active';
        let item = document.createElement("li");
        item.innerHTML = `<span id="orders-${orderID}" class="${activeText}" onclick="loadOrder('${orderID}')">${orderID}</span>`;
        ordersContainer.appendChild(item);
      }
    });
  });

  function loadOrder(orderID) {
    document.getElementById("intro-section").style.display = "none";
    document.getElementById("loader").style.display = "flex";
    document.getElementById("item-details").style.display = "none";
    if (orderChannel != null) {
      orderChannel.detach();
    }
    orderId = orderID;

    if (orders) {
      const activeSidebarItem = document.getElementsByClassName('active');
      for (let i=0; i < activeSidebarItem.length; i++) {
        activeSidebarItem[i].className = activeSidebarItem[0].className.replace("active", "");
      }
      const newActiveItem = document.getElementById(`orders-${orderID}`);
      newActiveItem.className += "active";
    }

    orderChannel = realtime.channels.get(`app:order:${customerId}:${orderId}`, { params: { rewind: '1' } });
    orderChannel.subscribe((msg) => {
      document.getElementById("loader").style.display = "none";
      document.getElementById("item-details").style.display = "flex";
      document.getElementById("title").textContent = orderId;
      document.getElementById("date").textContent = new Date(msg.data.creationDate['@ts']).toLocaleString();

      const city = msg.data.deliveryAddress.city;
      const state = msg.data.deliveryAddress.state;
      const street = msg.data.deliveryAddress.street;
      const zipCode = msg.data.deliveryAddress.zipCode;
      document.getElementById("delivery-address").textContent = `${street} \n ${city} \n ${state} \n ${zipCode}`;
      let list = document.getElementById('cart');
      list.innerHTML = "";
      const cart = msg.data.cart;
      let totalPrice = 0;
      for (let i = 0; i < cart.length; i++) {
        let item = cart[i];
        let entry = document.createElement('li');
        totalPrice += item.price;
        let productName = item.name || item.product['@ref'].id;
        entry.appendChild(document.createTextNode(`product: ${productName} - price: ${item.price} - quantity: ${item.quantity}`));
        list.appendChild(entry);
      }
      document.getElementById("total-price").textContent = totalPrice;
    });
  }

  /* Hide or show the logged in / anonymous panels based on the session cookie */
  let loggedIn = document.cookie.indexOf('username') >= 0;
  document.getElementById('panel-anonymous').setAttribute('style', "display: " + (loggedIn ? 'none' : 'flex'));
  document.getElementById('panel-logged-in').setAttribute('style', "display: " + (loggedIn ? 'inline' : 'none'));
</script>

</html>

In terms of functionality, this is effectively the same as the index.html code. We populate the sidebar based on the contents of the channel app:orders:CUSTOMERID. Then, when an order is selected, subscribe to the channel app:order:CUSTOMERID:ORDERID and populate the fields in the right part of the page.

Fauna to Ably order functionality

With that setup, we just have to get the data into those Ably channels. Again, this work will be very similar to the existing work we made for mapping Fauna to Ably for Products. We will also need to consider the attached customer ID on the order when constructing the Ably Channel name.

In the faunaHandler.js file, let’s do the same actions as before for the product updates. We’ll do an initial fetch to see what orders exist and then update Ably channels with the current states of the list of orders per customer ID and each order. Add the following to the faunaHandler.js file just above the module.exports bit:

/* Order functions */
const allOrders = {};

let allOrdersQuery = q.Map(
    q.Paginate(q.Documents(q.Collection('orders'))),
    q.Lambda(x => q.Get(x))
);
client.query(allOrdersQuery)
.then((orders) => {
    for (const order of orders.data) {
        const orderId = order.ref.value.id;
        const customerId = order.data.customer.id;
        if (!allOrders[customerId]) allOrders[customerId] = [];
        allOrders[customerId].push(orderId);
        const publicOrder = publicOrderConversion(order.data);
        ablyClient.channels.get(`app:order:${customerId}:${orderId}`).publish('update', publicOrder);
    }
    for (const [customerId, orders] of Object.entries(allOrders)) {
        ablyClient.channels.get(`app:orders:${customerId}`).publish(`order`, orders);
    }
})
.catch((err) => console.error(
    'Error: [%s] %s',
    err.name,
    err.message
));

Try running the server again with npm run start, load up localhost:3000, and try buying a few items. Click the orders button in the nav bar, and hopefully you should see all your orders populated in the sidebar!

Conclusion

We now have a website where users can log in, browse products, see realtime updates on new products being added and changed, and purchase them. Customers can then view orders they’ve made, also reflecting in realtime the exact history of each user’s purchases and being entirely secure only to allow said user to access their orders.

There’s so much more that can be done with a site like this, but hopefully, with the core building blocks developed here, most realtime features can be developed. As seen by the repetition between the product and order updates, many of the same patterns can be used to seamlessly replicate and interact between clients and systems such as Fauna for the user.

You can find all the code from this blog on GitHub. If you have any Ably questions, you can reach out to them on their Discord, or you can reach out to the Fauna team on Discord or here.

Building Multiplayer Games with Unity

Tom Camp — Thu, 21 Jul 2022 16:41:00 +0000

Some of the funnest games are those you can play with your friends. Being able to explore worlds, defeat monsters, and overcome puzzles and challenges together can be amazing experiences.

Creating multiplayer functionality in games can be a massive challenge however. Creating architectures that allow for reliable communication of clients, cheat detection, handling of networking issues and more requires a lot of thought and planning.

In this blog, we'll be exploring a lot of the considerations required in multiplayer gaming and demonstrating some of the best ways to make things work smoothly and easily.

Introduction

In order to best demonstrate multiplayer functionality in Unity, rather than creating an entire game from the ground up, I got in touch with raywenderlich to see if I could extend their Tower Defense tutorial series with multiplayer functionality. It’s an incredibly detailed and informative tutorial of a tower defense game, and I strongly recommend checking it out to fully understand the context of this blog post.

With this core, single-player game, we have a solid basis for adding in multiplayer functionality. Core features and considerations we'll cover in this blog are:

How to communicate placement and upgrading of turrets
How to keep clients in sync with an authoritative source
How to handle clients falling out of sync with one another, be it due to latencies, computer issues, and more
How to handle re-connection of clients to a game

We'll be using Unity to develop in, and Ably to implement reliable, scalable communication between devices.

Getting started

Firstly, you’ll need to have Unity installed. The best way to do this is to get the Unity Hub from the Unity website, and then from that install the most recent version of Unity. This was originally created for 2020.3.28f1.

With Unity in hand, download our base project from GitHub on the starting-point branch, then load it up in Unity. Before starting anything, you’ll want to adjust the dimensions of the game window so that it renders correctly. In the main viewer click the ‘Game’ tab at the top, and then change the aspect ratio to 4:3.

Creating a Lobby

One key thing is that as this game will be multiplayer, we will need a way for players to uniquely identify instances of the game to join, play, and watch. To do this, we’ll create a Scene prior to the actual game in which players can input a unique ID for a game they’d like to join, and then click a button to load the game.

In the base template, there’s already a Scene added in Assets/Scenes called Lobby. This is a simple Scene with a text input to allow for a player to type the game they wish to join (as long as all players input the same ID they’ll enter the same game), and a button to submit this.

Currently, this button is non-functional however. What we want is for it to run a script, which will load the actual game scene, and convey the unique game’s ID inputted here for the game to use.

Persisting the Game State

To handle the game ID persistence we can create a new C# script which we’ll call StateManagerBehaviour. In this script place the following code:

using UnityEngine;

public class StateManagerBehaviour : MonoBehaviour
{
    public static StateManagerBehaviour Instance;
    public string GameID;

    void Awake()
    {
        if (Instance == null)
        {
            Instance = this;
            DontDestroyOnLoad(gameObject);
        }
        else if (Instance != null)
        {
            Destroy(this.gameObject);
        }
    }
}

Here we have a script that will stop whatever it is attached to from being destroyed between Scenes. This means that any variables set, such as the current game’s ID, can be placed here and persisted.

Within the Lobby Scene, create an Empty GameObject and call it StateManager. Attach the StateManagerBehaviour script to it. This object and the script attached will now persist between scenes!

Changing scenes

Now that we can persist a game ID, we need to make the submit button functional. The button will need to:

Update the value of GameID in the StateManagerBehaviour to match what is put into the input box
Load the game’s scene

To do this, create a new script attached to the button called JoinGame, and add the following code to it:

using UnityEngine;
using UnityEngine.SceneManagement;
using UnityEngine.UI;
using TMPro;

public class JoinGame : MonoBehaviour
{
    [SerializeField]
    private Button startButton;

    [SerializeField]
    public TMP_InputField gameIDField;

    void Start()
    {
        startButton.onClick.AddListener(() => {
            if (gameIDField.text == "") return;
            StateManagerBehaviour.Instance.GameID = gameIDField.text;
            SceneManager.LoadScene("GameScene");
        });
    }
}

Select the JoinGameButton in the Hierarchy, and drag and drop the InputField into the Game ID Field of the JoinGame script, as well as the JoinGameButton itself into the Start Button field.

The button should now be functional! This should now mean we can load up the game scene from this Lobby scene when we build the project!

Before testing it however we’ll need to check that both scenes are added to our build settings. Go to File -> Build Settings…, and add both scenes to the Scenes in Build section.

With that, you can try running the Lobby scene, and if all is well you should be able to input a value for the game ID, and hit the button to load in the Tower Defense game!

Adding in multiplayer capabilities

With the lobby added, we can start looking at how to include multiplayer functionality. As the core user interaction is the placement of monsters to defeat the hordes of enemies, we’ll look at communicating that first between clients.

For our communication layer, we’ll be using Ably. Ably’s a realtime pub/sub platform, which is an incredibly powerful paradigm for any form of fan-in or fan-out messaging. From the game, each client will send messages to one of Ably’s servers, and other clients in the game who have subscribed for updates will receive these messages.

To use Ably you’ll need to sign up for a free account. Once you have the account, you can get an API key from your account dashboard, which will allow clients to make use of Ably programmatically. For this section, we’ll be using the API key within our code just to get going, but you should eventually replace it with Token Authentication, which ensures users only ever have access to short-lived tokens which you have full control over in terms of permissions granted.

Getting the Ably Unity Package

With an account created and an API key in hand, we now need to get one of the client libraries Ably provides in order to easily interact with the system. Ably provides a Unity client library as part of their C# library. If you go to the Ably C# GitHub repository, you can download the most recent version of the Unity Package from the releases.

At the time of writing, the most up-to-date version is 1.2.8 and can be downloaded here.

With the unitypackage in hand, we need to import it into our Unity project. In the top menus, select Assets -> Import Package -> Custom Package…, and select the unitypackage you just downloaded. In the options, untick the JsonNet items. If you forget to at this step, just delete the folder that’ll appear in the UnityPackages directory.

If you get errors about the JsonNet being the wrong version, go to Edit - Project Settings. In here go to the Player tab, and then under ‘Configuration’, ‘Assembly Version Validation’ untick it.

Sharing turret upgrades between clients

With the Ably library added, we can now use it to share the addition of monsters between clients. As we’ll be using Ably in multiple scripts, let’s create an object to hold a single instance of the Ably client library for us. In the GameScene scene, add an empty game object called AblyManager, and create a script to attach to it called AblyManagerBehaviour. Within this script, add the following:

using IO.Ably;
using IO.Ably.Realtime;
using UnityEngine;

public class AblyManagerBehaviour : MonoBehaviour
{
    private AblyRealtime realtime = new AblyRealtime(
        new ClientOptions { Key = "INSERT_ABLY_API_KEY_HERE" }
    );
    public IRealtimeChannel gameChannel;

    // Start is called before the first frame update
    void Awake()
    {
        gameChannel = realtime.Channels.Get(StateManagerBehaviour.Instance.GameID);
    }
}

Make sure to replace the INSERT_ABLY_API_KEY_HERE text with the Ably API key you got earlier from your account dashboard.

Here, we are connecting to Ably with the API key to authenticate us and then instantiating an Ably Channel. Ably Channels are the means by which clients can indicate the clients they wish to communicate with over Ably. They’re uniquely identified by a name, in this case, specified as StateManagerBehaviour.Instance.GameID.

We’re using the state object we specified earlier to ensure that only clients who wish to participate in a certain game, identified by the GameID, will be using the Ably Channel.

Sending and receiving messages

The Ably instance we’ve instantiated will be able to be used by our other scripts to communicate between clients. Open up the PlaceMonster script, and add in the following to the top of the class:

private AblyManagerBehaviour ablyManager;

We can instantiate this reference to the AblyManager within the Start function:

ablyManager = GameObject.Find("AblyManager").GetComponent<AblyManagerBehaviour>();

With this reference to the AblyManager, we can now make use of its Ably Channel. What we need is to send a message whenever a player clicks to add or upgrade a monster, so that clients will be able to subscribe and represent these actions themselves.

Publish player input

To do this, we’ll need to change up the process of adding a monster. Instead of adding it as part of a click, we will want the click to send a message. Replace the contents of the onClick function with the following:

void OnMouseUp()
{
    if (CanPlaceMonster())
    {
        ablyManager.gameChannel.Publish("spot:" + name, “0”);
    }
    else if (CanUpgradeMonster())
    {
        MonsterData monsterData = monster.GetComponent<MonsterData>();
        Debug.Log(monsterData.levels.IndexOf(monsterData.getNextLevel()));
        ablyManager.gameChannel.Publish("spot:" + name, monsterData.levels.IndexOf(monsterData.getNextLevel()).ToString());
    }
}

Here we are checking on the clientside if we can actually perform the action we’re attempting, in order to avoid sending unnecessary messages. Assuming we can, we send a message to the Ably Channel, with a name field corresponding to the spot we’re interacting with, and a data field containing the level we’re wanting to set the monster to. The name field is useful as the Ably Client library can filter incoming messages by name, which will make it easier to only handle messages intended for each monster in each monster’s script.

Subscribe to player input

With the actions being sent to the Ably Channel, we also need to have our clients subscribe to receive these messages. To hold messages as they arrived, we’ll be using a queue.

The main reason for doing this over just handling messages as they come in is due to the fact the Ably Subscribe functionality will make use of a thread to actively listen for messages. This means that when a message is received, the handling of that message would occur on this new thread rather than the main thread, which means we won’t have access to any of Unity’s code paths or components from it. By using a queue, we can handle the messages we receive from the main thread at a later time from the Update function.

Let’s add a queue at the top of the class:

private Queue actions = new Queue();

Now we have a queue to hold our messages in, let’s add a subscribe function to listen to the Ably Channel within the Start function, just after the ablyManager allocation:

ablyManager.gameChannel.Subscribe("spot:" + name, message =>
{
    // Need to uniquely identify actions to avoid unintentional upgrade when trying to place, etc.
    actions.Enqueue(message.Data);
});

Here we’re subscribing to the Ably Channel, and only acting on messages with the name corresponding to “spot:” + name. This means we’ll only be running adding messages to the queue if they’re intended for this monster with this monster’s name.

Now that we should be receiving messages, we need to act upon them. Within the Update function we can start to go through the queue:

// Update is called once per frame
void Update()
{
    if (actions.Count == 0) return;

    PlaceOrUpgradeMonster(int.Parse((string) actions.Dequeue()));
}

Here we’re checking if the queue has any actions to be performed, and if so we’ll be calling a new function, PlaceOrUpgradeMonster, passing the level of monster that’s been requested for the location. This is important due to the fact we may have multiple clients requesting to upgrade the same tile at the same time, so we want to ensure that the intended level of upgrade is considered to avoid unintentional further upgrades to a tile.

For the contents of PlaceOrUpgradeMonster, we can just slightly adjust the old contents of OnMouseUp, taking into consideration the intended level of upgrade and if it’s still possible or not.

private void PlaceOrUpgradeMonster(int index)
{
    if (CanPlaceMonster() && index == 0)
    {
        monster = (GameObject)Instantiate(monsterPrefab, transform.position, Quaternion.identity);
        AudioSource audioSource = gameObject.GetComponent<AudioSource>();
        audioSource.PlayOneShot(audioSource.clip);

        gameManager.Gold -= monster.GetComponent<MonsterData>().CurrentLevel.cost;
    }
    else if (CanUpgradeMonster())
    {
        MonsterData monsterData = monster.GetComponent<MonsterData>();
        if (monsterData.getNextLevel() == monsterData.levels[index])
        {
            monster.GetComponent<MonsterData>().increaseLevel();
            AudioSource audioSource = gameObject.GetComponent<AudioSource>();
            audioSource.PlayOneShot(audioSource.clip);

            gameManager.Gold -= monster.GetComponent<MonsterData>().CurrentLevel.cost;
        }
    }
}

This is a great moment to run the game and see if it works! Hopefully with a single client the game should work much the same as before, where a click on a tile places a monster. If you have two clients running however using the same game ID, the actions of each client should now (hopefully) affect the other client!

Synchronizing clients

One useful feature of Ably is that all messages in a channel are guaranteed to be ordered the same way for all subscribers, which means that you can be certain that all clients will receive the same messages in the same order, ensuring you won’t have any desynchronization due to that.

However, there are still various other synchronization issues that we need to address:

How do we ensure that all clients start the game at the same time
How do we handle latency, which can result in messages being received at different times for different clients

Synchronizing start time

Ensuring that all the players start at the same time is a fairly easy one to handle; we can add a button to the UI which players can use to start the game. This will send a message to all clients, indicating to start the game. Each message sent through Ably will also have a timestamp attached to it, which can be used to ensure all clients are in agreement as to when a game should have started.

Add a button to the UI of the game, and call it ‘StartButton’. Edit the text element attached to it to have the text ‘Start!’.

As this will be an aspect game’s flow, add the following to the top of the AblyManagerBehaviour script:

public bool started = false;
public Button startButton;
public void StartGame()
{
    startButton.enabled = false;
    started = true;
}

Within the Inspector of the button we created, select the AblyManager within the On Click section of the Inspector. From there, go to the AblyManagerBehaviour dropdown for the function dropdown, and select the StartGame function. The button should now on click set paused to false.

We will now need to check if the game has started within the SpawnEnemy script. Firstly, add the Within the SpawnEnemy script’s Update function, add the following to the top:

if (!ablyManager.started)
{
    // This doesn't account for any time previously passed
    lastSpawnTime = Time.time;
    return;
}

If the game hasn’t started yet, we simply keep the lastSpawnTime set to the current time to avoid enemies eventually instantly spawning, and then skip trying to place any enemies.

We also need to instantiate the ablyManager within SpawnEnemy, so at the top add the following line:

private AblyManagerBehaviour ablyManager;

Next, replace the contents of Start:

void Start()
{
    lastSpawnTime = Time.time;
    gameManager = GameObject.Find("GameManager").GetComponent<GameManagerBehaviour>();
    ablyManager = GameObject.Find("AblyManager").GetComponent<AblyManagerBehaviour>();
}

If you run the game now, locally the game should only start once the player clicks the start button. We still need to share this start action with all other players however.

To do this, let’s create a function within the AblyManagerBehaviour class which will send a message when the button is clicked, and another function which will call the StartGame function once a client receives the message:

public void SendStartGame()
{
    gameChannel.Publish("start", "");
}

Within the Awake function in the AblyManagerBehaviour we can then add a subscribe function to listen for this message, much as we did in the PlaceMonster class:

gameChannel.Subscribe("start", (msg) =>
{
    StartGame();
});

If you now change the StartButton’s On Click Behaviour to use this new SendStartGame function, you should hopefully now have the same Behaviour as you had previously locally, but all clients should now react to any client pressing the button.

Ensuring synchronization is maintained

Whilst the above indicator to start is quite easy to set up, ensuring that all clients remain synchronized for the entire game is a much harder challenge. There are a few base scenarios you need to consider:

What if a client lags/freezes, resulting in it falling behind in terms of gameplay?
What if a message is delayed in reaching some clients, to the point it’ll result in different outcomes between clients?

Both these scenarios can result in different outcomes in a game between clients, which isn’t something that’s acceptable. There are a few ways to overcome these issues.

Full state communication

Imagine you have a central server which is responsible for running the ‘true’ version of the game, and is trusted as the defacto truth of what happens. Whenever something happens on it, it can communicate the new state of the game, such as where enemies are, where monsters are, and the health of both the cookie and the enemies.

If a client were to fall behind, it can then simply check what the most recent version of the state is that was communicated, and adjust the local representation of the game to match.

There will be small periods of time where clients can fall out of sync, but the moment the current state of the game is communicated again, all clients can quickly adjust to the current state and resync. This means that even if there’s latencies in communication resulting in some ‘sync’ states reaching clients out of sync, they’ll simply be out of sync for a small period, but this won’t impact what they can do in the game, nor the final outcome all clients will see, as that’s determined and communicated by the server.

This is however fairly heavy on the communication side of things in comparison to our current approach of only sending messages for players actually interacting with the game.

Fast-forward and delayed inputs

One of the simplest things we can do to ensure a fairly reliable synchronized state is a combination of 2 techniques.

Firstly, if a client falls behind in terms of actions due to their machine freezing momentarily say, it can continue to simulate the game as normal up until the point it receives messages which are actions from other players. These actions will have a timestamp attached to them, indicating when the message was received by Ably. If we compare the timestamps of these messages to the timestamp attached to initial ‘start’ message, we can see how much time has passed from our defacto source of truth, the Ably servers, which is shared by all clients.

If we’re tracking how many times the Update function has been called since the start of the game, we can interpret how far behind the local game is behind the ‘true’ state of the game. If we then loop through the ticks of the game rapidly up until we’re back to the ‘true’ state, applying the updates received by messages at the appropriate times, we should then have our delayed client back in sync.

To handle the likelihood of a message arriving with a delay for some users, we can handle this for the vast majority of cases by simply introducing a delay in the game itself on placing monsters down and them becoming operational (aka able to fire). If we make this say 1 second, then it doesn’t matter if a monster’s placement message is delayed by any time up to 1 second in reaching a client, as all the other clients will already be waiting for 1 second from the timestamp attached to the message before allowing the monster to shoot, so they’ll all start interacting with enemies at the same time.

This isn’t perfect, and if you wanted to ensure for any amount of delay that clients can maintain synchronization you’d need to implement some form of rewind mechanism to play back through actions from the last ‘synced’ point in time, storing the exact state of the game at these moments locally to play on from implementing ‘delayed’ actions at the correct times to ensure they’ve happened when they should happen. For this blog though, we’ll be going with the above implementation.

Allowing clients to catch up with the current state

The easiest way to allow for clients which have fallen behind the current state to catch up is to make use of the Time.timeScale variable that’s part of Unity, which effects how often Update and FixedUpdate run. We can scale this up when we need a client to catch up to quickly run through events until the client has caught up with the current state, and then returning back to normal speed.

To detect if a client has fallen behind though, we’ll need to start tracking how many ticks have occurred since a game has started locally, and also when according to Ably the game started. Add the following to the AblyManagerBehaviour class to start tracking the number of ticks since starting:

public int ticksSinceStart = 0;

void FixedUpdate()
{
    if (started)
    {
        ticksSinceStart++;
    }
}

With that added, we also can start tracking the start time according to Ably by checking the timestamp attached to the ‘start’ message all the clients receive. Change the StartGame and Awake functions in AblyManagerBehaviour to be the following:

public DateTimeOffset? startTimeAbly;

// Start is called before the first frame update
void Awake()
{
    gameChannel = realtime.Channels.Get(StateManagerBehaviour.Instance.GameID);
    gameChannel.Subscribe("start", (msg) =>
    {
        StartGame(msg.Timestamp);
    });
}

public void StartGame(DateTimeOffset? timestamp)
{
    if (!started) startTimeAbly = timestamp;
    started = true;
}

With this startTimeAbly should contain the timestamp the game started at.

Now that the elapsed ticks are being tracked since the start of the game, in addition to the time according to Ably the game started, we need to start using the timestamps of messages which come in from Ably to the client to determine if our current game state is correct for the elapsed time.

The first thing we need for that is to start passing the timestamp from messages from Ably that come in to the PlaceMonster class to the queue we consume from. Let’s create an object within the PlaceMonster file to represent both the existing level of monster to upgrade to, in addition to the timestamp:

public class PlaceMonsterData
{
    public int monsterID;
    public DateTimeOffset? timestamp;
}

Let’s replace the subscribe call in the Start function in PlaceMonster to use the new structure:

ablyManager.gameChannel.Subscribe("spot:" + name, message =>
{
    PlaceMonsterData pmd = new PlaceMonsterData();
    pmd.monsterID = int.Parse((string)message.Data);
    pmd.timestamp = message.Timestamp;
    actions.Enqueue(pmd);
});

Next, change the Update function to make use of the new data structure:

void Update()
{
    if (actions.Count == 0) return;
    PlaceMonsterData pmd = (PlaceMonsterData) actions.Peek();
    DateTimeOffset? startTime = ablyManager.startTimeAbly;
    DateTimeOffset? msgTime = pmd.timestamp;
    TimeSpan? diffTime = msgTime - startTime;
    int ticksSince = ablyManager.ticksSinceStart;
    float timeFromTicks = ticksSince * (1000 * Time.fixedDeltaTime);

    if (!diffTime.HasValue)
    {
        PlaceOrUpgradeMonster(pmd.monsterID);
        return;
    }

    if (timeFromTicks < diffTime.Value.TotalMilliseconds)
    {
        Time.timeScale = 20;
        return;
    }
    else
    {
        Time.timeScale = 1;
        actions.Dequeue();
        PlaceOrUpgradeMonster(pmd.monsterID);
    }
}

Let’s break down what we’re doing here. We’ll check the start time and current tick from the AblyManagerBehaviour class, and compare that to the current next element in the queue. We make use of Peek to check the next element as we may not want to consume it if we’re out of sync in this current update loop.

We work out the amount of time that’s passed since the start of the game by comparing the start timestamp and the action’s timestamp and storing it in diffTime.

We can then work out how much effective time has elapsed since the start of the game by counting the number of ticks that’ve occurred, and multiplying it by the amount of time each tick should occur (by default once every 20ms). This value is held in Time.fixedDeltaTime, so we can just use that to calculate timeFromTicks.

We can then compare the actual time elapsed (diffTime) and the local game time elapsed (timeFromTicks). If the game time is behind, we can speed up the rate of the game using Time.timeScale. Here it’s set to 20, but you can fiddle with it to match whatever feels best to you.

If we reach a time elapsed in which the game has caught up with the true ‘state’, we can finally enact on the message by placing a monster, and also ensuring that the timeScale is returned to 1.

You should now be able to test this out by running the game. Start in the Lobby scene, enter a room code, and hit the start button. You should find that at first the enemies should move at a normal pace, and monster placement works the same as usual.

If you perform an action which causes the game to pause however, such as say tasking to another application, when you return you should see the game progressing as normal up until you try to place a new monster. The game should then pick up pace to catch up with the true state the game should be in!

This works for ensuring we become eventually consistent with the main state, but is dependent on a new action occurring to bring the client back to the real present state. As the current only way to invoke an action is with a monster placement, this sync could happen right away, or not at all before the end of the game. An easy fix for this would be to just introduce a fairly regular ping from a client to ensure all the clients are consistently receiving timestamps from Ably to sync on.

Delaying monster placement

We’ve handled the scenario of a client falling behind, but we’ve yet to address messages being delayed, and thus say a monster not existing for one client when it does for another, resulting in a desync of damage done to enemies.

Ideally we’d have a foolproof way to handle this scenario, such as by making use of checkpoints in the game state locally to rewind to moments which can cause a desync, such as a monster being placed, and replaying the game quickly from that point forward.

In this demo however, we’re going to do something far simpler, but which will work for the vast majority of scenarios; delaying the placement of a monster from when the action occurs. That is to say that, if we receive a message to place a monster with timestamp ‘1000’, we wouldn’t place the turret in the game until say ‘2000’ in the local game. That means that so long as all the clients receive this message in the 1 second period between the true occurrence of the message and any of them actually enacting on it, they should all perform the action at the exact same time in the game.

To do this, we can adjust the shooting logic to not shoot until a second after the timestamp associated with the upgrade/placement. This should from the player’s perspective make the actual placement of monsters still feel smooth and instantaneous, with the actual ‘delay’ in action being hidden by a delay in a monster firing.

First, we’ll need to get the timestamp added to the ShootEnemies class. Add the following variables to the top of the class:

private AblyManagerBehaviour ablyManager;
public DateTimeOffset? Timestamp;

Next, add a reference to the AblyManagerBehaviour in the Start function:

ablyManager = gameObject.GetComponentInChildren<AblyManagerBehaviour>();

Finally, add a consideration for the timestamp to the start of the Update function:

if (timestamp.HasValue)
{
    DateTimeOffset? startTime = ablyManager.startTimeAbly;
    DateTimeOffset? msgTime = timestamp.GetValueOrDefault();
    TimeSpan? diffTime = msgTime - startTime;
    int ticksSince = ablyManager.ticksSinceStart;
    float timeFromTicks = ticksSince * (1000 * Time.fixedDeltaTime);
    if (timeFromTicks >= diffTime.Value.TotalMilliseconds + 1000)
    {
        timestamp = null;
    }
    else
    {
        return;
    }
}

With that added, we need a way to pass the timestamp to a monster’s ShootEnemies class. Update the PlaceOrUpgradeMonster function in PlaceMonster to be the following:

private void PlaceOrUpgradeMonster(int monsterLevel, DateTimeOffset? timestamp)
{
    if (CanPlaceMonster() && monsterLevel == 0)
    {
        monster = (GameObject)Instantiate(monsterPrefab, transform.position, Quaternion.identity);
        monster.GetComponent<ShootEnemies>().timestamp = timestamp;
        AudioSource audioSource = gameObject.GetComponent<AudioSource>();
        audioSource.PlayOneShot(audioSource.clip);

        gameManager.Gold -= monster.GetComponent<MonsterData>().CurrentLevel.cost;
    }
    else if (CanUpgradeMonster())
    {
        MonsterData monsterData = monster.GetComponent<MonsterData>();
        if (monsterData.getNextLevel() == monsterData.levels[monsterLevel])
        {
            monster.GetComponent<MonsterData>().increaseLevel();
            monster.GetComponent<ShootEnemies>().timestamp = timestamp;
            AudioSource audioSource = gameObject.GetComponent<AudioSource>();
            audioSource.PlayOneShot(audioSource.clip);

            gameManager.Gold -= monster.GetComponent<MonsterData>().CurrentLevel.cost;
        }
    }
}

With this we are allowing for the timestamp to be passed from the Update function, and we then use it to assign the timestamp value in the ShootEnemies class.

Finally, change the Update function in PlaceMonster class to pass a the timestamp to the PlaceOrUpgradeMonster function:

…
if (!diffTime.HasValue)
{
    PlaceOrUpgradeMonster(pmd.monsterID, msgTime);
    return;
}

if (timeFromTicks < diffTime.Value.TotalMilliseconds)
{
    Time.timeScale = 20;
    return;
}
else
{
    Time.timeScale = 1;
    actions.Dequeue();
    PlaceOrUpgradeMonster(pmd.monsterID, msgTime);
}

With that, we should now have decent handling for clients both falling behind the current state, in addition to clients receiving delayed messages.

Conclusion

We have a multiplayer Tower Defence game which can handle network and client instability. There’s a load of additional features, improvements, and fixes that can still be applied at this stage, but it’s a strong starting point to take further and experiment with. A few things I’d recommend trying out are:

Making use of Token Authentication for clients, to keep authentication details away from untrusted users
Having a central server to further coordinate users, and perform additional validation, provide scoreboards, and more
Extending the synchronization work to support rewinding the state of a game to a previously known state
Giving clients names so players can uniquely identify themselves
Providing players their own money to buy monsters with rather than a shared bank

There is a version of the game available from the browser if you’d like to give it a try.

If you create anything based on this work, I’d love to hear about it in the comments, on Twitter, or even directly via email.

Reliable WebSockets-based pub/sub with Spring Boot

Tom Camp — Fri, 09 Apr 2021 13:18:12 +0000

Spring Boot is a powerful application framework for creating production-grade Spring based Applications. It allows for developers to very quickly and easily get sites up and running, with databases and more.

In this tutorial, we'll be going over WebSockets, Pub/Sub, and how to make the most out of these in your Spring Boot applications.

What are WebSockets?

Although traditional REST-based communication can be enough for many use-cases, sometimes it's not quite enough to provide realtime experiences to your users. Think of chat applications; new updates are constantly occurring, clients are going online and offline, and all of this is expected to be indicated to all the clients as quickly as possible. If we were to rely on the client to make a request for these updates, it'd need to be requesting extremely frequently to maintain a realtime experience, and in doing so consume a lot of resources and bandwidth.

This is where WebSockets come in. WebSockets allow for bi-directional communication between the clients and the server. Once a client established a connection to your server, the server is free to send updates to the clients whenever an update is available. This ends up being far more efficient, and also ensures updates are being sent to clients as soon as possible.

What is Pub/Sub?

Publish/Subscribe (Pub/Sub) is a paradigm in which you decouple clients and servers from one another. Rather than a server having to keep track of which clients to send messages to, you instead have clients subscribe to entities known as 'channels', and the server (or clients, depending on your use case!) publish messages to that channel. Some middleware or server is then responsible for routing messages published to a channel to all clients who've subscribed to it.

By introducing this middleware, you suddenly don't have to worry about maintaining state on your Spring Boot server, and are able to have inter-client communication. By separating your publishing server from this central pub/sub server(s), you can easily extend communication to be inter-client, handle multiple servers publishing to clients, and more.

Implementing these concepts

With these concepts in mind, in this blog we're going to look at creating a simple collaborative TODO list. Anyone will be able to add new TODOs, remove them, and mark them completed/incomplete. These events will be communicated over WebSockets, and make use of Ably to handle our Pub/Sub. We'll make use of H2 to store our data, and wrap it up in a Docker container to make it easily deployable.

Getting started

Firstly, let's set up a basic Spring Boot application. We can use the Spring Initializr with Spring Data JPA, H2 Database, Lombok added. H2 Database will provide us with a simple database, and Spring Data JPA will allow us to easily interact with it using Hibernate. Lombok will make it easier to write concise and readable classes.

Generate the base Spring Application using the Spring Initializr and store the output in your local workspace. This application will contain the dependencies we specified, as well as a basic file called DemoApplication.java. This is the entry point to our application. If you want to try running this to make sure it works, just run ./mvnw spring-boot:run. It should successfully build and run, although as we've not added anything yet it won't do anything.

Creating a database

Now we have our template ready, let's first set up the database. Create a new folder in your Java folder called todoitem. We'll be putting out database classes into it.

Firstly, we'll define what a 'todoitem' will look like. Create inside the todoitem folder 'TodoItem.java', and add the following:

@Entity
@Data
@NoArgsConstructor
@RequiredArgsConstructor

public class TodoItem
{
    @Id
    @GeneratedValue
    private Long id;

    @NonNull
    private String text;

    @NonNull
    private String username;

    private boolean completed = false;
}

To break this down:

@Entity -- This indicates that this class is a JPA entity
@data -- This is a Lombok annotation that automatically generates getters and more for each value
@RequiredArgsConstructor -- This is another Lombok annotation that automatically generates a constructor which expects values for all variables which aren't assigned and need a value. In this case, both text and username are marked @NonNull, and so the generated constructor will require both those fields.
@NoArgsConstructor -- Like the above annotation, except it creates a constructor which requires no arguments

The id is simply a unique identifier for the todo item, which is automatically generated and identified as the id for JPA due to the @id and @GeneratedValue annotations.

Text and username represent the todo item's content and who made the todo item respectively.

Completed simply represents whether the todo item has been completed or not.

With that small bit of code, we've defined our todoitem database item. We now need to generate a JPA Repository to interact with the database. Thankfully JPA makes this easy by providing JpaRepository, which we can extend. This will provide all the interfaces we need to add, remove, and change TodoItems in our database.

Create a new file in todoitem called TodoItemRepository, and add the following code:

@Repository
public interface TodoItemRepository extends JpaRepository<TodoItem, Long> {

}

That's all we need, all methods we'll need to interact with the database are automatically generated for us!

Creating a Service for our Database

Although we could now simply make direct usage of this class for making database changes, usually you'd also create a Service class to handle interactions with our repositories. Let's create TodoITemService.java in todoitem, and add the following to it:

@Service
public class TodoItemService {
    @Autowired
    private TodoItemRepository todoItemRepository;

    public List<TodoItem> findAll() {
        return todoItemRepository.findAll();
    }

    public Optional<TodoItem> find(Long id) {
        return todoItemRepository.findById(id);
    }

    public TodoItem create(String text, String username) {
        TodoItem copy = new TodoItem(text, username);
        return todoItemRepository.save(copy);
    }

    public Optional<TodoItem> updateCompletionStatus(Long id, boolean status) {
        return todoItemRepository.findById(id)
        .map(oldItem -> {
        oldItem.setCompleted(status);
        return todoItemRepository.save(oldItem);
        });
    }

    public void delete(Long id) {
        todoItemRepository.deleteById(id);
    }
}

Within this we're autowiring the TodoItemRepository we created, and then providing some functions to interact with the repository.

That's it, we now have everything we need for handling our representation of our todo list's items!

Creating a REST Controller

With our database made, let's look at how we can open up interacting with it to the world. To do this, we'll be creating a REST Controller, which effectively opens up endpoints which clients can GET/POST/PUT/DELETE to. We'll then perform actions on our database depending on the request and endpoint.

Add TodoItemController.java to our todoitem folder, and start off by adding the following:

@RestController
@RequestMapping("api")
public class TodoItemController {
    @Autowired
    private TodoItemService todoItemService;
}

Here we've defined this class a RestController with the annotation, and specified that all the mappings we're going to provide will fall under 'api'. For example, if we create a mapping for '/getAll' in this class, the endpoint will actually be '/api/getAll'.

We've also autowired TodoItemService in, as that's what we'll be using from our mappings to interact with our database.

Next, let's start creating our mappings. Add the following functions:

@GetMapping
public ResponseEntity<List<TodoItem>> findAll() {
    List<TodoItem> items = todoItemService.findAll();
    return ResponseEntity.ok().body(items);
}

@GetMapping("/{id}")
public ResponseEntity<TodoItem> find(@PathVariable("id") Long id) {
    Optional<TodoItem> item = todoItemService.find(id);
    return ResponseEntity.of(item);
}

@PostMapping
public ResponseEntity<TodoItem> create(@CookieValue(value = "username") String username, @RequestBody Map<String, String> json) {
    if (json.get("text") == null)
    {
        return ResponseEntity.badRequest().body(null);
    }
    TodoItem newItem = todoItemService.create(json.get("text"), username);
    URI location = ServletUriComponentsBuilder.fromCurrentRequest()
        .path("/{id}")
        .buildAndExpand(newItem.getId())
        .toUri();

    JsonUtils.JsonUtilsObject object = io.ably.lib.util.JsonUtils.object();
    object.add("text", json.get("text"));
    object.add("username", username);
    object.add("id", newItem.getId().toString());

    return ResponseEntity.created(location).body(newItem);
}

@PutMapping("/{id}/complete")
public ResponseEntity<TodoItem> completed(
    @PathVariable("id") Long id) {

    Optional<TodoItem> updated = todoItemService.updateCompletionStatus(id, true);

    return updated
        .map(value -> {
            return ResponseEntity.ok().body(value);
        })
        .orElseGet(() -> ResponseEntity.notFound().build());
}

@PutMapping("/{id}/uncomplete")
public ResponseEntity<TodoItem> uncompleted(
    @PathVariable("id") Long id) {

    Optional<TodoItem> updated = todoItemService.updateCompletionStatus(id, false);
    return updated
        .map(value -> {
            return ResponseEntity.ok().body(value);
        })
        .orElseGet(() -> ResponseEntity.notFound().build());
}

@DeleteMapping("/{id}")
public ResponseEntity<TodoItem> delete(
    @CookieValue(value = "username") String username,
    @PathVariable("id") Long id) {

    Optional<TodoItem> todoOptional = todoItemService.find(id);
    if (todoOptional.isEmpty()) {
        return ResponseEntity.noContent().build();
    }

    if (todoOptional.get().getUsername().equals(username)) {
        todoItemService.delete(id);
    }
    return ResponseEntity.noContent().build();
}

Here we're defining various endpoints for our application, and using our todoItemService to update our database.

We're also replying on a cookie we'll have on the client-side, username, to determine who the client is. This isn't providing any real authentication, but will work for now for acting as faux-authentication.

To test is this is working, let's start the server with ./mvnw spring-boot:run. The server should be running at localhost:8080.

Send the following curl to create a new todo item:

curl -X POST -H 'Content-Type: application/json' -d '{
"text": "Go to the shops!"
}' --cookie "username=Bob" http://localhost:8080/api -i

Then, we can retrieve our todo items with:

curl -X GET http://localhost:8080/api -i

You should get a response of:

[{"id":1,"text":"Make toast!","username":"Bob","completed":false}]

Feel free to try deleting todos and completing/uncompleting them.

Now in order to provide the username cookie to our clients, let's create a simple Controller called AuthController.js:

@RestController
public class AuthController
{
    /* Set a cookie when the user logs in */
    @RequestMapping(value = "/login", method = RequestMethod.GET)
    public String login(@RequestParam(name = "username", defaultValue = "anonymous") String username, HttpServletResponse response) throws IOException {
        /* Login the user without credentials. This is an over simplified authentication system to keep this tutorial simple */
        response.addCookie(new Cookie("username", username));
        response.sendRedirect("/");
        return "redirect:/";
    }

    /* Clear the cookie when the user logs outs */
    @RequestMapping(value = "/logout", method = RequestMethod.GET)
    public String logout(HttpServletRequest request, HttpServletResponse response) throws IOException {
        for (Cookie cookie : request.getCookies()) {
            if (cookie.getName().equalsIgnoreCase("username")) {
                cookie.setValue(null);
                cookie.setMaxAge(0);
                cookie.setPath(request.getContextPath());
                response.addCookie(cookie);
            }
        }
        response.sendRedirect("/");
        return "redirect:/";
    }
}

All this does it set and unset the username cookie depending on what the client is requesting.

Creating a basic webpage

Now that we've created an REST API to interact with our database, we can make a simple webpage which'll allow users to:

Choose a username
Create new TODOs
Marking existing TODOs complete/not complete
Able to delete only their own TODOs

We'll send any changes a client makes to the server to validate + action in the database. We will then have the server publish a message over Ably containing the change. The clients will be subscribed to these updates, and update their UI depending on messages they receive.

In order to dynamically generate the page when first serving it from the server to contain the existing todo items, we'll be using Thymeleaf. Add Thymeleaf to your dependencies in your pom.xml file:

<!--    Thymeleaf for dynamic html    -->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-thymeleaf</artifactId>
</dependency>

Next, create a new folder in your resources/templates folder called todolist, then add a file called index.html to it. Fill it with:

<html>
<head>
    <meta content="text/html;charset=utf-8" http-equiv="Content-Type">
    <meta content="utf-8">
    <link rel="stylesheet" href="/index.css">
    <script src="https://cdn.ably.io/lib/ably.min-1.js" type="text/javascript"></script>
</head>
<body>
<div class="topnav">
    <div id="panel-anonymous" class="login-container">
        <form action="/login">
            <input type="text" name="username" placeholder="Username">
            <button type="submit">Login</button>
        </form>
    </div>
    <div id="panel-logged-in" class="login-container">
        <p><a class="active" href="/logout">Logout</a></p>
    </div>
</div>

<div id="myTODOS" class="header">
    <div id="panel-anonymous-msg" class="login-container">
        <h2>Sign in to create your own TODOs</h2>
    </div>
    <div id="panel-logged-in-msg" class="login-container">
        <h2>Todo List</h2>
        <input type="text" id="message" name="message" />
        <div class="row">
            <input id="publish" class="addBtn" type="submit" value="Add new TODO!">
        </div>
    </div>
</div>

<ul class="row" id="todo-list">
    <li th:each="todoitem: ${todoitems}"
        th:inline="text"
        th:id="${todoitem.id}"
        th:class="${todoitem.completed} ? 'checked' : ''">
        [[${todoitem.text +  ' -- Added by ' + todoitem.username}]]
        <span class="close">×</span>
    </li>
</ul>
</body>
<script type="text/javascript" src="/auth.js"></script>
<script type="text/javascript" src="/ably.js"></script>
<script type="text/javascript" src="/todolist.js"></script>
</html>

This page has a reference to the Ably client library which we'll be using for implementing subscriptions over WebSockets. It also contains a login box, as well as an input for new TODOs. The unordered list is using thymeleaf to populate itself with existing todo items which'll be provided to it by the server.

It also references 3 js files we're yet to create: auth.js, ably.js and todolist.js. Let's add them now to our resources/static folder.

The index.css file should be:

* {
box-sizing: border-box;
margin: 0;
}

.topnav {
overflow: hidden;
background-color: #e9e9e9;
}

.topnav a {
float: left;
display: block;
color: black;
text-align: center;
padding: 14px 16px;
text-decoration: none;
font-size: 17px;
}

/* Navbar links on mouse-over */
.topnav a:hover {
background-color: #ddd;
color: black;
}

/* Active/current link */
.topnav a.active {
background-color: #333333;
color: white;
}

/* Style the input container */
.topnav .login-container {
float: right;
}

/* Style the input field inside the navbar */
.topnav input[type=text] {
padding: 6px;
margin-top: 8px;
font-size: 17px;
border: none;
width: 150px; /* adjust as needed (as long as it doesn't break the topnav) */
}

/* Style the button inside the input container */
.topnav .login-container button {
float: right;
padding: 6px 10px;
margin-top: 8px;
margin-right: 16px;
background-color: #555;
color: white;
font-size: 17px;
border: none;
cursor: pointer;
}

.topnav .login-container button:hover {
background: #ccc;
}

/* Add responsiveness -- On small screens, display the navbar vertically instead of horizontally */
@media screen and (max-width: 600px) {
.topnav .login-container {
float: none;
}
.topnav a, .topnav input[type=text], .topnav .login-container button {
float: none;
display: block;
text-align: left;
width: 100%;
margin: 0;
padding: 14px;
}
.topnav input[type=text] {
border: 1px solid #ccc;
}
}

/* Remove margins and padding from the list */
ul {
margin: 0;
padding: 0;
}

/* Style the list items */
ul li {
cursor: pointer;
position: relative;
padding: 12px 8px 12px 40px;
background: #eee;
font-size: 18px;
transition: 0.2s;

/* make the list items unselectable */
-webkit-user-select: none;
-moz-user-select: none;
-ms-user-select: none;
user-select: none;
}

ul li:nth-child(odd) {
background: #f9f9f9;
}

ul li:hover {
background: #ddd;
}

ul li.checked {
background: #888;
color: #fff;
text-decoration: line-through;
}

ul li.checked::before {
content: '';
position: absolute;
border-color: #fff;
border-style: solid;
border-width: 0 2px 2px 0;
top: 10px;
left: 16px;
transform: rotate(45deg);
height: 15px;
width: 7px;
}

.close {
position: absolute;
right: 0;
top: 0;
padding: 12px 16px 12px 16px;
}

.close:hover {
background-color: #f44336;
color: white;
}

.header {
background-color: #ED760A;
padding: 30px 40px;
color: white;
text-align: center;
}

.header:after {
content: "";
display: table;
clear: both;
}

input {
margin: 0;
border: none;
border-radius: 0;
width: 75%;
padding: 10px;
float: left;
font-size: 16px;
}

/* Style the "Add" button */
.addBtn {
padding: 10px;
width: 25%;
background: #d9d9d9;
color: #555;
float: left;
text-align: center;
font-size: 16px;
cursor: pointer;
transition: 0.3s;
border-radius: 0;
}

.addBtn:hover {
background-color: #bbb;
}

The auth.js file contains our very simple login system, where if we have a cookie representing the username we're considered logged in and update the UI appropriately:

/* Change the display based on logged in status */
let loggedIn = document.cookie.indexOf('username') >= 0;
document.getElementById('panel-anonymous').
setAttribute('style', "display: " + (loggedIn ? 'none' : 'block'));
document.getElementById('panel-anonymous-msg').
setAttribute('style', "display: " + (loggedIn ? 'none' : 'block'));

document.getElementById('panel-logged-in').
setAttribute('style', "display: " + (loggedIn ? 'block' : 'none'));
document.getElementById('panel-logged-in-msg').
setAttribute('style', "display: " + (loggedIn ? 'block' : 'none'));

The todolist.js file contains our logic for sending requests to our server to change todo list items:

/* Create a new list item when clicking on the "Add" button */
function addNewTODO(id, todoText, username) {
    var ul = document.getElementById("todo-list");
    var li = document.createElement("li");

    var t = document.createTextNode(todoText + ' -- Added by: ' + username);
    li.appendChild(t);
    li.id = id;
    ul.appendChild(li);

    /* Create a close icon */
    var span = document.createElement("SPAN");
    var txt = document.createTextNode("\u00D7");
    span.className = "close";
    span.appendChild(txt);
    li.appendChild(span);

    span.onclick = () => {
        publishToSpring('DELETE', '/api/' + id);
    }
}

let closeButtons = document.getElementsByClassName('close');
for(let i = 0; i < closeButtons.length; i++) {
    let closeButton = closeButtons[i];
    closeButton.onclick = function() {
        publishToSpring('DELETE', '/api/' + closeButton.parentElement.getAttribute('id'));
    }
}

function removeTODO(id) {
    var div = document.getElementById(id);
    div.style.display = "none";
}

function completeTODO(id) {
    document.getElementById(id).classList.add("checked");
}

function incompleteTODO(id) {
    document.getElementById(id).classList.remove("checked");
}

/* Send new TODO to server */
document.getElementById('publish').onclick = sendMessageToSpring;

const node = document.getElementById("message");
node.addEventListener("keyup", ({key}) => {
    if (key === "Enter") {
        sendMessageToSpring();
    }
});

function sendMessageToSpring() {
    let username = document.cookie.indexOf('username');
    let todoText = document.getElementById("message");
    let message = {
        'text': todoText.value,
        'username': username
    }
    todoText.value = "";
    publishToSpring('POST', '/api', JSON.stringify(message));
}

/* Mark a list element as complete/incomplete */
var list = document.querySelector('ul');
list.addEventListener('click',
(ev) => {
    if (ev.target.tagName === 'LI') {
        clickEvent(ev.target.id);
    }
}, false);

function clickEvent(id) {
    if (document.getElementById(id).classList.contains("checked")) {
        publishToSpring('PUT', '/api/' + id + '/uncomplete');
    } else {
        publishToSpring('PUT', '/api/' + id + '/complete');
    }
}

function publishToSpring(action, url, data) {
    var xhr = new XMLHttpRequest();
    xhr.open(action, url, true);
    xhr.setRequestHeader("Content-Type", "application/json");
    xhr.send(data);
}

getCurrentTodos();

function getCurrentTodos() {
    let request = new XMLHttpRequest();
    request.open('GET', '/api');
    request.responseType = 'json';
    request.send();
}

Finally, our ably.js file contains our Pub/Sub logic. It establishes a connection to the Ably Pub/Sub service using a yet-to-be-implemented auth endpoint to retrieve a token, and then subscribes to the channel 'default' for any messages published. This channel is what we'll be publishing messages to from the server eventually. Depending on the message, we call the appropriate function in our todolist.js file:

/* Set up a Realtime client that authenticates with the local web server auth endpoint */
let realtime = new Ably.Realtime({ authUrl: '/auth' });
let channelName = 'default';
let channel = realtime.channels.get(channelName);

/* Subscribe to changes from an Ably channel */
channel.subscribe(function(message) {
    if (message.name === 'add') {
        addNewTODO(message.data.id, message.data.text, message.data.username);
    } else if (message.name === 'remove') {
        removeTODO(message.data);
    } else if (message.name === 'complete') {
        completeTODO(message.data);
    } else if (message.name === 'incomplete') {
        incompleteTODO(message.data);
    }
});

Great, we now have a web page which can display our existing TODOs in a list, and allows the user to submit requests to our server.

To have the server provide this webpage when the client tries to load localhost:8080/, let's create a new Controller, TodoListController.js:

@Controller
public class TodoListController {
    @Autowired
    private TodoItemService todoItemService;

    @GetMapping("/")
    public String todoPage(Model model) {
        List<TodoItem> listItems = todoItemService.findAll();
        if (listItems == null) {
            listItems = new ArrayList<>();
        }
        model.addAttribute("todoitems", listItems);

        return "todolist/index";
    }
}

Although we're yet to implement publishing of messages from our server through Ably so our clients can react and update their UIs, we should be able to still update the server through the webpage. Comment out the ably.js file, and start up the server with ./mvnw spring-boot:run. If you load up localhost:8080, you should see our webpage. Login in the top right panel, and then try sending a message or 2 through the submission box. If you then do curl -X GET http://localhost:8080/api -i in your terminal, you should see the TODOs you submitted with your username attached. Feel free to then try to complete, uncomplete, and delete TODOs and check to make sure the server's correctly reflected the change.

Implementing WebSocket based Pub/Sub for broadcasting events

Finally, we can start implementing our WebSocket based Pub/Sub solution so that clients are updated in realtime whenever a change is made to the server's database.

We'll be using Ably for our Pub/Sub messaging to simplify the implementation. It'll handle our connection recovery, ensure reliable message delivery, and in general do all of the hard work involved in creating a Pub/Sub distributed system.

To start implementing this, you'll need a free Ably account, and get your API key for one of your apps. Add this to your application.properties file:

ABLY_API_KEY: YOUR_API_KEY

Next, let's add Ably to our pom.xml, as well as gson for handling JSON objects:

<!--    Ably    -->
<dependency>
    <groupId>io.ably</groupId>
    <artifactId>ably-java</artifactId>
    <version>1.2.3</version>
</dependency>
<dependency>
    <groupId>com.google.code.gson</groupId>
    <artifactId>gson</artifactId>
    <version>2.8.6</version>
</dependency>

You may remember that when instantiating Ably in our ably.js file, we were attempting to call /auth on our server to obtain a token to authenticate with Ably. Now that we have access to an API key, we can create our auth endpoint. Add the following to the top of your AuthController class:

@RestController
public class AuthController {
    private AblyRest ablyRest;

    @Value( "${ABLY_API_KEY}" )
    private void setAblyRest(String apiKey) throws AblyException {
        ablyRest = new AblyRest(apiKey);
    }

    /* Issue token requests to clients sending a request to the /auth endpoint */
    @RequestMapping("/auth")
    public String auth(HttpServletRequest request, HttpServletResponse response) throws AblyException {
        String username = null;
        Cookie[] cookies = request.getCookies();
        if (cookies != null) {
            for (Cookie cookie : cookies) {
                if (cookie.getName().equalsIgnoreCase("username")) {
                    username = cookie.getValue();
                    break;
                }
            }
        }
        Auth.TokenParams tokenParams = getTokenParams(username);
        return createTokenRequest(tokenParams, response);
    }

    public Auth.TokenParams getTokenParams(String username) throws AblyException {
        Auth.TokenParams tokenParams = new Auth.TokenParams();
        tokenParams.capability = Capability.c14n("{ '*': ['subscribe'] }");
        if (username != null) {
            tokenParams.clientId = username;
        }
        return tokenParams;
    }

    public String createTokenRequest(Auth.TokenParams tokenParams, HttpServletResponse response) {
        Auth.TokenRequest tokenRequest;
        try {
            tokenRequest = ablyRest.auth.createTokenRequest(tokenParams, null);
            response.setHeader("Content-Type", "application/json");
            return tokenRequest.asJson();
        } catch (AblyException e) {
            response.setStatus(500);
            return "Error requesting token: " + e.getMessage();
        }
    }
...
}

This generates a token with permission to subscribe to updates, and passes it back to the client to use.

Finally, we can make use of Ably in our TodoItemController to send messages to our Ably channel whenever a change successfully is applied to our database. Replace the file with the following:

@RestController
@RequestMapping("api")
public class TodoItemController {

    @Autowired
    private TodoItemService todoItemService;

    private AblyRest ablyRest;

    @Value( "${ABLY_API_KEY}" )
    private void setAblyRest(String apiKey) throws AblyException {
        ablyRest = new AblyRest(apiKey);
    }

    private final String CHANNEL_NAME = "default";

    @GetMapping
    public ResponseEntity<List<TodoItem>> findAll() {
        List<TodoItem> items = todoItemService.findAll();
        return ResponseEntity.ok().body(items);
    }

    @GetMapping("/{id}")
    public ResponseEntity<TodoItem> find(@PathVariable("id") Long id) {
        Optional<TodoItem> item = todoItemService.find(id);
        return ResponseEntity.of(item);
    }

    @PostMapping
    public ResponseEntity<TodoItem> create(@CookieValue(value = "username") String username, @RequestBody Map<String, String> json) {
        if (json.get("text") == null) {
            return ResponseEntity.badRequest().body(null);
        }
        TodoItem newItem = todoItemService.create(json.get("text"), username);
        URI location = ServletUriComponentsBuilder.fromCurrentRequest()
            .path("/{id}")
            .buildAndExpand(newItem.getId())
            .toUri();

        JsonUtils.JsonUtilsObject object = io.ably.lib.util.JsonUtils.object();
        object.add("text", json.get("text"));
        object.add("username", username);
        object.add("id", newItem.getId().toString());

        publishToChannel("add", object.toJson());
        return ResponseEntity.created(location).body(newItem);
    }

    @PutMapping("/{id}/complete")
    public ResponseEntity<TodoItem> completed(
        @PathVariable("id") Long id) {

        Optional<TodoItem> updated = todoItemService.updateCompletionStatus(id, true);

        return updated
            .map(value -> {
                publishToChannel("complete", Long.toString(id));
                return ResponseEntity.ok().body(value);
            })
            .orElseGet(() -> ResponseEntity.notFound().build());
    }

    @PutMapping("/{id}/uncomplete")
    public ResponseEntity<TodoItem> uncompleted(
        @PathVariable("id") Long id) {

        Optional<TodoItem> updated = todoItemService.updateCompletionStatus(id, false);

        return updated
            .map(value -> {
                publishToChannel("incomplete", Long.toString(id));
                return ResponseEntity.ok().body(value);
            })
            .orElseGet(() -> ResponseEntity.notFound().build());
    }

    @DeleteMapping("/{id}")
    public ResponseEntity<TodoItem> delete(
        @CookieValue(value = "username") String username,
        @PathVariable("id") Long id) {

        Optional<TodoItem> todoOptional = todoItemService.find(id);
        if (todoOptional.isEmpty()) {
            return ResponseEntity.noContent().build();
        }

        if (todoOptional.get().getUsername().equals(username)) {
            todoItemService.delete(id);
            publishToChannel("remove", Long.toString(id));
        }
        return ResponseEntity.noContent().build();
    }

    /* Publish an update to the TODO list to an Ably Channel */
    private boolean publishToChannel(String name, Object data) {
        try {
            ablyRest.channels.get(CHANNEL_NAME).publish(name, data);
        } catch (AblyException err) {
            System.out.println(err.errorInfo);
            return false;
        }
        return true;
    }
}

Your application should now be fully functional! Make sure you've uncommented the ably.js file, and then run the application with ./mvnw spring-boot:run, and start using it from localhost:8080! You should find changes made from one client updates the display of another.

Dockerizing

Finally, let's dockerize it. Firstly, install Docker.

Next, package our application as a JAR:

./mvnw clean package

We'll reference this JAR when building our Docker Container.

Now create a Dockerfile in the base directory with the following content:

FROM adoptopenjdk:11-jre-hotspot
COPY ./target/*.jar /usr/app/application.jar
WORKDIR /usr/app
ENTRYPOINT ["java", "-jar", "application.jar"]

All our Dockerfile is doing is taking the JAR we compiled earlier, and defining it as our entrypoint for running this application.

We also want a docker-compose.yml file to define how the structure of our docker container a its settings:

version: '2'

services:
    app:
        image: 'spring-boot-ably:latest'
        build:
            context: .
        ports:
            -- "8080:8080"
        container_name: app
        environment:
            -- ABLY_API_KEY=INSERT_ABLY_API_KEY_HERE

\
You'll want to replace the INSERT_ABLY_API_KEY_HERE with the Ably API key you got earlier.

With this done, you can now run docker compose up, and you should see your basic application running with Docker!

Conclusion

With all this done, we now have our collaborative TODO list! By making use of WebSockets and the Pub/Sub paradigm we've made what can be quite a challenging issue, regular updates being transmitted to clients, into something fairly simple. With just the few lines we've included for handling our WebSockets, we've ensured distribution to all subscribers with no additional load on our server.

Ably itself also ensures ordering or messages, as well as guarantees on message delivery, ensuring that our clients don't miss messages or receive them in the wrong order.

In our next tutorial, we'll be implementing a robust authentication system using Auth0. You can find the source code to this tutorial on GitHub.

Forem: Tom Camp

Announcing the winners of the Ably Realtime Hackathon

Collaboration experiences

Realtime dashboards

React bonus category

Want to see all the hackathon submissions?

Create a realtime closed captioning solution with Deepgram and Ably

Overview

Technology

Setting Up the project environment

1. Initializing the project

2. Installing necessary packages

3. Creating a .env.local file

Setting up the transcription server

1. Creating the server directory and file:

2. Importing necessary modules:

3. Initializing Deepgram and Ably clients:

4. Setting Up Ably Channels:

5. Subscribing to Presence events:

6. Creating a Deepgram live transcription session:

7. Setting Up Deepgram event listeners:

8. Handling received transcripts:

9. Subscribing to client messages:

Running the server

Setting Up the client

1. Instantiating Ably React Hooks

2. Creating the Ably authentication endpoint

Updating App.tsx

3. Setting Up Ably:

4. Sending Audio to Ably

5. Stopping recording

6. Subscribing to the Ably Channel for captioning:

Running the completed project

Conclusion

Handling Massive Fan-out with AWS Lambdas

Lambdas in realtime experiences

WebSockets + Lambda: The perfect match for realtime experiences

WebSockets + Lambda example: A chat application

Demo: Making a chat app with Lambdas and WebSockets

Creating an AWS account

Lambda

Setting up the Pub/Sub broker

Creating the client-side chat app

Running the server

Conclusion

Cursors Everywhere: An experiment on shared cursors for every website

The Making of the Plugin

The Plan

Starting with the Basics - Sharing X and Y Coordinates

Capturing Cursor Movements

Improving the Original Approach: Relative Positioning

The Result

How it's done professionally: Miro

Stepping Up the Game: Smooth Transitions and Fewer Messages

1: Position Interpolation and Message Batching for Bandwidth Efficiency

Position Interpolation

2.1: Message Batching

2.2: Improving Batching Methods with the Spaces SDK

Taking Into Account the Element Under the Cursor

Element Identification

Including Element in Cursor Position

Replaying Cursor Movements

Still Not Quite There...

Improving Websites for Cursor Tracking

1. Adding Better Identification on Elements

2. Mapping Elements Across Different Views

Conclusion

Building realtime experiences with Amazon Aurora

What scenarios need Aurora and WebSockets?

How does Aurora help?

How do WebSockets help?

Architecture: Ensuring scalable realtime experiences

Architecture example: Chat

Demo: Making a chat app with Aurora and WebSockets

Step 1: AWS account

EC2

Aurora setup

Step 2: Adding the chat interface to the EC2 instance

Express

Ably

3. Creating a `.env.local` file