Forem: SandraMartin

Astro-Storyblok Previews Site with Netlify's Branch Deploys Feature

SandraMartin — Mon, 20 Mar 2023 17:45:46 +0000

Description: This article will tell you how to set up your Astro-Storyblok projects so that your previews site deploys to a Netlify branch-deploy, while keeping your main production site in SSG mode. Bonus: Github action to keep preview branch in-sync with production.

Introduction

If you are using Storyblok as your headless CMS for a static site such as a blog, you might want to set up a workflow where editors can click 'save' in Storyblok, and then see their changes. This would mean setting up the visual editor to work in Storyblok, which as I explained in a previous article, requires your Astro project to be in SSR mode.

What if you want to take advantage of Astro's static website mode (SSG) for your production site, but you would still like previews to work in Storyblok? How can we do this if previews require the project to be in SSR mode?

If you are using Netlify for deployments, there is a way to create an Astro project that builds the site in SSG mode for production environments and SSR mode for a previews branch , then deploys each version to a different Netlify URL using the Netlify branch-deploy feature.

You can also set up a Github action to keep the preview branch always up-to-date with any changes deployed to your main (production) branch!

Here's a walkthrough for how to set this up.

1. Create a branch-deploy in Netlify so you can have a special branch just for previewing content when clicking the 'save' button in Storyblok

Resource: Netlify Branches and deploys

Create Previews Branch

In your project, create a branch off of main called previews.

Go into your Netlify dashboard Deploys -> Deploy Settings -> Branches and deploy contexts.

Click Edit Settings.
Select 'Let me add individual branches'
In the Additional branches input, add your branch that you created called previews.

Now you have a branch that you will use for your preview environment. You're going to set up your Astro project to build in SSR mode just on this preview branch.

By the way, in Netlify terminology, a branch that you create which corresponds to a deploy that isn't from the main production branch is called a branch-deploy. There are different Netlify deploy-contexts, and branch-deploy is one of them (in addition to production and deploy-previews.

Create Environment Variables

You need to make sure that you have an environment variable that tells Netlify that this branch you created is the preview environment. Later, when you go into your Astro project, you'll set up the project to build differently based on these variables.

In Netlify, go to Deploys -> Deploy Settings -> Environment variables.

Create an environment variable called VITE_ENVIRONMENT.
In the Branch deploys, create a variable with the value preview. (Of course you can name this variable differently if you prefer.)

Now Netlify can be told based on the VITE_ENVIRONMENT variable to either build the branch in SSG mode for production, or build it in SSR mode for the previews branch.

2. In the Storyblok Visual Editor settings, add your Netlify preview branch URL to the Preview URLs

Log in to your Storyblok space. In the left side navigation menu, find settings. From there, navigate to Visual Editor.

This is where you will tell Storyblok to connect the Visual Editor to that special branch you created, the previews branch-deploy.

In the section Preview URLs, there are inputs to add multiple preview domains. Add previews where it asks for Name and the Netlify url for that deployed previews branch where it asks for Location.

You can look at the Netlify URL to know which deploy-context (i.e. which branch-deploy) the preview site is building from. Since you called your branch-deploy previews in Netlify, you'll see something like https://previews--NETLIFY_URL. If you were looking at a Netlify deploy-preview, it would look like https://deploy-preview-32--NETLIFY_URL (where the number is the number of the deploy-preview).

3. Set up your `astro.config.mjs` file so that it uses conditional logic to build your site in `static` mode if its in production, `server` mode in the preview environment

Now you'll head into your Astro project to add some necessary configuration. This is where the environment variable you set up in Netlify is going to come into play.

Set up `.env`

In Netlify, you created the environment variable VITE_ENVIRONMENT with the deploy-context of Branch deploys being given the preview value.

In your project, you'll need to create a .env file to supply your project with that variable value so it can work in dev mode. In the .env file, add VITE_ENVIRONMENT="preview".

Add configuration to `astro.config.mjs`

There are four properties in the config object given to defineConfig that will need to be written as ternary conditionals. They are output, accessToken (in the Storyblok settings object), and bridge (also in Storyblok settings), and adapter.

For output, you'll give a value that is dependent on whether the deploy-context is preview or not. Here's how it would look:

output:  import.meta.env.VITE_ENVIRONMENT  ===  "preview"  ?  "server"  :  "static",

This will tell the project to build in server mode (i.e. SSR) if it's deploying to the preview branch. If it's not, it will build in static mode (SSG).

The integrations array in the configuration object contains a storyblok method with a settings object passed in. You'll need to add ternaries for the accessToken and bridge like so:

storyblok({
    accessToken:
        import.meta.env.VITE_ENVIRONMENT  ===  "preview"
        ?  "HodE5R9wdAKERDFNiq7LGStl" : "UUtz6RYjUvOC0tMf1MteHQtz",
    bridge:  import.meta.env.VITE_ENVIRONMENT  ===  "preview"  ? true :  false,
    apiOptions: {
        region:  "us",
    },
    components: {
        page:  "storyblok/Page",
        feature:  "storyblok/Feature",
        grid:  "storyblok/Grid",            
        ...
        },
    }),

This tells the Storyblok integration to use the Preview token if the site is deploying in SSR, and the Public token if the site is deploying in SSG. (These Storyblok access tokens are created in your Storyblok space in Settings > Access Tokens.)

In the article I previously referenced, which is where I show how to set up the Visual Editor, I discuss how to add the Netlify adapter. The Netlify adapter is needed for the SSR deploy-context. You can read that article for more information if you need it on setting up SSR. You will need to set up the Netlify adapter.

You can tell your project to use the Netlify adapter only in SSR mode:

adapter: import.meta.env.VITE_ENVIRONMENT === 'preview' ? netlify() : undefined,

Here's an example of how the astro.config.mjs file might look. There might be differences depending on your project integrations, but this is a general example:

import { defineConfig } from  "astro/config";
import  storyblok  from  "@storyblok/astro";
import  netlify  from  "@astrojs/netlify/functions";
import  tailwind  from  "@astrojs/tailwind";

export  default  defineConfig({
    output:  import.meta.env.VITE_ENVIRONMENT  ===  "preview"  ?  "server"  :  "static",
    adapter: import.meta.env.VITE_ENVIRONMENT === 'preview' ? netlify() : undefined,
    integrations: [
        tailwind(),
        storyblok({
            accessToken:
                import.meta.env.VITE_ENVIRONMENT  ===  "preview" ?  "HodE5R9wdAKERDFNiq7LGStl" : "UUtz6RYjUvOC0tMf1MteHQtz",
                bridge:  import.meta.env.VITE_ENVIRONMENT  ===  "preview" ? true :  false,
                apiOptions: {
                region:  "us",
                },
                components: {
                    page:  "storyblok/Page",
                    feature:  "storyblok/Feature",
                    grid:  "storyblok/Grid",
                },
            }),
        ],
});

4. Get page data from Storyblok using the public token in SSG mode, the preview token in SSR mode.

The Astro pages that pull in content data from Storyblok can pull in either draft content or published content. You need to tell your project to build the pages using draft content when in the preview deploy-context, and published content when in production.

For every page in your Astro pages folder, you will do something like this when getting data from the Storyblok API:

const { data } =  await  sbApi.get("cdn/stories/home", {
    version:
        import.meta.env.VITE_ENVIRONMENT  ===  "preview"  ?  "draft"  : "published",
});

This will conditionally build the project using either draft or published content based on that environment variable value in Netlify.

Here is an example of a home page in an Astro project that does this:

---

import { useStoryblokApi } from  "@storyblok/astro";
import  StoryblokComponent  from  "@storyblok/astro/StoryblokComponent.astro";
import  Layout  from  "../layouts/Layout.astro";

const  sbApi  =  useStoryblokApi();
const { data } =  await  sbApi.get("cdn/stories/home", {
    version:
        import.meta.env.VITE_ENVIRONMENT  ===  "preview"  ?  "draft"  : "published",
});
const  story  =  data.story;
---

<Layout>
<StoryblokComponent  blok={story.content} />
</Layout>

For creating dynamic routes, you can use a conditional setup to tell your project to use Astro's getStaticPaths method in SSR mode only. Checkout this Discord comment for an example by user Nikitos.

5. Create a Github action that automatically merges any changes made to main to the previews branch too

The last thing you might want to do is to set up a continuous integration so that your preview branch stays in-sync with the main branch. That way, when there is a deploy to production, you don't have to manually update the previews branch - it will happen automatically.

You can use a Github Action to do this. Here's what you need to do.

Create a .github folder in your project at the root. Inside that folder, add a workflows folder.
Create a file called auto-merge.yml inside the workflows folder

Add the following code to that file:

name: Merge main to previews
on:
    pull_request:
        branches: [main]
        types: [closed]
jobs:
    merge-main-to-previews:
    if: github.event.pull_request.merged == true
    timeout-minutes: 2
    runs-on: ubuntu-latest
    steps:
        - uses: actions/checkout@v3
        - name: Set Git config
        run: |
            git config --local user.email "actions@github.com"
            git config --local user.name "Github Actions"
        - name: Merge main to previews
        run: |
            git fetch --unshallow
            git checkout previews
            git pull
            git merge --no-ff main -m "Auto-merge main to previews"
            git push

Now, anytime a pull request is merged to the main (production) branch, it will also merge those changes into the previews branch!

Getting the Visual Editor to work for Storyblok + Astro

SandraMartin — Wed, 01 Feb 2023 02:39:59 +0000

There’s a relatively new integration between Astro and Storyblok. It’s an SDK that helps developers build a site in Astro and use Storyblok as the CMS. Check out the Github readme for astro-storyblok to get started.

As I was getting my own Astro-Storyblok project set up, I ran into a few bumps in the road. Those bumps had to do with getting storyblok's visual editor working.

Trying out the example project - it works great!

There's an example Astro+storyblok that has been created by the storyblok developer relations team. It's a Stackblitz. When I created an example Storyblok space and connected it to that Stackblitz, I saw the visual editor working great!

Trying out my own project - it doesn't work!

However, when I created my own project just like the Stackblitz, put it on Github, deployed my project to Netlify, then connected it to Storyblok just like I did with the Stackblitz example, I ran into a problem.

The visual editor didn't work.

I would click "save", but the changes I had made did not show on the page. Everything in my project was the same as in the Stackblitz. The only real difference in my project setup was that my project was a Github repo.

The Stackblitz and my Github project were both built in Astro's SSG mode - they were built as static sites. But for the visual editor to work, the Github project that Storyblok integrates with needs to be in Astro's SSR mode - it needs to be a server-side rendered site.

So I changed my Github project to SSR mode, and that fixed the problem.

Walkthrough

I'll walk you through the steps for how to get this starter project working, and hopefully this is helpful.

Here are the things I needed to do:

Build a website with Astro that is deployed with Netlify (I'll just use the storyblok-astro example project for simplicity)
Use the storyblok headless CMS to edit content on the website
See the edits to my site update in the storyblok visual editor when I click "save"
Publish changes so that the website builds with those new changes when I click "publish" in the storyblok editor

Create a storyblok space

First I went into the storyblok dashboard and added a new space. I chose New Space and then clicked Create Space.

In the blok library, there were already four bloks created. These bloks correspond the the storyblok components that I need to create in my Astro project.

Create the project

I looked at the Stackblitz project to guide me as I created a project in VS Code. I copied over the same files, since they correspond to the example space that Storyblok gives you when you create a new space in Storyblok. Here is my Github repo so you can see the files .

Notice that the files in the storyblok folder correspond to the bloks in my storyblok space:

src
--layouts
--pages
--storyblok
    --Feature.astro
    --Grid.astro
    --Page.astro
    --Teaser.astro

Install dependencies

Since I copied the files from the Stackblitz, which means there is a package.json with the project dependencies, I could just type npm install to install all the needed dependences. But if I were building this from scratch, I would need the following:

The @storyblok-astro dependency, which is the SDK to make Astro work with storyblok

npm install @storyblok/astro

The @astrojs/netlify dependency. This is the Netlify adapter that I must use with Astro to make my Astro project build in Netlify in SSR mode.

npm install @astrojs/netlify

Set up SSR

To set up SSR in my Astro project, I needed to add the line output: "server" to the defineConfig function in the astro.config.mjs file. This tells Astro to use server-side rendering.

import { defineConfig } from "astro/config";
import storyblok from "@storyblok/astro";

export default defineConfig({
  output: "server",
});

Also, since I'm using the Netlify adapter, I need to import the adapter package and add a line to the astro.config.mjs file:

import { defineConfig } from "astro/config";
import netlify from "@astrojs/netlify/functions";

export default defineConfig({
  output: "server",
  adapter: netlify(),
});

Configure project to work with storyblok

Integrating with storyblok requires a few steps. I already installed @storyblok/astro but I would need to set up the astro.config.mjs to include the storyblok integration.

I needed to add the storyblok()function inside the intregrationsproperty of the config object. That storyblok() function takes an object with several options:

storyblok({
    accessToken: "<YOURACCESSTOKEN>",
    bridge: true,
    apiOptions: {
        region: "us",
    },

For the accessToken above, be sure to use the preview token if you want to use this with the visual editor in a preview branch/environment. If you are using this in production, you would use the public token. I'm using the preview token in my example project because I want it to work with the visual editor.

Then I'll add the components that are needed to connect to the storyblok bloks in the storyblok space (Astro knows to look for these components in the src folder).

components: {
    page: "storyblok/Page",
    feature: "storyblok/Feature",
    grid: "storyblok/Grid",
    teaser: "storyblok/Teaser",
},

Here's the entire config as looks now in the project:

import { defineConfig } from "astro/config";
import storyblok from "@storyblok/astro";
import tailwind from "@astrojs/tailwind";
import netlify from "@astrojs/netlify/functions";

export default defineConfig({
  output: "server",
  adapter: netlify(),
  integrations: [
    storyblok({
      accessToken: "LSTMzttTLabrQLn9YBzQowzz", // preview token
      bridge: true,
      apiOptions: {
        region: "us",
      },
      components: {
        page: "storyblok/Page",
        feature: "storyblok/Feature",
        grid: "storyblok/Grid",
        teaser: "storyblok/Teaser",
      },
    }),
    tailwind(),
  ],
});

Deploy site to Netlify

The project is now all ready in Github to be deployed. I went into Netlify and clicked on Add new site. Then I selected Import an existing project and chose Github for where to grab the project from.

I found my repo, selected it to deploy, and clicked Deploy site. Netlify then did its magic and built the site, giving me a url that I could click on to see my site.

Make the storyblok visual editor work

Now I was able to go into storyblok and connect the site to the visual editor. In storyblok, I head into the settings for my space and click on Visual Editor.

I then added the newly deployed site's URL to the the Visual Editor settings in Storyblok where it says Location (default environment).

Now, when I look at Content in my storyblok space, and then click on Home, I see my website in the visual editor.

And the best thing is, when I click on a blok in the right hand window, I can edit that block to have different text. When I click "save", I see the page in the visual editor refresh, and my change is there!

Create webhook to publish storyblok edits to the site

The last thing I need to set up is the ablity to trigger a build with my new changes when I click the publish button in the storyblok editor. To do this, I needed a webhook from Netlify.

In Netlify, I went to deploys / deploy settings / build hooks. I then created a build hook, giving it the name storyblok_publish), and then copied the hook.

Then I went into the Storyblok space settings - Webhooks and added the webhook where it says Story published & unpublished.

Now when I clicked 'publish' in the Storyblok editor, it would trigger a deploy to my site with the changes.

Once the site builds, I see that it has the changes I made in storyblok!

Fetch Hosted Audio Streams In The Browser

SandraMartin — Tue, 21 Jun 2022 15:23:08 +0000

Today we'll look at how to fetch an audio stream from a URL, send the stream data to Deepgram, and receive transcriptions in return. This is particularly useful when you want to get live stream data and send it in real-time (and also receive a transcription back from Deepgram in real-time chunks).

This will require three main steps:

Open a WebSocket channel to Deepgram
Request the hosted audio stream data from the URL where the stream is hosted (using Fetch)
Pass the stream in incremental chunks to Deepgram

If you want to see the audio stream transcribed by Deepgram's speech-to-text API, I recommend signing up for a free account and getting an API key here.

Open WebSocket Channel

The very first thing we need to do is connect to Deepgram with a browser WebSocket.

const socket = new WebSocket('wss://api.deepgram.com/v1/listen', [
  'token',
  'YOUR_DEEPGRAM_API_KEY',
])

Next we'll write several event listeners. Events are actions that occur within a programmed system; the system is designed to inform us of those events so that we can write code in reaction to them. The WebSocket API includes several events that fire throughout the process of the socket opening, data being received, and the socket closing. We'll use an event listener to react to those events.

We write this WebSocket logic first because then we can listen for the onopen event to take place, and once it does, we make the fetch request to receive the audio file:

socket.onopen = () => {
  // Fetch stream
  // Send stream to Deepgram
}

socket.onmessage = (message) => {
  // Receive transcript from Deepgram
  // Do something with transcript
}

socket.onclose = () => {
  console.log({ event: 'onclose' })
}

socket.onerror = (error) => {
  console.log({ event: 'onerror', error })
}

Fetch Hosted Audio Stream

The next step will be to write the fetch request. The browser's global fetch() method takes a string parameter that is the URL to where we will be making the request.

The method returns a promise, so we can chain .then() to wait for the response and get the response body.

socket.onopen = () => {
  const url =
    'https://stream.live.vc.bbcmedia.co.uk/bbc_radio_fourlw_online_nonuk'

  fetch(url)
    .then((response) => response.body)
    .then((body) => {
      // Send stream to Deepgram
    })
}

Read Chunks and Send to Deepgram

The response.body we receive from the URL will be a ReadableStream, which is a "readable stream of byte data" (Mozilla). This just means it is data that can be read. And in order to read the data, we must break it down - similar to the way a human must break a sentence down into words in order to comprehend its meaning.

How do we break down the stream data? The Streams API was created for this purpose. We'll use it to take the stream and break it into chunks, which are the single pieces of data that are written to or read from a stream.

We can write a function to consume the readable stream and turn it into chunks to send to Deepgram. There are different ways to do this, but all of them will need to do the following:

Create a reader using the getReader() method
Use the read() method of the reader interface to get access to each chunk in the queue (returned as a promise)

Here is one way to write this function:

async function readAllChunks(readableStream) {
  // Create reader:
  const reader = readableStream.getReader()
  while (true) {
    // Read chunks:
    const { done, value } = await reader.read()
    if (done) {
      break
    }
    // Send chunks to Deepgram:
    socket.send(value)
  }
}

Since the .read() method returns a promise that resolves to an object, we can destructure that object into its two properties: done and value. Then we can send that value on to Deepgram for transcription.

The Streams API specification provides other useful examples for how to consume a ReadableStream.

Now that we have the logic to read the stream as chunks, we need to invoke the function and pass in the readable stream:

socket.onopen = () => {
  const url =
    'https://stream.live.vc.bbcmedia.co.uk/bbc_radio_fourlw_online_nonuk'

  fetch(url)
    .then((response) => response.body)
    .then((body) => {
      // Invoke function that sends readable stream:
      readAllChunks(body)
    })
}

Finishing Up

We have accomplished what we set out to do, which is to get a hosted audio stream and send it to Deepgram to be transcribed in realtime. When the transcript is returned, we can do whatever it is we intended to do with it.

Here's the code in its entirety. I've also included an example in my Stackblitz account that puts the text from the stream onto the browser page. Be sure to add an API key to make it work.

const socket = new WebSocket('wss://api.deepgram.com/v1/listen', [
  'token',
  'YOUR_DEEPGRAM_API_KEY',
])

socket.onopen = () => {
  const url =
    'https://stream.live.vc.bbcmedia.co.uk/bbc_radio_fourlw_online_nonuk'
  fetch(url)
    .then((response) => response.body)
    .then((body) => {
      readAllChunks(body)
    })
}

async function readAllChunks(readableStream) {
  const reader = readableStream.getReader()
  while (true) {
    const { done, value } = await reader.read()
    if (done) {
      break
    }
    socket.send(value)
  }
}

socket.onmessage = (message) => {
  const received = JSON.parse(message.data)
  const transcript = received.channel.alternatives[0].transcript
  if (transcript && received.is_final) {
    document.querySelector('#captions').textContent += transcript + ' '
  }
}

socket.onclose = () => {
  console.log({ event: 'onclose' })
}

socket.onerror = (error) => {
  console.log({ event: 'onerror', error })
}

Have questions? We're happy to help @DeepgramDevs.

Enhance Your Audio With Dolby.io For Higher Quality Transcripts

SandraMartin — Fri, 03 Jun 2022 20:53:24 +0000

Your recordings may not always be great quality - there might be a graininess to them, or background noise that interferes with what the listener is trying to focus on. While Deepgram may still perform fairly well, it's always true that better source audio results in a higher chance for accurate transcripts. For this tutorial, I'll use a low-quality audio file from the Library of Congress.

An excellent tool for improving the quality of audio is Dolby.io's Media Enhance API. With this API, all I have to do is make a POST request with the audio file, and Dolby.io can analyze it to remove the interfering sounds and the crackling or static you often hear with these types of recordings. I can even specify what type of content the audio is, such as an interview, podcast, or voice recording, and Dolby.io can enhance it even further for that type of content.

Before We Start

Before jumping into coding, I'll be sure to get an API key from each of the APIs I'll be using today. I'll head to Dolby.io and then Deepgram.com to get keys.

I'll also install the dependencies with the following command in my project directory:

npm install axios @deepgram/sdk

Create a Node.js Project

I'll create an index.js file and require axios to help with making API requests:

const axios = require('axios')

I intend to send an audio file to Dolby.io for enhancement, wait for it to be processed, and wait for that file to come back to me. Since there will be an unknown amount of time involved in each step of the process, I need to write asynchronous functions for each step. Here are the steps:

Start the Enhance Job

The first asynchronous function will be called startEnhanceJob

const startEnhanceJob = async (url) => {}

I need to make the audio file available to Dolby.io by putting it in cloud storage. Dolby offers the option of me putting it in their temporary cloud storage, but I have to use the URL format they expect, which will start with dlb://. I'll write some JavaScript to create that Dolby.io URL format:

const startEnhanceJob = async (url) => {
  const dlbUrl =
    'dlb://out/' + url.split('/').slice(-1)[0].split('.')[0] + '.wav'
}

Then I will make the POST request with the audio file to Dolby.io and receive a job ID for that enhance job (which I'll need in the next step).

const startEnhanceJob = async (url) => {
  // create a dolby URL
  const dlbUrl =
    'dlb://out/' + url.split('/').slice(-1)[0].split('.')[0] + '.wav'

  // POST request
  const { data } = await axios({
    method: 'post',
    url: 'https://api.dolby.com/media/enhance',
    headers: {
      'x-api-key': 'YOUR_DOLBYIO_API_KEY',
      'Content-Type': 'application/json',
      Accept: 'application/json',
    },
    data: {
      input: url,
      output: dlbUrl,
      content: { type: 'interview' },
    },
  })

  return { jobId: data.job_id, dlbUrl }
}

Notice that I added content: { type: interview } since the audio file I'm sending is an interview.

Check the Enhance Job and Report Progress

It will take some amount of time for the enhancement job to run. I need to track the progress so that I know when the file is ready. I'll write two functions in this step: checkEnhanceJob and waitUntilJobCompletes.

For checkEnhanceJob, I'll take the job ID that was returned from the startEnhanceJob function, and I'll use it to make a GET request to the Dolby.io Enhance API to get progress on the enhancement job:

const checkEnhanceJob = async (job_id) => {
  const { data } = await axios({
    method: 'GET',
    url: 'https://api.dolby.com/media/enhance',
    params: { job_id },
    headers: {
      'x-api-key': api_key,
      'Content-Type': 'application/json',
      Accept: 'application/json',
    },
  })

  // Returns a number between 0 and 100
  return data.progress
}

Then I'll write a function that uses the checkEnhanceJob result in a loop to show a countdown as progress is being made on the enhance job. It will wait 2000ms (2 seconds) between each loop:

const waitUntilJobCompletes = async (job_id) => {
  let progress = await checkEnhanceJob(job_id)
  while (progress < 100) {
    await new Promise((r) => setTimeout(r, 2000))
    progress = await checkEnhanceJob(job_id)
    console.log(progress)
  }
  return
}

Get Enhanced File URL

Once the enhancement of the file is complete, I need to output that new file to a URL that I can use (in this project, I'll be using it to pass on to Deepgram for transcription).

I'll write a function that will make a POST request to put the output of the enhance job at the URL I created for temporary storage of the file. I'll also console.log the file URL so I can test it now and see how it sounds.

const getNewFileUrl = async (dlbUrl) => {
  const { data } = await axios({
    method: 'POST',
    url: 'https://api.dolby.com/media/output',
    data: { url: dlbUrl },
    headers,
  })

  console.log(data.url)
  return data.url
}

Run the Enhance Logic

I wrote each step of the enhancement job, but now I need to write a main function that runs every step, i.e., every function I wrote.

I also need to add the URL of the audio file I want to enhance. I've chose a file from the Library of Congress called "Interview with Lillie Haws, New York, New York, November 12, 2001".

Here is the main function:

const main = async () => {
  // start enhancement of file:
  const { jobId, dlbUrl } = await startEnhanceJob(
    'https://tile.loc.gov/storage-services/media/afc/911/afc2001015_sr298a01.mp3'
  )

  // track progress as it is processing:
  await waitUntilJobCompletes(jobId)

  // get the output URL:
  const url = await getNewFileUrl(dlbUrl)
}

main()

When it runs, I'll see the values of my loop that I printed counting up to completion of the enhancement job. And when it finishes, I'll see a very long URL that I can use to listen to my file.

If I click on the link, I'm taken to the hosted audio file. It sounds so much better than the original! Now I'm ready to transcribe it with Deepgram.

Transcribe With Deepgram

I'll be using Deepgram's API for transcribing Pre-Recorded Audio. Deepgram has a Node.js SDK, so I'll require it in my index.js file. I'll also create a new instance of Deepgram by giving it my Deepgram API key:

const { Deepgram } = require('@deepgram/sdk')
const deepgram = new Deepgram(process.env.DEEPGRAM_KEY)

I will take the file URL that I received from Dolby.io and send that to Deepgram for transcription. It is the temporarily stored file that I assigned to the url variable in the main function (in the last section).

I'll also specify that I would like Deepgram to add punctuation. I can do this by adding { punctuate:true } to the request:

const response = await deepgram.transcription.preRecorded(
  { url },
  { punctuate: true }
)

Now I can run the whole function, and I'll see that Deepgram transcribes the enhanced file. I'll console.log the response from Deepgram so I can actually see the transcription now:

// main function
const main = async () => {
  const { jobId, dlbUrl } = await startEnhanceJob(
    'https://tile.loc.gov/storage-services/media/afc/911/afc2001015_sr298a01.mp3'
  )
  await waitUntilJobCompletes(jobId)
  const url = await getNewFileUrl(dlbUrl)
  const response = await deepgram.transcription.preRecorded(
    { url },
    { punctuate: true }
  )
  console.log(response.results.channels[0])
}

main()

And now I have a full transcript of the audio file from the Library of Congress.

Conclusion

Today I used Dolby.io and Deepgram to enhance an audio file and transcribe the speech of the interview into text. These two APIs seem like a great combination for many future projects!

If you enjoyed my post, follow me at Twitter to continue the conversation.

Project Upgrade: Voice Controlled To-Do List App with Deepgram and Vue 3

SandraMartin — Mon, 02 May 2022 20:25:41 +0000

Recently I wrote about a project I did to help me learn Pinia, Vue 3's new official state management system. I built a basic to-do list app:

It dawned on me that a fun way to jazz up this project would be to use Deepgram to make the app voice-powered so that a user can speak commands to add, delete, or check-off items on the list.

I'm inspired by my colleague Bekah's series about updating portfolio projects. A voice-based to-do list app would be a lot more interesting than a regular to-do list app!

Project Overview

The project I originally did can be found in this repo, and the accompanying blog post is here. Check it out to build the standard to-do list project with Vue 3 and Pinia.

In this iteration of the project, I'll continue to use Pinia to manage global state, but I'll add Deepgram so I can use Deepgram's speech-to-text API to help me power the voice-control feature. If you want to build this voice-control feature along with me, I've created a starting branch here.

Deepgram Live Streaming Logic

There are several articles on Deepgram Docs about how to use Deepgram in Javascript to convert an audio stream of speech into text, including one I wrote about how to use it with Vue 3. I won't go over in detail here how I built Vue composables to integrate Deepgram's speech-to-text API. Use these resources or take a look at my repo (in the composables folder) to learn how I did this using Vue 3 and the Deepgram Node SDK:

Additional Resources

Focus: Voice-Control Feature

For today's project, I will focus on dealing with the transcript of text that I get back from Deepgram, analyzing it for speech commands to add, delete, and check off items in the to-do list. This logic will all be part of a voice-control feature.

Create the Component and Connect Deepgram

The voice-control feature that I build today will be one component with two main elements:

a button that, when clicked, turns on or off voice-control mode (i.e., toggles the connection to Deepgram)
a div that contains text which identifies the status of whether Deepgram is connected and listening, if it misheard what was said, or if it is not connected.

Here's a minimally styled version of this project that highlights the component I'll be building today:

In the components folder with the other Todo components, I'll create a component called TodoSpeech.vue. It will be a child component of TodoApp.vue.

In TodoSpeech.vue, I'll start by adding the button element and status div to the template. I'll use a speech bubble emoji on the button, and I'll create an event listener that I plan to program to toggle the Deepgram connection when the button is clicked.

<!-- in template: -->
<button @click="toggleListen()">💬</button>
<div>{{ deepgramStatus }}</div>

I'll also create a deepgramStatus ref, which I'll program to update when Deepgram is connected and ready to receive audio data. The initial value will report that Deepgram is not connected:

// in script:
let deepgramStatus = ref('Deepgram Not Connected')

I'll connect to Deepgram using the composable useDeepgramSocket (already built in a previous post), which I import in the script. I also run the composable in the setup function so that it runs at the beginning of the component lifecycle. I destructure four properties off of the composable: DG_socket, DG_transcript, openStream, closeStream.

<script>
import { ref } from "vue";
import useDeepgramSocket from "@/composables/useDeepgramSocket";

export default {
  setup() {
    const { DG_socket, DG_transcript, openStream, closeStream } = useDeepgramSocket();
    let deepgramStatus = ref("Deepgram Not Connected");

    return { deepgramStatus };
  },
};
</script>

The openStream and closeStream methods that I destructured off of the useDeepgramSocket composable will toggle on and off when the button is clicked and toggleListen runs. Notice that I created the isListening ref to update the toggle status:

let isListening = ref(false)

function toggleListen() {
  if (!isListening.value) {
    openStream()
    isListening.value = true
  } else {
    closeStream()
    isListening.value = false
  }
}

I'll use a watcher to watch the Deepgram socket's status in the composable. That way, the div in the template will update when I click the button and Deepgram is connecting, connected, or not connected:

watch(DG_socket, () => {
  if (DG_socket.value === 'Connecting') {
    deepgramStatus.value = 'Connecting'
  } else if (DG_socket.value === 'Not Connected') {
    deepgramStatus.value = 'Voice Controls Off'
  } else if (DG_socket.value === 'Closing connection...') {
    deepgramStatus.value = 'Closing connection...'
  } else {
    deepgramStatus.value = 'Listening'
  }
})

Here is the entire component now. I should be able to click the button and see the status change to show the status of the Deepgram connection:

<template>
  <div>
    <button @click="toggleListen()">💬</button>
    <div>{{ deepgramStatus }}</div>
  </div>
</template>

<script>
import { ref, watch } from "vue";
import useDeepgramSocket from "@/composables/useDeepgramSocket";

export default {
  setup() {
    const { DG_socket, DG_transcript, openStream, closeStream } =
      useDeepgramSocket();

    let deepgramStatus = ref("Deepgram Not Connected");
    let isListening = ref(false);

    function toggleListen() {
      if (!isListening.value) {
        openStream();
        isListening.value = true;
      } else {
        closeStream();
        isListening.value = false;
      }
    }

    watch(DG_socket, () => {
      if (DG_socket.value === "Connecting") {
        deepgramStatus.value = "Connecting";
      } else if (DG_socket.value === "Not Connected") {
        deepgramStatus.value = "Voice Controls Off";
      } else if (DG_socket.value === "Closing connection...") {
        deepgramStatus.value = "Closing connection...";
      } else {
        deepgramStatus.value = "Listening";
      }
    });

    return { deepgramStatus, toggleListen };
  },
};
</script>

Analyze Deepgram Response for Voice Commands

The logic to add, delete, or check-off an item from the list will be based on what the speaker says. I need to program this application to listen for commands such as "add!" or "delete!".

To do that, I will need to understand the form of the text transcript I'll be getting back from Deepgram.

Deepgram Audio Stream to Text

When I started using Deepgram to transcribe audio streams, I was surprised that Deepgram was able to send back the text almost immediately, after each phrase or sentence rather than waiting until the socket closes and sending back the entire transcription.

How is it able to send a response back continuously, as the audio is streaming, and how is it able to send meaningful chunks, such as at the end of a sentence or a pause in speech? This is thanks to two of its features: Endpointing and Interim Results.

To put it very simply, Endpointing is how Deepgram uses pauses in speech to process that speech into meaningful text chunks. It can hear longer silences in speech and use them to identify finished thoughts, i.e., phrases or sentences.

Interim results is Deepgram analyzing speech as accurately and as quickly as possible with the little bits of info it gets on the fly; then, after it gets more data to work with, it corrects the transcription. That is why as one or two words come back, one might be wrong, but then when the sentence comes back, the whole thing is more accurate.

In my case, I'm using the Interim Results property is_final to identify a complete voice command. When is_final is true, it means I'm dealing with a full statement or a complete command such as "Add walk the dog to the list!" The is_final property breaks the transcript up at the end of a fully-processed statement:

// in useDeepgramSocket.js:
const transcript = received.channel.alternatives[0].transcript
if (transcript && received.is_final) {
  DG_transcript.value = transcript + ''
}

This is a pretty cool feature because it means that a user can speak a command, such as "Add walk the dog to the list", and Deepgram can identify that it is the end of the command based on the flow of the speech. It will send me that sentence back to deal with in text form, and then I can use logic to dig in and find the specific command word, such as add or delete. Nice!

Add To-Do Item With Voice-Control

Now I'll write logic to analyze a voice command for the words "add to do", and if the command has those words (such as in the command "ADD TO DO walk the dog!"), the item will be added to the todoList array in the Pinia store.

I also want to keep track of the number of commands given, so if Deepgram doesn't transcribe the command correctly due to problems interpreting the speech, I can check that count value against the number of items in the store todoList array, and report back to the user that the command was misunderstood and didn't make it in.

Here is pseudo-code for what I need to write:

function addTodo(command) {
  // Create array of Regular Expression words to identify in the text string, such as [/^add to do/, /^ad to do/]
  // Loop through regex values using .find()
  // Turn the command into a standardized string - lower case, no final punctuation, trim whitespace
  // Check if the command string contains the regex value using .test
  // If so, remove the command phrase ADD TO DO to create new string that is just the todo item
  // add todo to the Pinia store todo-list
  // reset the count of uttered commands to match the length of the todo list array in the store
}
watch(utterance, () => {
  // watch for a command and if there is one, add it to the list
})

PHEW! That is a lot of logic to get through. Best to take it one step at a time.

Create a Set of Regular Expressions

I want to analyze the command string such as "Add to do walk the dog" for the three words "add to do". I will:

Create a regular expression by enclosing it within slashes /-/
Use the character ^ before the command phrase to identify that the phrase should be at the beginning of the string

So the regular expression for "add to do" that I will use is /^add to do/.

However, since there is the possibility that Deepgram could transcribe this with 'ad' instead of 'add' (seems unlikely, but I want to be prepared), I'll also use /^ad to do/. In fact, I'll match to anything that could be a homophone.

I need an array to create a set of all the options. (The reason I don't include something like /^add two do/ is because it's not grammatically correct, and Deepgram is smart enough to know not to transcribe things as ungrammatical.)

const addRegEx = [/^add to do/, /^ad to do/, /^add to dew/]

Use the Array Method .find()

I'll use the array method .find to loop through the array of regular expressions. It will search for the first item in the array that matches. If it finds one, there's no need to continue looping through since all we need is one match:

function addTodo(command) {
  const addRegEx = [/^add to do/, /^ad to do/, /^add to dew/]
  // loop through array to find first match:
  addRegEx.find((reg) => {})
}

Write a Method to Standardize the Command String

I want the command to be lowercase, to not include periods, commas, question marks, or quotation marks, and to not have extra whitespace at the end. (I could entirely turn off punctuation in Deepgram, but I like the transcript to include apostrophes in words such as "she's.")

I'll create a function to take the command string and standardize it:

function standardizeUtterance(command) {
  const punctuation = /[.,?"]+/g
  const change = command.toLowerCase().replace(punctuation, '').trim()
  return change
}

Notice that I used another regular expression, /[.,?"]+/g. This paired with the replace method will search for any of those punctuation marks throughout the entire string and replace them with"" (which is nothing).

Then I use this standardizeUtterance function inside the addTodo function. I'll add it before the loop so that it doesn't run for every loop:

function addTodo(command) {
  const addRegEx = [/^add to do/, /^ad to do/, /^add to dew/]
  // clean up utterance
  const item = standardizeUtterance(command)
  addRegEx.find((reg) => {
    // use item for more logic
  })
}

Test that the Item Matches the Regular Expression

I'll write an if statement to say that if the string starts with the regular expression command such as /^add to do/, then do something else (the something else will involve adding it to the to-do list).

I use the method .test, which is a javascript method used to match a regular expression with a string, returning true or false depending on if there is a match or not.

// inside addTodo method:
addRegEx.find((reg) => {
  if (reg.test(item)) {
  }
})

Remove the Command Phrase from the String

If there is a match, I will want to add the string to the to-do list. But right now the whole string also contains the command phrase "add to do" as in the sentence "Add to do walk the dog." I do not want "add to do" to be part of the string that goes into the to-do list array in the store.

I'll write a function that takes the string and removes the phrase. I'll need to give it both the full command, and the regular expression (which is the command phrase). The method replace will search for the phrase and replace it with "", i.e., nothing.

function removeCommandPhrase(command, reg) {
  const change = command.replace(reg, '').trim('')
  return change
}

Then I add it to addTodo:

// inside addTodo method:

addRegEx.find((reg) => {
  if (reg.test(item)) {
    // remove command phrase ADD TO DO
    const todo = removeCommandPhrase(item, reg);
    }
  });
}

Add the To-Do Item to the To-Do List Array

I'm almost done! This is the most important step. I can add the to-do item to the list in the store.

I have to go back up to the start of the script and import the store. I also need to run the store function and set it to a variable that I can use (I'll set it to store)

<script>
import { ref, watch } from "vue";
import useDeepgramSocket from "@/composables/useDeepgramSocket";
// import store:
import { useTodoListStore } from "../store/useTodoListStore";

export default {
  setup() {
    const { DG_socket, DG_transcript, openStream, closeStream } =
      useDeepgramSocket();
     // run function and set to variable:
    const store = useTodoListStore();
    ...
    function addTodo(command) {
    ...
    });
}

Then inside addTodo I will use the store.addTodo function that is already in the store as an action (I created it in the first post in this series.)

// inside addTodo method:

addRegEx.find((reg) => {
  if (reg.test(item)) {
    // remove command phrase ADD TO DO
    const todo = removeCommandPhrase(item, reg)

    // add to store
    store.addTodo(todo)
  }
})

Now the logic is there to add the item to the to-do list. I just need to do one more thing to make it show up on the screen.

Watch For a Command, Add it To The List

Right now, the addTodo function with all the logic to add an item to the list never runs. I need to make it run somehow.

Originally, I set up Deepgram to turn on when the button is clicked. When the logic runs to create the WebSocket connection to Deepgram, I also get a value from it that I defined as DG_transcript. This value holds the transcript string that comes back after a user says something such as "Add to do walk the dog".

I want to trigger addTodo to run every time a new command is said. And I know I'll be using that transcript for logic to keep track of how many times a command has been said (I'll go over that in the next section). So I'm going to create a ref in this TodoSpeech.vue component called utterance, which will stay in sync with the DG_transcript from the useDeepgramSocket.js composable.

// inside setup() function in TodoSpeech.vue
const { DG_socket, DG_transcript, openStream, closeStream } =
  useDeepgramSocket()
// create ref
let utterance = ref(DG_transcript)

Now the utterance ref is in sync with the transcript that comes from Deepgram. I can watch that utterance ref for changes, and if there is a change, the addTodo function will run.

watch(utterance, () => {
  if (utterance.value !== '') {
    addTodo(utterance.value)
  }
})

Now when I use the voice-control feature to add an item to the list, I see it show up on the screen. Woo-hoo!

Dealing with Misunderstood Commands

Automatic Speech Recognition technology has gotten really good, but there can still be mistakes. People might mumble or slur some of the words, or the garbage truck could be making a lot of noise in the background. Deepgram's ASR technology can handle a lot, but I have to expect that sometimes it will mishear something.

What if I say "Add to do walk the dog", but Deepgram mishears it as "And to do walk the dog"? I want to be ready for that and for any other situation. What if the user forgets to add the command and just says "Walk the dog!"

I'm going to program my app to show a message to the user when this happens. It will display "I didn't catch that" when it doesn't hear a match to the command phrase.

The logic for this will depend on counting how many times a command has been spoken. If the speaker says one thing, and it gets added to the list, then the count should be one. But if the speaker says another thing after that, and it doesn't get added to the list, then the count is at two, but the list is at one. So there was a misunderstanding.

Here is the logic that will run:

watch(utterance, () => {
  if (utterance.value !== '') {
    count.value++
    addTodo(utterance.value)
    alertMisunderstood()
  }
})

If a command is given, the count is increased by one. Then theaddTodo function runs.

Inside addTodo, I check for a match. If there is a match, it means that an item is added to the list. After it is added to the list, I will reset the count to match the number of items in the list:

function addTodo(command) {
  const addRegEx = [/^add to do/, /^ad to do/, /^add to dew/]
  // clean up utterance
  const item = standardizeUtterance(command)
  addRegEx.find((reg) => {
    if (reg.test(item)) {
      // remove command phrase ADD TO DO
      const todo = removeCommandPhrase(item, reg)
      // add to store
      store.addTodo(todo)
      // reset count
      count.value = store.todoList.length
    }
  })
}

However, in the if statement above, I test for a match. If it does NOT find a match, none of that logic inside of it runs. So that would result in the count having increased, but the number of items in the to-do list not having increased.

In that case, I need to write logic to notice that discrepancy and alert the user:

function alertMisunderstood() {
  // if count doesn't equal todo list length, the command was misunderstood
  if (count.value !== store.todoList.length) {
    deepgramStatus.value = "I didn't catch that"
  }
}

I don't want the phrase "I didn't catch that" to remain on the screen forever. I'll have it disappear after a second and return to "Listening":

function alertMisunderstood() {
  // if count doesn't equal todo list length, the command was misunderstood
  if (count.value !== store.todoList.length) {
    deepgramStatus.value = "I didn't catch that"
    setTimeout(() => {
      deepgramStatus.value = 'Listening'
    }, 1000)
  }
}

Now, everything in the watcher is set up to make sure that when a user gives a command, it is either added to the to-do list, or reported back with a message to the user that the command was misunderstood.

watch(utterance, () => {
  if (utterance.value !== '') {
    count.value++
    addTodo(utterance.value)
    alertMisunderstood()
  }
})

Delete To-Do Item With Voice-Control

The hard part of this is done. I walked through the step-by-step logic to analyze voice-control command strings. Now that I want to delete an item, I can use the same logic.

I'll write out the pseudo-code and supply my logic. I won't go through it step-by-step again since the only thing that will be different is writing the step to remove an item from the store to-do list instead of add an item.

Anyone following along with this post and building the voice-control feature could now take some time to write a deleteTodo function. I recommend copying the pseudo-code below (the commented-out steps) and then writing each step of the logic.

Pseudo-Code:

function deleteTodo(command) {
  // Create an array of Regular Expression words to identify in the text, such as [/^delete/];
  // Loop through regex values using .find
  // Turn the command into a standardized string - lower case, no final punctuation, trim whitespace
  // Check if the command string contains the regex value using .test()
  // If so, remove the command phrase DELETE to create new string that is just the todo item
  // Loop through store to-do list and for each item, check if that item matches the new string that is just the todo item
  // if item in store todo list matches, delete from store to-do list
  // reset the count of uttered commands to match the length of the todo list array in the store
}

watch(utterance, () => {
  // watch for a command to run delete logic
})

Here is the code I wrote for this voice-controlled delete logic:

function deleteTodo(command) {
  const deleteRegEx = [/^delete/]
  // clean up utterance
  const item = standardizeUtterance(command)
  deleteRegEx.find((reg) => {
    if (reg.test(item)) {
      // remove command phrase DELETE
      const todo = removeCommandPhrase(item, reg)
      store.todoList.forEach((storeTodo) => {
        // if item in store todo list matches this utterance,
        if (storeTodo.item === todo) {
          // delete from store
          store.deleteTodo(storeTodo.id)
          // reset count
          count.value = store.todoList.length
        }
      })
    }
  })
}

And in the watcher:

watch(utterance, () => {
  if (utterance.value !== '') {
    count.value++
    addTodo(utterance.value)
    deleteTodo(utterance.value)
    alertMisunderstood()
  }
})

Check-Off To-Do Item With Voice-Control

The logic to check off an item on the list is exactly the same as the logic to delete an item, except instead of calling store.deleteTodo(storeTodo.id), I will call store.toggleCompleted(storeTodo.id).

Here is the logic to check-off an item with voice-control:

function checkOffTodo(command) {
  const checkOffRegEx = [/^check off/]
  const item = standardizeUtterance(command)
  checkOffRegEx.find((reg) => {
    if (reg.test(item)) {
      const todo = removeCommandPhrase(item, reg)
      store.todoList.forEach((storeTodo) => {
        if (storeTodo.item === todo) {
          // toggle completed in store:
          store.toggleCompleted(storeTodo.id)
          count.value = store.todoList.length
        }
      })
    }
  })
}

And the watcher:

watch(utterance, () => {
  if (utterance.value !== '') {
    count.value++
    addTodo(utterance.value)
    deleteTodo(utterance.value)
    checkOffTodo(utterance.value)
    alertMisunderstood()
  }
})

Conclusion

That concludes this post on how to add voice-control to a Vue 3 to-do list app. It was well worth the trouble to turn a classic to-do list project into a more exciting voice-based app that uses Deepgram's speech-to-text API.

Feel free to reach out with questions on Twitter. Happy coding!

Build a To-do List App with Pinia and Vue 3

SandraMartin — Fri, 15 Apr 2022 15:36:37 +0000

I was building a Vue 3 project for my recent blog series on how to build a full-stack live streaming web app. I wanted to use Vuex to manage some global state properties. It was my first time using Vuex with Vue 3 since I began my journey to learn the Composition API.

When I arrived at the Vuex documentation page, I saw this:

Well, that was a surprise! I had been hearing the word "Pinia" in relation to Vue but didn't know exactly what it was. Pinia is now the official state management library for Vue!

I pushed onwards with using Vuex in that project but made a mental note to come back soon to Pinia to find out what it is all about.

Soon is now! Today I will learn a little about Pinia by building a to-do list. I'll show how I build it and provide some of my thoughts about the experience. Let's dive in!

The Project

Here is a screenshot of the final project. It's a to-do list that lets me add, delete, and check off an item on the list.

The project repo can be found here.

Getting Started with Pinia

I'll create my Vue project (making sure to select Vue 3 since I want to use the Composition API). Pinia also works with Vue 2, but I've personally gone totally in on Vue 3 (and haven't looked back - check out my series on Vue 3 to read about my journey).

vue create todo-pinia

After I cd into the project folder, I'll install pinia:

npm install pinia

Then I'll go into the main.js file and import createPinia. This creates a Pinia instance to be used by my application. The .use() tells the Vue app to install Pinia as a plugin.

import { createApp } from 'vue'
import { createPinia } from 'pinia'
import App from './App.vue'

createApp(App).use(createPinia()).mount('#app')

In the src folder, in components, I'll create the three components that will make up my todo list app - TodoApp.vue (the parent component), TodoForm.vue (a child component), and TodoList.vue (another child component).

Here is the plan for how these components will be organized in the browser:

In each component, I can quickly scaffold out the basic code structure for my template and script. I do that with an extension in VS Code called Vue VSCode Snippets. Since I have that, I just type the letters vbase-3, and the code writes itself for me:

Now I'll import each component to where it needs to be -TodoForm.vue and TodoList.vue into the TodoApp.vue - and I'll import the TodoApp.vue component into App.vue. I like to write the name of the component in each to start so I can see them on the screen.

Here's my screen now. The layout is there, but no logic or styles yet:

Pinia - What is it?

Next, I'll create a store and set up my global state with Pinia.

The concept of a Pinia store is the same as it is for Vuex or Redux - it is a place to hold global state, and it makes it easy for any component in the project to track changes to that global state.

Not all state needs to go in the store - just state properties that I want to make available throughout the app. This is especially useful when I want to share state between two sibling components like the TodoForm.vue and TodoList.vue because I can avoid sending props down ('prop drilling') and emitting events up through the parent.

Define a Pinia Store

I will create a store folder in src, and in the folder, I'll make a file called useTodoListStore.js. I'm naming it starting with the word 'use' because a common convention of Vue 3, both for Pinia store files and for Vue composables, is to start the file name with 'use'.

I can have as many stores as I want; in fact, I should have separate stores for separate logical concerns, similar to how Vue 3 composables are built around distinct logical concerns. Each store should be in a different file.

However, since this is such a small project, I only need one store - one store for the to-do list logic.

I will first import the defineStore function from Pinia. Under the hood, this is going to create the useStore function that I will need in my components to retrieve the store I made.

import { defineStore } from 'pinia'

I set it to a const and use the keyword export since I'll need to be able to import it into my components.

This defineStore function will take two arguments: a string (the unique name of the store) and an object (options such as state, getters, and actions).

import { defineStore } from 'pinia'

export const useTodoListStore = defineStore('todoList', {
  // state
  // getters
  // actions
})

State, Getters, and Actions

The options that I pass to the defineStore function are my store's state, getters, and actions. Unlike Vuex, there is no longer the need for mutations. This makes me happy!

I always found mutations confusing because it felt like I was repeating myself when I had to write an action to commit a mutation, which would then make the state change. Pinia has gotten rid of that middleman, and instead, the flow is just action -> change state.

I already have a mental model around the way methods, data, and computed work in Vue 2. The methods make stuff happen, the data contains my state properties, and the computed returns an automatically updated property that has had a calculation performed on it.

Pinia's options follow the same mental model - I can think of the state as being like data in the Vue Options API, the actions like methods, and the getters like computed properties.

I really like this change, and it's one of the first things that made me think, "Wow, I think I'm really going to like Pinia!"

Create Initial State

Now I'll start creating a global state object in my useTodoListStore.

The state is actually a function, and it's recommended that I use an arrow function (this is because Pinia has excellent Typescript integration, and using an arrow function will allow Typescript inference to work on the state properties).

I'll add a todoList property, which will be an array meant to contain each to-do item (each item is going to be an object, but there's nothing in the todoList array at the moment).

import { defineStore } from 'pinia'

export const useTodoListStore = defineStore('todoList', {
  state: () => ({
    todoList: [],
  }),
})

Actions - Add and Delete an Item

I can also set up my first action. I know the main logic to start will be adding an item to the to-do list. I'll write a function addTodo that will perform the logic of pushing an item object into the todoList array.

Individual actions are methods within the actions object in the store.

I will also add an id property to state since I will want each item to have an id that increments each time a new item is pushed into the toDoList array:

import { defineStore } from 'pinia'

export const useTodoListStore = defineStore('todoList', {
  state: () => ({
    todoList: [],
    id: 0,
  }),
  actions: {
    addTodo(item) {
      this.todoList.push({ item, id: this.id++, completed: false })
    },
  },
})

Maybe while I'm here, I should go ahead and write an action to delete an item from the to-do list since I know I'll want to have a delete feature. Under the last line of code in the addToDo action, I'll add a deleteTodo:

deleteTodo(itemID) {
  this.todoList = this.todoList.filter((object) => {
    return object.id !== itemID;
  });
},

Input Form to Add an Item

I'll jump back into the TodoForm.vue component now. I want to write a form to enter a to-do item. I'll use the dev-tools to check that the item is getting into the state I set up in the Pinia store.

In the template, I'll create the basic form:

<!-- TodoForm.vue -->

<template>
  <form @submit.prevent="">
    <input v-model="todo" type="text" /><button>Add</button>
  </form>
</template>

The input has a v-model="todo" which I'll connect to a ref in the script to make this property reactive so it updates as the user types the item into the input:

// TodoForm.vue

<script>
import { ref } from "vue";
export default {
  setup() {
    const todo = ref("");
    return { todo };
  },
};
</script>

I haven't added a method yet for the @submit event listener because I need to set up the logic in the script first. The submit button is going to trigger a function to add an item to the todo list, so I'll need to somehow invoke the addTodo action in the store.

Access Pinia Store from a Component

To use a Pinia store in a component, I need to import the store and then set a const store to the invoked store function:

// TodoForm.vue

import { useTodoListStore } from '@/store/useTodoListStore'
export default {
  setup() {
    const todo = ref('')
    // use Pinia store:
    const store = useTodoListStore()

    return { todo }
  },
}

Now I will have access to state, actions, and getters in the store through that const store.

I'll write a method in the TodoForm.vue component that will be triggered when the submit button is clicked. I want that method to do two things: add an item to the todoList array in the store, and clear the todo ref so it returns to being an empty string after the item is added to the list:

// in setup function in script in TodoForm.vue:

function addItemAndClear(item) {
  if (item.length === 0) {
    return
  }
  // invokes function in the store:
  store.addTodo(item)
  todo.value = ''
}

And I'll make sure that function is added to the form's @submit event listener in the template:

<form @submit.prevent="addItemAndClear(todo)">

I'll type npm run serve in the terminal to start up the Vue development server.

Now I can open the Vue dev-tools and see that the item is being added to the todoList array in the store.

Reactive Properties in Pinia

In the previous section, I used an action from the Pinia store - addTodo - in my todoForm.vue component. In this section, I'll use a state property in the todoList.vue component, and I need it to be reactive to changes that might happen. I'll be using it in the component template, and it has to be reactive so it updates in sync with the state change.

There's an important function I'll want to use that comes with the Pinia library - storeToRefs. Each to-do list item displayed in the todoList component will actually come from the store, and since the store's state is an object, I will use this helper method to destructure the returned object without losing reactivity. It is similar to Vue 3's utility function toRefs. I'll demonstrate its usage as I build the next feature.

Todo List - Show Item

I want access to the todoList that's in the store (which now has data to represent the items I've added to the list), so in the todoList.vue component I'll need to bring in the store, just like I did in todoForm.vue. I'll also set const store to the invoked store function.

Then I need to wrap the todoList property that I want to pull from the store in the function storeToRefs:

<script>
import { useTodoListStore } from "../store/useTodoListStore";
import { storeToRefs } from "pinia";
export default {
  setup() {
    const store = useTodoListStore();
    // storeToRefs lets todoList keep reactivity:
    const { todoList } = storeToRefs(store);

    return { todoList };
  },
};
</script>

Now I can use todoList in my template, and it will stay in sync with the store. I'll write a v-for loop to create the list:

<template>
  <div v-for="todo in todoList" :key="todo.id">
    <div>{{ todo.item }}</div>
  </div>
</template>

And the list is displaying now:

To-do List - Mark as Completed

I want to add some styles to each item to show if the to-do item has been completed.

First, I need the logic to toggle an item to be complete or not complete. Right now, in the store, each item that is added to the list also has a completed property set to false:

// useTodoListStore.js

this.todoList.push({ item, id: this.id++, completed: false })

I can write an action in the store to toggle that to true:

toggleCompleted(idToFind) {
      const todo = this.todoList.find((obj) => obj.id === idToFind);
      if (todo) {
        todo.completed = !todo.completed;
      }
    },

In the todoList.vue component, I'll add a checkmark emoji as a span to the template with an event listener to listen for a click on the checkmark. The Unicode is ✔ for a checkmark.

<div v-for="todo in todoList" :key="todo.id">
    <div>
      <span>{{ todo.item }}</span>
      <span @click.stop="toggleCompleted(todo.id)">&#10004;</span>
    </div>
  </div>

However, I need to make sure that I have brought toggleCompleted into the component. Since it's an action method and not a reactive state property, I won't use storeToRefs for toggleCompleted:

<script>
import { useTodoListStore } from "../store/useTodoListStore";
import { storeToRefs } from "pinia";
export default {
  setup() {
    const store = useTodoListStore();
    const { todoList } = storeToRefs(store);
    // destructuring action method doesn't require using storeToRefs:
    const { toggleCompleted } = store;

    return { todoList, toggleCompleted };
  },
};
</script>

To add the styles, I first will add a dynamic class to the to-do item span in the template:

<span :class="{ completed: todo.completed }">{{ todo.item }}</span>

And CSS to change the look of the item as it is toggled true and false:

/* CSS Styles */

.completed {
  text-decoration: line-through;
}

To-Do List - Delete Item

I had already added the deleteTodo function to the store, so I can jump into writing the delete feature in the todoList.vue component.

I'll do the same thing I did in the previous section, bringing in the store's action deleteTodo and using a cross mark emoji for the delete button. I won't explain every step since I just need to repeat what I did in the previous section for marking an item complete, but this time hooking it up to the delete action. But I'll show the code.

Here's the todoList.vue component after I added the delete feature:

// todoList.vue

<template>
  <div v-for="todo in todoList" :key="todo.id">
    <div>
      <span :class="{ completed: todo.completed }">{{ todo.item }}</span>
      <span @click.stop="toggleCompleted(todo.id)">&#10004;</span>
      <span @click="deleteTodo(todo.id)">&#10060;</span>
    </div>
  </div>
</template>

<script>
import { useTodoListStore } from "../store/useTodoListStore";
import { storeToRefs } from "pinia";
export default {
  setup() {
    const store = useTodoListStore();
    const { todoList } = storeToRefs(store);
    const { toggleCompleted, deleteTodo } = store;

    return { todoList, toggleCompleted, deleteTodo };
  },
};
</script>

<style>
.completed {
  text-decoration: line-through;
}
</style>

And here is the store now that I have all the logic working:

// useTodoListStore

import { defineStore } from 'pinia'

export const useTodoListStore = defineStore('todoList', {
  state: () => ({
    todoList: [],
    id: 0,
  }),
  actions: {
    addTodo(item) {
      this.todoList.push({ item, id: this.id++, completed: false })
    },
    deleteTodo(itemID) {
      this.todoList = this.todoList.filter((object) => {
        return object.id !== itemID
      })
    },
    toggleCompleted(idToFind) {
      const todo = this.todoList.find((obj) => obj.id === idToFind)
      if (todo) {
        todo.completed = !todo.completed
      }
    },
  },
})

I've finished a barebones to-do list app with Pinia, minus styling. This code is available on the 'just-pinia' branch of my project repo for anyone who would like to see it in its entirety.

Bonus Section: Add Typescript

One of the best features of Pinia is that it works very well with Typescript. I first chose to build the to-do list without Typescript so I could just focus on how to use Pinia, but I also want to demonstrate how it works with Typescript since that is a huge advantage of Pinia.

Setting up Vuex with Typescript was always challenging for me because of the need to create custom complex wrappers. It wasn't easy to just dive in.

But with Pinia, I don't have to do that. I can just add Typescript to my project and start using it.

I'll add Typescript to my existing project with this command:

vue add Typescript

When it prompts me to make some choices, I'll be sure to say yes to "Convert all .js files to .ts". That way it will turn the store file into a .ts file.

Then I'll delete the HelloWorld file because I don't need that. I might need to delete one of the extends properties from the .eslintrc.js file.

I'll go to the store file and see that Typescript is pointing out all the missing types I need to add.

I'm not going to go through how to use Typescript since this blog post isn't meant to teach how to write Typescript. But I'll add the types and show how my store looks after I revise it to include Typescript:

import { defineStore } from "pinia";

interface ToDoItem {
  item: string;
  id: number;
  completed: boolean;
}

export const useTodoListStore = defineStore("todoList", {
  state: () => ({
    todoList: [] as ToDoItem[],
    id: 0,
  }),
  actions: {
    addTodo(item: string) {
      this.todoList.push({ item, id: this.id++, completed: false });
    },
    deleteTodo(itemID: number) {
      this.todoList = this.todoList.filter((object) => {
        return object.id !== itemID;
      });
    },
    toggleCompleted(idToFind: number) {
      const todo = this.todoList.find((obj) => obj.id === idToFind);
      if (todo) {
        todo.completed = !todo.completed;
      }
    },
  },
});

In the components, I'll need to add lang="ts" to the script and import defineComponent. The export will need to be wrapped in the defineComponent function.

<script lang="ts">
import { defineComponent } from "vue";
export default defineComponent({
...
});
</script>

And that's how I would add Typescript to my project after-the fact; although I highly recommend starting the project from the beginning with Typescript, since it will help with the developer experience of catching errors and thinking about types.

The Typescript version of the to-do list can be found in my repo on the branch called pinia-typescript.

Conclusion

I went through creating a to-do list using just Pinia and then I also showed how to build one with Typescript. I've since added styles and an alert feature to the application, and the most updated code can be found on the main branch of the project repo.

I hope this blog post has been helpful. I'm very excited about Pinia because of how straightforward it was to jump in and start using, especially with Typescript.

If you have any questions, feel free to reach out on Twitter!

Asynchronous Logic to Write a Vue 3 and Deepgram Captions Component

SandraMartin — Fri, 01 Apr 2022 19:52:00 +0000

This is the final post of my series, "How to Build a Live Streaming Web Application with Amazon IVS and Deepgram."

For today's post, I recommend getting a Deepgram API key to use in this project, which anyone can get by heading to the Deepgram console.

Introduction

Today's post will cover how to use Vue 3 composables to power a text-captions component that integrates with Deepgram's speech-to-text API. Some of the things I'll cover today are:

Using async and await to write a composable that fetches a temporary API key from Deepgram.
Using Vue 3's watch method to react to data that is updating in real-time as Deepgram sends a text transcription back through a browser WebSocket.
Writing logic that is sensitive to the order things occur - i.e., asynchronous logic that flows between the component and the composable.

This post assumes some knowledge of Vue 3, in particular Vue composables. For a refresher on Vue 3, check out my series Diving Into Vue 3.

Today I will build the AudioCaptions.vue component. (For the VideoPlayer component, see my previous post in the series.) Here is the page with minimal styling. I've put a red box around the AudioCaptions.vue component:

Where it says "Deepgram Not Connected," there will be text captions that display in real-time along with the video stream.

Here is a diagram of what I will build today:

This feature will rely on Vue 3's Composition API, especially Vue Composables, to put Deepgram captions on the screen.

Composables and Asynchronous Logic

Composables are a feature of the Vue 3 Composition API; custom composables are the ones I build myself with the intention of encapsulating reusable, stateful logic.

I feel like it is somewhat of an art learning how to write composables. The key to writing them well is making them as generic as possible so that they can be reused in many contexts.

For example, I could write a function that does everything I need it to do to create text captions on the screen using the Deepgram API - the function would include logic to get an API key, turn on the browser microphone, get the audio stream from the microphone, and then send the stream through a WebSocket. I could call the composable useDeepgram.

However, there are several logical concerns in that one large useDeepgram function that could be broken out into other composable functions. While it's easier just to write it all in one file, it means I could only use it in situations that are exactly like this project.

The challenge of breaking it apart is that the logic to get the transcription from Deepgram depends on certain things happening first, such as the API key arriving and the microphone being turned on. When I break that logic apart into separate functions, I have to be conscious of the order that those functions run, the state that gets updated in multiple functions (and making sure the functions stay in sync), and the conventions for writing asynchronous logic. Not to mention the challenge of updating the component in real-time with the data that comes through the WebSocket.

The point is that writing composables in the real world can be challenging, so learning some strategies for dealing with more complicated situations, particularly asynchronous logic, is worth it. Because the beauty of composables is that if you write them well, you have a clean, reusable function that you'll return to again and again.

Composable Using Async and Await

Here is the AudioCaptions.vue component right now, before I add the feature logic:

<template>
  <div>
    <p>Status Will Go Here</p>
  </div>
</template>

<script>
export default {
  setup() {
    return {};
  },
};
</script>

In the template where it says "Status Will Go Here," I plan to add a reactive variable. That value will update to show the audio captions after everything is working. For now, I've just hard-coded that text.

useDeepgramKey Composable

The first composable I'm going to write will be called useDeepgramKey.js, and its purpose will be to fetch a temporary API key. If I fetch a temporary API key from Deepgram, I can use the key in the browser and not worry about exposing the key since the key will expire almost immediately. Read more about this feature in a blog post that Kevin wrote about protecting your Deepgram API key.

On the backend, I have set up an endpoint to receive the fetch request from the composable. That endpoint can be seen in the server.js file in my repo here.

Now I'll create the useDeepgramKey.js composable.

Tip #1 - Use async and await to write a composable that returns a promise.

I will do three things to make this composable run asynchronously:

Write the composable as an async function using export default async to make the composable itself know to wait for the fetch request to finish.
Encapsulate the fetch request in its own async function called async function getKey(), which is called inside the composable using the await keyword.
In the component AudioCaptions, use a .then() when I call the composable so that I get access to the returned state after the Promise completes.

Here is the composable to start. The key will update to be the API key when that arrives from the backend, and DGStatus will update with a message if there is an error.

import { ref } from "vue";
let key = ref("");
let DGStatus = ref("Deepgram Not Connected");

export default async function useDeepgramKey() {
  return { key, DGStatus };
}

Now I'll write an async function that will perform all the logic of getting the temporary key. I'll name it getKey() and I will use a try-catch block to make the fetch request and handle any errors:

async function getKey() {
  try {
    const res = await fetch("http://localhost:8080/deepgram-token", {
      headers: { "Content-type": "application/json" },
    });
    if (res) {
      const response = await res.json();
      // update with temporary api key:
      key.value = response.key;
      return key;
    }
  } catch (error) {
    if (error) {
      // update to show error message on screen:
      DGStatus.value = "Error. Please try again.";
    }
  }
}

To make sure this runs, I need to call the function in the composable. I will add await getKey() to the async function that will be exported. Using await is to go along with async that I used on the composable function itself. These two keywords together tell the composable that it must wait until the getKey function resolves.

Here is the composable in its entirety:

import { ref } from "vue";
let key = ref("");
let DGStatus = ref("Deepgram Not Connected");

async function getKey() {
  try {
    const res = await fetch("http://localhost:8080/deepgram-token", {
      headers: { "Content-type": "application/json" },
    });
    if (res) {
      const response = await res.json();
      // update with temporary api key:
      key.value = response.key;
      return key;
    }
  } catch (error) {
    if (error) {
      // update to show error message on screen:
      DGStatus.value = "Error. Please try again.";
    }
  }
}

export default async function useDeepgramKey() {
// call function:
  await getKey();
  return { key, DGStatus };
}

I can console.log(key.value) to make sure the key is arriving successfully.

Then I'll go back to AudioCaptions.vue to wire up a reactive reference that will update to show the error status message if the key does not arrive. I'll create a ref called deepgramStatus and replace the hardcoded "Status Will Go Here" with that variable.

<template>
  <div>
    <p>{{ deepgramStatus }}</p>
  </div>
</template>

<script>
import { ref } from "vue";
export default {
  setup() {
    let deepgramStatus = ref("Deepgram Not Connected");
    return { deepgramStatus };
  },
};
</script>

I also need to call the composable function in the component. If I don't call it, the logic won't run. Since it is a promise, I will use a .then() method on it to get the result, which will be the key and DGStatus values. I only need the DGStatus, so I'll set that to the deepgramStatus ref.

Here's the AudioCaptions.vue script now:

<script>
import { ref } from "vue";
import useDeepgramKey from "@/composables/useDeepgramKey";
export default {
  setup() {
    let deepgramStatus = ref("Deepgram Not Connected");

    // use .then() to wait for promise resolution
    useDeepgramKey().then((res) => {
      deepgramStatus.value = res.DGStatus.value;
    });

    return { deepgramStatus };
  },
};
</script>

If I want to see the error message, I can delete a character in the fetch request URL, making it http://localhost:8080/deepgram-toke, which is incorrect. That will cause the fetch request to fail, and I'll see the error message.

Composable That Relies on Asynchronous Events in Other Composables

Now I will begin to tackle the useDeepgramSocket composable. This composable will take an audio stream from the browser microphone and send it to Deepgram by way of a browser WebSocket. It relies on two other composables to do this:

useDeepgramKey - I need to get the temporary API key from the composable I just made, useDeepgramKey, to send it in the request to Deepgram; otherwise, Deepgram won't be able to fulfill the request.
useMicrophone - I need to get an audio stream from the browser microphone. That audio data will be sent to Deepgram to be transcribed into text that will be put onto the screen as captions.

I haven't created the useMicrophone composable yet, so I'll make a quick detour right now to write that composable.

useMicrophone Composable

The useMicrophone composable will rely on the browser Media Stream API and the getUserMedia method to request permission to use the browser microphone of the user and pull the audio from it. Since there are several other blog posts in Deepgram Docs about this nifty API, I won't go into detail about how it works. Check out Brian's post for a general introduction to it.

This composable is also going to use an async function since the getUserMedia method requires waiting for the user to give permission to use the microphone. The time involved means that this method returns a promise. I already know how to write this type of composable since I just did it in the last section.

I'll make the composable an async function and I'll also write the logic to get the audio stream as an async function. Here is the composable in its entirety:

async function getAudio() {
  try {
    const mediaStream = await navigator.mediaDevices.getUserMedia({
      audio: true,
    });
    const mediaRecorder = new MediaRecorder(mediaStream, {
      audio: true,
    });
    return mediaRecorder;
  } catch (e) {
    console.error(e);
  }
}

export default async function useMicrophone() {
  const microphone = await getAudio();
  return { microphone };
}

Now it's ready for me to use in the next composable I will write.

useDeepgramSocket Composable

First, I'll import this composable into AudioCaptions.vue and call it. That way, everything I write in useDeepgramSocket will run and I can check my progress as I build this composable.

<script>
import { ref } from "vue";
import useDeepgramKey from "@/composables/useDeepgramKey";
import useDeepgramSocket from "@/composables/useDeepgramSocket";
export default {
  setup() {
    let deepgramStatus = ref("Deepgram Not Connected");

    useDeepgramKey().then((res) => {
      deepgramStatus.value = res.DGStatus.value;
    });

    // call this so the composable runs as I work on it
    useDeepgramSocket();

    return { deepgramStatus };
  },
};
</script>

I know I need to have access to the temporary API key from useDeepgramToken and to the microphone from useMicrophone. I will start by setting up my composable to show that I have access to them within the same scope.

Both composables return a promise. That means I will need to use syntax that will make the functions run but wait for the promise to resolve before moving on to the next thing.

Tip #2 - Use `.then()` to chain each composable that returns a promise to run asynchronously if returned values need to be in the same scope

Here's what I mean:

import useDeepgramKey from "./useDeepgramKey";
import useMicrophone from "./useMicrophone";

export default function useDeepgramSocket() {

  // chain .then() methods for each composable:
  useDeepgramKey().then((keyRes) => {
    useMicrophone().then((microphoneRes) => {

      let apiKey = keyRes.key.value;
      let microphone = microphoneRes.microphone;

      console.log(apiKey);
      console.log(microphone);

      // WEBSOCKET FUNCTION WILL GO HERE

    });
  });
  return {};
}

I have named the result argument in each .then() a name that shows which composable they came from - keyRes and microphoneRes, which makes it easy for me to see what each of them represents. The keyRes is a ref, so I must drill all the way down to the .value property. The microphoneRes is a Vue 3 readonly property, which is why I don't have to drill down as far.

Now that I have the values, I can write a function that encapsulates the logic to open the WebSocket.

openDeepgramSocket Function

I will write a function called openDeepgramSocket that will do the following:

Create the socket with new WebSocket(URL, deepgram protocols).
Open the socket with socket.onopen. When it opens, I'll add an event listener to the microphone to take in the audio stream and send it through the socket.
Have socket.onclose listen for when the channel closes.

I will also create a reactive reference called DGStatus_socket to update the status of the transcription along the way. That value will be returned to the AudioCaptions.vue component as the text captions.

Here is the function:

function openDeepgramSocket(apiKey, microphone) {
  const socket = new WebSocket(
    "wss://api.deepgram.com/v1/listen?punctuate=true",
    ["token", apiKey]
  );

  socket.onopen = () => {
    if (microphone.state != "recording") {
      DGStatus_socket.value = "Connected to Deepgram";
      console.log("Connection opened.");

      microphone.addEventListener("dataavailable", async (event) => {
        if (event.data.size > 0 && socket.readyState == 1) {
          socket.send(event.data);
        }
      });

      microphone.start(200);
    }
  };

  socket.onmessage = (message) => {
    const received = JSON.parse(message.data);
    const transcript = received.channel.alternatives[0].transcript;
    if (transcript && received.is_final) {
      DGStatus_socket.value = transcript + "";
      // shows the transcript in the console:
      console.log(DGStatus_socket.value);
    }
  };

  socket.onclose = () => {
    console.log("Connection closed.");
  };
}

I have to make sure to call the function in the composable:

export default function useDeepgramSocket() {
  useDeepgramKey().then((keyRes) => {
    useMicrophone().then((microphoneRes) => {
      let apiKey = keyRes.key.value;
      let microphone = microphoneRes.microphone;

      // Call function:
      openDeepgramSocket(apiKey, microphone);
    });
  });
  return {};
}

Now I see the transcript coming back to me because I have added a console.log to show it:

I'm ready to put that transcript onto the screen as the captions!

Vue watch to Update Transcript Status

I will use the reactive reference DGStatus_socket in the composable useDeepgramSocket to update the captions in AudioCaptions.vue. To do that, I need to return it from the composable and then destructure it in the component AudioCaptions.vue.

Here is the useDeepgramSocket composable where I return the DGStatus_socket value (excluding the large openDeepgramSocket function):

import { ref } from "vue";
import useDeepgramKey from "./useDeepgramKey";
import useMicrophone from "./useMicrophone";

// create status ref
let DGStatus_socket = ref("");

function openDeepgramSocket(apiKey, microphone) {
...
}

export default function useDeepgramSocket() {
  useDeepgramKey().then((keyRes) => {
    useMicrophone().then((microphoneRes) => {
      let apiKey = keyRes.key.value;
      let microphone = microphoneRes.microphone;

      openDeepgramSocket(apiKey, microphone);
    });
  });

  // return status ref to component
  return { DGStatus_socket };
}

In AudioCaptions.vue, I destructure the DGStatus_socket so I have access to it:

const { DGStatus_socket } = useDeepgramSocket();

Is it working? Not yet. I have to update the deepgramStatus ref that is connected to the template if I want to see those captions on the screen.

Tip #3: Use watch to update a value in the component and trigger a side effect in-sync with that change

According to the Vue documentation, watch is used in "cases where we need to perform 'side effects' in reaction to state changes - for example, mutating the DOM or changing another piece of state based on the result of an async operation."

This example of putting the captions on the screen fits that description exactly. I want the deepgramStatus value to update if DGStatus_socket from the composable useDeepgramSocket changes, and I want that state change to trigger the effect of the text updating in the DOM.

I will add a watcher to the AudioCaptions component:

watch(DGStatus_socket, () => {
      deepgramStatus.value = DGStatus_socket.value;
    });

And this is what the component in its entirety looks like now:

<template>
  <div>
    <p>{{ deepgramStatus }}</p>
  </div>
</template>

<script>
import { ref, watch } from "vue";
import useDeepgramKey from "@/composables/useDeepgramKey";
import useDeepgramSocket from "@/composables/useDeepgramSocket";
export default {
  setup() {
    let deepgramStatus = ref("Deepgram Not Connected");

    useDeepgramKey().then((res) => {
      deepgramStatus.value = res.DGStatus.value;
    });

    const { DGStatus_socket } = useDeepgramSocket();

    watch(DGStatus_socket, () => {
      deepgramStatus.value = DGStatus_socket.value;
    });

    return { deepgramStatus };
  },
};
</script>

And with that, I have my captions powered by Deepgram! Check out the code for this post on my repo branch deepgram-composables.

Conclusion

Today I built the final component of my project, a full-stack video streaming application with text captions.

This post contained the barebones logic for the captions feature, but in my actual project, I have added styling to improve the user experience, and I've added buttons to turn the captions on or off. Check out the repo here.

Here is the final project:

It's been a great experience learning about Amazon IVS and Deepgram, and I've gotten the chance to get a better taste of how to take advantage Vue 3's composition API.

If you enjoyed this series, please follow me on Twitter to receive updates on future series I have in the works!

How to Write Vue 3 Composables for a Third-Party API Integration

SandraMartin — Fri, 25 Mar 2022 14:47:20 +0000

This post is a continuation of the series "How to Build a Live Streaming Web Application with Amazon IVS and Deepgram." If you want to start at the beginning, check out the first post in the series.

Composables

In Vue.js, the term 'composables' refers to composition functions, a key feature of Vue 3's Composition API. While the API itself includes many composition functions that are core to its design, such as the setup() function or the reactivity functions ref() and reactive(), composables are those composition functions that I write myself to be used throughout my own project as needed.

Composables are functions that encapsulate stateful logic, which means they are like little packages that are focused around performing one logical concern, and they keep track of state that changes due to the function running.

For example, I could write a composable function that toggles a menu open or closed. That logic could be used throughout an application, and it would need to keep track of the state of the menu being opened or closed. I would just need to import the composable into whatever file I need it and run the function.

Composable Example `useMenu.js`

import { readonly, ref } from 'vue'

const isOpen = ref(false)
const toggleMenu = () => {
  isOpen.value = !isOpen.value
}

export default function useMenu() {
  return {
    isOpen: readonly(isOpen),
    toggleMenu,
  }
}

Today I am going to introduce how to write a composable in Vue 3 to bring in the Amazon IVS video player SDK and to connect to my Amazon IVS streaming channel.

If you have wanted to learn more about how to write Vue 3 composables to use third-party technologies, this will be useful to you. In this post, I'll cover:

Writing a Vue composable to bring in an external script
Writing a Vue composable that is dependent on another composable, utilizing the Javascript function setInterval
Amazon IVS integration using Vue 3 composables

Background to the Series

In the first post in the series, I explained how to build a live streaming web application using just Vanilla JS and HTML. That post covers setting up an Amazon IVS account, which is necessary to get the third-party technology example (Amazon IVS) in today's post working. Setting up an Amazon IVS channel is necessary to build the second composable in this tutorial, but the first one can be built without doing that.

In the second post, I started building a full-stack application using Vue 3. I set up Vue-Router, Vuex, and a backend server which all helped me build a feature to restrict access to the streaming page of my site. Visitors to the site are required to enter a code, and that code is verified on the backend, resulting in the user being able to navigate to the streaming page. There is no need to read that post to be able to go through this tutorial on composables.

Today I'll build the video streaming page in Vue 3, focusing on building this feature with Vue composables. If you want a refresher on Vue composables, check out my post Reusability with Composables.

The code for today's tutorial can be found in this Github repo, the branch named "amazonIVS-composables."

Now I'll get into it!

Files Organization

This project has two main views, the landing page and the streaming page. Today I'll be working entirely in the streaming page. In my project, I have named this file StreamChannel.vue, and it is in the Views folder.

The StreamChannel.vue will eventually be made up of two components - one for the video player that relies on the Amazon IVS technology and one for the closed-captions that relies on the Deepgram speech-to-text technology. Today I'll only build the video player component.

I'll set up two folders to start - a components folder and a composables folder, both in the src folder. In the components folder, I'll create a VideoPlayer.vue component.

The composables folder is where I will put the composition functions that contain the logic that makes the video player work. A common practice in Vue 3 is to name composables so that they begin with 'use'. The name will identify what the composable does. The two composables I will be writing today are useIVSPlayer and useIVSChannel. Here is what they will do:

useIVSPlayer.js - this composable will bring in the Amazon IVS video player script so that the HTML video element is enhanced with the Amazon IVS Player Web SDK.
useIVSChannel.js - this composable will check at an interval if the player in the useIVSPlayer script has loaded, and if it has, it will create a connection to my Amazon IVS channel, updating state to show that the channel is connected.

My Github repo for this project shows how I have set up these folders.

VideoPlayer Component

The VideoPlayer.vue component will be a video player that shows the live stream. The Amazon IVS video player script looks for an HTML <video> element with a specific ID and then takes control of that element to bring in its own specially made video player with Amazon IVS optimizations. So the first composable I write will be a function that brings in the Amazon IVS player with a script.

In the VideoPlayer.vue file, I will start by writing the HTML I need in the Vue template so that I have a basic video player. I've given it a height and a width that I prefer, and the id="video-player" so that I can use that id later to bring in the Amazon IVS player. The attributes that the html <video> element supports are listed here.

<template>
  <div>
    <p class="status">AWS Channel {{ IVSStatus }}</p>
    <video
      width="520"
      height="440"
      id="video-player"
      controls
      playsinline
    ></video>
  </div>
</template>

The IVSStatus will be set to a ref property. For now, I'll make that property a string 'Is Not Connected,' but later, it will be hooked up to the status of the channel, and it will update to say 'Is Connected' when the channel is available.

Here is my setup function in the script block with that ref:

<script>
import { ref } from "vue";

export default {
  name: "VideoPlayer",
  setup() {
    let IVSStatus = ref("Is Not Connected");

    return { IVSStatus };
  },
};
</script>

The last thing I need to do to be able to see this video player is add the component to the StreamChannel.vue page in src/views/StreamChannel.vue:

<template>
  <div>
    <h1>Stream Channel</h1>
    <video-player />
  </div>
</template>

<script>
import VideoPlayer from "@/components/VideoPlayer";
export default {
  name: "StreamChannel",
  components: {
    VideoPlayer,
  },
};
</script>

Here is how the page will look:

Now I am ready to write the first composable, the useIVSPlayer.js composition function, which will do the logic to bring in the Amazon IVS player.

Composable to Bring in An External Script

The useIVSPlayer.js composable will bring a script into my StreamChannel.vue component. The docs at Amazon IVS say that I need this script so that the player is brought in:

<script src="https://player.live-video.net/1.8.0/amazon-ivs-player.min.js">

One way to bring in an external script is to add the script to the <head> in my index.html page:

<head>
    <meta charset="utf-8" />
    ...
    <script
      type="text/javascript"
      src="https://player.live-video.net/1.8.0/amazon-ivs-player.min.js"
    ></script>
  </head>

Then I can type "IVSPlayer" in the console, and I should see the module there.

If I choose this way to bring in the script, the module will be available on every page of my application. However, sometimes it is preferable to make a third-party technology only available on the page where it is needed. In that case, I need to remove that script from the <head> and bring it in a different way.

If I only want to bring in the script on the StreamChannel.vue page, I need to write logic to build out the script tag with the src of the player URL. I will use a Vue composable, which is just a Javascript function, to build out this logic. Writing it as a composable makes it reusable, so I can easily copy it into other projects or bring it into other components in my application as needed.

This useIVSPlayer.js composable will:

be a Promise, since I need to account for the small amount of time it will take to load the script
use createElement to create the script tag and setAttribute to add the src
append the script to the head with appendChild
use the global event listener onload to trigger the promise being resolved

Here is the composable:

export default new Promise((res) => {
  const script = document.createElement("script");
  script.setAttribute(
    "src",
    "https://player.live-video.net/1.6.1/amazon-ivs-player.min.js"
  );
  document.head.appendChild(script);
  script.onload = () => res();
  script.onerror = () => {
    throw "IVS PLAYER ERROR";
  };
});

I start with export default because I need to be able to import this logic into my VideoPlayer.vue component.

Now in VideoPlayer.vue I can import the composable. I am going to use a .then() method because useIVSPlayer is a promise. The .then() method will wait for the promise to resolve before doing whatever I write inside the .then().

For now, I will check that the player is available and console.log that it is ready. Later, I'll add logic inside the .then() to bring in my streaming channel.

Here is the setup function now in the VideoPlayer.vue component:

setup() {
    let IVSStatus = ref("Is Not Connected");

    useIVSPlayer.then(() => {
      if (window.IVSPlayer) {
        console.log("player loaded");
      }
    });
    return { IVSStatus };
  },

For now, I will keep IVSStatus as "Is Not Connected" because even though I have brought in the Amazon IVS player, I still need to hook up the video player to my channel stream. I'll do that in the next section.

Composable to Play Channel Stream

Now I want to build a composable that will load my channel stream into the IVS player that I just brought in. This composable will do the following:

Check that the IVS Player script is loaded and then create a new player that I can use for my stream.
Load my channel stream into the player by adding the playback URL.
Turn the player on with the play() method.
Check that the stream is connected and loaded. This will be done with setInterval since I don't know how long the delay might be.

First, I'll write my composable with an export default so I can import it into other files as needed. I'll also bring in ref from vue so I can track the state of the channel being loaded. I'll create a ref variable called playerIsLoaded and set it to false to start:

import { ref } from "vue";

export default function useCreateIVSPlayer() {
  let playerIsLoaded = ref(false);

  return { playerIsLoaded };
}

Everything I need to do in this composable is dependent on the IVS player (the one I brought in with the useIVSPlayer composable) being loaded. So I'll wrap all my logic in an if statement to check that it is loaded and supported.

if (IVSPlayer && IVSPlayer.isPlayerSupported) { 
    // all logic here
  }

I'll use the player SDK's method create() to create a player for my channel. Then I'll attach the player to the HTML video element in my VideoPlayer.vue component with the SDK's attachHTMLVideoElement() method and I'll use .load() to load my channel's playback URL. I'll use play() to play the channel stream:

const player = IVSPlayer.create();
    player.attachHTMLVideoElement(document.getElementById("video-player"));
    player.load(
      "PLAYBACK_URL"
    );
    player.play();

(The playback URL is unique to my channel, so it has to be taken from the Amazon IVS console. See my walkthrough in this post for more information.)

Now I need to check that the channel stream is loaded. This won't happen instantaneously, but it should load relatively quickly. I don't want to use a setTimeoutbecause I don't know how long it will take and I'm concerned about the idea of adding a super long timeout if I don't have to. I'll use setInterval to check for the stream being loaded.

setInterval is a DOM API method that repeatedly calls a function until some other trigger happens to turn it off. In this case, the other trigger will be the channel being loaded.

The way to turn it off is to use clearInterval. I'm going to assign setInterval to a variable called checkInterval. The callback of setInterval will run every 500 milliseconds. Inside that callback, it will check that the channel has loaded, and once it has, it will set playerIsLoaded to true and clear everything by passing checkInterval to clearInterval.

Here's the logic I just described:

let checkInterval = setInterval(() => {
      if (player.core.isLoaded) {
        playerIsLoaded.value = true;
        clearInterval(checkInterval);
      }
    }, 500);

I'll return the ref playerIsLoaded from the composable, so I have access to it in the VideoPlayer.vue component. I want to watch that value so that when it changes, the IVSStatus value in the VideoPlayer.vue template updates to show that the channel is connected.

Here is the composable in its entirety:

import { ref } from "vue";

export default function useIVSChannel() {
  let playerIsLoaded = ref(false);

  if (window.IVSPlayer && window.IVSPlayer.isPlayerSupported) {
    const player = window.IVSPlayer.create();
    player.attachHTMLVideoElement(document.getElementById("video-player"));
    player.load(
      "PLAYBACK_URL"
    );
    player.play();

    let checkInterval = setInterval(() => {
      if (player.core.isLoaded) {
        playerIsLoaded.value = true;
        clearInterval(checkInterval);
      }
    }, 500);

    return { playerIsLoaded };
  }
}

The last thing I need to do to get this working is go back to the VideoPlayer.vue component and run the composable function inside setup and update IVSStatus based on the channel being connected, which I'll do in the next section.

Run the Composable

In VideoPlayer.vue, I will run the useIVSChannel composable inside setup. Actually, I'll run it inside the .then() that I already wrote earlier, which will cause useIVSChannel to run after useIVSPlayer has resolved. (I have to remember to import useIVSChannel from the composables folder if I want to use it.)

useIVSPlayer.then(() => {
      const { playerIsLoaded } = useIVSChannel();

    });

I deconstruct playerIsLoaded off of useIVSChannel so that I can watch that reactive reference. I'll use Vue's watch method to make a side effect occur when the playerIsLoaded value changes to true (i.e., when the channel is connected). The side effect will be that the IVSStatus will update to "Is Connected":

watch(playerIsLoaded, () => {
        if (playerIsLoaded.value) {
          IVSStatus.value = "Is Connected";
        }
      });

Here is the entire script for the VideoPlayer.vue component:

<script>
import { ref, watch } from "vue";
import useIVSPlayer from "../composables/useIVSPlayer";
import useIVSChannel from "../composables/useIVSChannel";

export default {
  name: "VideoPlayer",
  setup() {
    let IVSStatus = ref("Is Not Connected");

    useIVSPlayer.then(() => {
      const { playerIsLoaded } = useIVSChannel();
      watch(playerIsLoaded, () => {
        if (playerIsLoaded.value) {
          IVSStatus.value = "Is Connected";
        }
      });
    });
    return { IVSStatus };
  },
};
</script>

However, nothing will happen in my video player in the browser if I have not turned on my stream. In the first post in this series, I showed how to set up OBS to use their software on my computer to create a stream, which I connected to an Amazon IVS channel.

If I turn on my stream in OBS, I should see myself in the video player in my application now!

Conclusion

Vue composables are useful for writing standalone logic that can be reused, but if one composable depends on another, it can be tricky. In this post, I showed how a composable can be written as a promise and how setInterval can be used to check if something the composable depends on has happened yet. These are two ways to write composables that may have asynchronous events occurring.

In the next post, I'll show how to write composables for using Deepgram's API to create text captions for my stream. I'll write a composable to use the browser MediaStreams API (a great example of a composable that can be reused in different contexts). I'll also show how to use fetch in a composable to get a token from the backend.

I hope you'll join me for the next post. Follow me on Twitter so you don't miss it!

Build a Live Streaming Web Application: Vue and Express Setup

SandraMartin — Fri, 18 Mar 2022 14:52:24 +0000

Introduction

In my last post, I introduced how to build a vanilla Javascript and HTML live streaming web application with text captions. It included a barebones frontend that was just one page - the video player and captions.

Today I'm going to start building a full-stack application using the same technology, except instead of vanilla Javascript, I will use Vue 3.

The full-stack application will be expanded from the barebones version to include features that are often necessary for real-world applications, such as:

a landing page that asks for a code to enter the site
a router (Vue Router) to implement routes and a navigation guard so users can't access beyond the entry page unless they enter the correct code
a state management library (Vuex) which will update state to allow access to the stream page if the correct code is entered
a backend server (node.js and express) with two endpoints, one for getting a Deepgram temporary key so I can keep my Deepgram API key secure, and one for verifying the secret code the user must input to enter the site

I recommend reading the first post to go through how to set up the video streaming technologies, Open Broadcaster Software, and Amazon IVS. And for an introduction to Vue 3, it might be worth checking out my series Diving Into Vue 3.

What I'll Build Today

In this post, I'll set up the router (Vue Router) and Vuex so that I can put a navigation guard on the entry page. I will also set up the backend server so I can keep my entry code secure.

Here is the diagram to show the data flow and structure for what I will build today. The focus will be on building the protected entry with a navigation guard. If it seems complicated in the diagram, it won't by the end of the post because I plan to walk-through building this feature step-by-step.

Setting up the Vue Project

After installing the Vue CLI, I can create a new Vue project with this command:

vue create NAME-OF-PROJECT

I'll be sure to choose 'Manually select features' so I can add some dependencies during the creation of the project:

These are the presets I selected. I definitely recommend selecting Router and Vuex since those are required for this project, but the others are just my personal preference:

I'll select Vue 3, and then I'll cd into the folder.

I can install all the rest of the dependencies now, or I can install them one by one as I need them. Here is a list of the dependencies I'll be using in this project:

vue-router@4 (already installed when I manually selected presets)
vuex@next (already installed when I manually selected presets)
express
dotenv
cors
body-parser
@deepgram/sdk

I can install them all at once with this command:

npm install vue-router@4 vuex@next express dotenv cors body-parser @deepgram/sdk

The Folders Structure

A Vue application that is created using the Vue CLI starts out with this structure:

The src contains the folders that will make up the frontend, and I will add two files to the root of the project which will be required for the backend. Those two files will be a server.js file and a .env file. The final structure can be seen here at the root level of the project in its GitHub repo.

SRC Folder

I'll go over the SRC folder in more detail because the way I organize this folder reflects my thinking about how I'll make the application work when considering how the pieces should fit together.

Here is the file structure of the SRC for the final project (these folders can be set up now or as I progress through building the project):

I'll go over the pieces starting from the bottom.

main.js

This is the file where Vue will be initialized, and also where Vue Router and Vuex (the store) will be brought into the project as plugins by way of app.use().

App.vue

This is the top-most parent component, i.e., the root component. It holds all the Vue code (coming in as code in this file or code in child components) that will be injected into the index.html file, inside the div that has id="app".

views

These are the pages that will be the main routes of the project. There will be a page that the user first lands on where they must enter a code. I will name it EnterCode.vue. And there will be a page that shows the live stream, with the video player and audio captions. I'll name that page StreamChannel.vue.

Here is a screenshot of what the views will look like by the end of this post. A demo of the finished project can be found in the first post, which shows the styled version with the video live stream page.

store

This is the folder that contains the Vuex store, a state management file. Some state properties need to be widely available throughout the application, not just in one component or one parent-child component. Vuex makes those state properties that I have put in the store available in any Vue files throughout the project.

router

This is the folder that contains the routes and creates the router, so that each view (page) can be navigated to when a user clicks on the link to that page. Using Vue Router means that a user can switch between page views without the page being reloaded every time the route changes.

composables

This folder contains composition functions that run the logic to make stuff happen, such as getting the user's microphone or bringing in the Amazon IVS player. Composition functions, or 'composables', are a Vue 3 way of encapsulating logic that can then be run inside the setup function inside a component. Read more about composables here.

components

The components folder contains two components that will make up the StreamChannel page - the VideoPlayer component and the AudioCaptions component. I'll build these components in the next post in this series.

Set up Vue Router and the Views

The first thing I will do is create the two main pages I intend to make as part of this application - the landing page and the live stream page.

I will create a views folder and put those two main page files in the folder:

For each file, I'll be sure to put in some HTML that will display on the page, so I can see that my router is working:

<template>
  <div>
    <h1>Enter Code</h1>
  </div>
</template>

When I created my project with the Vue CLI, I opted to add the Vue Router dependency, which means I have a router already created for me - the index.js file in the router folder. The index.js already has two routes set up - home and about.

If I start up the development server, I see that the links at the top of the page will take me between the two routes.

Since my project is only going to have two pages, I am just going to change the home and about views of the bootstrapped project to be the EnterCode view and the StreamChannel View.

In the router file, the three things each route object needs to have are:

the path - the URL path for the route
the name - the name I want to give this route (optional, but in my case, I do need it)
the component - the component (view file) that will be loaded for this route

Here is the code to set that up in the router:

import { createRouter, createWebHistory } from 'vue-router'
import EnterCode from '../views/EnterCode.vue'

const routes = [
  {
    path: '/',
    name: 'EnterCode',
    component: EnterCode,
  },
  {
    path: '/stream-channel',
    name: 'StreamChannel',
    component: () =>
      import(/* webpackChunkName: "about" */ '../views/StreamChannel.vue'),
  },
]

const router = createRouter({
  history: createWebHistory(process.env.BASE_URL),
  routes,
})

export default router

I need to go back to App.vue and change the router-links to match my new routes. Even though I've named the landing page EnterCode.vue, I'll refer to it has Home for the user. Notice that the to="" attribute contains the path:

<template>
  <nav>
    <router-link to="/">Home</router-link> |
    <router-link to="/stream-channel">Channel</router-link>
  </nav>
  <router-view />
</template>

Now I see that my two main pages can be accessed (without reloading the page!) when I click the link to switch to each route:

Vue Router Navigation Guards

I want the landing page to have a form input where the user has to enter a code, and if the code is correct, they can move on to the channel page. See the code for this input here.

But if the code they enter is incorrect, I want to keep them from being able to navigate to that page. Right now, the channel page is completely open, and I can access it just by clicking the Channel link.

I can set up a navigation guard to perform some logic when the router-link is clicked (Read the docs for more information about navigation guards).

The logic will check to see if the correct code has been entered. If it has, the router will navigate to the channel page. If not, it will send them back to the home (EnterCode.vue) page.

To add a navigation guard, I use the beforeEnter guard:

{
 path: "/stream-channel",
 name: "StreamChannel",
 component: () => import("../views/StreamChannel.vue"),
 beforeEnter(to, from, next) {
   // ADD LOGIC HERE to check state of allowed access
   next();
 },
},

The next() function will move the navigation forward to the route. I can use an if statement to keep that from happening unless certain circumstances are met.

If I include a name of a route in the next() function, the navigation will move to the route I have identified by name. In this example, it would cause navigation to stay on the EnterCode page since I have identified that next should move me to that route.

beforeEnter(to, from, next) {
  // ADD LOGIC HERE to check state of allowed access
  next({ name: "EnterCode" });
},

I need to have some way of keeping track of the state of whether the correct code was entered or not. I will use the state management system, Vuex, which means I can now put this code in my router since the very next thing I will do is set up the store so there is a state property for allowAccess:

{
 path: "/channel",
 name: "StreamChannel",
 component: () => import("../views/StreamChannel.vue"),
 beforeEnter(to, from, next) {
   if (store.state.allowAccess === true) {
     next();
   } else {
     next({ name: "EnterCode" });
     alert("Please enter the secret code");
   }
 },
}

I'll probably see this error in the browser now since I've referenced store.state.allowAccess, but the router file doesn't know what the store instance is.

To make it go away, I need to import the store into the router by adding the following import statement to the router folder's index.js file.

import store from '../store'

The GitHub repo has the code in its entirety for this stage of the project.

Set up Vuex to Manage State

Even though this is a small application, I am choosing to include a Vuex store because it is common to use Vuex for the purpose of keeping track of whether a user is authenticated/logged-in or not.

In this project, I will use the Vuex store to keep track of whether a correct entry code has been entered or not by the user. The state property will be called allowAccess, and its default state will be false. It will change to true when a user enters the correct code.

Here is the store with the state property I have created:

import { createStore } from 'vuex'

const store = createStore({
  state() {
    return {
      allowAccess: false,
    }
  },
})
export default store

I need to connect the input on the landing (EntryCode) page with this property in state. When the user types a correct code into that input, the submit button triggers a function that checks if the code is correct, and then if it is, dispatches an action to the store, which will cause allowAccess to update to true.

The way Vuex causes state to change is through this pattern:

Vue Component dispatch action -> Store action commit mutation -> Store mutation change state

Here is that flow in the actual code in my project.

The Vue component EnterCode form submit button triggers submitCode():

<form @submit.prevent="submitCode">
  <label for="code"> Code: </label>
  <input v-model="code" type="password" name="code" value />
  <button type="submit" name="button" class="dg-btn">Submit</button>
</form>

The submitCode() method dispatches the verifyCode action (which is passed as a string 'verifyCode') with a payload of true or false:

function submitCode() {
  if (code.value === 'pb') {
    store.dispatch('verifyCode', true)
  } else {
    store.dispatch('verifyCode', false)
  }
  code.value = ''
}

The verifyCode action in the store commits the verifyCode mutation, sending the true or false payload, referred to here as status:

actions: {
 verifyCode({ commit }, status) {
   commit("verifyCode", status);
 },
},

The verifyCode mutation changes state so allowAccess equals the status payload of true or false

mutations: {
 verifyCode(state, status) {
   state.allowAccess = status;
 },
},

The last thing to do is navigate with the router to the StreamChannel page if a correct code has been entered (i.e., state for allowAccess in the store has changed to true) or alert the user if they have entered the wrong code.

//StreamChannel.vue

function submitCode() {
  if (code.value === 'PB') {
    store.dispatch('verifyCode', true)
  } else {
    store.dispatch('verifyCode', false)
  }
  // Navigate if correct code, alert if not correct
  if (store.state.allowAccess) {
    router.push({ name: 'StreamChannel' })
  } else {
    alert('Incorrect code')
  }
  code.value = ''
}

This version of the project, with the setup I just did for Vue Router and Vuex, can be seen in its entirety in the repo branch titled vuex-no-server-yet.

Security

The example so far is not very secure because I check for the correct code on the frontend, and that entry code is right there in plain Vue (no pun intended) in the client:

if (code.value === "pb")

The more secure way to handle this would be to create a backend server file with an endpoint to check if the code is correct and store that code securely in the server, or even better (so it does not get exposed in GitHub) in a .env file.

In the next section, I'll introduce that more secure method of checking the code. I'll still use everything I just set up for Vuex, but instead of checking the code in the EnterCode.vue file, I'll use a fetch request to send that code to the backend for verification.

Create the Server with Express

Now I'll set up my server file, which I'll use today for making the entry code more secure, and which I'll use in the next post for setting up the Deepgram token endpoint.

At the root of the project, I'll create a server.js file and a .env file.

I'll add this code to create my basic server.

// bring in node modules
require('dotenv').config()
const express = require('express')
const app = express()
var bodyParser = require('body-parser')
const cors = require('cors')

// identify port number
const port = 8080

// express use() function to add third-party middleware
app.use(cors())
app.use(bodyParser.json())

// ENDPOINT WILL GO HERE

// Connect host to port
app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`)
})

Dependencies

Here is a brief explanation of each dependency I'm using in this server file.

express

Express is a node.js framework that gives me utility methods and middleware to help with setting up routing endpoints (the endpoints deal with requests that come in from the frontend).

body-parser

Body-parser is middleware that can take an incoming request body and parse the data. I'll be using the JSON parser because the secret entry code will be sent from the frontend as JSON.

cors

This is another middleware package that will help to handle requests from the frontend, specifically cross-origin requests.

Since the client and server have a different origin from each other (such as localhost:8080 vs. localhost:8081), I would need to add a CORS response header access-control-allow-origins with information about permitted origins. The CORS middleware will add that header automatically for every request that is sent to the server.

dotenv

This is a very important node module. It allows me to use environment variables in my server file, pulling the values for those variables from the .env file. The .env file will never be tracked by git, so I can put anything super-secret in the .env file and not worry that it will end up on Github for the world to see.

Run the server

Now I can start up the server to check that everything is working so far.

I will add a script command to the package.json file to make it easier to start up the server. In my package.json, in the "scripts" object, I'll add one for "start":

"scripts": {
  "serve": "vue-cli-service serve",
  ...
  "start": "node server.js"
},

Now, in the terminal, when I'm in the project folder, I can type npm run start to start the backend server, and if I open another terminal, I can type npm run serve to start the frontend development server.

Add the secret code value to `.env`

Before I set up the endpoint, I want to put the expected code value in the .env file so that I can access it in the server using process.env.SECRET_CODE.

In the .env file, I will add the secret code that I want users to type into the input on the frontend to be able to enter the site. I can add any value I want. In this case, I'll just make the value 'code' (in the earlier section, I used 'pb' as the code, but I'm using 'code' here to make it more obvious that this is the code):

SECRET_CODE="code"

Create the secret-code endpoint

Now I'll start on the backend creating the endpoint. The endpoint will expect a value to come in from the frontend (the code entered by the user). Since a value is being sent back, this will be a POST request endpoint.

app.post('/secret-code', async (req, res) => {})

The app.post() method is available because of express, and it requires the route path as a slash and whatever I want to name the path, in this case "/secret-code".

When the code is sent from the frontend to the backend, it comes in as part of the request body (which is an object), so I will get the code value from req.body. Because I plan to send it back in a JSON object as { code: "code" }, the value will come back as req.body.code.

if (req.body.code._value === process.env.SECRET_CODE)

However, the value that is sent back is put into another object, which I see when I examine the shape of the req.body.code by using console.log. I see this:

The _value property actually contains the value that I want to match to my code value I have put in the .env file. So I need to access it with req.body.code._value. This is how I can check that there is a match:

app.post('/secret-code', async (req, res) => {
  if (req.body.code._value === process.env.SECRET_CODE) {
    res.status(200).json('Correct code')
  } else {
    res.status(200).json('Incorrect code')
  }
})

The res.status() method will send back the result status code, and the .json() method will make sure it returns to the frontend as JSON data.

Now I can go back to the frontend and write a fetch request that will send the secret code that I need for this verification.

Fetch POST request

To write a fetch POST request, I will use the fetch API.

To write a fetch request, I use the fetch() method with a first argument of the endpoint path (including the port number) and the second argument an options object:

fetch('http://localhost:8080/secret-code', requestOptions)

The options object will give information about the type of request (POST), the headers (to tell that the content will be of the type JSON) and the request body (I will send a JSON object that contains the code):

const requestOptions = {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ code: code }),
}

The fetch request will send the code to the backend server, and since that will take a little bit of time, I will use a promise to wait for the response, doing the following action once that response has been returned. To use a promise, I will attach a .then() to the request, which means that anything inside the .then() will happen after the response is returned from the fetch request.

Once the response returns, I will turn it back into JSON and then take that response data to verify if it is correct or not. Depending on if it is correct, I update the state in the store.

Remember, for a verified response of true, I am expecting the backend server to return a response with a JSON string of "Correct code":

fetch('http://localhost:8080/secret-code', requestOptions)
  .then((response) => response.json())
  .then((data) => {
    if (data === 'Correct code') {
      //dispatch payload to store
    }
  })
  .then(() => {
    if (store.state.allowAccess) {
      // navigate to channel or alert that code is wrong
    }
  })

I've created a branch in the repo called vuex-with-server where the code from this step can be viewed in its entirety.

Conclusion

In this post, I set up a Vue project and added some of the common features used in a full-stack application: routes, a state management library, and a backend server with endpoints.

Now I am ready to build the Channel page, which will contain the video player for my live stream and the audio captions that display the transcript.

Previously, I went over how to build a live stream web application with Amazon IVS, using only vanilla javascript and HTML. But now, I want to use the Vue framework to help me build this in a way that is more reusable, and that incorporates common practices for using Vue.js.

In my next post, I will introduce how to build the Amazon IVS player using Vue 3 composables, which will give me a chance to discuss more specifically some of the new features of Vue 3.

I will also introduce how to use Vue 3 composables to create the Deepgram captions, and I'll include a more secure way of connecting to Deepgram by requesting a Deepgram token from the backend.

I hope you'll join me for my next post. As always, feel free to reach out on Twitter.

How to Build a Live Streaming Web Application with Amazon IVS and Deepgram

SandraMartin — Fri, 11 Mar 2022 21:50:37 +0000

In this series, I will build a live streaming web application with text captions. Companies such as Twitch, StreamYard, Facebook Live, and many others provide live streaming on the web as a product, so I got curious about how I might go about building my own version.

The main technologies I will use to build the live stream and text captions functionality are:

Open Broadcast Software - an open-source software used to capture video streams
Amazon IVS - an AWS service that can receive a video stream from OBS and put that stream into the browser, optimizing the entire process
Deepgram - a speech-to-text API that can receive an audio stream and return a text-transcript

I will build two versions of this project. I'll use the following to build each project:

HTML/Javascript - The first version I build (the vanilla version) will be focused on creating a front-end with a very limited back-end, emphasizing the most barebones approach to getting the application working.
Vue.js/Node.js - For the second version (the framework version), I will use Vue.js, a Javascript framework that gives me the tools I need to more easily include important features for security like routing and navigation guards. I will build a full-stack video streaming application with a node.js server to help me add a layer of security for some of the data I need to protect.

Here is a gif to demonstrate the final project:

Project One: Vanilla Video Stream Player

Now I'll start by building the first version of the project. I'll build a 'vanilla' video streaming player in the sense that I will only use HTML and Javascript on the front-end, and the only back-end will be the work I do to get Amazon IVS set up to receive the OBS stream.

I want to keep it as simple as possible, focusing on how to build a video streaming player in the browser that includes text captions. This means I will not take into account real-world requirements such as hiding API keys or creating an entry page to restrict access to the video stream. Later, in the Vue.js version I build, I'll add those pieces, but to start, I just want to get the video player working - I want it to play my live stream and display text captions for what I'm saying as I stream to viewers.

Here is a diagram to demonstrate the core technology for the video streaming part of the project:

The diagram presents this flow: the webcam takes in the video stream --> OBS captures that video stream so it can be sent along to Amazon IVS --> Amazon IVS provides a service to take in the stream, optimize it, and send it in a format to the browser so that it can be used in an HTML video player --> the HTML video element plays the optimized video stream.

Here is a diagram to demonstrate the core technology for the text captions part of the project:

The general flow for the text captions technology will be something like this:

The browser Media Streams API gets permission to use the browser microphone --> the microphone takes in an audio stream of the audio that plays from the live stream --> The Deepgram API opens a web socket channel in the browser to send the audio stream to Deepgram --> the browser receives the JSON object in return that contains the text-transcript --> Javascript puts the text onto the page as captions as the video is playing.

Now that I have a high-level picture of how this project will be built, I am ready to build the barebones front-end video stream application. (In the next post in this series, I will build the Vue.js/Node.js full-stack application with added functionality.)

Setting up Open Broadcast Software

The first thing I need is software to capture my video stream on my computer. I'll use the Open Broadcast Software.

What is OBS?

For anyone serious about streaming, OBS is a powerful tool. It is a free open source software that gives many configuration options for capturing and editing a stream. I can edit every aspect of my stream and create scenes made up of multiple sources such as images, text, etc. I can mix audio, switch between scenes with transitions, adjust the layouts, and so much more.

The stream I capture in OBS can be connected to a streaming platform such as Twitch, YouTube, or others, and it will deliver my stream to that platform; however, for this project, my goal is to stream to a web application that I make myself.

OBS takes some effort to learn, but I only need to familiarize myself with a few parts of it if I'm going to set it up to capture a stream and connect to Amazon IVS (Amazon IVS is not a streaming platform - it is more like an SDK that helps make the stream easier to handle when I build my front-end).

Set up Live Streaming with OBS

To set up OBS for my project, I will:

Go to obsproject.com and choose the operating system I use. I'll download the software.
Run the OBS software. In the Sources panel, I'll click the plus sign to add a new source. I'll select Video Capture Device, and in the window that pops up, I'll select the camera I want to use to capture my stream (my computer camera or webcam).

Make sure the source is selected in the sources panel (I may have other sources that I have set up), then I'll click on Settings in the far-right Controls panel.
Select Stream in the left column of the window that opens up. The Service will remain Custom, but I notice that I could select a streaming platform such as Twitch or YouTube if I weren't planning to build my own streaming application.
There is nothing more to do until I create the Amazon IVS channel. But I know that later I will take the Server and the Stream Key information from Amazon IVS for the specific channel I create in the AWS console.

Setting up Amazon IVS

In this step, I will create an Amazon IVS channel, which is where my video stream from OBS will feed into once I connect them.

What is Amazon IVS?

The 'IVS' in Amazon IVS stands for Interactive Video Service. The website for Amazon IVS describes it as a "managed live streaming solution" that I can use to send "live streams to Amazon IVS using streaming software" and "make low-latency live video available to any viewer around the world." In addition, I "can easily customize and enhance the audience experience through the Amazon IVS player SDK."

So what does this mean?

The fact is, building a video player browser can be very complicated. Amazon IVS takes away much of the challenge, allowing me to focus on the design of my front-end rather than the nitty-gritty of the video player. If I did it all without Amazon IVS, I could use the HTML native video tag, but then there would be much to do to optimize the video stream that comes through (there's an interesting article about this here). A developer could dedicate their entire career to getting good at building stuff that manages audio and video streams in the browser.

Amazon IVS will optimize the stream to make it work for viewers watching it in my web application anywhere in the world. It also provides an SDK for the video player, which I can bring into the browser by adding a script. That script will take control of the video element in my HTML and add all the magic that Amazon IVS does under the hood. The Amazon IVS video player is built for the purpose of streaming live video, so I don't have to build my own complicated video player.

One important thing to consider is cost. AWS is not free, and while it is very cost-effective for a bigger streaming platform like Twitch (the Twitch streaming technology is powered by Amazon IVS), an individual developer like myself building a small project for fun might not find it to be the best option.

The good news is a new user of Amazon IVS can enjoy the free tier, which gives the following:

Five hours of live video input for a basic channel per month
100 hours of SD live video output per month

This is enough to build this project and not be charged, as long as I am careful about turning off my stream in OBS when I'm not using it. (Yes, I did forget to do this one time and clocked several hours in Amazon IVS.) Be sure to read through the pricing details and be vigilant about turning off the stream when you don't need it to be on.

Set up Amazon IVS

Now I'll set up a channel in Amazon IVS. The channel will take my video stream from the OBS software on my computer and make that stream available in a video player that I will bring into the browser with the Amazon IVS SDK (so many acronyms!).

In order to do this, I'll need to create an AWS account. This will require billing information.

In addition, I'll need to set up AWS Identity and Access Management (IAM), which adds a 'policy' to my account that allows me to create an AWS IVS channel. This is standard for doing anything in AWS - the first step is to configure IAM so that users of the AWS console have specific permissions. I am the only user of my console, so I'm not worried about restricting any permissions in my account.

This guide walks through how to set up the IAM permissions so that a user can create an AWS IVS channel.

Now I can navigate to IVS to create a channel. In the top search bar, I can type 'IVS' to find Amazon Interactive Video Service.

This takes me to the Amazon IVS console. I will click the Create channel button to create my channel.

I can name my stream and stick with the Default configuration. Then I'll scroll down and click Create channel.

This will create the channel and then put me on that channel's page in the console. This is where I can configure the channel and get the information I need to connect my video stream in OBS to this channel. I need to find this section of the page:

There are three pieces of information I am going to need for my project. Two are to connect OBS to Amazon IVS, and one is to bring the stream from Amazon IVS (with all its optimizations) into the browser video player:

Ingest server - put this in OBS settings for my stream
Stream key - put this in OBS settings for my stream
Playback URL - use this as the src for my script that I put in the video player

I have already set up OBS, so I can just go back to the settings for my stream and add the Ingest server and Stream key. The Playback URL will be used later.

Now, if I go back to the OBS controls and click on Start Streaming, my stream should be fed to Amazon IVS, and I am able to see it in the Amazon IVS channel page where it says Live stream:

Connect Front-end Video Player to Amazon IVS

The back-end is done (AWS takes care of most of the work). Now I can build the front-end, which I will do using vanilla Javascript and HTML.

In the <head> tag of my HTML document, I will include the script for the Amazon IVS player. Amazon IVS explains how to do this setup here, for those who want to go straight to the source.

<head>
  <meta charset="UTF-8" />
  <title>Video Stream Demo</title>
  <script src="https://player.live-video.net/1.6.1/amazon-ivs-player.min.js"></script>
</head>

This will load the IVS Player, and I will have access to the IVSPlayer variable in the global context. I can type that variable into the console to take a look at the module that has been loaded. There are quite a few properties that could be of use to me, depending on my project's needs.

In the <body> tag, I will include a <video> player that has an id of video-player (this id can be renamed, as long as the javascript I write to find this element looks for that specific id).

<body>
  <video
    width="520"
    height="440"
    id="video-player"
    controls
    playsinline
  ></video>
</body>

In the browser, I see the video player, but there is no stream coming through. That is because I have only brought in the Amazon IVS player; I have not yet connected the player to my stream channel.

I will use javascript to put my stream channel into the player.

<script>
if (IVSPlayer.isPlayerSupported) {
  const player = IVSPlayer.create();
  player.attachHTMLVideoElement(document.getElementById("video-player"));
  player.load("PLAYBACK_URL");
  player.play();
}
</script>

Where it says PLAYBACK_URL in the code example, I need to put the string for my playback URL, which I can find in the Amazon IVS console for my channel.

Then I can turn on my stream in OBS, and I should see my stream in the browser!

Use Deepgram to Create Text Captions

The second part of this project, after getting the live stream video player working, is creating text captions. The captions will display what is being said in the live stream as I am streaming.

I will need to do two things: use my computer's microphone to listen to the audio that is being outputted from the live stream, and then send that audio stream to Deepgram to turn it into a text transcription.

What is the Media Streams API?

The browser contains several APIs for working with audio and video. I need to use one that lets me gain access to the user's microphone. If I can gain that access, I can record the audio from the live stream and send it on to Deepgram to get the text transcript.

The Media Streams API contains many interfaces and methods for working with audio and video data. There is already a really great guide for how it works here, so I won't go over all the details. I just need to understand that the Media Streams API has so much that I can use when I'm working with audio or video data in the browser. In fact, I'm pretty sure the Amazon IVS SDK uses it under the hood as part of their video player.

Get Audio with Media Streams API

I will use the getUserMedia method from this API. To get access to the user's microphone, I can write this javascript:

<script>
//Get access to user's microphone
navigator.mediaDevices.getUserMedia({ audio: true }).then((res) => {
  mediaRecorder = new MediaRecorder(res, {
    audio: true,
  });
});
</script>

This will cause the browser to ask for permission to use the microphone.

If the user gives permission, then I'll have access to the live stream audio to send to Deepgram.

Create Text Captions with Deepgram API

In the next step, I will use the Deepgram API to take the audio data and turn it into text.

What is Deepgram?

Deepgram is an ASR technology (ASR stands for Automatic Speech Recognition). It uses pretty advanced AI and deep learning technology to take speech from audio files or streams and turn it into text. There are probably a million ways to use this technology in a project. It's a fun API to get comfortable with for this reason.

If I'm going to use Deepgram in my project, I need to create an account here. This will give me an API key and $150 in free credit, so I won't need to enter billing information just to get started (unlike AWS).

Now I can connect to the Deepgram socket with my API key.

Connect to Deepgram to Get Transcription

I want to get the transcription and display it under the video player, so I will create an HTML element for that transcript. I'll give it the id of captions.

<p id="captions"></p>

I'm going to follow the tutorial my colleague Kevin Lewis wrote about getting live speech transcriptions in the browser. He explains that I need to connect to Deepgram with a WebSocket. I have to make sure I have access to the microphone before I open the WebSocket, so I will put the logic to connect to Deepgram inside the .then() that is attached to the getUserMedia function call.

navigator.mediaDevices.getUserMedia({ audio: true }).then((stream) => {
  ...
  const socket = new WebSocket("wss://api.deepgram.com/v1/listen", [
    "token",
    "YOUR_KEY_HERE",
  ]);
});

I will put my API key where it says "YOUR_KEY_HERE".

Once the socket is open, I can add an event listener that listens for when there is audio data that has come through the microphone. When that happens, I can take that audio data and send it through the Deepgram socket to Deepgram.

socket.onopen = () => {
  mediaRecorder.addEventListener('dataavailable', async (event) => {
    if (event.data.size > 0 && socket.readyState == 1) {
      socket.send(event.data)
    }
  })
  mediaRecorder.start(1000)
}

Deepgram will send the transcribed audio back to me as text. It will come in the form of a JSON object, so I need to drill down to the transcript property using dot notation. I will use document.querySelector(#captions) to put the transcript onto the screen under the video element.

socket.onmessage = (message) => {
  const received = JSON.parse(message.data)
  const transcript = received.channel.alternatives[0].transcript
  if (transcript && received.is_final) {
    document.querySelector('#captions').textContent += transcript + ' '
  }
}

Here is all the Javascript code for building the text captions feature:

    <script>
    // Get access to user's microphone
    navigator.mediaDevices.getUserMedia({ audio: true }).then((stream) => {
      if (!MediaRecorder.isTypeSupported("audio/webm"))
        return alert("Browser not supported");
        const mediaRecorder = new MediaRecorder(stream, {
          mimeType: "audio/webm",
        });

      // Open connection to Deepgram
      const socket = new WebSocket("wss://api.deepgram.com/v1/listen", [
        "token",
        "YOUR_KEY_HERE",
      ]);

      // Listen for audio data coming from microphone and send it to Deepgram
      socket.onopen = () => {
        mediaRecorder.addEventListener("dataavailable", async (event) => {
          if (event.data.size > 0 && socket.readyState == 1) {
            socket.send(event.data);
          }
        });
        mediaRecorder.start(1000);
      };

      // Put the transcript onto the screen in the #captions element
      socket.onmessage = (message) => {
        const received = JSON.parse(message.data);
        const transcript = received.channel.alternatives[0].transcript;
        if (transcript && received.is_final) {
          document.querySelector("#captions").textContent += transcript + " ";
        }
      };

      socket.onclose = () => {
        console.log({ event: "onclose" });
      };

      socket.onerror = (error) => {
        console.log({ event: "onerror", error });
      };
    });
    </script>

And here is the HTML:

<html lang="en">
  <head>
    <meta charset="UTF-8" />
    <title>Video Stream Demo</title>
    <!-- Amazon IVS SDK video player -->
    <script src="https://player.live-video.net/1.6.1/amazon-ivs-player.min.js"></script>
  </head>
  <body>
    <video
      width="520"
      height="440"
      id="video-player"
      controls
      playsinline
    ></video>
    <p id="captions"></p>
    <!-- scripts -->
  </body>
</html>

Now I can start my live stream, and the text captions will be displayed under the video player!

Conclusion

In this tutorial, I built a 'vanilla' live stream player with text captions. I demonstrated how to use the technologies Amazon IVS and Deepgram using fundamentals of web development - HTML and Javascript.

But most front-end developers rely on frameworks to build projects like these. And there are other considerations I need to make in regards to keeping my Deepgram API key secure and limiting who has access to this website.

In the next part of the series, I will improve this project by building it using Vue.js (specifically Vue 3) for the front-end, and node.js for the back-end. I will include some of the real-world practices for building a full-stack application. I'll need a server file so I can incorporate more security, which I'll build with node.js, and I'll build an entry page with VueRouter navigation guards so that users must enter a code to see my live stream.

Vue.js is my favorite Javascript framework, and I have written a series on Diving Into Vue 3, which is worth checking out if you want to come along with me for the rest of this series to build a full-stack live stream application in Vue.js.

Please follow me on Twitter if you find my tutorials useful!

Diving Into Vue 3 - Reusability with Composables

SandraMartin — Fri, 25 Feb 2022 16:12:37 +0000

Introduction

This is the fifth and final post of my 'Diving Into Vue 3' series. Today I will combine what I have learned so far with a demonstration of how to use the Composition API to take advantage of its biggest strength: writing reusable code. This post will:

review everything I've learned so far by walking through how I build an example component, focusing on challenges of working with the DOM and using lifecycle methods.
introduce how to use a template ref to keep track of an element in the DOM.
demonstrate how to refactor the project to use composition functions (i.e. composables).

Don't forget there are four previous posts in this series that might be useful to you:

If you don't need the walk-through for building the example project, feel free to jump to the section on reusability, where I show how to refactor the project to use composables.

Example Component

I am going to build a single-file component that has a mast with an image on the left and text on the right. The problem I need to address is that I want to change the size of the text based on the image being resized.

Here's the demo:

To achieve this, I will:

listen for resizing of the window with an event listener.
track the image size.
update the text size if the image gets to a certain size.

The repo to go along with this example can be found here. There are several branches to show the progression of how the project gets refactored.

Vue 2

I won't go over how I built the project in Vue 2, but if it helps, the completed project in Vue 2 can be viewed here.

Resizing the window will show how the text size changes as the width of the image changes.

Vue 3

Here's how to build the component in Vue 3. The html in the template is exactly the same as the Vue 2 project:

<template>
  <div class="mast">
    <div class="container">
      <div class="image-container">
        <img ref="imageRef" src="../assets/meatball.jpeg" />
      </div>
      <div ref="textRef" class="text-container">
        <p>
          Meatball, 9. Barks at Amazon guy. Likes sharing your apple slices.
          Wants you to grab the toy but won't let you have it.
        </p>
      </div>
    </div>
  </div>
</template>

In the script section, I'll need to add the setup function, and then I will define the variables for the data I'll be tracking. Since elements in the DOM will depend on each other to either trigger a change or react to a change, I will need to make them reactive using ref so everything stays in sync. Here's how I do that:

<script>
import { ref } from "vue";
export default {
  name: "Mast",
  setup() {
    let imageWidth = ref(0);

    //template refs
    let imageRef = ref(null);
    let textRef = ref(null);

    return { imageRef, textRef };
  },
};
</script>

The important data to keep track of is the imageWidth because that value is what I will use to determine if the text size should change.

The imageWidth value has to come from the image element in the DOM. It will be based on the actual size of the image at a point in time, so I will need to connect to the actual DOM element using a template ref.

Template Refs

I think of template refs as the Vue way of using Javascript to hook into a DOM element, such as the method document.getElementById() or document.querySelector().

In Vue 2, the way to do that is to add ref="nameOfRef" as an attribute on the element that I am targeting, then in the script, I could perform some action on it using this.$refs.nameOfRef.

In Vue 3, template refs are now part of the reactive API. If I want to set up a template ref, I still need to add ref="nameOfRef" as an attribute on the element that I want to hook into.

<img ref="imageRef" src="../assets/meatball.jpeg" />

The difference now is that in the script, I need to define the template ref as a reactive reference variable wrapped in ref. And I MUST return it in the return object of the setup function so that it connects to that DOM element in the template. If I don't, it won't work.

setup() {
    //template refs
    let imageRef = ref(null);
    let textRef = ref(null);

    return { imageRef, textRef };
  },

Also, I need to be aware that I won't be able to actually access the ref to do something with it until the component has mounted - which brings me to the next topic.

Lifecycle Hooks

Now that I have the data set up I can add the logic to listen for the resize event.

I want to track the size of the image, which will change depending on if the window is resized. Since I'm dealing with a visual element, I need to consider timing of when that element will appear in the browser. It won't appear until the component has mounted.

The hooks that I'll need for setting up the event listener (and destroying it) are onMounted and onUnmounted, which are the equivalent to mounted and unmounted in Vue 2.

In onMounted, I have access to the template ref, so I will first set the initial value of the imageWidth based on the width of the actual image, which I pull from the template ref. I will also put a listener on the window to track the resizing event so that as the window is resized, the resizeHandler function runs.

Everything currently resides in the setup function for now, but will be refactored later and moved into composables:

// inside setup function:

onMounted(() => {
  //set initial value
  imageWidth.value = imageRef.value.offsetWidth

  //add listener to track resize
  window.addEventListener('resize', resizeHandler)
})

The resizeHandler sets the imageWidth value to the imageRef's width. I have to remember that with refs in the script, I have to unwrap the value using .value:

// inside setup function:

function resizeHandler() {
  //tracking of width changes
  imageWidth.value = imageRef.value.offsetWidth
}

Since I'm listening for the resize event starting when the component mounts, I need to be sure to destroy the listener when the component unmounts:

// inside setup function:

onUnmounted(() => {
  //remove listener
  window.removeEventListener('resize', resizeHandler)
})

watch

I now have the data set up so that the imageWidth updates in-sync with the imageRef's width as the event listener fires the resizeHandler function.

The last thing I need to do is make something happen as a side effect of the imageWidth increasing or decreasing. Vue offers watch and watchEffect as part of the API for watching a reactive property and causing a side effect to occur based on changes to the property.

In this case, I will use watch because I only need to track the imageWidth value since a change to imageWidth is what I'm using to cause the text size to change.

// inside setup function:

watch(imageWidth, () => {
  //initiate side effects to change text size when window width changes
  if (imageWidth.value < 150) {
    textRef.value.style.fontSize = '.8em'
    textRef.value.style.lineHeight = '1.3'
  }
  if (imageWidth.value < 200 && imageWidth.value > 150) {
    textRef.value.style.fontSize = '1em'
    textRef.value.style.lineHeight = '1.4'
  }
  if (imageWidth.value > 200) {
    textRef.value.style.fontSize = '1.3em'
    textRef.value.style.lineHeight = '1.5'
  }
})

Here is the finished example code using Vue 3 (and before I refactor it to use composables). Now that everything is working, I will refactor my code to make it more reusable.

Reusability in The Composition API

Many people would say that the biggest advantage of using Vue 3's Composition API is its emphasis on organizing code by logical concern rather than by option types like in Vue 2. If I'm building a small application that is only going to have minimal logic in a component, the Options API, or even just putting all my logic in the setup function, is fine. But as a component grows larger, it can be challenging to follow the data flow.

For example, a UI component such as a dropdown menu has to deal with opening and closing the dropdown, keyboard interactions, pulling in data to populate the menu, and more. All that logic in one component spread out among the options like methods, watch, mounted, etc., can be hard to decipher.

Vue 2 does offer approaches for separating out logic, such as mixins and utility functions. But Vue 3's whole philosophy is designed around the idea of writing code that is reusable, focused around logical concern, and easy to read. The most fundamental way it does this is through composition functions (i.e. composables).

Composables

The advantage of organizing code by logical concern encapsulated in a composable function is that it becomes easier to read, but it also becomes easier to reuse in other parts of the project or even in other projects.

I feel that the ultimate goal should be to write the most agnostic code possible in a composable, i.e. code that can be recycled in different contexts and isn't so dependent on the one unique context it starts out in.

It does take time and practice to get better at this skill, but the good news is, Vue 3 is the perfect framework to work at it because using the Composition API really emphasizes this approach to coding.

With that in mind, I'll think about how I can refactor my project to take advantage of composables.

useWindowEvent

A common situation is having to listen for an event on the window, such as a resize event. I see an opportunity to write a composable that can be reused when I want to add or destroy an event listener on the window.

In my project, in the onMounted hook I currently have:

window.addEventListener('resize', resizeHandler)

And in the unMounted hook:

window.removeEventListener('resize', resizeHandler)

I can create a composable function that accepts an event-type, a handler, and a string saying 'add' or 'destroy', and write logic that will set up the window event listener. I will put this file in a folder called ~/composables. The Vue 3 convention is to name composable files with the prefix 'use' as in useWindowEvent.

Here is the composable useWindowEvent.js:

export default function useWindowEvent(event, handler, addOrDestroy) {
  if (addOrDestroy === 'add') {
    window.addEventListener(event, handler)
  }

  if (addOrDestroy === 'destroy') {
    window.removeEventListener(event, handler)
  }
}

Now in my project, I import it into the component where it will be used:

import useWindowEvent from '../composables/useWindowEvent'

Then I invoke the function with the arguments that I set it up to receive:

useWindowEvent('resize', resizeHandler, 'add')

This is just a small composable, and it doesn't really make my life that much easier since I didn't have to write very much code anyways to set up the listener on the window.

But there is a significant advantage to creating reusable code. I know the composable is written to work, so I'm less likely to have little errors or typos since I'm reusing code that has been tested and used before. Because I've tested it, I can feel confident reusing it in many contexts.

Consistency is another benefit. I keep functionality consistent by using the composable in multiple places, rather than having to reinvent the wheel every time, potentially introducing differences (and problems).

And now that I have created a useWindowEvent, I could try to make it to work for all kinds of elements, not just the window. If I spend some time improving it so that it can add an event listener to any type of element, then I have a really useful composable that I can reuse.

useResizeText

The main feature of my project is that the text resizes based on the image element's width. I can turn this into a composable that can be reused in cases where I want text to resize based on some other element.

In my goal to write it in a way that is more agnostic, I can think of the element that is watched (the image) as the trigger element, and the element that changes (the text) as the react element. In the resizeText composable, I'll refer to them as the triggerElement and the reactElement, but in the Mast.vue component they are the imageRef and the textRef. These are more specific references to the context of my project, while triggerElement and reactElement are more general since I would like the composable to be reused if I ever need it in a different project.

I create the composable file called useResizeText.js. I anticipate that I'll need to accept two arguments, the triggerElement and the reactElement (which come in from Mast.vue as the imageRef and the textRef):

//useResizeText.js:

export default function useResizeText(triggerElement, reactElement) {
  return { elementWidth }
}

I've included the return object because any data from the composable that I want to make available in the component (or another file) must be included in it. I'll return the elementWidth to the component so I can put it in my template in Mast.vue and see the resize logic working in real-time.

In the Mast.vue component, I will call the composable. I have to send in the template refs so the composable can compute the text size based on those DOM elements. I will destructure the composable so that I get the returned elementWidth.

Inside setup in Mast.vue:

//destructure to get data sent back from the composable
//get updated width for template
const { elementWidth } = useResizeText(imageRef, textRef)

I will return elementWidth to the template so that I see that number reacting to the window resizing. I also return imageRef and textRef because that is required for the template refs to stay in-sync between the script and the template.

Here is everything in the setup function:

setup() {
    //template refs
    let imageRef = ref(null);
    let textRef = ref(null);
    //destructure to get data sent back from the composable
    //get updated width for template
    const { elementWidth } = useResizeText(imageRef, textRef);
    return { imageRef, textRef, elementWidth };
  },

The composable itself is mostly the same as it was when I wrote the logic in the setup function, with a few small updates.

To make sure I don't get an error when I set the elementWidth to the imageRef/triggerElement offsetHeight value, I use an 'if' statement to make sure the triggerElement exists:

if (triggerElement.value) {
  elementWidth.value = triggerElement.value.offsetWidth
}

I also set the initial text styles as soon as the component mounts and then run that setTextStyles function again inside the watch every time the elementWidth (the image's width) changes.

Here is the full code for the resizeText.js composable:

import { ref, watch, onMounted, onUnmounted } from 'vue'
import useWindowEvent from './useWindowEvent'

export default function useResize(triggerElement, reactElement) {
  let elementWidth = ref(0)

  //handler to send into useWindowEvent
  function resizeHandler() {
    if (triggerElement.value) {
      elementWidth.value = triggerElement.value.offsetWidth
    }
  }

  //set initial values for elementWidth and text styles
  onMounted(() => {
    if (triggerElement.value) {
      elementWidth.value = triggerElement.value.offsetWidth
      setTextStyles()
    }
  })

  //function to set text styles on mount and in watcher
  function setTextStyles() {
    if (elementWidth.value < 150) {
      reactElement.value.style.fontSize = '.8em'
      reactElement.value.style.lineHeight = '1.3'
    }
    if (elementWidth.value < 200 && elementWidth.value > 150) {
      reactElement.value.style.fontSize = '1em'
      reactElement.value.style.lineHeight = '1.4'
    }
    if (elementWidth.value > 200) {
      reactElement.value.style.fontSize = '1.3em'
      reactElement.value.style.lineHeight = '1.5'
    }
  }

  //add and destroy event listeners
  useWindowEvent('resize', resizeHandler, 'add')
  onUnmounted(() => {
    useWindowEvent('resize', resizeHandler, 'destroy')
  })

  //watch elementWidth and set text styles
  watch(elementWidth, () => {
    setTextStyles()
  })

  return { elementWidth }
}

This refactoring makes Mast.vue much easier to read because the logic for resizing the text and for adding a window event listener are separated out into composables.

However, my ultimate goal is to make composables that are more reusable in general. There is more I can do to make the resizeText composable reusable in other projects.

For example, I could set it up to take a breakpoints object, so that I don't have to always use the same hardcoded width sizes to influence the text.

I could also rework it accept a styles object for the text styles so that I'm not required to use the same hardcoded values for text styles for any component that uses the composable. Something like this in the component:

//constants
const breakPoints = { small: '100', medium: '150', large: '200' }
const textStyles = {
  fontSize: { small: '.8em', medium: '1em', large: '1.3em' },
  lineHeight: { small: '1.3', medium: '1.4', large: '1.5' },
}

Here is the full example.

There are still many ways to improve this composable to make it more agnostic, but this gives a general idea of the process that goes into making a composable more reusable.

Conclusion

This concludes my series on Diving into Vue 3. I have learned the fundamentals that will allow me to jump into building projects using the Composition API. I feel so much more confident in Vue 3 now, and I'm also really excited about it.

I hope you have enjoyed this series. There is always more to learn, so stay tuned for future posts about Vue topics.

Questions? Comments? Just want to say hi? You can find me on Twitter!

Diving Into Vue 3 - The Reactivity API

SandraMartin — Fri, 18 Feb 2022 16:34:11 +0000

This is the fourth post in my 'Diving Into Vue 3' series. Today I hope to give a clear and practical description of how to use the new Reactivity APIs, focusing on ref and reactive. I'll also introduce how to use related helpers such as toRef, toRefs, and isRef.

The other posts in this series that have already come out are:

But first, a little background on reactivity. Feel free to skip ahead to the section on how to make data properties reactive if you just want to learn about ref and reactive.

What is reactivity?

The term reactivity in relation to Vue generally refers to a feature where what you see on the screen automatically updates in-sync with any changes to the state. It's the Vue 'magic' that makes the template re-render instantly if a data property changes.

When talking about reactivity in JavaScript or in programming in general, the term means programming something to work the way Vue does by implementing a design pattern called the Observer Pattern, which is explained in Design Patterns for Humans as :

whenever an object changes its state, all its dependents are notified.

Vue automatically updating the DOM when a data property changes is a result of Vue being built using the Observer Pattern - Vue state is an object with properties that have dependencies, so if one of those properties changes, its dependents react to the change by updating if they need to, and that triggers the re-render in the browser.

JavaScript on its own is not reactive, as shown in this example:

let numWorkers = 50
let numManagers = 4
let totalEmployees = numWorkers + numManagers

console.log(totalEmployees) // 54

numWorkers = 48

console.log(totalEmployees) // Still 54

Vue is reactive because the core Vue.js team built it to be. So in the following example, totalEmployees will automatically update anytime numWorkers or numManagers (two properties in the state object) changes:

  data() {
    //returns the state object
    return { numWorkers: 4, numManagers: 6 }
  },
  computed: {
    totalEmployees() {
      // returns whatever the total is based on current state for numWorkers and numManagers
      return this.numWorkers +  this.numManagers
    }
  }

Reactivity in Vue 2

The reactivity system in both Vue 2 and Vue 3 is based on state being an object, but there are big differences in how the properties are made reactive.

In Vue 2 the data option returns an object:

  data() {
    return {
      numWorkers: 4,
      numManagers: 6
    }
  }

Under the hood, Vue 2 uses Object.defineProperty to define all the data properties on a component instance, converting them to getters and setters. There's a deep dive into the Vue 2's reactivity system in the Vue.js docs that's worth spending some time with.

Because defining the properties happens at the time of the component instance's initialization, it results in some small drawbacks:

data properties cannot be added or deleted after component instance initialization. They have to be present during initialization for them to be reactive
If the data property is an array, it is not possible to set an item directly to the array through assignment by using the array index (as in arr[0] = value), and it also isn't possible to update the length of the array (as in arr.length = 0)

This isn't a major problem because the Vue.$set method can be used in cases where these updates need to be made after component instance initialization. However, Vue 3's reactivity system is so improved that now these issues are no longer a problem, making it unnecessary to use Vue.$set.

Reactivity in Vue 3

Vue 3's reactivity system had a major rewrite from what it was in Vue 2. The fundamental idea of tracking all the data properties and their dependencies so they can update automatically is still the same, but Vue 3 now uses the JavaScript Proxy API to achieve this (instead of Object.defineProperty like in Vue 2).

There's a rabbit hole to go down for anyone who wants to learn more about the Vue 3 reactivity system, and the Vue.js docs are fantastic. Even though I'm not going to explain it all (there's no way I could!), there are a few things I think are helpful to understand.

The docs state:

Proxy is an object that encases another object and allows you to intercept any interactions with that object.

Awareness of reactive properties using proxies is helpful when debugging code in the console. If I console.log a reactive property such as this property company:

const company = reactive({
  employees: ['Tom', 'Sara', 'Joe'],
  managers: ['Julie', 'Jorge'],
})

in the console, I see:

Clicking on it will open the object to show that there is a Handler and a Target. A proxy always contains a Handler and a Target, and since Vue 3 uses proxies, I find it helpful to be comfortable with the shape of this data as a proxy.

The Target is where to look for the actual values. It contains the data that I might be looking for. The Handler contains the special logic for making the data properties reactive. The Handler contains methods like get and set.

The Handler is the rabbit hole if you want to learn about reactivity. The Target is where I need to look for my data values.

Because reactive data is wrapped in a proxy, something to get used to when working with the data is the idea of having to 'unwrap' the data object to get to the value. After reading a lot of different resources about working with Vue 3 reactive data, I now feel comfortable with the idea that using strategies to 'unwrap' the data, such as destructuring or drilling down to the value property, are using the metaphor of unwrapping because Vue 3 reactive data is wrapped in a Proxy object.

How to make data properties reactive

As I said earlier, if I want to make data properties reactive in Vue 2, I have to return them in an object inside the data option of the Options API.

  data() {
    return {
      president: "Mickey Mouse",
      vicePresident: "Donald Duck"
    }
  }

If I am using the Vue 3 setup function (see my post on the setup function if you need an explainer on that), I can make data reactive by using the reactive or ref helpers.

ref

For this first example, I will use ref. I'm using ref because "Mickey Mouse" and "Donald Duck" are strings, and the recommendation is to use ref with primitive values (i.e. Javascript types that are not objects, such as strings, numbers, etc.)

First, I import ref:

<script>import { ref } from "vue";</script>

Then in the setup function, I set my variable to the ref() helper, which takes in the initial value. I must include the data in the return object if I want it to be available to the template.

  setup() {
    let president = ref("Mickey Mouse");
    let vicePresident = ref("Donald Duck");

    return { president, vicePresident };
    },

An important difference between ref and reactive is that if I want to do something to the value of my ref properties inside the setup function, I have to unwrap the object to access that value. So if I want to change the value of president, I will change president.value:

  function changePresident() {
    president.value = 'Goofy'
  }

I don't have to worry about unwrapping the values for president and vicePresident in the template. Vue can shallow unwrap those for me. 'Shallow unwrap' means the first level of properties in an object are available in the template without having to use .value (but nested properties would still need to be unwrapped).

<template>
  <div>
    <p><b>President:</b> {{ president }}</p>
    <p><b>Vice President:</b> {{ vicePresident }}</p>
  </div>
</template>

FYI, it's fine not to use ref if I don't need the data to be reactive, just writing the data like this:

setup() {
  let president = "Mickey Mouse"
  let vicePresident = "Donald Duck"

  return { president, vicePresident };
},

But it would mean the data isn't reactive, so I can't ever see updates to the data. If I use a method to change the data, I would never see that update change anything on the screen, and I would have to be happy with Mickey Mouse and Donald Duck showing as president and vice president forever.

There are times when you don't need the data to be reactive, so in those instances, just don't use ref or reactive!

reactive

I can use reactive for the same example, but I would only do so if I wanted the data to start out in the form of an object rather than separate string values. So in Vue 2, if I have this:

data() {
  return {
    executiveTeam: {
      president: "Mickey Mouse",
      vicePresident: "Donald Duck",
    },
  };
},

To change this to Vue 3 using reactive, I will first import reactive:

import { reactive } from 'vue'

In the setup function, I will create an object for executiveTeam and define the properties on the object. I can set the object to const since the object itself won't change, just the properties inside.

setup() {
  const executiveTeam = reactive({
    president: "Mickey Mouse",
    vicePresident: "Donald Duck",
  });

  return { executiveTeam };
},

And if I want to update the data, I do not have to unwrap it with .valuelike I do with ref.

function changePresident() {
  executiveTeam.president = 'Goofy'
}

This is because reactive is used with objects, and objects pass values by reference (which lends itself better to reactivity). Reactive references (ref) are used for primitive types, and primitives in Javascript pass values by value, so Vue has to wrap them in an object to make them reactive. Since ref properties are wrapped to make them reactive, they have to be unwrapped down to the .value to get the value. Read more about this concept in the Composition API RFC if this concept is something you want to understand more deeply.

However, because I'm returning the object executiveTeam and I want to access the properties president and vicePresident on that object in the template, I will have to drill down into the executiveTeam object to get each property I need:

<template>
  <div>
    <p><b>President:</b> {{ executiveTeam.president }}</p>
    <p><b>Vice President:</b> {{ executiveTeam.vicePresident }}</p>
  </div>
</template>

I cannot destructure the object that I return because if I do, the properties inside executiveTeam will lose reactivity. I'll demonstrate this in the next example to make this more clear.

When using reactive to give an object's properties reactivity, like this:

const executiveTeam = reactive({
  president: 'Mickey Mouse',
  vicePresident: 'Donald Duck',
})

I cannot destructure to try and return those properties by their key, as in:

//LOSES REACTIVITY:
let { president, vicePresident } = executiveTeam

return { president, vicePresident }

This is where toRefs comes in handy.

toRefs

The helper toRefs will allow me to turn each of the properties in the object into a ref, which means I won't have to use executiveTeam.president in the template; I'll be able to just write president. Here's the full example now using toRefs:

<script>
import { reactive, toRefs } from "vue";
export default {
  setup() {
    const executiveTeam = reactive({
      president: "Mickey Mouse",
      vicePresident: "Donald Duck",
    });

    //toRefs allows me to destructure
    let { president, vicePresident } = toRefs(executiveTeam);

    return { president, vicePresident };
  },
};
</script>

Since toRefs turns each property into a ref, I need to go back to unwrapping them down to their value using .valueif I want to do something to them in the setup function:

function changePresident() {
  president.value = 'Goofy'
}

toRef

Just like toRefs, the helper toRef is used to turn reactive object properties into reactive references (ref), but I would use toRef if I just need to turn one property in a reactive object into a ref:

setup() {
  const executiveTeam = reactive({
    president: "Mickey Mouse",
    vicePresident: "Donald Duck",
  });
  //toRef used to turn just one property into a ref
  let presidentRef = toRef(executiveTeam, "president");

  const changePresident = () => {
    presidentRef.value = "Goofy";
  };

  return { presidentRef, changePresident };
},

I will have to use .value if I want to update the ref's value inside the setup function, but in the template, Vue will unwrap president for me:

<template>
  <div>
    <h1>Company Roles</h1>
    <p><b>President:</b> {{ presidentRef }}</p>
    <button @click="changePresident">Change President</button>
  </div>
</template>

It can be challenging to remember which variables are reactive properties and which ones are ref. Something that helps is to use a naming convention where I add the suffix Ref to anything that is a ref, such as presidentRef. I don't have a lot of experience with Vue 3 yet, but for the time being, I plan to use that naming convention to see if it helps me get a better handle on the distinction between ref and reactive properties.

isRef

Vue 3 also provides the helper isRef that I can use to check if something is a ref.

console.log(isRef(executiveTeam.president)) //false
console.log(isRef(presidentRef)) //true

My Thoughts on the Vue 3 Reactivity API

This topic of ref and reactive has been the most challenging for me in my goal to learn Vue 3. There is more nuance to how these helpers are used in practice, and it becomes too much for an introduction post to try to cover all the different situations where I might have to make informed decisions about using ref and/or reactive and all the other helpers.

The Vue.js team is aware that this is one of the challenges of Vue 3 - the question of when to use ref or reactive does not always receive a simple answer. In the Composition API RFC they state:

Understandably, users may get confused regarding which to use between ref and reactive. First thing to know is that you will need to understand both to efficiently make use of the Composition API. Using one exclusively will most likely lead to esoteric workarounds or reinvented wheels.

I have come across many resources that suggest using just ref or just reactive to start. But I think it is worth the effort to learn the nuances of both. I agree with the Vue.js team: it's better to understand both ref and reactive if I'm going to use Vue 3 to its fullest potential. And that's what I plan to do.

While using just ref for primitives and reactive for objects is one suggested approach (suggested by the Vue.js team here), I would encourage you to dig deeper into the docs and resources out there to learn more about the Reactivity APIs. For a nuanced feature such as this, it's important to understand why certain approaches can be taken.

Conclusion

Please join me for my next post about reusability in Vue 3, including a discussion of composition functions (i.e. Vue composables). In my opinion, composables are the best thing about this new Composition API and they make it worth putting in the time to learn the harder concepts of Vue 3.

Please reach out on Twitter and let me know if you are enjoying this series on Vue 3.

Forem: SandraMartin

Astro-Storyblok Previews Site with Netlify's Branch Deploys Feature

Introduction

1. Create a branch-deploy in Netlify so you can have a special branch just for previewing content when clicking the 'save' button in Storyblok

Create Previews Branch

Create Environment Variables

2. In the Storyblok Visual Editor settings, add your Netlify preview branch URL to the Preview URLs

3. Set up your astro.config.mjs file so that it uses conditional logic to build your site in static mode if its in production, server mode in the preview environment

Set up .env

Add configuration to astro.config.mjs

4. Get page data from Storyblok using the public token in SSG mode, the preview token in SSR mode.

5. Create a Github action that automatically merges any changes made to main to the previews branch too

Getting the Visual Editor to work for Storyblok + Astro

Trying out the example project - it works great!

Trying out my own project - it doesn't work!

Walkthrough

Create a storyblok space

Create the project

Install dependencies

Set up SSR

Configure project to work with storyblok

Deploy site to Netlify

Make the storyblok visual editor work

Create webhook to publish storyblok edits to the site

Fetch Hosted Audio Streams In The Browser

Open WebSocket Channel

Fetch Hosted Audio Stream

Read Chunks and Send to Deepgram

Finishing Up

Enhance Your Audio With Dolby.io For Higher Quality Transcripts

Before We Start

Create a Node.js Project

Start the Enhance Job

Check the Enhance Job and Report Progress

Get Enhanced File URL

Run the Enhance Logic

Transcribe With Deepgram

Conclusion

Project Upgrade: Voice Controlled To-Do List App with Deepgram and Vue 3

Project Overview

Deepgram Live Streaming Logic

Additional Resources

Focus: Voice-Control Feature

Create the Component and Connect Deepgram

Analyze Deepgram Response for Voice Commands

Deepgram Audio Stream to Text

Add To-Do Item With Voice-Control

Create a Set of Regular Expressions

Use the Array Method .find()

Write a Method to Standardize the Command String

Test that the Item Matches the Regular Expression

Remove the Command Phrase from the String

Add the To-Do Item to the To-Do List Array

Watch For a Command, Add it To The List

Dealing with Misunderstood Commands

Delete To-Do Item With Voice-Control

Pseudo-Code:

Check-Off To-Do Item With Voice-Control

Conclusion

Build a To-do List App with Pinia and Vue 3

The Project

Getting Started with Pinia

Pinia - What is it?

Define a Pinia Store

State, Getters, and Actions

Create Initial State

Actions - Add and Delete an Item

Input Form to Add an Item

Access Pinia Store from a Component

Reactive Properties in Pinia

Todo List - Show Item

To-do List - Mark as Completed

To-Do List - Delete Item

Bonus Section: Add Typescript

Conclusion

Asynchronous Logic to Write a Vue 3 and Deepgram Captions Component

Introduction

Composables and Asynchronous Logic

Composable Using Async and Await

useDeepgramKey Composable

3. Set up your `astro.config.mjs` file so that it uses conditional logic to build your site in `static` mode if its in production, `server` mode in the preview environment

Set up `.env`

Add configuration to `astro.config.mjs`

Tip #2 - Use `.then()` to chain each composable that returns a promise to run asynchronously if returned values need to be in the same scope

Composable Example `useMenu.js`

Add the secret code value to `.env`