Forem: NLP App Developer & Advocate

Configure Symbl.ai to Listen for Action Items in a WebSocket Call

NLP App Developer & Advocate — Fri, 28 May 2021 21:59:35 +0000

Symbl.ai, a conversation intelligence platform for developers to create new experiences around conversation data, empowers developers to extend beyond mere automated speech recognition to contextual insights. Contextual insights are the results of Symbl.ai's Natural Language Processing algorithms to recognize speaker intentions. Among a few of the most common intentions speakers express in speech are follow-ups, questions or action items.

In short, action-items as a conversational entity recognized by Symbl.ai's platform are a reflection of a speaker's call to action during his or her conversation at any time.

First Step

The first step to access action items as Symbl.ai's contextual insights is to sign up. Register for an account at Symbl.ai (i.e., platform.symbl.ai). Grab both your appId and your appSecret. With both of those you receive your x-api-key.

If you would like to sign your JSON Web Tokens with your appId together with appSecret in a cURL command executed in the terminal, here is a code snippet.

curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

If you would like to authenticate into Symbl.ai's developer platform to signing the JWTs in Postman, the added benefit is that Symbl.ai's public workspace includes almost all of its APIs, including a section called "Labs" for its most experimental developer APIs.

Second Step

The second step is to familiarize yourself with Symbl.ai's documentation on its Streaming API. To review, loop through each of the following items from Symbl.ai's live speech-to-text tutorial: 1) creating a WebSocket, 2) setting its listeners, 3) creating an audio stream, or 4) handling the audio stream. If you wouldn't like to review these concepts, then paste the following copied code directly into your console:

/**
 * The JWT token you get after authenticating with our API.
 * Check the Authentication section of the documentation for more details.
 */
const accessToken = ""
const uniqueMeetingId = btoa("user@example.com")
const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;

const ws = new WebSocket(symblEndpoint);

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
  }
  if (data.type === 'message_response') {
    for (let message of data.messages) {
      console.log('Transcript (more accurate): ', message.payload.content);
    }
  }
  if (data.type === 'message' && data.message.hasOwnProperty('punctuated')) {
    console.log('Live transcript (less accurate): ', data.message.punctuated.transcript)
  }
  console.log(`Response type: ${data.type}. Object: `, data);
};

// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};

// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};

// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Websockets How-to', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'example@symbl.ai',
      name: 'Example Sample',
    }
  }));
};

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });

/**
 * The callback function which fires after a user gives the browser permission to use
 * the computer's microphone. Starts a recording session which sends the audio stream to
 * the WebSocket endpoint for processing.
 */
const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send audio stream to websocket.
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};


handleSuccess(stream);

Third Step

The third step Symbl.ai's WebSocket in your browser. Open up an instance of Chrome. Open up the console. Paste the above copied code directly into your console. After hitting enter, your WebSocket's messages start to pile up.

Fourth Step

After having run the Streaming API in the browser, you receive Symbl.ai's transcription from automated speech recognition in real-time. However, Symbl.ai enables you as a developer to extend far beyond merely automated speech recognition to contextual insights. In the code you run in the browser, you configure the WebSocket's event listener to capture contextual insights in real-time.

Navigate to the event listener ws.onmessage. Inside of the event listener, the ws.onmessage method provides you the ability to track events such as those events that are sent or received by the WebSocket. In the event listener there is a stream of raw data following on the WebSocket's protocol.

 if (data.type === 'topic_response') {
    for (let topic of data.topics) {
      console.log('Topic detected: ', topic.phrases)
    }
  }
  if (data.type === 'insight_response') {
    for (let insight of data.insights) {
      console.log('Insight detected: ', insight.payload.content);
       if (insight.type === "action_item" ) {
          console.log("Insight detected is an Action Item!!!")
      }
    }
  }

After adding the new log to your WebSocket's ws.onmessage method, the following is the full code:

/**
 * The JWT token you get after authenticating with our API.
 * Check the Authentication section of the documentation for more details.
 */
const accessToken = ""
const uniqueMeetingId = btoa("user@example.com")
const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;

const ws = new WebSocket(symblEndpoint);

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
  }
  if (data.type === 'message_response') {
    for (let message of data.messages) {
      console.log('Transcript (more accurate): ', message.payload.content);
    }
  }
if (data.type === 'topic_response') {
    for (let topic of data.topics) {
      console.log('Topic detected: ', topic.phrases)
    }
  }
  if (data.type === 'insight_response') {
    for (let insight of data.insights) {
      console.log('Insight detected: ', insight.payload.content);
       if (insight.type === "action_item" ) {
          console.log("Insight detected is an Action Item!!!")
      }
    }
  }
  if (data.type === 'message' && data.message.hasOwnProperty('punctuated')) {
    console.log('Live transcript (less accurate): ', data.message.punctuated.transcript)
  }
  console.log(`Response type: ${data.type}. Object: `, data);
};

// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};

// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};

// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Websockets How-to', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'example@symbl.ai',
      name: 'Example Sample',
    }
  }));
};

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });

/**
 * The callback function which fires after a user gives the browser permission to use
 * the computer's microphone. Starts a recording session which sends the audio stream to
 * the WebSocket endpoint for processing.
 */
const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send audio stream to websocket.
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};


handleSuccess(stream);

What's Next?

If you would like to add a listener for real-time sentiment analysis to your Symbl.ai configurations, Symbl.ai provides you the ability to listen to polarity scores on sentiments from messages in real-time. A basic knowledge of WebSockets is a first step. After logging sentiments, the next step is create a way to capture the data in real-time. If you would like to skip these blogs, feel free to download the code from Symbl.ai's GitHub where you can find the real-time sentiment analysis repo with instructions.

Sentiment analysis is only one way to handle the conversation data from Symbl.ai's enabled voice. Another way to handle Symbl.ai's enabled voice is to connect an API. In particular, action_items enable developers to create automated workflows from the detected insights in real-time so that those detected insights appear in external Third Party SaaS dashboards in real-time.

Imagine, for instance, creating JIRA tickets in real-time through a POST request after a Symbl.ai's insight for an action_item is detected in a live conversation. With Symbl.ai, you as a developer are empowered to connect, transform or visualize conversations in ways not yet imagined or implemented in real-life, since Symbl.ai's core product extends far beyond merely automated speech recognition.

Join Our Community

Symbl.ai invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai's GitHub.

Real-Time Sentiment Analysis on Messages

NLP App Developer & Advocate — Fri, 09 Apr 2021 15:34:03 +0000

Symbl.ai's API for sentiment analyzes messages in real-time, returning their polarity with a suggestion. If the polarity is below .5, the suggestion is negative. If above, the suggestion is positive. The suggestion, however, may be modified. If, for instance, you want to program a switch to toggle suggestions, the switch statement could return results for granulated polarities, since Symbl.ai's API for sentiment is fully programmable.

In the following blog you create a web app with which to map sentiments directly to message IDs in real-time with Symbl.ai's sentiment analysis API over a WebSocket in JavaScript running locally on a Python server. The result is a table of message IDs with sentiments.

A table of message IDs with sentiments may not seem like a whole lot but for a fully programmable API, there can be nothing better than generic functionality. So it is important to disclaim at the outset that the web app wastes absolutely no time on a user interface beyond what is required to demonstrate the generic functionality of the API.

Create the Web APP

In the web app, create the following files: index.html with a source folder containing an index.js file together with a style.css file.

In the style.css file, add the following lines:

body {
  font-family: sans-serif;
}

In the index.html, add the following lines:

<!DOCTYPE html>
<html>

<head>
    <title>Parcel Sandbox</title>
    <meta charset="UTF-8" />

    <script src="src/index.js">
    </script>
</head>

<body>
    <div id="app"></div>

    <button type="button" onclick="openSocket()">Click Me!</button>

    <div id="table-parent">
    </div>
</body>

</html>

The index.html achieves two goals. It creates a button for triggering the openSocket() method. The second is a table. In the openSocket() you program the JavaScript WebSocket for Symbl.ai's WebSocket for its platform. In the table you program a log, logging the message ID with its polarity score. You do not return to these two files later.

Sentiment Analysis: WebSocket in JavaScript & Events

The first step to sentiment analysis on messages with Symbl.ai is to establish a WebSocket. The second step is make a call to the sentiment API as the WebSocket's handler for events handles events the speaker makes.

To establish a WebSocket connection, the first step is to sign up for a free account at Symbl.ai. Register for an account at Symbl (i.e., https://platform.symbl.ai/). Grab both your appId and your appSecret. With both of those you authenticate either with a cURL command or with Postman so that you receive your x-api-key. Here is an example with cURL:

curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

After signing you, you receive free credits with which to make API calls. To make API calls on a WebSocket connection with Symbl.ai, create four const in your index.js file:

 const accessToken = "";
  const uniqueMeetingId = btoa("devrelations@symbl.ai");
  const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;
  const ws = new WebSocket(symblEndpoint);

The WebSocket connection requires both an unique meeting ID, as well as an accessToken (i.e., x-api-key you generate in cURl or Postman with a request containing an appId together an appSecret).

After configuring the endpoint for a WebSocket with Symbl.ai, add the following methods for handling events, 1) ws.onmessage, 2) ws.onerror, 3) ws.onclose.

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  console.log(event);
};
// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};
// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};

After these methods are created, create a method called onopen in the following way:

// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Establish a WebSocket Connection', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'devrelations@symbl.ai',
      name: 'Developer Relations',
    }
  }));

The onopen method contains many parts, the most important of which is speechRecognition where encoding is set to LINEAR16 while the hertz is set to 44100. To read more about best practices for streaming audio integrations, checkout the following blog: https://symbl.ai/best-practices-for-audio-integrations-with-symbl/.

The last but not least step is to configure the WebSocket to access the client's device (i.e., microphone).

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });
const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send to websocket
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};
handleSuccess(stream);

A detailed examination of a WebSocket's access to client's device is beyond the scope of the current blog, since our focus is on real-time sentiment analysis on messages.

The following is the full code for establishing the WebSocket connect:

const uniqueMeetingId = btoa('email@address.com');
const accessToken = '';
const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;
const ws = new WebSocket(symblEndpoint);
// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  console.log(event);
};
// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};
// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};
// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Websockets How-to', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'example@symbl.ai',
      name: 'Example Sample',
    }
  }));
};
const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });
const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send to websocket
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};
handleSuccess(stream);

Run the code directly in your browser's console without any reference to the web app. If you were able to log messages in the console, you successfully established a WebSocket connection. The next step is to configure onmessage to log the polarity score on those messages for realtime sentiment analysis.

Symbl.ai's Real-Time Sentiment Analysis API

The next step is to configure onmessage to log the polarity score on those messages for realtime sentiment analysis. The first step to analyze sentiments is to log the message IDs. You reconfigure onmessage to log message IDs.

Logging Message IDs

Our objective now is to make a call to the following API endpoint:

https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true

You note that passing ?sentiment=true into the API is the query parameter for a request to return messages with values for polarities on message IDs. Since the API endpoint requires no more than a ${conversationId}, the first step is to assign the conversationId to a constant.

  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
    const conversationId = data.message.data.conversationId;
console.log('onmessage event', event);
    // You can log sentiments on messages from data.message.data.conversationId

With the conversationId the next step is to configure an HTTP request to make a call to the API for sentiment analysis every time that the WebSocket logs an event. To configure an HTTP request to make a call to the API, make the call by configuring the headers as well as the authorization.

   const request = new XMLHttpRequest();
    request.responseType = "text";
    const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
    request.open("GET", sentimentEndpoint)
    request.setRequestHeader('Authorization', `Bearer ${accessToken}`);
    request.setRequestHeader('Content-Type', 'application/json');
    request.onreadystatechange=(e)=> {
      console.log(request.responseText)
    }
    request.send()
  }

With the request configured, the API endpoint makes a call every time the WebSocket handles an event that a speaker triggers. If you want, run the code in your console. It logs polarity values for message IDs. However, these logs do not map one to the other. The following is the full code for establishing the WebSocket connect:

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
    const conversationId = data.message.data.conversationId;
    console.log('onmessage event', event);
    // You can log sentiments on messages from data.message.data.conversationId 
    const request = new XMLHttpRequest();
    request.responseType = "text";
    const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
    request.open("GET", sentimentEndpoint)
    request.setRequestHeader('Authorization', `Bearer ${accessToken}`);
    request.setRequestHeader('Content-Type', 'application/json');
    request.onreadystatechange=(e)=> {
      console.log(request.responseText)
    }
    request.send()
  }
};

Refactoring the Code

It is time to refactor our code to conform to the web app built earlier, as well as log sentiments on messages. In the web app an element called table-parent is identified as <div id="table-parent">. The call to the API endpoint feeds both the message IDs together with the scores for polarity directly into the table-parent in real-time.

Refactoring the API call

Refactor the API call in the following way:

if (conversationId) {
        // You can log sentiments on messages from data.message.data.conversationId
        const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
        const response = await fetch(sentimentEndpoint, {
          method: 'GET',
          mode: 'cors',
          cache: 'no-cache',
          headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${accessToken}`
          }
        });

Configuring the Table

const resp = await response.json();
        if (response.ok) {
          let rows = "";
          for (let message of resp.messages) {
            if (cacheTable.indexOf(message.id) === -1) {
              console.log('Polarity: ', message.sentiment.polarity.score);
            }
            rows += `
              <tr>
                <td>${message.id}</td>
                <td>${message.sentiment.polarity.score}</td>
              </tr>
            `
            cacheTable.push(message.id);
          }
          let tableHtml = `
            <table>
              <thead>
                <tr>
                  <th>ID</th>
                  <th>Polarity</th>
                </tr>
              </thead>
              <tbody>
              ${rows}
              </tbody>
            </table>
          `;
          debugger;
          document.querySelector("#table-parent").innerHTML = tableHtml;
        }
      }
    }

The parent-table updates when a user speaks or listens.

Refactored Code

The following is the fully refactored code for mapping polarity values to message IDs in a table over a WebSocket connection in JavaScript with Symbl.ai's real-time sentiment analysis API:

ws.onmessage = async (event) => {
    // You can find the conversationId in event.message.data.conversationId;
    const data = JSON.parse(event.data);
    if (data.type === 'message' && data.message.hasOwnProperty('data')) {
      console.log('conversationId', data.message.data.conversationId);
      conversationId = data.message.data.conversationId;
      console.log('onmessage event', event);
    }
    if (data.type === 'message_response') {
      for (let message of data.messages) {
        console.log('Transcript (more accurate): ', message.payload.content);
      }
      if (conversationId) {
        // You can log sentiments on messages from data.message.data.conversationId
        const sentimentEndpoint = `https://api.symbl.ai/v1/conversations/${conversationId}/messages?sentiment=true`;
        const response = await fetch(sentimentEndpoint, {
          method: 'GET',
          mode: 'cors',
          cache: 'no-cache',
          headers: {
            'Content-Type': 'application/json',
            'Authorization': `Bearer ${accessToken}`
          }
        });
        const resp = await response.json();
        if (response.ok) {
          let rows = "";
          for (let message of resp.messages) {
            if (cacheTable.indexOf(message.id) === -1) {
              console.log('Polarity: ', message.sentiment.polarity.score);
            }
            rows += `
              <tr>
                <td>${message.id}</td>
                <td>${message.sentiment.polarity.score}</td>
              </tr>
            `
            cacheTable.push(message.id);
          }
          let tableHtml = `
            <table>
              <thead>
                <tr>
                  <th>ID</th>
                  <th>Polarity</th>
                </tr>
              </thead>
              <tbody>
              ${rows}
              </tbody>
            </table>
          `;
          debugger;
          document.querySelector("#table-parent").innerHTML = tableHtml;
        }
      }
    }
    if (data.type === 'topic_response') {
      for (let topic of data.topics) {
        console.log('Topic detected: ', topic.phrases)
      }
    }
    if (data.type === 'insight_response') {
      for (let insight of data.insights) {
        console.log('Insight detected: ', insight.payload.content);
      }
    }
    if (data.type === 'message' && data.message.hasOwnProperty('punctuated')) {
      console.log('Live transcript: ', data.message.punctuated.transcript);
    }
    // console.log(`Response type: ${data.type}. Object: `, data);
  };

To run the code locally, you have to avoid CORS. At the same time, you need to create an HTTP server in Python. Run the following line of code:

python3 -m http.server 8000

Python3 enables an http server to run locally on your host. With the application running, hit the click button. After hitting the click button, you should see message IDs mapped to values for polarity in real-time. If you run Command + Option + J, the following logs appear in your console.

API's Rigor with Symmetrical Augmentations

Symbl.ai's API is rigorous. It provides for sentiments in a way that other APIs do not. Symbl.ai's sentiment analysis, for instance, provides symmetrical augmentations for adverbial enhancements. If, for instance, you check the sentiment for "It is good", the score is .8. If you check the sentiment for "It is really good", the score is .9. What is true for positivity is true for negativity.

Conclusion

If you were able to successfully integrate Symbl’s API directly into JavaScript’s own software for enabling real-time conversations so that your transcribed a conversation live from the browser, congratulations!
If you look closely at the data, the conversationId may be applied to new API calls to access AI insights for action items, topics, etc… You can hit these API end points with cURL commands, Postman, or take a look at the section on Further Developer down below for ideas.

Community

Stuck? Free feel to ask us any questions on our Slack Channel or send us an email at devrelations@symbl.ai