Forem: Symbl.ai

Conversation Intelligence Quest: Python CI Challenge

Akanksha Bhasin — Thu, 20 Jan 2022 18:22:29 +0000

Test Out Your Python Skills on Building Conversation Intelligence Functions

Hey fellow devs!

We are thrilled to announce a new Python CI coding challenge on Qualified.io. Put your Python skills to the test and win some cool prizes and bragging rights by building conversation intelligence functions. In case you are not familiar with what Conversation Intelligence (CI) is, it’s essentially AI that can understand natural human conversations, explicitly and contextually.

📜 The Challenge:

In this cool challenge, you will take on the identity of a “Python Agent’ and venture into the mystic land of Convotopia. Combining your Python skills and CI APIs (provided by Symbl.ai), you will build functions for extracting the undiscovered secrets (conversation insights) hidden deep within Convotopia. Prepare yourself to take on greater challenges as you venture deeper into Convotopia.

The challenge is divided into 5 Levels, with each passing level you gain super powers to unlock the next level and be a Python hero in Convotopia.⛳

Start the challenge here: bit.ly/SymblCIQuest-Q1

⏳ Time Commitment:

The challenge will take approximately 1 hour and 30 mins (or less) to complete all levels, depending on your level of expertise with Python and/or conversation intelligence APIs (provided by Symbl.ai).

Why should you commit 1.5 hours of your life? You can win some cool prizes completing the challenge (see section below for details).

Approx time for each level:

Level 1. Access to Convotopia Land - 20 mins
Level 2. Connect with the locals - 20 mins
Level 3. Are you connecting with the locals? - 20 mins
Level 4. Let the extraction begin - 30 mins
Level 5. What is the ‘Talk of the town’ in Convotopia? - 30 mins

👨🏻‍💻👩🏻‍💻 Who Can Participate:

Our Challenge is open to programmers (or not) from all across the globe. If you are interested in building Python functions and exploring AI functions, this is the challenge for you.

🏆 Prizes:

Complete the challenge and win some cool prizes:

$50 Amazon gift card - The first 4 participants to complete the challenge with 100% by 15th February.
$25 Amazon gift card - not among the first 4 to complete the challenge or didn’t score 100%? No worries, every week, between 15th February to 30th March we will randomly select a winner who completed 80% or more of the challenge to win.
Symbl.ai Developer Swag - first 50 participants who complete 80% of the challenge by March 30th.
Certificate of completion - All participants who successfully complete the challenge, will receive a certificate. (bragging rights to share on social).

🔥 How to participate?

Head over to the Challenge to register yourselves & get started - Link ✔️
Join our slack channel 🚩
Publish a social media post about your participation and experience, and tag @symbldotai and @symbldev .📢
Add the #BuildWithSymbl #SymblQuest hashtag to your posts and share it with the community. ✨
Fill in the Challenge completion swag form.🎁

🤗 Join our Symbl.ai Community!

Become a part of our Slack community and do follow us on Twitter, Linkedin, Youtube, Github, Facebook and Instagram! ❤️

📩 Contact us

If you have any questions, please don't hesitate to drop me an email at (akanksha.bhasin@symbl.ai)

Happy Hacking!
May the source be with you! ✊🏻

✍🏽 By Akanksha Bhasin
DevRel @symbl.ai

Symbl.ai's Developer Month💻

Akanksha Bhasin — Thu, 20 Jan 2022 18:21:42 +0000

             Welcome to Symbl.ai’s Developer Month 👩‍🔬 👨‍🔬

Woot, woot!✨
Symbl.ai has launched Developer Month!❤️

To celebrate the thriving Developer Community all across the Globe we have launched Symbl.ai’s Developer Month! Join us to celebrate creativity, innovation & collaboration among the devs in these amazing 2 months.👩‍💻👨‍💻

How you can participate?
It's super easy!!
Join our events and fun activities such as meme contest, happy hours, Twitter Giveaway and more!
Build with Symbl.ai APIs or showcase your projects and learnings. Feel free to tag us and use the #SymblDevMonth hashtag along your journey; we would love to amplify your work!📢

How to join our Community?🌻
Join our Symbl.ai community here.
🔗Slack Group: https://bit.ly/symbldotai-slack
🔗Twitter: @SymblDev and @symbldotai
🔗Github: https://github.com/symblai

👉 Signup on the platform: https://bit.ly/Symbl-signup

What’s Next?

⭐Upcoming events in July

...

🚩July 14: Meme Contest

A one week contest hosted on slack to unleash your creativity skills! You are free to use any meme generator to create cool memes related to programming and development. The results will be announced on July 21 and top 5 submissions will receive exclusive Symbl swag.

🚩July 15: Hackathon Ideation Session 3: Transform One Iteration

App: Extract Time-Series Analysis of Sentiments from Calls with Symbl.ai’s Python SDK, Pandas, & Matplotlib.

🚩July 25: Giveaway

A twitter giveaway will run on Symbl.ai's twitter handle and 3 lucky winners will be awarded a prize.
Lot of cool swags in your way! Stay tuned!!

🚩July 30: Happy Hours

Happy hours are community connect sessions hosted for fun and to initiate engagement in the community.
Get a chance to network with folks all around the globe and build lifelong connections! Cool suprises are also lined up for you.😍

. . .

⭐Highlights from June

🚩June 16: Giveaway -

A Giveaway was hosted on Twitter on 16th June. 5 Winners were rewarded with Symbl.ai’s tshirt and exclusive stickers.

Giveaway Link

🚩June 17: Symbl.ai 101 -

Getting started with Symbl.ai on Postman and common integration and use cases were discussed.

Link to the session

🚩June 22: Slack AMA -

We are really glad to launch Symbl.ai’s AMA series. We had with us our 1st Guest - Toshish Jawale, co-founder and CTO of Symbl.ai
The AMA was hosted on #general channel of slack on Conversation Intelligence 101.

Link to the AMA discussion

🚩June 24: Symbl.ai Visualize session -

The audience was taught the basics of Symbl.ai as well as a demo was demonstrated on websockets and Real-Time Sentiment Analysis with Emojis was also shown.😃😐😞

Link to the session

🚩June 25: Developer Spotlight Series -

We are really glad to announce the first ever Symbl.ai's Developer Spotlight Series.
We had our first guest with us - Team Tier 4 who recently won the Symbl.ai’s Track Prize in Hackon 2.0.

Link to the discussion

✍🏽 By Akanksha Bhasin
May the source be with you!

Feel free to reach out to me via Twitter or Linkedin! ❤️

Symbl.ai’s Hacktoberfest 2021: Thank you for participating!

Akanksha Bhasin — Tue, 23 Nov 2021 07:19:06 +0000

It was our pleasure participating in the Hacktoberfest for the first time and we were absolutely thrilled to see the participation from our fellow developers of Symbl.ai Community. 👩‍💻👨‍💻

It feels like we just announced Symbl.ai’s Hacktoberfest a while back, but it has officially come to a close.

⭐ Some Amazing Stats from our 3 repos that were a part of hacktoberfest:

Symbl Agora - 32 contributions (12 issues + 20 PRs)
Symbl Twilio - 20 contributions (15 issues + 5 PRs)
Symbl Docs - 15 contributions (6 issues + 9 PRs)

On behalf of me and my Symbl.ai team, we thank each and every one of you for your contributions in open source. Your involvement and enthusiasm blew us away, with participants ranging from school kids, professionals, community members to people completely new to our ecosystem.

🏆 We would also like to appreciate our Top Contributors during the Hacktoberfest.

🚩 Huge shout out to our Project Maintainers and stakeholders from Symbl.ai Team: Vishal, Shivam, Akanksha, Rishabh, Marcelo, Vaibhav, Janhavi and Pema.

⏳ We’re currently in the process of reviewing all submissions, and everyone who completed our Hacktoberfest challenge will receive a limited edition T-shirt, and stickers depending on the contribution level.

✅ Participants who meet all the requirements will receive an email in the coming week with a swag form and shipping instructions. We appreciate your patience – they’re coming soon!

🤔 Anything you’d like to see at Symbl.ai’s Hacktoberfest next year? Let me know via email (akanksha.bhasin@symbl.ai).

💻 Hacktoberfest is over, but you still can contribute to our repositories anytime. Read more about us on Symbl.ai Docs + Open Source repos.🚀

🌻 Join our Symbl.ai community here

🔗 Slack Group: https://bit.ly/symbldotai-slack
🔗 Twitter: @SymblDev and @symbldotai
🔗Akanksha Bhasin’s Twitter

Happy Hacking!

Cheers,
Akanksha

Configure Symbl.ai to Listen for Action Items in a WebSocket Call

NLP App Developer & Advocate — Fri, 28 May 2021 21:59:35 +0000

Symbl.ai, a conversation intelligence platform for developers to create new experiences around conversation data, empowers developers to extend beyond mere automated speech recognition to contextual insights. Contextual insights are the results of Symbl.ai's Natural Language Processing algorithms to recognize speaker intentions. Among a few of the most common intentions speakers express in speech are follow-ups, questions or action items.

In short, action-items as a conversational entity recognized by Symbl.ai's platform are a reflection of a speaker's call to action during his or her conversation at any time.

First Step

The first step to access action items as Symbl.ai's contextual insights is to sign up. Register for an account at Symbl.ai (i.e., platform.symbl.ai). Grab both your appId and your appSecret. With both of those you receive your x-api-key.

If you would like to sign your JSON Web Tokens with your appId together with appSecret in a cURL command executed in the terminal, here is a code snippet.

curl -k -X POST "https://api.symbl.ai/oauth2/token:generate" \
     -H "accept: application/json" \
     -H "Content-Type: application/json" \
     -d "{ \"type\": \"application\", \"appId\": \"<appId>\", \"appSecret\": \"<appSecret>\"}"

If you would like to authenticate into Symbl.ai's developer platform to signing the JWTs in Postman, the added benefit is that Symbl.ai's public workspace includes almost all of its APIs, including a section called "Labs" for its most experimental developer APIs.

Second Step

The second step is to familiarize yourself with Symbl.ai's documentation on its Streaming API. To review, loop through each of the following items from Symbl.ai's live speech-to-text tutorial: 1) creating a WebSocket, 2) setting its listeners, 3) creating an audio stream, or 4) handling the audio stream. If you wouldn't like to review these concepts, then paste the following copied code directly into your console:

/**
 * The JWT token you get after authenticating with our API.
 * Check the Authentication section of the documentation for more details.
 */
const accessToken = ""
const uniqueMeetingId = btoa("user@example.com")
const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;

const ws = new WebSocket(symblEndpoint);

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
  }
  if (data.type === 'message_response') {
    for (let message of data.messages) {
      console.log('Transcript (more accurate): ', message.payload.content);
    }
  }
  if (data.type === 'message' && data.message.hasOwnProperty('punctuated')) {
    console.log('Live transcript (less accurate): ', data.message.punctuated.transcript)
  }
  console.log(`Response type: ${data.type}. Object: `, data);
};

// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};

// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};

// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Websockets How-to', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'example@symbl.ai',
      name: 'Example Sample',
    }
  }));
};

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });

/**
 * The callback function which fires after a user gives the browser permission to use
 * the computer's microphone. Starts a recording session which sends the audio stream to
 * the WebSocket endpoint for processing.
 */
const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send audio stream to websocket.
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};


handleSuccess(stream);

Third Step

The third step Symbl.ai's WebSocket in your browser. Open up an instance of Chrome. Open up the console. Paste the above copied code directly into your console. After hitting enter, your WebSocket's messages start to pile up.

Fourth Step

After having run the Streaming API in the browser, you receive Symbl.ai's transcription from automated speech recognition in real-time. However, Symbl.ai enables you as a developer to extend far beyond merely automated speech recognition to contextual insights. In the code you run in the browser, you configure the WebSocket's event listener to capture contextual insights in real-time.

Navigate to the event listener ws.onmessage. Inside of the event listener, the ws.onmessage method provides you the ability to track events such as those events that are sent or received by the WebSocket. In the event listener there is a stream of raw data following on the WebSocket's protocol.

 if (data.type === 'topic_response') {
    for (let topic of data.topics) {
      console.log('Topic detected: ', topic.phrases)
    }
  }
  if (data.type === 'insight_response') {
    for (let insight of data.insights) {
      console.log('Insight detected: ', insight.payload.content);
       if (insight.type === "action_item" ) {
          console.log("Insight detected is an Action Item!!!")
      }
    }
  }

After adding the new log to your WebSocket's ws.onmessage method, the following is the full code:

/**
 * The JWT token you get after authenticating with our API.
 * Check the Authentication section of the documentation for more details.
 */
const accessToken = ""
const uniqueMeetingId = btoa("user@example.com")
const symblEndpoint = `wss://api.symbl.ai/v1/realtime/insights/${uniqueMeetingId}?access_token=${accessToken}`;

const ws = new WebSocket(symblEndpoint);

// Fired when a message is received from the WebSocket server
ws.onmessage = (event) => {
  // You can find the conversationId in event.message.data.conversationId;
  const data = JSON.parse(event.data);
  if (data.type === 'message' && data.message.hasOwnProperty('data')) {
    console.log('conversationId', data.message.data.conversationId);
  }
  if (data.type === 'message_response') {
    for (let message of data.messages) {
      console.log('Transcript (more accurate): ', message.payload.content);
    }
  }
if (data.type === 'topic_response') {
    for (let topic of data.topics) {
      console.log('Topic detected: ', topic.phrases)
    }
  }
  if (data.type === 'insight_response') {
    for (let insight of data.insights) {
      console.log('Insight detected: ', insight.payload.content);
       if (insight.type === "action_item" ) {
          console.log("Insight detected is an Action Item!!!")
      }
    }
  }
  if (data.type === 'message' && data.message.hasOwnProperty('punctuated')) {
    console.log('Live transcript (less accurate): ', data.message.punctuated.transcript)
  }
  console.log(`Response type: ${data.type}. Object: `, data);
};

// Fired when the WebSocket closes unexpectedly due to an error or lost connetion
ws.onerror  = (err) => {
  console.error(err);
};

// Fired when the WebSocket connection has been closed
ws.onclose = (event) => {
  console.info('Connection to websocket closed');
};

// Fired when the connection succeeds.
ws.onopen = (event) => {
  ws.send(JSON.stringify({
    type: 'start_request',
    meetingTitle: 'Websockets How-to', // Conversation name
    insightTypes: ['question', 'action_item'], // Will enable insight generation
    config: {
      confidenceThreshold: 0.5,
      languageCode: 'en-US',
      speechRecognition: {
        encoding: 'LINEAR16',
        sampleRateHertz: 44100,
      }
    },
    speaker: {
      userId: 'example@symbl.ai',
      name: 'Example Sample',
    }
  }));
};

const stream = await navigator.mediaDevices.getUserMedia({ audio: true, video: false });

/**
 * The callback function which fires after a user gives the browser permission to use
 * the computer's microphone. Starts a recording session which sends the audio stream to
 * the WebSocket endpoint for processing.
 */
const handleSuccess = (stream) => {
  const AudioContext = window.AudioContext;
  const context = new AudioContext();
  const source = context.createMediaStreamSource(stream);
  const processor = context.createScriptProcessor(1024, 1, 1);
  const gainNode = context.createGain();
  source.connect(gainNode);
  gainNode.connect(processor);
  processor.connect(context.destination);
  processor.onaudioprocess = (e) => {
    // convert to 16-bit payload
    const inputData = e.inputBuffer.getChannelData(0) || new Float32Array(this.bufferSize);
    const targetBuffer = new Int16Array(inputData.length);
    for (let index = inputData.length; index > 0; index--) {
        targetBuffer[index] = 32767 * Math.min(1, inputData[index]);
    }
    // Send audio stream to websocket.
    if (ws.readyState === WebSocket.OPEN) {
      ws.send(targetBuffer.buffer);
    }
  };
};


handleSuccess(stream);

What's Next?

If you would like to add a listener for real-time sentiment analysis to your Symbl.ai configurations, Symbl.ai provides you the ability to listen to polarity scores on sentiments from messages in real-time. A basic knowledge of WebSockets is a first step. After logging sentiments, the next step is create a way to capture the data in real-time. If you would like to skip these blogs, feel free to download the code from Symbl.ai's GitHub where you can find the real-time sentiment analysis repo with instructions.

Sentiment analysis is only one way to handle the conversation data from Symbl.ai's enabled voice. Another way to handle Symbl.ai's enabled voice is to connect an API. In particular, action_items enable developers to create automated workflows from the detected insights in real-time so that those detected insights appear in external Third Party SaaS dashboards in real-time.

Imagine, for instance, creating JIRA tickets in real-time through a POST request after a Symbl.ai's insight for an action_item is detected in a live conversation. With Symbl.ai, you as a developer are empowered to connect, transform or visualize conversations in ways not yet imagined or implemented in real-life, since Symbl.ai's core product extends far beyond merely automated speech recognition.

Join Our Community

Symbl.ai invites developers to reach out to us via email at developer@symbl.ai, join our Slack channels, participate in our hackathons, fork our Postman public workspace, or git clone our repos at Symbl.ai's GitHub.