Any advice for a new Engineering Team Lead? It’s a Backend Team (just Incase being specific will help).

Opesanya Adebayo — Tue, 23 Jul 2019 05:25:42 +0000

I Can Tell Your Personality From Your Tweets

Opesanya Adebayo — Tue, 05 Jun 2018 09:55:05 +0000

In this tutorial, we’ll be looking at the Watson Personality Insights service. I assume you already have an IBM Cloud account. If you don’t, go through my Getting Started with IBM Watson tutorial to create one. It’s pretty straightforward.

The service essentially uses data supplied from social media, blog posts or other sources of textual data to identify psychological traits which determine purchase decisions, intent and behavioral traits. It’s quite an interesting service. You can get more information from the documentation.There are also demos here and here from the official documentation that you can play around with. You should have a swell time!

Let’s get our hands dirty with some code. In this tutorial, we’ll make a simple application that takes in a person's Twitter handle and returns a personality profile.

First step is to create a Personality Insights service. Visit the IBM Cloud Catalog, select Personality Insights and create the service. Then, create service credentials, and we are ready to go! If you have any issue creating the service and generating credentials, visit my Getting Started with IBM Watson tutorial.

To start using the Node.js SDK for IBM Watson, open your terminal (or command line), navigate to your project folder, and run npm i watson-developer-cloud -- save. You should have watson-developer-cloud added to your package.json file.

Once this is done, you can now write the following code to your project to use the Personality Insights service. Please note that I am loading my credentials from a .env file, and i’m using the dotenv module. You can run npm i dotenv --save to add this to your package.json file and be able to use it in your project. Your code should look like this now:

require("dotenv").config();
const PersonalityInsightsV3 =require('watson-developer-cloud/personality-insights/v3');
const personality_insights = new PersonalityInsightsV3({
    username: process.env.PERSONALITY_INSIGHTS_USERNAME,
    password: process.env.PERSONALITY_INSIGHTS_PASSWORD,
    version_date: process.env.PERSONALITY_INSIGHTS_VERSION_DATE

We’ll have to create a function that takes a Twitter handle and fetches tweets from that account. This means we’ll need the Twitter npm module. Go ahead and run npm i twitter --save to add the module to your package.json file. Create a file and save it as fetchTweets.js. It will contain our code to retrieve tweets.

let Twitter = require('twitter');
require("dotenv").config();

let client = new Twitter({
  consumer_key: process.env.TWITTER_CONSUMER_KEY,
  consumer_secret: process.env.TWITTER_CONSUMER_SECRET,
  access_token_key: process.env.TWITTER_ACCESS_TOKEN,
  access_token_secret: process.env.TWITTER_ACCESS_TOKEN_SECRET
});

const fetchTweets = (username) => {
  return new Promise((resolve, reject) => {

    let params = {
      screen_name: username,
      count: 200,
      include_rts: false,
      trim_user: true,
      exclude_replies: true,
      tweet_mode: "extended"
    };

    let tweets = [];

    const fetchTweets = (error, newTweets) => {
      if (error) {
        reject(Error(error));
      }
      // Filter out tweets with only relevant info
      filteredTweets = newTweets.map(function (tweet) {
        return {
          id: tweet.id_str,
          language: tweet.lang,
          contenttype: 'text/plain',
          content: tweet.full_text.replace('[^(\\x20-\\x7F)]*', ''),
          created: Date.parse(tweet.created_at),
          reply: tweet.in_reply_to_screen_name != null
        };
      });
      // check if tweets are actually retrieved and get more tweets if yes.
      if (newTweets.length > 1) {
        tweets = tweets.concat(filteredTweets);
        params.max_id = tweets[tweets.length - 1].id - 1;
        client.get('statuses/user_timeline', params, fetchTweets);
      } else {
        // if there are no more tweets to retrieve, return already retrieved tweets
        resolve(tweets);
      }
    };
    client.get('statuses/user_timeline', params, fetchTweets);

  });
};

module.exports = fetchTweets;

Now we can access tweets with these few lines of code. Let’s now edit the file that will contain code to collect the retrieved tweets and send them to the Personality Insights service. Edit personalityInsights.js. It should have the following code:

require("dotenv").config();
const readline = require('readline');
const PersonalityInsightsV3 = require('watson-developer-cloud/personality-insights/v3');
const personality_insights = new PersonalityInsightsV3({
  username: process.env.PERSONALITY_INSIGHTS_USERNAME,
  password: process.env.PERSONALITY_INSIGHTS_PASSWORD,
  version_date: '2017-10-13'
});

const fetchTweets = require('./fetchTweets');


const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout
});

rl.question('Please enter a twitter Handle for Watson to analyze...', (handle) => {

  console.log("Your results should show up soon. Thank you.");

  fetchTweets(handle).then((tweets) => {
    let params = {
      // Content items are tweets.
      content_items: tweets,
      consumption_preferences: true,
      raw_scores: true,
      headers: {
        'accept-language': 'en',
        'accept': 'application/json'
      }
    };
    personality_insights.profile(params, function (error, personalityProfile) {
      if (error && error.code == 400) {
        reject(Error("Ouch! You either do not have sufficient tweets, or your language is not supported. Sorry."));
      } else
        console.log(JSON.stringify(personalityProfile, null, 2));

    });

    rl.close();

  }).catch(err => console.error(err));

});

Let’s examine the code a little bit. In line 15, we include the fetchTweets.js module we created earlier, so that we can access the function to retrieve tweets in the personalityInsights.js file. We then call the function in line 27 and pass the handle the user entered to it, so we can retrieve the tweets and send them to the Personality Insights service.

Now you cN run node personalityInsights in your terminal to run your application. You should see something like this:

{
  "word_count": 15223,
  "processed_language": "en",
  "personality": [
     . . .
  ],
  "needs": [
     . . .
  ],
  "values": [
     . . .
  ],
  "behavior": [
     . . .
  ],
  "consumption_preferences": [
     . . .
   ],
  "warnings": []
}

Our result is a combination of four major parts:

Personality results based on Big5 personality traits and facets
Needs, which describe at a high level those aspects of a product that are likely to resonate with the author of the input text
Values, which describe motivating factors that influence the author’s decision-making
Consumption Preferences, which indicate the author’s likelihood to prefer different products, services, and activities.

You can get a lot more detail from the official documentation.

I know it's difficult to make sense of all that json stuff, so If you need a textual analysis of the resulting JSON response from the Personality Insights service, all you need to do is run this JSON response through the personality-text-summary npm module. Let's see how you can do that.

Go back to your terminal and run npm i personality-text-summary --save. This should add the module to your package.json file. Next step is to update our code so we can use it. We should have the following after updating:

require("dotenv").config();
const readline = require('readline');
const PersonalityInsightsV3 = require('watson-developer-cloud/personality-insights/v3');
const personality_insights = new PersonalityInsightsV3({
  username: process.env.PERSONALITY_INSIGHTS_USERNAME,
  password: process.env.PERSONALITY_INSIGHTS_PASSWORD,
  version_date: '2017-10-13'
});
const PersonalityTextSummaries = require('personality-text-summary');
const v3EnglishTextSummaries = new PersonalityTextSummaries({
  locale: 'en',
  version: 'v3'
});

const fetchTweets = require('./fetchTweets');


const rl = readline.createInterface({
  input: process.stdin,
  output: process.stdout
});

rl.question('Please enter a twitter Handle for Watson to analyze...', (handle) => {

  console.log("Your results should show up soon. Thank you.");

  fetchTweets(handle).then((tweets) => {
    let params = {
      // Content items are tweets.
      content_items: tweets,
      consumption_preferences: true,
      raw_scores: true,
      headers: {
        'accept-language': 'en',
        'accept': 'application/json'
      }
    };
    personality_insights.profile(params, function (error, personalityProfile) {
      if (error && error.code == 400) {
        reject(Error("Ouch! You either do not have sufficient tweets, or your language is not supported. Sorry."));
      } else
            console.log(getTextSummary(response));
            // console.log(JSON.stringify(personalityProfile, null, 2));

    });

    rl.close();

  }).catch(err => console.error(err));

});


const getTextSummary = personalityProfile => {
  let textSummary = v3EnglishTextSummaries.getSummary(personalityProfile);
  if (typeof (textSummary) !== 'string') {
    console.log("Could not get summary.");
  } else {
    return textSummary;
  }
};

If everything went well, you should have a result like this:

That’s it. We’ve come to the end of this tutorial. The code for this tutorial is available on Github.

I’ll be working on something really interesting in the next post, so get ready for a great time!

Forem: Opesanya Adebayo

Any advice for a new Engineering Team Lead? It’s a Backend Team (just Incase being specific will help).

I Can Tell Your Personality From Your Tweets