Forem: John Vitor Constant Lourenco

ServiceNow and OpenAI

John Vitor Constant Lourenco — Mon, 26 Feb 2024 02:58:49 +0000

Overview

Creating a chat completion is an easy task, but if you need to ride with context it can become a complex task.

In this guide I will show you an easy way to create a chat completion on Servicenow consuming the OpenAI endpoint without losing de context of comments of the task.

What you'll learn:

How to consume the OpenAI endpoint using the Servicenow
How to not loose task comments context in chat completion

What you'll need to know:

Javascript
Script includes
Bussiness rules
Outbound REST Messages
Table and form creation

Creating REST Message

Let's start creating the REST Messages
- Open the table REST Messages[sys_rest_message]
- New Record
- Name: OpenAI
- Endpoint: https://api.openai.com/
- Save

Then create a new HTTP Method in the related List:

Name: Create chat completion
HTTP method: POST
Authentication type: No authentication (because we will pass auth parameters on request headers)
HTTP Headers (Name/Value):
Authorization: Bearer ${token}
Content-Type: application/json
Content:

{
"model": "gpt-3.5-turbo",
"temperature": 1,
"stream": false,
"messages": [
    {
        "role": "system",
        "content": "You are a helpful assistant"
    },
    {
        "role": "user",
        "content": "What is Caesar cipher"
    }
]
}

Variable Substitutions (Related list)
- Name: token
- Escape type: No escaping
- Test value: your OpenAI token

Then click on Test and verify if it works.

Saving OpenAI credentials

We ill use api_key_credentials table to save our OpenAI token

Create a new Record on api_key_credentials
- Name: OpenAI
- API Key: your OpenAI token

Creating OpenAI table

Now we will create a table to save all Requests sent to OpenAI endpoint.
OpenAI completion endpoint give us a json like this:

{
  "id": "chatcmpl-8feVV3Y9dngfD793tUwLzJCsnuFWi",
  "object": "chat.completion",
  "created": 1704937849,
  "model": "gpt-3.5-turbo-0613",
  "choices": [
    {
      "index": 0,
      "message": {
        "role": "assistant",
        "content": "Caesar cipher is a simple form of substitution cipher where each letter in the plaintext is shifted a certain number of positions down or up the alphabet. For example, a shift of 3 would mean that A becomes D, B becomes E, and so on. The cipher is named after Julius Caesar, who allegedly used it to communicate secret messages."
      },
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 20,
    "completion_tokens": 69,
    "total_tokens": 89
  },
  "system_fingerprint": null
}

Then we will create a table with these fields to handle the response and do additional things (like save which Task record and user called this REST endpoint:

Table Fields:
- u_http_status_code: String
- u_model: String
- u_temperature: Integer
- u_last_input: String
- u_raw_input: String
- u_id: String
- u_object: String
- u_created: Integer
- u_raw_output: String
- u_output_content: String
- u_prompt_tokens: Integer
- u_completion_tokens: Integer
- u_total_tokens: Integer
- u_finish_reason: String
- u_estimated_prompt_token: Integer
- u_cost: Decimal
- u_gliderecord: Reference
- u_user: Reference

Creating script include

Now we need to create script include

Name: global.OpenAI

var OpenAI = Class.create();
OpenAI.prototype = {
    initialize: function() {
        var gr = new GlideRecord("api_key_credentials");
        gr.addQuery("name", "OpenAI");
        gr.setLimit(1);
        gr.query();
        gr.next();
        this.key = gr.api_key.getDecryptedValue();
    },


    createChatCompletion: function(input, record, alternativePrompt, customModel, customTemperature) {

        var req = record || '';
        var prompt = alternativePrompt || "You are a helpful assistant";
        var model = customModel || "gpt-3.5-turbo";
        var temperature = customTemperature || 1;

        var messages = [{
            "role": "system",
            "content": prompt
        }, ];

        if (typeof(input) == 'object') {
            for (var i = 0; i < input.length; i++) {
                messages.push({
                    "role": input[i].role,
                    "content": input[i].content,
                });
            }
        } else {
            messages.push({
                "role": "user",
                "content": input,
            });
        }

        var response;
        var httpStatus;
        var responseBody;
        try {
            var r = new sn_ws.RESTMessageV2('OpenAI', 'Create chat completion');
            r.setStringParameterNoEscape('token', this.key);
            var body = {
                "model": model,
                "temperature": temperature,
                "stream": false,
                "messages": messages
            };

            r.setRequestBody(JSON.stringify(body));
            response = r.execute();
            httpStatus = response.getStatusCode();
            responseBody = response.getBody();
            var responseBodyObj = JSON.parse(responseBody);
            var gr = new GlideRecord('u_open_ai');


            var lastInput = messages[messages.length - 1].content;

            gr.initialize();
            gr.setValue('u_raw_input', JSON.stringify(body));
            gr.setValue('u_estimated_prompt_token', this.getTokenEstimate(body));
            gr.setValue('u_raw_output', responseBody.toString());
            gr.setValue('u_gliderecord', req);
            gr.setValue('u_last_input', lastInput);
            gr.setValue('u_user', gs.getUser().getID());
            gr.setValue('u_http_status_code', httpStatus);
            gr.insert();

            return {
                'responseBody': responseBody,
                'httpStatus': httpStatus,
                'content': responseBodyObj.choices[0].message.content
            };
        } catch (ex) {
            return {
                'responseBody': responseBody,
                'httpStatus': httpStatus,
            };
        }
    },

    getTokenEstimate: function(obj) {
        function countTokens(str) {
            return str.split(' ').length +
                str.split(/[^a-zA-Z0-9]/).length;
        }

        return obj.messages
            .map(function(message) {
                return countTokens(message.content);
            })
            .reduce(function(a, b) {
                return a + b;
            }, 0);
    },

    createChatCompletionToString: function(input, record, alternativePrompt, customModel, customTemperature) {
        return JSON.stringify(this.createChatCompletion(input, record, alternativePrompt, customModel, customTemperature));
    },

    type: 'OpenAI'
};

Now let's look at what each code block do:

    initialize: function() {
        var gr = new GlideRecord("api_key_credentials");
        gr.addQuery("name", "OpenAI");
        gr.setLimit(1);
        gr.query();
        gr.next();
        this.key = gr.api_key.getDecryptedValue();
    },

In function initialize we retrieve the OpenAI key and decrypt it in execution time. WARNING! Depending on your instance configuration, roles, and Application, you will need to change how you retrieve this key. Another option is to save the key into a System Property variable and retrieve it using gs.getProperty(); (But I don't like this approach because you'll save your key in a raw text)

  createChatCompletion: function(input, record, alternativePrompt, customModel, customTemperature) {
        var req = record || '';
        var prompt = alternativePrompt || "You are a helpful assistant";
        var model = customModel || "gpt-3.5-turbo";
        var temperature = customTemperature || 1;

        var messages = [{
            "role": "system",
            "content": prompt
        },];

        if (typeof(input) == 'object') {
            for (var i = 0; i < input.length; i++) {
                messages.push({
                    "role": input[i].role,
                    "content": input[i].content,
                });
            }
        } else {
            messages.push({
                "role": "user",
                "content": input,
            });
        }
    },

In this function we create a chat completion using these parameters:

input: Is the input string or message object array. Input string can be used to get a simple answer from GPT, and message obejct array can be used to create a complex multi-turn request for GPT
record (optional): Is GlideRecord which called this function, this parameter is created only to associate the request with the document.
alternativePrompt (optional): We can change the default prompt using this parameter;
customModel (optional): As name says, if you want to change the default model, use this parameter;
customTemperature (optional): As name says, if you want to change the default Temperature, use this parameter; This initial part of code initialize the default parameters to sendto OpenAI endpoint, and check if input is string or message array object, if input is a string, the code convert to message array object to send to OpenAI completion endpoint.

  var response;
        var httpStatus;
        var responseBody;
        try {
            var r = new sn_ws.RESTMessageV2('OpenAI', 'Create chat completion');
            r.setStringParameterNoEscape('token', this.key);
            var body = {
                "model": model,
                "temperature": temperature,
                "stream": false,
                "messages": messages
            };

            r.setRequestBody(JSON.stringify(body));
            response = r.execute();
            httpStatus = response.getStatusCode();
            responseBody = response.getBody();
            var responseBodyObj = JSON.parse(responseBody);
            var gr = new GlideRecord('u_open_ai');


            var lastInput = messages[messages.length - 1].content;

            gr.initialize();
            gr.setValue('u_raw_input', JSON.stringify(body));
            gr.setValue('u_estimated_prompt_token', this.getTokenEstimate(body));
            gr.setValue('u_raw_output', responseBody.toString());
            gr.setValue('u_gliderecord', req);
            gr.setValue('u_last_input', lastInput);
            gr.setValue('u_user', gs.getUser().getID());
            gr.setValue('u_http_status_code', httpStatus);
            gr.insert();

            return {
                'responseBody': responseBody,
                'httpStatus': httpStatus,
                'content': responseBodyObj.choices[0].message.content
            };
        } catch (ex) {
            return {
                'responseBody': responseBody,
                'httpStatus': httpStatus,
            };
        }
    },

This part of code initializes de variables to create a POST request to OpenAI completion endpoint, then do a POST. After that we save the request response to u_open_ai table (How it is saved changes depending on the http status code)

Optional:

getTokenEstimate: function(obj) {
        function countTokens(str) {
            return str.split(' ').length +
                str.split(/[^a-zA-Z0-9]/).length;
        }

        return obj.messages
            .map(function(message) {
                return countTokens(message.content);
            })
            .reduce(function(a, b) {
                return a + b;
            }, 0);
    },

This function try to estimate how many token message array object will consume. This will be useful when we start to deal with very large context window (context larger than gpt can handle 16k+ for gpt-3.5-turbo-0125 or 128k+ for gpt-4-0125-preview).

Next steps:

How to call this function from GlideRecord forms such as Incident, Requested Item or Change Request
How to handle with very large context window (I will use the "recursive summary" approach).
How to consume internal databases like Knowledge Bases or E-mails using embeddings clusterization and search.
EXTRA: How to use the GeminiAI from Google.

Criando um tutorial no formato de Codelabs

John Vitor Constant Lourenco — Mon, 14 Mar 2022 03:44:53 +0000

Início

Problema Inicial

Quem nunca se deparou com dezenas de páginas de documentação e se perdeu no momento de tentar aprender algo novo?
Não importa o quão profundo, especial ou acertivo o seu guia é, o usuário irá perder o interesse na sua documentação ou guia depois de 30 segundos tentando se encontrar no meio de tantas páginas.
Portanto, diversas "Bigtechs" criaram sites e ferramentas para guiar o usuário e facilitar o aprendizado de determinada linguagem e ferramenta.

Objetivo

Google Developers Codelabs é um site que provê tutoriais guiados e intuítivos para auxiliar no aprendizado de desenvolvedores, e este guia tem como objetivo te ensinar a criar um Codelabs próprio, para engajar melhor a sua equipe ou comunidade.

Ferramentas utilizadas no Windows

Pré-Requisitos

Instalação do Go no Windows

Baixe a última versão do Go no Windows (1.17.8 no momento da criação deste tutorial) e instale como qualquer outro programa normal.

Para verificar se está instalado, basta digitar no Terminal:

go version

e verificar se a resposta é a versão do Go.

Instalação do Node.Js no Windows

Baixe a última versão do Node.js (16.14.0 no momento da criação deste tutorial) e instale como qualquer outro programa normal.

Para verificar se está instalado, basta digitar no Terminal:

Node --version

e verificar se a resposta é a versão do Node.js que você instalou.

Para verificar a instalação do npm, digite no Terminal:

npm -v

e verifique se a reposta é a versão equivalente do npm para aquele Node.js.

Instalação do claat no Windows

Abra o terminal de sua preferência e digite:

go install github.com/googlecodelabs/tools/claat@latest

Uma vez instalado digite claat -h no terminal e verifique se a resposta são os comandos do claat.

Gerando o HTML

O codelabs transforma um arquivo de markdown em um guia interativo, onde cada passo é representado pelo título do arquivo. Ou seja, basta criar um arquivo .md na sua pasta de preferencia.

Uma vez gerado, basta inserir um cabeçalho no arquivo de markdown como no exemplo abaixo e escrever sua documentação ou guia como qualquer outro arquivo .md convencional, lembrando que o Codelabs irá nomear o arquivo de acordo com o "id" do cabeçalho, o nome do guia de acordo com o "H1" e as páginas serão divididas de acordo com o "H2" do .md

author: John Vitor Constant de Oliveira Lourenço
summary: Documentação de APIs com o Readme
id: como-criar-um-codelabs
categories: codelabs,markdown
environments: Web
status: Published
feedback link: um link onde os usuários podem dar o feedback (talvez o repositório git)
analytics account: Google Analytics ID

O arquivo de markdown utilizado neste guia pode ser obtido aqui.

Uma vez criado o arquivo .md, basta navegar até a pasta do arquivo pelo Terminal:

cd "C:\Workbench\Sites\Como criar um codelabs"

e rodar o comando do claat:

claat export como-criar-um-codelabs.md

Finalizando, ele irá criar uma pasta com o arquivo .html já no formato de Codelabs.
Tanto o .md quanto o html gerado neste guia pode ser acessado aqui.

Fazendo deploy da aplicação

Em breve