An Easy way to get started programming with LLMs

Liam Griffiths — Thu, 22 Aug 2024 20:01:00 +0000

So you're interested in trying to build something cool with LLMs and are looking for a way to get started without too much fuss. I'd like to share a little bit of what I've been working on at Substrate and step through how to use it to start making useful things right away.

I'm using JavaScript here, but this can also be done with Python too!

To follow along you can download the Substrate SDK and get an account which comes with a bunch of free credit.

The basic setup is to npm install substrate, then open up a new JavaScript file.

Let's start with a basic interaction with an LLM. For now, we don't have to care about the specific model - we just want to get something working.

import { Substrate, ComputeText } from "substrate"

const substrate = new Substrate({ apiKey: "YOUR_API_KEY" })

const computeText = new ComputeText({ prompt: "tell me story about an owl" })
const result = await substrate.run(computeText)

console.log(result.json)

If you run that you should see the story the LLM created in just a few moments. Nice!

Let's introduce some prompt-engineering now and see if we can get the LLM to answer a question for us and see how we can get the response formatted in a useful way.

const question = "What are the most popular events at the Olympics?";

const computeText = new ComputeText({ 
  prompt: `
    Answer the following question, but do so with a numbered list with no more than 5 items.

    === Question
    ${question}`,
})
const result = await substrate.run(computeText)

console.log(result.json)

Here we've expanded our prompt to add in some additional instructions and provided our question to the LLM in a clear way.

Here's what it gave me:


Here are the top 5 most popular events at the Olympics:

1. **Track and Field (Athletics)**: Events like the 100m dash, long jump, high jump, and marathon are consistently among the most-watched and followed sports at the Olympics.

2. **Swimming**: Swimming events such as the 100m freestyle, 200m individual medley, and relay events have a huge following due to the world-class athletes competing in this sport.

3. **Gymnastics**: Artistic gymnastics, rhythmic gymnastics, and trampoline events are extremely popular, with the women's all-around competition being one of the most-watched events.

4. **Diving**: The precision and skill required for diving events make them thrilling to watch, with the platform and springboard competitions drawing large audiences.

5. **Figure Skating**: Figure skating events, including men's and women's singles, pairs, and ice dance, are highly anticipated and watched, especially during the winter Olympics.

Now let's see if we can extract this text into a more structured format. LLMs are really good at this and it can help us integrate the data into a program we'd like to build.

We're going to use a variant of ComputeText called ComputeJSON for this that lets us define a schema for the output. And in order to chain these together in one request, we'll need a way to do that too - so let's also import a set of helper functions from sb as well.

import { Substrate, ComputeText, ComputeJSON, sb } from "substrate"

Now let's setup ComputeJSON to extract the event name and a description as labeled fields.

const question = "What are the most popular events at the Olympics?";

const computeText = new ComputeText({ 
  prompt: `
    Answer the following question, but do so with a numbered list with no more than 5 items.

    === Question
    ${question}`,
})

const computeJSON = new ComputeJSON({
  prompt: sb.interpolate`Extract the event name and event description from the following: ${computeText.future.text}`,
  json_schema: {
    type: "object",
    properties: {
      events: {
        type: "array",
        items: {
          type: "object",
          properties: {
            name: { type: "string" },
            description: { type: "string" },
          }
        }
      }
    }
  }
})

const result = await substrate.run(computeJSON)
console.log(result.get(computeJSON).json_object);

Ok there's a little more going on here, so let me explain.

In the prompt for ComputeJSON we'd like to use the future result of the ComputeText response. We're using the helper sb.interpolate to do that. It works just like normal interpolation, but it let's us also use future too. In the json_schema field we're also defining the structure of the JSON data we'd like in the end.

You might have also noticed that we're now also using substrate.run(computeJSON) too. This will run both of these LLM calls as we have wired them together by including the computeText.future.text value as an input. The SDK is smart enough to figure out we'll need to run both.

Finally when we console.log at the end we're using a helper result.get(<node>) to make selecting a specific value from the result a little easier.

The output I got from running that last example looks like this:

{
  "events": [
    {
      "name": "track and field (athletics)",
      "description": "events like the 100m dash, long jump, high jump, and marathon are consistently among the most-watched and followed events at the olympics."
    },
    {
      "name": "gymnastics",
      "description": "artistic gymnastics, rhythmic gymnastics, and trampoline events have a huge following due to their technical difficulty and the athleticism required."
    },
    {
      "name": "swimming",
      "description": "the 100m freestyle, 200m individual medley, and relay events are always highly anticipated and closely watched."
    },
    {
      "name": "diving",
      "description": "the platform diving and springboard events showcase incredible skill and precision, making them a fan favorite."
    },
    {
      "name": "figure skating",
      "description": "the men's and women's singles, pairs, and ice dance events are highly popular due to their artistic expression and technical complexity."
    }
  ]
}

Not bad!

We've just created a pretty simple pipeline that chains two LLMs together and produced some fun structured data.

This pipeline is a two node graph in the end. And on Substrate you can create even more complex graphs by chaining as many different models together as you like. Models that output many kinds of data too: text, images, video, and audio.

I'll be blogging here a little more to share some fun ideas and ways of using Substrate to work with AI models like we did here with some very simple code. Stay tuned!

Building a static React App to share encrypted notes

Liam Griffiths — Sat, 28 Jul 2018 15:49:50 +0000

I started writing this little web app as a way to learn about setting up a website on Netlify and to try out some new libraries. It was a lot of fun to work on and has turned out to be quite useful for sharing little messages or passwords. I wanted to share it here to get some feedback and hopefully inspire some other people out there working on similar projects :)

The site is basically a little clone of the service privnote.com - and it works by encrypting a message in the browser and stores the encrypted message in a redis server. The password to decrypt the note never leaves the browser and the site gives you a link to share with someone else with the password appending to the hash section of the url - that way the password you share with them never leaves their browser either. When the other person opens the link the web app will fetch the note and decrypt it on the client side. I set it up so that the encrypted note on the server is deleted as soon as it is read from the database.

The source code is available on my GitHub page - feel free to check it out. I wrote out some instructions there on how to deploy it yourself if you want to run your own version of it.

Forem: Liam Griffiths

An Easy way to get started programming with LLMs

Building a static React App to share encrypted notes