Forem: Lizzie Siegle

Building a WNBA Analytics Dashboard with Streamlit, LangChain, and Cloudflare Workers AI

Lizzie Siegle — Fri, 18 Oct 2024 19:57:30 +0000

In this tutorial, we'll walk through the process of creating an interactive WNBA (Women's National Basketball Association) analytics dashboard. This powerful tool combines data visualization, AI-driven insights, and a chatbot interface to provide a comprehensive view of WNBA player statistics and team information.

Project Overview

This WNBA analytics dashboard offers the following features:

Player statistics visualization
Team comparisons
Interactive map showing WNBA team locations
AI-powered chatbot for WNBA-related queries
Data filtering and sorting capabilities
Responsive design for various screen sizes

We'll be using the following technologies:

Python
Streamlit for the web application framework
LangChain for natural language processing
Cloudflare Workers AI for machine learning capabilities
Pandas for data manipulation
Plotly for interactive charts
Folium for map visualizations

Step 1: Setting Up the Environment

First, let's set up our development environment:

Create a new Python virtual environment

python3 -m venv venv                                                            
source venv/bin/activate

Include the required packages at the top of your Python file. From the GitHub repo, you can download the requirements.txt file and run pip install -r requirements.txt to install them.

import base64
from dotenv import load_dotenv
import json
import os
import requests
import time
import webcolors

import streamlit as st
import folium
from streamlit_folium import st_folium

from geopy.geocoders import Nominatim
from geopy.exc import GeocoderTimedOut, GeocoderServiceError

from langchain.memory import ConversationBufferMemory
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.runnables import RunnablePassthrough

import numpy as np
import pandas as pd
from pathlib import Path

import plotly.graph_objects as go

Set up a Cloudflare account and obtain the necessary credentials for Workers AI. When you login to the dashboard, your account ID is the string of characters that follow https://dash.cloudflare.com/ To get a Workers AI auth token, click on AI on the lefthand sidebar followed by the blue Use Rest API button. Then, click Create a Workers AI API Token.

Add them to a .env file and reference them by adding these lines beneath the import statements.

load_dotenv()

# Cloudflare Workers AI setup
ACCOUNT_ID = os.getenv('CF_ACCOUNT_ID')
AUTH_TOKEN = os.getenv('CF_AUTH_TOKEN')

Configuring the Streamlit Page

Let's set up the basic Streamlit page configuration and add some custom CSS for styling:

st.set_page_config(page_title="WNBA Player Analytics Dashboard, AI Insights, && AI Assistant", page_icon="🏀", layout="wide")

# Custom CSS (truncated for brevity)
st.markdown("""
<style>
    .hover-link {
        color: #1E90FF;
        text-decoration: none;
        transition: color 0.3s ease;
    }
    .hover-link:hover {
        color: #FF4500;
        text-decoration: underline;
    }
    /* ... more custom CSS -> see https://github.com/elizabethsiegle/wnba-analytics-dash-ai-insights ... */
</style>
""", unsafe_allow_html=True)

Data Collection and Preprocessing

Create a function to fetch and preprocess WNBA player data:

def fetch_player_data(season):
    url = f"https://www.basketball-reference.com/wnba/years/{season}_per_game.html"
    dataframes = pd.read_html(url, header=0)
    df = dataframes[0]
    df = df[df.G != 'G'].fillna(0)  # Remove header rows and fill NaNs
    df = df.drop(['G'], axis=1)

    # Convert percentage columns to float
    percentage_columns = ['FG%', '3P%', '2P%', 'eFG%', 'FT%']
    for col in percentage_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col].astype(str).str.rstrip('%'), errors='coerce') / 100

    # Convert other numeric columns to float
    numeric_columns = ['Age', 'GS', 'MP', 'FG', 'FGA', '3P', '3PA', '2P', '2PA', 'FT', 'FTA', 'ORB', 'DRB', 'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS']
    for col in numeric_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col], errors='coerce')

    # Ensure 'Player' and 'Team' columns are strings
    df['Player'] = df['Player'].astype(str)
    df['Team'] = df['Team'].astype(str)

    # Handle any remaining problematic columns
    for col in df.columns:
        if df[col].dtype == object:
            df[col] = df[col].astype(str)

    return df

# Usage
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
player_data = fetch_player_data(selected_season)

This function fetches data from basketball-reference.comfor a given season and performs necessary data cleaning, processing it for our use.

Building the User Interface

Now, let's set up the basic structure of our Streamlit app and sidebar controls:

st.set_page_config(page_title="WNBA Analytics Dashboard", page_icon="🏀", layout="wide")
st.title("WNBA Player Analytics Dashboard")
# Sidebar for filters
st.sidebar.header('Filter Options')
st.sidebar.success("Filter players by season, team, and position to explore the data.")
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
# Team and position selection
teams = sorted([team for team in player_data.Team.unique() if team != 'TOT'])
selected_teams = st.sidebar.multiselect('Team', teams, teams)

positions = ['C', 'F', 'G', 'F-G', 'C-F']
selected_positions = st.sidebar.multiselect('Position', positions, positions)
# Fetch and display data
player_data = fetch_player_data(selected_season)
st.write(player_data)

This creates a simple dashboard with a season selector and displays the raw data.

Step 4: Adding Data Visualization

Here, we have a helper function to clean percentage data, converting string percentages to floats and handling various data types. This dictionary maps user-friendly names to column names in the dataset.

We create a 2x2 grid layout for organizing dashboard components.
The Chart Section (col1):

allows users to select a statistic and chart type (Pie or Bar).
filters data based on a minimum value slider.
creates and displays a chart of the top 5 players for the selected statistic. The stats Section (col2):
displays quick stats like average, highest player, and number of players shown.
shows a styled table of top players for the selected statistic.

Key Features include interactive elements (dropdowns, radio buttons, slider) for user customization and dynamic chart creation based on user selection.

AI Insights Section (col3) adds an AI-powered feature to the dashboard:

It creates a button that, when clicked, generates AI insights about the chart displayed above.

When the button is clicked, it checks if the necessary data is available in the session state.

If data is available, it creates a DataFrame of the top 5 players and their stats.

It then calls the generate_insights() function (which you'd need to implement using a language model like GPT-3 or GPT-4) to analyze this data.

The generated insights are displayed to the user.

A warning is shown to remind users that the insights are AI-generated and should be verified.

This feature demonstrates how to integrate AI capabilities into a data dashboard, providing users with quick, automated analysis of the visualized data.

Implementing Player Comparison

Add a section for comparing two players:

st.markdown('<div class="player-comparison-section">', unsafe_allow_html=True)
    st.subheader("🏀 Player Comparison (players must have played in the same season)")

    # Allow users to select players to compare
    players = player_data['Player'].unique()

    # Find the indices of Caitlin Clark and Angel Reese
    caitlin_index = players.tolist().index('Caitlin Clark') if 'Caitlin Clark' in players else 0
    angel_index = players.tolist().index('Angel Reese') if 'Angel Reese' in players else 0

    player1 = st.selectbox("Select first player", players,index=caitlin_index, key='player1')
    player2 = st.selectbox("Select second player", players, index=angel_index, key='player2')

    def normalize(value, min_value, max_value):
        try:
            value = float(value)
            return 100 * (value - min_value) / (max_value - min_value) if max_value > min_value else 50
        except (ValueError, TypeError):
            return 0  # or some default value for non-numeric entries

    if player1 and player2:
        # Get data for selected players
        stats1 = player_data[player_data['Player'] == player1].iloc[0]
        stats2 = player_data[player_data['Player'] == player2].iloc[0]

        # Select stats to compare
        stats_to_compare = ['PTS', 'AST', 'TRB', 'STL', 'BLK', 'FG%', '3P%', 'FT%']
        # Convert columns to numeric, replacing non-numeric values with NaN
        for stat in stats_to_compare:
            player_data[stat] = pd.to_numeric(player_data[stat], errors='coerce')

        normalized_stats = {}
        for stat in stats_to_compare:
            min_val = player_data[stat].min()
            max_val = player_data[stat].max()
            normalized_stats[stat] = [
                normalize(stats1[stat], min_val, max_val),
                normalize(stats2[stat], min_val, max_val)
            ]

        # Create a radar chart
        fig = go.Figure()

        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][0] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player1
        ))
        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][1] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player2
        ))

        fig.update_layout(
            polar=dict(radialaxis=dict(visible=True, range=[0, 100])),
            showlegend=True,
            legend=dict(
                font=dict(size=16),  # Increase font size
                itemsizing='constant',  # Make legend items a constant size
                itemwidth=30,  # Adjust item width
                yanchor="top",  # Anchor to the top
                y=0.99,  # Position at the top
                xanchor="right",  # Anchor to the right
                x=0.99,  # Position at the left
                bgcolor="rgba(255, 255, 255, 0.5)",  # Semi-transparent background
                bordercolor="Black",  # Border color
                borderwidth=2,  # Border width
            ),
            title=dict(
                text=f"{player1} vs {player2} Comparison",
                font=dict(size=24)  # Increase title font size
            ),
            width=700,  # Adjust as needed
            height=700  # Adjust as needed
        )

This creates a radar chart comparing two selected players across multiple statistics.

Adding an Interactive Map

Let's add a map showing WNBA team locations:

@st.cache_data
def create_wnba_map():
    # WNBA teams, their locations, and home page URLs
    wnba_teams = {
        'Atlanta Dream': ('Atlanta, GA', 'https://dream.wnba.com/'),
        'Chicago Sky': ('Chicago, IL', 'https://sky.wnba.com/'),
        'Connecticut Sun': ('Uncasville, CT', 'https://sun.wnba.com/'),
        'Dallas Wings': ('Arlington, TX', 'https://wings.wnba.com/'),
        'Indiana Fever': ('Indianapolis, IN', 'https://fever.wnba.com/'),
        'Las Vegas Aces': ('Las Vegas, NV', 'https://aces.wnba.com/'),
        'Los Angeles Sparks': ('Los Angeles, CA', 'https://sparks.wnba.com/'),
        'Minnesota Lynx': ('Minneapolis, MN', 'https://lynx.wnba.com/'),
        'New York Liberty': ('Brooklyn, NY', 'https://liberty.wnba.com/'),
        'Phoenix Mercury': ('Phoenix, AZ', 'https://mercury.wnba.com/'),
        'Seattle Storm': ('Seattle, WA', 'https://storm.wnba.com/'),
        'Washington Mystics': ('Washington, D.C.', 'https://mystics.wnba.com/')
    }

    # Create a map centered on the United States
    m = folium.Map(location=[39.8283, -98.5795], zoom_start=4)

    # Geocoding to get coordinates
    geolocator = Nominatim(user_agent="wnba_app")
    # Team name to abbreviation mapping
    team_abbr = {
        'Atlanta Dream': 'ATL', 'Chicago Sky': 'CHI', 'Connecticut Sun': 'CON',
        'Dallas Wings': 'DAL', 'Indiana Fever': 'IND', 'Las Vegas Aces': 'LVA',
        'Los Angeles Sparks': 'LAS', 'Minnesota Lynx': 'MIN', 'New York Liberty': 'NYL',
        'Phoenix Mercury': 'PHO', 'Seattle Storm': 'SEA', 'Washington Mystics': 'WAS'
    }

    # Add markers for each team
    for team, (city, url) in wnba_teams.items():
        try:
            location = geocode_with_retry(geolocator, city)
            if location is None:
                # Use fallback coordinates if geocoding fails
                lat, lon = fallback_coordinates[team]
            else:
                lat, lon = location.latitude, location.longitude
            # Get team abbreviation and color
            abbr = team_abbr.get(team, 'ATL')  # Default to ATL if not found
            hex_color = team_colors.get(abbr, '#000000')  # Default to black if color not found
            rgb = webcolors.hex_to_rgb(hex_color)
            closest_folium_color = closest_color(rgb)

            # Create popup HTML with team info and link
            popup_html = f"""
            <b>{team}</b><br>
            {city}<br>
            <a href="{url}" target="_blank">Visit Team Website</a>
            """

            folium.Marker(
                [lat, lon],
                popup=folium.Popup(popup_html, max_width=300),
                tooltip=team,
                icon=folium.Icon(color=closest_folium_color, icon='basketball', prefix='fa')
            ).add_to(m)
        except Exception as e:
            st.warning(f"Couldn't add marker for {team}: {str(e)}")

    return m

This creates an interactive map with markers for each WNBA team.

Step 8: Adding a Chatbot Interface

Finally, let's add a chatbot for answering WNBA-related questions:

    st.markdown('<div class="chatbot-section">', unsafe_allow_html=True)
    st.subheader("🏀 Chat💬 w/ WNBA AI Assistant powered by LangChain && Cloudflare Workers AI🤖")

    # Add a loading message
    chat_loading = st.empty()
    chat_loading.info("Chat is initializing... This may take a few moments.")

    # Initialize the LLM and conversation chain
    @st.cache_resource
    def initialize_chat(filtered_data: pd.DataFrame):
        llm = CloudflareWorkersAI(
            account_id=ACCOUNT_ID,
            api_token=AUTH_TOKEN,
            model="@cf/meta/llama-2-7b-chat-int8"
        )
        # Convert filtered_data to a string representation
        data_context = filtered_data.to_string()

        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a knowledgeable assistant specializing in WNBA statistics, players, and teams. 
            Provide accurate and helpful information about the WNBA.
            Here's the current WNBA data you have access to:
            {data_context}
            Use this data to answer questions, but don't mention the data directly unless asked."""),
            ("human", "{input}"),
            ("ai", "{agent_scratchpad}")
        ])

        memory = ConversationBufferMemory(return_messages=True, output_key="agent_scratchpad")
        def get_chat_history(inputs):
            return memory.chat_memory.messages

        chain = (
            RunnablePassthrough.assign(
                agent_scratchpad=get_chat_history,
                data_context=lambda _: data_context[:100] + "..." # Truncate for brevity 
            )
            | prompt
            | llm
        )

        return chain, memory, data_context

    # Initialize the chat
    chain, memory, data_context = initialize_chat(filtered_data)

    # Remove the loading message
    chat_loading.empty()

    # Initialize chat history
    if "messages" not in st.session_state:
        st.session_state.messages = []

    # Display chat messages from history on rerun
    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])

    # React to user input
    if user_input := st.chat_input("Ask me anything about WNBA stats, players, or teams!"):
        # Display user message in chat message container
        st.chat_message("user").markdown(user_input)
        # Add user message to chat history
        st.session_state.messages.append({"role": "user", "content": user_input})

        # Get AI response
        with st.spinner("Thinking..."):
            try:
                response = chain.invoke({
                    "input": user_input,
                    "data_context": data_context
                })

                if not response or response.strip() == "":
                    response = "I apologize, but I couldn't generate a response. This could be due to an issue with the AI model or the input. Please try asking your question in a different way or try again later."
            except Exception as e:
                response = f"An error occurred: {str(e)}"
                st.error(f"Debug: Error details: {e}")

        # After getting the response from the model
        if isinstance(response, list) and len(response) > 0 and hasattr(response[0], 'content'):
            response_text = response[0].content
        elif isinstance(response, dict) and 'content' in response:
            response_text = response['content']
        elif isinstance(response, str):
            response_text = response
        else:
            response_text = str(response)


        # Display assistant response in chat message container
        with st.chat_message("assistant"):
            st.markdown(response_text)
        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": response_text})

        # Update memory
        memory.chat_memory.add_user_message(user_input)
        memory.chat_memory.add_ai_message(response_text)


        # Add some styling to make the chat interface look better
        st.markdown("""
        <style>
        .stChatFloatingInputContainer {
            bottom: 20px;
            background-color: #f0f2f6;
            padding: 10px;
            border-radius: 10px;
            box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
        }
        </style>
        """, unsafe_allow_html=True)

This creates a chatbot interface that can answer questions about WNBA statistics using the provided data.

Conclusion

We've now built a comprehensive WNBA analytics dashboard with data visualization, AI insights, and a chatbot interface. This project demonstrates the power of combining data analysis with machine learning to create interactive and informative tools for sports analytics.

The complete code can be found here on GitHub.

Happy coding, and enjoy exploring WNBA statistics with your new analytics dashboard!

Building a WNBA Analytics Dashboard with Streamlit, LangChain, and Cloudflare Workers AI

Lizzie Siegle — Fri, 18 Oct 2024 19:57:30 +0000

Project Overview

This WNBA analytics dashboard offers the following features:

Player statistics visualization
Team comparisons
Interactive map showing WNBA team locations
AI-powered chatbot for WNBA-related queries
Data filtering and sorting capabilities
Responsive design for various screen sizes

We'll be using the following technologies:

Python
Streamlit for the web application framework
LangChain for natural language processing
Cloudflare Workers AI for machine learning capabilities
Pandas for data manipulation
Plotly for interactive charts
Folium for map visualizations

Step 1: Setting Up the Environment

First, let's set up our development environment:

Create a new Python virtual environment

python3 -m venv venv                                                            
source venv/bin/activate

Include the required packages at the top of your Python file. From the GitHub repo, you can download the requirements.txt file and run pip install -r requirements.txt to install them.

import base64
from dotenv import load_dotenv
import json
import os
import requests
import time
import webcolors

import streamlit as st
import folium
from streamlit_folium import st_folium

from geopy.geocoders import Nominatim
from geopy.exc import GeocoderTimedOut, GeocoderServiceError

from langchain.memory import ConversationBufferMemory
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.runnables import RunnablePassthrough

import numpy as np
import pandas as pd
from pathlib import Path

import plotly.graph_objects as go

Set up a Cloudflare account and obtain the necessary credentials for Workers AI. When you login to the dashboard, your account ID is the string of characters that follow https://dash.cloudflare.com/ To get a Workers AI auth token, click on AI on the lefthand sidebar followed by the blue Use Rest API button. Then, click Create a Workers AI API Token.

Add them to a .env file and reference them by adding these lines beneath the import statements.

load_dotenv()

# Cloudflare Workers AI setup
ACCOUNT_ID = os.getenv('CF_ACCOUNT_ID')
AUTH_TOKEN = os.getenv('CF_AUTH_TOKEN')

Configuring the Streamlit Page

Let's set up the basic Streamlit page configuration and add some custom CSS for styling:

st.set_page_config(page_title="WNBA Player Analytics Dashboard, AI Insights, && AI Assistant", page_icon="🏀", layout="wide")

# Custom CSS (truncated for brevity)
st.markdown("""
<style>
    .hover-link {
        color: #1E90FF;
        text-decoration: none;
        transition: color 0.3s ease;
    }
    .hover-link:hover {
        color: #FF4500;
        text-decoration: underline;
    }
    /* ... more custom CSS -> see https://github.com/elizabethsiegle/wnba-analytics-dash-ai-insights ... */
</style>
""", unsafe_allow_html=True)

Data Collection and Preprocessing

Create a function to fetch and preprocess WNBA player data:

def fetch_player_data(season):
    url = f"https://www.basketball-reference.com/wnba/years/{season}_per_game.html"
    dataframes = pd.read_html(url, header=0)
    df = dataframes[0]
    df = df[df.G != 'G'].fillna(0)  # Remove header rows and fill NaNs
    df = df.drop(['G'], axis=1)

    # Convert percentage columns to float
    percentage_columns = ['FG%', '3P%', '2P%', 'eFG%', 'FT%']
    for col in percentage_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col].astype(str).str.rstrip('%'), errors='coerce') / 100

    # Convert other numeric columns to float
    numeric_columns = ['Age', 'GS', 'MP', 'FG', 'FGA', '3P', '3PA', '2P', '2PA', 'FT', 'FTA', 'ORB', 'DRB', 'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS']
    for col in numeric_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col], errors='coerce')

    # Ensure 'Player' and 'Team' columns are strings
    df['Player'] = df['Player'].astype(str)
    df['Team'] = df['Team'].astype(str)

    # Handle any remaining problematic columns
    for col in df.columns:
        if df[col].dtype == object:
            df[col] = df[col].astype(str)

    return df

# Usage
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
player_data = fetch_player_data(selected_season)

This function fetches data from basketball-reference.comfor a given season and performs necessary data cleaning, processing it for our use.

Building the User Interface

Now, let's set up the basic structure of our Streamlit app and sidebar controls:

st.set_page_config(page_title="WNBA Analytics Dashboard", page_icon="🏀", layout="wide")
st.title("WNBA Player Analytics Dashboard")
# Sidebar for filters
st.sidebar.header('Filter Options')
st.sidebar.success("Filter players by season, team, and position to explore the data.")
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
# Team and position selection
teams = sorted([team for team in player_data.Team.unique() if team != 'TOT'])
selected_teams = st.sidebar.multiselect('Team', teams, teams)

positions = ['C', 'F', 'G', 'F-G', 'C-F']
selected_positions = st.sidebar.multiselect('Position', positions, positions)
# Fetch and display data
player_data = fetch_player_data(selected_season)
st.write(player_data)

This creates a simple dashboard with a season selector and displays the raw data.

Step 4: Adding Data Visualization

We create a 2x2 grid layout for organizing dashboard components.
The Chart Section (col1):

allows users to select a statistic and chart type (Pie or Bar).
filters data based on a minimum value slider.
creates and displays a chart of the top 5 players for the selected statistic. The stats Section (col2):
displays quick stats like average, highest player, and number of players shown.
shows a styled table of top players for the selected statistic.

Key Features include interactive elements (dropdowns, radio buttons, slider) for user customization and dynamic chart creation based on user selection.

AI Insights Section (col3) adds an AI-powered feature to the dashboard:

It creates a button that, when clicked, generates AI insights about the chart displayed above.

When the button is clicked, it checks if the necessary data is available in the session state.

If data is available, it creates a DataFrame of the top 5 players and their stats.

It then calls the generate_insights() function (which you'd need to implement using a language model like GPT-3 or GPT-4) to analyze this data.

The generated insights are displayed to the user.

A warning is shown to remind users that the insights are AI-generated and should be verified.

This feature demonstrates how to integrate AI capabilities into a data dashboard, providing users with quick, automated analysis of the visualized data.

Implementing Player Comparison

Add a section for comparing two players:

st.markdown('<div class="player-comparison-section">', unsafe_allow_html=True)
    st.subheader("🏀 Player Comparison (players must have played in the same season)")

    # Allow users to select players to compare
    players = player_data['Player'].unique()

    # Find the indices of Caitlin Clark and Angel Reese
    caitlin_index = players.tolist().index('Caitlin Clark') if 'Caitlin Clark' in players else 0
    angel_index = players.tolist().index('Angel Reese') if 'Angel Reese' in players else 0

    player1 = st.selectbox("Select first player", players,index=caitlin_index, key='player1')
    player2 = st.selectbox("Select second player", players, index=angel_index, key='player2')

    def normalize(value, min_value, max_value):
        try:
            value = float(value)
            return 100 * (value - min_value) / (max_value - min_value) if max_value > min_value else 50
        except (ValueError, TypeError):
            return 0  # or some default value for non-numeric entries

    if player1 and player2:
        # Get data for selected players
        stats1 = player_data[player_data['Player'] == player1].iloc[0]
        stats2 = player_data[player_data['Player'] == player2].iloc[0]

        # Select stats to compare
        stats_to_compare = ['PTS', 'AST', 'TRB', 'STL', 'BLK', 'FG%', '3P%', 'FT%']
        # Convert columns to numeric, replacing non-numeric values with NaN
        for stat in stats_to_compare:
            player_data[stat] = pd.to_numeric(player_data[stat], errors='coerce')

        normalized_stats = {}
        for stat in stats_to_compare:
            min_val = player_data[stat].min()
            max_val = player_data[stat].max()
            normalized_stats[stat] = [
                normalize(stats1[stat], min_val, max_val),
                normalize(stats2[stat], min_val, max_val)
            ]

        # Create a radar chart
        fig = go.Figure()

        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][0] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player1
        ))
        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][1] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player2
        ))

        fig.update_layout(
            polar=dict(radialaxis=dict(visible=True, range=[0, 100])),
            showlegend=True,
            legend=dict(
                font=dict(size=16),  # Increase font size
                itemsizing='constant',  # Make legend items a constant size
                itemwidth=30,  # Adjust item width
                yanchor="top",  # Anchor to the top
                y=0.99,  # Position at the top
                xanchor="right",  # Anchor to the right
                x=0.99,  # Position at the left
                bgcolor="rgba(255, 255, 255, 0.5)",  # Semi-transparent background
                bordercolor="Black",  # Border color
                borderwidth=2,  # Border width
            ),
            title=dict(
                text=f"{player1} vs {player2} Comparison",
                font=dict(size=24)  # Increase title font size
            ),
            width=700,  # Adjust as needed
            height=700  # Adjust as needed
        )

This creates a radar chart comparing two selected players across multiple statistics.

Adding an Interactive Map

Let's add a map showing WNBA team locations:

@st.cache_data
def create_wnba_map():
    # WNBA teams, their locations, and home page URLs
    wnba_teams = {
        'Atlanta Dream': ('Atlanta, GA', 'https://dream.wnba.com/'),
        'Chicago Sky': ('Chicago, IL', 'https://sky.wnba.com/'),
        'Connecticut Sun': ('Uncasville, CT', 'https://sun.wnba.com/'),
        'Dallas Wings': ('Arlington, TX', 'https://wings.wnba.com/'),
        'Indiana Fever': ('Indianapolis, IN', 'https://fever.wnba.com/'),
        'Las Vegas Aces': ('Las Vegas, NV', 'https://aces.wnba.com/'),
        'Los Angeles Sparks': ('Los Angeles, CA', 'https://sparks.wnba.com/'),
        'Minnesota Lynx': ('Minneapolis, MN', 'https://lynx.wnba.com/'),
        'New York Liberty': ('Brooklyn, NY', 'https://liberty.wnba.com/'),
        'Phoenix Mercury': ('Phoenix, AZ', 'https://mercury.wnba.com/'),
        'Seattle Storm': ('Seattle, WA', 'https://storm.wnba.com/'),
        'Washington Mystics': ('Washington, D.C.', 'https://mystics.wnba.com/')
    }

    # Create a map centered on the United States
    m = folium.Map(location=[39.8283, -98.5795], zoom_start=4)

    # Geocoding to get coordinates
    geolocator = Nominatim(user_agent="wnba_app")
    # Team name to abbreviation mapping
    team_abbr = {
        'Atlanta Dream': 'ATL', 'Chicago Sky': 'CHI', 'Connecticut Sun': 'CON',
        'Dallas Wings': 'DAL', 'Indiana Fever': 'IND', 'Las Vegas Aces': 'LVA',
        'Los Angeles Sparks': 'LAS', 'Minnesota Lynx': 'MIN', 'New York Liberty': 'NYL',
        'Phoenix Mercury': 'PHO', 'Seattle Storm': 'SEA', 'Washington Mystics': 'WAS'
    }

    # Add markers for each team
    for team, (city, url) in wnba_teams.items():
        try:
            location = geocode_with_retry(geolocator, city)
            if location is None:
                # Use fallback coordinates if geocoding fails
                lat, lon = fallback_coordinates[team]
            else:
                lat, lon = location.latitude, location.longitude
            # Get team abbreviation and color
            abbr = team_abbr.get(team, 'ATL')  # Default to ATL if not found
            hex_color = team_colors.get(abbr, '#000000')  # Default to black if color not found
            rgb = webcolors.hex_to_rgb(hex_color)
            closest_folium_color = closest_color(rgb)

            # Create popup HTML with team info and link
            popup_html = f"""
            <b>{team}</b><br>
            {city}<br>
            <a href="{url}" target="_blank">Visit Team Website</a>
            """

            folium.Marker(
                [lat, lon],
                popup=folium.Popup(popup_html, max_width=300),
                tooltip=team,
                icon=folium.Icon(color=closest_folium_color, icon='basketball', prefix='fa')
            ).add_to(m)
        except Exception as e:
            st.warning(f"Couldn't add marker for {team}: {str(e)}")

    return m

This creates an interactive map with markers for each WNBA team.

Step 8: Adding a Chatbot Interface

Finally, let's add a chatbot for answering WNBA-related questions:

    st.markdown('<div class="chatbot-section">', unsafe_allow_html=True)
    st.subheader("🏀 Chat💬 w/ WNBA AI Assistant powered by LangChain && Cloudflare Workers AI🤖")

    # Add a loading message
    chat_loading = st.empty()
    chat_loading.info("Chat is initializing... This may take a few moments.")

    # Initialize the LLM and conversation chain
    @st.cache_resource
    def initialize_chat(filtered_data: pd.DataFrame):
        llm = CloudflareWorkersAI(
            account_id=ACCOUNT_ID,
            api_token=AUTH_TOKEN,
            model="@cf/meta/llama-2-7b-chat-int8"
        )
        # Convert filtered_data to a string representation
        data_context = filtered_data.to_string()

        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a knowledgeable assistant specializing in WNBA statistics, players, and teams. 
            Provide accurate and helpful information about the WNBA.
            Here's the current WNBA data you have access to:
            {data_context}
            Use this data to answer questions, but don't mention the data directly unless asked."""),
            ("human", "{input}"),
            ("ai", "{agent_scratchpad}")
        ])

        memory = ConversationBufferMemory(return_messages=True, output_key="agent_scratchpad")
        def get_chat_history(inputs):
            return memory.chat_memory.messages

        chain = (
            RunnablePassthrough.assign(
                agent_scratchpad=get_chat_history,
                data_context=lambda _: data_context[:100] + "..." # Truncate for brevity 
            )
            | prompt
            | llm
        )

        return chain, memory, data_context

    # Initialize the chat
    chain, memory, data_context = initialize_chat(filtered_data)

    # Remove the loading message
    chat_loading.empty()

    # Initialize chat history
    if "messages" not in st.session_state:
        st.session_state.messages = []

    # Display chat messages from history on rerun
    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])

    # React to user input
    if user_input := st.chat_input("Ask me anything about WNBA stats, players, or teams!"):
        # Display user message in chat message container
        st.chat_message("user").markdown(user_input)
        # Add user message to chat history
        st.session_state.messages.append({"role": "user", "content": user_input})

        # Get AI response
        with st.spinner("Thinking..."):
            try:
                response = chain.invoke({
                    "input": user_input,
                    "data_context": data_context
                })

                if not response or response.strip() == "":
                    response = "I apologize, but I couldn't generate a response. This could be due to an issue with the AI model or the input. Please try asking your question in a different way or try again later."
            except Exception as e:
                response = f"An error occurred: {str(e)}"
                st.error(f"Debug: Error details: {e}")

        # After getting the response from the model
        if isinstance(response, list) and len(response) > 0 and hasattr(response[0], 'content'):
            response_text = response[0].content
        elif isinstance(response, dict) and 'content' in response:
            response_text = response['content']
        elif isinstance(response, str):
            response_text = response
        else:
            response_text = str(response)


        # Display assistant response in chat message container
        with st.chat_message("assistant"):
            st.markdown(response_text)
        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": response_text})

        # Update memory
        memory.chat_memory.add_user_message(user_input)
        memory.chat_memory.add_ai_message(response_text)


        # Add some styling to make the chat interface look better
        st.markdown("""
        <style>
        .stChatFloatingInputContainer {
            bottom: 20px;
            background-color: #f0f2f6;
            padding: 10px;
            border-radius: 10px;
            box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
        }
        </style>
        """, unsafe_allow_html=True)

This creates a chatbot interface that can answer questions about WNBA statistics using the provided data.

Conclusion

The complete code can be found here on GitHub.

Happy coding, and enjoy exploring WNBA statistics with your new analytics dashboard!

Building a WNBA Analytics Dashboard with Streamlit, LangChain, and Cloudflare Workers AI

Lizzie Siegle — Fri, 18 Oct 2024 19:57:30 +0000

Project Overview

This WNBA analytics dashboard offers the following features:

Player statistics visualization
Team comparisons
Interactive map showing WNBA team locations
AI-powered chatbot for WNBA-related queries
Data filtering and sorting capabilities
Responsive design for various screen sizes

We'll be using the following technologies:

Python
Streamlit for the web application framework
LangChain for natural language processing
Cloudflare Workers AI for machine learning capabilities
Pandas for data manipulation
Plotly for interactive charts
Folium for map visualizations

Step 1: Setting Up the Environment

First, let's set up our development environment:

Create a new Python virtual environment

python3 -m venv venv                                                            
source venv/bin/activate

Include the required packages at the top of your Python file. From the GitHub repo, you can download the requirements.txt file and run pip install -r requirements.txt to install them.

import base64
from dotenv import load_dotenv
import json
import os
import requests
import time
import webcolors

import streamlit as st
import folium
from streamlit_folium import st_folium

from geopy.geocoders import Nominatim
from geopy.exc import GeocoderTimedOut, GeocoderServiceError

from langchain.memory import ConversationBufferMemory
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
from langchain_core.prompts import ChatPromptTemplate, PromptTemplate
from langchain_core.runnables import RunnablePassthrough

import numpy as np
import pandas as pd
from pathlib import Path

import plotly.graph_objects as go

Set up a Cloudflare account and obtain the necessary credentials for Workers AI. When you login to the dashboard, your account ID is the string of characters that follow https://dash.cloudflare.com/ To get a Workers AI auth token, click on AI on the lefthand sidebar followed by the blue Use Rest API button. Then, click Create a Workers AI API Token.

Add them to a .env file and reference them by adding these lines beneath the import statements.

load_dotenv()

# Cloudflare Workers AI setup
ACCOUNT_ID = os.getenv('CF_ACCOUNT_ID')
AUTH_TOKEN = os.getenv('CF_AUTH_TOKEN')

Configuring the Streamlit Page

Let's set up the basic Streamlit page configuration and add some custom CSS for styling:

st.set_page_config(page_title="WNBA Player Analytics Dashboard, AI Insights, && AI Assistant", page_icon="🏀", layout="wide")

# Custom CSS (truncated for brevity)
st.markdown("""
<style>
    .hover-link {
        color: #1E90FF;
        text-decoration: none;
        transition: color 0.3s ease;
    }
    .hover-link:hover {
        color: #FF4500;
        text-decoration: underline;
    }
    /* ... more custom CSS -> see https://github.com/elizabethsiegle/wnba-analytics-dash-ai-insights ... */
</style>
""", unsafe_allow_html=True)

Data Collection and Preprocessing

Create a function to fetch and preprocess WNBA player data:

def fetch_player_data(season):
    url = f"https://www.basketball-reference.com/wnba/years/{season}_per_game.html"
    dataframes = pd.read_html(url, header=0)
    df = dataframes[0]
    df = df[df.G != 'G'].fillna(0)  # Remove header rows and fill NaNs
    df = df.drop(['G'], axis=1)

    # Convert percentage columns to float
    percentage_columns = ['FG%', '3P%', '2P%', 'eFG%', 'FT%']
    for col in percentage_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col].astype(str).str.rstrip('%'), errors='coerce') / 100

    # Convert other numeric columns to float
    numeric_columns = ['Age', 'GS', 'MP', 'FG', 'FGA', '3P', '3PA', '2P', '2PA', 'FT', 'FTA', 'ORB', 'DRB', 'TRB', 'AST', 'STL', 'BLK', 'TOV', 'PF', 'PTS']
    for col in numeric_columns:
        if col in df.columns:
            df[col] = pd.to_numeric(df[col], errors='coerce')

    # Ensure 'Player' and 'Team' columns are strings
    df['Player'] = df['Player'].astype(str)
    df['Team'] = df['Team'].astype(str)

    # Handle any remaining problematic columns
    for col in df.columns:
        if df[col].dtype == object:
            df[col] = df[col].astype(str)

    return df

# Usage
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
player_data = fetch_player_data(selected_season)

This function fetches data from basketball-reference.comfor a given season and performs necessary data cleaning, processing it for our use.

Building the User Interface

Now, let's set up the basic structure of our Streamlit app and sidebar controls:

st.set_page_config(page_title="WNBA Analytics Dashboard", page_icon="🏀", layout="wide")
st.title("WNBA Player Analytics Dashboard")
# Sidebar for filters
st.sidebar.header('Filter Options')
st.sidebar.success("Filter players by season, team, and position to explore the data.")
selected_season = st.sidebar.selectbox('Season', list(reversed(range(1997, 2025))))
# Team and position selection
teams = sorted([team for team in player_data.Team.unique() if team != 'TOT'])
selected_teams = st.sidebar.multiselect('Team', teams, teams)

positions = ['C', 'F', 'G', 'F-G', 'C-F']
selected_positions = st.sidebar.multiselect('Position', positions, positions)
# Fetch and display data
player_data = fetch_player_data(selected_season)
st.write(player_data)

This creates a simple dashboard with a season selector and displays the raw data.

Step 4: Adding Data Visualization

We create a 2x2 grid layout for organizing dashboard components.
The Chart Section (col1):

allows users to select a statistic and chart type (Pie or Bar).
filters data based on a minimum value slider.
creates and displays a chart of the top 5 players for the selected statistic. The stats Section (col2):
displays quick stats like average, highest player, and number of players shown.
shows a styled table of top players for the selected statistic.

Key Features include interactive elements (dropdowns, radio buttons, slider) for user customization and dynamic chart creation based on user selection.

AI Insights Section (col3) adds an AI-powered feature to the dashboard:

It creates a button that, when clicked, generates AI insights about the chart displayed above.

When the button is clicked, it checks if the necessary data is available in the session state.

If data is available, it creates a DataFrame of the top 5 players and their stats.

It then calls the generate_insights() function (which you'd need to implement using a language model like GPT-3 or GPT-4) to analyze this data.

The generated insights are displayed to the user.

A warning is shown to remind users that the insights are AI-generated and should be verified.

This feature demonstrates how to integrate AI capabilities into a data dashboard, providing users with quick, automated analysis of the visualized data.

Implementing Player Comparison

Add a section for comparing two players:

st.markdown('<div class="player-comparison-section">', unsafe_allow_html=True)
    st.subheader("🏀 Player Comparison (players must have played in the same season)")

    # Allow users to select players to compare
    players = player_data['Player'].unique()

    # Find the indices of Caitlin Clark and Angel Reese
    caitlin_index = players.tolist().index('Caitlin Clark') if 'Caitlin Clark' in players else 0
    angel_index = players.tolist().index('Angel Reese') if 'Angel Reese' in players else 0

    player1 = st.selectbox("Select first player", players,index=caitlin_index, key='player1')
    player2 = st.selectbox("Select second player", players, index=angel_index, key='player2')

    def normalize(value, min_value, max_value):
        try:
            value = float(value)
            return 100 * (value - min_value) / (max_value - min_value) if max_value > min_value else 50
        except (ValueError, TypeError):
            return 0  # or some default value for non-numeric entries

    if player1 and player2:
        # Get data for selected players
        stats1 = player_data[player_data['Player'] == player1].iloc[0]
        stats2 = player_data[player_data['Player'] == player2].iloc[0]

        # Select stats to compare
        stats_to_compare = ['PTS', 'AST', 'TRB', 'STL', 'BLK', 'FG%', '3P%', 'FT%']
        # Convert columns to numeric, replacing non-numeric values with NaN
        for stat in stats_to_compare:
            player_data[stat] = pd.to_numeric(player_data[stat], errors='coerce')

        normalized_stats = {}
        for stat in stats_to_compare:
            min_val = player_data[stat].min()
            max_val = player_data[stat].max()
            normalized_stats[stat] = [
                normalize(stats1[stat], min_val, max_val),
                normalize(stats2[stat], min_val, max_val)
            ]

        # Create a radar chart
        fig = go.Figure()

        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][0] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player1
        ))
        fig.add_trace(go.Scatterpolar(
            r=[normalized_stats[stat][1] for stat in stats_to_compare],
            theta=stats_to_compare,
            fill='toself',
            name=player2
        ))

        fig.update_layout(
            polar=dict(radialaxis=dict(visible=True, range=[0, 100])),
            showlegend=True,
            legend=dict(
                font=dict(size=16),  # Increase font size
                itemsizing='constant',  # Make legend items a constant size
                itemwidth=30,  # Adjust item width
                yanchor="top",  # Anchor to the top
                y=0.99,  # Position at the top
                xanchor="right",  # Anchor to the right
                x=0.99,  # Position at the left
                bgcolor="rgba(255, 255, 255, 0.5)",  # Semi-transparent background
                bordercolor="Black",  # Border color
                borderwidth=2,  # Border width
            ),
            title=dict(
                text=f"{player1} vs {player2} Comparison",
                font=dict(size=24)  # Increase title font size
            ),
            width=700,  # Adjust as needed
            height=700  # Adjust as needed
        )

This creates a radar chart comparing two selected players across multiple statistics.

Adding an Interactive Map

Let's add a map showing WNBA team locations:

@st.cache_data
def create_wnba_map():
    # WNBA teams, their locations, and home page URLs
    wnba_teams = {
        'Atlanta Dream': ('Atlanta, GA', 'https://dream.wnba.com/'),
        'Chicago Sky': ('Chicago, IL', 'https://sky.wnba.com/'),
        'Connecticut Sun': ('Uncasville, CT', 'https://sun.wnba.com/'),
        'Dallas Wings': ('Arlington, TX', 'https://wings.wnba.com/'),
        'Indiana Fever': ('Indianapolis, IN', 'https://fever.wnba.com/'),
        'Las Vegas Aces': ('Las Vegas, NV', 'https://aces.wnba.com/'),
        'Los Angeles Sparks': ('Los Angeles, CA', 'https://sparks.wnba.com/'),
        'Minnesota Lynx': ('Minneapolis, MN', 'https://lynx.wnba.com/'),
        'New York Liberty': ('Brooklyn, NY', 'https://liberty.wnba.com/'),
        'Phoenix Mercury': ('Phoenix, AZ', 'https://mercury.wnba.com/'),
        'Seattle Storm': ('Seattle, WA', 'https://storm.wnba.com/'),
        'Washington Mystics': ('Washington, D.C.', 'https://mystics.wnba.com/')
    }

    # Create a map centered on the United States
    m = folium.Map(location=[39.8283, -98.5795], zoom_start=4)

    # Geocoding to get coordinates
    geolocator = Nominatim(user_agent="wnba_app")
    # Team name to abbreviation mapping
    team_abbr = {
        'Atlanta Dream': 'ATL', 'Chicago Sky': 'CHI', 'Connecticut Sun': 'CON',
        'Dallas Wings': 'DAL', 'Indiana Fever': 'IND', 'Las Vegas Aces': 'LVA',
        'Los Angeles Sparks': 'LAS', 'Minnesota Lynx': 'MIN', 'New York Liberty': 'NYL',
        'Phoenix Mercury': 'PHO', 'Seattle Storm': 'SEA', 'Washington Mystics': 'WAS'
    }

    # Add markers for each team
    for team, (city, url) in wnba_teams.items():
        try:
            location = geocode_with_retry(geolocator, city)
            if location is None:
                # Use fallback coordinates if geocoding fails
                lat, lon = fallback_coordinates[team]
            else:
                lat, lon = location.latitude, location.longitude
            # Get team abbreviation and color
            abbr = team_abbr.get(team, 'ATL')  # Default to ATL if not found
            hex_color = team_colors.get(abbr, '#000000')  # Default to black if color not found
            rgb = webcolors.hex_to_rgb(hex_color)
            closest_folium_color = closest_color(rgb)

            # Create popup HTML with team info and link
            popup_html = f"""
            <b>{team}</b><br>
            {city}<br>
            <a href="{url}" target="_blank">Visit Team Website</a>
            """

            folium.Marker(
                [lat, lon],
                popup=folium.Popup(popup_html, max_width=300),
                tooltip=team,
                icon=folium.Icon(color=closest_folium_color, icon='basketball', prefix='fa')
            ).add_to(m)
        except Exception as e:
            st.warning(f"Couldn't add marker for {team}: {str(e)}")

    return m

This creates an interactive map with markers for each WNBA team.

Step 8: Adding a Chatbot Interface

Finally, let's add a chatbot for answering WNBA-related questions:

    st.markdown('<div class="chatbot-section">', unsafe_allow_html=True)
    st.subheader("🏀 Chat💬 w/ WNBA AI Assistant powered by LangChain && Cloudflare Workers AI🤖")

    # Add a loading message
    chat_loading = st.empty()
    chat_loading.info("Chat is initializing... This may take a few moments.")

    # Initialize the LLM and conversation chain
    @st.cache_resource
    def initialize_chat(filtered_data: pd.DataFrame):
        llm = CloudflareWorkersAI(
            account_id=ACCOUNT_ID,
            api_token=AUTH_TOKEN,
            model="@cf/meta/llama-2-7b-chat-int8"
        )
        # Convert filtered_data to a string representation
        data_context = filtered_data.to_string()

        prompt = ChatPromptTemplate.from_messages([
            ("system", """You are a knowledgeable assistant specializing in WNBA statistics, players, and teams. 
            Provide accurate and helpful information about the WNBA.
            Here's the current WNBA data you have access to:
            {data_context}
            Use this data to answer questions, but don't mention the data directly unless asked."""),
            ("human", "{input}"),
            ("ai", "{agent_scratchpad}")
        ])

        memory = ConversationBufferMemory(return_messages=True, output_key="agent_scratchpad")
        def get_chat_history(inputs):
            return memory.chat_memory.messages

        chain = (
            RunnablePassthrough.assign(
                agent_scratchpad=get_chat_history,
                data_context=lambda _: data_context[:100] + "..." # Truncate for brevity 
            )
            | prompt
            | llm
        )

        return chain, memory, data_context

    # Initialize the chat
    chain, memory, data_context = initialize_chat(filtered_data)

    # Remove the loading message
    chat_loading.empty()

    # Initialize chat history
    if "messages" not in st.session_state:
        st.session_state.messages = []

    # Display chat messages from history on rerun
    for message in st.session_state.messages:
        with st.chat_message(message["role"]):
            st.markdown(message["content"])

    # React to user input
    if user_input := st.chat_input("Ask me anything about WNBA stats, players, or teams!"):
        # Display user message in chat message container
        st.chat_message("user").markdown(user_input)
        # Add user message to chat history
        st.session_state.messages.append({"role": "user", "content": user_input})

        # Get AI response
        with st.spinner("Thinking..."):
            try:
                response = chain.invoke({
                    "input": user_input,
                    "data_context": data_context
                })

                if not response or response.strip() == "":
                    response = "I apologize, but I couldn't generate a response. This could be due to an issue with the AI model or the input. Please try asking your question in a different way or try again later."
            except Exception as e:
                response = f"An error occurred: {str(e)}"
                st.error(f"Debug: Error details: {e}")

        # After getting the response from the model
        if isinstance(response, list) and len(response) > 0 and hasattr(response[0], 'content'):
            response_text = response[0].content
        elif isinstance(response, dict) and 'content' in response:
            response_text = response['content']
        elif isinstance(response, str):
            response_text = response
        else:
            response_text = str(response)


        # Display assistant response in chat message container
        with st.chat_message("assistant"):
            st.markdown(response_text)
        # Add assistant response to chat history
        st.session_state.messages.append({"role": "assistant", "content": response_text})

        # Update memory
        memory.chat_memory.add_user_message(user_input)
        memory.chat_memory.add_ai_message(response_text)


        # Add some styling to make the chat interface look better
        st.markdown("""
        <style>
        .stChatFloatingInputContainer {
            bottom: 20px;
            background-color: #f0f2f6;
            padding: 10px;
            border-radius: 10px;
            box-shadow: 0 4px 6px rgba(0, 0, 0, 0.1);
        }
        </style>
        """, unsafe_allow_html=True)

This creates a chatbot interface that can answer questions about WNBA statistics using the provided data.

Conclusion

The complete code can be found here on GitHub.

Happy coding, and enjoy exploring WNBA statistics with your new analytics dashboard!

Build a Route Generator App with Cloudflare Workers AI, LangChain, Streamlit, and Mapbox

Lizzie Siegle — Tue, 10 Sep 2024 22:20:06 +0000

In this tutorial, you will learn how to create a web app that generates optimized tourist routes using Cloudflare Workers AI, Mapbox, LangChain, and Streamlit. The app allows users to select landmarks in a chosen city and get the shortest route between them, solving the Traveling Salesman Problem.

Set Up Your Dev Environment and Environment Variables

On the command line, make a new folder called tsp.py. Create a virtual environment by running



python3 -m venv venv
source venv/bin/activate

Next, install the required libraries with



pip install python-dotenv geopy langchain langchain-core langchain-community markdown pandas requests streamlit streamlit-searchbox folium streamlit-folium

You need a Mapbox API token--you can get one here and a Cloudflare Workers AI token, which you get by clicking AI on the lefthand side of your dashboard, followed by clicking the blue Use REST API button then the blue Create a Workers AI API Token button.

You can find your Cloudflare Account SID on that same page or in the dashboard URL--it's the string of characters following dash.cloudflare.com/.

Make a .env file and replace those values below:



MAPBOX_TOKEN=
CLOUDFLARE_API_TOKEN=
CLOUDFLARE_ACCOUNT_ID=

The app uses environment variables for secure access to APIs like Mapbox and Cloudflare Workers AI. They are accessed in Python code using dotenv and os (and would be replaced by lines like mapbox_token = st.secrets["MAPBOX_TOKEN"] if you were to deploy to Streamlit.

At the top, include the following import statements and variables to reference the environment variables:



from dotenv import load_dotenv
import folium
from langchain.chains import LLMChain
from langchain.prompts import PromptTemplate
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
import os
import requests
import streamlit as st
from streamlit_folium import folium_static

load_dotenv()

mapbox_token = os.getenv('MAPBOX_TOKEN')
cf_account_id = os.getenv('CLOUDFLARE_ACCOUNT_ID')
cf_api_token = os.getenv('CLOUDFLARE_API_TOKEN')

Create User Interface with Streamlit

The UI is created using Streamlit. Add the title and description with the code below:



st.title('Route Me🚴‍♀️🚶‍♀️‍➡️🏃‍♀️')

st.markdown("""
This app uses Cloudflare Workers AI, LangChain, and Mapbox to solve the Traveling Salesman problem. 
1. Enter a city🏙️
2. Pick landmarks🌁🗽
3. Generate the shortest path.
4. Explore! 🗺️
""")

We also add a footer for aesthetic and branding purposes:



footer_html = """
<div class="footer">
    Made with ❤️ in SF🌉 with Cloudflare Workers AI ➡️ 👩🏻‍💻 
    <a href="https://github.com/your-repo">code here on GitHub</a>
</div>
"""
st.markdown(footer_html, unsafe_allow_html=True)

Fetch Cities and Landmarks

We use Mapbox's searchbox API to fetch information about the input city and corresponding landmark information. The app provides a list of cities and landmarks according to the user input.

The following code searches for cities:



def find_city(city_inp: str):
    url = "https://api.mapbox.com/search/searchbox/v1/suggest"
    params = {"q": city_inp, "access_token": mapbox_token, "types": "place"}
    res = requests.get(url, params=params)

    # Extract suggestions
    suggestions = res.json().get('suggestions', [])
    return [(f"{s['name']}, {s['place_formatted']}", s['mapbox_id']) for s in suggestions]

This function retrieves details about the input city:



def retrieve_city(city_id: str):
    url = f"https://api.mapbox.com/search/searchbox/v1/retrieve/{city_id}"
    params = {"access_token": mapbox_token}
    res = requests.get(url, params=params)
    return res.json().get('features', [])[0]  # Get first result

This function retrieves landmarks:



def retrieve_landmark(name: str, proximity: str):
    url = "https://api.mapbox.com/search/searchbox/v1/forward"
    params = {
        "q": name,
        "access_token": mapbox_token,
        "proximity": proximity,
        'types': 'poi'
    }
    res = requests.get(url, params=params)
    return res.json()['features'][0]

Use Cloudflare Workers AI and LangChain for Landmarks

We use Cloudflare Workers AI to generate a list of landmarks. This is done through LangChain’s prompt chain.



def lmchain():
    llm = CloudflareWorkersAI(account_id=cf_account_id, api_token=cf_api_token, model='@cf/meta/llama-3.1-8b-instruct')
    prompt = PromptTemplate(
        template="Return a list of 7 landmarks in {city}.",
        input_variables=["city"]
    )
    return LLMChain(llm=llm, prompt=prompt)

Traveling Salesman Problem and Optimized Routes

Ultimately, this app solves the Traveling Salesman problem by utilizing Mapbox's optimized-trips API. It calculates the shortest route between landmarks selected by the user.



def travelingsalesman(landmarks_df):
    coordinates = ";".join([f"{row['longitude']},{row['latitude']}" for _, row in landmarks_df.iterrows()])
    url = f"https://api.mapbox.com/optimized-trips/v1/mapbox/cycling/{coordinates}"
    params = {"access_token": mapbox_token, "geometries": "geojson"}
    res = requests.get(url, params=params)
    return res.json()

Display the Map

We use Folium to generate the map and display it in the Streamlit app with the streamlit-folium library.



def create_route_map(landmarks, optimized_coords):
    m = folium.Map(location=[landmarks['latitude'].mean(), landmarks['longitude'].mean()], zoom_start=12)
    folium.PolyLine(optimized_coords, color='red').add_to(m)
    return folium_static(m)

Generate Textual Route Description

The LLM generates a textual description of the optimized bike/walk/run route:



def make_route(city, landmarks):
    chain = lmchain()
    return chain.run({"city": city, "landmarks": "\n".join(landmarks['Name'])})

Download Map and Route Options

The app allows users to download the route map and description:



st.download_button(label='Download Route Map', data=map_to_html(st.session_state.route_map), file_name='route_map.html')
st.download_button(label='Download Route Description', data=route_html, file_name='route_description.html')

The complete Python file should look like this:



from dotenv import load_dotenv
from geopy import distance
import io
from langchain.chains import LLMChain
from langchain_core.prompts import PromptTemplate
from langchain.output_parsers import CommaSeparatedListOutputParser
from langchain_community.llms.cloudflare_workersai import CloudflareWorkersAI
import markdown
import os
import pandas as pd
import requests
import streamlit as st
from streamlit_searchbox import st_searchbox
from typing import List
import uuid
import folium
from folium.plugins import Draw
from streamlit_folium import folium_static

# Display the title and description
st.title('Route Me🚴‍♀️🚶‍♀️‍➡️🏃‍♀️')

st.markdown("""
    <style>
        .footer {
            position: fixed;
            bottom: 0;
            left: 0;
            width: 100%;
            background-color: #f1f1f1;
            text-align: center;
            padding: 10px 0;
            color: #000;
            font-size: 14px;
            box-shadow: 0 -2px 5px rgba(0,0,0,0.1);
        }
        .content {
            padding-bottom: 150px; /* Adjust this value to create space between content and footer */
        }
    </style>
""", unsafe_allow_html=True)
# Footer HTML
footer_html = """
    <div class="footer">
        Made with ❤️ in SF🌉 with Cloudflare Workers AI ➡️ 👩🏻‍💻 <a href="https://github.com/elizabethsiegle/bike_walk_route_map_generator">code here on GitHub</a>
    </div>
"""

# Inject the footer into the app
st.markdown(footer_html, unsafe_allow_html=True)

# Define markdown content directly
markdown_content = """
This app uses [Cloudflare Workers AI](https://developers.cloudflare.com/workers-ai/), [LangChain](https://langchain.dev/),  landmark/city data from Mapbox, [Folium](https://python-visualization.github.io/folium/latest/) for visualizing maps and routes, and [Streamlit](https://streamlit.io/)/[Streamlit Folium](https://folium.streamlit.app/) to tackle the Traveling Salesman problem!

1. Enter a city🏙️ you wish to visit
-> get a few must-visit landmarks in your chosen city
2. Pick the landmarks🌁🗽 you want to visit.
3. Generate the shortest path between these landmarks.
4. Explore! 🗺️
"""
st.markdown(markdown_content)

# Load environment variables
load_dotenv()
mapbox_token = st.secrets["MAPBOX_TOKEN"] # os.environ.get('MAPBOX_TOKEN')
cf_account_id = st.secrets["CLOUDFLARE_ACCOUNT_ID"]# os.environ.get('CLOUDFLARE_ACCOUNT_ID')
cf_api_token = st.secrets["CLOUDFLARE_API_TOKEN"] # os.environ.get('CLOUDFLARE_API_TOKEN')

token = str(uuid.uuid4())

# find cities
def find_city(city_inp: str) -> List[tuple]:
    if len(city_inp) < 3:
        return []

    # searchbox api = return list of suggestions for city
    url = "https://api.mapbox.com/search/searchbox/v1/suggest"
    params = {"q": city_inp, "access_token": mapbox_token, "session_token": token, "types": "place"}

    res = requests.get(url, params=params)
    if res.status_code != 200:
        return []

    try:
        suggestions = res.json().get('suggestions', [])
        results = []
        for s in suggestions:
            print(f"s[name] {s['name']} s[place_formatted] {s['place_formatted']}")
            results.append((f"{s['name']}, {s['place_formatted']}", s['mapbox_id']))

        return results
    except Exception as e:
        st.error(f"Error fetching city suggestions: {e}")
        return []

# Function to retrieve city details
@st.cache_data
def retrieve_city(id):
    url = f"https://api.mapbox.com/search/searchbox/v1/retrieve/{id}"
    params = {"access_token": mapbox_token,"session_token": token}
    res = requests.get(url, params=params)
    if res.status_code != 200:
        return []
    try:
        features = res.json().get('features', [])
        if not features:
            st.warning("No features returned for the city.")
            return []
        return features[0]
    except Exception as e:
        st.error(f"An error occurred: {e}")
        return []

# Function to retrieve landmark details
@st.cache_data
def retrieve_landmark(name, proximity):
    mapbox_url = "https://api.mapbox.com/search/searchbox/v1/forward"
    params = {"access_token": mapbox_token, "q": name, "proximity": proximity, 'types': 'poi', 'poi_category': 'tourist_attraction,museum,monument,historic,park,church,place_of_worship'}

    res = requests.get(mapbox_url, params=params)
    print(f'res.json {res.json()}')
    if res.status_code != 200:
        return []

    try:
        return res.json()['features'][0]
    except Exception as e:
        print(f"Error retrieving landmark: {e}")
        return []

# find city w/ searchbox
city_id = st_searchbox(find_city, key="city")

# Function to make landmark chain
@st.cache_resource
def lmchain():
    outp_parser = CommaSeparatedListOutputParser()
    form_instructions = outp_parser.get_format_instructions()

    llm = CloudflareWorkersAI(account_id=cf_account_id, api_token=cf_api_token, model='@cf/meta/llama-3.1-8b-instruct',)
    prompt = PromptTemplate(
        template="""Return a comma-separated list of the 7 best landmarks in {city}. Only return the list. {form_instructions}""",
        input_variables=["city"],
        partial_variables={"form_instructions": form_instructions},
    )

    chain = LLMChain(llm=llm, prompt=prompt, output_parser=outp_parser)
    return chain

# Function to get landmark locations
@st.cache_data
def get_landmarks(landmarks, long_city, lat_city):
    data = []
    for lm in landmarks:
        features = retrieve_landmark(lm, f"{long_city},{lat_city}")
        if not features:
            continue

        coor = features['geometry']['coordinates']
        long, lat = coor
        dist = distance.distance((lat_city, long_city), (lat, long)).km

        if dist <= 7:
            data.append([lm, long, lat, True])

    return pd.DataFrame(data=data, columns=['Name', 'longitude', 'latitude', 'Include'])

@st.cache_data
def travelingsalesman(chosen_landmarks):
    profile = "mapbox/cycling"
    coordinates = ";".join([f"{row['longitude']},{row['latitude']}" for _, row in chosen_landmarks.iterrows()])

    # optimized trips API -> optimized route to hit all landmarks
    url = f"https://api.mapbox.com/optimized-trips/v1/{profile}/{coordinates}"
    params = {"access_token": mapbox_token, "geometries": "geojson"}  # Request GeoJSON format

    res = requests.get(url, params=params)
    if res.status_code != 200:
        st.error(f"Error: API request failed with status code {res.status_code}")
        return None, []

    try:
        json_response = res.json()

        if 'trips' not in json_response or not json_response['trips']:
            st.error("Error: No trips found in the API response")
            return None, []

        trip = json_response['trips'][0]

        if 'geometry' not in trip:
            st.error("Error: No geometry found in the trip data")
            return None, []

        geometry = trip['geometry']

        if isinstance(geometry, dict) and 'coordinates' in geometry:
            optimized_coords = [(coord[1], coord[0]) for coord in geometry['coordinates']]
            return json_response, optimized_coords
        else:
            st.error("Error: Unexpected geometry format in the API response")
            return None, []
    except Exception as e:
        st.error(f"Error in travelingsalesman: {str(e)}")
        st.write("JSON Response:", json_response)  # Debug: Print the JSON response
        return None, []

def create_route_map(landmarks, optimized_coords):
    # Create a map centered on the mean of all coordinates
    all_lats = landmarks['latitude'].tolist() + [coord[0] for coord in optimized_coords]
    all_lons = landmarks['longitude'].tolist() + [coord[1] for coord in optimized_coords]
    center_lat = sum(all_lats) / len(all_lats)
    center_lon = sum(all_lons) / len(all_lons)

    m = folium.Map(location=[center_lat, center_lon], zoom_start=12)
    Draw(export=True).add_to(m)

    # Add markers for each landmark
    for _, row in landmarks.iterrows():
        folium.Marker(
            [row['latitude'], row['longitude']],
            popup=row['Name']
        ).add_to(m)

    # Add the optimized route line if coordinates are available
    if optimized_coords:
        folium.PolyLine(
            optimized_coords,
            weight=6,
            color='red',
            opacity=0.8
        ).add_to(m)
    else:
        st.warning("No optimized route available. Displaying landmarks only.")

    # Fit the map to the bounds of all coordinates
    sw = min(all_lats), min(all_lons)
    ne = max(all_lats), max(all_lons)
    m.fit_bounds([sw, ne])

    return m

# Function to create a bike route
@st.cache_data
def make_route(city, landmarks, _llm):
    prompt = PromptTemplate(
        template="""You are an experienced tour guide in {city}. You love telling more about landmarks in a short way. Create a bike route for {city} in markdown, using headings with ##, passing by the following landmarks: {landmarks}. End with the introduction of the next landmark {end}, as if it was the next destination, but don't discuss it.""",
        input_variables=["landmarks", "city", "end"]
    )
    chain = LLMChain(llm=_llm, prompt=prompt)
    landmarks_string = "\n".join([f"{row['Name']}" for _, row in landmarks.iloc[:5, :].iterrows()])

    # Handle case where there are fewer than 6 landmarks
    if len(landmarks) < 6:
        part_one = chain.run({'city': city, 'landmarks': landmarks_string, 'end': ""})
        return part_one
    else:
        part_one = chain.run({'city': city, 'landmarks': landmarks_string, 'end': landmarks.iloc[5]['Name']})

    prompt = PromptTemplate(
        template="""You are an experienced tour guide in {city}. You love telling more about landmarks in a short way. Create a bike route for {city} in markdown, using headings with ##, passing by the following landmarks: {landmarks}. Start your explanation with 'Continuing from {previous}'.""",
        input_variables=["landmarks", "city", "previous"]
    )

    chain = LLMChain(llm=_llm, prompt=prompt)
    landmarks_string = "\n".join([f"{row['Name']}" for _, row in landmarks.iloc[5:, :].iterrows()])
    part_two = chain.run({'city': city, 'landmarks': landmarks_string, 'previous': landmarks.iloc[4]['Name']})

    return part_one + " " + part_two

# Function to convert markdown data to HTML
@st.cache_data
def to_html(data, filename='route'):
    return markdown.markdown(data)

# New function to convert Folium map to HTML string
def map_to_html(map_object):
    map_html = io.BytesIO()
    map_object.save(map_html, close_file=False)
    return map_html.getvalue().decode()

if 'route' not in st.session_state:
    st.session_state.route = None

# Function to generate a bike route
def gen_route(city, stops):
    route = make_route(city['properties']['full_address'], stops, CloudflareWorkersAI(account_id=cf_account_id, api_token=cf_api_token))
    st.session_state.route = route

def create_route_map(landmarks, optimized_coords):
    # Create a map centered on the first landmark
    m = folium.Map(location=[landmarks.iloc[0]['latitude'], landmarks.iloc[0]['longitude']], zoom_start=12)

    # Add markers for each landmark
    for _, row in landmarks.iterrows():
        folium.Marker(
            [row['latitude'], row['longitude']],
            popup=row['Name'],
            tooltip=row['Name']
        ).add_to(m)

    # Add the optimized route line if coordinates are available
    if optimized_coords:
        folium.PolyLine(
            optimized_coords,
            weight=2,
            color='red',
            opacity=0.8
        ).add_to(m)
    else:
        st.warning("No optimized route available. Displaying landmarks only.")

    return m

# Update the main app logic
if city_id:
    city = retrieve_city(city_id)
    if city:
        coords = city['geometry']['coordinates']
        long, lat = coords
        landmarks = lmchain().run({"city": city['properties']['full_address']})

        if 'landmark_locations' not in st.session_state:
            st.session_state.landmark_locations = get_landmarks(landmarks, long, lat)

        user_inp = st.data_editor(
            st.session_state.landmark_locations,
            hide_index=True,
            disabled=('Name', 'longitude', 'latitude'),
            column_config={'longitude': None, 'latitude': None},
            key='user_input',
            use_container_width=True
        )

        st.session_state.landmark_locations.update(user_inp)

        selected_landmarks = st.session_state.landmark_locations[st.session_state.landmark_locations['Include']]

        output, optimized_coords = travelingsalesman(selected_landmarks)
        if output is not None and optimized_coords:
            dist = output['trips'][0]['distance']
            conv_fac = 0.000621371
            miles = dist * conv_fac
            st.write(f"Total distance: {miles:.3f} mi")

            waypoints = [wp['waypoint_index'] for wp in output['waypoints']]
            stops = selected_landmarks.iloc[waypoints, :]

            # Store the route map in session state
            st.session_state.route_map = create_route_map(stops, optimized_coords)

            # Display the route map
            folium_static(st.session_state.route_map)

            st.button('Generate route!', on_click=lambda: gen_route(city, stops))
        else:
            st.error("Unable to generate the optimized route. Please try again or select different landmarks.")

# Show generated route and offer map download
if 'route' in st.session_state and st.session_state.route:
    route = st.session_state.route
    st.markdown(route)

    # Offer route map download if available
    if 'route_map' in st.session_state:
        map_html = map_to_html(st.session_state.route_map)
        st.download_button(
            label='Download Route Map',
            data=map_html,
            file_name='route_map.html',
            mime='text/html'
        )

    # Offer route description download
    route_html = to_html(route)
    st.download_button(
        label='Download the route description!',
        data=route_html,
        file_name='cf-workers-ai-tourist-route.html',
        mime='text/html'
    )
st.markdown('</div>', unsafe_allow_html=True)
# Footer HTML
footer_html = """
    <div class="footer">
        Made with ❤️ in SF🌉
    </div>
"""
The [complete code can be found here on GitHub](https://github.com/elizabethsiegle/bike_walk_route_map_generator).

# Inject the footer into the app
st.markdown(footer_html, unsafe_allow_html=True)

Lastly, run the file with streamlit run tsp.py!
app gif

So much fun is to be had for developers with Mapbox, LangChain, Cloudflare, and Streamlit. Let me know online what you're building!

Summarize articles with Cloudflare Workers AI LoRAs

Lizzie Siegle — Fri, 12 Jul 2024 18:14:09 +0000

This tutorial will go over how to make a Streamlit web app in Python that takes in a URL and, with Workers AI LoRA adapters, summarizes the article in the tone of the user's choosing, such as gen z or humorous.

It's free to use most AI models hosted on Cloudflare!

Test it out yourself here.

The complete code can be found here on GitHub.

LoRAs

Fine-tuning is a process that takes a pre-trained model and further trains it on a specific dataset to adapt it to a particular task or domain, enhancing its performance and relevance to that task. LoRAs are a more efficient approach to fine-tuning that adds a small number of low-rank matrices to the model's parameters during fine-tuning.

This means that LoRAs offer a more efficient and scalable way to adapt LLMs to specific tasks without the heavy computational and memory demands of traditional fine-tuning.

Setup

Since this project requires Python package installations, make a new project directory and virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir lora-summarize 
cd lora-summarize
python3 -m venv venv 
source venv/bin/activate 
pip install streamlit
pip install bs4
pip install requests
pip install load_dotenv

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir lora-summarize
cd lora-summarize
python -m venv venv 
venv\Scripts\activate 
pip install streamlit
pip install bs4
pip install requests
pip install load_dotenv

The last command uses pip, the Python package installer, to install the packages that you are going to use in this project, which are:

The load_dotenv library to load environment variables from a .env file
The streamlit library to make webapps in Python, easily connecting the frontend and backend
The bs4 library to extract text from a webpage
The requests library to hit the Cloudflare endpoint for a LLM hosted on Cloudflare

This project needs a Cloudflare API Token and your Cloudflare Account ID to use an LLM. After making a Cloudflare account, click Workers&Pages on the left. Then, you can find your Account ID on the right side.

Get a Cloudflare API Key by clicking AI on the left, followed by Use Rest API. Finally, click Create a Workers AI API Token. Grab it and in the root directory, make a .env file and put them in there as such:

CF_API_TOKEN=XXXX
CF_ACCOUNT_ID=XXXX

Now it's time to code the Streamlit app.

Build the Streamlit App

I love how the Streamlit app lets you build your web page from the top-down.

At the top of a Python file called app.py, add the following code to import the required libraries, load the .env secret credentials, and create the variable for the Cloudflare URL to hit in Python:

import streamlit as st
from bs4 import BeautifulSoup
import json
import os
import requests
# Load API secrets
from dotenv import load_dotenv
load_dotenv()
CLOUDFLARE_ACCOUNT_ID = os.environ.get("CF_ACCOUNT_ID")
CLOUDFLARE_API_TOKEN= os.environ.get("CF_API_TOKEN")
url = f'https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1'

Then comes the meat of the app: Make a main function, include some Streamlit modules to display markdown on the page, write text to the page, get text input asking for a news URL to summarize, and display a selectbox with options the user can select for the tone of the summary.

When the user clicks the enter button, a spinner is displayed while Beautifulsoup extracts paragraph tags from the input web page URL.

def main():
    st.markdown("""
        <style>
            .big-font {
                font-size:40px !important;
                color:green;
            }
        </style>
    """, unsafe_allow_html=True)
    st.markdown('<p class="big-font"<p>AI🤖 News🗞️ Summarizer</p>', unsafe_allow_html=True)
    st.write(":blue[This Python🐍 web🕸️ app is built👩🏻‍💻 w/ [Streamlit](https://streamlit.io/) && [Cloudflare Workers AI](https://ai.cloudflare.com/)]")

    news_link = st.text_input('Please enter a news link to summarize') # news_link = "https://www.npr.org/2024/07/08/g-s1-8731/emma-navarro-coco-gauff-wimbeldon"
    tone = st.selectbox(
        ':green[What tone do you want the news summary to take?]',
        ('humorous🤣', 'majestic🌊', 'academic📚', '✨inspirational✨', 'dramatic🎭', 'gen z👧🏻')
    )
    st.write("You selected: ", tone)
    if st.button('Enter') and tone is not None and news_link is not None:
        with st.spinner('Processing📈...'):
            resp1 = requests.get(news_link)
            soup = BeautifulSoup(resp1.text, 'html.parser')

            # Extract text data from website
            text_data = ''
            for tag in soup.find_all(['p']):
                text_data += tag.get_text()

            print('text_data' , text_data)

That text extraction of paragraphs represents the body of the input URL. It is included (with the selected tone of the summary) in the prompt passed to the LoRA model adapter that hits the LLM (Mistral) model hosted on Cloudflare via GET request. To run inference with public LoRAs, you just need to define the LoRA name in the request. The response returned is the summary of the selected tone. It is then parsed and displayed. Finally, a footer is displayed at the bottom of the webpage.

            # Define the headers
            headers = {
                'Authorization': f'Bearer {CLOUDFLARE_API_TOKEN}',
                'Content-Type': 'application/json'
            }

            # Define the data
            data = {
                "messages": [
                    {
                        "role": "user",
                        "content": f"Summarize the following content from a news article in a {tone} tone: {text_data}"
                    }
                ],
                "lora": "cf-public-cnn-summarization"
            }

            # Make the POST request
            response = requests.post(url, headers=headers, data=json.dumps(data))

            # Parse the response
            response_data = response.json()
            summary = response_data["result"]["response"]
            print("summary ", summary)
        html_str = f"""
        <p style="font-family:Comic Sans; color:Pink; font-size: 18px;">{summary}</p>
        """
        st.markdown(html_str, unsafe_allow_html=True)

    st.write("Made w/ ❤️ in SF 🌁 || ✅ out the [👩🏻‍💻GitHub repo](https://github.com/elizabethsiegle/cf-ai-lora-news-summarizer)")

The complete app.py code is below:

import streamlit as st
from bs4 import BeautifulSoup
import json
import os
import requests
# Load API secrets
from dotenv import load_dotenv
load_dotenv()
CLOUDFLARE_ACCOUNT_ID = os.environ.get("CF_ACCOUNT_ID")
CLOUDFLARE_API_TOKEN= os.environ.get("CF_API_TOKEN")
url = f'https://api.cloudflare.com/client/v4/accounts/{CLOUDFLARE_ACCOUNT_ID}/ai/run/@cf/mistral/mistral-7b-instruct-v0.1'

def main():
    st.markdown("""
        <style>
            .big-font {
                font-size:40px !important;
                color:green;
            }
        </style>
    """, unsafe_allow_html=True)
    st.markdown('<p class="big-font"<p>AI🤖 News🗞️ Summarizer</p>', unsafe_allow_html=True)
    st.write(":blue[This Python🐍 web🕸️ app is built👩🏻‍💻 w/ [Streamlit](https://streamlit.io/) && [Cloudflare Workers AI](https://ai.cloudflare.com/)]")

    news_link = st.text_input('Please enter a news link to summarize') # news_link = "https://www.npr.org/2024/07/08/g-s1-8731/emma-navarro-coco-gauff-wimbeldon"
    tone = st.selectbox(
        ':green[What tone do you want the news summary to take?]',
        ('humorous🤣', 'majestic🌊', 'academic📚', '✨inspirational✨', 'dramatic🎭', 'gen z👧🏻')
    )
    st.write("You selected: ", tone)
    if st.button('Enter') and tone is not None and news_link is not None:
        with st.spinner('Processing📈...'):
            resp1 = requests.get(news_link)
            soup = BeautifulSoup(resp1.text, 'html.parser')

            # Extract text data from website
            text_data = ''
            for tag in soup.find_all(['p']):
                text_data += tag.get_text()

            print('text_data' , text_data)

            # Define the headers
            headers = {
                'Authorization': f'Bearer {CLOUDFLARE_API_TOKEN}',
                'Content-Type': 'application/json'
            }

            # Define the data
            data = {
                "messages": [
                    {
                        "role": "user",
                        "content": f"Summarize the following content from a news article in a {tone} tone: {text_data}"
                    }
                ],
                "lora": "cf-public-cnn-summarization"
            }

            # Make the POST request
            response = requests.post(url, headers=headers, data=json.dumps(data))

            # Parse the response
            response_data = response.json()
            summary = response_data["result"]["response"]
            print("summary ", summary)
        html_str = f"""
        <p style="font-family:Comic Sans; color:Pink; font-size: 18px;">{summary}</p>
        """
        st.markdown(html_str, unsafe_allow_html=True)

    st.write("Made w/ ❤️ in SF 🌁 || ✅ out the [👩🏻‍💻GitHub repo](https://github.com/elizabethsiegle/cf-ai-lora-news-summarizer)")

if __name__ == "__main__":
    main()

What's Next for LoRAs and Cloudflare Workers AI

It's free to use many open source LLMs hosted on Cloudflare. They offer so many possibilities to builders and developers of all backgrounds for building generative AI applications.

LoRAs are helpful because they provide a cost-effective, efficient, flexible, and scalable method for adapting LLMs to specific tasks, making advanced AI capabilities more accessible for a wide variety of applications.

Let me know online what you're building with AI!

Create AI-generated art via SMS with Replicate in Python

Lizzie Siegle — Thu, 21 Sep 2023 21:22:37 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

It's fun to get creative! Read on to learn how to create AI-generated images via SMS using Replicate and Twilio Programmable Messaging.

Do you prefer learning via video more? Check out this TikTok summarizing this tutorial.

Prerequisites

A Twilio account - sign up for a free Twilio account here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
Replicate account to use the Stable Diffusion text-to-image model – make a Replicate account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access.

⚠️ ngrok is needed for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. You can also choose to automate ngrok as shown in this article.

Replicate

Replicate offers a cloud API and tooling so you can more easily run machine learning models, abstracting away some lower-level AI concepts and handling infrastructure so you can focus more on your own applications. You can run open-source models that others have published, or package and publish your own, either publicly or privately.

Configuration

Since you will be installing some Python packages for this project, you need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir replicate-prompt-to-image-sms 
cd replicate-prompt-to-image-sms 
python3 -m venv venv 
source venv/bin/activate 
pip install replicate flask twilio

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir replicate-prompt-to-image-sms  
cd replicate-prompt-to-image-sms  
python -m venv venv 
venv\Scripts\activate 
pip install replicate flask twilio

Grab your default Replicate API Token or create a new one here. Run this command in the terminal in your current folder:

export REPLICATE_API_TOKEN={replace with your api token}

Now it's time to write some code to receive a prompt via SMS and return an AI-generated image!

Turn a Prompt into an AI-generated image via SMS with Replicate

Make a file called app.py and place the following import statements at the top.

import replicate
from flask import Flask, request
from twilio.twiml.messaging_response import MessagingResponse

Next, make a Flask app so your app can receive the inbound text message to your Twilio phone number. That string is then passed to Replicate 's stable diffusion model to generate photo-realistic images given that prompt.

app = Flask(__name__)
@app.route('/sms', methods=['GET', 'POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip()
    output = replicate.run(
        "stability-ai/stable-diffusion:27b93a2413e7f36cd83da926f3656280b2931564ff050bf9575f1fdf9bcd7478",
        input={"prompt": inb_msg}
        )
    res_prompt = "prompt was " + inb_msg
    msg = resp.message(res_prompt)
    img = output[0]
    msg.media(img)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

On the command line, run python app.py to start the Flask app. Now it's time to set up a Twilio phone number so you can text it a prompt and get an AI-generated image back!

Configure a Twilio Number for the SMS Chatbot

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL with the /sms path in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio phone number is configured so that it maps to your web application server running locally on your machine and your application can run. Text a prompt of a picture you'd like to generate to your Twilio number and you should get an AI-generated image of whatever you sent over SMS!

Instead of ngrok, you could alternatively use another tunneling service like Tailscale–my teammate Rishab reviewed this post, and kept getting a 403 error with ngrok. The solution was to switch port numbers with ngrok–it seems that Tailscale has more privileges to the network settings compared to ngrok. You can view the complete code on GitHub here.

What's Next for Replicate and Twilio Programmable Messaging?

There is so much fun for developers to have around building with LLMs! Replicate offers so many models to play with like sdxl, a text-to-image generative AI model that creates beautiful 1024x1024 images, or you can fine-tune your own. I can't wait to see what you build with AI–let me know online what you're working on!

Build an AI Personal Trainer with LangChain Agents and SendGrid

Lizzie Siegle — Tue, 19 Sep 2023 19:53:32 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

Can AI help me run a faster marathon?

Problem: I play a lot of tennis and run, bike, and walk as my forms of transit, so I have never trained for a half marathon or 10k. However, I KNOW I must train to run a marathon. As a developer, I turned to code, wondering how AI could help me run a faster marathon.

Solution: Read on to learn how to build a personal marathon training plan generator Streamlit-hosted app from personal workout data using the Strava API, LangChain agents, chains, few-shot prompt templating, PromptTemplates, OpenAI, and SendGrid. This can also be applied to personal training, nutrition, and more!

In the Streamlit page above, a user inputs their Strava API token so the app can access their workout data, the date they want to begin training, and the marathon (or race) day. When they click Submit, a custom training plan is generated.

You can test the app out here using a Strava API token generated by following the directions under Strava Activity Webhook Authentication in this tutorial on querying data with LangChain agents.

Do you prefer learning via video more? Check out this TikTok relating to this tutorial--massive thank you to Twilio Community Engagement Manager Sandra Mendez for spearheading this video.

Understanding LangChain Agents

This meaty app uses many features of LangChain, such as agents: agents use LLMs to decide what actions should be taken and they have access to tools like web search or calculators which can be packaged into a logical loop of operations.

For more background on LangChain, you can read this blog post here.

Prerequisites

A free SendGrid account
Python installed - download Python here
An email address to test out this project
OpenAI Account – make an OpenAI Account here
A Strava account - sign up for a Strava account if you don't have one already
A Strava app and API key - follow directions here under Setup the Strava API
A Strava Auth token - follow directions here under Strava Activity Webhook Authentication You need a Strava app and API key–following the directions above will provide you with the necessary Strava Client Secret in order to generate a Strava Access Token to make a request to access your Strava data. Different Strava API app values include:
Client ID: Your application ID
Client Secret: Your client secret (please keep this confidential)
Authorization/Access token: Your Strava API app's authorization token will need to be regenerated every six hours (should be confidential) Next, after making an OpenAI account if you don't have one already, you can get an OpenAI API Key here by clicking on + Create new secret key. Then, grab a SendGrid API key here.

The Python application will need to have access to these keys, so in your root directory (called strava-langchain) make a .env file to store the API keys safely. The application we create will be able to import this key as an environment variable soon.

In the .env file add the following lines of text, making sure to replace <YOUR-OPENAI-API-KEY> and <YOUR-SENDGRID-KEY> with your actual keys. STRAVA_CLIENT_SECRET should have been used above to generate a Strava Access Token:

OPENAI_API_KEY= <YOUR-OPENAI-KEY>
SENDGRID_API_KEY = <YOUR-SENDGRID-KEY>
STRAVA_CLIENT_SECRET= <YOUR-STRAVA-CLIENT-SECRET>

Your Strava Access Token will be placed into a Streamlit form. Streamlit is an open-source Python app framework that helps developers quickly create web apps for data science and machine learning.

In the root directory run the following commands to install the project's required libraries after making a virtual environment:

python3 -m venv venv 
source venv/bin/activate

Make a requirements.txt file containing the following:

chromadb==0.4.5
google-search-results==2.4.2
langchain==0.0.261
numpy==1.25.0
openai==0.27.2
pandas==2.0.3
python-dotenv==1.0.0
PyYAML==6.0
reportlab==4.0.4
Requests==2.31.0
sendgrid==6.10.0
streamlit==1.25.0
tabulate==0.9.0
urllib3==2.0.4

Install them on the command line by running pip install -r requirements.txt.

Then run mkdir data_files to make a folder called data_files to store the activity data.

We will import these in our code soon.

Few-shot prompting Examples

There are a few ways one could go about making an AI personal marathon trainer, workout trainer, or recommendation system. My college classmate-turned Redis AI engineer Sam Partee suggested using few-shot prompting, in which a few explicit examples (or "shots") guide the Large Language Model (LLM) to respond in a certain way.

Create a file named stapp.py and include some examples of marathon training plans. I used OpenAI to create these plans and then formatted them a bit because most marathon plans I found online were not copy-and-pasteable.

examples = [
  {
    "training_start_date" : "July 19",
    "marathon_date" : "November 8",
    "pd_run_output": "The average distance is 3.71 miles and the maximum distance is 9.52 miles. The average moving time is not available and the maximum moving time is not available. Other running statistics include total elevation gain, with an average of 47.6 and a maximum of 200.6.",
    "num_workouts": "112",
    "plan": 
"""
Week 1: 
Day 1, {training_start_date}: Cross-training - Swim
Day 2: Easy run - 4 miles at an easy pace
Day 3: Cross-training - Elliptical 
Day 4: Medium run - 5 miles at a medium pace 
...
{marathon_date}: Marathon Day!
Note: The above plan assumes that the student is currently able to comfortably run 4 miles at an easy pace. Adjustments may need
to be made if the student is not yet at this level. Additionally, the plan includes cross-training and rest days to
ensure proper recovery and prevent injury.
"""
  }
]

The complete code for that file is here on GitHub. Now let's make some custom tools to help LangChain suggest personalized workouts.

Make Custom Tools to assist LangChain Agents

Using agents allows LLMs access to tools with which LLMs can search the web, do math, run code, and more. Though LLMs are powerful, they still have trouble with certain tasks–that's why you may sometimes need to make custom tools.

LangChain offers a large selection of pre-built tools, but only so many problems can be solved using existing tools.

Create a file named request.py and include the following import statements.

import re
from datetime import date, datetime
import os
import base64
from typing import List, Union
import pandas as pd  # dataframe

from reportlab.platypus import SimpleDocTemplate, Paragraph
from reportlab.lib.pagesizes import letter
from reportlab.lib.styles import getSampleStyleSheet
from reportlab.lib.units import inch

import requests
import urllib3
from dotenv import dotenv_values
import streamlit as st

from langchain import LLMChain, LLMMathChain
from langchain.agents import Tool, AgentExecutor, LLMSingleActionAgent, AgentOutputParser, create_pandas_dataframe_agent, initialize_agent, AgentType
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI  # , GPT4All
from langchain.prompts import BaseChatPromptTemplate
from langchain.prompts.few_shot import FewShotPromptTemplate
from langchain.prompts.prompt import PromptTemplate
from langchain.schema import AgentAction, AgentFinish, HumanMessage

from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import (
    Mail, Attachment, FileContent, FileName, FileType, Disposition)

import stapp  # file
urllib3.disable_warnings(
    urllib3.exceptions.InsecureRequestWarning
)
config = dotenv_values(".env")

Next, we set some variables such as the Strava URL to retrieve a user's Activity data, have the app access its API keys, and then chain LLMs together: the llm variable passes the desired gpt-4-32k model to use with ChatOpenAI and sets the temperature to 0.2–play around with this value. A higher temperature value means the LLM output will include more randomness. Lastly, pass LLM to a math chain to perform complex math problems.

activities_url = "https://www.strava.com/api/v3/athlete/activities"
os.environ["OPENAI_API_KEY"] = config.get('OPENAI_API_KEY')
os.environ["SENDGRID_API_KEY"] = config.get('SENDGRID_API_KEY')
llm = ChatOpenAI(model_name='gpt-4-32k', temperature=0.2)
llm_math_chain = LLMMathChain.from_llm(llm=llm, verbose=True)

Now make some functions for the custom tools to help the LLM convert meters to miles, calculate the number of days between two dates, and get the number of days in a Pandas dataframe object.

def convert_to_miles(num):
    return ((num/m_conv_factor)*1000)/1000.0

def calc_days_btwn(training_start_date, marathon_date): 
    training_start_date = training_start_date.split(',')[1].strip()
    training_start_date = datetime.strptime(training_start_date, '%Y-%m-%d') 
    training_start_date = datetime.date(training_start_date)

    delta = marathon_date - training_start_date
    return delta.days-1

#number of workouts in Strava data
def num_rows_in_dataframe(df):
    return len(df.index)

Beneath that, add a helper function to help validate user emails.

email_regex = r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,7}\b'
def validate_email(email):
    # pass the regular expression and the string into the fullmatch() method
    if(re.fullmatch(email_regex, email)):
        return True
    else:
        return False

It's time to make a Custom Prompt Template in LangChain to make constructing prompts using dynamic inputs easier.

Make Custom Prompt Template

LangChain provides a set of default prompt templates to generate prompts for different tasks. However, for some projects like this one, developers might want to create a prompt template with specific dynamic instructions for the LLM–in instances like that, you can create a custom prompt template similar to the one below right inside request.py beneath validate_email:

#Set up a prompt template
class CustomPromptTemplate(BaseChatPromptTemplate):
    template: str
    # The list of tools available
    tools: List[Tool]

    def format_messages(self, **kwargs) -> str:
        # Get the intermediate steps (AgentAction, Observation tuples)
        # Format them in a particular way
        intermediate_steps = kwargs.pop("intermediate_steps")
        thoughts = ""
        for action, observation in intermediate_steps:
            thoughts += action.log
            thoughts += f"\nObservation: {observation}\nThought: "
        # Set the agent_scratchpad variable to that value
        kwargs["agent_scratchpad"] = thoughts
        # Create a tools variable from the list of tools provided
        kwargs["tools"] = "\n".join([f"{tool.name}: {tool.description}" for tool in self.tools])
        # Create a list of tool names for the tools provided
        kwargs["tool_names"] = ", ".join([tool.name for tool in self.tools])
        formatted = self.template.format(**kwargs)
        return [HumanMessage(content=formatted)] 

class CustomOutputParser(AgentOutputParser):
    def parse(self, llm_output: str) -> Union[AgentAction, AgentFinish]:
        # Check if agent should finish
        if "Marathon Day" in llm_output:
            return AgentFinish(
                return_values={"output": llm_output.split("Marathon Day:")[-1].strip()},
                log=llm_output,
            )
        elif marathon_date in llm_output:
            return AgentFinish(
                # Return values is generally always a dictionary with a single `output` key
                # It is not recommended to try anything else at the moment :)
                return_values={"output": llm_output.split(marathon_date)[-1].strip()},
                log=llm_output,
            )
        # Parse out the action and action input
        regex = r"Action\s*\d*\s*:(.*?)\nAction\s*\d*\s*Input\s*\d*\s*:[\s]*(.*)"
        match = re.search(regex, llm_output, re.DOTALL)
        if not match:
            try:
                action = match.group(1).strip()
                action_input = match.group(2)
                # Return the action and action input
                return AgentAction(tool=action, tool_input=action_input.strip(" ").strip('"'), log=llm_output)
            except ValueError as e:
                response = str(e)
                if not response.startswith("Could not parse LLM output: `"):
                    raise e
                response = response.removeprefix("Could not parse LLM output: `").removesuffix("`")

That helps shape the output. Let's work on receiving the input.

Make a Streamlit web app in Python

To make a webpage to receive input about the user for their personal marathon training plan, we will use Streamlit.

In request.py, we set a title, subheader, the dates that are first displayed when the date input widget is rendered, and a Streamlit form containing two text_input fields for the user's Strava token and email, make the date input widget, and then write the input dates to the page and split up the input dates into two variables. This code goes directly beneath response = response.removeprefix("Could not parse LLM output: ").removesuffix("").

st.title('Personal Marathon Training plan generator')
st.subheader('enter details below')
today = datetime.now()
jul_30 = date(today.year, 7, 30)
dec_31 = date(today.year, 12, 31)

with st.form('my_form'):
    strava_token_input = st.text_input('Strava API token')
    email = st.text_input('Email to send plan to')
    dates = st.date_input(
        "Select your training dates for the year: it should be from the first training date to the marathon date",
        (jul_30, date(today.year, 12, 10)),
        jul_30,
        dec_31
    )
    st.write('Training date: ', dates[0], "Marathon date: ", dates[1])
    training_start_date = dates[0]
    marathon_date = dates[1]
    submitted = st.form_submit_button('Submit')

Run the command streamlit run request.py in your terminal. An application should open up on your localhost:8080:

Beneath the last line marathon_date = dates[1], add the following code to validate the user input:

if submitted:

        if not validate_email(email):
            st.error("invalid email", icon="🚨")
            st.cache_data()
            VALIDANSWERS = False
        elif training_start_date > marathon_date:
            st.error("Start date must be earlier than end date", icon="🚨")
            st.cache_data()
            VALIDANSWERS = False     
        try:
            r = requests.get('https://www.strava.com/api/v3/activities?access_token=' + strava_token_input)
            r.raise_for_status()
            VALIDANSWERS = True
        except requests.exceptions.HTTPError as e:
            print(e)
            st.error("Check Strava token")
            st.cache_data()
            VALIDANSWERS = False
        if VALIDANSWERS:

Get Strava data into Pandas and CSV form

The parameters and headers need to be set in order to call a Strava API request. my_dataset contains data from just the first page of user activities, so to retrieve more activity data, loop through four pages of Strava activities and add them to my_dataset before calling pd.json_normalize on it to turn the semi-structured JSON data into a flat table to work with it.

Directly beneath the if VALIDANSWERS: line, add the following code:

 header = {'Authorization': 'Bearer ' + strava_token_input} 
            params = {'per_page': 200, 'page': 1} #max 200 per page, can only do 1 page at a time
            my_dataset = requests.get(activities_url, headers=header, params=params).json() #activities 1st page
            page = 0
            for x in range(1,5): #loop through 4 pages of strava activities
                page +=1 
            params = {'per_page': 200, 'page': page}
            my_dataset += requests.get(activities_url, headers=header, params=params).json() 

            activities = pd.json_normalize(my_dataset)

Next, create a new Pandas Dataframe object with specific columns containing the categories you want to consider: this tutorial is only looking at the activity name, type (bike ride, run, swim, tennis, etc), distance, moving time, total elevation gain, and start date.

The following code goes beneath activities = pd.json_normalize(my_dataset) and creates a CSV file of the activities data after removing items from 2021 so it only includes workouts from 2022 and 2023. You can replace this with another year! It then makes a CSV file named data_files/runs.csv containing the runs and no other activity types.

cols = ['name', 'type', 'distance', 'moving_time', 'total_elevation_gain', 'start_date']
            activities = activities[cols]
            activities = activities[activities["start_date"].str.contains("YEAR-YOU-WANT-TO-IGNORE-WORKOUTS") == False] 
            activities.to_csv('data_files/activities.csv', index=False)

            runs = activities.loc[activities['type'] == 'Run']
            runs.to_csv('data_files/runs.csv', index=False)

Now, convert meters to miles and the moving time seconds to minutes and hours with the code below:

 data_run_df = pd.read_csv('data_files/runs.csv')
            data_activity_df = pd.read_csv('data_files/activities.csv')
            m_conv_factor = 1609

            data_run_df['distance'] = data_run_df['distance'].map(lambda x: convert_to_miles(x))
            data_activity_df['distance'] = data_activity_df['distance'].map(lambda x: convert_to_miles(x))
            #convert moving time secs to mins, hours
            data_run_df['moving_time'] = data_run_df['moving_time'].astype(str).map(lambda x: x[7:]) #slices off 0 days from moving_time
            data_run_df.to_csv('data_files/runs.csv')
            data_activity_df = pd.read_csv('data_files/activities.csv')

Next, this data will be passed to a LangChain agent.

Generate a Personal Marathon Plan with LangChain Agents

In the same request.py file, make an array called tools containing the tools the agent has access to. Each tool gets a name, function to run, and description. The description is optional, but recommended because it's used by an agent to determine tool use.

Beneath creating the data_activity_df variable, add the tools array:

tools = [
                Tool(
                    name = "rows in csv",
                    func = lambda df: num_rows_in_dataframe(df),
                    description="use to get the number of rows in csv file to calculate averages from running data"
                ),
                Tool(
                    name="Calculator",
                    func=llm_math_chain.run,
                    description="useful for when you need to answer questions about math"
                ),
                Tool(
                    name="days_between",
                    func=lambda day1, day2=marathon_date: calc_days_btwn(day1, day2),
                    description="useful for when you need to calculate the number of days between two dates of type date, like between the training start date and the marathon date"
                )
            ]

Now, create some Pandas dataframe agents to interact with the Strava data. These agents are similar to CSV agents which load data from CSV files instead of dataframes and perform queries. You could use both and see how the output differs.

Agents can be chained together to build more complex applications. Another agent used below is of type ZERO_SHOT_REACT_DESCRIPTION. Zero-shot means the agent functions only on the present action — it lacks memory. It uses the ReAct framework to decide which tool to use, solely based on the tool’s description.

The code runs the agents, telling them what to do and saving the output to variables for use later. Be specific with the prompts here and play around with different questions or commands.

Beneath the tools array, add:

pd_agent_run = create_pandas_dataframe_agent(OpenAI(temperature=0.3), data_run_df, verbose=True) 
pd_agent_activity = create_pandas_dataframe_agent(OpenAI(temperature=0.3), data_activity_df, verbose=True)
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
num_workouts = agent.run(f'Calculate the number of days between these two dates representing the number of workouts in the plan: {marathon_date} and {training_start_date}. You have access to tools')
pd_run_output = pd_agent_run.run("Calculate average distance and maximum distance in miles and average moving time and maximum moving time in minutes. Additionally, calculate any other running statistics from the data you think would be helpful to consider in marathon training.")

Next, make a prompt template, a "reproducible way to generate a prompt" containing a text string ("the template"). It accepts a set of parameters from the user and generates a prompt. This mirrors the items in stapp.py's examples array.

prefix contains directions and context to the LLM and suffix is the user input and output indicator. Finally, these, along with the input variables which are unique to the student the plan is being made for, are passed into FewShotPromptTemplate.

coach_template = """training_start_date:{training_start_date}, marathon_date:{marathon_date}, pd_run_output: {pd_run_output}, num_workouts: {num_workouts}, plan: {plan}
            """
example_prompt = PromptTemplate(input_variables=["training_start_date", "marathon_date", "pd_run_output", "num_workouts", "plan"], template=coach_template)

prefix = """
            You are a marathon trainer tasked with crafting one personalized marathon training plan containing  according to your student's previous runs and activities.
            For each week you should recommend 2 cross-training activities and no more than one run greater than 14 miles per week. The longest run you recommend should be 20 miles around 2 weeks before {marathon_date}.
            There should be {num_workouts} workouts for {num_workouts} days, and there should be easy workouts and rest days in the week leading up to {marathon_date}.
            Slightly modify the following example marathon training plans based on your student and the days beginning on {training_start_date} up until {marathon_date} to create one personalized training plan with each workout on a new line: 
            """
suffix = """
            training_start_date: {training_start_date}, marathon_date: {marathon_date}, pd_run_output: {pd_run_output}, num_workouts: {num_workouts}
            """
few_shot_prompt_template = FewShotPromptTemplate(
                examples=stapp.examples, 
                example_prompt=example_prompt, 
                prefix=prefix,
                suffix=suffix, 
                input_variables=["marathon_date", "training_start_date", "pd_run_output", "num_workouts"]
            )

Create a custom output parser object to help structure the response from the LLM and LLM chain, the most common type of chain. The LLM chain consists of the model and prompt. For this case it's an LLM but it could be a ChatModel. The LLM is passed to a LLMSingleActionAgent to complete a single action and structure the response from the LLM.

This agent is passed to an AgentExecutor. An executor differs from an agent in that it is a running instance of an agent used to execute tasks according to the agent’s decisions and configured tools. In short, agents decide which tools to use and the executors execute them. The output of the Executor (which is given the user input) is the personalized marathon training plan. Beneath the FewShotPromptTemplate() function, add the following code:

output_parser = CustomOutputParser()

#LLM chain consisting of the LLM and a prompt
llm_chain = LLMChain(llm=llm, prompt=few_shot_prompt_template)
tool_names = [tool.name for tool in tools]
agent = LLMSingleActionAgent(
    llm_chain=llm_chain, 
    output_parser=output_parser,
    stop=["\Marathon Day:"], 
    allowed_tools=tool_names
)
agent_executor = AgentExecutor.from_agent_and_tools(agent=agent, tools=tools, verbose=True)
plan = agent_executor({"marathon_date":marathon_date, "training_start_date": training_start_date, "pd_run_output": pd_run_output, "num_workouts": num_workouts})
print('plan being print ', plan)

Email the Plan as a PDF using Twilio SendGrid

We access the part of the plan containing the dates, replace new lines with breaks because the ReportLab Python library to generate and format PDFs ignores \n, and send an outbound email using Twilio SendGrid–the email body says "good luck at your marathon on {marathon_date}" with the plan attached.

plan = plan['output']
plan = str(plan).replace('\n','<br />') #each workout on new line
message = Mail(
                from_email='langchain_sendgrid_marathon_trainer@sf.com',
                to_emails=email,
                subject='Your AI-generated marathon training plan',
                html_content='<strong>Good luck at your marathon on %s</strong>!\n\nYour plan is attached.'%(marathon_date))

The email attachment is formatted and styled using ReportLab. The following code returns a stylesheet with a few basic heading and text styles to format the PDF attachment which is in the form of a SimpleDocTemplate document. The document is built using the Paragraph class that lets us format the text with inline font style (and optional color changes) using an XML style markup.

The encoded file is passed as an attachment to the SendGrid email. If the email is sent successfully, the Streamlit app displays a success message; otherwise, it displays an error message.

styleN = getSampleStyleSheet()['Normal']
            story = []

            pdf_name = 'plan.pdf'
            doc = SimpleDocTemplate(
                pdf_name,
                pagesize=letter,
                bottomMargin=.4 * inch,
                topMargin=.6 * inch,
                rightMargin=.8 * inch,
                leftMargin=.8 * inch)
            P = Paragraph(plan, styleN)
            story.append(P)

            doc.build(
                story,
            )
            with open(pdf_name, 'rb') as f:
                data = f.read()
                f.close()
            encoded_file = base64.b64encode(data).decode()

            attachedFile = Attachment(
                FileContent(encoded_file),
                FileName('personal_ai_generated_marathon_training_plan.pdf'),
                FileType('application/pdf'),
                Disposition('attachment')
            )
            message.attachment = attachedFile
            sg = SendGridAPIClient()
            response = sg.send(message)
            if response.status_code == 202:
                st.success("Email sent! Check your email for your personal training plan")
                print(f"Response Code: {response.status_code} \n Message sent!")
            else:
                st.warning("Email not sent--check email")

Run streamlit run request.py in your terminal. Fill in your information (like Strava Access Token, the email where you want to receive the generated plan, the training start date, and marathon race date!) on the Streamlit app, click Submit, and wait for your personal marathon training plan in your email.

If the Submit button is clicked, we check the input (dates and email). If the inputs are either:

invalid: the Streamlit app displays an error message and caches the function so the user can try valid input.
valid: the Streamlit app displays a success message and it's time to use the user's Strava API token to get their Strava workout data.

Play around with different agents, prompting, and different parameters like temperature (which affects the risks the agent takes and creativity of the output).

The complete code above can be found here on GitHub.

What's Next for Agents and SendGrid

So will I use this to train for my marathon? I haven't coded this much since college (or had this much fun doing so) and this was a great way to learn many of LangChain's features, but time will tell –work travel, tennis practices, and AI hackathons make consistent training difficult. I do want to continue trying different ways of changing the output--this is a lot of code which I feel like could be unnecessary, but it yielded the best results.

There's so much developers can do with communication APIs and AI–this is just the tip of the iceberg. You can use a different LLM, look at different Strava data, try different agents (or use chains instead!), embeddings, use LLaMA 2 or other similar models on Replicate, and more. You could use SMS or Twilio Conversations to collect user input instead of Streamlit, or make the LLM recommend other workouts for different goals or meals based on nutrition data!

Let me know online what you're building (and send running tips!)

Build an AI SMS Chatbot with Replicate, LLaMA 2, and LangChain

Lizzie Siegle — Tue, 12 Sep 2023 22:16:09 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

Recently, Meta and Microsoft introduced the second generation of the LLaMA LLM (Large Language Model) to help developers and organizations to build generative AI-powered tools and experiences. Read on to learn how to build an AI SMS chatbot that answers questions like Ahsoka (from Star Wars) using LangChain templating, LLaMa 2, Replicate, and Twilio Programmable Messaging!

Do you prefer learning via video more? Check out this TikTok summarizing this tutorial in 1 minute!

Prerequisites

A Twilio account - sign up for a free Twilio account here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
Replicate account to host the LlaMA 2 model – make a Replicate account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access.

Replicate

Replicate offers a cloud API and tools so you can more easily run machine learning models, abstracting away some lower-level machine learning concepts and handling infrastructure so you can focus more on your own applications. You can run open-source models that others have published, or package and publish your own, either publicly or privately.

Configuration

Since you will be installing some Python packages for this project, you will need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir replicate-llama-ai-sms-chatbot  
cd replicate-llama-ai-sms-chatbot  
python3 -m venv venv 
source venv/bin/activate 
pip install langchain replicate flask twilio

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir replicate-llama-ai-sms-chatbot  
cd replicate-llama-ai-sms-chatbot   
python -m venv venv 
venv\Scripts\activate 
pip install langchain replicate flask twilio

Grab your default Replicate API Token or create a new one here.

On the command line run

export REPLICATE_API_TOKEN={replace with your api token}

Now it's time to write some code!

Code

Make a file called app.py and place the following import statements at the top.

from flask import Flask, request
from langchain import LLMChain, PromptTemplate
from langchain.llms import Replicate
from langchain.memory import ConversationBufferWindowMemory
from twilio.twiml.messaging_response import MessagingResponse

Though LLaMA 2 is tuned for chat, templates are still helpful so the LLM knows what behavior is expected of it. This starting prompt is similar to ChatGPT so it should behave similarly.

template = """Assistant is a large language model.

Assistant is designed to be able to assist with a wide range of tasks, from answering simple questions to providing in-depth explanations and discussions on a wide range of topics. As a language model, Assistant is able to generate human-like text based on the input it receives, allowing it to engage in natural-sounding conversations and provide responses that are coherent and relevant to the topic at hand.

Assistant is constantly learning and improving, and its capabilities are constantly evolving. It is able to process and understand large amounts of text, and can use this knowledge to provide accurate and informative responses to a wide range of questions. Additionally, Assistant is able to generate its own text based on the input it receives, allowing it to engage in discussions and provide explanations and descriptions on a wide range of topics.

Overall, Assistant is a powerful tool that can help with a wide range of tasks and provide valuable insights and information on a wide range of topics. Whether you need help with a specific question or just want to have a conversation about a particular topic, Assistant is here to assist. 

I want you to act as Ahsoka giving advice and answering questions. You will reply with what she would say.
SMS: {sms_input}
Assistant:"""

prompt = PromptTemplate(input_variables=["sms_input"], template=template)

Next, make a LLM Chain, one of the core components of LangChain. This allows us to chain together prompts and make a prompt history. The model is formatted as the model name followed by the version–in this case, the model is LlaMA 2, a 13-billion parameter language model from Meta fine-tuned for chat completions. max_length is 4096, the maximum number of tokens (called the context window) the LLM can accept as input when generating responses.

sms_chain = LLMChain(
    llm = Replicate(model="a16z-infra/llama13b-v2-chat:df7690f1994d94e96ad9d568eac121aecf50684a0b0963b25a41cc40061269e5"), 
    prompt=prompt,
    memory=ConversationBufferWindowMemory(k=2),
    llm_kwargs={"max_length": 4096}
)

Finally, make a Flask app to accept inbound text messages, pass that to the LLM Chain, and return the output as an outbound text message with Twilio Programmable Messaging.

app = Flask(__name__)


@app.route("/sms", methods=['GET', 'POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip()
    output = sms_chain.predict(sms_input=inb_msg)
    print(output)
    resp.message(output)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

On the command line, run python app.py to start the Flask app.

Configure a Twilio Number for the SMS Chatbot

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL with the /sms path in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio phone number is configured so that it maps to your web application server running locally on your computer and your application can run. Text your Twilio number a question relating to the text file and get an answer from that file over SMS!

You can view the complete code on GitHub here.

What's Next for Twilio, LangChain, Replicate, and LLaMA 2?

There is so much fun for developers to have around building with LLMs! You can modify existing LangChain and LLM projects to use LLaMA 2 instead of GPT, build a web interface using Streamlit instead of SMS, fine-tune LLaMA 2 with your own data, and more! I can't wait to see what you build–let me know online what you're working on!

Build a SMS Bot that Answers questions over Docs with LangChain in Python

Lizzie Siegle — Thu, 15 Jun 2023 20:33:43 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

With Natural Language Processing (NLP), you can chat with your own documents, such as a text file, a PDF, or a website. Read on to learn how to build a generative question-answering SMS chatbot that reads a document containing Lou Gehrig's Farewell Speech using LangChain, Hugging Face, and Twilio in Python.

LangChain Q&A

LangChain is an open-source tool that wraps around many large language models (LLMs) and tools. It is the easiest way (if not one of the easiest ways) to interact with LLMs and build applications around LLMs.

LangChain makes it easy to perform question-answering of those documents. Picture feeding a PDF or maybe multiple PDF files to a machine and then asking it questions about those files. This could be useful, for example, if you have to prepare for a test and wish to ask the machine about things you didn’t understand.

Prerequisites

A Twilio account - sign up for a free Twilio account here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
Hugging Face Account – make a Hugging Face Account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access. > ⚠️ ngrok is needed for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. You can also choose to automate ngrok as shown in this article.: ## Configuration Since you will be installing some Python packages for this project, you will need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir lc-qa-sms 
cd lc-qa-sms 
python3 -m venv venv 
source venv/bin/activate 
!pip install langchain
!pip install requests
pip install flask
pip install faiss-cpu
pip install sentence-transformers
pip install twilio
pip install load_dotenv

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir lc-qa-sms  
cd lc-qa-sms 
python -m venv venv 
venv\Scripts\activate 
pip install langchain
pip install requests
pip install flask
pip install faiss
pip install sentence-transformers
pip install twilio
pip install load_dotenv

Set up Hugging Face Hub

The Hugging Face Hub offers over 120k models, 20k datasets, and 50k demos people can easily collaborate in their ML workflows. As mentioned earlier, this project needs a Hugging Face Hub Access Token to use the LangChain endpoints to a Hugging Face Hub LLM. After making a Hugging Face account, you can get a Hugging Face Access Token here by clicking on New token. Give the token a name and select the Read Role.

Alternatively, you could use models from, say, OpenAI.

On the command line in your root directory, run the following command on a Mac to set the token as an environment variable.

export HUGGINGFACEHUB_API_TOKEN=replace-with-your-huggingfacehub-token

For the Windows command, check out this blog post on environment variables.

Now let's build that LangChain question-answering chatbot application.

Answer Questions from a Doc with LangChain via SMS

Inside your lc-qa-sms directory, make a new file called app.py.

At the top of the file, add the following lines to import the required libraries.

import requests
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains.question_answering import load_qa_chain
from langchain import HuggingFaceHub
from flask import Flask, request, redirect
from twilio.twiml.messaging_response import MessagingResponse

Beneath those import statements, make a helper function to write to a local file from a URL, and then load that file from the local file storage with LangChain's TextLoader library. Later the file will be passed over to the split method to create chunks.

def loadFileFromURL(text_file_url):
    output_file = "lougehrig.txt"
    resp = requests.get(text_file_url)
    with open(output_file, "w",  encoding='utf-8') as file:
      file.write(resp.text)

    # load text doc from URL w/ TextLoader
    loader = TextLoader('./'+output_file)
    txt_file_as_loaded_docs = loader.load()
    return txt_file_as_loaded_docs

In this tutorial, the file will be Lou Gehrig's famous speech, which is why the output_file is named "lougehrig.txt".

You could alternatively use a local file.

Next, add the following code for a helper function to split the document into chunks. This is important because LLMs can't process inputs that are too long. LangChain's CharacterTextSplitter function helps us do this–setting chunk_size to 1000 and chunk_overlap to 10 keeps the integrity of the file by avoiding splitting words in half.

def splitDoc(loaded_docs):
    # split docs into chunks
    splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=10)
    chunked_docs = splitter.split_documents(loaded_docs)
    return chunked_docs

Now convert the chunked document into embeddings (numerical representations of words) with Hugging Face and store them in a FAISS Vector Store. Faiss is a library for efficient similarity search and clustering of dense vectors.

def makeEmbeddings(chunked_docs):
    # Create embeddings and store them in a FAISS vector store
    embedder = HuggingFaceEmbeddings()
    vector_store = FAISS.from_documents(chunked_docs, embedder)
    return vector_store

That makeEmbeddings function will make it more efficient to retrieve and manipulate the stored embeddings to conduct a similarity search to get the most semantically-similar documents to a given input which the LLM needs to best answer questions in the following helper function.

def askQs(vector_store, chain, q):
    # Ask a question using the QA chain
    similar_docs = vector_store.similarity_search(q)
    resp = chain.run(input_documents=similar_docs, question=q)
    return resp

The final helper function defines and loads the Hugging Face Hub LLM to be used with your Access Token and starts the request on a similarity search embedded with some input question to the selected LLM, enabling a question-and-answer conversation.

def loadLLM():
    llm=HuggingFaceHub(repo_id="declare-lab/flan-alpaca-large", model_kwargs={"temperature":0, "max_length":512})
    chain = load_qa_chain(llm, chain_type="stuff")
    return chain

declare-lab/flan-alpaca-large is a LLM. You could use others from Hugging Face Hub such as google-flan-t5-xl. They are trained on different corpuses and this tutorial uses flan-alpaca-large because it's faster.

Lastly, make a Flask app using the Twilio REST API to call the helper functions and respond to inbound text messages with information pulled from the text file.

app = Flask(__name__)
@app.route('/sms', methods=['POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip() #get inbound text body
    chain = loadLLM()
    LOCAL_ldocs = loadFileFromURL('https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt')
    LOCAL_cdocs = splitDoc(LOCAL_ldocs) #chunked
    LOCAL_vector_store = makeEmbeddings(LOCAL_cdocs)
    LOCAL_resp = askQs(LOCAL_vector_store, chain, inb_msg)
    resp.message(LOCAL_resp)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

Your complete app.py file should look like this:

import requests
from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter
from langchain.embeddings import HuggingFaceEmbeddings
from langchain.vectorstores import FAISS
from langchain.chains.question_answering import load_qa_chain
from langchain import HuggingFaceHub
from flask import Flask, request, redirect
from twilio.twiml.messaging_response import MessagingResponse

def loadFileFromURL(text_file_url): #param: https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt
    output_file = "lougehrig.txt"
    resp = requests.get(text_file_url)
    with open(output_file, "w",  encoding='utf-8') as file:
      file.write(resp.text)

    # load text doc from URL w/ TextLoader
    loader = TextLoader('./'+output_file)
    txt_file_as_loaded_docs = loader.load()
    return txt_file_as_loaded_docs

def splitDoc(loaded_docs):
    # split docs into chunks
    splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=10)
    chunked_docs = splitter.split_documents(loaded_docs)
    return chunked_docs

def makeEmbeddings(chunked_docs):
    # Create embeddings and store them in a FAISS vector store
    embedder = HuggingFaceEmbeddings()
    vector_store = FAISS.from_documents(chunked_docs, embedder)
    return vector_store

def askQs(vector_store, chain, q):
    # Ask a question using the QA chain
    similar_docs = vector_store.similarity_search(q)
    resp = chain.run(input_documents=similar_docs, question=q)
    return resp

def loadLLM():
    llm=HuggingFaceHub(repo_id="declare-lab/flan-alpaca-large", model_kwargs={"temperature":0, "max_length":512})
    chain = load_qa_chain(llm, chain_type="stuff")
    return chain


app = Flask(__name__)
@app.route('/sms', methods=['GET', 'POST'])
def sms():
    resp = MessagingResponse()
    inb_msg = request.form['Body'].lower().strip() #get inbound text body
    chain = loadLLM()
    LOCAL_ldocs = loadFileFromURL('https://raw.githubusercontent.com/elizabethsiegle/qanda-langchain-sms-lougehrig/main/lougehrig.txt')
    LOCAL_cdocs = splitDoc(LOCAL_ldocs) #chunked
    LOCAL_vector_store = makeEmbeddings(LOCAL_cdocs)
    LOCAL_resp = askQs(LOCAL_vector_store, chain, inb_msg)
    resp.message(LOCAL_resp)
    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

On the command line, run python app.py to start the Flask app.

Configure a Twilio Number for the SMS Chatbot

Now, your Flask app will need to be visible from the web so Twilio can send requests to it. ngrok lets you do this. With ngrok installed, run ngrok http 5000 in a new terminal tab in the directory your code is in.

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL with the /sms path in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio phone number is configured so that it maps to your web application server running locally on your computer and your application can run. Text your Twilio number a question relating to the text file and get an answer from that file over SMS!

The complete code can be found here on GitHub.

What's Next for Twilio and LangChain

There's so much you can do with document-based question-answering. You can use a different LLM, use a longer document than a text file containing Lou Gehrig's famous speech, use other types of documents like a PDF or website (here's LangChain's docs on documents), store the embeddings elsewhere, and more. Other than a SMS chatbot, you could create an AI tutor, search engine, automated customer service agent, and more.

Let me know online what you're building!

Build a Basketball SMS Chatbot with LangChain Prompt Templates in Python

Lizzie Siegle — Wed, 31 May 2023 22:00:33 +0000

This blog post was written for Twilio and originally published on the Twilio blog.

As I've played with the OpenAI API (and DALL·E and fine-tuning models) and gotten more into ML, developers I meet (as well as my wonderful coworker Craig Dennis who advised me on this tutorial) keep telling me to use LangChain, a powerful and flexible framework for developing applications powered by language models.

Read on to learn how to build an SMS chatbot using LangChain prompt templates, OpenAI, Twilio Programmable Messaging, and Python.

Large Language Models

Large Language Models are trained on large quantities of textual data (i.e. in ChatGPT's case, the entire internet up to 2021–so you can provide more context or data that the model is missing with prompt engineering and Prompt Templates–more on those later) to produce human-like responses to dialogue or other natural language inputs.

To yield these natural language responses, LLMs use deep learning (DL) models, which use multi-layered neural networks to process, analyze, and make predictions with complex data.

LangChain

LangChain came out late last year and already has over 43,000 stars on GitHub and a thriving community of contributors.

It is many things, but ultimately at its core is an open-source framework that simplifies the development of applications using large language models (LLMs), like OpenAI or Hugging Face. Developers can use it for chatbots, Generative Question-Answering (GQA), summarization, and more.

With LangChain, developers can “chain” together different LLM components to create more advanced use cases around LLMs. Chains can consist of multiple components from several modules:

Prompt Templates: Prompt templates are templates for different types of prompts. Like “chatbot” style templates, ELI5 question-answering, etc
LLMs: Large language models like GPT-3, Hugging Face, BLOOM, etc
Agents: Agents use LLMs to decide what actions should be taken. Tools like web search or calculators can be used, and all are packaged into a logical loop of operations.
Memory: Short-term memory, long-term memory. LangChain recognizes the power of prompts and has built an entire set of objects for them. It also provides a wrapper around different LLMs so you can easily change models, swapping them out with different templates. The chat model could be different, but running and calling it is the same–a very Java-like concept! ## LangChain Twilio Tool Recently, LangChain came out with a Twilio tool so your LangChain Agents are able to send text messages. For example, your LLM can understand the input in natural language, but Agents can let you complete different tasks like calling an API.

You'll need your own Twilio credentials and to install it with pip install twilio. The code for the tool would look something like this:

from langchain.utilities.twilio import TwilioAPIWrapper

twilio = TwilioAPIWrapper(
     account_sid="YOUR-ACCOUNT-SID",
     auth_token="YOUR-TWILIO-AUTH-TOKEN",
     from_number="YOUR-TWILIO-NUMBER,"
)
twilio.run("hello world", "NUMBER-TO-TEXT")

This tutorial will, however, show you how to use LangChain Prompt Templates with Twilio to make a SMS chatbot.

LangChain Prompt Templates

"Prompts” refer to the input to the model and are usually not hard-coded, but are more often constructed from multiple components. A Prompt Template helps construct this input. LangChain provides several classes and functions to make constructing and working with prompts easy.

Prompts being input to LLMs are often structured in different ways so that we can get different results. For Q&A, you could take a user’s question and reformat it for different Q&A styles, like conventional Q&A, a bullet list of answers, or even a summary of problems relevant to the given question. You can read more about prompts here in the LangChain documentation.

Prompt templates offer a reproducible way to generate a prompt. Like a reusable HTML template, you can share, test, reuse, and iterate on it, and it will update. When you update a prompt template, it updates for anyone else using it or the whole set of apps that could use it.

Prompt templates contain a text string (AKA “the template”) that can take in a set of parameters from the user and generate a prompt.

Importing and initializing a LangChain PromptTemplate class would look like so:

from langchain import PromptTemplate

template = """Question: {question}

Answer: """
prompt = PromptTemplate(
        template=template,
    input_variables=['question']
)

# user question
question = "Which NBA team won the finals in 1996?"

Since OpenAI LLMs lack data after September 2021, its models can't answer anything that occurred after without additional context. Prompt templates help provide additional context, but differ from fine-tuning - fine-tuning is like coaching the model with new data to get certain output, but prompt engineering with Prompt Templates provides the model specific data to help it get the output you want.

Let's use this template to build the chatbot.

Prerequisites

A Twilio account - sign up for a free one here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
OpenAI Account – make an OpenAI Account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access.

⚠️ ngrok is needed for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. You can also choose to automate ngrok as shown in this article.

Configuration

Since you will be installing some Python packages for this project, you will need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir lc-sms 
cd lc-sms 
python3 -m venv venv 
source venv/bin/activate 
!pip install langchain
!pip install openai
pip install Flask
pip install twilio
pip install load_dotenv

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir lc-sms 
cd lc-sms 
python -m venv venv 
venv\Scripts\activate 
pip install langchain
pip install openai
pip install Flask
pip install twilio
pip install load_dotenv

The last command uses pip, the Python package installer, to install the three packages that you are going to use in this project, which are:

The OpenAI Python client library, to send requests to OpenAI's GPT-3 engine. You could also use a different LLM, listed here in the LangChain docs.
The Twilio Python Helper library, to work with SMS messages.
The Flask framework, to create the web application in Python.
The load_dotenv library to load environment variables from a .env file
The LangChain library to building applications with LLMs through composability As mentioned above, this project needs an OpenAI API Key to use the LangChain endpoints to OpenAI. After making an OpenAI account, you can get an OpenAI API Key here by clicking on + Create new secret key. The Python application will need to have access to this key, so we are going to make a .env file where the API key can safely be stored. The application we create will be able to import this key as an environment variable soon.

Create a .env file in your project’s root directory and enter the following line of text, making sure to replace <OPENAI_API_KEY> with your actual key:

OPENAI_API_KEY= <YOUR-OPENAI-KEY>

Make sure that the OPENAI_API_KEY is safe and that you don't expose your .env file in a public location such as GitHub.

Now, your Flask app will need to be visible from the web, so Twilio can send requests to it. ngrok lets you do this: with ngrok installed, run ngrok http 5000 in a new terminal tab in the directory your code is in.

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio phone number is configured so that it maps to your web application server running locally on your computer. Let's build that application now.

Generate text through LangChain with OpenAI in Python via SMS

Inside your lc-sms directory, make a new file called app.py.

Copy and paste the following code into app.py to start off the ChatGPT-like SMS app to import the required libraries.

from flask import Flask, request, redirect
from dotenv import load_dotenv

from twilio.twiml.messaging_response import MessagingResponse

from langchain.llms import OpenAI
from langchain import PromptTemplate

import os

Then, make the OpenAI LLM object (which could be another LLM--this is where LangChain can make reusability easier!), passing it the model name and API key from the .env file, and create a Flask application.

load_dotenv()


llm = OpenAI(
    model_name="text-davinci-003",
    openai_api_key=os.environ.get('OPENAI_API_KEY')
)


app = Flask(__name__)

Make the start of the /sms webhook containing a TwiML response to respond to inbound SMS with.

@app.route("/sms", methods=['GET', 'POST'])
def sms_reply():
    # Start our TwiML response
    resp = MessagingResponse()

Add the following template code below to help shape the queries and answers from the LLM. In this tutorial, it's Warriors basketball-themed. (Yes, I know the 2023 NBA Finals are the Heat versus the Nuggets. A girl can dream.)

 template = """Answer the question based on the context below. If the question cannot be answered using the information provided, answer with "I don't know, but the Warriors are the best team in the NBA".
    Context: Steph Curry has won 4 NBA Finals series. His Golden State Warriors defeated the Cleveland Cavaliers three times and the Boston Celtics once. 

In 2015 Steph Curry and the Warriors defeated the Cleveland Cavaliers. The Cavs featured LeBron James, Kyrie Irving and not much else! 
In 2017 Steph Curry, Kevin Durant and the Warriors defeated the Cavs again. The Cavs still had Lebron and Kyrie.
In 2018 the Warriors, featuring Steph and KD again, defeated the Cavs for the third time in four years. The Cavs still had Kyrie and Lebron. 
In 2022 Steph and the Warriors defeated the Boston Celtics for his fourth title. The Celtics featured Jayson Tatum and Jaylen Brown. Steph Curry and the Golden State Warriors lost one NBA Finals series to the Cleveland Cavaliers and one to the Toronto Raptors. 

In 2016 Steph, Klay, Draymond and the rest of the Warriors lost to the Cleveland Cavaliers.  The Cavs starred LeBron James and Kyrie Irving. 
In 2019 the Steph and the Warriors, missing an injured KD, lost to the Toronto Raptors. The Raptors featured Kawhi Leonard in his only season in Canada alongside Kyle Lowry and Pascal Siakham. Steph Curry has only won one NBA Finals MVP up to this point in his career. . 

In 2022 Steph Curry averaged 31 points, 6 rebounds and 5 assists per game to win the MVP award in the Warriors 6-game defeat of the Boston Celtics.
In 2015 Andre Iguodala won the MVP in the Warriors defeat of the Cavs.
In both 2017 & 2018 Kevin Durant was Finals MVP in the Warriors victories over the Cavs. Steph Curry's 4-2 NBA Finals record puts him ahead of many NBA greats including Larry Bird (3-2) and LeBron James (4-6). Steph still comes up short of the greatest NBA Finals winners including Bill Russell (11-1) and Michael Jordan (6-0).  
    Question: {query}
    Answer: """
    basketball_query_template = PromptTemplate(
        input_variables=["query"],
        template=template
    )

Get the user's question via inbound text message, print out the answer by passing the question to the prompt template and formatting it, and then pass that to Twilio as the text message to send back to the user.

question = request.form['Body'].lower().strip()
    print(llm(
        basketball_query_template.format(
            query=question
        )
    ))
    resp.message(openai(
        basketball_query_template.format(
            query=question
        )
    ))

    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

In a new terminal tab (while the other terminal tab is still running ngrok http 5000), run python app.py. You can now text your Twilio number (questions about the Warriors because we provided the model Warriors context) configured from above.

The complete code can be found here on GitHub.

What's Next for Twilio and LangChain

LangChain can be used for chatbots (not just Warriors/basketball-themed), chaining different tasks, Generative Question-Answering (GQA), summarization, and so much more. Stay tuned to the Twilio blog for more LangChain and Twilio content, and let me know online what you're building with AI!

How to Fine-Tune an OpenAI ML Model with Node.js

Lizzie Siegle — Tue, 02 May 2023 17:40:54 +0000

This blog post was written for Twilio and originally published on the Twilio blog.
OpenAI, the creator of ChatGPT, offers a variety of Machine Learning models developers can use with different capabilities and price points. Their API lets developers interface with the models, adding features like text completion, code completion, text or image generation, speech-to-text, and more to their applications.

ChatGPT's model is powerful (this Twilio tutorial went over how to Build a Serverless ChatGPT SMS Chatbot with the OpenAI API), but still limited. This tutorial will go over how to fine-tune an existing model from OpenAI with your own data so you can get more out of it.

What is Fine-tuning?

OpenAI's models have been trained on a large amount of text from the web, but their training data is still finite and not as current as they could be: they still miss information on specific topics.

Transfer learning is a machine learning method where you use knowledge that was gained from solving one problem and apply it to a new but related problem. A model developed for a task is reused as the starting point for a model on a second task–for example, a model trained to specifically recognize pizza could be edited to recognize calzones.

Fine-tuning is similar to transfer learning in that it lets you modify and customize an existing ML model.

Why Fine-tune?

Fine-tuning helps you achieve better results on a wide number of tasks. Once a model is fine-tuned, you won't need to provide samples via prompt anymore, helping you save on costs and enabling lower-latency requests.

Assuming the original task the model was trained for is similar to the new task, fine-tuning a model that has already been designed and trained allows developers to take advantage of what the model has already learned without having to develop it from scratch.

Prerequisites

OpenAI Account - create an OpenAI Account here
Node.js installed - download Node.js here

After making an OpenAI account, you'll need an API Key. You can get an OpenAI API Key here by clicking on + Create new secret key.

Save that API key for later.

Make a New Node.js project

To get started, create a new Node.js project in an empty directory:

mkdir finetune-model
cd finetune-model
npm init --y

Get started with OpenAI

Make a .env file in your root directory and add the following line:

OPENAI_API_KEY=YOUR-OPENAI-API-KEY

Replace YOUR-OPENAI-API-KEY with the OpenAI API Key you took note of earlier. Now you will be able to access this API Key in your code with process.env.OPENAI_API_KEY.

Next, install the openai and fs npm packages:

npm install openai fs

Now it's time to prepare your data!

Prepare Custom Data

Training data is how GPT-3 learns what you want it to say.

Your data should be in a JSONL document, where each line is a prompt-completion pair corresponding to a training sample.

Make a jsonl-file called newdata.jsonl and include prompt-completion pairs that look something like this:

{"prompt": "<prompt text> ->", "completion": "<ideal generated text>\n"}
{"prompt": "<prompt text> ->", "completion": "<ideal generated text>\n"}
{"prompt": "<prompt text> -> ", "completion": "<ideal generated text>\n"}

The data I used includes fun facts about me and can be found here on GitHub.

OpenAI recommends having at least a couple hundred examples and found that each doubling of dataset size leads to a linear increase in model quality. Save at least thirty prompt-completion pairs (the more the better). This is not enough to make a fine-tuned model for a production-ready product, but is enough for the purposes of this tutorial.

Save your jsonl file, and on the command line, use OpenAI's CLI data preparation tool to help you easily convert your data into the jsonl file format.

openai tools fine_tunes.prepare_data -f newdata.jsonl

With your prepared data, it's time to write some code to fine-tune an existing OpenAI machine learning model.

Fine-tune an OpenAI Model

In this tutorial, you'll use davinci, but OpenAI currently lets you fine-tune its base models which also include curie, babbage, and ada. These are their original models and they lack any instruction after training (which text-davinci-003 has, for example).

In a file called finetune.js, add the following code to the top which will be used for each step:

const { Configuration, OpenAIApi } = require("openai");
const fs = require("fs");
const configuration = new Configuration({
  apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(configuration);

Beneath that, your first function uploads the jsonl file so you can use it next.

async function uploadFile() {
    try {
        const f = await openai.createFile(
            fs.createReadStream("newdata.jsonl"),
            "fine-tune"
        );
        console.log(`File ID ${f.data.id}`);
        return f.data.id;
    }
    catch (err) {
        console.log('err uploadfile: ', err);
    } 
}
uploadFile();

Once you run the file with node finetune.js, take note of the ID printed out for the next step and comment out the uploadFile() function call.

Beneath the uploadFile() function, add the following function:

async function makeFineTune() {
    try {
        const ft = await openai.createFineTune({
            training_file: 'YOUR-FILE-ID',
            model: 'davinci'
        });
        console.log(ft.data);
     }
    catch (err) {
        console.log('err makefinetune: ', err.response.data.error);
    }
}
makeFineTune();

Replace YOUR-FILE-ID with the file ID from the previous step, and rerun finetune.js to run the new function. After running it, comment out the makeFineTune() function call.

Wait a few minutes to let it process. You can check-in on the status of the fine tune, and additionally get the model ID, by calling the listFineTunes API method as shown below:

async function getFineTunedModelName() {
    try {
        const modelName = await openai.listFineTunes();
        console.table(modelName.data.data, ["id", "status", "fine_tuned_model"]);

    }
    catch (err) {
        console.log('err getmod: ', err)
    }
 }
getFineTunedModelName();

The fine-tuned model is not ready for use when status is running and fine_tuned_model is null, but once you see status is succeeded and fine_tuned_model is something like davinci:ft-personal-2023-05-01-22-05-28, then comment out getFineTunedModelName(), copy the model name (beginning with davinci), and you're good to go and use the model!

Test your Fine-Tuned OpenAI model

Add the following code to your finetune.js file.

async function run() {
    try {
        const comp = await openai.createCompletion({
            model: 'YOUR-FINETUNED-MODEL-NAME',
            prompt: 'What is Lizzie Siegle's favorite food', //replace this prompt according to your data
            max_tokens: 200
        });
        if (comp.data) {
            console.log('choices: ', comp.data.choices)
        }
    } catch (err) {
        console.log('err: ', err)
    }
}
run();

Replace YOUR-FINETUNED-MODEL-NAME with the model name from the previous step.

Then, rerun node finetune.js and you should see your new custom model deployed in action making a new completion according to the prompt you passed in!

Alternatively, you can test out your fine-tuned model in the OpenAI Playground.

Complete code can be found here on GitHub.

What's Next for Fine-tuning OpenAI models?

That's how to customize one of OpenAI's existing models to fit your own specific use case and data!

For next steps, you can

add more data to improve completions and make the model production-ready
try fine-tuning a different model other than davinci
use all the other Completions parameters like temperature, frequency_penalty, presence_penalty, etc., on your requests to fine-tuned models
fine-tune a model in another programming language
and more!

Build a ChatGPT-like SMS Chatbot with OpenAI and Python

Lizzie Siegle — Thu, 12 Jan 2023 21:48:48 +0000

This blog post was written for Twilio and originally published on the Twilio blog.
ChatGPT went viral recently. This conversational machine learning (ML) chatbot developed by OpenAI can answer questions, admit its mistakes, challenge incorrect premises, generate stories and poetry, and more. Read on to learn how to build a ChatGPT-like SMS chatbot using the OpenAI API and Twilio Programmable Messaging with Python.

Read this blog post if you'd like to learn how to build the same application but using Node.js.

ChatGPT and Python

GPT-3 (short for “Generative Pre-training Transformer 3”) is a natural language processing (NLP) model trained on human-generated text. Given some text input, it can generate its own human-like text based in a variety of languages and styles. Here, I ask it to "give me some bars about SendGrid."

You can test it out in the browser here.

The GPT-3 model uses a transformer architecture. This multi-layer neural network is good for processing sequential data, like text. Language-related tasks it can perform include translation, summarization, and question answering, as well as text generation comparable to human text generation.

To make a ChatGPT-like application via SMS with Python, you must use the OpenAI API.

Prerequisites

A Twilio account - sign up for a free one here
A Twilio phone number with SMS capabilities - learn how to buy a Twilio Phone Number here
OpenAI Account – make an OpenAI Account here
Python installed - download Python here
ngrok, a handy utility to connect the development version of our Python application running on your machine to a public URL that Twilio can access. This is needed for the development version of the application because your computer is likely behind a router or firewall, so it isn’t directly reachable on the Internet. You can also choose to automate ngrok as shown in this article. ### Configuration Since you will be installing some Python packages for this project, you will need to make a new project directory and a virtual environment.

If you're using a Unix or macOS system, open a terminal and enter the following commands:

mkdir chatgpt-sms-python
cd chatgpt-sms-python
python3 -m venv venv
source venv/bin/activate
pip install openai twilio flask python-dotenv

If you're following this tutorial on Windows, enter the following commands in a command prompt window:

mkdir chatgpt-sms-python
cd chatgpt-sms-python
python -m venv venv
venv\Scripts\activate
pip install openai twilio flask python-dotenv

The last command uses pip, the Python package installer, to install the four packages that you are going to use in this project, which are:

The OpenAI Python client library, to send requests to OpenAI's GPT-3 engine.
The Twilio Python Helper library, to work with SMS messages.
The Flask framework, to create the web application in Python.
The python-dotenv package, to read a configuration file. As mentioned above, this project needs an OpenAI API Key. After making an OpenAI account, you can get an OpenAI API Key here by clicking on + Create new secret key. The Python application will need to have access to this key, so we are going to make a .env file where the API key can safely be stored. The application we create will be able to import this key as an environment variable soon.

Create a .env file in your project’s root directory and enter the following line of text, making sure to replace with your actual key:

OPENAI_API_KEY= <YOUR-OPENAI-KEY>

Make sure that the OPENAI_API_KEY is safe and that you don't expose your .env file in a public location such as GitHub.

Now, your Flask app will need to be visible from the web so Twilio can send requests to it. ngrok lets you do this. With ngrok installed, run ngrok http 5000 in a new terminal tab in the directory your code is in.

You should see the screen above. Grab that ngrok Forwarding URL to configure your Twilio number: select your Twilio number under Active Numbers in your Twilio console, scroll to the Messaging section, and then modify the phone number’s routing by pasting the ngrok URL in the textbox corresponding to when A Message Comes In as shown below:

Click Save and now your Twilio Phone Number is configured so that it maps to your web application server running locally on your computer. Let's build that application now.

Build your ChatGPT-like SMS Python App

Inside your chatgpt-sms-python directory, make a new file called app.py.

Copy and paste the following code into app.py to start off the ChatGPT-like SMS app. It imports the required modules, sets the OpenAI API key from the .env file, and creates a Flask app.

from flask import Flask, request
from twilio.twiml.messaging_response import MessagingResponse
import os
import openai
openai.api_key = os.getenv("OPENAI_API_KEY")
app = Flask(__name__)

Directly beneath, add the following code:

@app.route("/sms", methods=['POST'])
def chatgpt():
    """get incoming message"""
    inb_msg = request.form['Body'].lower()
    print(inb_msg)
    response = openai.Completion.create(
        model="text-davinci-003",
        prompt=inb_msg,
        max_tokens=3000,
        temperature=0.7
    )
    """Respond to incoming calls with a simple text message."""
    # Start our TwiML response
    resp = MessagingResponse()
    # Add a message
    resp.message(response["choices"][0]["text"])
    print(response["choices"][0]["text"])

    return str(resp)

if __name__ == "__main__":
    app.run(debug=True)

Inside of the /sms webhook, this code creates a variable inbMsg from the inbound text message users will text in and prints it out. It then calls the openai.Completion.create method to use one of their language models to generate text based on inbMsg.

In order to specify our completion, you pass in some properties: model identifies the OpenAI language model used to generate an answer for the text which we assign to the prompt property. In this tutorial, you use the text-davinci-003 language model. It's the same language model used in the background by ChatGPT. The OpenAI docs list the other language models they offer for use.

Optional properties max_tokens and frequency_penalty specify the maximum completion length and the effort the model will make to not repeat itself. You can see more optional properties for completion here in the OpenAI documentation.

You can view the complete app on GitHub here.

Run the Python Flask app by running the following command

python app.py

Now take out your phone and text your Twilio Phone Number a question or prompt so that OpenAI can answer it or generate text!

What's Next for Twilio and ChatGPT?

The development possibilities offered by ChatGPT and Twilio are endless! You can build an SMS chatbot with Python, call an AI friend, chat with an AI chef over WhatsApp, use the OpenAI API with Node.js and Twilio Serverless, and more. Let me know what you're working on with OpenAI or Python–I can't wait to see what you build.