Forem: Albert Bennett

Data Structures in C# - The complex ones... and some other ones as well

Albert Bennett — Sat, 16 Mar 2024 15:34:34 +0000

Hi and welcome to my post!
Today I'll be going through the more complex data structures.
I think most people reading this will be familiar with some of the more common data structures such as arrays, lists and dictionaries but, I wonder how many would be familiar with the likes of binary trees, stacks and linked lists.
In this post I'll show you the wonderful world of data structures.

Feel free to leave a like if you learned something new or found out something interesting.

There is a lot in this post so I've added link to the individual header in this post for reference:

Binary Tree
Stacks
Queues
DataTables
Hashsets

Binary Trees

Here we go starting of with the more complex of the data structures.
Before we get into the code implementation of a flavor of binary tree lets first define the common elements of binary trees:

Node: This object represents the connection between itself and other nodes in the binary tree
Root Node: This is the first node in the tree and has no parent but, can have children
Child: Any node that has a parent, is a child of that parent.
Sibling: This is any node that lays on the same depth as the current node and has the same parent node
Parent: This is any node that has children, in a binary tree a parent can have up to two children
Leaf: This is a child that has no children
Depth: This is the position in the tree that the node is in

Here is a diagram of an example full binary tree:

In our example tree above, 0 is the root node and is the parent of both 5 and 10. 5 is the parent node of both 3 and 7. 3 and 7 are siblings as they both have 5 as a parent and are at the same depth. 3, 7 and 10 are all leaf nodes as they don't have any children of their own.
With all of that out of the way lets go on to the types of binary trees. Binary trees come into five distinct flavors:

Pathological binary tree: A pathological Binary tree is a binary tree where each node has only one child. These kinds of binary trees act similar to linked lists. Example below:
Full binary tree: These kinds of binary trees can have from 0 to 2 children. See example below, it the same diagram as above in the explainer on the elements of a binary tree:
Complete binary tree: This is a type of binary tree where all nodes are filled on all depths and is skewed to the left, except for the last depth.
Balanced binary tree: This kind of binary is where the left and right side of the binary tree differ in depth by at most one.
Perfect binary tree: This is a binary tree where all nodes have two children and all leaves are at the same depth.

Super! So we have the different types of binary trees defined, next on to how this kind of data structure is constructed in code. to clarify we will be building a full binary tree, as that is the most common type of binary tree that I have experienced.

To start we will define what a node looks like.

    internal class Node(Node? parent, int value)
    {
        readonly int value = value;
        readonly Node? parent = parent;
        Node? leftChild;
        Node? rightChild;

        public int Value { get { return value; } }
        public Node? Parent { get { return parent; } }
        public Node? LeftChild { get { return leftChild; } }
        public Node? RightChild { get { return rightChild; } }

        // This a recursive method to add the new nodes to our tree
        public void Add(int value)
        {
            /*If the new value is < our value either add a child node
             or check the left child node*/
            if (value < this.value)
            {
                if (leftChild != null)
                {
                    leftChild.Add(value);
                }
                else
                {
                    leftChild = new Node(this, value);
                    Console.WriteLine($"parent:{this.value}, left child: {value}");
                }

                return;
            }

            /*If the new value is >= our value either add a child node
                or check the right child node*/
            if (rightChild != null)
            {
                rightChild.Add(value);
            }
            else
            {
                rightChild = new Node(this, value);
                Console.WriteLine($"parent:{this.value}, right child: {value}");
            }
        }
    }

And that's it... that's our tree...
As you can see from above, for a node we can create one with no parent to be our root node from here when we add to the root node it checks it's children if they are null it creates on based on the value to be added or it will call add on one of it's child nodes. The adding of the child nodes is a O(N) complexity where N is the number of nodes in the binary tree:
I've written the below to test this out:

List<int> values = new() { 3, 7, 8, 6, 10 };
Node rootNode = new(null, 5);

Console.WriteLine($"Root node value: {rootNode.Value}");
Console.WriteLine(string.Join(',', values));

foreach(int i in values)
{
    rootNode.Add(i);
}

Console.ReadKey();

The output:

Now there are a few bits we can add to the node structure to help to query it. First we will add a new enum to tell us if the node is a root, child or leaf node:

    internal enum NodeTypes : byte
    {
        Root = 0,
        Child = 1,
        Leaf = 2
    }

Added to the node class:

        public NodeTypes Type
        {
            get
            {
                return parent == null ? NodeTypes.Root :
                    leftChild != null || rightChild != null ?
                        NodeTypes.Child : NodeTypes.Leaf;
            }
        }

These types will help us query the nodes and let us know if we have reached the end of any branch of a tree.
Next we will write a method to let use know if a node is a sibling of another node:

        public bool IsSibling(Node node)
        {
            if (node.Type == NodeTypes.Root)
                return false;

            return node.parent == this.parent;
        }

Finally the below method will let us search for a value in the binary tree and return the node that contains the value. The search is a O(N) complexity where N is the number of nodes in the binary tree:

        /* This is another recursive method where each node is checked until either
           the tree has been searched or the value has been found */
        public Node? FindNodeWithValue(int value)
        {
            if (this.value == value)
            {
                return this;
            }
            else if (Type != NodeTypes.Leaf)
            {
                Node? foundNode = null;

                if (leftChild != null)
                {
                    foundNode = leftChild.FindNodeWithValue(value);
                }

                if (foundNode == null)
                {
                    if (rightChild != null)
                    {
                        return rightChild.FindNodeWithValue(value);
                    }
                }

                return foundNode;
            }

            return null;
        }

Essentially every node is searched until the value is found, now there are other methods for searching a binary tree including depth first search.

Now, there are other kinds of tree not just binary trees. A binary tree is just a tree structure with up to two children per node. You also have oct trees (8 children per node) that are used for partitioning 3D space for the likes of defining the level of detail on terrain in a video game or quad trees (4 children per node) that are used for image compression.

Stacks

So stacks, this is a generic collection that works on the principal last in first out. This means that the last item in is the first item in the collection. Also they don't support an indexer.
They can be defined as follows:

Stack<int> stack = new Stack<int>();

Items can be added to a stack by pushing them onto it:

Stack<int> stack = new Stack<int>();

stack.Push(5);
stack.Push(6);
stack.Push(7);

Console.WriteLine($"{string.Join(',', stack)}");

To view the top item in the stack you can use peek. the stack:

Stack<int> stack = new Stack<int>();

stack.Push(5);
stack.Push(6);
stack.Push(7);
stack.Push(8);
stack.Push(9);
stack.Push(10);

Console.WriteLine($"Stack: {string.Join(',', stack)}");

int peeked = stack.Peek();
Console.WriteLine($"Item: {peeked}");

You can remove items from the stack by using stack pop. Using stack pop will only remove the first item in the collection and return it.

Stack<int> stack = new Stack<int>();

stack.Push(5);
stack.Push(6);
stack.Push(7);

Console.WriteLine($"Stack: {string.Join(',', stack)}");

int removed = stack.Pop();
Console.WriteLine($"Item removed: {removed}");

Console.WriteLine($"Stack: {string.Join(',', stack)}");

You can also remove all items in the stack with clear.

The main use cases for stacks is when you need to make use of the last item added to the collection. So it could be useful in scenarios where you need to manage the history of a item such as browsing history or you need to keep an audit log or track of what had happened to a particular object to get to the state that it is in now.

Queues

Queues work on the principal of first in first out and also don't support an indexer.
Queues can be instantiated as follows:

Queue<string> names = new Queue<string> ();

Items are added to the queue using enqueue. This will add items to the end of the collection similar to a list.

Queue<string> names = new Queue<string> ();

names.Enqueue("Tom");
names.Enqueue("Timmy");
names.Enqueue("Thomas");
names.Enqueue("Tommity");
names.Enqueue("Tommiton");
names.Enqueue("Tombert");
names.Enqueue("Tomantha");

Console.WriteLine($"{string.Join(',', names)}");

Like with a stack you can view the item in the front of the collection without removing it using peek.

string peeked = names.Peek();
Console.WriteLine($"Peeked: {peeked}");

Similar again to stacks you can remove the first item in the collection with dequeue:

Queue<string> names = new Queue<string> ();

names.Enqueue("Tom");
names.Enqueue("Timmy");
names.Enqueue("Thomas");
names.Enqueue("Tommity");
names.Enqueue("Tommiton");
names.Enqueue("Tombert");
names.Enqueue("Tomantha");

Console.WriteLine($"{string.Join(',', names)}");

string dequeued = names.Dequeue();
Console.WriteLine($"dequeued: {dequeued}");
Console.WriteLine($"{string.Join(',', names)}");

You can also clear a queue by calling the clear method.

The main use case for queues is for well managing messages such as in message queues in a Publish Subscribe architecture. In it queues are use to store messages to mitigate system interrupts so that when the consuming resource is free to take on a new task the queue is dequeued and the first item is sent to the consumer.

Data tables

Data tables as the name suggest is a code representation of a table with columns and rows that you can use to define the relationship between the different data elements and query the data in memory.

As you'd imagine Data Tables are formed from several different elements:

Data Table: defines the table in memory
Data Row: defines a row in the Data Table
Data Column: defines a column in the Data Table
Data Relationship: defines the relationship between two columns in Data Tables (like a foreign key to table)
Data Set: defines the collection of the data objects (tables, relations, etc)

Super! Lets start with an example of a simple table that some customer information in it:

using System.Data;

const string CUSTOMER_TABLE_NAME = "Customers";

const string ID_COLUMN_NAME = "customer_id";
const string CUSTOMER_FIRST_NAME_COLUMN_NAME = "customer_first_name";
const string CUSTOMER_LAST_NAME_COLUMN_NAME = "customer_last_name";

DataSet ds = new DataSet();

CreateCustomerTable(ds);

static void CreateCustomerTable(DataSet ds)
{
    //Creates our data table
    DataTable dataTable = new DataTable(CUSTOMER_TABLE_NAME);

    //Defines the columns in the data table
    dataTable.Columns.Add(ID_COLUMN_NAME, typeof(int));
    dataTable.Columns.Add(CUSTOMER_FIRST_NAME_COLUMN_NAME, typeof(string));
    dataTable.Columns.Add(CUSTOMER_LAST_NAME_COLUMN_NAME, typeof(string));

    addDataToCustomerTable(1, "John", "Tomson", dataTable);
    addDataToCustomerTable(2, "Tom", "Johnson", dataTable);

    //Adding our table to the data set
    ds.Tables.Add(dataTable);
}

//The method is used to add a new row to the table
static void addDataToCustomerTable(int id, string customerFirstName, string customerLastName, DataTable table)
{
    DataRow row = table.NewRow();

    row[ID_COLUMN_NAME] = id;
    row[CUSTOMER_FIRST_NAME_COLUMN_NAME] = customerFirstName;
    row[CUSTOMER_LAST_NAME_COLUMN_NAME] = customerLastName;

    table.Rows.Add(row);
}

That is the basics of creating a new data table. However, next we'll add two new tables and define a relationship between them. In the example I'll be defining an Item table and an Online Order table.

using System.Data;

const string CUSTOMER_TABLE_NAME = "Customers";
const string ONLINE_ORDER_TABLE_NAME = "OnlineOrders";
const string ITEM_TABLE_NAME = "Items";

const string CUSTOMER_ID_COLUMN_NAME = "customer_id";
const string CUSTOMER_FIRST_NAME_COLUMN_NAME = "customer_first_name";
const string CUSTOMER_LAST_NAME_COLUMN_NAME = "customer_last_name";

const string ORDER_ID_COLUMN_NAME = "online_order_id";
const string QUANTITY_COLUMN_NAME = "quantity";

const string ITEM_ID_COLUMN_NAME = "item_id";

const string ITEM_DESCRIPTION_COLUMN_NAME = "description";

DataSet ds = new DataSet();

CreateCustomerTable(ds);
CreateOnlineOrderTable(ds);
CreateItemTable(ds);

CreateTableRelationships(ds);

void CreateTableRelationships(DataSet ds)
{
    ds.Tables[ONLINE_ORDER_TABLE_NAME].ParentRelations.Add(
        ds.Tables[CUSTOMER_TABLE_NAME].Columns[CUSTOMER_ID_COLUMN_NAME],
        ds.Tables[ONLINE_ORDER_TABLE_NAME].Columns[CUSTOMER_ID_COLUMN_NAME]);

    ds.Tables[ONLINE_ORDER_TABLE_NAME].ParentRelations.Add(
        ds.Tables[ITEM_TABLE_NAME].Columns[ITEM_ID_COLUMN_NAME],
        ds.Tables[ONLINE_ORDER_TABLE_NAME].Columns[ITEM_ID_COLUMN_NAME]);
}

static void CreateOnlineOrderTable(DataSet ds)
{
    //Creates our data table
    DataTable dataTable = new DataTable(ONLINE_ORDER_TABLE_NAME);

    //Defines the columns in the data table
    DataColumn idColumn = new DataColumn(ORDER_ID_COLUMN_NAME, typeof(int));
    idColumn.Unique = true;
    idColumn.AutoIncrement = true;

    dataTable.Columns.Add(idColumn);
    dataTable.Columns.Add(CUSTOMER_ID_COLUMN_NAME, typeof(int));
    dataTable.Columns.Add(ITEM_ID_COLUMN_NAME, typeof(int));
    dataTable.Columns.Add(QUANTITY_COLUMN_NAME, typeof(string));

    addDataToTable(CreateDataForOrderTable(1, 1, 15), dataTable);

    //Adding our table to the data set
    ds.Tables.Add(dataTable);
}

static void CreateCustomerTable(DataSet ds)
{
    //Creates our data table
    DataTable dataTable = new DataTable(CUSTOMER_TABLE_NAME);

    //Defines the columns in the data table
    DataColumn idColumn = new DataColumn(CUSTOMER_ID_COLUMN_NAME, typeof(int));
    idColumn.Unique = true;
    idColumn.AutoIncrement = true;

    dataTable.Columns.Add(idColumn);
    dataTable.Columns.Add(CUSTOMER_FIRST_NAME_COLUMN_NAME, typeof(string));
    dataTable.Columns.Add(CUSTOMER_LAST_NAME_COLUMN_NAME, typeof(string));

    addDataToTable(CreateDataForCustomerTable("John", "Tomson"), dataTable);
    addDataToTable(CreateDataForCustomerTable("Tom", "Johnson"), dataTable);

    //Adding our table to the data set
    ds.Tables.Add(dataTable);
}

static void CreateItemTable(DataSet ds)
{
    //Creates our data table
    DataTable dataTable = new DataTable(ITEM_TABLE_NAME);

    //Defines the columns in the data table
    DataColumn idColumn = new DataColumn(ITEM_ID_COLUMN_NAME, typeof(int));
    idColumn.Unique = true;
    idColumn.AutoIncrement = true;

    dataTable.Columns.Add(idColumn);
    dataTable.Columns.Add(ITEM_DESCRIPTION_COLUMN_NAME, typeof(string));

    addDataToTable(CreateDataForItemTable("Apple"), dataTable);
    addDataToTable(CreateDataForItemTable("Orange"), dataTable);

    //Adding our table to the data set
    ds.Tables.Add(dataTable);
}

static Dictionary<string, object> CreateDataForCustomerTable(string firstName, string lastName)
{
    return new Dictionary<string, object>
    {
        { CUSTOMER_FIRST_NAME_COLUMN_NAME, firstName },
        { CUSTOMER_LAST_NAME_COLUMN_NAME, lastName }
    };
}

static Dictionary<string, object> CreateDataForOrderTable(int customerId, int itemId, int quantity)
{
    return new Dictionary<string, object>
    {
        { CUSTOMER_ID_COLUMN_NAME, customerId },
        { ITEM_ID_COLUMN_NAME, itemId },
        { QUANTITY_COLUMN_NAME, quantity }
    };
}

static Dictionary<string, object> CreateDataForItemTable(string description)
{
    return new Dictionary<string, object>
    {
        { ITEM_DESCRIPTION_COLUMN_NAME, description }
    };
}

//The method is used to add a new row to the table
static void addDataToTable(Dictionary<string, object> data, DataTable table)
{
    DataRow row = table.NewRow();

    foreach (KeyValuePair<string, object> kvp in data)
    {
        row[kvp.Key] = kvp.Value;
    }

    table.Rows.Add(row);
}

In the code sample above both the Item table and Customer table can join the Online Order table see entity relationship diagram below:

Nice one!
With the relationships defined if we try to add something to our online order table with out it being in either customers or our item times first we should get errors as that data doesn't exist in the joining tables.

The below should throw an exception when we try to add a new row into the online order table as there is no primary key in the customer table with an id of 3, as such we get a foreign key constraint issue.

That is the basics of data tables in C#. The DataTable object in C# is actually quite robust. There are many other features of it such as data binding and setting grid views for displaying the table. Overall if you need to have some control over data as you would have in a database(to lesser extent of course) the DataTable is a good way to do this in C# as it mimics some of the same features that a table in the database has.

Hashset

A Hashset is an collection that prevents duplicates by ensuring each element in the collection is unique. Kind of like keys in a dictionary.

As Hashset is instantiated in much the same way as a list or other generic collection:

HashSet<int> hashset = [1,2,3,4,5];

When I try to add a new element to the Hashset that already exist if doesn't add a new entry into the collection:

HashSet<int> hashset = [1,2,3,4,5];

Console.WriteLine($"Hashset elements: {string.Join(',', hashset)}");

int toAdd = 1;

Console.WriteLine($"Attempting to add {toAdd}");
hashset.Add(toAdd);

Console.WriteLine($"Hashset elements: {string.Join(',', hashset)}");

The other thing I'd mention is that with Hashset they do have most of the same functionality as other generic collections such as adding and removing elements. The main use case for them is to ensure you aren't storing duplicate elements in a collection

I topic isn't something I see much of online and to be honest before exploring this topic and seeing the other data structures I might not have considered using most of them. In my day job a lot of the time when I need a go to collection it's either dictionaries, lists or arrays. Rarely had I seen Hashsets or DataTables. Maybe in some older applications but, I can't remember the last time I needed to implement a binary tree.

Thanks for reading my post.
Feel free to leave a like if you learned something new or found out something interesting and I'll see you in the next post >>

Common Sorting Algorithms in C# - From My Experience

Albert Bennett — Sat, 02 Mar 2024 19:45:21 +0000

Hi folks, and welcome to my blog.
If you’ve learned something new or found something useful feel free to leave a like, or share the post..

Recently I’ve been delving into the more academic side of software engineering. In my day to day job I’ve been finding myself mostly just going through the motions re-using the same tools day in day out or following the same patterns and strategies that had been in place for years and not really fully understanding the why behind them, well not in much depth anyways. I knew the basics and how it was implemented and why it was needed but not the underlying structures or how efficient they are or if there were other ways of doing the same things that could be used to improve the underlying systems.

As such I’ve decided to start reading and studying up on some of the core concepts that form the backbone of some of these systems and up first is sorting algorithms.
In this post I’m going to go over the most common sorting algorithms that I had found being used and their complexity.

Bubble Sort

Bubble sort is probably the simplest of the sorting algorithm (that I know of).
In it you essentially check each element in the array against each other and swap them if needed. Eventually with enough swaps the array will be arranged in order. See below for an example:

Here is one way to implement it, in C#:

        public static int[] BubbleSort(int[] array, bool ascending)
        {
            //Returning early as you need at least two numbers to swap
            if (array.Length <= 1)
            {
                return array;
            }

            //The maximum number of swaps possible
            int maxSwapCount = array.Length - 1;

            //Iterate through all elements in the array
            for (int i = 0; i < maxSwapCount; i++)
            {
                /*Iterates through the newly swapped array, with each
                  iteration at least 1 element will be in the correct place*/
                for (int j = 0; j < maxSwapCount - i; j++)
                {
                    int currentNumber = array[j];

                    int nextIndex = j + 1;
                    int nextNumber = array[nextIndex];

                    if ((ascending && currentNumber > nextNumber) || currentNumber < nextNumber)
                    {
                        array[j] = nextNumber;
                        array[nextIndex] = currentNumber;

                        Console.WriteLine(string.Join(',', array));
                    }

                }
            }

            return array;
        }

The complexity of the bubble sort is O(N²) as the array needs to be cycled through by two separate for loops each as such it has a quadratic complexity and that grows exponentially.

Bubble sorts are fairly inefficient and really there aren’t too many use cases to use it over other sorting algorithms. Now there are a few tricks that can be implemented to speed up a bubble sort such as reducing the number of iterations under the assumption that at least one number will be in the correct position per inner iteration. I’ve seen them mostly being used on smaller datasets that need to be sorted in either descending or ascending order. Things like the responses from an API where the items in an array need to be sorted by time or ID.

Quick Sort

The quick sort is a recursive algorithm. In it a pivot value is determined; this is used to determine how to partition or split the array of elements into values that are greater than the pivot or values that are less than the pivot. From here the process continues recursively on the sub arrays until the array has been sorted. Below is an example of the output and swaps made in a quicksort:

Below is an example of how a Quick sort can be implemented:

        public static int[] QuickSort(int[] array)
        {
            if (array.Length <= 1)
            {
                return array;
            }

            return QuickSort(array, 0, array.Length - 1);
        }

        static int[] QuickSort(int[] array, int leftIndex, int rightIndex)
        {
            int pivot = array[leftIndex];
            int currentLeftIndex = leftIndex;
            int currentRightIndex = rightIndex;

            //While the two sorting indices don't meet, this is to localise the sorting to a given range and essential define our partition for the Quick Sort
            while (currentLeftIndex <= currentRightIndex)
            {
                /* Comparing if the left value is less than the pivot value
                   then moving the left index along until the left value >= the pivot*/
                while (array[currentLeftIndex] < pivot)
                {
                    currentLeftIndex++;
                }

                /* Comparing if the right value is greater than the pivot value
                 then moving the right index along until it is <= the pivot value */
                while (array[currentRightIndex] > pivot)
                {
                    currentRightIndex--;
                }

                //Swap the values when the number on the left is less then the number on the right
                if (currentLeftIndex <= currentRightIndex)
                {
                    Console.WriteLine($"Swapping {array[currentLeftIndex]} at {currentLeftIndex} with {array[currentRightIndex]} at {currentRightIndex}");

                    (array[currentRightIndex], array[currentLeftIndex]) = (array[currentLeftIndex], array[currentRightIndex]);
                    currentLeftIndex++;
                    currentRightIndex--;
                }
            }

            /*If the starting index on the left side is less than the current right index then sort again. This is done using our new indices that will act as a partition of the array and localise sorting to that range.
              as there are more swaps to be made before the array has been sorted */
            if (leftIndex < currentRightIndex)
            {
                QuickSort(array, leftIndex, currentRightIndex);
            }

            /*If the starting index on the right side is greater than the current left index then sort again. This is done using our new indices that will act as a partition of the array and localise sorting to that range.
              as there are more swaps to be made before the array has been sorted */
            if (rightIndex > currentLeftIndex)
            {
                QuickSort(array, currentLeftIndex, rightIndex);
            }

            return array;
        }

In terms of complexity the Quick Sort is in the worst case O(N²). However on average the performance of the Quick Sort is closer to O(N log N) where N is the total number of partitions.

Merge Sort

A merge sort is a recursive method to sort an array. It works by subdividing the original array over and over again until each subsequent array has only one element in it:

From here the arrays are merged back together sorted by swapping the elements in the joining arrays in a specified sorting direction(ascending or descending). In the example below the numbers are sorted by ascending order.

Codewise how this can be done is by first splitting the array into two parts representing the left side and right side until the array contains only one element in it. Next the arrays are rejoined back together again sorted using the sorting algorithm.

        public static int[] MergeSort(int[] array, bool ascending)
        {
            if (array.Length == 0)
            {
                return [];
            }

            //Determine the middle index of the array
            int middle = array.Length / 2;

            int[] leftArray = array.Take(middle).ToArray();
            int[] rightArray = array.Skip(middle).Take(array.Length - middle).ToArray();

            if (leftArray.Length > 1)
            {
                MergeSort(leftArray, ascending);
            }

            if(rightArray.Length > 1) 
            {
                MergeSort(rightArray, ascending);
            }

            MergeArray(leftArray, rightArray, array, ascending);

            return array;
        }

The merge sort algorithm is fairly straightforward. Essentially, compare the left side and right side and then if left > right (for ascending ordering) the sorted array at the current index equals the number from the left array else it contains the number from the right array. Then if not all elements from either array had been sorted they are added to the end of the sorted array first using the left elements then the right elements as this was how we were comparing them (left to right).

 static void MergeArray(int[] leftArray, int[] rightArray, int[] sortedArray, bool ascending)
 {
     int leftIndex = 0;
     int rightIndex = 0;
     int sortedIndex = 0;

     while (leftIndex < leftArray.Length && rightIndex < rightArray.Length)
     {
         if (ascending)
         {
             /* If sorting by ascending order we need to set the number in the sorted at
                the current index to the lowest value between the two arrays */
             sortedArray[sortedIndex++] = leftArray[leftIndex] <= rightArray[rightIndex] ?
                 leftArray[leftIndex++] :
                 rightArray[rightIndex++];
         }
         else
         {
             /* If sorting by descending order we need to set the number in the sorted at
                 the current index to the highest value between the two arrays */
             sortedArray[sortedIndex++] = leftArray[leftIndex] >= rightArray[rightIndex] ?
                 leftArray[leftIndex++] :
                 rightArray[rightIndex++];
         }
     }

     //If there are any missing elements in the swapped arrays then add them to the sorted array
     while (leftIndex < leftArray.Length)
     {
         sortedArray[sortedIndex++] = leftArray[leftIndex++];
     }

     while (rightIndex < rightArray.Length)
     {
         sortedArray[sortedIndex++] = rightArray[rightIndex++];
     }

     Console.WriteLine($"{string.Join(',', leftArray)} + {string.Join(',', rightArray)} = {string.Join(',', sortedArray)}");
 }

To test the algorithm I’ve implement the following in the program.cs file:

int[] numbers = new int[] { 10, 5, 16, 99, 87, 45, 20, 35,17 };

Console.WriteLine($"Input: {string.Join(',', numbers)}");

int[] sortedArray = MergeSortAlgorithim.MergeSort(numbers, true);

Console.WriteLine($"Output: {string.Join(',', sortedArray)}");
Console.ReadKey();

Console log:

In terms of complexity the merge sort algorithm is one of the more efficient ones that we had explored in the blog post.

If you want the Big O notation of it, it is O(N log N).

O represents the complexity of operations. Where N is the length of the array as we need to search the array to sort it. Log N represents the merge process per split array as we need to compare each element in each of the split arrays in the MergeArray method with each other.

Speed Comparisons

Out of pure curiosity, I decided to create a little speed test using the Stopwatch for the algorithms and code written above with the inclusion of the most common method I have seen (in recent times) for sorting an array of integers and that is using Linq Orderby.

Here is the code I wrote to test the performance of the algorithms in the post:

 internal class SpeedTest
 {
     const string LINQ_ORDERBY = "Linq OrderBy";
     const string BUBBLE_SORT = "Bubble Sort";
     const string QUICKSORT = "Quicksort";
     const string MERGESORT = "Mergesort";

     public static void TestAlgorithims(int minElements, int maxElements)
     {
         int[] testArray = generateRandomArray(minElements, maxElements);

         //Console.WriteLine($"Length of Array: {testArray.Length}");

         Stopwatch stopWatch = new();

         testLinqSort(ref testArray, stopWatch);
         testBubbleSort(ref testArray, stopWatch);
         testQuickSort(ref testArray, stopWatch);
         testMergeSort(ref testArray, stopWatch);
     }

     private static void testMergeSort(ref int[] testArray, Stopwatch stopWatch)
     {
         stopWatch.Start();
         MergeSortAlgorithim.MergeSort(testArray, false);
         stopWatch.Stop();

         writeResults(stopWatch, MERGESORT, testArray.Length);
         stopWatch.Reset();
     }

     private static void testQuickSort(ref int[] testArray, Stopwatch stopWatch)
     {
         stopWatch.Start();
         QuickSortAlgorithim.QuickSort(testArray);
         stopWatch.Stop();

         writeResults(stopWatch, QUICKSORT, testArray.Length);
         stopWatch.Reset();
     }

     private static void testBubbleSort(ref int[] testArray, Stopwatch stopWatch)
     {
         stopWatch.Start();
         BubbleSortAlgorithim.BubbleSort(testArray, false);
         stopWatch.Stop();

         writeResults(stopWatch, BUBBLE_SORT, testArray.Length);
         stopWatch.Reset();
     }

     private static void testLinqSort(ref int[] testArray, Stopwatch stopWatch)
     {
         stopWatch.Start();
         testArray.OrderBy(x => x);
         stopWatch.Stop();

         writeResults(stopWatch, LINQ_ORDERBY, testArray.Length);
         stopWatch.Reset();
     }

     private static void writeResults(Stopwatch stopWatch, string sortingAlgorithim, int arrayLength)
     {
         TimeSpan ts = stopWatch.Elapsed;

         //string elapsedTime = String.Format($"Total Nanoseconds: {ts.TotalNanoseconds}");

         // Console.WriteLine($"Execution Time for {sortingAlgorithim} {elapsedTime}");
         Console.WriteLine($"{sortingAlgorithim} : {arrayLength} : {ts.TotalNanoseconds}");
     }

     private static int[] generateRandomArray(int minElements, int maxElements)
     {
         Random r = new();

         int count = r.Next(minElements, maxElements);
         int[] testArray = new int[count];
         int currentIndex = 0;

         while (currentIndex < count)
         {
             testArray[currentIndex] = r.Next(0, maxElements);
             currentIndex++;
         }

         return testArray;
     }
 }

Here is the output from the testing (the green columns are the most efficient for that number of elements). The time is measured in nanoseconds (one nanosecond = one billionth of a second). You’ll notice from the table below that there are two entries for 4 elements for each algorithm, this is because during testing I noticed that the first test for each algorithm would always have inflated times but, seem to be corrected on the second set of tests:

As you can see from the testing, Linq Orderby is much more efficient at larger datasets then smaller ones afterwards it’s the bubble sort for what seems to be datasets of less than 100. Viewing the graph overall on average Linq does come on top followed by, mergesort, quicksort and lastly bubble sort.

In conclusion, use Linq Orderby. (・‿・)ノ⌒*:･ﾟ✧

Viewing the graph above, all other algorithms seem to be outperformed by Linq Orderby after the number of elements in the array grew beyond 100 elements.
Now is any of the performance testing a definitive guide on what to use when, no, I don’t think so but, it does seem to indicate what could work well under specific conditions.

If you’ve learned something new or found something useful feel free to leave a like, or share the post and I will see you in the next post >>

Sources:

Displaying charts in an adaptive card

Albert Bennett — Sun, 24 Jul 2022 20:27:49 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)

Today is going to be another quick tutorial on how you can display a chart on an adaptive card for a chat bot. It's a fairly straight forward, I had wanted to do this for my series on creating a Twitter sentiment analysis chatbot but couldn't find anything on it until now.
You can see the code in the repo linked below:

https://github.com/Albert-Bennett/TwitterSentimentAnalysisChatBot

Now on to the coding.

Introduction

Like with most things programming related there are many ways of achieving this but, they all involve generating an image of a chart and using that in the AdaptiveImage element to display the image. The easiest way that I have found is to use a third party API to generate the chart and send it back via a get request.
Here is a link to the documents for the API: https://documentation.image-charts.com/
And here is the URL that we can use to generate the graph:
https://image-charts.com/chart?cht=p3&chs=300x300&chd=t:{0}&chl={1}&chan
The API is exactly what I need for it more-over it's free :).
Below is a snipping of the code that I used to add a chart to the adaptive card for the Twitter sentiment analysis bot.

        AdaptiveImage GetStatisticsChart(Dictionary<string, int> tweetSentimentAnalysis, int numberOfTweetsFound)
        {
            return new AdaptiveImage
            {
                Size = AdaptiveImageSize.Large,
                Url = GetPieChartUrl(tweetSentimentAnalysis, numberOfTweetsFound)
            };
        }

        Uri GetPieChartUrl(Dictionary<string, int> tweetSentimentAnalysis, int numberOfTweetsFound)
        {
            string seriesNames = string.Join('|', tweetSentimentAnalysis.Keys);
            string percentages = string.Empty;

            foreach (var val in tweetSentimentAnalysis.Values)
            {
                percentages += $"{Math.Round(((float)val * 100) / numberOfTweetsFound)},";
            }

            percentages = percentages.Remove(percentages.Length - 1);

            return new Uri(string.Format(baseUrl, percentages, seriesNames));
        }

The important part of the above is the that the URL of the AdaptiveImage is set to the URI of the API endpoint with the specifics to generate your graph. The query parameters that I interpolated were the series and what percentage of the pie chart was covered by it. In my example and as shown in the image below there are two series separated in the URL by a '|' each with a value of 50 separated by ','. According to the docs on the API the colors of each series can be defined as well but, I chose not to as it is a little more complex than just setting the hex codes for the series.

The other thing to note and you can see that when you go through the docs is that you can also change the type of chart to generate in my example I chose to generate a pie chart as it presented the data better.

Sentiment Analysis for Tweets in a Chat Bot - Part 3 The Chat Bot

Albert Bennett — Fri, 22 Jul 2022 07:56:00 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)

Hi and welcome to the third and possibly final part on my series on how to create a chatbot to perform sentiment analysis on tweets searched for.

In this post I'll go through how you can create a chatbot to interact with the function app that we developed in the pervious part. You can check out the repos for this project by clicking on the links below:
https://github.com/Albert-Bennett/TwitterSentimentAnalysisFunctionApp
https://github.com/Albert-Bennett/TwitterSentimentAnalysisChatBot

Setting up our project
We are starting simple enough by creating an echo bot using the template project to get these make sure to install them:

You can find a link to package here: https://marketplace.visualstudio.com/items?itemName=BotBuilder.botbuilderv4
With the template installed we can start by creating an echo bot. It is one of the most barebones templates for bots, all it does is repeat back your input.
To test the bot what you will need to do is to run the Bot Framework Emulator and run your chat bot project. With the chat bot up and running you should see something like this:

From here just copy the local host URL appended with /api/messages and paste into the bot framework emulator and you should see something like this when it has been successfully connected:

And that's our basic bot setup and testing so we know everything is working correctly at this point.

Connecting to our function app
To start we need to figure out what we need to send on to our app. There was nothing complex with it, everything that it needs to search can be sent in the query parameters. So our service needs to send a get request to a URL similar to this: http://localhost:7071/api/SentimentAnalysisFN?hashtags=[search terms]&max_tweets=[max number of tweets to find]
Super so with this the only method in our service should look like this:

        public async Task<TwitterSentimentResponse> GetSentimentAnalysisForTweets(string searchTerm, int? maxResults = 10)
        {
            string url = $"{baseUrl}?hashtags={searchTerm}&max_tweets={maxResults}";

            using (var client = _httpClientFactory.CreateClient())
            {
                var response = await client.GetAsync(url);

                if (response.IsSuccessStatusCode)
                {
                    using var contentStream = await response.Content.ReadAsStreamAsync();

                    return await JsonSerializer.DeserializeAsync<TwitterSentimentResponse>(contentStream);
                }
            }

            return null;
        }

In case your wondering how I modeled the data for the response I copied it from the function app 😑. Of course I ran into an issue when deserializing the response from the function app. I thought serializing an enum in the response would return an int. Instead it returns a string.

The adaptive card
This part is a bit complex but, more from a structure and modeling point of view. To start off we need to decide how to present the data to the user. Here is a quick sketch of how I want the data to be shown:

I think it looks neat and with that we can start developing our solution.
Adaptive cards are constructed in a hierarchical structure meaning that we had a root component and add children to it of varying types like image cards and text input.
With that in mind here is the code to structure the adaptive card kinda like the mock image:

        public static Attachment GetAnalysisCard(TwitterSentimentResponse data)
        {
            var cardBodyElements = new List<AdaptiveElement>
            {
                GetStatisticsElement(data.tweetSentimentAnalysis, data.numberOfTweetsFound)
            };

            cardBodyElements.AddRange(GetPopularTweetCards(data.mostPopularTweets));

            AdaptiveSchemaVersion defaultSchema = new(1, 0);

            AdaptiveCard card = new(defaultSchema)
            {
                Body = cardBodyElements
            };

            return CreateAdaptiveCardAttachment(card.ToJson());
        }

        static IEnumerable<AdaptiveElement> GetPopularTweetCards(TweetData[] mostPopularTweets)
        {
            List<AdaptiveElement> result = new List<AdaptiveElement>();

            foreach (TweetData tweet in mostPopularTweets)
            {
                result.AddRange(GetPopularTweetCard(tweet));
            }

            return result;
        }

        private static IEnumerable<AdaptiveElement> GetPopularTweetCard(TweetData tweet)
        {
            return new List<AdaptiveElement>
            {
                new AdaptiveTextBlock
                {
                        Size = AdaptiveTextSize.Medium,
                        Weight = AdaptiveTextWeight.Bolder,
                        Text = tweet.text,
                        Wrap = true
                },
                new AdaptiveColumnSet
                {
                    Columns = new List<AdaptiveColumn>
                    {
                        new AdaptiveColumn
                        {
                            Items = new List<AdaptiveElement>
                            {
                                new AdaptiveTextBlock
                                {
                                    Spacing = AdaptiveSpacing.None,
                                    IsSubtle = true,
                                    Wrap = true,
                                    Text = $"Likes: {tweet.public_metrics.like_count}"
                                },
                                new AdaptiveTextBlock
                                {
                                    Spacing = AdaptiveSpacing.None,
                                    IsSubtle = true,
                                    Wrap = true,
                                    Text = $"Retweets: {tweet.public_metrics.retweet_count}"
                                }
                            }
                        },
                        new AdaptiveColumn
                        {
                            Items = new List<AdaptiveElement>
                            {
                                new AdaptiveTextBlock
                                {
                                    Spacing = AdaptiveSpacing.None,
                                    IsSubtle = true,
                                    Wrap = true,
                                    Text = $"Replies: {tweet.public_metrics.reply_count}"
                                },
                                new AdaptiveTextBlock
                                {
                                    Spacing = AdaptiveSpacing.None,
                                    IsSubtle = true,
                                    Wrap = true,
                                    Text = $"Quote count: {tweet.public_metrics.quote_count}"
                                }
                            }
                        }
                    }
                }
            };
        }

        static AdaptiveColumnSet GetStatisticsElement(Dictionary<string, int> tweetSentimentAnalysis, int numberOfTweetsFound)
        {
            return new AdaptiveColumnSet
            {
                Columns = new List<AdaptiveColumn>
                {
                    new AdaptiveColumn
                    {
                        Items = GetCardStatistics(tweetSentimentAnalysis, numberOfTweetsFound),
                        Width = "stretch"
                    },
                    new AdaptiveColumn
                    {
                        Items = new List<AdaptiveElement>
                        {
                            new AdaptiveTextBlock
                            {
                                Spacing = AdaptiveSpacing.None,
                                IsSubtle = true,
                                Wrap = true,
                                Text = $"Number of Tweets found: {numberOfTweetsFound}"
                            }
                        }
                    }
                }
            };
        }

        static List<AdaptiveElement> GetCardStatistics(Dictionary<string, int> data, int numberOfTweetsFound)
        {
            List<AdaptiveElement> result = new List<AdaptiveElement>();

            foreach(string sentiment in data.Keys)
            {
                result.Add(new AdaptiveTextBlock
                {
                    Spacing = AdaptiveSpacing.None,
                    IsSubtle = true,
                    Wrap = true,
                    Text = $"{sentiment}: {(data[sentiment] * 100) / numberOfTweetsFound }%"
                });
            }

            return result; 
        }

        static Attachment CreateAdaptiveCardAttachment(string jsonData)
        {
            var adaptiveCardAttachment = new Attachment()
            {
                ContentType = "application/vnd.microsoft.card.adaptive",
                Content = JsonConvert.DeserializeObject(jsonData),
            };

            return adaptiveCardAttachment;
        }

Although there is a lot of code above it all just boils down to structuring the card in a way that I think presents that data in an nice way to the user. It's just a lot of... for each statements and ifs to process the response from the sentiment analysis. See snapshot below for the output in the bot:

The wiring up
The last part is just wiring up and sequencing the calls to the various services to make the bot... work. Here is how the main bot should look like.

        protected override async Task OnMessageActivityAsync(ITurnContext<IMessageActivity> turnContext, CancellationToken cancellationToken)
        {
            var userInput = turnContext.Activity.Text;
            var response = await _twitterAnalysisAppService.GetSentimentAnalysisForTweets(userInput, 2);

            if (response == null)
            {
                string errorText = "There was an issue contacting the function app";
                await turnContext.SendActivityAsync(MessageFactory.Text(errorText, errorText), cancellationToken);
            }
            else
            {
                var adaptiveCard = AdaptiveCardConstructor.GetAnalysisCard(response);
                await turnContext.SendActivityAsync(MessageFactory.Attachment(adaptiveCard));
            }
        }

All it's doing is getting the users input from the chat bot and passing it on the service that handles the interaction with the function app and then passing that result on to the adaptive card creator to create the output for the bot. There is also a little error handling as well.

And that's it really. You should have a chat bot that you can use to do sentiment analysis on tweets that you search for. Of course there are some tweeks... that can be done, firstly because I was doing a lot of testing I had set the max number of tweets to search for to be 2 this can be increased to whatever you want. I'd suggest a much higher number like 10 or even 50. It's up to you the higher the number of searched tweets the more accurate the statistics at the end (although is only analyzing a snapshot of all tweets containing the search term so...). You can also increase the number of popular tweets to be returned as well, I think 2 - 5 is fine. There are issues you can run into with the adaptive cards if they get too big, as in they can get cut off or throw errors.
There are a number of improvements that can also be made across the system, things like asking the user for how many popular tweets to return and how many to search for could be a good idea as well as removing the limitation of only searching for tweets with a particular hash tag. I did it for flavour reasons and also it was the first thing that came to mind when I was thinking about a twitter search function. You can also go full on with the Azure services, by using things like an app service to host the function app and a bot service to host the bot and also adding app insights to implement some sort of logging.

Thanks for reading and I hope to see you guys next time.

Sentiment Analysis for Tweets in a Chat Bot - Part 2 Integrations

Albert Bennett — Fri, 22 Jul 2022 07:56:00 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)

Hi and welcome to part 2 of a series of posts that I am writing all about how you can use Azure, luis.ai and the twitter API to create a chat bot that can be used to search and do sentiment analysis on some tweets.

In this part I'll be going on to the coding and integration side of things using a function app.
You can check out the repos for this project by clicking on the links below:
https://github.com/Albert-Bennett/TwitterSentimentAnalysisFunctionApp
https://github.com/Albert-Bennett/TwitterSentimentAnalysisChatBot

The function app
To start we will need to create a new function app. Mine is setup with a http trigger and Open API. The Open API part isn't really required for this project but, I like to have automated documentation where I can just in case I need to refer back to the functionality or various other details of the function app as I go along.

It's also worth noting that I will be using .net 6.0 for the project.

The request and response
So, the plan for the function app is that it accepts a get request that defines a set of hash tags and a number defining the max number of tweets to search for in the twitter API.
To do this we will make sure that the http trigger accepts a get request and defines variables based on query parameters.

           int maxTweets = 0;

            if (!int.TryParse(req.Query["max_tweets"], out maxTweets))
            {
                return new BadRequestObjectResult("max_tweets must be a number greater than 0 and less than 100");
            }

            string hashtagQueryParam = req.Query["hashtags"];

            string[] hashtags = hashtagQueryParam.Split('\u002C');

With the query parameters defined we need to decide what will be returned from the function app on a successful response. I think the best would be to return the top X most popular tweets from the ones that we got back from twitter, as well as details on the sentiment analysis of the found tweets. To do this we will structure the response object as such:

    public enum TweetSentiment
    {
        Neutral = 0,
        Positive = 1,
        Negative = 2,
InConclusive = 3
    }

    internal class FNResponse
    {
        public TweetData[] MostPopularTweets { get; set; } 
        public int NumberOfTweetsFound { get; set; }
        public Dictionary<TweetSentiment, int> TweetSentimentAnalysis { get; set; } 
    }

We are using a enum to manage the response of the sentiment analysis as it is possible for there to be no results for one or more of the analysis categories (positive, negative, neutral, in conclusive). The 'InConclusive' results is only returned for tweets that the analysis can't\ isn't done on or if there was an error when doing the sentiment analysis on that tweet.

Twitter API
Next we are going to need to check what endpoint we need to call and what the response for that action is, on the Twitter API. We need these details so we know what we have to call in the API and how to model our data objects.
From looking at the documentation we can see that the endpoint that we need to call is: https://api.twitter.com/2/tweets/search/recent
To this we will need to add the URL encoding for the hashtag symbol that being %23. This is because the # is a special character and can cause issues when adding it to a URL, also the standard Twitter API doesn't have an option to search for a specific hashtag just a content search so to search for hastags we need to add it to the search query.
Next looking at the response object from the docs and testing out the endpoint through Postman we can see that we need to model our data as such.

    public class TwitterSearchResult
    {
        public TweetData[] data { get; set; }
        public TwitterSearchResultMetaData meta { get; set; }
    }

    public class TweetData
    {
        public TweetPublicMetrics public_metrics { get; set; }
        public string text { get; set; }
    }

    public class TweetPublicMetrics
    {
        public int retweet_count { get; set; }
        public int reply_count { get; set; }
        public int like_count { get; set; }
        public int quote_count { get; set; }

        /// <summary>
        /// This method is just going to add up all of the public metrics to get an arbitrary popularity metric
        /// </summary>
        /// <returns></returns>
        public int GetPopularity()
        {
            return retweet_count + reply_count + like_count + quote_count;
        }
    }

    public class TwitterSearchResultMetaData
    {
        public string next_token { get; set; }
    }

We don't need the ID from the tweet object as we won't be using it but, we do need to know the public metrics (likes, replies, etc) and of course the text from the tweet. There is other data the gets returned from the twitter API when calling that endpoint but, we don't need so so I did model the data for it.
With all of this info we can build up a good idea of what we want to send to twitter and what we want to get back.

The Twitter Service
Super! with all of this information we can now start building up the services that we need to get the function app to do what we want.
For starters we need our TwitterService.

    public class TwitterService : ITwitterService
    {
        readonly IHttpClientFactory _httpClientFactory;

        readonly string bearerToken;
        readonly string baseUrl;

        public TwitterService(IHttpClientFactory httpClientFactory)
        {
            _httpClientFactory = httpClientFactory;

            bearerToken = Environment.GetEnvironmentVariable("TwitterBearerToken");
            baseUrl = Environment.GetEnvironmentVariable("TwitterBaseUrl");
        }

        public async Task<TweetData[]> FindTweetsByHashtag(FNRequestBody request)
        {
            int maxResults = request.MaxTweets;
            string paginationToken = null;
            string hashtagQuery = GetHashtagQuery(request.HashTags);

            List<TweetData> tweetsFound = new();

            while (tweetsFound.Count < maxResults && (paginationToken != null || tweetsFound.Count == 0))
            {
                var twitterSearchResult = await GetNextSetOfTweets(paginationToken, hashtagQuery);

                if (twitterSearchResult != null)
                {
                    paginationToken = twitterSearchResult.meta.next_token;

                    if (tweetsFound.Count + twitterSearchResult.meta.result_count > maxResults)
                    {
                        var diff = maxResults - tweetsFound.Count;

                        for (int i = 0; i < diff; i++)
                        {
                            tweetsFound.Add(twitterSearchResult.data[i]);
                        }
                    }
                    else
                    {
                        tweetsFound.AddRange(twitterSearchResult.data);
                    }
                }
                else
                {
                    return tweetsFound.ToArray();
                }

            }

            return tweetsFound.ToArray();
        }

        string GetHashtagQuery(string[] hashTags)
        {
            string query = string.Empty;

            foreach (string hashTag in hashTags)
            {
                var tag = hashTag.StartsWith('#') ?  hashTag : $"#{hashTag}";
                query = string.IsNullOrEmpty(query) ? tag : query + $" {tag}";
            }

            return query;
        }

        async Task<TwitterSearchResult> GetNextSetOfTweets(string paginationToken, string query)
        {
            string paginationSubQuery = string.IsNullOrEmpty(paginationToken) ? string.Empty : $"&next_token={paginationToken}";

            string url = HttpUtility.UrlEncode($"{baseUrl}?query={query}&tweet.fields=public_metrics{paginationSubQuery}");

            var message = new HttpRequestMessage(HttpMethod.Get, url)
            {
                Headers =
                {
                    { "Authorization", $"Bearer {bearerToken}" }
                }
            };

            using (var client = _httpClientFactory.CreateClient())
            {
                var response = await client.SendAsync(message);

                if (response.IsSuccessStatusCode)
                {
                    using var contentStream = await response.Content.ReadAsStreamAsync();

                    return await JsonSerializer.DeserializeAsync<TwitterSearchResult>(contentStream);
                }
            }

            return null;
        }

Simply put, the twitter service connects to twitter. It creates the endpoint and sends on the query request to the twitter API. It then sends back data to us that we can use with luis to do the sentiment analysis. The only gotcha moment with the API call is that the twitter API only sends back 10 results at a time. To get around this they also send on a pagination token. We can use that token to get the next set of results on and on until we have the max amount that has been requested.

luis Data Modeling
This next part is a little easier as we can test out the luis prediction endpoint that we gathered from the previous step in Postman. With this we can see what the returned object is from the luis API.

With this info we can model our data as such:

    public class LuisQueryResult
    {
        public LuisPrediction prediction { get; set; }
    }

    public class LuisPrediction
    {
        public LuisSentiment sentiment { get; set; }
    }

    public class LuisSentiment
    {
        public string label { get; set; }
    }

Although the response object from luis was much more complex and there are query parameters that we can add to get more data, I'd like to keep the responses back as lean as possible. Besides for our project all we really need is the sentiment analysis portion of the response.

The luis Service
This is going to be much the same as the twitter service. We need to build up a query and send that as a get request to the endpoint and then transform the results.

    public class LuisService : ILuisService
    {
        readonly IHttpClientFactory _httpClientFactory;

        readonly string subscriptionKey;
        readonly string appId;
        readonly string baseUrl;
        readonly string endpoint;

        public LuisService(IHttpClientFactory httpClientFactory)
        {
            _httpClientFactory = httpClientFactory;

            subscriptionKey = Environment.GetEnvironmentVariable("SubscriptionKey");
            appId = Environment.GetEnvironmentVariable("AppId");
            baseUrl = Environment.GetEnvironmentVariable("LuisBaseUrl");
            endpoint = Environment.GetEnvironmentVariable("LuisEndpoint");
        }

        public async Task<Dictionary<TweetSentiment, int>> GetSentimentAnalysisOnTweets(TweetData[] tweets)
        {
            Dictionary<TweetSentiment, int> sentimentAnalysis = new Dictionary<TweetSentiment, int>();

            foreach (TweetData tweet in tweets)
            {
                var analysis = await GetSentimentAnalaysis(tweet.text);

                if (sentimentAnalysis.ContainsKey(analysis))
                {
                    sentimentAnalysis[analysis]++;
                }
                else
                {
                    sentimentAnalysis.Add(analysis, 1);
                }
            }

            return sentimentAnalysis;
        }

        async Task<TweetSentiment> GetSentimentAnalaysis(string text)
        {
            string url = $"{baseUrl}{appId}{endpoint}?subscription-key={subscriptionKey}&query={HttpUtility.UrlEncode(text)}";

            using (var client = _httpClientFactory.CreateClient())
            {
                var response = await client.GetAsync(url);

                if (response.IsSuccessStatusCode)
                {
                    using var contentStream = await response.Content.ReadAsStreamAsync();

                    var result = await JsonSerializer.DeserializeAsync<LuisQueryResult>(contentStream);

                    switch (result.prediction.sentiment.label)
                    {
                        case LuisConstants.NegativeResult:
                            return TweetSentiment.Negative;

                        case LuisConstants.PositiveResult:
                            return TweetSentiment.Positive;

                        default:
                            return TweetSentiment.Neutral;
                    }
                }
            }

            return TweetSentiment.InConclusive;
        }
    }

The only gotcha ya' in this service was the addition of an 'InConclusive' sentiment. this is for when the luis API returns back an unsuccessful response from something like a bad request of for querying the API too often in a short time span.

Putting it all together
Finally on to gluing it all together. What we need to do is use the two services to build up the response body and use the popularity metric that we added to the public_metric object to return the X most popular tweets, with the sentiment analysis done.

        public async Task<IActionResult> Run(
            [HttpTrigger(AuthorizationLevel.Function, "get", Route = null)] HttpRequest req)
        {
            int maxTweets = 0;

            if (!int.TryParse(req.Query["max_tweets"], out maxTweets))
            {
                return new BadRequestObjectResult("max_tweets must be a number greater than 0 and less than 100");
            }

            if (maxTweets <= 0)
            {
                return new BadRequestObjectResult("max_tweets must be greater than 0");
            }
            else if (maxTweets > 100)
            {
                return new BadRequestObjectResult("max_tweets must be less than 100");
            }

            string hashtagQueryParam = req.Query["hashtags"];

            string[] hashtags = hashtagQueryParam.Split('\u002C');

            if (hashtags == null || hashtags.Length == 0)
            {
                return new BadRequestObjectResult("You must include hashtags to search in the request body");
            }

            var foundTweets = await _twitterService.FindTweetsByHashtag(hashtags, maxTweets);

            if (foundTweets != null)
            {
                var maxNumberOfPopularTweets = int.Parse(Environment.GetEnvironmentVariable("MaxNumberOfPopularTweets"));
                var mostPopularTweets = foundTweets.OrderByDescending(x => x.public_metrics.GetPopularity()).Take(maxNumberOfPopularTweets).ToArray();

                var fnResponse = new FNResponse
                {
                    MostPopularTweets = mostPopularTweets,
                    NumberOfTweetsFound = foundTweets.Length,
                    TweetSentimentAnalysis = await _luisService.GetSentimentAnalysisOnTweets(foundTweets)
                };

                return new return new OkObjectResult(fnResponse);
            }

            return new BadRequestObjectResult($"No tweets found searching for the following hashtags: {hashtags}.");
        }
    }

There is really not much to our entry point just, some data validation and then the assembly of the various data points.
With that all done, the result should look something like this.

The last last part I'm going to go over is the local setting file where I have placed all of the configuration for the function app in case you want to try and get the bot working by yourself.

{
    "IsEncrypted": false,
  "Values": {
    "AzureWebJobsStorage": "UseDevelopmentStorage=true",
    "FUNCTIONS_WORKER_RUNTIME": "dotnet",
    "TwitterBearerToken": "[your twitter bearer token]",
    "TwitterBaseUrl": "https://api.twitter.com/2/tweets/search/recent",
    "MaxNumberOfPopularTweets": 3,
    "SubscriptionKey": "[your azure account subscription key]",
    "AppId": "[your luis app id]",
    "LuisBaseUrl": "https://[the name of your luis app].cognitiveservices.azure.com/luis/prediction/v3.0/apps/",
    "LuisEndpoint": "/slots/production/predict"
  }
}

For demo purposes placing the config here is fine but, in a more production environment these should be held in something like KeyVault. Unless you are testing locally, in which case it's also fine to keep them locally so long as the local.settings file doesn't get committed to the repo and you are using non production settings.

With this series of posts I'm just going over the key parts to the application. There are other code tasks that I had done and can be seen in the repos for this project. If you need a crash course in it or on function apps I have a few posts where I got over it that can be found below:

With that thanks for reading my post and I'll see you in part 3 for the finale and the bot side of this chat bot.

Sentiment Analysis for Tweets in a Chat Bot - Part 1 The Setup

Albert Bennett — Fri, 22 Jul 2022 07:56:00 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)

Hi and welcome to my blog post.
Today I'd like to show you how you can use the Twitter API, luis.ai and the Microsoft Bot Framework to create a chat bot that does sentiment analysis on tweets posted on twitter. To be fair though the chat bot part is more for display and demonstration purposes. The function App with the twitter integration is where the actual magic ✨ happens.
See below for links to the GitHub repos:
https://github.com/Albert-Bennett/TwitterSentimentAnalysisFunctionApp
https://github.com/Albert-Bennett/TwitterSentimentAnalysisChatBot

This is going to to a three part series.
In this part I'm going to explain how to set up various resources so that we can start programming in part 2. If you'd like you can click on the thumbnail below for a demonstration of the chat bot in action.

Getting Started
For this part you'll need access to an Azure account and you'll need to create a twitter developer account. This part shouldn't be to complex just a lot a forms and procedures.

Step 1 - Twitter Authentication App
With the Twitter API it is fairly straight forward. Just go here: Twitter API Signup Page and sign up if you haven't already.
When that's done you'll need to create a new project and an app in that project. With the app created you should be shown that app's authentication details. This would include:

An API Key
An API Key Secret
A Bearer Token

❗ As soon as that screen pops up copy the - API Key, API Key Secret and Bearer Token to a new notepad page or something. As soon as you transition off that page those details will be hidden. Although you can regenerate them at anytime, you will need to save those details for the Twitter integration later on.
You can read through and explore the docs if you want at this point now that you have your bearer token you can access some of the API endpoints and try them out. The Twitter docs are amazing, everything is fairly easy to find and they have an in browser testing feature where you can see what the endpoints are going to look like built from the fields that you have selected.

Step 2 - Azure Resources
We need to create some resources in Azure to get us started. Mostly to support the luis app.
Firstly you need to create a new resource group in Azure to hold all of the resources that we will be using in this application. This is fairly straight forward, just search in Azure for resource group and create a new one.
It is important to make sure that the region that you select to create the resource group in is close to where you are and that all resources that you create are in the same region this is so that data doesn't have to travel far in-between the differnt applications and services.
Next we need to create a new 'language understanding' resource in Azure in our new resource group.
This resource is essentially luis in Azure. We need it so that when we create a new app in luis we can associate it with this resource.

Step 3 - luis app
We are going to be using luis.ai in this application to preform the sentiment analysis on the tweets that we get back from Twitter. luis.ai is a NLP (natural language processor) meaning that it is used to process natural language not just a set of commands but, unstructured language like this and these words as well.
Firstly you'll need to sign in to the luis service and set the authoring resource to be the new authoring resource that should have been created with the language understanding resource from the previous step.
From here we can create a new app.

We don't need to create entities, utterances or anything for our app to do the sentiment analysis. What we do need is to click train at the top of the screen.

With the model trained the publish button should be enabled.
Next we need make sure that when the luis model gets published it has setup to do sentiment analysis. This is done by just changing the publish settings which is found when you click publish.
I've set mine up to use the production slot when it gets deployed just to make the calls in the function app a little simpler.
Although we can test the luis app before publishing it, the sentiment analysis is something inbuilt in luis and I don't think that what we do in the app can influence the sentiment analysis of the tweets from twitter. We still need to test the service though to make sure everything has been setup correctly.

Last but not least while we are in luis, if you go to the manage page at the bottom there is an example query. We will need this for step 2 as it will for the foundation of our luis endpoint.

With that setup and tested it's on to the next step.

Thanks for reading and I'll see you in the next one!

If you like this post and are interested in learning a little more about the Microsoft bot framework feel free to look through my other blog posts:

How to add a retry policy to a Timer Function

Albert Bennett — Thu, 14 Jul 2022 20:25:40 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)

Hi and welcome to my blog post.
It's a quick one today. I'm going to show you how to add a retry policy on a timer function. I had seen an article shared on Microsoft's twitter account a short while ago and wanted to share some of the info. It seems like a really cool feature and, one that I am excited about. Having worked with function apps on an off for the past 5 years or so. Also I'll link their article on it below.

What is a retry policy
A retry policy is an action that retries a request a number of times until either the maximum number of retries has occurred or the endpoint returns a successful response.
It is mostly used to handle transient errors in an application. Things such as drops in connections, minor unexpected errors that kind of thing.

Implementation
This retry policy isn't available on all kinds of Azure functions. For the example today, I'll be showing it on a timer function but. To my knowledge this binding only works for Timer, Event Hubs functions but, there are other ways of adding a retry policy to an Azure function.
For timer functions adding a retry policy is very straight forward. Simply add the following 'FixedDelayRetry' binding to your function app:

        [FunctionName("Function1")]
        [FixedDelayRetry(5, "00:00:10")]
        public void Run([TimerTrigger("0 */5 * * * *")]TimerInfo myTimer, ILogger log)
        {
            log.LogInformation($"C# Timer trigger function executed at: {DateTime.Now}");

            int i = Random.Shared.Next(4);

            if (i < 4)
            {
                log.LogError($"Thrown an error because number generated was < 4: {i}");
                throw new FunctionException("Not working :(");
            }
        }

I've added extra logic to the function simply to test out the retry policy.
Important things to note is that in the binding you need to specify the number of retries that you want to be attempted and also the time delay between them. For this example I have set it to retry a maximum of 5 times with a 10 second delay between each attempt.

It's also worth noting that this feature is only available on version 3 function apps and above.

As promised here is the original article:
https://docs.microsoft.com/en-us/azure/azure-functions/functions-bindings-error-pages?tabs=fixed-delay%2Cin-process&pivots=programming-language-csharp

Or if you want to find out more about the retry pattern:
https://docs.microsoft.com/en-us/azure/architecture/patterns/retry

Thanks for reading and I hope to see you guys next time.

How to: Save conversation data in an Microsoft Chatbot

Albert Bennett — Wed, 08 Dec 2021 23:13:30 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)

Hi and welcome to my blog post.
Today I'd like to show you how to get started with saving conversation data with chatbots using the Microsoft bot framework.
Feel free to check out the repo for a working sample:
https://github.com/Albert-Bennett/state-management-chatbot

Getting Started
To get us started you need to create an object to hold the data that you need to persist throughout the conversation. In the repo above I've used this one:

    public class ConversationData
    {
        public ConverstationStates CurrentState { get; set; } = ConverstationStates.Welcome;
        public string Name { get; set; }
        public string Age { get; set; }
    }

The enum in my class (ConverstationStates) is just used to keep track of what response to return to the user after certain stages of the conversation. For the sample bot all that is kept track of is the users name, age and the current conversation state, although you can use this technique to store more complex data types.

Modifying the Startup.cs class
Next you'll need to modify your startup.cs class to be make the ConversationState object injectable into the chatbot. We need to be able to do this to use it later on when we want to save the data.

        private static void CreateConversationalState(IServiceCollection services)
        {
            var storage = new MemoryStorage();

            var conversationState = new ConversationState(storage);
            services.AddSingleton(conversationState);
        }

This method needs to get called at the end of your ConfigureServices method. The main take away from this is the creation of the MemoryStorage object, this is just a reference to local storage to store the conversation data in. Also note the creation of the ConversationState object, this object is kind of like a wrapper that allows us to access read and write for our conversation data. Thus letting us persist the data for the duration of the users session.

Getting our Data
This is two lines of code for us.

var conversationStateAccessors = _conversationState.CreateProperty<ConversationData>(nameof(ConversationData));
var conversationData = await conversationStateAccessors.GetAsync(turnContext, () => new ConversationData());

It is worth noting that I am injecting in the ConversationState (_conversationState) object into the bot class and utilizing it here to either create a new ConversationData object or get the current one that the user is using.

Saving the Data
The last task that we need to do to persist the users data is to save it. Below I have put this logic in the OnTurnTask method of the bot.

        public override async Task OnTurnAsync(ITurnContext turnContext, CancellationToken cancellationToken = default)
        {
            await base.OnTurnAsync(turnContext, cancellationToken);

            await _conversationState.SaveChangesAsync(turnContext, false, cancellationToken);
        }

This method gets called when there is something happening in the bot such as sending out a response and is normally one of the last methods to finish execution before returning back to the user. I have placed the logic here to make sure that the conversation data is getting saved in one place.

Sample in Action
If you'd to see the bot in action, I have linked the video of it working below.

Also feel free to check out the repo if you want to try it yourself. The link is here: https://github.com/Albert-Bennett/state-management-chatbot
You'll also need to download the bot framework emulator v4 to get the bot running locally.

That's about it for persisting conversation data in a chat bot. Of course there are ways of persisting the data for longer that the users session, maybe by using something like CosmosDB. But, I think that the snippets of code that I have shown, show off the basics of how the process works.

Thanks for reading and I'll see you in the next one!

If you like this post and are interested in learning a little more about the Microsoft bot framework feel free to look through my other blog posts:

Lazy vs Eager Initialization

Albert Bennett — Tue, 07 Dec 2021 20:23:14 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)

Hi and welcome,
Today I'm going to go though the uses and differences of Lazy vs Eager initializing of objects.
As a bit of a disclaimer, I haven't seen lazy objects being used too often nowadays. Today was the first time that I had seen it in about 4 or so years but, I thought that it was an interesting topic to write a blog on.

Introduction
What Lazy initializing of an object means is to initialize an object when it is first accessed, instead of eager initializing an object.

Pros and Cons of using Lazy Initializing
Below are a few of the advantages and disadvantages of using Lazy initialization vs Eager initialization.

pro: Lazy initializing an object can save on resources (memory and computation mostly).
pro: With lazy initializing you can defer the instantiation of the object. For example you can instantiate a large object or collection after some intensive processing which can help improve overall performance by free up resources.
con: It's a bit overkill if the object that you are lazy loading is small and you know that you will need to use it soon after it has been defined.
con: Can waste lots of resources if managed incorrectly such as wrapping many small objects instead of managing them in a Lazy enumerable object or statically.

Code Sample
See below for how you can define a lazy object in C#:

    public class TestLazyInitialization
    {
        Lazy<MassiveObject> bigObject;

        public MassiveObject BigObject
        {
            get
            {
                return bigObject.Value;
            }
        }

        public void MyFunction()
        {
            bigObject = new Lazy<MassiveObject>(() => new MassiveObject
            {
                ReallyBigString = "This is a massive object"
            });
        }
    }

    public class MassiveObject
    {
        public string ReallyBigString { get; init; }
    }

The creation of the object inside of the lazy object happens when the Value property of the lazy object (bigObject.Value) gets accessed, in this case when a consumer access the BigObject property of the TestLazyInitialization class.

Notes

Eager initializing an object is to initialize an object when it gets defined. Something like this:

MassiveObject bigObject => new();

Or if you are old skool (⌐■_■)

MassiveObject bigObject = new MassiveObject ();

Thanks for reading my post and I'll see you next time
ᕕ( ᐛ )ᕗ

C# 10.0 - Useful features to know

Albert Bennett — Fri, 03 Dec 2021 21:49:28 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)

Ho, ho, ho merry... something 🎅

It is that time of year again MS ignite has past by and few new features\ updates have been announced for our favorite programing language C#. Today I'd like to show you how you can use a few of the new features that I have found very useful over the past month or so.

Field Initializers in Structs
I am really happy about this update to struct instantiation.
In the past if you wanted to set a default value for a property in a struct you might need to do something like this. See example below:

    public class TestClass
    {
        public void CreateThing()
        {
            var thing = new Thing("Name");
        }
    }

    public struct Thing
    {
        public Thing(string name)
        {
            Name = name;
            Id = 5;
        }

        public string Name { get; set; }
        public int Id { get; set; }
    }

Notice how we can't set the value of a property at property declaration nor can we set the value inside of a parameterless constructor as both would result in an error. This is why we needed to create a constructor with a parameter to set the default value for the 'Id' property.
Now in C# 10 you can do the same in fewer lines of code and no need for a new constructor.

    public class TestClass
    {
        public void CreateThing()
        {
            var thing = new Thing();
        }
    }

    public struct Thing
    {
        public string Name { get; init; } = "name";
        public int Id { get; init; } = 5;
    }

Record Structs
So, records have been around since C# 9. A record is essentially an immutable reference type that is used when your data needs to be both immutable and encapsulate some kind of complex data value. It has all of the properties and behaviors of a class. See example of how to define a record below:

public record UserId(string Id);

The change here for C# 10 is that a record can now be declared as a struct as well. See example below:

public record struct UserId(string Id);

The advantage here is that the record can now contain all of the same properties and behaviors that a struct has all in one line of code.

Namespace declaration
This has been simplified in C# 10.
Normally this is how namespaces were defined in previous versions of C#.

namespace testLib
{
    //some class
}

In C# 10 it has been simplified to:

namespace testLib;

//some data types

In my opinion it really makes code files look a lot cleaner by removing a set of braces and a level of indentation. It's not something that comes up so often but... it's nice when it does.

Thanks for reading my post on C# 10.0 features that I think are useful.
I wrote this post as part of the C# Advent Calendar event for day 5. Feel free to check out the other blog post already revealed during this event: https://csadvent.christmas/

The entire list of updates for C# 10 can be found here:

https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csharp-10

Flight Bot: A chat bot to avoid a minor annoyance

Albert Bennett — Wed, 10 Nov 2021 15:52:53 +0000

If you learned something new feel free to connect with me on linkedin or follow me on dev.to :)

Hi there and welcome to another one of my blogs.
This time I'd like to talk about a project that I had built for a Hackathon that was being hosted at work. The theme of the Hackathon was open, I mean there was a few category's to choose from to get some inspiration but.. . I decided to go my own way with it via the open category.

The Projects Backstory
The tldr version is that I had tried to book flights a couple of years ago to go to visit Bran's Castle in Romania for Halloween and had gotten frustrated with how many sperate flight finding websites that I needed to search to book a flight. Up to that point I hadn't ever booked a flight and had gotten overwhelmed with the amount of choice, in the end I decided not to go.

Project Demo
Here is the flight finding bot in action (links to demo video)!

And the github links for the curious:
Project repo:- https://github.com/Albert-Bennett/flightbot
API repo:- https://github.com/Albert-Bennett/FlightBot.DatabaseSeeding

Project Tech Stack
Here is a list of what I had used to get the project working:

The Microsoft Bot Framework (used to build the bot and conversation model)
luis.ai (used to extract data from the users free text, although given the use case for luis in the project was limited it was still better that trying to parse the string manually)
geonames API (used to search for a airports close to the users location and used to confirm search results. Alternates to this was the google maps api but, anyone could register as an airport on it so... I didn't use it)
Amadeus API (the API that was used to find the flights, originally I had planned to use the skyscanner API but, they wouldn't let me access it)
Azure SQL (used to map IATA codes. IATA codes are surprisingly hard to get a hold of and seeing as how I needed to use the geonames API to search for airports it didn't make much sense to not create a DB to hold those mappings. Also, there was a lot of APIs about there for searching for IATA codes but, a lot of them didn't have a free trial period)
Azure Functions (used to create an API to update the IATA DB and also to search for IATA codes based on geoname id, name, location or IATA code (just in case someone wanted to search for a location based on IATA code))

Challenges
The main challenge of this project was that part way through it my partner showed me an app for booking flights and how easy it was to use. It was a flight comparison website using the Amadeus API as well. It made me rethink the design of the chat bot, as well as a target audience for something like this. I think as programmers it is easy for us to take for granted how much of an advantage being tech savvy is, a previous role of mine training people to use computers at an employment center really made me appreciate how much more accessible the world is to us. Take someone like my mum for a moment, she is 65 years old, she has a phone and all she can use it for is making calls, taking pictures and on occasion accidently hang up on someone before getting frustrated by the amount of options that she has been presented with and struggling to figure out what had happened.
I'm not trying to put her down here. She tries her best but, it is not what she is use to. As such I thought the advantage of using the format of a chat bot over an application is in its simplicity, it's straightforwardness so to speak. In my opinion a chat bot can be made much more accessible to the less tech literate by directing them toward a singular goal, presenting them with very straight forward questions and acting on their fallibility with follow-ups to the inputs and re-affirmations such that they don't get lost in the process. Although accommodating for something like this was much more work then what I had originally planned. State management was a particular pain point given how the flow of a more conversation heavy application needs to be able to remember certain parts of information and yet be able to go back to a previous state of the conversation.

Let me know what you guys think below I'd like to hear about what opinions that you have around the topics of chat bots and accessibility.
Thanks for reading my blog post.

A very confusing puzzle game I made in Unity

Albert Bennett — Mon, 18 Oct 2021 16:57:26 +0000

If you liked what you read feel free to connect with me on linkedin or follow me on dev.to :)
Here is a link to where you can download the game: https://play.google.com/store/apps/details?id=com.MindOfBobGames.ThePenteract&gl=IE

Introduction

My career started in the mobile games dev industry. It is where I had spent the first few years of my career, as such I made a few games in my time...
Most of what I had created was POCs to see if the core gameplay mechanics would work out. One of these POCs was The Penteract.

History of the Project

Before the game was released on to the Google Play Store it was sitting in a folder on my laptop for several years. The original POC for this game was built but, apart from that there wasn't anything else to it. I had a general idea for where I wanted it to go and what I had wanted to do with. Finally last year I got around to finishing it off to a point where I could be proud of what I had done.

Concept

The initial idea behind it was a 3D puzzle game that has the player trapped inside of a box where they had to make there way to the center of said box. The player's goal is to solve puzzles on the doors leading to other coloured rooms with different symbols on the walls. The aim of the game was for the player to reach the center of the Penteract before it collapsed in on itself. The problems that I had run into was all around planning the mechanics around the concept and how to display them.

Development Issues

Conceptually it seems simple however, its implementation was not. I had many issues with implementing the original concept mostly around how to make it fun and rewarding to players. I could have changed the concept but the main benefit that I could see from this approach is that the puzzles could be made using a tool, allowing me to add 1,000s of different variations on the main set of puzzles also having only a few models to UV map and animate was a huge bonus. Below I've gone into greater detail of these issues.

Problem 1: Atmosphere

With the restrictions in place that being a limited number of assets being present in the game, I felt that the atmosphere needed to be on point. I wrote a short story for what The Penteract was, taking inspiration from the physics concept of 5 dimensional space. I felt that it was a strong theme to set the game in a dark SIFY world as well. Where the device was a 5 dimensional cube, in which the occupants can go through time and space. This also could be used to explain why some puzzles and cube configuration can repeat as well as solving the limited assets problem.

For the rooms I had to used various bright colours, so the colour palette was quite broad except for the door panels. These were all grey to stand again some of the brighter coloured rooms they had been animated using a render target so that when the lights would flicker they would still show up so that the player would still know where they were.

Music was another key part of the atmosphere, this done quite simply using AI because I can't make music :( and also it was a huge time saver for me. The service that I had used has now been discontinued but I believe that https://soundraw.io is fairly close to what I had used. I focused on slower beats with a SIFY twang to them.

Problem 2: Making the game fun

With the main pieces in place for the atmosphere the gameplay aspects fell into place.
Firstly if the player could solve puzzles to make it to the center of the Penteract then where does the challenge come in to it, would it all not be the same number of puzzles to get to the center. Sure I could have used a random number of puzzles to solve the issue but that felt like a cheap way of solving the problem. Instead I decided to do something much more convoluted. The puzzle system itself is simple, the colours and symbols on the walls dictate the number of points allocated to that particular puzzle. The rooms are designed so that in each room:

Two doors give no points and don't change the state of the Penteract.
One room grants the user the highest number of points thus bringing them closer to winning
One room gives the user a small number of points again progressing their win condition
Two rooms that help to breakdown the Penteract in either minor or major ways.

To limit player frustration the breakdown of the Penteract is done in levels from a little broken where 20% of all doors that give the player <1 point can open to a broken room to flashing lights and timers restriction on puzzles. I really like the timer restrictions, and I animated the puzzles to look like a clock counting down using rectangles. I feel like the timer restrictions really help to add pressure to the player where the broken rooms helps the player to choose the more correct options.
The one real issue I had and still have is that to facilitate all of this the way the player can figure out which doors can help progress their game state is very convoluted as I needed to use colours to show the player that the rooms they were going into was different from each other. In the end I also placed a set of randomly flashing patterns above a puzzle to help but... it didn't help. I'd go into how the actually system and how the combinations fit together but, the code has been lost. It is also why the images in the blog post are all from the store page.

Problem 3: The puzzles

It worth noting that there are two types of puzzles in the game intrinsic (door puzzles) and extrinsic (room layouts).
The Penteract is a mobile game as such I needed the intrinsic puzzles suited to touch controls that can have many, many variations whilst still maintaining the same format and also can be very recognizable to the player especially as the player could be put under a little pressure. So right of the bat complex puzzles were gone.
So I thought of a series of short puzzles that most people could do in quick succession: mostly either word, math or pattern based puzzles. I had recently played some mobile games that used a combination of them to compete with other players, so why not...
The variations of the puzzles come by means of pre generating them using a WPF application that exports out a JSON file where each object in the file defines another variation of the puzzle. The way the game makes use of these files is by randomly selecting a puzzle format checking the difficulty of the game and then randomly selecting a puzzle in the file. In the game I think there is about 7,000 different variations of the set of puzzles.

Apart from these issues the rest of the game was fine, I mean programmatically speaking there isn't much complicated about it.

Thanks for reading all about a game I had siting in a folder that I finished last year and the challenges that I had faced when completing it. I thought that it might good for me to share some of my experiences in the games dev world, I rarely talk about it nowadays. I'm not sure if I'd return to it though but maybe, if the right idea/ project came along...

Forem: Albert Bennett

Data Structures in C# - The complex ones... and some other ones as well

Binary Trees

Stacks

Queues

Data tables

Hashset

Common Sorting Algorithms in C# - From My Experience

Bubble Sort

Quick Sort

Merge Sort

Speed Comparisons

Displaying charts in an adaptive card

Introduction

Other solutions

Sentiment Analysis for Tweets in a Chat Bot - Part 3 The Chat Bot

Sentiment Analysis for Tweets in a Chat Bot - Part 2 Integrations

Sentiment Analysis for Tweets in a Chat Bot - Part 1 The Setup

How to add a retry policy to a Timer Function

How to: Save conversation data in an Microsoft Chatbot

Lazy vs Eager Initialization

C# 10.0 - Useful features to know

Flight Bot: A chat bot to avoid a minor annoyance

A very confusing puzzle game I made in Unity

Introduction

History of the Project

Concept

Development Issues

Problem 1: Atmosphere

Problem 2: Making the game fun

Problem 3: The puzzles