Forem: Michael Berry

Python and Headless: How to make interactive, adaptable, affordable reports

Michael Berry — Wed, 16 Jun 2021 19:31:01 +0000

In this article, I'll show you how to pull data from a headless CMS and create a content type report using the Python framework: Dash.

Seeing is believing

I am a visual learner. I really appreciate when a graph or diagram accompanies data, or when a visual aid gets to "the point" in a presentation. Not everyone is as much of a fan of charts as I am, but representing data visually is universally helpful. For example, which is easier to understand in a presentation:

these side-by-side numbers: 101 | 443 | 102 | 320 | 82

this graph:

With just the delimited list of numbers, you don't get a good idea of how the values compare to one another. The graph makes differences and similarities more obvious.

Let's say that you are a new manager or member of a marketing team that uses a headless CMS to produce content. Knowing what types of content the team has been producing will be useful for better understanding their current content strategy. However, when you are looking at content models in a headless CMS, you're normally met with a long running list of names and numbers:

To assess the kinds of content in the project, you're forced to scan the entire content type listing and mentally compare the values. Without a visual aid, it's hard to tell what topics are prevalent in your project. Furthermore, once you've finished your analysis, you can't easily share your findings with other members of your team. Making them believe your analysis is correct is much easier with a visual report.

But what if you don't have the time or resources to integrate a commercial reporting tool into your project? Luckily, there are open source frameworks and libraries for building web analytic applications, like Dash.

What is Dash?

Dash is framework built upon the popular Plotly graphing library, that allows you to build interactive visual applications and render them in a browser. Although it uses JavaScript libraries behind the scenes, Dash apps are written purely in Python and don't require familiarity with JavaScript.

If you know some Python, Dash is well suited to create reports from headless CMS data because it supports a wide variety of well-documented graph types, and the open-source edition is completely free.

How to create a report from a headless CMS

Let's create a content report using the Kentico Kontent headless CMS, the open source Kontent Python SDK, and Dash.

In our sample scenario, we'll create three reports about a coffee shop website, and aggregate them in a tabbed report.

In our report we want to show:

what type of content exists in the project
the audiences our articles are targeting
the timeline showing when they were published.

These reports will give us more insight into the "who, what, when" strategy surrounding our previously published articles.

Application setup

First we need to install Dash, our SDK, and dependencies. I recommend that you do this in a virtual environment. In the terminal run:

pip install dash
pip install kontent_delivery
pip install pandas

Alternatively, you can download and then install this requirements.txt using pip install -r /path/to/requirements.txt.

By the end of the exercise, we will have a file structure that looks like this:

kontent-dash-blog (root)/
    reports/
        article_timeline.py
        content_types.py
        personas.py
    report_dashboard.py
    requirements.txt

Dash and Kontent Initialization

Once we've installed the requirements, we need to:

setup our Flask application for running Dash
initialize our Kontent SDK delivery client using our Kontent project ID
setup some methods to build our three reports

We'll do this in a file called report_dashboard.py and create it in the root of the project:

import dash
import flask
from kontent_delivery.client import DeliveryClient
# report methods
from reports.content_types import build_types_chart
from reports.personas import build_personas_pie
from reports.article_timeline import build_post_timeline

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']

# dash
server = flask.Flask(__name__)
app = dash.Dash(server=server, external_stylesheets=external_stylesheets)

# kontent
client = DeliveryClient("your_Kontent_Project_ID")

# custom methods to create reports
bar = build_types_chart(client)
pie = build_personas_pie(client)
timeline = build_post_timeline(client)

Report 1: bar chart showing content types

In this chart, we'll call the Kontent API and count how many content types exist of a given type. We'll create the file content_types.py in a reports directory:

import plotly.express as px

def build_types_chart(client):
    type_response = client.get_content_types()
    item_response = client.get_content_items()

    types = {}

    # add Kontent type counts to a dictionary
    for content_type in type_response.types:
        types[content_type.codename] = 0

    # increment content type count per item of respective type
    for item in item_response.items:
        types[item.content_type] += 1 

    data = {"content types":types.keys(), "count": types.values()}

    fig = px.bar(data, x="content types", y="count")

    return fig

In the build_types_chart method, we create key value pairs from the codename of our content types, and a count of how many items are published of that type. With that data, we use Plotly Express to create and return a bar graph to our Dash application.

Report 2: pie chart showing article personas

In our second chart, we want to show what percentage of personas our article content targets. Let's assume we know that the article content type uses a taxonomy group called personas:

We'll create the file personas.py in the reports directory:

from kontent_delivery.builders.filter_builder import Filter
import plotly.express as px

def build_personas_pie(client):
    # only return "article" content items
    item_response = client.get_content_items(
        Filter("system.type", "[eq]", "article")
    )

    # get the taxonomy group that articles use
    personas_response = client.get_taxonomy("personas")
    chart_personas = {}

    # identify personas to be counted
    def get_taxonomy_terms(terms):
        for persona in terms:
            chart_personas[persona.name] = 0
            if persona.terms:
                get_taxonomy_terms(persona.terms)


    get_taxonomy_terms(personas_response.terms)

    # increment persona count per item
    for item in item_response.items:
        for persona in item.elements.personas.value:
            chart_personas[persona.name] += 1 

    data = {"personas":chart_personas.keys(), "count": chart_personas.values()}

    fig = px.pie(data, names="personas", values="count",
                title='Assigned article personas')
    return fig

In the build_personas_pie method, we pull all of our article content items. We then create key value pairs from the name of taxonomy terms that exist in our Personas taxonomy group, and a count of how many items are tagged with a particular term. With that data, we use Plotly Express to create and return a pie graph to our Dash application.

Report 3: timeline showing date posted

Information about when these articles were published could provide more context about what our content strategy was previously, or how it changed over time. Let's assume we have a "posted date" element on our articles that we can use to create our timeline:

We'll create the file article_timeline.py in the reports directory:

from kontent_delivery.builders.filter_builder import Filter
import pandas as pd
import plotly.express as px

def build_post_timeline(client):
    # only return "article" content items
    item_response = client.get_content_items(
        Filter("system.type", "[eq]", "article")
    )

    # setup two arrays for Plotly scatter
    chart_items = []
    chart_post_date = []

    # populate arrays with respective item name and post date
    for item in item_response.items:
        chart_items.append(item.name)
        chart_post_date.append(item.elements.post_date.value)

    df = pd.DataFrame(dict(item=chart_items, post_date=chart_post_date))

    # use column names of df for the different parameters x, y, ...
    fig = px.scatter(df, x="post_date", y="item",
                    title="Article post timeline",
                    labels={"Post Date":"Date item posted"} # customize axis label
                    )

    return fig

Like the other methods, we retrieve the article items from Kontent, but in this example we take an extra step to convert two arrays into a dictionary and pass them into a pandas DataFrame. This is only for the sake of matching the basic dot plot example in the Plotly documentation.

Tying it all together in a tabbed report with Dash

It's time to pull all the visual aids together in a single report and add some interactivity with Dash. We created the report_dashboard.py file earlier to add the Kontent client and our chart building methods, and now we'll add a layout with some interactive components:

# ... other imports from before
# add these for components:
import dash_html_components as html
import dash_core_components as dcc
from dash.dependencies import Input, Output

external_stylesheets = ['https://codepen.io/chriddyp/pen/bWLwgP.css']

# dash
server = flask.Flask(__name__)
app = dash.Dash(server=server, external_stylesheets=external_stylesheets)

# kontent
client = DeliveryClient(config.project_id, options=config.delivery_options)

bar = build_types_chart(client)
pie = build_personas_pie(client)
timeline = build_post_timeline(client)

# -- end of "Dash and Kontent Initialization" code --

# -- start of "Tying it all together..." code --

# adding the dash layout
app.layout = html.Div([
    # dash tabs component that wraps individual tabs
    dcc.Tabs(id='tabs-example', value='tab-1', children=[
        dcc.Tab(label='Content Types', value='tab-1'),
        dcc.Tab(label='Article Personas', value='tab-2'),
        dcc.Tab(label='Article Post Timeline', value='tab-3'),
    ]),
    # div used as a canvas to show reports
    html.Div(id='tabs-example-content')
])

# onclick events that call our chart building methods
@app.callback(Output('tabs-example-content', 'children'),
              Input('tabs-example', 'value'))
def render_content(tab):
    if tab == 'tab-1':
        return html.Div([
            dcc.Graph(figure=bar)
        ])
    elif tab == 'tab-2':
        return html.Div([
             dcc.Graph(figure=pie)
        ])
    elif tab == 'tab-3':
        return html.Div([
             dcc.Graph(figure=timeline)
        ])

# run the application when report_dashboard.py is called
if __name__ == '__main__':
    app.run_server(debug=True)

In the final code file of our report, we use Dash's core HTML components to create a simple two div layout. The first div contains Dash "Tab" components that we use to toggle between graphs, and the second div acts as a canvas to show the graphs we generated.

Following the layout, we use Dash callbacks to generate a different graph depending upon which tab we click. More information about Dash's core components, including "Graph" and "Tabs" can be seen in the Dash documentation here.

At this point, you can run report_dashboard.py in your terminal and watch Dash spin up a localhost server for your newly created reporting application. Follow the link in the terminal, and VOILA! You've created your own interactive, custom report for free.

You can check out a live hosted example or the full source code on GitHub.

Conclusion

In this article we discussed what Dash is, and how you can use it to create interactive reports from your headless CMS data. Now you know with a little Python knowledge, and an open-source framework like Dash, you don't have to miss out on having headless CMS reports.

Want more Dash and headless samples? Check out this Dash + Kontent app for making modular content graphs.

Are you having issues with the samples from the article? Get in touch on our Discord.

Can you evolve your documentation from fossil to fantastic?

Michael Berry — Tue, 22 Dec 2020 13:17:57 +0000

We can all agree that repeating the same information over and over wastes time and that writing comprehensive and accessible documentation is essential. But have you thought about how you’re creating that documentation? Are you still dusting off printed instructions or digging up ancient word documents? Is there a more modern approach?

In my opinion: "Boilerplates" are beautiful. They make using a new framework, language, or platform possible without starting from scratch, and can vastly simplify starting a new project. To simplify my documentation creation process, I've made a Kontent + Docusaurus boilerplate that will handle styling, navigation, and structure, while also allowing me to maintain my content using a headless CMS.

In this article, I’ll talk about the benefits of using Kentico Kontent, Facebook’s Docusaurus, and my Kontent + Docusaurus boilerplate to create scalable, attractive, easy-to-manage documentation.

Why my documentation went “the way of the dinosaurs”

I wear a lot of hats in my organization: I’m a manager, support engineer, content creator, internal trainer, to name a few. Shifting attention between varying tasks and functions is a common occurrence and can be pretty overwhelming. To save time (and my sanity), documenting my processes and responsibilities helps me stay organized and allows me to delegate tasks to my team more effectively. But until recently, I didn’t have a great way to create this resource. I’ve used our crowded company wiki, Word documents, PowerPoint presentations, and a lot of information is given verbally from memory. Something had to change as this approach locked me into a perpetual loop of updating static presentations and documents with no hope of delegation.

How does it help you?

Maybe you’re not managing a team or on-boarding anyone, but I’m confident at some point you’ve had to walk a teammate or multiple colleagues through a process a few times. Let’s be fair: your colleagues don’t want to take valuable time away from you. They just need answers, and those answers happen to live in your head, are hiding in your notes, or are just not documented in an accessible location.

Furthermore, updating and maintaining documentation or guides doesn’t have to (and normally shouldn’t) rely upon developers, specialized, or senior team members. You should be able to write the original copy, hand-off the maintenance to a colleague or team, and move on to other tasks that are burning for your attention.

The boilerplate: Our cloning DNA

Creating a new piece of documentation shouldn’t be an ordeal; you don’t need another project vying for your time. Using a template as a starting point is a good way to make sure starting the documentation isn’t a major line item blocking out your calendar. So clone this repository and save time—no prehistoric mosquitoes required (you knew a Jurassic Park reference was coming).

This boilerplate comes with:

A navigation sidebar with collapsible sub-sections
Attractive pre-made styles, including light & dark mode
A minimal amount of content to act as a “starting point”
Quick and easy setup instructions in the Boilerplate’s documentation or GitHub README

Kontent + Docusaurus boilerplate pictured above.

The headless CMS: Advancing our reptile brains

When documentation is required, jumping into a simple word processor is second-nature to a lot of us. It’s comfortable, it’s quick, it’s familiar. This doesn’t mean it is the best approach for the job, though, and I argue that breaking this habit is key for making scalable and accessible documentation. Using a headless CMS addresses scalability head-on and future-proofs your documentation.

I work in Kentico Kontent all the time, so it is a go-to comfort pick for me when choosing a CMS, but those who aren’t fluent in Kontent will still reap major benefits from using it:

It’s a Software-as-a-Service platform, meaning no managing upgrades, hosting, or security patches.
Kontent’s search bar, filters, and “Home” allows you to quickly locate content.
Inserting code snippets into the Rich Text Editor using a WYSIWYG editor is easy.
Reusable content and content links mean no more copy-pasting swaths of documentation between sections.
Its features, API, and SDKs are well documented.

Sample content item in Kentico Kontent pictured above.

These are just a few of the benefits when using Kentico Kontent, but if you need more proof, the official Kontent documentation uses Kontent, and there is a two-part article series that talks about why.

Docusaurus: More Barney than an apex predator

In addition to the list of benefits from Docusaurus’ website, the plugins and presets provided by the Docusaurus team make getting up and running fast. Its creators’ focus on documentation sites, along with 20,000 GitHub stars and being backed by Facebook, is what really drew me to using Docusaurus as the front-end application for my documentation. It’s also a nice fit because:

It’s a static site generator making it fast and secure.
It can be deployed and hosted virtually anywhere for free.
The Docusaurus team wrote easy-to-read documentation about everything from Installation *to *Lifecycle APIs.

This React and MDX markdown powered project has T-Rex-level power in its extensibility and out-of-the-box functionality, but the features and configuration are intuitive, so its learning curve isn’t intimidating. For example, in just three terminal commands you can have their classic preset site running. Docusaurus is definitely a friendly dinosaur.

So...is it actually fantastic?

For me personally, I’m a lot happier to point a colleague to up-to-date stylized on-boarding resources than having to muddle together PowerPoint presentations, but this depends on your use case. If you just need a few quick, static paragraphs to document something, Kontent + Docusaurus might be overkill. But, if you have living documentation that needs frequent updates, reusable content, and structured navigation, it could be a better choice than passing around Word documents or jamming information into overflowing wikis.

Additionally, since the content is available over an API in a headless CMS, this documentation can evolve with the times; Create a voice-app that pulls the documentation from Kontent, integrate personalization to better serve the visitor, add analytics or commenting with machine learning to gauge the helpfulness of a page. The possibilities and complexity can scale with the project needs. I’d say that’s pretty fantastic!

Kentico Kontent architecture possibilities pictured above.

Can I see this evolution in action?

Absolutely! The documentation explaining how to set up the Kontent + Docusaurus Boilerplate was written using Kontent and Docusaurus. You can also set up your own project using the instructions in the Kontent + Docusaurus Boilerplate GitHub repository.

Don’t wait for a meteorite

In this article, we highlighted the benefits of replacing old documentation with Kontent and Docusaurus and how a boilerplate can streamline your documentation creation. So don’t wait for a cataclysmic event, blow up your ancient documentation habits, and give Kontent and Docusaurus a try!

Machine learning: How to ensure labels are smart, not lucky

Michael Berry — Fri, 12 Jun 2020 13:55:02 +0000

Without labels and categories organizing our lives, the world would be chaos. However, what you consider to be a “sensible label” for something may not be the term others would use, and choosing the “correct” terms turns into a game of chance. So how can you guarantee that your project’s content is organized with appropriate and consistent categories?

In this article, I'll demonstrate how to use ML.NET to automate adding taxonomy terms (tags) to content items in the headless CMS Kentico Kontent.

Let’s turn that game of “lucky labeling” into a smarter solution!

No data science or machine learning degree required

I was initially intimidated by the algorithms, statistics, and other mathematical complexities surrounding creating machine learning models, but the fears were quickly dispelled when I discovered Microsoft’s ML.NET Model Builder Extension for Visual Studio. This extension uses automated machine learning to find the best machine learning algorithm for your scenario and data set. Once it chooses the best algorithm, you’re able to select that algorithm, train your model, and evaluate it all within a graphical user interface. The big takeaway: It requires zero machine learning/data science experience.

When is ML.NET categorization helpful in a Kontent project?

Applying machine learning for content categorization is helpful if:

You have similar taxonomy terms in your project and editors are struggling to decide which term best fits their content.
There are a lot of taxonomy terms in the project, and you need to identify which ones can be removed.
You have a large data set and need to identify what terms you should add to your project.

We'll focus on scenario #1: You have similar taxonomy terms in your project, and editors are struggling to decide which term best fits their content.

Categorizing the Chaotic Netflix Catalog

Let's take a look how ML.NET can automate content categorization. We'll aim to automatically suggest Netflix categories for movies based upon their titles, descriptions and ratings using a .NET Core 3.1 console application. Once the application is working, we'll extend the functionality to the headless CMS. For the sake of simplicity, I’ve focused on a non-hierarchical labeling structure where one taxonomy term is assigned to a movie, and have broken the steps down into the following stages:

Training the model
Integrating the headless CMS
Analyzing the content
Importing the suggested term

Training the model

The first step for creating a ML.NET model is to find a suitable dataset in the supported SQL Database, CSV, or TSV formats. We'll download a CSV file. We need to download and install Microsoft’s ML.NET Model Builder Extension for Visual Studio. Once downloaded, open the console application in Visual Studio, right-click the project, and use “Add” to add “Machine Learning” to the project.

After machine learning has been added, the extension provides six machine learning templates to choose from. Per Microsoft’s scenario examples and descriptions, “Issue Classification” best fits since we are trying to predict labels in 3+ categories, similar to Microsoft's GitHub Issue Classification example.

The next steps are for training the model. Select the CSV mentioned earlier, select the "listed_in" column as the column to predict, and choose the "description", "rating", and "title" columns the model should base its suggestion upon.

Click the “Train” button at the bottom of the screen. This opens a screen to specify how long to train the model. Microsoft has some training recommendations here: https://docs.microsoft.com/en-us/dotnet/machine-learning/automate-training-with-model-builder#how-long-should-i-train-for. In my project, I chose 120 seconds and it managed a 57.44% accuracy. Clicking “Evaluate” allowed me to test the model against some of the data in the CSV and broke down the top 5 prediction results.

After a few tests, I wasn’t happy with the 57% accuracy results, suggesting that the Microsoft training time recommendation was too short for the seventeen different classification categories in the CSV. That inspired me to re-run the training for 1 hour, which resulted in an improved ~66% accuracy, so don't be afraid to increase the training time.

Click the “Code” button to generate and add two machine learning projects to the project's solution. Once the ML projects are there, consume the model using:

// Add input data
var input = new ModelInput();
input.Title = "The Nightmare Before Christmas";
input.Description = "Jack Skellington, king of Halloween Town, " +
                    "discovers Christmas Town, but his attempts to bring Christmas to his home causes confusion.";
input.Rating = "PG";

// Load model and predict output of sample data
ModelOutput result = ConsumeModel.Predict(input);
Console.WriteLine(result.Prediction);

Run the console application, and it will correctly produce the output “Children & Family Movies,” demonstrating that the ML.NET model is ready to be linked to content in a headless CMS.

Integrating the headless CMS

In a real-world scenario, the project may exist before the need for “Smart” labeling arose, but in our case the demand for machine learning and a headless CMS was driven by some new, fancy Netflix data.

To set up Kontent, create a new project that will contain a single content type named "Movie" and a single taxonomy group called "Listed in." The content type and taxonomy group consist of elements and terms from the CSV used to build the machine learning model:

Title: a simple text element
Rating: a multiple-choice element contains all 11 content ratings (rated TV-G to rated R) in the CSV
Release Date: a date & time element
Description: a rich text element
Listed in: a taxonomy group containing all 17 of “Listed in” terms in the CSV

Once the content type and taxonomy terms are present, start making some sample content item variants using not-yet-released movie titles, ratings, and descriptions. I leave them in the “Draft” workflow step to ensure you are able to upsert data to the items from the console application.

Analyzing the content

Now it is time to pull that content from Kentico Kontent and feed it into the console application to make a prediction. For this, install the Kentico Kontent .NET SDK NuGet package, create a class to store your API keys, and a MovieListing Class that returns a strongly typed Movie:

//file location: MLNET-kontent-taxonomy-app/Configuration/KontentKeys.cs
public class KontentKeys
    {
        public string ProjectId { get; set; }
        public string PreviewApiKey { get; set; }
        public string ManagementApiKey { get; set; }
    }

//file location: MLNET-kontent-taxonomy-app/MovieListing.cs
class MovieListing
    {
        IDeliveryClient client;

        public MovieListing(KontentKeys keys)
        {
            client = DeliveryClientBuilder
                .WithOptions(builder => builder
                .WithProjectId(keys.ProjectId)
                .UsePreviewApi(keys.PreviewApiKey)
                .Build())
                .Build();
        }       

        public async Task<DeliveryItemListingResponse<Movie>> GetMovies()
        {
            DeliveryItemListingResponse<Movie> response = await client.GetItemsAsync<Movie>(
                new EqualsFilter("system.type", "movie"),
                new ElementsParameter("title", "rating", "description", "listed_in")
                );

            return response;
        }       
    }

//file location: MLNET-kontent-taxonomy-app/Models/Movie.cs
public partial class Movie
    {
        public const string Codename = "movie";
        public const string DescriptionCodename = "description";
        public const string ListedInCodename = "listed_in";
        public const string RatingCodename = "rating";
        public const string ReleaseDateCodename = "release_date";
        public const string TitleCodename = "title";

        public string Description { get; set; }
        [JsonProperty("listed_in")]
        public IEnumerable<TaxonomyTerm> ListedIn { get; set; }
        [JsonProperty("rating")]
        public IEnumerable<MultipleChoiceOption> Rating { get; set; }
        [JsonProperty("release_date")]
        public DateTime? ReleaseDate { get; set; }
        public ContentItemSystemAttributes System { get; set; }
        [JsonProperty("title")]
        public string Title { get; set; }
    }

Instantiate this class from the main Program.cs, set and pass the API keys to it, and return a DeliveryItemListingResponse to loop through. Set up the ML.NET consumption logic in a separate class for easier maintainability and a cleaner main program:

//file location: MLNET-kontent-taxonomy-app/TaxonomyPredictor.cs
//generated using: https://github.com/Kentico/kontent-generators-net
class TaxonomyPredictor
    {
        public string GetTaxonomy(Movie movie)
        {
            // Add input data
            var input = new ModelInput();

            input.Title = movie.Title;
            input.Rating = movie.Rating.ToList().First().Name;
            input.Description = movie.Description;

            // Load model and predict output of sample data
            ModelOutput result = ConsumeModel.Predict(input);
            Console.WriteLine("Listing best match: " + result.Prediction);

            //formatting to meet Kontent codename requirements
            //ex: Children & Family Movies => children___family_movies
            var formatted_prediction = result.Prediction.Replace(" ", "_").Replace("&", "_").ToLower();

            return formatted_prediction;
        }
    }

Instantiate a "predictor" from Program.cs. The TaxonomyPredictor.GetTaxonomy(Movie) method can now be used to suggest “listed in” terms when looping through the list of movies returned by MovieListing.GetMovies().

//file location: MLNET-kontent-taxonomy-app/Program.cs

MovieListing movieListing = new MovieListing(keys);
TaxonomyPredictor predictor = new TaxonomyPredictor();

var movies = movieListing.GetMovies();

foreach (Movie movie in movies.Result.Items)
{
    if (movie.ListedIn.Count() < 1)
    {
        string formatted_prediction = predictor.GetTaxonomy(movie);

        Console.WriteLine(formatted_prediction);
    }
}

This will produce the “best match” taxonomy term in the console when run.

Importing the suggested term

Finally, it is time to automate upserting the suggested taxonomy terms to the headless CMS. Use the Kentico Kontent .NET Management SDK in a separate class called TaxonomyImporter:

//file location: MLNET-kontent-taxonomy-app/TaxonomyImporter.cs
class TaxonomyImporter 
    {
        ManagementClient client;

        public TaxonomyImporter(KontentKeys keys)
        {
            ManagementOptions options = new ManagementOptions
            {
                ProjectId = keys.ProjectId,
                ApiKey = keys.ManagementApiKey,
            };

            // Initializes an instance of the ManagementClient client
            client = new ManagementClient(options);
        }

        public async Task<string> UpsertTaxonomy(Movie movie, string listing_prediction)
        {

            MovieImport stronglyTypedElements = new MovieImport
            {
                ListedIn = new[] { TaxonomyTermIdentifier.ByCodename(listing_prediction) }
            };

            // Specifies the content item and the language variant
            ContentItemIdentifier itemIdentifier = ContentItemIdentifier.ByCodename(movie.System.Codename);
            LanguageIdentifier languageIdentifier = LanguageIdentifier.ByCodename(movie.System.Language);
            ContentItemVariantIdentifier identifier = new ContentItemVariantIdentifier(itemIdentifier, languageIdentifier);

            // Upserts a language variant of your content item
            ContentItemVariantModel<MovieImport> response = await client.UpsertContentItemVariantAsync(identifier, stronglyTypedElements);

            return response.Elements.Title + " updated.";
        }
    }

Instantiate the importer in Program.cs to use the UpsertTaxonomy method when looping through the list of movies. Create a strongly typed MovieImport model that inherits from Movie in order to accommodate what the Kentico Kontent Management SDK expects when upserting content item variants:

//file location: MLNET-kontent-taxonomy-app/Models/MovieImport.cs
public partial class MovieImport : Movie
    {
        [JsonProperty("listed_in")]
        public new IEnumerable<TaxonomyTermIdentifier> ListedIn { get; set; }
        [JsonProperty("rating")]
        public new IEnumerable<MultipleChoiceOptionIdentifier> Rating { get; set; }
    }

Once this is done, add logic to use environment variables, appsettings.json, and configuration binding using a combination of .NET Core configuration options make your API keys hidden and secure! Below is the completed Program.cs file, configuration files, and result of running the program:

//file location: MLNET-kontent-taxonomy-app/Program.cs
class Program
{
    static void Main(string[] args)
    {
        Console.WriteLine("Starting program...");

        var environmentName = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");

        var configuration = new ConfigurationBuilder()
            .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
            .AddJsonFile($"appsettings.{environmentName}.json", optional: true, reloadOnChange: true)
            .Build();

        //file location: MLNET-kontent-taxonomy-app/Configuration/KontentKeys.cs
        var keys = new KontentKeys();

        ConfigurationBinder.Bind(configuration.GetSection("KontentKeys"), keys);

        MovieListing movieListing = new MovieListing(keys);
        TaxonomyPredictor predictor = new TaxonomyPredictor();
        TaxonomyImporter importer = new TaxonomyImporter(keys);

        var movies = movieListing.GetMovies();

        foreach (Movie movie in movies.Result.Items)
        {
            if (movie.ListedIn.Count() < 1)
            {
                string formatted_prediction = predictor.GetTaxonomy(movie);
                var upsertResponse = importer.UpsertTaxonomy(movie, formatted_prediction).Result;

                Console.WriteLine(upsertResponse);
            }
        }

        Console.WriteLine("Program finished.");
    }       
}

//file location: MLNET-kontent-taxonomy-app/Configuration/KontentKeys.cs
public class KontentKeys
    {
        public string ProjectId { get; set; }
        public string PreviewApiKey { get; set; }
        public string ManagementApiKey { get; set; }
    }

//file location: MLNET-kontent-taxonomy-app/appsettings.json
{
  "KontentKeys": {
    "ProjectId": "<YOUR PROJECT ID>",
    "PreviewApiKey": "<YOUR PREVIEW API KEY>",
    "ManagementApiKey": "<YOUR MANAGEMENT API KEY>"
  }
}

Imagine the possibilities

In this article, I solved inconsistent content tagging using the ML.NET Model Builder, a .NET Core console application, and a Kentico Kontent project.

In my project, I focused on a flat taxonomy structure and straightforward classification scenario, but this is just one of almost endless machine learning possibilities. For example, combining machine learning with webhook notifications will allow automatic improvements to created content. With this combination, it's possible to automatically write descriptions for uploaded assets or suggest the best SEO keywords for blog posts.

You can find the full source code for my application, supporting data files, and instructions on how to test your own copy of the project here:

Get the code

I hope this inspired you to stop being lucky, and start being consistent using machine learning! 😃

How to Keep It Stupid Simple: Use .NET Core Razor Pages

Michael Berry — Thu, 26 Mar 2020 13:07:43 +0000

With recent improvements to the .NET platform and the ever-evolving CMS market, guidance on creating feature-rich websites is available at every turn. But what if you don’t have time for complicated tutorials and just need a modern, simple, easy-to-manage site?

With the release of .NET Core 3.x, advances in ASP.NET Core MVC and the emergence of Blazor preview, there is a lot of excitement in the .NET community. Although these improvements are wonderful, the robustness of MVC and the newness of Blazor may make some projects feel over-complicated. In addition to development complexity, accounting for CMS installations and upgrades can add extra overhead to a project. In the following article, I’ll discuss how ASP.NET Core Razor Pages and Content as a Service (CaaS) can help prevent overhead and complexity. If you’re a strong believer in the KISS principle like I am, I hope this article shows that your .NET development can be stupid simple!

Why Razor Pages?

Straightforward structure

ASP.NET Core Razor Pages takes a page-focused approach to the project structure. It co-locates a page’s view and its PageModel (logic pertaining to a view) in a “Pages” directory. If you’ve modeled your content and identified that the majority of it is structured around the concept of a “page,” then Razor Pages may be the perfect framework for your project.

For example, in my sample coffee shop project, I have a lot of basic informational pages about coffee and cafes. If I wanted to make changes to the “Cafes” page, I’d navigate to ~/Pages/Cafes/Index.cshtml to edit the view or ~/Pages/Cafes/Index.cshtml/Index.cshtml.cs to edit the logic.

This single-responsibility-style structure makes finding the code for specific pages less difficult since pages are found in the “Pages” folder.

Familiarity

If you’ve ever used Web Forms, the above description of layout and logic being colocated should sound very familiar. Razor Pages is sometimes viewed as the successor to Web Forms due to its structure, but it comes with modern features of .NET Core. So if you’ve been dreading the transition from Web Forms to MVC in order to use .NET Core features, I’m sure Razor Pages will be a welcoming experience.

Options

Routing can be very confusing or prone to errors. This being said, instead of using MVC’s less straightforward controller -> action -> view approach, default routing in a Razor Pages project uses the location of a page within the project to determine its URL.

For example, a request to “http://domain/Articles” will be handled by ~/Pages/Articles/Index.cshtml and its respective .cs file, while a request to “http://domain/articles/detail” will be handled by ~/Pages/Articles/Detail.cshtml and its .cs file.

Although this behavior helps simplify routing, it doesn’t prevent you from having custom routes. For example, in my sample project, I opted to use an @page directive to append a segment to my article URLs in order to return a specific article and present a practical address to visitors.

In the Articles/Detail.cshtml view, the directive @page “{UrlPattern}” is used.

This UrlPattern value is used in the page’s logic to return content from my headless CMS’s API. We’ll discuss my choice of Content-as-a-Service/headless CMS choice later in this article, but for now, rest assured that the @page directive allows me to add a segment to a URL such as “http://domain/Articles/Detail/coffee-processing-techniques.”

In addition to page directives, Razor Pages projects can use AddRazorPages and AddRazorPagesOptions on the service collection in the Startup class. This opens up a lot of routing possibilities described in the official Microsoft documentation for Razor Pages routing. To showcase this, I used custom routing for my homepage. I wanted to organize the homepage in a sub-folder similar to the other pages in my site, and I didn’t want “/Home” in the URL. This isn’t a complicated routing scenario, but you can see how this approach allows for more flexible routing.

Although I’ve focused on the page-centric structure thus far, that doesn’t mean that your project is limited to pages. Functionality that is distinct from pages or that can be reused across multiple pages can be implemented using ViewComponents. For example, in my project, I created a Components folder within Pages that contains the logic and view for displaying contact information. To use this component in my site’s footer, I just call

C# @await Component.InvokeAsync("Contact")

in my shared layout.

Although this adds a folder within Pages that isn’t directly related to any given page, this component is responsible for rendering content to pages, so it still seems sensible to have it in this location.

These are a few examples of how Razor Pages can allow developers to reap the numerous benefits of modern features provided in .NET Core while also keeping their project as simple as possible. This being said, writing code is only a fraction of the effort it takes to create a great website. How can the theme of “simplicity” extend to the content management of the project?

Why CaaS?

For starters, what is “CaaS?” Content as a Service is an approach to creating content in a Software as a Service content management system and delivering that content over multiple channels using an API. You might be thinking, “That sounds a lot like a headless CMS...” Exactly! The CaaS approach to content creation leverages a headless CMS but also includes various services, bundles, or offerings in addition to giving editors a CMS and developers an API. For the sake of this article, you can substitute “CaaS" for “headless CMS” if it better fits your experience.

No CMS maintenance

If you’ve ever managed an on-premise CMS or have been involved in CMS deployment and maintenance, you know that it comes with a fair amount of overhead. Installations, managing maintenance windows, doing upgrades, applying bug fixes, upgrading infrastructure... The list goes on and on. These tasks take time away from development and content creation. With a Content-as-a-Service offering, the vendor takes care of the CMS and infrastructural maintenance, which allows you to focus on development.

Omnichannel

Even though we’re trying to keep our projects simple, the world is not a simple place. Customers have complex needs. In 2020 and beyond, customers are going to expect their content to be available on websites, mobile applications, voice assistants, and other channels. By creating your content in a system or service that is platform agnostic and can be delivered to any channel over an API, you’re ensuring the future expansion of your project to these other mediums is possible.

Structured content

In addition to the omnichannel benefits, working with CaaS makes development easier by ensuring content is structured. It’s a common practice for CaaS products to send API responses in either JSON or XML format. This streamlines the development process by making content returned from the API predictable and consistent across the project. If you’ve done proper content modeling in the early stages of development, there shouldn’t be any guessing involved when using a response from your CaaS API. Minimizing guesswork with uniform, structured content delivery will definitely make the development process smoother.

Why Kentico Kontent?

For my sample project, I chose to use Kentico Kontent as my CaaS offering. In addition to meeting all of the requirements listed in the Why CaaS? section above, Kentico Kontent’s open-source .NET SDK and supporting tools further improve the development experience.

.NET SDK

In 2020, having a clean API is not the pinnacle of a pleasant developer experience. I’d argue that many developers leveraging an API service jump into SDK documentation almost as quickly as the base API documentation itself. With this in mind, Kentico Kontent’s .NET SDK has sensible naming conventions, extensive documentation, and dependency injection support that makes setting up the SDK and querying content a breeze. For example, other than the code related to dependency injection and strong typing, my call for retrieving an “About Us” item from Kentico Kontent is a single line of code:

This code returns a structured JSON response containing all the details about my “About Us” content item that I created in the Kentico Kontent user interface, then casts it to my AboutUs model.

Setting up the application to use dependency injection with the SDK can be accomplished in a single service registration in the Services.cs file that uses a standard configuration in an appsettings.json file:

Startup.cs

public class Startup
{
    public Startup(IConfiguration configuration)
    {
        Configuration = configuration;
    }

    public IConfigurationRoot Configuration { get; }

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddDeliveryClient(Configuration);
    }
}

appsetings.json

{
    "DeliveryOptions": {
        "ProjectId": "<Kontent_ProjectId_Value>"
    }
}

Although I’ve focused on a minimal setup in the above examples, don’t worry! If your project needs more advanced functionality, the Kentico Kontent .NET SDK has lots of features and options, making it scale comfortably with increasing demands or complexity.

Model generator

When using a Model-View-ViewModel design pattern like Razor Pages, creating and updating models can be a considerable amount of upfront and ongoing work, especially if your content creators or project managers change the content structure or requirements. To account for this, Kentico Kontent has an open-source tool that automatically generates strongly-typed models for your project. If you rely on strongly-typed models for IntelliSense, hate manually editing models, and don’t mind using a CLI tool to take on this otherwise tedious task, you’re going to love that Kentico Kontent provided the Kontent model generator for .NET projects.

Not just for developers

In addition to the development perks, a headless CMS will benefit editors and their managers too. If you’re interested in content creation features, check out this page, or you can try it out for yourself. No manuals required!

Summary

In this article, we discussed what aspects of ASP.NET Core Razor Pages make it a great framework for creating simple, but modern websites in both structure and feature sets. We also described how pairing Razor Pages with a CaaS offering like Kentico Kontent can further reduce development overhead. I hope that demonstrating my sample project showed you how we can keep it simple in 2020 and beyond!

Lessons Learned: Using a Headless CMS without an SDK

Michael Berry — Thu, 21 Nov 2019 14:12:11 +0000

How complicated is it for a beginner to implement a headless CMS powered site without an SDK? I chose Python and Flask to build a section of a sample site. Did I succeed? And what did I learn in the process?

Why Python and Flask?

Earlier this year, I found myself spending a lot of time manually grinding through administrative tasks that felt repetitive, time consuming, and tedious. I began to search for the best way for a non-developer to start automating tasks. After a few days of browsing, I noticed that I always seemed to end up on a blog, forum, or community post talking about Python. I picked up a few books, took an online introductory course, and started writing scripts to help me with my work.

After completing some basic tutorials, I wondered if I could somehow connect these newly learned skills to the Content-as-a-Service product that my team supports, Kentico Kontent. Furthermore, could I do it without an SDK since Kontent does not currently have one for Python? To test this, I decided to recreate the articles section of the Dancing Goat sample project that comes with new Kontent subscriptions and catalog some of the lessons I learned from the project.

Learn the Terminology

If you've decided to use Kentico Kontent for your project, reviewing and bookmarking the terminology we use throughout the documentation and product UI is an important step. If you're diving into a project and haven't established a firm grasp on the differences between content types, content items, and components, then the Kentico Kontent documentation won’t deliver its true value.

Lesson learned: taking the time to “learn the lingo” makes working in the Kentico Kontent ecosystem easier.

Establish Your Project Scope

Scope creep has always been an issue of mine, and I don't think I'm alone. When I began this project, I naively wanted to recreate the Dancing Goat site in its entirety while leveraging all of Kentico Kontent’s features. This wasn't an effective plan.
Although no Kentico Kontent features are too difficult to implement, starting small and focusing just on the home page and articles in the sample site gave me the best opportunity to learn. I could see the API in action, learn about the Flask microframework, and understand how content modeling was done in the sample project.

Lesson learned: starting small and then scaling up prevents the project from becoming overwhelming and provides the opportunity to learn the Kentico Kontent basics.

Structure Your Content

Knowing how the content types and content items connected to each other in my sample project saved me a significant amount of time. Since mapping out the home page and article relationships was so useful in a pre-generated project, I know it will be even more beneficial for a new project.
I've seen a fair share of real customer projects that have redundant entries because they didn't consider the relationships between their content prior to development. This section on content modeling from the Kentico Kontent documentation will likely be helpful in avoiding this pitfall.

Lesson learned: prevent duplicative content by understanding what is reusable and which content is related.

Create Sample/Starter Content

I remember feeling a bit overwhelmed by the size of the Delivery API responses from my project. This is because I was using the sample site which has a lot of content items and linked items. The sample site is a great reference tool, but may be more complex than most projects need to be.
Building out a minimal set of content types and items related to the core functionality in your project can save time when developing your front-end application. Creating only a few content items in the project will make reading and using the JSON responses provided by the Delivery API more manageable. You'll also have an easier time remembering the codenames of your content items, which you will be using a lot during testing. Keep it simple, then scale up.

Lesson learned: only build a few content items of each type during the early phases of the project to make analyzing JSON responses more straightforward.

Identify the Necessary API Requests

I started using the POSTman REST client to test and document the GET requests I needed to make in my project. This was immensely helpful since I needed to construct requests in my front-end application using a Python library. It can also be helpful in other scenarios since:

You'll familiarize yourself with the Delivery API’s response structure.
You'll be more aware of the API's filtering capabilities.
You can more easily determine if there is an issue with your code, the content, or our API endpoint during debugging. By running tests in a REST client, you separate your code from our product.

Lesson learned: understanding the API calls in advance provides a helpful roadmap for building a front-end application and helps during debugging.

Decide If You Are Going to Use an SDK

I knew there wasn't a Python SDK for Kentico Kontent. I needed to find documentation on how to call API endpoints using Python and Flask. This may not be your situation, but it would be a shame if halfway through a JavaScript, .Net, PHP, Ruby, or Java project, you or your developers stumbled upon an SDK that would have made your lives a lot easier.

For projects that will use an SDK, it is important to review the SDK documentation, stay up to date on current GitHub issue submissions, and check release notes for the chosen SDK. The SDK GitHub repositories and their accompanying documentation are updated frequently, including breaking change documentation when a new version of an SDK is released.

Lesson learned: collect and review relevant documentation before starting your project, and watch the GitHub repositories that you are using for your project.

Understand the Conventions in Your Selected Framework/Programming Language

Let's be honest, I started hacking away trying to get my front-end application to work. The approach should have been to explore Flask and Python conventions first. Understanding what options I had for structuring my application, determining routes, error handling capabilities, and configuration file conventions should have come sooner rather than later in the project.

I'm sure Flask and Python purists could have a field day critiquing my sample project, but when I started applying some conventions, my application started to become more manageable. If I had known this, I could have saved a lot of time!

Lesson learned: learning about framework conventions prior to development makes the project easier to navigate and manage.

Formulate a Resolver Strategy (for Types and Inline Items)

Early on, it was clear that not using strongly typed models made my project very messy and hard to read. Once I implemented my own type helper, similar to the CustomTypeProvider in the Kentico Kontent .Net sample site, I was able to improve the readability of my code. To help with strongly typing models, Kentico created model generators for the .Net, JavaScript, and Java SDKs. This didn't help me with my Python project, but it is good to know that these tools are available for projects that use the SDKs.

In addition to strongly typing models, you'll want to resolve linked items, components, and text links that appear in Rich Text elements. In my project, I had to create a custom resolver. This resolver scrapes the content in my Rich Text elements for object and anchor tags and replaces the references with actual links or content. This seems confusing at first, but after spending some time looking at the JSON returned by the API for individual content items, you'll see that Rich Text elements come with accompanying “links” and “modular_content” values that can be used for inline link and item resolution.

Luckily, the Kentico Kontent documentation has a section on dealing with structure in rich text, and there are built-in methods in the Kentico Kontent SDKs that handle this resolution for the most part. If you decide to use an SDK, a resolver strategy will be more straightforward. Otherwise, spend some time looking at the structure of the API's JSON responses for linked items and rich text.

Lessons learned: plan on implementing strongly typed models from the beginning of the project, and review how inline content items appear in the Delivery API responses.

Start Building

I hope that after reading about the steps that helped me with my project, you feel more prepared to take on a new Kontent project of your own. As a Python and programming novice, I was able to get a Kentico Kontent blog up and running without an SDK, so I am sure you will create some killer apps that enable a flawless content strategy. You can check out my project here on GitHub, and feel free to leave comments and suggestions.
If Vue.js is more interesting or relevant to you, my colleague Chris Jennings also wrote an excellent "Lessons learned" blog post that may be helpful.