Forem: Bulent Osman Yusuf

Create a bot with RSS and webhooks for Mastodon, a Twitter alternative

Bulent Osman Yusuf — Tue, 22 Nov 2022 14:07:09 +0000

Looking for a Twitter alternative? Here's two ways to set up a bot to automatically publish content from your Contentful space to your Mastodon server.

NB: This post was first published on the Contentful Blog.

In recent weeks there’s been a huge surge in interest for an alternative to Twitter. Whether folks want to stretch their wings by learning about new social media platforms, or they’re looking for a simple Twitter replacement, an open source service called Mastodon is trending as the clear favorite.

But there are things to take into account when using Mastodon for the first time. It has its own rules, vocabulary, and ways of operation. Tweets are now “toots.” You no longer have a universal timeline like Twitter, for example, but a thing called a federated timeline. And there’s even an edit button!

In this post, we’ll take a quick tour of Mastodon – and, by extension, a mysterious place called the fediverse (oooooooh). We’ll also show you two simple ways to set up a bot to automatically publish content from your Contentful space to your Mastodon server of choice. Ready? Let’s go!

What is Mastodon?

The Mastodon project first went live in October 2016, and is the creation of software engineer Eugen Rochko. The mascot of the service is an adorable wooly mammoth. The concept is a decentralized social network built from free and open source software, where individuals have the ability to quickly spin up and self-host their own social networking services – while still being able to interact and communicate with one another.

Growth in the past few weeks has been absolutely stellar. According to a toot by Eugen on 7 November, “Hey, so, we’ve hit 1,028,362 monthly active users across the network today. 1,124 new Mastodon servers since Oct. 27, and 489,003 new users. That’s pretty cool.”

Cool is an understatement!

What about the Fediverse?

Mastodon as a service embodies all the characteristics of a fediverse, which is shorthand for federated universe; a set of distinct but interconnected web publishing servers (also known as a node or instance) sharing the same protocols for communication. You have the option to follow and interact with people within your own server (your local timeline) but also with the denizens of neighboring servers which might be dedicated to different interests and specialisms (a federated timeline).

For example, Person A could start a social media node using the Mastodon protocol dedicated to lovers of apples, while Person B could start one dedicated to those who prefer oranges. Users of either social network can freely interact with each other, talk about the joys of fruit salad, and attract followers and comments from both places. But they can also choose to confine themselves to their own network about their preferred fruit. Either option is entirely valid.

Is it free to use Mastodon?

Yes. Mastodon is funded entirely by sponsors who cover the cost of operating individual servers or the salaries of volunteers running them. It’s ad-free and there are definitely no algorithms – no mathematical alchemy behind the scenes serving up content it thinks you might like – just a stream of toots that you’ll see posted in real time.

But there are enough similarities to Twitter to make it recognizable to users of that other service, for example, the use of hashtags, notifications, and followers. And you can access it from your desktop or use mobile apps available for iOS and Android devices, both of the official and unofficial variety.

Free speech is pivotal to Mastodon, but there are guardrails against hate speech and misinformation. The Mastodon Server Covenant is a list of server owners who pledge to provide active moderation against racism, sexism, homophobia and transphobia, plus several other user-first commitments on privacy and terms of service. This offers newcomers reassurance up front that they’re venturing into a safe space, free from trolls and fake news.

Let’s build a bot for Mastodon!

Now that you know a little bit about what makes this Twitter alternative so intriguing in its own right, let’s roll up our sleeves and build something for it!

We’re not ready to create a server of our own on Mastodon just yet, but let’s kick it off with functionality that’s smaller and simpler (but no less useful): a bot that automagically notifies the community when a new post is published to the Contentful blog.

This is a relatively straightforward process because Mastodon provides access to its data over a REST API, just like Contentful does. You have either the option of webhooks or RSS fields (or both) to tie it all together. With some simple alterations, you can follow these same steps to sync your content between other social or microblogging services like Twitter, Instagram, Tumblr, etc.

Wait, what’s a bot?

Did you know? More than half of all web traffic is generated by bots. Why? Because internet bots are able to perform tasks that are simple and repetitive much faster than a human ever could. Wikipedia has a good definition: a bot is a software application that runs automated tasks (scripts) over the internet, such as messaging or publishing content.

What’s a webhook?

Webhooks are unidirectional mechanisms that applications can use to talk to each other when a particular event occurs. Think of a webhook as an automated, one-way communication from one application to another. The first application sends the information to the receiver without waiting for a response. Then, it relies on the receiver to process this information. Read all about it in our ultimate guide to webhooks.

What’s an RSS feed?

RSS (RDF Site Summary or Really Simple Syndication) allows people to subscribe to newly published content via an RSS reader so that they don’t need to manually check websites or channels. An RSS feed takes the form of a standard XML (Extensible Markup Language) file, built of content and tags that define the content, and it looks a bit like HTML. XML is both human and machine readable. Check out the Contentful blog RSS feed here.

Step 1: Sign up for a Mastodon account and choose a server

Let’s start by signing up for an account with Mastodon. The first thing you need to do is pick a server. There are multiple servers available, filterable by topic (Animation! Astronomy! FOSS!) and region, and the ones on this index belong to the Mastodon Server Covenant.

You can always switch servers later if you change your mind. We’re going with mastodon.online, a newer server administered by Mastodon directly. This is the account we have created for the bot (which, being a bot, is limited only to republishing content from the blog and is not moderated by a human).

Step 2: Generate a token in Mastodon

Now that you have a Mastodon account and have familiarized yourself with the service, go to settings > profile > development and select “New Application.”

Fill out some fields describing your project, then, under “Scopes” only select write:statuses among all the checkboxes that you see.

Click “submit,” then a unique token will be generated. Hang on to this information, you’ll need it in a bit.

Step 3 / Option A: Create a webhook in Contentful

a) Sign up for a Contentful account

Our assumption for this tutorial is that you already have a Contentful account, together with a Space that’s already populated with a content model.

If you don’t already have one, then this is the place to go to sign up for an account. And if you want to quickly spin up a blog, we have a roundup of some useful templates here.

Come back when you’re ready to syndicate your content!

b) Create a webhook in Contentful

So, let’s summarize. Our goal is to notify our Mastodon account of changes within specific parameters, e.g. that we have added a new post to the Contentful blog.

The most straightforward approach is to create a webhook in your Contentful Space that communicates directly with Mastodon through the magic of APIs.

Go to your webhooks via your “Settings” menu.

You’ll see a list of your webhooks; create a new one using the “Add Webhook” button.

Now, this is where you’ll configure your webhook.

Name: Whatever you want! We chose “Contentful blog post to Mastodon.”

Webhook URL: This is the URL to the Mastodon API. Use the following format, replacing [YOUR_INSTANCE] and [YOUR_MASTODON_TOKEN] with the values from Steps 1 and 2.

[YOUR_INSTANCE]/api/v1/statuses?access_token=[YOUR_MASTODON_TOKEN]
Trigger: By default, “Trigger for all events” is selected. DO NOT leave it as it is. Change it to “Select specific triggering events” and make it trigger only when an entry is published. You can see what it looks like below.

Headers: You don’t need to add additional headers, but you need to change the “Content type” to application/json.

💡Heads up — you can do all the above steps to set up your webhook programmatically using the Contentful API. You can find the code to do this in this repo.💡

Finally, and this is a crucial step, you need to figure out what you will “toot” every time the webhook triggers.

Remember, we want to automatically send a toot every time an entry is published. You define the text of your toot by using a custom payload on your webhook.

This depends on your content model. In our example, this is what our content model looks like:

Based on our content model, this would be our payload:

{
  "status": "New post! {/payload/fields/title/en-US}. \n [YOUR_WEBSITE]/{/payload/fields/slug/en-US}"
}

The values between the brackets will be replaced by dynamic data from Contentful. Make sure to replace YOUR_WEBSITE with the URL of your blog, e.g., contentful.com/blog.

Step 3 / Option B: Use a third-party automation service

In some scenarios, you may prefer to sync up the two services using a third-party automation tool.

This might be because your content model is in the middle of being revised, for example, or environmental governance rules in your org won’t permit you to create webhooks willy-nilly. We get it, it happens.

For the purposes of this tutorial, we’ll go ahead and create an account with IFTTT, though Zapier would be equally effective.

We’ll need to specify two things for our project to work properly; a trigger and the action. Then we’ll top it off with some details for the web request.

a) This is the trigger

Within IFTTT, click on “Create” or visit https://ifttt.com/create, and follow the prompts to create a dedicated applet.

In the list of services, search for “RSS.”

Choose “New feed item” as your trigger.

Paste the URL of your RSS feed (ours is here), and then create the trigger.

b) And this is the action

Now onto choosing an action; this is what IFTTT will do whenever there’s a new item added to your RSS feed.

We’ll choose “webhook” to send the contents of our RSS entry to the Mastodon API.

Next, choose the “webhook action” to “Make a web request” (it’s the only option), because as we said earlier, we’ll be sending a request to the Mastodon API.

c) Make a web request

Just a few more things you need to add and then we’re done.

First, the URL that will receive the web request, it will be something like:

https://[YOUR_INSTANCE]/api/v1/statuses?access_token=[YOUR_MASTODON_TOKEN]
Then, change the “Method” to “POST” by selecting in the drop down.

Then, for the “Content Type,” select application/json.

Then, in the “Body” section, put a JSON object with the “status” property, that will define a template for what your toots will look like.

In our example, { "status": "{{EntryTitle}} -> {{EntryUrl}}" }, our toots will have a simple format of blog post title and then the URL.

Step 4: Testing, testing

To test your new bot, you should go ahead and publish a sample blog post.

Alternatively, you can live by the seat of your pants and wait until the next time you publish something for real...

Either way, if everything has been set up correctly, you should now have a working bot that fires off a toot every time a new post appears on your site.

Here’s a screenshot of our Contentful bot working its magic on Mastodon – and hey, it’s already got some followers!

Wrapping up: Is Mastodon the best Twitter alternative?

So there you have it. Now you know what Mastodon is, what the fediverse is, and how to tie it all together with your Contentful space.

Is Mastodon really the best Twitter alternative available? We’ll leave that to you to judge, but with this simple little project you can really get your teeth into this intriguing platform and evaluate it for yourself.

Get started! Start building with a free Contentful account, no credit card required. Sign up.

Developer Showcase Spotlight: Low-code examples of building blogs

Bulent Osman Yusuf — Thu, 06 Oct 2022 11:21:26 +0000

Wanna create a blog? Tools are readily available that make it super easy and fun. Contentful, Gatsby and Netlify are your building blocks for building blogs, and in this post I show a few examples.

This post was first published on the Contentful Blog.

My role at Contentful is to manage the company blog, and I love it. We have a dedicated team of developers, designers, writers, and editors all working together to deliver great content –– and I get to pester all of them on a daily basis.

But when the Developer Showcase was launched early this summer, it got me thinking about building a Jamstack site of my own; a sandbox to play in, learn something new, and occasionally break stuff.

The idea was simple: to create a basic blog from modular components, and to do it using existing resources without getting too deep into the code. I wanted to bring together standalone elements like a content management system, a static site generator and hosting provider through the magic of APIs. The grand result would be a site that was performant and secure.

In the end, I got a bit carried away and wound up creating several blogs plus a portfolio to bring them all together. The result is called Get Some Jam. And that’s what’s been so utterly delightful about this project.

Approaching it as a content editor searching for low-code solutions, I found many options for achieving my goal and couldn’t settle on just one. I’m going to tell you all about it in this post, so pull up a seat and make yourself comfortable.

First things first, set up a GitHub account

If you leave here with only one takeaway (other than the fact that Contentful is amazing and you should totally be using it for your next project), it’s that a GitHub account is your passport to many exotic locations on the magical electronic interwebs.

I used to think that GitHub was just a place where developers would store code and collaborate, but it’s a lot more than that. What’s also wonderful about GitHub is that an account functions as a universal identifier on the web, so you don’t need to create accounts with multiple services and then try to remember passwords for each of them. One GitHub account will grant you secure access to everything.

From my personal GitHub account, for example, I created and synced up free accounts with Contentful (content platform), Gatsby (static site generation and deployment), Netlify (deployment), and Vercel (deployment), and between those four entities I was able to build four different websites in about a week.

The only outlier to this process was Google, which has its own authentication protocol for their services. That came into play for this project when it came to buying and managing domains. But Google makes doing this so easy that I really can’t object.

Oh yes, and the OTHER useful thing about GitHub is that it’s your de-facto place for storing and delivering files that go into the upkeep of a website. Which leads us to my first example.

1. Building a low-code blog with Gatsby and Gatsby Cloud

The tech stack for this first blog is very simple and consists of two components: Gatsby and Contentful.

This is because Gatsby offers more than a framework for static site generation, there’s also the option of a fully featured deployment and hosting service called Gatsby Cloud.

So, my first port of call was the official GitHub repository where Contentful maintains a starter blog template built using Gatsby, which has push button deployment for Gatsby Cloud. This template is basic but properly formatted with all the necessary features of a functional blog. Things like an index page, formatting for individual posts and key visuals, plus timestamps, authors, and tagging.

It also comes with a preconfigured content model to populate your Contentful space, and sample posts you can delete later. Much of the heavy lifting is done already for those who are still learning the concept of structured content. My thanks to David Fateh for maintaining this great resource.

Bonus: How to add inline images

While adding some sample entries of my own to this blog, I found a setting that wasn’t to my liking. You’re not able to show inline images in the body of a post. This is because images are a Rich Text Asset that’s not defined (and therefore not queryable) in the content model when setting up the blog for the first time.

After a bit of Googling, I found some helpful blog posts articulating how to resolve the issue. One by Paul Scanlon over on the Gatsby blog and another by Andreea Macoveiciuc on Medium. After a bit of trial and error, I wedged an approximation of their code into my blog-post.js file in starter-gatsby-blog/src/templates/blog-post.js and now images are visible.

Here it is if you’d like to use it yourself:

body {
   raw
   references {
      ... on ContentfulAsset {
         contentful_id
         title
         description
         gatsbyImage(width: 1000)
          __typename
      }
   }
}

I did attempt to have this merged into the main Starter Gatsby Blog repo on GitHub, but it causes issues at startup because it’s an unexpected element in the schema. As long as you use this code after the blog has been generated you’ll be golden.

Would you like to build something more elaborate with Contentful and Gatsby? Check out this guide for a fully featured homepage starter.

2. Building a low-code blog with NextJS and Vercel

Moving on to something slightly more complicated — but not by much — the tech stack for this second blog is three components: Contentful, NextJS and Vercel. In this instance, we’re switching to a new framework and a new deployment service.

Again, this is a basic but functional blog, with a pre-populated content model which looks like:

title – Text field (type short text)

content – Rich Text field

excerpt – Text field (type Long text, full-text search)

coverImage – Media field (type one file)

date – Date and time field

slug – Text field. You can optionally go to the settings of this field, and under Appearance, select Slug to display it as a slug of the title field.

author – Reference field (type one reference)

If you really want to dig into the process yourself, you can follow the instructions on the GitHub repo and create the content model manually from within your Contentful space. OR you could just hit that big blue “deploy” button and have it taken care of for you. Can you guess which option I chose? =D

Not much else to add here, except that this template might be really useful if you’re intent on showcasing high-definition images for each post. You can see our very own Brittany Walker spinning up this same template in a video tutorial here.

3. Building a low-code blog with RemixJS and Vercel

At first glance, this third blog looks exactly the same as the Gatsby Starter Blog. It’s actually a fork and rebuilt from the ground up using the RemixJS framework. For this site, I opted to host it again on Vercel.

This starter blog template is the work of Contentful Software Engineer Marco Link, and has pride of place on the Developer Showcase alongside some other projects of his making.

One tweak that I made to the blog after creating it was to change the date formatting from DE to EN. You can do that by visiting the file to-readable-date.ts in contentful-remix-starter-blog/app/utils and changing the value “de-DE” to “en-GB”.

Would you like to build something more elaborate with Contentful and RemixJS? Check out this guide for a fully featured portfolio starter.

4. Building a low-code portfolio with Gatsby and Netlify

Phew, that’s a lot of blogs! The last piece of the project is to bring them all together in one place, with all the relevant links and a bit of discourse about how they were made and the technologies used. This portfolio site is called “Get Some Jam,” and the pun is very much intended.

I’d been searching the web for a suitable template, and eventually I found the Mate portfolio starter by software engineer Ema Suriano. This is a beautiful single page site with a customizable and responsive design regardless of whether you access it on a desktop, smartphone, or magic mirror. If you end up using this template yourself, be sure to buy them a coffee.

It also has a handy feature to import posts from a Medium account, so that I could offer meta-commentary on the process of creating these posts and lessons learned alongside the links to the sites themselves. I expect that no one will find these posts interesting except perhaps my mother (and even that’s a stretch), but it felt good to document the personal eureka moments as I went along.

Next steps after the sites are built

That’s the story behind my submission of Get Some Jam to the Developer Showcase. Thank you for your attention. And since you’ve made it this far, I’ve two more nuggets of wisdom to share from these low-code examples.

Add webhooks for automating previews and updates

We recently published a detailed guide to using webhooks in Contentful. The ultimate guide, in fact. Just to add my two cents; this is one of the first things to investigate after you’ve set up your site with Contentful and your framework/deployment service of choice. It automates many of the key steps you would normally take when managing a site, such as web page previews or pushing them live when you hit the publish button in Contentful.

Keep an eye on Lighthouse scores

Another fantastic feature that’s available on Gatsby Cloud and Netlify is Lighthouse scores. Essentially, Lighthouse is a tool from Google that analyzes web apps and web pages, collecting modern performance metrics and insights to guide developer best practices. Each time you deploy an update, your new build is given a score between 1–100 for each of the following criteria: Performance, Accessibility, Best Practices, Search Engine Optimization and Progressive Web Apps. This post on the Netlify blog is a great introduction.

Wrapping up

Sitting on the other side of my little project, I have a renewed appreciation for the Contentful content platform. Yes, it’s scalable and versatile and fast and reliable — all of these things are true. But it’s also accessible enough that a non-technical person can roll up their sleeves and build something for themselves.

And this is made possible thanks to a terrific group of developers and creatives who have shared their efforts for the betterment of the community. I proffer a tip of the hat to them all.

Get started! Start building with a free Contentful account, no credit card required. Sign up.

Content migration from Drupal 7 to Contentful: A step-by-step guide

Bulent Osman Yusuf — Wed, 20 Jul 2022 14:36:51 +0000

NOTE: This article originally appeared on the Contentful Blog. Sharing here on behalf of the author Patrick Geers.

Folks, we have an important question. Are you considering migrating your content from Drupal to Contentful? Then this post will show you how. Carpe diem!

Drupal 7 is officially reaching end of life on November 1, 2023. After that point, all support for this legacy content management system will come to a close.

So how should you plan for the change? Sure, you could start using the latest version of Drupal. Or — and maybe this is a wild and crazy idea, but hear us out — you could move everything over to a new CMS.

After all, if you’ve taken this long to move on from Drupal 7, there’s likely a good reason why you were taking your time.

Let’s hazard a guess: Was it a feeling of dissatisfaction with using a monolithic CMS? Is it too complex to maintain? Do you desire a conscious uncoupling? To upgrade to something more versatile for your content needs?

It’s time to use Contentful! We have a lot of resources on this site to outline the benefits of adopting a headless CMS. But rather than dive into a discussion about content strategy and data migration, this post explains how to migrate content from Drupal 7 to Contentful.

We’ll be outlining the workflow of a migration project step by step, together with examples, migration tools, and command line prompts.

Before we dive in, there are two prerequisites in order for this guide to be useful. First, you have worked with Drupal 7. Second, you have some knowledge of the command line interface in Contentful.

Ready? Let’s go!