Forem: agentofuser

Headless Ghost and Fission

agentofuser — Wed, 24 Feb 2021 16:08:56 +0000

We're exploring a number of "Headless" combinations at Fission. The first one is with the Ghost Content Management System (CMS). We use Ghost to run our blog and want to start using it to integrate with and power our home page, so it's a great place for us to start.

The basic pattern of "Headless" is that your authors and editors login to a server based system with all the authoring and editing tools they are used to, but then the "head" – the usual template system and user facing website – isn't used. Instead, a modern front end framework is custom designed and built to take the published content of the CMS and host it. This fits great with the Fission app publishing platform.

After we've got the basics of a headless publishing workflow setup with Fission, we'll move on to using our Webnative framework to add personalization at the edge, but that will come in later posts.

Aside from Ghost, we're also looking at Headless WordPress and Headless Drupal. Let us know in the forum if you've got other Headless projects that you'd like to combine with Fission!

Build a Next.js Blog powered by Headless Ghost and Fission

You want to have a modern static website/app built with React, deployed to a CDN (even a distributed one), and loading super fast for visitors and search engines.

You want to avoid the headaches and costs of managing and securing servers for traditional content management systems (CMSs) like Ghost and WordPress.

That's what static site generators (like Next.js, Gatsby, and others) are for, right?

But, you don't want to have to retrain every writer and content creator on your team to use Markdown and Git. Ghost and WordPress are familiar, pretty, and usable. They have been iterated over years to be approachable by almost anyone.

What do you do? That's right you cut the CMS' head off 🔪

More politely, you decouple the interface used for authoring content from the systems used for rendering and distributing it.

With CMS and website decoupled, you can protect the CMS inside an intranet without exposing it to outside danger, or even use it as a desktop word processor running only on your machine, as I'll show you in this tutorial.

To make things easier we've made a template Next.js website that connects to Ghost and pulls content from it. Follow along and try it out. Feedback welcome :)

This starter template is based on the official blog-starter-typescript from Next.js.

We changed it only enough to make it possible to fetch and publish blog posts from the Ghost CMS, while keeping the ability to write Markdown files in the git repo. It's a purely additive change.

There's also a GitHub Action to build and publish the static website to Fission. It works automatically on pushes to the git repo, and can be triggered manually after updating the content on Ghost. (The default Ghost webhook isn't customizable to trigger a GitHub Action, but that can be added as a plugin.)

The Markdown blog posts are stored in /_posts as files with frontmatter support. Adding a new Markdown file in there will create a new blog post.

The Ghost blog posts are fetched using Ghost's Content API library.

✨ Getting Started

Let's start by making this starter yours:

Click Use this template at the starter page on GitHub

That will make a copy of the starter into a new repo under your account with a fresh git history. You can pick a different name for it too; in that case replace nextjs-blog-starter-typescript-ghost with your chosen name below.

Clone the repo (replace with your own URL):

  git clone git@github.com:fission-suite/nextjs-blog-starter-typescript-ghost.git

cd into the repo:

  cd nextjs-blog-starter-typescript-ghost

Install dependencies

  yarn

All command-line instructions and directory paths from now on assume the current directory is the root of the cloned repo.

👻 Setting Up Ghost

Public or Local Ghost?

Running a local Ghost instance is good for testing. But also, if you have no need for a Ghost instance running all the time out on the internet (for example, if you are the only author) you can do this for your production website, using Ghost locally for its nice interface if you prefer that to editing Markdown files. No servers to secure, no bills to pay.

Setting Up Local Ghost

If you have a Ghost instance running already somewhere, move on to the next step. If you don't, you can set one up on your own machine with Docker.

To create a local Ghost instance with Docker, run the following at the root of your repo:

yarn ghost-local-create

Ghost data will be stored at ./ghost, which is in .gitignore by default. In a private repo you can choose to commit that too and have your Ghost content versioned and available whenever you need it.

There are other scripts like ghost-local-start, ghost-local-stop, and ghost-local-remove which you might find handy to manage the Docker container.

After Docker downloads and sets up Ghost, it will be accessible on your browser at http://localhost:3001.

Next you need to create an admin account on your newly-created Ghost. To do that, visit the admin interface at http://localhost:3001/ghost and follow the wizard.

Exposing the Ghost Content API

I'll use http://localhost:3001 for the examples, but you can replace that with your Ghost URL if you have an instance already running somewhere else.

Go to the admin interface at http://localhost:3001/ghost
On the left-hand sidebar, click on "Integrations"
On the Integrations screen, click on +Add custom integration
Give it a name, like nextjs, and click "Create" then "Save"

Now the important part:

On the same screen, you'll find two fields we need: Content API Key and API URL
Copy those into a new .env.local file, like this:

# .env.local
# replace values with your own
GHOST_API_URL=http://localhost:3001
GHOST_API_KEY=2a9356e4a5214c883ba886e58e

⚠️ This file is ignored by git by default. Don't commit env.local to git unless you know what you're doing.

Alright! Ghost part's done.

💻 Running Next.js Locally

Next.js is the missing static website head to our *head*less Ghost. Let's stitch them together! This should be enough:

yarn dev

Your blog should be up and running at http://localhost:3000! (If anything unexpected happens, please post an issue.)

Now you can change the Next.js website code and the content on Ghost, and iterate quickly on them in the browser.

Note: live-reload works for Next.js code and Markdown files; to see changes to Ghost content you need to refresh the page.

🌐 Deploying to Fission

When you're ready to publish, the first step is exporting your website to a set of static files:

yarn build

That should create a directory at ./out with all your ready-to-publish files.

Next we use the Fission CLI to send that out onto the internets.

🔰 Fission CLI Install and Sign Up

To install the Fission command-line interface using brew, run:

brew tap fission-suite/fission
brew install fission-cli

For more ways of installing the Fission CLI, please check the documentation.

If you don't have a Fission account, you can create one without leaving the command-line by running:

fission setup

🌱 Register New Fission App

You can pick a subdomain or let Fission choose a random one for you.

To host the Next.js website at a random subdomain on .fission.app, run:

fission app register

To choose your own subdomain, use the --name option like this:

fission app register --name my-beautiful-subdomain

That will create a fission.yaml file. This one is safe to commit to git, and you should do that if you want to use the GitHub Action to build and deploy the website for you. Make sure there is a line saying build: ./out in it. That's the directory where Next.js puts the exported website files.

🚀 Aand... Launch!

One last step:

fission app publish

And you're done! Your website should be up at a random URL returned to you by the Fission CLI or at my-beautiful-subdomain.fission.app if you used the --name option. Yay!

🤖 (Semi-)Automatic Deployment with the GitHub Action

If you don't want to run the build locally and deploy to Fission every time you make a change to the Next.js code or Markdown files (who does?), this starter comes with a GitHub Action that automates that for you.

To get it working you need to set up 3 secrets for your repo. To do that, go to the "Settings" > "Secrets" screen on your GitHub repo, then create a "New repository secret" for each of these:

GHOST_API_URL
GHOST_API_KEY
FISSION_KEY

The values for the GHOST_API_ fields are the same ones you used for the .env.local file above.

The FISSION_KEY was created and stored locally for you by the Fission CLI when you set it up. Here's how you get it:

cat ~/.config/fission/key/machine_id.ed25519 | base64

With all 3 secrets set up, you can trigger the deploy action manually by clicking on "Run workflow" > "Run workflow", or see it in action after your next git push.

Note:

⚠️ To fetch the content from Ghost, the GitHub Action must be able to access the URL entered in the GHOST_API_URL secret.

If you are running a local Ghost instance on your machine, http://localhost:3001 won't be visible to the outside.

A simple way of exposing your local Ghost instance to a publicly-accessible URL is by using one of the open source alternatives to ngrok or ngrok itself.

With ngrok you do something like this:

ngrok http 3001

⚠️ This is great for testing, but not secure. Look into using https if you're going to rely on this and don't want your Ghost API key and data to leak.

Why "(semi-)" automatic then? 🤔

Glad you asked. The GitHub Action can detect when you push to the GitHub repo because there's a built-in "on push" event that can be used to trigger it.

But when you create a new post on Ghost or update an existing one, GitHub needs a way of finding out about it so it can run the deploy action. There's no git push event happening, so the trigger is usually a webhook (i.e. a POST request you send to a GitHub API URL.)

🙂 Luckily, Ghost comes with the built-in ability to send webhooks when content changes happen.

🙃 *Un*luckily, though, Ghost doesn't let you customize the payload that goes in the webhook requests it sends, and GitHub requires a certain field to be present telling it which ref (usually a branch) you are referring to.

Bottom line: after changing things on Ghost, you need to trigger a build manually.

You can do that via the GitHub interface ("Actions" > "Continuous Deployment" > "Run workflow" > "Run workflow"), or by running the handy script we added to this starter:

GITHUB_USER=your-username GITHUB_REPO=your-repo GITHUB_AUTH_TOKEN=your-auth-token trigger-github-deploy-action.sh

Here's how you can get an auth token.

This situation is definitely not ideal, and can be solved with Ghost plugins, but we wanted to stick to the basic install.

🙏 Show your support

Please give a ⭐️ if you liked this project! We appreciate it :)

The Complete Beginner's Guide to Deploying Your First Static Website to IPFS

agentofuser — Sun, 12 May 2019 12:59:36 +0000

This is going to be like the quickest tutorial ever. Underwhelming almost.

You don't need to know anything about IPFS or distributed nothing, not even static site generators.

Ready? Ok, the first step is you gotta open up your terminal and type this in:

mkdir -p dwebsite/public
cd dwebsite
echo '<h1>Hello, worlds!</h1>' >> public/index.html

yarn global add @agentofuser/ipfs-deploy
# or: npm install -g @agentofuser/ipfs-deploy
ipd

You with me? Ok, typed that. What else? Nothing.

What? Yep. You're done here.

Now you sit back and watch as the victory parade logs by 😎

ℹ 🤔 No path argument specified. Looking for common ones…
✔ 📂 Found local public directory. Deploying that.
✔ 🚚 public weighs 24 B.
✔ 📌 It's pinned to Infura now with hash:
ℹ 🔗 QmQzKWGdjjQeTXrruYL2vLkCqRP8TyXnG1a9QEJjDM8WTY
✔ 📋 Copied HTTP gateway URL to clipboard:
ℹ 🔗 https://ipfs.infura.io/ipfs/QmQzKWGdjjQeTXrruYL2vLkCqRP8TyXnG1a9QEJjDM8WTY
✔ 🏄 Opened web browser (call with -O to disable.)

And there you have it. Your very own l33t #dwebsite live on the hashlinked Merkleverse. Check it out. Share with your friends.

Sweet, huh? 🍬

Slow down, what exactly just happened?

Alright, that was a bit much to take in at once. Let's rewind a little and look at it in slow-motion, with backstage commentary:

1. Where's the stuff?

Truth be told, I could have called ipd ./public, passing the directory to be deployed (public) explicitly.

But then you wouldn't see the "hard thinking" emoji as ipfs-deploy probes smartly about for one of the many, often undocumented, build destinations commonly used by static site generators.

ℹ 🤔 No path argument specified. Looking for common ones…
✔ 📂 Found local public directory. Deploying that.

Yes, I actually combed through staticgen.com, installed a bunch of those static site generators, and built little test sites just so I could claim that "zero-config" headline. It's the little things, you know.

This is what ipfs-deploy looks for when we're too lazy to type it out:

const guesses = [
  '_site',         // jekyll, hakyll, eleventy
  'site',          // forgot which
  'public',        // gatsby, hugo
  'dist',          // nuxt
  'output',        // pelican
  'out',           // hexo
  'build',         // metalsmith, middleman
  'website/build', // docusaurus
  'docs',          // many others
]

As you can see from this blog's domain name, I'm a fan of Gatsby, but ipfs-deploy serves all in need. Bring your own SSG and we'll slap an interplanetary jetpack on its back and send it flying. 🚀

2. The upload

This is what we're here for, that is, putting the website into space (figuratively, for now). So a few seconds later, ipfs-deploy delivers:

✔ 📌 It's pinned to Infura now with hash:
ℹ 🔗 QmQzKWGdjjQeTXrruYL2vLkCqRP8TyXnG1a9QEJjDM8WTY

Jackpot! That's the money shot right there.

That little rambling of a hash is the crux of the whole dweb judo. The magic utterance that summons by name your website from the cavernous depths of the connected dungeons, caring not where it lies, but only what it is.

Intrinsic addressing, unbound by location or route.

Welcome <grave pause> to the distributed future.

Wait a minute, I thought I heard you say "distributed" 🧐

Oh, you perspicacious reader.

You're right: if IPFS is supposed to be a peer-to-peer protocol, why are we even uploading at all, right? To a server!? 🤢

Shouldn't we just announce to the network that we have the hash and then wait to serve it ourselves to other peers as they request it?

Yeah, you can do that. Then you close your laptop, wifi goes down, 💩 happens, and poof ✨ there goes your website.

This is just like torrents: you need at least one seeder to be available for content to be reachable. If your website has tons of people who run their own IPFS node visiting it and re-serving it to others, then on average your uptime will be pretty high.

But there aren't that many such visitors around yet (hopefully Brave will fix that for us) and besides, it's a brand new website! 🐣 Poor thing wasn't born famous. Give it some time.

It would be a different story if browsers had a decent #asyncUX and visitors could easily tell them to queue the download for when a peer is available and then notify when it's ready (like a seamless "read it later" flow).

But as it stands, if there's no one live the moment a request is made, things just hang and then time out. Definitely not space-ready.

So we need a high-uptime seeder on our side. Or, in IPFS lingo, a "pinner."

Zero-Config Pinning with Infura.io

A big design goal of ipfs-deploy is to let you have that first happy experience of just seeing something you made up on IPFS as fast as possible.

One way to do that is to run a local IPFS daemon and have you serve the content yourself.

But as we saw above, that would be a little gimmicky as it doesn't represent what an actual deployment that you can share with your friends would look like. Gotta have a stable pinner.

Pinning stuff with decent uptime costs money though, so most pinning services understandably require you to at least sign up for a free tier before they'll agree to host your website.

Not Infura.io, though!

By some magic of careful rate-limiting, clever abuse prevention, growth capital, or reckless abandon, they let you just upload stuff out of the blue, unauthenticated, and they'll serve it for you indefinitely. (Even against your will it seems, as there is no clear way of *un*pinning things at the moment.)

So we owe that slick on-ramp to their generosity: thank y'all at Infura, and please keep it up!

Also, if you own a pinning service yourself and would like to be part of the zero-config welcome package, please consider adding an "even freer" tier that doesn't require signup.

Newly-created static websites don't take up much room, have very little traffic, and are a great gateway into further IPFS consumption.

You did it! 🏁

If you got this far, condragulations. You rock! 🗿

Not only did you deploy your first IPFS website, you can now boast you actually understand how it works by waving your hands and saying "oh, it's pretty much like git + bittorrent you know, easy peasy!"

If you diligently followed the instructions, and somehow things blew up in your face, that's on me, not you. This is my commitment: your first happy experience, or it's a bug!

So please tell me what went wrong and I'll smooth it out for you.

We're all about removing friction around here 🧹🥌

If you haven't had enough, stick around for some extra credit.

And if you are content with where we got for now, please share that feeling with others by spreading and upvoting this guide far and wide :) Thank you!

Bonus chapter

Free Redundancy with Pinata.cloud

Having a stable pinner is cool and all, but isn't much different from regular web hosting. (In the "distributedness" sense, that is. In the "content-addressableness" sense, it's night and day.)

One way to get a taste of the distributed nature of IPFS is by adding a second pinner, and ipfs-deploy makes that easier.

We're going to deploy both to Infura.io and Pinata.cloud so that visitors can download from both at the same time, or from one in case the other fails.

Resilience! 🤹

Pinata.cloud is a dedicated IPFS pinning service that gives you more control over what is being hosted.

It allows you to delete pins and to add metadata that you can later use to filter and manage your deployments.

There is a 1 GB free tier which is plenty enough for dev blogs, landing pages, documentation, and #YangGang fanpages. It does require signup, but it's pretty straightforward and doesn't require credit card or personal information.

After signing up and getting your API keys, go to your website's directory and copy your keys into a .env file like so:

# dwebsite/.env
IPFS_DEPLOY__PINATA__API_KEY=paste-the-api-key-here
IPFS_DEPLOY__PINATA__SECRET_API_KEY=and-the-secret-api-key-here

⚠ You don't want to make that information public ⚠, so when you're doing this in a repository that you're going to host publicly, make sure to add the .env file to your .gitignore:

echo .env >> .gitignore

One last config: to deploy to Pinata, you'll need to forward port 4002 on your router to your machine.

Why? Because deploying to Pinata works like this:

We first start a temporary local IPFS node,
Pin the website locally,
And send the hash to Pinata.
Pinata then connects to our local node as a peer,
Requests the hash we sent it,
Then downloads and hosts it itself.

Because of step 4, we need to be able to listen to external connections.

Manually forwarding ports is a pain, I know, but support for NAT traversal is coming this quarter to js-ipfs, so that is one less hoop we'll need to jump soon 🤞

By contrast, Infura exposes their IPFS node's HTTP API, so we can upload to it directly as an HTTP client without running a local node or listening for connections.

Pinata's custom API strikes a different point in the flexibility-zeroconfigurability spectrum.

Thankfully, with ipfs-deploy we can have both ✌️

Now that Pinata is set up, let's get back to the show. Here's what you run to deploy to both pinning services:

ipd -p infura -p pinata

And this is what you get:

ℹ 🤔 No path argument specified. Looking for common ones…
✔ 📂 Found local public directory. Deploying that.
✔ 🚚 public weighs 24 B.
✔ 📌 It's pinned to Infura now with hash:
ℹ 🔗 QmQzKWGdjjQeTXrruYL2vLkCqRP8TyXnG1a9QEJjDM8WTY

Nothing new so far. Now to Pinata (featuring new emojis!):

✔ ☎️ Connected to temporary local IPFS node.
✔ 📶 Port 4002 is externally reachable.
✔ 📌 Pinned to temporary local IPFS node with hash:
ℹ 🔗 QmQzKWGdjjQeTXrruYL2vLkCqRP8TyXnG1a9QEJjDM8WTY
✔ 📌 It's pinned to Pinata now with hash:
ℹ 🔗 QmQzKWGdjjQeTXrruYL2vLkCqRP8TyXnG1a9QEJjDM8WTY
✔ ✋️ Stopped temporary local IPFS node.

And there it is: same hash, different locations.

✔ 📋 Copied HTTP gateway URL to clipboard:
ℹ 🔗 https://ipfs.infura.io/ipfs/QmQzKWGdjjQeTXrruYL2vLkCqRP8TyXnG1a9QEJjDM8WTY
✔ 🏄 Opened web browser (call with -O to disable.)

You can see the same content on whichever gateway you want by replacing "ipfs.infura.io" with:

Or any of the ones listed here: https://ipfs.github.io/public-gateway-checker

I've said it before, but I find this so cool that it bears repeating:

Intrinsic addressing, unbound by location or route.

Or: calling data by what it is, not where it is.

That's cryptographic hashing for ya 💪

The telling-your-friends part 🗣

Alright, we're done here, right? Feeling all great and distributed. Now go tell your friends about it over the phone 📞

— Hey Jessie, guess what?

— What?

— I've got my website up on the dwebs!!

— Yay friend, how cool! I bet that was super hard.

— Nah, there's this npm package, y'know...

— Yeah, yeah, lemme see the website, where's it at?

— Oh it's at i-p-f-s-dot-i-o-slash-i-p-f-s-slash... Er...

— Slash what?

— It's uh... You got a pen? It's uh... uppercase Q, lowercase m, uppercase Q,
uppercase T... Ugh... How about I text you the URL?

— Ok, sure, but what if I'm just some random person who sees your URL on a billboard? You won't be able to text me then, will you?

— Hm, I guess not.

— You know what, how about you call me when you have something memorable I can type in the browser?

— Wow, harsh.

Ok that very lifelike conversation went south pretty fast.

It turns out content-addressing on its own doesn't gel quite well with the limited human memory buffer.

Plus, friends can be tough.

What do we do then?

A pretty URL

Human-readable naming for IPFS websites is definitely an area that needs smoothing out.

But if (and that's a big if) you keep within the constraint of using the free Cloudflare IPFS Gateway for now, ipfs-deploy wraps it all together in a pretty neat way.

Here is actual footage of me deploying interplanetarygatsby.com:

ipd -p infura -p pinata -d cloudflare

And this is what I see after the uploads are over:

✔ 🙌 SUCCESS!
ℹ 🔄 Updated DNS TXT interplanetarygatsby.com to:
ℹ 🔗 dnslink=/ipfs/QmSNf4sScZUmpNqBWAAs9S5tC4XkQRNepA3KbF4aipGGeq

It makes me smile every time at the sheer effortlessness of it 😌

There are some one-time steps you need to take to get there though:

Buy a domain
Sign up for a Cloudflare account
Move your domain's DNS zone to Cloudflare
Hook up your domain to their IPFS gateway
Get your API keys

Between actually performing those configuration steps and waiting for DNS information to propagate, this whole thing can take a couple of hours.

That's why I put that part in this bonus chapter and left the base instructions zero-config. In time, we'll remove more and more friction and make that first happy experience ever happier 😃⬅️🧹🥌

So after steps 1-5 are done, the last thing you need to do is add your domain and Cloudflare API credentials to your website's .env file:

# dwebsite/.env
IPFS_DEPLOY_SITE_DOMAIN=example.com
IPFS_DEPLOY_CLOUDFLARE__API_KEY=paste-your-cloudflare-api-key-here
IPFS_DEPLOY_CLOUDFLARE__API_EMAIL=the-email-you-used-to-sign-up

Now go ahead, fire away with ipd -d cloudflare, and tell all those billboards about it! 📣📣📣

៚

Postface: A Call for Curlers 🧹🥌

While writing this guide, I happened upon the "curling stone" emoji and felt an instant connection to it.

To be honest, I've always found the idea of curling a little... odd. But a whole team sport dedicated to frantically removing friction from a path just so that this peculiar artifact can gracefully and effortlessly glide its way onto a goal is so... moving.

That's how I want ipfs-deploy to feel to the user. A silk-smooth experience landing them right on the distributed web without breaking a sweat, and a wild sweeping crowd cheering them on.

If that sounds like your kind of sport, say hello in the issues and let's polish some stuff!

Foreword: Space-Ready Publishing with Gatsby and IPFS

agentofuser — Sat, 04 May 2019 13:37:00 +0000

Interplanetary Gatsby is a blog about making space-grade websites.

“Static” and “offline-first” don’t really do them justice.

In space, there is no such thing as “online” —this naïve illusion of near-simultaneity bore by the small window of low-latency communications that graced our species in the few decades that followed millenia of foot, horse, and sea, and preceded our belated (re)launch into the deeper waters of the cosmic ocean.

For light is slow, and so far as our minds have been able to see, there is no short-term escape to its grasp on the reach of our interstellar voice.

Asynchronicity is all that is or ever was or ever will be.

Mars is 3.2 light-minutes away at its closest and 22.2 light-minutes away at its farthest.

Imagine clicking a link to a news article and sitting there for a minimum of 6 minutes for it to load.

It doesn’t matter how hand-optimized the page is or how high your bandwidth. That’s the fixed cost that the very first bit will have to crawl through before it gets to you.

How do you design around this constraint?

You cache the hell out of everything. You prefetch. You reach out to people on your own planet before capitulating to the space link. You change the user experience to decouple fetching from reading, you queue and notify instead of blocking and spinning.

These are the things I want to explore with Gatsby and IPFS—the InterPlanetary File System.

They are the perfect tools for this job, and I invite you to join me on this journey into the vastness of the Merkleverse.