Forem: Payload CMS

How To Build An E-commerce Site With Next.js 💰

Jacob Fletcher — Thu, 07 Sep 2023 20:05:12 +0000

The last time you were tasked with building an e-commerce store, you were likely overcome with dread and fear. Your mind likely raced to one of the following pitfalls:

Time to market
Microservice Hell
Customer-facing UI
Feature anxiety
Vendor lock-in
API overages
Monthly fees
Data ownership
Database access

With Payload's new E-commerce Template you can focus more on business strategy and less on technology with a beautifully designed, production-ready website right alongside your Payload app.

Your APIs are your own and your data belongs to you. You are not beholden to a third-party service that may charge you for API overages, on top of a monthly fee, and who may limit your access to your own database.

Running an online store will never cost you more than your server (plus payment processing fees).

And you can go from initial clone to first sale within minutes.

Imagine actually wanting to work on your e-commerce site, being more than happy to invite marketers in to dream up new landing pages for your latest campaigns, and where editors create completely unique layouts for all types of content, all by themselves.

This template is right for you if you are selling:

Physical products like clothing or merchandise
Digital assets like ebooks or videos
Access to content like courses or premium articles

Everything you need is ready to go, including:

Admin Dashboard
Authentication
Shopping Cart
Checkout
Payment Processing
Customer Accounts
Forms
Paywall Content
Layout Builder
Dark Mode
SEO

And much more.

Quick Start

Option 1 (Recommended)

Head to Payload Cloud, clone the E-commerce Template, and deploy it on a 30-day free trial. This process may take a few minutes, and once it's done you'll have a fully working e-commerce store, complete with a live domain. This will have created a new repo for you in your GitHub account. You can clone this repo down for local development, and when you push, your commits will automatically deploy.

Option 2

The code for this template is completely open source and can be found here. To quickly clone this template, run the following command in your terminal:

npx create-payload-app my-project -t ecommerce

Then follow the instructions in the README to get started. You can always deploy this repo to Payload Cloud later.

Admin Dashboard

As with all Payload apps, you get an enterprise-grade admin dashboard out of the box. This dashboard is where your store admins can manage all users, orders, products, pages, and more. You can fully customize the look and feel and it can even be white-labeled to match your brand. The admin panel is also where your editors will use the layout builder to generate unique landing pages and product layouts.

Authentication

Authentication with Payload was already easy, but it just got even easier. In this template you get fully working customer login and create account flows, complete with password reset and email verification. Forms, field states, error handling, cookies, redirects, it's all ready to go out of the box. With this template you don't need to worry about building these flows yourself, or rely on a third-party service (that you'd likely have to pay for).

Customer Accounts

Users never have to leave your site to manage their account, which is a huge win for customer retention. They simply login to see their past purchases, review their order history, and update their account information. Users never have to navigate between platforms, they can do everything in one place. You can also use this data to send personalized emails to your customers, like abandoned cart reminders or product recommendations.

Shopping Cart

One of the hardest parts of any e-commerce app is the shopping cart. They need to work very well in order to increase conversion and seamlessly funnel users into your checkout process. A good shopping cart should:

Allow users to shop without logging in
Save anonymous users' carts to browser storage
Sync saved carts to users' accounts upon login
Persist the same cart across all devices

Checkout

The template integrates with Stripe to provide a fully working payment processing flow.

In full transparency, Stripe does take a small cut of every transaction, but to be fair, they provide a valuable service by handling 100% of the financial and accounting services. It is entirely possible to integrate with other payment processors, you would just need to build out the integration yourself.

Selling Digital Assets

Gating content behind a paywall is a great way to increase your revenue. If you are selling digital assets, easily control access to those assets behind a user's purchase history. If your product is a video course or premium content, for example, you can restrict access to that course or content to only users who have purchased it. Or if you're selling an ebook, easily restrict access to a download link until the user has purchased it.

Layout Builder

Your marketers will be able to dream up new landing pages and product layouts without feature anxiety. Editors will be able to create completely unique layouts for all types of content, configurable on-the-fly directly in the CMS. They can seamlessly navigate between the admin panel and the live site to see their changes in real-time.

This template comes pre-configured with a few heroes and blocks to get you started, but you can easily add more.

SEO

Meta tags, open-graph tags, structured data, sitemap generation, the works, it's all included. You can easily customize the SEO settings for each page and product from the admin panel.

Get Started

Use this template to get to market fast. Use it to enjoy building your next big e-commerce store. Or to create FOMO amongst your developer friends. Deploy in minutes on Payload Cloud or clone it from GitHub. Or, run npx create-payload-app. Happy building.

What is Payload?

Payload is a fully open-source, TypeScript-based headless CMS and application framework that gets developers what they need and gets out of the way. Built with Node.js and React, Payload is code-first, so you can enjoy a backend and a CMS-grade admin UI without having to learn another CMS. Get started in one line $npx create-payload-app or have us do the work.

Error Management Magic: Introducing the Sentry Plugin for Payload

Jessica Chowdhury — Thu, 27 Jul 2023 13:38:15 +0000

This integration brings Sentry's real-time error tracking, alerting, and performance monitoring features directly into your Payload project. We want to empower your developers to diagnose and resolve issues with unmatched efficiency.

yarn add @payloadcms/plugin-sentry
// OR
npm i @payloadcms/plugin-sentry

What is error tracking and why does it matter?

Error tracking is the process of meticulously collecting and analyzing errors that occur within your application. With a strong error tracking solution, you open the door to a plethora of benefits that will elevate the overall quality of your project.

A good error tracking solution will allow you to...

Prioritize and address errors ensuring that the most important errors are fixed first
Identify recurring errors and patterns earlier and prevent them from snowballing
Save valuable time when it comes to debugging and troubleshooting errors
Increase application reliability and reduce downtime
Improve the overall developer and user experience

Ultimately, error tracking serves as a key component in driving the overall quality of your software. By addressing issues promptly, improving reliability, and optimizing the development process, you can deliver a product that takes software development to new heights.

Sentry: The Error Monitoring Powerhouse

Sentry is a leading, feature-rich, open-source platform for error and performance monitoring. With minimal setup, you can start capturing valuable data from your application and access it through Sentry's comprehensive and easy to use dashboard.

Here is a glance at their awesome features:

Error tracking: Get detailed reports with stack traces, environment data, and user info
Performance monitoring: Including user interactions, network requests, and other performance metrics
Release tracking and health monitoring: Keep tabs on releases and monitor application health with ease
Cross-project visibility: Sentry lets you manage and monitor errors across all multiple apps from one central platform
Custom dashboards: Access the metrics that matter most to you and personalize the dashboard to your needs
Notifications and email alerts: Add and customize alerts so you'll be the first to know when something is happening

And that's just the tip of the iceberg. Sentry's features empowers you to manage errors with extreme efficiency, saving time and resources while delivering a top user experience.

Step-by-Step Plugin Integration Guide

Let’s walkthrough the setup process for the Sentry-Payload plugin:

1. Installing the Plugin

Navigate to your Payload app then install the plugin using Yarn or NPM:

yarn add @payloadcms/plugin-sentry
// OR
npm i @payloadcms/plugin-sentry

2. Creating a Sentry Account

If you don't have a Sentry account already, it's time to create one. Head over to Sentry and create an account. The registration process is straightforward and will only take a few minutes.

3. Locating Your DSN

The DSN (Data Source Name) is a unique identifier that connects your application to Sentry. After creating your Sentry account, you can find your project's DSN in the project settings or configuration section.

4. Add DSN to your Payload Config file

By this step, you should have the plugin installed and your DSN identifier ready. Navigate to your payload.config() file and add the plugin along with your DSN:

const config = buildConfig({
  plugins: [
    sentry({
      dsn: 'your-dsn-string-here',
    }),
  ]
});

5. Configure additional options (not required)

These options allow you to fine-tune the Sentry integration to your specific needs:

init : ClientOptions | optional

Sentry allows a variety of options to be passed into the Sentry.init() function, see the full list of options here.

requestHandler : RequestHandlerOptions | optional

Accepts options that let you decide what data should be included in the event sent to Sentry, checkout the options here.

captureErrors: number[] | optional

By default, Sentry.errorHandler will capture only errors with a status code of 500 or higher. To capture additional error codes, pass the values as numbers in an array.

Pass any of these options to the plugin like so:

const config = buildConfig({
  plugins: [
    sentry({
      dsn: 'your-dsn-string-here',
      options: {
        init: {
          debug: true,
          environment: 'development',
          tracesSampleRate: 1.0,
        },
        requestHandler: {
          serverName: false,
          user: ["email"],
        },
        captureErrors: [400, 403, 404],
      }
    }),
  ]
});

Take it for a spin

Once you have the plugin installed and configured, Sentry will be ready to start capturing errors for you. So let’s take this plugin for a test drive by intentionally triggering an error:

Start your Payload app with yarn dev
Navigate to the login page in your browser
Instead of using your valid credentials, enter an incorrect password
This should throw the following error: AuthenticationError: The email or password provided is incorrect
Go to sentry.io and select the project associated with the DSN in your payload.config()
See the new error in your dashboard, click into it for more detail

Wrap Up

A robust error tracking solution is crucial to delivering a reliable and performant application. Sentry provides developers with the ultimate toolbox, comprehensive error and performance monitoring, custom dashboards, real-time notifications and much more.

This plugin aims to make integrating Sentry into your Payload project a breeze. With 3 lines of code, you can instantly improve your error handling workflow. Give it a go and as always, reach out to us if you have any questions or feedback! Happy tracking 👋

How to Customize the Look and Feel of Payload with CSS

PatrikKozak — Mon, 12 Jun 2023 17:56:30 +0000

One of the hidden gems of Payload is its Custom CSS option.

Its potential is completely limitless and lets you design / customize the appearance of every element of the Payload dashboard in a way that better represents your brand.

Styling a CMS can be a pain and is usually limited. While the defaults out of the box are great, some projects deserve more finesse. Payload gives you that freedom.

Our Custom CSS feature is a poster child for this approach. Wanna tweak the platform’s look? Go for it. Turn it into a kaleidoscope if you want (please don’t)... Because at the end of the day, your CMS should be a seamless extension of your brand—bonus points for making your marketing team happy.

Block, Element, Modifier (BEM) Naming convention

The entirety of the Payload admin UI utilizes the BEM (Block, Element, Modifier) naming convention. This widespread approach mitigates the usual challenges posed by CSS’s cascading nature, making your CSS more manageable and scalable. It also maintains a reliable selector convention, ensuring that they won’t unpredictably change on you.

Demo

The first step involves providing your base Payload config with a path to your custom stylesheet, ensuring that Payload can recognize and apply your custom styles.

import { buildConfig } from 'payload/config';
import path from 'path';


const config = buildConfig({
    admin: {
        css: path.resolve(__dirname, 'relative/path/to/stylesheet.scss'),
    },
});

Once you’ve set the path, you’ll need to add your stylesheet to your directory. This will serve as the space where you define your custom styles.
First up, we’ve got a snapshot of our standard Payload Admin UI - unopinionated out of the box but highly customizable.

The image below showcases the same UI, now elegantly enhanced with some custom CSS. I’ve updated a few small things like fonts and colors to maybe match an existing brand more closely. With minimal changes, you can see the admin panel take on a different look and feel.

How did I do this? Starting with the browser's inspector tool, I pinpointed elements I planned to modify and inserted the necessary classes and styles into my custom stylesheet.

I took a hard look at the UI’s current color theme. It was doing the job, but it wasn’t in sync with the brand image I was after. So, I changed it. By adjusting a few css variables, I’m able to change the look and feel of the dashboard to match any brand.

Swapping out the font styles is easy, you simply just need to set them in your css file. Finally, I made a few adjustments to the colors of the button and the banner backgrounds, creating a more prominent contrast.

@import '~payload/scss';

html {
  &[data-theme=dark] {
    --theme-elevation-0: rgb(24, 20, 48);
    --theme-elevation-50: rgb(37, 33, 61);
    --theme-elevation-100: rgb(50, 46, 74);
    --theme-elevation-150: rgb(63, 59, 87);
    --theme-elevation-200: rgb(76, 72, 100);
    --theme-elevation-250: rgb(89, 85, 113);
    --theme-elevation-300: rgb(102, 98, 126);
    --theme-elevation-350: rgb(115, 111, 139);
    --theme-elevation-400: rgb(128, 124, 152);
    --theme-elevation-450: rgb(141, 137, 165);
    --theme-elevation-550: rgb(154, 150, 178);
    --theme-elevation-600: rgb(167, 163, 191);
    --theme-elevation-650: rgb(180, 176, 204);
    --theme-elevation-700: rgb(193, 189, 217);
    --theme-elevation-750: rgb(206, 202, 230);
    --theme-elevation-800: rgb(219, 215, 243);
    --theme-elevation-850: rgb(232, 228, 255);
    --theme-elevation-900: rgb(255, 255, 255);
    --theme-elevation-950: rgb(255, 255, 255);
    --theme-elevation-1000: rgb(255, 255, 255);

    --theme-bg: rgb(24, 20, 48);
  }
}

body, h2, h5 {
  font-family: monospace;
  font-weight: bold;
}

.btn {
  &--style-primary {
    background-color: rgb(0, 213, 110);
    color: var(--theme-elevation-900);
  }

  &--style-primary.btn--disabled {
    background-color: rgb(102, 255, 212);
  }

  &--disabled {
    background-color: rgb(102, 255, 212);
  }
}

.login .logo path {
  fill: rgb(0, 213, 110);
}

.template-default {
  .nav {
    &__brand .icon path {
      fill: rgb(0, 213, 110);
    }
  }

  .before-dashboard {
    .banner {
      // BEM modifier selector (.banner--type-success)
      &--type-success {
        background: rgb(0, 213, 110);
        color: white;
      }

      // BEM element selector (.banner__content)
      &__content svg {
        background-color: #2A2544;
      }
    }
  }
}

A quick note: see how we are nesting the targeted elements inside its parents selectors, this is what allows us to override their styles. CSS specificity is key here, it gives you complete control of the styles.

For instance, consider our button styles. By employing a combined class selector, .btn--style-primary.btn--disabled, we increased specificity to target only the disabled primary buttons—a testament to the power of CSS specificity used in tandem with BEM.

If you want to target something else with more specificity, I recommend first inspecting the DOM. This helps you identify the selectors currently controlling the style. With this knowledge, you can craft a more specific selector to accurately target and style your desired elements, utilizing CSS specificity to your advantage.

Wrap Up

The Custom CSS feature in Payload extends beyond being a simple customization tool–it’s a pathway to enrich UX and seamlessly integrate your CMS with your brand aesthetics.

We invite you to explore this feature, adapt it, and truly make it your own!

Learn More

To learn more about PayloadCMS and Custom CSS, take a look at the following resources:

Like what we're doing? Give us a star on GitHub

We're trying to change the CMS status quo by delivering editors with a great experience, but first and foremost, giving developers a CMS that they don't hate working with. All of our new features are meant to be extensible and work simply and sanely.

Payload + Nodemailer: Free and Extensible Email Integration

Jessica Chowdhury — Fri, 26 May 2023 15:45:03 +0000

Email is a necessary evil in our lives. We don’t love it, but we need it.

With Payload, you don't have to abandon what you're already familiar with. Stick to your trusty SMTP if you like to keep things simple. Or, if you're a fan of Gmail or Outlook, go ahead and integrate them with ease. You can even bring in other powerful email service tools like SendGrid, Resend, HubSpot and more.

Integrating email with Payload is free, flexible and highly extensible. No matter what kind of emails you need to send – from newsletters, transactional and marketing emails, to those crucial authentication emails – we've got you covered.

In this post, we’ll walk through the process of configuring email with Payload and cover everything you need to get up and running. Let’s dive in!

How it works

Payload utilizes Nodemailer to produce a versatile email transporter which can then be used anywhere in your application.

For those who are new to Nodemailer, it is a powerful module in the Node.js ecosystem that greatly simplifies the process of sending email. We recommend taking a look at the Nodemailer docs if you want to learn more.

If you have used Nodemailer before, this process will be familiar. Simply create a new transport and pass it to the email property in your payload.init() function.

Once you add your email configuration, you can send emails from anywhere in your application simply by calling Payload.sendEmail({}). Neat, huh?

Configuration

The email property takes the following options:

fromName - required
fromAddress - required
logMockCredentials - will output your credentials to the console on startup
transportOptions - pass in your options and let Payload create the transport for you
transport - manual create a transporter using nodemailer.createTransport({})

Important: You only need to use transportOptions OR transport in your email configuration. Not both.

There are two ways to create a Nodemailer-compatible transport:

Define your transportOptions and Payload do it for you
Manually configure a transport or import a separate package to do this for you

To get setup, navigate to where you are calling payload.init(), this is usually in `src/server.ts.

After adding your email options, your payload.init() should look something like this:

export const email = {
  fromName: 'Admin',
  fromAddress: 'admin@example.com',
  logMockCredentials: true,
  transportOptions: {},   
  transport: {},
}

const start = async (): Promise<void> => {
  await payload.init({
    secret: process.env.PAYLOAD_SECRET,
    mongoURL: process.env.MONGODB_URI,
    express: app,
    email,
  })

  app.listen(8000)
}

start()

Important: We always recommend storing sensitive data as environment variables and not directly in your code to prevent security vulnerabilities.

Mock email handler

If you do not provide a transport or transportOptions, Payload will initialize an ethereal capture service. Ethereal is a free email caching service which captures all outbound emails. Using this service can be really useful for testing emails when you’re working in a development environment.

To use this service, logMockCredentials must be set to true. This will output the ethereal credentials to your console after startup, you will then use these to login to ethereal.email and view any emails that are sent during development.

transportOptions

Pass any valid Nodemailer options to transportOptions and Payload will create the transporter for you.

You can use transportOptions to configure:

SMTP

export const email = {
  fromName: 'Admin',
  fromAddress: 'admin@example.com',
  transportOptions: { 
    host: process.env.SMTP_HOST,
    auth: {
      user: process.env.SMTP_USER,
      pass: process.env.SMTP_PASS
    },
    port: 587,
    secure: false,
  }
}

An email service

Nodemailer will automatically provide the connection details (host, port, etc) for several well known email services. For example if you want to use Gmail, you simply need to provide the service name like this:

export const email = {
  fromName: 'Admin',
  fromAddress: 'admin@example.com',
  transportOptions: {
    service: 'gmail',
    auth: {
      user: process.env.GMAIL_USER,
      pass: process.env.GMAIL_PASS,
    },
  }
}

An external transport, a nodemailer plugin or similar

import nodemailerSendgrid from 'nodemailer-sendgrid'

export const email = {
  fromName: 'Admin',
  fromAddress: 'admin@example.com',
  transportOptions: nodemailerSendgrid({
    apiKey: process.env.SENDGRID_API_KEY,
  }),
}

Nodemailer has created packages that integrate popular email vendors for you, such as SendGrid:

transport

This option allows you to manually create a transport, this supports SMTP and email services.

You can make use of nodeMailer.createTransport({}) for support of well known email services and browse this list of options that you can define.

import nodemailer from 'nodemailer'
import payload from 'payload'

const transport = await nodemailer.createTransport({
  service: 'outlook',
  auth: {
    user: process.env.OUTLOOK_USER,
    pass: process.env.OUTLOOK_PASS,
  },
})

const email = {
  fromName: 'Admin',
  fromAddress: 'admin@example.com',
  logMockCredentials: true,
  // Passes your custom transport
  transport,
}

More examples of using nodeMailer.createTransport({}) can be found in the Nodemailer documentation.

Sending Email

Once you have configured your transporter, you can start sending emails from anywhere inside your Payload project by calling payload.sendEmail({}).

payload.sendEmail({}) takes properties of to, from, subject, and html:

import payload from 'payload'

payload.sendEmail({
  from: 'sender@example.com',
  to: 'receiver@example.com',
  subject: 'Message subject title',
  html: '<p>HTML based message</p>',
})

Dynamic Email Content

There are many ways to include data directly from your project into your emails. Whether it is using hooks, making API requests, fetching data from globals or anything else you can think of.

For example, sending order details when there is a new submission to the Orders collection:

import payload from 'payload'
import type { CollectionConfig } from 'payload/types'

const Orders: CollectionConfig = {
  slug: 'orders',
  hooks: {
    afterChange: [
      ({ doc, operation, req }) => {
        const { customerEmail, items, total } = doc
        if (operation === 'create') {
          req.payload.sendEmail({
            to: customerEmail,
            from: 'sender@example.com',
            subject: 'Welcome To Payload',
            html: `<h1>Thank you for your order!</h1>
              <p>Here is your order summary:</p>
              <ul>
                ${items.map(item => `<li>${item.name} - ${item.price}</li>`)}
              </ul>
              <p>Total: ${total}</p>
            `,
          })
        }
      },
    ],
  },
  fields: [],
}

export default Orders

Automatically trigger email dispatch

Payload’s collection and field hooks allow you to define specific conditions which will trigger an email to be sent.

Like sending an email every time you receive a newsletter signup:

import payload from 'payload'
import type { CollectionConfig } from 'payload/types'

const NewsletterSignups: CollectionConfig = {
  slug: 'newsletter-signups',
  hooks: {
    afterChange: [
      ({ doc, operation, req }) => {
        if (operation === 'create') {
          req.payload.sendEmail({
            to: doc.email,
            from: 'sender@example.com',
            subject: 'You have joined our newsletter list!',
            html: '<p>Thanks for signing up</p>',
          })
        }
      },
    ],
  },
  fields: [],
}

export default NewsletterSignups

Or sending a welcome email to new users:

import payload from 'payload'
import type { CollectionConfig } from 'payload/types'

const Users: CollectionConfig = {
  slug: 'users',
  auth: true,
  hooks: {
    afterChange: [
      ({ doc, operation }) => {
        if (operation === 'create') {
          payload.sendEmail({
            to: doc.email,
            from: 'sender@example.com',
            subject: 'Welcome To Payload',
            html: '<b>Hey there!</b><br/>Welcome to Payload!',
          })
        }
      },
    ],
  },
  fields: [],
}

export default Users

Authentication Emails

Payload makes auth-enabled collections super simple to integrate with email by handling forgotPassword and verify for you.

Each auth-enabled collection has forgotPassword and verify options that you can pass generateEmailSubject and generateEmailHTML functions to. The function accepts one argument containing { req, token, user }.

import payload from 'payload'
import type { CollectionConfig } from 'payload/types'

const Users: CollectionConfig = {
  slug: 'users',
  auth: {
    verify: {
      generateEmailSubject: () => 'Verify your email',
      generateEmailHTML:  ({ token }) => `<p>Verify your account here ${process.env.PAYLOAD_PUBLIC_SITE_URL}/verify?token=${token}.</p>`,
    },
    forgotPassword: {
      generateEmailSubject: () => 'Reset your password',
      generateEmailHTML: ({ token }) => `<p>Reset your password here ${process.env.PAYLOAD_PUBLIC_SITE_URL}/reset-password?token=${token}.</p>`,
      },
    },
  },
  fields: [],
}

export default Users

Templates

Payload doesn't ship a default HTML templating engine, so you are free to add whatever suits you best.

Make your email templates highly dynamic by using Handlebars, a templating language that combines HTML, plain text and expressions. The expressions are included in the html template surrounded by double curly braces.

<table border="0" width="100%">
  <tbody>
      <td>
        <!-- HEADLINE -->
        <h1>{{headline}}</h1>
      </td>
    </tr>
    <tr>
      <td>
        <!-- CONTENT -->
        {{{content}}}
      </td>
    </tr>
  </tbody>
</table>

Here is a simple but powerful function that ties everything together in one function:

import fs from 'fs'
import Handlebars from 'handlebars'
import inlineCSS from 'inline-css'
import path from 'path'
import payload from 'payload'

const template = fs.readFileSync(path.join(__dirname, 'template.html'), 'utf8')
const getHTML = Handlebars.compile(template)

export const sendEmailWithTemplate = async (args): Promise<any> => {
  const { from, to, subject, data } = args

  const templateData = {
    ...data,
    apiURL: process.env.PAYLOAD_PUBLIC_SERVER_URL,
    siteURL: process.env.PAYLOAD_PUBLIC_SITE_URL,
  }
  const preInlinedCSS = getHTML(templateData)

  const html = await inlineCSS(preInlinedCSS, {
    url: ' ',
    removeStyleTags: false,
  })

  await payload.sendEmail({
    from,
    to,
    subject,
    html,
  })

  return null
}

The template.html file that is being used in sendEmailWithTemplate can be any HTML file of your choice. You can find this template in our email example, feel free to use this as a starter template and add your own custom CSS.

Example

We have an example repo where you can see these code snippets being used and try them out in real time.

Wrap Up

Payload aims to provide developers with a powerful but simple solution when it comes to configuring and managing email. By harnessing the capabilities of Nodemailer, we can offer flexibility in choosing email providers and accommodate various email needs.

I hope this post provides value and gives you a good introduction to email integration with Payload.

Always feel free to reach out if you have any questions or feedback!

Learn More

Like what we're doing? Give us a star on GitHub

SQL vs. NoSQL - cutting through the Tech Twitter noise

James Mikrut — Thu, 11 May 2023 14:42:51 +0000

We've done a lot of thinking around database support lately, and even hosted a community discussion on Discord. Right now we're MongoDB-only, but our most upvoted roadmap item is adding support for additional databases. And transitioning from NoSQL thinking to an SQL mindset has really spotlighted the concrete differences.

If you've ever wondered exactly what NoSQL is good for, and alternatively, why SQL might be a better fit for certain applications, and you're tired of Tech Twitter telling you what to think, then read on for a real-world comparison.

In this post, I'm going to cover the big challenges that we'll face when adapting our features and functionalities over from NoSQL to SQL, and along the way, I'll highlight the strengths and weaknesses of each paradigm with some objective ramifications.

Data Definition Language

Since we're currently solely on MongoDB, we don't need to do things manually like adding a new table column when you add a new field to your Payload config. But with SQL, we need to think about that (and make it painless).

DDL, or Data Definition Language, is how SQL databases define and maintain their structure. Even when you use something like Prisma, which takes care of a lot of the headache for you, you need to become intimately familiar with how Prisma migrations work. There, Prisma translates its own schema syntax into SQL migrations for you.

Don't know what any of this means? Go ahead and try to simply follow the "Create a column in a table" example. If you know SQL and if this is not new to you, then you're probably okay with this. But there's work involved here.

When you decide you want to store a new field in a NoSQL database like MongoDB, you literally have to do nothing besides just start storing the data. No migrations, no generated DDL, no schema / database "sync".

That's obviously a big win for NoSQL, but that also means that the NoSQL structure is significantly more loosey-goosey and that can lead to errors. Lots of devs (not to mention project requirements) straight up require the structural constraints of SQL, and allowing any type of data to be stored in a NoSQL document can present issues in terms of data integrity. So there are definitely pros and cons to both approaches.

Migrations

Fascinatingly, another very hotly requested Payload feature is a first-party way of handling migrations. And that got us thinking. When you look at the Prisma migration resources above, clearly, they are using the word "migration" to describe the "migration" of your database shape from A to B. This is the expected and most widely used meaning of "migration".

But we're on MongoDB, and that doesn't apply. But developers are still asking for migration support. There is a specific distinction here that I'd like to make. Migrations can mean many things in many different contexts.

Changing database shape

Most often, they are referred to as simply modifying your database structure (DDL). This is a requirement for SQL in general. For example, adding a column to a table is a manual ALTER that needs to be run at some point.

Transforming existing data

But there's a whole different side to the story. What about if you need to change or transform your existing data? What if you had a full_name field, and you needed to split it up into two separate fields, first_name and last_name? Stuff like this is super common, and requires manual, programmatic management. Whenever anyone has asked Payload for migration support, this is what they meant.

Due to this second aspect of "migrations", that means that the concept is relevant for both NoSQL and SQL databases alike. Application frameworks like Laravel handle this very well, with a full, true "migrations" workflow that allows you to handle both DDL and data-based transformations within the same paradigm, all based on database transactions. I personally think this is by far the best way to solve this and this is an often-overlooked aspect of ORMs in general. Some ORMs come with great programmatic migration support, while others solely handle the DDL side (the pain in the ass side) in raw, generated SQL. Excuse my French, but I've been spoiled here by MongoDB, and I spoiled myself intentionally.

If you end up listening to our community planning call, you'll see that we cover our goals and future vision in-depth on that call. We want to model ourselves after Laravel's consistent (solve for both) approach by not outputting raw SQL, but rather, outputting TS files that give you granular, functional control.

Relational data and performance

One of the biggest wins for SQL in the minds of engineers is its heavily structured relational architecture. Without a doubt, relationships are a first-class citizen in SQL. Much of the querying language itself is dedicated specifically to combining and relating data from separate tables.
But relationships can certainly be done in NoSQL, so what's the deal? What are the real-world differences?

First up, there is an entire array of functionality within SQL that actually enforces relational data integrity. Within MongoDB, those types of constraints would all need to be written manually, and you'll need to ultimately "own" the integrity of your data. In NoSQL, you might have a relationship to another document. You could go delete that document, but unless you've handled this yourself, the original relationship would remain, yet be completely invalid. That's not ideal, and that places more work on you.

Let's take Payload, for example. Surprise, surprise. We have a relationship field, and it can store IDs to other related documents which are seamlessly merged in when you retrieve documents from the DB. We leverage the dataloader pattern to batch together all "populations" required for a given query, returning them all super fast and with as few separate queries to the DB as possible.
We actually even outperform SQL-based frameworks quite a bit. In a purely relational test, we were 3x faster than Directus and 7x faster than Strapi while both were running Postgres, and we were on MongoDB.

Without knowing what's under the hood of either of those platforms, my takeaway here is that relationships are possible in both NoSQL and SQL. If you were to write out queries deliberately for very specific joins between tables, and have the ability to define queries so that a single query is made vs. dynamic populations, then SQL will likely outshine MongoDB in terms of relational performance. But - your application logic will determine which is faster, and it's easy to get it wrong with either side.

One thing to note here is that if you are using a GraphQL API, or any type of dynamic / expressive querying language, it's impossible to make a single query that returns all populated "relations". This is because it's up to the shape of the query to determine what to "populate". This is known as the N+1 problem. You first need to load the main document, and only then can you determine what to load after. If this applies to you, then most relational performance gains from SQL are non-applicable.

Handling complex data structures

One absolutely massive win that NoSQL has over SQL is that you can store complex data directly in a single document. It can be stored in the same shape as you want it to be in when you read it back from the database. Some SQL databases like Postgres have pretty great JSON support via JSON columns, but if you over-use JSON columns in a SQL database, you're not taking advantage of the strengths of SQL in the first place and might as well go with NoSQL.

So what's it look like to store a complex data structure in SQL, without losing the benefits of SQL? For a simple example, take Payload's Group field:

{
  "id": "some-id-here",
  "myGroup": {
    "mySubField": "hello",
    "anotherSubField": "goodbye"
  }
}

The above field can be stored directly in NoSQL as-is, which is fantastic. But what about SQL-based databases? Do we transform the nested architecture to a flattened column structure, using a double underscore to separate field "paths"?

id	myGroup__mySubField	myGroup__anotherSubField
`some-id-here`	`hello`	`goodbye`

This could functionally perform the same, but we'd need to "transform" the data on its way out from the database somehow back to its intended shape. Doable, but not nearly as elegant.

Alternatively, we could "join in" another table that includes columns that represent properties within the group. That'll become a bit less performant, because we're now doing a join that we didn't need to do in NoSQL land.

Finally, do we just say "screw it" and embed the group and all of its properties, from that point on, into a single JSON column?

Field-based localization

One specific feature that we support, which is simple in NoSQL, but more difficult in SQL is field-based localization. You might have 30 locales, and any given text field needs to be translated into each of the 30 locales.

{
  "id": "some-id-here",
  "myLocalizedField": {
    "en": "hello",
    "es": "hola",
    "de": "hallo"
  }
}

Again, in MongoDB, this is a cake-walk. Storing the locales all on a parent document is great, because they are fundamentally a part of that parent document.

But in SQL we've gotta get crafty again. You might have 30 locales or more. Does that mean 30 columns for every single text field, all in one database?

id	myLocalizedField__en	myLocalizedField__es	myLocalizedField__de
`some-id-here`	`hello`	`hola`	`hallo`

If you have too many columns in SQL, you'll start to see performance implications. But this needs to be solved one way or another. Again, here, we could jump to a JSON column but then we're losing a lot.

Array-based structures

In NoSQL, storing arrays is dead simple. Just save it on the doc and move on.

{
  "id": "some-id-here",
  "myArray": [
    {
      "subField": "hello"
    },
    {
      "subField": "goodbye"
    }
  ]
}

The well-accepted and correct way to handle array-based data in SQL is to simply create a new table for each array-based field (like Payload's Arrays and Blocks fields). Then you'd just join in each row of array data when you retrieve the record back. It's more complicated, sure, and we're losing some performance here because we have to make joins. But it's logical and at least the answer is clearer than the last few examples.

Takeaways

Ultimately I think having a healthy dose of skepticism when reading anything that Tech Twitter or Reddit tells you will do you a great deal of good as an engineer. The truth is that there are certain cases where SQL will perform better for you, and NoSQL will be a better fit elsewhere. But it's up to your own unique requirements.

Overall, I think it's safe to say that SQL will win for your project if you:

Know the shape of your data well in advance
Are confident that your schema won't change often or are comfortable dealing with lots of DDL migrations
Have a relatively "flat" and not complex schema
Require heavily relational structures

On the flip-side, NoSQL will win out if you:

Frequently manage complex, nested data structures like Groups, Arrays, Blocks, localized field data, and similar
Use relationships solely for simple structures like Categories, Tags, Authors, etc.
Want to avoid dealing with DDL stuff entirely and make your life easier
Are okay with manually enforcing data integrity and handling edge cases yourself, or are using a framework that handles that stuff for you on top of NoSQL

Ah, life. Everything's a compromise. As much as I'd like to go hard and declare a victor, I totally understand why Payload SQL support has been asked for, although I personally think MongoDB is still a better fit for most of our users. In the end, you'll be able to pick whatever's right for you.

Planning session recording

We recorded the whole Discord planning session that took place last week. To hear more about how we're approaching future SQL support in Payload, including ORM evaluation and more, give it a listen.

Where can you help?

First up, we want to hear from you! If you haven't yet, make your voice and vote known on our Roadmap GitHub Discussion.

From there, once we have our future database adapter pattern established, anyone can build an adapter. We'd love to see some community-generated database and ORM support and that will only speed up / pressure test what we're building.

We're really appreciative of the engagement we saw on that call and are super pumped about doing more of those in the future. The next one will likely be regarding replacing Webpack, but we'll see. Follow us on Twitter to keep in the loop or join our Discord to keep an eye on the conversation there.

Launch Week Day 5 - Replace Shopify in Your Headless CMS Stack

Sean Zubrickas — Fri, 07 Apr 2023 17:09:37 +0000

To wrap up our second launch week, we're releasing our Ecommerce starter kit. It demonstrates the strength of Payload and how it can be used to power much more than simple website content.

Today, the typical custom ecommerce build workflow is to combine a headless CMS with an ecommerce vendor like Shopify or Medusa. I refuse to discuss Woocommerce so ... you're welcome. We've built projects like that in the past, and while doable, you start to have a kaleidoscope of different vendors / functions / API requests / spaghetti.

One of the big takeaways from this week's launches is that we want to start to see the web get simpler, not more complex. We want to promote efficiency and reduce the time it takes for developers to build products, and from our own experience, building anything custom with ecommerce is quite the nightmare in today's ecosystem.

Let's take Sanity and Shopify for example (a popular combination for custom ecommerce). Is it strange to you that both platforms can manage content, but you still need both platforms regardless, even though there's a lot of overlap?
Let's consider an MVP ecommerce build. If you combine Shopify and Sanity, you're really only leveraging Shopify for a few aspects of your MVP:

Your cart
The checkout experience
Processing payments
Customer authentication

If you leverage these parts of Shopify, and then manage content in Sanity, here's how the workflow will look.

From the engineering perspective

Fetch data from Shopify for pricing, product meta, attributes, etc.
Fetch product landing page content from Sanity
Combine the two into product landing pages / archives
Manage separate environments for dev / stage / prod in both Sanity and Shopify, and then somehow keep them in sync
Create accounts for customers in Shopify, and build the cart from Shopify
In the cart, merge product meta from Shopify with whatever content is necessary from Sanity

That's not too bad really. Halfway to microservices hell, but it's doable. We've done similar many times before. But now let's think about it from the marketing / business perspective.

From the marketing / business perspective

I need to add a new product. What do I do first? Build the product in Shopify, then go build the landing page in Sanity?
Where do I manage product prices again? That's done in Shopify.
Where do I manage the product thumbnail image? That's content, right? So Sanity. Wait, no, that's done in Shopify.
I need to add a new attribute to a product. It needs a description. Content, right? OK - probably sanity. Wait. Nope, that's product meta. Gotta do that in Shopify.
Rage due to confusion

This is harder than it needs to be

When my team built projects like this in the past, frankly they didn't turn out great. We had a hard time ourselves bringing in new devs to the code because it was all a network of inter-connected mysteries. If it's remotely hard for engineers, it's certainly going to be hard for the admin end-users.

But there's a better way. Payload offers so much more than a typical SaaS CMS, which makes this all significantly easier. Let's look at the following examples.

Payload gives you authentication
You own your APIs and your database so you can write hooks to sync product meta automatically
You can integrate a payments provider like Stripe seamlessly
You can use Stripe webhooks thanks to the fact that you can open your own endpoints (on your own server)
You can do literally ALL product management directly in one place (Payload)

Payload can eliminate Shopify from the mix entirely. Replace Shopify with Stripe, and then you have literally zero overlap between the platforms.

When an admin logs in to manage products, everything is directly in Payload. When an engineer goes to build a new feature, everything is directly in Payload.

Announcing our Ecommerce Starter Kit

Today, we're officially announcing our Ecommerce starter kit which gives you everything you need to get an ecommerce store up and running instantly with Payload + NextJS. Oh, and it's completely free and open-source.

No Shopify necessary. No microservices hell. You can even instantly deploy the starter to Payload Cloud, which hooks you up with everything you need to run in production.
With this starter, you get all of the following:

Deep integration with Stripe, including automatic two-way sync via Stripe webhooks
Manage content for your entire ecommerce store in one place
Next.js frontend ready to go with products, attributes, pricing, and more
Customer authentication built with Payload and tied automatically to Stripe customers
Shopping cart out of the box
Lots, lots more

This is specifically cool for agencies

Coming from an agency background, I share many of the struggles that engineers in an agency face.

Each project you take on is subtly different, and you might find yourself scrambling to find the perfect tech stack for each one. You can quickly end up managing a smattering of products, each with entirely different stacks.

Project A needs a simple donation form, so let's go with Contentful and a simple Stripe payment form
Project B needs to sell licenses to a digital product, so let's use Sanity and Shopify for licensing and digital downloads
Project C is just a brochure marketing site, so let's use Webflow
Project D is an enterprise site, so let's use Contentstack
Project E has subscriptions, so let's build a custom backend with Laravel and use Stripe Subscriptions

Each one of these projects will be a learning experience and your engineering efficiency will suffer big time. And then fast forward a year—you have to onboard new engineers into these projects completely separately, and you'll end up with a bunch of engineers that never really understand any of the projects because they're all totally different. Revisit a project a few months later and you'll need to spend 2 days getting back up to speed with the mania that you built.

I think one of Payload's biggest shining points is that we can handle every single project in the list above. The fact that we're half application framework and half CMS is what enables all of this.

In an agency setting, imagine if you could flatten your stack. What if each project could build off the last, seamlessly? What if you could write a feature, and then the next time a client comes asking for a similar feature, you literally just re-use what you already have, but bill them as if you had to build it?\

That's the dream. And that's what we're trying to give you.

This ecommerce kit is the beginning of lots more to come, but it's one that we're very excited about. It'll make you build better products, save money, and be happy with your engineering workflow to top it all off.

Give the new ecommerce kit a shot today.

Launch Week Day 4 - The Future of Headless

Sean Zubrickas — Fri, 07 Apr 2023 17:09:10 +0000

The tech ecosystem, especially frontend, changes at light speed. It gets overwhelming to try and keep up. But I kinda love change and I think it's once again time for change in the CMS world.

Don't wanna read this whole monologue? Then skip right to the good stuff. Spoiler alert, you can now deploy Payload serverlessly on Vercel within an existing NextJS app.

Steven Tey and I dove into the topics covered in this blog post on a livestream yesterday.

Back in ~2015, I was an early advocate for headless CMS because I wanted our agency to be able to use React for the frontends of the sites we built. I had a big pitch I'd give to my clients about the value of "headless" CMS, and they bit. But now we're taking a second to re-evaluate that pitch.

In my pitches, I would fire off the benefits of headless CMS, focusing on ideas like separation of concerns, more maintainable architecture, and the ability to deliver omnichannel content from a single source of truth.

But in retrospect, I had to gloss over the unfortunate side-effects of headless. The simple fact is that I was pushing my technical agenda on our clients simply because I wanted to use React. Granted, we built some killer stuff for our clients and React enabled that.

Lots of the other upsides to headless CMS were true, for sure. A CMS should obviously be API-first nowadays. That's a given that doesn't even need to be stated anymore. But I can count on one hand the amount of times we used a headless CMS for OmNiChAnNeL content. Most all of our headless CMS builds were just used to power a single website. Actually reading about how some of the big headless CMS position themselves as thought leaders around decentralized / omnichannel / content hub / etc. just makes me laugh at this point.

Anyway, our move to headless certainly posed struggles for us along the way, especially early on. Here's a few:

Keeping CMS changes in sync with the frontend

When we'd update a content model on the CMS side, we'd need to very deliberately think about how to deploy all of our changes to the CMS at the exact same time as our changes to our frontend to minimize downtime and prevent breaking code.

Doesn't matter if we used Contentful, Sanity, Wordpress, whatever—the CMS being separate meant that we had to orchestrate deployments with caution.

Sharing TypeScript types

A good CMS (like Payload 😈) can generate TypeScript interfaces for you which could theoretically be re-used on your frontend. But, with the separation of the CMS and the frontend, you have a few equally bad options to share these types:

Manually copy / paste types from the CMS to the frontend repo
Publish a third-party types package that can be leveraged for both backend and frontend (still need to copy / paste)
Use a monorepo and get wild with scripts to automate your copy/pasting

Rough edges to do with "preview"

Back in the monolithic CMS days you'd log into your CMS and then browse around on your frontend. You'd have a nice little "admin bar" that was purpose-built to give connectivity to your frontend and your backend, but we've lost a lot of that with the move to headless.

There are "headless" CMSs like Storyblok who give you a full frontend visual editor, but... is that still headless? Cool in any way you look at it for sure, but I question if a CMS can call itself "headless" if it literally gives you a preview environment for your website - the "head".

You need to have a strong engineering team

Lots of the pushback I would hear from my clients, and rightfully so, is something like this:

"What happens if you and your team go away? Will we be stuck with a beautiful engineering masterpiece with no one smart enough / cost-effective enough to work on it? If we use WordPress, we can grab a developer off the street that works for cheap."

This always reminded me of Ducatis. Beautiful machines, but RIP if you have to work on one. If you've got a crazy dev environment that has three separate services all combined, sharing code in separate NPM packages, the onboarding for a developer coming into the project is gonna be steep.

Let's take a second to remember that we're doing this to ourselves. All for the sake of a website in most all cases.

It's time for things to get simpler

Over the past quarter, the Payload team has been thinking about all of the above quite a bit. My thesis on the topic is that the whole web dev ecosystem perpetually swings back and forth from complexity to simplicity. Someone comes up with new tech, and it's complex, but worth it. And then people get tired of the complexity and want something simpler. And then someone makes it simpler. Repeat for the rest of our lives.

I think right now we are swinging back toward simplicity. Case in point: I'm personally tired of manually solving for headless CMS "rough edges" if we're just building a simple website.

Many of you already know that one of Payload's core pillars is simplicity. Is there a way that we can solve some of the above complexity issues with Payload?

Where we're going at Payload

We envision a way to make Payload embeddable in just about any environment - be it an Express server, serverless functions, or edge functions. We want it to be portable so that you can install it within your website itself, no matter if your website is running Svelte, Next, Remix, Vue, Gatsby, whatever. If your environment can run Node, it should be able to run Payload.

We want to promote simplicity once again. Building a website? Make it easy on yourselves, not harder. Install Payload in the same repo as your frontend. Share types, simplify dev ops, take some of the rocket science away.

To realize this vision, the Payload team and I have dedicated much of the last few months to simplifying, streamlining, and modularizing our codebase so that it can be run directly in as many different types of projects as possible.

The first steps

We've simplified our code so that the Payload Local API is as barebones as possible. No HTTP overhead, no GraphQL bloat, just Node methods that are strongly typed and hit your database directly. It's almost like an ORM but it gives you an admin panel as well. This allows you to bring Payload into any project, even if it's not Express-based.

Yesterday, we announced that you can now deploy Payload serverlessly, within an existing NextJS app, on Vercel.

This is a first step, but it's a monumental shift from the "headless CMS" paradigm. You can now use Payload directly in Next server components to get strongly typed data, hitting your database directly, without any HTTP overhead. It's kinda like we're giving an optional "head" back to the headless CMS.

// Here's a NextJS server component
const Page = async ({ params: { slug } }) => {
  const payload = await getPayload();

  // Zero HTTP overhead - this doesn't hit a REST API,
  // it goes straight to your database
  const pages = await payload.find({
    collection: "pages",
    where: {
      slug: {
        equals: slug,
      },
    },
  });

  const page = pages.docs[0];

  if (!page) return notFound();

  return (
    <React.Fragment>
      <AdminBar adminBarProps={{ collection: "pages", id: page.id }} />
      <Hero {...page.hero} />
      <Blocks blocks={page.layout} />
    </React.Fragment>
  );
};

Being able to deploy Payload within your app, wherever your app is running, will solve for basically all of the above complexities that I mentioned. It'll be faster, simpler, and require less black magic.

Future goals

We're going to keep chasing this idea of simplicity. In the future, you'll be able to do this for any type of app. It doesn't need to be React or even Next. Our goal is to let you install Payload wherever you need it, and simplify your stack.

We're also already working on being able to deploy Payload within edge functions. We've got a bit of work to go on that, but it'll happen.

Serverless not your thing? Can't deal with cold starts?

Then you can combine Payload and NextJS into a single server-based repo as well. You can even deploy it on Payload Cloud.

There are edge cases, of course

I've been going on about how most people always deploy CMS in a website, blah blah. But that doesn't mean that Payload is going to lose track of our API-based DNA. There are lots of cases where deploying a CMS separately will still be the move, and any CMS should have omnichannel capabilities nowadays via REST / GraphQL / etc. no matter where or how it's deployed. I just think that is a given at this point.

Conclusion

It's time the headless CMS paradigm moved back toward simplicity. With our launch of the next-payload package, we're taking an important first step towards restoring simplicity.

I'm not sure of what the word should be that describes what I'm trying to get to but I hate coining new words for obvious concepts.

Some other CMS use the word "composable", but I feel like that's obvious if you're working with an API-based CMS. But what is true composability? Microservice / SaaS hell? Yes, let me go sign up for a SaaS CMS, then go sign up for a forms provider, then go sign up for Hubspot, then go sign up for a mail service, and then deploy my site on Next / Vercel. All just for a website? I don't care how big the site is, that doesn't sit right with me. Actually that makes me kinda want to puke.

Let's fix the complexities inherent with headless CMS and see how far we can take this idea of simplicity. Let's make our jobs easier. That'll allow us to deliver better products.

Launch Week Day 3 - Bulk Operations

Sean Zubrickas — Wed, 05 Apr 2023 14:23:52 +0000

The long awaited new feature lets editors edit, delete and publish many documents of a collection all at once.

We’ve improved the editor experience by adding selection in the collection list view and new controls to Edit, Delete, and if drafts are enabled, Publish, and Unpublish. We also added API endpoints to handle the changes and also extended the local API for programmatic access. The new features are available in v1.6.24 and later.

Delete, Publish, Unpublish Many

The workflow for these are all much the same, select a bunch of documents and go for it. You’ll get a prompt to make sure you are happy with your life choices before submitting. The changes are all handled in one request. After submitting, the UI shows the number of updated documents. If there are any errors that occur due to validation or any faults in a hook on the backend, the number of documents that didn’t get updated due to error is also shown. The results are more detailed in the HTTP response or local API for your apps to deal with.

Edit Many

The editing experience for multiple documents in a collection allows the editor to choose which fields to update. You can access all fields within a nested structure with the field selector and input your changes before saving. Fields that are hidden, unique or read-only are not available, and you can also configure the admin.disableBulkEdit option to remove certain fields.

When drafts are enabled for a collection, you also have the choice to publish or save as draft. That gives you three ways to submit:

Save updates the published document or most recent draft document in place without changing any statuses.
Publish takes your changes, adds it to the most recent document version and publishes the results.
Draft saves a draft version of your changes along with the most recent version of each document and does not affect what you currently have published.

Filter and Select All

Instead of selecting individual rows in the list, an editor can select all documents in the list. The select-all option can be used with the existing search and filter options to only include all documents matching certain criteria. With this, the bulk edit, publish, unpublish and delete workflows will use the query to perform the operation against all the matching items.

API Changes

The new functionality extends to the backend and local API as well.

There are new REST endpoints for collections to allow for bulk delete and updates:

PATCH /api/pages
DELETE /api/pages Each endpoint takes a query in the same form as a GET request would to make our changes against. The local APIs have also been extended to take either an id or where query.

const result = await payload.update({
      collection: 'posts',
      where: {
        id: { in: ids },
      },
      data: {
        title: 'Doc Title',
      },
    });

// result: 
// { 
//    docs: [{...}]
//    errors: []
// }

Try it out

Update your own Payload project or head on over to the public demo and give it a shot.

Launch Week Day 2 - Community Help

Sean Zubrickas — Tue, 04 Apr 2023 15:03:07 +0000

Being able to find solutions easily and leverage an active community all sharing knowledge together has a huge impact on both the quality of your code and the quality of your life.

With this in mind, we’re excited to introduce our new Community Help archive.

Community Help builds upon our active and supportive communities on Discord and GitHub. With over 500 resolved threads, there's a good chance that your question has been asked before, a similar issue has been encountered or a code example that you are looking for has been shared.

But the problem up until now was that GitHub Discussions are terrible for SEO for some reason, and Discord help threads are locked behind needing to download yet another app.

So to make this information way more accessible, we’re directly fetching all this data from Discord/GitHub and displaying it on the Payload website’s Community Help page. (p.s. don’t forget we are open source so feel free to check out how we did this!)

To help you find what you need faster, we've integrated Algolia Search to index every question, answer, and comment. This lets you use keywords to search on any part of the thread so you won’t miss out on anything relevant. You can also view the thread directly in your browser without needing to visit or sign up for external sites.

The first thing most of us do when we need help is turn to a search engine (i.e…. we bing it….just kidding). But when’s the last time you ever landed on a GitHub discussion from Google? Think about it… You might land on an issue, but rarely on a discussion.

With all of our GitHub Discussions and Discord community help threads output as static pages in our NextJS site, over time, we’re also hoping to start to see way more results from the Payload community directly in your Google search results.
This section of our site will continue to evolve and become more powerful as our archive of answers expands.

We’re extremely grateful for our enthusiastic and engaging community and this would not be possible without them. A few of you are absolute maniacs about helping each other and we just want to take a second to say - - THANK YOU.

Head over to the Community Help page and let us know what you think!

Launch Week Day 1 - Payload Cloud is here

Sean Zubrickas — Tue, 04 Apr 2023 15:01:21 +0000

Payload Cloud is now available, and completely free through beta (July 1). It's by far the most seamless way to deploy Payload for production.

If you browse through our Discord community help forums, a big slice of the questions people ask are around deployment-related issues.

We want to make it easier for those of you that don't want to deal with that racket (me included). Of course, Payload will always be open-source and able to be self-hosted, but starting today, you've got a better option.

What is Cloud?

Today, we're launching the public beta of Payload Cloud. It's a deployment platform that works just like Vercel, but for regular Mongo + Express apps that run a server. Connect your GitHub repo and boom — we deploy everything you need for production instantly. It's not a SaaS CMS — instead, it's your CMS, deployed for you. We just give you the plumbing.

Every piece of infrastructure that we deploy — database, S3 storage, compute, email — is exposed to you, and you can do with it what you need. We just make it easy and tailor the experience to CMS-related expectations. No more vendor hell.

Everything is free until July 1

We are releasing Cloud as a beta, and that means you get everything for free until July 1. We'll foot the infrastructure bills for you until then in exchange for you helping us ensure our infrastructure is solid.

What Cloud gives you

Even on the smallest Standard tier, we've done the hard stuff for you. Out of the box, you get the following:

Cloudflare CDN

Your app will sit behind Cloudflare which means that you get everything that Cloudflare provides, like DDoS protection, automatic SSL certificates, a global CDN with caching, and custom domain support.

We've worked tirelessly with Cloudflare to even allow you to use apex domains, which is rare.

I should take a second here to shine some light on this.

Have you ever tried to deploy an app to Heroku or DigitalOcean's App Platform on your root domain? Let's say, example.com - NOT a sudomain like www.example.com. With most DNS providers, you can't. That's because example.com is an "apex" domain, and with most DNS providers, you need to point your apex domain to a static IP like a DigitalOcean droplet. So you're forced to jump through all sorts of hoops or just make a compromise.

But with Payload Cloud, we've configured our Cloudflare implementation with static IP support — so if you want to deploy your CMS and your frontend together, and host it directly on an apex domain, Payload Cloud gives you a static IP to point your domain to.

MongoDB Atlas

We work closely with Mongo and have partnered with them to automatically provision a database for you, running on Atlas. Payload Cloud allows you to rely on the Atlas platform's speed, stability, and scale instantly. Your database is yours. We give you the credentials. Do with it what you will.

S3 file storage

Store your files on S3 with your own bucket access. Among other things, the new @payloadcms/plugin-cloud package will handle storing all of your files on S3 seamlessly for you when deployed to Payload Cloud. As with Mongo Atlas, we give you the creds, and you can do what you want with your files. They're yours.

App layer with automatic deploys

No more pm2. No more push / pull to a DigitalOcean droplet. Just connect your repo, and automatically deploy your code on push. Manage environment variables, monitor runtime logs, and more completely out of the box.

Deploy your frontend, too

I have a theory about the state and future of headless CMS. The "headless" concept came about because people wanted to bring their own frontend. Of course, the omnichannel content aspect of headless is great, but let's not kid ourselves—many of you are just powering the content for a single website / app.

That's the funny thing about web development—the whole ecosystem is like a pendulum. It swings back and forth from complexity to simplicity and right now I think we're swinging back to simplicity.

If the majority of us are all just powering websites with headless CMS, how about we allow ourselves to combine the frontend and backend again? The CMS can still be API-based and "headless", but if we could combine the two, we can regain some of the seamlessness that we had with monolithic solutions. It's more "composable" than "headless".

That would allow us to once again do stuff like:

Build more efficiently and get our jobs done with less effort
Deploy updates to the CMS and the frontend in tandem with each other
More easily manage environments and feature sets
Share TS interfaces from backend to frontend without having a third shared utils NPM package or similar
Build a better admin UX between frontend and backend using things like showing an "admin bar" while the user is logged in and browsing the frontend

One of the coolest parts about Payload Cloud is that if you want, you can deploy your entire stack (CMS and frontend) directly on one service. You no longer need to deploy your CMS separately from your frontend if you don't want to. This can be huge for cases where you, as engineers, want to simplify and work as efficiently as possible.

At its core, Payload Cloud is built to deploy Express + Mongo apps. Nothing more. So that means you can combine your Next + Payload apps directly in one and deploy it once. Put Remix and Payload in the same server. Run Sveltekit and Payload next to each other. Roll your own React frontend with Vite and deploy it seamlessly next to Payload.

Make your job easier. There are lots of possibilities here.

Through the rest of this week, we've got more announcements that make Payload able to be "composed" within your other apps, but this is definitely one of the side-effects of Payload Cloud that I'm most excited about.

What's coming

This is the very beginning for Payload Cloud. Over the next few months until our exit from public Beta on July 1, we've got lots more coming.

Free tier

First up, we want to launch our free tier. To do so, we need to keep working on how to reduce our own cost for this, because surprise surprise, there is no free lunch. Someone's paying for free stuff and it's gonna be us. We're happy to do so, because we want to make the developer experience for Payload as beautiful as possible. But we need to make sure that this is scalable and affordable, so we're going to try and revisit this as soon as we can.

Email out-of-the-box

We're working with Resend to give you a full, modern email service for every Cloud installation. This is almost done and will be a fast-follow to our beta launch today.

Programmatic CDN and purging

We want give you a way to utilize Cloudflare's CDN for everything, programmatically, across your app + frontend. Even Payload API responses. In the future, the @payloadcms/plugin-cloud package will automatically cache all applicable API responses, and will dynamically invalidate CDN cache when documents change. We'll also be exposing a way for you, as a developer, to programmatically purge CDN cache as you need.

On-demand backups

Right now, MongoDB Atlas manages automated backups for you, so backups do exist out of the box already today. But we want to expose them to you via our UI, to allow you to manage and create your own backups at your discretion. We also want to extend the "backup" idea to connect to files as well, so that when you take a backup, or one is scheduled, we give you a tidy little package containing your DB and your files, ready to go for use somewhere else.

Environment management

We're going to be releasing a full suite of environment management controls that will allow you to sync your DB and your files from project to project seamlessly. That will allow you to build multiple environments like dev, stage, and prod, while being able to seamlessly migrate files + data from one environment to another.

Usage monitoring UI

We have usage monitoring and reporting on our side, but we want to build this into the admin UI so that you can be informed of your usage in a simple and consistent manner directly within the Payload Cloud UI.

This is only the beginning

Over time, Payload Cloud will continue to evolve to be the best way to deploy Express + Mongo apps. It's only going to get better from here, and we need your help in making it so.

Ready to dive in?

Deploy your app in minutes — get started with Payload Cloud!

Redirects in Payload — Retaining SEO Value and Avoiding 404s

Sean Zubrickas — Wed, 08 Mar 2023 13:20:42 +0000

If you're re-platforming to Payload (hooray!) you need to have a proper approach for redirects in place to ensure a successful migration.

Having bulletproof redirect functionality in place will ensure you retain your current SEO value on existing pages and avoid any 404s. Google (mainly) and other search engines have gotten much more strict about how they rank websites, specifically those that have 404s, or pages that lead to nowhere.

Additionally, the marketing team has likely done a lot of work building up their SEO ranking, and it's extremely important to keep that in place after a migration.

Surprisingly, many headless content management systems don't offer redirect functionality out of the box, and you're stuck with the hassle of building this yourself.

We've been there, and that's exactly why we built our redirects plugin. It's easy for you to integrate and even easier for marketers or content managers to use which lets you focus on the rest of your project.

To make it easy to familiarize yourself with our redirects plugin, we've built an example with it already installed. Clone down the Payload repo if you haven't already, and look in the examples folder for redirects. There is a straightforward readme that will get you up and running in minutes.

For the purposes of this post, I've already got our redirects example up and running so let's dive in!

As you can see in the screenshot below, the redirects plugin creates a new collection called Redirects, which is where you'll create and manage all of your redirects.

Now, if you take a look at the example below, you'll see our payload.config is pretty straightforward. We have a pages collection, and when we go to setup our redirects we will have the ability to select documents from that collection.

import redirects from '@payloadcms/plugin-redirects'
import path from 'path'
import { buildConfig } from 'payload/config'

import { Pages } from './collections/Pages'
import { Users } from './collections/Users'
import { MainMenu } from './globals/MainMenu'

export default buildConfig({
  collections: [Pages, Users],
  cors: ['http://localhost:3000', process.env.PAYLOAD_PUBLIC_SITE_URL],
  globals: [MainMenu],
  typescript: {
    outputFile: path.resolve(__dirname, 'payload-types.ts'),
  },
  plugins: [
    redirects({
      collections: ['pages'],
    }),
  ],
})

If you click into one of the redirects you'll see two fields. The fields themselves are pretty straightforward - you have a From URL field, which is the URL string that will be matched up against the requested path, and the To URL field, which is conditional, and provides the option to link to a custom URL or internal doc.

The nice thing is that don't need to know all the ins and outs of how the plugin works, but we suspect you're going to dig around anyway and we're all for that.

Wrap up

We understand that redirects aren't the most exciting thing in the world, but being able to simply drop this into your project will make things much easier. With our redirects plugin, you've got a solution for almost any scenario you'll be up against.

I hope you found this post helpful and that you can start utilizing this powerful plugin for your Payload project. Cheers.

Learn More

To learn more about the official redirects plugin, Payload CMS and Next.js, take a look at the following resources:

Get Started

yarn add @payloadcms/plugin-redirects

Like what we're doing? Give us a star on GitHub

We're trying to change the CMS status quo by delivering editors with a great experience, but first and foremost, giving developers a CMS that they don't absolutely despise working with. All of our new features are meant to be extensible and work simply and sanely.

Stop by GitHub and give us a star!

Announcing the Write for the Community Program

Sean Zubrickas — Thu, 16 Feb 2023 12:41:43 +0000

Payload's Write for the Community program offers a unique opportunity to reward developers for contributing to an open-source project.

Our community has been instrumental in the breadth of features we've added to Payload over the past few years. We'd like to keep that momentum going and make it a worthwhile endeavor for all involved. That's precisely why we created the Write for the Community program.

The WFC program evolved from a handful of successful collaborations with a few of our contributors. This program will encourage developers and content creators to share their knowledge and expertise with the Payload community. By contributing articles, tutorials, and other types of content, participants can help make Payload more accessible, user-friendly, and feature-rich, and get rewarded in the process.

What's in it for me, exactly?

Whether it's an article, repo, or both, we'll make it worth your while. Exact compensation will be determined by the value of the contribution. We're determined to expand our library of resources, and this program will help make that possible.

In addition to the monetary rewards, you'll have the opportunity to showcase your skills and expertise, gain exposure, and build your professional network. By contributing to the Payload community, contributors can establish themselves as thought leaders and experts on Headless CMS and web development.

Read all about the program here.

Like what we're doing? Give us a star on GitHub

Stop by GitHub and give us a star!

Get up and running with one line

Getting started with Payload is easy—and free forever. Just fire up a new terminal window and run the following command:

npx create-payload-app