Forem: Theo Pendle

Implementing Hash-Based Strict CSP on AEM

Theo Pendle — Sun, 23 Jun 2024 12:28:34 +0000

As always, the full solution is available on Github. Scroll to the bottom of the article for the link.

Introduction to CSP

As a reminder, CSP stands for Content Security Policy: a security standard that helps prevent cross-site scripting (XSS), clickjacking, and other code injection attacks by controlling what resources a user agent is allowed to load for a given page using the Content-Security-Policy header.

Perhaps the most critical directive of CSP is the script-src directive, which controls what JS code the browser can load and execute.

The highest level of protection is achieved by using the so-called 'strict' CSP. If you want to know more about strict vs non-strict CSPs, including examples and rationales, please visit these excellent resources:

There are two methods for implementing a strict CSP: nonces or hashes.

Defining the requirements

Let's first list the features of the ideal CSP solution to help us pick the right approach:

It should provide the highest level of security (ie: strict CSP)
It should work for both author and publish instances (ie: it should not rely on a publish-only dispatcher)
It should be easy to maintain
It should be cacheable
It should work for <script> tags that:
1. point to internal clientlibs
2. contain inline JS
3. point to external clientlibs (eg: external analytics script)

Hashes vs Nonce-nse

While there are certainly cases where the nonce approach makes sense, in my opinion, hashes are a superior solution for most CMS use cases. That's because using nonces presents the following disadvantages:

Because it requires the HTML document to contain these nonces that are unique to each request, it makes the result impossible to cache and fails requirement n.4. Since caching is a critical performance optimization, this is usually disqualifying.
It's not a useful mechanism for protection against untrusted external scripts, so it fails requirement 5.iii. This kind of protection would require an integrity check which is, you guessed it, a hash (which we can re-use for our CSP!)

Hashes, by comparison, can be cached because they are specific to the script content, which typically only changes with a software release and can be used to validate the integrity of scripts from untrusted sources.

Therefore, we can conclude that a hash-based CSP is the best solution for most AEM use cases.

If the above rationale doesn't apply to your use case, then this Medium article by Saravana Prakash can show you how to achieve nonce-based CSP this in AEM.

Solution design

Now that we know what approach to take, let's design the solution.

Where do scripts come from anyway?

There are typically 3 ways in which <script> tags are added to a page's HTML:

Added directly via HTL files that make up the Page component (eg: customfooterlibs.html or customheaderlibs.html)
Added indirectly via the Page Policy:
Added as dependencies to clientlibs defined in points 1 and 2.

So the sequence of events should be:

Let AEM add all the <script> tags in the page HTML
For each <script> tag:
- Calculate the hash using one of the following:
  - The inline content of the script
  - The content of the references clientlib
  - The integrity attribute of the untrusted script
Add it to the tag
Add it to the CSP header

Transformers to the rescue!

Photo by Aditya Vyas on Unsplash

Unfortunately I'm not talking about Optimus Prime, but rather a SAX output pipeline that will use a Transformer to add the CSP hashes to the <script> tags on the HTML page.

Implementation

In this section I will highlight the most important parts of the solution. For a complete solution, see the Github link at the bottom of the article.

Creating the transformer

The transformer handles the following cases:

Inline scripts

Example:

<script>
    console.log('Hello, World!');
</script>

If the <script> tag is inline, the hash can be calculated using the innerText of the element.

Clientlib scripts

Example:

   <script src="/etc.clientlibs/demo/clientlibs/clientlib-site.min.js">

If the <script> tag points to a clientlib served from AEM, the hash can be calculated using the content of the clientlib.

External scripts

Example:

   <script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js" integrity="sha384-YvpcrYf0tY3lHB60NNkmXc5s9fDVZLESaAA55NDzOxhy9GkcIdslK1eN7N6jIeHz" crossorigin="anonymous"></script>

Here is the code for the transformer. I've been very generous with the comments to explain the rationale behind each step. This snippet refers to some POJOs and configuration that you can see in the Github diff at the end of the article.

@RequiredArgsConstructor
@Slf4j
public class CspHashTransformer extends DefaultTransformer {

    @Getter
    private final String hashingAlgorithm;

    private final HtmlLibraryManager htmlLibraryManager;

    private SlingHttpServletRequest request;
    private SlingHttpServletResponse response;
    private ContentSecurityPolicy csp;
    private TransformerElement currentElement;

    @Override
    public void init(final ProcessingContext context, final ProcessingComponentConfiguration config) throws IOException {
        super.init(context, config);
        request = context.getRequest();
        response = context.getResponse();
        csp = new ContentSecurityPolicy();

        // We initialize the CSP with a strict-dynamic directive to allow for our trusted scripts to
        // load other scripts without being blocked by the browser.
        csp.addScriptSrcElem("'strict-dynamic'");
    }

    /**
     * Process the start of an element. If the element has a src attribute pointing to a clientlib, calculate the hash
     * and add it to the element as an integrity attribute and to the CSP header.
     *
     * @param namespaceUri  the namespace URI of the element
     * @param localName     the local name of the element
     * @param qualifiedName the qualified name of the element
     * @param attributes    the attributes of the element
     * @throws SAXException if an error occurs during processing
     */
    @Override
    public void startElement(final String namespaceUri, final String localName,
                             final String qualifiedName, final Attributes attributes) throws SAXException {

        currentElement = new TransformerElement(namespaceUri, localName, qualifiedName, attributes);

        log.debug("Start processing element {}", currentElement);

        addIntegrityAttributeAndCspForSrc();

        super.startElement(currentElement.namespaceUri(), currentElement.localName(), currentElement.qualifiedName(), currentElement.attributes());
    }

    /**
     * Called by the SAX parser when it encounters character data. Used to append the character data to the inner text
     * of the current element.
     *
     * @param ch     the character array being read
     * @param start  the start index of the character array
     * @param length the length of the character array
     * @throws SAXException if an error occurs during processing
     */
    @Override
    public void characters(final char[] ch, final int start, final int length) throws SAXException {
        if (currentElement != null) {
            currentElement.innerText().append(ch, start, length);
        }
        super.characters(ch, start, length);
    }

    /**
     * Process the end of an element. If the element has inner text, calculate the hash and add it to the CSP header.
     *
     * @param namespaceUri  the namespace URI of the element
     * @param localName     the local name of the element
     * @param qualifiedName the qualified name of the element
     * @throws SAXException if an error occurs during processing
     */
    @Override
    public void endElement(final String namespaceUri, final String localName,
                           final String qualifiedName) throws SAXException {

        if (currentElement == null) {
            return;
        }

        log.debug("End processing element {}", currentElement);

        addCspForInnerText();

        super.endElement(namespaceUri, localName, qualifiedName);
    }

    private void addIntegrityAttributeAndCspForSrc() {

        // Get the source of the script
        final String src = currentElement.attributes().getValue("src");
        if (src == null) {
            log.debug("No src attribute found for element {}", currentElement);
            return;
        }

        // Attempt to find a clientlib associated with the src attribute
        final HtmlLibrary clientlib = getHtmlLibrary(src);

        // Attempt to read the integrity attribute
        final String integrity = currentElement.attributes().getValue("integrity");

        // If no clientlib can be found using the src, then assume the src is external
        final boolean isExternal = clientlib == null;

        // For security reasons, we consider that an external script without an integrity attribute is invalid. It will
        // not be added to the CSP and therefore will fail to load/execute in the browser.
        if (isExternal && integrity == null) {
            log.error("Integrity attribute missing from external src <{}>. Hash cannot be calculated.", src);
            return;
        }

        // Re-use the integrity hash if possible, else calculate the hash from the clientlib content
        final String hash = isExternal
                ? integrity
                : getHashFromClientlib(clientlib);

        // If no hash can be calculated, then the script will not be added to the CSP and therefore will fail to load
        if (hash == null) {
            log.debug("No clientlib or external hash found for found for src <{}>. Hash cannot be calculated.", src);
            return;
        }

        // For internal script, add the integrity attribute containing the hash. Security-wise this does not provide any
        // benefit as the CSP will already enforce the hash, but it is good practice to include it so that you can
        // easily identify which script corresponds to which hash for debugging puposes.
        if (!isExternal) {
            addIntegrityAttribute(hash);
        }

        // Finally, add the hash to the CSP
        addHashToCsp(hash);
    }

    private String getHashFromClientlib(final HtmlLibrary clientlib) {
        try (final InputStream inputStream = clientlib.getInputStream(true)) {
            final String hash = calculateHashAndEncodeBase64(inputStream);
            log.debug("Hash for <{}>: <{}>", clientlib.getPath(), hash);
            return hash;
        } catch (final IOException e) {
            log.error("Could not read clientlib <{}>", clientlib.getPath(), e);
            return null;
        }
    }

    private void addCspForInnerText() {
        final String innerText = currentElement.innerText().toString();
        if (innerText.isEmpty()) {
            log.debug("Element {} has no inner text", currentElement);
            return;
        }

        final String hash = calculateHashAndEncodeBase64(innerText);
        addHashToCsp(hash);
    }

    /**
     * Adds the hash as an integrity attribute to the current element.
     *
     * @param hash the hash to add
     */
    private void addIntegrityAttribute(final String hash) {
        final AttributesImpl attributes = new AttributesImpl(currentElement.attributes());
        attributes.addAttribute(currentElement.namespaceUri(), "integrity", "integrity", "0", hash);
        currentElement.attributes(attributes);
    }

    /**
     * Adds the hash to the Content-Security-Policy header.
     *
     * @param hash the hash to add
     */
    private void addHashToCsp(final String hash) {
        csp.addScriptSrcElem("'" + hash + "'");
        response.setHeader("Content-Security-Policy", csp.toString());
    }

    /**
     * Find the clientlib associated with the src attribute if such a clientlib exists.
     *
     * @param src the src attribute of the element
     * @return the clientlib associated with the src attribute, or null if no such clientlib exists
     */
    private HtmlLibrary getHtmlLibrary(final String src) {
        final String path;
        try {
            path = new URI(src).getPath();
        } catch (final URISyntaxException e) {
            log.error("src attribute element {} is not a valid URI", currentElement, e);
            return null;
        }

        // Find true path of clientlib in /apps (or /libs, via overlay)
        final String appsPath = path
                .replace("etc.clientlibs", "apps")
                .replace(".min.js", "");

        final Resource resource = request.getResourceResolver().resolve(appsPath);
        if (resource instanceof NonExistingResource) {
            log.error("Could not find resource using path <{}>", path);
            return null;
        }

        return htmlLibraryManager.getLibrary(LibraryType.JS, resource.getPath());
    }

    private String calculateHashAndEncodeBase64(final InputStream inputStream) {
        try {
            final MessageDigest digest = MessageDigest.getInstance(hashingAlgorithm);
            final byte[] buffer = new byte[4096];
            int bytesRead;

            while ((bytesRead = inputStream.read(buffer)) != -1) {
                digest.update(buffer, 0, bytesRead);
            }

            final byte[] hash = digest.digest();
            final String hashString = Base64.getEncoder().encodeToString(hash);
            return hashAndAlgorithm(hashString);

        } catch (final IOException e) {
            log.error("Error reading input stream for hashing", e);
            return null;
        } catch (final NoSuchAlgorithmException e) {
            log.error("Encryption algorithm not found", e);
            return null;
        }
    }

    private String calculateHashAndEncodeBase64(final String string) {
        try {
            final MessageDigest digest = MessageDigest.getInstance(hashingAlgorithm);

            final byte[] hash = digest.digest(string.getBytes(StandardCharsets.UTF_8));
            final String hashString = Base64.getEncoder().encodeToString(hash);
            return hashAndAlgorithm(hashString);

        } catch (final NoSuchAlgorithmException e) {
            log.error("Encryption algorithm not found", e);
            return null;
        }
    }

    private String hashAndAlgorithm(final String hash) {
        return hashingAlgorithm.toLowerCase().replace("-", "") + "-" + hash;
    }
}

Adding the transformer to the pipeline

To create a pipeline that includes our transformer, we need to create a Rewriter by adding node at /apps/demo/config/rewriter/links-pipeline with the following properties:

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0"
          jcr:primaryType="nt:unstructured"
          contentTypes="text/html"
          generatorType="htmlparser"
          order="1"
          paths="[/content]"
          serializerType="htmlwriter"
          transformerTypes="[csp-hash-transformer]">
    <generator-htmlparser
            jcr:primaryType="nt:unstructured"
            includeTags="[SCRIPT]"/>
</jcr:root>

The result

Now, if we load scripts onto our page using customfooterlibs.html:

<!-- This HTL include demonstrates the loading of a clientlib -->
<sly data-sly-use.clientlib="core/wcm/components/commons/v1/templates/clientlib.html">
    <sly data-sly-call="${clientlib.js @ categories='demo.base', async=true}"/>
</sly>

<!-- This script element demonstrates the loading of an external script -->
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.3/dist/js/bootstrap.bundle.min.js"
        integrity="sha384-YvpcrYf0tY3lHB60NNkmXc5s9fDVZLESaAA55NDzOxhy9GkcIdslK1eN7N6jIeHz"
        crossorigin="anonymous"></script>

<!-- This script element demonstrates the inline script hashing -->
<script>
    console.log('This was logged from an inline script!');
</script>

<!-- This inline script loads an external script to demonstrate the 'dynamic' principle -->
<script>
    const script = document.createElement('script');
    script.src = 'https://cdn.jsdelivr.net/npm/jquery@3.7.1/dist/jquery.min.js';
    document.body.appendChild(script);

    addEventListener("load", (event) => console.log('jQuery version:',$().jquery));
</script>

We should receive a CSP like this (yours will vary depending on your clientlibs/dependencies):

Content-Security-Policy: script-src-elem 'strict-dynamic' 'sha256-47DEQpj8HBSa+/TImW+5JCeuQeRkm5NMpJWZG3hSuFU=' 'sha256-XjA+iLg5j0FvhFkZc7LcXfbQJ0b3gvw2c2jj9vv65q0=' 'sha256-Uv8dzRTPRZ2++L/ZWgfN9lPjdvzsDYVS4rEfvWCA0x0=' 'sha256-3dfW5u+XJXRPqcC3F8wewmnAr6oxejxP7ArjOE38P2Q=' 'sha256-5hrKOpQWBa1NuajxV3udxJCgNMQMD/lUApbmGxMmpuM=' 'sha256-wlCSQBL9yeqVFrMGUIlSAc0Wfb1JydFIkk8wiBq/o5M=' 'sha256-WJ3od+zqoblT5apcuXdUh4o1UWwVnb5AjQhmHWIu2OY=' 'sha384-YvpcrYf0tY3lHB60NNkmXc5s9fDVZLESaAA55NDzOxhy9GkcIdslK1eN7N6jIeHz' 'sha256-QFYsdZ/eGhCq89XHZ7IOsy5A9dTKeyvduAx2RnCqvAA=' 'sha256-Fb8sXaPGzkkQJOoKLIpn0I5s+VOOiBlZMYGgv8wHxZI=' 'sha256-g2cCN9gX44Hp5lFL/iomg3hI3LeG/LRkzeNJfQZjJGI=';

And you should see that the demonstration scripts have executed as expected in the browser console:

This was logged from an inline script!
jQuery version: 3.7.1

What about the other CSP directives?

Good question! There are indeed dozens of other directives you can use to fine-tune your CSP.

This article will not give you a comprehensive strategy for dealing with all your CSP requirements, it only shows you how to automate the configurationscript-src-elem directive.

Thankfully, using multiple CSP headers is a valid approach, so you can add the rest of the directives anywhere you like as additional CSP headers. Just make sure you understand how multiple CSP headers interact with each other to avoid surprises.

Conclusion

As promised, all the code is available in one easy-to-read diff on Github. You can find it here.

If you have any comments or ideas about this article, the topic matter or the format, don't hesitate to leave a comment or to reach out to me on LinkedIn!

Four-eyes Validation in AEM

Theo Pendle — Sun, 14 Jan 2024 14:51:53 +0000

Prevent initiators from approving their own workflows

💡 As always, scroll to bottom of the article to find all the source code on Github!

Pre-requisites

In order to get the most out of this article, make sure you are at least familiar with the following concepts:

Workflows (see: Work with Workflows)
Customizing workflows and steps (see: Workflow Step Reference, Custom Process Step)
Participant steps and choosers

Use cases

It is a common business process to include an approval of some kind before any corporate content is published, committed, filed or broadcasted. A four-eyes validation adds a certain level of confidence as it means that at least two individuals have seen a piece of content (the creator and an approver) before it goes live.

Web content management is no exception, and AEM supports the ability to approve content before release using workflows. More specifically, I will be referring to the Request for activation workflow.

As a reminder, the Request for activation workflow is used when a user who does not have crx:replicate permissions attempts to publish a page. By default, the content must be approved by a member of the administrators group:

However, depending on how your organisation envisions its content delivery pipeline, the supported functionality may be not be sufficient. Let's look at some examples:

Centralized authoring

In this model, web content is managed by a centralized web team that receives briefs from business teams and converts them into AEM web pages.

Once the content is ready, the team which issued the brief is asked to approve the content to make sure it meets their expectations.

Pros	Cons
Content management stays in the hands of the expert AEM authors	The whole organisation relies on the web team for any updates. This creates a bottleneck.

De-centralized authoring

In this model, some teams may be allowed to manage their own limited domain. For example, HR can publish new job offers while sales can maintain its own marketing materials.

In this case, some team members are responsible for creating content, which then get approved by their managers before publication.

Pros	Cons
Simpler content can be managed independently by the topic experts	If a manager is busy or unavailable, delivery will slow down
	Managers may end up doing a lot of little-value-added approval work

Democratized authoring

In this model, teams are empowered to deliver content for which they are responsible in the most independent and streamless manner.

Content can be approved by any member of the team, except the creator.

If you are reading this, you are probably a software engineer familiar with the concept of code review. This model follows the same principles!

Pros	Cons
Simpler content can be managed independently by the topic experts
Creator and approver roles are flexible
Delivery velocity is maximised. No need to call the manager in order to approve typo fixes or style changes, etc.

Hybrid model

Of course, it is common to want to mix and match authoring models. Some content might be managed by the web team as per the Centralized model, whereas other content may be managed by the business teams as per the Democratized model. Yet other content might be deemed sensitive and require a manager-level approval as per the De-centralized model, etc.

Problem statement

The Centralized and De-centralized models described above are relatively easy to implement in AEM using the Request for activation workflow as the initiator (person who requests publication) and approver are in two distinct user groups.

However, in the Democratized authoring model, the initiator and approver are in the same user group. This is not supported by AEM. The reason is that the ParticipantStepChooser which is responsible for telling AEM which user should be assigned to a particular step can only return a single participant ID (user or group).

This means that (according to the diagram above), if allan requests the publication of a new job offer page, and the approval is assigned to the hr group, then allan could approve his own request.

Obviously, this breaches the four-eyes principle. Let's look at how we can implement this use case.

Solution

💡 The code below focuses on business logic. Some utility code has been abstracted. See the Github diff at the bottom of the article to see the exhaustive source code.

Concept

The solution design relies on the following idea: we can create a special group (an exclusion group) at runtime which includes all members of an existing group or groups, minus the initiator. Members of this exclusion group can then perform the approval, and finally the exclusion group is removed when the workflow ends.

Creating the exclusion group

To create the exclusion group, we will implement a custom workflow process step for that purpose, as per Adobe's documentation: Custom Process Step.

import com.adobe.granite.workflow.WorkflowException;
import com.adobe.granite.workflow.WorkflowSession;
import com.adobe.granite.workflow.exec.WorkItem;
import com.adobe.granite.workflow.exec.WorkflowProcess;
import com.adobe.granite.workflow.metadata.MetaDataMap;
import com.theopendle.core.workflow.WorkflowUtil;
import com.theopendle.core.workflow.queries.GroupWithId;
import com.theopendle.core.workflow.queries.UsersOfGroup;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang.StringUtils;
import org.apache.jackrabbit.api.security.user.Authorizable;
import org.apache.jackrabbit.api.security.user.Group;
import org.apache.jackrabbit.api.security.user.User;
import org.apache.jackrabbit.api.security.user.UserManager;
import org.apache.jackrabbit.oak.spi.security.principal.PrincipalImpl;
import org.apache.jackrabbit.value.StringValue;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceResolverFactory;
import org.osgi.framework.Constants;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;

import javax.jcr.RepositoryException;
import java.util.*;
import java.util.stream.Collectors;

import static com.theopendle.core.workflow.WorkflowUtil.setWorkflowVariable;

@Slf4j
@Component(property = {
        "process.label" + "=Create exclusion group",
        Constants.SERVICE_DESCRIPTION + "=Workflow step to create exclusion groups created earlier in the workflow",
        Constants.SERVICE_VENDOR + "=Theo Pendle",
})
public class CreateExclusionGroup implements WorkflowProcess {

    public static final String USER_ID_ADMIN = "admin";
    public static final String GROUP_ID_ADMIN = "administrators";
    public static final String PN_EXCLUSION_GROUP_ID = "exclusionGroupId";
    public static final String PN_GROUPS = "groups";

    @Reference
    private ResourceResolverFactory resourceResolverFactory;

    @Override
    public void execute(final WorkItem workItem, final WorkflowSession workflowSession, final MetaDataMap metaDataMap) throws WorkflowException {
        final String initiatorId = workItem.getWorkflow().getInitiator();

        // In case the initiator is the admin user, allow them to self-approve
        if (initiatorId.equals(USER_ID_ADMIN)) {
            log.warn("Initiator is admin. No exclusion group will be created.");
            setWorkflowVariable(workItem, PN_EXCLUSION_GROUP_ID, GROUP_ID_ADMIN);
            return;
        }

        final Map<String, String> arguments = WorkflowUtil.readArguments(metaDataMap);
        if (!arguments.containsKey(PN_GROUPS)) {
            throw new WorkflowException(String.format("No <%s> argument passed to step", PN_GROUPS));
        }

        final Set<String> groups = Arrays.stream(arguments.get(PN_GROUPS).split(","))
                .filter(StringUtils::isNotBlank)
                .collect(Collectors.toSet());
        if (groups.isEmpty()) {
            throw new WorkflowException(String.format("<%s> argument contains an empty list", PN_GROUPS));
        }

        try {

            try (final ResourceResolver resolver = resourceResolverFactory.getServiceResourceResolver(Map.of(
                    ResourceResolverFactory.SUBSERVICE, "user-management"))) {

                final UserManager userManager = resolver.adaptTo(UserManager.class);
                if (userManager == null) {
                    throw new WorkflowException(String.format("Could not retrieve <%s>", UserManager.class));
                }

                final Authorizable initiatorAuthorizable = userManager.getAuthorizable(initiatorId);
                if (initiatorAuthorizable == null) {
                    throw new WorkflowException(String.format("Could not find initiator of the workflow with ID <%s> ", initiatorId));
                }

                // Find all users belonging to specified groups
                final Set<Authorizable> users = groups.stream()

                        // If the initiator is not a member of the group provided, then discard it
                        .filter(groupId -> userInGroup(userManager, groupId, initiatorAuthorizable))

                        // Get all members of the eligible groups
                        .flatMap(groupId -> getUsersOfGroup(userManager, groupId).stream())

                        // Remove the initiator
                        .filter(user -> !user.equals(initiatorAuthorizable))

                        .collect(Collectors.toSet());

                if (users.isEmpty()) {
                    throw new WorkflowException(String.format("No other users found in groups <%s> (initiator <%s>)", groups, initiatorAuthorizable.getPrincipal().getName()));
                }

                // Create exclusion group
                final String exclusionGroupId = "demo-exclusion-group-" + UUID.randomUUID();
                final PrincipalImpl exclusionPrincipal = new PrincipalImpl(exclusionGroupId);
                final Group exclusionGroup = userManager.createGroup(exclusionPrincipal, "demo/exclusion");

                // Add properties to group so it can easily be found and recognized
                final String userIds = users.stream()
                        .map(user -> {
                            try {
                                return user.getPrincipal().getName();
                            } catch (final RepositoryException e) {
                                return null;
                            }
                        })
                        .filter(Objects::nonNull)
                        .collect(Collectors.joining(", "));

                for (final Map.Entry<String, String> entry : Map.of(
                        "workflowInstance", workItem.getWorkflow().getId(),
                        "includesGroups", String.join(",", groups),
                        "excludesUser", initiatorAuthorizable.getPrincipal().getName(),

                        // Name the group after the users that it contains so that authors know who is eligible to approve
                        // without needing access to the AEM user/group admin interfaces
                        "profile/givenName", userIds
                ).entrySet()) {
                    exclusionGroup.setProperty(entry.getKey(), new StringValue(entry.getValue()));
                }

                // Add users to exclusion group
                for (final Authorizable user : users) {
                    exclusionGroup.addMember(user);
                }

                // Commit the creation of the exclusion group
                resolver.commit();

                // Store the group ID so it can easily be found later for deletion
                setWorkflowVariable(workItem, PN_EXCLUSION_GROUP_ID, exclusionGroupId);

                log.info("Created exclusion group with ID <{}>", exclusionGroupId);

            } catch (final LoginException e) {
                throw new WorkflowException("Could not log to service user.", e);
            } catch (final RepositoryException e) {
                throw new WorkflowException("Unexpected error while fetching user and/group", e);
            } catch (final PersistenceException e) {
                throw new WorkflowException("Unexpected error while saving exclusion group", e);
            }

        } catch (final Exception e) {
            throw new WorkflowException(String.format("Unexpected error while running <%s>", this.getClass()), e);
        }
    }

    private boolean userInGroup(final UserManager userManager, final String groupId, final Authorizable userAuthorizable) {
        try {
            final GroupWithId groupWithId = new GroupWithId(groupId);
            final Iterator<Authorizable> iterator = userManager.findAuthorizables(groupWithId);

            if (!iterator.hasNext()) {
                log.error("Group <{}> passed to workflow via argument <{}> does not exist", groupId, PN_GROUPS);
                return false;
            }

            final Authorizable group = iterator.next();
            if (!group.isGroup()) {
                log.error("Principal <{}> passed to workflow via argument <{}> is not a group", groupId, PN_GROUPS);
                return false;
            }

            return ((Group) group).isMember(userAuthorizable);

        } catch (final RepositoryException e) {
            log.error("Unexpected error while searching for Authorizables", e);
            return false;
        }
    }

    private Set<User> getUsersOfGroup(final UserManager userManager, final String groupName) {
        try {
            final UsersOfGroup usersOfGroup = new UsersOfGroup(groupName);

            final Iterator<Authorizable> iterator = userManager.findAuthorizables(usersOfGroup);

            final Set<User> users = new HashSet<>();
            while (iterator.hasNext()) {
                final Authorizable authorizable = iterator.next();

                if (authorizable.isGroup()) {
                    log.info("Ignoring authorizable <{}>, member of group <{}> as it is not a user",
                            authorizable.getPrincipal().getName(), groupName);
                    continue;
                }

                users.add((User) authorizable);
            }

            return users;

        } catch (final RepositoryException e) {
            log.error("Unexpected error while searching for Authorizables", e);
            return Collections.emptySet();
        }
    }
}

Assigning the approval step

Now that the exclusion group has been created, we can create a dynamic participant step chooser that assigns the next step to the exclusion group (see Dynamic Participant Step - Example Participant Chooser Service
):

import com.adobe.granite.workflow.WorkflowException;
import com.adobe.granite.workflow.WorkflowSession;
import com.adobe.granite.workflow.exec.ParticipantStepChooser;
import com.adobe.granite.workflow.exec.WorkItem;
import com.adobe.granite.workflow.metadata.MetaDataMap;
import lombok.extern.slf4j.Slf4j;
import org.osgi.service.component.annotations.Component;

import static com.theopendle.core.workflow.WorkflowUtil.getWorkflowVariable;

@Slf4j
@Component(service = ParticipantStepChooser.class, property = {
        ParticipantStepChooser.SERVICE_PROPERTY_LABEL + "=Exclusion group"
})
public class ExclusionGroupParticipantStepChooser implements ParticipantStepChooser {

    @Override
    public String getParticipant(final WorkItem workItem, final WorkflowSession workflowSession, final MetaDataMap metaDataMap) throws WorkflowException {
        final String exclusionGroupId = getWorkflowVariable(workItem, CreateExclusionGroup.PN_EXCLUSION_GROUP_ID, String.class);

        if (exclusionGroupId == null) {
            throw new WorkflowException(String.format("No exclusion group found in workflow metadata map via property <%s>",
                    CreateExclusionGroup.PN_EXCLUSION_GROUP_ID));
        }

        return exclusionGroupId;
    }
}

Cleaning up the exclusion group

Of course, we don't want hundreds of exclusion groups to build up over time, so let's make sure to delete the group once the approval step is over.

We can do this by implementing another custom process step:

import com.adobe.granite.workflow.WorkflowException;
import com.adobe.granite.workflow.WorkflowSession;
import com.adobe.granite.workflow.exec.WorkItem;
import com.adobe.granite.workflow.exec.WorkflowProcess;
import com.adobe.granite.workflow.metadata.MetaDataMap;
import lombok.extern.slf4j.Slf4j;
import org.apache.jackrabbit.api.security.user.Authorizable;
import org.apache.jackrabbit.api.security.user.UserManager;
import org.apache.sling.api.resource.LoginException;
import org.apache.sling.api.resource.PersistenceException;
import org.apache.sling.api.resource.ResourceResolver;
import org.apache.sling.api.resource.ResourceResolverFactory;
import org.osgi.framework.Constants;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;

import javax.jcr.RepositoryException;
import java.util.Map;

import static com.theopendle.core.workflow.WorkflowUtil.getWorkflowVariable;

@Slf4j
@Component(property = {
        "process.label" + "=Delete exclusion groups",
        Constants.SERVICE_DESCRIPTION + "=Workflow step to delete exclusion groups created earlier in the workflow",
        Constants.SERVICE_VENDOR + "=Theo Pendle",
})
public class DeleteExclusionGroups implements WorkflowProcess {

    @Reference
    private ResourceResolverFactory resourceResolverFactory;

    @Override
    public void execute(final WorkItem workItem, final WorkflowSession workflowSession, final MetaDataMap metaDataMap) throws WorkflowException {
        final String exclusionGroupId = getWorkflowVariable(workItem, CreateExclusionGroup.PN_EXCLUSION_GROUP_ID, String.class);

        if (exclusionGroupId == null) {
            throw new WorkflowException(String.format("No exclusion group found in workflow metadata map via property <%s>",
                    CreateExclusionGroup.PN_EXCLUSION_GROUP_ID));
        }

        try (final ResourceResolver resolver = resourceResolverFactory.getServiceResourceResolver(Map.of(
                ResourceResolverFactory.SUBSERVICE, "user-management"))) {

            final UserManager userManager = resolver.adaptTo(UserManager.class);
            if (userManager == null) {
                throw new WorkflowException(String.format("Could not retrieve <%s>", UserManager.class));
            }

            final Authorizable exclusionGroup = userManager.getAuthorizable(exclusionGroupId);
            if (exclusionGroup == null) {
                throw new WorkflowException(String.format("Could not find exclusion group with ID <%s>", exclusionGroupId));
            }

            exclusionGroup.remove();
            resolver.commit();
            log.info("Deleted exclusion group <{}>", exclusionGroupId);

        } catch (final LoginException e) {
            throw new WorkflowException("Could not log to service user.", e);
        } catch (final RepositoryException e) {
            throw new WorkflowException("Unexpected error while fetching user and/group", e);
        } catch (final PersistenceException e) {
            throw new WorkflowException("Unexpected error while saving exclusion group", e);
        }
    }
}

Updating the workflow model

Now that we have the steps we need, let's put them together in the Request for activation workflow model.

Here is an image of the updated workflow model. See the Github link at the bottom of the article for the exact configuration:

Result

Once the workflow model is updated and synchronized, the use case should be satisfied.

For the purposes of the demo, I have created some test groups and users, see the Github link at the bottom of the article for details.

Watch me demo the feature in the GIF below:

Steps in the demo:

We log in as allan and start the Request for activation workflow on a page
We confirm that allan is not notified of the approval step (he cannot approve his own request)
We log in as betty
We confirm that betty has received a notification about the approval step
We approve the content
We log back in as allan and confirm that the content was indeed published

Conclusion

You should now be able to create four-eyes approvals in workflows for colleagues within the same AEM user group!

This article focused on the popular use case of page activation, but of course the principle is applicable to any approval you might need to implement.

You can view the source code for the implementation and the demo on Github.

I've created a diff so you can see just the changes to make this feature work here.

Don’t hesitate to contact me on LinkedIn if you have any questions/ideas!

AEM: Connect AEM to an External database (Oracle example)

Theo Pendle — Sat, 22 Apr 2023 08:43:05 +0000

It can happen when implementing AEM that you suddenly realize that the JCR database is not the best solution to store certain types of data.

For example, the JCR-SQL2 dialect is missing a lot of features that you'd expect from an typical database, so queries can become cumbersome. These limitations can be a real bummer if you're trying to store thousands of products, blog posts or whatnot. For that reason, you may want to store certain data in an external database such as Oracle, MySQL, Neo4J, etc.

In enterprise environments, Oracle is often favored for RDBMS needs due its advanced features and support. With that in mind, I've decided to use Oracle as the basis for this tutorial, but the principles apply to any other external database.

There are essentially 3 components required to integrate AEM with an external database:

A JDBC driver bundle, which lets Java communicate with the database
A data source configuration, which represents the connection details of your database
A client, some Java class which actually runs a query

Follow this tutorial to find out how to set this all up. I've provided a Github link at the bottom of the article if you want to see a concrete implementation of this tutorial.

Wrapping the oracle JDBC driver

I've seen many articles online that simply mention "create an OJDBC wrapper bundle" or give step-by-step instructions to manually create the bundle using Eclipse plugins. I would prefer to have an automated wrapper that can easily be changed and/or bundled with the rest of your AEM application, so let's use Maven to create our bundle.

Here is a POM file to wrap the ojdbc8 driver:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <parent>
        <groupId>com.theopendle</groupId>
        <artifactId>demo</artifactId>
        <version>1.0.0-SNAPSHOT</version>
        <relativePath>../pom.xml</relativePath>
    </parent>

    <artifactId>demo.ojdbc-bundle</artifactId>
    <packaging>bundle</packaging>
    <name>Demo - OJDBC bundle</name>
    <description>OSGi wrapper for Oracle JDBC jar.</description>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.felix</groupId>
                <artifactId>maven-bundle-plugin</artifactId>
                <version>3.5.0</version>
                <extensions>true</extensions>
                <configuration>
                    <instructions>
                        <Bundle-License>non-free</Bundle-License>
                        <Bundle-Vendor>Oracle</Bundle-Vendor>
                        <_exportcontents>*</_exportcontents>
                        <Export-Package>
                            oracle.core.*;version="${project.version}",
                            oracle.jdbc.*;version="${project.version}",
                            oracle.jpub.*;version="${project.version}",
                            oracle.net.*;version="${project.version}",
                            oracle.security.*;version="${project.version}",
                            oracle.sql.*;version="${project.version}"
                        </Export-Package>
                        <Import-Package>*;resolution:=optional</Import-Package>
                        <Private-Package>!*</Private-Package>
                        <Embed-Dependency>*;scope=compile|runtime;type=!pom;inline=true</Embed-Dependency>
                    </instructions>
                </configuration>
            </plugin>
        </plugins>
    </build>

    <dependencies>
        <dependency>
            <groupId>com.oracle.database.jdbc</groupId>
            <artifactId>ojdbc8</artifactId>
            <version>21.7.0.0</version>
        </dependency>
    </dependencies>
</project>

If you're not familiar with the maven-bundle-plugin plugin, here are some pointers:

The <Export-Package> tag contains the list of packages in the ojdbc8 JAR file that you might conceivably need to call from your Java code. We are making the classes from those packages available in OSGi so they can be imported by your Java code.
The <Import-Package>*;resolution:=optional</Import-Package> marks all imports as optional so we aren't forced to satisfy them with another bundle.
The <Embed-Dependency>*;scope=compile|runtime;type=!pom;inline=true</Embed-Dependency> tag tells Maven to embed all compile or runtime dependencies (in this case our only dependency is ojdbc8) into this bundle.

If you add this POM to your project as a Maven submodule, and embed it in your allpackage, you can easily deploy this bundle to AEM as a part of your application. See the Git diff at the bottom of the article for specifics.

The end result should be a new bundle running in the OSGi container, like so:

Now that the diver is available in OSGi, let's use it to open a database connection.

Configuring the DataSourcePool

The com.day.commons.datasource.poolservice.DataSourcePool Java class is used to to create connections to a database. It pulls its configuration from the com.day.commons.datasource.jdbcpool.JdbcPoolService configuration factory.

You can configure your datasource by going to /system/console/configMgr/com.day.commons.datasource.jdbcpool.JdbcPoolService but here is my version as cfg.json file:

{
  "jdbc.password": "test",
  "jdbc.driver.class": "oracle.jdbc.driver.OracleDriver",
  "datasource.name": "oracle",
  "jdbc.connection.uri": "jdbc:oracle:thin:@localhost:1521:test",
  "jdbc.validation.query": "SELECT 1 FROM DUAL",
  "default.readonly": false,
  "default.autocommit": false,
  "jdbc.username": "test",
  "pool.max.wait.msec": 1000,
  "pool.size": 10
}

Testing the connection

Now everything should be ready for you to start making queries to the database, so let's give it a go.

In your core java module, write a class that opens a connection to the database. If you used my configuration above, you can run use the Connection::isValid method to run the validation query, in my case SELECT 1 FROM DUAL.

Here is an example of an service that will let us easily open connections in the future. For now it does nothing but check the validity of the connection on startup.

@Slf4j
@Component(service = OracleDataSource.class, immediate = true)
public class OracleDataSource {
    public static final String DATA_SOURCE_NAME = "oracle";

    @Reference
    private DataSourcePool dataSourcePool;

    @Activate
    public void activate() {
        try {
            final DataSource dataSource = (DataSource) dataSourcePool.getDataSource(DATA_SOURCE_NAME);

            try (final Connection connection = dataSource.getConnection()) {
                if (connection == null) {
                    log.error("Could not establish connection to <{}>", DATA_SOURCE_NAME);
                    return;
                }
                log.info("Connection is valid: <{}>", connection.isValid(1000));
            }

        } catch (final SQLException e) {
            log.error("Could not establish connection to <{}>", DATA_SOURCE_NAME, e);

        } catch (final DataSourceNotFoundException e) {
            log.error("Could not find data source with name <{}>", DATA_SOURCE_NAME, e);
        }
    }
}

Now if you build and install your core bundle, you should see a log like this:

11.03.2023 14:00:28.568 *INFO* [OsgiInstallerImpl] com.theopendle.core.services.OracleDataSource Connection is valid: <true>

Conclusion

In this tutorial you learned how to bundle a JDBC driver and use it to open a connection to an Oracle database.

I usually like to tackle specific use cases in my tutorials to make them more engaging and because I try to anticipate what developers will be Googling 😉

However, there are several general takeaways in this article:

Bundling a JAR at build-time using a POM file in a Maven submodule is applicable for any dependency, not just a JDBC driver. And it is much easier and better practice than for your applications to realy on bundles that must be built by hand and deployed to OSGi as an installation step of your AEM instances.
Using a OSGi service to represent your data source and provide methods like opening and validating connections, handling timeouts, connection pool maintenance, etc. is a much better practice than to have those elements implemented in business logic.

See this link to a Git diff to see all the changes that make this feature possible.

I hope this was useful for you! If you liked it, don't hesitate to follow me here or on LinkedIn 😀

Custom Sling Injector With Annotations

Theo Pendle — Sat, 19 Nov 2022 20:13:52 +0000

How to inject anything anywhere with Sling

In this article I will show you how to build a custom injector for your Sling models with a real use-case. The source code is available on Github if you scroll to the bottom of the article.

The use case: hungry for cookies

One common use case I've seen is the need to read cookies in order to determine some behaviour in a set of components. Let's have a look at how we can use injection to write a re-usable solution to this requirement.

The objective will be to achieve something like this:

    ...

    // Read the cookie value here
    @CookieValue
    private String sessionId;

    @PostConstruct
    protected void init() {

        // Use the cookie value here
        log.info("Session ID <{}>", sessionId);
    }

    ...

There are essentially three parts to a custom injector:

The annotation which you will use to inject an object into a field (or somewhere else)
An annotation processor which will feed the attributes of your annotation to the Sling framework
The injector itself, which is responsible for providing the object

Let's start by creating our annotation. I will give code examples with an extra-generous serving of comments to explain each line 👍

Creating the annotation

To create your annotation, write the following Java @interface:


// This annotation belongs on a method (eg: getter), field or parameter (eg: constructor parameter)
@Target({ElementType.FIELD, ElementType.PARAMETER})

// Retained at runtime, as opposed to the Lombok @Getter which is discarded for example
@Retention(RetentionPolicy.RUNTIME)

// Declares an annotation as a custom inject annotation.
@InjectAnnotation

// This string will tell Sling which injector to use for this value
@Source("cookie-value")
public @interface CookieValue {

    // Default to OPTIONAL injection strategy as we cannot rely on the cookie being present
    InjectionStrategy injectionStrategy() default InjectionStrategy.OPTIONAL;
}

Processing the annotation

Now that we have an annotation to indicate that a value should by read from a cookie, we must tell the Sling framework how to interpret it.

The most important piece of information for Sling to know is the injectionStrategy should, by default be OPTIONAL. If not, then any Sling model that attempts to read a cookie and fails, would throw an exception.

To create our annotation processor, we must implement the InjectAnnotationProcessor2 interface.


@AllArgsConstructor
public class CookieValueProcessor implements InjectAnnotationProcessor2 {

    private CookieValue annotation;

    // This is the only we wish to take from the CookieValue annotation
    @Override
    public InjectionStrategy getInjectionStrategy() {
        return annotation.injectionStrategy();
    }

    // We can leave all these as default
    @Override
    public String getName() {
        return null;
    }

    @Override
    public String getVia() {
        return null;
    }

    @Override
    public boolean hasDefault() {
        return false;
    }

    @Override
    public Object getDefault() {
        return null;
    }

    @Override
    public Boolean isOptional() {
        return null;
    }
}

Injecting the cookie value

Finally we are ready to write the logic to read a value from a cookie. To do that, we must write the injector. This class will also server to bind the annotation to the processor from the previous steps:


/**
 * Injects the value of a cookie derived from a SlingHttpServletRequest.
 */
@Slf4j
@Component(service = {Injector.class, StaticInjectAnnotationProcessorFactory.class})
public class CookieValueInjector implements Injector, StaticInjectAnnotationProcessorFactory {

    @Override
    public String getName() {
        return "cookie-value";
    }

    @Override
    public Object getValue(final Object adaptable,
                           final String name,
                           final Type type,
                           final AnnotatedElement element,
                           final DisposalCallbackRegistry callbackRegistry) {

        // Injectors are cycled through, so it is important to perform this check.
        // We only want to trigger our code if the annotation being processed is indeed CookieValue
        final CookieValue annotation = element.getAnnotation(CookieValue.class);
        if (annotation == null) {
            return null;
        }

        // We cannot get a cookie from a resource adaptable, so let's make sure we indeed have a request to read from
        if (!(adaptable instanceof SlingHttpServletRequest)) {
            log.error("Cannot adapt <{}> to cookie value", adaptable.getClass().getSimpleName());
            return null;
        }

        final SlingHttpServletRequest request = (SlingHttpServletRequest) adaptable;

        final Cookie cookie = request.getCookie(name);
        if (cookie == null) {
            log.debug("Could not read value from cookie <{}> as no such cookie exists", name);
            return null;
        }

        return cookie.getValue();
    }

    // This rather clunky method in conjunction with CookieValueProcessor to pass the values of the annotation to the
    // Sling framework. Especially critical for the InjectionStrategy!
    @Override
    public InjectAnnotationProcessor2 createAnnotationProcessor(final AnnotatedElement element) {
        // Check if the element has the expected annotation
        final CookieValue annotation = element.getAnnotation(CookieValue.class);
        if (annotation != null) {
            return new CookieValueProcessor(annotation);
        }
        return null;
    }
}

And there you have it! Now lets use what we've just created in a Sling model.

Using our injector

In order to test our work so far, I'll inject a cookie value into a very basic Sling Model:


@Model(adaptables = SlingHttpServletRequest.class)
public class HelloWorldImpl implements HelloWorld {

    // Here we use our injector to tell Sling to read the value of the "sessionId" cookie.
    @CookieValue
    private String sessionId;

    @Getter
    private String message;

    @PostConstruct
    protected void init() {
        message = "Session ID: " + sessionId;
    }
}

Deploy this component to your local instance, place it on a page and you should see the following result:

No worries! The cookie doesn't exist yet. Let's create it by running the following code in the console of your browser:

document.cookie = "sessionId=123456789"

Refresh your page and you should see the value of sessionId:

And that's how you write a custom injector! 💉

Kicking it up a notch

But what if we want to de-couple the field name from the cookie name? And what if we want to inject not a String but rather a boolean value, such as a marketing consent flag?

Well stick around to find out.

Injecting a custom type

Changing the type of the injected value is relatively easy. Since the getValue() method of the Injector interface returns an Object, Sling basically casts the return value to the type of the field into which the injection happens.

However, it is our job to find out which type should be returned. So let's add this piece of logic before to our injector class:

// Check which type we should return, and convert accordingly
final String stringValue = cookie.getValue();
if (type.equals(String.class)) {
    return stringValue;
}
if (type.equals(Boolean.class)) {
    return Boolean.parseBoolean(stringValue);
}

// If the type of the field is not something we expect, then return null
log.error("Cookie value can only be injected as type <{}>", type.getClass());
return null;

Reading a custom cookie name

To make it possible to specify the name of the cookie to read from, we will have to give our annotation another attribute. Add this method to the CookieValue annotation:

/**
 * The name of the cooke to read the value from
 */
String cookie() default "";

Then add the following logic to the injector:

// Use the cookie name if provided, else use the name of the annotated element (eg: field name)
final String cookieName = StringUtils.isNotBlank(annotation.cookie())
        ? annotation.cookie()
        : name;

Finally, we can use the injector in our model:

@CookieValue(cookie = "demo_marketing_consent")
private Boolean marketingConsent;

Deploy and run the following line of code to create the cookie:

document.cookie = "demo_marketing_consent=true"

And now your boolean value is injected too:

More examples

The source code for this tutorial is on GitHub.

I've created a diff here so you can see exactly what changes to make. It includes another example of a custom annotation that lets you do this:


// Inject the tags associated to the page on which this component is placed
@PageTag
private List<Tag> tags;

// Inject a single tag from a custom page property
@PageTag(name = "customTag")
private Tag customTag;

Don’t hesitate to contact me on LinkedIn if you have any questions/ideas or if you just want to discuss all things Sling or AEM! 👍

AEM: Deploying replication agents as code

Theo Pendle — Sun, 02 Oct 2022 09:38:38 +0000

Use case

Let's imagine you want to install AEM with a pretty standard, simple topology. You may have 4 environments (local, dev, uat, prod) and each environments consists of one author replicating to two publishers.

That means you have 8 replication agents to set up, 12 if you add a dispatcher, 20 if you want reverse-replication, etc.

Configuring these replication agents can start to become a bit of a chore, especially if you are using custom replication and transport (receiver) users. You should btw, it's in the security checklist.

I've often seen AEM customers configure the replication agents manually as if it's some sort of AEM right-of-passage. But actually, you can add the replication agents into your git repository and then deploy them like anything else.

So let's give it a go 😀

Using a run mode

The first thing is to make sure that your AEM instances are running with run modes that allows us to distinguish by environment. If you don't know what a run mode is, please read this official documentation.

For example:

Your local Author might run with author,local
Your DEV Author might run with author,dev

We will attach our replication agents to these environment run modes, so make sure you have them active.

Creating a local agent

Now let's create the agents.

Let's start with the local environment. Here we'll create just one agent, which replicates from your local author instance to your local publish instance.

The easiest way to go about this is probably to start by copying the /etc/replication/agents.author node to /etc/replication/agents.local, then delete the agents you're not interested in. In this case, we'll remove everything except publish:

As you can see in the screenshot, I also renamed the agents.local jcr:title.

Now, if you navigate to /etc/replication, you will see a new addition to the list!

This interface is quite handy for configuring the agent, so you can do so now. In my case I'm going to start off basic, using the admin user everywhere (yes, it's naughty, I know, we'll fix it later).

Make sure to test your agent (http://localhost:4502/etc/replication/agents.local/publish.test.html) once you've configured it to guarantee that it all works as expected:

If you pull the /etc/replication/agents.local node into your repo, you should have an XML file that represents the replication agent:

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0" xmlns:rep="internal"
    jcr:mixinTypes="[rep:AccessControllable]"
    jcr:primaryType="cq:Page">
    <jcr:content
        cq:template="/libs/cq/replication/templates/agent"
        jcr:description="Agent that replicates to the default publish instance."
        jcr:primaryType="nt:unstructured"
        jcr:title="Publish"
        sling:resourceType="cq/replication/components/agent"
        enabled="true"
        logLevel="info"
        retryDelay="60000"
        serializationType="durbo"
        ssl="default"
        transportPassword="\{ba9f6a513c4aba03b33bd50d23a2032e6326c84d8a1d72fadfad8ee4301a2e5a}"
        transportUri="http://localhost:4503/bin/receive?sling:authRequestLogin=1"
        transportUser="admin"
        userId="admin"/>
</jcr:root>

Great! 🙂 With this, your fellow developers can deploy the replication agent to their local instances!

Creating a new module

I've opted to create a new module called ui.replication to host the replication agents.

I'll let you decide if you want to do the same. If you just want to get to the end of this tutorial to prove a concept, then you can skip this step.

I will not explain how this is done as it's simple Maven stuff, but scroll to the end of this article for a Github link if you want to see what it looks like.

Creating agents for other environments

Imagine you now want the same publish replication agent on other environments. For example, you'd like to be able to publish pages from from your DEV Author instance to your DEV Publish instance.

Simply copy /etc/replication/agents.local to /etc/replication/agents.dev and replace:

The jcr:title property of /etc/replication/agents.dev/.content.xml
The transportUri and transportPassword of /etc/replication/agents.dev/publish/.content.xml

Here is an example:

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0" xmlns:rep="internal"
    jcr:mixinTypes="[rep:AccessControllable]"
    jcr:primaryType="cq:Page">
    <jcr:content
        cq:template="/libs/cq/replication/templates/agent"
        jcr:description="Agent that replicates to the default publish instance."
        jcr:primaryType="nt:unstructured"
        jcr:title="Publish"
        sling:resourceType="cq/replication/components/agent"
        enabled="true"
        logLevel="info"
        retryDelay="60000"
        serializationType="durbo"
        ssl="default"
        transportPassword="\{8621078f9b44db113fd091bc3e1bbbf39f2775907f227d726f1bd301f2198be9}"
        transportUri="http://dev.publish:4503/bin/receive?sling:authRequestLogin=1"
        transportUser="admin"
        userId="admin"/>
</jcr:root>

Deploy to your local instance again and go to /etc/replication. You should now see the LOCAL and DEV agents:

This is normal. All agents will be deployed, but only the agents that match your run modes will be active. For example, here is my local Author:

The principle is the same for UAT, PROD or any environment you like. Just make sure that those instances also have matching run modes.

⚠️ Combining run modes

Of course, we only want our replication agent to be active on the author instance. Therefore we would need to distinguish which agent is active on the author and which is on the publish. This is possible, but a bit tricky.

Typically in AEM, you can combine run modes with an AND logical operator. For example for configurations:

/apps/demo/osgiconfig/config.author applies to all author instances
/apps/demo/osgiconfig/config.dev applies to all dev instances
/apps/demo/osgiconfig/config.author.dev applies to an instance if it has both the author and the dev run mode

However, replication agents combine run modes with an OR logical operator (don't ask me why). For example:

/etc/replication/agents.author applies to all author instances
/etc/replication/agents.dev applies to all dev instances
/etc/replication/agents.author.dev applies to an instance if it has either the author or the dev run mode

Unfortunately, this leaves you with these two options:

If you only want replication agents on author, then deploy the replication agents only to author. This is the approach I use for this tutorial, by not including the ui.replication module in the all package (see Github link at the bottom of the article).
If you want some agents on author and other on publish, then you must create a separate run modes, (eg: author-dev, publish-dev) and use those in your replication agents (eg: /etc/replication/agents.author-dev).

Using replication users

At this point you should have working, environment-specific replication agents.

The problem now is that we are using full admin privileges for replication, which can lead to security vulnerabilities. As mentioned before, Adobe considers this important enough to publish in the security checklist.

To paraphrase Adobe's explanation, there are two users that work to replication content:

The replication user, which reads content from the source instance (eg: author).
The transport user that writes content to the target instance (eg: publish)

Both users should have permissions only on the content that is eligible for replication.

I'll let you decide what those permissions will be for your application. But for the purpose of this article, let's imagine you have two users:

replication-user on the author instance
transport-user on the publish instance

Password encryption

The replication-user is the easiest to use. You can do so by setting the following property in your replication agent XML file:

userId="replication-user"

For the transport-user, its a bit more complicated. Since the author instance will have to authenticate on the publish instance, we need to provide the password in the XML file.

In order to avoid exposing the password when it is pushed to your code repository, it must first be encrypted.

To do this, go to /system/console/crypto on your author instance and encrypt the password for transport-user, or use the following curl command:

curl -u admin:admin -F"datum=password-goes-here" http://localhost:4502/system/console/crypto/.json

Take resulting encrypted value (curly braces included) and set the transportPassword property, like so:

transportPassword="{String}{537dc0a2c2de05f46fa9d65fb884c5b243fb2948e2f4b0ab4826c0912dc2154e}"

The {String} prefix tells AEM to consider the decrypted value as a string.

The end result looks like this:

/etc/replication/agents.local

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0" xmlns:rep="internal"
    jcr:mixinTypes="[rep:AccessControllable]"
    jcr:primaryType="cq:Page">
    <jcr:content
        cq:template="/libs/cq/replication/templates/agent"
        jcr:description="Agent that replicates to the default publish instance."
        jcr:primaryType="nt:unstructured"
        jcr:title="Publish"
        sling:resourceType="cq/replication/components/agent"
        enabled="true"
        logLevel="info"
        retryDelay="60000"
        serializationType="durbo"
        ssl="default"
        transportPassword="{String}{537dc0a2c2de05f46fa9d65fb884c5b243fb2948e2f4b0ab4826c0912dc2154e}"
        transportUri="http://localhost:4503/bin/receive?sling:authRequestLogin=1"
        transportUser="transport-user"
        userId="replication-user"/>
</jcr:root>

/etc/replication/agents.dev

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:cq="http://www.day.com/jcr/cq/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0" xmlns:nt="http://www.jcp.org/jcr/nt/1.0" xmlns:rep="internal"
    jcr:mixinTypes="[rep:AccessControllable]"
    jcr:primaryType="cq:Page">
    <jcr:content
        cq:template="/libs/cq/replication/templates/agent"
        jcr:description="Agent that replicates to the default publish instance."
        jcr:primaryType="nt:unstructured"
        jcr:title="Publish"
        sling:resourceType="cq/replication/components/agent"
        enabled="true"
        logLevel="info"
        retryDelay="60000"
        serializationType="durbo"
        ssl="default"
        transportPassword="{String}{9ceef3f083eb276082f17a81dcaf9b241ec79767df44ebf8d32a3d894c333652}"
        transportUri="http://dev.publish:4503/bin/receive?sling:authRequestLogin=1"
        transportUser="transport-user"
        userId="replication-user"/>
</jcr:root>

Conclusion

Congratulations, you can now deploy your replication agents as code! 😀

The source code for this tutorial is on Github.

I've created a diff so you can see just the changes to make this feature work here.

Don’t hesitate to contact me on LinkedIn if you have any questions/ideas!