Forem: Fabric

5 Challenges for Magento E-Commerce Web Development

Dana — Tue, 04 Jan 2022 00:28:32 +0000

Magento is one of the most popular e-commerce platforms today. Businesses that want to create a custom shopping experience, a platform that grows with them, or a platform that provides many out-of-the-box features generally lean toward Magento, as it promises all that and more.

However, every developer who has learned and worked with Magento, including myself, can tell you a handful of reasons why it’s very challenging: It’s very hard to learn, has frequent quirky bugs and issues, and is very slow.

In this article, we’ll cover the features Magento offers to e-commerce developers. Then we’ll go over five challenges e-commerce developers face when working with Magento. In the end, we’ll take a look at Magento’s alternatives.

Overview of Magento

When an e-commerce manager or developer looks for different e-commerce frameworks and platforms to choose from, Magento is one of the first choices they find that fits many requirements.

Magento is an open source e-commerce platform released in 2008 that is built with PHP. Magento lets developers and business owners create a platform that grows with them as necessary. Adobe recently acquired Magento in 2018, and since then have made many improvements, most notably on Magento’s documentation.

Magento is a highly customizable e-commerce framework. When you need to add new features, you can create modules or themes that add to, modify, or override existing features. You have the ability to change the platform per your requirements.

However, in a lot of use cases, you won’t need to add new features. Magento comes with many features that most businesses find more than enough.

Furthermore, Magento has a huge community. As an open source platform, it has over 1,400 contributors that help resolve bugs and issues, as well as add enhancements to the platform. A huge part of Magento’s growth is the community that has contributed to it countless times.

Magento E-Commerce Development Challenges

This is the part that most developers and teams miss when they look for the right e-commerce platform and land on Magento. Although Magento has a lot of advantages, it comes with many other challenges that most developers, teams, and businesses discover when it’s too late.

Here’s a summary of the difficulties e-commerce developers face when using Magento:

Challenge	Impact	Description
Steep learning curve	High	Its complex architecture makes it hard to find experienced Magento e-commerce developers.
Slow development and websites	High	Magento’s e-commerce web development workflow is slow, leading to a bad user experience.
Breaking updates	Medium	Magento occasionally has breaking minor patches.
Costly Resources	Medium	Magento’s resources, such as developers, hosting, and deployment lead to a high “e-commerce tax”.
Lacking bug fixes	Low	Many bugs are labeled as “Cannot Reproduce.”

1. Steep learning curve

In 2018, I was hired at a company for the sole purpose of learning Magento. Although I was a pretty skilled PHP developer back then, it was so hard to learn. I’ve been using it every day since then, and I still discover new quirks and unexpected errors or issues.

As I mentioned earlier, Magento’s documentation has improved since Adobe acquired it. However, it’s still not easy to learn because its architecture is complex.

Whether it’s something as simple as using Bootstrap in your theme (there are about 500 questions on Magento StackExchange just about that) or as complex as creating a module that customizes core functionalities or adds to them, everything in Magento is done the hard way.

Because it’s hard to learn, it’s hard to find expert Magento e-commerce developers. You’ll either have to hire someone to learn it and become an expert; hire experts that are not easy to find and, consequently, are costly; or rely on freelancer developers, which comes with its own set of problems.

I once was hired as a freelance developer to fix bugs and issues made by previous freelance developers. Thus, finding and hiring developers to do the job is definitely costly and one of the biggest challenges of using Magento.

2. Slow development and websites

This challenge deals directly with Magento e-commerce web development. Because of Magento’s complex architecture, two additional problems arise: slow development workflow and slow e-commerce websites.

The slightest change you have to make might require you to compile the generated code or the static assets for the website—even if it’s something as simple as changing the background color of the page. To see it in action, you have to deploy the static content. That basically means compiling Less, CSS, JavaScript, or other frontend assets for the changes to appear on the website.

This process becomes more time-wasting as you add more modules and themes because Magento will need to compile all of them even if the change is in just one. This means that developers take longer to implement or release features in Magento compared to the time it would take in other platforms.

Furthermore, the frontend of your store is pretty slow. This leads to a bad user experience and, ultimately, a bad search engine optimization (SEO) and lower ranking on search engines.

To make your website faster, you can use some of Magento’s configurations, like bundling and minifying assets. However, you’ll most likely need to use additional tools and services, like Varnish and Redis for your website to actually be considerably fast.

3. Breaking updates

After almost every major or minor release, multiple patches are released to fix the issues these releases bring with them. What makes it worse is that a lot of these problems involve security issues.

If you take a look at Magento’s Security Center, you’ll find many patches and some for the same release version.This means that whenever Magento has a new release, you should be careful before updating your platform, as the update can introduce bugs and security issues into it. This also means that using Magento requires constant maintenance for unexpected changes and bugs.

4. Costly resources

The previous points have highlighted this issue quite a bit. Magento requires a lot of costly resources. From the developers you need to hire (whether freelance or in-house) to the hosting, deployment, and maintenance, the costs pile up with time.

You’ll realize just how much you’ve put into Magento more and more as your business grows. You’ll need more experienced e-commerce developers, especially if you need complex features. You’ll need more cloud, storage, and hosting resources. The list of additional costs can heavily increase with time.

5. Lacking bug fixes

Again, this is another drawback of Magento’s complex system. Many bugs that might occur in your development or production environment could be hard to reproduce by Magento’s team of developers, which is why there are now 1,500-plus closed issues with the label “Cannot Reproduce” on GitHub.

Admittedly, a lot of these bugs and issues occur because of modules you find online or even create yourself. This is because a small change you make might affect something deep in the system that you don’t know about, and might even never find out about, or require a lot of debugging.

Hence, all these factors mean that although Magento has a strong community that supports it in solving bugs and issues, some bugs might not get a fix, and you’ll have to figure out a way to resolve them on your own.

Magento Alternatives

Magento’s many challenges all go back to its complex architecture. The system is tightly coupled and advanced in a way that makes it hard to learn, causes issues when you try to modify it, and slows down the platform and development process.

A better alternative to Magento’s monolithic architecture is microservices-based commerce. Microservices-based commerce is the set of e-commerce services you can usually get from an e-commerce platform or framework, except they’re all independent. You can use them together or couple and integrate them with other external services.

The move from a large system with lots of features and services that are tightly knit to smaller modules or services that are independent of one another but can still work harmoniously together removes the complications and challenges that occur in platforms like Magento.

fabric is a platform that lets you do exactly that—providing a set of microservices that you can integrate as necessary.

With fabric’s headless e-commerce APIs that provide the core functionalities of an e-commerce store through an API independent of the frontend, you have more flexibility in your platform. You can easily make changes to the frontend and the look and feel of your store without worrying about what goes on under the hood. Similarly, you can add new services and integrations to the backend without it affecting the frontend of your store.

Key Takeaways

Magento offers a lot of built-in features and is very extensible. However, Magento’s challenges often outweigh the advantages of using it.
The 5 developer challenges for Magento e-commerce development include a steep learning curve, slow development and websites, breaking updates, high costs, and the lack of bug fixes.
Microservices-based commerce is the better alternative for monolithic platforms like Magento.
With fabric, you get the same advantages you would need from Magento, including its extensibility and wide set of features, but with fewer e-commerce developer challenges.

Building Our E-Commerce Platform with Serverless FaaS

Robert Gibb — Sun, 26 Sep 2021 22:57:33 +0000

This post was published here on behalf of Devashish Pandey, a lead software development engineer at fabric who previously worked on platform and API development at SecureDB and StormDB.

With so many "something-as-a-service" options at our disposal for infrastructure, in 2017 we opted for function-as-a-service (FaaS) to build our platform of e-commerce services. Our mission was, and remains, to build for e-commerce services what AWS built for web services, or what we call the commerce fabric of the internet.

Building with FaaS and only paying for function execution time rather than constantly running servers (i.e. serverless) allowed us to serve customers early on without incurring high infrastructure costs. These customers were large, household names and referrals from our founding team who had transformed digital at Staples. Part of this transformation involved moving from the monolithic e-commerce platform IBM Websphere to a service-oriented architecture with open-source and custom-built software.

E-commerce veterans who knew our founding team wanted to transform digital for their companies, which led us to build our platform of modular commerce services in 2017. Using serverless FaaS with AWS Lambda supported these services reliably and efficiently, and it continued to support them through 2020 when we raised our $9.5M seed round and attracted customers outside of our circle of friends.

But as we raise more money (most recently our $100M Series B) and onboard global brands like GNC, life on the serverless backend is not as easy as it once was. As a result, some challenges that were inconsequential at the time are now more pronounced. In this post, I will explain how we direct these serverless challenges and show you how serverless FaaS powers our platform of commerce services.

Our Serverless Architecture

The current iteration of the fabric platform can be hosted in either AWS, GCP, Azure, and even Knative since all of these are supported by the serverless framework we use. However, in 2017 when we adopted serverless, AWS had the most mature offering so we made the simple choice in choosing Lambda for FaaS.

Flash forward to today and we still use Lambda and sparingly route requests through AWS Fargate, a container-as-a-service (CaaS) environment. Fargate is mainly used when we can't meet latency service level agreements (SLAs) with Lambda due to cold start problems. However, since Lambda powers 95% of all processes within our platform, we've found ways to direct these issues which I'll talk about below.

As for the way our products are structured, a data store layer is used, followed by a business layer in which most logic is programmed as you would with any other backend scenario. Lambda orchestrates event-driven workflows at the business and data storage layers.

The actual product offerings consist of commerce applications like a product information manager (PIM) and order management system (OMS), referred to as Co-Pilot Apps in the diagram below. (Co-Pilot is the name of the UI that merchandisers, marketers, and other business users interact with.) We also offer e-commerce storefronts that fast-track storefront development. Each type of offering has a separate yet connected business layer.

On the top level, each layer is supported by Lambda in a serverless manner. API gateway routes interface between them, and the logic layer and external services layer interface through integrations.

Every action, process module, and element of the platform is isolated into standalone functions in Lambda. The resulting layout achieves full separation of concern, with a single gateway route connecting with a single layer action/process at any given time. This cancels out the most common issues with resource sharing and allows each action/module to scale independently.

Managing Serverless

There are numerous tools engineers can use to manage serverless setups of the type I've discussed thus far. However, mainly because it was one of the earlier market representatives, we decided to use the Serverless Framework to manage serverless infrastructure through infrastructure as code (IaC).

Adopting the Serverless Framework as our infrastructure modeling and standardization tool enables quick spring-ups of environments. Further, the vendor-agnostic nature of the framework leaves room for any migration toward a cloud-agnostic, client-requested offering down the road. Using the Serverless Framework, we could support this offering while maintaining the same variables for mushroomed fabric platform instances.

In addition to these benefits, we found getting started with the Serverless Framework incredibly easy, even for a robust platform of applications and APIs like ours. If you want to try it yourself, just launch your terminal and install the node package manager. After that, a simple serverless command sets your environment up for configuration and management through the framework's CLI. A serverless.yml file is then created, which is where it all started for us in 2017!

Below is what a typical serverless.yml file configuration looks like using the Serverless Framework. The scripting language used is proprietary serverless.js, specific for a node.js app running on a MongoDB data store layer. This setup is similar to our own.

The provider section of the config file allows for the full exploration of cloud services providers. However, we continue to use AWS as our go-to provider as we've found that other providers cause capacity bottlenecks as we continue supporting more of the world's largest retailers and brands. After AWS is selected, each function across the implementation is associated with a handler and specific API gateways.

Using this template has dramatically improved the accuracy and speed of infrastructure configurations as we scale with serverless.

Serverless Challenges

As good as the story sounds so far, we have experienced our fair share of challenges with serverless and continue to experiment with ways to maintain business efficiencies while improving capacity and performance.

Capacity

While our current setup has enough capacity to support existing customers, there's always concern about having enough serverless infrastructure to support more, larger customers---especially during holiday shopping bursts. To get ahead of this, we are experimenting with augmenting Lambda with Amazon EKS using serverless containers with AWS Fargate.

Cloud-agnostic support is an option we are exploring as well. But before allowing customers to dictate their cloud provider, we need to better understand capacity with GCP and Azure. With customers calling the shots for infrastructure, capacity issues could also arise with AWS. For instance, if a customer selects an AWS region of deployment outside USA East and West, burst concurrency diminishes.

Performance

What makes serverless efficient for your business can cause inefficiencies with application performance, particularly when using Lambda functions. These inefficiencies occur during the cold start duration when Lambda is downloading your code and starting a new execution environment.

To reduce inefficiencies, we have tested and implemented several performance optimizations. One workaround we use for avoiding cold starts is periodic warmup events that counteract environment timeouts. We are also exploring Lambda@Edge (Lambda + Amazon Cloudfront) to offset any latency caused by Lambda. However, while doing this, we are trying not to defeat the cost-benefit variable for which Lambda was chosen in the first place.

Stick with Serverless?

Despite these challenges, we remain loyal to serverless infrastructure and putting product development ahead of server management. As one of the fastest-growing tech startups in e-commerce this year, serverless has served us well in supporting new customers while enabling fast iterations of new commerce services like Subscriptions and Loyalty Management.

While AWS Lambda is not yet perfect, we have noticed that, as fabric grows, so too does Lambda in terms of capacity. This gives us breathing room to test iterations of Lambda and other serverless offerings from AWS while continuing conversations and experimentation around matching the right architecture with the right infrastructure.

Hardening E-Commerce APIs to Secure Billion-Dollar Retailers

Niloufar Heidari — Sun, 12 Sep 2021 20:02:36 +0000

At fabric we are building a modular and headless commerce platform with the vision of becoming for commerce services what AWS is for web services, and due to the headless nature of our platform, APIs run the show. So much so that a multi-billion dollar equipment supplier just signed a multi-million dollar contract with us to consume our commerce APIs.

API security was a concern before I joined fabric in January 2021, but as we attract more billion-dollar retailers such as Restoration Hardware, GNC, and Staples—including more general market interest with our recent $100M Series B—API security is now a high priority.

In this post, I will show you our general approach for securing APIs to support these major retailers. But first, let’s briefly review why API security matters.

Why is API Security Important in Retail?

At fabric we use APIs to connect internal platform services such as cart, checkout, and pricing, and external services such as Stripe (payments), Alavara (tax), and Algolia (search). These connections mean data transfer and compromised APIs are behind major data breaches. With mainstream retail, personally identifiable and payment information is widely required to conduct trade and, as e-commerce grows, the attack surface grows with it.

The API Security project by OWASP supports this:

“A foundational element of innovation in today’s app-driven world is the API. From banks, retail and transportation to IoT, autonomous vehicles and smart cities, APIs can be found in customer-facing, partner-facing and internal applications. By nature, APIs expose application logic and sensitive data such as Personally Identifiable Information (PII) and because of this have increasingly become a target for attackers. Without secure APIs, rapid innovation would be impossible.”

Below I will describe the tactics we use to secure our APIs. With these tactics, the top ten API security threats top ten API security threats identified by OWASP are directed. Note: The OWASP API security project is nascent and needs contributors!

How We Secure APIs

These are the different RESTful APIs we use:

Customer storefront APIs: Our Storefront API protects e-commerce store frontends (including mobile apps). It is a modular, customizable, and secure API gateway that provides a single data source for all your UIs.

Third-party APIs: These are APIs provided by fabric to third-party vendors like Zendesk or RevCascade to access order and inventory modules. For example, mapping APIs are used to update inventory with new products from merchants.

Merchant application APIs: This is a customer API that is unique for each customer.

All of our API traffic is securely transmitted using HTTPS and modern TLS protocols. In addition to applying security best practices for data in transit via APIs, we take these security measurements:

Internal Vulnerability Testing

Using vulnerability assessment tools (e.g. QRadar, Qualys, and Nessus) and considering new security enhancements throughout the software development life cycle (SDLC) is essential for keeping APIs secure. At fabric, our continuous vulnerability management process makes security a key aspect of API development.

Security tooling is part of our CI/CD pipeline and automated security-focused unit tests run against every deployment. Additionally, we run monthly vulnerability scans to identify and categorize API security issues for all of our customers. If we identify any critical security vulnerabilities, they’re patched within a week.

Third-Party Pen Tests

Third-party penetration tests reputable firms (e.g. SpiderLabs, Cobalt, and FireEye) provide an objective expert analysis of a system’s security. A third-party pen test can expose vulnerabilities an internal audit and automated testing alone can miss.

In addition to leveraging automated scanning tools, third-party pen testers provide the human expertise necessary to identify misconfigurations and exploits that could otherwise go unnoticed.

For example, automated tooling is great at identifying common vulnerabilities and exposures (CVEs), but an attack that exploits business logic in an e-commerce site’s frontend APIs may go undetected. Simply put: third-party testing manipulates and attempts to hack an API in the same ways a real attacker would. This human perspective makes it possible to detect and correct misconfigurations and logic flaws before an attacker exploits them.

At fabric, we conduct third-party pen tests for our entire API annually and before every major release.

Authentication and Authorization

Validating users are who they say they are (authentication) and only allowing access to resources they have permissions for (authorization) is fundamental to API security. fabric authenticates users and then provides them with a JSON Web Token (JWT). Authorization of those requests is secured using Okta’s implementation of the industry-standard OAuth 2.0 protocol.

End-to-end, the process works like this:

A client sends credentials (username and password) to the fabric login API.

The credentials are verified by comparing what the client provides to a salted hash. (Credentials are encrypted using a robust hashing algorithm.)

If the credentials are valid, the API signs and returns a JWT. If the credentials are invalid, the login request is rejected. Rate-limiting on the login API prevents brute force attacks.

The client verifies the token and sends the signed token with every subsequent request. This eliminates the need for the client to include credentials for any other requests in the session.

OAuth 2.0 authorization ensures that the client can only access API endpoints they have adequate permissions for.

API Throttling

DDoS attacks can prevent an e-commerce site from processing legitimate orders and are one of the biggest threats facing large retailers. API throttling allows us to limit the number of API requests per second and prevent DDoS from bringing a site down.

Of course, with any throttling, there is the risk of false positives. To mitigate this risk, we use automation to optimize the throttling of APIs and reduce false positives while still providing protection from malicious attacks. Additionally, limits are based on both steady-state rates and bursts to account for different behaviour patterns.

To implement API throttling, we use AWS API Gateway. At a high level, the throttling process works like this:

Clients call an API.

If the number of requests is less than the throttling limit, the request is processed normally.

If the number of requests exceeds the throttling limit, the user receives an error and must wait to send additional requests.

API Keys

Like API throttling, API keys help limit abuse of our API and protect e-commerce sites that depend on it. API keys include a unique identifier (key ID) that allows us to meter access and prevent attacks.

Keys are created for each fabric customer and tied to specific entities. Clients must include a key ID and secret key in their requests. By tracking key IDs, we can hone in on malicious behaviour and block it at the source.

CORS Support

By default, modern web clients only allow servers to execute scripts from their own origin (an origin is a domain, scheme, or port). This is known as the same-origin policy. From a security perspective, the same-origin policy has obvious benefits. For example, it prevents a malicious site from executing JavaScript that reads data from another webpage a user is logged into.

Of course, there are plenty of legitimate reasons a site or API may need to call resources with a different origin. Cross-Origin Resource Sharing (CORS) solves this problem and allows servers to securely access resources from specific, pre-defined domains.

To implement CORS, we use the AWS CloudFront CDN and AWS CORS for our REST APIs. Our implementation ensures only authorized cross-origin scripts can be executed.

Here’s a simple example of how we use CORS in practice:

A client uses the fabric Commerce API gateway to load their website from an S3 Bucket at example.website.s3.us-east-1.amazonaws.com.

To load resources and allow users to interact with the online store and shop, the site uses JavaScript to make authenticated HTTP GET and PUT requests against the same example.website.s3.us-east-1.amazonaws.com bucket.

Typically, a modern web browser would block the requests to example.website.s3.us-east-1.amazonaws.com because they’re from a different origin. However, with CORS, we can explicitly allow specific cross-origin requests (in this case, example.website.s3.us-east-1.amazonaws.com), and the site will work as intended.

Conclusion

What we’ve discussed here covers a majority of the API attack surface. Still, we also perform regular code security audits; maintain compliance with standards including PCI DSS, SOC2 Type II, NIST 800, and OWASP Top 10; and implement robust identity and access management to secure our APIs.

In addition to directing common API security issues, we document our security approach to help customers understand how their data and the data of their customers is protected when using our commerce platform.

If you’re developing an API-based platform for retailers (or for users within another industry), implementing the tactics described in this blog post will harden the platform instances you deploy for your customers. It will also secure shopping-related data at scale and help your customers earn trust among shoppers in this evolving digital landscape.

E-Commerce Frontend Vs Backend: What's the Difference?

Indraneil Khedekar — Tue, 07 Sep 2021 15:52:57 +0000

The frontend of an e-commerce website is the part that online shoppers see and interact with. This is also referred to as the digital storefront. The backend of an e-commerce website is the part that handles all the data such as prices, promotions, product images, order details, and fulfillment.

In short, the frontend defines how the online shopping experience feels, and the backend defines how it functions.

As the touchpoints between buyers and sellers grow, the need for consistent shopping experiences across e-commerce websites and other sales channels grows as well. A 2017 study by Google found that retailers implementing multi-channel strategies saw 80% more store visits.

Understanding the difference between the e-commerce backend and frontend and how they work together can help you design a seamless omnichannel shopping experience for your customers. Understanding how separating them in a “headless commerce” setup can then help with scaling that experience.

Frontend vs Backend

Frontend

The e-commerce frontend, or client-side, is what people (i.e. clients) see when they visit your site. It’s the part of your site that shoppers interact with directly and includes features like fonts, colors, drop-down menus, image sliders, shopping carts, search bars, and product detail pages.

Frontend development ensures that an e-commerce website (or other digital sales channel) is functional and visually appealing by prioritizing the user experience. Frontend developers use programming languages like JavaScript, HTML, and CSS to create and manage a website’s frontend.

Backend

The e-commerce backend, or server-side, is the data access layer that holds products, orders, and customer information. It helps websites operate by processing, storing, and transferring their data to and from the customer-facing side. These actions are carried out by backend components like the web server, application server, and database which make the frontend interactive. They make it so customers can view products, place orders, and manage their purchases using their own devices.

Traditional Frontend and Backend in E-Commerce

Traditional e-commerce platforms such as Oracle ATG, Salesforce Commerce Cloud, and Shopify Plus bring the front and back ends together into tightly coupled systems. This approach makes it easy for merchants to deploy online stores but has several downsides.

These coupled, monolithic platforms lack flexibility. Adding new applications to such systems can compromise their integrity due to the dependencies between their front and back ends.

For instance, Shopify merchants that use apps to add functionality to their stores often struggle with site performance issues. Something as simple as adding a small popup widget can decrease page speed, introduce vulnerabilities, and cause JavaScript and CSS conflicts that can potentially “break” the frontend.

The good news is there's an alternative that overcomes the rigidity of legacy e-commerce systems. The need for better functionality, easier integrations, and consistent shopping experiences across channels has fueled the rise of a new approach to building e-commerce systems called headless commerce.

This approach separates the front and back ends of an e-commerce solution. It allows all the different frontends and the backend to exist independently, resulting in fewer dependencies so brands and retailers can scale to match varying business needs across channels.

Headless Frontend and Backend in E-Commerce

In a headless architecture, the front and back end communicate using APIs. This allows the backend to connect with multiple frontends across varying touchpoints, which is important since online touchpoints exist far beyond the web browser. They include every device used to make a purchase or a purchase decision.

Headless systems can connect with a diverse range of digital touchpoints such as:

Smartwatches
Smart refrigerators
Smart speakers and voice assistants
In-store kiosks
Digital signage

And that’s not all. The e-commerce backend in a headless system can be built differently as well. Unlike a tightly coupled system that holds the commerce, cart, and order management functionality in a single structure, the backend system can be constructed using modules. This approach gets rid of complex dependencies and helps websites adapt to changing needs without interruptions.

Benefits of a headless frontend and backend

Safer upgrades

Separating the content (frontend) and delivery (backend) systems gives you more control over both aspects. You’re able to adjust each part independently without jeopardizing the entire system. You can add and remove components and test new technologies with little to no downtime.

Better personalization

Independent frontends allow you to experiment with design changes easily. They also help you offer personalized shopping experiences. You can modify store designs to suit individual customer preferences and display tailored content on the fly.

Easier omnichannel

A headless approach makes it easy for you to sell across different touchpoints. With an independent front and back end, you’re able to design unique shopping experiences across all of your sales channels. You can do it without slowing down your digital storefront.

More freedom

An API-based approach lets you experiment with platforms and technologies. Don’t like your store design? Slap on a responsive, Vue-based storefront. Battling a buggy product management system? Move your catalog to fabric PIM. You can explore different options until you find what works for you.

Key Takeaways

The front and back ends are two parts of the e-commerce architecture that define its design and functionality.
The outdated approach of coupling the front and back end lacks the flexibility needed for a successful omnichannel commerce strategy.
A decoupled front and back end makes it easier to sell across channels, improves site performance, and gives you more freedom to make design changes.
You can use frontend solutions for e-commerce such as fabric XM and backend solutions such as fabric PIM to facilitate a seamless omnichannel experience.

Making E-Commerce More Accessible: 6 Tips for Developers

James Konik — Tue, 07 Sep 2021 13:26:29 +0000

Ignoring accessibility is a way to drive customers away from your e-commerce site. While it’s easy for developers to focus on new features and bug fixing, putting effort into inclusive, accessible sites will have huge benefits.

Accessibility impairment is more common than you might think. According to Shopify data, 2.3 percent of people live with a reading disorder, 8 percent of men have a form of color blindness, and 7 percent of people live with a severe dexterity disability.

In this article, I’ll highlight six areas where developers can make e-commerce sites more accessible.

Current Accessibility Standards

Accessibility standards exist to outline what to work toward clearly. The Web Content Accessibility Guidelines is a great resource for web content accessibility. It provides a single, shared global standard to meet the needs of individuals, organizations, and governments.

In some cases, accessibility standards are legally required. Applicable rules depend on your location, the products you sell, and the size of your business.

How to Implement Accessibility

If you’re just beginning to implement accessibility, a11y-101 is a great starting point. After implementation, ensure the product works well on screen readers and can be used via keyboard navigation. User testing, although pricier, is also an effective option.

Automatic audit tools, like the a11y command line tool, identify issues quickly and easily. The HTML_CodeSniffer is useful when testing for specific compliance guidelines.

See the result of using the a11y command-line tool on fabric’s site. Installing took thirty seconds, and running the tool is even quicker. Tools like this can significantly improve your accessibility without sacrificing time.

Let’s take a look at six specific tips for addressing problem areas to improve the accessibility of your e-commerce channels.

1. Improve Product Carousels

A carousel is a visually appealing way to show off products. This love, however, is not universal. They have several accessibility problems. First, their controls are unclear, particularly for the visually or cognitively impaired. Second, a screen reader may not associate controls with the required action. A control system that is obvious to a designer may not be intuitive to users.

Carousels that autoplay are an issue for those who use a keyboard to select screen elements. Users may not reach the control they want in time. Those who need more time to read are also frustrated when content changes automatically. Furthermore, the element chosen may also revert to the first available one when the carousel updates, making the whole site difficult to use.

If your client insists on using a carousel, you can still take steps to improve its accessibility. For example, see this sample code from the W3C Web Accessibility Initiative (WAI) under this license. It features readable text, optimizes for keyboard focus, and communicates changes to all users.

Although carousels are not ideal, you can still improve your site’s accessibility with them. In the below snippet, a live region is being created and then used later in the code. Doing so announces to users with visual impairments changes to the highlighted item.


    // In the init function, we add a live region to announce the slide number when using the previous/next buttons

function init(set) {

    //…… other code here

    var liveregion = document.createElement('div');
    liveregion.setAttribute('aria-live', 'polite');
    liveregion.setAttribute('aria-atomic', 'true');
    liveregion.setAttribute('class', 'liveregion visuallyhidden');
    carousel.appendChild(liveregion);

    //…… further code
}

// later in the setSlides function, we decide what happens when the slide is updated

function setSlides( args ) {

    //…… other code here

    if (announceItem) {
      carousel.querySelector('.liveregion').textContent = 'Item ' + (new_current + 1) + ' of ' + slides.length;
    }

    //…… further code

}

2. Make Visual Elements Accessible

Using only visual elements to present your product information excludes potential customers who cannot see them. Making information available to everyone is critical for accessibility. For images, that means including an alt tag, as seen in the following code:


<img src="coffee-maker.png" alt="blue coffee maker with glass mug and simple control panel" width="300" height="200" />

Video presents challenges to many users. But, with the right tweaks., you can make them available to everyone. Here are some changes to consider:

Captions: Captions make videos easier to follow, especially for the hard of hearing. Open captions are included in the actual video, while closed captions use a separate text file for screen readers and the visually impaired.
Transcripts: Providing a video’s full text as a separate transcript is an excellent way to increase accessibility. Transcripts also benefit those who prefer skimming through text, as opposed to a strictly visual format. Some media players, such as Able Player, automatically generate transcripts from closed captions.
Audio descriptions: Spoken audio describing a video helps visually impaired users understand what is happening. This can be presented as a separate, optional file or played alongside the video.
Accessible video players: If your frontend channels utilize a video player, make these more accessible too. Look to include supporting captions and audio descriptions, allow videos to be paused, have clear controls, and use high-contrast colors.

3. Replace or Improve Captchas

Captchas are designed to prevent bots and AI from accessing websites. However, AI is pretty good at them, scoring 98.4 percent on distorted text problems. In response, captchas are becoming harder, but this added complexity is causing difficulties for those with impairments.

Captchas are the biggest problem cited by screen reader users. Many visually impaired people cannot solve a visual captcha, leaving them unable to access the protected content. Although an audio captcha is a good alternative, they can also have accessibility problems. Furthermore, not all systems have audio available.

There are alternative ways to identify bots beyond captchas, so you can avoid using them altogether. One solution is to time how long a form on your e-commerce site takes to complete. Spambots can fill and submit forms instantly. By excluding any messages completed in under ten milliseconds, you can filter most of these bots out.

Another solution is to use server-side AI to analyze messages and filter IP addresses. For example, the honeypot involves hiding a form field so users cannot see it, but spambots still attempt to fill it in. It lets you identify spambots and filter out their responses accordingly.

See how to make a form field hidden:


<input name="name_bot_trap" type="hidden" value="">

Not all spambots will fall for this, however. Check out other techniques here and here.

Captchas don’t just stop bots; they prevent real people from using your site. Thus, replacing or improving captchas removes another barrier to user accessibility.

4. Use High-Contrast Themes

For users with visual impairments, some visual elements don’t stand out, affecting the UI. One way to improve accessibility for color-blind users is to use contrasting colors. For instance, selecting colors on opposite sides of the color wheel contrasts nicely for everyone.

Some users also find excess brightness difficult to look at and prefer darker themes. Thus, some sites allow users to enable Dark Mode. You can offer this with CSS and JavaScript. Browser extensions, like Dark Reader, offer solutions. This extended tutorial explains writing a theme switcher in React.

There are several types of color blindness, and a color scheme that works for one user may not work for another. Thus, you can also conduct testing with color-blind users. Alternatively, you can take a screenshot of your site and test it in a color blindness simulator to see how your site looks and identify any issues to fix.

5. Provide Input Assistance

Form inputs need to be clear and understandable. They should alert users to any errors but also offer solutions. For example, making a field “required” is impairs accessibility. Using an asterisk to indicate a required field confuses screen reader users, who just read out “asterisk.” If you mark a field as required in HTML, screen readers will pick up on it but still read out the asterisk.

Adding the text “required” is a good compromise. You can use the <aria-hidden> tag to mask the asterisk. This displays the asterisk while making sure readers ignore it:


<label for="input_four">First Name <i aria-hidden="true">*</i><i class="visible-hidden">required</i></label>
<input id="input_four">

The visible-hidden is a CSS class to hide the “required” text from visual rendering. See a11y-101’s approach to hiding text:


.visible-hidden {
    clip: rect(1px, 1px, 1px, 1px);
    height: 1px;
    overflow: hidden;
    position: absolute;
    white-space: nowrap;
    width: 1px;
}

.visible-hidden:focus {
    clip: auto;
    height: auto;
    overflow: auto;
    position: absolute;
    width: auto;
}

Although that’s a good start, older readers may not support the <aria-hidden> tag.

Another aspect to consider is where to place an indication of “errors” on forms. Readers should be able to find it. For example, a small line of text at the top or bottom of your form can easily be cut off from the user’s screen. Providing text that explains how to fix specific errors automatically read by screen readers is ideal. Like carousels, you should aim to prevent screen updates resetting keyboard navigation.

By handling errors sensibly, you ensure users will continue to use your e-commerce site.

6. Keep Customers On-Site

Affiliate links can confuse users by taking them away from your site to a different one. For instance, driving users to a commercially driven, flashy site can be stressful.
To improve accessibility, ensure it’s clear where links lead.

Those using screen readers navigate using the tab key to move around the page. They must tab through affiliate links, making this process slow. Allowing users to skip links helps speed up the process.

fabric’s Experience Manager lets you build a headless CMS that keeps data and backend services separate from the presentation layer. This allows you to deliver a consistent, accessible experience across multiple channels, ensuring users don’t get confused as they move between channels.

Key Takeaways

In an increasingly diverse, complex e-commerce landscape, it’s more important than ever to consider the differing needs of your users. This is particularly true for the millions of users who have difficulties with site usability. Therefore, improving your site’s accessibility is essential.

fabric can help build an e-commerce site to meet user accessibility needs and deliver a great experience for everyone.

Scaling Multi-Channel E-Commerce with Microservices and a Headless CMS

James Konik — Fri, 30 Jul 2021 20:43:56 +0000

Having a multi-channel e-commerce solution helps you reach new customers by giving you distribution on a variety of channels. Brands and retailers can now sell through traditional catalogs and POS, online stores, third-party sellers, and even social media.

From a technical perspective, supporting a multi-channel e-commerce workflow isn’t easy. Doing it well demands you take the right technical approach and design an architecture that can handle rapid change without limiting your potential for growth.

Microservices, APIs, and a headless CMS can form the basis for solving this problem. In this guide, I’ll explain the benefits of using this kind of architecture, and I’ll share some examples of how such a system works in practice, with workflows and code samples.

What is Multi-Channel E-Commerce?

Multi-channel e-commerce is selling your products in more than one place online. We usually think of online stores when considering e-commerce, but there are plenty of other ways to reach customers.

Apps, social media, and marketplaces such as Amazon that allow third-party sellers are all good places to sell goods online. Tablets in physical locations, such as stores or trade events, are also popular.

Selling in multiple channels brings new challenges. Your backend infrastructure needs to support everything it’s connected to, and your developers need to be ready to handle the extra work of building and supporting these channels.

What is a Headless CMS?

A headless CMS is a content management system that keeps the backend independent from frontend delivery channels.

The backend makes its services available to these channels via APIs. These all use the same underlying data, so things like product or customer information can stay the same across all of them.

For example, if you have a product available on both Amazon and Google Shopping, your headless API can route its pricing data from a common source like a PIM, meaning the price, product descriptions, and images are identical on both channels.

What are Microservices?

Microservices are encapsulated services that each have their own interface. They can be updated and modified without affecting other services. If the interface stays consistent, internal changes shouldn’t affect other components.

For example, if you want to upgrade the security on your orders, that’s easier if your orders are handled by a set of microservices. If you were tied into a monolith, such as Oracle ATG, you’d have to upgrade the whole platform.

With microservices, each individual service can be deployed and scaled on its own container. With AWS, for example, you pay for what you use, and if one service is suddenly in demand, Amazon’s cloud can manage the upsurge without problems and without affecting the rest of your backend.

That’s useful if some part of your infrastructure sees a spike in traffic. Perhaps you have a holiday product that gets mentioned in an article. A lot of people will look at your catalog. A few days later (hopefully), a lot of them will come back and buy the product. Amazon can let you ride the wave of traffic to your catalog and then handle the increased demand on your ordering system.

Building vs. Buying

Once you understand the requirements of multi-channel e-commerce, it becomes clear that you need some sort of headless, API-driven solution, and microservices offer clear benefits when it comes to scaling and modifying features.

The hard question is whether you should build these services yourself or use a third-party solution.

Building multi-channel e-commerce microservices

Here’s an example of how data might flow through a microservices-based setup, with an API gateway receiving calls and a service mesh routing calls from your checkout service to the packing and accounts services.

To set up an architecture like this, you first need to decide what services you want it to provide and how they will connect. Once the essentials are in place, you can build from there. Let’s look at a couple of the key components you’ll need.

Order management system (OMS)

An order management system deals with taking orders and handling different payment methods, as well as updating inventory, invoicing, and passing information to your delivery and accounting teams.

When an order is placed, data flows from a frontend application to your backend infrastructure, which is responsible for distributing it among the other services that need it. Your product database will be updated, as will your accounts, and the warehouse will need to know what to deliver.

As orders involve the interaction of multiple services, they are a good test of your architecture, as well as being essential to get right.

Product information manager (PIM)

Storing product information is a key component of any e-commerce business with large and growing volumes of data to manage. The fields and relationships between them may evolve as new sales channels provide new ways to present them.

The PIM provides information on products to your customers and allows for manual updates by staff, as well as automatic updates via other services, such as inventory.

With a headless architecture, you can keep all this in one place so that updates will be reflected everywhere. You can add fields and redefine existing ones as required, and also expand your range of services to support new kinds of queries.

API gateway and service mesh

Another key component to this architecture is your API gateway. It routes API calls to the correct services from the various frontend channels. You can change the routing, which allows you to easily switch between different services or versions of them.

For example, you could swap in a newer version of a service for testing or swap it out if you need to roll back to an earlier version. You can also route services from existing monoliths to newer microservices during a migration.

A service mesh acts as an intermediary between your services and controls the data flow between them. A service mesh can handle logging, load balancing, and encryption.

It can also enforce call limits on your services and provide a layer of security, letting you choose what data services are allowed to send to others. You may want to track which of your channels are calling particular services to help you identify issues or usage spikes.

Other services

Aside from the core services and gateway layers, there are many other supporting services that make up a headless, e-commerce microservice platform.

A personalization service can bring together the information you collect across all your channels and use it to customize the user experience on each one. A promotions and loyalty service can apply discounts to all your channels or specific ones. You could allow users to collect points or earn rewards from social media shares and then spend them on your web store, for example.

Example: Processing Orders from Multiple Channels

Let’s look at an example of how a microservice might deal with a call to its API. I’ll describe a workflow for processing orders using a prebuilt API, but if you build your own, it should be similar.

When the customer has selected their items and verified their order details, the order is sent via the checkout service as a post request to the checkout endpoint.

All channels can send the same request, regardless of their platform or UI. We can add or remove new channels without changing the backend, provided they send requests in the same format.

The post request is sent to:

POST /api-order/checkout

To send it in JQuery (assuming you have a web frontend), you can do something similar to this:


$.ajax({
    type: "POST",
    contentType: "application/json",
    url: "/api-order-checkout",
    data: JSON.stringify(data),
    dataType: 'json',
    success: function (data) { //using data formatted as in the section below
        //let the user know it has completed successfully
    },
    error: function (e) {
        //let the user know it has gone wrong
    }
});

The request is formatted like this:


{
    "cartId": "5f1b4158bb24210008b9a3f0",
    "customerEmail": "test@fabric.inc",
    "customerPhoneNumber": {
        "number": "07780811973",
        "kind": "Mobile"
    },
    "paymentDetails": [
        {
            "transactionDetails": {
                "paymentType": "CARD",
                "cardNumber": "4111111111111111",
                "expDate": "1122",
                "cvv": "999",
                "cardHolderFullName": "Joe Demo",
                "metadata": {}
            },
            "paymentIdentifier": {
                "cardIdentifier": "1111"
            },
            "paymentMethod": "Visa",
            "paymentKind": "Fabric User",
            "amount": "11.27",
            "currency": "USD",
            "conversion": 1,
            "billToAddress": {
                "name": {
                    "first": "Joe",
                    "last": "Demo"
                },
                "email": "demo@fabric.inc",
                "phone": {
                    "number": "07780811973",
                    "kind": "Mobile"
                },
                "street1": "600 CONGRESS AVE",
                "street2": "",
                "city": "AUSTIN",
                "state": "TX",
                "country": "US",
                "zipCode": "1003"
            }
        }
    ],
    "estimatedTax": {
        "itemsTaxes": [
            {
                "lineItemId": 1,
                "amount": 2
            },
            {
                "lineItemId": 2,
                "amount": 2
            }
        ],
        "shipToTaxes": [
            {
                "shipToId": "5f1b41892e8f1c0008fc20e8",
                "amount": 1.5
            }
        ]
    },
    "shipFrom": {
        "street1": "888 Broadway",
        "street2": "",
        "city": "New York",
        "state": "NY",
        "country": "USA",
        "zipCode": "46282"
    }
}

The API gateway will route this data to the relevant service. If the request is received and processed correctly, the backend will send a response formatted as follows:


{
"checkoutComplete": true,
"orderId": "4679-1016-66250"
}

So the frontend can present a confirmation message to the user, and of course, this will vary depending on the sales channel you’re using.

The backend can send further requests to other services, as determined by you and managed by your service mesh. As shown in the application diagram above, these could include calls to the shipping or promotions services, but the options are endless. To users, the interface is simple, while the backend implementation can be as complex as it needs to be.

Key Takeaways

Headless, microservices-based architectures have several advantages. Their modular nature makes them easy to change and gives you the flexibility to evolve your systems over time. They are also scalable, allowing you to allocate resources efficiently as you grow.

Building them yourself is time-consuming and expensive, so using a prebuilt solution can give you a headstart. These building blocks can get you up and running quickly and keep you ahead of the competition. With your architecture in place, you can add features and support new services, confident that your setup can handle their requirements.

Design Patterns for Modern Day Commerce Using Microservices

James Konik — Fri, 30 Jul 2021 20:07:33 +0000

E-commerce businesses are using microservices to build a set of reusable components for their stores. These services make it easier to deliver your content to multiple channels at scale by operating independently from the frontend.

In this post, I’ll talk about several design patterns you can implement and explain what they offer. I’ll also mention common use cases.

Understanding Software Design Patterns

Software design patterns are defined ways to solve common problems. They help developers understand how the components of a system relate to each other and interact. There’s no “perfect” design pattern—each has pros and cons and is helpful in specific situations.

Most developers spend years of practice getting these patterns right. However, by applying them properly, you can achieve results. There are five design patterns for modern e-commerce architectures:

The strangler pattern: A useful way to migrate from legacy software to a more modern approach.
The ambassador pattern: This gives you an encapsulated approach to handling network issues.
The sidecar pattern: This helps you add functionality without getting too tightly coupled to the rest of your software.
API interfaces: These help software services and components communicate.
Function chains: These help code to handle sequential tasks.

While implementation is the most difficult part, knowing the names and intentions of each pattern is an essential first step. At the end of this guide, you will have a starting point for deciding when each is right for your e-commerce platform.

The strangler pattern

Named after the strangler fig tree, the strangler pattern is moving from one platform to another gradually. You do this by replacing parts of your software one by one until eventually the old system is fully strangled. In practice, you can break it down into three steps:

Transformation: you create new versions of your services, replacing them one at a time
Co-existence: where you run the new services and older services together
Elimination: you replace everything you need and can retire the old system

Using the strangler pattern allows continuous delivery of new features and high code coverage. It also facilitates a modular, test-driven approach enabling you to isolate issues and ensure each service you deliver works well.

It’s a great way to shift to a new software setup, such as from a monolith to microservices. It lets you break the work down into manageable chunks, driving results quickly. Furthermore, you can distribute tasks to different teams to increase buy-in and accountability across your engineering organization.

On the other hand, it can take time. However, you can mitigate that by having teams work in parallel. Setting up your team right is as important as nailing the technical side of things.

The ambassador pattern

In the ambassador pattern, the ‘ambassador’ service is dedicated to communication. You create a proxy process or service that handles network requests for the rest of your application.

With an ambassador service in place, you can add features like monitoring, logging, and call rerouting. It’s useful to translate requests from one format to another—for example, multi-channel e-commerce, where you distribute products to many different frontend consumers.

If you’re using a mix of legacy and modern software, it can help bridge the gap, ensuring your network meets modern security and accountability standards. From an organizational viewpoint, it lets you assign a team to the proxy service itself, allowing you to divide responsibility.

While the ambassador pattern can quickly tie disparate systems together, it isn’t ideal if network latency is a concern. It can increase inter-service communication and higher memory and CPU usage.

If you have issues moving away from your monolith or legacy software you’re stuck maintaining, the ambassador can be a great way to circumvent those weaknesses. It allows you to add features to old software without having to rewrite everything.

The sidecar pattern

In the sidecar pattern, you move a specified set of functionality into a separate component. This component exists alongside the primary application, generally sharing the same lifecycle.

The sidecar is hosted together with its parent and can even run in the same process. That means there is little to no latency when the sidecar communicates with the parent, and it has access to the same resources.

It can, however, use a different programming language or framework than the primary service, and multiple sidecars can use different languages. That means you can use the right tool for the job when adding extra functionality or cater to the strengths and preferences of various team members.

Often the sidecar process can handle peripheral functions such as logging or network connectivity, while the main application handles core functions. Thus, if you need to move or reconfigure the application, teams can focus on the sidecar without having to change the main application.

It’s a simple pattern with many potential applications. For example, in an e-commerce setting, you could use it to log financial transactions. As detailed records are vital in e-commerce, you can have an independent one to add to and build on over time.

You can also use the sidecar pattern for handling network operations like adding modern encryption to a legacy service. This can let you partially modernize old e-commerce systems without resorting to a complete rewrite.

API interfaces

An application programming interface, or API, is a way for software components to communicate with each other using a defined set of calls. Web services or microservices commonly use APIs.

In addition to their use over networked communication, you can also use them for communication between microservices on the same host. There are several patterns commonly seen in API interfaces.

REST is the most recognizable. It’s a staple of computer science courses and standard for a vast number of websites and services. It includes a set of verbs, which implement the CRUD pattern. RESTful services are stateless and cacheable, making them ideal for the web.

In headless commerce, APIs allow multiple frontend applications to communicate with your backend services. Websites, apps, and software deployed on any other platform can send API requests to the same location. This lets you work on each component separately, making improvements and additions without worrying about the effect on the whole ecosystem.

Function Chain

You can build serverless functions on the cloud that are self-contained and stateless and can execute on demand. Services like Amazon Web Services, Microsoft Azure, and Google Cloud let you create these, so you don’t have to worry about hardware issues.

You can organize serverless functions into a function chain. In this pattern, each function invokes the next when it completes. This pattern is ideal if a user action kicks off a series of tasks that are slow to process. The first function can respond to the user, so they aren’t left waiting.

There are a few issues to consider when applying this pattern. Ideally, functions are independent and replaceable, but here the functions are dependent on each other. This breaks object-oriented design principles, but it’s necessary for certain applications. You can use a queuing system to call the functions in sequence, making them more independently operable and scalable.

Function chains are useful to implement sequential tasks that are well defined. For example, you may want to call a function chain after a user places an order to process data through different microservices and push it into each subsequent data store.

These tasks can each happen independently and in the background. This way, your e-commerce store’s UI stays snappy while the backend functions might take minutes to complete.

Key Takeaway

The more you know about software patterns and how to apply them to e-commerce microservices, the better you can leverage this existing knowledge to solve problems.

Using the Strangler Pattern to Break Down Your E-Commerce Monolith

James Konik — Wed, 30 Jun 2021 19:24:10 +0000

Monolithic platforms like Shopify, Oracle ATG, and BigCommerce are easy to get up and running. However, they lock you into various platform-specific limitations. As a result, you're subject to each platform’s constraints and may feel stuck using one-size-fits-all software.

You face similar challenges if you’ve built your own e-commerce monolith. When more components of the application are intertwined, it is harder to modify features, scale components, or divide the work. After years of using one of these systems, technical debt builds up and changes are hard to implement.

Microservices solve such problems. You can modularize your e-commerce business. Adapting quickly to the wealth of new opportunities to engage with your customers is a competitive advantage. That's why modern tech companies like Netflix, Amazon, and Etsy have all switched to microservices.

Migrating from monolith to microservices provides many technical and business challenges. One approach that helps to replace your monolith is the strangler pattern. It does so one service at a time.

In this post, you’ll learn how to use the strangler pattern to break down each piece of your e-commerce monolith until you’ve replaced it with discrete, scalable microservices.

Why Microservices?

Microservices have several advantages over a monolithic approach:

Technical flexibility

One-size-fits-all platforms are fast and convenient when you get started, but as your business develops, issues start to arise.
With microservices, you are free to implement the programming languages and third-party services that are best for each piece of your application. Because they’re scalable and modular, your decisions don’t tie you down.

Flexibility

Microservices are independent and decoupled from your frontend. That means you can easily build new ways to reach your customers and wire them up to your services without having to reinvent the wheel each time.
With your own services, you can ensure each service matches your needs, defining the data format, and functionality. You can also control how user access is structured. Flexibility also prevents bottlenecks as your teams are not dependent on each other.

Security

Attackers that gain access to your monolithic platform often gain access to everything - customer, product, and order data. Microservices can create silos between services, keeping them more secure.

Performance

Scalability and the ability to focus resources on your more heavily trafficked services dynamically mean you can improve performance. For example, Best Buy improved its catalog API sync time from 24 hours to under 15 minutes.

What is the Strangler Pattern?

The strangler pattern takes its name from the strangler fig tree. Like the tree, you use your existing application as a base. Then, you build a new service that replaces one specific element of it. When it’s done you retire the old part of the application.

You continue this service by service until the new microservices replace your entire monolith. You ‘strangled’ the old code and thus can abandon it completely.

There are three phases in the strangler lifecycle:

Transform the application by creating new versions of existing services.
Co-exist with the old application running alongside an ever-increasing number of microservices.
Eliminate the old when the new services completely replace the old system.

Strangler vs. waterfall approach

Besides the strangler pattern there is also the waterfall replacement pattern, the more common approach to migrating microservices. The waterfall pattern requires you to commit to a long development and deployment cycle, which increases the risk of bugs and lowers your velocity.

The waterfall method can take well over a year to deliver results, but you can make progress in bursts of six months or less by using the more agile strangler method. It naturally divides work into attainable targets, which means developers are motivated to complete tasks that deliver visible results.

With the strangler pattern, you can use each new microservice as soon as it is ready. There’s no need to build a completely new system. Some recommend rolling out changes monthly. For one, you can enjoy the advantages quickly. Secondly, it also lets your developers move on to the next project. If issues occur, they can roll pieces of the system back more easily than the whole thing.

The strangler pattern is often the better option. It allows you to migrate each piece of your infrastructure step-by-step in a manageable, low-risk way and deliver business value faster.

The downside to the strangler pattern

There are some caveats. Migrating a complete application one step takes a while. Analyzing successful projects using the strangler pattern suggests to remaining consistent and ensuring your management team commits to the long-term project.

Microservices benefit from robust development practices (like continuous integration and deployment) and having engineers that are familiar with them. Finding the right team is just as important as picking the right languages and frameworks. It’s important to make sure your developers are on board with the new processe. If they understand the advantages of the changes, buy-in will be higher.

One way to mitigate the risks of migrating to microservices is to use an established e-commerce platform like fabric. We've built fabric on microservices and can work with your team to ensure a smooth migration.

Example: Using the Strangler Pattern in E-Commerce

Let’s look at how to implement the strangler pattern. I use the monolith provided billing service feature and move it to a new microservice. Then, I'll show you how to migrate important data and features without breaking your existing application.

Step 1: Decide where to start the migration

The service you choose to start with depends on your need and the platform you’re using. There are two common approaches.

Critical features that you need to upgrade: If it's impossible to update your legacy codebase or you’re facing performance issues that cost you customers (like a slow checkout experience), starting here might be best. Ensure everything is watertight before deploying. It takes more time but has more impact.
Pick something less critical, to test the process: It's a great idea if your developers are implementing microservices for the first time as it’s less risky. However, it will drive less value for the business.

Let’s look at moving an e-commerce billing address service from a monolithic platform like Oracle ATG to a microservice hosted on AWS. Billing services that are slow or prone to errors might be preventing you from collecting revenue from customers. Thus, they’re often mission-critical in online stores.

I’ll only include a couple of attributes, but you can replicate this general pattern with whatever data you need.

Step 2: Build a middle layer

An API gateway can manage service calls and filter them to either your existing monolith or new services. AWS includes a guide on how to do it. A service mesh is also useful for load balancing and routing to manage data flow between your services.

With the middle layer in place, you can switch between your new and old services seamlessly. You could also theoretically A/B test the old and new path or roll back if something goes wrong during the migration.

Step 3: Build a new service and migrate data

Next, replace the old billing portion of your application with a new microservice. Here’s some code based on this example from Amazon’s AWS documentation that uses EC2 and Docker to deploy a billing microservice.

This task definition creates a new Docker container running on AWS to replace an Oracle ATG billing address service.


{
    "containerDefinitions": [
        {
            "name": "[billing-address-service]",
            "image": "[account-id].dkr.ecr.[region].amazonaws.com/[service-name]:[tag]",
            "memoryReservation": "256",
            "cpu": "256",
            "essential": true,
            "portMappings": [
                {
                    "hostPort": "0",
                    "containerPort": "3000",
                    "protocol": "tcp"
                }
            ]
        }
    ],
    "volumes": [],
    "networkMode": "bridge",
    "placementConstraints": [],
    "family": "[billing-address-service]"
}

You’ll need to transfer data from your existing monolith’s database to your new microservice’s backend database in most cases.

For example, the Oracle ATG billing address service described above uses profile data. You can copy this from your Oracle application’s backend to AWS, as demonstrated below.

This sample code uses Amazon SQS to queue up a series of data transfer jobs. This is critical if you have lots of data to move. Because the process may take a long time, you don’t want a single failed transfer to break the whole process.


const { SQSClient, SendMessageCommand } = require("@aws-sdk/client-sqs");

const REGION = "us-west-1"; //replace with your region

const params = {

DelaySeconds: 12,
  MessageAttributes: { // add other fields if you need them, remove what you don’t.
    profileId: {
      DataType: "Number",
      StringValue: "543",
    },
    address: {
      DataType: "String",
      StringValue: "29 Acacia Road, Springfield",
    },
  },
  MessageBody:
    "Clothing Store Product Data.",

  QueueUrl: "SQS_QUEUE_URL", // your queue URL

}

const sqs = new SQSClient({ region: REGION});

const run = async () => {
  try {
    const data = await sqs.send(new SendMessageCommand(params));
    console.log("Success, product sent. MessageID:", data.MessageId);
  } catch (err) {
    console.log("Error", err);
  }
};
run();

Once you’ve queued up each row, you need a service to receive and process the data. It will add each row to your new database and delete messages from the queue as they are successfully processed.

import  { SQSClient } from "@aws-sdk/client-sqs";

const REGION = "us-west-1"; //replace with your region

const sqsClient = new SQSClient({ region: REGION });
export  { sqsClient };

import {
  ReceiveMessageCommand,
  DeleteMessageCommand,
} from  "@aws-sdk/client-sqs";
import { sqsClient } from  "./libs/sqsClient.js";

// Set the parameters
const queueURL = "SQS_QUEUE_URL"; //Your queue URL

const params = {
  AttributeNames: ["SentTimestamp"],
  MaxNumberOfMessages: 10,
  MessageAttributeNames: ["All"],
  QueueUrl: queueURL,
  VisibilityTimeout: 20,
  WaitTimeSeconds: 0,
};

const run = async () => {
  try {
    const data = await sqsClient.send(new ReceiveMessageCommand(params));
    if (data.Messages) {
      var deleteParams = {
        QueueUrl: queueURL,
        ReceiptHandle: data.Messages[0].ReceiptHandle,
      };
      try {
        // TODO: Save your data here
        database.save(data);
        // Then delete the queued message
        const data = await sqsClient.send(new DeleteMessageCommand(deleteParams));
        console.log("Message deleted", data);
      } catch (err) {
        console.log("Error", err);
      }
    } else {
      console.log("No messages to delete");
    }
    return data; // For unit tests.
  } catch (err) {
    console.log("Receive Error", err);
  }
};
run();

Once your data is transferred into the new microservice, and you’ve verified it is correct, you can begin testing the new service. If everything works correctly, you can retire part of your monolith and repeat the process with each part of your application.

Obviously, this migration process varies widely depending on the monolithic e-commerce platform you’re using. Still, hopefully, this gives you a sense of how you can apply the strangler pattern in your application.

Conclusion

The strangler pattern can help move away from your legacy software platform in a structured, low-risk way. It helps make meaningful improvements quickly without breaking your existing e-commerce store.

You can thus roll out new features and turn your platform into scalable, robust services that form the core of your e-commerce business. The decoupled nature of microservices lets you use them across multiple channels.

If you’d like to avoid doing a complete migration yourself, fabric can help. With industry-leading expertise in e-commerce microservices, they can help you migrate to their suite of e-commerce microservices using a secure, scalable method like the strangler pattern.

Moving from BigCommerce to Microservices-Based E-Commerce

James Konik — Fri, 25 Jun 2021 01:53:34 +0000

BigCommerce includes many features and services, but if you want to take every opportunity to engage with your customers, you need a solution of which you’re in complete control.

There’s no better way to do that than replacing it with microservices. After all, this is the approach Amazon took to scale and become so successful. You can move one step at a time, adding reusable core components that act as a foundation for multiple frontend sales channels.

In this post, we’ll discuss replacing BigCommerce with a microservices-based alternative, looking at the reasons for making the change. Then we'll discuss the specifics, along with a few examples of what to do code-wise.

Note: We've written before about migrating away from classic e-commerce monoliths like Salesforce Demandware. As you'll see, many of the same concepts and migration practices apply to modern monoliths like Shopify Plus.

Why Microservices?

Microservices have several advantages over a pre-built, one-size-fits-all solution.

Security

Microservices are modular and interchangeable by design. That means your developers can work on them without affecting other parts of your system. It also means you can quickly make and test changes and slot in older or alternate versions of your code.
Being able to respond fast and hone in on potential problem areas are key advantages of microservices. By contrast, a large platform may take time to fix holes, and the process of upgrading might be out of your control.

Flexibility

BigCommerce offers you several templates and has plenty of features, but you are also at the mercy of any changes it makes.
With your own microservices, you can update them when you like. You can choose the fields and data format to work with and decide how your services connect and share information.
You can make every decision related to your business yourself. That may seem daunting, but specialists can help, even if you’re using your own solution.

Costs

While BigCommerce is extendable, many of the upgrades aren’t cheap. Even one can significantly add to your costs, and the more you have, the bigger the hit.
Estimates suggest ongoing costs for microservices can drop by up to 90%, and that’s before you factor in the productivity gains for your developers.

Replacing BigCommerce

BigCommerce’s headless architecture is ideal when shifting to microservices as you can pick and choose what you replace relatively quickly. Furthermore, by taking a modular approach and focusing on one service at a time, you can gradually transition.

Moving one service at a time makes it easy to identify any issues and roll back any mistakes. If you have trouble with any of your new services, just swap the old one back in, then test and improve the new one until it does what you expect.

Step 1: Decide where to start with the migration

There isn’t a fixed approach to migrating from one platform to another, and you might take several valid routes. Let’s look at some of BigCommerce’s services and decide what to implement using microservices.

Product information management (PIM)

Product Information Management is one of the most important tasks in e-commerce, and being able to update and access this data is crucial. BigCommerce has several apps available to help with it.

With microservices, though, you can customize everything from storing the data you need in the format of your choice to making it available to every channel you use. So you aren’t limited to finding the closest fit from a range of options.

fabric’s PIM system gives you a ready-made solution, allowing you to make changes using an intuitive UI and supporting features like multiple product hierarchies, auditing, and one-click migration.

Order management system (OMS)

Your order management system needs to handle payments, store the user’s order, and make sure it gets dealt with. In addition, logistical issues, like stock management, need it to run flawlessly.

BigCommerce has its own OMS with a defined workflow and several extensions available in its marketplace.

Your own OMS will be a crucial component of your business. It will take time to build and to get everything right. But a customized system built to your specifications can make the rest of your business run smoothly and efficiently.

fabric has its own OMS, and using this as a starting point for your system can save you a lot of time and ensure you nail the basics.

Other supporting services

PIM and OMS are essential to nearly all e-commerce businesses. The other services you pick will depend on which of BigCommerce’s capabilities you currently use and want to improve or replace.

BigCommerce offers many features out of the box. You might want to replicate its customer review system or perhaps control user access for your staff.

If you want to take things slowly, starting with a small, non-essential service can be a great way to get going.

Step 2: Build a middle layer

As you switch from BigCommerce to your new microservices, you need to make sure the system sends requests to the right place. You can do this with an API Gateway. Setting one up lets you choose which services deal with requests, so you can pick and choose what your new services handle.

Swap out legacy services

Once the middle layer is in place, you can retire old services and add your new ones. You can also swap things back and forth between new and existing services for testing or even filter requests so that some are dealt with by BigCommerce, and some go to your new setup.

Step 3: Migrate data from BigCommerce to microservices

When switching over, you’ll need to get your data out of BigCommerce and into your new backend database. It’s easy to export data from BigCommerce: you can generate a CSV file that contains everything, or you can pick and choose the fields to export.

As well as getting the fields out, you need to decide on your new data structure. Will you use identical field names for the fields you want, or will you change them? You may want to allow for additional fields too.

If you want to extract data from BigCommerce dynamically, you can do that via its APIs. The latest version of its REST API is three, faster and more efficient, though it doesn’t yet include everything in the older version two.

There’s also a GraphQL API, though this is still in early access and feature-incomplete. Being able to build your own full-featured API in any language is one advantage of microservices.

Let’s look at the data format for shopping cart items:

Field Name	Data Type
id	string
customerId	integer
email	string
currency	object
isTaxIncluded	boolean
baseAmount	number
discountAmount	number
cartAmount	number
coupons	array[object]
discounts	array[object]
lineItems	object
createdTime	string
updatedTime	string
locale	string

Here’s the JavaScript for adding this code to a message queue, adapted from the AWS documentation. You can also find code there to receive the data, which doesn’t need much adaptation.

I’ve picked out a few attributes from the list above. You can add more, using the same format, or remove any you don’t want.


// Imports for AWS
const { SQSClient, SendMessageCommand } = require("@aws-sdk/client-sqs");

// Set this to your region
const REGION = "us-east-1"; 

// The parameters, swap in what you need, and out what you don’t
const params = {
  DelaySeconds: 10,
  MessageAttributes: {
    id: {
      DataType: "String",
      StringValue: "475",
    },
    customerId: {
      DataType: "String",
      StringValue: "562567",
    },
    baseAmount: {
      DataType: "Number",
      StringValue: "79.99",
    },
discountAmount: {
      DataType: "Number",
      StringValue: "59.99",
    },
  },
  MessageBody:
    "Shopping cart information.",
  QueueUrl: "SQS_QUEUE_URL", // Your queue URL, e.g. 'https://sqs.REGION.amazonaws.com/ACCOUNT-ID/QUEUE-NAME'
};

const sqs = new SQSClient({ region: REGION });

const run = async () => {
  try {
    const data = await sqs.send(new SendMessageCommand(params));
    console.log("Success, message sent. MessageID:", data.MessageId);
  } catch (err) {
    console.log("Error", err);
  }
};
run();

Step 4: Test and repeat

As you replace each service, you should test thoroughly, making sure everything works correctly. That makes it easier to fix issues, as you can be reasonably sure that they relate to your recent changes.

Once everything is working, move onto the next service and repeat the process. This iterative, step-by-step approach lets you stay focused and get the right balance between working on new services and putting them through their paces in the real world.

Conclusion

Developing a new system yourself allows you to tune everything to your liking. It also lets your developers build the system the way they want and learn how it works as they put it together.

Taking advantage of existing solutions like BigCommerce also has benefits. It’s cheaper, quicker and many have tested the solution.

fabric specializes in microservices, and its solutions can help you make the switch easily. Its products are API-driven, allowing you to extend fabric core features easily and grow your services as your business expands.

Moving from Shopify Plus to Microservices-Based Commerce

James Konik — Thu, 13 May 2021 22:01:27 +0000

If you’re selling products online, there’s a good chance you’re using a one-size-fits-all platform. These platforms offer many advantages but they also have limitations. If you want to move beyond these limitations, microservices can help.

Shopify Plus is a one-size-fits-all platform that allows you to use headless systems while retaining most of its features. So if you already have a Shopify Plus setup, you can keep the frontend and migrate the backend to a microservices-based alternative for more flexibility.

In this post, we'll explain why businesses are moving away from monolithic e-commerce platforms like Shopify Plus and how they are doing it from a high level.

Why Microservices?

Shopify is a fantastic system in many ways, but if you use it exclusively you have to accept its limitations. Moving all or part of your system away from it can avoid these problems. Let’s look at some of the areas microservices can improve.

Security

Shopify has been in the news regarding data theft before. Managing your services means you’re free to address those concerns and implement additional security.
Its fraud protections aren’t as comprehensive as some services, so moving to a different payment system that matches your requirements can give you and your customers peace of mind.
Working with APIs means you can control the data flow in and out of your backend. It also means you can develop it incrementally while building on what you already have. That means you don’t have to reinvent the wheel when you want to offer something new on the frontend.

Technical Limitations

One issue with Shopify Plus is its lack of multi-store architecture, though this feature has been promised in the future and some plugins may help with it. If you can’t wait, you can build a backend that can handle data across all your outlets.
Shopify Plus's SEO approach is good but hard to modify. Using a custom frontend can allow you to change the URL structure, the robots.txt file, and give you better control over your subdomains.
It’s a similar situation with Shopify’s checkout. It’s hard to customize, leaving you with a fixed approach. What’s there isn’t bad; it’s just that you can’t deviate from it significantly or add specific features easily.
Shopify Plus can also slow down as you add features; its plugins don’t have to be certified. Handling services independently means you can ensure they continue to perform at the level you require.

Flexibility

Shopify allows you to get up and running quickly, but you’re tied into the way it does things. With microservices, you can take a modular approach, get the essentials in place, and add whatever you need on top.
Shopify apps aren’t always geared to work well internationally and can cost a lot if you have multiple local versions of your store. Data for multiple stores is also tricky to combine.
Shopify payments can have restrictions for products, as well as countries and currencies. You need a local business and bank account in each territory you have a store. That makes changing and expanding difficult.
With microservices, you can add an API query doing whatever it is you need—you aren’t limited to what Shopify offers. If you need to support a new feature, you can do that.

Replacing Shopify Plus

You can take advantage of the modularity of microservices to move from Shopify bit by bit. Identify what you can benefit most from replacing and build it. If you do that repeatedly, you can move away from Shopify step by step, improving your service with minimal disruption.

Step 1: Decide Where to Start the Migration

Let’s look at some of Shopify Plus’s services and decide what to implement using microservices.

Product Information Management (PIM)

Your Shopify installation will include lots of product-specific data. Moving this to a separate backend, with an API letting you retrieve, modify, and create new products, is a great place to start.

Decoupling this data from Shopify means you can use it on any of the new channels you create. It gives you complete control over what’s shared, and the formatting and structure can be defined however you like.

Fabric has a prebuilt PIM system that will help you make the transition quickly and easily and offers you plenty of flexibility. As well as helping you get set up, Fabric can also help with data migration.

Order Management System (OMS)

Your order management system includes your user’s shopping cart as well as payments. You have to smoothly pass the data on to your payment handler and delivery setup while allowing users to track and potentially cancel the whole process. There’s a lot involved, and everything needs to run like clockwork.

Taking control of this area means you can evolve it over time and perfect everything, adding features for the various frontend services you want to use while ensuring the core functionality is rock-solid.

Fabric can help with this, too. Its OMS promises a friction-free experience that can drive customer satisfaction and make life easier for everyone involved in your order and delivery chain.

Other Supporting Services

Shopify Plus offers a whole suite of features and services. Some of these will matter to you more than others—priorities differ from business to business. When picking a feature to include, you should consider what you’d like to make available to channels other than Shopify and which areas have the most scope for improvement.

These might include store management, customer personalization, or analytics. Take a look at the features Fabric offers to see what it can do for you in those areas.

Step 2: Build a Middle Layer

To integrate your new services with your Shopify installation, you can set up an API gateway. That will let you reroute requests to your new backend.

You could also adopt a service mesh model, which handles communication between your application components and lets you manage data flow between your existing application and external microservices.

With a new layer in place, you can look at replacing services one by one, thoroughly testing at each stage, and ensuring the new process is fully integrated before moving on to the next.

Step 3: Migrate Data from Shopify Plus to Microservices

To move data from your old system, you can export it to a CSV file, then use JavaScript to shift it across to your new microservice. You can do this as you go, create a service, copy your data into it, and then adjust your API gateway to handle future requests from Shopify.

Exporting Data from Shopify Plus

Shopify Plus has a simple export system, letting you easily create a CSV file. Its documentation explains how to do this for a product, customer, and order data, among other things. You can find an example file here.

Shopify product data includes the following fields:

Required	Required but can be blank	Optional
Handle	Body	SEO Title
Title	Vendor	SEO Description
Published	Type	Google shopping metafields
Status	Tags	Cost per item
	Image Src
	Image Position
	Image Alt Text
	Gift Card

There are more fields for products that have additional options and variants. Consult the documentation for details.

Your new system won’t have to use the same fields, so you can pick and choose which are relevant to you and which, if any, are required in your system. That kind of flexibility is one of the advantages of working with microservices. You can have a compact, faster system or one that contains whatever extra information you need.

Importing Data into Microservices

You can use JavaScript to transfer your data into your new system. Here’s an example showing how to import data using the latest version of Amazon SQS. You can find more details in Amazon’s documentation.


const { SQSClient, SendMessageCommand } = require("@aws-sdk/client-sqs");

const REGION = "us-east-1"; //or whatever your region is

const params = {

DelaySeconds: 10,
  MessageAttributes: {
    Title: {
    DataType: "String",
    StringValue: "Sneakers",
    },
    Vendor: {
    DataType: "String",
    StringValue: "Comfy Sneakers Inc",
    },
    InventoryQty: {
    DataType: "Number",
    StringValue: "146",
    },
    Value: {
    DataType: "Number",
    StringValue: "79.99",
    },
  },
  MessageBody:
    "Product Data.",

  QueueUrl: "SQS_QUEUE_URL", //replace this with your queue URL

}

const sqs = new SQSClient({ region: REGION});

const run = async () => {
  try {
    const data = await sqs.send(new SendMessageCommand(params));
    console.log("Success, product sent. MessageID:", data.MessageId);
  } catch (err) {
    console.log("Error", err);
  }
};
run();

Step 4: Test and Repeat

As you expand your microservices and move functionality away from Shopify, it’s essential to test thoroughly. This should be done at every stage. You need to have a rigorous testing system to ensure everything is still working smoothly, including the area being moved and everything else.

Advantages of the Iterative Approach

Taking a section by section approach means you can focus on a particular service and know where to look should any problems arise. That makes issues easier to fix and the overall adjustment smoother.

In addition to finding errors, it’s helpful to have metrics that measure the speed of transactions. If you can connect these to sales and customer retention, all the better.

Conclusion

Shopify Plus is an excellent platform with much to like about it, but the world is moving away from one-size-fits-all software toward more flexible, customized solutions.

Building services yourself is very achievable, but there is plenty of support available so you don’t have to go it alone entirely. An existing solution can save you time and money, and, as these services mature, the argument for that route gets stronger.

Consider taking advantage of prebuilt or customized microservices with Fabric to quickly build a system that offers adaptability for new challenges and opportunities based on solid, reusable foundations.

Building a Hybrid SQL + NoSQL E-Commerce Data Model

John G — Tue, 20 Apr 2021 15:52:33 +0000

E-commerce databases track unpredictable human actors across a complicated monetary system: at the end of the day, you want to deliver a product to a person you never see after getting a payment authorized by a third party. Processing payments, adding products, tracking orders, and handling rebates are all complicated features your e-commerce database has to handle in order to reach that goal.

This article will build on previous articles, talking about a scalable SQL data model and a NoSQL data model, to show how you can build a hybrid data model. You’ll learn how to utilize the strengths of various database engines specifically to handle more complicated e-commerce interactions.

Imagine a user returning one item out of a dozen that were discounted after being purchased in bulk. The user has forgotten their password, and the returned package has arrived at the wrong supplier warehouse. When this return arrives, your database model has to handle a complicated web of working relationships to put everything back in the right place. You might also need to trace the chain of events that led to this outcome so you can send the customer a satisfaction survey.

This article will make a database that can shine in this situation.

Modeling Your Users

The foundation of any e-commerce site are its users. They perform every action, and each one is a complicated, out-of-your-control person. Users move, change payment methods, forget to inform the businesses they frequent, then request a refund.

Fortunately, you only need to know a very fixed set of facts about users. There are rarely new types of things you need to know about people, but they are queried in a variety of ways. Some common access patterns for user data are adding new addresses or updating email addresses or passwords.

Because they feature a fixed set of data with a wide variety of access patterns, user data is a great candidate for a structured SQL data store. Also, user information is one of the best sources of analytical data for your business. For example, you’re likely to want to perform ad hoc queries about the users buying your products, further cementing the suitability of SQL for your user data.

The following user model implements some complicated patterns that an e-commerce site has to support, like resetting a password, adding multiple shipping and billing addresses, as well as tracking how long users spend on various pages and what they clicked to get there.

When you’re building out your model, be aware that you don’t need to collect all the information up front. You don’t need to collect a user’s shipping or billing address when they first sign up for your application. For example, Fabric keeps the onboarding process simple, only asking for an email, name, and password at sign up. However, as we’ve seen in our e-commerce site breakdowns, many businesses complexify this step, making it hard for people to checkout online.

Next, let’s look at what you’ll show users.

Your Product Catalog

A product catalog represents every item that’s available for your users to purchase: objects, subscriptions, or donation tiers. All of these options have different structures and need to keep track of slightly different things. Subscriptions have features or access levels, while objects have sizes, colors, and images. There’s a lot of flexibility and variety as to what can live in a product catalog.

Mixing in NoSQL

Unlike SQL data, which is structured, normalized, and optimized to save on storage, NoSQL data is denormalized and optimized to save on CPU. This means it’s a bad choice for data analysis applications (like tracking users on a page) but a great candidate for high throughput applications when you fully understand the access patterns.

NoSQL can be perfect for hosting product information. Most of the time, product information is simply displayed to a user, and the stock available might be updated if an item is purchased. NoSQL databases are great at creating instantiated views populated with data from a single round trip query.

With NoSQL, your goal is to store all the data that your application uses for a page inside a single document. Here’s an example:

With this data model, when a customer requests a product page, your application requests and receives everything it needs to render the page in a single round trip. Images, prices, options, and dimensions for every variant that a customer can inspect are all stored in a single document. This means the data layer doesn’t spend CPU time finding and mashing together various tables, making NoSQL requests very horizontally scalable.

One of the other benefits of NoSQL is that data is returned in a format the web understands (JSON), which can save you the trouble of writing a custom Object Relationship Mapper (ORM) for your products.

What to Cache

One of the data storage options available to developers is in-memory storage, like Redis. With any caching strategy, the goal is to preserve CPU and disk I/O by moving transactions formerly performed in your data layer into faster access hardware like RAM.

The features that make the product catalog a good candidate for a NoSQL data store also make it a good candidate for caching in an in-memory data store. The data is read heavy and rarely changes, so caches will save disk time while maintaining accuracy.

Caching is also a good solution for session management, and in the e-commerce context, storing a user’s current cart. Rather than looking up a cookie in your database or returning all the items in their cart every time a user makes a request, you can push that workload to your caching layer and achieve a request response time that would require significantly more resources if handled by your database.

Payment Models

Payment systems allow users to purchase and receive your products by linking your website to various services that enable payment. This is a vital moving part of any e-commerce website, and it has to function well to ensure customer satisfaction. A broken checkout flow is often a dealbreaker.

This step also has the most stringent security requirements for what kinds of data are stored and how. In general, don’t store much credit card information, and be sure you encrypt any identifiable payment information you do save, like the card identifier. The less data your payment model stores, the better. It reduces your legal liability and helps you stay in compliance with Payment Card Industry-Data Security Standards (PCI-DSS).

The ideal payments table does not contain much information. When a user finally clicks Buy after entering their payment information, your application can mostly bypass your data layer. Instead, it will pass the request to a payment processor, which will handle it and return the result.

If you think you need to store payment information for future use, your payment processor very likely has an option for storing that data and will provide you a unique, not personally identifiable token to use next time.

Sending Your Product to a Customer

Order models are the part of your database that tracks the process of getting a product to a customer, from the moment they first express interest all the way through getting the item into their hands. An order can involve ongoing access to a website or handling a discrete event, like sending a product to a client.

A smooth order flow is vital for creating a user experience that people are willing to repeat. The order is where all our other database schemas get put to use.

Here’s an order for a product:

You’ll notice that discount and price information have been duplicated from our NoSQL data model into the cart_item table. The NoSQL model represents the discounts that are shown to a customer when they enter a page, and the saved price and discount are the discounts that were actually applied to the item.

If a price or discount changes, you’ll want to be sure to keep a record of what prices were actually paid. Be aware of fault lines like this between your NoSQL and SQL components.

Wrapping Up

Revisiting the initial scenario presented in this article, let’s see how the model fares all stitched together. The user has forgotten their password, but our user’s model handles updating it. The returned package has arrived at the wrong supplier warehouse, but we tracked the original source in the order, so we can forward it to the correct destination. Calculating the refund is tricky, but because we tracked discount information on an order level, we can run an ad hoc query to figure out the appropriate rebate.

When planning your data model, remember that SQL is optimized for efficiently storing structured data and performing arbitrary queries on data, while NoSQL is optimized for efficiently delivering unstructured data.

There are dozens of challenges that an e-commerce website can face that aren’t covered by this model, like handling subscriptions, applying a discount to an item based on user behavior, and tracking inventory. Fabric has a team of experts who’ve built e-commerce data models and websites for years, offering simple solutions to these complicated problems.

What’s an Example of Good E-Commerce Database Design?

Shanika Wickramasinghe — Sun, 18 Apr 2021 18:02:06 +0000

What's an Example of Good E-Commerce Database Design?

Databases are vital tools for storing, managing and retrieving information, and they are critical for building an e-commerce system. A well-structured database is the heart of a good e-commerce system that manages all the interactions of the system.

A good e-commerce database design should include the following characteristics.

Simple and Functional Database Structure
The database table structure should be relatively simple but needs to cover all the required functionality without compromising the user experience.

High Performance
The database queries should execute fastly and efficiently to facilitate live customer interactions and provide a satisfactory shopping experience. Therefore, the selected database should consist of good indexing and performance optimization options.

High Availability and Scalability
A good database design should be highly available with automatic snapshots and enable automatic scaling to support future platform growth as well as sudden traffic spikes.

Based on the above characteristics, an e-commerce database design boils down into three major aspects as the following.

The Scope of the Database.
Type of the Database.
Database Infrastructure.

The scope refers to the planned functionality of the database. The underlying table structure of the database, its relationships, and indexes all depend on the functionality of the e-commerce platform. The database type can vary from a relational database to a NoSQL database or a hybrid approach depending on the requirements and the underlying data structure.

The Scope of the Database.

The main consideration when designing the database is identifying the functionalities offered by the e-commerce platform. These functionalities can be further divided as core-functions and extra-functions (additional-functions).

Core-functions are the functions that are necessary for facilitating the day-to-day operations of the e-commerce platform.

User Management
Product/Inventory Management
Shopping Cart Function
Payment Management
Shipping/Logistics Management

Extra/Additional functions are the nice-to-have functions for the e-commerce platform that will enhance the user experience for both end-users (customers) and administrators (business). Following are some functions that come under this category.

Marketing Functions
Help Desk/Support
Advanced Analytics
Third-Party Integrations

Database Structure for Core-Functions

In this section, let's see how to structure the database to facilitate the core functions. The below table structure is an example of a database design that covers the core functionality of an e-commerce platform.

This example covers all the basics of an e-commerce platform. There, the table fields and indexes depend on the design of the overall platform, and it contains three separate sections for user management, product management, and shopping process. Let's have a closer look at each section.

User Management

We have created a user table that contains all the user details along with user_payment and user_address tables to store multiple addresses and payment details of users. This structure offers more granular control over data while eliminating duplicate records.

Another way to manage users is by creating two separate user tables for end-users and administrators and assigning relationships according to their requirements, as shown below.

Product Management

Managing products is not simply about maintaining a list of products. You also have to manage the inventory, discounts, categories, and other attributes of the products. So, always focus on simplifying the data structure while reducing duplicates. In the above table structure, the main product table contains information about the products.

There are two other separate tables called discount, product_inventory, and product_category that are connected to it through database relationships. This approach provides the greatest level of flexibility to the database. For instance, we can simply query the product_inventory table to check for inventory without going through all the data associated with other related tables. This is also a good place to utilize indexes to increase the performance of the database.

Shopping Process

This is the most critical and complex part when it comes to designing the database. The shopping process will guide a user to search the products, add the desired products to the shopping cart, and finally complete the transaction using a payment provider.

The heart of the e-commerce process connects users with products. A good chunk of design effort should be exhausted to streamline the shopping process. In the above example, there are shopping_session and cart_item as temporary data stores that only store the shopping session information of the current user until the order is confirmed and the data is moved to permanent storage tables with the payment details. (order_details, order_items, and payment details.). You can check out this article for a detailed explanation of how to design a shopping cart database.

As shown in this section, the scope of the database structure is determined by the overall functionality of the platform. Therefore, it is paramount that you properly define the required functionality before diving into designing the database. This way, you can create a clearly defined data structure with enough flexibility to support future expansions.

Type of the Database

The next consideration is to determine the type of database. To have the best e-commerce database design, you must first consider two main database types: RDBMS or NoSQL databases.

Relational database: This is similar to spreadsheets and uses tables, columns, and rows to organize and retrieve data. It is built using the standard query language (SQL), and all the data is related to each other.

Example: MySQL, PostgreSQL, MariaDB, Microsoft SQL, Amazon RDS, Azure SQL Database

Non-relational or NoSQL database: This is a nontabular database with a flexible schema that works well for storing unstructured data. Contrary to its name, a non-relational database can store related data. It does so by nesting related data within a single data structure instead of splitting them between tables. There are various NoSQL databases, and the most popular ones are document stores and key stores.

Example: MongoDB, Apache Cassandra, Amazon DynamoDB, Azure CosmosDB, Couchbase.

Both these database types are solid options for any e-commerce platform, and the choice is up to the designer and depends on the requirements of the platform. A relational database will provide a simple and robust platform to create the database, while NoSQL offers better data flexibility, scalability, and slightly better performance. In the coming section, we'll explore an example of e-commerce database design for each type.

E-Commerce Database Design Examples

Relational Database Design

Many e-commerce sellers use a relational database design centered around the following tables:

Products table
Customers/Users table
Orders table

Additional tables can be added as required to support shipping, categories, product reviews, etc. We have covered the database structure in the previous section, where the scope of the database was defined.

Below, you can see a diagram of a simple e-commerce database design built using MySQL.

MySQL e-commerce database design

While a standard relational database design will work for many e-commerce stores, there are situations where it is advantageous for retailers to follow another approach.

NoSQL database design

Most of the world's largest online retailers, including Walmart and eBay, use NoSQL databases to power their e-commerce systems. The reason is NoSQL databases providing the required performance and scalability to effectively manage large catalogs and unstructured data like user data and images.

A good example of an e-commerce database built using NoSQL technology is CouchBase, which uses a document store as its database type. With a document database, data is simpler and easier to access as an entire product can be stored in a single document instead of storing across multiple tables.

Below, you can see an example of data modeling for different products with CouchBase's NoSQL database.

CouchBase NoSQL data structure

source

Hybrid Database

Another exciting option is to use a combined solution utilizing an RDBMS for structured data such as user details, order details, payment details, and a NoSQL database for unstructured data such as product details or marketing information.

Hybrid database structure

Creating a Hybrid database structure is relatively more complex than using traditional database types (RDBMS or NoSQL) as a hybrid approach will utilize the application to combine data from both sources. This allows the developer to combine the strengths of both database types. However, this kind of approach is only suitable for truly complex application architectures that offer users a myriad of functions. For instance, if your purpose is to speed up an RDBMS, it's easier to configure a caching server such as Memcached rather than going for a hybrid approach.

Database Infrastructure

It is possible to select a database type and design the data structure, but we also need infrastructure to run the database. It depends on the deployment of the e-commerce platform. One option is to provision a server, install the database software and then manage all the aspects from security to maintenance. The other option is to use a managed database service like Amazon RDS, Azure SQL for RDBMS, Amazon DynamoDB, or Azure CosmosDB for NoSQL. Among these two options, the managed approach saves time and does not require any maintenance as these are SaaS offerings.

Suppose you need complete control over your data, database configurations, and server security policies or deploy the database on a private network. In that case, the only option is to configure and maintain the database manually. However, with more and more businesses utilizing cloud providers nowadays, the best option would be to use a managed database. It is more applicable for an e-commerce platform where scalability and availability play a vital role in the success of the platform. So, it's almost always advisable to use a SaaS database.

A managed database will offer peace of mind for any developer as it is backed by a reputed cloud provider with a service level agreement to provide maximum uptime. It will also enable the developer/administrators to focus more on creating and optimizing the database without dealing with server or database maintenance. A slight downside of a managed platform would be the higher cost associated with it when compared to managing own servers. However, when considering the total cost of ownership (TCO) and operational expenditure (OpEx), a SaaS solution is the ideal option for a growing e-commerce database.

PIM Software for Multichannel Sellers

Product Information Management (PIM) is a critical component of the core functionality of any good e-commerce platform. It provides a flexible solution for managing all your product data inside a single database in the cloud. Moreover, PIM is an excellent option for retailers looking to sell on multiple channels. Having a centralized database in the cloud, PIM allows easy management and delivery of product information to other marketplaces.

Another main advantage of PIM is the ability to collect and organize data in a variety of formats. Different platforms have their own catalog structure, and PIM automatically sends data to these channels in the proper format.

Fabric's PIM is one of the best examples of e-commerce database design using PIM software. It serves as a single source for centralizing all your data in the cloud, allowing businesses to benefit from increased productivity while ensuring all information is accurate and up to date across various channels.