Forem: Appsmith

Automatically Generate a CRUD App from Any Database in Just a Few Clicks

Appsmith — Thu, 23 Nov 2023 10:40:37 +0000

This tutorial will show you how to quickly generate a complete graphical application to create, read, update, and delete (CRUD) data from almost any database, including MySQL, PostgreSQL, and MongoDB, in just a few clicks.

Appsmith is an internal app platform that reduces the labor of generating user-friendly interfaces to your data, so that you can quickly iterate on new CRUD application ideas and build apps your organization can rely on.

What is a CRUD application?

CRUD is an acronym for the basic operations that you perform on a database to interact with the data in it:

Create a new row of data in a table
Read one or more rows from a table
Update one or more rows in a table
Delete one or more rows from a table

Every application that stores data in a database is built on these basic operations. Examples of this are:

Social media apps: create posts, read posts, update posts (add likes or comments), and delete posts
Project management apps: create tasks, read tasks (to see the status of a task), update tasks (with comments), and delete tasks (once completed)
Calendar apps (such as Google Calendar): create appointments, read appointments (to see what you are doing today), update appointments (if you have to shuffle meetings around), and delete appointments (if somebody has to cancel a meeting)

Choosing the right CRUD generator to build your ideas and internal tools

No matter what kind of application you are building, it must be able to perform CRUD operations. Connecting an interface to your database and manually building all of the pages, forms, fields, and interactivity required to make your application work typically involves a lot of work. Before you start developing, you need to decide whether to start from scratch or build using existing tools.

A number of tools exist to generate CRUD interfaces, but most of them are specific to a particular database platform, are not cross platform, or require further development work to work for end users. The best tool for the job allows you to quickly spin up new apps and completely customize their functionality.

Newer app platforms satisfy both of these requirements as they allow you to massively automate certain repetitive steps in the software development process while still having all the versatility of building from scratch.

Appsmith automatically builds database CRUD interfaces for any database with just a few clicks

Appsmith is one such app platform that leverages automation to allow developers to quickly spin up applications, including database front ends. Then you can use custom SQL queries and front-end JavaScript logic to interact with back-end datasources including MySQL, PostgreSQL, and many other popular databases.

To speed up this process even more, we’ve created a new feature that automates the creation of CRUD apps in Appsmith even further. We’ve named this feature “Generate New Page” because it creates an entirely new page in your application with all the widgets you’d need to perform CRUD operations on your database.

Now, Appsmith can inspect your database and automatically generate all of the forms and inputs required to interact with it on the front end. From there, you can tweak the application using a drag-and-drop WYSIWYG interface to further customize.

How to implement a CRUD application with Appsmith in under five minutes

Assuming that you already have your database, you can create a complete application to create, read, update, and delete data with three simple steps.

Step 1: Get Appsmith

Getting started with Appsmith only takes a minute. You can sign up for the free cloud-hosted version to get up and running immediately or self-host Appsmith on your own infrastructure.

Step 2: Create a new Appsmith application

After you sign into Appsmith, create a new application and give it a name.

Step 3: Click “Generate new page” to create your database UI

From here, you can add whatever datasource you want to your Appsmith application. For this tutorial, you can use the mock users database that ships with Appsmith. Do this by clicking Show all datasources under the Datasources section, clicking Create new, and then selecting the users database.

Then click Show all datasources again, hit Generate new page for the users datasource, and fill in the information needed to generate your front end.

Et voilà! Now you have your own complete, fully functional CRUD front end for this database.

From here, you can tweak your new database CRUD UI to your heart’s content, including hiding fields, changing field types, making text areas dropdowns, and almost anything else you could imagine. That way, you’re completely in charge of how your new app looks and behaves.

Save time and effort when prototyping reliable production apps

Appsmith’s ability to generate a new page, automatically populated with form inputs and wired up to perform CRUD operations on the database you feed in, is a real time saver for developers.

This is perfect for interacting with databases at any level, whether you’re quickly spinning up a database front end to debug, building an internal tool that allows technical and non-technical team members to easily update a database without mistakes, or creating a full-fledged application that interacts with multiple databases.

If you want to quickly spin up your own CRUD applications from a database, sign up for the free cloud-hosted version of Appsmith today and give this feature a try for yourself!

Git in Appsmith: The Details behind Our Implementation

Appsmith — Wed, 15 Nov 2023 15:05:43 +0000

As we discussed in the previous article in this series, we’ve spent a lot of effort over the last year polishing Git in Appsmith so that it’s ready for you to use, just the way you’re used to using Git. Since Appsmith is open-source, we’re happy to share the ins and outs of this process and what we’ve learned while developing this feature.

The biggest problem we had to solve was moving from the standard [local repository]—[remote repository] model in Git to our [web client]—[Appsmith server]—[remote repository] model. Another major difficulty was reliably (and efficiently!) converting Appsmith applications that are serialized to a database into the source files that Git expects to work with. In this article, we will examine our Git implementation and the solutions to these problems.

The Git implementation is more complex in Appsmith because of the interaction between Appsmith servers and web clients and the conversion from database entities to source files. This extra step is hidden from the user.

Why Git instead of another VCS (or something home-grown)?

We chose Git because it is almost synonymous with “version control” in modern software engineering, and for good reason. Git was the first successful open-source, decentralized source control solution created all the way back in 2005 by Linus Torvalds, the creator of the Linux kernel. He has a great talk about this from all the way back in 2007, but I will highlight just a few differences between Git and its predecessors.

Git looks at commits as a collection of every source file in the filesystem stamped with a SHA-1 hash, rather than each file having its own version as most previous *v*ersion control systems (VCSs) do. This makes a lot more sense, because developers don’t really care about individual files themselves, but rather the functionality of the entire application.

Another major improvement introduced by Git was cheap branching. In previous VCSs, branching was a huge hassle that often took hours or days of planning. In Git, a branch is just a pointer to a commit, which makes it extremely cheap to create. All Git has to do on the back end is store the SHA-1 value of the commit that a branch points to in a file, which is around 40 bytes (not kilobytes, just bytes). This branching model, combined with the tree data structure to organize commits on the back end, made parallel development the default.

The last major innovation from Git was the idea of decentralized development. Since Linus wanted an open-source solution to manage the Linux kernel, he shied away from previous solutions that relied on a centralized server. With previous VCSs, if you didn’t have an internet connection to the centralized server, you could not commit changes. But with Git, you could have an entire repository that you could commit to just on your local machine without any internet connection and then push your changes to a remote repository for redundancy and sharing when you wanted.

Git is backed by a strong open-source community, is widely supported by different major hosting providers like GitHub and Bitbucket, allows plugins to customize workflows, and has a nearly two-decade-long track record of doing the job of source control well. These are some of the main reasons why Git is so ubiquitous today.

How we implemented Git in Appsmith

This was a massive undertaking for the Appsmith team, but we need to acknowledge that we are really standing on the shoulders of giants.

Without Git itself, we’d be having to implement our own VCS paradigms (and probably learn a lot of lessons that others have already learned), and without the JGit project, an implementation of Git in Java maintained by the Eclipse foundation, we wouldn’t have a stable foundation for the required back-end operations in Git.

However, even though leaning on high-quality third-party libraries saved us a lot of development toil, we still had plenty of problems to solve to make Git and Appsmith compatible.

Aligning Appsmith and Git to work together effectively

Working with Git on an app platform like Appsmith needs to make sense to users who are comfortable working on the command line. The biggest challenges were moving from the [local repository]—[remote repository] model of Git to our [web client]—[Appsmith server]—[remote repository] model and transitioning from applications stored as documents in MongoDB to source files on a filesystem.

These difficulties led to the following design decisions to make sure that Git and Appsmith could function properly together:

The Appsmith server, not a user’s web client, stores the local repository.
This local repository still communicates normally with the remote repository (like GitHub) for push and pull operations.
Changes are transferred between the Appsmith server and the web client for users to see.
We maintain a filesystem on the Appsmith server that mirrors the entry in the application database.

Most of the issues we experienced were caused by the extra complexity when transferring data between the web client and the Appsmith server as that doesn’t exist in the traditional Git workflow.

Authentication issues

Early on, authentication issues were a major blocker, because we wanted to find authentication methods with the right balance of convenience and flexibility. We initially gravitated towards user-based authentication methods (such as OAuth), but later settled on key-based authentication so that we could support as many hosting providers as possible with a single solution.

Managing application source files on the Appsmith server

The major components of Appsmith applications are stored in Appsmith’s database. This includes everything that makes up an application -- datasources, queries/JS Objects, widgets, and pages. This presents a few problems as Git only works with files on a filesystem, so we need to take this data and write it to files reliably and safely.

Our first concern was the security of our users. One of the primary ways users accidentally leak credentials is by committing them to a VCS repository. For that reason, when storing datasources in source control, we store as much of the configuration as possible in a .json file that can be committed to Git, while excluding any secrets. We then store the encrypted secrets only in the Appsmith database where they are safe from accidental disclosure.

We also serialize queries and JS Objects and write them to files. For a particular page on an application, each query has its own directory, where we store the query in a text file combined with any relevant metadata as a separate .json file. We write JS Objects to .js files containing the object code.

a) The text file contains the actual query while the b) metadata.json file contains information about that query, including the datasource it is built on.

Widgets are front-end React-based components that handle user interactions and trigger back-end queries and other actions. As of October 2023, Appsmith offers 40+ widgets out of the box. These widgets are stored in source control as a collection of .json files that describe the subcomponents of the overall widget.

This is a subset of the files stored in source control for one of the more complex widgets in Appsmith: the tabs widget.

All of these components (along with their associated metadata) are stored as text files, .js files, or .json files under a particular page directory in source control, corresponding to the different components of Appsmith applications. This filesystem is maintained by the Appsmith server, which transparently converts the contents of its database into source files for Git to use whenever changes are made.

Getting to the point where we could convert every ****single part of an application stored in Appsmith’s MongoDB database to and from source files like this was easier said than done. This is something that we had to continually polish to make sure all of the components and metadata were being imported and exported correctly throughout the entire application.

Performance

In a normal Git workflow, all development takes place on the machine holding the local repository and performance considerations over a network are only relevant for push and pull operations.

As development takes place on a user’s web client, that information needs to be transmitted to Appsmith’s server, which is serving as the local repository. This means that all operations are network-dependent, which is why it was so important to make these operations as performant as possible.

Early on, many operations on the Appsmith server were simply too slow, which caused the Appsmith client to time out on certain requests. This caused partially-completed operations or operations completing out of order, ultimately leading to corruption. This is the main reason that Git functionality was so unreliable in the early days and we spent a lot of time fixing this issue.

Ignoring unchanged files

One operation that took particularly long was the conversion of the application from its database representation to its filesystem representation. These filesystem operations were I/O-heavy and therefore took a long time to complete, often causing timeouts. This made it look like operations were failing, even though they were working correctly on the server, just not quickly enough.

We realized that the most efficient way to resolve this was to avoid the write operation entirely, if possible. Each component of the application stored in the database has its own timestamp and we use this information to tell if that component has been altered since the previous commit. If it hasn’t, then we just skip the write operation for that component to save time.

Ignoring metadata

A more complicated problem was ignoring changes that didn’t make sense for a user. There is a massive amount of metadata for the various components of an application that wouldn’t necessarily make sense in a git diff, because it doesn’t correspond with changes that users made to the application.

The problem is that this data was intermixed with changes that users did make. We had to rework the architecture of the files that were stored in source control to pull out this metadata into its own file that could then be ignored in client requests to further speed up operations.

These improvements cut down the response time for requests to the Appsmith server by about 4x. This turned our Git functionality from something that sometimes worked into something that could be relied on as part of mission-critical workflows.

The road ahead

There is still one major issue, although it affects user experience rather than functionality. Whenever we upgrade a version of Appsmith (which happens regularly on the cloud-hosted version, since we’re always improving), that causes a change in the underlying metadata of the application. Then when a user makes their next commit, in addition to the changes they made to their application, they see other (often major) changes to files they didn’t touch.

This can be very confusing and cause the user’s changes to be drowned out by these other changes. We’re aware that there is a lot of customer pain around this, so we’re trying to better communicate these metadata changes to users by keeping automatic commits separate from user commits, but that is still a work in progress.

As we mentioned before, we also want to improve the output of git diff so that users can get an idea of the exact changes made rather than just where the changes were made in the application. At our recent Hackathon, one team created a demo of this functionality, although we still have a lot of work to do before it’s ready to be released.

We are focusing on auto-deployment functionality as well. We want users to be able to automatically create development, testing, and production branches to neatly extend to different environments in a CI/CD workflow. We see this as critical to any robust development process.

Further ahead, we will also be adding branch protections in the local Git repository. These can already be enabled in the remote repository, but we want to add that functionality to the local repository as well to prevent Appsmith applications from entering a bad state.

Git in Appsmith is a huge leap forward for productivity and reliability in internal apps

Modern software development practices would be impractical, if not impossible, without Git. Nobody would think to develop an application without first having robust and reliable source control in place. Just because internal app platforms allow users to create applications much more quickly and efficiently than other tools doesn’t mean they shouldn’t offer native Git integrations.

But the truth is that many internal app platforms don’t bother, because solving these performance issues is so difficult. That’s why solving these issues and getting a reliable feature was a monumental achievement for our team.

And now that Git is available in Appsmith out of the box, your team can quickly iterate and test your apps, roll back if there’s a problem, and always see the current state of your codebase using the same industry-standard tools as in any other IDE. Read on to the next article to learn how to best leverage Git specifically for your applications built in Appsmith.

Git in Appsmith: Every Developer Has Been Saved by Git — So, Why Isn’t it a Feature of App Platforms?

Appsmith — Mon, 13 Nov 2023 14:55:08 +0000

Version control (which, these days, almost always means Git) is vital to modern software development. Git is the mechanism that all (responsible) developers use to manage changes to their code and collaborate with others. Without Git, we just wouldn't be able to rapidly build and iterate software the way we do today, with constant experimentation by teams spread across the globe.

And yet, until last year, Git has not been a feature of app platforms. This three-part series explains why it took so long for Git to appear in app platforms, how Appsmith finally implemented this feature, and best practices you can adopt when bringing Git workflows to your own internal app development.

What is Git and why is it so useful to developers?

Git is an open-source distributed version control system. If you've done any software development in the last couple of decades, you'll already be familiar with the reasons Git is so popular:

It lets you easily track code changes: Developers need to be creative to solve tough problems. Being able to easily experiment while keeping everything organized and reversible encourages this.
It enables smooth collaboration: Team members can collaborate on Git repositories, no matter where they are in the world. Git allows you to work on multiple branches of the same codebase at the same time, so that changes can be made without breaking others’ work.
It helps you catch mistakes before they enter production: Git shows the history of every change that's been made to your code. This makes code review easier and helps developers spot problems before they are deployed to production. If a problem does sneak through, it's possible to quickly revert back to a previous version.

Git is a force multiplier for development teams, greatly improving the engineering process. Development teams using Git can be more creative and take greater risks, all while placing themselves under much less stress. From a product perspective, Git lets you evolve your code rapidly to respond to market needs and easily maintain different versions for different use cases.

Why don't other internal apps platforms have Git?

To be upfront, we think the reason Git hasn't become a cornerstone feature of other internal apps platforms is because they consider it too difficult a problem to solve, even if it would greatly improve the experience for the developers building with their products.

App platforms and low-code interface builders store their application logic, interfaces, and data in a database, rather than in files. Git, on the other hand, works on a filesystem. Bridging these two systems to track the changes to code generated by these platforms may not seem complicated, but once you factor in performance, safeguards against corruption, and implementation of additional Git-native functionality like authentication, branching, and merging, things can get pretty hairy.

Some platforms skirt around these problems by having ”Git-compatible” or ”Git-like” functionality provided by a home-baked version control system that avoids these pitfalls by just not providing certain functionality, but they are often locked behind enterprise plans and, frankly, they just aren't Git.

If you want version control and integration with your existing tools, syntax that every developer is familiar with, access control, thousands of compatible tools including GitHub, BitBucket, and Visual Studio Code, and plugins for almost everything... it has to be Git. There is just no substitute.

Every developer has a story about Git saving the day (or their project, or their job…)

Not using Git when it's available would be considered a massive oversight by any development team. So, despite the challenges of implementation, why hasn't its absence as a feature been considered a massive oversight by the developers of app platforms? Surely their own development teams are aware of its benefits — we've all been saved by Git at least once!

Git saves the enterprise: AWS spot instances nearly wipe out a month’s work

We've collected a couple of stories from our team to demonstrate just how important version control is. This story comes from the Appsmith internal Slack channel and shows how Git can help save a business from unexpected data loss (and an innocent mistake from a developer):

...So we had an enterprise customer who was building their internal tools on Appsmith. To save costs, they had decided to do this on an AWS spot instance, which they thought was surprisingly cheap! What the customer didn't know was that spot instances have no data persistence. If they are shut down or rebooted, or crash, or AWS decides to re-provision them, everything is gone (surprise!).

This particular spot instance had a pretty good uptime: it made it a whole month into development before it was unexpectedly wiped. All development progress was destroyed, or it would have been, if they hadn't been using Git. They were back up and running in an hour after setting up a new instance and syncing their repositories. Their Appsmith apps were all intact. As I understand it the phrase ”spot instance” is currently banned in that customer's meeting room.

Git goes hand-in-hand with backup best practices that stipulate that you should store backups in three separate physical locations to protect against data loss. As Git is distributed, it's easy to make sure that it is synchronized off site, where it is protected from theft, fire, or the dreaded rm -rf.

Git saves the developer...from Git (and themselves)

Another story from our support channels, again showing how Git is already making a big impact for developers building with Appsmith:

Git is popular because it's the best at what it does, but that doesn't mean it's without problems. Many of these problems are, however, not technical, but derive from PEBKAC, usually when two developers are working on a project and merge incompatible changes. Or a solo developer can do this too, I guess, by merging two of their own feature branches without double-checking things.

We had a user, not a corporate user, but someone on our Discord we were helping out. In this case, a merge of two Appsmith feature branches made a total mess. This developer was a little green and had maybe over-extended himself, and wasn't sure how to resolve the issue. We were able to jump in and help him sort out the mess and salvage the project in time for him to deliver it to his client without a delay. In a sense it was Git saving the developer from Git, letting us piece together a working version from a very messy, bad state.

Troubleshooting and code reviews for your internal apps are made much easier with Appsmith’s Git functionality. You can view exactly what has changed and pinpoint where an error or conflict has occurred.

Now imagine how these scenarios would have played out without Git. Nightmarish!

We took up the challenge — Appsmith's platform is now the only one with Git

As you've probably figured out by now, we're talking about Git because we're excited to have finally brought it to Appsmith. No compromises: it's as Git as Git gets.

This wasn't an easy journey. While this functionality was in high demand, early versions were frustrating to use. In our earliest implementations, it wasn't even possible to pinpoint where the conflicts were in a file. Even members of the Appsmith development team would avoid using our early Git implementations. We even had a rule for our internal “Hackathons” that using the Git feature was banned because it kept breaking! So we know why other app platforms had avoided fully implementing Git: it really was a challenge to get it working right.

It took three developers five months of rigorous work to get our Git implementation polished and working to our high standards, and, as you can see above, our customers are already benefiting. When we see disasters mitigated and workflows sped up with Git, we know it was totally worth the investment, and we're hoping, now that we have real Git integrated into our app platform, others will follow.

Read on to part 2 of this series to find out more about how we solved the big problems that were preventing Git from proliferating in this space, or skip ahead to part 3 for tips and best practices for using Git for your own internal app development.

New to Appsmith? You can sign up to our free cloud hosted app platform, or self-host using Docker.

The Ultimate Guide to Building Internal Tools in 2024

Appsmith — Wed, 08 Nov 2023 16:44:14 +0000

Welcome to Appsmith's guide to internal tools for 2024. This guide explains what internal tools are and how your business relies on this software, provides a list of the current most popular business tools for each category, and explains the long-term advantages of building your own internal tools to match your unique business processes and industry best practices.

What are internal tools and why are they important to your business?

Internal tools is the term used for the software your business uses to support its internal operations. These tools encompass data entry tools to query and update your core business data such as customer and employee records; tools to log, track, and respond to customer support requests; tools that move data between systems; and any other software used to run or manage your business processes.

The goal of these tools is to streamline and optimize the operation of your business by:

Automating repetitive tasks and complex workflows to improve productivity
Providing user-friendly interfaces that non-technical staff can use to interact with data
Connecting and displaying disparate information from different tools in one location
Solidifying your business processes around the “best” way of doing things, while not hindering the evolution and improvement of your workflows.

For larger organizations with multiple teams, internal tools should also ensure that the data required for each of your departments to operate is readily available without having to wait for information to be provided by another.

Some successful examples of this include:

Airbnb built a reporting tool to simplify running experiments by closing off all the pitfalls and automating the heavy analytical lifting.
Facebook has a tool to enable staff to build their dashboards.
SaaS startup Fyle built an internal tool to reduce the support team’s dependence on engineering.
Oscar, a significant insurance provider, solved siloed systems to track and manage authorization requests with a utilization management app.
Amazon CDK is an excellent example of an internal tool (which they’ve since open-sourced) for defining resources for your cloud application using well-known programming languages.

Your business will come to rely on the internal tools you choose. They will either define your business processes or need to adapt to your existing processes so as not to disrupt the operation and productivity of your business. If you're an established business, you no doubt already rely on several. Think about your accounting software, or maybe you use a spreadsheet to track employee hours — both are internal tools.

You must choose carefully from the innumerable business products intended to help your internal processes. The most popular solutions are designed for broad use cases, while others are niche products designed for specific industry use cases. You will most likely need several tools for different tasks, and they must play nicely with each other.

Many businesses forgo these pre-built options and build their own to ensure that their tools are completely optimized and integrated with their use cases. We'll discuss the ups and downs of building your own tools further along in this article.

Choosing from the different types of internal tools — what's popular now

Most internal tools take the form of a database and a user interface to create, read, update, and delete data in it. Built on top of this will be task-specific functions to automate and speed up the process of interacting with the data in the system. While general-purpose tools like spreadsheets are useful, they are outclassed by tools that are tailored to a specific purpose.

For example, an employee timekeeping system will include a database containing records of when staff worked and an interface for managing these records. This is all easily implemented in a spreadsheet, but someone has to spend time to keep it up to date. A good choice of a product to improve on this would be a timekeeping system with the function for employees to quickly clock in and out using a PIN so that records don't have to be manually entered, removing the need for manual management and increasing the accuracy of the data.

The more complex the process an internal tool is designed to model or manage, the more useful these additional automated functions will become. Several primary categories of internal tools have emerged over the years to address specific business problems. Due to the positive outcomes seen by the businesses who adopt them, each has become a necessity for modern businesses looking to grow.

Customer relationship management (CRM)

What is it? Customer relationship management software is used to manage customer interactions.
How does it benefit your business? You can ensure that your customers are getting the most value from your business by tracking user and prospect contact details, identifying qualified leads, responding to requests, and managing marketing campaigns.
Popular solutions include SalesForce and Zoho CRM.

Content management system (CMS)

What is it? Content management systems enable teams to collaborate on projects involving media such as blog posts, videos, and image content for websites and other content outlets.
How does it benefit your business? Users can create, manage, modify, and publish content in a user-friendly interface, speeding up the creative process and allowing for the approval of content before it is published externally.
Popular solutions include WordPress for managing website content and SchoolNow for managing content in education environments.

Customer success

What is it? Customer success tools aim to bring your customer data and communications together in one place for analysis.
How does it benefit your business? A unified overview of your customers will help you find out which customers might be at risk of churn, identify which customers might have potential for upsells, and tailor user flows and messaging for the greatest effect.
Popular solutions include HubSpot and ChurnZero.

Workflow management

What is it? Workflow management tools provide a way to automate, standardize, audit, and improve business workflows.
How does it benefit your business? By automating day-to-day processes and providing a standardized methodology, you can save time and better ensure adherence to internal standards and regulatory compliance.
How you build your workflow management tools depends on the other tools that you wish to integrate and your unique business needs. Many internal app platforms can integrate with third-party tools and provide the foundation for custom, automated workflows.

Ticketing and issue tracking systems

What is it? Ticketing and issue tracking software is widely used to manage long processes with multiple people and steps involved and to track customer service requests.
How does it benefit your business? With a ticketing system in place, it becomes easier to track open issues and reduce the time that customers or employees spend waiting to get their issues resolved.
Popular solutions ****include Zendesk and Help Scout.

Inventory management

What is it? Inventory management tools give an overview of stock items via tables, charts, and lists.
How does it benefit your business? Inventory management software helps businesses track their item stock and maintain the optimal amounts without holding too much of any item.
Inventory management software often comes as a feature of e-commerce platforms or as a custom solution that integrates data from separate e-commerce and in-person point-of-sales systems.

Data mapping

What is it? Data mapping tools capture, transform, and join data from multiple sources and sanitize it to maintain consistent formatting. They are often used to bridge data between two different systems and provide analytics and summary information.
How does it benefit your business? Data mapping is utilized to combine different data sources and develop automated data pipelines to make sense of dispersed data.
Data mapping tools are another category that is usually based on custom implementations. This is usually done using automated scripts and data pipeline tools managed by technical teams within organizations. By providing a user interface to these technical tools, non-technical users can monitor and manage these processes, freeing up your technical teams for other, less laborious tasks.

Data entry, create-read-update-delete (CRUD) applications, and database UIs

What is it? Data entry tools allow users to collect and manage different documents, forms, and data. Data entry interfaces usually include text inputs, file uploads, and features to capture media.
How does it benefit your business? Data entry tends to be a time-consuming and error-prone manual process. By standardizing this with a purpose-specific interface that validates data as it is entered, you can reduce mistakes and speed up the process by providing a consistent interface.
Most data entry tools are bespoke to a given task or data source, with forms built to make the process as fast as possible — for example, by placing required input fields first and providing keyboard shortcuts to further increase input speed. Historically, desktop solutions such as Microsoft Access and Claris FileMaker have been popular for this task, while modern browser-based solutions can build and serve database UIs for any device.

Dashboards and data visualization

What is it? Dashboards help you make sense of your company's most essential data. Typically, dashboards include graphs, charts, and button groups that are used to perform basic database operations.
How does it benefit your business? Dashboards empower you to find solutions to your most pressing business concerns by combining data from multiple sources and presenting it in a customizable, visual format.
Popular solutions ****for building data dashboards include Tableau and Microsoft Power BI.

Admin panels

What is it? Admin panels let users perform quick actions. For example, a panel could allow users to manage transactions and communicate with customers through a single interface.
How does it benefit your business? Like data entry tools, admin panels speed up common tasks and provide a streamlined means of accomplishing multi-step workflows that is less error-prone.
Admin panels are usually developed in house, providing a way for users to interact with different internal tools through a streamlined interface. These interfaces can also trigger complex scripted tasks, providing time-saving shortcuts.

All of the off-the-shelf tools mentioned above “just work” but are rigid in their implementation. They are not multipurpose and come with their own individual costs. There's also the question of where the data you place in the hands of third-party vendors is stored and how carefully it is handled. These are the primary concerns that drive many organizations to build their own tools that can handle all of their internal processes in one place and be hosted on their own secure infrastructure.

Should you build or buy internal tools?

That isn't to say that building your own tools is the best choice for everyone. Some businesses do not have, or do not wish to spend, the resources to build and maintain these tools. Other businesses only need a few off-the-shelf tools as their processes are quite generic (for example, a coffee shop probably just needs some basic point-of-sales and accounting software). Some businesses opt for a hybrid approach, choosing off-the-shelf tools that can later be extended with custom functionality.

Just as the choice between off-the-shelf solutions will be dictated by your requirements, so is the decision to build your own tools. It's important to identify a path of adopting or building tools that optimizes everyone’s time, especially engineering time.

You need to think about whether the available off-the shelf solutions will accommodate:

Your data: You should not have to discard useful information to make it “fit” in a tool. If you build your own tools, they will always be able to accommodate the information you require.
Your internal processes: If you are starting out, you can probably adapt some processes to your new tools so long as it doesn't impact their efficiency. If you have well-established and already optimized workflows, you do not want to have to discard years of ”muscle memory” — the ability for your business to reflexively react quickly and accurately to an incoming order, customer request, or other scenario — by adopting tools that do not work with them.
Your organizational structure: You don't want to have to grant your interns the keys to the kingdom to perform low-level tasks. Tools must let you control who has access to what and when.
You, your customers, and regulatory security concerns: There is ever-increasing regulation about what data you can store and where it can be stored. For example, businesses with customers in the European Union may face restrictions about where data about their customers can be physically stored. If you are using a SaaS product, you need to know where its developers host their data and what they plan on doing with it once you've given it to them.
Your future plans: A tool might seem appropriate (and affordable) now, but will it still be suitable once your business has grown? Choosing a free commerce platform that doesn't perform stock control may seem pragmatic when you're operating a market stall, but what about when you grow beyond that and need to manage hundreds of incoming orders? Changing that system later could be disruptive — and costly.

The build versus buy question is an important topic to explore for any business looking to establish or update their internal tools and infrastructure — so much so that we've dedicated a whole space to this discussion.

It's always wise to see what others in your market space are doing (particularly those that are succeeding!). Businesses with a lot of moving parts will need more, tightly integrated tools, rather than broadly targeted generic tools. This is usually best served by bespoke tools that can grow with an organization's teams and customers.

If you're a fan of brevity, these success stories and our in-depth discussion can be summarized as: buy tends to optimize for the short term, while build optimizes for the long term.

What should you consider before you start building an internal tool?

Your primary consideration in regard to your internal tools should be the people using them. Whether comparing off-the-shelf products or designing your own, the goal of internal tooling is to increase efficiency and reduce drag while solidifying your processes around these improvements. Paramount to this is that your users find the tools easy to use and appropriate to their given task. The principles are the same as building a customer-facing product — if it doesn’t solve their problems, it’s unlikely to be adopted.

You should also look at the larger picture of what you are building. When you are in control of the development roadmap of your own tools, something that may not be initially feasible for cost or time reasons can be added later.

Consider also the lifetime of the project — the person working on it now may not be the same person working on it in five years' time, so building with a consistent, adaptable, and easily adoptable toolchain is important. Identify any third-party platforms and libraries that can be leveraged to reduce the amount of work required, and research the components you plan to use and make sure they are scalable and support the functionality that you require (or will require down the track).

To avoid wasting development time, you should clearly define the problem you are solving for your users and the roadblocks you are trying to remove for them. You should then simulate the workflow — sketch out how you want it to work, and consult your stakeholders to make sure that they are satisfied that this design will be an improvement over the way they currently work.

Put simply, make sure you're building the right thing. You don't want to invest in tools that you later find aren't right for the job.

Rapid development using internal tools platforms solves the biggest challenges of "build"

Building with an internal app platform is a great compromise between buying and building from scratch.

Appsmith is an app platform that you can build your internal business tools on. We're build, not buy — but with a few differences.

We take the toil out of build by providing the following:

A consistent set of tools: Keep your tech teams in sync and working with the same tools so that there is less conflict and faster development. Appsmith is based on industry-standard JavaScript and React, so it's easy to learn and can integrate with many popular libraries.
Proven data back ends with custom front ends: Modern businesses are data-driven. You should not have to risk moving your data into a third-party provider's infrastructure. When you build with Appsmith, your data stays on your infrastructure, so you can ensure that it is handled in a secure and compliant manner.
Battle-tested, user-friendly design patterns: Most applications follow a similar set of design patterns for a reason — they're well proven and they work. Appsmith provides pre-built templates and a responsive canvas that can read the developer’s intent to assist them with quickly creating applications that look good and behave well — without requiring pixel pushing and a user interface design degree.
“What you see is what you get” components: Usually it can take hours or days to build a prototype that actually looks like something. This takes minutes in Appsmith due to our pre-built components, and what you see on your development canvas is what your users will see on their devices.
A developer-first environment: Experiment with new ideas safely with Appsmith's Git functionality by isolating your development and testing environments. Lean on pre-built components and libraries, or code away and implement your own functionality.

We also avoid the vendor lock-in concerns of buying by allowing you to:

Bring your own data: Appsmith lets you build on top of your existing data, so no information has to be culled to fit with the design decisions of off-the-shelf tools.
Build it your way: Appsmith provides proven UI elements and data tools to build with — it adapts to what your business needs rather than forcing you to change how you do things. Build your own custom interfaces that grant users access only to the data they need to do their jobs, reducing mistakes.
Host Appsmith anywhere: You can use our cloud hosted version or deploy to your own infrastructure with our easy-to-deploy Docker image.
Suggest features and help to guide Appsmith’s future: Appsmith's community keeps us at the forefront of internal tools with feature requests for the latest third-party integrations and robust community support.
Predict pricing based on your usage: We saw a problem with how other platforms were priced, so we decided to fix it with capped, usage-based pricing so that our customers always know what they're spending, and what they're getting.
Solve issues faster: When you’re in control of your data and your tools you can correct data issues yourself without having to wait on external support teams. If you need extra support or development questions answered, Appsmith is always available to help.
Adapt for the future: Want to update your processes after discovering a better way of doing things? Appsmith lets you update your apps and test them before deploying them to production. You're never stuck doing something the "wrong" way.

Appsmith can be used to build all of the common (and uncommon) internal tools you need. CRM, CMS, and customer success tools can be built with forms that include the specialized information that your business considers valuable. Workflow, support ticketing, and inventory management processes can all be optimized around tools that allow employees to respond quickly to developing situations. Different data can be mapped by consolidating data sources, then visualized in dashboards and admin panels so that decision makers always have up-to-date information.

You can evaluate Appsmith’s internal tool builder for free and scale up to Enterprise when you're ready

There is a lot to think about when deciding on the tools you will build your business on. For some businesses who have generic internal processes, the choice is clear: buy pre-built tools that match those processes. For other businesses, it's less simple — there are more questions to answer, and the answers will largely depend on whether the business wishes to fit the mold provided by third-party vendors or build to optimize and innovate their workflows.

Whichever options you are considering, you should evaluate them. It's easy to evaluate an off-the-shelf solution, but historically it's been difficult to evaluate the “build” option due to the time it takes to spin up a prototype and test its effectiveness. That's why Appsmith provides a cloud-hosted, cost-free development platform that can be up and running in minutes. From there, you can connect to your data, experiment with building your own internal tools with our drag-and-drop interface, and often have something ready to test within hours.

Once you've begun building and using your own internal tools, you can further reduce the load on your tech teams by upgrading to Appsmith's Enterprise offering. We'll provide support and advice on how to best develop your apps, as well as images and code updates for your self-hosted platform — leaving you with bespoke, reliable internal applications that will scale with your business.

Building Responsive UI Tools: Everyone Keeps Making the Same Mistakes

Appsmith — Mon, 23 Oct 2023 10:30:55 +0000

Design decisions have direct effects on how users interact with your application. We’ve found that this was especially true when creating Appsmith’s canvas, which is used to build the user interfaces (UIs) for applications.

UI requirements can define many aspects of an application, from its architecture to its dependencies, so these decisions are especially important at the earliest phases of development.

In this three-part series, I discuss the design decisions behind our “intent-driven auto layout” functionality, how it fits into our vision of what a truly great UI looks like, and where we are taking it in the future.

To start, let’s look at what we at Appsmith (and most others) have been doing wrong when designing UI-building interfaces for our users.

The critical mistake that app platforms make — giving the user too much freedom

When building UIs for internal tools platforms, most platform development teams make one critical mistake: they believe that (like themselves) users want to tinker with their UIs, so they provide a multitude of knobs to control exactly how the UI looks, often down to positioning elements at the pixel level.

What users actually want is a fast way to build attractive, functional and intuitive user interfaces for their apps. Including as much granular control as possible over UI elements adds layers of complexity, which drastically slows down the creation of the UI and often makes the final product difficult to use.

We’ve seen similar patterns show up in text editors as well. Earlier tools like Microsoft Word provided a lot of features to be “creative,” whereas newer tools like Notion have very few text options, rich formatting tools, etc. But this is actually good because it allows writers to focus on the text instead.

Allowing too much freedom when building can result in products that are non-functional or very difficult to use for non-professionals or non-experts. Adding intelligent guardrails can solve a lot of problems.

Even if users explicitly ask for more customization and control, we’ve found that it’s better to understand what they want to achieve with their design and better accommodate these requirements in our design tools. This is generally a much safer, more scalable option than adding more bells and whistles. Sometimes good design isn’t about giving the user what they say they want, but uncovering their true need, which often takes a lot more consultation, planning, and work up front.

Appsmith was guilty of this too — how can freedom not be a good thing?

Despite this, we at Appsmith did initially adopt the view of giving users more knobs and more complexity so that they could control every aspect of their UI. And, because of this, we allowed users to do things with their UIs that were ultimately harmful to their application.

For example, early versions of Appsmith didn’t enforce any sizing constraints on widgets. Users could squash labels and buttons to make them completely unusable or expand them to the size of the entire canvas. These are things that should never be done if the goal is to create a functional UI.

Initially, we didn’t think this was a problem that needed solving. Our thought process was that developers would never intentionally build a bad UI, so there was no reason for us to spend time adding guardrails. While this was true for some, the reality is that not all developers are interested in UI design.

Our users also had to constantly make small design decisions that, when added up, detracted from the usability of their applications. These weren’t a problem… until they were. Once their apps reached a certain level of complexity, or they suddenly had to be supported on a different screen size, our users would have to spend a lot of time fixing things.

We tried to optimize for a low barrier to entry when creating UIs rather than scalability and maintenance down the line. We tried to implement fewer rules so that users could do anything, rather than adding more rules to encourage good design patterns. And as issues started to pile up, we realized this was a mistake.

Like other major players in the internal-apps space, we thought just giving users the ability to define different layouts for desktop and mobile devices would be enough to solve this problem. But the problem here was that users had to spend a massive amount of time managing two separate UIs for one application. And this wasn’t a good use of their time.

As we gained experience and learned from the developers using Appsmith to create the UIs for their applications, we started to realize the drawbacks to these early design decisions. We saw that “possibilities are endless” often means shifting responsibility to end users who have to “figure it out.” But the truth is that users are often too busy to worry about smaller decisions like this.

Too much freedom when building UIs in our app studio was negatively impacting the quality of the applications being developed by our users, causing frustration and wasting their time. It was causing UIs that were ugly by default, when it should have been doing the opposite.

Users need a streamlined UI-building experience

Internal app developers need to be able to produce reliable, functional, and easy-to-use applications, such as CRUD interfaces and data dashboards, quickly. Users adopt an internal tools platform because they want to reduce busy work — if they wanted to customize everything in their app, they would use an IDE and build their app from scratch or use a library like React. Once we realized the significance of this, we started to see more customization for the UI as a liability.

It is also clear that Appsmith users expect a generic app to be mobile-friendly by default. Historically with app-building tools, users have had to babysit two (or more) different layouts for the same app. Doing something as innocuous as adding a button often created a bunch of problems on the mobile version as the entire layout sometimes had to be reconfigured to make space for another widget.

It was clear that our initial design wasn’t able to support the experience that developers expected. It relied on absolute positioning, meaning that users had to drag widgets onto the screen and make manual adjustments to get things to look the way they wanted. While this UI-building interface was intuitive and easy to use, building complex UIs with it was laborious and error-prone.

As we explored the existing solutions in the internal tools space, we found that many others solved this problem by exposing the complexity of CSS flexboxes to their end users. When building their apps, users would then have to consider the devices they wanted to target, the positioning properties of the UI elements — including horizontal and vertical stack, horizontal alignment, sizing, etc. — and then configure and preview them.

We didn’t want to take this road for Appsmith, and decided that we would find a better solution for our users. We would allow them to use a drag-and-drop interface to quickly build and preview their own user interfaces that would automatically work on any screen size.

Users have to consider many properties when adding basic widgets and constantly adjust them without the benefit of intuitive dragging and dropping.

How we at Appsmith decided to solve this problem

Previously, our tool encouraged poor UI design because that was much easier to do than good design. The reason was that every new knob we added was one more thing that could go wrong and one more thing that developers had to worry about.

After many months of research, consultation with our users, and leveraging our experience in the field, we had drafted and validated a viable plan to implement a smart drag-and-drop interface: one that could place UI elements and widgets where the user wants them to appear, rather than at a specific pixel coordinate.

We wanted to adopt the auto-layout functionality already in the space, but combine it with an intuitive drag-and-drop interface that users were accustomed to. That way, they didn’t need to think about the complexity of CSS flexboxes when designing their UI.

We’ve labeled this approach intent-driven design. Instead of giving users more knobs to customize every little intricacy of their UI, we decided to take the knobs away entirely. We wanted users to specify what they wanted through an intuitive drag-and-drop interface and use algorithms and heuristics behind the scenes to offload the routine work of positioning everything appropriately on the canvas.

Internal app developers are generally most concerned with their back-end queries and logic – they want their application to interface smoothly with their data by allowing for fast data retrieval and accurate data entry. The benefit of Appsmith for them has always been that complex tech stacks can be quickly spun up without even having to think about the massive complexity at each layer. So now we’re just extending this principle to UI design.

Why should users be forced to learn UI design to build a good app?

User interface design is a field of its own. Professional designers study for years to be able to build user interfaces that are visually appealing and accessible to broad audiences. We don’t think that internal app developers should need to have this experience to build their own great user interfaces — so we’re using our own to automate the process.

We’re already seeing results from our continued work on our UI-building interface: Appsmith users don’t have to spend hours pushing pixels around and babysitting multiple layouts. They are able to create apps quickly with good-looking UIs that just work — even across different screen sizes.

UI responsiveness has been an interesting problem to tackle, and one that we hope will set the standard for the entire internal tools space. I've only touched on the work that went into this. In our upcoming article, we will delve deeper into how we pivoted our UI-building functionality and how you can avoid our mistakes by prioritizing your design choices upfront.

Negating Build versus Buy Trade-Offs When Choosing Internal Tools

Appsmith — Thu, 05 Oct 2023 09:08:55 +0000

Businesses rely on internal tools like CRMs, CMSs, and other data entry and retrieval tools to perform their daily tasks quickly and efficiently. Disrupting their workflows unnecessarily could have catastrophic consequences for an operating business, so any changes to and investment in them must be justified to stakeholders. The decision to adopt a new tool must factor in its longevity, flexibility, and ability to reliably handle your use cases.

The prevailing view today is that off-the-shelf SaaS products are almost always the best solution for internal tools as in-house development is simply too costly in terms of time and resources. However, this build versus buy question is becoming increasingly irrelevant as a new class of tools is popping up to sidestep many of these trade-offs.

These new tools – internal app platforms – allow teams to get the best of both worlds by building their own customized solutions without having to bear the full cost of in-house development.

The need for quality internal tools

Many industry- and business-specific processes are not optimized for efficiency. These inefficiencies could come up when employees manually update databases, rely on information in spreadsheets, or rely on information spread across multiple systems.

For example, suppose a customer support rep needs to solve a customer issue by working across multiple systems: a ticketing system, an internal database of customer transactions, and a CRM filled with customer data. The rep might spend a lot of time digging through these different sources of data and entering (sometimes duplicate) information into them. The time when support reps are trying to work with these clunky tools and processes is time when they aren’t serving customers.

All of these inefficiencies are pain points that need a solution. Quality internal tools can act as that solution by providing simple, streamlined interfaces to enter data with error-checking and intuitive dashboards to consolidate data from multiple sources into one source of truth. Your teams can use these tools to partially or entirely automate certain processes and get more done in less time with fewer mistakes and headaches.

Should you build or buy your internal tools?

The question of whether to build or buy is prevalent for the software tools involved in any business process. Most, if not all, businesses have bespoke internal processes that have evolved over years of operations to deliver their required results in the best way possible. These processes are often a competitive differentiator for the business and have been adapted to deal with regulatory, regional, physical, or other factors specific to a given use case.

Off-the-shelf tools with rigid features may require changing these processes, which is costly. Conversely, fully bespoke solutions require additional resources to develop, test, and maintain. It’s worth sitting down and thinking about which of these trade-offs are worth it.

The advantages of buying off-the-shelf software to manage your internal processes

This is essentially outsourcing the building and maintenance of your internal tools. The main reason to outsource anything is to avoid paying the opportunity cost of doing it yourself. With this strategy, you don’t have to dedicate knowledge, time, energy, and money toward becoming an expert on solving a problem.

Instead, you can just trust another provider who has prior experience in handling the problem, freeing up your resources to be used elsewhere. You trust that they have the necessary expertise to build a solution that will provide an optimal (or near-optimal) process based on industry best practices.

Buying is particularly beneficial when you’re trying to solve a well-defined problem that will be the same across most companies and there’s no advantage to differentiating your company by doing things differently than everyone else. Examples include using Slack for communication or Shopify to manage an e-commerce business.

When an off-the-shelf tool does things exactly the same way that you do, buying that tool is entirely appropriate — and recommended. It’s when your process diverges from the norm that you can run into issues.

The difficulties of buying off-the-shelf software to run your business

The biggest issues with buying off the shelf are that the tools may not be flexible or affordable enough.

Flexibility

Commercial tools must cater to the widest audience possible, meaning that they won’t always be a good fit for your use case. This pitfall is often encountered when a tool seems like a good fit but lacks one key piece of functionality that is either missing initially or emerges as a requirement later.

For example, a business selling digital goods may choose an online sales tool that does not support stock control. Later, it starts selling physical items as well, but because its chosen platform caters only to digital sellers, the platform has no interest in adding this particular feature. Now, the business must manually manage stock and disable products when they run out rather than it happening automatically. Alternatively, it must start all over again with a different provider.

It’s generally difficult to extend off-the-shelf solutions without hacks and manual intervention, which then require maintenance or tweaking to work with future updates, defeating the entire purpose of buying a ready-made solution. The only clean ways to get around these issues are to either align your processes exactly to these available tools or buy tools with well-defined APIs that can be built on or smoothly integrated with other tools, as discussed later.

Lock-in

Another downside of buying tools is the vendor lock-in. When you outsource critical business processes and data to other companies, you must trust that they won't change the terms of the agreement in the future. Plans may start out affordable, but they aren’t guaranteed to stay that way. If you want to change providers down the line, you might face a tough decision, especially if you are heavily dependent on a particular tool for your data and workflows.

Additionally, every new tool that you use will have its own back end, which means another data silo. Each additional data silo can create process gaps or inefficiencies if they don’t expose their data through well-defined APIs that can be easily consolidated into a single source of truth. Each additional process gap can drive a need for yet more tools to address it. While many individual SaaS products are affordable, this overall cost can get out of hand as teams add more and more tool subscriptions across the company. SaaS bloat can be expensive.

The advantages of building your own internal business tools

Longevity and flexibility are the biggest advantages of building your own internal tools. Owning your infrastructure allows you to avoid the hazards of rigid tools and unpredictable future costs.

You don’t need to manipulate your processes or data to make them compatible with off-the-shelf software or factor in third-party dependencies to be performant. You can instead prioritize what you need — not what your provider thinks will make them the most money.

You also can use APIs so that your internal tools have a single source of truth by sharing the same back-end datasources. This can dramatically reduce the data siloing that is all too common in most organizations.

Practically speaking, it makes sense to buy the tools that are already an exact fit for your business and for processes that are unlikely to change. On the other hand, there is growing weight behind the argument that you should build the more unique and dynamic internal tools your company relies on to stay efficient and competitive. However, there are challenges to doing this.

The difficulties of building from scratch

There are two different ways to build your internal tools: using a platform with prebuilt foundations and components or building everything from scratch.

When building your own internal tools from scratch, you do have to pay the opportunity cost incurred by not buying. You have to assign resources — time, money, and energy — into building your tools instead of focusing on the job your customers hired you to do.

This investment is easy to underestimate. Therefore, it’s important to know exactly what ongoing development, deployment, and maintenance tasks you’re committing to when building your own apps from scratch:

Create a development environment your team can access
Install your framework (React, Django, etc.), making sure to avoid running into dependency and environment issues
Start building a UI from scratch, including code for boxes, labels, navigation bars, etc.
Write logic for access control and authorization from scratch or by leveraging third-party libraries
Configure your back-end data sources from scratch for each tool separately
Spend a minimum of three months building and refining the tool (we’ve heard this so much, even when leaning on third-party libraries)
Configure your production servers
Build CI/CD pipelines to test and deploy code to production
Release the tool to users within your company
Gather feedback and handle feature requests, and then loop through the later stages of this process again for every feature request

This is a resource-intensive process that comes with the additional risk of building a subpar user interface that requires constant ongoing maintenance, diverting resources from more profitable endeavors.

Tools built from scratch may also face neglect due to the fact that there are usually bigger strategic priorities and customer-facing issues to address. Without proper maintenance, a tool’s usefulness can be compromised over time, wasting most of the upfront work.

Internal app platforms bridge the gap between build and buy

To address the problems inherent with buying off the shelf or building your own software, a new category of developer-first low-code software platforms has emerged. These app platforms solve the problems of buying off-the-shelf software because you can build the exact tools you need and don’t have to pay the (sometimes exorbitant) costs to maintain them. You are still building your own tools, and you therefore own them.

Alongside this, they solve the challenges of building your own software from scratch by drastically minimizing the time and energy required to build your own tools. These platforms automate many of the repetitive steps involved with developing a new application, allowing you to focus on your core business logic. You are still building, just not from scratch.

Building on a flexible app platform also allows you to take a hybrid approach where you build certain tools and buy others. As long as the tools you buy have well-defined APIs (through REST or GraphQL, for example), you can easily integrate them into your bespoke applications.

An example of this hybrid approach is integrating Zendesk for customer support, internal databases for holding customer data, and Jira for issue tracking. Support representatives could receive customer inquiries (logged via Zendesk), look up relevant information, implement minor service requests like adjusting account settings, and log issues in Jira for things that need engineering attention — all within a single, custom interface.

This is an example of a high-value, custom internal tool, which would usually only be provided by a costly third-party platform or require an extended period of development if built from scratch. Because situations like this happen fairly regularly in any company, it is worth streamlining and automating them with internal app platforms.

App platforms also facilitate rapid iteration and improvement of your internal processes. You can quickly demonstrate prototypes, gather feedback, iterate, ship, and deploy updates to your applications. This is all because they’re just more efficient on the back end. Standardizing your internal tools on a single platform will ensure your teams can work smoothly together while also cutting down SaaS bloat across the company.

Building with an internal app platform is a great compromise between buying and building from scratch as long as you trust the provider of your internal app platform.

Appsmith solves the biggest pain points of building from scratch

While internal app platforms solve most of the problems of building from scratch, they do not solve all of them and sometimes come with their own baggage — the biggest being poor transparency and high costs.

Appsmith — an open-source internal app platform

At Appsmith, we’re addressing these problems by open-sourcing the core of our codebase. We have built a future-proof, reliable, open-source platform that can scale with your business. Since we open-sourced our platform, a thriving community has grown around Appsmith, further contributing to the project and providing support to our users.

With our Git functionality, you can export and import your applications to and from GitHub, Bitbucket, or any other hosting provider. In other words, you truly own your code and could spin your applications up without us at any time, so vendor lock-in is impossible. On top of this, we’ve promised to always implement fair and transparent usage-based pricing for our Business and Enterprise versions.

We’ve prioritized building a number of quality-of-life features that developers appreciate, including:

Making every aspect of debugging as efficient and streamlined as possible
Integrating templates to rapidly prototype (with more always under development)
Implementing and improving intent-driven auto-layouts to quickly build UIs
Allowing you to run custom scripts with a button click using our cURL connector
Letting you embed your Appsmith apps wherever you want through iframes

We also address the maintenance and deployment concerns associated with building your own tools. Our free cloud-hosted version of Appsmith is always up to date on reliable, managed infrastructure. If you want to host on your own infrastructure, you can easily deploy our Docker image. We offer clustering for Kubernetes out of the box and support auto scaling across all major cloud services, like AWS, Azure, and Google Cloud Platform.

Many commercial application platforms roll out updates and force you to update your apps to remain compatible. If you’re self-hosting Appsmith, we stay hands off, so you can run any version for as long as you want. By using our Git functionality, you can develop and test your apps on a newer version of Appsmith in a different branch and only update your production instances when you’re ready.

Appsmith drives business results

Companies have highly effective internal tools with Appsmith to streamline and improve their operations. The team at HIFI was able to build a custom CRM to process customer applications 50% faster, saving hours of manual work per week. Instawork developed a 360-degree customer dashboard that combined data from internal and external sources, allowing them to resolve support issues 37% faster. You can see other companies we’ve worked with on our case studies page.

Our open-source community and internal team is always available on our Discord channel. However, if you require support for your mission-critical internal tools, you can chat with a team member today to see if our Enterprise plan is right for your own use cases. And if you want to start building on Appsmith for free, you could try our free cloud-hosted community version.

October Roundup: Widget Improvements, New Appsmith functions, and ARM Architecture Support

Appsmith — Tue, 02 Nov 2021 07:49:30 +0000

We’re back again this month with updates from the last 30 days. We like to work hard! We’ve shipped many features, fixed bugs, and launched Hacktoberfest with a series of fun events. You can check them out here.

Widget, UI, and UX Improvements

#1 Menu Buttons on Table Widget

Tables on Appsmith are one of the most loved widgets, and yes, we’ve upgraded them again! We can use menu buttons inside the table widget on any particular column to create menus and customize them. Open the column settings from the table’s property pane and set the column type to the menu button to see them in action! And just like that, we should be able to see a menu button on our table. Additionally, you can add more items to this by configuring the Menu Items property. That’s not all, and you can further customize the items to have icons, border radius, box shadows, and more!

#2 A New Sparkling White UI

Notice anything new? Yes, we have improved our whole design system to help developers focus more on their application editing on Appsmith. Right from searching through apps on the dashboard to building them using queries and widgets, everything is white!

#3 Option to Hide/Unhide Properties from Property Pane‍

At Appsmith, we consistently focus on adding new properties and features to widgets, and sometimes it can get a bit overwhelming to see all of these at the same time! To make it less cumbersome, we’ve added an option to hide and unhide a few properties. Developers can concentrate on what’s important. To use this feature, toggle the dropdown arrow inside the property pane. You will see an option to collapse the various options within the pane. For example, in the picture below, only the header option has been made visible.

New Appsmith Functions!

You can bid goodbye to refresh buttons! Developers had to use refresh buttons on Appsmith Appsmith to be able tore-run the queries. We recognized this pain point. Now there’s a cool new feature to periodically run APIs and DB queries. You can configure these by using the setInterval and clearInterval functions!

The setInterval function executes a trigger callback at a given interval. Here’s how you can use it while dynamically binding queries onto widgets:

setInterval(callbackFunction: Function, interval: number, id?: string)

// setInterval(() => { Query1.run() }, 10000, "myTimer");

The clearInterval function stops executing the trigger callback started with the setInterval method.

clearInterval(id: string)

// clearInterval("myTimer");

Appsmith Deployments

1 Helm Chart Support

Helm charts are now added for Appsmith deployments on Kubernetes deployments. These include application deployments with basic default support. Additionally, developers can add custom configuration for persistent volume data, custom ingress controller and secure with TLS certificate

2 Support for Deploying Appsmith on ARM Architecture

Previously, many developers had reported issues regarding docker-based installation of Appsmith on ARM architectures. Now, we had rectified them and had added extensive support for M1 MacBook users. Following are the changes that we made to the scripts for docker installation:

We heavily refactored Dockerfile to build the image that is able to support both AMD64 and ARM64 architecture
We have updated docker.env.sh with additional environment variables and renamed the existing MongoDB credential variables
We added a document to build an image for multiple architectures using Docker buildx

Quit Buggin! Cheers to our Updated Debugging Experience

We have improved the debugging experience by adding hint and errors messages for different cases when working on queries and binding them onto widgets. To make these messages more efficient, we added a new attribute named readableError to the ActionExecutionResult object, returned to the client post after actions are executed. With this, we should be able to understand and solve complex errors quickly. Not just that, we have seen many APIs failing because of using duplicate headers or query parameters, so we have added a few hint messages to help you debug faster.

We’ve got a host of other bug fixes and updates too, be sure to check out our release notes here.

See you next month with more updates! Do follow us on Twitter, Youtube, and Linkedin to stay up to date.

Our New JS Snippets Feature Helps You Write JavaScript in the Appsmith Platform

Appsmith — Mon, 18 Oct 2021 15:52:14 +0000

Building an app on Appsmith is great fun. Why? Because it takes minutes to build a clean and consistent UI, and less than a few hours to make data connections. We might be tooting our own horn, but what we love about Appsmith (ask our in house devs!) is that we can use JavaScript anywhere on the platform to add additional transformations, functionalities, and actions using the moustache syntax. We’re aware that not everyone is a JavaScript enthusiast, everyone prefers a language of their choice! So we’ve come up with our new feature: “JS Snippets.” With this, you can get some help with writing JavaScript inside Appsmith. In this blog, I will tell you how we made this feature and how you can use it!

The Story!

We often see many questions in our community forums and discord channels seeking help with binding nested APIs, write data transformations, configure chained actions. Hence, we thought of building a library of snippets inside Appsmith applications that you can use directly and customize based on your data source.

Initially, we thought of the issue in a straightforward way, but at Appsmith, we always like to go the extra mile. So, we brainstormed this idea multiple times and decided to extend it to customize and add snippets particular to widgets and data sources. But, in order to do this kind of customization, we need a database application that validates and filters these snippets. So we built an internal Appsmith app that manages all the JS Snippets. This is how the application looks like:

Here, as we can see, on the left, we see a list of all the available snippets today. When these are clicked, we will see the details of the snippet and have options to verify it and publish it live directly to the JS Snippets library.

To customise these snippets, we define three formats for each of them.

Snippet: The example code snippet written in JS
Args: Customisable variables inside the snippets and their expected data type,
Template: The dynamic template that helps devs customise their snippet with any variable on Appsmith

Here’s an example of how we ideated customizable JS snippet using these three forms for merging two queries in a table:

Snippet:

{{ 
fetchOrders.data.map((orderObj) => {
   const user = fetchUsers.data.find((userObj) => userObj.id == orderObj.refId) || {};
   return { ...orderObj, ...user };
})
}}

Args:

[
  {
    "identifier": "fetchOrders",
    "name": "Query1",
    "type": "OBJECT"
  },
  {
    "identifier": "fetchUsers",
    "name": "Query2",
    "type": "OBJECT"
  },
  {
    "identifier": "orderObj",
    "name": "row1",
    "placeholder": true,
    "type": "OBJECT"
  },
  {
    "identifier": "userObj",
    "name": "row2",
    "placeholder": true,
    "type": "OBJECT"
  },
  {
    "identifier": "id",
    "name": "key1",
    "type": "VAR"
  },
  {
    "identifier": "refId",
    "name": "key2",
    "type": "VAR"
  },
  {
    "identifier": "user",
    "name": "recordForMerge",
    "placeholder": true,
    "type": "VAR"
  }
]

Here’s a screenshot of how the snippet is rendered on Appsmith:

Users can now directly copy this snippet onto the widget, or customize the arguments, test it out, and use it in the widget.

How to use JS Snippets on Appsmith

We can find JS snippets inside the Appsmith applications from Omnibar. Click on the Search/Omnibar from the top navigation and choose the Use snippets option. Here, you can filter snippets based on data sources and UI widgets.

The other easy way is to access snippets from the slash commands. Just type in / inside the bindings, and choose the insert snippet option.

Special credits to Arun Vijayan, Sumit, Ajinkya, and Nikhil Nandagopal who had worked so hard on this feature!

Hope you’ll build some super cool applications using the JS Snippets on Appsmith! If you do, don’t forget to tell us about it, we would love to feature your work! Your contributions and feedback help us make Appsmith better and we really appreciate it.

Join the community! Come chat with us on Discord, or jump in on Github directly!

You can also follow us on Twitter and Linkedin.

Appsmith Roundup: Powerful Widget Grouping Options, Undo-Redo, and New Widgets

Appsmith — Wed, 06 Oct 2021 14:30:40 +0000

Powerful Widget Options

From grouping widgets to copy-pasting them to undoing or redoing them on the canvas, we’ve got everything covered for you to customize your applications on Appsmith. This makes your app-building experience much more accessible, smoother, and of course, fun!

To use the widget grouping feature, you have to select all required widgets on the canvas with your pointer. This will create a widget group highlighted in a rectangular dotted box; next, you can drag the rectangular box anywhere across the canvas to move them all in one go. Additionally, widget groups can also be duplicated and copied, or deleted. You can find all these options on the top-right corner of widget groups.
‍

There’s no such thing as error-free creation; it’s common to make mistakes while building apps. Often, we accidentally delete widgets while building the UI and have to redo everything. With the new undo-redo feature, you can make changes in the canvas using simple keyboard shortcuts.

Undo - Command / Control + Z

Redo - Command / Control + Shift + Z

We also resolved some critical issues for copy-pasting widgets. Now, the widget won’t overlap, and all the functionalities such as deleting them, customizing them would be working as expected!

New Widgets and Customisation Options

We got four more new widgets last month! A stat box widget to display all the metrics on your dashboards, a checkbox widget to configure multiple checkbox inputs, a beautiful audio recorder widget to record audio on your applications, and finally, a tree select widget for selecting options in a nested tree-like structure. Cool right? Here are the quick previews of these new widgets.

Additionally, we’ve upgraded all the widgets styling options to give developers more control over customization. Now, you can add background colour, shadows, borders, border-radius to almost all the widgets where there is a chance of customization. Of course, we’ve tried them. We have a new product designer who joined us recently who had built this beautiful dashboard on Appsmith.

Don’t forget to chat with us on our discord channel if you want to see more such internal tools and designs!

Configuring Page

We’ve added a new option to configure all your pages in the application in one place. Using this, you can rename, reorder, duplicate, and delete pages within few simple clicks. To access it, click on the application name and select the page option from the dropdown to configure pages under the application. Below is a screenshot that shows how easy it is to manage your application pages on Appsmith!

Introducing JS Snippets!

JavaScript is like fuel at Appsmith. We can use JS anywhere across the application, from binding data to writing functions to implement complex JS transformations using the moustache bindings. But not everyone is a JS expert; even we sometimes google to get stuff done. Hence to make it handier we’ve introduced a super cool feature, the JS Snippets. Some common use-cases can help you with fundamental transformations, functions, binding, and many more.

Here’s how you can access this:

Open the Quick search and navigation on the top menu bar.
Select use snippets
Search for Snippets ‍

Before you use them, you can also test them out with your connected data.

We’ve got a host of other bug fixes and updates too, be sure to check out our release notes here.

See you next month with more updates! Do follow us on Twitter, Youtube, and Linkedin to stay up to date.

Building an E-commerce Store using Appsmith and Postgres

Appsmith — Wed, 15 Sep 2021 21:10:48 +0000

Purchasing products online has become a common practice across the world. In fact, according to a recent survey, approximately 36% of U.S. consumers are now buying retail goods online. And this number is expected to grow! With the number of digital buyers rising, and the ease and convenience of online shopping, e-commerce portals are the obvious answer. Several big chain retailers are re-evaluating their presence online and continuing their digital transformation. Building an eCommerce store is an important part of this transformation. Fortunately, building an e-commerce platform is now as easy as 123, thanks to Appsmith.

In this blog, we will take you through the process of building the store.

Set up Postgres on Appsmith

The first step is to build an e-com store on Appsmith is to connect to a database to store the information regarding all the products and orders. For this, you can connect to the existing demo database that we’ve used to build this application. You could also use your own PostgresDB in case customization is required. Now, follow the steps below to connect the PostgresDB on Appsmith.

Create a new account on Appsmith (it’s free!), if you are an existing user, log in to your Appsmith account.
Create a new application by clicking on the Create New button under the Appsmith dashboard.
We’ll now see a new Appsmith app with an empty canvas and a sidebar with two directories: Datasources and Widgets.
Click on the + icon next to the data sources section and choose the PostgresDB integration.
Rename the data source to Store DB, and we can do this by clicking on the existing one and choosing the rename option.
Use the following details to connect with the PostgresDB (demo or your own) for this application.
After entering the credentials, you can test the data source by clicking on the Test button to check if the connection is valid or not.
Then, click on the Save button to save the data source permanently on your organisation if the connection is valid.

Host Address: db.ircvafghkrkovtjtnllv.supabase.co
Port: 6543
Database Name: postgres

Username: postgres
Password: appsmithdemo123#

Note: The connected data source has two tables with the following fields:

orders - [id int8, contact_no varchar, address text, total_items json, cart_value int8]
products - [id int8, product_name varchar, product_description text, product_image text, price float4, quantity int4]

If you’re using new PostgresDB instances, copy and run the following queries by creating a new query on the previously created Supabase Store DB data source.

Manage Products with a CRUD App

The first and foremost thing for any eCommerce application is to build a CRUD (CREATE, READ, UPDATE, DELETE) application to manage the store items. Usually, for making this, we need to spend a lot of time creating queries and building UIs. But at Appsmith, we have a cool feature where you can generate a CRUD application from your database using just one click. To learn more about this, read this blog here.

Now follow the below steps to build a CRUD app on the products table from the connected Postgres data source.

Click on the + icon next to the Datasource section from the entity.
Choose the Supabase Store DB data source and click on the Generate New Page button.
Here you’ll be asked to select a table from the data source, choose the products table, and set the searchable item to the product name.
Now, click the Generate Page button; this will automatically create all the UI and queries and build a completely working CRUD app. Yeah, I know, it’s pretty awesome!

Note that this will be created under a new page with a unique name, we can change the page name to Product Admin, and this is how it looks like:

Creating product: We can add new products by clicking on the New Row button. A modal pop up with a form to add details of the product.

Read products: All the products are listed on the table; you can order by any parameter listed on the table and customize it based on different use-cases.

Update product: To update a product, we can select the row from the table, update the Form placed on the right, and hit the save button.

Delete product: To delete the product, scroll horizontally on the table and use the delete button.

Awesome, that’s really easy. In the next section, let’s create a new page. We display all the products and add functionality to add products into a cart and checkout the order.

Create Store Page and Implement Cart using the Appsmith Context Object

Now that we have an admin page to manage all our products, let's add a new table widget to display all the products and implement the cart functionality. Follow the below steps:

Firstly, let's rename this page to Store Home Page, click on the + icon next to data sources, and create a new query from the Supabase Store DB.
Rename the new query to getAllProducts and paste the following SQL in the query body pane.

SELECT * FROM public." products";

This query is used to fetch all the products from the connected data source.
Next, click on the + icon next to Widgets and drag and drop a new table widget onto the canvas. You can click on the table name on the top right to open the property pane to configure table data and its columns.
On Appsmith, it's super easy to bind the queries, use the moustache bindings and query name; similarly, open the table's property pane and use the following snipper in the Table Data property: {{getAllProducts.data}}. And, just like that, we should see all the products populated on the table.
Here, the table data is only filled with the products, but we should be able to have one more column where users can add the selected products to the cart. For this, open the table property pane, and select the ADD A NEW COLUMN option. This will create a new column on the table, rename it to Add to Cart and set the column type to Button.
Open the column settings and set the label to: Add to Cart, now when clicked the item should be saved to a local store, to implement this logic, toggle JS on the onClick property and use the following code snippet.

{{
(function() {
    storeValue('cart', [...(appsmith.store.cart || []), Table1.selectedRow.id])
})()
}}

Here, we have a function, that stores the ID of the selected rows into a new array called cart when the button is clicked. To learn more about appsmith local store, read the documentation here.

Awesome! Now, let’s display all the products that are added by the users in the Appsmith local store on a list widget. Follow the below steps:

Drag and drop a list widget from the Widgets section, and open its property pane by clicking on the cog icon on the top right.
Paste the following code snippet in the Items property.

{{

(function() {
const groupedItems = _.groupBy((appsmith.store.cart || []).map(id => getAllProducts.data.find(product => product.id == id)), 'id');
const items = Object.keys(groupedItems).map(id => {
 return {...groupedItems[id][0], totalQuantity: groupedItems[id].length }
});
return items;
})()

}}

In the above code snippet, first we are grouping all the product items using the ID. Next, we’re mapping the grouped items to get the quantity of the items added, this will allow us to handle multiple items in a single cart. Following is the screenshot of the property pane of the list widget.

As we can see, in the evaluate value pane, we have the products in the cart, and also the quantity gets added to the totalQuantity variable directly from the function.

Next, we’ll add a text box where we calculate the total price in the cart and display it to the users, this drag and drop a new text box widget from the widget sections, and paste the following in the Value property by opening the property pane:

${{ (function() {

const groupedItems = _.groupBy((appsmith.store.cart || []).map(id => getAllProducts.data.find(product => product.id == id)), 'id');

const items = Object.keys(groupedItems).map(id => {
 return {...groupedItems[id][0], totalQuantity: groupedItems[id].length }
});

return items.reduce(function (previousValue, currentValue) {
  return previousValue + currentValue.price * currentValue.totalQuantity;
}, 0);
})() }}

Here, we’ve considered the same grouping function, but instead of counting the items, we’re simple calculating the total price by summing them up.

Checking Out the Cart

We now have the complete cart, the next step is to store all the orders in a new table, and show them on a Product Admin Page. Follow the below steps to implement this:

First, drag and drop a button and set the value to checkout, let’s open a new modal to gather all the information from the users. So we can set the onClick property to Open a Modal and create a new one.
Next, add three text widgets, and input widgets, for us to gather information of their order. Following is how the UI looks like:

Let’s name the input widgets to follow, to use them in the query while creating orders:

Name Widget: userName

Delivery Address Widget: userAddress

Contact Number: userContact

Next, we’ve added the cart price again for UX purposes so that the users can have an idea of their cart value when checking out. Lastly, when the confirm button is clicked, let’s create a new query submitOrder to post the query details.

INSERT INTO public."orders" ("name", "contact_no", "address", "total_items", "cart_value") VALUES ('{{userName.text}}', '{{userContact.text}}', '{{userAddress.text}}', '{{ (function() {

const groupedItems = _.groupBy((appsmith.store.cart || []).map(id => getAllProducts.data.find(product => product.id == id)), 'id');

const items = Object.keys(groupedItems).map(id => {
 return {...groupedItems[id][0], totalQuantity: groupedItems[id].length}
});

return items;
})() }}', {{userAmount.text}});

Here we're using the same details that we've collecting from the Modal, we copied the same function that was used for the cart to post cart items into the orders table.

Lastly, we've created one more page, to display all the items that are in the orders table. For this, we simply use a table widget and customise the data. Following is how the UI looks like:

Building this app from scratch, including writing snippets of code is likely to take 30 minutes! Isn’t that simple?

If you liked this tutorial, and are planning to build this, let me know. I’d love to help you make it as complex as you’d like. Write to me at vihar@appsmith.com