Forem: Katerina Skroumpelou

Monorepos - Why Speed Matters

Katerina Skroumpelou — Wed, 20 Mar 2024 20:52:22 +0000

In the ever-evolving landscape of software development, efficiency and speed are vital. As projects grow in complexity, developers and teams need tools that can keep up without sacrificing quality or performance.

Nx is a suite of powerful tools designed to optimize your development workflow, which sets the building blocks for a fast CI. Nx is always innovating in many ways to make developers’ lives easier, but this post is exclusively focused on the things Nx has done in the past year to make development faster and faster.

Why speed matters

The ability to iterate quickly and efficiently is vital for any software project. Speed in the development process offers several critical advantages:

Faster feedback loops: Quick iterations mean immediate feedback, allowing teams to adapt, learn, and improve their work on the fly.
Reduced time to market: Accelerating the development process can significantly cut down the overall time to market, providing a competitive edge which reclaims revenue that would have otherwise been lost.
Decreased developer frustration: No more waiting for builds and tests to complete. A streamlined workflow keeps morale high and productivity higher.

If you’re using Nx already, you’re already familiar with

Affected - identifying and running tasks only on projects impacted by code changes,
Nx Replay - our powerful cache and
Nx Agents - the concept of Parallelization and Distribution.

But let’s see all the extra things we did this past year to make everything faster.

Speed at the core

Rustifying Nx

The Nx daemon has seen significant enhancements, notably through the use of Rust to calculate file hashes behind the scenes. This improvement not only speeds up the start-up times but also optimizes performance even without the daemon, especially on CI environments where the daemon isn't used. The benchmark results at this repo showcase the remarkable speed improvements, making Nx competitive with native code solutions while maintaining the accessibility and flexibility of Node.js. Nx is still Node-first, so contributions are easier and only the most performance-critical parts of Nx are native code.

Task Hasher and archive file innovations

The introduction of a task hasher written in Rust, alongside the use of an archive file to store workspace file hashes (.nx/cache), has significantly reduced the need for repetitive file system accesses. This innovation means that running multiple Nx commands in CI is much faster, as file hashing becomes unnecessary after the initial run.

The Archive file

The archive file is a binary file that contains the workspace file hashes with their last modified time. Every time Nx starts (ie, running nx run project:target) it gets all the files with their last modified time, and compares it to the archive. If the file exists in the archive, then Nx does not access the file system to read the file to hash (reading individual files is slower than just getting a list of files from a directory). So running multiple nx commands in CI is quick to start because Nx does not need to constantly hash files.

Nx Replay

Nx Replay enables caching and reusing of task results. It’s our well known Nx remote cache! It allows developers to avoid re-running expensive tasks by retrieving the cached results from a remote cache. This significantly improves build and test performance, as well as developer productivity. Nx Replay is also critical to the functioning of Nx Agents, which rely on the remote cache to ensure that the results of a task will be shared with every agent that needs them. By using Nx Replay, developers can optimize their workflows and reduce the time spent waiting for tasks to complete.

With Nx Replay, you can see significant speed improvements in your CI pipelines for modified PRs. What’s also important is that if a task has been executed in CI, a developer running that same task locally can reuse the task result instead of actually running the task. So you will also see improvements locally.

Nx Agents

Nx Agents represent the pinnacle of task distribution optimization, ensuring that tasks are executed as efficiently as possible based on the specific requirements of each change. Some features that make up this effort are:

Easy integration with existing providers
- Distribution is handled on the Nx Cloud infrastructure and all you need is a single line. What’s more, all results are played back to your original CI provider script which triggers the Nx Cloud distribution, so that you can make use of the resulting artifacts
Efficient task distribution
- Save compute resources and reduce costs, minimizing idle time and compute waste
- Dynamic sizing based on PR size
Tusky - our AI solution - coming soon
- You set your desired cost/speed ratio, and you forget about any more configuration. We ensure maximum speed up to limits you set yourself.

You can read more about Nx Agents here.

TaskAtomizer

The TaskAtomizer splits your Cypress or Playwright e2e tests by file. This significantly enhances granularity for caching, parallel execution, and flaky test identification. This granular approach ensures that individual test results can be cached and only the necessary tests rerun, greatly reducing CI pipeline times and facilitating more accurate flaky test detection.

Addressing flaky tests with test deflaking

Flaky tests can be a significant bottleneck in the CI process. Nx tackles this issue head-on by intelligently re-running only the flaky tasks, rather than the entire pipeline. This approach not only saves time but also provides developers with more confidence in their CI pipeline's reliability.

Nx creates a hash of all the inputs for a task whenever it is run. If it encounters a task that fails with a particular set of inputs and then succeeds with those same inputs, Nx knows for a fact that the task is flaky.

New Nx features that tie in with our core speed improvements

Module Federation

With the help of Nx and the Module Federation setup that Nx offers, you can split up large Angular apps into smaller “vertical slices”. This can significantly speed up your builds and app startup time. Nx has revolutionized the use of Module Federation, especially in how static remotes are built and served. We make use of Nx’s task orchestration, allowing users to fine tune the number of builds happening in parallel to improve local startup time, manage machine resources better, allow for scaling.

First-Class Playwright support

With first-class support for Playwright through @nx/playwright, Nx offers out-of-the-box generators to run Playwright tests efficiently. This integration is especially powerful with features like TaskAtomizer, enhancing the testing process's speed and reliability.

Conclusion

Nx provides an unparalleled toolkit for developers and teams looking to optimize their development workflows, and we keep making it faster. By intelligently leveraging modern technologies and innovative optimizations, Nx delivers speed, efficiency, and reliability, allowing teams to focus on what matters most: building great software.

Learn more

#nx Follow

Introducing @nx/nuxt: Enhanced Nuxt.js Support in Nx

Katerina Skroumpelou — Tue, 06 Feb 2024 16:35:05 +0000

We're excited to introduce a new way to enhance your Nuxt development workflow! After the Vue plugin, we're introducing our new Nx plugin for Nuxt, @nx/nuxt. Designed for Nuxt developers and existing Nx users alike, this integration brings the best of both worlds into your development ecosystem, enabling you to leverage Nx's powerful capabilities seamlessly within your Nuxt projects.

Why Consider Nx for Your Nuxt Projects?

Using Nx with your Nuxt.js projects presents the following advantages:

Monorepo Management: Simplify the management of multiple projects within a single repository, facilitating code sharing and reducing overhead.
Modular Development: Break down your Nuxt app into manageable, independent modules that can be developed, tested, and deployed in isolation.
Enhanced Caching: Accelerate your development with Nx's intelligent caching, automatically configured for your Nuxt projects.
Nx generators: Nx provides generators for scaffolding new Nuxt applications, with support for Jest, Storybook, and e2e test generation with Cypress or Playwright.
Automated upgrades: Nx offers a set of migrators that help you upgrade your projects.

Getting Started with Nx and Nuxt.js

Whether you're initiating a new project or integrating into an existing one, @nx/nuxt offers a straightforward setup process:

Starting a New Nx Workspace with Nuxt

Creating a new Nx workspace optimized for Nuxt is as simple as running:

npx create-nx-workspace@latest --preset=nuxt

Our setup wizard will guide you through the initial configuration, ensuring your workspace is tailored to your needs:

npx create-nx-workspace@latest

 >  NX   Let's create a new workspace [https://nx.dev/getting-started/intro]

✔ Where would you like to create your workspace? · my-org
✔ Which stack do you want to use? · vue
✔ What framework would you like to use? · nuxt
✔ Integrated monorepo, or standalone project? · integrated
✔ Application name · my-app
✔ Test runner to use for end to end (E2E) tests · playwright (also cypress)
✔ Default stylesheet format · scss (also css, less)
✔ Set up CI with caching, distribution and test deflaking · github

This command will create a new Nx workspace with a single Nuxt application, complete with essential features and ready for development.

Enhancing an Existing Nuxt Project with Nx

Integrating Nx into an existing Nuxt.js project has never been easier, with the help of the nx init command. This command will add Nx to your project without the need to disrupt your current setup.

How It Works

When you run nx init in your existing Nuxt.js project, Nx does the following:

Installs @nx/nuxt: Adds the necessary Nx and @nx/nuxt dependencies to your project, enabling Nx's features while keeping your existing setup intact.
Understands Existing Configurations: Nx automatically recognizes your nuxt.config.js or nuxt.config.ts file, ensuring that all your custom configurations, scripts, and commands are preserved and utilized.
Minimal Configuration: Only a minimal nx.json file is added to your project. This file is used to configure the @nx/nuxt plugin if needed, but in most cases, your existing Nuxt.js configurations will suffice.

To begin the integration process, simply navigate to the root of your existing Nuxt.js project and run:

npx nx init

This approach offers several key benefits for teams looking to adopt Nx:

Zero Disruption: Your project will continue to use its existing configurations, and the existing configuration entrypoint files. There's no need to learn new configuration syntaxes or reconfigure your project to start using Nx.
Immediate Value: Instantly gain access to Nx's powerful developer tools and build system, without significant changes to your project.
Future Flexibility: As your project grows, Nx is ready to scale with you. You can gradually adopt more Nx features and plugins over time, at a pace that suits your team.

Using Nx to run your Nuxt app

Nx scans your workspace to look for Nuxt configuration files (eg. nuxt.config.ts). It uses these files to understand where your Nuxt projects live, and uses them to set up tasks that can be invoked through Nx, like serve and build. So, in your Nx workspace, you can then run:

nx serve my-nuxt-app

and

nx build my-nuxt-app

and these commands will call the nuxt CLI under the hood, enhanced with Nx's features.

You can see a visual representation of your task dependencies by running

nx graph

You can also see how Nx configures your tasks, by running:

nx show project my-nuxt-app --web

Using Nx Console

You get access to all these features through our VSCode and WebStorm Nx Console extension.

You can use Nx Console to visualize tasks, and understand where each inferred task (like build and serve in Nuxt's case) is coming from, with our codelens-like feature, as an alternative to the --web flag on the nx show project command.

Nx Console is also very convenient for generating code and running tasks, since it offers a graphical user interface for all the amazing features of the Nx CLI.

What Does Nx Bring to Your Nuxt Development?

With @nx/nuxt, your Nuxt projects gain automatic recognition of build and serve processes. There's no need to deal with unfamiliar configurations; Nx intuitively understands your Nuxt project structure and optimizes accordingly.

Modularize and Scale with Ease

One of the most compelling aspects of using Nx with Nuxt.js is the ability to modularize large applications into manageable libraries or components. This not only makes your codebase more organized and maintainable but also significantly enhances your development workflow and CI processes.

Breaking Down a Monolithic Nuxt App

Large Nuxt applications can become challenging to maintain and scale over time. By adopting Nx, you can structure your Nuxt app as a collection of smaller, focused libraries. Each library can encapsulate specific functionalities or features, such as UI components, utilities, or business logic.

Independent Development and Testing

This modular structure allows teams to work on different aspects of the application in parallel, reducing bottlenecks and improving collaboration. Furthermore, you can run tests, linters, and other checks independently for each library, making your development process more efficient and targeted.

For instance, if you want to create a new Vue UI library, you can use the following command:

nx generate @nx/vue:lib my-shared-ui

This command creates a my-shared-ui library within your workspace, which can then be used across your Nuxt app and potentially other applications within the same workspace.

Enhancing CI with Modular Builds

On the CI front, Nx's modular approach makes things much faster. You can configure your CI pipeline to build, test, and deploy only the affected libraries and applications, thanks to Nx's advanced dependency graph analysis. This results in faster CI runs and more efficient resource utilization.

Sharing Code Between Applications

Nx's workspace model facilitates code sharing between projects, which is particularly useful in monorepos containing multiple front-end projects. With Nx, sharing UI components, utilities, or services between these applications becomes straightforward.
To share code, simply import the library into your Nuxt and Vue applications as needed. Nx takes care of the rest, ensuring that dependencies are correctly managed and that your applications remain buildable and testable.

Imagine a scenario where your workspace contains a Nuxt application for your public-facing website and a Vue application for an internal tool. You can create a shared library for common UI components, such as buttons, inputs, and modals, and use these components in both applications. This not only reduces duplication but also ensures consistency across your projects.

// Importing a shared UI component in your Nuxt app
import { MyButton } from '@my-org/my-shared-ui';

// Importing the same component in your Vue app
import { MyButton } from '@my-org/my-shared-ui';

Visualizing Your Project Structure

Nx provides a clear overview of your project's structure and dependencies, making it easier to manage complex applications. The Nx Console extension for VSCode, for instance, offers a graphical interface to visualize and run tasks, enhancing your development experience.

In your workspace, you can run

nx graph

and see the structure of your projects:

Embracing Nx in Your Nuxt Journey

Whether you're starting a new Nuxt project or looking to enhance an existing one, Nx offers a compelling set of tools and features to streamline your development process. From modularization to caching, the integration of Nx into your Nuxt projects promises a more efficient, scalable, and enjoyable development experience. By embracing Nx's capabilities in your Nuxt development, you're not just optimizing your current workflow; you're future-proofing your development process. As your projects grow and evolve, Nx's modular architecture and powerful tooling will continue to provide value, making your development experience more enjoyable and productive.

Learn more

#nx Follow

Storybook Interaction Tests in Nx

Katerina Skroumpelou — Thu, 03 Aug 2023 15:20:27 +0000

In Nx 16.6 we are introducing our new generators for Storybook interaction tests! These new generators replace the default Cypress tests we used to generate along with a project’s Storybook configuration, particularly for those already using Storybook. The intention is that if a user chooses to use Storybook and generate Storybook configuration, to integrate in that experience Storybook Interaction testing, and skip generating Cypress tests, to keep everything in one place, in an integrated experience.

Understanding Storybook Interaction Tests

Interaction tests allow users to verify the functional aspects of UIs. This is done by supplying the initial state of a component, simulating user behavior such as clicks and form entries, and finally checking if the UI and component state update correctly. Very much like e2e tests are doing.
In Storybook, this workflow occurs in your browser, which makes it easier to debug failures since you’re running tests in the same environment you develop components.

How it works

You write a story to set up the component’s initial state, simulate user behavior using the play function, and then use the test runner to confirm that the component renders correctly and that your interaction tests with the play function pass. Storybook’s Test runner is a standalone utility — powered by Jest and Playwright — that executes all of your interaction tests, and runs parallel to your Storybook.

Setting Up Storybook Interaction Tests on Nx

You can read our detailed guide on how to set up Storybook interaction tests on Nx, here: https://nx.dev/packages/storybook/documents/storybook-interaction-tests.

Writing Interaction Tests in Storybook

An interaction test is defined inside a play function connected to a story. The story simulates the user’s behavior once it loads in the UI and verifies the underlying logic.
Under the hood, Storybook’s @storybook/addon-interactions mirrors Testing Library’s user-events API. So, you can use the same queries and assertions that you would use for Testing Library, like we already do with our unit tests.
For complex flows, it can be worthwhile to group sets of related interactions using the step function. This allows you to provide a custom label that describes a set of interactions.

Debugging and Reproducing Errors

Storybook provides an interactive debugger that displays the step-by-step flow of your interactions, and provides UI controls to pause, resume, rewind, and step through each interaction.

If an error occurs during a story’s play function, it’ll be shown in the interaction addon panel to help with debugging. And since Storybook is a web app, anyone with the URL can reproduce the error with the same detailed information without any additional environment configuration or tooling required.

Executing and Automating Tests

Storybook only runs the interaction test when you’re viewing a story. Therefore, as a Storybook grows, it becomes unrealistic to review each change manually. The Storybook test-runner automates the process by running all tests for you. This can be executed via the command line or on CI environment.

What should I choose? Interaction tests or E2E tests?

Setting up interaction tests with Nx and Storybook provides an extra layer of confidence in the functionality of your components. It ensures that they not only look right but also behave correctly in response to user interactions.
Storybook interaction tests provide a unique advantage over traditional e2e tests, especially when considering the development setup. With Storybook already in place, you essentially have a controlled environment set up for each of your components. This allows you to write interaction tests almost immediately, without the overhead of setting up and navigating through a full application environment, as is the case with e2e tests.
Moreover, since Storybook isolates each component, you can ensure that the tests are solely focused on individual component behavior rather than application-level concerns. This results in faster test execution, easier debugging, and more granular feedback during the development process. In essence, with Storybook’s interaction tests, you get many of the benefits of e2e tests but with a setup that’s quicker, more focused, and integrated right into your component development workflow.

Useful Links

Learn more

Also, if you liked this, click the ❤️ and make sure to follow Katerina and Nx on Twitter for more!