Forem: dbroemme

An Introduction to Lambdas in Ruby

dbroemme — Wed, 28 Jun 2023 11:00:00 +0000

Lambdas are a powerful programming construct used in many languages. These functions allow developers to write code that is more concise, efficient, and easier to maintain, making Lambdas an essential part of any developer's toolkit.

In this article, we'll explore how you can use Lambda functions in Ruby to simplify your code and make it more powerful.

What Is a Lambda Function?

Lambdas are expressions whose value is a function. For this reason, they are also called anonymous functions. The really nice part about this is that we can reference them using a variable in our code, just like a string, integer, or any other object.

Everything in Ruby is an object, and Lambdas are no exception. Specifically, a Lambda is an instance of the Ruby Proc class. They behave the same way as a Proc, except that a Lambda validates the number of parameters. Its use case is targeted towards specific functions, albeit those without a proper name. In fact, the Ruby Proc class has a lambda? method which returns true if argument handling is rigid (as is the case for a Proc generated by a Lambda).

Side-note: Because of its popularity, it is also important to clarify that AWS Lambda is a different concept to a Lambda in Ruby. AWS Lambda is a cloud-computing service that implements functions without managing the underlying server infrastructure. You can simply write the function in any number of programming languages, including Ruby.

How to Define and Invoke a Lambda Function in Ruby

The canonical form of a Lambda expression is as follows:

a_lambda = lambda { puts "Hello world!" }

More commonly in Ruby, you will see the -> shorthand notation used to define a Lambda expression.

a_lambda = -> { puts "Hello world!" }

If you simply puts a_lambda, here is what you get.

puts a_lambda
=> #<Proc:0x0000000108f77bb0 10.rb:6 (lambda)>

Lambdas can use a block definition notation as well.

a_lambda = lambda do
  puts "Hello world!"
end

The Lambda is invoked as shown below, and the output is as expected:

a_lambda.call
=> Hello world!

The mechanism to invoke a Lambda is the same as it is with a Proc. However, it is different from the way in which traditional functions are called. Note that, if you simply reference a_lambda in your code, it will not invoke the function. This is equivalent to referencing a variable but doing nothing with it.

Lambdas can also accept parameters. The following Lambda returns the square of a number:

a_lambda_with_params = lambda {|val| val**2 }

The call method can take parameters to match your Lambda signature:

three_squared = a_lambda_with_params.call(3)
puts "Three squared is #{three_squared}"

This outputs the following:

Three squared is 9

Ruby offers other syntax options for invoking a Lambda. All of the following statements are equivalent:

val1 = a_lambda_with_params.call(3)
val2 = a_lambda_with_params.(3)
val3 = a_lambda_with_params[3]
val4 = a_lambda_with_params===3

The last few notation alternatives seem quite obtuse, but they work nonetheless.

Now consider that we want to calculate the square of a set of numbers. We might typically write Ruby code such as this:

numbers = [1, 2, 3, 4, 5]
squares = numbers.map { |n| n**2 }
puts squares.join(", ")

In this code, what is the expression passed to the map function? It is a block that accepts a single parameter. Thus, it is equivalent to an anonymous function or Lambda. We can rewrite our code as follows:

numbers = [1, 2, 3, 4, 5]
square_lambda = lambda { |n| n**2 }
squares = numbers.map &square_lambda
puts squares.join(", ")

This outputs the following:

1, 4, 9, 16, 25

Notice the ampersand that precedes the Lambda expression passed to map. Why is this needed? Well, the ampersand tells Ruby that this is the special block parameter. If we omit this, Ruby thinks that square_lambda is a 'regular' parameter. Using that approach, we would get the following error.

in `map': wrong number of arguments (given 1, expected 0) (ArgumentError)

One of the main benefits of a Lambda is the convenience and ease of using the function only once. We could also define a traditional method here, but if this is the only place it would be used, a Lambda function provides a cleaner solution for such a 'disposable' function.

Another primary benefit of Lambdas is that they provide a portable mechanism to execute code anywhere in your program. This supports some interesting use cases that we'll explore next.

Lambda Use Cases in Ruby

There are a number of use cases that work well as Lambda functions in Ruby.

First Class Functions

Lambda functions can be passed to other functions for use. This concept is referred to as a first-class function. Consider a use case where you need to execute code at a later point in time, once a certain condition is met.

# Example: Use a lambda to defer execution of code until later
def do_something_later(&a_lambda)
  @deferred_action = a_lambda
end

# Later, when the condition is met, call the deferred action
if some_condition_is_met
  @deferred_action.call
end

Callbacks

Callbacks are another related use case for which Lambdas are useful. Lambdas can be used to implement logic invoked at certain points in the application lifecycle, or a specific object’s lifecycle.

A good example of this pattern is found in MongoDB's Mongoid framework. You can set a :default at the field level, and Lambdas allow you to defer execution until runtime.

Consider a document with a last modified timestamp. The first implementation runs when the class is loaded. However, execution of the Lambda is deferred until the document is created — the outcome you likely want in your application.

# Static implementation at class load time
field :last_modified, type: Time, default: Time.now

# The implementation below defers the lambda execution until document creation time
field :last_modified, type: Time, default: ->{ Time.now }

The Visitor Pattern

The visitor pattern in Ruby is another use case that can take advantage of the flexibility of Lambdas. If the visitor logic is not known a priori or otherwise fits the Lambda use cases, visitors can be implemented as Lambda functions.

Consider the following code snippet for a visitor to a hierarchy of nodes representing mathematical expressions.

class PrintVisitor < Visitor
  NODE_TYPES = {
    IntegerNode => ->(node) { puts "IntegerNode: #{node.value}" },
    FloatNode => ->(node) { puts "FloatNode: #{node.value}" },
    ExpressionNode => ->(node) {
      puts "ExpressionNode:"
      node.left.accept(self)
      node.right.accept(self)
    }
  }
  def visit(node)
    handler = NODE_TYPES[node.class]
    handler.call(node)
  end
end

This implementation uses Lambda functions to avoid defining separate methods for each element subclass. Instead, the Visitor class defines a single visit method that can handle any element subclass by using a Lambda function to call the appropriate method.

This may be a bit over-engineered, but if you consider cases where the visitor logic is not predefined, Lambdas become much more useful.

Functional Programming Paradigms

Lambdas in Ruby enable developers to adopt a functional programming paradigm by creating anonymous functions that can be used as arguments in other functions. This approach allows for creating complex functionality by combining smaller, reusable functions. The use of Lambdas in Ruby code results in concise, readable code. They also leverage key functional programming concepts, such as closures and higher-order functions.

Lambdas can support functional programming features such as currying if you want to take them to that level. Currying permits functions to be partially applied and reused in diverse contexts.

Metaprogramming

Lambdas are used in metaprogramming to generate code dynamically. Consider the following code that adds a method:

# Example: Use a lambda for dynamic code generation
def create_method(name, block)
  define_method(name, &block)
end

a_lambda = -> { puts "Hello, world!" }
create_method(:hello, a_lambda)

hello   # Output: "Hello, world!"

Lambdas as Closures

A powerful capability of Lambdas is their ability to create a closure. This encapsulates the function as well as the runtime environment, including variables from the surrounding scope. Variables are closed (or bound) to their values in a Lambda. This is why it is called a closed expression or closure. The execution context is stored in an instance of a Ruby Binding object.

Consider the following example that creates a Lambda which implements an incremental counter:

# Example: Use a lambda as a closure to capture runtime variables
def create_counter(start)
  lambda { start += 1 }
end

counter = create_counter(0)
puts counter.call   # Output: 1
puts counter.call   # Output: 2

The Impact of Lambdas on Performance

Performance can be a consideration if you have extremely tight operational requirements. In a tight loop, for example, you will see that Lambdas perform a bit slower than other Ruby code. It is easy to understand why. A Proc object is created to implement a Lambda (as opposed to the runtime engine delegating control to a method).

In the vast majority of use cases, you will likely not notice much difference in performance (considering that computers can execute thousands of instructions in the time it takes to perform a single database query). Nonetheless, this is something to keep in mind.

Wrapping Up

In this post, we explored some of the fundamentals of Lambdas and Lambda use cases for Ruby.

Lambdas provide a powerful fundamental programming construct for Ruby developers. They can be convenient one-time use functions, implementation mechanisms for callbacks, and used for several other use cases. Lambdas can also come in handy on projects that adhere to functional programming paradigms.

Happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!

Audit Logging in Ruby and Rails

dbroemme — Wed, 19 Apr 2023 13:09:18 +0000

An audit log is a documented record of events and activity within an information technology system. Audit logging is critical for application owners and administrators to track activity within their systems.

In this post, we'll first dive into what auditing entails and what to consider when audit logging. We'll then explore some options for implementing audit logs, including PaperTrail, Audited, AuditLog, AppSignal, and a custom implementation.

Let's get started!

Auditing and Audit Logs: An Introduction

Auditing provides an understanding of system behavior using verifiable, well-defined events. It has a different goal than application logging, which records general-purpose messages often used to diagnose or troubleshoot issues within an application.

Auditing is typically a foundational requirement for many systems, yet it is often one of the last concerns of development teams. The use of simple, effective patterns upfront can change this trend.

One motivation for teams to tackle this capability early on is the feedback it provides in terms of metrics for testing and evaluating systems. The observability of audit events can highlight system issues or even missed requirements.

Audit logs typically have a defined structure that includes the following event information:

Timestamp
Event type
Description
User or system process that initiated or requested the event
Impacted entities in the system
Any additional relevant detail

Any system, service, or device can have its own audit log. A collection of audit logs spanning a particular dimension is considered an audit trail of related activity. An audit trail could be specific to a user, application, or system entity. For example, the AWS CloudTrail service records all activity within an AWS account.

Audit logs support, among others, the following key functions and use cases:

Compliance and regulatory requirements
Security reviews and investigations of potential security breaches
Determining what users make particular changes in a system and when

Design Considerations for Audit Capabilities

Audit logging solutions typically involve a centralized log management system. Each application instance can directly send audit events to the centralized repository, or the solution may include agents running on compute nodes that facilitate this communication.

Distributed architectures can make auditing challenging because there are many sources of audit events throughout the infrastructure. Centralized solutions aim to funnel all of the disparate audit events through a central pipeline for processing.

Audit management systems provide storage and search capabilities for the collection of audit events. There are several persistence options, although many Ruby and Rails-based solutions use database persistence to store audit events as a natural extension of ActiveRecord capabilities.

As an alternative, cloud data streaming services can also be used. For example, AWS Kinesis is designed to handle extremely large volumes of streaming data and is well suited for the audit use case. Another option is to store audit data in AWS S3 buckets and then query it using AWS Athena.

What Constitutes an Audit Event in Ruby and Rails?

Countless events occur within a system, so one of the most important decisions is determining what exactly to audit. Audit events can be based on functional requirements (e.g., a user submits an order) or non-functional requirements (e.g., a security-relevant event occurs, such as a failed login attempt). Both types are important and serve different purposes. Certainly, security auditing is of foremost concern in today’s operating environment.

The level of detail is another key consideration. A single business transaction may involve updates to multiple database tables. While a change in an individual database entity could be the subject of an audit event, by itself this may not contain enough information about the overarching event taking place. It is often beneficial to audit the business function in addition to, or even in place of, individual database rows. Business events typically occur within the service or controller level code.

Some use cases require all of the detail you have. For example, requirements may dictate that all user HTTP requests must be audited. You may need to know which users queried for particular information in comprehensive security audits. As is often the case in software engineering, let your requirements be your guide in making these design decisions.

Ruby Audit Log Implementation Options

Most Ruby gems in this space are focused on the Rails use case, as it has ORM capabilities to identify changes to ActiveRecord instances and store them as audit events. This enables basic audit capabilities without much development effort. Keep in mind, however, the level of detail of your audit events. You may need to add code to specifically audit business events in your service or controller classes.

AppSignal also has a new logging feature as part of its integration for Ruby. Applications can log messages and structured event data from their applications which are stored on managed AppSignal servers. Once you've signed up, you can use AppSignal dashboards to search, filter, and view your logs.

PaperTrail

The PaperTrail gem is a popular choice for audit logging in Ruby. It is focused on tracking changes to ActiveRecord model classes using a versions table. It allows you to see how a model looks at any point in its lifecycle. Several configuration options determine what type of model changes result in creating an audit event, or version. For example, you can choose to audit all changes or only create audit events where designated attributes are modified.

If you have an application that requires maintaining a history of changes over time, you will find this gem particularly useful. There is a good deal of overlap between auditing and versioning capabilities, and PaperTrail provides a nice solution for both. A versions method is added to each model class.

You can easily see the changes between versions as well, as shown in this code snippet from the PaperTrail documentation:

widget = Widget.create name: 'Bob'
widget.versions.last.changeset # reads object_changes column
# {
#   "name"=>[nil, "Bob"],
#   "created_at"=>[nil, 2015-08-10 04:10:40 UTC],
#   "updated_at"=>[nil, 2015-08-10 04:10:40 UTC],
#   "id"=>[nil, 1]
# }
widget.update name: 'Robert'
widget.versions.last.changeset
# {
#   "name"=>["Bob", "Robert"],
#   "updated_at"=>[2015-08-10 04:13:19 UTC, 2015-08-10 04:13:19 UTC]
# }
widget.destroy
widget.versions.last.changeset
# {}

As long as your controller has a current_user method, it will track who is responsible for changes using the following callback:

class ApplicationController
  before_action :set_paper_trail_whodunnit
end

One drawback is the lack of a well-defined pattern for custom audit events. A workaround is to create a database entity that captures relevant audit information, although that somewhat defeats the purpose of this gem’s paradigm.

Audited

The Audited gem is another implementation similar in scope to PaperTrail. It now only focuses on ActiveRecord support. An opt-in approach is used to denote which model classes should be audited, as shown below.

class User < ActiveRecord::Base
  audited
end

Model classes get a revisions method similar to PaperTrail’s versions. Audited models also get an audits method that provides change-tracking information.

However, this choice suffers from the same drawback as PaperTrail: a lack of support for custom audit events.

AuditLog

AuditLog takes a different approach. Rather than centering the design around model classes, it uses explicit calls to create audit events, as shown in the code below.

AuditLog.audit!(:update_password, @user, payload: { ip: request.remote_ip })
AuditLog.audit!(:sign_in, @user, payload: { ip: request.remote_ip })
AuditLog.audit!(:create_address, nil, payload: params)

If you want to audit individual model entities, you can use code similar to the following to accomplish this.

class TicketsController < ApplicationController
  def index
    audit! :list_tickets, nil
  end

  def create
    if @ticket.save
      audit! :create_ticket, @ticket, payload: ticket_params
    else
      render :new
    end
  end

  def update
    if @ticket.save
      audit! :update_ticket, @ticket, payload: ticket_params
    else
      render :edit
    end
  end
  ...
end

PaperTrail and Audited can do this automatically for you. However, in this code sample, you can also see the use of action-based auditing in the index method. This equates to an audit event capturing a view tickets operation.

AuditLog also provides a simple web interface to view audit information. While your operational requirements may dictate a more comprehensive reporting capability, this is a nice feature to have right out of the box.

Audit Logging with AppSignal

AppSignal provides a managed logging mechanism that offloads the storage of log data and provides a comprehensive set of search and filter capabilities. Each application has a default log source, or you can create additional sources as shown below.

The following code shows an example of creating an audit log event from a sample blog application. The logger constructor takes a group name as a parameter that can be used to identify event sources. This code example comes from an articles controller class where new articles are created.

logger = Appsignal::Logger.new("articles")
logger.info("Created new article #{@article.id}", { article_id: @article.id, date: DateTime.now })

AppSignal dashboards can be used to view log data, both at the aggregate level as well as detailed audit log information.

You can drill down into specific log events and view the structured data included with the event, as shown below.

Custom Audit Logging Implementation

You can also roll your own implementation if you have specific requirements that don’t fit one of these solutions. In most cases, you will need to implement additional reporting capabilities using the stored audit data.

For this, you may choose to:

Use another Ruby gem.
Push the data to a data warehouse or cloud service for analysis and long-term storage.

Choosing an Audit Logging Implementation

Here are a few key considerations to help you choose between audit solutions:

What types of events do you need to audit? Are they primarily database-driven events? If so, a solution like PaperTrail is a great choice. If you have higher-level business events, then you need support for custom audit events similar to the pattern implemented by AuditLog.
Do gems with higher activity levels give you more confidence? If so, PaperTrail and Audited are two great choices. AuditLog provides more flexibility, but it has less of an installed user base.
What volume of audit events do you need to capture? Will the storage of audit events in the same application database add significant overhead to your application? If so, consider using AppSignal for minimal impact to your application's performance.
Remember that you also need query and reporting capabilities. The out-of-the-box capability here varies widely across the different options. AppSignal provides a comprehensive search and filter capability over log data.

Wrapping Up

Audit logging is a fundamental capability for almost all business applications. We've seen that it is beneficial to establish a design pattern early on for the observability of a system as development progresses.

There are some great implementations available for Ruby and Rails developers: PaperTrail, Audited, AuditLog, and AppSignal. Evaluate your requirements using the criteria discussed above to determine the best choice for your needs.

Happy audit logging!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!

Monitor Ruby Application Performance with Magic Dashboards

dbroemme — Tue, 13 Sep 2022 13:04:50 +0000

Application teams must understand what their customer experience is like. This is true not only from a general perspective (in terms of usability and responsiveness) but also on a day-to-day, minute-by-minute basis.

In particular, when you work with distributed systems, errors are inevitable. Site traffic fluctuates throughout the day, and any one of a system’s dependencies could also encounter an issue at any time.

In this article, we'll use magic dashboards to help monitor and resolve performance issues within a Ruby on Rails application.

But before we dive into magic dashboards, let's see what we should look out for when designing our apps.

Things to Consider When Building A Ruby App

As an application owner or team member, you'll want to know if there are any problems with your application before the customer does. This enables you to take corrective action immediately and hopefully avoid any disruption to your users.

There are a few key application questions you need to be able to answer at any point in time:

Is the page response time acceptable? According to Google, these days anything slower than two seconds will cause customers to leave your site and go elsewhere.
Are your users experiencing any errors? If so, what types of errors? What is the error rate?
Are any ongoing operational issues affecting your application? This could be with the network, storage, or security services. As we all know, cloud providers have outages that impact our application’s health as well.

Observability is key, yet it is often the last thing engineers think about. Besides, there is already time built into the schedule for operational readiness. It's called the weekend before the product launch!

Joking aside, you have to build your app first before something exists to be observed. This lends itself to delaying performance testing until close to the end of the development lifecycle. You don’t want to spend too much time performance testing when the product isn’t code-complete, because then you’ll have to do it all over again later.

The same is true for security testing. If you conduct it too early, a vulnerability could still be introduced after testing, but before a product goes into production.

So, we know monitoring is critical, but all of these concerns are fair points. Monitoring doesn’t get the attention it deserves until late in the process. That’s why AppSignal created magic dashboards.

Magic Dashboards in AppSignal

AppSignal understands the importance of metrics, dashboards, and your app's performance — but also that engineers have minimal time to work on these before launch.

Our magic dashboards are called "magic" because the metrics collection and associated dashboards are created automatically for you, simply when you connect your app and integrated components.

Magic Dashboards: An Example Ruby on Rails App

Let's find out how we can monitor and resolve performance issues within a Ruby on Rails app using magic dashboards.

Our application uses a simple machine learning (ML) model for cryptocurrency price prediction. An asynchronous job updates the model daily with the latest price data and improves accuracy over time. This Rails app has both web pages and a REST API, as shown in the architecture diagram below.

After we deploy our Rails app, magic dashboards are automatically created for the Rails Puma web server and the Sidekiq asynchronous jobs. Other supported integrations for magic dashboards include MongoDB and the Erlang VM.

You can use the Puma magic dashboard to evaluate performance based on threads, pool capacity, and puma workers. The Sidekiq magic dashboard monitors queue length, queue latency, job duration, job status, and memory usage.

Magic dashboards are detected and created based on the use of event-based metrics such as a Sidekiq job run, as well as minutely probes. The minutely probe feature allows you to register a Ruby block or class to send custom metrics to AppSignal. Magic dashboards provide out-of-the-box integration for supported components, but you can leverage this mechanism to send your custom metrics to AppSignal.

An Example of Custom Minutely Probes

Sidekiq integration is already built in, but imagine you want to monitor a proprietary background job mechanism. You can use the following class example. The probe class obtains a connection that is then used on each call to get the desired metrics.

# config/initializers/appsignal.rb or a file that's loaded on boot

# Creating a probe using a Ruby class
class BackgroundJobLibraryProbe
  def initialize
    # This is only called when the minutely probe gets initialized
    require "background_job_library"
    @connection = BackgroundJobLibrary.connection
  end

  def call
    stats = @connection.fetch_queue_stats
    Appsignal.set_gauge "background_job_library_queue_length", stats.queue_length
    Appsignal.set_gauge "background_job_library_processed_jobs", stats.processed_jobs
  end
end

# Registering a Class probe
Appsignal::Minutely.probes.register(
  :background_job_library_probe, BackgroundJobLibraryProbe
)

The call to Appsignal::Minutely.probes.register takes two parameters — a name for the probe and its implementation, which can be either a lambda or a class that implements the call method.

Configuration and System Requirements for AppSignal

Simply use AppSignal with your Rails application to get magic dashboards. If you haven’t already installed AppSignal, include the appsignal gem in your Gemfile and run a bundle install.

gem 'appsignal'

Then run the appsignal install command to configure your environment. This configuration can be stored in a config file or environment variables, and connects your application to your AppSignal account and dashboards.

bundle exec appsignal install appsignal-license-key

After you install and run your server, you will see a few emails similar to the following.

The system requirements for this example are:

AppSignal gem 2.9.0 or higher, which includes support for the minutely probe.
Puma integration - requires version 3.11.4 or higher.
Sidekiq integration - requires the Redis gem 3.3.5 or higher. For this integration, the AppSignal gem 2.9.5 or higher is recommended.

Performance Testing our Price Prediction API

The above example application uses a simple neural network implemented using the Ruby FANN gem to predict the next day's Bitcoin price. Percentage price changes from the last ten days are used as inputs to the model.

The REST API doesn't take any parameters, as it currently only predicts the price for tomorrow. The output is a simple JSON document, as shown below.

http://localhost:3000/crypto/predict_api

{
  "day":"2022-07-22",
  "price":23080.95619
}

A Sidekiq job gets the market price at the beginning of each day and updates the ML model. This job is scheduled to run every five minutes, enabling it to catch up to any outages or errors quickly.

You can find all of the code from this article on GitHub. Please note that nothing in this article or the software constitutes investment advice. Consult a financial advisor before making any investment decisions regarding cryptocurrency.

We'll use JMeter to simulate load on the REST API for performance testing. This allows us to generate traffic and evaluate performance using our magic dashboards. Our first test uses 5 concurrent clients, each making 100 requests.

The statistics show a fairly high range in response time, with an average of 342ms, but the P99 is 852ms. It seems that we can do better.

Improving Rails App Performance with the Puma Magic Dashboard

Our Puma magic dashboard includes a graph of the thread pool capacity, and we can see that it touches zero during our test run. This would explain why some requests take longer than others, so we'll increase the number of puma threads to 10.

Keep in mind, we did nothing to create the dashboard or this graph. AppSignal automatically created this for us.

After running the test with the increased puma thread count, the response times are much more consistent, and the dashboard confirms that pool capacity stays within acceptable levels.

Now it is time to turn up the dial. The number of JMeter concurrent clients is increased to 50, with a ramp-up time of 5 seconds. This test shows a poor response time and a number of API errors. The Puma magic dashboard again shows the available puma threads reaching zero.

Looking at the API code, we find that the model is being loaded from the database each time. This is not very efficient, so we change the Sidekiq job to not only grab the new daily price, but also run the ML model and save the prediction in the database.

We deploy this change.

Better API Performance with Sidekiq's Magic Dashboard

Now let's check our Sidekiq magic dashboard. Unfortunately, there is a bug in the code, but at least we can identify and fix it quickly.

With the Sidekiq PriceUpdateJob now working, the API is modified so that it only needs to retrieve the predicted price from the database. This improves the API performance, but we still see some API errors and lengthy response times.

Back to the Puma Magic Dashboard

A glance at the dashboard highlights that we have not yet configured Puma to use additional workers. A Puma worker is an OS-level process that can run several threads. The total thread count is calculated as the number of workers multiplied by the maximum threads. First, we use 2 workers, but the available threads are still exhausted. So we'll increase the amount to 4 workers.

Our API performance is now very good. The average response time is 419ms, and there are no API errors.

The magic dashboard confirms that there is still available thread capacity.

AppSignal's magic dashboards give us instant insights into the capacity of the Puma web server. The Sidekiq and Active Worker dashboards provide similar insights into our application's asynchronous jobs.

What We've Learned from Magic Dashboards

During our Rails application performance testing, the Puma magic dashboard helped us easily identify that the number of threads was insufficient for our desired throughput level. It shows the thread pool capacity, number of workers over time, and the total number of threads.

While the Sidekiq job did not have any performance issues, the magic dashboard helped us quickly identify that there was an error after deployment, which we were able to fix and redeploy rapidly. All of these monitoring capabilities were set up for us automatically by AppSignal.

AppSignal's Dashboard Features for Ruby and Rails Apps

Dashboards in AppSignal have namespaces, such as "web" application or "background" jobs, so you can create numerous monitoring views of your application. The built-in summary dashboard shows an overview of your application's health, including throughput, response time, and the latest errors.

Note that, as with any dashboard, you can edit what graphs and metrics are shown, as well as change the layout and the configuration of selected graphs.

Anomaly detection is a powerful feature. It allows you to define thresholds that send notifications when a metric value goes over or below a given value, such as free memory or the error rate.

Wrapping Up: Monitor Your Ruby App Today with AppSignal

Observability and performance are critical to the success of our applications. However, we often wait until the last minute to deal with these topics.

AppSignal's magic dashboards provide extensive out-of-the-box metrics, dashboards, and insights that are set up automatically. This allows you to rapidly tune your application’s performance and reach a state of operational readiness.

Using Scientist to Refactor Critical Ruby on Rails Code

dbroemme — Wed, 25 May 2022 11:11:04 +0000

Ask any software engineer to review key portions of production code, and inevitably, they will point out three things that need to be refactored. So why does so much bad, brittle, or misunderstood code remain running in production?

The answer is simple: engineers are afraid to touch it. Refactoring tasks get identified and added to the backlog, but rarely make it into the current sprint.

There are numerous reasons for this. The code may have been written by an engineer who left the team years ago, and no one completely understands it. In other cases, the capability is critical to the business. No one wants to be responsible for a potential outage or loss of revenue.

In this post, we'll examine how you can use Scientist to migrate, refactor, and change critical Ruby production code with confidence.

But first, you might ask — can't we use tests to dig up code issues?

This Is What Rails Testing Is For, Right?

Yes, and no. It is often difficult to gain complete confidence in code changes before deployment. The unit and system tests pass. It's good to go, right?

The reality is that there is no substitute for the real world, i.e. production. What if the data quality is bad or tests are missing? How can you know if the new software will perform well enough to handle production throughput?

Teams with public services sometimes find that they need to deal with “bugwards compatibility” issues. When a bug has existed in production for a while, clients may code in a way that depends on consistent incorrect behavior. Customers often use software in unexpected ways.

Observe Production Changes in Ruby and Rails with Scientist

If production is the best place to gain confidence in a change, then consider observing how code behaves there. This may sound scary at first, as the idea of “testing in production” contradicts classic software engineering practices.

However, the good news is that it’s easy and safe to do so in Ruby and Rails using the Scientist gem. Scientist's name is based on the scientific method of conducting experiments to verify a given hypothesis. In this case, our hypothesis is that the new code does the job.

The reason we can safely take this approach stems from the fact that experiments still use the result of the existing code. New code is only evaluated for observation and comparison purposes, both for accuracy and performance. We mitigate the test coverage concerns discussed earlier by evaluating performance using real-world data and parameters. Experiments typically evaluate a chosen sample rate of requests to minimize the impact on production. However, you can evaluate every request if desired.

Let's now take a quick look at how Scientist works in a branch by abstraction way.

The Branch by Abstraction Pattern in Ruby's Scientist

Scientist's approach begins with the Branch by Abstraction pattern described by Martin Fowler as making “a large-scale change to a software system in a gradual way.”

We introduce an abstraction layer to isolate the code being updated. This layer decides which implementation to use so that the experiment is transparent to the rest of the system. The technique is related to using a feature flag that determines the code path.

The Scientist gem, which originated from Github, implements this pattern using an experiment. The existing code is referred to as the control, and the new implementation is the candidate. Both code paths are run in randomized order, but only the control result is returned to the client.

Using Scientist to Refactor a Ruby Service

Consider a Ruby service that returns the largest prime factor for a given number. Assume that we've identified optimizations to prune the required set of candidates, speeding up the service.

However, service owners want to be sure no bugs were introduced. They also want to observe any performance improvements. Introduce the following code, modifying clients to call this method:

require 'scientist'

def largest_prime_factor(number)
  science "prime-factors" do |experiment|
    experiment.use { find_largest_prime_factor(number) }     # old way
    experiment.try { improved_largest_prime_factor(number) } # new way
  end  # returns the control value
end

At this point, only the use (control) expression is invoked. To make the experiment worthwhile, define a custom Experiment class to enable it (100% of the time below) and publish the results (in this case, just logging). Scientist generates fantastic data but it doesn’t do anything with it by default. That part is left up to you.

require 'scientist/experiment'
require 'pp'

class MyExperiment
  include Scientist::Experiment

  attr_accessor :name

  def initialize(name)
    @name = name
  end

  def enabled?
    true
  end

  def raised(operation, error)
    p "Operation '#{operation}' failed with error '#{error.inspect}'"
    super # will re-raise
  end

  def publish(result)
    pp result
  end
end

The results of the experiment will be logged, and we can make improvements over time based on the feedback. Once the new code meets the requirements and confidence is high, accomplish cutover to the new implementation by simply replacing the science code with a delegation to the new implementation.

LabTech Simplifies Scientist Experiments in Ruby on Rails

You can use the LabTech gem in your Rails application to easily configure Scientist and handle the results.

Applications that use AppSignal can use the Appsignal.instrument custom instrumentation helper to track how long Scientist events take to complete. Wrap it around the different experiment code blocks to see events appear in the performance timeline.

Now, going back to LabTech — the web page below simply accepts a number to factor.

Getting started is easy provided you have access to the console. First, add the LabTech gem to your Gemfile and run a bundle install.

gem 'lab_tech'

Tables store results and experiment configuration, so run a database migration.

rails lab_tech:install:migrations db:migrate

The abstraction layer is the same, except the LabTech module is used. The full code is available on GitHub.

def largest_prime_factor(number)
    LabTech.science "prime-factors" do |experiment|
      ...
    end
end

At this point, the experiment is disabled, so use the console to enable it in all cases or for a percentage of the time.

bin/rails console
LabTech.enable "prime-factors"
LabTech.enable "prime-factors", percent: 5

We can now run tests and the experiment will be evaluated. For a textual view of results, use either of the following commands from the Rails console.

LabTech.summarize_results "prime-factors"
LabTech.summarize_errors "prime-factors"

After a few successful runs and one manufactured error, here is an example of what the results summary looks like. There is an overview of successes and failures, as well as an ASCII chart showing performance differences.

--------------------------------------------------------------------------------
Experiment: prime-factors
--------------------------------------------------------------------------------
Earliest results: 2022-04-27T02:42:45Z
Latest result:    2022-05-01T17:27:39Z (5 days)

3 of 4 (75.00%) correct
1 of 4 (25.00%) mismatched

Median time delta: +0.000s  (90% of observations between +0.000s and +0.000s)

Speedups (by percentiles):
      0%  [                         ·         █               ]    +2.4x faster
      5%  [                         ·         █               ]    +2.4x faster
     10%  [                         ·         █               ]    +2.4x faster
     15%  [                         ·         █               ]    +2.4x faster
     20%  [                         ·         █               ]    +2.4x faster
     25%  [                         ·         █               ]    +2.4x faster
     30%  [                         ·         █               ]    +2.4x faster
     35%  [                         ·         █               ]    +2.4x faster
     40%  [                         ·         █               ]    +2.4x faster
     45%  [                         ·         █               ]    +2.4x faster
     50%  [ · · · · · · · · · · · · · · · · · █ · · · · · · · ]    +2.4x faster
     55%  [                         ·         █               ]    +2.4x faster
     60%  [                         ·         █               ]    +2.4x faster
     65%  [                         ·         █               ]    +2.4x faster
     70%  [                         ·                        █]    +6.9x faster
     75%  [                         ·                        █]    +6.9x faster
     80%  [                         ·                        █]    +6.9x faster
     85%  [                         ·                        █]    +6.9x faster
     90%  [                         ·                        █]    +6.9x faster
     95%  [                         ·                        █]    +6.9x faster
    100%  [                         ·                        █]    +6.9x faster
--------------------------------------------------------------------------------

The Blazer gem provides a nice way to analyze the results easily. It is simple to install and allows SQL queries to run against tables. The query here shows that the candidate implementation is significantly faster than the original.

In the example prime factoring service, the speedup in the improved implementation comes from a heuristic that eliminates some possible factors to consider. As we consider higher numbers and find a prime factor, we can stop searching after we get to our target number divided by that factor. The new code path only adds one statement to accomplish this.

We can also see the reduction in execution time using a Blazer query against the LabTech tables.

Use Cases and Limitations of Scientist

Optimal use cases for Scientist include searches, calculations, and code that has no side effects. Code that includes transactional updates or external integrations such as email does not fit cleanly into the model because the capability is run twice (both the old and new implementations).

This is not a trivial limitation, as it does eliminate several use cases. However, there are some workarounds if the experiment is critical to your success. Consider whether the side effects are relevant or if duplication is an issue. For example, it may not matter in some cases whether two emails are sent during the evaluation. Another option is to have the new code determine the result but not persist it. This would prevent any meaningful performance comparisons. However, it would allow you to verify accuracy.

Other limitations stem from Scientist’s focus on return values. In some cases, valid results may exhibit differences over time, whether they simply include timestamps in the response or certain factors vary. In many cases, we can write custom comparison logic in the experiment to verify accuracy beyond basic string comparisons.

Finally, a limitation of LabTech is that it has not been ported to Rails 7 as of the time of writing.

Best Practices for Effective Scientist Experiments in Rails

Consider these items when implementing your experiments:

In Rails projects, Scientist can either be configured in an initializer or a wrapper like the Rails LabTech gem. Most Rails applications already have a database, so LabTech leverages ActiveRecord to store results.
To avoid slowing down development and testing, enable your experiment only in staging and production environments.
To minimize any potential impact on production, only run the experiment on a percentage of requests. LabTech supports this out of the box as an optional parameter when you enable the experiment (it is initially disabled by default). Using pure Scientist, this logic is easy to code in the experiment’s enabled? method.
Some logic is resource-intensive, so a low sampling rate may be a good place to start. As you gain confidence with the results, ramp up the percentage of requests being evaluated.
You can add context attributes to get the most from your results. The experiment context can be set to a Symbol-keyed Hash of data that is then made available in published results, e.g.:

experiment.context :user => user

Wrap-Up: Observe and Monitor Your Ruby App with Scientist

In this post, we explored how to use the Scientist gem to change, migrate, and refactor Ruby code in production.

We examined Scientist's origins in the Branch by Abstraction pattern, then dived into refactoring. Next, we saw how LabTech could help with gathering results and your Scientist configuration.

We then touched on some limitations of Scientist before finally outlining a few best practices.

You must observe and monitor what is happening in your system. Integrate Scientist into your development process to make critical changes in your Ruby code with greater confidence.

Happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!