Forem: Honeybadger

A guide to feature flags in Laravel using Laravel Pennant

Honeybadger Staff — Tue, 16 Apr 2024 07:00:00 +0000

This article was originally written by Ashley Allen on the Honeybadger Developer Blog.

As a developer, you want to be able to release code early and often. It helps you to get feedback from users, and it helps you avoid having to deal with large, complex Git merges. However, it's not always so straightforward.

What happens if you want to test a new feature you've been working on with a subset of users? Alternatively, maybe you want to hide a feature that isn't quite ready for production or enable a feature for a specific user or team on a specific subscription level.

This is where feature flags come in. They allow you toggle features in your applications based on given conditions.

In this article, we'll discuss what feature flags are, as well as the advantages and disadvantages of using them. We'll also take a look at how to use feature flags in Laravel using the first-party Laravel Pennant package. We'll then delve into testing feature flags in your Laravel applications.

By the end of this article, you should have a good understanding of what feature flags are and feel confident enough to start using them in your own applications.

What are feature flags?

Feature flags, sometimes referred to as "switches" or "feature toggles", are a concept in the software development world that involves toggling features on or off based on certain conditions at runtime. Generally, they allow you to programmatically toggle a feature without needing to redeploy an application. This might be based on a user’s subscription level or toggled by a user in an admin panel.

To provide an example of what a feature flag is, let's say you're building a blogging platform and working on a new social-sharing feature. This feature will allow users to share their blog posts directly to their social media platforms after publishing them. Imagine that you want to test it with a subset of users before releasing it to everyone. You could use a feature flag to toggle the feature on or off for a specific user or group of users. This means the feature will only be available to those users, and everyone else will not see it. After a phase of initial testing, if you're happy with the stability of the feature, you can then release it to everyone.

The advantages of feature flags

To get a better understanding of what feature flags achieve, let's take a look at some of the use cases and advantages of using them.

Enabling features for specific users or organizations

Imagine that you have a large enterprise application that offers three different features: email, chat, and video conferencing. You could use a feature flag for each of these features which only allows the users or organizations to access the features if they're enabled.

For example, you may have one organization using your application that only wants access to your email feature. Using feature flags in your code, you could hide the chat and video conferencing features from them.

You might also want to add an admin panel that your sales team (or similar) can use to enable these features at a later date if the organization decides they want to use them.

Similarly, if you have a software-as-a-service (SaaS) application that offers different subscription levels, you could use feature flags to enable or disable features based on the subscription level of the user. This means that as a user upgrades their subscription, they will automatically gain access to the features available at that level.

A/B testing

Another use case for feature flags is A/B testing. This is where you test two different versions of a feature to see which one performs better. For example, you might want to test two different versions of a landing page to see which one converts better.

You could use a feature flag to determine which version of the page to show to the user. Using an A/B testing tool or analytics platform, you could then assess which version of the page is the more optimized version.

Complements trunk-based development

You'll likely be used to using Git Flow as your branching strategy when working on projects. Let's quickly recap the basics of Git Flow. This involves having a master (or sometimes called main) branch that contains all the production-ready code that has been released or is due to be released. There is also a develop branch, which tends to contain all the work in progress and will be implemented in the next release. There may also be a staging branch that contains the code currently in the staging environment, such as a separate server where the application can be tested before release. Typically, before each release, changes from the develop or staging branch (depending on whether a staging branch is being used) will be merged into a release branch (such as release/v1.1.0) that will then be merged into the master branch once the release is ready.

Git Flow places a focus on "feature branches". This is where you'll create a new Git branch off the develop (or similar) branch called something like feature/auth-system. The feature branches typically focus on adding a particular feature and are only merged once the feature is production-ready and the team is comfortable with releasing it. These feature branches can exist for weeks or months, meaning that the code being added to them can become stale and out of date. This can lead to merge conflicts when you try to merge the feature branch back into the develop branch, which can be time-consuming and frustrating to deal with. In cases of large merge conflicts, it may also lead to bugs being introduced into the codebase if the conflicts aren't dealt with correctly.

Another issue is that if you have two separate feature branches, you have no idea how the changes implemented in each one will affect the other until they've been merged into the develop branch. This can then lead to the developers needing to make more changes to the code to fix any issues that arise from the merge.

Another branching strategy that proposes a different approach to Git Flow is trunk-based development (TBD). TBD proposes that you should only have a single master branch that contains all the production-ready code. It emphasizes creating short-lived branches that are merged into the master branch as soon as possible. The branches don't necessarily need to include an entire feature. Instead, they can be used to implement a small part of a feature.

For example, let's say you're building a profile page your application. You might follow this type of workflow:

Create a branch from master.
Add a basic profile page that allows you to update your profile name.
Merge back into master.
Create a branch from master.
Add the functionality to the profile page to update your profile picture.
Merge back into master.
Create a branch from master.
Add functionality to the profile page to enable two-factor authentication.
Merge back into master.
And so on...

As you can see, the branches are focused and short-lived. However, because the code is constantly being merged back into the master branch, it might be constantly leading to new code being released to production, even if the feature isn't ready yet. This is where feature flags can really complement the TBD workflow.

You could create a feature flag for a profile page that is disabled by default. This means you can merge as much profile page-related code into the master branch as you want, but the feature won't be available to users until you're confident that the profile page feature is production-ready. This means you can keep your branches short-lived and avoid merge conflicts and avoid having to release code that isn't ready yet. As soon as the feature is complete, you can then enable the feature flag and release it to everyone.

It's worth noting that while TBD can be a good approach to take, it's not necessarily suitable for all projects. It's important to consider the pros and cons of each branching strategy and choose the one that's right for your project. It requires you to have a very good-quality test suite and continuous integration (CI) process in place to reduce the chances of introducing bugs into your codebase or accidentally releasing code that isn't ready yet without hiding it behind a feature flag.

The disadvantages of feature flags

Although feature flags can be useful, they also have some disadvantages. Let's take a look at some of them.

Add extra complexity to your code

One of the main disadvantages of using feature flags is that they can add extra complexity to your code. Here is an example:

if (Feature::active('telecommunications')) {
    // Telecommunications is enabled...
} else {
    // Telecommunications is disabled...
}

In this example, we create two branches in our code's flow, thereby increasing the complexity of the code. Even though it is only a simple "if" statement, it means that we must now write tests for both branches to ensure that they work as expected. This can be time-consuming but is necessary to reduce the likelihood of bugs being introduced into our code. For example, if we only wrote tests for the "Telecommunications is enabled" branch, we wouldn't know if the "Telecommunications is disabled" branch was working as expected, and we might accidentally run some code that should only be run if the feature is enabled.

Additionally, as the feature grows, it's likely that you'll end up with a lot of conditions like this scattered around your application. Therefore, it's important to be careful and avoid adding too many feature flags, or you will end up with a lot of extra code that is only used in certain circumstances. Remember, the more conditionals your code has, the more difficult it will be to test your code.

Remembering to remove the feature flags

Continuing with our telecommunications feature, let's imagine that you are using feature flags to hide a feature from users while you are working on it. Once the feature is finished, you may want to leave it in for a short period of time. By doing so, it means you'll be able to disable the feature if any major bugs or issues are detected and until you're confident that it's stable enough to be released to everyone.

After you're happy with the stability of the feature, you'll need to remember to delete the feature flag. This might include removing the feature flag definition, removing the feature flag check from your code, and removing any tests written for the feature flag. If you forget to remove the feature flag, you'll end up with a lot of dead code in your application that isn't being used.

Feature flags in Laravel

Now that we have an understanding of what feature flags are, let's take a look at how we can use them in Laravel.

One way to implement feature flags in Laravel is to use Laravel Pennant. Pennant is a first-party package created and maintained by the Laravel core team. It provides a simple way to define and check feature flags in Laravel applications and comes with several handy features out of the box.

Let's dive into the benefits of using Pennant and how we can use it in our applications.

Benefits of using Pennant

After working on many Laravel projects in the past, I've seen that each team has its own approach to implementing feature flags; some better than others. Pennant provides a standardized way of implementing feature flags, meaning that if you move between Laravel projects, you'll be able to quickly get up to speed with how feature flags are implemented if they're using Pennant.

Additionally, Pennant encourages us to abstract the logic of the feature flag away using a consistent approach. It allows us to check the value of the feature flag without needing to know how or where the value is determined. This means that we can change the implementation of the feature flag without having to make changes to our application code. To demonstrate this point, let's take a look at an example.

Imagine that we have a "telecommunications" feature in our application and want to disable it for all users. To do this, we may want to store the value in a config field and access it in our code:

if (config('features.telecommunications.enabled')) {
    // Telecommunications is enabled...
} else {
    // Telecommunications is disabled...
}

There may be code like this throughout your application, including in controllers, middleware, views, service classes, and so on. Let's say we wanted to update this functionality so that the feature flag is stored in the database instead of a config file, or we want to enable it for a specific user. We'd need to go through our application and update all the places where we're checking the value of the feature flag and change them to match the new logic. This can be a time-consuming and error-prone process.

Instead, we could abstract the logic of the feature flag away using Pennant (we'll take a look at how to do this later). We can then check the value of the feature flag:

if (Feature::active('telecommunications')) {
    // Telecommunications is enabled...
} else {
    // Telecommunications is disabled...
}

As a result, we'll be able to change the logic used in the feature flag definition in a single place without needing to update our application code. This makes it much easier to maintain and update.

Installing Laravel Pennant

To get started with using Pennant, we'll first need to install the package using Composer by running the following command in our Laravel project's root:

composer require laravel/pennant

Once the package has been installed, we'll need to publish the package's config file and database migration using the following command:

php artisan vendor:publish --provider="Laravel\Pennant\PennantServiceProvider"

Running this command should create two new files:

config/pennant.php - The package's config file.
database/migrations/2022_11_01_000001_create_features_table.php - The package's database migration that will be used to create the features table in our database.

We can then run the database migration using the following command:

php artisan migrate

Your database should now have a new features table.

Drivers

Out of the box, Pennant supports two different drivers: array and database.

Both of these drivers are used to store the result after checking a feature flag. When a feature flag is first checked, the result will be calculated at runtime and then stored. This means that the next time the feature flag is checked, the result will be read from the storage driver instead of being calculated again. This helps to improve the performance of your application.

If you choose to use the array driver, the result will be stored in an in-memory array. This means that at the end of the request lifecycle, the result will be lost and will need to be recalculated on the next request.

If you choose to use the database driver, the result will be stored in the features table in your database. This means that the result will be persisted between requests in a centralized location. As a result, the database driver is ideal if your application is running on multiple servers or if you're using a serverless environment. Furthermore, if you're using the database driver, it reduces the need to calculate the result of the feature flag on every request, which can help to improve the performance of your application if you have any complex logic to determine the result of a flag. It's worth noting that even if you're using the database driver, Pennant will still make use of an in-memory cache to store the results of the feature flags. This means that if you need to check the value of a feature flag multiple times in a single request, it will only result in a single database query.

You can select the driver you'd like to use by setting it in the PENNANT_STORE environment variable. For example, if you wanted to use the array driver, you could set the following in your .env file:

PENNANT_STORE=array

Creating a feature flag

Now that we have an understanding of the drivers, let's take a look at how to create a feature flag.

You can create feature flag definitions like so:

use Laravel\Pennant\Feature;

Feature::define(
    'telecommunications',
    fn (): bool => config('features.telecommunications.enabled')
);

In this example, we're defining a feature flag called telecommunications that will return the value of the features.telecommunications.enabled config value. This means that if the config value is true, the feature flag will return true. If the config value is false, the feature flag will return false.

By default, Laravel passes in the authenticated user to the closure in your definition. You can access them by type hinting the App\Models\User class. For example, let's say you wanted to create a feature flag that only enables the telecommunications feature if the user is an administrator. You could do so like this:

use Laravel\Pennant\Feature;
use App\Models\User;

Feature::define(
    'telecommunications',
    fn (User $user): bool => $user->isAdmin(),
);

The logic defined in the closures can also become more complex if needed. For instance, imagine that you're redesigning your application's user interface (UI). You may want to display the new UI to the admins of your application, any users who have joined your beta program, or randomly selected users. You could do so like this:

use App\Models\User;
use Illuminate\Support\Lottery;
use Laravel\Pennant\Feature;

Feature::define(
    'new-ui',
    fn (User $user): bool|Lottery => match (true) {
        $user->isAdmin(), $user->hasJoinedBetaProgram() => true,
        default => Lottery::odds(0.5),
    }
);

In the code example, we're returning true from the closure if the user is an admin or has joined the beta program. Otherwise, we're returning an instance of Laravel's Lottery class, which will randomly determine true or false with a 50% chance of either.

To use these feature flags in your code, you'll need to register them in your Laravel application. You can do this by adding them to the boot method of a service provider. You may want to create your own service provider specifically for holding feature flag definitions (such as a FeatureServiceProvider). However, for the purposes of this example, we'll add them to the boot method of the app/Providers/AppServiceProvider.php file:

namespace App\Providers;

use App\Models\User;
use Illuminate\Support\Lottery;
use Illuminate\Support\ServiceProvider;
use Laravel\Pennant\Feature;


final class AppServiceProvider extends ServiceProvider
{
    // ...

    public function boot(): void
    {
        Feature::define(
            'new-ui',
            fn (User $user): bool|Lottery => match (true) {
                $user->isAdmin(), $user->hasJoinedBetaProgram() => true,
                default => Lottery::odds(0.5),
            }
        );
    }
}

Now that we've registered the feature flag definition, we'll be able to check its value in our code. We'll look at how to do this later in the article.

Class-based feature flags

As your feature flags become more complex, you may want to consider lifting the code out of the closure and into a class. This can help to keep your code clean and maintainable.

Continuing with our new-ui feature flag, let's take a look at how we could lift the code out of the closure and into a class.

To begin with, we can create a new feature flag class by running the following Artisan command:

php artisan pennant:feature NewUI

Running this command should create a new app/Features/NewUI.php file that looks like this:

namespace App\Features;

use Illuminate\Support\Lottery;

class NewUI
{
    /**
     * Resolve the feature's initial value.
     */
    public function resolve(mixed $scope): mixed
    {
        return false;
    }
}

We'll update the class to look like so:

namespace App\Features;

use App\Models\User;
use Illuminate\Support\Lottery;

class NewUI
{
    public function resolve(User $user): bool|Lottery
    {
        return match (true) {
            $user->isAdmin(), $user->hasJoinedBetaProgram() => true,
            default => Lottery::odds(0.5),
        };
    }
}

In the code example, we've moved the logic from the closure into the resolve method of the class. We've also type hinted the $user parameter as the App\Models\User class. You may have also noticed that we've added bool|Lottery as the return type. This specifies that the method will return either a bool or a Lottery instance.

Defining the feature flags inside classes like this removes the need to use the Feature::define() method in our code or to register the feature flag definition in a service provider.

Using feature flags

Now that we know how to create feature flag definitions, let's take a look at how we can use them in our code.

To check the value of a feature flag, we can use the Feature::active() method:

use Laravel\Pennant\Feature;

if (Feature::active('telecommunications')) {
    // Telecommunications is enabled...
} else {
    // Telecommunications is disabled...
}

There may be times when you want to check that at least one feature is active. To do this, we can use the Feature::someAreActive() method:

use Laravel\Pennant\Feature;

if (Feature::someAreActive(['telecommunications', 'email'])) {
    // At least one of the features is enabled...
} else {
    // None of the features are enabled...
}

Similarly, you may want to check that all the features are active. To do this, we can use the Feature::allAreActive() method:

use Laravel\Pennant\Feature;

if (Feature::allAreActive(['telecommunications', 'email'])) {
    // All the features are enabled...
} else {
    // Not all the features are enabled...
}

There may also be times when you want to check whether a feature is inactive. Although you could use the !Feature::active(), Pennant provides a useful Feature::inactive() method that helps to make your code read as plain English as possible. You can use this method like so:

use Laravel\Pennant\Feature;

if (Feature::inactive('telecommunications')) {
    // Telecommunications is disabled...
} else {
    // Telecommunications is enabled...
}

By default, when checking the value of a feature flag, the value will be checked for the authenticated user. However, there may be times when you want to check the value of a feature flag for a different user. To do this, you can use the Feature::for() method:

use App\Models\User;
use Laravel\Pennant\Feature;

// Find another user that we want to check the feature flag for...
$anotherUser = User::find(2);

if (Feature::for($anotherUser)->active('telecommunications')) {
    // Telecommunications are enabled for the given user...
} else {
    // Telecommunications are disabled for the given user...
}

If you're using a class-based feature flag, you can pass the class's name to the methods we've shown above:

use App\Features\NewUI;
use Laravel\Pennant\Feature;

if (Feature::active(NewUI::class)) {
    // New UI is enabled...
} else {
    // New UI is disabled...
}

Toggling feature flags

As we've briefly covered, the results of the feature flags are stored either in an in-memory array or the database depending on which driver you're using. This means that the logic for a feature flag will typically only be run once.

However, you may want to update these stored values to change the result of a feature flag. For example, let's say that after a user has upgraded their subscription, you want to enable a new feature for them. To do this, you can use the Feature::activate() method:

use Laravel\Pennant\Feature;

Feature::activate('telecommunications');

Similarly, imagine a user downgrades their subscription so you want to disable a feature. You can use the Feature::deactivate() method like this:

use Laravel\Pennant\Feature;

Feature::deactivate('telecommunications');

As mentioned previously, Pennant will scope the feature flag to the authenticated user by default. However, you may want to activate or deactivate a feature flag for a different user. For instance, your application may have an admin panel that allows employees at your company to enable features at the click of a button for specific clients after they've upgraded their subscription.

To change the scope of the feature flag and activate it for a different user, you can use the Feature::for() method, followed by activate or deactivate:

use App\Models\User;
use Laravel\Pennant\Feature;

// Find another user that we want to activate the feature flag for...
$anotherUser = User::find(2);

Feature::for($anotherUser)->activate('telecommunications');

If you've been building a feature for your application and have only activated it for a subset of users, you may want to activate it for everyone after you're happy with its stability. To do this, you can use the Feature::activateForEveryone() method:

use Laravel\Pennant\Feature;

Feature::activateForEveryone('telecommunications');

Likewise, you can call the Feature::deactiveForEveryone() method if you want to deactivate the method for everyone. You may want to do this if you come across a large bug in your code and want to prevent access to that feature until it's fixed. You can do so like this:

use Laravel\Pennant\Feature;

Feature::deactivateForEveryone('telecommunications');

Using feature flag middleware

Pennant ships with an EnsureFeaturesAreActive middleware class. You can use it to only allow access to given routes in your application if the feature flag is active. If the feature flag is not active, the middleware will return a 400 Bad Request response.

Let's take a look at an example of how we can use middleware. Imagine we have several routes in our application that are related to the telecommunications feature and that we only want to allow access to those routes if the feature is active. We can do so like this:

use App\Http\Controllers\TelecommunicationsController;
use Illuminate\Support\Facades\Route;
use Laravel\Pennant\Middleware\EnsureFeaturesAreActive;

Route::middleware([EnsureFeaturesAreActive::using('telecommunications')])->group(function () {
    Route::get('/telecommunications', [TelecommunicationsController::class, 'index']);

    // ...
});

In the code example, we've wrapped the telecommunications routes inside a route group using the group method. We've also applied the EnsureFeaturesAreActive middleware to that group and passed the name of the feature flag to the using method.

The EnsureFeaturesAreActive middleware requires that all the feature flags passed to the using method are active.

Here is an example:

use App\Http\Controllers\TelecommunicationsController;
use Illuminate\Support\Facades\Route;
use Laravel\Pennant\Middleware\EnsureFeaturesAreActive;

Route::middleware([EnsureFeaturesAreActive::using('telecommunications', 'new-ui')])->group(function () {
    Route::get('/telecommunications', [TelecommunicationsController::class, 'index']);

    // ...
});

In the example above, we've specified that the authenticated user must have both the telecommunications and new-ui feature flags active to access the routes inside the route group. If either of them are inactive, the middleware will return a 400 Bad Request response and prevent access.

Using feature flags in Blade views

Pennant also ships with a useful @feature Blade directive that allows you to check the value of a feature flag in your Blade views and render different content based on the result.

Let's take a look at what it might look like inside your Blade views:

@feature('telecommunications')
    Telecommunications is active
@else
    Telecommunications is inactive
@endfeature

In the code example above, we're using the @feature to check if the telecommunications feature is enabled. If it is, we'll render the "Telecommunications is active" text. Otherwise, we'll render the "Telecommunications is inactive" text.

A great use case for this directive is for only rendering a menu item in your application's navigation if the feature is active. For example, let's say you have a telecommunications feature in your application and only want to show the link for it in your navigation menu if the feature is active. You could do so like this:

@feature('telecommunications')
    <a href="{{ route('telecommunications.index') }}">Telecommunications</a>
@endfeature

Using the `HasFeatures` trait

Pennant ships with a handy Laravel\Pennant\Concerns\HasFeatures trait that you can add to your User models. This trait provides a features method on the model that you can then use to check the status of the feature flags for the user.

Using the features method gives us access to all the same methods that we'd have access to if we were using the Feature class directly. This means that we can use the active, inactive, someAreActive, allAreActive methods, and so on.

Let's take a look at how we can use it in our App\Models\User model.

We'll first need to add the Laravel\Pennant\Concerns\HasFeatures trait to our User model:

namespace App\Models;

use Illuminate\Foundation\Auth\User as Authenticatable;
use Laravel\Pennant\Concerns\HasFeatures;

class User extends Authenticatable
{
    use HasFeatures;

    // ...
}

If we wanted to check if a feature is active for our authenticated user, we could do so like this:

auth()->user()->features()->active('telecommunications');

Similarly, if we wanted to check if a feature is active for another user, we could do the following:

User::find(1)->features()->active('telecommunications');

Rich-value feature flags

Although you'll typically return Boolean values (true or false) from your feature flag definitions, Pennant also supports rich-value feature flags. This means that you can return any value from your feature flag definitions, such as a string or an array.

This can be particularly handy if you're using Pennant to implement some form of A/B testing. For instance, imagine that you have a button in your application that your users can click to upgrade their account to a higher tier. You might want to test different text for the button to see which one performs better.

So you may want to create a feature flag definition like this:

Feature::define(
    'upgrade-button-text',
    fn (User $user): bool => Arr::random([
        'Upgrade now',
        'Upgrade to PRO!',
    ])
);

In this feature flag definition, we're declaring an upgrade-button-text feature flag that will return a random string from the array in the closure.

We can then get the value of the feature flag by using the Feature::value() method. If we wanted to output the value of the feature flag in our Blade views, we could do so like this:

<a href="/upgrade">
    {{ Feature::value('upgrade-button-text') }}
</a>

Testing feature flags

Writing tests for your applications is an important part of the development process. It helps to give you confidence that your code works, reduces the likelihood of bugs, and makes it easier to make changes to your code in the future without fear of breaking the existing functionality.

When it comes to testing feature flags, there are two main approaches that you can use:

Write a single test that tests the feature flag definition and the code that uses the feature flag.
Write separate tests for the feature flag definition and write separate tests for the code that uses the feature flag.

Let's say that you choose option one and write a single test that tests the feature flag definition and the code that uses the feature flag, such as a controller. In the setup of your test, you need to make sure that the feature flag's conditions can be passed. For example, let's take the following Telecommunications feature flag definition:

namespace App\Features;

use App\Models\User;

class Telecommunications
{
    public function resolve(User $user): bool
    {
        return $user->isAdmin() || $user->has_joined_beta_program;
    }
}

If you wanted to write a test for a controller that was using this feature flag, you'd need to make sure that the user you created for your test is either an admin or has joined the beta program. In this case, it's a relatively simple feature, so it's easy to do. However, if the feature flag were more complex, it could be more difficult to set up the conditions for the feature flag to pass.

If the feature flag has a complex definition, and you're using it in multiple places in your code, it may lead to you writing a lot of duplicate setup code in your tests. This can be time-consuming and error-prone. It also means that if you need to update the feature flag definition, you'll need to update all the tests that use it.

You may want to use this approach for some mission-critical parts of your application, where you want to be as confident as possible that the feature flag is working as expected.

Another benefit of this approach is that you're never directly interacting with Pennant in your test code. This means that if you choose to use a different approach in the future for your feature flags (such as moving away from using Pennant and creating your own feature flag manager), you won't need to update your tests because you're testing that the behavior hasn't changed.

The other approach that you can take to test your application is to write separate tests for the feature flag definition and the code that uses the feature flag. This would involve writing tests for the feature flag definition itself and separate tests for the code that uses the feature flag. This allows you to test the feature flag definition in isolation and then mock the result of the feature flag in your tests for the code that uses it. As a result, this approach can be quicker to set up and can help to reduce the amount of duplicate code in your tests.

Let's take a look at how to use this approach.

Imagine that we're using our Telecommunications feature flag class defined above. We'll be using it with the EnsureFeaturesAreActive middleware to only allow access to the telecommunications routes if the feature flag is active. The routes may look something like so:

use App\Features\Telecommunications;
use App\Http\Controllers\TelecommunicationsController;
use Illuminate\Support\Facades\Route;
use Laravel\Pennant\Middleware\EnsureFeaturesAreActive;

Route::middleware([EnsureFeaturesAreActive::using(Telecommunications::class)])->group(function () {
    Route::get('/telecommunications', [TelecommunicationsController::class, 'index'])->name('telecommunications.index');

    // ...
});

For the purposes of this example, the TelecommunicationsController will be very basic and just return a view:

namespace App\Http\Controllers;

use Illuminate\Contracts\View\View;

class TelecommunicationsController extends Controller
{
    public function index(): View
    {
        return view('telecommunications.index');
    }

    // ...
}

We'll first write a test for our controller. We'll want to test that a 200 OK response is returned if the feature flag is active, and a 400 Bad Request response is returned if the feature flag is inactive.

Let's take a look at what the tests may look like and then we'll discuss what's being done:

namespace Tests\Feature\Http\Controllers\TelecommunicationsController;

use App\Features\Telecommunications;
use App\Models\User;
use Illuminate\Foundation\Testing\LazilyRefreshDatabase;
use Laravel\Pennant\Feature;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

class IndexTest extends TestCase
{
    use LazilyRefreshDatabase;

    #[Test]
    public function view_can_be_returned(): void
    {
        // Override the feature so we're forcing it to be enabled.
        Feature::define(
            feature: Telecommunications::class,
            resolver: true
        );

        $this->actingAs(User::factory()->create())
            ->get(route('telecommunications.index'))
            ->assertOk()
            ->assertViewIs('telecommunications.index');
    }

    #[Test]
    public function error_is_returned_if_the_feature_is_disabled_for_the_user(): void
    {
        // Override the feature so we're forcing it to be enabled.
        Feature::define(
            feature: Telecommunications::class,
            resolver: false
        );

        $this->actingAs(User::factory()->create())
            ->get(route('telecommunications.index'))
            ->assertBadRequest();
    }
}

In the first test (view_can_be_returned), we're overriding the Telecommunications feature flag using the Feature::define() method to force the value to always be true. This means that when the feature flag is checked when we make our request, we'll already have a true value stored for the user. This is simulating a scenario where the user satisfies a condition for the feature to be enabled. As a result, the 200 OK response will be returned with the correct view.

In the second test (error_is_returned_if_the_feature_is_disabled_for_the_user), we're doing the opposite and overriding the Telecommunications feature flag again to force the value to be false. This simulates a scenario where the user doesn't satisfy a condition for the feature to be enabled. As a result, the 400 Bad Request response will be returned.

Now that we've tested our controller that's using the feature flag, let's test the feature flag itself. We'll want to test the following scenarios:

true is returned if the user is an admin.
true is returned if the user has joined the beta program.
false is returned if the user is not an admin and has not joined the beta program.

Let's look at what our tests might look like and then discuss what's being done:

namespace Tests\Feature\Features\Telecommunications;

use App\Features\Telecommunications;
use App\Models\User;
use Illuminate\Foundation\Testing\LazilyRefreshDatabase;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

class HandleTest extends TestCase
{
    use LazilyRefreshDatabase;

    #[Test]
    public function true_is_returned_if_the_user_is_an_admin(): void
    {
        $user = User::factory()
            ->create([
                'role' => 'admin',
                'has_joined_beta_program' => false,
            ]);

        $this->assertTrue(
            (new Telecommunications())->resolve($user)
        );
    }

    #[Test]
    public function true_is_returned_if_the_user_has_joined_the_beta_program(): void
    {
        $user = User::factory()
            ->create([
                'role' => 'user',
                'has_joined_beta_program' => true,
            ]);

        $this->assertTrue(
            (new Telecommunications())->resolve($user)
        );
    }

    #[Test]
    public function false_is_returned_if_the_user_is_not_an_admin_and_has_not_joined_the_beta_program(): void
    {
        $user = User::factory()
            ->create([
                'role' => 'user',
                'has_joined_beta_program' => false,
            ]);

        $this->assertFalse(
            (new Telecommunications())->resolve($user)
        );
    }
}

For the purposes of this example, I've used a basic role field on the User model to determine if the user is an admin or not. However, in a real-world application, you'd likely have a more complex way of determining the user's role. For more details on how to do this, you can check out A Complete Guide to Managing User Permissions in Laravel Apps.

In the first test, we're creating a user that is an admin and has not joined the beta program. We're then creating a new instance of the Telecommunications feature flag class and calling the resolve method with the user. We're then asserting that the result returned is true.

In the second test, we're doing something similar and checking that the result is true if the user is not an admin but has joined the beta program.

In the final test, we're creating a user that has the role of user and has not joined the beta program. We're then asserting that the feature flag returns false for this user.

Using this approach is a great way to test your feature flags in isolation. If the feature flag definition changes in the future, you'll only need to update the test for the feature flag itself, rather than all the tests that use the feature flag. This helps to keep your test code clean and maintainable.

Conclusion

In this article, we've taken a look at what feature flags are, as well as the advantages and disadvantages of using them. We've also taken a look at how to use feature flags in Laravel using the first-party Laravel Pennant package. We then delved into how you can test feature flags in Laravel applications.

Hopefully, you should now have a good understanding of what feature flags are and feel confident enough to start using them in your own applications.

Options for passwordless authentication in Django apps

Honeybadger Staff — Thu, 11 Apr 2024 07:00:00 +0000

This article was originally written by Muhammed Ali on the Honeybadger Developer Blog.

Passwordless authentication is gaining popularity as a secure and user-friendly alternative to traditional password-based authentication in Django applications. This article will explore various options for implementing passwordless authentication in Django, highlighting their advantages, limitations, and considerations for choosing the right option.

In this article, we will delve into three passwordless authentication methods: email-based authentication, authentication with OAuth, and authentication with magic links.

For email-based authentication, the article will provide a step-by-step guide on how to implement it in Django. Next, we will walk through configuring Google APIs and setting up the necessary code and templates to enable Google OAuth authentication in a Django application.

Finally, this article will introduce authentication with magic links using the django-sesame library. It will explain how to send magic login links to users' emails and authenticate specific views.

Brief overview of traditional password-based authentication in Django

Traditional password-based authentication in Django follows a standard username-password approach. Users create an account with a unique username and password combination, which is stored securely in a database. This ensures that the user's credentials are protected, and hackers cannot gain access to sensitive information. When users attempt to log in, they provide their credentials, and Django verifies the entered password against the stored hash of the user's password. This verification process is crucial in ensuring that only authorized users can access the platform.

Limitations and drawbacks of password-based authentication

While password-based authentication has been widely used, it is not without limitations and drawbacks. Here are some common challenges associated with this approach:

Password Management Issues: Users often struggle with password management, including remembering multiple passwords or creating strong and unique passwords. This can lead to password reuse across different platforms, increasing the risk of security breaches if one account is compromised.
Vulnerability to Phishing Attacks: Password-based authentication is susceptible to phishing attacks, where malicious actors trick users into providing their passwords on fake login pages. Users may unknowingly divulge their credentials, compromising their accounts' security.
Password Database Vulnerabilities: Storing passwords securely is crucial, but if not implemented properly, there can be vulnerabilities in the password storage mechanism. Inadequate hashing algorithms, lack of salting, or improper encryption can expose passwords to unauthorized access in case of a data breach.
Forgotten Passwords and Reset Procedures: Users frequently forget their passwords, leading to frustration and creating additional workload for support teams. Password reset procedures, such as email verification or security questions, can also introduce vulnerabilities if not properly implemented.
User Experience and Friction: Password-based authentication can create friction for users, especially when they need to remember complex passwords or go through lengthy login processes. Cumbersome login experiences can lead to user abandonment or dissatisfaction.

First, we will create a Django project, and then run migrations to set up the user authentication table. Run the following command to start a Django project:

django-admin startproject project

Navigate to the root of your project and run the following command to apply the migration for authentication:

python manage.py migrate

Email-based authentication

One popular passwordless authentication method is email-based authentication. Instead of passwords, users receive one-time login links via email. This approach eliminates the need for users to remember passwords and mitigates the risk of password-related vulnerabilities. We'll explore the implementation process of generating and verifying tokens for authentication, along with security considerations to ensure the integrity of an email-based authentication system.

For the illustration, we will have an application that authenticates the login of a particular user using a link sent to the user’s email. The message will contain a randomly generated link that can be used only once for login.

To start, go to project/settings.py and paste the following configuration to send an email from the Django app. Since this is in development, we will print out the email on the terminal.

# Email Backend Configuration (for development)
EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'  # Print emails to the console for testing

# Your SMTP email configuration (for production)
# EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
# EMAIL_HOST = 'your-smtp-host'
# EMAIL_PORT = 587
# EMAIL_USE_TLS = True
# EMAIL_HOST_USER = 'your-smtp-username'
# EMAIL_HOST_PASSWORD = 'your-smtp-password'

Now create a new file named project/views.py and paste the following code to send the email containing the login link.

from django.shortcuts import render, redirect
from django.contrib.auth.tokens import default_token_generator
from django.contrib.auth import get_user_model, login
from django.contrib.sites.shortcuts import get_current_site
from django.utils.http import urlsafe_base64_encode, urlsafe_base64_decode
from django.utils.encoding import force_bytes, force_str
from django.template.loader import render_to_string
from django.core.mail import send_mail
from django.contrib.auth.decorators import login_required

User = get_user_model()

def send_login_link(request):
    if request.method == 'POST':
        email = request.POST.get('email')
        user = User.objects.get(email=email)

        if user:
            # Generate a one-time use token for the user
            token = default_token_generator.make_token(user)

            # Create a unique link for the user to log in
            uid = urlsafe_base64_encode(force_bytes(user.pk))
            domain = get_current_site(request).domain
            login_link = f'http://{domain}/login/{uid}/{token}/'
            print("login,", login_link)

            # Send the login link via email
            subject = 'Your Login Link'
            message = render_to_string('login_link.html', {'login_link': login_link})
            from_email = 'noreply@example.com'
            send_mail(subject, message, from_email, [email])

        return render(request, 'email_sent.html')
    return render(request, 'send_login_link.html')

The code above checks if the request method is 'POST' using request.method == 'POST'. If it is a POST request, it retrieves the user's email address from the request using request.POST.get('email'). It then retrieves the user object associated with the provided email address using User.objects.get(email=email). If a user is found, the code proceeds to generate a one-time use token for the user using default_token_generator.make_token(user). It creates a unique login link for the user by encoding the user's primary key using urlsafe_base64_encode(force_bytes(user.pk)), gets the current site's domain using get_current_site(request).domain, and appends the token to the URL. The login link is then sent to the user via email using the send_mail function. The email contains a subject, rendered message from the login_link.html template, the sender's email address, and the recipient's email address. Finally, it renders the email_sent.html template and returns it as a response.

Notice that some HTML files were used in the code. Create them (i.e., project/templates/login_link.html, project/templates/email_sent.html, project/templates/send_login_link.html) and paste in the following:

project/templates/login_link.html will display the email for the login link:

<!-- login_link.html -->
<p>Hello,</p>
<p>Click the link below to log in to our website:</p>
<p>{{ login_link }}</p>
<p>If you did not request this login link, please ignore this email.</p>
<p>Best regards,</p>
<p>Your Website Team</p>

project/templates/email_sent.html will display default message when email has been sent:

<!-- email_sent.html -->
<p>An email with the login link has been sent to your email address. Please check your inbox.</p>

project/templates/send_login_link.html will display the form for the user to submit their email:

<!-- send_login_link.html -->
<form method="post">
    {% csrf_token %}
    <input type="email" name="email" placeholder="Enter your email" required>
    <button type="submit">Send Login Link</button>
  </form>

Now we will work on the view that will handle the authentication when the login link is clicked. In the project/views.py file, paste the following code:

# ...
def login_with_link(request, uidb64, token):
    try:
        uid = force_str(urlsafe_base64_decode(uidb64))
        user = User.objects.get(pk=uid)
    except (TypeError, ValueError, OverflowError, User.DoesNotExist):
        user = None

    if user and default_token_generator.check_token(user, token):
        # Log the user in without requiring a password
        user.backend = 'django.contrib.auth.backends.ModelBackend'
        login(request, user)

        return redirect('user_email')  # Replace 'home' with the URL to redirect after login

    return render(request, 'login_link_invalid.html')

When the function is called, it first attempts to decode the uidb64 parameter, which is a base64-encoded string representing the user's primary key. The force_str function is used to convert the decoded byte string into a Unicode string. This decoded value is assigned to the variable uid.

Next, the code attempts to retrieve the user from the user model using the User.objects.get(pk=uid) statement. It uses the decoded uid as the primary key to fetch the corresponding user object. If the user does not exist or if any exceptions occur during the decoding or retrieval process, the user variable is set to None.

After retrieving the user, the code checks two conditions to determine whether the login is valid. First, it verifies that the user object exists. Then, it uses the default_token_generator.check_token(user, token) function to validate the provided token against the user.

If both conditions are met and the user is valid, the code proceeds to log in the user without requiring a password. To do this, it sets the authentication backend for the user to the default model backend using the statement user.backend = 'django.contrib.auth.backends.ModelBackend'. This step ensures that the user's authentication is handled correctly by Django's authentication system.

The project/templates/login_link_invalid.html fill will display the error message when the link is invalid or is used more than once:

<p>The login link is invalid. Please try again.</p>

Finally, for the views, we will create a view that will display the user’s email when they have successfully logged in. In the project/views.py file, paste the following code:

@login_required
def user_email(request):
    email = request.user.email
    return render(request, 'user.html', {'email': email})

Now paste the following in project/templates/user.html. The code below receives the email context from the user_email view and displays it if the user is authenticated.

<!-- user_email.html -->

  {% if user.is_authenticated %}
  <p>Your Email: {{ email }}</p>
  {% endif %}

Now update the urls.py file to accommodate the views we have just created:

from django.contrib import admin
from django.urls import path
from . import views #new 

urlpatterns = [
    path('admin/', admin.site.urls),
    path('send-login-link/', views.send_login_link, name='send_login_link'), # new
    path('login/<str:uidb64>/<str:token>/', views.login_with_link, name='login_with_link'), #New
    path('user-email/', views.user_email, name='user_email'), #new
]

Now we will let Django know where the templates are located. You can do this in the settings.py file. Update TEMPLATES.DIRS with the following code:

'DIRS': [BASE_DIR / 'project/templates'],

To log in with an email, we need to be registered. I will use the Django superuser for this illustration. You can create a superuser by running the following command and filling in the prompt.

python manage.py createsuperuser

Now you can run the server with the following command and go to http://127.0.0.1:8000/send-login-link/ and submit the email you just created.

python manage.py runserver

You will then receive the email on your terminal containing the login link.

Note: The rest of the tutorial will start from this repo.

Authentication with OAuth in Django

Open Authorization (OAuth) is an open standard protocol that enables secure authorization and delegated access to resources on behalf of a user. Common use cases of OAuth include social media login integration, accessing APIs of service providers, and single sign-on (SSO) implementations.

When it comes to integrating third-party login providers, OAuth has emerged as a widely adopted protocol. Implementing OAuth for user authentication offers numerous benefits, including a simplified user experience, enhanced security, and access to user profile information. In this comprehensive guide, we will dive into the practical steps required to integrate OAuth into a Django application using the django-allauth package. By following along, you will gain background knowledge on how to leverage OAuth providers, such as Google, Facebook, or GitHub, to authenticate users and access their profile information securely.

In this section, we will set up the application to authenticate with Google, but it is almost the same process for other providers. To use OAuth in Django, you need to install django-allauth. Do this by running the following command:

pip install django-allauth

Now add the following to the INSTALLED_APPS list in the settings.py file. The code lets Django know about the django-allauth library. 'allauth.socialaccount.providers.google' indicates that you want to use Google for authentication. You can check the documentation to see other options available for authentication.

INSTALLED_APPS = [
#  ...

    'django.contrib.sites',
    'allauth',
    'allauth.account',
    'allauth.socialaccount',
    'allauth.socialaccount.providers.google',
]

Now add the following configuration at the end of your settings.py file. The AUTHENTICATION_BACKENDS setting specifies the authentication backends to be used. The 'django.contrib.auth.backends.ModelBackend' and 'allauth.account.auth_backends.AuthenticationBackend' backends allow users to authenticate using both the traditional Django authentication system and social media accounts, such as Google. The SOCIALACCOUNT_PROVIDERS setting focuses on configuring the Google OAuth provider. The SITE_ID setting represents the ID of the current site in a multi-site Django setup. The LOGIN_REDIRECT_URL and LOGOUT_REDIRECT_URL settings define the URLs to redirect users after successful login and logout operations.

AUTHENTICATION_BACKENDS = [
    'django.contrib.auth.backends.ModelBackend',
    'allauth.account.auth_backends.AuthenticationBackend',
]

SOCIALACCOUNT_PROVIDERS = {
    'google': {
        'SCOPE': [
            'profile',
            'email',
        ],
        'AUTH_PARAMS': {
            'access_type': 'online',
        }
    }
}
SITE_ID = 1

LOGIN_REDIRECT_URL = '/'
LOGOUT_REDIRECT_URL = '/'

Now paste the following in project/templates/home.html. The following code checks if the user is authenticated (i.e., logged in) and then displays a welcome message with the username. Otherwise, it provides a link labeled "Login with Google" that, when clicked, redirects the user to the Google OAuth login page. The {% provider_login_url 'google' %} expression generates the appropriate URL for initiating the authentication flow with Google provided by the django-allauth library.

{% load socialaccount %}
<html>
<body>
<h1>My Google OAuth Project </h1>
{% if user.is_authenticated %}
  <p>Welcome, You are logged in as {{ user.username }}</p>
{% else %}
  <a href="{% provider_login_url 'google' %}">Login With Google</a>
{% endif %}
</body>
</html>

Now we will configure the URLs needed to access the application and log in. To do this, paste the following code in your urls.py file.

from django.urls import path, include
from django.views.generic import TemplateView
from django.contrib.auth.views import LogoutView

urlpatterns = [
#...
path('', TemplateView.as_view(template_name="home.html")),
path('accounts/', include('allauth.urls')),
path('logout', LogoutView.as_view()),
]

'accounts/': This path is used to handle URLs related to user accounts. It includes the URLs provided by the django-allauth library, which is commonly used for user authentication and account management in Django.
'logout': This path is used to handle the URL for user logout functionality.

Next, we will run the migration command to apply the new models from django-allauth to the database:

python manage.py migrate

Configure the Google API

Now we will configure a new Google APIs project. We need to do this so that Google will be aware of our account and grant access to their API. Start by going to the Google Developer APIs Console and creating a new project. Next, click the “Credentials” tab on the left-hand side. Here, you will set credentials to request user consent so that the app can access user data. Follow the image below to set this up.

You will now be prompted to a page to select the application type. Select “web application” and add the following:

http://127.0.0.1:8000 under ”Authorized JavaScript origins”.
http://127.0.0.1:8000/accounts/google/login/callback/ under ”Authorized redirect URIs”.

Then, click the “CREATE” button, and you will be provided with the Client ID and Client Secret. Save this information, as you will need it later.

Run application

Before using the authentication method, we need to add a Site for the domain and OAuth provider (in this case, Google) to the admin.

To access the admin, run the following command to create a superuser:

python manage.py createsuperuser

Start the server:

python manage.py runserver

Open http://127.0.0.1:8000/admin and log into Django Admin with the details you just created. On the ”Sites" tab, click ”Add” and input 127.0.0.1:8000 as the Domain name and Display name.

Now we will add our Google OAuth credentials to the site. On the “Social Applications“ tab, click ”Add” and input the following:

Provider: Google
Name: “Passwordless Django” (You can use anything here.)
Client id: <client-ID-from-Google>
Client secret: <client-secret-from-Google>
Sites: 127.0.0.1:8000

Now log out of the admin account and go to the home page 127.0.0.1:8000. You will be able to access the application and log in with Google accounts.

If you need more information about your users from Google, there is a process that your app needs to go through to be verified. This process is outlined in detail in the OAuth Client Verification docs. Once your app is verified, you can access a wider range of user information.

Authenticating with magic links

Similar to email-based authentication, a link (magic link) will be sent to the user’s email address for logging in. Additionally, you can use magic links to authenticate a specific view instead of the entire application. For instance, suppose you have an employment website. You could grant job seekers access to the list of available jobs, but restrict their ability to apply until they request a link for application (magic link).

In this section, we will use a tool called Django-sesame for this. Since we have seen how the link for authentication works (you can learn this in the docs), I will focus on authenticating a specific view.

We will start by installing Django-sesame:

pip install django-sesame

Navigate to the root of your project and run the following command to apply the migration for authentication:

python manage.py migrate

Go to project/settings.py and paste the following configuration to send an email from the Django app and set Django-sesame as the authentication backend.


AUTHENTICATION_BACKENDS = [
    "django.contrib.auth.backends.ModelBackend",
    "sesame.backends.ModelBackend",
]

EMAIL_BACKEND = 'django.core.mail.backends.console.EmailBackend'  # Print emails to the console for testing

Now create a new file project/views.py and paste the following code to send the email containing the authentication link for a secret page.

from django.shortcuts import render
from django.contrib.auth.models import User
from django.urls import reverse
import sesame.utils
from .forms import EmailForm
from sesame.decorators import authenticate
# Create your views here.
def index(request):

    if request.method == 'POST':
        form = EmailForm (request. POST) #th
        if form.is_valid ():
            #here we, want to email the user the magic link
            # first check, user exists w/ that email

            email = form.cleaned_data['email']
            user = User.objects.filter(email=email).first()
            if user is None:
                #we can add an error here.
                return render(request, 'index.html', {'form':form})

            link = request.build_absolute_uri(reverse('secret-page'))
            print (link)

            link += sesame.utils.get_query_string(user) #add token to link (http://127.0.0.1:8000/secret/)
            user.email_user(
                subject = 'Your magic login link',
                message = f"""Dear {user.username} - Authenticate secret page with the link: {link}"""
            )

    context = {"form": EmailForm()}
    return render(request, 'index.html', context)           

@authenticate
def secret_page(request):
    return render(request, 'secret_page.html')

The index function is the view that handles the HTTP requests to the root URL ("/") of the application. It expects both GET and POST requests. If the request method is POST, it will validate a form named EmailForm, which will be created later. This form that will accept the user's email address.

If the form is valid, it retrieves the email field value and checks if a User object with that email exists. If a user is found, it generates a magic login link using request.build_absolute_uri() and sesame.utils.get_query_string(), and then sends an email to the user containing the link. The link that is sent only authenticates the view with the [@authenticate](https://django-sesame.readthedocs.io/en/stable/howto.html#view-decorator) decorator. Thus, if a view has a @login_required decorator, this link will not be able to access such a view.

The secret_page function is decorated with @authenticate, indicating that the user must be authenticated to access this view. When accessed, it renders the secret_page.html template.

Now we will create the EmailForm form. In project/forms.py, paste the following code:

from django import forms

class EmailForm(forms.Form):
    email = forms.EmailField()

Notice in the view code that some HTML files were used; create them (project/templates/index.htmlandproject/templates/secret_page.html) and paste in the following.

In project/templates/index.html, paste the following:

<form action="{{request.path}}" method="POST">
{% csrf_token %}
<p>
    {{ form }}
    <input type="submit" value="Send log in link">
</p>
</form>

In project/templates/secret_page.html, paste the following:

<h1>This is a secret page </h1>

Finally, you can update the project/urls.py file with the following code:

from django.contrib import admin
from django.urls import path
from . import views

urlpatterns = [
    path('admin/', admin.site.urls),
    path('', views.index, name='index'),
    path('secret-page/', views.secret_page, name="secret-page"),
]

Enter the following code to test run the migration command:

python manage.py migrate

Then, enter the following code to create a superuser:

python manage.py createsuperuser

Finally, run the server and of to http://127.0.0.1:8000/ on your browser and input the registered user. Then, the email with the access link will be sent to your command line.

Conclusion

This article provided a comprehensive overview of passwordless authentication options in Django, allowing developers to choose the most suitable method for their applications based on security requirements and user experience considerations. By implementing passwordless authentication, developers can enhance the security and usability of their Django applications.

Selecting the most suitable passwordless authentication option for your Django app depends on various factors, such as the app's security requirements, user experience goals, technical complexity, and integration capabilities. Evaluating the strengths and weaknesses of each option will help you make an informed decision that aligns with your specific needs.

The character encoding cheat sheet for JS developers

Honeybadger Staff — Wed, 10 Apr 2024 07:00:00 +0000

This article was originally written by Samson Omojola on the Honeybadger Developer Blog.

As a JavaScript developer, you work with text data all the time. However, computers don't naturally understand text in the same way as humans. Instead, text data must be converted into binary data, which computers can understand and manipulate. Character encoding is the process of mapping characters to binary data so that computers can work with text. Character encoding is important because it enables text to be stored and transmitted electronically. It is used in a wide range of applications, from web development to data storage and communication. For different computers and devices to work together, they must use the same encoding scheme. In the context of JavaScript development, character encoding is particularly important, as JavaScript is often used to manipulate and display text data in web applications.

In this comprehensive article, we will explore character encoding in JavaScript, including both Node.js and the browser side. We will start by providing an introduction to character encoding and the Unicode character encoding standard, which has become the de facto standard for encoding text data in modern computing. Next, we will take a closer look at ASCII encoding, the encoding standard that preceded Unicode, and how Unicode has largely replaced it. We will then dive into UTF-8 encoding, which is the most widely used character encoding standard in JavaScript and other modern programming languages. By the end of this article, you will have a solid understanding of character encoding in JavaScript and be equipped with the knowledge necessary to effectively work with text data in your JavaScript applications. Let's get started!

Understanding Unicode

Unicode is a character encoding standard developed to solve the problem of previous encoding schemes being unable to represent all characters in all languages. It provides a unique number, called a code point, for each character, symbol, and emoji in every language.
Unicode has become the de facto standard for character encoding in modern computing and has largely replaced older encoding standards, such as ASCII. To work with Unicode in JavaScript, it's important to understand its structure and how it differs from other encoding standards.

ASCII encoding

The American Standard Code for Information Interchange (ASCII) was the first widely used character encoding standard. It was developed in the 1960s and based on the English alphabet and some common punctuation marks. However, ASCII had several limitations, including its inability to represent characters from other languages and limited character set. As a result, it has largely been replaced by Unicode. While you may not encounter ASCII much in your JavaScript development, it's important to be aware of its history and limitations to better understand how character encoding has evolved over time.

ASCII

UTF-8 encoding

The character encoding standard UTF-8 is based on Unicode. It is the most widely used character encoding standard in modern computing, including in JavaScript. UTF-8 utilizes variable-length encoding, which means that it uses varying numbers of bytes to represent different characters. This allows it to represent all characters in all languages while being efficient in terms of storage and transmission. In JavaScript, UTF-8 is used to encode text data in strings and other data structures. Understanding how UTF-8 works is essential for working with text data in JavaScript applications.

UTF-8

UTF-16 encoding

UTF-16 is another character encoding standard that is based on Unicode. It was designed to utilize fixed-length encoding, with each character represented by two bytes. While UTF-16 is still used in some applications, it is less common than UTF-8. One advantage of UTF-16 is that it is more efficient for some languages, such as Chinese and Japanese, which have a large number of characters. However, a disadvantage of UTF-16 is that it requires more storage space than UTF-8 for most text data. Additionally, some operating systems and applications don’t support UTF-16, which can cause compatibility issues. In JavaScript, UTF-16 is used in some contexts, such as when working with the DOM or certain APIs. However, UTF-8 is generally preferred for most text data.

It’s worth mentioning that in a web browser, character encoding is typically handled using the HTML meta tag. The meta tag is included in the head section of each HTML page to ensure that non-ASCII characters are displayed correctly.

<!DOCTYPE html>
<html>
<head>
  <meta charset="utf-8">
  <title>My Web Page</title>
</head>
<body>
  <p>こんにちは</p>
</body>
</html>

However, JavaScript can be used to dynamically load or generate content on a webpage, such as text or images. In these cases, it may be necessary to specify the type of character encoding for the dynamically generated content, which can’t be done with a meta tag.

const string = 'こんにちは、世界！'; // Japanese string
const encodedString = encodeURIComponent(string);
console.log(encodedString);

In this example, we start by creating a string variable called string that contains a Japanese greeting, "こんにちは、世界！", which means "Hello, world!" in English. We then use the encodeURIComponent() function to encode the string variable as a URI component using UTF-8 encoding. This function encodes each character in the string as a sequence of one to four bytes, depending on its Unicode code point. Finally, we log the encodedString variable to the console using the console.log() function, which will output the URI-encoded version of the original string to the console:

%E3%81%93%E3%82%93%E3%81%AB%E3%81%A1%E3%81%AF%E3%80%81%E4%B8%96%E7%95%8C%EF%BC%81

In this output, each non-ASCII character in the original string is represented as a sequence of percent-encoded bytes, which allows the string to be transmitted over the internet or stored in a database without losing any information. The % symbol is followed by the two-digit hexadecimal representation of each byte in the encoded sequence.

To decode a URI-encoded string in JavaScript, you can use the decodeURIComponent() function. It converts the percent-encoded bytes back into their original characters.

const decodedString = decodeURIComponent(encodedString);
console.log(decodedString);

In the code above, the variable encodedString contains the URI-encoded version of the Japanese greeting string. We use the decodeURIComponent() function to decode the percent-encoded bytes in encodedString back into their original characters using UTF-8 encoding. This function reverses the process of URI encoding and returns the decoded string. Finally, we log the decodedString variable to the console using the console.log() function, which will output the original Japanese greeting string:

こんにちは、世界！

In this output, the decoded string is identical to the original Japanese greeting that we started with, which confirms that the encoding and decoding process worked correctly.

Ajax requests

You can also send and receive data using AJAX. In this case, it's important to specify the character encoding used by the server. This can be done using the charset option of the Content-Type header:

const xhr = new XMLHttpRequest();
xhr.open('GET', '/data', true);
xhr.setRequestHeader('Content-Type', 'application/json; charset=utf-8');
xhr.onload = () => {
  const data = JSON.parse(xhr.responseText);
  console.log(data);
};
xhr.send();

Browser forms and cookies

Working with cookies and form data in a web browser also requires handling character encoding. Cookies should be URL-encoded to ensure that they can be properly transmitted and decoded by the server. Form data can be sent using either the GET or POST methods, and the character encoding should be specified using the enctype attribute of the form:

document.cookie = `name=${encodeURIComponent('John Doe')}; expires=${expires}; path=/`;
<form action="/submit" method="post" enctype="application/x-www-form-urlencoded">
  <input type="text" name="name" value="John Doe">
  <button type="submit">Submit</button>
</form>

Character encoding in Node.js

Node.js is a popular JavaScript runtime that allows you to build server-side applications using JavaScript. When working with text data in Node.js, it's important to understand how character encoding works. In Node.js, text data is typically represented as strings. However, in JavaScript, strings are encoded using the UTF-16 standard, which can cause issues when working with other character encodings. When reading and writing text files in Node.js, you can specify the type of character encoding using the fs module. For example, to read a file as UTF-8, you can use the following code:

const fs = require('fs');

fs.readFile('myfile.txt', 'utf-8', (err, data) => {
  if (err) throw err;
  console.log(data);
});

In the code above, we are using the Node.js built-in module fs, which stands for "file system", to read the contents of a file called myfile.txt. We start by importing the fs module using the require function and assigning it to a variable called fs. Then, we use the readFile method provided by the fs module to read the content of myfile.txt. This method takes three arguments: the path to the file to be read (in this case, myfile.txt), the encoding to be used to read the file (in this case, 'utf-8'), and a callback function to be called when the file has been read.

The callback function takes two parameters: an err parameter, which contains an error object if an error occurred while reading the file, and a data parameter, which contains the content of the file as a string if the file was successfully read. In the callback function, we first check if an error occurred while reading the file by checking whether the err parameter is null. If an error occurred, we throw the error object; otherwise, we log the content of the file to the console using the console.log function.

When working with HTTP requests and responses in Node.js, you can also specify the type of character encoding using the Content-Type header. For example, to send a response as UTF-8, you can use the following code:

const http = require('http');

http.createServer((req, res) => {
  res.writeHead(200, { 'Content-Type': 'text/html; charset=utf-8' });
  res.write('Hello, 世界!');
  res.end();
}).listen(3000);

The code above sends a response: ‘Hello’. Note that in the Content-Type header, we specify the character encoding of the response.

When working with databases in Node.js, you may also need to specify the character encoding for the connection. For example, when using the mysql module, you can specify the character encoding in the connection options:

const mysql = require('mysql');

const connection = mysql.createConnection({
  host: 'localhost',
  user: 'me',
  password: 'secret',
  database: 'mydb',
  charset: 'utf8mb4' //we specify the character encoding for the mysql connection here
});

connection.connect();

Finally, it's important to be aware of character encoding errors in Node.js. When working with text data, it's possible to encounter errors due to incorrect encoding or malformed data. To handle these errors, you can use the iconv-lite module, which provides utilities for encoding and decoding text data:

const iconv = require('iconv-lite');
const buf = iconv.encode('Hello, world!', 'utf-8');
const str = iconv.decode(buf, 'utf-8');
console.log(str); // Output: Hello, world!

In the code above, we are using the iconv-lite library to encode and decode a string using the utf-8 character encoding. First, we import the iconv-lite library using the require function and assign it to a variable called iconv. Then, we create a new Buffer object called buf by encoding the string 'Hello, world!' using the utf-8 encoding with the iconv.encode function. Next, we create a new string called str by decoding the buf buffer using the utf-8 encoding with the iconv.decode function. Finally, we log the str string to the console using the console.log function.

Best practices for character encoding

When working with character encoding in JavaScript, there are a few best practices to keep in mind:

Use UTF-8 encoding for all text data. This is the most widely used and compatible encoding standard, and it ensures that your data can be properly read and displayed across various platforms and languages.
Avoid using non-ASCII characters in variable and function names. While this is technically allowed in JavaScript, it can cause encoding mismatches and make your code harder to read and understand.
When concatenating strings, always use the + operator or template literals (${}) instead of the += operator. This can prevent encoding mismatches and ensure that your code works correctly with non-ASCII characters.
When handling user input, always validate and sanitize the data to prevent encoding attacks and other security vulnerabilities. This can be done using, for example, the OWASP ESAPI or DOMPurify library.
When working with external APIs or databases, always verify the expected character encoding and convert your data to UTF-8 if necessary. This can prevent encoding mismatches and ensure that your data is properly stored and retrieved.

Debugging character encoding issues

Debugging character encoding issues can be tricky, but several techniques can be used to identify and fix these issues. One common issue is encoding mismatches, where data is encoded in one format but decoded in another. This can lead to garbled or corrupted data and may be difficult to identify.

One approach to debugging encoding issues is to use a tool like the UTF-8 Validator, which can help identify common issues with encoding. Additionally, you can use console.log statements to output the encoded and decoded data, and then compare them to see if there are any differences.
Another helpful technique is to use a tool like the iconv library, which can convert data between different character encodings. This can be particularly helpful when working with data from external sources that may use a different encoding than your application.

Conclusion and further reading

In this article, we've covered the basics of character encoding in JavaScript, including the different encoding standards, how they work, and how to work with them in Node.js and web browsers. We've also covered some best practices for working with character encoding in JavaScript and provided tips and techniques for debugging encoding issues.
If you want to learn more about character encoding in JavaScript, there are several resources. The Unicode Consortium's website provides detailed information about the Unicode standard, while the Mozilla Developer Network has extensive documentation on character encoding in JavaScript. Additionally, there are several books on JavaScript that cover this topic in depth, such as "JavaScript: The Definitive Guide" by David Flanagan and "Eloquent JavaScript" by Marijn Haverbeke.

Building a Sinatra app in Ruby

Honeybadger Staff — Tue, 09 Apr 2024 07:00:00 +0000

This article was originally written by Aestimo Kirina on the Honeybadger Developer Blog.

Sinatra is a lightweight web application framework written in Ruby. It provides a simple and easy-to-use syntax for building web applications. The framework focuses on being minimalistic, allowing developers to quickly create web applications without having to deal with a lot of the boilerplate code and relatively rigid way of doing things that accompany larger and more popular frameworks like Rails.

In this article, we'll use the Sinatra framework to build a simple URL-shortening app where a user can input a long URL and generate a short URL that redirects to the original URL when clicked. The app will also have a feature for keeping track of how many times a shortened URL is clicked.

Takeaways

By following this tutorial, you'll learn the following:

What Sinatra is and why you should use it
How to leverage its powerful routing features
How to work with a database
How to Add views.

Pre-requisites

To follow along with this tutorial, it's necessary to have the following:

Ruby installed on your development environment. Head here in case you don't have it installed and need a step-by-step guide. We'll be using Ruby 3.2.2 for this tutorial.
A Sinatra installation. If you don't have Sinatra installed, don't worry; we'll cover this in this tutorial.
Some basic experience with using Ruby.

Now, let's get started by understanding why we need Sinatra in the first place.

Why Sinatra?

With such full-featured Ruby frameworks as Ruby on Rails around, you might be wondering why anything else would be needed for your app development needs. Although Rails can handle almost anything you could throw at it, there are instances where having something else makes a lot of sense. Specifically, Sinatra is a great choice for app development because:

It's lightweight: Sinatra has very minimal dependencies, which makes it very lightweight compared to something like Rails. This makes it an easy framework to wrap your head around and allows you to have more control over how you architect your app’s structure.
It has powerful routing features: Sinatra’s routing system is flexible and expressive, allowing you to easily define handlers for common HTTP verbs, such as GET, POST, PUT, and DELETE. With Sinatra, you can also define custom and dynamic routes using route patterns, named parameters, and route segments, which can help you build powerful web applications, especially microservices and RESTful APIs.
It's modular: Sinatra is designed to be very simple and minimalistic. However, the framework follows a modular design, which allows you to pick and choose the components that meet your specific needs. If you need features like authentication and session management, Sinatra supports middleware that will take care of these and more. This allows for extreme developer flexibility that a more popular framework like Ruby on Rails might not give you.
It leverages the extensive Ruby ecosystem: As a Ruby framework, Sinatra seamlessly integrates with Ruby’s large and vibrant ecosystem. You can leverage the extensive Ruby gem library to obtain gems for almost anything you can think of instead of building from scratch. When you hit a roadblock in your app development journey, the Ruby community will be there to help.

Of course, there's more to Sinatra than the features we've highlighted here, but these will suffice for now. Next, let's go over the app we'll be building using the framework.

An overview of the app we'll be building

Our goal is to build a simple URL-shortening app using Sinatra. However, it makes a lot of sense to have an overview of the app's functionality since this will guide us on the build process. First, we should understand how URL shortening works, which we will cover next.

How URL shortening works

Generally speaking, URL shorteners work as outlined below:

The user enters a long URL, such as 'https://www.this-is-a-long-url-example.com'.
The app generates a random string, which is appended to a base URL and then associated with the long URL entered by the user, such as 'https://localhost/hn78JH'.
Then, both the long URL and it's shortened counterpart are saved to the app's database.
When a website visitor clicks the shortened URL, they are redirected to the original long URL. An optional visits counter can be attached to this flow so that it's incremented automatically with each visit.

Now that we know how the URL-shortening process works, let's kickstart our build.

Quick note: The full app code repo is available here.

Building a URL shortener using Sinatra

We will progressively build our URL-shortening app following the outline we discussed in the previous section on how URL shortening works.

Let's begin by installing the Sinatra gem, but first, it's important that we explain a very important concept that will determine how we build the app: modular vs. classical Sinatra apps.

Modular vs. classical app structure

When it comes to building Sinatra apps, you can go one of two ways: classical or modular. Without going into too many details, classical apps are structured in such a way that all app functionality lives within a single file. Modular apps feature different levels of abstraction, but more importantly, the main application (or applications since they can be more than one) is sub-classed from Sinatra::Base. You can read more on this here. While it is possible to build sophisticated apps using the classical approach, we'll go the modular route for the URL-shortener app.

Installing Sinatra and initializing the project

Installing Sinatra is as simple as running the command gem install sinatra. Then, create a new directory in an appropriate local folder to hold the project files:

mkdir sinatra_url_shortener

Now let's cd into the newly created app directory and create a basic starter file:

cd sinatra_url_shortener && touch app.rb

Next, open the project in your favorite editor and modify app.rb as shown below:

# app.rb

require 'sinatra'

get '/' do
  'Sinatra rocks!'
end

The code above is a very simple Sinatra app composed of a single file with a require statement to include Sinatra and a simple route definition telling the app to respond with the text "Sinatra rocks!" when someone hits the home or root page.

Assuming you're still in the project's root, run the script with the ruby app.rb command and then visit http://127.0.0.1:4567/ to see a Sinatra app running:

Of course, this is a far cry from the vision we have for our app, but it's a great start. Next, let's continue the build by adding a Gemfile and some configuration files.

Adding a `Gemfile`, Rack configuration and `Rakefile`

Since our goal is to build a modular Sinatra app with proper abstractions in place, let's continue the build by adding a Gemfile to hold the app's gem dependencies in a separate, dedicated file. We'll also need a Rack configuration file, config.ru, which will define how the app will be run:

# Gemfile

source 'https://rubygems.org'

gem 'dotenv'
gem 'json'
gem 'pry'
gem 'rake'
gem 'sassc'
gem 'sinatra'
gem 'sinatra-activerecord'
gem 'sinatra-contrib'
gem 'sinatra-static-assets'

group :development do
  gem 'foreman'
  gem 'rubocop'
  gem 'sqlite3', '~> 1.3', '>= 1.3.11'
end

group :test do
  gem 'rack-test'
  gem 'rspec', require: 'spec'
end

group :production do
  gem 'pg'
end

We won't go into the details of what each gem does, but we’ll highlight a few important ones:

sinatra - because we are building a Sinatra application.
sinatra-activerecord - since we'll be using Active Record as the ORM wrapper around the app's database.
sinatra-contrib - which will handle automatic app reloading in development since Sinatra does not come with this functionality by default.
rake - to help us run app-specific tasks, such as rake db:create in the command line.
rack-test and rspec - for testing purposes.
sassc - to use SCSS stylesheets and compile them into CSS (We'll use this later to style our app a bit).
json - useful for working with the JSON data format, including returning JSON data via API endpoints.

Next, add a Rakefile in the app's root and edit it as follows:

# Rakefile

require 'sinatra'
require './app'
...

run App

Finally, for this section, we'll need to modify the app.rb file as shown below:

# app.rb
require 'sinatra'
require 'sinatra/reloader'
require 'sinatra/activerecord'
require './helpers'
require 'securerandom'

class App < Sinatra::Base
  configure do
    set :json_encoder, :to_json
    set :erb, layout: :layout
  end

  before do
    headers['Access-Control-Allow-Methods'] = 'GET, POST, PUT, DELETE, OPTIONS'
    headers['Access-Control-Allow-Origin'] = '*'
    headers['Access-Control-Allow-Headers'] = 'accept, authorization, origin'
  end

  options '*' do
    response.headers['Allow'] = 'HEAD,GET,PUT,DELETE,OPTIONS,POST'
    response.headers['Access-Control-Allow-Headers'] =
      'X-Requested-With, X-HTTP-Method-Override, Content-Type, Cache-Control, Accept'
  end

  get '/' do
    erb :index
  end
end

In brief, here's what we've just defined:

configure block - We set the JSON encoder and a layout file that will be used by the app's views (similar to the application.html.erb layout file in a Rails app).
before and options blocks - These are generally called filter blocks, and we use them here to define some headers.
Finally, we define a root route that uses a view file, index.erb, which we'll get into a bit later.

Next, let's work on the app's database and the Link model.

Adding a database connection and Active Record model

In development, using SQLite gives us an easy-to-use file-based database that will suffice for our simple URL-shortening service.

To define the database connection, first create a folder called config in the app's root. Then, inside it, create two files: application.rb and database.yml.

Let's edit application.rb as shown below:

# config/application.rb

require 'active_record'
require 'sqlite3'
require 'yaml'

config_path = File.join(__dir__, 'database.yml')
ActiveRecord::Base.configurations = YAML.load_file(config_path)
ActiveRecord::Base.establish_connection(:development)

# Loads all models
Dir["#{__dir__}/../models/*.rb"].each { |file| require file }

Then the database.yml:

# config/database.yml

development:
  adapter: sqlite3
  database: db/app.sqlite3

production:
  adapter: postgresql
  encoding: unicode
  pool: 5
  host: <%= ENV['DATABASE_HOST'] %>
  database: <%= ENV['DATABASE_NAME'] %>
  username: <%= ENV['DATABASE_USER'] %>
  password: <%= ENV['DATABASE_PASSWORD'] %>

To finish this section, run bundle exec rake db:create to create an SQLite database under the db folder in the app's root.

Next, let's work on the URL model.

Defining the `ShortLink` model and Active Record migration

Let's start with the migration to create the short_links table. In the terminal in the app's root, run the following command:

bundle exec rake db:create_migration NAME=create_short_links

This will create a migration file under db/migrate, which we need to edit as follows:

# db/migrate/XXXXX_create_short_links.rb

class CreateShortLinks < ActiveRecord::Migration[7.0]
  def change
    create_table :short_links do |t|
      t.string :short_url
      t.string :long_url
      t.integer :visits, default: 0

      t.timestamps
    end
    add_index :short_links, %i[short_url long_url]
  end
end

Then, run bundle exec rake db:migrate, which will create the short_links table in the database. Let's now turn our attention to the model.

Create a new file, models/short_link.rb, and edit it accordingly:

# models/short_link.rb

# frozen_string_literal: true
class ShortLink < ActiveRecord::Base
  validates :short_url, presence: true
  validates :long_url, presence: true
end

We won't add any URL validation rules, but this would be a good challenge to test your Ruby skills, right? Go ahead and add such a rule and see how it works out for you. In the meantime, your app structure should look like this:

.
├── app.rb
├── config
│   ├── application.rb
│   └── database.yml
├── config.ru
├── db
│   ├── app.sqlite3
│   ├── migrate
│   │   └── XXXXXXXXXXX_create_short_links.rb
│   └── schema.rb
├── Gemfile
├── Gemfile.lock
├── helpers.rb
├── models
│   └── short_link.rb
├── public
│   └── css
├── Rakefile
├── stylesheets
│   └── style.scss
└── views

Working with Sinatra views

For us to work with the URLs, we'll need a view with a form in it that allows a user to input a long URL and have a short version generated once they submit the form. First, we need to add(or modify) the relevant routes to app.rb:

# app.rb

...

class App < Sinatra::Base
  ...

  get '/' do
    @link = ShortLink.new

    erb :index
  end

  post '/links' do
    generated_string = SecureRandom.alphanumeric(12)

    link = ShortLink.create(
      long_url: params[:long_url],
      short_url: generated_string
    )

    link = ShortLink.create(
      long_url: params[:long_url],
      short_url: short_url_string
    )

    redirect to("/show/#{link.id}")
  end

  get '/show/:id' do
    @link = ShortLink.find(params[:id])

    erb :show
  end

  get '/links/:short_url' do
    @link = ShortLink.find(params[:short_url])
    link_visits = @link.visits.to_i
    @link.update(visits: link_visits + 1)

    redirect to("http://localhost:9292/#{link.long_url}")
  end

  ...

end

Let's break down what's going on. Beginning with the root route, we define an instance variable called @link, which we'll use in the form in the view. We also define a POST route, which deals with creating the ShortLink model and saving the data to the database and then uses the redirect to helper to redirect to the created ShortLink show view.

A point of note is that we use Ruby's SecureRandom to generate a string to represent our short URL string. There are many ways to generate unique strings in Ruby, so take some time to explore other options and use the one that works best for you.

Next, under the views folder, create two files: index.erb and layout.erb. Start with the layout file and edit it with a simple ERB boilerplate as shown below:

<!-- views/layout.erb -->

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <meta http-equiv="X-UA-Compatible" content="ie=edge">
  <title>Sinatra URL Shortener App</title>
</head>
<body>
  <%= yield %>
</body>
</html>

Notice the use of the <%= yield %> statement like the one used in a Rails app layout; it will render any dynamic content we have.

Next, open the index.erb file and edit it as shown below:

<section class="section is-primary">
  <div class="container">
    <div class="box">
      <h1 class="title is-2">Create Short URL</h1>
      <form action="/links" method="POST">
        <div class="field">
          <label class="label">Name</label>
          <div class="control">
            <input type="text" name="short_link[long_url]" value="#{@link.long_url}" class="input">
          </div>
        </div>
        <div class="field">
          <div class="control">
            <input type="submit" value="Save Link" class="button is-link">
          </div>
        </div>
      </form>
    </div>
  </div>
</section>

Note: We are using Bulma styles here, but you can use whatever you like for styling your app.

Next, we need a view template to render show view. Create a new file, show.erb, under the views folder and edit it as shown below:

<!-- views/show.erb -->

<section class="section">
  <div class="container">
    <h2>Short Link Details</h2>
    <p>Long URL: <%= @link.long_url %></p>
    <p>Short URL: <%= @link.short_url %></p>
    <p>No. of visits: <%= @link.visits %></p>
  </div>
</section>

Running the app

At this point, you have a functioning app that takes a long URL and generates a shortened URL string version. You can run the app using the command rackup in the terminal and visit http://localhost:9292/ to see the app running:

Create a few links; then, try to visit the shortened URL version and see how the redirect happens.

Wrapping up and next steps

In this tutorial, we have learned how to build a simple URL-shortening app using Sinatra. As you've seen, Sinatra is a very powerful Ruby framework that can be used to build all sorts of apps. However, it is especially suited for micro-services and APIs. With the lessons you've learned as while following the tutorial, challenge yourself to build something even more exciting.

Happy coding!

Reducing your Python app's memory footprint

Honeybadger Staff — Fri, 05 Apr 2024 07:00:00 +0000

This article was originally written by Michael Barasa on the Honeybadger Developer Blog.

Many developers focus on developing core application functionalities and pay little or no attention to memory management until they run out of memory, and their apps start crashing, freezing, or experiencing random performance downgrades.

Computers have limited RAM, and it’s always best to make effective use of allocated resources. Trying to run a high memory-consuming app on a low-spec computer could cause it to crash, negatively impacting the user experience. Furthermore, a high memory footprint may also affect the performance of other apps and background services. When running a high memory-consuming app on the cloud, where resources are measured and charged for their use, you will likely end up with an expensive bill.

A high memory footprint can lead to undesirable consequences. Keep reading to learn what memory management entails and discover tips on lowering your Python app's memory footprint.

What is memory management, and why is it important?

Memory management is a complex process involving freeing and allocating computer memory to different programs, ensuring that the system operates efficiently. For example, when you launch a program, the computer has to allocate enough memory, and when the application is closed, the system frees memory and allocates it to another program.

Memory management has numerous benefits. First, it ensures that applications have the required resources to operate. The computer allocates memory to active processes and releases memory from inactive programs, which indicates effective memory utilization.

Second, proper memory management contributes to system stability. Since the computer handles memory allocation automatically, applications will always have access to the required memory, which reduces issues, such as random crashes and shutdowns. Memory management techniques, such as garbage collection, can assist in preventing memory leaks.

Third, memory management leads to better performance optimization. By continuously releasing and allocating memory, applications always have access to resources, which means they can quickly launch and execute.

Each application has a memory footprint, which refers to the amount of memory it consumes. A high memory footprint indicates an app is using a lot of memory, while a low footprint means it has low consumption.

Although computer systems can manage memory automatically, as a developer, you still have to keep your app's memory footprint in check. Using memory-intensive functions and inefficient data structures could cause your software to run out of memory, freeze, and even crash.

In the following section, we’ll explain how to measure your app's memory consumption. Later, we’ll discuss tips for lowering your memory footprint.

How to measure memory usage in Python

You can use any of the following methods to measure the amount of memory your application is using.

The psutil library

psutil is a Python library for fetching useful information about system utilization and active processes. Among other uses, the psutil library allows you to monitor memory, CPU, disk, and network usage.

To demonstrate how the psutil library works, we will use the following Python program that checks whether an integer is a prime number.

number = int(input("Please enter a number: "))

if number == 1:
    print(num, "is not a prime number")
elif number > 1:
   # check for factors
   for i in range(2,number):
       if (number % i) == 0:
           print(number,"is not a prime number")
           print(i,"times",number//i,"is",number)
           break
   else:
       print(number,"is a prime number")

# if the input number is less than or equal to 1, it is not prime
else:
   print(number,"is not a prime number")

We can check the above program's memory footprint by importing the psutil module and adding the following function in the code.

import psutil # import Python psutil module

def memory_usage():
    process = psutil.Process()
    usage = process.memory_info().rss 
    # Using memory_info() to check consumption
    return usage # Returning the memory in bytes

When you include and run the above function, it will show that the program uses 25886KB of memory.

Resource module

We can also use the resource module, specifically the getrusage() function, to check the amount of memory a program is using. We use it as follows:

import resource

def memory_usage():
    usage = resource.getrusage(resource.RUSAGE_SELF).ru_maxrss
    return usage

The sys module

The sys module also has getsizeof([]) and getallocatedblocks() methods, which allow you to check a program's memory footprint and the allocated number of memory blocks, respectively. The sys library can provide valuable insight for debugging purposes.

Here is how you can use the sys module in your code.

import sys

def memory_usage():
    usage = sys.getsizeof([])
    return usage

Third-party libraries

Apart from in-built functions, you can also utilize third-party libraries, such as memory_profiler, pympler, or objgraph, to measure an app's memory footprint.

Common causes of high memory usage

A large memory footprint can lead to undesirable consequences, including random freezes, crashes, and, ultimately, a bad user experience. We’ll cover the common causes of high memory usage in the following sections.

Memory leaks

The term memory leak refers to a situation where memory is allocated to a particular task but is not released upon completion of the process. This means that your application is not running efficiently. The amount of available memory is also reduced significantly.

Memory leaks can lead to performance downgrades. Apart from your application freezing or crashing, other background services may become inoperable. Furthermore, as more apps demand memory, the computer system may be forced to close down certain processes.

External dependencies

Although third-party libraries allow us to add numerous functionalities to our applications without creating everything from scratch, they may cause high memory consumption in an app.

For example, some libraries do not free up memory spaces when a task is completed or continuously run unnecessary background processes, which strains the available resources.

Large datasets

Python is a popular programming language for data analysis, machine learning, and artificial intelligence. Training AI algorithms require a considerable amount of data and memory. If you train an AI model on an underpowered CPU, it may crash or cause your computer to freeze.

Unoptimized code

Not using the garbage collector effectively, defining and storing too many objects in memory, and using the wrong datatypes could increase your app's memory footprint.

Tips to lower your app’s memory footprint

Now that we know the common causes of high memory usage and how to measure memory consumption, let's look at how to lower your app's memory footprint.

Use generators instead of lists

Although extremely useful, lists usually consume lots of memory, especially when they store many values. When the list is called, each value is loaded into memory and used by the application. Generators are like lists, but with one distinction; they support lazy loading. Thus, values stored in generators are retrieved only when needed.

Let's compare the memory consumption of lists and generators.

Here is a list that stores values between 0 and 999:

import sys

list = [i for i in range(1000)] # Stores values from 0 to 999
print(list) # We print values in the list
print(sum(list)) # We calculate the sum of the values in the list
print(f"The list consumes {sys.getsizeof(list)} bytes") # We check the amount of memory the list has taken.

When you run the above code, it shows that the list consumes about 920 bytes of memory.

In the following code sample, we use a generator instead of a list.

import sys

generatorlist = (i for i in range(1000)) # Stores values from 0 to 999
print(generatorlist) # We print values in the list
print(sum(generatorlist)) # We calculate the sum of the values in the list
print(f"The generator consumes {sys.getsizeof(generatorlist)} bytes") # We check the amount of memory the list has taken.

When the above code is executed, the generator consumes only 104 bytes of memory. Thus, generators are significantly more efficient than lists.

Read data in smaller chunks

As discussed, dealing with large datasets can be memory intensive. The computer has to allocate enough resources to process and store all file contents, meaning there is a chance of your application slowing down, freezing, or even crashing completely.

You can lower an app's memory footprint by reading data in smaller chunks compared to loading entire datasets in memory. This technique allows you to analyze data quickly without experiencing major performance issues.

For example, the following code is not memory efficient since we are loading our entire datasets ('employee.csv') into memory.

import pandas as pd

def readEmployeeData():
    df = pd.read_csv('employees.csv')['FIRST_NAME']
    print(df.value_counts())

We can save memory by defining a chunksize, or the number of rows our program should read from the dataset in one go, as demonstrated below.

import pandas as pd

def readDataInChunks():
    result = None
    for chunk in pd.read_csv("employees.csv", chunksize=200): #Setting the chunksize to 200 rows
        employees = chunk["FIRST_NAME"]
        chunk_result =  employees.value_counts()
        if result is None:
            result = chunk_result
        else:
            result = result.add(chunk_result, fill_value=0)

    result.sort_values(ascending=False, inplace=True)
    print(result)

readDataInChunks()

In the above code, we read and compute information from a smaller dataframe or chunk, which is more memory efficient. We save the results from the computation in a list and then proceed to analyze the next chunk of data, until we've analyzed the entire dataset.

Use memory-efficient dependencies

Before importing and using a third-party library in your project, research its key features and reliability. Ask questions, such as how much memory the library uses, and determine whether there are possible memory leaks. Being involved in online tech communities, such as Stack Overflow, can help you access valuable information much faster.

Use memory-profiling tools

It's a good idea to use memory profiling tools, such as memory_profiler, valgrind, and pympler, to measure an app’s memory footprint before pushing your application to production. This step ensures you're not caught off-guard and avoid negatively impacting the user experience.

For example, let's see how we can use memory_profiler to analyze memory consumption.

We can simply install memory_profiler with the following command.

$ pip install -U memory_profiler

Once the dependency is installed, add the @profile annotation above the function you wish to analyze.

import pandas as pd

@profile #Adding the @profile annotation
def readDataInChunks():
    result = None
    for chunk in pd.read_csv("employees.csv", chunksize=20):
        employees = chunk["FIRST_NAME"]
        chunk_result =  employees.value_counts()
        if result is None:
            result = chunk_result
        else:
            result = result.add(chunk_result, fill_value=0)

    result.sort_values(ascending=False, inplace=True)
    print(result)

readDataInChunks()

We can then execute the program with the following command.

python -m memory_profiler example.py

Alongside the program's log results, you should see the following output. You can use this information to optimize certain portions of your code.

Line #    Mem usage    Increment  Occurrences   Line Contents
=============================================================
     9   56.156 MiB   56.156 MiB           1   @profile
    10                                         def readDataInChunks():
    11   56.160 MiB    0.004 MiB           1       result = None
    12   57.566 MiB    1.133 MiB           4       for chunk in pd.read_csv("employees.csv", chunksize=20):
    13   57.555 MiB    0.062 MiB           3           employees = chunk["FIRST_NAME"]
    14   57.555 MiB    0.090 MiB           3           chunk_result =  employees.value_counts()
    15   57.555 MiB    0.000 MiB           3           if result is None:
    16   57.152 MiB    0.000 MiB           1               result = chunk_result
    17                                                 else:
    18   57.562 MiB    0.121 MiB           2               result = result.add(chunk_result, fill_value=0)
    19
    20   57.570 MiB    0.004 MiB           1       result.sort_values(ascending=False, inplace=True)
    21   57.621 MiB    0.051 MiB           1       print(result)

Conclusion

As you work on a software project, having a low memory footprint should be on the top of your list and not just an afterthought. Applications with low memory consumption can experience fewer crashes and freezes and, thus, improve the overall user experience.

Using generators instead of lists, avoiding memory-intensive libraries, and reading data in smaller chunks are some helpful tips for lowering your app's memory footprint.

A comprehensive guide to file operations in Go

Honeybadger Staff — Wed, 03 Apr 2024 07:00:00 +0000

This article was originally written by Adebayo Adams on the Honeybadger Developer Blog.

Proper file management systems date to the 1800s, and it is safe to say that the internet and even computers would not be able to work properly if they could not store, read, write, and delete files. In this article, you will learn everything you need to know about file operations in Go, from writing, reading, and creating files to creating and deleting directories in Go applications.

Prerequisites

The primary prerequisite to follow along with this article is a basic understanding of Go. You can check out my Go Beginners Series to get more familiar with it or for a quick brush-up on your Go skills.

File operations in Go

Go's file handling is quite straightforward due to its built-in package os; it provides access to most of the operating system's features, including the file system. It allows you to perform file operations without needing to change the code for it to work with different operating systems.

The packages enable you to perform various file operations in your applications, from writing, reading, and creating files to creating and deleting directories. It also provides helpful error messages whenever it encounters errors while performing file operations.

We’ll explore how to read files in Go in the next section.

Reading files in Go

Reading files is probably the most frequent file operation you'll perform in Go, as many use cases require it. In this section, we will explore how to read different types of files in Go.

Let's start with plain text files.

Reading `.txt` files in Go

The os package provides a ReadFile function that makes reading files straightforward in Go. For example, I have a data.txt file in my project folder, and I can read and print it out with the following code:

package main

import (
    "fmt"
    "os"
)

func main() {
    filepath := "data.txt"
    data, err := os.ReadFile(filepath)
    if err != nil {
        fmt.Println("File reading error", err)
        return
    }
    fmt.Println(string(data))
}

The code above defines a filepath variable with a string value of the file that I'm trying to read, data.txt. It then reads the file with the ReadFile function and stores the result and error in the data and err variables. Finally, it checks for errors before printing the result in string format by wrapping it with the string helper function. It should return the following output in the terminal:

1. fmt
2. net/http
3. os
4. io
5. time
6. encoding/json
7. strings
8. strconv
9. sync
10. testing
11. database/sql
12. encoding/csv
13. regexp
14. math
15. bufio
16. flag
17. path/filepath
18. image
19. html/template
20. crypto

And that's it! You've just read your first file in Go! Let's explore how to read a file line-by-line in the next section.

Reading a `log` file line-by-line in Go

In this section, we will explore a real-life use case of reading files in Go. Let's imagine that you're building an application that helps users deploy Go applications. You need to have a way for users to see the deployment errors so that they have an idea of what to fix before trying to re-deploy their application.

We will write a code that reads a log file line-by-line and prints only the amount of lines requested by the user. This way, users can choose to see only the last five logs and don't have to scroll through all the logs to find the issue with their deployment.

I have a sample log.txt file that looks like this:

2023-07-11 10:00:00 - Successful: operation completed.
2023-07-11 10:05:12 - Error: Failed to connect to the database.
2023-07-11 10:10:32 - Successful: data retrieval from API.
2023-07-11 10:15:45 - Error: Invalid input received.
2023-07-11 10:20:58 - Successful: file upload.
2023-07-11 10:25:01 - Error: Authorization failed.
2023-07-11 10:30:22 - Successful: record update.
2023-07-11 10:35:37 - Error: Internal server error.
2023-07-11 12:45:59 - Error: Server overloaded.
2023-07-11 12:50:06 - Successful: session created.
2023-07-11 12:55:17 - Error: Invalid input parameters.
2023-07-11 13:00:30 - Successful: software update installed.
2023-07-11 13:05:46 - Error: Access denied.
2023-07-11 13:10:53 - Successful: report generated.
2023-07-11 13:16:01 - Error: Unexpected exception occurred.
2023-07-11 13:20:13 - Successful: user registration.
2023-07-11 13:25:28 - Error: Disk read/write failure.

The code for doing this will look like this:

package main

import (
    "bufio"
    "fmt"
    "os"
)

func printLastNLines(lines []string, num int) []string {
    var printLastNLines []string
    for i := len(lines) - num; i < len(lines); i++ {
        printLastNLines = append(printLastNLines, lines[i])
    }
    return printLastNLines
}

func main() {
    filepath := "log.txt"
    file, err := os.Open(filepath)
    if err != nil {
        fmt.Println("Error opening file:", err)
    }
    defer file.Close()

    var lines []string
    scanner := bufio.NewScanner(file)
    for scanner.Scan() {
        lines = append(lines, scanner.Text())

    }
    if err := scanner.Err(); err != nil {
        fmt.Println("Error reading file:", err)
    }

    // print the last 10 lines of the file
    printLastNLines := printLastNLines(lines, 3)
    for _, line := range printLastNLines {
        fmt.Println(line)
        fmt.Println("________")
    }
}

The code above uses the os' package Open function to open the file, defers its Close function with the defer keyword, defines an empty lines slice, and uses the bufio's NewScanner function to read the file line-by-line while appending each line to the lines array in a text format using the Text function. Finally, it uses the printLastNLines function to get the last N lines of the lines array. N is any number of the user's choosing. In this case, it is 3, and the code uses a for loop to print each line with an horizontal line between each one.

Note: The defer keyword ensures that the file closes if there is an error; if not, it closes the file after the function has finished running.

Using the sample log.txt file, the code above should return the following:

2023-07-11 13:16:01 - Error: Unexpected exception occurred.
________
2023-07-11 13:20:13 - Successful: user registration.
________
2023-07-11 13:25:28 - Error: Disk read/write failure.
________

Next, let's explore how to read .json files in Go.

Reading `.json` files in Go

Reading and using data from .json files is also a popular use case in programming, so let's learn how to do that. For example, you have a .json configuration file that looks like the following:

{

      "database_host": "localhost",
      "database_port": 5432,
      "database_username": "myuser",
      "database_password": "mypassword",
      "server_port": 8080,
      "server_debug": true,
      "server_timeout": 30

  }

You also need to read for your application to work correctly, which can be done by reading and decoding the .json file:

package main

import (
    "encoding/json"
    "fmt"
    "os"
)

type Config struct {
    DBHost        string `json:"database_host"`
    DBPort        int    `json:"database_port"`
    DBUsername    string `json:"database_username"`
    DBPassword    string `json:"database_password"`
    ServerPort    int    `json:"server_port"`
    ServerDebug   bool   `json:"server_debug"`
    ServerTimeout int    `json:"server_timeout"`
}

func main() {
    filepath := "config.json"
    var config Config

    file, err := os.Open(filepath)
    if err != nil {
        fmt.Println(err)
    }
    defer file.Close()

    decoder := json.NewDecoder(file)
    err = decoder.Decode(&config)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(config)
}

The code above uses the os' package Open function to open the file, defers its Close function with the defer keyword, and uses the json's NewDecoder function to read the file. It then uses the Decoder function to decode it into an object and checks for errors before printing the config details in the terminal:

{localhost 5432 myuser mypassword 8080 true 30}

That's it! You can now split the config object up and use it however you like in your application.

Next, let's look at how to read a .csv file.

Reading `.csv` files in Go

Comma-separated values (CSV) is one of the most popular file formats. Let's explore how to read the following .csv file in this section:

Name,Email,Phone,Address
John Doe,johndoe@example.com,555-1234,123 Main St
Jane Smith,janesmith@example.com,555-5678,456 Elm St
Bob Johnson,bobjohnson@example.com,555-9876,789 Oak St

Go provides an encoding/csv package that can be used to read .csv files:

package main

import (
    "encoding/csv"
    "fmt"
    "os"
)

func main() {
    filepath := "data.csv"

    file, err := os.Open(filepath)
    if err != nil {
        fmt.Println(err)
    }
    defer file.Close()

    reader := csv.NewReader(file)
    records, err := reader.ReadAll()
    if err != nil {
        fmt.Println(err)
    }

    fmt.Println(records)
}

The code above uses the os' package Open function to open the file, defer its Close function with the defer keyword, and create a reader variable that stores the result of the NewReader function. It then uses the ReadAll function to read the file, check for errors, and print the result in the terminal. The output should look like this:

[[Name Email Phone Address] [John Doe johndoe@example.com 555-1234 123 Main St] [Jane Smith janesmith@example.com 555-5678 456 Elm St] [Bob Johnson bobjohnson@example.com 555-9876 789 Oak St]]

Let's explore how to read bytes from files in the next section.

Reading bytes from files in Go

In some cases, you might want to read a specific number of bytes from files in Go:

package main

import (
    "fmt"
    "os"
)

func main() {
    filepath := "data.txt"

    file, err := os.Open(filepath)
    if err != nil {
        fmt.Println(err)
    }
    defer file.Close()

    data := make([]byte, 10)
    _, err = file.Read(data)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println(data)
    fmt.Println(string(data))
}

The code above uses the os' package Open function to open the file, defer its Close function with the defer keyword, and create a data variable that has the value of 10 bytes using the make function. It then checks for errors before printing the data out in bytes and string formats:

[49 46 32 102 109 116 10 50 46 32]
1. fmt
2.

Now that we've explored how to perform various read operations on different file formats in Go, let's explore how to handle write operations.

Writing and manipulating files in Go

Reading and writing files is relatively straightforward in Go, as it provides developers with a lot of functions to perform such tasks without needing to download a third-party library. In this section, we will explore different Go writing tasks and how to handle them properly.

Creating a file in Go

Go's os package provides a Create function that creates files with any extension. For example, suppose you have an application that helps users to deploy applications. In this case, you might want to create an empty log.txt file containing the logs of the application immediately after the user creates a new project:

package main

import (
    "fmt"
    "log"
    "os"
)

func main() {
    file, err := os.Create("log.txt")
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()
    fmt.Print("File created successfully")
}

The code above uses the os's Create function to create an empty log.txt file, checks for any error, and prints a success message to the user. The user should see the following output in the terminal:

File created successfully

Before continuing with writing files in Go, let's explore the different file opening flags in Go, as we will use some of them in the later in the article.

File-opening flags in Go

Go provides file-opening flags represented by constants defined in the os package. These flags determine the behavior of file operations, such as opening, creating, and truncating files. The following is a list of the flags and what they do.

os.O_RDONLY: Opens the file as read-only. The file must exist.
os.O_WRONLY: Opens the file as write-only. If the file exists, its contents are truncated. If it doesn't exist, a new file is created.
os.O_RDWR: Opens the file for reading and writing. If the file exists, its contents are truncated. If it doesn't exist, a new file is created.
os.O_APPEND: Appends data to the file when writing. Writes occur at the end of the file.
os.O_CREATE: Creates a new file if it doesn't exist.
os.O_EXCL: Used with O_CREATE, it ensures that the file is created exclusively, preventing creation if it already exists.
os.O_SYNC: Open the file for synchronous I/O operations. Write operations are completed before the call returns.
os.O_TRUNC: If the file exists and is successfully opened, its contents are truncated to zero length.
os.O_NONBLOCK: Opens the file in non-blocking mode. Operations like read or write may return immediately with an error if no data is available or the operation would block.

These flags can be combined using the bitwise OR (|) operator. For example, os.O_WRONLY|os.O_CREATE would open the file for writing, creating it if it doesn't exist.

When using these flags, it's important to check for errors returned by file operations to handle cases where the file cannot be opened or created as expected.

Let's look at how to write text to files in the next section.

Writing text to files in Go

The os package also provides a WriteString function that helps you write strings to files. For example, you want to update the log.txt file with a log message:

package main

import (
    "log"
    "os"
)

func main() {
    file, err := os.OpenFile("log.txt", os.O_WRONLY|os.O_CREATE, 0644)
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    data := "2023-07-11 10:05:12 - Error: Failed to connect to the database. _________________"
    _, err = file.WriteString(data)
    if err != nil {
        log.Fatal(err)
    }

}

The code above uses the OpenFile function to open the log.txt file in write-only mode and creates it if it doesn't exist. It then creates a data variable containing a string and uses the WriteString function to write string data to the file.

Appending to a file in Go

The code in the previous section deletes the data inside the file before writing the new data every time the code is run, which is acceptable in some cases. However, for a log file, you want it to retain all the previous logs so that the user can refer to them as many times as needed to, for example, perform analytics.

You can open a file in append mode like this:

package main

import (
    "log"
    "os"
)

func main() {
    file, err := os.OpenFile("log.txt", os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0644)
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    data := "\n 2023-07-11 10:05:12 - Error: Failed to connect to the database.\n __________________ \n"
    _, err = file.WriteString(data)
    if err != nil {
        log.Fatal(err)
    }

}

The code above uses the os.O_APPEND to open the file in append mode and will retain all the existing data before adding new data to the log.txt file. You should get an updated file each time you run the code instead of a new file.

Writing bytes to files in Go

Go allows you to write bytes to files as strings with the Write function. For example, if you are streaming data from a server and it is returning bytes, you can write the bytes to a file to be readable:

package main

import (
    "log"
    "os"
)

func main() {
    file, err := os.OpenFile("data.bin", os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0644)
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    data := []byte{0x48, 0x65, 0x6C, 0x6C, 0x6F, 0x2C, 0x20, 0x57, 0x6F, 0x72, 0x6C, 0x64, 0x21, 0x0A}
    _, err = file.Write(data)
    if err != nil {
        log.Fatal(err)
    }

}

The code above opens the data.bin file in write-only and append mode and creates it if it doesn't already exist. The code above should return a data.bin file containing the following:

Hello, World!

Next, let's explore how to write formatted data to a file section.

Writing formatted data to a file in Go

This is one of the most common file-writing tasks when building software applications. For example, if you are building an e-commerce website, you will need to build order confirmation receipts for each buyer, which will contain the details of the user's order. Here is how you can do this in Go:

package main

import (
    "fmt"
    "log"
    "os"
)

func main() {
    username, orderNumber := "Adams_adebayo", "ORD6543234"
    file, err := os.Create(username + orderNumber + ".pdf")
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()
    item1, item2, item3 := "shoe", "bag", "shirt"
    price1, price2, price3 := 1000, 2000, 3000

    _, err = fmt.Fprintf(file, "Username: %s\nOrder Number: %s\nItem 1: %s\nPrice 1: %d\nItem 2: %s\nPrice 2: %d\nItem 3: %s\nPrice 3: %d\n", username, orderNumber, item1, price1, item2, price2, item3, price3)
    if err != nil {
        log.Fatal(err)
    }

}

The code above defines two variables, username and orderNumber, creates a .pdf based on the variables, checks for errors, and defers the Close function with the defer keyword. It then defines three variables, item1, item2, and item3, formats a message with the fmt's Fprintf all the variables, and writes it to the .pdf file.

The code above then creates an Adams_adebayoORD6543234.pdf file with the following contents:

Username: Adams_adebayo
Order Number: ORD6543234
Item 1: shoe
Price 1: 1000
Item 2: bag
Price 2: 2000
Item 3: shirt
Price 3: 3000

Writing to `.csv` files in Go

With the help of the encoding/csv package, you can write data to .csv files easily with Go. For example, you want to store new users' profile information in a .csv file after they sign up:

package main

import (
    "encoding/csv"
    "log"
    "os"
)

func main() {
    file, err := os.OpenFile("users.csv", os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0644)
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    writer := csv.NewWriter(file)
    defer writer.Flush()
    data := []string{"Adams Adebayo", "30", "Lagos"}
    err = writer.Write(data)
    if err != nil {
        log.Fatal(err)
    }

}

The code above opens the users.csv file in write-only and append mode and creates it if it doesn't already exist. It will then use the NewWriter function to create a writer variable, defer the Flush function, create a data variable with the string slice, and write the data to the file with the Write function.

The code above will then return a users.csv file with the following contents:

Adams Adebayo,30,Lagos

Writing JSON data to a file in Go

Writing JSON data to .json files is a common use case in software development. For example, you are building a small application and want to use a simple .json file to store your application data:

package main

import (
    "encoding/json"
    "log"
    "os"
)

func main() {
    file, err := os.OpenFile("users.json", os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0644)
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    data := map[string]interface{}{
        "username": "olodocoder",
        "twitter":  "@olodocoder",
        "email":    "hello@olodocoder.com",
        "website":  "https://dev.to/olodocoder",
        "location": "Lagos, Nigeria",
    }

    encoder := json.NewEncoder(file)
    err = encoder.Encode(data)
    if err != nil {
        log.Fatal(err)
    }

}

The code above opens the users.csv file in write-only and append mode and creates it if it doesn't already exist, defers the Close function, and defines a data variable containing the user data. It then creates an encoder variable with the NewEncoder function and encodes it with the Encoder function.

The code above then returns a users.json file containing the following:

{"email":"hello@olodocoder.com","location":"Lagos, Nigeria","twitter":"@olodocoder","username":"olodocoder","website":"https://dev.to/olodocoder"}

Writing XML data to files in Go

You can also write XML data to files in Go using the encoding/xml package:

package main

import (
    "encoding/xml"
    "log"
    "os"
)

func main() {
    type Person struct {
        Name string `xml:"name"`
        Age  int    `xml:"age"`
        City string `xml:"city"`
    }

    file, err := os.OpenFile("users.xml", os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0644)
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    data := Person{
        Name: "John Doe",
        Age:  30,
        City: "New York",
    }

    encoder := xml.NewEncoder(file)
    err = encoder.Encode(data)
    if err != nil {
        log.Fatal(err)
    }

}

The code above defines a Person struct with three fields, opens the users.xml file in write-only and append mode and creates it if it doesn't already exist, defers the Close function, and defines a data variable that contains the user data. It then creates an encoder variable with the NewEncoder function and encodes it with the Encoder function.

The code above should return a user.xml file that contains the following contents:

<Person><name>John Doe</name><age>30</age><city>New York</city></Person>

Renaming files in Go

Go enables you to rename files from your code using the Rename function:

package main

import (
    "fmt"
    "os"
)

func main() {
    err := os.Rename("users.xml", "data.xml")
    if err != nil {
        fmt.Println(err)
    }

}

The code above renames the users.xml file created in the previous section to data.xml.

Deleting files in Go

Go enables you to delete files with the Remove function:

package main

import (
    "fmt"
    "os"
)

func main() {
    err := os.Remove("data.bin")
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println("File deleted")
}

The code above deletes the data.bin file from the specified path.

Now that you understand how to write and manipulate different types of files in Go, let's explore how to work with directories.

Working with directories in Go

In addition to files, Go also provides functions that you can use to perform different tasks in applications. We will explore some of these tasks in the following sections.

Creating a directory

Go provides a Mkdir function that you can use to create an empty directory:

package main

import (
    "fmt"
    "os"
)

func main() {
    err := os.Mkdir("users", 0755)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println("Directory Created Successfully")
}

The code above creates a users folder in the current working directory.

Creating multiple directories in Go

You can create multiple directories in Go using the MkdirAll function:

package main

import (
    "fmt"
    "os"
)

func main() {
    err := os.MkdirAll("data/json_data", 0755)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println("Directory Created Successfully")
}

The code above will create a data directory and a json_data directory inside it.

Note: If a data directory already exists, the code will only add a json_data directory inside it.

Checking if a directory exists in Go

To avoid errors, checking if a directory exists before creating a file or directory inside is good practice. You can use the Stat function and the IsNotExist function to do a quick check:

package main

import (
    "fmt"
    "os"
)

func main() {
    if _, err := os.Stat("data/csv_data"); os.IsNotExist(err) {
        fmt.Println("Directory does not exist")
    } else {
        fmt.Println("Directory exists")
    }

}

The code above returns a message based on the results of the check. In my case, it will return the following:

Directory exists

Renaming directories in Go

You can also use the Rename function to rename directories:

package main

import (
    "fmt"
    "os"
)

func main() {
    err := os.Rename("data/csv_data", "data/xml_data")
    if err != nil {
        fmt.Println(err)
    }

}

The code above renames the data/csv_data directory to data/xml_data.

Deleting an empty directory in Go

You can use the Remove function to delete folders in your applications:

package main

import (
    "fmt"
    "os"
)

func main() {
    err := os.Remove("data/json_data")
    if err != nil {
        fmt.Println(err)
    }

}

The code above removes the json_data directory from the data directory.

Deleting a directory with all its content in Go

Go provides a RemoveAll function that allows you to remove all the directories and everything inside them, including files and folders:

package main

import (
    "fmt"
    "os"
)

func main() {
    err := os.RemoveAll("users")
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println("users directory and all it's content has been removed")
}

The code above deletes the users directory and everything inside it.

Note: It's good practice to check if the directory exists before attempting to delete it.

Get a list of files and directories in a directory in Go

You can retrieve a list of all the files and directories in a directory using the ReadDir function:

package main

import (
    "fmt"
    "os"
)

func main() {
    dirEntries, err := os.ReadDir("data")
    if err != nil {
        fmt.Println(err)
    }

    for _, entry := range dirEntries {
        fmt.Println(entry.Name())
    }
}

The code above returns a list of all the directories and files inside the data folder.

Now that you know how to work with directories in Go applications, let's explore some of the advanced file operations in the next section.

Advanced file operations in Go

In this section, we will explore some of the advanced file operations you might encounter in Go applications.

Writing compressed data to a file in Go

Working with compressed files is uncommon, but here's how to create a .txt file inside a compressed file using the compress/gzip package:

package main

import (
    "compress/gzip"
    "fmt"
    "log"
    "os"
)

func main() {
    file, err := os.OpenFile("data.txt.gz", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
    if err != nil {
        log.Fatal(err)
    }
    defer file.Close()

    gzipWriter := gzip.NewWriter(file)
    defer gzipWriter.Close()

    data := "Data to compress"
    _, err = gzipWriter.Write([]byte(data))
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println("File compressed successfully")
}

The code above creates a data.txt.gz, which contains a data.txt file in the working directory.

Writing encrypted data to a file in Go

When building applications that require secure files, you can create an encrypted file with Go's crypto/aes and crypto/cipher packages:

package main

import (
    "crypto/aes"
    "crypto/cipher"
    "fmt"
    "log"
    "os"
)

func main() {
    // file, err := os.OpenFile("encrypted.txt", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
    file, err := os.Create("encrypted.txt")
    if err != nil {
        log.Fatal(err)
        fmt.Println("Error")
    }
    defer file.Close()

    key := []byte("cacf2ebb8cf3402964356547f20cced5")
    plaintext := []byte("This is a secret! Don't tell anyone!🤫")

    block, err := aes.NewCipher(key)
    if err != nil {
        log.Fatal(err)
        fmt.Println("Error")
    }

    ciphertext := make([]byte, len(plaintext))
    stream := cipher.NewCTR(block, make([]byte, aes.BlockSize))
    stream.XORKeyStream(ciphertext, plaintext)

    _, err = file.Write(ciphertext)
    if err != nil {
        log.Fatal(err)
        fmt.Println("Error")
    }
    fmt.Println("Encrypted file created successfully")
}

The code above creates an encrypted.txt file containing an encrypted version of the plaintext string:

?Э_g?L_.?^_?,_?_;?S???{?Lؚ?W4r
W?8~?

Copying a file to another directory in Go

Copying existing files to different locations is something we all do frequently. Here's how to do it in Go:

package main

import (
    "fmt"
    "io"
    "os"
)

func main() {
    srcFile, err := os.Open("data/json.go")
    if err != nil {
        fmt.Println(err)
    }
    defer srcFile.Close()

    destFile, err := os.Create("./json.go")
    if err != nil {
        fmt.Println(err)
    }
    defer destFile.Close()

    _, err = io.Copy(destFile, srcFile)
    if err != nil {
        fmt.Println(err)
    }
    fmt.Println("Copy done!")
}

The code above copies the json.go file in the data directory and its contents and then creates another json.go with the same in the root directory.

Get file properties in Go

Go allows you to get the properties of a file with the Stat function:

package main

import (
    "fmt"
    "os"
)

func main() {
    fileInfo, err := os.Stat("config.json")
    if err != nil {
        fmt.Println(err)
    }

    fmt.Println("File name:", fileInfo.Name())
    fmt.Println("Size in bytes:", fileInfo.Size())
    fmt.Println("Permissions:", fileInfo.Mode())
    fmt.Println("Last modified:", fileInfo.ModTime())

    fmt.Println("File properties retrieved successfully")
}

The code above returns the name, size, permissions, and last modified date of the config.json file:

File name: config.json
Size in bytes: 237
Permissions: -rw-r--r--
Last modified: 2023-07-11 22:46:59.705875417 +0100 WAT
File properties retrieved successfully

Get the current working directory path in Go

You can get the current working directory of your application in Go:

package main

import (
    "fmt"
    "os"
)

func main() {
    wd, err := os.Getwd()
    if err != nil {
        fmt.Println(err)
    }

    fmt.Println("Current working directory:", wd)

}

The code above will return the full path of my current working directory:

Current working directory: /Users/user12/Documents/gos/go-files

Conclusion

That's it! You now have the knowledge to tackle most of the file operations you'll encounter when building Go applications. In this article, we explored how to read, write and manipulate files and directories in Go.

We also explored how to solve many file operations problems by combining different Go packages and functions. Thank you so much for reading! If you have any questions, suggestions, or corrections, feel free to connect with me on Twitter and LinkedIn, and I'll try my best to work on them.

A guide to soft deletes in Laravel

Honeybadger Staff — Tue, 02 Apr 2024 07:00:00 +0000

This article was originally written by Ashley Allen on the Honeybadger Developer Blog.

There may be times in your web applications that you want to allow users to delete data without permanently removing it from the database. For example, you may want to allow an admin to delete another user's account but still keep the user's data in the database in case the admin made a mistake. This would then allow the admin to restore the deleted account if needed. This is where soft deletes are useful.

In this article, we'll cover what soft deletes are, the advantages and disadvantages of using them, and how to use them in your Laravel application. We'll explore how to prepare your model and database for soft deletes, how to soft delete and restore models, how to delete models permanently, and how to query soft-deletable models. We'll then take a look at how to test soft-deletable models and a common pitfall to avoid when using soft deletes with the DB facade.

By the end of the article, you should have a good understanding of what soft deletes are and how to start using them in your own Laravel applications.

What are soft deletes?

In their simplest form, soft deletes are a way of deleting data in an application without removing it from the database. Instead, a flag is set on the row in the database to indicate that it has been deleted. In Laravel, the flag is a timestamp column called deleted_at.

When a query is made to retrieve data from the database, we can determine whether the data has been "deleted" by checking the value of the deleted_at column. If the value is null, then the data has not been deleted. If the value is not null and is set to a timestamp, then the data has been deleted.

Here are some situations where you may want to use soft deletes:

Allowing a user to delete their account, but still keep their data in the database in case they change their mind and want to restore their account.
Using it for analytical purposes, such as keeping track of usage statistics even after a related model has been removed by a user.

An easy way of thinking about soft deletes is that it's like a "recycle bin" for your application. When you're using your computer, you can delete files, and they're sent to your "recycle bin". Thus, they're "deleted" in the sense that they can't be accessed from their original location, but they're still there. You can then choose to permanently delete them from your "recycle bin" or restore them to their original location.

However, it's important to note that if an application uses soft deletes, it doesn't mean the application also has to support restoring the data or permanently deleting it. The soft deletes be added for the developer's own purposes, rather than making it a feature of the application that users know about.

The advantages of using soft deletes

Using soft deletes in your Laravel application has a number of advantages. Let's take a look at some of them.

Data recovery and restoration

As we've already briefly mentioned, having the functionality to restore data that has been "deleted" is a huge advantage of using soft deletes.

As humans, we all make mistakes. Therefore, it's inevitable that at some point, a user will accidentally delete data. If the data was permanently deleted, it would likely be extremely difficult to recover. However, if the data was soft-deleted, it could potentially be restored, assuming the application had a "restore" feature.

Data auditing

Depending on the type of application you're building and the data you're storing, you may want to keep track of data even after it's been deleted. You might do this for analytical purposes, such as keeping track of usage statistics. Alternatively, you might need to do this for legal or regulatory compliance reasons, by keeping track of data even when it's no longer needed by a user. For example, imagine you have an e-commerce website with an admin panel for the store owner to manage products and sales. If customers request the deletion of their accounts, you may need to keep track of some of their data (such as their orders) for tax purposes, but delete the rest of their data.

Two-step deletion

You may want your application to have a two-step deletion process for some types of data. For example, you have an application that has multiple roles: "admin", "manager", and "user".

You may want to allow managers to delete a user. A manager deleting a user would effectively delete the user from the application, but their data would still be kept in the database. It would then be up to an admin user to approve the permanent deletion of the user from the database or restore them. This sort of approach can be useful when you need to be absolutely certain that a user's data should be deleted and makes use of multiple users to verify the deletion.

The disadvantages of using soft deletes

Although there are many advantages to using soft deletes, there are also some disadvantages. Let's take a look at some of them.

Increased database storage

As we've already covered, soft-deleted rows aren't removed from the database. Instead, they're marked as deleted by using a specific column in the table. This means that the data is still stored in the database.

As a result, the amount of data you're storing in the database will likely increase much quicker than if you were permanently deleting the data. Although this may not be an issue for smaller applications, it could become an issue for larger applications that have a lot of data and may increase the database infrastructure costs.

Increased chance of accidentally querying "deleted" data

Later in this article, we'll cover a common "gotcha" that I've seen many developers encounter. This is when a developer attempts to query a table in the database that has soft deletes enabled but forgets to include a check for the soft-delete flag field. This can result in the developer accidentally including "deleted" data in their queries, which can cause unexpected results.

Therefore, although soft deletes can be useful, they can add an extra layer of complexity (albeit a small one) to your application that you need to be aware of, especially when writing more complex database queries.

Data privacy issues

Trying to find a balance between allowing soft deletion and adhering to data privacy laws can be difficult. Let's take a look at an example. Imagine that a user based in the European Union (EU) has an account on a web application that allows users to delete their account. However, the application may also provide the ability for the user to restore their deleted account if they change their mind. This means that the application needs to keep the user's data in the database, even after they've deleted their account. However, keeping the data in the database may lead to the application breaching the rules of the General Data Protection Regulation (GDPR), which is a data privacy law in the EU. The user may be under the impression that their data has been completely wiped from your system, but in reality, it's still there.

If you need to keep the user's data (except for any personally identifying information), you may be able to still adhere to the rules by anonymizing the data. For example, you could replace the user's name, email address, phone number, address, and so on with a series of random characters. However, this isn't always possible, especially if the user may want to restore their account afterward and continue using the application with their old, correct data.

It's important to note that this blog post does not serve as any legal advice surrounding data privacy or security. If you're unsure about anything related to data privacy or security, you should seek legal advice from a professional who is more knowledgeable in this area.

Not a replacement for an audit log

Making data soft-deletable in the database can be a quick and useful way of tracking data that has been deleted. For example, in the context of a model in a Laravel application using the default columns, we could use the following columns:

created_at - The date and time the model was created.
updated_at - The date and time the model was last updated.
deleted_at - The date and time the model was soft-deleted.

Through these columns, we can form a brief timeline of the model's lifecycle. This can be useful for simple debugging purposes and for the developer to get a quick overview of the model. Depending on where it's being used, it can also be used to present useful information to the user. However, this isn't a replacement for a proper audit log.

If your application requires more in-depth data tracking, you should consider using an audit log. Although our created_at, updated_at, and deleted_at fields can tell us the times of these events, we don't have any extra context. They can't really help us answer any of these questions:

Who created, updated, or deleted the model?
If the model was updated, which fields were updated?
What were the fields changed from and to?
If the model was deleted, why was it deleted?
Was it deleted by the user or automatically by the system?

As a result of using an audit log, we would more likely be able to answer these questions. However, this doesn't mean that you can't use soft deletes and an audit log together. Soft deletes can still be useful for all the reasons we've already covered, and an audit log can be used to provide more context around the data.

Preparing the model and database

To get started with using soft deletes in your Laravel application, you'll need to prepare your soon-to-be soft-deletable model and its database table.

Laravel provides a handy Illuminate\Database\Eloquent\SoftDeletes trait that you can add to any model that you wish to make soft-deletable. This trait adds some useful methods to the model, such as the following:

forceDelete - This method permanently deletes the model from the database.
forceDeleteQuietly - This method permanently deletes the model from the database without firing any model events.
restore - This method restores the soft-deleted model in the database.
restoreQuietly - This method restores the soft-deleted model in the database without firing any model events.
trashed - This method returns a Boolean to indicate whether a given model has been soft-deleted.

It also registers some model events triggered when the model is being soft-deleted, force deleted, and restored.

Another huge benefit of using this trait is that it registers a global scope (using the Illuminate\Database\Eloquent\SoftDeletingScope class that's included) that automatically excludes any soft-deleted models from your queries. This class also registers some query macros that you can use to include soft-deleted models in your queries. We'll go into more depth about these query macros later in this article.

Now that we have some context about the SoftDeletes trait, let's add it to our model. For the purposes of this article, we'll imagine we are building a blogging application and have an Author model that we wish to make soft-deletable. We'll add the SoftDeletes trait to the model like so:

declare(strict_types=1);

namespace App\Models;

use Illuminate\Database\Eloquent\Model;
use Illuminate\Database\Eloquent\SoftDeletes;

final class Author extends Model
{
    use SoftDeletes;

    // ...
}

Now that we've added the soft delete functionality to the model, we need to prepare the database table so that we can keep track of whether a model has been soft-deleted. As we've already mentioned, Laravel does this using a deleted_at timestamp column. Thus, let's add this column to our authors table using a migration.

We'll create a new migration by running the following Artisan command:

php artisan make:migration add_deleted_at_to_authors_table --table=authors

This command will create a new migration class with a file path, such as database/migrations/2023_07_10_111209_add_deleted_at_to_authors_table.php. Let's update the migration so that we can add the new deleted_at column. To do this, we can use a handy softDeletes method that's available on the Blueprint class provided by Laravel. Likewise, we can use the dropSoftDeletes method to remove the column.

The migration should look something like this:

declare(strict_types=1);

use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;

return new class extends Migration
{
    public function up()
    {
        Schema::table('authors', function (Blueprint $table) {
            $table->softDeletes();
        });
    }

    public function down()
    {
        Schema::table('authors', function (Blueprint $table) {
            $table->dropSoftDeletes();
        });
    }
};

We can now run the migration using the migrate Artisan command to add the deleted_at column to the authors table:

php artisan migrate

Soft deleting and restoring models

Your model and database should now be ready to use soft deletes. Let's take a look at how we can soft-delete and restore models.

To soft-delete a model, you can call the delete method on the model instance. Since the model is using the SoftDeletes trait, the delete method will automatically set the deleted_at column to the current date and time rather actually remove the row from the database. As an example, to soft delete an Author model (which we'll assume has an ID of 1), we can do the following:

$author = Author::find(1);

$author->delete();

The Author model will now be soft-deleted. It's worth noting that calling delete on an already soft-deleted model will update the model's deleted_at column to the current date and time. Although this may not cause any issues for your application, it's worth keeping in mind if you need to keep a record of the original time that a model soft-deleted, such as for analytical purposes.

If we wanted to restore the model (so that it's no longer soft-deleted), we can call the restore method on the model instance:

$author->restore();

If you'd like to check whether a given model is soft-deleted, you can use the trashed method on the model instance. If the model is soft-deleted, it will return true; otherwise, it will return false. Here is an example:

$author = Author::find(1);

$author->trashed(); // false

$author->delete();

$author->trashed(); // true

Deleting models permanently

If you want to permanently remove a model from the database (sometimes referred to as "hard-deleting"), you can use the forceDelete method on the model. This can be performed on both soft-deleted and non-soft-deleted models like so:

$author = Author::find(1);

$author->forceDelete();

It's important to remember that once this method has been called, the row will be permanently removed from the database and cannot be restored.

Querying soft-deletable models

As we've already briefly mentioned, the SoftDeletes trait registers a global scope that automatically excludes any soft-deleted models from your queries by default. This means that if you run queries, such as Author::find(1), Author::all(), or Author::where('name', 'John')->get(),soft-deleted models will not be returned.

In most cases, this is the desired behavior. However, there may be times when you want to include soft-deleted models in your query or only want to return soft-deleted models. Laravel registers some query macros that you can use to achieve this. Let's take a look at them.

Including soft-deleted models

There may be times when you want to retrieve all the models in the database table, regardless of whether they're soft-deleted. An example of when you might want to do this would be if you're writing a query to analyze the usage statistics of a user's account.

To include soft-deleted models in your query, you can use the withTrashed method on the query builder instance. For example, if you want to retrieve all the Author models, including the soft-deleted models, you could write the following query:

$authors = Author::withTrashed()->get();

Likewise, if you wanted to retrieve a single model that may or may not be soft-deleted, you could use the withTrashed method like so:

$author = Author::withTrashed()->find(1);

Only soft-deleted models

There may be times when you only want to fetch the soft-deleted models from the database and exclude any non-soft-deleted models. For example, you might want to do this if you're building a "recycle bin" feature for your application that allows users to restore or permanently delete soft-deleted models.

To only retrieve soft-deleted models, you can use the onlyTrashed method on the query builder instance. For example, if you wanted to retrieve only the soft-deleted models, you could write the following query:

$authors = Author::onlyTrashed()->get();

Likewise, if you wanted to retrieve a single soft-deleted model, you could use the onlyTrashed method like so:

$author = Author::onlyTrashed()->find(1);

Using the `DB` facade

So far, our code examples have revolved around using functionality provided by the SoftDeletes trait to query our soft-deleted models. However, there may be times when you want to write a query using the DB facade. For example, if you're building an in-depth report for your application using a more complex query, you may want to do this.

If you do choose to query a soft-deletable model using the DB facade, there's a common "gotcha" that you need to be aware of. I've seen this affect many developers, including myself, so it's worth mentioning here.

The SoftDeletes trait registers the SoftDeletingScope query scope on the soft-deletable model, which means it can be applied to the Illuminate\Database\Eloquent\Builder object used when building an Eloquent query (as shown above). However, when you use the DB facade, you're not using the Illuminate\Database\Eloquent\Builder object; instead, you're using an Illuminate\Database\Query\Builder object.

Thus, the SoftDeletingScope is not applied to the query, and if you write a query using the DB facade, soft-deleted models will be returned in the results. This means you'll need to make sure to manually exclude soft-deleted models from your query.

Let's take a look at a basic example of this. Imagine we have the following query using the DB facade to get all the authors:

$authors = DB::table('authors')->get();

If we ran the above code, the soft-deleted and non-soft-deleted models would all be fetched from the database. If we wanted to update the query to exclude soft-deleted models, we could do the following:

$authors = DB::table('authors')
    ->whereNull('deleted_at')
    ->get();

As you can see in the code example, we've used the whereNull method to exclude any soft-deleted models from the query.

It's important that you remember to add this when building queries. A great way of checking that you've written queries correctly is to ensure that you have a good-quality test to assert that no soft-deleted models are returned in the query. We'll cover this in more depth later in the article.

Soft-deleted models and route model binding

In Laravel, you can use "route model binding" when defining routes to automatically inject a model instance into your controller method. This makes it easy to fetch a model instance from the database and have it ready for use directly in your controller. For example, if you wanted to fetch an Author model instance from the database and have it available in your controller, you could define the following route:

Route::get('/authors/{author}', [AuthorController::class, 'show']);

In the show method of the AuthorController, we can then type-hint the Author model and have it injected into the method like so:

public function show(Author $author)
{
    // ...
}

If a user now navigated to /authors/1, the $author variable would contain the Author model instance for the author with an ID of 1. This is a really handy feature!

However, by default, route model binding ignores soft-deleted models. This means that if you were to navigate to /authors/1, but the author with an ID of 1 was soft-deleted, you would receive an HTTP 404 response.

In most cases, this is likely the desired behavior. However, there may be times when you'd like to include soft-deleted models in the binding. For example, you may want to build an admin panel that allows you to view and restore soft-deleted models. To do this, you can use the withTrashed method on the route model binding definition. For example, we could update the route definition to the following:

Route::get('/authors/{author}', [AuthorController::class, 'show'])->withTrashed();

It's important to note that this won't use only soft-deleted models in the bindings. Instead, it will include both soft-deleted and non-soft-deleted. Therefore, if you'd only like to include soft-deleted models, you'll need to add a check to your controller to ensure that the model is soft-deleted. For example, we could update the show method in the AuthorController to the following:

public function show(Author $author)
{
    // If the author is not soft-deleted, return a 404 response.
    if (!$author->trashed()) {
        abort(404);
    }

    // Continue as normal...
}

Testing soft deletes

Like any other part of your application, it's important that you remember to test your code. It can help to give you confidence that your code runs as expected and can reduce the chance of bugs being introduced when you make changes in the future.

Thanks to some helpful methods provided by Laravel, we can easily test our soft-deleted models. Let's take a look at some common tests you might want to write for your soft-deleted models and tips for writing them.

Assert whether a model is soft-deleted

A common assertion that you'll likely want to make when testing soft-deleted models is testing whether a model is soft-deleted. There are multiple ways that you can achieve this in Laravel.

First, as we've already covered, you can use the trashed method on the model to check whether the model is soft-deleted. For example, let's imagine we have a route and controller (accessible by DELETE /authors/{author} using the route name authors.destroy) that allows a user to soft-delete a model. We could write the following test to assert that the author is soft-deleted:

use App\Models\Author;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

#[Test]
public function author_can_be_soft_deleted(): void
{
    $author = Author::factory()->create();

    $this->delete(route('authors.destroy', $author))
        ->assertOk();

    $this->assertTrue($author->fresh()->trashed());
}

Alternatively, we could use the assertSoftDeleted helper method provided by Laravel. We could rewrite the above test to use this method like so:

use App\Models\Author;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

#[Test]
public function author_can_be_soft_deleted(): void
{
    $author = Author::factory()->create();

    $this->delete(route('authors.destroy', $author));
        ->assertOk();

    $this->assertSoftDeleted($author);
}

Similar to our two approaches above, if we wanted to assert that a model is not soft-deleted, we could use the assertNotSoftDeleted helper method provided by Laravel. For example, we might have a route and controller (accessible by POST /authors/{author}/restore using the route name authors.restore) that we can use to restore soft-deleted authors. If we wanted to write a test to assert that a model is not soft-deleted, we could write the following:

use App\Models\Author;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

#[Test]
public function author_can_be_restored(): void
{
    $author = Author::factory()
        ->trashed()
        ->create();

    $this->post(route('authors.restore', $author))
        ->assertOk();

    $this->assertNotSoftDeleted($author);
}

You may have noticed in the test above that we called a trashed method on the Author model's factory. This method is provided by Laravel and allows us to create soft-deleted models. This can be really useful when you want to quickly create a soft-deleted model in your tests.

If you'd like to test that a model has been permanently deleted from the database, you can use the assertModelMissing helper method provided by Laravel. For example, if we had a route and controller (accessible by DELETE /authors/{author}/force using the route name authors.force) that we could use to permanently delete a soft-deleted model, we could write the following test to assert that the model has been permanently deleted:

use App\Models\Author;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

#[Test]
public function soft_author_can_be_permanently_deleted(): void
{
    $author = Author::factory()
        ->trashed()
        ->create();

    $this->delete(route('authors.force', $author))
        ->assertOk();

    $this->assertModelMissing($author);
}

If the route only allows soft-deleted models to be permanently deleted and returns an HTTP 404 response if we attempt to permanently delete a non-soft-deleted model, we could use the assertModelExists to write the following test to assert that the model is not permanently deleted:

use App\Models\Author;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

#[Test]
public function non_soft_author_cannot_be_permanently_deleted(): void
{
    $author = Author::factory()->create();

    $this->delete(route('authors.force', $author))
        ->assertNotFound();

    $this->assertModelExists($author);
}

Assert that soft-deleted models are included or excluded from queries

You'll likely want to write tests that assert that soft-deleted models are included or excluded from your queries. These types of tests can provide peace of mind and give you confidence that you're not accidentally including soft-deleted models in your queries.

Let's take the following basic controller method as an example that returns a view containing all the authors (excluding soft-deleted authors):

class AuthorController extends Controller
{
    public function index()
    {
        return view('authors.index', [
            'authors' => Author::all(),
        ]);
    }
}

To ensure that we're not accidentally including soft-deleted authors in our query, we could write the following test:

use App\Models\Author;
use Illuminate\Database\Eloquent\Collection;
use PHPUnit\Framework\Attributes\Test;
use Tests\TestCase;

#[Test]
public function authors_view_is_returned(): void
{
    // Create two non-soft-deleted authors. They should be
    // included in the results.
    $authors = Author::factory()
        ->count(2)
        ->create();

    // Create a soft-deleted author. They should not be
    // included in the results.
    Author::factory()
        ->trashed()
        ->create();

    $this->get(route('authors.index'))
        ->assertOk()
        ->assertViewIs('authors.index')
        ->assertViewHas(
            key: 'authors',
            value: fn (Collection $authors): bool =>
                $authors->pluck('id')->toArray() === [
                    $authors[0]->id,
                    $authors[1]->id,
                ]
        );
}

As we can see in the test, we're explicitly checking that the Collection being passed to the view only includes the two non-soft-deleted authors. By doing this, we can be sure that we're not accidentally including soft-deleted authors in our query.

In fact, this general approach of creating a model that doesn't satisfy one of the query constraints is quite handy when testing any type of query. You can use it across your all application's tests to ensure that your query constraints are working as expected and that you're only returning the models that you expect to return. This can be particularly useful when testing complex queries with many conditional constraints.

Conclusion

This article should have given you an insight into what soft deletes are, as well as the advantages and disadvantages of using them. You should now be able to use soft deletes in your own Laravel applications and have extra confidence in your ability to test them. You should also be aware of a common pitfall to avoid when using soft deletes and writing database queries using the DB facade.

The definitive guide to object-oriented programming in PHP

Honeybadger Staff — Mon, 01 Apr 2024 07:00:00 +0000

This article was originally written by Adebayo Adams on the Honeybadger Developer Blog.

Organizing your code properly is a time-saver when building large applications and working across teams as a programmer. You should always strive to write organized, reusable, and, most importantly, maintainable code so that you can work efficiently across teams with fewer conflicts during development. In this article, you will learn everything you need to know about object-oriented programming and its benefits in your PHP code.

Prerequisites

To follow along with this article, you only need to know the basics of PHP. You should be familiar with some concepts, such as variables, functions, conditional statements, and loops.

Object-oriented programming in PHP

Object-oriented programming (OOP) is a programming paradigm that enables developers to structure and treat their applications like real-world objects by using classes to bundle data and functions in a single unit and using it throughout the application, thereby improving readability, reusability, maintainability, and more.

Aside from being one of the most popular programming paradigms, it is also the most used in PHP. Most PHP frameworks and CMSs, such as Laravel, Drupal, Symphony, October, CodeIgniter, and many others use OOP at their core. Even parts of the world's most popular website builder, WordPress, uses OOP.

Next, let's explore how to use classes in PHP.

Classes

As mentioned in the previous section, object-oriented programming is based on classes, and we'll explore them in this section. For example, if you want to create a User class, you can use the class keyword:

<?php
class User {

}
?>

That's it! You've just created your first class in PHP.

Class properties

PHP allows you to add properties to classes. Class properties are variables you define within the class curly brackets { } and can be used inside and outside the class. You can define a property inside a class:

class User {
    public $name = 'John Sanders';
    public $age = 25;
    protected $email = 'jsanders@ml.com'
    private $ssn = '123-45-6789';
}

The code above defines two public properties ($name and $age), one protected property ($email), and one private property ($ssn) inside the User class. All class properties are defined by prefixing the property with public, protected, or private keywords.

There are three levels of access in PHP. Public properties can be accessed inside and outside the class, and protected properties can be used inside the class that defines them and the classes that inherit them. In contrast, private properties can only be accessed within the class that defines them.

Class methods

You can add functions inside classes by defining them within the class' curly braces { }:

class User {
    // same as above...
    function greet() {
        echo 'Hello,  ' . $this->name . ' and welcome to the site!';
    }
}

The code above defines a greet function that prints a message to the user. Functions can also be either public or private, and you can define public, protected, and private functions inside the class by prefixing them with either public, protected, or private keywords:

class User {
    // same as above...

    public function getSSN() {
        return $this->ssn;
    }

    private function setSSN($ssn) {
        $this->ssn = $ssn;
    }
}

The code above declares one public function, getSSN, and one private function, setSSN. The $this keyword is used to access the class properties and methods inside the class, which means you have to use $this->age instead of $age to access the public age property.

Creating objects

The User class you created in the previous sections doesn't do anything yet; to use the class, you need to create an object from the class. For example, to create an object based on the User class, you use the new keyword:

$user = new User;

The code above created an object, user1, using the User class as a blueprint. You can now use the properties and functions of the user class like this:

echo $user->name . ' is ' . $user->age . ' years old.';
echo "<br>";
echo $user->greet();
echo "<br>";
echo $user->getSSN();
echo "<br>";

The code above uses the single arrow (->) syntax to access the properties and functions of the user object and prints out the following in the browser:

The class you created in this section is not scalable because every object created with it will always have the same name, age, email, and SSN values. Let's explore how to make it dynamic in the following sections.

The `__construct` method

PHP provides a magic method, __construct, also known as the constructor method. It is used inside classes to initialize properties of new objects. Let's rewrite the User class to use a constructor method:

class User {
    public $name;
    public $age;
    public $email;
    private $ssn;

    function __construct($name, $age, $email, $ssn) {
        $this->name = $name;
        $this->age = $age;
        $this->email = $email;
        $this->ssn = $ssn;
    }

    function greet() {
        echo 'Hello,  ' . $this->name . ' and welcome to the site!';
    }

    public function getSSN() {
        return $this->ssn;
    }

    private function setSSN($ssn) {
        $this->ssn = $ssn;
    }
}

The code above creates a dynamic User class by requiring every object to provide its name, age, email, and SSN properties upon creation. You can now create an object on the User class:

$user1 = new User('Akanni William', 35, "williamak@ml.com", '123-45-6789');

$user2 = new User('Alabi Brandon', 25, "brandonal@cl.com", '987-65-4321');

echo $user1->name . ' is ' . $user1->age . ' years old.';
echo "<br>";
echo $user1->greet();
echo "<br>";
echo $user1->getSSN();

echo "<br>"; 
echo "<br>";

echo $user2->name . ' is ' . $user2->age . ' years old.';
echo "<br>";
echo $user2->greet();
echo "<br>";
echo $user2->getSSN();

The code above will print the following result using the name, age, email, and SSN properties upon object creation:

With this syntax, you can now create as many objects as you want with different details. Additionally, you will get an error if you do not provide all the properties listed inside the constructor for every object you create.

Note: Class names are case-insensitive, which means you can create a user object like this: $user2 = new user(... or like this: $user1 = new USer(.... However, you should always try to be consistent with naming by using the conventional camel-case, which starts with an uppercase letter.

Chaining class methods

PHP helps developers to write concise code by allowing chain class methods. For example, if you have a class that handles login and logout, you might also want to change the user's status accordingly. Here is how you can do that in PHP:

class Twita {
    function __construct(public $username, public $offlineStatus)
    {
        $this->username = $username;
        $this->offlineStatus = $offlineStatus;
    }

    function login(){
        echo "Welcome, " . $this->username . " to your account ";
        echo  "<br>";
        return $this;
    }

    function logout(){
        echo $this->username . "has logged out";
        echo  "<br>";
        return $this;
    }

    function setOnlineStatus(){
        $this->offlineStatus = false;
        echo $this->username. "is now online";
        echo  "<br>";
        return $this;
    }

    function setOfflineStatus(){
        $this->offlineStatus = true;
        echo $this->username. "is now offline";
        echo  "<br>";
        return $this;
    }
}

The code above declares a Twita class with username and offlineStatus properties and login, logout, setOnlineStatus, and setOfflineStatus methods. To chain the methods, you can use the single arrow (->) syntax:

$twit = new Twita('Abolade William', true);
$twit->login()->setOnlineStatus()->logout()->setOfflineStatus();

You should get a response that looks like this:

Note: For method chaining to work, all the functions need to return $this at the end of each function.

Common class constants

The following magic constants in PHP are especially useful when working with classes:

The CLASS constant returns the name of the class where it is used.
The FILE constant returns the full path of the file where it is used.
The LINE constant returns the line where it is used.
The METHOD constant returns the name of the method where it is used.

Here's an example of the constants inside a class:

Class Vehicle {
    public $name;
    public $color;
    public $price;
    public $brand;

    function __construct($name, $color, $price, $brand) {
        $this->name = $name;
        $this->color = $color;
        $this->price = $price;
        $this->brand = $brand;
    }

    public function getClassConstant(){
        echo "Name of class is: " .__CLASS__ . "<br>";
    }

    public function getFileConstant(){
        echo "Full path of file is: " . __FILE__ . "<br>";
    }

    public function getLineConstant(){
        echo "Line of code is: " . __LINE__ . "<br>";
    }

    public function getMethodConstant(){
        echo "Name of method is: " . __METHOD__ . "<br>";
    }
}

The code above defines a class Vehicle with four properties ($name,$color,$price, and $brand) and four methods, (getClassConstant, getFileConstant, getLineConstant, and getMethodConstant) that returns the respective values of the constants. You can now use the Vehicle class like this:

$vehicle = new Vehicle('Lexus', 'Black', 100000, 'Lexus');
$vehicle->getClassConstant();
$vehicle->getFileConstant();
$vehicle->getLineConstant();
$vehicle->getMethodConstant();

The code above should return the following:

The constants declared in the block of code above are particularly useful, especially when building developer tools and you need to print out the line of code, file, or function preventing the code from running correctly.

Comparing objects

If, for any reason, you want to compare objects of a class in PHP, you can do so with the equals operator (==) and the strict equals operator (===):

class Numbers {
    public $num1;
    public $num2;
    public $num3;

    function __construct($num1, $num2, $num3) {
        $this->num1 = $num1;
        $this->num2 = $num2;
        $this->num3 = $num3;
    }

    function add() {
        return $this->num1 + $this->num2 + $this->num3;
    }

    function multiply() {
        return $this->num1 * $this->num2 * $this->num3;
    }

    function divide() {
        return $this->num1 / $this->num2 / $this->num3;
    }

    function subtract() {
        return $this->num1 - $this->num2 - $this->num3;
    }
}

$numbers1 = new Numbers(10, 5, 2);
$numbers2 = new Numbers(10, 5, 2);

echo ($numbers1 == $numbers2) ? 'True' : 'False'; // True
echo ($numbers1 === $numbers2) ? 'True' : 'False'; // False

The code above defines a Numbers class that takes in three numbers and functions that perform basic arithmetic operations on them. Then, it creates two instances of the class and compares them with the equals and strict equals operators. The first comparison will return true because both instances hold the same values; however, the second check is strict and will return false because the objects are of different instances.

Encapsulation in PHP

Encapsulation in programming refers to bundling related data and functionalities in a single unit. This approach improves reusability, readability, and maintainability.

Encapsulation can be achieved in PHP by using classes to organize your code. For example, if you are building an e-commerce application, you can have a class that handles user authentication and another that handles the checkout process. This allows multiple developers to collaborate and work on a project with minimal or no conflict while simultaneously focusing on different parts of the code.

Class inheritance

When developing applications using an object-oriented paradigm, you can write even lesser code using class inheritance in PHP. For example, when building an e-commerce application, you might have a class called Product with properties like name, price, description, and image and methods like formatPrice and addToCart. You can create the class like this:

class Product {
    public $name;
    public $price;
    public $desc;
    public $image;

    function __construct($name, $price, $desc, $image) {
        $this->name = $name;
        $this->price = $price;
        $this->desc = $desc;
        $this->image = $image;
}
    final function formatPrice(){
        return "$". $this->price . "<br>";
    }
    public function addToCart(){
        return $this->name . " has been added to cart  <br> <br>";
    }
}

The code above defines a class Product with four properties ($name, $price, $desc, and $image) and two methods (formatPrice and addToCart).

Note: The final keyword that precedes the formatPrice functions means that the function cannot be overridden inside the classes that inherit the Product class.

You can now create multiple classes based on the Product class:

class Laptop extends Product {
    public $cpu;
    public $ram;
    public $storage;

    function __construct($name, $price, $desc, $image, $cpu, $ram, $storage) {
        parent::__construct($name, $price, $desc, $image);
        $this->cpu = $cpu;
        $this->ram = $ram;
        $this->storage = $storage;
    }

    public function addToCart(){
        return $this->name . " has been overidden  <br> <br>";
    }

    public function formatLaptopSpecs(){
        return $this->name . "<br>" . "i" . $this->cpu . " Processor <br>" . $this->ram . "GB RAM <br>" . $this->storage . "GB SSD <br> <br>";

    }
}

class Phone extends Product {
    public $os;
    public $ram;
    public $storage;

    function __construct($name, $price, $desc, $image, $os, $ram, $storage) {
        parent::__construct($name, $price, $desc, $image);
        $this->os = $os;
        $this->ram = $ram;
        $this->storage = $storage;
    }

    public function formatPhoneSpecs(){
        return $this->name . "<br>" . $this->os . "<br>" . $this->ram . "GB RAM <br>" . $this->storage . "GB <br> <br>";

    }
}

class TV extends Product {
    public $screenSize;
    public $resolution;
    public $refreshRate;

    function __construct($name, $price, $desc, $image, $screenSize, $resolution, $refreshRate) {
        parent::__construct($name, $price, $desc, $image);
        $this->screenSize = $screenSize;
        $this->resolution = $resolution;
        $this->refreshRate = $refreshRate;
    }

    public function formatTVSpecs(){
        return $this->name . "<br>" . "Screen Size: " . $this->screenSize . "<br>" . "Resolution: " .$this->resolution . "<br>" . "Refresh Rate: " .$this->refreshRate . "<br> <br>";
    }
}

The code above defines three different products with additional properties and methods by extending the Product class. The extends keyword is used to extend the Product class, also known as the parent class in this case.

Note: The Laptop class overrode the Product class' addToCart function by defining another addToCart function of its own.

You can now use the classes like this:

$tv = new TV('LG TV', 1000, 'A TV', 'image.jpg', '55"', '4K', '60Hz');
echo $tv->formatTVSpecs();
echo $tv->addToCart();

$phone = new Phone('iPhone 12', 1000, 'A phone', 'image.jpg', 'iOS', 8, 256);
echo $phone->formatPhoneSpecs();
echo $phone->addToCart();

$laptop = new Laptop('Macbook Pro', 1000, 'A laptop', 'image.jpg', '5', 16, 512);
echo $laptop->formatLaptopSpecs();
echo $laptop->addToCart();

The code above defines one instance of the classes that extends the Product class and will return the following result:

Note: A class that extends another class is called a child class, while a class that is being extended is called a parent class.

Benefits of inheritance

Inheritance allows you to reuse code by creating new classes that inherit the attributes and methods of existing classes. This can save a lot of time since you don't have to write the same code repeatedly. It can make code easier to maintain because if a bug is found in the parent class, it only needs to be fixed once, and all the child classes will automatically inherit the fix.

Inheritance creates a hierarchical class structure, making the code easier to understand and organize. You can use the parent-child relationship to group classes into logical categories, making navigating and maintaining the codebase easier. Finally, it enables you to add new features or functionality to an application without modifying the existing code.

Static members

PHP allows you to define properties and methods bound to the class and not instances using the static keyword. We'll explore how to do that in this section.

Static properties

Static properties are only available to use on the class itself and not the instances. For example, if you want to keep track of how many times the User class has been used, you can do so like this:

class User {
    public $name;
    public $email;
    public $password;

    function __construct($name, $email, $password) {
        $this->name = $name;
        $this->email = $email;
        $this->password = $password;
    }
    static $usersCount = 0;
}

The code above defines a static property, usersCount, inside the User class. You can now access the property by prefixing it with the self:: syntax:

$user = new User('Abolade William', 'willabolade@ml.com', 'password');
echo User::$usersCount; // 0

The code above will return the value of usersCount, 0.

Note: You must add the $ to the property.

Let's take a look at static methods next.

Static methods

If you want to update the usersCount every time an instance of User is created, you can use the static method:

class User {
    // same as above...

    static function updateUsersCount(){
        self::$usersCount++;
        return self::$usersCount . " users registered";
    }
}

The code above defines an updateUsersCount method that updates the usersCount property. You can now call the updateUsersCount function on the class each time an instance of the User class is created:

$user = new User('Abolade William', 'willabolade@ml.com', 'password');
echo User::updateUsersCount(); // 1 users registered
echo User::$usersCount; // 0

Note: You can also access static members inside the class with the class name instead of self:: like so: User::$usersCount.

Abstract classes and methods

PHP allows you to define classes with functions that are not defined yet but forces classes that extend them to define their implementation. Abstract classes and methods are created by prefixing the class and method names with the abstract keyword.

For example, if you want to force every child class that extends the Product class to implement a formatSpec function, you can do so with an abstract class:

abstract class Product {
    public $name;
    public $price;
    public $desc;
    public $image;

    function __construct($name, $price, $desc, $image) {
        $this->name = $name;
        $this->price = $price;
        $this->desc = $desc;
        $this->image = $image;
    }

    public function addToCart(){
        return $this->name . " has been added to cart  <br> <br>";
    }

    abstract function formatSpec();
}

The code above defines an abstract Product class with an abstract formatSpec method. You cannot create an object based on an abstract class. You need to create classes that extend it.

Note: An abstract class must have at least one abstract method.

You can now create a child class that extends the abstract class:

class Laptop extends Product {
    public $cpu;
    public $ram;
    public $storage;

    function __construct($name, $price, $desc, $image, $cpu, $ram, $storage) {
        parent::__construct($name, $price, $desc, $image);
        $this->cpu = $cpu;
        $this->ram = $ram;
        $this->storage = $storage;
    }

    public function formatSpec(){
        return $this->name . "<br>" . "i" . $this->cpu . " Processor <br>" . $this->ram . "GB RAM <br>" . $this->storage . "GB SSD <br> <br>";
    }
}

The code above defines a Laptop class that extends the Product class and implements the formatSpec function inside it.

Note: You will get an error if you don't implement the abstract methods inside the classes that extend abstract classes.

Interfaces

PHP takes abstraction to the next level by allowing you to define functions that classes must implement inside an interface. An interface is similar to an abstract class but can only have public functions and cannot contain properties and defined methods. Here's an example of an interface:

interface ProductInterface {
    public function addToCart();
    public function formatSpec();
}

The code above defines a ProductInterface with two methods that the child class has to implement. A child class that implements the ProductInterface will look like this:

class Phone implements ProductInterface {
    public $name;
    public $price;
    public $desc;
    public $image;
    public $os;
    public $ram;
    public $storage;

    function __construct($name, $price, $desc, $image, $os, $ram, $storage) {
        $this->name = $name;
        $this->price = $price;
        $this->desc = $desc;
        $this->image = $image;
        $this->os = $os;
        $this->ram = $ram;
        $this->storage = $storage;
    }

    public function addToCart(){
        return $this->name . " has been added to cart  <br> <br>";
    }

    public function formatSpec(){
        return $this->name . "<br>" . $this->os . "<br>" . $this->ram . "GB RAM <br>" . $this->storage . "GB <br> <br>";
    }
}

The code above defines a Phone class based on the ProductInterface using the implements keyword.

Note: The child class must implement all the methods inside the interface that it implements, or the code will throw an error.

Polymorphism

The word polymorphism originated from two Greek words, poly, which means different or many, and morph, which means form or shape. Polymorphism is a fundamental concept in OOP that allows objects of different classes to be used interchangeably while retaining their unique behavior. It enables you to write more flexible, modular, and easier-to-maintain code.

Polymorphism in PHP is achieved by using abstract classes or interfaces to write code that can be easily extended and modified without modifying the existing code. It also allows you to write more generic code that can work with objects of different classes rather than writing separate code for each class. This reduces code duplication and increases code reusability, making your code more efficient and maintainable.

Traits

PHP allows you to define and use functions that don't fit in a class with traits. A trait is a collection of functions that can be used inside classes without inheriting, extending, or implementing a class or interface.

For example, you can define a trait that contains an addBrandLogo function, which can be used in all other classes to add a brand logo to product images:

trait AllProductsFunctions {
    public function addBrandLogo(){
        return "Brand logo added";
    }
}

The code above defines a trait (AllProductsFunctions) with a method (addBrandLogo) inside it. This trait can now be used inside other classes using the use keyword:

  class Phone implements ProductInterface {
// same as before...
    use AllProductsFunctions;
// same as before...
}

The function inside the AllProductsFunctions trait can now be used on all objects based on the Phone class. You can use multiple traits in a class by separating the traits' names with a comma.

Note: You can override a trait method by defining a method with the same name inside the class, and a trait method with the same name will override the method that the class inherited.

Benefits of object-oriented programming

Now that you have learned all the basics and advanced concepts you need to know to begin object-oriented programming in PHP and how it helps you to design and create scalable, maintainable, and reusable applications, let's explore some of the significant benefits of OOP in the following sections.

Code reusability

One of the primary benefits of OOP in PHP is code reusability. This means that you can reuse code blocks and functions throughout applications without having to rewrite the entire code from scratch. With OOP, you can create objects that encapsulate a specific set of functionalities and reuse these objects across multiple applications or even within the same application. This helps to reduce the development time and effort required to create complex applications while improving the overall quality of the code.

Scalability

OOP in PHP also makes it easier to scale applications as they grow. By using objects to encapsulate different parts of the application, you can modify or add new functionalities without affecting other parts of the codebase. This makes it easier to maintain and update applications while reducing the risk of introducing bugs or errors. Additionally, OOP makes it easier to modularize code, allowing multiple developers to work on specific parts of the application without needing to understand the entire codebase.

Encapsulation

Encapsulation is a crucial principle of OOP in PHP that helps improve an application’s security and maintainability. With encapsulation, you can hide the internal workings of an object from the rest of the application. This means that the object can only be accessed through its public interface, which helps to prevent unauthorized access to critical parts of the application. Encapsulation also makes it easier to modify or update the internal workings of an object without affecting other areas of the codebase.

Inheritance

Inheritance is another important concept in OOP that allows developers to create new classes based on existing classes. With inheritance, you can inherit the properties and methods of a parent class and then modify or extend them to create new functionality. This makes reusing code easier, reduces redundancy, and creates more efficient applications. Additionally, inheritance helps to promote consistency across different parts of an application, which makes it easier to maintain and update the codebase.

Testability

Finally, OOP in PHP makes it easier to test and debug applications. By encapsulating code into objects, you can isolate specific parts of the application and test them in isolation from other areas of the codebase. This helps to identify bugs and errors more quickly and makes it easier to fix them without affecting the rest of the application. Additionally, OOP makes it easier to write automated tests that can be used to test the application at scale, which helps to improve the overall quality of the code.

Using OOP in PHP offers a range of benefits that can help you create more scalable, maintainable, and reusable applications. As such, OOP has become a popular programming technique for building modern web applications and will likely remain a vital part of the developer toolkit for years.

Conclusion

And that's it! I hope this article achieved its aim of teaching you object-oriented programming in PHP. You learned about classes in PHP, including class properties, class methods, chaining class methods in PHP, common class constants in PHP, and class inheritance. We also explored how to use static members, abstract classes and methods, interfaces, polymorphism, and traits.

Finally, you learned some of the significant benefits of object-oriented programming in PHP.

A step-by-step guide to building a Ruby gem from scratch

Honeybadger Staff — Tue, 26 Mar 2024 07:00:00 +0000

This article was originally written by Julie Kent on the Honeybadger Developer Blog.

If you have worked with Ruby or Rails, you have probably heard of or used gems. Gems are libraries that can be used across code bases to provide useful functionality. While seemingly mystical, you can easily create your own gems and publish them for others to use. This article will de-mystify gems and show you how to build your own.

tl;dr

This article discusses how to create a Ruby gem from scratch, without third-party tooling; keep reading to learn the minimum information that Ruby needs to create a new gem. Or—to generate a complete boilerplate Ruby gem fast—you can use Bundler.

What are Ruby gems?

To put it simply, a Ruby "gem" is a self contained Ruby library. The code within these libraries often does something helpful that can easily be integrated into other projects. For example, there are Ruby gems for various purposes, including the following:

Authentication
Debugging
Styling
Integrating with databases
Logging and error handling

Personally, I think whoever came up with the term "gem" deserves a marketing award. It makes gems seem so magical, but don't worry, we will demystify how simple it is to publish your own.

The difference between a Ruby gem and RubyGems

Before we get started with our tutorial on how to create and publish your own gem, I thought it'd be helpful to clarify one thing first. While there are Ruby gems, there is also something called RubyGems; notice the capitalization and lack of a space in the second term. RubyGems is a package manager for Ruby, essentially a tool designed to easily manage the installation of gems, and a server for distributing them. You may not have realized the difference because, since Ruby v1.9, RubyGems has been bundled with the standard Ruby package. However, if you've ever used the command gem in your CLI, then surprise! You have been utilizing RubyGems.

What our gem will do

We'll build a gem to convert a temperature from Celsius to Fahrenheit. Of course, this is simple functionality, but it's also not useless and has enough complexity that you'll gain a solid understanding of the basic building blocks involved in creating a gem.

Set up

First, open your shell of choice and create a new directory with the name of your gem. I'll name mine "julies_temperature_check_gem":

mkdir julies_temperature_check_gem
cd julies_temperature_check_gem

Next, create a subdirectory called "lib"; this is where the code for your package will be installed. The convention is to have one Ruby file with the same name as your gem:

mkdir lib
touch lib/julies_temperature_check_gem.rb

We will also need to create a ".gemspec" file (we'll discuss this in the next section):

touch julies_temperature_check_gem.gemspec

Understanding the .gemspec file

Every gem needs to have a .gemspec file. This file serves as the interface to RubyGems.org, and it houses important information, such as what's in the gem, who made it, and the version. Certain fields are required in every .gemspec file:

authors
files
name
summary
version

There are also a number of optional fields that you can include in your .gemspec file. The following fields are recommended by RubyGems:

description
email
homepage
license
licenses
metadata
required Ruby version

I believe most of these fields are self-explanatory, but if you'd like to learn more about any of the fields above or the additional optional fields that aren't part of the recommended set, you can check them out here.

Let's open our gemspec file in the editor and add the required fields (of course, edit the obvious ones with your name, version, etc.)

Gem::Specification.new do |s|
  s.name        = "julies_temperature_check_gem"
  s.version     = "0.0.0"
  s.summary     = "Converts Celsius to Fahrenheit"
  s.authors     = ["Julie Kent"]
  s.files       = ["lib/julies_temperature_check_gem.rb"]
end

Writing our code

Let's write the code for our gem! Open the Ruby file you created during the set-up process. We'll create a simple class and our method to perform the conversion.

class JuliesTemperatureCheckGem
  def self.convert_temperature(input)
    puts "#{input.to_f * 9 / 5 + 32}"
  end
end

Build and install

Because we have created our gemspec file, we can build a gem from it! All we need to do is run the following:

gem build julies_temperature_check_gem.gemspec

You should see the following output:

gem build julies_temperature_check_gem.gemspec
  Successfully built RubyGem
  Name: julies_temperature_check_gem
  Version: 0.0.0
  File: julies_temperature_check_gem-0.0.0.gem

Cool! Now let's install it! To do so, run the following:

gem install ./julies_temperature_check_gem-0.0.0.gem

You should see the following output:

Successfully installed julies_temperature_check_gem-0.0.0
Parsing documentation for julies_temperature_check_gem-0.0.0
Installing ri documentation for julies_temperature_check_gem-0.0.0
Done installing documentation for julies_temperature_check_gem after 0 seconds
1 gem installed

Testing and publishing

This is the moment of truth! We need to open IRB and see if it works!

irb
3.1.2 :001 > require "julies_temperature_check_gem"
=> true
3.1.2 :002 > JuliesTemperatureCheckGem.convert_temperature(5)
41.0
=> nil

Amazing! We have created our very own gem. If we wanted to, we could publish our gem to RubyGems.org so that others in the Ruby community could use it if desired. Since this is such a simple example, we won't actually this, but the steps are very simple.

First, you need to sign up for a RubyGems account. You can do so here.
Then, from your console, run the following: gem signin You'll be asked to enter your credentials.
Once you are signed in, you just need to push the gem up to RubyGems.org by running gem push julies_temperature_check_gem-0.0.0.gem.

What about Bundler?

One question that might be on your mind is, "What about Bundler?" For those who aren't familiar, Bundler is a tool that helps manage RubyGems dependencies in Ruby libraries. If you've ever worked on a Rails project, chances are you're familiar with Bundler and running commands like bundle install.

You can also use Bundler to write your own gems. It helps speed up the process, as it creates a scaffold directory and initializes a Git repository automatically. To start, all you need to do is run the following:

bundle gem <name of gem>

I highly recommend using Bundler if you want to build your own gem. You can read more about how to use Bundler here.

Wrap-up

Congratulations! You now know what it takes to create a gem and better understand why gems (and RubyGems) are important in the Ruby ecosystem. If you are excited about gems, there are a lot of opportunities to help with open source gems. Here's a good place to get started!

Setting up a WebSocket server in Node.js

Honeybadger Staff — Mon, 25 Mar 2024 07:00:00 +0000

This article was originally written by Samson Omojola on the Honeybadger Developer Blog.

In today's digital landscape, real-time communication has become increasingly important for web applications. Traditional HTTP-based communication, while suitable for certain scenarios, is not designed for persistent, low-latency connections. This is where WebSockets come into play. WebSockets provide a bi-directional communication channel between a client and a server, enabling real-time data transfer over a single TCP connection. In this article, we will dive into the world of setting up a WebSocket server using Node.js. By harnessing the power of JavaScript on the server-side, we can build robust WebSocket servers that cater to the demands of real-time web applications.

Advantages of WebSockets

Full-Duplex Communication: Unlike traditional HTTP, where the client initiates a request and the server responds, WebSockets allow for simultaneous communication in both directions. This bidirectional nature facilitates real-time updates, live collaboration, and interactive applications.
Lower Latency: WebSockets eliminate the need for repeated HTTP requests, reducing network overhead and latency. The persistent connection established by WebSockets allows for instant data transmission, providing a more responsive and interactive user experience.
Lightweight Protocol: The WebSocket protocol is designed to be lightweight and efficient. It utilizes a simple handshake process during the initial connection and minimizes the overhead associated with headers and cookies, making it an optimal choice for real-time communication.
Scalability: WebSocket connections are designed to handle a large number of concurrent clients, making them suitable for applications that require high scalability. By efficiently managing connections, WebSocket servers can support thousands or even millions of simultaneous connections.
Cross-Domain Support: WebSockets can easily communicate across different domains, enabling developers to build applications that span multiple servers or services. This flexibility is especially useful in scenarios where data needs to be shared between different origins.

Installing dependencies and initializing a WebSocket server in Node.js

Installing required dependencies

Before setting up a WebSocket server in Node.js, we need to install the necessary dependencies. Fortunately, Node.js has a vibrant ecosystem with various WebSocket libraries available. In this article, we will focus on using the popular ws library, which provides a simple and efficient WebSocket implementation for Node.js.

To install the ws library, follow these steps:

Open your terminal or command prompt.
Navigate to your project directory.
Run the following command:

npm install ws

Initializing a WebSocket server

Once we have the ws library installed, we can proceed to initialize a WebSocket server in Node.js. The server will listen for incoming WebSocket connections, handle communication with clients, and facilitate the exchange of real-time data.

To create a basic WebSocket server, perform the following steps:

Create a new JavaScript file, such as server.js, in your project directory.

Require the ws module at the beginning of the file:

const WebSocket = require('ws');

Define a variable to hold the WebSocket server instance:

const wss = new WebSocket.Server({ port: 8080 });

Here, we are creating a WebSocket server that will listen on port 8080. You can modify the port number based on your requirements.

Attach event listeners to handle WebSocket connections and messages:

wss.on('connection', (ws) => {
  // Code to handle new WebSocket connections
});

wss.on('message', (message) => {
  // Code to handle incoming messages
});

In the first event listener, connection, we can define the logic to handle new WebSocket connections. This code block will be executed each time a client establishes a WebSocket connection with the server.

Similarly, in the second event listener, message, we can define the logic to handle incoming messages from clients. This code block will be executed whenever the server receives a WebSocket message from a connected client.

Start the WebSocket server by adding the following line at the end of the file:

console.log('WebSocket server is running...');

This will print a message in the console indicating that the WebSocket server is up and running.

Handling WebSocket connections

Once a client establishes a WebSocket connection with the server, it is crucial to handle and manage these connections effectively.

Inside the connection event listener, you can access the WebSocket connection object (ws) representing the newly connected client. You can perform various operations, such as sending initial data, tracking connected clients, or performing authentication.

wss.on('connection', (ws) => {
  // Send initial data to the client
  ws.send('Welcome to the WebSocket server!');

  // Track connected clients
  // ...
});

To handle disconnections, you can listen for the WebSocket close event:

ws.on('close', () => {
  // Code to handle client disconnection
});

This event will be triggered when a WebSocket connection is closed by the client or due to some error. You can use this event to clean up resources associated with the client or update the list of connected clients.

Handling WebSocket messages

WebSocket communication primarily revolves around exchanging messages between the client and server. Let's explore how to handle incoming messages from clients in Node.js.

Inside the message event listener, you can access the received message and take appropriate actions based on its contents.

wss.on('message', (message) => {
  // Handle incoming messages
  console.log('Received message:', message);

  // Send a response to the client
  ws.send('Message received successfully!');
});

We can perform custom logic based on the received message, such as updating the application state, broadcasting the message to other connected clients, or triggering specific actions.

To send a message to a specific client or all connected clients, you can use the send method of the WebSocket connection object (ws).

// Send a message to the client that triggered the event
ws.send('Hello client!');

// Broadcast a message to all connected clients
wss.clients.forEach((client) => {
  client.send('Broadcast message!');
});

By utilizing the send method, you can establish seamless bidirectional communication with clients, enabling real-time updates and interactive features in your application.

Data serialization and deserialization in WebSocket communication

WebSocket communication involves the exchange of data between the client and server in a structured format. To ensure compatibility and seamless communication, it is essential to serialize and deserialize data appropriately.

Serializing data

Serialization refers to the process of converting data objects into a string representation that can be transmitted over the WebSocket connection. JavaScript Object Notation (JSON) is a widely used data interchange format that provides a simple and human-readable way to represent structured data.

To serialize data into JSON format, you can use the JSON.stringify() method provided by JavaScript. Here's an example of serializing an object:

const data = { name: 'John', age: 25 };
const serializedData = JSON.stringify(data);

The JSON.stringify() method converts the data object into a JSON string, which can be sent as a message over the WebSocket connection.

Deserializing data

Deserialization is the reverse process of serialization, where the received string representation of data is converted back into its original object form. In WebSocket communication, you need to deserialize incoming JSON strings to access and process the data.
To deserialize a JSON string into an object, you can use the JSON.parse() method. Here's an example:

const receivedMessage = '{"name":"Jane","age":30}';
const deserializedData = JSON.parse(receivedMessage);

The JSON.parse() method converts the receivedMessage string into a JavaScript object, allowing you to access its properties and perform further operations.

Sending serialized data

To send serialized data over a WebSocket connection, you can use the send() method provided by the WebSocket connection object (ws). By converting the data into a JSON string, you can ensure its compatibility and efficient transmission.

const data = { name: 'John', age: 25 };
const serializedData = JSON.stringify(data);

ws.send(serializedData);

The send() method sends the serialized data as a WebSocket message to the connected client.

Receiving and deserializing data

When receiving messages from clients, you will need to deserialize the received JSON string to access and process the data.

wss.on('message', (message) => {
  const receivedData = JSON.parse(message);
  // Process the received data
});

Here, the message parameter represents the received WebSocket message. By deserializing the message using JSON.parse(), you can obtain the original data object for further processing.

Advanced features in WebSocket server implementation

WebSocket servers can incorporate advanced features to enhance functionality, security, and scalability. Next, we will explore two important aspects: handling multiple channels and implementing authentication mechanisms.

Handling multiple channels

WebSocket servers often need to handle communication across multiple channels or rooms. Channels allow grouping clients based on their interests, topics, or specific interactions. This enables targeted message broadcasting and efficient data management.

To implement multiple channels in your WebSocket server, you can utilize data structures like dictionaries or arrays to store and manage clients associated with each channel. When a message is received, you can then broadcast it selectively to the clients subscribed to that particular channel.

Here's an example:

const channels = {};

wss.on('connection', (ws) => {
  // Join a specific channel
  const channel = 'general';
  if (!channels[channel]) {
    channels[channel] = [];
  }
  channels[channel].push(ws);

  // Handle incoming messages
  ws.on('message', (message) => {
    // Broadcast message to all clients in the channel
    channels[channel].forEach((client) => {
      client.send(message);
    });
  });

  // Handle disconnections
  ws.on('close', () => {
    // Remove client from the channel
    channels[channel] = channels[channel].filter((client) => client !== ws);
  });
});

In this example, clients joining the WebSocket server are assigned to the 'general' channel by default. However, you can modify the logic to allow clients to specify their desired channel during connection or through specific messages.

Implementing authentication mechanisms

WebSocket servers often require authentication mechanisms to ensure secure communication and restrict access to authorized clients. Authentication enables identifying and verifying clients before granting them access to specific resources or channels.
Implementing authentication in your WebSocket server involves validating client credentials, such as tokens or session information, before accepting their connection. You can utilize libraries like JSON Web Tokens (JWT) or integrate with existing authentication solutions (e.g., OAuth) to authenticate clients.

Here's a simplified example demonstrating authentication using JWT:

const jwt = require('jsonwebtoken');

wss.on('connection', (ws, req) => {
  const token = req.headers.authorization.split(' ')[1];

  // Verify and decode JWT token
  jwt.verify(token, 'your_secret_key', (err, decoded) => {
    if (err) {
      // Handle authentication failure
      ws.close();
      return;
    }

    // Authentication successful
    // Handle further operations or channel assignment
  });
});

In this example, the client is expected to send a JWT token in the authorization header. The server verifies the token using a secret key and performs appropriate actions based on the authentication result.

Real-world use case: real-time stock market data streaming

WebSocket servers are widely used in real-time applications that require continuous streaming of data, such as stock market tracking platforms. Let's explore a real-world use case of a real-time stock market data streaming application, highlighting how WebSocket servers are utilized.

Scenario

Imagine a stock market tracking platform that provides real-time updates on stock prices, market trends, and user-customized watchlists. The application aims to deliver up-to-the-second stock market data to users, allowing them to make informed investment decisions. To demonstrate this example, let's use a real data source, such as the Alpha Vantage API, to fetch real-time stock market data. We'll integrate the axios library to make HTTP requests to the API and emit the data to the clients.

const http = require('http');
const socketIO = require('socket.io');
const axios = require('axios');

const server = http.createServer();
const io = socketIO(server);

// API Configuration
const API_KEY = 'YOUR_ALPHA_VANTAGE_API_KEY';
const API_BASE_URL = 'https://www.alphavantage.co/query';

io.on('connection', (socket) => {
  console.log('New WebSocket connection established.');

  socket.on('subscribe', (symbols) => {
    console.log(`Subscribed to symbols: ${symbols}`);

    // Fetch real-time data for each symbol
    symbols.forEach((symbol) => {
      fetchRealTimeData(symbol)
        .then((data) => {
          socket.emit('data', data);
        })
        .catch((error) => {
          console.error(`Error fetching data for ${symbol}: ${error}`);
        });
    });
  });

  socket.on('disconnect', () => {
    console.log('WebSocket connection closed.');
    // Additional cleanup or logic as needed
  });
});

async function fetchRealTimeData(symbol) {
  try {
    const response = await axios.get(API_BASE_URL, {
      params: {
        function: 'GLOBAL_QUOTE',
        symbol: symbol,
        apikey: API_KEY,
      },
    });

    const data = response.data['Global Quote'];
    return {
      symbol: data['01. symbol'],
      price: data['05. price'],
      change: data['09. change'],
    };
  } catch (error) {
    throw error;
  }
}

server.listen(3000, () => {
  console.log('WebSocket server is listening on port 3000.');
});

Above, we import the necessary modules: http, socket.io, and axios. These modules are required for creating the HTTP server, handling WebSocket connections, and making API requests, respectively.

We create an HTTP server using http.createServer() and initialize Socket.IO with the created server using socketIO(server).

We define constants for the Alpha Vantage API key (API_KEY) and base URL (API_BASE_URL) that will be used to fetch real-time stock market data.

The io.on('connection') event handler is triggered whenever a new WebSocket connection is established. Inside this handler, we log a message indicating the new connection.

The socket.on('subscribe') event handler is triggered when a client sends a subscription request. It receives an array of symbols to subscribe to.

Inside the subscribe event handler, we loop through each symbol in the array and call the fetchRealTimeData function.

The fetchRealTimeData function is an async function that uses the axios library to make an HTTP GET request to the Alpha Vantage API. It fetches the global quote for the specified symbol.

If the API request is successful, we extract the relevant data from the response and create an object with the symbol, price, and change values.

The data object is emitted to the client using socket.emit('data', data), sending the real-time stock market data to the subscribed client.

If an error occurs during the API request or data retrieval, an error is thrown and logged to the console.

The socket.on('disconnect') event handler is triggered when a WebSocket connection is closed. We log a message indicating the disconnection.

Finally, we start the HTTP server and listen on port 3000. Upon server start, we log a message confirming that the WebSocket server is listening.

By integrating the Alpha Vantage API and utilizing Socket.IO, this code enables real-time stock market data streaming to clients connected to the WebSocket server. Clients can subscribe to specific symbols and receive continuous updates of the corresponding stock data in real-time.

Conclusion

In this article, we explored initializing WebSocket servers, handling WebSocket connections and messages in Node.js, data serialization and deserialization in WebSocket communication, and the implementation of a WebSocket server in Node.js for a real-world use case: real-time stock market data streaming.
WebSocket servers offer several advantages for real-time applications, including real-time data updates, reduced network overhead, scalability, and customization. They enable us to build robust and interactive applications that require instant data streaming and real-time collaboration.
By harnessing the power of WebSocket technology, we can create dynamic and responsive applications that deliver real-time data updates and enhance user experiences. Thanks for reading!

Working with DynamoDB in Laravel

Honeybadger Staff — Fri, 22 Mar 2024 07:00:00 +0000

This article was originally written by Samson Omojola on the Honeybadger Developer Blog.

DynamoDB is a fully managed NoSQL database service provided by Amazon Web Services (AWS). It is designed to deliver high performance and scalability while ensuring low latency for read and write operations. DynamoDB offers seamless scalability, automatic data replication, and built-in security features, making it an excellent choice for applications with varying workloads and data access patterns. In this article, I'll show you how to integrate DynamoDB into your Laravel application and how best to work with it.

Why use DynamoDB with Laravel?

Flexibility and Scalability: DynamoDB's flexible schema allows you to store and retrieve data without predefined tables or fixed schemas. This flexibility aligns well with Laravel's expressive and dynamic nature, enabling you to model and store data in a way that suits your application's needs. Additionally, DynamoDB automatically scales its provisioned throughput to handle varying levels of traffic, making it suitable for applications with unpredictable workloads.
High Performance: DynamoDB is designed for low-latency read-and-write operations, even at scale. It achieves this by employing an SSD-backed storage system and distributing data across multiple servers for parallel processing. Laravel, with its efficient routing and caching mechanisms, complements DynamoDB's performance capabilities, allowing you to build responsive and high-performing applications.
Seamless Integration: Laravel provides a comprehensive ecosystem of packages and libraries, including AWS SDK for PHP. This SDK offers easy integration with DynamoDB, allowing you to leverage Laravel's familiar syntax and conventions while working with the database. By using DynamoDB with Laravel, you can harness the power of both technologies seamlessly.
Serverless Architecture: DynamoDB is a serverless database service, which means you don't have to worry about managing infrastructure or provisioning resources. It automatically handles tasks such as hardware provisioning, replication, and backups. When combined with Laravel's support for AWS Lambda and other serverless deployment options, you can build and deploy scalable applications without the need for traditional server management.
Cost-Effectiveness: DynamoDB's pricing model is based on provisioned throughput and storage usage, allowing you to optimize costs based on your application's needs. With Laravel's ability to scale horizontally, you can efficiently manage resources and control costs as your application grows.

Setting up DynamoDB in Laravel

Installing the AWS SDK for PHP

To work with DynamoDB in Laravel, you need to install the AWS SDK for PHP, which provides a set of libraries for interacting with various AWS services. You can install the SDK using a composer by running the following command in your Laravel project directory:

composer require aws/aws-sdk-php

Configuring AWS credentials for Laravel

Before using DynamoDB, you need to configure your AWS credentials in Laravel. The AWS SDK looks for credentials in the environment variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. You can set these variables in your .env file or directly in your server environment.

AWS_ACCESS_KEY_ID=your_access_key_id
AWS_SECRET_ACCESS_KEY=your_secret_access_key

Configuring the DynamoDB service in Laravel

To configure the DynamoDB service in your application, you need to define the connection settings in the config/database.php file. Add a new entry to the 'connections' array, specifying the driver as 'dynamodb' and providing the necessary configuration options.

'dynamodb' => [
    'driver' => 'dynamodb',
    'key' => env('AWS_ACCESS_KEY_ID'),
    'secret' => env('AWS_SECRET_ACCESS_KEY'),
    'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
],

Make sure to set the 'key', 'secret', and 'region' options appropriately, either directly in the configuration file or by referencing the corresponding environment variables.

Enabling DynamoDB cache (optional)

Optionally, you can enable caching for DynamoDB queries in Laravel to improve performance. Laravel provides various caching drivers, such as Redis and Memcached, which can be used in combination with DynamoDB. You can configure the cache driver in the config/cache.php file and set the 'default' option to your preferred caching driver.

'default' => env('CACHE_DRIVER', 'file'),

Additionally, you can specify the DynamoDB cache store in the config/cache.php file:

'stores' => [
    'dynamodb' => [
        'driver' => 'dynamodb',
        'key' => env('AWS_ACCESS_KEY_ID'),
        'secret' => env('AWS_SECRET_ACCESS_KEY'),
        'region' => env('AWS_DEFAULT_REGION', 'us-east-1'),
        'table' => 'laravel_cache',
    ],
],

This configuration sets up DynamoDB as the cache store using the specified credentials and region. You can also specify a custom table name for the cache.

By following these steps, you will have successfully set up DynamoDB in your application and now be ready to create DynamoDB tables, define models, and perform CRUD operations in your application using DynamoDB as the underlying database.

Creating DynamoDB tables and models

Defining the DynamoDB table schema

When working with DynamoDB, you need to define the schema for your tables. The table schema includes the primary key, attribute definitions, and any secondary indexes you want to create. In Laravel, you can define the table schema using AWS SDK for PHP.

Generating Laravel models for DynamoDB tables

To interact with DynamoDB tables in Laravel, you can generate models that represent the tables and handle the CRUD operations. Laravel provides a convenient way to generate models using the php artisan command-line tool. You can create a new model using the following command:

php artisan make:model DynamoDB/YourModelName

This command will generate a new model file in the app/DynamoDB directory. You can customize the model's properties and methods according to your specific requirements.

Creating migrations for DynamoDB tables

In Laravel, migrations are used to create and modify database tables. However, since DynamoDB is a NoSQL database, it doesn't have traditional migrations like SQL-based databases. Instead, you can use migrations to manage the creation and modification of DynamoDB tables.
To create a migration for a DynamoDB table, you can use the dynamodb:table command provided by the AWS SDK for PHP. This command generates a migration file that you can modify to define the necessary table schema and configuration. You can create a migration using the following command:

php artisan dynamodb:table create_your_table_name

This command will generate a migration file in the database/migrations directory. Open the migration file and customize it according to your table's schema, attributes, and indexes. Once you have defined the migration, you can run it using Laravel's migration command:

php artisan migrate

This will create the DynamoDB table based on your migration file.

Basic CRUD operations

Inserting data into DynamoDB

To insert data into a DynamoDB table, you can utilize the generated model's methods. Typically, this is done by creating a new instance of the model and setting the desired attributes before calling the save() method. For example:

$record = new YourModelName;
$record->attribute1 = 'Value 1';
$record->attribute2 = 'Value 2';
$record->save();

The code above creates a new record in the DynamoDB table with the specified attribute values.

Retrieving data from DynamoDB

To retrieve data from a DynamoDB table, you can use various querying methods provided by Laravel's Eloquent ORM. For example, you can retrieve all records from a table using the all() method:

$records = YourModelName::all();

You can also retrieve records based on specific conditions using methods such as where(), orWhere(), and whereIn(). These methods allow you to filter the records based on attribute values or perform more complex queries.

Updating data in DynamoDB

To update data in a DynamoDB table, you can retrieve the record using the model's querying methods and modify the desired attributes. Then, you can call the save() method to persist the changes to the database. For example:

$record = YourModelName::find($id);
$record->attribute1 = 'New Value 1';
$record->save();

The above code finds the record with the specified ID and updates the value of attribute1.

Deleting data from DynamoDB

To delete data from a DynamoDB table, you can use the delete() method provided by Laravel's Eloquent ORM. You need to retrieve the record first and then call the delete() method on the model instance. For example:

$record = YourModelName::find($id);
$record->delete();

These operations form the foundation for interacting with your DynamoDB data and building more complex functionalities in your application.

Querying and indexing in DynamoDB

Querying data using primary keys

DynamoDB's primary key consists of the partition key and, optionally, the sort key. You can query data efficiently by specifying the exact values or a range of values for the partition and sort keys. In Laravel, you can use the model's querying methods to perform these queries. For example:

// Query using the partition key
$record = YourModelName::where('partition_key', '=', 'value')->get();

// Query using the partition and sort keys
$records = YourModelName::where('partition_key', '=', 'value')
    ->where('sort_key', '>', 'start_value')
    ->where('sort_key', '<', 'end_value')
    ->get();

These queries retrieve records that match the specified partition and sort key values.

Performing conditional queries

DynamoDB allows you to perform conditional queries based on attribute values. Laravel provides methods such as where(), orWhere(), and whereIn() to specify these conditions. For example:

// Conditional query using attribute values
$records = YourModelName::where('attribute', '=', 'value')
    ->orWhere('attribute', '>', 'value')
    ->get();

This query retrieves records that match the specified conditions.

Creating and using secondary indexes

DynamoDB supports secondary indexes, which enable efficient querying based on non-primary key attributes. In Laravel, you can define secondary indexes in the migration file for your DynamoDB table. Once the index is created, you can query the data using the index attributes. For example:

// Query using a secondary index
$records = YourModelName::where('index_attribute', '=', 'value')->get();

By defining and utilizing secondary indexes, you can optimize querying based on specific attributes other than the primary key.

Performing advanced queries using DynamoDB expressions

DynamoDB expressions provide a powerful way to perform advanced queries in Laravel. Expressions allow you to use functions, operators, and conditionals to filter and manipulate data. You can use the whereRaw() method in Laravel's querying methods to include DynamoDB expressions in your queries. For example:

$records = YourModelName::whereRaw('#attr > :value', ['attr' => 'attribute', 'value' => 10])->get();

This query uses a DynamoDB expression to retrieve records where the value of the 'attribute' attribute is greater than 10.

Using these querying and indexing techniques, you can efficiently retrieve data based on specific keys, perform conditional queries, utilize secondary indexes, and even execute advanced queries using DynamoDB expressions. These capabilities allow for flexible and optimized data retrieval in your application.

Conclusion

Working with DynamoDB in Laravel offers a powerful and scalable solution for storing and accessing data in your web applications. DynamoDB's flexible schema, automatic scaling, and low-latency performance make it a suitable choice for various use cases.

In this article, we explored the fundamentals of DynamoDB and why it is beneficial to use it with Laravel. We discussed basic CRUD operations and querying and indexing techniques. By harnessing the power of DynamoDB in conjunction with the Laravel framework, you can build scalable, high-performance applications that handle large datasets and provide a seamless user experience.

Thanks for reading!

The complete guide to dates and times in Go

Honeybadger Staff — Thu, 21 Mar 2024 07:00:00 +0000

This article was originally written by Adebayo Adams on the Honeybadger Developer Blog.

Dates and times are one of the most essential features of a programming language, and Go is no exception. Whether you are building a social media or a command-line application, handling dates and times correctly in your application as a developer is necessary to maintain data integrity and consistency for your users.

In this article, you will learn how to use the time standard library to format and perform date and time operations in Go.

Prerequisites

The only prerequisite to follow along with this article is knowing the basics of the Go language. You can read my Go Beginners Series for a quick brush-up.

Go `time` package

One of the main reasons developers love to work with Go is its robust standard library, which contains several packages and tools you need to build any feature into your applications without employing any third-party libraries or tools. One of these packages is the time package, which allows you to handle dates and times in your application effectively without installing a third-party package.

In the following sections, we will explore how to use this package to perform various date and time tasks in Go, such as parsing and formatting dates, calculating durations, working with time zones, and handling date arithmetic. But first, let's look at the two main functions you'll use most of the time when handling dates and times in Go.

The `Parse` function

The Parse function converts a date string into a time object so you can perform other tasks with it. For example, you can get a birthday date input from your application's frontend as a string and need to extract the month and day to determine the zodiac sign. The Parse function takes a date string for conversion and a layout string to determine the output.

The `Format` function

The Format function is used to convert a date object into a string so you can display it in a way your users understand. For example, you can do a date calculation in your code and send it back to your application's frontend in a specific date format based on your user's time zone. The Format function takes in the layout string, which determines the output.

Working with dates: Parsing, formatting, and manipulating

There are several use cases where you need to parse, format, and manipulate dates in your Go application, from formatting them to make sense to your audience to calculating the time that a bidding offer has been on and when it'll expire. We will explore some common date-related tasks you'll encounter in your day-to-day job and how to solve them.

Parsing dates

As mentioned above, parsing dates involves formatting a date string typically from a user to a format your Go code will understand.

Parsing dates as `time` objects

A widespread use case of parsing dates involves when you get a birthday date from your frontend application, which will usually be in a string format, and want to convert it into a date object to perform more programmatic operations on the date. You can do so as follows:

package main

import (
    "fmt"
    "time"
)

func main() {
    dateStr := "2002-03-01"
    layout := "2006-01-02"
    parsedDate, err := time.Parse(layout, dateStr)
    if err != nil {
        fmt.Println("Error:", err)
        return
    }
    fmt.Println(parsedDate)
}

The code above defines the file as part of the package main, imports the fmt and time packages from the Go standard library, and defines the dateStr and layout variables—which represent the date string from the user and how the date should be formatted, respectively. Finally, the dateStr string is converted to a time object with the Parse function. The code should return the following result:

2002-03-01 00:00:00 +0000 UTC

Extracting date components from the `time` object

Another use case of parsing dates is when you need to extract the month, day, or year from the date string. You can do this like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    dateStr := "03-01-2002"
    layout := "01-02-2006"
    parsedDate, err := time.Parse(layout, dateStr)
    if err != nil {
        fmt.Println("Error:", err)
        return
    }
    year, month, day := parsedDate.Date()
    fmt.Printf("Year: %d, Month: %s, Day: %d\n", year, month, day)
}

The code above does the same thing as the previous one, but it uses a different layout and also prints out the year, month, and day like so:

Year: 2002, Month: March, Day: 1

Formatting dates

As mentioned above, formatting dates involves formatting a Go time object typically gotten from a finished function to a format that your users will understand. However, to grasp how to format dates properly, you need to understand how layouts work in Go.

Let's explore that in the next section.

Understanding layouts in Go

In Go, time layouts are used to define the format of time values when parsing or formatting strings. Time layouts consist of predefined layout strings representing different time components. The following code displays how to format Go time objects with some commonly used time layouts:

package main

import (
    "fmt"
    "time"
)

func main() {
    date := time.Now()
    fmt.Println("The ANSIC layout:", date.Format(time.ANSIC))
    fmt.Println("The UnixDate layout:", date.Format(time.UnixDate))
    fmt.Println("The RubyDate layout:", date.Format(time.RubyDate))
    fmt.Println("The RFC822 layout:", date.Format(time.RFC822))
    fmt.Println("The RFC822Z layout:", date.Format(time.RFC822Z))
    fmt.Println("The RFC850 layout:", date.Format(time.RFC850))
    fmt.Println("The RFC1123 layout:", date.Format(time.RFC1123))
    fmt.Println("The RFC1123Z layout:", date.Format(time.RFC1123Z))
    fmt.Println("The RFC3339 layout:", date.Format(time.RFC3339))
    fmt.Println("The RFC3339Nano layout:", date.Format(time.RFC3339Nano))
}

The code above defines a date variable that contains the current date and prints it out in various ways using different layouts. The code should return the current date as follows:

The ANSIC layout: Mon Jun 19 10:13:58 2023
The UnixDate layout: Mon Jun 19 10:13:58 WAT 2023
The RubyDate layout: Mon Jun 19 10:13:58 +0100 2023
The RFC822 layout: 19 Jun 23 10:13 WAT
The RFC822Z layout: 19 Jun 23 10:13 +0100
The RFC850 layout: Monday, 19-Jun-23 10:13:58 WAT
The RFC1123 layout: Mon, 19 Jun 2023 10:13:58 WAT
The RFC1123Z layout: Mon, 19 Jun 2023 10:13:58 +0100
The RFC3339 layout: 2023-06-19T10:13:58+01:00

So you can now use any of the formats that fit your above to parse and extract data from the time object instead of writing the layout as a string like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    dateStr := "2002-04-01T12:40:45.15Z"

    parsedDate, err := time.Parse(time.RFC3339, dateStr)
    if err != nil {
        fmt.Println("Error while parsing date :", err)
    }
    year, month, day := parsedDate.Date()
    fmt.Println("Day :", day, "Month :", month, "Year :", year)
    fmt.Println(parsedDate)
}

The code above uses one of the predefined layouts in Go to parse the date and extract the year, month, and day values from the time object. The code should return the following result:

Day : 1 Month : April Year : 2002
2002-04-01 12:40:45.15 +0000 UTC

Let's explore how to manipulate dates in the next section.

Manipulating dates

You can manipulate dates in different ways, depending on your application needs. The most common use cases for date manipulation are adding to and subtracting from dates.

Adding to dates

Go enables you to add to dates using the AddDate function like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    subscribedOn := time.Now()
    renewOn := subscribedOn.AddDate(1, 2, 7)
    fmt.Println("This account will be due for renewal on:", renewOn.Format(time.RFC822))
}

The code above creates a subscribedOn variable that contains the current date and a renewOn variable that uses the AddDate function to update the date by 1 year, 2 months, and 7 days. Finally, the code prints a message with the updated date formatted with the RFC822 layout.

The AddDate function takes in the number of years, months, and days. You should have 0 for the values you don't want to add to.

The result should look like the following but with the current date at runtime:

This account will be due for renewal on: 26 Aug 24 12:13 WAT

Subtracting from dates

To subtract from dates, you can use the AddDate function but with negative values like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    now := time.Now()
    previousDate := now.AddDate(0, 0, -14)
    fmt.Println("2 weeks ago was", previousDate.Format(time.RFC822))
}

The code above uses the AddDate function with negative values to get the date of 14 days ago and prints a message with the value as follows:

2 weeks ago was 05 Jun 23 12:19 WAT

Now that you know how to parse, format, and manipulate dates in Go, let's explore how to work with times in Go in the next section.

Time operations: Arithmetic, duration, and intervals

As a software engineer, there are several use cases involving working with times in your application. Whether adding and subtracting dates, delaying code execution, or measuring the time it took a function to run, knowing how to perform these tasks effectively is essential to building performant applications.

This section will explore how to perform common time-related tasks in Go.

Time arithmetic operations

The most common time-related tasks in programming involve some arithmetic operations. In the following sections, let's explore how to perform arithmetic operations on times in Go.

Time addition

You can add to times in Go with the Add function. For example, if you want a user's posts to be deleted automatically after 24 hours, you can do that like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    postCreated := time.Now()
    deletePost := postCreated.Add(24 * time.Hour)
    fmt.Println("Post will be deleted automatically on", deletePost.Format("02 January 2006 15:04:05"))
}

The code above defines a postCreated variable that contains the current time and a deletPost variable that has a time value that is 24 hours higher than the postCreated date. The code prints a message with a formatted deletePost value as follows:

Post will be deleted automatically on 20 June 2023 13:23:44

Time subtraction

Go allows you to subtract from times using the Add function with negative values. For example, if you want to deduct an hour from a given time, you can do that like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    givenDate := time.Now()
    minusOneHour := givenDate.Add(-24 * time.Hour)
    fmt.Println("One hour ago was: ", minusOneHour.Format(time.RFC822))
}

The code above subtracts 24 hours from the current time and prints a message based on the formatted result. The result should look like the following but based on the time at runtime:

One hour ago was:  18 Jun 23 13:39 WAT

Time duration

Durations are essential to working with times in Go because they represent the length of intervals. Let's explore how to work with durations in the following sections.

Duration construction

Here's how to construct a duration in Go:

package main

import (
    "fmt"
    "time"
)

func main() {
    duration := 3 * time.Hour
    fmt.Println("Duration:", duration)
}

The code above creates a duration variable that contains a three-hour duration that you can now perform other time operations on. The code will return the following:

Duration: 3h0m0s

Duration conversion

The time package allows you to convert durations to different units, and it offers methods such as Hours(), Minutes(), Seconds(), and Nanoseconds(), which return the respective duration in the specified unit. You can use them like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    duration := 2*time.Hour + 30*time.Minute

    fmt.Println("Duration in Hours:", duration.Hours())
    fmt.Println("Duration in Minutes:", duration.Minutes())
    fmt.Println("Duration in Seconds:", duration.Seconds())
    fmt.Println("Duration in Nanoseconds:", duration.Nanoseconds())
}

The code above creates a duration variable that contains a value of 2 hours and 30 minutes and prints out the value in hours, minutes, seconds, and nanoseconds like so:

Duration in Hours: 2.5
Duration in Minutes: 150
Duration in Seconds: 9000
Duration in Nanoseconds: 9000000000000

Manipulating time intervals

Time intervals represent a specific range of time between two points. Go provides the concept of intervals through the time.Time type and allows easy manipulation using the time package.

Interval checking

You can compare time values to check if they fall within a specific interval. The time package provides the Before, After, and Equal methods to perform these comparisons like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    start := time.Now()
    end := time.Date(2023, time.August, 31, 23, 59, 59, 0, time.UTC)
    current := time.Now()

    if current.After(start) && current.Before(end) {
        fmt.Println("Current time is within the interval.")
    } else if current.Equal(start) || current.Equal(end) {
        fmt.Println("Current time is equal to the interval.")
    } else {
        fmt.Println("Current time is outside the interval.")
    }
}

The code above defines a start variable containing the current date, an end variable equal to the 31st of August 2023, and a current variable equal to the current date. The code then checks if the current date is before, after, or similar to the end date and prints a result based on that like so:

Current time is within the interval.

Interval duration

Many developers are obsessed with how long it takes their programs to run; if you are like that, you can get the time it took your function to complete as follows:

package main

import (
    "fmt"
    "time"
)

func loop() {
    for i := 0; i < 10; i++ {
        fmt.Println(i)
    }
}

func main() {
    start := time.Now()
    loop()
    end := time.Now()

    duration := end.Sub(start)
    fmt.Println("It took loop function:", duration, "to finish")
}

The code above defines a loop function that runs 10 times and a duration variable that uses the Sub to get the duration it took for the function to finish. The code prints a message based on this process as follows:


0
1
2
3
4
5
6
7
8
9
It took loop function: 70.667µs to finish

Converting time zones

Go's time package provides the tools to convert time values between different time zones accurately. Below is an overview of the key functionalities that Go provides to tackle time zone operations.

Loading time zone data: Go relies on the International Organization for Standardization Time Zone Database to handle time zone information. The time package loads this database during runtime, allowing you to access and use it for time zone conversions. Here's the entire database of global time zones.
Time zone initialization: Go provides the LoadLocation function, which takes a time zone identifier (such as "America/New_York" or "Europe/London") and returns a *time.Location value representing the corresponding time zone. This Location value can then be used to convert times to and from that time zone.
Converting time zones: Go's time struct provides the In function, which enables developers to convert a time value from one time zone to another once the Location value is obtained.

Now that you understand what Go provides for you to tackle handling time zones, let's see how to do it in code. For example, if you want to convert a specific date to Los Angeles, USA, time, you can do that like so:

package main

import (
    "fmt"
    "time"
)

func main() {
    dateStr := "2023-06-13T23:31:00"
    layout := "2006-01-02T15:04:05"
    timeZone := "America/Los_Angeles"
    loc, _ := time.LoadLocation(timeZone)
    parsedDate, err := time.ParseInLocation(layout, dateStr, loc)
    if err != nil {
        fmt.Println("Error:", err)
        return
    }
    fmt.Println("The current date and time in Los Angeles, USA is:", parsedDate)
}

The code above parses a specific date to Los Angeles, USA, time. This code will return the following result, which contains the difference in hours:

The current date and time in Los Angeles, USA is: 2023-06-13 23:31:00 -0700 PDT

Common date and time operations in Go

Now that you know how the time package works in Go, let's explore some of the common date and time problems and how to solve them in Go.

Parse a date string and calculate the duration until a specific future date

Here's how to calculate the duration until a specific date from a given date string:

package main

import (
    "fmt"
    "time"
)

func main() {
    dateStr := "2023-06-13"
    layout := "2006-01-02"
    parsedDate, err := time.Parse(layout, dateStr)
    if err != nil {
        fmt.Println("Error:", err)
        return
    }
    futureDate := time.Date(2025, time.January, 1, 0, 0, 0, 0, time.UTC)
    duration := futureDate.Sub(parsedDate)
    fmt.Println("There's", duration.Hours(), "hours to go until 2025")
    fmt.Println("There's", duration.Minutes(), "minutes to go until 2025")
}

The code above will return the minutes and hours it'll take before the given and future dates like so:

There's 13632 hours to go until 2025
There's 817920 minutes to go until 2025

Parse the date string and check if it is a leap year

Here's how to check if a given date's year is a leap year:

package main

import (
    "fmt"
    "time"
)

func main() {
    dateStr := "2024-06-13"
    layout := "2006-01-02"
    parsedDate, err := time.Parse(layout, dateStr)
    if err != nil {
        fmt.Println("Error:", err)
        return
    }
    year := parsedDate.Year()
    isLeapYear := (year%4 == 0 && year%100 != 0) || (year%400 == 0)
    if isLeapYear {
        fmt.Println("The year" + " " + fmt.Sprint(year) + " " + "is a Leap Year")
    } else {
        fmt.Println("The year" + " " + fmt.Sprint(year) + " " + "is a not Leap Year")
    }
}

The code above will convert a given date, extract the year, check if the year is a leap year, and print a message based on the result like so:

The year 2024 is a Leap Year

Get the start and end of a month

Here's how to get the start and end date of a month in Go:

package main

import (
    "fmt"
    "time"
)

func main() {
    currentTime := time.Now()
    firstDayOfMonth := time.Date(currentTime.Year(), currentTime.Month(), 1, 0, 0, 0, 0, time.UTC)
    lastDayOfMonth := firstDayOfMonth.AddDate(0, 1, -1)

    fmt.Println("First day of the month:", firstDayOfMonth)
    fmt.Println("Last day of the month:", lastDayOfMonth)
}

The code above will return the start and end dates of the month at runtime as follows:

First day of the month: 2023-06-01 00:00:00 +0000 UTC
Last day of the month: 2023-06-30 00:00:00 +0000 UTC

Calculate the difference between two dates

Here's how to calculate between two dates in Go:

package main

import (
    "fmt"
    "time"
)

func main() {
    firstDate := time.Date(2022, time.June, 1, 0, 0, 0, 0, time.UTC)
    secondDate := time.Date(2023, time.June, 1, 0, 0, 0, 0, time.UTC)

    duration := secondDate.Sub(firstDate)
    fmt.Println("Difference in days:", duration.Hours()/24)
}

The code above will return the difference between both dates in days like this:

Difference in days: 365

Format time in a relative time format

Here's how to format time in a relative time format—for example, two hours ago—in Go:

package main

import (
    "fmt"
    "time"
)

func main() {
    pastTime := time.Now().Add(-2 * time.Hour)
    duration := time.Since(pastTime)

    var formattedTime string
    if duration.Hours() < 24 {
        formattedTime = fmt.Sprintf("%.0f hours ago", duration.Hours())
    } else {
        days := int(duration.Hours()) / 24
        formattedTime = fmt.Sprintf("%d days ago", days)
    }

    fmt.Println(formattedTime)
}

The code above will return the time in a relative format like this:

2 hours ago

Format a duration in hours, minutes, and seconds with leading zeros

Here's how to format a duration with leading zeros to make it more readable in Go:

package main

import (
    "fmt"
    "time"
)

func main() {
    duration := time.Hour*2 + time.Minute*3 + time.Second*4
    formattedDuration := fmt.Sprintf("%02d:%02d:%02d", int(duration.Hours()), int(duration.Minutes())%60, int(duration.Seconds())%60)
    fmt.Println(formattedDuration)
}

The code above will return the given duration as follows:

02:03:04

Parse a date string and extract the day of the week

You can extract the day of the week from a date using the WeekDay function as described below:

package main

import (
    "fmt"
    "time"
)

func main() {
    dateStr := "2023-06-19"
    layout := "2006-01-02"
    parsedDate, err := time.Parse(layout, dateStr)
    if err != nil {
        fmt.Println("Error:", err)
        return
    }
    dayOfWeek := parsedDate.Weekday()
    fmt.Println("Day of the week:", dayOfWeek)
}

The code above will return the day of the given date like this:

Day of the week: Monday

Conclusion

As a software engineer, it is essential to know how to work with dates and times. In this article, you learned the time package in Go and how it helps you parse, format, and manipulate dates. You also explored how to perform arithmetic, interval, and duration tasks with times and reviewed solutions to common time and date tasks in Go.

And that's it! I hope this article achieved its aim of teaching you everything you need to know about dates and times and how to work with them in Go. Thank you so much for reading!

Forem: Honeybadger

A guide to feature flags in Laravel using Laravel Pennant

What are feature flags?

The advantages of feature flags

Enabling features for specific users or organizations

A/B testing

Complements trunk-based development

The disadvantages of feature flags

Add extra complexity to your code

Remembering to remove the feature flags

Feature flags in Laravel

Benefits of using Pennant

Installing Laravel Pennant

Drivers

Creating a feature flag

Class-based feature flags

Using feature flags

Toggling feature flags

Using feature flag middleware

Using feature flags in Blade views

Using the HasFeatures trait

Rich-value feature flags

Testing feature flags

Conclusion

Options for passwordless authentication in Django apps

Brief overview of traditional password-based authentication in Django

Limitations and drawbacks of password-based authentication

Email-based authentication

Authentication with OAuth in Django

Configure the Google API

Run application

Authenticating with magic links

Conclusion

The character encoding cheat sheet for JS developers

Understanding Unicode

ASCII encoding

UTF-8 encoding

UTF-16 encoding

Ajax requests

Browser forms and cookies

Character encoding in Node.js

Best practices for character encoding

Debugging character encoding issues

Conclusion and further reading

Building a Sinatra app in Ruby

Takeaways

Pre-requisites

Why Sinatra?

An overview of the app we'll be building

How URL shortening works

Building a URL shortener using Sinatra

Modular vs. classical app structure

Installing Sinatra and initializing the project

Adding a Gemfile, Rack configuration and Rakefile

Adding a database connection and Active Record model

Defining the ShortLink model and Active Record migration

Working with Sinatra views

Running the app

Wrapping up and next steps

Reducing your Python app's memory footprint

What is memory management, and why is it important?

How to measure memory usage in Python

The psutil library

Resource module

The sys module

Third-party libraries

Common causes of high memory usage

Memory leaks

External dependencies

Large datasets

Unoptimized code

Tips to lower your app’s memory footprint

Use generators instead of lists

Read data in smaller chunks

Use memory-efficient dependencies

Use memory-profiling tools

Conclusion

A comprehensive guide to file operations in Go

Prerequisites

File operations in Go

Using the `HasFeatures` trait

Adding a `Gemfile`, Rack configuration and `Rakefile`

Defining the `ShortLink` model and Active Record migration

Reading `.txt` files in Go

Reading a `log` file line-by-line in Go

Reading `.json` files in Go

Reading `.csv` files in Go

Writing to `.csv` files in Go

Using the `DB` facade

The `__construct` method