Forem: Adam Crampton

Quick Guide - Upgrading to Laravel 9 and PHP 8.1

Adam Crampton — Thu, 17 Feb 2022 23:25:46 +0000

Introduction

This article is a quick guide to the steps required for upgrading from PHP7.x to 8.1, and Laravel v8 to v9.

Since Laravel 9 now requires PHP 8, I thought this would be a useful reference for tackling the task.

Note: The environment used in this guide is Ubuntu 20.04 with Nginx. If using a different Linux distribution and/or Apache, the process and commands will be slightly different.

Approach

It goes without saying, but ensure you run through this process on a test environment first, to ensure a smooth production upgrade once certain that everything works.

PHP will need to be upgraded before Laravel, as we will run into Composer dependency issues if we try to do this the other way around.

Upgrading PHP

Follow these steps to get PHP 8.1 up and running in your environment.

Upgrade Steps

1. Let's start by making sure we have the PPA repository added, so we're able to install PHP 8.1. Run these two commands:

sudo apt install software-properties-common
sudo add-apt-repository ppa:ondrej/php -y

2. Now we update our packages.

sudo apt update -y && sudo apt upgrade -y

3. We can now install PHP8.1 along with the libraries required for Laravel.

sudo apt install php8.1 php8.1-fpm php8.1-cli php8.1-curl php8.1-zip php8.1-mysql php8.1-mbstring php8.1-xml php8.1-bcmath

Note: You may need to install additional modules for certain libraries such as php8.1-gd. Don't worry if you miss any though - Composer will pick up these incompatibilities later when you upgrade the Laravel core.

4. Next, we'll need to update the FPM path in our Nginx hosts.

sudo vim /etc/nginx/sites-available/some-host.com

In this example we're upgrading from 7.4, so we'll change this:

fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;

to this:

fastcgi_pass unix:/var/run/php/php8.1-fpm.sock;

5. Finally, we save the file and restart Nginx and FPM.

sudo systemctl restart nginx && sudo systemctl restart php8.1-fpm

Upgrading Laravel

This part can be quite tricky if you use a lot of libraries. My advice here is to check the status of the dependencies you use (i.e. in Packagist or GitHub), and ensure they are Laravel 9 compliant.

The other thing to keep in mind is that it's very common to run into dependency errors when running the upgrade, as many libraries require a version upgrade to support Laravel 9.

If this occurs, run composer update -vvv for a verbose output, and check to see what is required. Quite often all the information is there, and fixing the issue is usually a case of updating the version reference in composer.json to the latest.

Composer Edits

Per the documentation, you need to make some mandatory composer.json updates.

Details here: https://laravel.com/docs/9.x/upgrade#updating-dependencies

And as mentioned above, check off each dependency in your composer.json to ensure Laravel 9 compatibility.

Follow The Upgrade Guide

This part is self expanatory - run through the entire upgrade guide to see if any of your codebase is affected.

Details here: https://laravel.com/docs/9.x/upgrade

Most people have found the v8 - v9 migration fairly simple, with minimal breaking changes. In reality, this process only takes 15-20 minutes for non-complex codebases.

Run The Update

To kick off the update, run composer update. If you run into compatibility issues while attempting the update, check your dependencies and make the appropriate updates in composer.json.

Troubleshooting

Some tips if you are running into issues with the Composer update:

Double check PHP 8 has upgraded and FPM is configured with the correct path in the Nginx host configuration (it may still be using 7.x).
Run composer update -vvv for full verbose output - this should give you a clue as to which dependencies are having a problem.
Remove the dependencies and run an install from scratch - sudo rm -rf vendor && composer install.
If all else fails, remove all dependencies from your composer.json except for the ones that ship with the Laravel core. You should then be able to add them back one-by-one until you find the problematic library.

Wrapping Up

Thanks for reading the article, and best of luck if you're about to dive into the upgrade!

10 Step Guide: Varnish + Nginx + PHP On Ubuntu With SSL

Adam Crampton — Mon, 15 Feb 2021 21:03:12 +0000

Setting up Varnish with SSL can be tricky, largely because there isn't any one place to get all the information you need in a straightforward manner.

Having gone through the frustration of piecing it all together myself, I decided to write a quick guide that details the process from start to finish.

Initially, I attempted the SSL portion of the job by using Hitch - which ended up being a complete waste of time (mostly because of really poor documentation). HAProxy on the other hand ended up being a great solution, as well as having other really great features like load balancing included.

Let's get straight to it. Here's the stack I'm working through in this article:

Ubuntu 18.04
PHP 7.4
Nginx
Varnish 5.2.1
Letsencrypt SSL
HAProxy
Laravel 8

You can ignore the PHP and Laravel bits if you're using a different stack.

0. Getting Started

You will need:

Ubuntu 18.04 installation + sudo SSH access
DNS configured for the server block you are going to add to Nginx (it'll be example.com in this article)

1. Set Up Nginx

Install nginx using:

sudo apt update
sudo apt install nginx

Set the default ports:

sudo vim /etc/nginx/sites-available/default

server {
  listen 8080 default_server;
  listen [::]:8080 default_server;
  ...
}

Note: Don't create the server block for your site just yet, we'll get to that shortly.

2. Install PHP 7.4 + Required Extensions

If PHP isn't already installed, run the following commands to get PHP 7.4 along with extensions required for a Laravel (and most modern PHP apps) project:

sudo apt install software-properties-common
sudo add-apt-repository ppa:ondrej/php
sudo apt update
sudo apt install php7.4 php7.4-cli php7.4-fpm php7.4-json php7.4-common php7.4-mysql php7.4-zip php7.4-gd php7.4-mbstring php7.4-curl php7.4-xml php7.4-pear php7.4-bcmath

3. Install FPM + Composer

Install FPM and Composer if you require it:

sudo apt install php7.4-fpm
sudo apt install composer

4. Create Nginx server block

Now we can create the Nginx server block for our site:

sudo vim /etc/nginx/sites-available/example.com
Paste this in (or use your preferred boilerplate):

server {
    listen 8080;
    server_name example.com www.example.com;
    root /var/www/example.com/public;

    add_header X-Frame-Options "SAMEORIGIN";
    add_header X-XSS-Protection "1; mode=block";
    add_header X-Content-Type-Options "nosniff";

    index index.html index.htm index.php;

    charset utf-8;

    location / {
        try_files $uri $uri/ /index.php?$query_string;
    }

    location = /favicon.ico { access_log off; log_not_found off; }
    location = /robots.txt  { access_log off; log_not_found off; }

    error_page 404 /index.php;

    location ~ \.php$ {
        fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
        fastcgi_index index.php;
        fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
        include fastcgi_params;
    }

    location ~ /\.(?!well-known).* {
        deny all;
    }
}

Save the file, then create the symlink by running:

sudo ls -n /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/

Restart Nginx: sudo service nginx restart

5. Install Cerbot

Certbot needs to be installed to request a Let's Encrypt certificate for our site. Ensure your DNS records are pointing to this server and run the following:

sudo add-apt-repository ppa:certbot/certbot
sudo apt install python-certbot-nginx
sudo certbot --nginx -d example.com -d www.example.com
Select No Redirect option
Edit /etc/nginx/sites-available/example.com and remove SSL lines added by Certbot at the bottom of the file - it will look something like this:

listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

6. Prepare PEM File

Next we will need to create a PEM file for our SSL proxy to use further down the track. To do so, we'll append the private key and fullchain key values to a single file:

Switch to root (sudo su), then:

sudo cat /etc/letsencrypt/live/example.com/privkey.pem /etc/letsencrypt/live/example.com/fullchain.pem > /etc/ssl/private/exampledotcom.pem
exit

7. Install + Configure Varnish

Varnish can now be installed and configured:

sudo apt -y install varnish
Edit /lib/systemd/system/varnish.service and change the default port from 6081 to 80:

ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m

Edit /etc/varnish/default.vcl and ensure the backend default port is set to 8080
Run sudo systemctl daemon-reload && sudo service varnish restart

Varnish should now be working for your site for HTTP.

8. Install + Configure HAProxy

Install HAProxy: sudo apt-get install haproxy
Open for editing: sudo vim /etc/haproxy/haproxy.cfg
Add a front end binding, pointing towards PEM file created earlier:

frontend haproxynode
        bind *:443 ssl crt /etc/ssl/private/exampledotcom.pem
        mode http
        default_backend backendnodes

Add back end config + node:

backend backendnodes
        balance roundrobin
        option forwardfor
        http-request set-header X-Forwarded-Port %[dst_port]
        http-request add-header X-Forwarded-Proto https if { ssl_fc }
        server example.com 172.1.35.35:80 check

Important: IP address above must be that of the server this host sits on. use ifconfig to determine that address.

To test the config for errors, run sudo haproxy -c -f /etc/haproxy/haproxy.cfg:

9. Restart Everything

sudo service nginx restart && sudo service varnish restart && sudo service haproxy restart

10. Trust Proxy Config (Laravel only)

Add the server IP from step 8 to the $proxies property in App\Http\Middleware\TrustProxies:

protected $proxies = ['172.1.35.35'];

Note: If you are behind a load balancer where you cannot reliably determine the IP, you can do this:

protected $proxies = '*';

Useful Links

Ubuntu 18.04 + Nginx install guide: https://www.digitalocean.com/community/tutorials/how-to-install-nginx-on-ubuntu-18-04
Let's Encrypt Nginx Tutorial: https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-18-04
VCL Reference: https://varnish-cache.org/docs/5.2/users-guide/vcl.html
An excellent VCL configuration template can be found here: https://github.com/mattiasgeniar/varnish-5.0-configuration-templates

Thanks!

Hopefully this guide was helpful for you, thanks for taking a look.

If there are any corrections to the above, please leave a comment and I will update the article.

Capistrano And Laravel: Setting The App Version

Adam Crampton — Wed, 03 Feb 2021 05:01:21 +0000

If you're of the few folks out there that uses Capistrano to deploy their Laravel projects (we DO exist!), this article will hopefully be useful for you.

Versioning your app is great for a number of reasons, and there are some use cases for needing access to this value in a production environment.

In my particular case, I maintain a large dashboard which receives frequent updates, and having the version number displayed in the footer helps as a quick visual reference.

The Approach

Fortunately, this is not a particularly complicated task. Here's what we need to do:

Ensure an APP_VERSION parameter exists in the .env file stored in the secrets directory within the production environment.

Add a task to the deploy.rb file which will prompt the user for a version string, then rewrite the line in the .env file within the release directory.

Adding The Task

Here's the task code that I use for this purpose:

task :set_version do
    ask(:app_version, "")
    on roles(:laravel) do
        execute :sed, "-i 's/^APP_VERSION.*/APP_VERSION=#{fetch(:app_version)}/' #{release_path}/.env"
    end
end

Let's break down what's going on in the above snippet.

We are defining our task as set_version, which when executed, prompts the user for an app version string.
Once the value is received, it is stored in the app_version configuration variable.
This task sits within a laravel namespace in the file, which explains the on roles(:laravel) line.
The execute directive is for a 1-line bash command, using sed, to perform an inline edit of the .env file located in the root of the release path.
The sed command which perform a regex match for APP_VERSION and replaces the entire line with the new value.
Still within the sed command, we have fetch(:app_version), which is the configuration value we stored earlier (mentioned in point 1).

Finally, we can set the task to run like in the example below:

namespace :deploy do
    after :updated, "laravel:set_version"
end

Clearing Config Cache

You will of course need to clear the config cache within the release directory to pick up the change to the .env file.

It's likely you are already doing this in some way; my preferred method is to use a task like this:

 task :clear_config do
     on roles(:laravel) do
         within release_path do
             execute :php, "artisan config:cache"
             execute :php, "artisan view:cache"
             execute :php, "artisan route:cache"
             execute :php, "artisan cache:clear"
         end
     end
 end

Just be sure to clear the config cache after the app version task has been run.

Thanks!

I appreciate you taking the time to read this; hopefully it helps someone along the way.

Testing Laravel API endpoints with jwt-auth

Adam Crampton — Fri, 20 Mar 2020 04:02:30 +0000

JWT authentication with Laravel is made relatively headache-free using the fantastic jwt-auth library, which can be found here:
https://github.com/tymondesigns/jwt-auth

Once you have JWT up and running in your project, you will want to start thinking about building tests as you add functionality to your codebase.

The three-step approach I've taken in this article is as follows:

Create a default API user and setting the details in config
Add tests for token handling
Add tests for project endpoints, using token auth

When these tests pass, you can be sure that tokens can be issued, refreshed, and used on your protected endpoints.

1. Setting a default API user

While you can certainly use a factory to automate this, sometimes it makes sense to use pre-saved credentials to test auth.

To that end, we can create a default User model for this purpose (manually, or using a seeder), and store the email and password in the .env file.

The best way to make these .env variables available in tests is to create a config file (e.g. app\config\api.php):

/*
|--------------------------------------------------------------------------
| API specific config
|--------------------------------------------------------------------------
*/

return [
    // Default API user - mostly used for tests.
    'apiEmail' => env('API_USER_EMAIL'),
    'apiPassword' => env('API_USER_PASSWORD')
];

2. Testing the auth routes

The three endpoints we can test here are login, logout, and refresh. This will ensure we can:

Generate a token
Pass the token into a logout request
Regenerate a token on the fly

Logging in

Here we'll hit the auth login route with the default user's credentials (email and password) and get a token back.

To ensure that we're getting the proper response, we'll assert a 200 HTTP status check and the 3 necessary fields in the JSON response.

/**
* Login as default API user and get token back.
*
* @return void
*/
public function testLogin()
{
    $baseUrl = Config::get('app.url') . '/api/auth/login';
    $email = Config::get('api.apiEmail');
    $password = Config::get('api.apiPassword');

    $response = $this->json('POST', $baseUrl . '/', [
        'email' => $email,
        'password' => $password
    ]);

    $response
        ->assertStatus(200)
        ->assertJsonStructure([
            'access_token', 'token_type', 'expires_in'
        ]);
}

Logging Out

We can now use our token from the login test to request a logout action for the default user.

Since I have set up the logout method in the Auth Controller to return an exact message, we can assert an exact match on the return message.

Note: Add a use statement for Tymon\JWTAuth\Facades\JWTAuth to easily access previously generated tokens.

/**
* Test logout.
*
* @return void
*/
public function testLogout()
{
    $user = User::where('email', Config::get('api.apiEmail'))->first();
    $token = JWTAuth::fromUser($user);
    $baseUrl = Config::get('app.url') . '/api/auth/logout?token=' . $token;

    $response = $this->json('POST', $baseUrl, []);

    $response
        ->assertStatus(200)
        ->assertExactJson([
            'message' => 'Successfully logged out'
        ]);
}

Refreshing the token

Finally, we'll refresh the token for the default user.

/**
* Test token refresh.
*
* @return void
*/
public function testRefresh()
{
    $user = User::where('email', Config::get('api.apiEmail'))->first();
    $token = JWTAuth::fromUser($user);
    $baseUrl = Config::get('app.url') . '/api/auth/refresh?token=' . $token;

    $response = $this->json('POST', $baseUrl, []);

    $response
        ->assertStatus(200)
        ->assertJsonStructure([
            'access_token', 'token_type', 'expires_in'
        ]);
}

3. Testing the endpoints

Now that we're confident the token handling is working as expected, we can now use our default API user to authenticate to JWT protected endpoints.

Here's an example where we grab all data from the User model:

/**
* Get all users.
*
* @return void
*/
public function testGetUsers()
{
    $user = User::where('email', Config::get('api.apiEmail'))->first();
    $token = JWTAuth::fromUser($user);
    $baseUrl = Config::get('app.url') . '/api/users?token=' . $token;

    $response = $this->json('GET', $baseUrl . '/', []);

    $response->assertStatus(200);
}

Next Steps

Now that you have a basic framework for automated endpoint authentication, you can easily add your tests as you build out your project.

Hope you enjoyed this post!

Large file uploads to an S3 bucket done neatly in Laravel

Adam Crampton — Mon, 24 Feb 2020 02:35:42 +0000

Lately I've been looking at ways to reduce the amount of clutter in my Laravel controllers, and found macros can be super useful for wrapping reusable snippets of code behind an easily accessible facades.

If you'd like to learn more about Laravel macros, here's a great primer, with a list of "Macroable" classes:
https://tighten.co/blog/the-magic-of-laravel-macros

Dealing with large files

The best way to reliably upload large files to an S3 bucket in Laravel is with the Flysystem S3 adapter, specifically using the writeStream and putStream methods.

Special note: You are almost certainly going to need to tweak the host's php.ini configuration, specifically:

post_max_size
upload_max_filesize

Setting up

The first thing I would recommend, if you don't do this already, is to set up a Service Provider specifically for your macros. To do this:

Run php artisan make:provider MacroServiceProvider
Add App\Providers\MacroServiceProvider::class to your project's app.php config file

Next, install the Flysystem AWS S3 Adapter:
composer require league/flysystem-aws-s3-v3

Finally, you'll want to ensure these four values are set in your project's .env file:

AWS_ACCESS_KEY=YOURACCESSKEY
AWS_SECRET_ACCESS_KEY=YOURSECRETACCESSKEY
AWS_REGION=aws-bucket-region
AWS_S3_BUCKET=name.of.s3.bucket

Create the macro

For this macro, we are going to extend the built in File component so we can use its facade.

Within App\Providers\MacroServiceProvider, let's add a closure to the boot method with four parameters:

path: Path within the bucket to save the file
filename: Filename you want to save as
file: The file object from $request->file('input_name')
overWrite: Whether or not you want to overwrite the file if a file with the same name already exists

Here's the code:

<?php

namespace App\Providers;

use Aws\S3\S3Client;
use Illuminate\Support\ServiceProvider;
use Illuminate\Support\Facades\Config;
use Illuminate\Support\Facades\File;
use League\Flysystem\AwsS3v3\AwsS3Adapter;
use League\Flysystem\Filesystem;

class MacroServiceProvider extends ServiceProvider
{
    /**
     * Bootstrap services.
     *
     * @return void
     */
    public function boot()
    {
        File::macro('streamUpload', function($path, $fileName, $file, $overWrite = true) {
            // Set up S3 connection.
            $resource = fopen($file->getRealPath(), 'r+');
            $config = Config::get('filesystems.disks.s3');
            $client = new S3Client([
                'credentials' => [
                    'key'    => $config['key'],
                    'secret' => $config['secret'],
                ],
                'region' => $config['region'],
                'version' => 'latest',
            ]);

            $adapter = new AwsS3Adapter($client, $config['AWS_S3_BUCKET'], $path);
            $filesystem = new Filesystem($adapter);

            return $overWrite 
                    ? $filesystem->putStream($fileName, $resource) 
                    : $filesystem->writeStream($fileName, $resource);
        });
    }
}

Update your controller

Now all you need to do is grab the file from the request, set your options, and call the macro:

            // Set file attributes.
            $filepath = 'upload/destination/files';
            $file = $request->file('uploaded-file');
            $filename = $request->input('name'); // Hidden input with a generated value

            // Upload to S3, overwriting if filename exists.
            File::streamUpload($filepath, $filename, $file, true);

Aaand that's it! Happy uploading!

Catching Laravel Eloquent Exceptions: Easy Peasy Error Messages

Adam Crampton — Mon, 09 Sep 2019 09:08:39 +0000

The Problem

I recently found myself in a situation where I had a codebase that would throw exceptions from certain Eloquent queries - but in a sort of legitimate way.

Essentially, the scenario was that certain user accounts did not have relationships with certain models, but were critical for the controller to complete its work.

The Solution

This sort of thing should normally be dealt with by fixing the issue at the entry point (validation on user creation), however in this particular scenario, it was a much better idea to return an error to the user to let them know that human intervention was required.

So, to solve this, I needed to catch the exception, generate a human-readable message, and send it back to the front end. This functionality should not clutter the controllers, and should be easy to maintain.

Here's my step-by-step approach:

1. Set up a trait

Since we probably want this functionality to be available globally through the app, let's set up a trait - which are super useful for this kind of thing.


namespace App\Traits;

Trait GlobalUtility
{
// Code
}

2. Define the errors

Within the trait, we set up an array of error messages, with a meaningful key name for each. A default message is added to ensure something is always returned.

namespace App\Traits;

Trait GlobalUtility
{
    protected $customExceptionErrors = [
        'generic' => 'Sorry, there was a problem with this page. Please contact your Sales Account Manager for help.',
        'noDealerRelationship' => 'Sorry, your user account does not appear to be connected to a Dealer. Please contact your Sales Account Manager for help.'
    ];
}

3. Return logic

The return method is a simple check that:

Accepts a named exception parameter, which is aligned with the keys in the $customExceptionErrors array
Fetches the error message
Returns the error string or the generic version if none is found (or no parameter is passed)

namespace App\Traits;

Trait GlobalUtility
{
    protected $customExceptionErrors = [
        'generic' => 'Sorry, there was a problem with this page. Please contact your Sales Account Manager for help',
        'noDealerRelationship' => 'Sorry, your user account does not appear to be connected to a Dealer. Please contact your Sales Account Manager for help.'
    ];

    /**
     * Returns a custom error message for the exception.
     *
     * @param string $exceptionName
     * @return string
     */
    public function getExceptionError($exceptionName = 'generic')
    {
        return $this->customExceptionErrors[$exceptionName] ?: $this->customExceptionErrors['generic'];
    }
}

4. Add Trait and Exception Class to Controller

Next, the trait and exception class are added to the controller where the Eloquent query is being made.

After the namespace declaration, we add:

use App\Traits\GlobalUtility;
use Illuminate\Database\Eloquent\ModelNotFoundException;

And within the class:


use GlobalUtility;

5. Update the Query

Now we must ensure the query utilises a "orFail" method, like firstOrFail or findOrFail, then pop it into a try & catch blocks.

try 
{
    $dealer = $user->worksFor()->firstOrFail();
}
catch (ModelNotFoundException $e)
{

}

6. Configure the redirect

Within the catch statement, we add the redirect with a call to the trait method to grab the appropriate error. We'll set this as customError, which will be a value available via the Session later on.

try 
{
    $dealer = $user->worksFor()->firstOrFail();
}
catch (ModelNotFoundException $e)
{
    return redirect()->route('orders.history')->with([
        'customError' => $this->getExceptionError('noDealerRelationship')
    ]);
}

7. Add check and fetch to the Blade file

Finally, we need to ask the app if the customError value exists in the session, and if so, display.

{{-- Show custom defined errors (caught from exceptions) if they exist --}}
@if (Session::has('customError'))
    <div class="alert alert-danger">
        <strong>{{ Session::get('customError') }}</strong>
    </div>
@endif

And that's it! Hope someone finds this useful :)