Forem: adithyasrinivasan

Data Providers in PHPUnit

adithyasrinivasan — Wed, 30 Mar 2022 12:53:47 +0000

When I built a Salary Calculator for Germany using Laravel, I started covering the code with Unit Tests using PHPUnit. This particularly came in handy when I had to cover more edge cases as I was able to trust the calculation. One of the features that I didn't know until that point was Data Providers in PHPUnit.

What is a Data Provider?

When I write a test for a method, or a class, I write multiple assertions with the values for the same code. Data Providers are useful here when all that changes is the values passed to the method.

DataProviders are a framework for easily controlling how data can be provided from a source

Let's take an example of how you would write a test without data providers. Assume you have a method add($a, $b) in a class Operations that returns a value, summing up the numbers.

    public function test_add()
    {
        $actual = (new Operations)->add(1, 2);
        $this->assertEquals(3, $actual);
    }

When you want to add multiple assertions, you would be tempted to add another assertion like this:

    public function test_add()
    {
        $actual = (new Operations)->add(1, 2);
        $this->assertEquals(3, $actual);

        $actual = (new Operations)->add(3, 4);
        $this->assertEquals(7, $actual);
    }

    // Or a test per method
    public function test_add_first()
    {
        $actual = (new Operations)->add(1, 2);
        $this->assertEquals(3, $actual);
    }

    public function test_add_second()
    {
        $actual = (new Operations)->add(3, 4);
        $this->assertEquals(7, $actual);
    }

These add overhead if the signature of the method changes, or when you want to cover more cases.

How to use a Data Provider in PHPUnit

A Data Provider can simplify this, allowing you to pass multiple sets of values for the same test. From the official documentation:

A data provider method

must be public
and either return an array of arrays or an object that implements the Iterator interface and yields an array for each iteration step.

Let's convert the existing test case using a Data Provider. We will use a dataProvider annotation following the name of the data provider.

/**
* @dataProvider addProvider
*/
public function test_add($a, $b, $expected)
{
    $actual = (new Operations)->add($a, $b);
        $this->assertEquals($expected, $actual);
}

public function addProvider()
{
    return array(
            array(1, 2, 3),
            array(4, 5, 7),
    );
}

In this case, PHPUnit would pass the take the array, loop through the items (array again) and pass the arguments based on the position of the method parameters.

For the first item, it will take array(1, 2, 3), call test_add(1, 2, 3) and validate the assertions. Then the next item and so on.

Inline Data Providers in PHPUnit

In the test cases above with a Data Provider, we can simplify it even further by inlining the data provider. The annotation to use is testWith

/**
* @testWith array(
            array(1, 2, 3),
            array(4, 5, 7),
    );
*/
public function test_add($a, $b, $expected)
{
    $actual = (new Operations)->add($a, $b);
        $this->assertEquals($expected, $actual);
}

XML Sitemap is rendering as plain text

adithyasrinivasan — Sun, 23 Jan 2022 10:02:10 +0000

After adding <xhtml:link> to my sitemap file where I included <xhtml:link rel="alternate" hreflang="supported_language-code"> my XML sitemap file started rendering as a plain text file.

In the console window of Chrome, I saw an error as well which indicated the structure was off.

Caught TypeError: Cannot read properties of null (reading 'childNodes')at XMLDocument.ready (content.js:218)ready @ content.js:218

I had to adjust my <urlset> from this

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xhtml="http://www.w3.org/1999/xhtml">

to this, thanks to this Stackoverflow answer

<urlset xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd http://www.w3.org/TR/xhtml11/xhtml11_schema.html http://www.w3.org/2002/08/xhtml/xhtml1-strict.xsd"
        xmlns="http://www.sitemaps.org/schemas/sitemap/0.9"
        xmlns:xhtml="http://www.w3.org/TR/xhtml11/xhtml11_schema.html">

Strangely enough that while the XML was rendering properly, Google Search Console was displaying an issue for the sitemap

Your Sitemap or Sitemap index file does not properly declare the namespace!

The schema below fixed the Google issue (that's all we care about in 2021, right?) while the rendering issue still persists.

<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xmlns:xhtml="http://www.w3.org/1999/xhtml"
        xsi:schemaLocation="http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd">

Automating MySQL backups to S3 using Laravel

adithyasrinivasan — Sun, 05 Dec 2021 13:59:58 +0000

Having already setup automatic S3 snapshots for Kibana, the next item on the list is to automate MySQL backups. There are a couple of services like Snapshooter that can do this but I prefer doing it on my own.

To automate MySQL backups to AWS S3 with Laravel, I'm using Laravel Backup package by Spatie.

Setup

Install the package using composer require spatie/laravel-backup

If you see a similar error message like this, your PHP version is probably not supported by the package / dependencies. You might want to use composer require spatie/laravel-backup ^6

Argument 2 passed to Symfony\Component\Translation\Translator::addResource() must be an instance of Symfony\Component\Translation\mixed, array given, called in /var/www/html/vendor/nesbot/carbon/src/Carbon/AbstractTranslator.php on line 165

To publish the configuration file, run php artisan vendor:publish --provider="Spatie\Backup\BackupServiceProvider" - this will create a file under config called backup.php
Go to your AWS Console, create a new IAM User (you can see the steps linked on the Kibana guide) and add AmazonS3FullAccess to the permissions attached to the user.
Grab the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY
Create a S3 Bucket from AWS console (the bucket name goes under AWS_BUCKET and your default region under AWS_DEFAULT_REGION)
On the config\backup.php file, under destination key, change the value of disks array to include s3

'disks' => [
    's3',
],

Ensure that the default database is set to mysql

'databases' => [
    'mysql',
],

Now if you run the command on your command line php artisan backup:run --only-db --only-to-disk=s3 the process should kick off and you should see similar messages.

Starting backup...
Dumping database database_name...
Determining files to backup...
Zipping 1 files and directories...
Created zip containing 1 files and directories. Size is 900 MB
Copying zip to disk named s3...
Successfully copied zip to disk named s3.
Backup completed!

Once this is done, you can setup to run it using the scheduler on a frequency that you prefer. Go to your Console\Kernel.php and add an entry

protected function schedule(Schedule $schedule)
{        
    $schedule->command("backup:run --only-db --only-to-disk=s3")->weeklyOn(6, '7:00');

    //Other commands
}

Error messages

The problem was incorrect permissions assigned to the IAM User

Copying zip failed because: There was an error trying to write to disk named s3.

Copying zip failed because: Error executing "ListObjects" on....
AWS HTTP error: cURL error 6: Could not resolve host:

AWS SignatureDoesNotMatch error

adithyasrinivasan — Fri, 26 Nov 2021 15:08:34 +0000

I was setting up AWS from scratch again on Windows 10. I went into IAM -> Users -> Add User and obtained the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Then I set up both of them under environmental variables.

I opened up a command prompt and ran aws sts get-caller-identity and got this error message.

An error occurred (SignatureDoesNotMatch) when calling the GetCallerIdentity operation: The request signature we calculated does not match the signature you provided

After a bit of Googling and making sure that my system time was set correctly, I found out that sometimes AWS CLI has an issue if the AWS_SECRET_ACCESS_KEY has a + (plus symbol) on the starting part of the value. All I did was to delete the existing AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY, regenerate it and it worked.

Remote Laravel Jobs

adithyasrinivasan — Fri, 26 Nov 2021 09:48:57 +0000

Laravel has been my go-to framework for building almost anything in the past few years, including Arbeitnow. Laravel & PHP make it such a breeze for anyone to build platforms, APIs to power the modern web. When I'm on the job hunt, I don't look for just "Laravel Jobs" as I consider myself a framework agnostic engineer.

I do see a lot of people looking for Laravel jobs and companies hiring people with expertise in Laravel, although some times it comes with a PHP / Laravel / Symfony, etc., requirement. As is common in tech, most of these are remote jobs - with remote-first or hybrid options. While remote working is allowed, it is often restricted to remote in EU. Some of them offer visa sponsorship and relocation support as well, so check it out.

I have tried to include salary ranges for laravel developers as well, but please make sure that this may change at any time.

Remote Laravel Jobs

ABOUT YOU is one of the fastest growing eCommerce companies in Europe Hamburg’s first unicorn with a value more than USD 1 billion. As a fashion technology corporation, along with our strong team, our ambition is to digitalize the classic shopping stroll by creating an inspiring personalized shopping experience on the smartphone.

They are currently looking for skilled & passionate Fullstack Developers (m/f/d) across Medior, Senior and Lead levels to become part of one of our Tech teams, either working onsite in Hamburg, remotely in the EU or in our Tech Hub in Zagreb, Croatia (planned to open in Spring 2022).

About You Laravel Jobs

ScaleFast is a leading full-service eCommerce solution provider and a rapidly growing company. We offer an all-in-one eCommerce solution for fast-growing brands. We provide to our clients the tools, operations and expertise needed to expand globally in no time.

100% remote work on offer until the end of the pandemic, reverting to 2 days a week working from home post-pandemic

Scalefast Laravel Jobs

bookingkit is Europe's leading booking and reservation management software for providers of tours and activities. We have challenged ourselves to digitise and revolutionise the market directly. Our Tech Stack - PHP, Vue.js, MySQL, Redis, AWS, Cypress.io, Docker Remote option: Work from wherever you are, while enjoying quarterly team events with all bookingkids in Berlin.

BookingKit Laravel Jobs

Other Laravel Jobs

CaseKing is an international e-Commerce company with over 900 employees at 12 locations worldwide. We are one of the most successful companies for hardware and gaming equipment in Europe and the industry leader in Germany. These jobs are mostly located in Berlin, Germany.

Caseking Laravel Jobs

At felmo we believe that the best veterinary care for your pet should not require more effort than ordering pizza or a cab. We work every day to elevate and deliver high-quality in-home veterinary care to pet-parents and their pets. felmo ist eine mobile Tierarztpraxis für Hunde und Katzen. 🐶🐱 Wir haben keine stationäre Praxis. Unsere Tierärzte besuchen die Kunden Zuhause.

Felmo Laravel Jobs

Urban Sports Club is the largest and most flexible flat-rate sports offer with over 50 types of sports and over 10000 partner venues across Europe.

Currently, they are hybrid remote and allow working up to 2 months anywhere in EU. Headquarters is in Berlin, Germany.

Urban Sports Club Laravel Jobs

Die Dosing GmbH ist spezialisiert auf digitale Lösungen zur Arzneimittel-Therapiesicherheit.

This is located in Heidelberg, Germany.

Dosing Laravel Jobs

Brainspin GmbH ist ein kleiner, aber erfolgreicher, spezialisierter Medienpartner für Einzelhändler, die ihr Geschäft durchschnelle, schön gebaute, ästhetisch ansprechende Webshops und E - Commerce-Sites von Nutzern für Nutzer online bringen möchten. 4 Tage (€55.000 bis € 65.000 PA) - E - Commerce, Berlin

Brainspin Laravel Jobs

überdosis - 54,000 - 72,000 (EUR) We are a small, family-like team of creatives, designing and developing websites and apps for very different clients, for us and for the whole world as open source. We are looking for another PHP/Laravel developer with years of experience. Maybe that’s you?

überdosis Laravel Jobs

Company blurbs have been borrowed either from Google search results or their own websites. Please always check the company's direct job description for up to date information.

Job Board API

adithyasrinivasan — Sat, 20 Nov 2021 09:36:20 +0000

A public free Job Board API with jobs from Applicant Tracking Systems

I have just opened up an API for my job board and you can see more information on the link below.

Job Board API

A public free Job Board API with jobs from Applicant Tracking Systems - https://arbeitnow.com/blog/job-board-api/

This could serve as a good starting point for a side project, just like how I did last year for Arbeitnow. If you're using Laravel, you're already familiar with the structure of the API. It's in JSON and supports pagination. To navigate between pages, you can follow the next / prev URLs.

$http = Http::get('https://arbeitnow.com/api/job-board-api');
if ($http->ok()){
 // process the data
}

Decode GZIP response in Laravel

adithyasrinivasan — Sun, 31 Oct 2021 14:37:34 +0000

If you try to decode a GZIP response without decoding, you will have seen this error

XMLReader::read(): /var/www/html/:1: parser error : Document is empty {"exception":"[object] (ErrorException(code: 0): XMLReader::read(): /var/www/html/:1: parser error : Document is empty at

To decode a XML ZIP response in Guzzle, you can do this as recommended by the documentation

// Request gzipped data, but do not decode it while downloading
$client->request('GET', 'foo.xml.gz', [
    'headers' => ['Accept-Encoding' => 'gzip'],
    'decode_content' => false
]);

When set to a string, the bytes of a response are decoded and the string value provided to the decode_content option is passed as the Accept-Encoding header of the request.

// Pass "gzip" as the Accept-Encoding header.
$client->request('GET', 'foo.xml.gz', ['decode_content' => 'gzip']);

Laravel provides useful header methods via its HTTP client.

$response = Http::withOptions([
            'decode_content' => 'gzip'
        ])->get('foo.xml.gz');

or using headers, this can be

   $response = Http::accept('gzip')
                ->get('foo.xml.gz');

Laravel raw SQL update query

adithyasrinivasan — Mon, 18 Oct 2021 09:29:18 +0000

A really small mistake - I wanted to run a MySQL update statement on laravel (no bindings, no parameters, no models) so I wrote this

DB::update(`UPDATE table SET something = 1 WHERE id = 1`);

When I ran this, I got this error

ValueError: PDO::prepare(): Argument #1 ($query) cannot be empty

Was a bit surprised to see this. Upon reading the documentation, you're supposed to run the query this way, for queries not returning any values

DB::statement('UPDATE table SET something = 1 WHERE id = 1');

Also, I had used backticks `` for the original longer SQL statement so that seemed to have caused on issue.

Execute quickly with Laravel Tinker

adithyasrinivasan — Sun, 17 Oct 2021 20:56:13 +0000

Came across this on Twitter today on how to execute quickly with Laravel Tinker. Seemed like a handy script!

Laravel productivity tip! Add this little function to your bash profile to quickly execute anything with the Tinker command and get the results instantly. pic.twitter.com/qIB9pQCTWt

— Philo Hermans (@philo01 ) October 16, 2021

Here's the gist of it that you should add to your bash profile

function tinker()
{
    if [-z "$1"]
        then
            php artisan tinker
        else
            php artisan tinker --execute="dd($1);"
    fi
}

Then on your terminal, you can execute it with the shortcut, instead of php artisan tinker and directly as tinker 'User::first()'

The author of the tweet also provided this as an explanation:

When you enter “tinker” it will execute the bash function. If no arguments are given, it will simply run “php artisan tinker”. If you provide any argument, this argument is forwarded as the “execute” parameter for the tinker command. The “dd” fn will prettify the output.

S3 Snapshots for Elasticsearch via Kibana

adithyasrinivasan — Sun, 29 Aug 2021 10:51:11 +0000

I'm terrified of racking up big bills with AWS so I don't work a lot with it, aside from Amazon Simple Email Service (SES). For my ElasticSearch 7.14 cluster that I run alongside with Kibana, I wanted to setup Snapshot and Restore.

My understanding was that with Snapshots, you obtain a copy of your cluster data at that point and you can restore it when you would like to. I already had an AWS account setup before starting with this. For this, you needed to setup a storage repository and then configure via Kibana.

This is how I went about it.

Create a S3 Bucket (this is the storage repository we will use with Elasticsearch)
Create an AWS policy to allow the IAM User to access the S3 Bucket
Create an IAM User and attach the policy we created
Install the S3 Repository Plugin
Register the repository with Kibana
Setup an automatic policy with Kibana / Elasticsearch so it backs up data via cron

Create a S3 Bucket

Open AWS Console -> S3 -> Create Bucket
Enter a bucket name: elasticsearch-s3-bucket-snapshot , other defaults are good - make sure you select " Block all public access" and then go ahead with creating the bucket (or saving)

Enter a bucket name: elasticsearch-s3-bucket-snapshot, other defaults are good - make sure you select "Block all public access"

Create an AWS policy

Open AWS Console -> IAM -> Policies
Click on "Create Policy"
Switch to JSON editor and paste the following policy (that Elasticsearch recommends for S3 permissions)

{
  "Statement": [
    {
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation",
        "s3:ListBucketMultipartUploads",
        "s3:ListBucketVersions"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::snaps.example.com"
      ]
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::snaps.example.com/*"
      ]
    }
  ],
  "Version": "2012-10-17"
}

Click on "Next: Tags", tags can be default so click next again
Enter a policy name elasticsearch-s3-policy-snapshot and hit " Create policy"

Enter a policy name elasticsearch-s3-policy-snapshot and hit "Create policy"

Create an IAM User

Open AWS Console -> IAM-> Access Management -> Users
Click on "Add Users"
Enter an user name elasticsearch-s3-user (customizable), check Programmatic access and click Next: Permissions

Enter an user name elasticsearch-s3-user (customizable), check Programmatic access and click Next: Permissions

Switch to " Attach existing policies directly" , search and select " elasticsearch-s3-policy-snapshot"

Click " Next: Tags", defaults are fine. Review the details and click Create User.

Review screen for Elasticsearch IAM user

This will generate a "Access Key ID" and "Secret access key" - both of which we will need soon to setup repository in Kibana, so keep them handy. There's a CSV option to download.

Credentials for IAM User

Install the S3 Repository Plugin

On your machine or server, go to where the Elasticsearch bin folder is. Mine was at /usr/share/elasticsearch
Run sudo bin/elasticsearch-plugin install repository-s3, answer with y or Y when asked about permissions
Restart your cluster!
We will configure the client settins for the plugin. Using elasticsearch-keystore, set the credentials for your IAM User

bin/elasticsearch-keystore add s3.client.default.access_key
# enter your "Access Key ID", hit enter
bin/elasticsearch-keystore add s3.client.default.access_key
# enter your "Secret Access Key", hit enter

Register the repository with Kibana

Go to your Kibana dashboard -> Management -> Stack Management -> Snapshot & Restore -> Repositories -> " Register a repository"

Snapshot & Restore on Kibana

Name it as elasticsearch-repository-snapshot , select AWS S3 as repository type, click Next. If you don't see AWS S3, make sure you restarted your cluster!

Repository name & Type

Client name is default (this is part of your keystore secrets s3.client. default.access_key) and your S3 Bucket Name. Review other defaults and hit Register.

Repository client name & S3 Bucket name

Once this is done, it pops up a details screen for the repository. Click on Verify repository

Verify Repository to make sure everything is good

Setup an automatic policy with Kibana

On the same screen, switch to Policies -> Create Policy
Enter these details
- elasticsearch-weekly-snapshot as the name
- as the snapshot name, appends the date at the end
- Select elasticsearch-repository-snapshot
- Set frequency to every week
- Click Next

Details for Policy - Cron

Click Next, and on review screen, hit Create Policy

Review Policy

You can run the policy and when it's done, you will see the snapshot

Run now for the policy

You should see something like this once snapshot is complete

snapshot is complete

Use localized datetime from Carbon in Laravel

adithyasrinivasan — Sun, 22 Aug 2021 20:20:55 +0000

I've been using this great package laravel-localization for translations. I noticed a minor issue with how datetime is resolved for locales in Laravel. The default format($dateTimeFormat) wasn't kicking it and a look at documentation reveals that if you use Carbon\Carbon 2, there are more helper methods to make this easier. Using isoFormat($dateTimeFormat) takes locale into consideration.

You also may know formatLocalized() method from Carbon 1. This method still works the same in Carbon 2 but you should better use isoFormat() instead. It does mean that you might have change all your existing method calls to a new format but it's a much cleaner, neater solution.

->isoFormat(string $format): string use ISO format rather than PHP-specific format and use inner translations rather than language packages you need to install on every machine where you deploy your application. isoFormat method is compatible with momentjs format method, it means you can use same format strings as you may have used in moment from front-end or node.js. Here are some examples:

Official documentation here: https://carbon.nesbot.com/docs/#api-localization

Adding Scopes to Laravel Scout Queries

adithyasrinivasan — Fri, 16 Jul 2021 19:09:12 +0000

I included a new section for four day work week jobs and when building the logic for that, I wanted to make use of scopes that are present in the eloquent model. I use Laravel Scout to offload some of the work.

Scope scopeIsPublished has an easy boolean check against the database like this:

<?php

//model definition

 public function scopeIsPublished($query)
 {
   return $query->where('is_published', true);
 }

With Scout, I wanted to include this when a user searches for a job. So naturally, this was what I ended up with.

<?php

//logic
$query = Jobs::search('search text')
            ->onlyPublished()
            ->otherConditions();

I was surprised when the results were unfiltered so I looked a little into the support for scopes in Laravel Scout. On the model instance, ScoutBuilder exposes a handy callback for query so you can use it like this.

<?php

//logic
$query = Jobs::search('search text')
            ->query(function ($query){
                return $query->onlyPublished();
            })
            ->otherConditions();

This helped get the expected results and should work with all providers Algolia, Elasticsearch, Meilisearch, etc.