Forem: Nicholas Hubbard

How I used a named pipe to save memory and prevent crashes (in Perl)

Nicholas Hubbard — Wed, 15 Jan 2025 16:13:45 +0000

I recently ran into an interesting bug in my Slackware package manager sbozyp. In this post I will detail the bug and my solution.

Disclaimer: The code snippets in this post do not deal with every edge case and are only meant to convey the ideas relevant to this blog post. All important edge cases are dealt with in the actual code for sbozyp.

Sbozyp is a package manager for SlackBuilds.org, a repository of build scripts for Slackware packages. There is a very important subroutine in sbozyp named build_slackware_pkg() that (basically) takes the name of a package, builds it into a Slackware package, and returns the path to the built package. To build the package we must execute the packages SlackBuild script. A SlackBuild script always has the general form: { basic setup -> perform build procedure -> conglomerate built parts into a Slackware package }. The last step is always performed by a Slackware-specific tool called makepkg. Unfortunately there is no way (that I know of) to be able to determine the name of the Slackware package before executing the SlackBuild script. For this reason sbozyp must inspect the stdout of the SlackBuild script to look for a line that makepkg always outputs that looks like Slackware package $PATH created. It is also important of course that the user can see the output of the SlackBuild script as it may contain vital information. This means that both my program and the user needs to examine the output of the SlackBuild script. (See here for more information on SlackBuild scripts)

With the problem description out of the way we can now focus on the solution.

My original solution was an obvious one but had an interesting bug. I simply would use Perl's system command to execute the SlackBuild script with a shell, where I would pipe stdout to tee, writing the stdout to a temporary file. After the SlackBuild would execute I would read the temporary file looking for the Slackware package $PATH created line. This solved the problem of allowing both the user and sbozyp to read the stdout of the SlackBuild script. For 99% of builds this worked fine ... but then there was qemu. Qemu is a massive project that requires a huge compilation. My system uses a small 4GB tmpfs mounted at /tmp, and the temporary file that was being teed to ended up getting so large from all the qemu compilation output that tee (and thus sbozyp) crashed for "device out of space".

Here is a basic (not actually realistic to sbozyp) outline of that code:

sub build_slackware_pkg {
    my ($pkg_name) = @_;
    my $slackbuild_script = find_slackbuild_script($pkg_name);
    my $tmp_file = make_temp_file(DIR=>'/tmp');
    0 == system("set -o pipefail; $slackbuild_script | tee $tmp_file") or die;
    open my $fh, '<', $tmp_file or die;
    my $slackware_pkg;
    while (my $stdout_line = <$tmp_file>) {
        $slackware_pkg = $1 if $stdout_line =~ /^Slackware package (.+) created$/;
    }
    return $slackware_pkg;
}

To solve the problem I create a named pipe, fork, and use the child process to execute the SlackBuild script, teeing stdout to the named pipe. From the parent I read the named pipe looking for the magic Slackware package $PATH created line. Here is a (also not realistic to sbozyp) outline of this code:

use POSIX qw(mkfifo WNOHANG);

sub build_slackware_pkg {
    my ($pkg_name) = @_;
    my $slackbuild_script = find_slackbuild_script($pkg_name);
    my $tmp_dir = make_temp_dir(DIR=>'/tmp');
    mkfifo("$tmp_dir/fifo", 0700);
    my $pid = fork();
    if ($pid == 0) { # child
        0 == system("set -o pipefail; $slackbuild_script | tee $fifo") or die;
        exit 0;
    } else { # parent
        open my $fifo_r_fh, '<', $fifo or die;
        my $slackware_pkg;
        while (waitpid($pid, WNOHANG) == 0 or my $stdout_line = <$fifo_r_fh>) {
            $slackware_pkg = $1 if $stdout_line and $stdout_line =~ /^Slackware package (.+) created$/;
        }
        die if $? != 0; # $? has exit status of child
        return $slackware_pkg;
    }
}

This implementation saves disk space as the data is being teed to a named pipe instead of a regular file in /tmp. As data is written to the pipe it is immediately being read, throwing away all the data that sbozyp doesn't need.

Problem solved :)

(Thanks to thrig on the #perl Libera IRC channel for helping lead me to this solution)

EDIT

I got some great feedback from u/gorkish on reddit with a much simpler way to solve the problem by running the SlackBuild script with open instead of system. Here is the code:

sub build_slackware_pkg {
    my ($pkg_name) = @_;
    my $slackbuild_script = find_slackbuild_script($pkg_name);
    open(my $cmd, '-|', $slackbuild_script) or die $!;
    my $slackware_pkg;
    while (my $line = <$cmd>) {
        $slackware_pkg = $1 if $line =~ /^Slackware package (.+) created$/;
        print $line;
    }
    close $cmd;
    die if $? != 0;
    return $slackware_pkg;
}

4 More Unrelated Perl Tidbits

Nicholas Hubbard — Tue, 11 Apr 2023 14:39:40 +0000

Last year I wrote an article titled 4 Unrelated Perl Tidbits, where I talked about some random Perl facts I learned about from reading Programming Perl. In this article I will talk about 4 more random and interesting Perl features I have learned about since.

Built-Ins Can Be Overridden with Lexical Subroutines

Perl version 5.18 introduced lexical subroutines, which are often sometimes referred to as "my subs". An interesting characteristic of lexical subs is that unlike regular subroutines, they can override built-ins.

use v5.18;

my sub print {
    die "printing is banned\n";
}

print "Hello, World!\n";

__END__

$ perl tmp.pl
printing is banned

If anybody has seen a use of this feature then please comment below.

Recursive Anonymous Subroutines With SUB

Perl version 5.16 introduced the __SUB__ special token that holds a reference to the current subroutine. You can use __SUB__ to make a recursive call in an anonymous subroutine.

use v5.16;

higher_order_subroutine(
    sub {
        my $n = shift;
        if ($n <= 1) { return 1 }
        else         { return $n * __SUB__->($n-1) }
    }
)

Goto Searches For Labels In The Dynamic Scope

The following example shows that goto searches for its label argument from within the current dynamic scope. Note that this program just goes on forever printing hello from after LABEL:

sub foo {
    bar();
}

sub bar {
    goto LABEL;
}

sub baz {
  LABEL:
    print "hello from after LABEL\n";
    foo();
}

baz();

__END__

$ perl tmp.pl
hello from after LABEL
hello from after LABEL
hello from after LABEL
...

This is another feature that you should leave a comment about if you have seen a usage of it.

Regex Modifier For Only Portions Of The Regex

You can use the (?M:) pattern in a regex to turn on the modifier specified by M, only inside the parentheses. For example, the following two regexs are the same:

/foo/i
/(?i:foo)/

You can also turn a modifier off with (?-M:), which is shown in this example:

if ('FOO' =~ /(?-i:foo)/i) {
    print "matches\n"
} else {
    print "does not match\n"
}

__END__

$ perl tmp.pl
does not match

This feature is useful if you want to turn a modifier on/off for only a portion of the regex:

if ('fooBAR' =~ /(?-i:foo)bar/i) {
    print "matches\n"
} else {
    print "does not match\n"
}

__END__

$ perl tmp.pl
matches

An Interesting Perl Pattern That Doesn't Work

Nicholas Hubbard — Thu, 23 Mar 2023 18:24:00 +0000

I recently came up with a pattern that is supposed to use a closure to protect a configuration hash from being mutated by its callers. Unfortunately this pattern has a terrible flaw.

{
    my %config;

    sub config {
        return %config if %config;
        %config = create_config();
        return %config;
    }
}

The config() subroutine is a lexical closure over the %config hash. No other code in a program would be able to access the %config variable, as everything is defined in its own block.

As an aside, this pattern could just as easily be written with a state variable, but I find it harder to explain the pattern when done this way.

Blocks

To understand this pattern we first have to understand how blocks work. Here is a code example that shows how blocks work:

{
    my $var = 'foo';
    print "from inside the block where \$var is defined \$var = $var\n";

    {
        print "from the most inner block \$var = $var\n";
    }
}

print "from outside the block \$var = $var\n";

__END__

$ perl tmp.pl
from inside the block where $var is defined $var = foo
from the most inner block $var = foo
from outside the block $var =

This code shows that a block introduces a lexical scope. Variables defined in a lexical scope are only available in its defining scope, and scopes nested inside their defining scope. The program output shows that the $var variable is available in the scope it is defined in, and from the scope nested in its defining scope. However, outside of its defining scope, $var is not defined.

If we turn on strict we get a fatal compilation error for trying to use $var from outside its defining scope:

Global symbol "$var" requires explicit package name (did you forget to declare "my $var"?) at tmp.pl line 14.
Execution of tmp.pl aborted due to compilation errors.

You may be wondering if the config() subroutine is accessible from outside the block it was defined in. The answer is that it is indeed available, because subroutine declarations are always global to the current package. This code example shows this fact:

use strict;

{
    sub foo {
        print "hello from foo!\n";
    }
}

foo();

__END__

$ perl tmp.pl
hello from foo!

Closures

Now that we understand blocks we can understand closures. In Perl, a closure is a subroutine that has access to the lexical environment that it was defined in. Here is the classic example:

use strict;

{
    my $n = 0;        

    sub increment {
        $n += 1;
        return $n;
    }

    sub decrement {
        $n -= 1;
        return $n;
    }
}

print 'increment() -> ', increment(), "\n";
print 'increment() -> ', increment(), "\n";
print 'decrement() -> ', decrement(), "\n";
print 'increment() -> ', increment(), "\n";
print 'increment() -> ', increment(), "\n";
print 'decrement() -> ', decrement(), "\n";

__END__

$ perl tmp.pl
increment() -> 1
increment() -> 2
decrement() -> 1
increment() -> 2
increment() -> 3
decrement() -> 2

The increment() and decrement() subroutines are both able to access the $n variable, though no other subroutines in a larger program would be able to, due to the outer block. For this reason the increment() and decrement() subroutines are closures over $n;

The Problem

We should now have all the knowledge needed to understand the pattern that this article is about.

The idea of the pattern is that if the %config variable has already been set then we just return it, and otherwise we set its value before returning it. This means that %config will only be set the first time that we call config(), and on all subsequent calls it will simply be returned. Therefore config() can be thought of as a function constant ... right?

Here is a code example where our pattern works as expected:

use strict;

{
    my %config;

    sub config {
        return %config if %config;
        %config = create_config();
        return %config;
    }
}

sub create_config {
    print "hello from create_config()\n";
    return (foo => 12);
}

my %config1 = config();

$config1{foo} = 1004;

my %config2 = config();

print "%config2's foo key = $config2{foo}\n";

__END__

$ perl tmp.pl
hello from create_config()
%config2's foo key = 12

This output displays a couple of important points. First, we know that the create_config() subroutine was only invoked a single time, even though we invoked config() twice. We know this because the "hello from create_config()" message is only printed a single time. The other important thing to note is that because we got the output "%config2's foo key = 12", we know that our modification of %config1's foo key (which we set to 1004), did not effect the %config variable that our config() subroutine closes over. If it had, then %config2's foo key would associate to 1004.

So what is the problem? Well ... everything falls apart when the %config variable is set to a multi-dimensional data structure. The following code encapsulates the fundamental problem with our pattern:

use strict;

{
    my %config;

    sub config {
        return %config if %config;
        %config = create_config();
        return %config;
    }
}

sub create_config {
    return (foo => [1, 2, 3]);
}

my %config1 = config();

$config1{foo}->[0] = 1004;

my %config2 = config();

print "%config2's foo key = [", join(', ', @{$config2{foo}}), "]\n";

__END__

$ perl tmp.pl
%config2's foo key = [1004, 2, 3]

Uh oh! We were able to mutate the %config variable that config() closes over, which means that config() is not actually a constant function. Now we come to the fundamental problem of our pattern. Because in Perl multi-dimensional data structures are made up of references, and perl does not perform deep-copying by default, we are able to mutate the underlying references of multi-dimensional data structures.

Here is a code example that shows that Perl does not perform deep-copying:

use strict;

my @array1 = ([1, 2, 3], [4, 5, 6], [7, 8, 9]);

print '@array1 contents:', "\n";
for my $elem (@array1) {
    print "    $elem\n";
}

# copy @array1 to @array2
my @array2 = @array1;

print '@array2 contents:', "\n";
for my $elem (@array2) {
       print "    $elem\n";
}

__END__

$ perl tmp.pl
@array1 contents:
    ARRAY(0x13875e8)
    ARRAY(0x13d1ef8)
    ARRAY(0x13d1fe8)
@array2 contents:
    ARRAY(0x13875e8)
    ARRAY(0x13d1ef8)
    ARRAY(0x13d1fe8)

In this programs output we can see that @array1 and @array2 contain the exact same references, which means that Perl does not perform deep-copying. If Perl did perform deep-copying, then when we copied @array1 into @array2, Perl would have made (recursive) copies of all the references in @array1 into new refererences. Perl's lack of deep-copying is the fundamental flaw of our pattern, as it means that we can modify %config's references from its copies that are returned by config().

Solutions

There are many ways we can solve this problem. First, we could use lock_hash_recurse from the core Hash::Util module to lock %config. After locking %config, we would get an error if we tried to mutate any of its values.

We could also use Const::Fast from CPAN to make %config an actual read-only hash. Similarly to locking the hash, we would get an error if we tried to mutate %config.

Finally, we could use Clone from CPAN to return a deep-copy of %config from the config() subroutine. Unlike the other solutions, our code could freely modify copies of %config without getting any errors, but these modifications would not affect the actual %config.

Use Dist::Zilla to Create a Perl Distribution

Nicholas Hubbard — Sun, 23 Oct 2022 14:29:26 +0000

Dist::Zilla (dzil) is a program for creating Perl distributions. While the documentation for dzil is complete, it is not geared towards a beginner that has never created a Perl distribution before. This article provides a brief introduction to Dist::Zilla geared towards users that know little about Perl distributions in general.

What is a Perl distribution?

A Perl distribution is an archive of files that includes a Perl module. There are no official rules on what non-module files must be included in a distribution, but they often include (among other things) test scripts, a Makefile.PL, documentation, and the license. These distributions are commonly uploaded to CPAN, which is a place for Perl programmers to upload their Perl distributions for the purpose of sharing their code.

Why Dist::Zilla?

You may think that bundling together a Perl module with some other files is simple, but there are many things that need to be accounted for, and are prone to human error. There are also many possibilities for what somebody may want to include in a distribution, and how they want to include it. Dist::Zilla exists to be a one-stop solution to every possible problem involved in creating a Perl distribution.

Using Dist::Zilla

When you install Dist::Zilla, you will be provided with an executable named dzil. The most important command that dzil provides is build, which - when run in the projects root directory - outputs a distribution tarball. Other commands such as test and release are also provided, but when getting started with Dist::Zilla you will only need the build command.

The "dist.ini" File

Dist::Zilla is configured on a per-project basis through a file named dist.ini, which should be located at the root of the project's directory tree.

The beginning of a dist.ini file specifies required settings that every distribution should have. These settings include name, version, abstract, copyright_holder, and license. (There is also author, which isn't required but you probably want to add it as well.)

Here is an example:

name = App-Foo
version = 1.0
author = Jane Doe
copyright_holder = Jane Doe
license = Perl_5
abstract = the best software ever

After you specify these required settings, you can then configure your distribution by specifying what plugins you wish to use. Plugins are the mechanism that Dist::Zilla uses for providing features to your Perl distribution. If you have a dist.ini that doesn't specify any plugins, Dist::Zilla will produce an empty distribution with no files.

Let's look at example of the plugins that a simple distribution might use, then go over what a few of the plugins actually do:

[MetaResources]
homepage       = https://github.com/JaneDoe/App-Foo
bugtracker.web = https://github.com/JaneDoe/App-Foo/issues
repository.url = https://github.com/JaneDoe/App-Foo.git

[GatherDir]
[PruneCruft]
[ManifestSkip]
[MetaYAML]
[License]
[ExecDir]
[MakeMaker]
[Manifest]
[AutoPrereqs]
[TestRelease]

The MetaResources plugin adds resource entries to the distribution's metadata. MetaCPAN can use this information to provide useful links to the distribution's page.

The GatherDir and PruneCruft plugins tell Dist::Zilla that you want to include all the files in your project's directory into the distribution, excluding the ones you certainly don't want. The files you certainly don't want include build artifacts introduced by recent invocations of Dist::Zilla. The combination of these two plugins is used in almost every Dist::Zilla project.

The MakeMaker plugin will tell Dist::Zilla to produce an ExtUtils::MakeMaker-powered Makefile.PL. Dist::Zilla will deal with everything required to create a proper Makefile.PL, so you do not need to know anything about ExtUtils::MakeMaker. Unless you are doing something special, you almost certainly want to use this plugin.

The UploadToCPAN plugin will allow you to use the dzil release command to upload your distribution to CPAN.

It is important to note that each plugin takes effect in the order the plugins are specified in your dist.ini.

There are many plugins available for Dist::Zilla - over 1,200 thus far - so you will probably find one that can do just about anything you could possibly need for creating a distribution. Here is a link for a metacpan query for "Dist::Zilla::Plugin", that can be used to explore the Dist::Zilla plugin ecosystem.

Here are links to the documentation for the plugins in the example dist.ini that I did not explain:

Synopsis

Dist::Zilla can seem daunting at first, but it is actually quite straightforward and easy to use once you figure it out. The only difficult thing is figuring out what plugins you want to use.

How I use Yabsm to Manage my Btrfs Snapshots

Nicholas Hubbard — Thu, 06 Oct 2022 23:02:57 +0000

YABSM IS DEPRECATED. CONSIDER USING ALTERNATIVES SUCH AS BTRBK OR SNAPPER.

I am the author of Yabsm (yet another btrfs snapshot manager), and I will explain how I use it to manage my Btrfs snapshots.

This article is meant to supplement the official documentation, and assumes a basic understanding of Linux's Btrfs filesystem.

Please note that Yabsm can be configured to suit many different use cases other than the one described here.

Snapshots vs Backups

Before we go on, let's clear up the difference between a snapshot and a backup.

A snapshot is a read-only nested subvolume created with a command such as btrfs subvolume snapshot -r $SUBVOLUME $DEST. SNAPSHOTS ARE NOT RELIABLE BACKUPS! If a subvolume is corrupted then all snapshots of that subvolume may also be corrupted.

A backup is an incremental backup sent to some location via Btrfs's send/receive commands. These backups will not be corrupted if the subvolume being backed up is corrupted.

My Btrfs Filesystem

I like to have just one top-level Btrfs subvolume mounted at /. This allows me to snapshot my entire system (excluding nested subvolumes) by running btrfs subvolume snapshot -r / $DEST.

My Yabsm Configuration

My configuration is based on the philosophy that because snapshots are both valuable and cheap, it makes sense to take a lot of snapshots.

Here is my /etc/yabsm.conf:

yabsm_dir=/.snapshots/yabsm

subvol root_subvol {
    mountpoint=/
}

snap root {
    subvol=root_subvol

    timeframes=5minute,hourly,daily
    5minute_keep=36
    hourly_keep=72
    daily_times=15:00,23:59
    daily_keep=62
}

ssh_backup slackmac {
    subvol=root_subvol
    ssh_dest=slackmac
    dir=/.snapshots/yabsm-slacktop

    timeframes=daily
    daily_times=23:59
    daily_keep=365
}

local_backup easystore {
    subvol=root_subvol
    dir=/mnt/easystore/backups/yabsm-slacktop

    timeframes=daily
    daily_times=23:59
    daily_keep=365
}

Yabsm Dir

I use the traditional /.snapshots/yabsm directory as my yabsm_dir, which is the location that my snapshots will reside. Yabsm will also use this directory for storing data necessary for performing SSH and local backups.

Subvol

As I mentioned earlier, I only have one top-level Btrfs subvolume, so I only need to define one subvol in my Yabsm config, which I name root_subvol.

Snap

I define one snap named root that tells Yabsm I want to take snapshots of root_subvol in the 5minute, hourly, and daily timeframe categories.

In the 5minute timeframe category I keep 36 snapshots. This lets me go to any state of my machine in the last 3 hours in 5 minute increments. I use the 5minute category because it gives me a valuable safety net. How many times have you broken your code that was working 20 minutes ago? If you take 5minute snapshots then you can easily go back to the state of that code 20 minutes ago.

In the hourly timeframe I keep 72 snapshots, which allows me to go back 3 days in hourly increments. How many times have you broken code that was working 2 days ago? If you take hourly snapshots (and keep enough of them), you can go back through the state of your machine from 2 days ago, in 1 hour increments.

In the daily timeframe category I keep 62 snapshots taken at midnight (23:59) and midafternoon (15:00). This gives me two snapshots per day in the last month.

Please note that there is also a weekly and monthly timeframe category.

SSH Backup

I define one ssh_backup named slackmac that backs up my system to my old MacBook running Slackware.

The ssh_dest value is set to slackmac, which is a host defined in the yabsm user's $HOME/.ssh/config file. (Yabsm runs as a daemon process, using the special username yabsm.)

The dir value is set to the directory on slackmac where the backups will be located.

I perform this ssh_backup only in the daily timeframe category, backing up every night at midnight. I keep 365 of these backups so I can go back an entire year.

Local Backup

I define one local_backup named easystore that backs up my system to my EasyStore external hard drive.

The hard drive is mounted at /mnt/easystore, and I keep my backups in the /backups/yabsm-slacktop directory on the hard drive.

Just like my slackmac ssh_backup, I perform my local_backup only in the daily timeframe category, every night at midnight.

Finding Snapshots

Yabsm provides the find command that I use to jump around to different snapshots and backups. The find command takes two arguments, the first is the name of any of your snaps, ssh_backups, or local_backups. The second argument is a query. The different kinds of queries are all documented in the link above.

Instead of repeating the documentation, let's break down a practical example of the find command's usage.

How many times have you broken code that worked 30 minutes ago? Because I take 5minute snapshots I can easily get back the state of the code 30 minutes ago.

An example:

$ diff "$(yabsm find root back-30-mins)/$HOME/projects/foo/script.sh" $HOME/projects/foo/foo.sh

This command will show the diff output of the $HOME/projects/foo/foo.sh file with this same file that was snapshotted 30 minutes ago. We can use this output to help figure out what we messed up.

The command yabsm find root back-30-mins will output the path to a snapshot for the snap named root that was taken 30 minutes ago. In the example we use our shell's parameter expansion feature to create a string that appends the path to foo.sh to the output of the yabsm find command. This is a powerful pattern!

The find command can do more than find a snapshot taken N units ago, it can also:

Find the newest or oldest snapshot/backup.
Find a snapshot/backup taken on a specific day and time.
Find all the snapshots/backups taken before or after a certain time.
Find all the snapshots/backups taken between two times.
Find all snapshots/backups.

The output of yabsm find --help shows some examples:

usage: yabsm <find|f> [--help] [<SNAP|SSH_BACKUP|LOCAL_BACKUP> <QUERY>]

see the section "Finding Snapshots" in 'man yabsm' for a detailed explanation on
how to find snapshots and backups.

examples:
    yabsm find home_snap back-10-hours
    yabsm f root_ssh_backup newest
    yabsm f home_local_backup oldest
    yabsm f home_snap 'between b-10-mins 15:45'
    yabsm f root_snap 'after back-2-days'
    yabsm f root_local_backup 'before b-14-d'

Synopsis

Yabsm is a powerful tool for managing your Btrfs snapshots. If you are interested in using Yabsm, then I recommend you consult the official documentation.

Create a Lock File in Perl with File::Temp

Nicholas Hubbard — Thu, 29 Sep 2022 20:17:37 +0000

A lock can serve many purposes in regards to avoiding race conditions. In this article we will explore how to implement a lock using Perl's built-in File::Temp module.

The Problem

We will write a program called "duckduckperl" that has the following usage:

usage: duckduckperl [retrieve] [print]

The retrieve command retrieves the HTML of a DuckDuckGo search for the word "perl", and writes it to $HOME/duckduckperl.html, overwriting this file if it already exists.

The print command prints the content of $HOME/duckduckperl.html.

Here is a version of the program that has a potential race condition:

#!/usr/bin/env perl

use strict;
use warnings;

my $OUTPUT_FILE = "$ENV{HOME}/duckduckperl.html";

my $USAGE = 'usage: duckduckperl [retrieve] [print]' . "\n";

(@ARGV == 1) or die $USAGE;

my $ARG = $ARGV[0];

if    ($ARG eq 'retrieve') { retrieve_html() }
elsif ($ARG eq 'print'   ) { print_html()    }
else                       { die $USAGE      }

sub retrieve_html {

    my $html = `curl --silent https://duckduckgo.com/?q=perl`;

    unless ($? == 0) {
        die "duckduckperl: curl command exited with status $?\n";
    }

    open my $fh, '>', $OUTPUT_FILE
      or die "duckduckperl: error: cannot open $OUTPUT_FILE: $!\n";

    print $fh $html;

    close $fh;
}

sub print_html {

    -f $OUTPUT_FILE
      or die "duckduckperl: error: cannot find file $OUTPUT_FILE\n";

    open my $fh, '<', $OUTPUT_FILE
      or die "duckduckperl: error: cannot open $OUTPUT_FILE: $!\n";

    my $html = <$fh>;

    close $fh;

    print $html;
}

The only part of this code that is important to understand is the &retrieve_html subroutine, because this is where the potential race condition comes from. This subroutine curl's the URL that represents a DuckDuckGo search of the word "perl", and dies if it fails. If the curl command succeeds it writes the outputted HTML $HOME/duckduckperl.html, overwriting any data already in the file.

Imagine if we call duckduckperl retrieve twice, and for whatever (network related) reason the second instance finishes first. When we go to call duckduckperl print, we will get the output of the first call instead of the second call, which probably is not expected.

The Solution

To avoid this problem we will write &retrieve_html to first check for the existence of a lock file, and if it exists waits for it to be deleted before continuing. If the lock file doesn't exist then it creates it before retrieving and writing the HTML, and deletes it afterwards. This guarantees that multiple
instances of duckduckperl retrieve terminate in the order they were called.

Perl's built-in File::Temp module provides useful features for creating a lock file. The most important feature that we will use is automatic deletion of the lock file when the File::Temp object is destroyed (garbage collected). This feature is available if we use File::Temp's OO interface.

Here is the updated version of &retrieve_html that uses a lock file:

use File::Temp;

sub retrieve_html {

    my $seconds = 0;
    while (grep /DUCKDUCKPERLLOCK$/, glob('/tmp/*')) {
        if ($seconds > 120) {
            die 'duckduckperl: error: aborting after waiting 2 minutes for lock file to be deleted' . "\n";
        }
        sleep 1;
        $seconds++;
    }

    my $lock_fh = File::Temp->new(
        DIR      => '/tmp',
        TEMPLATE => 'XXXX',
        SUFFIX   => '.DUCKDUCKPERLLOCK',
        UNLINK   => 1
    );

    open my $fh, '>', $OUTPUT_FILE
      or die "duckduckperl: error: cannot open $OUTPUT_FILE: $!\n";

    my $html = `curl --silent https://duckduckgo.com/?q=perl`;

    unless ($? == 0) {
        die "duckduckperl: curl command exited with status $?\n";
    }

    print $fh $html;

    close $fh;
}

The first part of the subroutine checks for the lock file, which is a file in the /tmp directory matching the regex /DUCKDUCKPERLLOCK$/. If this file exists, we check every second for the next 2 minutes to see if it is deleted, before giving up and exiting the program. If the lock file does not exist, then we create it using File::Temp::new.

We configure our File::Temp object with 4 options. The DIR option specifies the directory that we want to place the file in. The TEMPLATE option specifies the template to be used for naming the file. The X's represents random characters that File::Temp will fill in to guarantee the file it creates has a unique name. The SUFFIX option gives the file name a suffix, which we set to .DUCKDUCKPERLLOCK. We use this suffix to identify the lock file when checking for its existence. Finally, the UNLINK option specifies that we want to delete the file when the File::Temp object is destroyed (garbage collected). Conveniently, even if the curl command fails and we make the program die, the File::Temp object is still destroyed, and the lock file is deleted.

Using this version of &retrieve_html, we can rest assured knowing that multiple instances of duckduckperl retrieve will terminate in the order they were invoked.

Named Subroutine Arguments in Perl

Nicholas Hubbard — Thu, 22 Sep 2022 01:13:39 +0000

Naming your subroutine arguments has benefits around increasing the readability of your code. Lets look at an example of code that can benefit from named arguments:

sub safe_open {

    my $file = shift @_;
    my $mode = shift @_;
    my $die_on_failure = shift @_;

    my $success = open(my $fh, $mode, $file);

    if ((not $success) and $die_on_failure) {
        die "error: failed to open '$file': $!\n";
    }

    return $fh;
}

In the following examples we can see that the third argument allows us to control if we should kill the program if the file cannot be opened:

my $fh = safe_open('/file/that/doesnt/exist', '>', 1);
print $fh "Hello, World!\n";

# OUTPUT:
# error: failed to open '/file/that/doesnt/exist': No such file or directory

my $fh = safe_open('/file/that/doesnt/exist', '>', 0);
print $fh "Hello, World!\n";

# OUTPUT:
# print() on closed filehandle $fh at ./scratch.pl line 22.

Imagine if somebody who has never worked with this code before comes across these calls to &safe_open. It will be impossible for them to know what the third argument stands for unless they go and look at the actual code of the subroutine.

To make it easier for people to understand the meaning of the arguments, we can name them!

sub safe_open {

    my %args = (
        FILE => '',
        MODE => '',
        DIE_ON_FAILURE => 1,
        @_
    );

    $args{FILE} or die "error: missing FILE arg in call to &safe_open\n";
    $args{MODE} or die "error: missing MODE arg in call to &safe_open\n";

    my $success = open(my $fh, $args{MODE}, $args{FILE});

    if ((not $success) and $args{DIE_ON_FAILURE}) {
        die "error: failed to open '$args{FILE}': $!\n";
    }

    return $fh;
}

Lets look at some examples of calling this subroutine:

my $fh = safe_open(FILE => '/file/that/doesnt/exist', MODE => '>', DIE_ON_FAILURE => 1);
print $fh "Hello, World!\n";

# OUTPUT:
# error: failed to open '/file/that/doesnt/exist': No such file or directory

my $fh = safe_open(FILE => '/file/that/doesnt/exist', MODE => '>');
print $fh "Hello, World!\n";

# OUTPUT:
# error: failed to open '/file/that/doesnt/exist': No such file or directory

my $fh = safe_open(DIE_ON_FAILURE => 0, FILE => '/file/that/doesnt/exist', MODE => '>');
print $fh "Hello, World!\n";

# OUTPUT:
# print() on closed filehandle $fh at ./scratch.pl line 25.

There are a few things to note from these examples.

The arguments must be explicitly named using ARG => value syntax.
The argument order does not matter.
The DIE_ON_FAILURE argument is optional and defaults to true.

Our &safe_open subroutine has these features because of the strategy used for constructing the %args hash.

The reason the argument order doesn't matter is because hashes are unordered and in the end we are constructing a hash out of the argument list.

An array can be used to construct a hash as long as it has an even number of elements. We leverage this to use the @_ array to construct the %args hash. Here is an example of this behavior:

my @array = ('foo', 12, 'bar', 13);

my %hash = @array;

print "\$hash{'foo'} = $hash{'foo'}\n";
print "\$hash{'bar'} = $hash{'bar'}\n";

# OUTPUT:
# $hash{'foo'} = 12
# $hash{'bar'} = 13

We are able to give default argument values because of the fact that, if a hash definition defines the same key multiple times, then the last definition is used. If the user leaves off an argument, then it is not redefined and stays set to the default. Consider this example:

my %hash = (
    FOO => 12,
    FOO => 13
);

print "\$hash{'FOO'} = $hash{'FOO'}\n";

# OUTPUT:
# $hash{'FOO'} = 13

Synopsis

We can use the subroutines argument array (@_) to construct a hash, which can give us significant benefits in terms of code readability. By constructing a hash from the argument array, all calls to the subroutine must explicitly name their arguments the argument order becomes irrelevant, and arguments can be given default values.

Create a Daemon in Perl with Schedule::Cron

Nicholas Hubbard — Thu, 08 Sep 2022 01:20:36 +0000

A daemon is a program that runs in the background for an indefinite period of time.

An important daemon on Unix-like operating systems is the cron scheduler that can be configured to perform tasks periodically. Though there are many different types of daemons, we will explore how to create a cron daemon.

My favorite CPAN module for creating a daemon is Schedule::Cron, which allows us to create a daemon that dispatches Perl subroutines at preconfigured intervals. The reason I like Schedule::Cron is that it is easy to learn, easy to understand, and it works.

This article is not a complete overview of everything that Schedule::Cron can do, so be sure to read the official documentation afterwards.

A Simple Daemon

Let's write a simple (and useless) daemon that logs the current time every 5 minutes, and rotates the log file every hour, keeping at most 4 old logs. The current logfile will be named $HOME/times.txt, and the old log files will be named $HOME/times.txt.{1,2,3,4}.

First we need a function to write the time to the current log file.

sub append_time {
    my $logfile = "$ENV{HOME}/times.txt";
    open my $fh, '>>', $logfile or die "cannot open file '$logfile': $!";
    my $time = localtime();
    print $fh "$time\n";
    close $fh;
}

Next we need a function to rotate the old log files.

use File::Copy 'move';

sub rotate_time_log {
    my $logfile      = "$ENV{HOME}/times.txt";
    my @old_logfiles = grep /^$ENV{HOME}\/times\.txt\.\d+$/, glob "$ENV{HOME}/*";
    # We don't need to rotate unless we have more than 4 old logfiles
    return if @old_logfiles <= 4;
    unlink "$logfile.4";
    for (my $i = 3; $i >= 1; $i--) {
        move $old_logfiles[$i], $logfile.$i+1;
    }
    move $logfile, "$logfile.1";
}

Understanding how these functions work is not important for learning about Schedule::Cron.

Now that we have our time logging functions, let's initialize our daemon object. To initialize the daemon object we will use the Schedule::Cron::new function.

The first argument to Schedule::Cron::new must be a reference to a subroutine that will be used as a default if we add a cron entry without specifying the function we want to run. This is only useful if there is only one function we want our daemon to run. We won't be using this feature, so we just set it to a function that kills the program.

There are many options we can pass to Schedule::Cron::new, the only one we will use is processprefix which is used to give a prefix to the name of our daemon process.

use Schedule::Cron;

my $cron_daemon = Schedule::Cron->new(
    sub { die "time-daemon: error: default Schedule::Cron function was called\n" },
    processprefix => 'time-daemon'
);

The most important Schedule::Cron method is add_entry, which takes a cron string and a coderef. When we eventually run the daemon it will schedule the coderef to be run at the interval specified by the cron string.

Personally I can never remember the syntax for cron strings. I use the website crontab.guru for getting an English translation of what my cron string means, which makes it easy to build my cron strings.

Lets schedule our &append_time subroutine to be run every 5 minutes, and our &rotate_time_log subroutine to be run every hour.

$cron_daemon->add_entry(
    '*/5 * * * *',
    \&append_time
);

$cron_daemon->add_entry(
    '0 */1 * * *',
    \&rotate_time_log
);

We are now ready to start up the daemon using the Schedule::Cron::run method. This method takes many options but the only one we will use is detach, which will cause daemon process to detach itself from the current process.

my $pid = $cron_daemon->run(detach => 1);

print "started the time-daemon as pid $pid\n";

That's all there is to it! The basic recipe to follow is: initialize a Schedule::Cron object, schedule subroutines to be run with add_entry, then start the daemon with the run method.

Perl Project Management with App::plx

Nicholas Hubbard — Sat, 03 Sep 2022 00:09:10 +0000

App::plx (Plx) is a tool for configuring per-project Perl development environments.

Plx has good documentation, so in this post I will give a general overview of some of the problems that it solves and how to get started using it.

This is not a complete overview on everything that Plx can do.

Why Plx?

Imagine you have a project that depends on a Perl version greater than that of the Perl you have available. You will need to install the correct version of Perl and figure out how to get your project to use this Perl instead of the system's Perl.

Next imagine a scenario where you are working on two different Perl projects that depend on a different version of the same CPAN module. You will need to figure out how to install both versions into different locations, and then you will need to figure out how to locate the correct version from the two different projects.

You also may want to install all of your CPAN modules into your projects root directory so it can have its own private collection of modules that you know the project owns. Plx allows you to do this through simple integration with local::lib.

Plx also allows you to override your users perl through use of shell configuration. Personally I dont like touching my shell like this, but its still an option.

These are some important problems that Plx solves.

Installation

Before you can use Plx you must install Plx.

If you already have a CPAN installer, such as cpanminus, then you should probably just use that to install Plx.

Plx can also be bootstrapped into a self contained script like so:

$ mkdir "$HOME/bin"
$ wget https://raw.githubusercontent.com/shadowcat-mst/plx/master/bin/plx-packed -O "$HOME/bin/plx"
$ chmod +x "$HOME/bin/plx"

This is my favorite way to install plx because I get a single executable that just works.

You can install Plx into any directory, I just chose "$HOME/bin" for simplicity. Just make sure you pick a directory in your $PATH.

Initialization

If you want to use Plx for your Perl project, you must first initialize the project to use Plx. To do this we must cd into the root directory of the project and then execute Plx with the --init flag.

The --init flag behaves differently depending on its argument.

When --init is called with a file path, it assumes it is a path to a Perl interpreter and sets up Plx to use it.

$ plx --init /path/to/some/perl

When called with perl as the argument it sets up Plx to use the first Perl in your PATH.

$ plx --init perl

The final and most exciting way to call --init is with a Perl version number. When called with a version number, Plx will look for a Perl of the given version first in your PATH and otherwise via Perlbrew.

$ plx --init 5.36.0

After initializing Plx you can execute your project code with a command like $ plx /path/to/project/script.pl, and Plx will execute the script with the Perl interpreter you specified with the plx --init command.

This is how Plx solves the problem of needing to use a different Perl than your systems built-in Perl.

Installing CPAN Modules

My favorite feature of Plx is that it allows you to install modules off of CPAN into a local::lib using cpanminus. This allows you to segregate your CPAN modules dependencies on a per-project basis.

To do this we must cd into the root directory of our project and run the following command.

$ plx --cpanm -Llocal Some::Module

This will install Some::Module into a project-local library located in a directory named local/lib at the root of the project.

This solves the problem of two projects requiring different versions of the same CPAN module. If both projects use Plx they can simply install their desired version into a local::lib.

Userstrap

What if you want to use your own Perl interpreter and a local::lib when you are working outside of a dedicated Plx project?

Plx has a --userstrap flag that will set this up for you automatically.

$ plx --userstrap /path/to/some/perl

Calling --userstrap essentially sets up your $HOME to be a Plx project and sets up a local::lib in $HOME/perl5, installs App::plx and App::cpanminus into the local::lib, and adds a line to your $HOME/.bashrc that sets up Plx for your Bash shell.

Now when you run Plx from outside a dedicated Plx project it will use $HOME as a sort of default Plx project. You can use --userstrap to prevent needing to use your system Perl, so you and can instead always use Plx.

Note that --userstrap requires that you use a Bash shell.

Plx is For Everybody

Plx is designed to not only provide a nice experience for Perl developers, but also to be usable by a sysadmin that isn't a Perl expert. Therefore Plx is configured through simple text files that can be manipulated by hand, and allows multiple commands to be run in a single Plx invocation via the --multi flag, which makes scripting Plx cleaner.

Synopsis

Plx is a tool for creating per-project virtual Perl environments. Plx lets us avoid a lot of headaches that come with developing multiple Perl projects on the same system.

A lot of what Plx does can be done by combining features of other CPAN modules, but Plx brings together these functionalities in a way that is easy to use and understand.

This blog post is only a brief introduction to Plx. Please go on to read the manual for more a more detailed overview of its features.

Bonus Tip for Emacs Users

If you lint your Perl code with the Perl interpreter using Flycheck, you will need to determine if the buffer is part of a Plx project so it runs the Perl interpreter through Plx (otherwise you will have @INC-related errors).

Use the following code to do this:

(require 'flycheck)
(require 'projectile)

(add-hook 'cperl-mode-hook 'flycheck-mode)
(add-hook 'cperl-mode-hook 'my/cperl-select-correct-flycheck-checker)

(flycheck-define-checker my/perl-plx
  ""
  :command ("plx" "-w" "-c"
            (option-list "-I" flycheck-perl-include-path)
            (option-list "-M" flycheck-perl-module-list concat))
  :standard-input t
  :error-patterns
  ((error line-start (minimal-match (message))
          " at - line " line
          (or "." (and ", " (zero-or-more not-newline))) line-end))
  :modes (perl-mode cperl-mode))

(defun my/cperl-select-correct-flycheck-checker ()
  "If the current buffer is part of a plx project then use the `my/perl-plx'
checker, otherwise use the `perl' checker."
  (let ((proj-root (projectile-project-root (string-remove-suffix "/" default-directory))))
    (if (and proj-root (file-directory-p (concat proj-root ".plx")))
        (flycheck-select-checker 'my/perl-plx)
      (flycheck-select-checker 'perl))))

How I use Emacs to write Perl

Nicholas Hubbard — Wed, 24 Aug 2022 23:42:00 +0000

This post is about how I use Emacs to write Perl. I do not claim to have the best Perl setup of all time or anything like that. The features I need to write Perl effectively are syntax highlighting, auto-indentation, linting, and code navigation.

I personally like to build my own IDE by bringing together unrelated packages, which is in contrast to full blown IDE packages, such as Devel::PerlySense or Perl::LanguageServer. These packages just aren't for me.

Basics

By default Emacs uses perl-mode instead of the more advanced cperl-mode. Both packages are built-in, so to use cperl-mode instead of perl-mode all you have to do is add the following line to your config.

(fset 'perl-mode 'cperl-mode)

The cperl-mode that was released with Emacs 28 improved the syntax highlighting for regular expressions and heredocs, and fixed a few other annoying bugs.

If you are using an Emacs version less than 28 then I would recommend downloading the cperl-mode off the Emacs 28 (at least) branch. I personally place this file in ~/.emacs.d/cperl-mode/cperl-mode.el, then I load it with the following code.

(add-to-list 'load-path "~/.emacs.d/cperl-mode")
(require 'cperl-mode)

By default cperl-mode replaces trailing whitespace with underscores. You can automatically delete this with whitespace-cleanup-mode, or you can use this elisp:

(setq cperl-invalid-face nil)

cperl-mode defaults to indenting blocks by 2 spaces. You can modify this by setting the cperl-indent-level to some value other than 2.

You probably want multi-line statements wrapped in parens to be indented like a block. By default cperl-mode indents this hash declaration in a strange way (to me):

my %hash = (
            'foo' => 1,
            'bar' => 2,
            'baz' => 3
           );

To instead indent like this:

my %hash = (
    'foo' => 1,
    'bar' => 2,
    'baz' => 3
);

Add this to your config:

(setq cperl-indent-parens-as-block t)
(setq cperl-close-paren-offset (- cperl-indent-level))

Linting

Linting Perl code helps to quickly find bugs caused by typos or little errors. My favorite Emacs linting package is Flycheck, which comes with built-in support for Perl.

By default Flycheck checks your code with the Perl interpreter, but it also comes with integration with Perl::Critic.

I like to lint the file everytime I save, and I like to display any errors immediately. Here is how I accomplish this with Flycheck.

(require 'flycheck)
(setq flycheck-check-syntax-automatically '(mode-enabled save))
(setq flycheck-display-errors-delay 0.3)

To enable flycheck mode in cperl-mode, simply turn it on with a hook.

(add-hook 'cperl-mode-hook 'flycheck-mode)

Now Emacs will underline any syntax errors, and you can view the message in the echo area by placing your cursor on the erroneus code.

I cannot tell you how many simple errors you will catch just by using Flycheck!

Code Navigation

For jumping between function definitions I use dumb-jump, which usually just works. I configure dumb-jump to use ag for its searching which makes it work very quickly.

(require 'dumb-jump)
(setq dumb-jump-force-searcher 'ag)
(add-hook 'xref-backend-functions #'dumb-jump-xref-activate)

I can then use dumb-jump by calling the xref-find-definitions function while my cursor is on the symbol I want to search for. This function is bound to M-. by default.

Shell

A lot of people use M-x compile to run their code, and one of the various debugger packages to run the Perl debugger. Personally I just use plain old Bash with the built-in M-x shell. This makes my work flow when it comes to running and debugging quite similar to that of a classic Perl vimmer who does all their work in a terminal.

I use the wonderful shx package for making M-x shell a more usable shell interface, and I use shell-pop for popping up shell buffers that are automatically cd'd to the current files directory.

(require 'shx)
(add-hook 'shell-mode-hook 'shx-mode)

(require 'shell-pop)
(setq shell-pop-autocd-to-working-dir t)
(global-set-key (kbd "M-SPC") 'shell-pop)

Closing Thoughts

Every 3rd-party package I described in this post is useful not only for Perl, but for programming in any language. This gives a uniform experience across different programming languages. If I instead used one of the Perl IDE packages then I wouldn't get the same uniform experience when using other languages.

Make Perl Count Arguments

Nicholas Hubbard — Wed, 17 Aug 2022 19:28:00 +0000

Unlike many programming languages, Perl doesn't check that functions are called with the correct number of arguments. This is because Perl subroutines are variadic by default, which makes a lot of programming tasks really easy. The downside is that a lot of the time functions only make sense if they are called with a certain number of arguments.

Most of the time if a subroutine is called with an incorrect number of arguments it is because the programmer made a simple typo. To make our lives as programmers easier, it would be nice if we could detect this situation and give a clear error message about what went wrong.

We will explore a solution to this problem.

WARNING: The examples in this artical call Perl from the command line. If you don't understand Perl command line syntax, only pay attention to what comes after the -E.

Lets look at an example.

package T;

sub num_diff {
    my $n1 = shift;
    my $n2 = shift;
    return abs($n1 - $n2);
}

1;

When called with two arguments our function behaves as expected.

$ perl -W -I. -MT -E 'say T::num_diff(7, 17)'
10

But what if we call num_diff with more than two arguments?

$ perl -W -I. -MT -E 'say T::num_diff(23, 21, 48)'
2

Perl has no issue if we call this function with 3 arguments, and happily returns us the difference between the first two arguments. This is bad! The difference between 3 numbers is certainly not the difference between the first 2.

A nice way to deal with this problem is to use signatures, which provides syntax for declaring a subroutines arguments. Lets rewrite num_diff using signatures.

package T;

use v5.20;
use feature 'signatures';

sub num_diff($n1, $n2) {
    return abs($n1 - $n2);
}

1;

Lets see what happens when we call num_diff with more than 2 arguments.

$ perl -I. -MT -E 'say T::num_diff(22, 33, 8)'
Too many arguments for subroutine 'T::num_diff' (got 3; expected 2) at -e line 1.

Awesome, our problem is solved! With signatures Perl can count our subroutine arguments and give us error diagnositics when we mess up.

Unfortunately though there are some downsides to signatures. First off signatures didn't exist until Perl version 5.20, so if you're use an old Perl signatures are not an option. The other downside is that signatures were experimental until Perl version 5.36, which is why the use feature 'signatures' statement is necessary.

I have been working on a Perl project that uses Perl version 5.16.3, so I cannot use signatures. To count arguments I wrote a function that I call as the first statement in subroutines that kills the program if it did not receive the correct number of arguments.

use Carp 'confess';

sub arg_count_or_die {

    # Carp::Confess unless $num_args is in range $lower-$upper

    my $lower    = shift;
    my $upper    = shift;
    my $num_args = @_;

    ($lower, $upper) = ($upper, $lower) if $lower > $upper;

    unless ($lower <= $num_args && $num_args <= $upper) {
        my $caller = ( caller(1) )[3];
        my $expected_plural = $lower == 1 ? '': 's';
        my $got_plural = $num_args == 1 ? '' : 's';
        my $arg_range_msg = $lower == $upper ? "$lower arg$expected_plural" : "$lower-$upper args";
        confess("yabsm: internal error: called '$caller' with $num_args arg$got_plural but it expects $arg_range_msg");
    }


    return 1;
}

Lets rewrite num_diff to use this function.

sub num_diff {

    arg_count_or_die(2, 2, @_);

    my $n1 = shift;
    my $n2 = shift;

    return abs($n1 - $n2);
}

Again, lets call num_diff with more than 2 arguments

$ perl -I. -MT -E 'say T::num_diff(22, 33, 8)'
my-program: internal error: call to 'T::num_diff' passed 3 args but expects 2 args at T.pm line 19.
        T::arg_count_or_die(2, 2, 22, 33, 8) called at T.pm line 27
        T::num_diff(22, 33, 8) called at -e line 1

We can see that by using Carp::Confess we get an excellent error message that shows the call stack that lead to the to our erroneus subroutine call. By prefixing the error message with my-program: internal error, if this error ever occurs our user knows that they found a bug and can send us the stack trace which will be very useful for debugging.

4 Unrelated Perl Tidbits

Nicholas Hubbard — Wed, 10 Aug 2022 17:18:00 +0000

I recently acquired a copy of Programming Perl, often referred to as "The Camel Book" in the Perl community. After reading the first 4 chapters I thought I would share a few Perl tidbits I found interesting.

While <> is True

Say we have the following file named logos.txt.

Onion
Camel

Raptor

And in the same directory we have the following program named scratch.pl.

open my $fh, '<', 'logos.txt' or die;
while (my $logo = <$fh>) {
    print $logo;
}
close $fh;

As expected our program prints the contents of logos.txt.

$ perl ./scratch.pl
Onion
Camel

Raptor

Something I never really considered though, is why doesn't the loop exit when <> reads the empty line in logos.txt? Shouldn't <> return an empty string which is a false value?

According to Programming Perl, the reason why the loop doesn't exit is because <> reads the newline at the end of the line so we actually get "\n" which is a true value. Turns out this is a fib, and the actual reason is that while (my $logo = <$fh>) ... expands into while (defined (my $logo = <$fh>)) ..., and "\n" is a defined value.

We can show this by deparsing the code with B::Deparse.

$ perl -MO=Deparse,-p,-sCi2 -ne 42
LINE: while (defined(($_ = readline(ARGV)))) {
  '???';
}
-e syntax OK

Heredocs Can Execute Shell Commands

Most Perl programmers know that if you single quote a heredocs terminating string you prevent variable interpolation.

my $var = 12;
print <<'EOS';
Hello
$var = 12
EOS

You can see in the programs output that $var was not interpolated.

$ perl ./scratch.pl
Hello
$var = 12

But did you know that if you backquote the terminating string then each line is executed as a shell command?

print <<`EOC`;
echo this is a shell command
echo this is also a shell command
EOC

When we run this program we can see that the echo commands were executed.

$ perl ./scratch.pl
this is a shell command
this is also a shell command

The Comma Operator

I always took commas for granted, never realizing they were actually an operator.

Did you ever wonder why lists return their last element when evaluated in scalar context? Turns out it is due to the comma operator.

In scalar context the comma operator "," ignores its first argument, and returns its second element evaluated in scalar context.

This means that in scalar context the list (11, 22, 33) will evaluate to 33. The first comma operator will throw away the 11 and then return (22, 33) evaluated in scalar context, which will be evaluated by throwing away the 22 and returning 33.

Auto-Incrementing Strings

In perl you can not only auto-increment numbers, but also strings. To increment a string it must match the regex /^[a-zA-Z]*[0-9]*\z/.

Single alphabet character strings are incremented intuitively.

$ perl -E 'my $v = "a"; say ++$v'
b
$ perl -E 'my $v = "B"; say ++$v'
C

What happens though if we increment a non-alphanumeric char. Will it give the next ASCII character? Turns out non-alphanumeric characters are treated as 0's when incrementing. Fortunately if we use warnings Perl will give us a heads up.

$ perl -W -E 'my $v = "-"; say ++$v'
Argument "-" treated as 0 in increment (++) at -e line 1.
1

What happens if we increment z, which is at the end of the alphabet? Do we wrap back around to a? This is where things get interesting. Perl increments the string just like you would in a regular number system. Just like 9 + 1 = 10 in decimal z + 1 = aa in … stringimal.

$ perl -E 'my $v = "z"; say ++$v'
aa

Here are some more examples to show you the magic of string auto-incrementing.

$ perl -W -E 'my $v1 = "foo"; say ++$v1'
fop
$ perl -W -E 'my $v1 = "az"; say ++$v1'
ba
$ perl -W -E 'my $v1 = "a9"; say ++$v1'
b0

Forem: Nicholas Hubbard

How I used a named pipe to save memory and prevent crashes (in Perl)

EDIT

4 More Unrelated Perl Tidbits

Built-Ins Can Be Overridden with Lexical Subroutines

Recursive Anonymous Subroutines With __SUB__

Goto Searches For Labels In The Dynamic Scope

Regex Modifier For Only Portions Of The Regex

An Interesting Perl Pattern That Doesn't Work

Blocks

Closures

The Problem

Solutions

Use Dist::Zilla to Create a Perl Distribution

What is a Perl distribution?

Why Dist::Zilla?

Using Dist::Zilla

The "dist.ini" File

Synopsis

How I use Yabsm to Manage my Btrfs Snapshots

Snapshots vs Backups

My Btrfs Filesystem

My Yabsm Configuration

Yabsm Dir

Subvol

Snap

SSH Backup

Local Backup

Finding Snapshots

Synopsis

Create a Lock File in Perl with File::Temp

The Problem

The Solution

Named Subroutine Arguments in Perl

Synopsis

Create a Daemon in Perl with Schedule::Cron

A Simple Daemon

Perl Project Management with App::plx

Why Plx?

Installation

Initialization

Installing CPAN Modules

Userstrap

Plx is For Everybody

Synopsis

Bonus Tip for Emacs Users

How I use Emacs to write Perl

Basics

Linting

Code Navigation

Shell

Closing Thoughts

See Also

Make Perl Count Arguments

4 Unrelated Perl Tidbits

While <> is True

Heredocs Can Execute Shell Commands

The Comma Operator

Auto-Incrementing Strings

Recursive Anonymous Subroutines With SUB