Forem: Waqar Ahmed

Getting Started with Git

Waqar Ahmed — Mon, 08 Apr 2024 11:42:51 +0000

What is Git?

Snapshots, Not Differences

The major difference between Git and any other VCS is that Git stores snapshots, while other system store file differences.

Most other versioning systems store data as changes to a base version of each file. However. Git stores data as snapshots of the project over time.

Git thinks of its data more like a series of snapshots of a miniature filesystem. With Git, every time you commit, or save the state of your project, Git basically takes a picture of what all your files look like at that moment and stores a reference to that snapshot.

To be efficient, if files have not changed, Git doesn’t store the file again, just a link to the previous identical file it has already stored. Git thinks about its data more like a stream of snapshots.

Nearly Every Operation Is Local

Most operations in Git need only local files and resources to operate — generally no information is needed from another computer on your network.

Git simply reads project data directly from your local database, no server access is necessary. This also means that there is very little you can’t do if you’re offline.

Git Has Integrity

Everything in Git is checksummed (SHA-1 hash) before it is stored and is then referred to by that checksum. So, any change in file or directory structure will change its checksum, which is detected by Git. This means it’s impossible to change the contents of any file or directory without Git knowing about it.

You can’t lose information in transit or get file corruption without Git being able to detect it.

Git stores everything in its database not by file name but by the hash value of its contents.

Git Generally Only Adds Data

When you do actions in Git, nearly all of them only add data to the Git database.
As with any VCS, you can lose or mess up changes you haven’t committed yet, but after you commit a snapshot into Git, it is very difficult to lose. Especially if you regularly push your database to another repository

The Three States Of Git

Git has three main states that your files can reside in: modified, staged, and committed:

Modified means that you have changed the file but have not committed it to your database yet.
Staged means that you have marked a modified file in its current version to go into your next commit snapshot.
Committed means that the data is safely stored in your local database.

This leads us to the three main sections of a Git project: the working tree, the staging area, and the Git directory.

Working tree, staging area, and Git directory (source: git-scm.com)

The working tree is a single checkout of one version of the project. These files are pulled out of the compressed database in the Git directory and placed on disk for you to use or modify.

The staging area is a file, generally contained in your Git directory, that stores information about what will go into your next commit. Its technical name in Git parlance is the “index”, but the phrase “staging area” works just as well.

The Git directory is where Git stores the metadata and object database for your project. This is the most important part of Git, and it is what is copied when you clone a repository from another computer.

The basic Git workflow goes something like this:

You modify files in your working tree.
You selectively stage just those changes you want to be part of your next commit, which adds only those changes to the staging area.
You do a commit, which takes the files as they are in the staging area and stores that snapshot permanently to your Git directory.

In the second article of this series, I will post about Git installation and configurations. The third would be about Git basics and getting a Git repository.

Source: waqar.org/getting-started-with-git

References

Chacon, S. and Straub, B. (2014) Pro Git. 2nd ed. Berlin, Germany: APress.
Book (2014) Git-scm.com. Available at: https://git-scm.com/book/en/v2/ (Accessed: March 22, 2024).

9 Essential commands to unlock the power of vim

Waqar Ahmed — Mon, 09 Aug 2021 11:58:49 +0000

Bram Moolenaar , the creator of vim, cloned and improved vi and released it in 1991. The name vim came from "vi imitations", later changed to "Vi improved". Gradually it became the most popular text editor among Linux users. In 2019 Stackoverflow developers survey vim was the fourth most popular text editor.

In their last survey, Stackoverflow revealed over 2.1 million people visited the ' How do I exit the Vim editor? ' question on Stack Overflow. So I am writing some top googled vim commands and the general know-how required to use vim.

Vim modes

Out of so many, the most frequently used are "command-line mode", "normal mode" and "insert mode".

Normal mode

You are in normal mode when you open a file. To go back to the "normal mode", press the Esc key.

Command-line mode

To run a command, go to the command-line mode by typing colon :.

Insert mode

Get into "insert mode" by pressing i in "normal mode".

Vim commands

Press Esc key to enter "normal mode" then press colon : to enter "command-line mode". At the bottom left corner of vim, a colon : appears. Now you can type any command and press the Enter key to execute the command.

vim command to open a file

Type vim preceded by file name to open a file in the vim text editor.

vim command to save and exit

Press the Esc to get out of "insert mode". Press colon : to get into the "command-line mode" and type wq to write and quit the file. If you have not made any changes to the file, you can exit by q command.

:q to quit
:wq to write and quit
:q! to quit without writing
:x to save and quit (similar to :wq)
:exit save and exit (similar to :x)

vim command to edit file

Press the i key in "normal mode" to start editing the file. Move around with arrow keys or use these keys:

l move right
k move up
j move down
h move left

vim command to go to the end of the file

Type capital G or shift+g to go to the end of the file in vim.

vim command to go to top of file

Type gg or 1G to go to the beginning of the file.

vim command to delete all lines

Use letter d to delete. Go into "command-line mode" by pressing : and type %d to delete all lines. You can also use range 1,$d. Just like regular expressions $ sign represents the end of the file.

vim command to show line numbers

In "command-line mode" type, set numbers or short command set nu to display line numbers of the file.

vim command to replace string

Type search string followed by the replacement string %s/search/replace/g.

vim command to delete lines with pattern

Inside "command-line mode" type, g/pattern/d

Run the :help command to check usage documentation. Add a query after :help to get specific help. Vim comes with a separate tutor program to teach vim usage. Run vimtutor in the terminal for this 20 minutes practice tutorial. Use the comments section to mention any other essential commands that I have missed.

Bootstrap VS Tachyons

Waqar Ahmed — Wed, 30 Jun 2021 17:13:45 +0000

Compared to Tachyons, Bootstrap is higher level framework, but Bootstrap is heavy. Bootstraps btn class adds over 400 lines of CSS code to the HTML element.
Customisation of Bootstrap is frustrating. To change the look and feel of Twitter, one would need to look into long pages of code adding even more lines and probably many !important directives.
Dakota Lee Martinez in his article provides a button example created with Tachyons, which is elegant and simple.

<a 
class="pa2 bg-light-gray link ba bw1 bg-hover-moon-gray black br3" 
href="#">
Link Button
</a>

Above code adds padding with class pa2, light grey background with bg-light-gray, link class removes hyperlink’s underline, class ba adds border and its width is set by bw1, bg-hover-moon-gray changes background on hover.

Tachyons Vs Bootstrap in the eyes of developers

In a comment of stackshare Bridget Sarah says Bootstrap is pain to override and you can tell from distance if you are using Bootstrap as every thing looks similar. She has found Tailwind CSS, which she finds better and faster than Bootstrap.
According to Daniel Eden, Designer at Facebook, Tachyons documents and limits properties of components which he calls subatomic design. These subatomic properties are composed to create components, rather than recreating components and its variants.

Tachyons developer, Adam Morse in this talk at DevShop London 2016, talks about motivation behind Tachyons. He discusses the problem of continuous over-riding your own written CSS code, writing tons of CSS code, struggle to keep all this info in your head and the need to refactor 200Kb CSS file. His answer to the problem is Tachyons. SUIT CSS (Style Tools for UI Components). was the initial inspiration that lead to creation of Tachyons. Unlike Bootstrap where you redefine a component multiple times, SUIT had a class which would not redefine itself or mutate later.

Tachyons Primer

Tachyons CSS Grids and Colour

To find more classes check Tachyons Styles page.
Cheatsheet with colours and relevant class names for text, backgrounds and borders.
Some friendly colour combinations by Tachyons.

Tachyons Installation and usage

CDN

Quickest and easiest method is to include tahyons in HTML head section.

<link rel="stylesheet" href="https://unpkg.com/tachyons@4.10.0/css/tachyons.min.css"/>

NPM

Check Tacyons npm or simply run following.

npm install --save-dev tachyons@4.10.0

or use Git repo and build locally

git clone git@github.com:tachyons-css/tachyons.git
cd tachyons
npm install && npm start

Automate server intrusion detection and banning with Fail2ban

Waqar Ahmed — Tue, 29 Jun 2021 18:23:31 +0000

A Fail2Ban installation monitors server access logs and automatically bans IP addresses of bots and attacking users in iptables. Fail2Ban analyses server logs and identifies a pattern where a suspicious user or bot is trying to access restricted areas on the server. Brute force attacks repeatedly fail and use combinations of login credentials on after another. Fail to ban can detect failed login attempts on SSH and Apache webserver.

Fail2Ban Installation

Installation is straight forward update apt-get and run instal fail2ban as a root user. Theses are Debian/Ubuntu commands for CentOS yum would replace apt-get and paths/locations might differ for configuration files. Check this tutorial for non-Debian operating systems.

sudo apt-get update
sudo apt-get install fail2ban

Fail2Ban Configuration

Fail2Ban can keep server admin updated with emails but to keep it simple, let’s not configure this additional feature. The configuration file /etc/fail2ban/fail2ban.conf is superseded by fail2ban.local file. So this local config file needs to be created to keep additional settings. Here is part of the default installation config file.

[Definition]
# Option: loglevel
# Notes.: Set the log level output.
#         CRITICAL
#         ERROR
#         WARNING
#         NOTICE
#         INFO
#         DEBUG
# Values: [ LEVEL ]  Default: ERROR
#
loglevel = INFO
logtarget = /var/log/fail2ban.log
# Options: dbfile
# Notes.: Set the file for the fail2ban persistent data to be stored.
#         A value of ":memory:" means database is only stored in memory 
#         and data is lost when fail2ban is stopped.
#         A value of "None" disables the database.
# Values: [ None :memory: FILE ] Default: /var/lib/fail2ban/fail2ban.sqlite3
dbfile = /var/lib/fail2ban/fail2ban.sqlite3
# Options: dbpurgeage
# Notes.: Sets age at which bans should be purged from the database
# Values: [ SECONDS ] Default: 86400 (24hours)
dbpurgeage = 86400

Fail2ban focuses on failed SSH login attempts by default. Using custom config file jails for HTTP web server like Apache, FTP and mail server can be enabled.

Fail2Ban Apache Jail Configuration

[apache-auth]
enabled  = true
port     = http,https
logpath  = /var/log/apache2/error.log
maxretry = 3
findtime = 600
bantime = 1h

So here we have set up fail2ban to look apache2 error logs for brute force attack using one password after another. Three failed login attempts within 600 seconds would lead to an IP address ban for an hour. Restart fail2ban for the changes to take effect sudo systemctl restart fail2ban. To enable fail2ban service to start with boot use sudo systemctl enable fail2ban.

Other Apache error log jails

Apache no script

[apache-noscript]
port     = http,https
logpath  = %(apache_error_log)s
ignoreip = localhost or an IP

Fail2ban analyses errors of not found .php, .asp, .pl scripts. Client attempting to exploit known scripts are banned.

Apache overflows

[apache-overflows]
logpath  = %(apache_error_log)s
maxretry = 2

Detects apache buffer overflow attempts by the client using long suspicious URLs.

Apache no home

[apache-nohome]
port     = http,https

Detects failures to find a home directory on a server.

Apache bot search

[apache-botsearch]
enabled  = true
port     = http,https
maxretry = 2

Alternative to apache-noscript filter this filter aims at blocking specific URLs, script or webservices.

Apache access log jails

Beware, access logs of busy websites can be huge. Setting Fail2ban to monitor access log can have a negative impact on server performance. So think about costs and benefits of access log jails before enabling them.

Apache bad bots

Bans bots identified as spammer robots crawling the web for email addresses.

[apache-badbots]
port     = http,https
logpath = %(apache_access_log)s

PHP URL fopen

Matches and blocks fopen URL PHP injection attacks

[php-url-fopen] 
port    = http,https

Webmail jails

Jails for roundcude, openwebmail and horde are available, just add the location of log file in these jails.

[roundcube-auth]
port     = http,https
logpath  = %(roundcube_errors_log)s
#if roundcube logs to journal then use following.
#backend = %(syslog_backend)s
[openwebmail]
port     = http,https
logpath  = /var/log/openwebmail.log
[horde]
port     = http,https
logpath  = /var/log/horde/horde.log

Fail2ban custom filters and jail for WordPress

Error logs have multiple failed requests to WordPress login URLs or files like wp_login.php. This blog post creates a custom regular expression to scan failed WordPress logins in the error log. Using this RegEx a new filter file is created in Fail2ban directory. Once the filter is in place a [wordpress] jail can be added in jail.local. You can easily create your own filters with custom RegEx. Try online regular expression services like regex101.com with your error log to create a regular expression.

Fail2ban client and commands

Use Fail2ban client to setup configuration or check status of jail and banned IP addresses.

Check status of Fail2ban

sudo fail2ban-client status

Check status of Apache jail

sudo fail2ban-client status apache

Ban an IP address directly with client

sudo fail2ban-client set apache banip x.x.x.x

Similarly unban an IP address

sudo fail2ban-client set apache unbanip x.x.x.x

Restart after editing jail configuration for changes to take effect

sudo systemctl restart fail2ban

Check firewall rules added to iptables

sudo iptables -L

Start fail2ban on boot

sudo systemctl enable fail2ban

I am also on Twitter. You can follow me there as I regularly tweet about my journey.

References

How To Protect an Apache Server with Fail2Ban on Ubuntu 14.04 — By Justin Ellingwood, Digitalocean.com/community
Fail2ban jail configuration — Github.com/fail2ban
Set Up Fail2ban To Protect An Apache Web Server — hitjethva, devops.ionos.com/tutorials
A Tutorial for Using Fail2ban to Secure Your Server — Linode.com

MySQL database management using the terminal

Waqar Ahmed — Mon, 28 Jun 2021 17:42:42 +0000

To log in to MySQL server via terminal, use login credentials and hostname.

mysql -h localhost -u username -p

MySQL server will prompt for the password. The -h hostname flag is optional use only when the host is different from the default value localhost.

Once logged in, you can run any SQL query in the MySQL console. However, select the database first.

show databases;

The above command will display a list of all databases. Run the use command to select a database.

use database_name;

Now you can run any SELECT, UPDATE, DELETE query on the table. You can CREATE and DROP the tables.

To see the list of all tables, run this command.

show tables;

Describe table command shows the columns and their attributes like names, data types, collation, primary key, index nullability.

describe table_name;
DESC table_name;

Similarly, describe user command displays user details and privileges.

describe user_name;

Things to remember in MySQL command-line

Before moving to the user and database creation command, there are a few things to remember.

Use a terminator; after every query.
For a multi-line query, hit enter to go to the new line.
To clear the SQL query without running it, use the clear \c flag.
To get more help, use the help \h flag.
To exit from MySQL, use the quit \q flag.

Root user VS non-root user

The MySQL root user has full access to the database server. Using root can seriously mess things up or delete critical stuff on the server without even warnings. It would be insecure to access MySQL as a root user from a website.

To create a non-root new MySQL user and give it only needed privileges and access to a database, follow these steps.

1. Create a new MySQL user

Create a new MySQL user with a password.

CREATE USER 'new_user'@'localhost' IDENTIFIED BY 'user_password';

2. Grant Database access to new user

GRANT ALL ON new_database.* TO 'new_user'@'localhost';

3. Reload MySQL privileges

To activate newly assigned privileges run

FLUSH PRIVILEGES;

To give privileges to a table only, use database.table_name notation. To see the privileges of new user run,

SHOW GRANTS FOR 'new_user'@'localhost'

Create a new MySQL Database

Log into MySQL shell using your credentials and run the following command to create a new database.

create database new_database;

Vagrant Development Environment Setup

Waqar Ahmed — Thu, 24 Jun 2021 05:52:14 +0000

What might happen to the development environment if you upgrade OS. Would Apache configuration need to be changed? Will MySQL work?

Also, programs start stacking up. You would need so many different programs like PHP, Python, MySQL, maybe nodeJS, which get jumbled on your computer.

Vagrant provides a separate OS box, a Virtual Machine loaded with PHP, Python-related development environments. It helps keep your OS free and clean.

Vagrant Development Environment

Vagrant gets rid of the many development and production environments. Usually, we develop on Mac and need to deploy on a Linux server. Vagrant helps keep the process quick and easy to deploy after coding.

WAMP, MAMP are still much easier to install and needs just a one-click setup. However, vagrant setup without advanced stuff like chef or puppet can be simple.

Head to the vagrant website to download the latest, then grab a virtual machine for your system from vagrant VirtualBox.

After install, you should be able to use the vagrant command in the terminal. Use vagrant init to create a project in the existing directory.

Vagrant will place the configuration file 'Vagrantfile' in the directory.

Vagrantfile is editable in nano or vi. The Vagrantfile is a big file. It contains extensive information about a couple of settings. So you would not need Google to search to find the specific 'settings' here.

You will need a vagrant box to start, search boxes here or try using vagrant boxes. Add it to the configuration.

config.vm.box = “base”
config.vm.box_url = “http://www”
config.vm.network :forwarded_port, guest: 80, host:8080

Vagrant's website has a getting started section, which can also help to select a box. Their default precise64 can be used directly at the time of project creation. It will set up Ubuntu 12.0 virtual box on your virtual machine.

vagrant init hashicorp/precise64
vagrant up // to start the virtual machine

The first time will take longer as Vagrant would download the specified box. Once, Vagrant is up and running. SSH into a new virtual box using this command.

vagrant SSH

You will SSH into a fresh installation of Ubuntu 12.0. Now you can use sudo apt-get install to install PHP or the LAMP Stack with Git, WGET, ZIP and other essential programs all in this virtual box, separate from your operating system.

To destroy VM, run vagrant destroy command

vagrant destroy

Instead of manually downloading and installing the whole Stack, you can use vagrant configuration to do that. Setup a bash script with all installation commands and add it into vagrant configuration Vagrantfile. The bash file will look like

#!/usr/bin/env bash

The above line is how to shell script must start.
Down below, we are updating/upgrading the Ubuntu repository package list before any installation.

sudo apt-get update
sudo apt-get -y upgrade
sudo add-apt-repository ppa:ondrej/php
sudo apt-get update
sudo apt-get install python-software-properties

Install Apache PHP, PHPs required extensions and other needed stuff.

-y will ok all yes/no prompts during the installation.

sudo apt-get install -y wget apavhe2 php php-curl

Apache configurations: enables mod_rewrite, sets document root to shared /vagrant directory and creates a symlink to /var/www.

sudo a2enmod rewrite
rm -rf /var/www
ln -fs /vagrant /var/www
sudo service apache2 restart

The vagrant project directory gets shared by both your local machine and the virtual machine. Whatever you save here remains available to both environments.

Edit Vagrantfile and add bash startup script.

config.vm.box_provision :shell, :path=> "bash_file_name.sh"

If the vagrant box is already running, Vagrantfile changes will not reflect. Run the vagrant provision command to read from the configuration and update box.

vagrant provision

Now, use your IDE on the local machine to create a PHP file in shared document root or create one on the virtual machine.

Check it in the browser. We have set up network port forwarding in the Vgrantfile configuration already. Type this URL in the browser.
http://localhost:8080/filename.php

I am also on Twitter. You can follow me there as I regularly tweet about my journey.

References

Back in 2015/16, I was a Jeffery Ways Laracasts subscriber. I have followed Laracasts vagrant simplified video. Video, in turn, uses a great resource from dev-metal.com, especially this post about a super simple vagrant development environment and LAMP installation.

Move a MySQL Database from one host to another using Terminal

Waqar Ahmed — Tue, 22 Jun 2021 17:03:29 +0000

There can be multiple ways for MySQL/MariaDB database export and import during database server migration or database backup. The quickest seems to be the command-line one. It seems much faster than PHPMyAdmin or SequelPro clients. I am sharing Linux commands for MySQL/MariaDB export and import.

Export MySQL Database

MySQL data backup program mysqldump creates a dump file of SQL statements for table data and database schema.

Log into MySQL on Linux shell. Depending on your MySQL server settings, use

mysqldump -u username -p database_name > db_dump.sql

mysqldump does not show a success message. Do ls in the directory where you ran the command to find DB-dump.sql. You can include or exclude specific content into a dump and have the gzipped compression. For details visit MySQL dump: Database backup program resources.

Moving MySQL dump to a new server

In terminal use scp to move database backup.

scp user@oldhost:/home/user/db_dump.sql user@newhost:/home/user

For more details, check this SCP command post to move the file from a server to another using the terminal.

Depending on your convenience and database size you can move SQL dump via FTP, SCP, or simply move the dump to a public_html directory and access it on HTTP. On your destination server run a WGET command to copy the dump file.

You will need to create a new database and a new database user account aside from the root user.

Import MySQL Database

On the Linux shell of the destination server, move to the directory where you have the database SQL dump file and run this command.

mysqlimport -u username -p database_name < db_dump.sql

you can also import in MySQL command line, log in to MySQL command line and run

mysql> use database_name;
mysql > source db_dump.sql

or run this one command in Linux shell

mysql -u username -p database_name < db_dump.sql

mysqldump | A database backup program

Do not run mysqldump inside the MySQL console. The mysqldump is not a MySQL command. It is a separate program to be run in a Linux shell.

The mysqldump command can also generate output in CSV, other delimited text, or XML format. mysqldump is convenient and flexible as you can view and edit the output before restoring. However, restoring data can be slow because executing SQL statements involves disk I/O operations. The best use cases for mysqldump are small sites, WordPress blog database or during the development phase of a project. For large-scale databases using physical backup and restore is more appropriate.

mysqldump can also populate databases by copying data from one MySQL server to another:

mysqldump --opt db_name | mysql --host=remote_host -C db_name

mysqldump options

--opt Enabled by default, --opt is shorthand for --add-drop-table --add-locks --create-options --disable-keys --extended-insert --lock-tables --quick --set-charset

--skip-add-locks Do not add locks
--skip-opt Turn off options set by --opt
--verbose Verbose mode
--where Dump only rows selected by given WHERE condition
--xml Produce XML output

mysqldump filter options

--tables add table name arguments following the option to dump only those tables
--ignore-table exclude tables from dump --ignore-table=db_name.tbl_name
--no-data, -d dump only the CREATE TABLE statement for the table

I am also on Twitter. You can follow me there as I regularly tweet about my journey.

Replace FTP with the SCP command in the terminal

Waqar Ahmed — Tue, 22 Jun 2021 14:45:47 +0000

SCP comes in handy to copy files to a remote server from a computer or vice versa. You can also copy server files to another server.

All you need is your server’s IP address and user credentials to start copying. Here are some common SCP commands use cases.

Copy file from your computer to a remote server

scp file.zip user@hostip:/home/directory

This command will copy file.zip in your terminals present working directory to the home directory of the server.

Copy file from a remote server to your computer

scp user@hostip:/home/directory/file.zip ~/Downloads

You are copying file.zip from the server’s home directory to your local downloads folder.

SCP between two remote hosts from your computer

Another use case of the SCP command is during site migration to a new server. Zip your site’s root directory and copy it to the new server.

Provide login credentials and file paths of both servers to SCP in your computer’s terminal.

scp user@remotehost1:/home/file.zip user@remotehost2:/home

SCP Options

Mention transfer port

Add -p flag to add port of remote host

scp -p 1122 file.zip user@hostip:/home/directory

Use file compression

Add a -C flag to enable the gzip compression in SSH transfer

scp -C localfile.zip user@hostip:/home/directory

More SCP Options

-v Verbose SCP operation
-p <port> Mention remote host port
-P Save the file’s modification and access times
-r Copy recursively
-c <cipher> Specify the cipher for data encryption

So if you are still using an FTP client, use the SCP command in the terminal for a secure transfer.

PHP Tricky True False Examples

Waqar Ahmed — Mon, 07 Jun 2021 14:06:46 +0000

The result of a PHP True False statement might be different from what looks like a simple and logical result.

PHP Comparison Operators == VS ===

PHP with loosely == operator will not compare type, so numeric strings will be converted to numbers and compared numerically. Below are two examples: for more on PHP comparisons, check the PHP.net page.

var_dump(0 == "a"); // 0 == 0 -> true
var_dump(10 == "1e1"); // 10 == 10 -> true

PHP strict comparison === will not convert string to a numeral. It compares both: type and the value. So examples above, comparing two of the different types, will always be false.

var_dump("1" === "01"); // 0 == "a" -> false
var_dump("10" === "1e1"); // 10 == "1e1" -> false

StackOverflow user Nick has added nice detailed comparison tables of == and === operators with TRUE, FALSE, NULL, 0, 1, -1, ‘0’, ‘1’, ‘-1’.

PHP TRUE, FALSE

In PHP: an undeclared variable, empty array, “0”, 0 and empty string are false while -1 is true. Here are some more TRUE, FALSE examples.

PHP STRINGS

var_dump((bool) "");        // bool(false)
var_dump((bool) "0");       // bool(false)
var_dump((bool) "1");       // bool(true)
var_dump((bool) "alpha");   // bool(true)

PHP INT, FLOAT

var_dump((bool) 1);           // bool(true)
var_dump((bool) -1);          // bool(true)
var_dump((bool) 2.3e5);       // bool(true)
var_dump((bool) -2);          // bool(true)
var_dump((bool) 0.0);         // bool(false)
var_dump((bool) -0.1);        // bool(true)

PHP ARRAYS

var_dump((bool) array());    // bool(false)
var_dump((bool) array(5));   // bool(true)

OTHERS

var_dump((bool) "false");   // bool(true)
var_dump((bool) NULL);      // bool(false)

PHP Functions

Return values of some commonly used PHP core functions might break the conditional flow by returning NULL or int() integer values. For example stripos() can return 0 which will be interpreted as false in an IF conditional block.

$text = 'abc xyz';
$pos = strpos($text, 'a');
var_dump($pos);  
//result: int(0)

This code will return true with the output of string position int(0). This return value zero evaluates to the false.

if (!strpos($text, 'a')) {echo "'a' not found in '$text'";}
//result: 'a' not found in 'abc xyz'

Above code will detect “a” but if block will evaluate to false as a’s position is 0.
Instead, explicitly check if the value returned is not FLASE

if(strpos($text, 'a') === FALSE){echo "'a' not found in '$text'";}

I am also on Twitter. You can follow me there as I regularly tweet about my journey.

Keeping code simple with regular expressions

Waqar Ahmed — Fri, 04 Jun 2021 20:31:31 +0000

A regular expression can save multiple conditionals, loops and string functions, making the code simpler. A one-liner regex code looks elegant and much more readable.

I am sharing some examples here. The first three are PHP and Javascript problems and their solution, followed by a RegEx solution.

The other three examples are about employing regex in SQL database, Apache, Nginx web servers and Linux shell.

Table of Content

Time to read an article
Gmail username validation
IP address validation
RegExp in SQL
RegEx in Apache, Nginx webserver
Linux Shell

Example 1:

Time to read an article

According to a study in the Journal of memory and Language( M Brysbaert), we read 238 words per minute. This function will return minutes to read the text input.

function minutesToRead($text){

   $total_words = str_word_count(implode(" ", $text));
   $minutes_to_read = round($total_words / 238);
   return max($minutes_to_read, 1);
}

echo minutesToRead($content) . ' min read'

Instead of breaking down the text into an array of words, we count the spaces \s in the text. We can also use \w+ to count the words.

PHP (regex)

function minutesToRead($text){

   $total_words = preg_match_all('/\s/', $text, $match);
   return max(round($total_words / 238), 1);
}

Javascript (regex)

function minutesToRead(text){

   const word_count = text.match(/\s/g).length;
   return Math.max(Math.round(word_count / 238), 1);
}

PHP preg_match_all matches all occurrences. In Javascript, the group flag \g is used to get all matches.

If the text has HTML tags, use PHP strip_tags to remove these tags in Javascript use one of these regular expressions to strip tags.

/<[\w\s"-.=%#;'“”!?…{}()\d:\/]+>/g
OR
/<[^<]+>/g

Example 2:

Gmail username validation

A username input needs checks for these rules:

begins with an English letter
only contains English letters, digits and dot (.)
minimum 6, maximum 30 characters long

A non-regex solution would need separate code blocks for each rule converting string to an array, using the filter function and several conditionals to implement all validation rules in the code.

For brevity, I will go straight to the solution using regular expression.

PHP

function isValidUsername($username){

    return preg_match("/^[a-z][a-z0-9.]{5,29}$/i", $username) === 1;
}

Javascript

function usernameIsValid(username){

   return /^[a-z][a-z0-9.]{5,29}$/i.test(username);
}

^[a-z] ensures username begins with a letter in the range of a-z.
[a-z0-9.] checks rest of the username only contains alphanumeric values and a dot.
{5,29} validates the length of the string is in the allowed range.
i flag is used for a case-insensitive match.

Example 3:

IP address validation

IPv4 address is a collection of four 8-bit integers (from 0 to the largest 8-bit integer 255) separated by a dot (.).

Examples:

192.168.0.1 is a valid IPv4 address

255.255.255.255 is a valid IPv4 address
257.100.92.101 is not a valid IPv4 address because 257 is too large to be an 8-bit integer
255.100.81.160.172 is not a valid IPv4 address because it contains more than four integers
1..0.1 is not a valid IPv4 address because it's not properly formatted
17.233.00.131 and 17.233.01.131 are not valid IPv4 addresses as both contain leading zeros

Javascript (without regular expressions)

function isIPv4Address(inputString) {

   let ip = inputString.split('.');
   return ip.filter((e)=>{return e.match(/\D/g) || e > 255 || 
   parseInt(e) != e;}).length == 0 && ip.length === 4;
}

PHP filter_var has an IP validator so, we do not need to write regex here.

PHP

filter_var("192.168.00.1", FILTER_VALIDATE_IP, FILTER_FLAG_IPV4);

Javascript (regex)

function isIPv4Address(inputString) {

   const ip = inputString.split('.');
   if(ip.length !== 4) {return false};
   return ip.every(e => /^([1-9]?[0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])$/.test(e));
}

The IP address split on dots into four strings. Regular expression validates each of the string is an 8-bit integer. Unlike non-regex solution, there is no string to int conversion.

[1-9]?[0-9] matches numbers between 0 to 99
1[0-9][0-9] matches numbers between 100 to 199
2[0-4][0-9] matches numbers between 200 - 249
25[0-5] matches number between 250 to 255
| is OR ^,$ marks the beginning and end of the regex

Example 4:

RegExp in SQL

For example, to extract initials from the name column of a table.

MySQL query

SELECT 
    id,
    name,
    REGEXP_REPLACE(name, '(.{1})([a-z]*)(.*)$','$1\.$3') AS REGEXP_name 
FROM students;

result

id  name                REGEXP_name

33  Lesa Barnhouse      L. Barnhouse
38  Kurtis Saulters     K. Saulters
40  Charisse Lake       C. Lake

(.{1}) group 1 matches the first character of the name
([a-z]*) group 2 matches alphabets up till space
(.*) group 3 matches the rest of the name up till the end
$1\.$3 prints value of group1, . and value of group3

Note: MySQL regular expressions support is not extensive, and character class tokens are different: like: [:alpha:] instead of standard \w. More details on MySQL RegExp manual and O'Reilly's cookbook.

Example 5:

RegEx in Apache, Nginx webserver

For example, a blog with URI articles.php?id=123 uses article_id to display the requested articles. Change it to SEO friendly URI like articles/category/title-of-article_123.html in the blog. Virtually all articles now have a separate page with the id and relevant keywords in the name.

The web server can regex match the new SEO URLs for id parameter, pass it to the script and display script output for the URL.

Apache2

RewriteRule "_([0-9]+).html$" "/articles.php?article_id=$1"

Nginx

rewrite "_([0-9]+).html$" "/articles.php?article_id=$1";

Example 6:

Linux shell

Regex can save the hassle of opening a file and searching or scrolling for a directive or setting in it. Instead, use a regular expression to match text pattern in a file and get matching lines straight in the terminal.

To find out the value of the AllowOverride directive in the apache configuration file.

grep -C 2 'AllowOverride' /etc/apache2/apache2.conf

-C 2 flag adds extra lines for context, AllowOverride matches the exact word. Command outputs this

<Directory /var/www/>
    Options Indexes FollowSymLinks
    AllowOverride None
    Require all granted
</Directory>

To find PHP maximum upload file size without opening long configuration file php.ini.

grep 'upload.*size' /usr/local/etc/php/php.ini

outputs upload_max_filesize = 2M

More grep information on gnu grep and manual page.

Conclusion

Learning some basic regex and exploring different use cases can help you build a knowledge of the possibilities regex brings.
Knowing where to use regular expressions in coding and problem-solving can help to write efficient code. Elegant, readable code is a bonus.

I will write a second article about regex basics. If you have any comment or a better regex, please share.

Header photo by Michael Dziedzic