Forem: Matthieu Cneude

Managing Local and Remote Filesystems with Vim and netrw

Matthieu Cneude — Fri, 06 Jan 2023 06:22:00 +0000

When I began to use Vim, everybody was using NERDTree. So I followed: to navigate through my filesystem, NERDTree would be the answer for many years to come.

Nowadays, I mostly use a fuzzy finder to find the files I want, but a file explorer can still be useful in some situations:

When I look at a new codebase. It helps to have a general overview of the filesystem, to understand the structure of the project.
When I need to create, update or delete files and directories.

The more I use Vim, the more I try to get rid of my plugins, to use Vim's native functionalities as much as possible. It's easier than keeping track of 39093209 plugins introducing breaking changes too often. That's why my interest for netrw grew overtime.

What's netrw? It's the file explorer shipped with most Vim packages. It's still a Vim plugin, but if you didn't compile Vim yourself, it's likely that you have it, too.

So we'll dive in netrw in this article. More specifically, we'll see:

Different ways to open netrw: in netrw's window, in a new window, or in a "project drawer".
How to browse through your files and directories.
How to change netrw's display, and filter your filetree.
How to mark files and directories, and perform some actions on them.
How to manage files and directory: create, open, rename, and delete.
How to open files with external programs.
How to bookmark specific directories.
How to explore filetrees on remote servers from your local, comfy Vim.

I assume here that you're somehow proficient with Vim. If you don't understand what's happening in this article, you can look at my series of articles to learn Vim. Also, I'm currently writing a book about Vim!

Are you ready to look at a different way of browsing your precious files? Let's go, then.

Basics

You can do many things with netrw, so let's look first at the very basics. First, for netrw to work, you need to set these options as follows:

:set nocp
:filetype plugin on

If you use Neovim, the options above are already set correctly.

If you want to disable netrw and never speak about it again, you can set these two variables:

:let g:loaded_netrw       = 0
:let g:loaded_netrwPlugin = 0

Useful if you want to use another file explorer, like NERDtree or similar.

Also, if you want to read the help for a specific keystroke inside a netrw buffer, you can run :help netrw-<keystroke>. For example: :help netrw-%. It also works for Ex commands; for example, :help netrw-:Sexplore

I encourage you to read the help if some keystrokes don't work as intended. One of netrw's big strength is its flexibility: you can customize it as you see fit, most of the time.

Vim's Help

:help netrw

Opening netrw

Let's first see how we can open our file explorer.

Opening In the Current Window

To open Vim directly in a netrw buffer, simply give a directory path as argument when you run vim in your terminal. For example:

vim /my/dir/

The last slash / is not mandatory for opening local directories, but it is for remote ones. It's also mandatory when you try to fetch a URL. To avoid confusion, I always add it.

Inside Vim, you can also use the :edit command followed by a directory name. For example, to open the directory of the file open in your current buffer, you can run:

:e .

The netrw buffer would then open in the current window, like any other buffer.

You can also use the command :Explore <dir> to open the directory <dir> in the current window. If you don't give a <dir>, you'll open the current file's directory. This command would also open a split window if these two conditions are met:

If the option "hidden" is not set.
If you modified the current buffer, but you didn't save it.

Opening In a Split Window

If you don't want to open a netrw buffer in your current window, you can also summon it in a split window:

:<size>Sexplore <dir> - Create a horizontal split window above the current window, of size <size>, and open the <dir> in a netrw buffer.
:<size>Hexplore <dir> - Create a horizontal split window below the current window, of size <size>, and open the <dir> in a netrw buffer.
:<size>Vexplore <dir> - Create a vertical split window on the left of the current window, of size <size>, and open the <dir> in a netrw buffer.
:<size>Vexplore! <dir> - Create a vertical split window on the right of the current window, of size <size>, and open the <dir> in a netrw buffer.
:Texplore <dir> - Open a window and a netrw buffer (browsing <dir>) in a new tab.

If you don't give any directory <dir> for the commands above, netrw will open the current file's directory.

As you can see, you can specify the size of the window you want to open. For example, :20Vex will open netrw in a new window on the right, with a size of 20. If you don't specify anything, the window will take half the current window. It's exactly the same behavior as regular Vim splits.

If you want to open your netrw windows with a fixed size, you can set the global option g:netrw_winsize. For example:

let g:netrw_winsize=20

Additionally, after opening a new file buffer using netrw, you can switch back and forth between netrw and the file buffer with the command :Rexplore.

Finally, if you want to open Vim's current working directory (the one displayed when you run :pwd), you can use a simple user command:

command! Oexplore exe 'Explore' getcwd()

If you want to know more about Vim user commands, I covered that in one of my article ;to learn Vim from the ground up.

Vim Help

:help netrw-explore

Opening a Project Drawer

Window splitting in Vim is a great feature. It's useful to copy and paste from one buffer to another, comparing two files, or... opening netrw.

But when you compare the behavior of netrw to other "project drawer" from other editors, it might seem odd that the file selected opens in the file explorer's window. In most editors, this "project drawer" is a special, independent, and smaller window, often opened on the left side. Any file you'd open in there would then pop up in a new window, or in a new tab.

The following commands try to replicate this behavior:

:<size>Lexplore <dir> - Toggle a vertical netrw window of size <size>. Add a bang ! to open it on the right end side.

This command has some peculiar qualities:

The window takes the full height of Vim, even if you have other horizontal split windows open.
When you try to open a file, a buffer will be created in the window you called netrw from, not in the window where the netrw's buffer is.

In short, it behaves more like these "project drawers" I've mentioned above. But even if it looks tempting to replicate what you might already know from other editor, I would recommend you not to use this command.

Vim has a specific window splitting system. The new window opened by :Lexplore is different, and make things a bit more confusing. For example, if you have already multiple split windows, in what window should the file you've selected be displayed? The one you come from? What was it? Or should it be the closest one to the netrw window?

Additionally, by trying to make this window special, it introduces some weird bugs, at least On My Machine™. For example, when moving from the netrw window to another window with another buffer, the netrw buffer would suddenly open there too, inexplicably.

Another nasty side effect: after running :Lexplore, any file you'll then try to open with netrw, whether you used :Lexplore or not, will always open in the last window you came from, not in the netrw window. Internally, :Lexplore set the global option g:netrw_chgwin to the window you come from, silently.

The mighty Drew Neil speaks about all of these inconveniences and more in one of his article, Oil and Vinegar - split window and the project drawer.

If you really want to open your file in another window, you can use one of the following keystroke instead of relying on :Lexplore:

P - Open the file under the cursor in the previously accessed window.
<count>C - Open the file in the window with ID <count>. Open in the current window if no <count> specified.

You can also run a couple of Ex commands:

:NetrwC - Set g:netrw_chgwin to the current window.
:NetrwC win#<number> - Set g:netrw_chgwin to the window number <number>.

The easiest way to open your file in the window you come from is to use P. You can also use 2C if you have only two window open, and you've opened netrw's window last.

Finally, to reset the global variable g:netrw_chgwin, you can run let g:netrw_chgwin=-1.

Vim's Help

:help netrw-P
:help netrw-C

Navigating in Vim

Let's see now how we can effectively browse in our filesystem with netrw.

Basics

Here are the basic keystrokes we can use to browse our filesystem:

ENTER - Open a buffer with the file (or directory) under the cursor in netrw's window.
o - Open the file (or directory) under the cursor in a new horizontally split window.
v - Open the file (or directory) under the cursor in a new vertically split window.
t - Open the file (or directory) under the cursor in a new tab.
- - Go up one directory level.
P - Open the file in the previously focused window.
p - Open the file in the preview window.

Managing the Display

Here are some useful NORMAL mode keystrokes to manage the display of your files and directories. You can make some of them global by setting the corresponding global variable.

CTRL-l - Refresh the directory listing.
i - Cycle through different listing display styles: "thin", "long", "wide", and "tree".
I - Toggle the banner on top.
s - Switch sorting between name, time, and size.
r - Sort in reverse order.
qf - Display file's size and timestamp of last modification.

Filtering the Display

What about keeping only what we need in our netrw window?

gh - Toggle showing or hiding dotfiles.
a - Cycle through three hiding modes determined by a regex.

If you want something more elaborate than gh, the keystroke a is here for you. It can hide or show files and directories using a regex; this regex is the value of the global variable g:netrw_list_hide.

Hitting a will cycle through three different display:

Display everything.
Hide the files matching the regex.
Show only the files matching the regex.

For example, to hide all JPG and PDF images, you can run:

:let g:netrw_list_hide="\.pdf$,\.jpg$"

It's also possible to hide every file specified in the .gitignore file, by adding the return of the function netrw_gitignore#Hide() to the hide list.

For example, to hide all PDF files, MD files, and files specified in .gitignore:

:let g:netrw_list_hide="\.pdf$,\.jpg$,".netrw_gitignore#Hide()

Last important thing: all marked files are considered part of the hide list. We'll speak about marked files just below in this article.

Listing the Browsing History

The browsing history is updated when you... browse in a netrw buffer. What a surprise!

You can move through the history of directories visited using these keystrokes:

u - Go to the preview directory.
U - Go to the next directory.

You can control the count of history's entries by setting the global variable g:netrw_dirhistmax. The default is 10.

By default, netrw will write its history in the file .netrwhist, in the first directory of the "runtimepath". If you want to change this directory, you can set a value to the variable g:netrw_home.

Marking Files And Directories

We saw that marked files are automatically part of the hide list when hitting the keystroke a in a netrw window. But what are these marked files?

They're simply files you've marked using specific marking keystrokes. Many netrw keystrokes and commands will act on the files marked.

These marked files end up in two different lists:

In a global marked file list. There's only one marked file list available.
In a local buffer marked file list. Each directory has one, so each directory has its own list of marked files.

So, how to mark (or un-mark) files? Here are some keystrokes to do so manually:

mf - Toggle the marking of the file or directory under the cursor.
mF - Un-mark all the marked files in the current directory.
mu - Un-mark all the marked files in every directory.
mr - Open a prompt inviting you to enter a regex, and mark all the files matching it.

You can also use the following command:

:MF <pattern> - Mark a list of files depending on the pattern <pattern>.

Notice that these regexes accept shell-style globs.

You can also mark the files which are already in another Vim list:

qL - Mark all the files part of the location list.
qF - Mark all the files part of the quickfix list.
mA - Mark all the files part of the argument list.
ma - Move marked files to the argument list.
cb - Move marked files to the buffer list.
cB - Mark all the files part of the buffer list.

Weirdly enough, there's no keystroke available to display the list of all marked files. But you can run the following command to do so:

:echo netrw#Expose("netrwmarkfilelist")

Even better: you can create a user command you can then map to a keystroke, if you want to. If you wonder what's a user command, I've covered that in this article.

Finally, here are some interesting keystrokes to act directly on the marked files:

mv - Prompt the user to enter a Vimscript command operating on all marked files, one at a time. Use % as filename placeholder.
mx - Prompt the user to enter a shell command operating on all marked files, one at a time. Use % as filename placeholder.
mX - Apply a shell command on every file at once.
mz - Compress (or uncompress) marked files.
mg - Apply Vimgrep to the marked files.
ms - Source all marked files.

We'll see many more ways to operate on marked files below.

Managing Files and Directories

We have now multiple options to open and display our files and directories using netrw. Next, let's see how to manage our filetree by adding or deleting elements.

Creating and Deleting

Here are the keystrokes you can use in a netrw window to add or delete files or directories. Under the hood, netrw will run specific shell commands to operate on the files of your liking:

d - Create a directory.
% - Create a new file (when the buffer created is saved).
D - Delete the file (or empty directory) under the cursor.

By default, you won't be able to remove non-empty directory. You can however decide to always delete directories recursively, by running:

let g:netrw_localrmdir='rm -r'

The keystroke D will also delete all marked files.

Renaming Files or Directory

What about renaming what we already have?

R - Rename (or move) files and directories. It's also possible to modify its path to move it.

This keystroke will also try to rename all marked files.

There is no problem to use this keystroke locally, but there's a big caveat when used on remote files. In that case, it will copy the file first, and then delete it. The two operations are independent; if the copy fail and the delete succeed, you'll lose the file.

We'll see the remote possibilities of netrw below.

Copying Files

Copying files or directories require three steps:

Select the target directory (where the files need to be copied) with the keystroke mt. If the banner is visible, the target path will be displayed there.
Mark the files you want to copy with mf.
Hit mc to effectively copy the files.

By default, netrw use the command cp under the hood. If it doesn't work, a message will point out what variable you need to set to use a different the command.

Moving Files

In netrw, moving files or copying them is a similar operation.

Again, a word of caution: if you move remote files, they'll be first copied and then deleted. You'll lose your files if the copy operation fails and the delete operation succeed.

Here are the three steps to move our files:

Select the target directory (where the files need to be copied) with the keystroke mt. If the banner is visible, the target path will be displayed there.
Mark the files you want to copy with mf.
Hit mm to move the files.

By default, netrw use the command cp under the hood. If it doesn't work, a message will point out what variable you need to set to change this command.

File Permissions

Using netrw, you can also change the file permissions using the keystroke gp. It uses the CLI "chmod" under the hood; if you want to change it, you need to give a value to the variable g:netrw_chgperm.

Opening Files with External Applications

If you want to open a specific file using an external application, here are some keystrokes to do so:

x - Open the file under the cursor with the user's preferred application.
X - Prompt for arguments, then use your shell to call the executable file under the cursor with the arguments.
gx - Can be used in any type of buffer (netrw and file buffers) to open the file (or URL) under the cursor with the user's preferred application.

The keystroke gx is handy to open a URL from your buffer while editing. I use it all the time!

By default, netrw uses an application which can open user's preferred applications, like xdg-open for example. You can change it by setting a value to g:netrw_browsex_viewer. For example:

:let g:netrw_browsex_viewer="xdg-open"`

If you set the value of this global variable to - (let g:netrw_browsex_viewer='-'), then netrw will try to run a function depending on the extension of the file. For example, if you try to open an HTML file, the function NFH_html() will be invoked, with the filepath as argument. You can then write some code in the body of the function to specify how to open the file.

You can find many examples in this file: $VIMRUNTIME/autoload/netrwFileHandlers.vim. If you don't know where your Vim runtime is (or what's the Vim runtimepath), you can look at this article.

Additionally, you can set different global variables to customize this functionality:

g:netrw_gx - Control how gx picks up the text under the cursor. The default is <cfile>.
g:netrw_nogx - Disable the gx keystroke for non-netrw buffers.
g:netrw_suppress_gx_mesg - Display messages from the browser open in Vim. Default is 0 (disabled).

Bookmarking Files and Directories

We also have the possibility to bookmark our favorite directories in netrw. Here are the most useful keystrokes to do so:

mb - Bookmark the currently browsed directory.
mB - Delete the last bookmark.
qb - List all bookmarks
<count>gb - Go back to the bookmark number <count>.

All the bookmarks are written in the file .netrwbook. This file is located by default in the first directory of the runtimepath. Similarly to the history, you can also set the variable g:netrw_home to change the directory of .netrwbook.

You can also use the following Ex commands to manage your bookmarks:

:NetrwMB - Bookmark file or directory under the cursor.
:NetrwMB <files_or_dirs> - Bookmark files and directories <files_or_dirs>

If you run :NetrwMB with marked files, they'll also be all bookmarked.

Remote Operations and Protocols

We've seen in details how to manage our files and directories using netrw on a local computer. We can also use it to explore and manage the file systems on remote computers, via SCP or FTP!

There are two categories of operations we can do on these remote files and directories:

Read and write.
Only read.

Keep in mind that writing on a remote system is potentially destructive.

Under the hood, netrw use the file netrw.vim in $VIMRUNTIME/autoload/netrw.vim. It uses autocommands to intercept read and write for URL-like filenames.

This file netrw.vim is mainly a wrapper sending commands to external program, like ftp or scp. You can also read using other protocols (like HTTP), but writing and browsing a remote filesystem is always done via FTP or SCP.

Our netrw will read and write from temporary files copied from the remote system on the local computer. For Unix-like system (including Linux), these files will end up in the /tmp directory.

We can use all the commands we've seen until now on remote files, too. There are two important differences, however:

As I mentioned already, if you want to move remote files or directories, you'll need to do a copy first and then a delete. If the copy fails and the delete succeed, the file or directory will be deleted from the system. Copying manually the file first, and then deleting the files in a second step is safer.
If you want to overwrite the command used by the keystroke d (to create a directory), you need to assign a value to g:netrw_mkdir_cmd, not g:netrw_localmkdir. The same logic apply for other operations; for example, to remove a directory: g:netrw_rmdir_cmd for a remote system, g:netrw_localrmdir for a local one.

To open a remote directory, you can use the following command:

vim <protocol>://<user>@<hostname>/<path>/

As we said earlier, the <protocol> can be "scp", "ftp", "http"...

As often, depending on the protocol you're using, netrw will internally call a shell command to access the directory.

Vim's Help

:help netrw-protocol

Using SCP

What's SCP? It's a protocol to exchange files between clients and servers using SSH. As a result, the exchanges are authenticated and encrypted.

Using the pattern we've seen just above, if I want to open my home directory on my remote server from my local computer, I can simply run the following:

vim scp://hypnos@185.193.160.40/

A netrw window will then open, browsing the home directory of the user hypnos on my server with IP 185.183.160.40. Even better: if you use OpenSSH, you can write this kind of configuration in your $HOME/.ssh/config file:

Host server
     HostName 185.193.160.40
     User hypnos

If you don't want to enter your password 3892 times while browsing, you can log to your remote server using one of your local SSH key. If you're on a Unix/Linux machine, look at man ssh-copy-id for more details.

We can also directly open a file with, for example, the command vim scp://server/myfile.

Speaking of files, we can also use the usual Ex commands inside Vim to edit, read or write them. For example:

:e scp://user@ip/myfile - Edit the file myfile in the current window.
:r scp://user@ip/myfile - Insert the content of myfile in the current buffer.
:w scp://user@ip/myfile - Write the content of the current buffer into myfile. Useful to copy a file from your local to your remote server!

Vim's Help

:help netrw-externapp
:help netrw-transparent
:help netrw-ex
:netrw-ssh-hack

Using FTP

You can also open files using the FTP protocol. Since this protocol is not secure at all (a third party can intercept the data, including username and password), I'd recommend you to use the SFTP protocol (for SSH FTP) instead.

On the surface it works similarly than the SCP protocol. Just call a URL like vim sftp://user@ip/myfile for example, and you're good to go.

There are two potential drawbacks, however:

It's significantly slower than SCP.
Some FTP implementations are noisy and add some junk to the filenames. You'll need to write a function NetReadFixup to clean them up automatically.

The benefit? You can do many more operations on files using SFTP compared to SCP.

You can also write a .netrc file (probably located in your home directory) to specify your favorite way to connect to a remote via FTP.

Obtaining a File

While browsing, you can use the O keystroke to fetch the remote file under the cursor. Vim won't open it in a new buffer. If some files are marked, they'll all be fetched.

These files are copied to the Vim's working directory. To display the path of this working directory, you can run :pwd.

Reading via HTTP

Using netrw, we can also fetch some content from HTTP. For example, try to run the following in Vim:

:e http://www.google.com/`

Congratulation! You're reading Google's HTML.

If you try to write the file, netrw will try to upload it back to the remote URL. On my machine, it runs the command curl -T http://www.google.com, but it might be different on yours.

You can also use :r <URL>/ to insert in the current buffer, instead of opening a new one

Customizing netrw's Mapping

The easiest way to customize netrw keystrokes is to declare your new mapping in the folder ftplugin. Again, if you wonder what it is, I've written about the Vim's runtimepath here.

For example, if you want to toggle the netrw banner by hitting B instead of I, you can add the following to the file $HOME/.vim/ftplugin/netrw.vim (or $XDG_CONFIG_HOME/nvim/ftplugin/netrw.vim for Neovim):

map <buffer> <silent> B I

It works because all netrw buffers have the filetype "netrw".

Master of Browsing

There are many other ways to customize netrw, but I think I've covered enough here to answer most needs.

What did we see in this article?

We can open the help for specific netrw <keystroke> by running the Ex command :help netrw-<keystroke>. It also works for Ex commands.
There is a command to open Netrw wherever you want, depending of the current window: on the side or on top of it.
You can go back and forth in netrw's browsing history using u and U.
It's possible to mark files and directories. The usual keystrokes will then act on these files.
Using netrw, we can create, delete, rename, or move files and directories.
It's also possible to open files using external applications or commands.
Bookmarking directories can be a good way to open quickly whatever you need.
One of the most powerful feature of netrw allows us to open remote filesystems using SCP or FTP.

The native plugin netrw might not be the most stable or the most complete file manager out there, but, to me, it's enough for the daily operations we all do on our precious files.

References

Learning to Play Vim: A Fun Guide to Learn the Best Editor

I began to write a very ambitious guide to learn Vim from the ground up. Thanks to great feedback from my readers, I'll be able to address the problems many beginners complain about when learning Vim. For example:

How to navigate in multiple files and projects in Vim?
How to debug in Vim?
How to search, find, and replace?

This guide will explain the most useful vanilla functionalities as well as some powerful plugins which will enrich your experience.

Help me make an impact in the Vim world! You can subscribe to the newsletter and tell me everything you want to see in the book. Early bird discount guarantees!

I reply to every email personally, so don't hesitate to ask as many questions as you want. It's always a pleasure to help.

Last but not least: I've also written a book about building your own Mouseless Development Environment, so if you're interested by that too, click on this shiny link.

How to Create Vim Text-Objects in Lua

Matthieu Cneude — Sat, 23 Jul 2022 10:48:43 +0000

"You don't get it. Vim is like a language! You'll speak Vim when you write! When you go to the market! You'll speak Vim with your cat! When you think, it will be the Word of Vim™ in your head! You'll see! It will change your life!"

This is the kind of argument any Vim hippy would sing to the poor heretics, trying to convert them to The Eternal Editor. A hippy like me, who's now writing an article about one of the main component of this "language", the text-object. Repeat after me: glory to the text-object!

It's powerful to use them, that's true; but we can do better. We could create new text-objects, like a god creating life! We could shape the Chaos to bring order in the galaxy!

The Greek god Hephaestus created the fire in his legendary forge. Like him, we also need some tools to create our own text-objects. What on Earth can we use? Vimscript?

With Neovim, we can use Lua to customize our favorite editor; we can also compile Vim with Lua support. To me, one of the best way to learn how to customize Vim is to take already written functions in Vimscript and transform them in Lua.

The benefits:

If you don't know Vimscript, analyzing the code to transform it in Lua will show you useful functions you can use for more customization. It's essential for harnessing Vim's power.
If you don't know Lua, you can learn it this way, too. It's a great programming language, used by many other tools, like the fantastic Pandoc for example.
You'll get a deeper understanding how Vim works, allowing your to improve your workflow.

So let's taste the power of Vimscript functions combined with Lua constructs, by implementing useful text-objects. More specificaly, we'll see in this article:

The general principles governing text-objects.
How to create a text-object representing a line.
How to create text-objects delimited by a pair of characters, like two commas.
How to create a very useful - and quite universal - text-object matching some level of indentation.

The text-objects we'll create are inspired by other authors from The Great Internet™. You'll find the sources of this inspiration at the end of the article.

I assume here that you're somehow proficient with Vim. If you don't understand what's happening in this article, you can look at my series of articles to learn Vim.

If it's the first time you write some Lua, I would encourage you to customize things further by modifying the examples we'll see in this article. To use the correct syntax, you should keep a quick reference on side, like this one.

Are you ready to dive into Vimscript, Lua, and text-objects? I'm sure you are! Let's go!

What's a Text-Object?

Before jumping into Vim to create our new delightful text-objects, let's first do what any developer should do when she needs to solve a problem: thinking. Let's decompose what a text-object is:

Text-objects are NORMAL mode keystrokes using two keys; the first one can only be an "a" (for "around") or an "i" (for "inside").
A text-object represents a specific string of characters, with a beginning and an end.
It can only be used after an operator, or after switching to VISUAL mode character-wise (using "v" in NORMAL mode).
The operator will act on the text-object, VISUAL mode will select the text-object itself.
To apply an operator on some text-objects, the cursor doesn't have to be on the text-object itself. In that case, the operator will act on the next text-object of the line.

If you never heard of the rule 5, it can change your life, and bring you glory and fortune in a couple of days. Nothing less! If you type di( in NORMAL mode, and your cursor is before a pair of parentheses, you'll move inside them. The operator d will then delete everything inside. Neat!

As I was saying just above, a text-object can begin with an "a" or an "i". I understand text-objects beginning with "a" as "around", even if Vim's help will refer to them as the indefinite article a, like delete a word. I prefer "around" because this kind of text-objects often include some characters around the "inside" text-object. It's a good mnemonic to remember the difference between the two.

Motions are looking a lot like text-objects, but they're not the same. Using an operator before a motion will perform an action from the cursor position to the end result of the motion. A text-object doesn't necessarily start at the cursor position; that's the biggest difference.

For example, daw will delete around a word, whatever the letter of the word your cursor is on. Using a motion, dw will delete from the cursor position to the next word.

Now that we explored the problem space, let's enter the solution space. To create a text-object with a start and an end, we could simply:

Select the characters we want.
Apply whatever operator on the selection.

That's great, because Vim has some mapping allowing us to do that easily.

Vim Help

:help text-object
:help motion

The Operator-Pending Mapping

Basics

When we hit an operator in NORMAL mode (like d, y, or c, for example), we switch silently to the OPERATOR-PENDING mode. It's in this mode that we'll input our motion (or text-object) we want to operate on. Then, we switch back to NORMAL mode, automagically.

To define new text-objects (or motions) which can be used in OPERATOR-PENDING mode, you can use an operator-pending mapping: omap or, if you don't want a recursive mapping, onoremap. Thanks to them, we only need to worry about selecting the characters included in our text-objects; we don't care about the operators themselves.

The Line: Our New Text-Object

Let's now create the first text-object of this article: a line.

The text-object al ("around the line) will delete everything, including the possible indentations at the beginning.
The text-object il (inside the line) won't delete the indentation.

Here's a first mapping for il:

onoremap <silent> il :<c-u>normal! $v^<cr>

In case you're not sure what that means, here are some explanations:

<silent> - Don't echo the Ex-command executed in the command-line window.
<c-u> - Delete everything already written in COMMAND-LINE mode (like the selection markers '<,'> for example).
normal! - This Ex-command allow executing NORMAL mode keystrokes.
$v^ - Go to the end of the line, switch to VISUAL mode, and select till the beginning of the line (without the eventual white spaces).

For example, if you hit yil or dil, your operators yank and delete will operate on our new text-object. If we only want to select it, we need another mapping for VISUAL mode:

xnoremap <silent> il :<c-u>normal! $v^<cr>

You can now hit vil and admire the potential of your infinite skills.

We have now the "inside" version of our text-object; what about the "around" one?

xnoremap <silent> al :<c-u>normal! $v0<cr>
onoremap <silent> al :<c-u>normal! $v0<cr>

These mappings follow the same principles, except that we select everything this time, including the eventual whitespaces at the beginning of the line. We could also use V instead of $v0.

Vim Help

:help omap-info

The Missing Text-Objects

Our new text-objects are great, but we can do better. This time, we'll create a whole series, delimited by different pair of characters.

Between Two Characters

It's easy to delete a pair of parentheses: a(, a), or even ab are there for you. As always, there are some "inside" variants, too. But what about deleting everything between two dots? Between two hyphens? Between two commas?

What about creating text-objects delimited by these characters: ,,.,;,:,+,-,=,~,_,*,#,/,|,\,&,$,??

To solve this problem, we could look at a simple example:

Hello, how are you, you?

If we want to change the characters between the two commas, we could:

Move the cursor to the first comma.
Select everything till the second comma.
Whether including the commas in the selection or not will be the difference between the "around" and the "inside" text-object.

This scenario requires our cursor to be between the two commas, at least for now. We'll improve this soon.

To move our cursor to the first or second comma, we can use f, F, t, and T in NORMAL mode. Here's a possible solution:

xnoremap <silent> i, :<c-u>normal! T,vt,<cr>
onoremap <silent> i, :<c-u>normal! T,vt,<cr>
xnoremap <silent> a, :<c-u>normal! F,ft,<cr>
onoremap <silent> a, :<c-u>normal! F,ft,<cr>

We could repeat these mappings for every character we want to include. But it would be quite difficult to read and maintain. Taking some inspiration from a Zsh snippet I've already covered in this article, we could:

Loop through all the different characters.
Loop through all the different modes we want to map (OPERATOR-PENDING mode and VISUAL mode).
Create the different mappings for each iteration.

Here's a possible implementation in Vimscript:

let s:chars = [ '_', '.', ':', ',', ';', '<bar>', '/', '<bslash>', '*', '+', '%', '`', '?' ]
for char in s:chars
    for mode in [ 'xnoremap', 'onoremap' ]
        execute printf('%s <silent> i%s :<C-u>normal! T%svt%s<CR>', mode, char, char, char)
        execute printf('%s <silent> a%s :<C-u>normal! F%svf%s<CR>', mode, char, char, char)
    endfor
endfor

I like to use Vimscript for simple configurations, but as soon as I need to reach for loops or conditionals (I hate the equality-operators-mess in Vimscript), I switch to Lua. If you use Neovim, or if you have Vim compiled with Lua, you can use the following:

function basic_text_objects()
    local chars = { '_', '.', ':', ',', ';', '|', '/', '\\', '*', '+', '%', '`', '?' }
    for _,char in ipairs(chars) do
        for _,mode in ipairs({ 'x', 'o' }) do
            vim.api.nvim_set_keymap(mode, "i" .. char, string.format(':<C-u>normal! T%svt%s<CR>', char, char), { noremap = true, silent = true })
            vim.api.nvim_set_keymap(mode, "a" .. char, string.format(':<C-u>normal! F%svf%s<CR>', char, char), { noremap = true, silent = true })
        end
    end
end

return {
    basic_text_objects = basic_text_objects
}

I would advise you to keep your Lua scrips in a lua/<namespace> folder; <namespace> could be your username, or anything else you'd like. For example, my scripts are in $XDG_CONFIG_HOME/nvim/lua/hypnos.

Then, you can call directly your new functions in your vimrc. If it's a Vimscript file, you'll need the prefix lua as follows:

lua require('<namespace>/text_objects').basic_text_objects()

To The Next Text-Object

Right now, we need our cursor between the two characters to act on our new text-objects. It would also be nice to be able to operate on the first pair of these characters when our cursor is before them. As we saw above in this article, it's already what we can do with parenthesis or quotes.

Let's consider the following example:

She felt, suddenly, deeply in love with Vim.

We could try to end up on the first comma of the pair whether the cursor is before them or between them. From there, we would select everything till the second comma. Here's a possible mapping:

onoremap <silent> i, :<C-u>silent! normal! f,F,lvt,<cr>

Let's imagine that the cursor is on the first d of suddenly, between the commas.

f, - Move to the second comma.
F, - Move to the first comma.
l - Move one character to the right (on the space after the comma).
vt, - Visually select everything till the next comma.

Now, let's imagine that our cursor is on the S of She, before the commas.

f, - Move to the first comma.
F, - There is no comma before the cursor to be found, so the Ex-command normal! fails.
l - Move one character to the right (on the space after the comma).
vt, - Visually select everything till the next comma

Note that the Ex-command normal! fails at the second step. By default, it would stop there, not executing the steps 3 and 4. That's why we add the Ex-command :silent! here; it will force normal! to execute till the end.

For the "around" text-object, we can follow the same principles. The only difference: we also select the two commas.

onoremap a, :<C-u>silent! normal! f,F,vf,<cr>

Here's the final result, in Lua:

function basic_text_objects()
    local chars = { '_', '.', ':', ',', ';', '|', '/', '\\', '*', '+', '%', '`', '?' }
    for idx,char in ipairs(chars) do
        for idx,mode in ipairs({ 'x', 'o' }) do
            vim.api.nvim_set_keymap(mode, "i" .. char, string.format(':<C-u>silent! normal! f%sF%slvt%s<CR>', char, char), { noremap = true, silent = true })
            vim.api.nvim_set_keymap(mode, "a" .. char, string.format(':<C-u>silent! normal! f%sF%svf%s<CR>', char, char), { noremap = true, silent = true })
        end
    end
end

Keep in mind that it's a naive approach. First, making the normal! command fails feels a bit like a hack. Additionally, it's quite difficult to understand. That said, sometimes a hack is good enough to answer our needs and move forward. I wouldn't use that in a plugin which could be used by others, but for my own use it's good enough.

Each time we've created our text-objects, we first stated the problem, and then tried to find a way toward a solution. That's a good iterative approach, but it also means that our text-objects might not cover all the possible scenario. We can still improve things afterward, if we see that our text-objects don't behave as expected in specific situations.

This is different from installing a plugin: you often have no idea how it works, and you can't modify them iteratively to answer your specific needs.

Text-Objects Based On Indentations

Let's create one more text-object, more complex this time. It can be useful for any developer.

The Basics

If you look at the text-objects already available in vanilla Vim, their boundaries are based on specific characters. There are other important elements you'll find in most codebases which can be used for a text-object: indentations.

Here's how to select our new text-objects:

Get the starting indentation of the current line. This will be the indentation of reference we'll compare every other line to.
Move up line by line, as long as the indentation is equal or higher to the starting indentation.
Stop when the next line doesn't fulfill the rule 2 anymore, and begin VISUAL mode line-wise.
Move down till the indentation is equal or higher to the starting indentation.

It means that our text-objects include all the lines having the same indentation as the one you start with, or higher. Since the behavior is more complex than other text-objects we've created, let's implement a function we'll call in our mappings:

function! IndentTextObject()
  let startindent = indent(line('.'))

  " Move up till we are at the top of the buffer
  " or till the indentation is less than the starting one
  let prevline = line('.') - 1
  while prevline > 0 && indent(prevline) >= startindent
    -
    let prevline = line('.') - 1
  endwhile

  " Begin linewise-visual selection
  normal! 0V

  " Move down till we are at the bottom of the buffer
  " or till the indentation is less than the starting one
  let nextline = line('.') + 1
  let lastline = line('$')
  while nextline <= lastline && indent(nextline) >= startindent
    +
    let nextline = line('.') + 1
  endwhile
endfunction

onoremap <silent>ai :<C-U>call IndentTextObject()<CR>
onoremap <silent>ii :<C-U>call IndentTextObject()<CR>
xnoremap <silent>ai :<C-U>call IndentTextObject()<CR>
xnoremap <silent>ii :<C-U>call IndentTextObject()<CR>

Again, I always prefer using Lua when things get slightly complex:

function select_indent()
    local start_indent = vim.fn.indent(vim.fn.line('.'))
    local prev_line = vim.fn.line('.') - 1

    while prev_line > 0 and vim.fn.indent(prev_line) >= start_indent do
        vim.cmd('-')
        prev_line = vim.fn.line('.') - 1
    end

    vim.cmd('normal! 0V')

    local next_line = vim.fn.line('.') + 1
    local last_line = vim.fn.line('$')
    while next_line <= last_line and vim.fn.indent(next_line) >= start_indent do
        vim.cmd('+')
        next_line = vim.fn.line('.') + 1
    end
end

function indent_text_objects()
    for _,mode in ipairs({ 'x', 'o' }) do
        vim.api.nvim_set_keymap(mode, 'ii', ':<c-u>lua select_indent()<cr>', { noremap = true, silent = true })
        vim.api.nvim_set_keymap(mode, 'ai', ':<c-u>lua select_indent()<cr>', { noremap = true, silent = true })
    end
end

return {
    indent_text_objects = indent_text_objects,
}

The core of the functions are the two while loops. They will:

Move the cursor up till the previous line has less indentation than the starting one.
Begin VISUAL mode line-wise (with normal! 0V) and go down line by line, selecting the ones at the same level of indentation or higher.

Note that we use the Ex-command - and + to move up and down. Yes, they're Ex-command: try :+ for example. You could also use normal! k or normal! <up>.

Improving the Implementation

Our little function works great, but it shows quickly its limits. Perhaps the most obvious: the "inside" and "around" text-objects have the exact same behavior. Horror and damnation!

Let's say that the "around" the text-object should select two more lines, the first lines having fewer indentations when we move up and down.

To illustrate, consider the following code. The symbol "┃" represents the cursor:

-- Comment
function super_function()
 ┃  if true then
       print('youpi')
       print('wuhu')
    end
end
-- Comment

Hitting dii would delete the whole if block but let everything else untouched. Hitting dai would delete everything, except the two comments.

From there, we have two solutions:

Create two different functions: one for the "around" text-object, one for the "inside".
Pass a boolean flag to use the "around" or "inside" text-object in the same function.

If we go with the first solution, we would have almost identical functions; it's likely we would need to change both each time we want to improve our text-objects. So let's go with the second solution. That said, if there are too many conditionals creeping in because there are more and more differences between the two text-objects, I would definitely split the function.

Here's a possible implementation:

function select_indent(around)
    local start_indent = vim.fn.indent(vim.fn.line('.'))
    local prev_line = vim.fn.line('.') - 1

    while prev_line > 0 and vim.fn.indent(prev_line) >= start_indent do
        vim.cmd('-')
        prev_line = vim.fn.line('.') - 1
    end
    if around then
        vim.cmd('-')
    end

    vim.cmd('normal! 0V')

    local next_line = vim.fn.line('.') + 1
    local last_line = vim.fn.line('$')
    while next_line <= last_line and vim.fn.indent(next_line) >= start_indent do
        vim.cmd('+')
        next_line = vim.fn.line('.') + 1
    end
    if around then
        vim.cmd('+')
    end
end

function indent_text_objects()
    for _,mode in ipairs({ 'x', 'o' }) do
        vim.api.nvim_set_keymap(mode, 'ii', ':<c-u>lua select_indent()<cr>', { noremap = true, silent = true })
        vim.api.nvim_set_keymap(mode, 'ai', ':<c-u>lua select_indent(true)<cr>', { noremap = true, silent = true })
    end
end

return {
    indent_text_objects = indent_text_objects,
}

Next, we could ignore the blank lines. Right now, we don't really care about them; but they will inevitably stop our moves up and down because their level of indentation will always be less than the starting one. That is, if your cursor wasn't on a line without indentation from the start, in which case you would select the entire buffer.

We can define a blank line as a line only containing white space (spaces, tabs, and newlines). To find out if the previous or next line is indeed blank, we could use a regex:

local blank_line_pattern = '^%s*$'

Regexes in Lua are a bit odd: where you would have \s for representing white spaces in many other regex engines, you have %s in Lua.

We now need to verify at each iteration if the next line is blank. We could create a small function for that:

local prev_blank_line = function(line) return string.match(vim.fn.getline(line), blank_line_pattern) end

We also need to modify the conditions for our two while loops. For example:

while prev_line > 0 and (prev_blank_line(prev_line) or vim.fn.indent(prev_line) >= start_indent) do

A last detail which might suits you. When your cursor is on a blank line while summoning your text-object with your agile fingers, you could ignore the keystroke. If you find the idea appealing, you could add the following before the first while loop:

if string.match(vim.fn.getline('.'), blank_line_pattern) then
    return
end

We've now patched the most obvious defects of our text-object. But we can do better! What about being able to give a count to select lower levels of indentation?

For example, dai would work as it does now, but d2ai would select a lower level of indentation, and d3ai would select even more.

To do so, we simply need to increase our starting indentation depending on the count given. Right now, we only know what's the level of indentation of the current line with our variable start_indent, but we don't know the amount of indentations for each level.

To get this information, we can look at the value of the option shiftwidth.

Here's the implementation:

local start_indent = vim.fn.indent(vim.fn.line('.'))
if vim.v.count > 0 then
    start_indent = start_indent - vim.o.shiftwidth * (vim.v.count - 1)
    if start_indent < 0 then
        start_indent = 0
    end
end

You might wonder about the calculation:

start_indent = start_indent - vim.o.shiftwidth * (vim.v.count - 1)

First, vim.v.count is the count given to the text-object. In Vimscript, it would be v:count.

Regarding the calculation itself, we need to multiply the count given by the amount of indentations per level (the shiftwidth option), and subtract it to the indentations of the current line. It gives us the amount of indentations one level above.

We decide that a count of 1 shouldn't change anything (dai is the same as d1ai), that's why we subtract 1 from the count. We also verify if our starting indentation is lower than 0, which could happen if we enter a count on a line without any indentation. In that case, We force the starting indentation to 0.

Here's the final function:

function select_indent(around)
    local start_indent = vim.fn.indent(vim.fn.line('.'))
    local blank_line_pattern = '^%s*$'

    if string.match(vim.fn.getline('.'), blank_line_pattern) then
        return
    end

    if vim.v.count > 0 then
        start_indent = start_indent - vim.o.shiftwidth * (vim.v.count - 1)
        if start_indent < 0 then
            start_indent = 0
        end
    end

    local prev_line = vim.fn.line('.') - 1
    local prev_blank_line = function(line) return string.match(vim.fn.getline(line), blank_line_pattern) end
    while prev_line > 0 and (prev_blank_line(prev_line) or vim.fn.indent(prev_line) >= start_indent) do
        vim.cmd('-')
        prev_line = vim.fn.line('.') - 1
    end
    if around then
        vim.cmd('-')
    end

    vim.cmd('normal! 0V')

    local next_line = vim.fn.line('.') + 1
    local next_blank_line = function(line) return string.match(vim.fn.getline(line), blank_line_pattern) end
    local last_line = vim.fn.line('$')
    while next_line <= last_line and (next_blank_line(next_line) or vim.fn.indent(next_line) >= start_indent) do
        vim.cmd('+')
        next_line = vim.fn.line('.') + 1
    end
    if around then
        vim.cmd('+')
    end
end

That's it! We've created a wonderful text-object with the power of our linked spirits. It's with joy and proud that I dub you Godly Blacksmith of the Text-Object©.

Mapping a Lua Function

Till now, we've used some Vimscript strings to map our Lua functions to our new text-objects. Neovim 0.7, which came out a couple of days before this article was published, allows us to directly map Lua functions without using any Vimscript.

You can try to look at :help vim.keymap.set for example to improve our code even further.

The Power of the Text-Object

As always, coming up with new ideas of text-objects depend on your needs. I always prefer solving the pain points I notice when writing in Vim, instead of trying to come up with ideas of improvements out of nothing. For example, I always wanted to have a text-object to manipulate functions; at the same time, I wanted something which could work with most programming languages. Having a text-object based on indentation levels is the best compromise to me.

So, what did we see in this article?

When you hit an operator in NORMAL mode, you switch to OPERATOR-PENDING mode. From there, you can give a motion or a text-object to operate on what your want.
A text-object is only a set of characters which can be acted upon, using operators.
It's easy to build basic text-objects thanks to the operator-pending mappings, omap and onoremap.
It's easier to write a function when you need more complex text-objects. You can then rely on the rich set of Vimscript functions to move your cursor and select what you want.
Vimscript is a language with many weird design decisions. I prefer using Lua as soon as my data flow goes on multiple branches.

Do you want more articles where we use Vimscript and Lua, to improve your customization power in Vim? Don't hesitate to give your thoughts, feedback, and improvements in the comment section. Like, share, and love.

References

Vim wiki - Indent text-object
Vim wiki - Creating new text-objects
Transactions Pending - Dylan McClure
Getting started using Lua in Neovim - Timothée Sterle
Learn X in Y minutes - Lua - Tyler Neylon

Learning to Play Vim: A Fun Guide to Learn the Best Editor

How to navigate in multiple files and projects in Vim?
How to debug in Vim?
How to search, find, and replace?

This guide will explain the most useful vanilla functionalities as well as some powerful plugins which will enrich your experience.

Help me make an impact in the Vim world! You can subscribe to the newsletter and tell me everything you want to see in the book. Early bird discount guarantees!

I reply to every email personally, so don't hesitate to ask as many questions as you want. It's always a pleasure to help.

Last but not least: I've also written a book about building your own Mouseless Development Environment, so if you're interested by that too, click on this shiny link.

A Guide to the Zsh Line Editor with Examples

Matthieu Cneude — Sun, 22 May 2022 13:04:35 +0000

Like every morning, you switch on your computer, launch your terminal, and begin to type weakly the first commands of the day. With a sigh of despair, you launch the 12938 docker containers of your 29374 coupled microservices with a simple docker compose up. Your misery accentuating your tiredness, you mistype the command three times, going back and forth between the NORMAL and INSERT Vi mode of your Zsh instance.

Did you ever ask yourself how the Zsh editor was able to give you an Emacs or a Vi mode? Have you ever wonder how to customize the editor to ease your tedious work? Well, me neither, for a long time. But we'll fix that together: in this article, we'll explore the capabilities of the Zsh Line Editor (ZLE), and how to customize it to increase our efficiency.

More precisely, we'll see:

What's ZLE and its keymaps.
How to bind any keystroke to any Zsh widget.
A glimpse of the TTY subsystem and why binding special keys can be such a challenge.
Some interesting Zsh built-in widgets.
How to create your own widgets.

I encourage you to fire Zsh while reading this article, and try the different commands we'll discuss. It's time: are you ready to uncover the veil of mystery surrounding the Zsh Line Editor?

What's the Zsh Line Editor?

The Zsh Line Editor (ZLE) is simply your command prompt. It's your interface to the shell interpreter, allowing you to write and edit your mind-blowing commands.

It also allows you to use keystrokes to execute ZLE commands, more commonly called widgets. That's great, because the term "command" is so overloaded that it doesn't mean anything anymore. Like "scalable", and of course "daffodil".

We've already seen some of the ZLE built-in widgets in the first part of this series of article, like vi-backward for example.

Zsh Keymaps

To understand how ZLE works, we first need to understand the concept of keymap.

A keymap is a set of keystrokes executing ZLE widgets. For each keymap, the mapping of keystrokes to widgets can be completely different.

Here are the most interesting keymaps available:

emacs - Emacs emulation
viins - Vi mode - INSERT mode
vicmd - Vi mode - NORMAL mode (also confusingly called COMMAND mode)
viopp - Vi mode - OPERATOR-PENDING mode
visual - Vi mode - VISUAL mode

For example, the ZLE widget vi-join (to join the current line with the next one) is bound to the keystroke CTRL+x CTRL+j in the emacs keymap, and to J in the vicmd keymap.

When we use our prompt, we use the global keymap. This is the current keymap loaded, for us to use its delightful keystrokes. This global keymap is often aliased to the main keymap, which is the keymap Zsh load by default when it launches. If you didn't specify any main keymap in your configuration, it will use the emacs one.

As a Vim Missionary, I would qualify this choice as "unforgivable heresy". Fortunately, We can alias this main keymap to viins (Vi INSERT mode) instead. We'll see how below.

Some other keymaps are sometimes used in specific Zsh modes. In that case, they are called local keymaps. For example, we've seen in the first article of this series how to bind keystrokes to the local keymap menuselect. It's only available when selecting something in a list, like selecting a completion for example.

If we define the same keystroke both in a local and in a global keymap, the first overwrite the second.

Binding Keystrokes to Zsh Widgets: the bindkey Command

That's nice to have many keymaps to choose from, but it's not enough. How do we bind custom keystrokes to these Zsh widgets?

Managing the Bindings

To manage our custom keystrokes, we need to use the bindkey command. First, let's see how we can display what was already bound by using:

bindkey - Output all the key bindings for the global keymap.
bindkey <keystroke> - Output the widgets bound to a specific keystroke <keystroke> in the global keymap. For example: bindkey '^l'.
bindkey -r <keystroke> - Delete the binding mapped to the keystroke <keystroke>. For example: bindkey -r '^l'

If you find that your keystroke is bound to the widget undefined-key, it means that it won't have any effect. This widget is the incarnation of pure void. When you gaze long into the undefined-key, the undefined-key also gazes into you.

The two keys CTRL and ALT are often used for keystrokes; therefore, it's useful to know how to tell Zsh to use one or the other in our key binding:

^ - Represent the CTRL key. For example: ^c for CTRL+c.
\e - Represent the ALT key. For example: \ec for ALT+c.

Note that it's true for most modern terminals out there, but not all. We'll come back to the joys of the escape characters soon.

One of the most common use of bindkey is, of course, to bind some custom keystrokes to some widget. To do so, you can follow this syntax:

bindkey <keystroke> <widget>

For example, if we want to clear the screen with CTRL+g, we would need to bind the widget clear-screen as follows:

bindkey '^g' clear-screen

We don't have to bind keystrokes to widgets, however. We can bind them to whole series of keystrokes, as we were typing them in ZLE. To do so, we can use the bindkey's option -s.

For example, if we want to bind ALT+h to the series of keystrokes CTRL+L (which clear the screen by default) and then writes "hello" because we're excessively polite, we can run the following:

bindkey -s '\eh' '^l hello'

How lovely!

Using bindkey with Specific Keymaps

What if you want to manage the bindings for precise keymaps? Here's a bunch of commands to do so:

bindkey -l - Output the list of all available keymaps.
bindkey -M <keymap> - Output all the keybindings for the keymap <keymap>. For example: bindkey -M viins.
bindkey -M <keymap> <keystroke> - Output the widget bound to the keystroke <keystroke> for the keymap <keymap>. For example: bindkey -M vicmd u.
bindkey -M <keymap> -r <keystroke> - Delete the binding mapped to the keystroke <keystroke> for the keymap <keymap>. For example: bindkey -M vicmd -r '^l'

What about aliasing the main keymap, for Zsh to launch with the one you want to use by default? Here are two handy commands you can use:

bindkey -e - Alias the main keymap to the emacs keymap (the default).
bindkey -v - Alias the main keymap to the viins keymap.

These commands are aliases; they are respectively equivalent to the following commands:

bindkey -A emacs main
bindkey -A viins main

Note that you can't bind the main keymap to the vicmd keymap, the equivalent of Vi NORMAL mode.

You can also uses bindkey -lL main to find out what keymap is aliased to the main one.

Fun fact: the viins keymap (Vi INSERT mode) has the keystroke ESC bound to the widget vi-cmd-mode. In this context, when you hit ESC, you'll switch the global keymap from viins to vicmd (Vi NORMAL mode). That's how you get the Zsh Vi mode.

Even funnier fact: the emacs keymap has also the keystroke CTRL+x CTRL+v (^X^V) bound to vi-cmd-mode. You can switch to Vi NORMAL mode from the emacs keymap! It finally proves, without any doubt, the superiority of Vi (and, by extension, Vim and Neovim) against Emacs.

Binding All The Keys!

We've seen quickly how to represent the special key CTRL or ALT to use them in our keystrokes. But what about the others?

Terminal and Escape Sequence

If you use the emacs keymap, you'll have most special keys bound to some sensible widgets, like the HOME key bound to beginning-of-line, or the END key bound to end-of-line. But, if you're a Vim Believer like I am, you'll reject this keymap with all the strength of your soul.

A question arise, then: how to bound these special keys to these widgets for the viins keymap? Or to other widgets, if we want to? How to represent these special keys for ZLE to know what we want to do?

It's where we enter the muddy swamp of the TTY subsystem. I don't want to drown us into too many details, so I'll try to keep it brief. You can always play with my sanity by asking me to write an article about that in the comment section, at the end of this article.

When you type some special keys, like BACKSPACE, HOME, or ENTER for example, the terminal receives a control character (or escape sequence). This is how the terminal "see" the key you've harshly hit.

The good news: most modern terminal emulators out there will translate your keystrokes with the same control characters (or escape sequence), which mean that we could directly bind these representations to the widgets we want.

In practice, for you to display these sequences and use them in your bindings, you can use CTRL+v in your terminal followed by the special key you want to bind. It will output verbatim the control character sent, without interpreting it.

For example:

When I hit CTRL+v in my terminal followed by the HOME key, I get the delightful escape sequence ^[[7~ (I would love to have that on a T-shirt, by the way).
I then bind it to the widget beginning-of-line for the keymap viins as follows: bindkey -M viins '^[[7~' beginning-of-line.

Now, when I hit the HOME key in Zsh while using the viins keymap, my cursor moves at the beginning of the line. Incredible feat the world will remember!

If you want to display multiple escape sequences without having to hit CTRL+v again and again, you can also run cat > /dev/null and hit whatever keystrokes you like.

This approach has one major drawback. Even if it's true that most modern terminal emulators receive the same escape sequences, the world is not only made with modern terminal emulators. These sequences might differ for:

Your TTYs (what you open when you hit CTRL+ALT+<FKEY> in Linux, <FKEY> being the keys from F1 to F12).
Your terminal multiplexer (like tmux or screen).

Because of these differences, there is another approach praised by many out there (even advised by the venerable Arch Linux Wiki): using terminfo.

A Common Terminal Interface

This "terminfo" is a common database aimed to provide an unified interface between the different terminals available. It's often used by CLIs to intercept the different escape sequences and perform some action. For example, Vi uses terminfo under the hood.

There is a problem with this approach, however: to use terminfo, we need to switch ZLE to "application mode", only used normally by some CLIs and TUIs running in the shell (but not all). Since the command line editor of ZSH does not run in application mode by default, we need to switch to it before using terminfo.

It means that any CLI running in the terminal will run in this application mode too. If some of these CLIs are not meant to run in this mode, some keystrokes might not work.

Because it feels more like a hack than a proper solution, I personally try to avoid this method.

Automatically Mapping Escape Sequences with zkbd

We can also use "zkbd" from Zsh contrib. You can do so by following this workflow:

Autoload zkbd by adding autoload -Uz zkbd to your zshrc.
Start a new Zsh process and run zkbd in your shell.
Answer the questions.

After answering all questions, zkbd will create a bunch of files with all the escape sequences bound to the good widgets. I never really used zkbd to be honest, but it might help to get you started.

Personal Configuration

I've written a piece of configuration showing both solutions proposed above (the one involving terminfo is commented out because I don't use it). You can try it, I'm pretty sure it will work quite well if you don't use some exotic terminal.

A last advice: if you use tmux, make sure that you have the line set -g default-terminal "tmux-256color" in your tmux configuration file for this config to work.

The Zsh Widgets

We've spoken a lot about keymaps and keybindings. What about the other side of the equation, the Zsh widgets themselves?

Built-in Widgets

There are many built-in widgets you can bind with the keystrokes of your dreams. Here are the different ways to get a complete list of these widgets:

You can run zle -la in your shell.
You can look at the manual page for ZLE by running man zshzle. Search for "Standard Widgets".
You can install the package zsh-doc and use a command for each widget, to find out its powers. For example: info zsh beginning-of-line.
You can look online.

But reading is cheap, trying is the way to enlightenment. How to try a widget directly, to see what it does?

To call the widget <widget>, we need to use the command zle <widget>. You can try to run zle end-of-line for example, to see that it doesn't work. How deceitful!

You'll be indeed rewarded with a voluptuous zle: widgets can only be called when ZLE is active. It means that widgets can be called only when you write and edit your commands in the editor, not when you send them to your TTY by hitting ENTER.

As a result, we can call widgets inside widgets, because widgets are always called when ZLE is available. And since we can call widgets using specific keystrokes, what about a widget which can run any widget? That's exactly the purpose of execute-named-cmd. It's bound by default to the keystrokes ESC x for the emacs keymap and : for the vicmd keymap (Vi NORMAL mode).

If you hit the good keystrokes depending of the global keymap you're using, a new prompt will appear. There, you can call any widget you want; they will act on whatever command you've began to type in your editor, if any. Even better: you can complete the widget's name you want to run in this new prompt by using TAB, as always.

Using execute-named-cmd, you can then run two other useful widgets if you want to know what widget is bound to what keystroke (and vice-versa):

where-is - Open yet another prompt where you can type the name of a widget. It will then output its keybinding (if any).
describe-key-briefly - Open yet another prompt where you can hit a keystroke. It will then output the widget bound to that keystroke (if any).

Widgets from Zsh User Contributions

There are also more widgets available via Zsh contrib (man zshcontrib), like the wonderful edit-command-line, allowing you to edit your commands in ~~Vim~~ your favorite editor, or select-quoted, useful to create Vi text-object representing quoted substring. More on that below.

I've covered many of these widgets in the first article of this series. Don't forget that you need to autoload them before using them. They are not always documented, but you can directly look at what you can use on the Zsh mirror repository.

For example, if you want to use the widget incarg from Zsh contrib to increment a number using a keystroke, you can add to your zshrc:

autoload -Uz incarg
zle -N incarg
bindkey -M vicmd '^a' incarg

Now, when your cursor is on a number and you hit CTRL+a, it will be incremented by the power of your spirit.

Writing your Own Widget

Using built-in widgets is definitely useful, but we can push the customization even further: what about widgets designed by your talented creativity?

Executing A Command While Writing Another

As we saw just above, we can create our own widget using zle -N <shell_function>. If an existing widget has already the same name, it will be overwritten. That's why built-in widgets have two names: one with a dot . prefix, and one without. If you overwrite the built-in widget end-of-line for example, you can still access the original one by using .end-of-line. As a result, it's considered good practice to never prefix your own widgets with a dot; it should be reserved to the most important widgets, the built-in ones.

Enough rambling. Let's build our first widget!

We can imagine that you're writing a command in your shell, and suddenly you wonder what you've modified in your project. The best would be to:

Run git diff.
Go back to the command you were writing.

Here's the widget:

function _git-diff {
    zle .kill-whole-line
    zle -U "git diff
$CUTBUFFER"
}

zle -N _git-diff
bindkey '^Xd' _git-diff

What happens in there?

We use zle to run the widget .kill-whole-line (we use the dot-prefixed name in case kill-whole-line was overwritten somewhere else). It will remove the command you were crafting with enthusiasm and love.
We run zle -U. This command inserts some string in ZLE, exactly like you would type them in the editor.
We declare the function _git-diff as a widget. We can run it in ZLE by hitting CTRL+x and then d.

It's worthwhile to explain further the second point:

The character between git diff and $CUTBUFFER is actually a carriage return, the control character ^M. I've made it by hitting CTRL+v and then ENTER.
The variable $CUTBUFFER contains whatever was deleted when using a widget beginning with kill-. It allows us to bring back the command previously deleted with .kill-whole-line.

The underscore _ prefixing the function's name indicates that the function is a widget. It's just a personal convention to know what shell function is a widget, and what isn't; you don't have to do the same with your widgets.

And voila! We've our first marvelous and useless little widget.

It's not the only implementation possible: we could try to run git diff first, and then bring back the command deleted as follows:

zle -U "git diff"
zle -U "$CUTBUFFER"

I'm afraid I've to deceive you once again; at the point, it's a miracle you're sill reading this article. The commands above won't do what we want them to do.

In fact, zle -U put strings on the stack of ZLE, and pull them back in the editor when the widget terminates. And since it's a stack (LIFO, Last In, First Out), $CUTBUFFER would be inserted in the editor before git diff! To fix that, you could swap the two declarations above, but it makes the widget more difficult to understand.

Here's another, cleaner solution:

function _git-diff {
    zle push-input
    BUFFER="git diff"
    zle accept-line
}

zle -N _git-diff
bindkey '^Xd' _git-diff

What happens there?

We use the widget push-input. It pushes the command on the edit buffer stack, and pull it back the next time ZLE is available.
We change the current buffer to git diff.
We use the widget accept-line which runs whatever command written in ZLE (in that case, git diff).
The widget push-input pull back the command from the buffer stack. It even keeps the cursor position!

A Widget to Prefix Your Commands

Let's take another example. Let's say that you want a widget to prepend "vim" to any command. A noble idea! Here's a possible solution:

function _vim {
    [[ ! $BUFFER =~ '^vi.*' ]] && BUFFER="vim $BUFFER" && zle end-of-line
}

zle -N _vim
bindkey -M vicmd '^Xv' _vim

If the regular expression ^vi.* is not matched, it:

Inserts vim at the beginning of the command.
Moves the cursor to the end of the line.

Quite easy, isn't it?

Managing Surrounding

I mentioned, in the first article of this series, a way to create new Vi text-objects and bind them to some specific widgets with the following commands:

autoload -Uz select-bracketed select-quoted
zle -N select-quoted
zle -N select-bracketed
for km in viopp visual; do
  for c in {a,i}${(s..)^:-\'\"\`\|,./:;=+@}; do
    bindkey -M $km -- $c select-quoted
  done
  for c in {a,i}${(s..)^:-'()[]{}<>bB'}; do
    bindkey -M $km -- $c select-bracketed
  done
done

We're now able to understand what's going on in there:

We loop through the two keymaps viopp (Vi OPERATOR-PENDING mode) and visual (Vi VISUAL mode).
We loop through a whole bunch of signs we want to consider as quotes (or brackets), and we add to each of them the prefix i or a (for inside and around, respectively).
We use the bindkey command to bind these new text-objects to both keymaps.

If you're not sure what's the Vi OPERATOR-PENDING mode, I've written about it in this article about Vim. It's can be used to create new text-objects. Here are the ones we bind to the select-bracketed widget: a(,i(,a),i),a[,i[,a],i],a{,i{,a},i},a<,i<,a>,i>,ab,ib,aB,iB.

If you want to know exactly what text-objects are created, you can add echo $c in both nested loops before the command bindkey.

The Power of the Zsh Line Editor

Well, that was quite a trip! Thanks to custom keybidings and widgets, the Zsh Line Editor is a valuable tool, allowing us to make our shell experience an effective one.

What did we see in this article?

The Zsh Line Editor (ZLE) is the command prompt where you can write and edit your commands.
The main keymap is the set of keystrokes which is loaded by default when Zsh is launched.
The global keymap is the one used to edit commands in Zsh. Local keymaps can be used in some specific Zsh modes, like selecting elements in a list.
A widget is a shell function which can be executed after hitting specific keystrokes. They can call other widgets, and they have access to special variables to manipulate the line editor.
We can look at the keystrokes already bound, and bind new ones to specific widgets with the bindkey command.
The terminal will interpret special keys with specific escape sequences. We can map them directly to some widgets, or use terminfo in application mode.
We can create our own widgets to customize even more what we can do in ZLE.

When you're annoyed by some functionalities which should be implemented while typing your commands in Zsh, it's a good opportunity to write a widget to answer your need. That's how you create a highly customized Mouseless Development Environment: by solving one little problem at a time.

Related Resources

Zsh Line Editor - Zsh Documentation
ZLE Standard Widgets - Zsh Documentation
Zsh Keybindings - Zsh Wiki
terminfo and termcap - David S. Lawyer
User defined widgets - Zsh Documentation

Becoming Mouseless

Do you want to build a Mouseless Development Environment where the Linux shell has a central role?

Switching between the keyboard and the mouse costs cognitive energy. This book will guide you step by step to set up a Linux-based development environment that keeps your hands on your keyboard.

Take the brain power you've been using to juggle input devices and focus it where it belongs: on what you create.

A Guide to Zsh Expansion with Examples

Matthieu Cneude — Wed, 23 Mar 2022 16:58:29 +0000

What would we do if we couldn't use the wonderful TAB key to expand our commands, filenames, or variables in our lovely shell? Not much, I'm afraid. We would still live in dark caves, hunting some dangerous Mammoth with knives made of stones!

Zsh is particularly powerful when you need some good old expansions. It supports common ones used by Bash too, and add many flags and modifiers on top. It comes in handy to manipulate quickly your commands without writing boring shell scripts.

We'll see, in this article, what we can do with Zsh expansions, and more specifically:

How to use glob operators and globbing flags.
What glob qualifiers we can use to expand filenames.
How to expand the Zsh history and how to modify these expansions.
How to expand parameters, using modifiers and flags.

In the examples of this article, the character > represents the shell's prompt. If you want to run the commands in your terminal (which I warmly recommend you to do), don't copy these > characters. Why not using the dollar character $ for the prompt, like every other articles out there? Because it's my article, I do what I want.

There are many, many ways to expand your filenames, history entries, or parameters with Zsh. I won't cover everything here, only what I find the most useful. The amount of information can still be overwhelming, however; that's why I recommend the following:

Take two or three useful tips from this article and try to apply them in your daily work.
When you're confident with your new knowledge, come back to this article and pick two or three more tips.
Rinse and repeat.

Last bit of information before diving in: I used Zsh 5.8 and GNU Bash 5.1.8 for testing the different examples. With that out of the way, let's begin to expand our souls.

Glob Operators

We can't use regular expressions with Bash or Zsh to expand our filenames. Instead, we can use glob operators to expand various filenames.

If you use the shell for long enough, I'm sure you're already familiar with some of them. Here's the list of glob operators for Zsh: *, (, |, <, [, and ?.

We'll see in this section how to use these glob operators. The examples given here follow this set of rules:

The first line is the non-expanded version of the command.
The second line is the expanded version of the command, after typing the non-expanded version and hitting the TAB key.
Each example uses the following directory structure:

.
├── a_dir_with_a_file
│   └── a_file
├── A_FILE_WITH_UPPERCASE
├── another_file
├── empty_dir
├── file_1
└── file_2

I encourage you to create the exact same file tree with these commands and try the examples in your shell:

mkdir a_dir_with_a_file empty_dir \
&& touch a_dir_with_a_file/a_file A_FILE_WITH_UPPERCASE another_file file_1 file_2

Trying by yourself will let you feel the power of Zsh under your fingertips. Nothing less!

Basic Glob Operators

These two basic glob operators work in Bash and Zsh:

* - Matches any string.
? - Matches any character.

For example:

> cat fil*
> cat file_1 file_2

> cat file_?
> cat file_1 file_2

> ls ./*file*
> ls ./a_dir_with_a_file ./another_file ./file_1 ./file_2

Character classes

Beyond the useful * and ?, we can also use character classes in both Bash and Zsh:

[] - Matches one of the enclosed character, or a range of characters separated with -.
[:<set>:] - Matches a specific set of characters.
[^<set>] or [!<set>] - Match any character which is not in the set <set>.

For example:

# Match every filename beginning with "f", "l", or "e"
> ls [fle]*
> ls empty_dir file_1 file_2 

# Match every filename not beginning with "a"
> ls [^a]*
> ls A_FILE_WITH_UPPERCASE empty_dir file_1 file_2 

# Match every filename beginning with "a"
> ls [a]*
> ls a_dir_with_a_file another_file 

# Match a filename beginning with any letter from "a" to "z"
> ls [a-z]*
> ls a_dir_with_a_file another_file empty_dir file_1 file_2

# Match a filename beginning with lowercase, followed by an underscore, and any other character
> ls [^[:upper:]]_*
> ls a_dir_with_a_file

Grouping

Grouping is only available in Zsh. To group, we can use the following characters:

() - Matches the enclosed pattern.
| - Equivalent of the boolean operator "OR". It needs to be used inside groups.

For example:

# Match any filename beginning with "fi" or "an"
> ls (fi|an)*
> ls another_file file_1 file_2

Globbing Flags

Globbing flags are only available in Zsh. They need to be added before any glob operator, and they need to have this syntax: (#<flag>).

Here are the three most useful flags:

i - Case insensitive.
I - Case sensitive.
l - Lowercase match lower case or uppercase; uppercase match only uppercase.

For example:

> cat a*
> cat a_dir_with_a_file another_file

> cat (#i)a*
> cat a_dir_with_a_file A_FILE_WITH_UPPERCASE another_file

> cat (#l)a*
> cat a_dir_with_a_file A_FILE_WITH_UPPERCASE another_file 

> cat (#l)A*
> cat A_FILE_WITH_UPPERCASE

Recursive Globbing

You can also use the recursive glob operator ** to expand the files in the working directory (like *) and every file in every subdirectory, recursively.

For example:

# Search filenames including the substring "file" in the working directory and its subdirectories
> ls ./**/*file*
> ls ./a_dir_with_a_file ./a_dir_with_a_file/a_file ./another_file ./file_1 ./file_2

The depth of the directory tree doesn't matter: Zsh will go through the whole tree and find whatever you want. You can also use the alternative glob operator *** if you also want to parse symlink's targets.

The recursive glob operator can be used in Bash too, but it will only expand files on the first level below the working directory. If you want it to behave like its Zsh counterpart, you need to set the option globstar with the following command shopt -s globstar.

Glob Qualifiers

Glob qualifiers are only available in Zsh. They are added after the glob operators to filter even further the filenames expanded. They are always between parenthesis.

You can use multiple qualifiers inside the parenthesis, separated with a colon :.

Basic Qualifiers

Here are some basic qualifiers:

^ - Negate all qualifiers following it.
- - The qualifier works on the target of the symbolic link (and not the symbolic link itself).
[<beginning>, <end>] - Choose what filenames should be expanded, depending on their positions. Can be negative to count from last match backward.

For example:

$ ls *
$ ls a_dir_with_a_file A_FILE_WITH_UPPERCASE another_file empty_dir file_1 file_2 

# Only expand the second and third file
> ls *([2,3])
> ls A_FILE_WITH_UPPERCASE another_file

Directory Expansion

These qualifiers will only expand to directories, except if they're negated with ^.

F - Expand to non-empty directories.
^F - Expand to empty directories and non-directories (plain files).
/^F - Expand to empty directories.

# Only expand non-empty directory
> ls *(F)
> ls a_dir_with_a_file

# Only expand plain files and empty directories
> ls *(^F)
> ls A_FILE_WITH_UPPERCASE another_file empty_dir file_1 file_2 

# Only expand empty directories
> ls *(/^F)
> ls empty_dir

Expanding By Filetype

We can also use qualifiers to expand specific file types (plain files or directory):

. - Expand to plain files
/ - Expand to directories

# Only expand plain files
> cat *(.)
> cat A_FILE_WITH_UPPERCASE another_file file_1 file_2 

# Only expand directories
> ls *(/)
> ls a_dir_with_a_file empty_dir

Expansions Depending On the Permissions

To only expand files with specific permissions, we can use the following qualifiers:

Depending on the owner:

* - Expand to executable plain files.
r - Expand to owner readable files.
w - Expand to owner writable files.
x - Expand to owner executable files.
U - Expand to files or directories owned by the current user.

Depending on the world:

R - Expand to world readable files.
W - Expand to world writable files.
X - Expand to world executable files>

Depending on the group:

A - Expand to group readable files
I - Expand to group writable files
E - Expand to group executable files
G - File or directory owned by the current user's group.

For even more control, we can specify the precise permissions:

f<spec> - Expand the files with access right matching the octal number <spec>. If <spec> is preceded by:
- = (default when none given) - <spec> must match the exact file-mode.
- + - At least one bit needs to be in the file-mode.
- - - The bits must not be set.

For example:

# Set the permissions for "file_1" to 777 (world can read, write, execute)
> chmod 777 file_1

# Expand files the user can read
> ls *(w)
> ls a_dir_with_a_file A_FILE_WITH_UPPERCASE another_file empty_dir file_1 file_2 

# Only expand the files with permission 777
> cat *(f777)
> cat file_1

# Only expand files when the permissions are always more than executable:
> cat *(f-1) 
> cat A_FILE_WITH_UPPERCASE another_file file_2

Expansions Depending on Date or Size

It's also possible to expand files depending on some dates:

m<unit><operator><n> - Expand to files modified <n> time ago.
- The <unit> can be M for month of 30 days, w for weeks, d for days (default when nothing is given),h for hour, m for minute, or s for second.
- The <operator> can be:
 - + - Files accessed more than <n> <unit> ago
 - - - Files accessed less than <n> <unit> ago.

The qualifier to expand only files with a specific size is similar:

L<unit><operator><n> - Expand to file with a certain size <n>.
- The unit can be K (Kilobyte), M (Megabyte), G (gigabyte), T (terrabyte).
- If <operator> is:
 - + - Files more than <n> bytes.
 - - - Files less than <n> bytes.
 - To match exactly the size, don't give an <operator>.

For example:

# Delete every files greater than 1Gb recursively
> rm **/*(LG+1)

# Delete all empty files recursively
> rm **/*(L0)

# Delete all files modified in the last hour
> rm **/*(mh-1)

# Delete all files modified more than one hour ago 
> rm **/*(mh+1)

Expanding A Precise Number of Files

Let's say that you had a vision: you know that you only want to expand a precise number of files. The following qualifier will grant your crazy wish:

Y<n> - Limit the expansion to <n> number of files

# Only expand two files
> ls *(Y2)
> ls A_FILE_WITH_UPPERCASE empty_dir

Expanding and Sorting

We can also sort our files if we want to:

o<sort> - Sort files depending on the value of <sort>
O<sort> - Like o, but sort in descending order

The value of <sort> can be:

n - Sort by name (the default).
L - Sort by size.
l - Sort by number of links.
a - Sort by last access.
m - Sort by last modification.
c - Sort by last inode change.
d - Files in subdirectories appear before.
N - Don't sort anything.

For example:

# Sort files from the smallest to the largest
> ls *(oL)

# Sort files from the largest to the smallest
> ls *(OL)

# Output the three biggest files (excluding directories) in the whole filetree
> ls **/*(.OL[1,3])

Expanding Depending on a Predicate

As far as I know, a predicate is not a dinosaur, but a command (or a function) returning a boolean. Here's a qualifier to filter the expansion depending on a predicate:

e:'<cmd>' - The estring will filter the files depending on a command returning true or false. The filename being tested is available via the special variable REPLY. You can also change the value of REPLY to generate new filenames.

Here are two examples to help you understand my complicated rambling:

# Expand every file except the ones called "file_2"
> cat *(e:'[[ $REPLY != file_2 ]]':)
> cat a_dir_with_a_file A_FILE_WITH_UPPERCASE another_file empty_dir file_1

# Expand only plain files (with the qualifier "."), except the ones called "file_2"
> cat *(e:'[[ $REPLY != file_2 ]]':.)
> cat A_FILE_WITH_UPPERCASE another_file file_1 

# Expand only plain files, and reassign the REPLY variable to expand even more files (even if they don't exist)
> touch *(e:'reply=(${REPLY}_{1..3})':.)
> touch A_FILE_WITH_UPPERCASE_1 A_FILE_WITH_UPPERCASE_2 A_FILE_WITH_UPPERCASE_3 another_file_1 another_file_2 another_file_3 file_1_1 file_1_2 file_1_3 file_2_1 file_2_2 file_2_3

The form e:'<cmd>' can take the form +<cmd> if you need to call a function instead of using a command.

# Create the function "condition"
> condition() { [[ $REPLY == file_2 ]] }

# Only expand file named "file_2"
> cat *(+condition)
> cat file_2

That was quite a ride! Let's now abandon the glob operators, their flags, and their qualifiers, to focus on the history of Zsh.

Expanding the Zsh History

To me, the best way to manage the shell's history is to use a fuzzy finder like fzf. That said, if you want to use vanilla Zsh to manipulate your history, you can use these three special characters:

! - The history character.
^ - The modification character.
# - The command character.

These characters are defined in the special variable $HISTCHARS.

It means that you can't use these three characters in your commands without properly escaping them. For example, If you run echo hello!! or echo "hello!!", the shell will expand the !!. In that case, you would need to use simple quotes (echo 'hello!!') or escaping the special characters with backslashes (echo "hello\!\!").

The Event Designators

The event designator is used to expand a command (or part of a command) from the history.

First, to display your shell's history, you can run the command history, or history -<number> (<number> indicating how many commands you want to output). For example,history -100 will display the last 100 commands you've executed in your shell.

Here are some nice and sweet event designators:

!<number> - Expand the <number>th entry from the history.
!-<number> - Expand the <number>th entry from the history, beginning by the most recent command.
!! - Expand the previous command executed.
!<string> - Expand the most recent command executed, starting with <string>.
!# - Expand to the current command you're typing.

For example:

# Expand the history's entry 123
> !123

# Expand the second most recent command.
> !-2

# Run a command for the following examples
> echo "first" "second" "third"

# Expand to the last echo command executed
> !echo
> echo "first" "second" "third"

# Expand to the last command executed
> !!
> echo "first" "second" "third"

# try to read install Neovim
> pacman -S neovim
error: you cannot perform this operation unless you are root.

# Repeat the previous command with sudo (super handy!)
> sudo !!
> sudo pacman -S neovim

Word Designators

Going down to the rabbit hole, we can add word designators to our event designator. The goal is to only expand parts of the command from the history.

The word designator needs to be separated from the event designator with a color :.

Here are my favorites:

0 - Expand the first input word of the command.
<number> - Expand the <number>th argument.
^ - Expand the first argument or option.
$ - Expand the last argument or option.
* - Expand to all the arguments or options. If there is none, expand to the NULL value.

You know what will follow? Yes! Examples! More of them!

> echo "first" "second" "third"

# Expand to the first word of the last command
> !!:0
> echo

# Expand to the second argument of the last command
> echo !!:2
> echo "second"

# Expand to the first argument of the last command
> echo !!:^
> echo "first"

# Expand to the last argument of the last command
> echo !!:$
> echo "third"

# Expand to all the arguments of the last command
> print !!:*
> print "first" "second" "third"

History Modifiers

Believe it or not, there's more. Additionally, you can add some modifiers to the word designator, again separated with :.

As always, here are my favorites:

a - Transform a file name into an absolute path.
A - Transform a file name into an absolute path and resolve the eventual symlinks.
h<number> - Keep the head of the path: everything except the last component of the path. Use <number> to keep the <number> of component of the pathname. In absolute path, the root directory / is the first component.
t<number> - Remove all leading path components except the final one (the tail). Add a <number> to keep a precise amount of trailing components.
r - Remove the filename extension.
l - Convert everything to lowercase.
u - Convert everything to uppercase.

# Run a command
> ls ~/a_dir_with_a_file/a_dir/a_file

# Expand to the absolute path
> !!:^:a
> ls /home/user/a_dir_with_a_file/a_dir/a_file

# Expand only the head (delete the tail) - similar to dirname
> !!:^:h
> a_dir_with_a_file/a_dir

# Expand only the first part of the head
> !!:^:h1
> a_dir_with_a_file

# Expand only the tail (delete the head) - similar to basename
> !!:^:t
> a_file

# Expand only the tail and one element before
> !!:^:t2
> a_dir/a_file

The Substitution Modifier

We can also substitute one substring with another using the substitution modifier:

s/<string>/<replacement>/<flag> - Substitute <string> with <replacement> using an optional <flag>.
gs/<string>/<replacement>/ or s/<string>/<replacement>/:g - Substitute globally.

If we use the metacharacter & in the replacement, it will be expanded with <string>. If you want to use the literal symbol & and not the metacharacter, you need to escape it with a backslash.

If the <string> is empty, it uses the previous <string> from the previous substitution.

For example:

> echo "hello" "hello" "bonjour"

> !!:s/hello/bonjour
> echo "bonjour" "hello" "bonjour"

> !!:gs/hello/bonjour
> echo "bonjour" "bonjour" "bonjour"

> !!:gs/hello/& dear friend
> echo "hello dear friend" "hello dear friend" "bonjour"

That's all for expanding the history. Next steps: expanding variables and parameters!

Parameters Expansion

Again, Zsh gives us many tools to expand variables and parameters.

The Dollar Character

As you probably know, the dollar character $ is used for variable expansion in Bash and Zsh. For example, if you define a wonderful variable var=hello, you can expand its value with $var.

Zsh, unlike Bash, can also expand arrays using the good old dollar.

For example:

# Create a variable "var" and assign the value "hello". Spaces between `=` are not authorized (don't do "var = hello").
> var=hello

# Expand the variable "var"
> echo "$var"
> echo hello

# Create an array with three elements
> array=("one" "two" "three")

# Expand the values of the array (only in Zsh)
> echo $array
> echo one two three

Parameters Expansion: Adding Braces

If the name of your variable is a substring of a word, you need to add braces to specify to the shell what's the name of the variable and what's some common string to display. For example:

# Create a variable "size"
> size=12

# The shell tries to expand the variable "sizeGb"
> echo "This is $sizeGb"
> echo This\ is\

# The shell knows that it needs to expand the variable "size" (and not "sizeGb")
> echo "This is ${size}Gb"
> echo This\ is\ 12Gb

That's not all: we can add modifiers to these braces, to manipulate the expansion of the variable (and not the value itself). Here's a list of the most useful ones, all available for Bash and Zsh :

${var:-default} - If var is not null, the value is expanded. Otherwise, default is expanded. Useful when you need a default value when the variable (or parameter) is empty.
${var:+default} - If var is not null, default is expanded. Otherwise, nothing is expanded.
${var#pattern} - If the pattern match the beginning of the value of var, the match is deleted and the rest is expanded. Use ## to match larger matching pattern.
${var%pattern} - If the pattern match the end of the value of var, the match is removed and the rest is expanded. Use %% to match larger matching pattern.
${var/<pattern>/<replacement>} - Replace the first pattern in var with replacement.
${var//<pattern>/<replacement>} - Replace all occurrences of pattern in var with replacement.

You guessed it already: I like examples. So here you go:

# Create a variable
> var=new-value

# Remove the prefix "new"
> echo "${var#new}"
> echo -value

# Change the value of the variable
> var="/home/user"

# Delete the first slash and everything before
> ls "${var#*/}"
> home/user

# Delete the last slash and everything before
> ls "${var##*/}"
> ls user

# Create a new variable "image"
> image=file.png

# Delete everything after the first dot
> echo "${image%.*}"
> echo file

# Delete everything before the first dot
> echo "${image#*.}"
> echo png

> var="hello hello hello"

# Replace the first word "hello" by "bonjour"
> echo "${var/hello/bonjour}"
> echo bonjour\ hello\ hello

# Replace every word "hello" by "bonjour"
> echo "${var//hello/bonjour}"
> echo bonjour\ bonjour\ bonjour

The following ones are only available with Zsh:

${var:/<pattern>/<replacement>} - Substitute <pattern> with <replacement>. The substitution only happens when pattern match the whole value of var.
${^array} - Substitute every element of the array with whatever comes before or after.
${=array} - Split a string to an array using the internal field separators. These separators are the values of the variable $IFS, by default whitepaces (spaces, tabs, and new lines characters).

# The pattern matches the entire value of the variable var, so everything is replaced with "bonjour"
> echo "${var:/hello hello hello/bonjour}"
> echo bonjour

# Create a new variable "array"
> array=("one" "two" "three")

# Expand the whole array and add the suffix ".png" to the last element
> echo "${array}.png"
> echo one\ two\ three.png

# Expand every element of the array with the suffix ".png".
# This command needs to be executed, not expanded (pressing the key "ENTER", not "TAB").
> echo ${^array}.png
one.png two.png three.png

> var="a big var"

# Assign the whole value of var to the array
> arr=(${var})

# Expand the first element of the array
> echo "$arr[1]"
> echo a\ big\ var

# Cut the value of var and assign it to the array
> arr=(${=var})

> echo "$arr[1]"
> echo a

> echo "$arr[2]"
> echo big

It's also possible to nest the pair of braces. Everything will be expanded from in out.

> var="/home/user/woupi/"

# 1. "${var#/home}" is expanded, output: "/user/woupi"
# 2. The output is expanded as follows: "<output>%woupi/"
# 3. The final output is "/user/"
> echo "${${var#/home}%woupi/}"
> echo /user/

> var="prefix-image.png"

# 1. "${var#prefix-}" is expanded, output: image.png 
# 2. "${output%.*}" is expanded, output: image
# 3. ".jpg" is added to the output, final output: image.jpg
> echo "${${var#prefix-}%.*}.jpg"
> echo image.jpg

Flags

You want more? That's not a problem. With Zsh, you can also add flags to parameter expansions. These flags are surrounded by parenthesis (), just after the opening curly braces {. You can use as many as you want, as your heart's content.

As always, here are the most interesting ones, with examples of course. For these examples, the expansion is not the result of hitting the key TAB, but by hitting the key ENTER.

C - Capitalise (first letter uppercase) every word.

> var="hello i like zsh"

> echo "${(C)var}"
Hello I Like Zsh

f - Join strings separated with newlines.
F - Split array with each element separated with newline.

# Assign all the files of the current directories and subdirectories to the variable "files"
> files=$(find)

> echo $files
.
./file1
./file2

> echo ${(f)files}
. ./file1 ./file2

> arr=(${(f)files})

# First index of array is 1
> echo $arr[1]
.

> echo $arr[2]
./file1

> echo ${(F)arr}
.
./file1
./file2

i - Sort an array (case-insensitive).

> arr=(file_02 file_01 B_FILE a_file 02_file 01_file)

> echo ${(i)arr}
01_file 02_file a_file file_01 file_02

O - Sort an array in descending order. Useful when combined with i.

> arr=(file_02 file_01 a_file 02_file 01_file)

> echo ${(Oi)arr}
file_02 file_01 a_file 02_file 01_file

L - Convert all letters to lowercase
U - Convert all letters to uppercase

> var="file_1 FILE_2"

> echo ${(U)var}
FILE_1 FILE_2

> echo ${(L)var}
file_1 file_2

t - Describe the type of the variable. It can be:
- scalar
- array
- integer
- float
- association

> arr=("one" "two")

> echo ${(t)arr}
array

# Create an associative array
> declare -A assoc=("key" "value")

> echo ${(t)assoc}
association

u - Only expand the unique words in an array, even if these words are repeated multiple times.

> arr=("one" "two" "two" "three")

> echo ${(u)arr}
one two three

k - Expand the keys instead of the values from an associative array.
v - Used with k, expand both key and value from an associative array

> declare -A assoc=("key" "value")

> echo ${assoc}
value

> echo ${(k)assoc}
key

> echo ${(kv)assoc}
key value

S - Adding this flag with #, ##, %, or %% will match a substring.

> var="/home/user/workspace/tvd"

> echo "${var#home*}"
/home/user/workspace/tvd

> echo "${(S)var#home*}"
//user/workspace/tvd

> echo ${(S)var##home*}
/

> echo ${(S)var%work*}
/home/user/space/tvd

> echo ${(S)var%%work*}
/home/user/

M - Adding this flag with #, ##, %, or %% will output the matched portion.

> var="/home/user"

> echo "${var#*/}"
home/user

> echo "${(M)var#*/}"
/

> echo "${var##*/}"
user

> echo "${(M)var##*/}"
/home/

You Can Now Become a Zsh Godly Figure

You can do many things with Zsh, even if it's not always easy to figure out what. With what we've seen in this article, we're now able to manipulate our expansions in more straightforward ways. No need to use annoying for loops anymore!

To summarize:

Glob operators can help you expand filenames, even recursively through a whole file tree. You can also add globbing qualifiers and flags to expand specific filenames.
Being able to run commands you've run in the past is really useful. Being able to modify the expansion of these commands on the fly is even better.
With Zsh, we can also expand parameters and variables easily using the famous dollar $, curly braces {}, and flags.

If it's not enough for you, I invite you to read the part of the Zsh's manual about expansion (man zshexpn). There's everything we've covered here, and much more, if your brain can handle it. Mine was on a strike after going through it.

Becoming Mouseless

Do you want to build a Mouseless Development Environment where the Linux shell has a central role?

Switching between the keyboard and the mouse costs cognitive energy. This book will guide you step by step to set up a Linux-based development environment that keeps your hands on your keyboard.

Take the brain power you've been using to juggle input devices and focus it where it belongs: on what you create.

A Guide to the Zsh Completion With Examples

Matthieu Cneude — Sat, 11 Dec 2021 13:07:28 +0000

"The completion offered by Zsh is great, but could it be better? Why not trying to understand how it works? I could then configure it for my own needs!"

This was my thinking a couple of months ago, after waking up a Monday morning full of energy and will to eat my breakfast. I was young and innocent, unaware of the consequences of this thought making its nest in my brain. What could go wrong? After all, it's only a completion system.

While reading the page of the Zsh manual concerning the completion, I saw the Demon of Complexity™ showing its nose. A dozen of questions were popping up, a twisted configuration syntax was unleashed, nonsensical descriptions were unfolding before my eyes.

When I see something complex, I can't stop myself trying to simplify it. It took me hours to begin to understand this completion system, but now I think I've a good grasp on the Beast.

This article is the result of my efforts. It aims to explain the Zsh completion system in a simple way; more precisely, we'll see:

How to enable the Zsh completion.
How does the completion system work.
What's the purpose of the zstyle module.
How to use styles to configure the Zsh completion.
What are the most useful styles we can use to customize the completion.
How to trigger a precise completion using keystrokes.
How to customize the completion menu using the module complist.
What are the Zsh options we can set to configure the completion.

As I explain in a previous article, I don't use any Zsh framework (like oh-my-zsh or prezto) to have a full control on my config.

Enough rambling! Let's dive into the fantastic and dangerous Zsh completion system.

Enabling the Zsh Completion System

To initialize the completion for the current session, you need to autoload the function compinit and to call it. To do so, add the following in your file .zshrc:

autoload -U compinit; compinit

If you wonder what's this autoload, I already wrote about it in this article.

You can also configure your completion with the shell command compinstall. It asks a series of question to decide what kind of completion you need, and it writes automatically your preferences in your .zshrc. In that case, you don't need to autoload and run compinit manually.

But I find this tool very limited, and the questions it asks are quite obscure. I prefer writing my own config for the completion system in a separate file and source it in my .zshrc. More on that below.

It's where things become slightly complicated. If you don't care about the details, you can copy and paste my simple config in a file and source it in your .zshrc.

How Does the Zsh Completion System Work

Zsh has two completion systems, the old compctl and the more recent compsys. We only look at compsys in this article.

Concretely, compsys is a collection of Zsh functions. You can find them in the official repository of Zsh. Three folders are specifically important:

Base - The core of the completion system, defining the basic completers.
Zsh - Functions for completing built-in Zsh commands (like cd for example).
Unix - Function for completing external commands.

There are also folders for commands only available in some systems, like Solaris, Redhat, or Mandrive for example.

When you type a command in Zsh and hit the TAB key, a completion is attempted depending on the context. This context includes:

What command and options have been already typed at the command-line prompt.
Where the cursor is.

The context is then given to one or more completer functions which will attempt to complete what you've typed. Each of these completers are tried in order and, if the first one can't complete the context, the next one will try. When the context can be completed, some possible matches will be displayed and you can choose whatever you want.

A warning is thrown if none of the completers are able to match the context.

There are two more things to know about completers:

The names of completers are prefixed with an underscore, like _complete or _expand_alias for example.
A few completers return a special value 0 which will stop the completion, even if more completers are defined afterward.

Configuring Zsh Completion With zstyle

You can configure many aspects of the completion system using the zsh module zstyle.

Modules are part of Zsh but optional, detached from the core of the shell. They can be linked to the shell at build time or dynamically linked while the shell is running.

What's zstyle?

You might think that zstyle is used to configure some display styles, but it's much more than that. It's easier to see it as a flexible way to modify the default settings of Zsh scripts (modules, widgets, functions, and so on).
The authors of these scripts need to define what are these settings, what pattern a user can use to modify them, and how these modifications can affect their code.

Here's the general way you can use zstyle to configure a Zsh module:

zstyle <pattern> <style> <values>

The pattern act as a namespace. It's divided by colons : and each value between these colons have a precise meaning.

In the case of the Zsh completion system, the context we saw earlier (what you've already typed in your command-line) is compared with this pattern. If there's a match, the style will be applied.

Don't worry if all of that sounds confusing: bare with me, I'll give some examples soon. For now, let's look a bit closer to the zstyle patterns we can use to configure our completion system.

See man zshcompsys for the list of styles for the completion system (search for "Standard Styles").

General zstyle Patterns for Completion

To configure the completion system, you can use zstyle patterns following this template:

:completion:<function>:<completer>:<command>:<argument>:<tag>

The substring separated with colons : are called components. Let's look at the ones used for the completion system in details:

completion - String acting as a namespace, to avoid pattern collisions with other scripts also using zstyle.
<function> - Apply the style to the completion of an external function or widget.
<completer> - Apply the style to a specific completer. We need to drop the underscore from the completer's name here.
<command> - Apply the style to a specific command, like cd, rm, or sed for example.
<argument> - Apply the style to the nth option or the nth argument. It's not available for many styles.
<tag> - Apply the style to a specific tag.

You can think of a tag as a type of match. For example "files", "domains", "users", or "options" are tags.

For the list of tags, see man zshcompsys (search for "Standard Tags")

You don't have to define every component of the pattern. Instead, you can replace each of them with a star *. The more specific the pattern will be, the more precedence it will have over less specific patterns. For example:

zstyle ':completion:*:*:cp:*' file-sort size
zstyle ':completion:*' file-sort modification

What happens if you set these styles?

If you hit TAB after typing cp, the possible files matched by the completion system will be ordered by size.
When you match files using the completion, they will be ordered by date of modification.

The pattern :completion:*:*:cp:* has precedence over :completion:* because it's considered more precise.

The * replace any character including the colon :. That's why the pattern :completion:*:*:cp:* is equivalent to :completion:*:cp:*. That said, I find the second form confusing: it's not clear what cp is. Is it a command? A function? A tag? In that case it's pretty obvious, but it's not always cp in the pattern. Personally, I always try to use the first form.

You can run zstyle in your shell to display the styles set in your current session as well as their patterns.

Examples of zstyles for the Zsh Completion

All of that is quite verbose and not very self-explanatory, so let's look at more examples. Here's a simple one:

zstyle ':completion:*' verbose yes

This zstyle command is composed of:

A pattern: :completion:*
A style: verbose
A value: yes

There is only one value given here, because the style verbose only accept one. But you can set more than one for some styles. Each of these values would be separated by a space. For example:

zstyle ':completion:*:*:cp:*' file-sort modification reverse

We give here two values to the style file-sort: modification (to order the matches by date of modification) and reverse (to reverse the order). As we saw before, the pattern :completion:*:*:cp:* indicates that we only set the style file-sort to the command cp.

Let's see now a couple of styles we can set to improve our Zsh completion.

For more zstyle, see man zshmodules (search for "zstyle").

Useful Style for the Zsh Completion System

I would recommend creating a new file completion.zsh somewhere and sourcing it directly in your file .zshrc as follows:

source /path/to/my/completion.zsh

Every style defined here should be called after autoloading and calling compinit. If you don't want a separate file, you can throw all this configuration into your .zshrc too.

The Essential

Defining the Completers

Let's first define the completer we'll use for our completion system. Here are some interesting ones:

_complete - This is the main completer we need to use for our completion.
_approximate - This one is similar to _complete, except that it will try to correct what you've typed already (the context) if no match is found.
_expand_alias - Expand an alias you've typed. It needs to be declared before _complete.
_extensions - Complete the glob *. with the possible file extensions.

Note that you can use _expand_alias with the keystroke CTRL+x a by default, without the need to use it as a completer. Simply type one of your alias in Zsh and try to use the keystroke.

You need to set the zstyle completer to define the completer you want to use. The order matter: the completion system will try each of these completer one after the other. The completion stop when some matches are found or when the completer stop it by itself.

For example, here are the completers I use:

zstyle ':completion:*' completer _extensions _complete _approximate

Since we're dabbling into the completion system, the completer _complete_help might come in handy. You can use it as a function you can call with CTRL+x h by default.

When you're not sure why you end up with some matches and not others, you can hit CTRL+x h (for help) before completing your command. It will display some information about what the completion system will do to complete your context.

Caching the Completion

Using a cache for the completion can speed up some commands, like apt for example. Let's add the following in our file to enable it:

zstyle ':completion:*' use-cache on
zstyle ':completion:*' cache-path "$XDG_CACHE_HOME/zsh/.zcompcache"

As you might have guessed, the style cache-path allows you to set the filename and location of the cache.

The Completion Menu

Instead of going through each match blindly to add what you want to your command, you can use a completion menu. For that, you need to set the style menu with the value select as follows:

zstyle ':completion:*' menu select

You can also use this menu only if a precise <number> of matches is found with select=<number>. It's even possible to start the menu selection only if the list of matches doesn't fit the screen, by using the value select=long. Using both values select=<number> and select=long is possible too.

Having the values select and interactive allows you to filter the completion menu itself using the completion system. You can also configure a keystroke to switch this interactive mode when the completion menu is displayed. More on that later in this article.

Adding the value search to the style will allow you to fuzzy-search the completion menu. Again, a keystroke can be configured to have this functionality on demand.

Formatting The Display

Colors and Decoration

Let's now improve the display of our completion menu using the style format.

Each completer can define different sequences (beginning with %) for different tags. For example:

zstyle ':completion:*:*:*:*:descriptions' format '%F{green}-- %d --%f'

As we saw quickly above, a tag is most of the time a type of match. More generally, you can see it as a type of information displayed during completion. Here, the descriptions tag is specific to the format style. It generates descriptions depending on the type of match. For example, if you have files displayed in the completion menu, the description for the tag "files" will be displayed too.

The value of format is used for displaying these descriptions. Here, we use in the style's value the sequence %d which will be replaced by the description itself.

Here's the result for local directories:

If you're using the completer _approximate, you can set the format of the possible corrections too. For example:

zstyle ':completion:*:*:*:*:corrections' format '%F{yellow}!- %d (errors: %e) -!%f'

Here's the result:

To format messages or warnings (for example when no match is found), you can add the following:

zstyle ':completion:*:messages' format ' %F{purple} -- %d --%f'
zstyle ':completion:*:warnings' format ' %F{red}-- no matches found --%f'

We can use the escape sequence %F %f in the style's value to use a foreground color. Here's a summary of sequences you can use:

%F{<color>} %f - Change the foreground color with <color>.
%K{<color>} %k - Change the background color with <color>.
%B %b - Bold.
%U %u - Underline.

For example, you can create a horrible display as follows:

zstyle ':completion:*:descriptions' format '%U%K{yellow} %F{green}-- %F{red} %BNICE!1! %b%f %d --%f%k%u'

Your descriptions are now a piece of art.

Grouping Results

To group the different type of matches under their descriptions, you can add the following:

zstyle ':completion:*' group-name ''

Without this style, all the descriptions will be at the top and the matches at the bottom:

With the value of this style set with an empty string, the matches will be grouped under the descriptions depending on their types:

If you're not satisfied by the order these descriptions are displayed, you can modify it too. For example:

zstyle ':completion:*:*:-command-:*:*' group-order alias builtins functions commands

Here, -command- means any word in the "command position". It means that we want the matches tagged alias to appear before builtins, functions, and commands.

Detailed List of Files and Folders

The style file-list can display the list of files and folder matched with more details, similar to the information you can display with ls -l.

For example:

zstyle ':completion:*' file-list all

Here's the result:

Colorful Completion List

To have nice colors for your directories and file in the completion menu, you can add the style list-colors.

zstyle ':completion:*:default' list-colors ${(s.:.)LS_COLORS}

It will set the value with the content of the environment variable LS_COLORS, normally used by the command ls --color=auto.

You apparently need to have the module zsh/complist loaded, but it worked without it on my system. I describe a bit more the module complist later in this article.

You can configure the colors of any completion menu even further using the environment variable ZLS_COLORS.

To configure the colors, search man zshmodules (search "Colored completion listings").

Smarter Completion System

Squeezing Slashes

By default, the completion system will expand // to /*/. For example, cd ~//Documents will be expanded to cd ~/*/Documents.

Usually, on Unix systems, // is expanded to /. If you prefer this behavior, you can set the style squeeze-slashes to true as follows:

zstyle ':completion:*' squeeze-slashes true

Directory Stack Completion

For the command cd, chdir and pushd, you can use a hyphen - not only for options but for a directory stack entry.

By default, the Zsh completion system will try to complete for a directory stack entry when you hit tab after a hyphen - while using one of these commands. If you prefer completing for an option, you can set the style complete-options to true as follows:

zstyle ':completion:*' complete-options true

If you're interested to learn more about the directory stack, I described a nice way to navigate through it in this article

Sorting Matched Files

You can sort the files appearing in the completion menu as follows:

zstyle ':completion:*' file-sort dummyvalue

If this style is not set (or set with a dummy value like in the example above), the files will be sorted alphabetically. You can use one of these values if you prefer another ordering:

size - Order files by size.
links - Order files depending on the links pointing to them.
modification or date or time - Order files by date of modification.
access - Order files by time of access.
change or inode - Order files by the time of change.

You can also add reverse to the values to reverse the order. For example:

zstyle ':completion:*' file-sort change reverse

If you add the value follow, the timestamp of the targets for symlinks will be used instead of the timestamp of the symlinks themselves.

Completion Matching Control

Setting the style matcher-list allows you to filter the matches of the completion with even more patterns.

For example, you can set this style for the completion to first try the usual completion and, if nothing matches, to try a case-insensitive completion:

zstyle ':completion:*' matcher-list '' 'm:{a-zA-Z}={A-Za-z}'

The completion can also try to complete partial words you've typed with the following style:

zstyle ':completion:*' matcher-list '' 'm:{a-zA-Z}={A-Za-z}' 'r:|[._-]=* r:|=*' 'l:|=* r:|=*'

This style would allow you, for example, to complete the file _DSC1704.JPG if you only try to complete its substring 1704.

The patterns themselves are quite... obscure. If you want a good intellectual challenge, you can look at the manual. I wish you good luck.

See man zshcompwid for patterns for the style matcher list (search for "COMPLETION MATCHING CONTROL").

Completion via Keystrokes

You can bind any completion style to a keystroke instead of using it with the general completion. For that, you need to use the completer _generic.

For example, if you want to expand aliases each time you hit CTRL+a, you can add the following lines:

zle -C alias-expension complete-word _generic
bindkey '^a' alias-expension
zstyle ':completion:alias-expension:*' completer _expand_alias

You can replace alias-expension by the name of your choice. The behavior depends of the completer you use; for expanding aliases, we use _expand_alias. Feel free to use whatever completer you want for your own needs.

The Module complist

I wrote already about the module "complist" in this article. We saw there how to move around in the completion menu using the keys "hjkl":

zmodload zsh/complist
bindkey -M menuselect 'h' vi-backward-char
bindkey -M menuselect 'k' vi-up-line-or-history
bindkey -M menuselect 'j' vi-down-line-or-history
bindkey -M menuselect 'l' vi-forward-char

You can configure many more keystrokes related to the completion menu using these commands:

accept-line - Validate the selection and leave the menu.
send-break - Leaves the menu selection and restore the previous command.
clear-screen - Clear the screen without leaving the menu selection.
accept-and-hold - Insert the match in your command and let the completion menu open to insert another match.
accept-and-infer-next-history - Insert the match and, in case of directories, open completion menu to complete its children.
undo - Undo.
vi-forward-blank-word - Move cursor to next group of match.
vi-backward-blank-word - Move cursor to previous group of match.
beginning-of-buffer-or-history - Move the cursor to the leftmost column.
end-of-buffer-or-history - Move the cursor to the rightmost column.
vi-insert - Toggle between normal and interactive mode. We've seen the interactive mode above.
history-incremental-search-forward and history-incremental-search-backward - Begin incremental search.

For example, I've configured CTRL+x i to switch to the interactive mode in the completion menu:

bindkey -M menuselect '^xi' vi-insert

See man zshmodules to configure complist (search for "THE ZSH/COMPLIST MODULE").

Completion Options

You can also use options to modify the Zsh completion system. To set an option, you need to use the command setopt. For example:

setopt MENU_COMPLETE

Here's a small selection of useful options for configuring the completion system:

ALWAYS_TO_END - Always place the cursor to the end of the word completed.
LIST_PACKED - The completion menu takes less space.
AUTO_MENU - Display the completion menu after two use of the TAB key.
AUTO_COMPLETE - Select the first match given by the completion menu. Override AUTO_MENU.
AUTO_PARAM_SLASH - When a directory is completed, add a trailing slash instead of a space.
COMPLETE_IN_WORD - By default, the cursor goes at the end of the word when completion start. Setting this will not move the cursor and the completion will happen on both end of the word completed.
GLOB_COMPLETE - Trigger the completion after a glob * instead of expanding it.
LIST_ROWS_FIRST - Matches are sorted in rows instead of columns.

See man zshoptions for the completion options (search for "Completion").

Ready to Complete?

The Zsh completion system is a complex beast, but I tried to do my best to simplify it. I hope it didn't cause any headache on your side.

What did we learn together in this article?

The completion system go through a series of completers first. They'll try to find matches depending on the context (the command you've typed) using different completion functions.
The Zsh module "zstyle" allows you to configure settings for a specific Zsh module or widget. Don't let the name style fools you: it can configure way more than visual styles.
You can configure the Zsh completion system using options and the module complist, but using zstyle is the most flexible way to tune it following your craziest wishes.

As you see, the completion system of Zsh is far from being simple. I've covered some of the basics here, but there's much more. Let me know if you want a follow-up article to dive even deeper in this madness.

Related Resources

Zsh Documentation - The Completion System
Zsh Documentation - Visual Effects
Author's configuration for the Zsh completion system
Completions from the prezto framework - Robby Russell, Sorin Ionescu, and contributors

Becoming Mouseless

Do you want to build a Mouseless Development Environment where the Linux shell has a central role?

Switching between the keyboard and the mouse costs cognitive energy. This book will guide you step by step to set up a Linux-based development environment that keeps your hands on your keyboard.

Take the brain power you've been using to juggle input devices and focus it where it belongs: on what you create.

What's The Vim Runtime? A Guide With Examples

Matthieu Cneude — Mon, 06 Dec 2021 10:46:37 +0000

Have you ever wonder what was The Beginning of All Things™?

In other words: have you ever wonder what Vim is doing when you start it? What files are sourced, why, and in what order? If you don't really care, I encourage you to reconsider your position. Knowing what Vim is doing at startup lets you speed it up, overwrites some defaults from the plugins you've installed, or even creates your own filetypes. In short, it gives you even more power to customize Vim to your precise needs.

In this article, we'll look at two main ideas:

What happens during Vim's startup.
What are the runtime paths, what's sourced from there, and when.

This article assumes that you have some advanced knowledge about Vim. If you don't, you can read my series of article to learn Vim from the ground up. More specifically, a lot of knowledge in this article is built on Vim for Expert Users.

Take your headlamp and a good rope, we'll go exploring today many paths in our filesystems.

Vim's Startup

When you start Vim, many things happen behind the curtain before your favorite cursor is displayed on the screen. I describe here a simplified version, trying to underline the most important steps.

Startup's Order

Here's what Vim will do when you run it:

Set the shell options (useful for the command :!, and for Vim's terminal).
Process the arguments given to the CLI. Buffers are created for all files given as arguments.
The value of the environment variable $VIMINIT (an Ex command) is executed.
Source the user's vimrc file.
The value of the environment variable $EXINIT (an Ex command) is executed.
If the option "exrc" is set, Vim will try to find and load a vimrc in the working directory. Keep in mind however that setting "exrc" is not secure.
Source filetypes, filetype plugins, and indent plugins. We'll see them below.
Source syntax highligting scripts.
Source plugin's scripts.
Load runtime's and plugin's scripts from the "after" folder.
Source the shada file (for Neovim) or the viminfo file (for Vim).
Execute the options given to Vim affecting the startup.

Let's add more precisions:

If you want to run more than one command using $VIMINIT, you can separate them with the symbol |.
Your vimrc file can be:
- In Vimscript for Vim (located at $HOME/.vimrc).
- In Lua or Vimscript for Neovim (located in $HOME/.config/nvim/init.vim, or $XDG_CONFIG_HOME/nvim/init.vim if $XDG_CONFIG_HOME is set).
Setting the option exrc can be dangerous: Vim can load potentially unsecure vimrc files you might have downloaded with other files.
You can use the option -u <another_vimrc> when you run Vim in your terminal, to load another vimrc file instead of the default one. You can also use of of these value instead of a vimrc for debug purposes:
- NORC - Don't load any vimrc but load your plugins.
- NONE - Don't load any vimrc nor plugins.
You can run any command using the options -c or +. For example: vim +"set shiftwidth=4|echo 'hello'"

Profiling Vim's Startup

If Vim does terrible things at startup and you don't understand how to fix it, you can use the option -V to have more details about the startup process.

If you're startup is too slow and you want to find the bottleneck, you can profile it with vim --startuptime <file>. It will write every files loaded (with timestamps) in the file <file>.

Special Environment Variables

Two environment variables are only defined when Vim starts:

$VIM - Used to locate various user files.
$VIMRUNTIME - Used to locate various support files for Vim. We'll speak more about the runtime below.

You can look at the value of these variable with the command :echo $VIM for example. If you define these environment variables in your shell, Vim won't overwrite them.

Vim Help

:help startup
:help slow-start
:help 'shell'
:help $VIM
:help $VIMRUNTIME

The Runtime Path

The runtime path is similar to the environment variable PATH in Unix-based systems. Vim will search in these paths to locate and source many different files during startup. To see all of these paths, you can look at the value of the option runtimepath, by running the command :set runtimepath?.

If you use Vim, it's likely that you'll see $HOME/.vim as the first path of the list. If you're a Neovim adept, it will be $HOME/.config/nvim.

The subdirectories of the runtime paths have different meanings according to their names. The Vimscript files inside these directories will be sourced at different times during startup, sometimes in different ways.

Important Runtime Paths

Here are some important runtime paths. Vim will parse them in this order:

For Vim:

$HOME/.vim
$VIM/vimfiles
$VIM/vimfiles/after
$HOME/.vim/after

For Neovim:

$XDG_CONFIG_HOME/nvim (or ~/.config/nvim if $XDG_CONFIG_HOME is not set)
$XDG_CONFIG_HOME/nvim/after

Subdirectories of the Runtime Paths

As I was writing above, the configuration files in the runtime paths contains different things depending of the subdirectory they're in. Here's an overview of the most useful of these subdirectories, and how you can use them to customize Vim even further.

I encourage you to look at the default configuration files to understand the meaning of these subdirectories. They are in the directory set in the variable $VIMRUNTIME. Don't forget that you can only read the value of $VIMRUNTIME after launching Vim.

You can also find examples in my own dotfiles.

ftplugin

The subdirectory ftplugin (for filetype plugin) allow you to load pieces of configuration each time you open a buffer with a specific filetype.

For example, let's say that you need a couple of mappings only for markdown files. You can:

Open a markdown file and verify that the filetype is indeed markdown, by running the command :set filetype?.
Create the file markdown.vim in $HOME/.vim/ftplugin for Vim, or $XDG_CONFIG_HOME/nvim/ftplugin for Neovim.
Add options and mappings only for markdown files in there. For example: setlocal spell will enable spelling each time you open a markdown file.

You need to keep in mind, however, that each time a buffer with the filetype markdown is created, the file ftplugin/markdown.vim will be sourced. As a result, you need to make sure that:

You're resetting any autocommand each time the file is sourced (using an autocommand group and :au!).
Every mapping you're creating have the special argument <buffer>, for example map <buffer> <leader>h echo "hello".
You're using setlocal instead of set to set options.
You're using the -buffer argument for any user command.

If you don't respect these rules, your pieces of configuration won't only be loaded for the buffers with a precise filetype, but for every buffer, globally.

autoload

The subdirectory autoload is also very useful. It lets you load your custom functions when you call them, instead of loading them when Vim starts. As a result, it can significantly speed up Vim's startup. These functions need to begin with the path of the file they're written in, for Vim to know where to find them when you call them.

For example, you can create a new file autoload/general.vim and use it to declare a new custom function MyFunction. This function needs to be called general#MyFunction, because it's declared in the file general.vim. To call it, you can run :call general#MyFunction.

You can also create subdirectories in the directory autoload. For example, you can create the file autoload/path/to/general.vim. In that case, you need to call your function path#to#general#MyFunction, describing the location of the file itself.

I encourage you to put all your custom functions in the "autoload" subdirectory.

syntax

The subdirectory syntax lets you create your own syntax files. They're used for code highlighting for example. You can first create a syntax group matching some regex, and then you link your syntax group to highlight groups. These highlight groups will decide of the text's color displayed.

Vim supports many syntaxes by default (you can see them in the directory $VIMRUNTIME/syntax), I never had to create my own syntax groups. It's however good to know that you can do it in case you need it.

colors

You can create highlight groups in the subdirectory colors. Each file represents a different color scheme. Again, you'll find different examples in the directory $VIMRUNTIME/colors. To change the color scheme of your highlighting, you can use the command :color <my_color_scheme>.

ftdetect

Have you ever dreamed to create your own filetypes? That's great, because Vim is a dream machine. You can create your filetypes in the subdirectory "ftdetect".

For example, if you have a bunch of file with the extension "new" and you want to create a filetype for them, you can:

Create the file ftdetect/new.vim
Write au BufRead,BufNewFile *.new set filetype=new in it.

You can also link some filetypes with other ones. For example, I wanted the files with the extension .yaml.dist to have the yaml filetype. So I've:

Created the file ftdetect/yaml.dist.vim
Written autocmd BufNewFile,BufRead *.yml.dist set filetype=yaml in it.

compiler

In Vim, you can use the command :make to run a specific compiler, depending of the filetype of your current buffer.

Under the hood, Vim will search a file named after the filetype of your current buffer, in the "compiler" directory. For example, if if you want to run the Golang compiler with precise options, you can create the file compiler/go.vim and add whatever you need.

The Directory after

The subdirectory "after" is a special one. You can think of it as another runtime path in the runtime path, where you can add the same subdirectories we saw above.

Everything in the after directory will be loaded after everything else. You can also see it when you look at the order of the most important runtime path at the beginning of this section.

From there, you can override anything you want: indentation, filetype plugins, or even the external plugins you've installed. For example, I can override Lisp indentation by creating the file after/indent/lisp.vim and setting whatever options or variables I want to modify.

I didn't cover the subdirectory indent by the way, but I'm sure you can figure out its purpose.

The Runtime Command

If you want to source manually some files from your runtime path, you can use the command :runtime <file>. For example, if you want to load every files from the colors subdirectory, you can run :runtime colors/**/*.vim.

You can also use :runtime! (with a bang !) to source everything.

Disabling Runtime Files

Finally, if you want to debug one of your runtime files, you can disable some of them by sourcing these specific files:

:source $VIMRUNTIME/ftoff.vim - Disable the detection of filetypes
:source $VIMRUNTIME/ftplugin.vim - Enable ftplugin
:source $VIMRUNTIME/ftplugof.vim - Disable ftplugin
:source $VIMRUNTIME/indent.vim - Enable indentation
:source $VIMRUNTIME/indoff.vim - Disable indentation

Vim Help

:help 'runtimepath'
:help autoload
:help ftplugin
:help syntax
:help new-filetype
:help after-directory

The Startup Has Been Revealed

Now that you know what Vim's doing when it's starting, you can add your pieces of configuration in the good runtime subdirectory for more flexibility. You can also try to speed up your startup if you find it too slow.

What did we see in this article?

Vim load specific configuration files and environment variables at startup
You can use the options -V or --startuptime to get more details regarding Vim's startup.
The environment variable $VIM and $VIMRUNTIME are set during startup.
The subdirectories in the runtime path have a meaning, sometimes affecting directly how they're loaded at startup, and when.
The runtime path subdirectory "after" is a special one: it's a runtime path too (accepting the same subdirectories), and you can overwrite every other loaded config files in there.
The command :runtime allows you to manually load config files from your runtime paths.

I encourage you to experiment with all these configuration files to improve your config. Personally, I find the subdirectories "autoload" and "ftplugin" especially handy.

Learning to Play Vim: A Fun Guide to Learn the Best Editor

How to navigate in multiple files and projects in Vim?
How to debug in Vim?
How to search, find, and replace?

This guide will explain the most useful vanilla functionalities as well as some powerful plugins which will enrich your experience.

Help me make an impact in the Vim world! You can subscribe to the newsletter and tell me everything you want to see in the book. Early bird discount guarantees!

I reply to every email personally, so don't hesitate to ask as many questions as you want. It's always a pleasure to help.

Last but not least: I've also written a book about building your own Mouseless Development Environment, so if you're interested by that too, click on this shiny link.

Do we have to deal with too much complexity?

Matthieu Cneude — Tue, 02 Nov 2021 16:08:52 +0000

I've been coding for 10+ years professionally (20 as a hobby) and something strikes me hard: it seems that our job gets more and more complex.

To be a software engineer, we need to know how databases work, trendy programming languages, the 39020 services proposed by the cloud, containerization and orchestration (docker & kubernetes), the command line, algorithms, good practices... the list goes on. On top, most opinions about software development are just that: opinions. Nobody really agree, we just speak forever about what to do without empirical arguments.

Is it possible nowadays to have one server, put a monolith on it, and maintain it as long as possible?

Is it possible to bring back simplicity in this ocean of tools, practices, workflow, and opinions?

A Vim Guide for Expert Users

Matthieu Cneude — Sat, 16 Oct 2021 16:07:46 +0000

This article is the sixth of the series aimed to teach Vim from the ground up:

You're now in your garden, sitting down in your favorite chair, contemplating a fabulous table full of the tastiest food. While delighted by your fantastic breakfast, a fresh breeze refresh your warm skin in this hot summer day. You hear the little birds singing in unison. The vivid smell of the young grass and the delicate and subtle aroma of your tea fill your nostrils. Everything is in perfect harmony; all your annoyances, problems, or negative thoughts are far away from this pure state of bliss.

Suddenly, you hear a strong and imposing voice from the sky. The clouds move quickly. You're flabbergasted: almost falling from your chair, you recognize the logo of the Sacred Vim formed by the cloud!

The voice, which was only a growl, begins to speak:

"My sweet friend! The time has come. You are now ready to begin the final step of your enlightenment. You will receive the Word of Vim!"

A bright and supernatural light break through the cloud and hit your table. You suddenly understand The Truth of Vim! Quickly, you take a bunch of napkins and begin to write (with some strawberry marmalade) all the knowledge which is given to you at this precise moment.

This article is a transcript of what was called later the Napkins of Power™. It helps customize Vim even further. More precisely, we'll see:

Special arguments you can use for your mappings.
How to create operator pending mappings.
The command :execute and its benefits when combined with the command :normal.
What are autocommands and autocommand groups.
Why and how to create custom functions.
What are user commands and how to use them.
What special strings you can use in your commands.
A complete example to illustrate most of the ideas described in this article.

We've covered many Vim functionalities since the beginning of this series of article, and this new one add even more on these foundations. But it can be daunting to try to add everything to your workflow at once; instead, it's better to add what can help you in your daily workflow.

You can create a virtuous circle: pick a new and useful functionally, use it in your day-to-day workflow, and, when you feel comfortable with it, reiterate.

Are your ready to receive The Expertise? The knowledge of this article is dangerous; use it with care, always to make the world a better place.

Verbose Commands

The more pieces of configuration and plugins you'll add to Vim, the more you'll wonder where they've been created in your config files. The command :verbose can help you: it will output at what line of what file a precise configuration have been declared.

More precisely, :verbose can output the declarations of:

Abbreviations
Options
Mapping
User commands

For example, I've set the option undodir in my vimrc. If I run :verbose set undodir?, I'll get the following output:

Last set from ~/.config/nvim/init.vim line 354

Keeping your configuration well organized and simple is the best strategy. But everything gets messy overtime, especially when experimenting with new configuration or plugins. I don't use verbose often but, when I need it, I'm always happy to have it.

Vim Help

:help :verbose-cmd

Mapping Special Arguments

We've seen the basics of mapping in a previous article, but we didn't speak about the special arguments it can take. Most of them can be used for abbreviations, too.

Here are the most useful ones:

<silent> - Doesn't output the mapping in the Vim command-line. If you want to also drop the output of the command linked to the mapping, add the command :silent.
<buffer> - The mapping's scope is reduced to the current buffer only. These mappings have the priority on the global ones.
<expr> - The mapping executes a Vimscript expression instead of a Vim command.
<unique> - The mapping fails if it already exists. It's useful if you don't want to override any mapping defined previously.
<Cmd> - The mapping can run a command without quitting the current mode you're in.

A special argument should be used as first argument for any mapping or abbreviation. The argument <Cmd> is an exception: it should be used just before the command itself.

To understand how it works, let's take some examples:

:nnoremap <silent> <leader><f6> :source $MYVIMRC<CR> - The command :source $MYVIMRC won't be displayed in the Vim command-line when hitting LEADER F6.
:iab <expr> cdate strftime('%Y-%m-%d') - The Vimscript function strftime is executed when the abbreviation cdate is used.
:inoremap <c-d> <Cmd>delete<cr> - Execute the command :delete without leaving INSERT mode.
:nnoremap <leader><f6> :silent :source $MYVIMRC<CR> - The output of the command :source $MYVIMRC will be dropped, thanks to the command :silent.

We'll see a use case for the special argument <buffer> below.

Vim Help

:help map-arguments

Operator Pending Map

Operators are NORMAL mode keystrokes which need to be combined with motions or text-objects. For example, d, c, or y are operators. I've written about them in the very first article of this series.

Vim lets you set a new motion for all existing operator with the command :onoremap. For example:

:onoremap ic i{

We've created here a new motion ic to use an operator inside curly brackets. You can try it out to delete the content between two curly brackets for example, by:

Placing the cursor inside the curly brackets.
Hitting dic in NORMAL mode.

The motions created with :onoremap always begins where your cursor is. That's why we need to place our cursor inside the curly brackets in our previous example. But it would be even better if we could act on the next curly brackets without worrying where the cursor is. The following command will make this dream a reality:

:onoremap nc :normal! f{vi{<cr>

Let's look at this example more closely:

:normal - Execute keystrokes as if you were in NORMAL mode (see Vim for Advanced Users).
f{ - find the next curly bracket on the line (see Vim for Beginners).
vi{ - Switch to VISUAL mode and select inside the curly brackets.
<cr> - Key notation for the ENTER key (carriage return, see Vim for Intermediate Users).

The operator will be applied on the selection made in VISUAL mode. To illustrate this idea, let's look at the following content:

My super┃line {with curly brackets}

The symbol ┃ represents the cursor position. If you hit the keystroke dnc, you'll delete what's inside the next curly bracket. The result will be:

My super line {}

The cursor will end up on the last curly bracket.

Vim Help

:help omap-info

The Command Execute

Let's look again at the mapping we've set above:

:onoremap nc :normal! f{vi{<cr>

The key notation <cr> is considered a special character when you create a mapping. It works with mappings, but it won't work with the :normal command by itself.

For example, you can try to run the following to replace the next occurrence of "emacs" with "vim":

:normal /emacs<cr>ciwvim

This time, Vim doesn't recognize <cr> as a special character, so the command won't work. To go around this limitation, you can use CTRL+V (see Vim for Adept Users). In that precise case, we would need to hit CTRL+V ENTER while writing our command. We would end up with something like this:

:normal! /emacs^Miwvim

The command :execute can solve our problem in a more elegant way. It lets you execute a command from a string. You can then use string constants for these special characters, all prefixed with a \. Here's an equivalent of our silly example:

:execute "normal! /emacs\<cr>ciwvim"

All the key notations you can use with :map or :abbreviate have their string constant equivalents.

When you give multiple arguments to execute, they're concatenated into one string and separated with spaces. If you don't want the spaces, use a dot instead. For example:

:execute 'echo "this" "is" "a" "str"."ing"'

The output will be: "this is a string".

Vim Help

:help execute
:help expr-quote

Autocommands

Basics

Autocommands can automatically run a command when a specific event happens. More precisely, it adds a command to a list of command linked to a precise event. When this event is fired, every command of the list of commands are executed.

An event can be opening Vim, reading any file, or writing a markdown file for example.

Here are the basics to manipulate autocommands:

:autocmd <event> <pattern> <cmd> or :au <event> <pattern> <cmd> - Add the command <cmd> to the list of commands executed automatically when the event <event> is fired. The pattern <pattern> filter the files where the autocommand should be applied.
:autocmd <event> or :au <event> - Output the list of commands executed when the event <event> is fired.
:autocmd! <event> or :au! <event> - Delete the list of autocommands of the event <event>.

To clarify all this jargon, here are some examples:

:autocmd BufWrite * echom "Write..." - Output "Write" each time any file is saved. The wildcard * means "every file".
:autocmd BufNew *.md echom "Read..." - Output "Read" each time a new markdown buffer is created. Unsurprisingly, The pattern *.md means every filename finishing with .md.

After running these commands, you can try to write a file (command :w) to see if it works. If you don't see the message in the command-line, run :messages to display them.

Multiple Events and Patterns

You can also create an autocommand with more than one event or pattern, separated with a comma. For example:

:autocmd BufNew,BufWrite *.md,*.js,*.php echom "Create or write md, js, or php..."

The command echom "Create or write md, js or php..." will run when a markdown, JavaScript, or PHP buffer is created (BufNew) or saved (BufWrite).

Note that the pattern can be a bit different depending on the events you listen to. For a description of all events available, see :help autocommand-events.

Finally, if you want the scope of the autocmd to be limited to the current buffer, you can use the special pattern <buffer>.

Autocommand Groups

Why Using Autocommand Groups?

As the name indicates, an autocommand group is a group of one or more autocommands. When you create an autocommand as we did above, it's added automatically to a default autocommand group without a name. You can create autocommand groups with names and add autocommands to it, too. You can think of it as namespaces for autocommands.

To understand this concept, let's see where it's useful to use autocommand groups. As we saw, each time you create an autocommand, it's added to the list of command triggered when a specific event occurs; even if this command is already part of the list.

Let's look at an example. First, let's add this autocommand to your vimrc:

autocmd BufWritePre * echom "Write..."

You can then try the following:

Source your virmc twice (by running :source $MYVIMRC twice).
Run :autocmd BufWrite.

This last command will output the list of commands executed when the event BufWrite is triggered. It will output something like this:

BufWritePre
    *         echom "Write..."
              echom "Write..."

The command echom "Write..." appears two times in the command list for the pattern *. As a result, each time the event occurs on any file, the command will run two times.

More often than not, we want to add our commands to the command list only once. Otherwise, it will impact performances each time the event is fired and, if the command is not idempotent, nasty bugs will pop up.

Using autocommand groups can solve this problem. Here are the basic commands you can use to manipulate these groups:

:augroup - Output all autocommand groups.
:augroup <name> or aug <name> - Call a new autocommand group named <name>. All autocommands created after this command will be part of the group.
:augroup! <name> or aug! <name> - Delete the group named <name>.
:augroup END - End the autocommand group. If you define an autocommand after this one, it won't be part of the group.

As always, here's an example:

:augroup messages
:autocmd BufWrite * echom "Write..."
:augroup END

The autocmd is now part of the autocommand group messages.

By itself, it doesn't solve our problem. If you add the three lines above in your vimrc and source multiple times, the autocommand in the group messages will be added to the autocommand list each time.

We saw above that you can delete autocommands with the command au! <event>. This will work if the autocommand is not in a named group. If you run au! BufWrite for example, it will delete every autocommand in the nameless autocommand group (the default one), but not the one in the group messages we've created above.

To solve our problem, we could delete every autocommand belonging to the group messages after creating the group itself:

:augroup messages
:au! messages
:autocmd BufWrite * echom "Write..."
:augroup END

If these lines are in your vimrc and you source it three times, every autocommand will be deleted from the group messages each time, before being added again. In short, our problem is solved: the command echom "Write..." will always appear once and only once in our list of command.

When we use au! between the initialization of the group (augroup messages) and the end of the initialization (augroup END), we don't have to indicate the name of the group. Vim will understand, in that case, that we want to delete every autocommand of the group declared just before. In short, the following commands are equivalent to the ones above:

:augroup messages
:au!
:autocmd BufWrite * echom "Write..."
:augroup END

Redeclaring an autocommand group won't recreate it, but it will add autocommands in the existing group instead. For example, you can do something like that:

augroup vimrc
au!
augroup END

augroup vimrc
    autocmd BufWrite * echom "Write..."
augroup END

Two things happen here:

A group vimrc is declared.
Autocommands are declared and added to the group vimrc.

In short, every autocommand added to the group vimrc is merged with the command :au!. You can even do better: when you create an autocommand, you can add it to an existing group directly. To do that, you can indicate the name of the group just between the command autocmd and the event name as follows:

augroup vimrc
au!
augroup END

autocmd vimrc BufWrite * echom "Write..."

You're now able to reload your vimrc as much as you want, your autocommands will only appear once in the autocommand list. It applies for other sourced files too, like the ones you might have in your folder ftplugin for example.

Ignoring Events

If you want to run a command without firing any event, you can use the command :noautocmd. To take our previous example, if you want to ignore the event BufWrite when running the command :w, you can run the following:

:noautocmd w

Vim Help

:help autocmd
:help autocommand-events
:help autocmd-events-abc

Custom Functions

Writing custom functions for Vim to make your craziest dreams come true should be the goal of any Vim Follower out there.

It's true that many functions are already available on the infinite Internet. You can simply copy and paste them without worrying how they work. That said, knowing the basics of Vimscript functions can allow you to adapt them to your needs.

Checking Existing Functions

Let's see first how you can display the functions already available:

:function or :fu - List all declared function.
:function /<pattern> or :fu /<pattern> - Filter all declared functions with the pattern <pattern>.

Creating Or Copying Functions

Looking at a simple function will help us understand how they work. Here's one:

function DeleteTrailingWS() abort
    normal mz
    %s/\v\s+$//ge
    normal `z
endfunc

This function delete trailing whitespaces in a whole buffer. Let's look at it in more details:

function - Keyword to declare a function. You can add a bang (function!) to overwrite a previously declared function with the same name.
DeleteTrailingWS - Name of the function. It should always begin with an uppercase letter.
abort - Stop the function when an error occurs.

Be careful if you use function! (with a bang): you might overwrite a function from one of your plugin. It can open the door to random bugs difficult to fix.

If you look at the body of our function, they are simply Vim commands. They are executed in order:

normal mz - Save the cursor position using the mark z.
%s/\v\s+$//ge - Delete every whitespace in the current buffer, using the substitute command.
- % - Range for the whole buffer.
- \v - Use the very magic mode.
- \s - Represent any whitespace.
- Flag e - Doesn't output an error if the search pattern fail.
normalz - Go back to the mark z (the cursor position when the function was invoked).

I've already written about all these commands in Vim for Advanced Users. For more details about Vim regexes, see Vim for Adept Users.

You can then call the function using the command :call as follows:

:call DeleteTrailingWS()

You can also create a new mapping for some of your function if you want to:

nnoremap <leader>ds call DeleteTrailingWS()

Autoloading Functions

You can create function in your vimrc directly as we did above, but it might create some problems. Imagine that the name of the function conflict with a function from one of your plugin: the bugs occuring can be difficult to debug.

Additionally, all the functions declared in your vimrc will be automatically loaded when you open Vim, even if you never use them. It would be more efficient to load them when you call them the first time.

It's where the autoload folder comes in handy. This folder is located in the Vim's runtime paths.

When Vim needs to find something, it will look at the Vim's runtime paths. The folder containing your vimrc is one of these paths, for example. To display all of them, you can run the command :set runtimepath?.

The folder autoload is one of these paths too. Any function created in this folder will have namespaces, and they will be loaded on demand. Exactly what we want!

You can create the autoload folder where your vimrc is. Then, you can create Vimscript files in there; the name of the file will be the namespace for your functions.

For example, you can create the file "general.vim" in the autoload folder. Then, you can write in the file the following function:

function general#DeleteTrailingWS() abort
    normal mz
    %s/\v\s+$//ge
    normal `z
endfunc

When you call the function with :call general#DeleteTrailingWS(), Vim will:

Look inside the autoload directory for a file called general.
Search for a function called DeleteTrailingWS inside this file.
Load and execute the function.

Additionally, you can easily display every function for the namespace <namespace> by running the command :function /<namespace>. For example, if you want to display all the functions for the namespace general, run :function /general.

For more fine-grained namespaces, you can add sub-directories in the autoload directory. For example, you can create the following:

autoload/my/super/namespace.vim

Then, you'll need to add the namespace "my#super#namespace" to the functions you create in the file namespace.vim. For example:

function my#super#namespace#DeleteTrailingWS() abort
    normal mz
    %s/\v\s+$//ge
    normal `z
endfunc

Vim Help

:help functions
:help autoload-function
:help call
:help 'runtimepath'

User Commands

Now that we're able to create our own functions, what about increasing our power with our own custom Vim commands? We'll then be able to run these user commands using the COMMAND-LINE mode, like any other command.

Basics

Like functions, custom user commands should always begin with an uppercase letter, to differentiate them from Vim's built-in commands.

Continuing our ritual, here are three useful commands to manipulate user commands:

:command or :com - Output all user commands.
:command <command> or :com <command> - Output all user commands starting with <command>.
:command <attributes> <name> <cmd> or :com <attributes> <name> <cmd> - Define a new user command with the name <name> running the command <cmd>. The attributes <attributes> indicate the number of arguments (among other things).

Similarly to custom functions, you can add a bang when you declare a command (:command!). In that case, if a command already exists with the same name, it will be overwritten.

Attributes for User Commands

There are four different categories of attribute you can use when creating a user command:

Argument handling
Range handling
Completion behavior
Special cases

We'll only cover the most important one in this article: the argument handling. It allows us to specify the number of argument a user command can take, with the attribute -nargs:

-nargs=0 - No argument allowed (default).
-nargs=1 - One argument is required.
-nargs=* - Any number of arguments allowed.
-nargs=? - 0 or 1 argument allowed.
-nargs=+ - One argument or more are required.

To indicate where the arguments should be inserted in the command, you need to use the placeholder <args>. For example, you can write the following in your vimrc:

function IScream(content)
   echom toupper(a:content)
endfunction

command -nargs=1 Scream call IScream(<args>)

As you can see, you can call the arguments of a function in its body using a:<arg_name>. To try your new user command, source your vimrc with :source $MYVIMRC and run the following:

:Scream "hello"

When a user command call a function which can take multiple arguments, you need to separate them with whitespaces and use the placeholder <f-args> instead of <args>.

If there is only one argument allowed, Vim will consider the whitespace as part of the argument itself.

Finally, if you need your user command to be only available in the current buffer, you can also add the attribute -buffer. It's mandatory if you create user commands in your runtime folder ftplugin.

Vim Help

:help user-commands

Special Strings for Vim Commands

Let's now look at special strings you can use in COMMAND-LINE mode. These placeholders will be replaced under the hood with their representations. Here's a list of the most useful ones:

% - Relative path of the current file.
<cword> - Word under the cursor.
<cWORD> - WORD under the cursor.
<cfile> - Filepath under the cursor.
<afile> - File open in the buffer when executing autocommands.
<sfile> - Filename of sourced file when used with command :source.

You can also use the following with %:

:p - Output the absolute path instead of the relative one. Also expand the tilda ~ to the home directory.
:. - Make the file path relative to the working directory.
:~ - Make the file path relative to the home directory (if possible).
:h - Keep the head of the file path (remove the last element).
:t - Keep the tail of the file path (remove everything except the last element).
:r - Keep the root of the file name (remove its extension).
:e - Remove everything except the extension of the filename.
:s?pat?sub? - Substitute the first occurrence of "pat" with "sub".
:gs?pat?sub? - Substitute all occurrences of "pat" with "sub".

These special strings only work when a command expects a filename as argument; as a result, it makes this functionality quite limited. Fortunately, You can use the function expand(<special_string>) to expand these placeholders in any command.

For example, you can try to run the following:

:echom expand("%")
:echom expand("%:p")
:echom expand("<cword>")

Here's a more useful example we already saw in the article Vim for Advanced Users:

nnoremap gX <silent> execute
            \ "!xdg-open" expand('%:p:h') . "/" . expand("<cfile>") " &"<cr>

You should now be able to understand this command:

<silent> - The mapping won't appear in the command line when used.
execute - Execute a string as a Vim command.
expand('%:p:h') - Output the head of the absolute path.
expand("<cfile>") - Output the filepath under the cursor in the current buffer.

In short, this mapping will open the relative filepath under the cursor using the CLI xdg-open.

This command shouldn't take more than one line, but the backslash \ allows us to write it on two lines for a better readibility. Its fancy name is "line continuation symbol".

If you're used to write shell scripts, remember that the line continuation symbol is not at the end of the line, but at the beginning of the next one.

Vim Help

:help cmdline-special
:help line-continuation

A Complete Example

Let's summarize most of what we saw in this article with a final example. We want to:

Create the user command DevDocs. This command will automatically open the website https://devdocs.io/ and search the word under the cursor.
Map the command to <leader>D in NORMAL mode. This mapping will be available for python, ruby, JavaScript, go, html, and PHP filetypes.

Here's a possible solution:

augroup vimrc
    autocmd!
augroup END

command! -nargs=? DevDocs call system('xdg-open https://devdocs.io/#q=<args>')

autocmd vimrc FileType python,ruby,javascript,go,html,php nnoremap <buffer><leader>D execute "DevDocs " . expand('<cword>')<CR>

We first declare an autocommand group vimrc.
We declare the user command DevDocs, accepting 0 or 1 argument.
We declare an autocommand linked to the event FileType. We indicate the filetypes which will trigger the autocommand.
We use the special argument <buffer> to make the mapping only available in the current buffer. Without that, the mapping would be available in every buffer regardless of the filetype.
The autocommand use expand('<cword>'), which is replaced by the word under the cursor.

The event FileType can be useful to assign precise mappings to a whole range of filetypes. We need, with this event, to give filetypes as autocommand patterns (like python or ruby for example). Remember that you can output the filetype of the current buffer with :set filetype?.

The binary xdg-open is only available for Linux-based systems. If you want the autocommand to work on macOS too, you can use the following:

command -nargs=? DevDocs call system('type open &>/dev/null && open https://devdocs.io/#q=<args> || xdg-open https://devdocs.io/#q=<args>')

The autocommand verifies if the binary open exists (for macOS) and, if it doesn't, it uses xdg-open.

Take Control of Your Vim Destiny

Creating your own functions, commands, and mapping for tedious operations will help you focus on more important tasks. Additionally, you'll bring more efficient in your whole workflow. How great is that?

Let's summarize what we saw in this article:

You can use the command :verbose to output where an abbreviation, option, mapping, or user command, is defined.
Special arguments are available for your mappings to extend its power.
Operator pending map allow you to define motions for operators.
The command :execute can execute a string as if it was a command.
Autocommands can run a defined command when an event is triggered.
You can use autocommand groups to organize your autocommand.
Autocommand groups are mandatory if you don't want to add multiple times the same command in the autocommand list. It can happen when a Vimscript file is loaded multiple time (like your vimrc).
Custom functions can execute numerous Vim commands one after the other.
It's better to autoload custom functions to be able to use namespaces and for Vim to start quicker.
User commands are commands you can personalize for your own needs.

A last tip: if you want the list of Vimscript functions you can use, split in different category (like "Variables" or "Date and Time"), you can look at :help function-list.

Learning to Play Vim: A Fun Guide to Learn the Best Editor

How to navigate in multiple files and projects in Vim?
How to debug in Vim?
How to search, find, and replace?

This guide will explain the most useful vanilla functionalities as well as some powerful plugins which will enrich your experience.

Help me make an impact in the Vim world! You can subscribe to the newsletter and tell me everything you want to see in the book. Early bird discount guarantees!

I reply to every email personally, so don't hesitate to ask as many questions as you want. It's always a pleasure to help.

Last but not least: I've also written a book about building your own Mouseless Development Environment, so if you're interested by that too, click on this shiny link.

A Vim Guide for Veteran Users

Matthieu Cneude — Tue, 17 Aug 2021 15:10:46 +0000

This article is the fifth of the series aimed to teach Vim from the ground up:

Can you picture an adventurer, going deeper and deeper into a mysterious cave, knowing what's waiting for her (mostly rocks and bats) but at the same time wishing to be surprised by some rare gems and abandoned treasures?

This is how I feel when I'm adventuring deeper and deeper into Vim's world. I know that I'll always find something valuable: a new keystroke, a fierce command, or a little subtlety, increasing the control I have over my favorite editor.

Over the last few articles, I tried to guide you in the Vim caves helping you discover the finest diamonds. We'll continue this endeavour in this article by exploring the following:b

Some keystrokes used for completion in INSERT mode.
Useful keystrokes we can use in COMMAND-LINE mode.
What are abbreviations and how to use them.
How to save the global options and mappings of the current Vim instance into a file.
How to save and restore Vim sessions.
What's the purpose of the viminfo file (or the shada file for Neovim).
How to redirect Vim command output into a file or a register.

Take your backpack, your headlamp, your rope, and let's go spot some new treasures for the mind.

Completion in Insert Mode

We have already looked at useful keystrokes in INSERT mode in the last article. There are other interesting ones you can use for completion. These keystrokes are all prefixed with another one, CTRL+x.

Vim needs to be compiled with the +insert_expand feature for these keystrokes to work.

The first couple of them allow you to scroll up and down your buffer without leaving INSERT mode:

CTRL+x CTRL+y - Scroll up
CTRL+x CTRL+e - Scroll down

The others are only useful if you want to complete what you've already typed. Here are, in my opinion, the most useful ones:

CTRL+x CTRL-l - Complete a whole line from the content of one of your buffer.
CTRL+x CTRL-f - Complete the filepath under the cursor. It expands environment variables if it contains a filepath too.
CTRL+x s - Complete with spelling suggestions.
CTRL+x CTRL+v - Complete with the command line history.
CTRL+x CTRL+i - Complete with the keywords in the current and included files. These files are in the option path.

You can also use the completion from a thesaurus or a dictionary, or even the omni-completion to auto-complete programming languages. If you want to dive deeper in these ideas, Vim's help is waiting for your curiosity to be unleashed.

Vim Help

:help i_CTRL-x
:help ins-completion
:help compl-omni-filetypes

Keystrokes in Command-Line Mode

There was a lot of love poured into INSERT and VISUAL mode keystrokes in this series of article. What about COMMAND-LINE mode? It's time to pay some respect to this old chap.

First, some good news: The keystrokes in INSERT mode will mostly work in COMMAND-LINE mode too. In one sentence, we already have a bunch of keys we can play with. How sweet is that?

Copying From the Buffers to the Command-Line

Did you ever have the urge to copy something from your buffer in COMMAND-LINE mode? The boring way would be to come back to your buffer in NORMAL mode, copy what you want in a register, come back to COMMAND-LINE mode, and spit what you want. Like in INSERT mode, you can use CTRL+r followed by the name of a register to copy its content in COMMAND-LINE mode.

Or you can use these keystrokes:

CTRL+r CTRL+f - Copy the filename under the buffer's cursor.
CTRL+r CTRL+w - Copy the word under the buffer's cursor.
CTRL+r CTRL+a - Copy the WORD under the buffer's cursor.
CTRL+r CTRL+l - Copy the line under the buffer's cursor.

Search

If the option incsearch is set, it will highlight the search you're doing while typing it. That's not all: you'll also be able to use CTRL+g and CTRL+t to go through every match without leaving COMMAND-LINE mode.

Multiple Commands on One Line

This one is not a keystroke but it can be useful nevertheless. You can run more than one command in one line if you separate them with a pipe. For example:

:set filetype?|echo "it's done!"

This won't work if the command accepts a pipe as an argument. For example: :!ls | wc.

Vim Help

:help Command-line-mode
:help ex−edit−index

Abbreviations

Let's now look at something totally different: Vim abbreviations. This feature can replace a word with another one automatically after pressing SPACE. You can use abbreviations in INSERT and COMMAND-LINE modes.

For example, if you often misspell "the" with "teh" (it happens to the best of us), you can automatically replace "teh" with "the".

Let's see the commands you can use to output and manipulate abbreviations:

:abbreviate or :ab - List all the abbreviations created.
:abbreviate <string> or :ab <string> - List abbreviations beginning with the string <string>.
:abbreviate <string> <replacement> or :ab <string> <replacement> - Create a new abbreviation for every mode.
:unabbreviate <ab> or :una <ab> - Remove the abbreviation <ab>.
:abclear or :abc - Remove all abbreviations.

When you list abbreviations using :ab, you'll see different letters in the first column of the output. They indicate in what mode you can use them:

i for insert mode.
c for command line mode.
! for both.

If you want to list or create abbreviations for one precise mode, you can use i or c at the beginning of the command:

:iab - List every abbreviation available in INSERT mode.
:cab <string> <replacement> - Create a new abbreviation only available in COMMAND-LINE mode.
iabc - Remove all abbreviations in INSERT mode.

It works a bit differently if you decide to use the long forms of the commands. For example, :iabbreviate won't work; instead, use "abbrev", like in :iabbrev, :cabbrev, or :iunabbrev.

You can use abbreviations for different purposes as shown in these different examples:

:iab @@ myemail@email.com - Insert your email
:iab BTW by the way - Expand an acronym
:iab sign Jane Doe™<cr>jane@doe.com - Insert your signature
:iab <expr> cdate strftime('%Y-%m-%d') - Insert the current date
:cab uuidgen r! uuidgen - Insert a UUID when running the command :uuidgen

Finally, if you've typed an abbreviation but you don't want to expand it when hitting SPACE, use CTRL+v and then SPACE instead.

Vim Help

:help abbreviations

Saving Vim's Options and Mapping in a File

Let's now go into the crux of this article: persisting your current options, keystroke mappings, or your whole sessions into a file.

If you were experimenting in Vim by changing a bunch of options and mappings but you don't remember what exactly, or if you wonder what command run when you hit a keystroke, you can output every option and global mappings (different from Vim's defaults) into a file.

To do so, you can run the command :mkexrc <filepath> or :mk <file> to output these options and mappings into the file <filepath>. If you use a relative path, it will be relative to the working directory you're in (to display it, use :pwd).

You can also add a bang ! to :mk to overwrite a file. For example, :mk! myfile will create the file myfile or overwrite it if it already exists.

From there, you can open the file you've written and copy whatever options or mapping you want in your vimrc. You can also source the file directly into another instance of Vim with the command :source <filepath>.

I've already written about the :source command in Vim for Intermediate Users in order to reload your vimrc. You can source any file containing some Vimscript with this command.

Vim Sessions

If you want to save and restore much more than options and global mappings, you can use Vim sessions.

Creating and Loading a Session

A session is constituted of all the views (a view is a collection of settings attached to a window) and the global settings of your current Vim instance.

Wouldn't it be fun to save your session in a file and open it later in another Vim instance? It would allow you to recover all your buffers, windows, and local settings! Life would be even more colorful then.

To do so, you can use the commands :mksession <filepath> or :mks <filepath>. You can then source the session in another instance of Vim with :source <filepath>.

As usual, you can add a bang ! to the command to overwrite the file if it already exists.

For example, here are some mappings I've written in my vimrc to manipulate sessions:

nnoremap <leader>ss :mksession! $VIMCONFIG/sessions/ - Mapping to save a session
nnoremap <leader>sl :source $VIMCONFIG/sessions/ - Mapping to load a session

Each time I press LEADER ss, I can type the name of the session and save it. All my sessions are gathered in the same directory.

Fine Tuning Vim's Sessions

The option sessionoptions contains a bunch of values separated with comma ,. These values will decide what will be saved in your session file and, as a result, what will be restored when the file is sourced.

Here are the most interesting values you can use:

blank - Save windows containing buffers without names.
buffers - Save hidden and unloaded buffers. Only buffers in windows are restored without this value.
curdir - Save the current directory.
folds - Save folds information.
globals - Save the global variables beginning with uppercase and containing at least one uppercase in their names.
help - Save the help windows.
localoptions - Save local options and mappings to a window or a buffer.
options - Save all options and mappings.
resize - Save the size of your windows.
tabpages - Save all the tab open. Without this value, only the current tab is saved.
terminal - Save the terminal window.

Note that the default global options or global mappings are not saved in a session file. It means that if you have an instance of Vim open and you've already modified some default values, loading a session won't reset them to their defaults.

Vim Help

:help :sessionoptions
:help views-sessions
:help :mksession

The files viminfo or shada

Vim also saves information automatically into a file when you close the editor. This file is different depending on what you're using:

For Vim, it's the viminfo file.
For Neovim, it's the shada file (for shared data). It can't be a symlink.

On Unix systems (Linux and macOS), each file can be found at these file paths:

viminfo - $HOME/.viminfo
shada - $XDG_DATA_HOME/nvim/shada/main.shada

You can change the name for your shada or viminfo file by setting the value of shadafile or viminfofile respectively. You can also set the value NONE if you don't want to use these files.

Each time you close a Vim instance, you can save the following:

The command-line history.
The search history.
The input-line history.
Contents of non-empty registers.
Lowercase marks for several files.
File marks (uppercase marks).
Last search or substitute pattern.
The buffer list.
The global variables (only if they're different from the default values).

You can configure what to save more precisely by changing the values of the following option:

For Vim, the option viminfo.
For Neovim, the option shada.

The option's values are separated with a comma ,. Here are the most useful:

! - Save and restore global variables (their names should be without lowercase letter).

You can add a number after each of these values:

< - Specify the maximum of lines saved for each register. All the lines are saved if this is not included. If you don't want to save your registers, use the value <0.
% - Save and restore the buffer list. You can specify the maximum number of buffer stored with a number.
' - Specify the maximum number of marked files remembered. It also saves the jump list and the change list we saw in a previous article.
/ and : - Number of search patterns and entries from the command-line history saved. The option history is used if it's not specified.
f0 - Don't store any uppercase mark.
s - Specify the maximum size of an item's content in KiB (kilobyte).
- For the viminfo file, it only applies to register.
- For the shada file, it applies to all items except for the buffer list and header.

For example, :set shada=!,'100,<50,s100 save global variables, a maximum of 100 files marked, a maximum of 50 lines per register, and a maximum of 100Kib for each item. The equivalent for vim would be :set viminfo=!,'100,<50,s100.

Vim writes the viminfo file when you close the editor, and it's loaded when you open a new instance. That said, you can also output, save, or load the file with these commands:

For Vim:

:oldfiles or :ol - Display all marked files stored in the viminfo file.
:rviminfo or :rv - Read the viminfo file.
:wviminfo or :wv - Write the viminfo file.

For Neovim:

:oldfiles or :o - Display all files with a mark stored in the shada file.
:rshada or :rs - Read the shada file.
:wshada or :ws - Write the shada file.

Add a bang to these commands (:rv! or :rs! for example) to allow everything set in your file overwriting everything in your current Vim instance.

The viminfo file is written in its one dialect. You can modify it directly if you feel even more adventurous. The shada file use the messagepack format.

Vim Help

:help viminfo
:help 'viminfo'
:help 'viminfofile'

Neovim Help

:help shada
:help 'shada'
:help 'shadafile'

Redirections

What if you want to save the output of Vim commands in a file or in a register? The command :redir is here to fulfill your desires. More precisely:

:redir > <file> - Write every command's output to the file <file>.
- Use :redir! (with a bang !) to overwrite the file.
- Use >> instead of > to append to the file.
:redir @<reg> - Write every command's output to the register <reg>.
:redir @<reg>>> - Append every command's output to the register <reg>.
:redir => <var> - Write every command's output to the variable <var>.
:redir END - End the redirection.

For example, if you want to set the variable sessionoptions in your .vimrc with the current value, you can do something like that:

:redir >> $VIMCONFIG/.vimrc
:set sessionoptions?
:redir END

After running :redir END, nothing more will be written into the file. You can only have one redirection going on, so declaring a new one will automatically close the previous one.

Let's look at another example. You can run the following to append the declaration of the option sessionoptions with the current value:

:redir @+>>
:set sessionoptions?
:redir END

Using the uppercase version of the named registers (the ones from a to z) will append the output of Vim's commands without the need to use >>. For example, :redir @A.

A final example, using a variable to store the output of the command this time:

:redir => my_var
:set sessionoptions?
:redir END
:echo my_var

Vim Help

:help :redir

Filtering Commands Output

It's nice to be able to save command output in Vim, but what if we want to filter them?

To do so, we can use the command :filter /<pattern>/ <cmd>: it will filter the command <cmd> according to the pattern <pattern>.

Let's take some examples:

:filter /content/ buffers - Only output the buffers with part of the filepath matching content.
:filter /archives/ oldfiles - Only output the marked files with part of the filepath matching archives. As you can see, the pattern doesn't have to match the whole line, only part of it. You can add a bang ! to the command filter (filter!) to inverse the match, that is, output everything which doesn't match the pattern.

This command doesn't work with all Ex commands. For example, it won't work with the command :register.

Vim Help

:help :filter

Next Step: Vim for Expert Users

What did we learn in this article?

You can use a new INSERT mode submenu with the keystroke CTRL+x followed by another keystroke.
You can use most keystrokes for INSERT mode in COMMAND-LINE mode. You can also use CTRL+r with a second keystroke to copy what's in your buffer directly in your command-line.
Abbreviations are meant to stop beating your keyboard because you can't stop doing the same spelling mistakes. It's not the only use case; the sky's the limit.
You can source any file containing Vimscript commands using the command :source.
You can save the current settings of your Vim instance with :mk <file>.
You can also save your current Vim session with the command :mks <file>.
Every time you close Vim, your viminfo file is written (if the viminfo options is not empty). If you use Neovim, the viminfo file is replaced with the shada file.
Every time you open Vim, the viminfo file (or shada file) is read.
You can use the redirection (command :redir) to persist in files or registers the output of the different Vim commands.
You can filter the output of some Vim commands using the command :filter.

Throughout this series, we've seen many functionalities The Glorious Vanilla Vim offers us on a tasty platter. But the real power of Vim lies in its flexibility. That's why in the next article, Vim for Expert Users, we'll become Apprentice Gods (or Goddesses) and look at different ways to shape our Vim world according to our deeper needs.

Becoming Mouseless

Do you want to build a Mouseless Development Environment where the Linux shell and Vim have a central role?

Switching between the keyboard and the mouse costs cognitive energy. This book will guide you step by step to set up a Linux-based development environment that keeps your hands on your keyboard.

Take the brain power you've been using to juggle input devices and focus it where it belongs: on what you create.

A Vim Guide for Adept Users

Matthieu Cneude — Sun, 11 Jul 2021 09:20:55 +0000

This is the fourth part of this series to learn Vim from the ground up:

Dave, your colleague developer, just read Vim for Advanced Users before going to bed. He wonders what's next: Vim for Unstoppable Creators? Vim for Godlike Developers? On these reassuring thoughts, he falls into a deep sleep.

He wakes up suddenly: he's not in his bed anymore! He's now in a massive room, probably a church, where hundreds of masked and hooded figures sing in unison. He's sitting in front of a desk, like everybody else. The smell of incense is powerful and, even if he's in an open space, it's warm and comfy.

The singing stop. A tall individual, hooded too, wearing a wonderful golden robe and a silver mask, stands on a stage in front of the gathering, not far from you.

She begins to speak:

"My friends! You've been chosen as the most fervent follower of our Guide, the powerful Vim! For the Glory of Vim!"

Everybody repeats the words of the hooded woman: "For the glory of Vim!"

She continues. "Today, when your formation will be done, I will knight you Glorious Vim Adept™. You'll be then borderline unstoppable, but beware! The Power needs to be used for The Greater Good!"

"The Greater Good!", repeats the gathering.

"My friends! Look under your desk. You have the Vector of Power, a Lenovo x220, with The Highness Vim running on it. Take it, it will be the link between you and my unbearable teaching. You'll become soon The Builders of the New Sweet World!"

"The Builder of the New Sweet World!", repeat the others.

There is indeed a computer under Dave's desk. He takes it and begins to read the table of content blinking before his excited eyes:

How to manipulate multiple quickfix and location lists.
What are digraphs and how to use them.
Useful keystrokes in INSERT mode.
Useful keystrokes in VISUAL mode.
Vim regular expressions.
Using shell commands in Vim.
Folding content.

Suddenly, the course itself begins to poor in Dave's brain! Frenetically, his fingers type in Vim everything his telepathic connection is showing him.

This article is a transcript of what Dave experienced in his own mind. It's not as powerful as telepathic teaching, but here are some tips and tricks you can use to remember what you'll read:

Try in Vim what's described while reading the article.
Create your own cheatsheet. It will help you remember all the shenanigans and you can even refer to it in your daily work.
Pick and choose new concepts from this article from time to time and try to integrate them to your own workflow.

What did Dave learn in his adventure?

Multiple Quickfix and Locations Lists

We saw in the last article how useful the quickfix lists are. But it didn't explain how to access the stack of these lists.

Indeed, each time we create a quickfix list, it will be added to a stack and the previous one will be pushed down. This stack can contain 10 quickfix lists maximum; each time you create a quickfix list, the 10th on the stack will disappear forever in the Forgotten Pit of Quickfix Lists.

This is true for location lists too: you'll have a stack containing 10 of them per window open.

You can use these commands to manipulate this stack:

:chistory or :chi - Show the stack of quickfix lists and point out the current one.
:colder or :col - Change the current quickfix list to the next older one.
:cnewer or :cnew - Change the current quickfix list to the next newer one.

You can add a count after :colder and :cnewer. For example, running :colder 3 will select the quickfix list which is 3 positions down the current one.

To change your current location list, you can replace the first letter c (quickfix) of the commands above with a l (location).

Vim Help

:help quickfix-error-lists

Digraphs

Vim allows you to insert special characters easily using digraphs. To insert these special characters, you need to know their representations, always two "normal" characters.

Here are some useful commands:

:digraphs or :dig - Display the digraphs available.
:digraphs <char1><char2> <number> - Create a new digraph represented with the characters <char1><char2>. The <number> is its decimal representation (Unicode character).

You can also use the following keystrokes in INSERT mode to spit these good digraphs:

CTRL+K <char1><char2> - Insert the digraph represented with the characters <char1> and <char2>.
<char1><BS><char2> - Insert the digraph represented with the characters <char1> and <char2> (only if the option digraph is set to true).

Here are some examples:

CTRL+K ->: →
CTRL+K TM: ™
CTRL+K Co: ©
CTRL+K Rg: ®
CTRL+K Eu: €
CTRL+K +-: ±
CTRL+K OK: ✓
CTRL+K XX: ✗

I owe my sanity to digraphs when I was trying to solve some problems in propositional logic full of "∧" (CTRL+k AN), "∨" (CTRL+k OR) or "∈" (CTRL+k (-).

A last tip: you can use the keystroke ga in NORMAL mode to display the digraph of the character under the cursor (if there is one).

Vim Help

:help digraph
:help digraph-table

Useful Keystrokes in Insert Mode

We saw many keystrokes for NORMAL mode throughout this series, but what about INSERT mode? We need more equality between modes in this world.

Keystrokes in INSERT mode are prefixed with CTRL, specifying to Vim that we don't want to insert some text.

Inserting and Deleting

CTRL+a - Insert the last content inserted.
CTRL+@ - Insert the last content inserted and quit INSERT mode.
CTRL+h - Delete the character before the cursor.
CTRL+w - Delete the word under the cursor.
CTRL+u - Delete everything before the cursor.
CTRL+t - Add one indentation.
CTRL+d - Delete one indentation.

Spitting Registers

We saw already in the previous article how to spit the content of a register in INSERT mode. Let's introduce some subtleties here:

CTRL+R <reg> - Spit the content of the register <reg> as if you typed it.
CTRL+R CTRL+R <reg> - Same as CTRL+R, but insert the text literally.
CTRL+R CTRL+P <reg> - Spit the literal content of the register <reg> with the correct indentation.

When you look at the content of your registers, you might find some weird characters like ^M or ^I. You can think of them as CTRL+m and CTRL+i, which means, if you recorded some macros while being in INSERT more, end-of-line and tabulation respectively.

The first keystroke CTRL+R <reg> will insert these end-of-lines and tabulations. For example, if you have the characters ^M in your register when you display them via :reg, it will become an end-of-line when you insert it.

If you use CTRL+R CTRL+R <reg>, you'll have the literal characters ^M inserted in your buffer. It's handy to spit a recording for a macro, modify it, and then save it back to your register <reg>. From there, you can execute your modified macro.

To know what the literal version of a key is, you can use CTRL+V (or CTRL+Q) in INSERT mode followed by the key. For example, CTRL+V ENTER will display ^M.

Since we're speaking about CTRL+V, you can also use it followed by the decimal, octal, or hexadecimal value of a character. It's another way to insert special characters without using digraphs. You can run man ascii to have access to the ASCII table in your shell and choose whatever character you want.

The keystrokes CTRL+V and the equivalent CTRL+Q can be used in COMMAND-LINE mode too.

Back to NORMAL mode

The keystroke CTRL+o in INSERT mode allows you to come back to NORMAL mode for one keystroke (or one command in COMMAND-LINE mode). When it's done, you'll be automatically back in INSERT mode.

Undo in Insert Mode

You can stop the undo sequence if you use CTRL+G u in INSERT mode.

For example, try to insert some text in INSERT mode and then undo your insertion with u in NORMAL mode. Everything you've inserted is now gone.

If you want to undo each word you've inserted instead of everything, you can use this command:

:inoremap <space> <C-G>u<space>

Now, each time you press SPACE in INSERT mode, you'll stop the undo sequence. When you hit u in NORMAL mode, it will undo what you've inserted after hitting the last SPACE. If you undo again, it will undo another word, and so on. There is a drawback, however: abbreviations won't work anymore in insert mode.

You can think of CTRL+G u as creating chunks of undo.

Vim Help

:help ins-special-keys
:help insert−index

Useful Keystrokes in Visual mode

There are also interesting keystrokes you can hit in the different variants of VISUAL mode.

Visual Mode and Visual Mode Linewise

o - Move your cursor to the opposite side of the selection (or the other end if you prefer). Doesn't work in VISUAL mode linewise.
R or S - Delete the selected lines and start INSERT mode.
U - Uppercase the selection.
gv - Switch back and forth between the previous selection and the current one.

Another tip: You can switch between VISUAL mode linewise and VISUAL mode blockwise without coming back to NORMAL mode with CTRL+v and SHIFT+v.

Visual Mode Blockwise

o - Move to the opposite corner of the selection.
O - Move to the opposite side of the selection.
I - Insert some content at the beginning of every line selected.
A - Append some content on every line selected after the highlighted area.
$A - Append some content at the end of every line selected.
c - Delete selected lines and begin INSERT mode on every line.

Vim Help

:help visual-index
:help blockwise-operators

Vim Regular Expressions

We're now at the crux of this article: Vim's regexes are really powerful for searching in your content or transforming it. As a Venerable Vim Adept, you need to harness its power.

This section won't go into the details of regexes in general. I assume that you know your basic metacharacters. If not, I've recorded videos about the basic of regexes using Grep (with exercises) if you're interested to learn more about them. It's not that hard and the benefits are huge.

If you know already some common regex engines like Perl, you won't be too much surprised by Vim's regexes. The basics are the same, but Vim introduces original concepts which are... interesting? If you want to quickly see the difference between Perl's regexes and Vim's, you'll find a summary by running :help perl-patterns.

In this article, I call "metacharacter" any character which has semantics: for example, the semantics for ^ is "beginning of the line".

The Concept of Atom

You'll see often this confusing notion of atom in Vim's help. It's just any metacharacter or group of metacharacters matching one character. For example, [A-Z] is an atom because it matches only one character from A to Z.

Magic and Nomagic

As we saw quickly in the previous article, Vim's regexes can have four different levels of magic: Very magic, magic, nomagic, and very nomagic. My advice in this madness: remember that very magic will allow you to use every regex metacharacter without escaping them, and that very nomagic oblige you to escape these metacharacters to use them.

The level of magic is set with the option magic. You might be tempted to change its value; please don't. Every plugin out there expect this option set to magic. Instead, we'll see different ways to change the magic level for each regex.

Because everybody has the level of magic set to "magic", you'll see many Vim regexes out there with a tonne of backslash to escape some metacharacters. That's why Vim's regexes often look so ugly, confusing, and hard to read. I propose this simple rule:

When you need a regex, use "very magic" by adding \v before your pattern.
When you don't need a regex, use "very nomagic" by adding \V before your pattern.

For example, :%s/\v(emacs)/\1 is bad/ will work flawlessly.

If you're curious what level of magic allows what metacharacter, take a look at :help \magic. You'll be granted with a wonderful table nobody remembers.

Character classes

Similarly to other regex engines, you'll have access to many character classes in Vim. You'll need to use the backslash preceding them whatever the level of magic you use.

Useful Character Classes

Here are the character classes I find the most useful:

\s or [:blank:] - whitespace characters.
[A-Z] or \u or [:upper:] - Uppercase.
[a-z] or \l or [:lower:] - Lowercase.
[0-9] or \d or [:digit:] - Digits.
\_ - Character class with end of line included.

For example, to illustrate a bit more the last one, if you want to include uppercase characters and line breaks in your regex, you can use \_u.

Most of the time, you can use the uppercase version of the character class to negate it. For example, \L is equivalent to [^a-z] (every character except the characters in the range a to z).

Character Classes Set Via Options

These character classes are interesting because you can change the characters they can match by changing the value of an option:

\f - Filename characters (option isfname)
\i - Identifier characters (option isident option)
\k - Keyword character (option iskeyword)
\p - Printable character (option isprint)

Changing these options can have consequences: they are sometimes used for other commands or keystrokes. For example, the keystroke gf use isfname under the hood.

Another tip: if you see for example the value 48-57 in these options, it means the ASCII characters from 48 to 57, which are the numbers from 0 to 9.

Zero-width

A zero-width lookaround assertion allows you to match a pattern looking forward of backward without adding it to the match. As a result, these metacharacters don't match any character by themselves. For example, ^ and $ are zero-width.

If you try to substitute them, you won't replace anything because there is nothing to replace. Instead, you'll insert some text. To understand that, you can try to run :%s/^/-> in a buffer for example.

You'll need to use the backslash for these metacharacters whatever the level of magic you're using. Here are the ones which might be useful:

\zs - Only match your pattern if what's before the metacharacter \zs match what's before your pattern.
\ze - Only match your pattern if what's after the metacharacter \ze match what's after your pattern.
\< - Match the beginning of a word.
/> - Match the end of a word.
\%^ - Match the beginning of the file.
\%$ - Match the end of the file.
\%V - Match inside the visual selection (or the previous one if you're not in VISUAL mode).

For example, in COMMAND-LINE mode after hitting / for searching:

\v^\s+\zsfor - Only match for if there are one or more whitespace before the pattern.
\<if\> - Only match the word if and not the substring if in cliff for example.
end\%$ - Only match the pattern end if it's just before the end of the current file.

Another concrete example: I'm using Zsh to edit command lines in Vim and I often end up with the following when I want to rename a file:

mv my-file-name.jpg my-file-name.jpg

The goal is to replace the hyphen - with underscores _ in the filename my-file-name.jpg. I could:

Switch to VISUAL mode
Select the second my-file-name.jpg
Run the following:

:'<,'>s/-/_/g

But it doesn't work. My substitute will replace every hyphen on the line thanks to the g flag, and, if I don't use it, I only replace the first hyphen on the line.

To solve the problem, I can use \%V:

:'<,'>s/\%V-/_/g

It works as expected because the pattern only match what's inside the selection I've made.

Regexes and Marks

What about using marks in our regexes?

\%<'m - Matches before the position of mark m.
\%>'m - Match after the position of mark m.

For example, /\%>'mcat\%<'a search the pattern cat between the mark m and a.

Vim Help

:help regex
:help /magic
:help holy-grail
:help pattern
:help pattern-atoms
:help pattern-overview

Using Your Shell Commands in Vim

Vim is powerful by itself, but coupling Vim with the shell is switching your life's GODLIKE mode. Additionally, if your development environment is built around the shell and your tools allow you to stay as much as you can on your keyboard, you'll be unstoppable.

Executing External Command

Here are the commands you can use to summon the Unfathomable Power of The Shell®:

:! <cmd> - Execute the shell command <cmd>.
:!! - Repeat the last command executed.

For example, We saw in this article that we can look at the ASCII table if we run man ascii in a shell. Now, you can look at it directly in Vim with :!man ascii.

What about inserting the output of a shell command in your buffer?

:read! <cmd> or :r! <cmd> - Execute the command <cmd> and insert the output in the current buffer.
:read!! or :r!!- Repeat the last command executed and insert the output in the current buffer.

Now, everybody in the universe can feel your brain radiating with The Power.

Vim Help

:help :!
:help :r!

Filter

Did you ever dream, during the warm summer nights, about feeding the lines of your Vim buffers into the input stream of your favorite command, and replace these lines with the resulting output?

Your dream will become a reality. Using :! with a range will help you fulfill your destiny. For example, you could:

Select a couple of lines in VISUAL mode
Run :'<,'>!grep <pattern>

Every line without the pattern <pattern> will disappear in a magical cloud. Show that to your friends, your family, or your boss, and they'll respect you forever.

This functionality is called "filter", but it's quite misleading to me. If I use 1,10:!sort, it won't filter anything, it will replace the input I gave to sort and it will replace it with the output.

Vim Help

:help filter

Folding

A good way to manage complexity is to hide what we don't need. In that regard, folding can be handy for complex codebases and long files.

Choosing Your Fold Method

The value of the option foldmethod will determine how you want to manage your folds. There are 6 in total:

manual - You manually define folds with the commands below.
indent - Folds are created depending on the indentation level.
expr - Folds are created depending on a Vimscript expression defined in foldexpr.
syntax - Fold are created depending on the syntax highlighting (if the syntax highlighting defines them).
diff - Fold unchanged text.
marker - Fold depending on markers.

For the foldmethod marker, here's an example I use in my .vimrc:

" Install Plugins ---------------------- {{{
"
" Some config here
"
" }}}
" Plugins Config ---------------------- {{{
"
" Some config here
"
" }}}

If you forgot how to set and unset options, I cover that in the first article of the series.

Keystrokes

All these keystrokes begins with z. When you look at this wonderful letter, you can let your imagination going into foreign worlds and see a fold. Folded: -. Unfolded: z. Think about unfolding an old manuscript full of hidden Vim knowledge. Impressed? Me too.

Creating and deleting folds

These keystrokes only work if your foldmethod is set to manual or marker.

zf - Create a fold. It can be used in VISUAL mode or with a motion.
zd - delete the fold under the cursor (but not the nested ones).
zD - Delete the fold under the cursor, including the nested ones.
zE - Eliminate every fold in the window. It deletes the markers if your foldmethod is set to marker. Brutal.

Opening and Closing Folds

zo - open the fold under the cursor.
zc - close the fold under the cursor.
za - Toggle the fold under the cursor (close it if it's open, open it if it's close).
zx - Undo opened and closed folds.

Uppercase variants of these keystrokes (zO for example) can be used to propagate the action to every nested fold.

Opening and Closing All Folds

zM - Close all folds
zR - Open all folds
zi - Toggle the use of folds (option foldenable).

Moving Through Folds

[z - Move to the start of the current fold.
]z - Move to the end of the current fold.
zj - Move downward to the start of the next fold.
zk - Move upward to the start of the next fold.

Commands

You can also use these commands to manipulate folds. Each of them accept a range as prefix.

:foldopen or foldo - Open folds.
:foldclose or foldc - Close folds.
:folddoopen <cmd> or :foldd <cmd> - Execute command <cmd> on all opened fold.
:folddoclosed <cmd> or :folddoc <cmd> - Execute command <cmd> on all closed fold

Using a bang ! for the first two ones (foldo! and foldc!) will open or close all nested folds too.

Vim Help

:help Folding
:help fold-methods
:help fold-commands

Are We There Yet?

Suddenly, Dave woke up in his bed. This was all a dream! The hooded figures, the telepathic Vim course, everything! But he has now a lot of knowledge about Vim he didn't have before falling asleep. Was it real? Was it a dream? Is our reality just a dream? Are we part of a weird Sim-like simulation? Are raspberries better than strawberries?

So many questions, so few answers! At least, we learned together the following:

We can have access to 10 global quickfix lists, and 10 location lists per window.
The NORMAL mode is powerful, but many useful keystrokes wait for our fingers to type them in INSERT and VISUAL modes.
Vim regular expressions can be more or less "magic", deciding what metacharacter you can use in your patterns without escaping them.
An atom in Vim regexes is any pattern matching one character.
Zero-width metacharacters don't match any character but help you narrow down your match depending on its context.
You can use marks in your regexes too.
You can run any shell command with :!.
You can spit output of shell commands in Vim with :read.
Folding can be useful when you've too much content on your screen.
Folding can also be used to apply a specific command on folded lines with :foldd and :folddoc.

This is not over. More discoveries will be made in the Name of Vim, and we'll all get Its Highest Power to build new applications which will save the world.

Becoming Mouseless

Do you want to build a Mouseless Development Environment where the Linux shell and Vim have a central role?

Switching between the keyboard and the mouse costs cognitive energy. This book will guide you step by step to set up a Linux-based development environment that keeps your hands on your keyboard.

Take the brain power you've been using to juggle input devices and focus it where it belongs: on what you create.

A Vim Guide for Advanced Users

Matthieu Cneude — Tue, 01 Jun 2021 13:27:48 +0000

Welcome to the third part of this series aimed to help you unleash a power never seen on Earth using the Almighty Vim.

We'll see together in this article:

Some nice keystrokes beginning with g.
What ranges are and how to use them.
The quickfix list and the location lists.
The marvelous substitute command.
The crazy useful :global (or :g) command.
What marks are and what you can do with them.
How to increase and decrease numbers with a single keystroke.
How to sort text with a nice command.

But before going into the juicy content of this article, I've some recommendations for you, dear reader:

Fire up your Vim and play around with the commands and keystrokes while reading. It's the best way for you to remember everything which blew you away. You'll then be able to be blown away whenever you like for the rest of your life.
Don't use a cheatsheet; write your own as you read. It will allow you to come back easily to the commands you find the most useful for your own needs.

Enough rambling. Let's begin!

Useful "g" Keystrokes

Let's begin gently with a mixed bag of keystrokes starting with g. There are many of these "g" commands in Vim, and we already saw some of them in the previous articles. Can you recall them?

You can use these keystrokes in NORMAL mode:

gf - Edit the file located at the filepath under your cursor.
- You can use CTRL+W CTRL+F to open the file in a new window.
- It can open the URL under your cursor if you have the plugin netrw.
gx - Open the file located at the filepath under your cursor.
- It will use the default application used by your OS for this filetype.
- It works with URLs too. It will open your favorite browser and load the website.
- It only works if you have the plugin netrw.
gi - Move to the last insertion you did and switch to INSERT mode.
- Pretty useful if you stopped your editing to look at some other file.
- It uses marks under the hood: more on that later in this article.
gv - Start VISUAL mode and use the selection made during the last VISUAL mode.
gn - Select the match of your last search:
1. Move to the last searched match.
2. Switch to VISUAL mode (if you weren't in VISUAL mode already).
3. Select the match.
4. You can continue to hit n (or gn) to select the area between the current match and the next match.
gI - Insert text at the beginning of the line, no matter what the first characters are.
- The keystroke I insert text just before the first non-blank characters of the line.
ga - Print the ascii value of the character under the cursor in decimal, hexadecimal or octal.
gu - Lowercase using a motion (for example, guiw).
gU - Uppercase using a motion (for example, gUiw).

To try out gf and gx, you can write for example https://www.google.com/ in Vim, place your cursor on it, and hit the keystrokes. Don't forget the trailing slash in your URL.

You'll soon discover an inconvenience with gx: when you use it on a filepath, Vim will hang till you close the file. That's why I've created the following mapping you can add to your .vimrc:

nnoremap gX :silent :execute
            \ "!xdg-open" expand('%:p:h') . "/" . expand("<cfile>") " &"<cr>

It maps the keystroke gX to use xdg-open with a relative filepath under your cursor. It will open the file with your favorite application in the background. The program xdg-open will only work on Linux-based systems; for MacOS, try open instead.

Vim Help

:help reference
:help g

Ranges

Now that we're done with the appetizers, let's discover the first real dish of our Vim feast: the ranges. You can apply them to many commands, making them terribly powerful.

Basics

In Vim's help, every command accepting a range will have the string of characters [range] in front of them. Multiple line specifiers are possible, separated by commas ,.

Here are the most interesting ranges you can use:

<number> - Any number <number> in your range refers to a line number.
. - Represent the current line (often the default range).
$ - Represent the last line of the current buffer.
% - Represent the entire file (same as 1,$).
* - Use the last selection you've made during the last VISUAL mode.

For example, using the command :d:

:1,40d - Delete line 1 to 40 included.
:2,$d - Delete every line from the second one till the end of the file.
:.,$d - Delete every line from the current one till the end of the file.
:%d - Delete every line.

You can also do some arithmetic with ranges if you want. For example, let's imagine your cursor is on the line 60: the range .,.+10 will be equivalent to the range 60,70.

Visual Mode and Range

If you switch to COMMAND-LINE mode after doing some selection in VISUAL mode, you'll see these two symbols appearing: '< and '> with a comma , in between. This is a range too!

The symbols '< represents the first line you've selected and '> the last line. Each of these are marks; again, we'll see more about marks below.

In practice, the ranges '<,'> and * are synonym, but you'll have more flexibility with the first. For example, you can execute a command from the first line you've selected till the end of the file with the range '<,$.

Vim Help

:help [range]
:help v_:
:help '<
:help '>

Vim's Quickfix And Location List

Now, let's talk about a very useful data structure available in Vim, the quickfix list.

Don't confuse the quickfix list and the quickfix window: these are two different entities. The first is a data structure, the second can display this data structure.

Quickfix Lists

You can think of a quickfix list as a set of positions in one or multiple files.

Basics

Let's take an example: what happens if you run the command :vimgrep hello *?

It will search the pattern "hello" in every file of your working directory.
It will populate a quickfix list with every position matching your pattern "hello".
It will move your cursor to the first position of the quickfix list.

If you want to know more about vimgrep and other tools you can search with, I wrote an article about that. Other commands (like :make or :grep) also populate automatically a quickfix list.

Let's expand the mystery around marks: these positions in the quickfix list are in fact hidden mark!

The quickfix list is often used to display specific errors in a codebase, often spit out from a compiler or a linter (via the command :make for example), but not only, as we just saw. I call the entries of a quickfix list "positions" to be more general, but sometimes Vim's help will refer to them as "errors". Don't be confused: it's the same idea.

Among other conditions, a quickfix list entry needs to have a filename for you to be able to jump to its position. Otherwise, the entry doesn't point to anything. Difficult to move to anything.

Useful Commands

Here are the commands you can use to navigate through the current quickfix list:

:cl or :clist - Display all valid entries of the current quickfix list. You can add a range as argument (only numbers).
:cc <number> - Move to the <number>th entry of the current quickfix list.
:cnext or :cn - Move to the next entry of the current quickfix list.
:cprevious or :cp - Move to the previous entry of the current quickfix list.
:cfirst or :cfir - Move to the first entry of the current quickfix list.
:clast or :clas - Move to the last entry of the current quickfix list.

Here are additional commands which make quickfix lists really powerful:

:cdo <cmd> - Execute a command <cmd> on each valid entry of the current quickfix list.
:cexpr <expr> or :cex <expr> - Create a quickfix list using the result of evaluating the Vimscript expression <expr>.
:caddexpr <expr> or :cadde <expr> - Appends the result of evaluating the Vimscript expression <expr> to the current quickfix list.

If you have no clue how to use the last two commands, you can do for example:

:cex [] - Empty the current quickfix list.
:cex system("<cmd>") - Populate your quickfix list with any shell command <cmd>. You can try it with ls for example.

The Quickfix Window

What about displaying the current quickfix list in a new buffer? You can do that with the following command:

:copen or :cope - Open a window (with a special buffer) to show the current quickfix list.

You can only have one quickfix window open. To move to the position of the selected entry of the quickfix list in the quickfix window, hit ENTER or .cc.

Location Lists

A location list is similar to a quickfix list, except that the first is local to a window and the second is global to your Vim instance. In other words, you can have multiple location lists available at the same time (one per window open), but you can only have access to one quickfix list.

The commands for location lists are similar to the ones for the quickfix lists; often, you'll only have to replace the first c (quicfix) of the command with l (location).

For example:

:lli or :llist - Display all valid entries of the current location list. You can add a range as argument (only numbers).
:ll <number> - Move to the entry <number> of the current location list.
:lnext or :lne - Move to the next entry of the current quickfix list.

To populate your location list you can also use the commands :lvimgrep or :lmake for example.

Often, Vim users will use the quickfix list for anything related to errors in their codebase, and the location list for search results. But it's up to you at the end of the day. With Vim, you're the master of your destiny.

Vim Help

:help quickfix
:help quickfix-window
:help location-list
:help location-list-window
:help expr
:help system()

Vim's Registers

The registers are another big dish in our Vim feast. You can think of registers as places where you can read or write some text. I like to think about them as Vim's clipboards.

Specifying a Register

Here's a command and a NORMAL mode keystroke to display and specify registers:

:registers or :reg - Display the content of your registers.
"<reg> - This keystroke specifies the register <reg> to be read or written.

How do you know when the register <reg> is read or written using the keystroke "? It depends of the keystroke you use afterward. For example:

To write the register a:
1. Hit "a in NORMAL mode to specify what register you want to write on.
2. Yank, change, or delete some content (for example by using dd in NORMAL mode) to write it to a.
To read the register a:
1. Hit "a in NORMAL mode to specify what register you want to read.
2. Use a put keystroke in NORMAL mode (for example p or P) to spit out the content of the register in your current buffer.

We take the example of register a here, but it will work for any writable register.

The Types of Registers

There are 10 different types of registers in Vim:

The unnamed register (") - Contain the last deleted, changed, or yanked content, even if one register was specified.
The numbered registers (from 0 to 9)
- 0 contains the content of the last yank.
- 1 to 9 is a stack containing the content you've deleted or changed.
  1. Each time you delete or change some content, it will be added to the register 1.
  2. The previous content of the register 1 will be assigned to register 2, the previoius content of 2 to 3...
  3. When something is added to the register 1, the content of the register 9 is lost.
- None of these registers are written if you've specified one before with the keystroke ".
The small delete register (-)
- Contains any deleted or changed content smaller than one line.
- It's not written if you specified a register with ".
The named registers (range from a to z)
- Vim will never write to them if you don't specify them with the keystroke ".
- You can use the uppercase name of each register to append to it (instead of overwriting it).
The read only registers (., % and :)
- . contains the last inserted text.
- % contains the name of the current file.
- : contains the most recent command line executed.
The alternate buffer register (#) - Contain the alternate buffer for the current window.
The expression register (=) - Store the result of an expression. More about this register below.
The selection registers (+ and *)
- + is synchronized with the system clipboard.
- * is synchronized with the selection clipboard (only on *nix systems).
The black hole register (_) - Everything written in there will disappear forever.
The last search pattern register (/) - This register contains your last search.

As you can see, even if you don't specify any register with the keystroke ", the content you delete, change, or yank will automatically overwrite one (or multiple) of them. So if you don't want the content you write to your registers to be silently overwritten by Vim, always write in the named registers.

Using a put command without specifying any register will spit the content of the unnamed register by default. But you might have this line in your vimrc:

clipboard+=unnamedplus

In that case, the content you change, delete, or yank will go directly in the unnamed register and the + register. Using put commands will directly output the + register too. Many find it useful (including me) to access your OS clipboard more easily, without the need to specify the + register for reading or writing it.

Appending a Recording

We've seen in the previous article that you can record your keystrokes using q. Now that you know how to use registers, you can manipulate these keystrokes:

If you made a mistake during the recording, you can spit the whole register, modify it, and save it back.
You can append to your recording by using the uppercase variant of your register. For example:
1. Hit qa and record whatever keystrokes you want. Stop the recording by hitting q again.
2. You realize that you forgot a couple of keystrokes.
3. Execute your keystrokes to be sure you're at the last position of your recording.
4. Hit qA to append to your register a. When you're done, hit q again.

You've just gained even more flexibility for your macros.

Using Registers in INSERT and COMMAND LINE modes

The magical keystroke " is great for NORMAL mode, but what about spitting the content of a register in INSERT mode or COMMAND LINE mode? For that, you can use CTRL+R <reg> to put the content of register <reg> in your current buffer.

For example, if you hit CTRL+R % in INSERT mode, you'll put the content of the register % in your current buffer.

The Insane Expression Register

If you don't know the expression register, I'm about to revolution your life. I hope you're ready.

Try this:

Switch to INSERT mode and hit the keystroke CTRL+r =. You'll move to Vim's command-line.
From there, you can type any Vimscript expression you want, like system("ls") we saw above, or 4+4.
Hit ENTER to run the expression, and you'll see the output of the shell command ls directly inserted in your buffer!

It's useful to evaluate some custom functions you've defined while staying in insert mode. If you use Neovim, you can use the function luaeval() to evaluate some Lua too.

Clearing a Register

A last little trick about registers: if you want to empty one, you can do:

qaq

Beginning a recording also deletes everything which is in this register. So you just need to stop the recording by hitting q again to have an empty register.

Vim Help

:help registers
:help clipboard
:help clipboard-unnamed
:help clipboard-unnamedplus

The Substitute Commands

Let's continue our ascension to become The Vim God (or Goddess). The substitute command is next on the menu.

Basics

If you're already familiar with the CLI sed, this command will remind you some good old memories.

A substitution is a way to replace some content with some other content, using a range and a count. The count will decide how many lines is affected by your substitution from the last line of your range.

Like many other commands, the default range is the current line if you don't specify it.

Here's the pattern of the command itself:

:s/pattern/replacement/flags

What does this mean?

The pattern is the search you want to match.
The replacement will replace the first match of the pattern on each line.
The flag modify the behavior of the command.

There's another element, represented here with a slash /: the separator. It doesn't have to be a slash, it can be any character except:

An alphanumerical character (a character included in the range a-z, A-Z, and 0-9).
A double quote".
A pipe |.

The replacement is not mandatory: if you omit it, the substitute command will delete the pattern matched.

You can also run :s without any range, pattern, replacement, flags, or count. In that case, it will repeat the last substitution you've done without the flags; you can add new flags (except &) and a count if you want to.

Let's see some examples to understand how it works:

:s/pattern/replacement/ - Substitute the first occurrence of pattern on the current line with replacement.
:s#pattern#replacement# - Equivalent substitution to the one just above. Handy if you have some URLs in your pattern or your replacement.
:s/pattern/ - delete the first occurrence of pattern on the current line.
:s/pattern/replacement/g - Substitute every occurrence of pattern on the current line.

You can also add a range as prefix and a count as suffix:

:%s/pattern/replacement/ - Substitute every first occurrence of pattern on each line of the current buffer.
:%s/pattern/replacement/g - Substitute every occurrence of pattern on each line of the current buffer.
:1,10s/pattern/replacement/ - Substitute every first occurrence of pattern on the first ten lines of the current buffer.
:s/pattern/replacement/ 10 - Substitute every first occurrence of pattern for the current line and the 10 next lines.
:1,10s/pattern/replacement/ 5 - Substitute every first occurrence of pattern on the first ten lines and on the five lines below the last line of the range.
:s g 10 - Repeat the last substitution without its flag, and add a new flag g. It will affect the 10 lines after the last line of the last substitute command.

Vim's "Magical" Patterns

A pattern in that case is a regular expression. But it's not the good old regex engine you're familiar with from high level programming languages. Vim has its own regex engine (actually, it has two!) which can be quite confusing.

Regexes in Vim are more or less magic. You think I'm kidding? I'm not. Here's what I find useful to remember:

To have access to all regex metacharacters, you can prefix your pattern with \v (very magic).
To have access to almost all regex metacharacters (except (, ) and |), use the command :sm instead of :s (substitute magic).
To have access to none of the metacharacters (except $), use :sno instead of :s (substitute nomagic).
To have access to none of the metacharacters, you can prefix your pattern with \V (Very nomagic).

For example, if I want to delete all opening parenthesis in my buffer, I can run one of these three equivalent commands:

:%s/\V(/
:%sm/(/
:%s/\(/

If you know your regex metacharacters, you might wonder what's the metacharacter ~. It's the latest substituted string in Vim's world. In fact, you can use many more metacharacters (called atoms) and character classes in your Vim's regex.

If don't know you're regex metacharacters, I've recorded a couple of videos explaining the basics of regular expressions (using grep).

The next article in this series dive into Vim's regexes a bit deeper.

Additional Commands

Here are two other commands I find useful:

:&& - Repeat the last substitute with its flags.
:~ - Repeat the last substitute command with the same replacement, but with the last used search pattern.

For example, let's say that you execute successively the following in Vim:

:s/pattern/replacement/
/hello
:~

The last command will substitute hello with replacement.

You can also use these useful keystrokes in NORMAL mode:

& - Repeat the last substitute, without its range and its flags.
g& - Repeat the last substitute, with its range as well as its flags, and replace its pattern with the last search pattern.

The Substitutes Flags

Here are some flags which can be useful:

& - Use the flag(s) from the previous substitute command.
c - Ask you to confirm each substitution.
g - Replace all occurrences in each line (not only the first one).
i - The pattern is case-insensitive.
I - The pattern is case-sensitive.
n - Only report the number of match without substitute.

You can now substitute like crazy to your heart's content!

Vim Help

:help :substitute
:help :sm
:help :sno
:help :s_flags

The Global Command

You don't have enough? You want even more power? Behold the Holy Global Command! It works similarly to the substitute command, except that it will execute a command instead of replacing a pattern.

Basics

Here's the pattern of the command itself:

:g/pattern/command

You can also prefix it with a range if you want to.

As an example, let's imagine that you have the urge to delete all the lines of your current buffer containing the pattern "useless". To do that, you can run:

:g/useless/d

Nice, but I can give you even more power.

Normal mode Commands

Do you know the command :norm? You can give to it some NORMAL mode keystrokes and it will apply them for you, as if you were hitting them in NORMAL mode. For example, the following will delete the word under the cursor:

:norm daw

Let's now combine a normal mode command with the global command:

:g/useless/norm gu$

This will lowercase every line containing the pattern useless.

When I first heard about that, I saw the Universe, the Big Bang, the cycle of creation and destruction. I understood life and death. I became blessed, chanting the power of Vim in every free cities.

A last tip: :norm use the key mappings you've defined. If you only want to use Vim's default mapping, you can use norm! instead.

If you write a plugin, always use norm!. You don't know what mappings have your users.

Vim Help

:help :global
:help :normal

Marks

It's time to answer the question haunting your incredible mind: what are marks?

Basics

You can think of a mark as a specific position in a buffer. To set one, you can use m in NORMAL mode followed by a character in the following ranges:

a-z - These marks are local to one buffer.
A-Z - These marks are global to multiple buffers.

If you have a "viminfo" file set via the option viminfo in Vim, or if you have a "shada" file set via the option shada in Neovim, these marks are persisted. This means that you can come back to them even if you closed the file.

If you wonder what are Vim's options and how to display their values, I wrote about it in the first article of this series.

There are also read-only marks in the range 0-9. They are only available when using a viminfo file (for Vim) or a shada file (for Neovim). They store the position of your cursor each time you quit a file: the mark 0 has the last position, the mark 1 has the position before the last one, and so on.

To move to a mark, you can use these keystrokes in NORMAL mode:

' - Move to the first non-blank character of the line where the mark  was set.
` - Move to the exact position where the mark  was set.
g'  - Move to the mark  without changing the jump list (I've written about the jump list in the previous article of this series).

Useful Commands

As you might imagine, there are also some useful commands for displaying or manipulating marks:

:marks - Display the marks set.
:marks <marks> - Display some specific marks <marks>.
:delmarks  or :delm  - Delete the mark .
:delmarks! or delm! - Delete all the marks in the range a-z.

For example, :marks <> will display the two marks < and >.

You can also use marks as ranges. For example:

` :`a,`bs/pattern/replacement/ `

This will substitute the first match of pattern with replacement from the exact position of the mark a to the exact position of the mark b.

Special Marks

Let's now introduce special marks with these keystrokes:

m< or m> - Set the marks '< and '> we saw above. It can be handy for the keystroke gv.
`[ - Move to the first character of the previously changed, deleted, or yanked content.
]` - Move to the last character of the previously changed, deleted, or yanked content.
`" - Move to the position were you've closed the current file for the last time.
`^ - Move to the position where you've used INSERT mode for the last time (this mark is used by the keystroke gi under the hood).

For every keystroke described above using a backtick, you can use a single quote ' instead with the differences we saw above.

Vim Help

:help mark-motions

Manipulating Numbers

After these register, substitution, and mark shenanigans, here's a lighter subject: adding (or subtracting) numbers.

Here's a bunch of NORMAL mode keystrokes to do exactly that:

CTRL+a - Increase the first digit or number on the line.
CTRL+x - Decrease the first digit or number on the line.

You can also use these keystrokes in VISUAL mode:

g CTRL+a - Same as CTRL+a unless you have several lines selected. In that case, the first number of each line will be incremented sequentially.
g CTRL+x - Same as CTRL+x unless you have several lines selected. In that case, the first number of each line will be decremented sequentially.

To illustrate a bit better the last two keystrokes, let's say that you have this amazing content in your beloved Vim:

Take some red flowers.
Add some eggs.
Forget about it and go back to your computer. `

If you select in VISUAL mode the last two lines and hit g CTRL+a, you'll get:

Take some red flowers.
Add some eggs.
Forget about it and go back to your computer. `

You can also prefix a count to the command to add a precise amount. For example, 12 CTRL+a will add 12 to the first number of the current line.

An important note: these keystrokes can also work on unsigned binary, octal, and hexadecimal numbers, as well as alphabetical characters. Their behaviors depend on the value of the option nrformats. For these keystrokes to behave as described in this article, you shouldn't have alpha as part of the value of nrformats, or you'll increment (or decrement) the first alphabetical character of the line.

Personally, I think it's better to exclude alphabetical characters, but in any case, I would recommend you to read Vim's help to learn more about that. As always.

Vim Help

:help CTRL-A
:help CTRL-X
:help nrformats

Sorting Text

What about a little dessert? A nice command to sort text directly in Vim, perhaps? Here are the commands:

:sort or :sor - Sort lines depending on a range. If no range is given, all lines are sorted.
:sort! or :sor! - Reverse the order.

You can add some options to this command. Here are the most useful ones:

i - Ignore Case.
n - Sort depending on the first decimal on the line.
f - Sort depending on the first float on the line.
/pattern/ - Sort depending on what comes after the match.
r - Combined with /pattern/, the sort depending on the matching pattern.

For example, if you have a CSV (with comma , as separator) and you want to sort every line depending on the second column, you can run something like this:

:sort /[^,]*,/

How about sorting the following list depending on the numbers of each line?

Take some red flowers (1). Forget about it and go back to your computer (3). Add some eggs (2).

Running :sort n will have the following result:

Take some red flowers (1). Add some eggs (2). Forget about it and go back to your computer (3).

As always, there are many subtleties involved here. If you want to dig deeper, you know what you should do: fire up this Vim's help and enjoy the depth of its infinity!

Vim Help

:help sort

Are You a Vim Master Now?

We can now make two assumptions:

Compared to your old self, you might be more of a master than before.
Compared to many, you still have a long road ahead of practice and learning.

But here's the most important: you shouldn't care about the second point. Always compare yourself to your past self, not to the others. Continue to walk on your own road, and you'll get your enlightenment. You'll then build your own Mouseless Development environment, you'll move to a comfy cave in the Himalaya, alone, living a life of vow, giving your whole soul to the study of the Almighty Vim.

What did we learn in this article?

The letter "g" in Vim is a bit like a magic wand:
- There are many useful keystrokes which begin with "g".
- The flag g is often used with the substitute command.
- The global command :g is really powerful to apply a command to some precise content.
Ranges allow you to execute many commands on a bunch of contiguous lines.
The quickfix list is a global list of positions in different files. You can apply any command to them with :cdo.
You can see the entries of a quickfix list using the quickfix window.
Location lists are similar to quickfix lists, except that you can have one location list (and one location window) per window open.
You can use the substitute command :s to replace a pattern (regex) with a replacement. Prefer :sm if you want to use a regex or :sno if you don't.
You can use marks to add some surgical precision in what you want to do.

To reward your tenacity and resilience for reading the whole article, I've a last tip for you: :help ex-cmd-index will list all the commands available in Vim.

Vim is easy to learn but hard to master: that's great, because we never stop improving!

Related Sources

At least one Vim trick you might not know - Hillel Wayne
Vim Macro Trickz - Hillel Wayne
Vim Search, Find and Replace: a Detailed Guide - Your obedient servant

Becoming Mouseless

Do you want to build a Mouseless Development Environment where the Linux shell and Vim have a central role?

Switching between the keyboard and the mouse costs cognitive energy. This book will guide you step by step to set up a Linux-based development environment that keeps your hands on your keyboard.

Take the brain power you've been using to juggle input devices and focus it where it belongs: on what you create.

The Basics of grep - Filtering All The Content You Want

Matthieu Cneude — Sun, 11 Apr 2021 12:10:09 +0000

The tool grep is a fantastic CLI (Command-Line Interface) allowing you to filter any pattern for a given input, including a file.

To know if you have GNU grep, you can run in your terminal grep --version. If you don't have GNU grep, I would recommend installing it.

This article is a summary of the following video I made. I would recommend you to watch it first, it's more complete with many examples and exercises:

If you find the videos I have on this channel helpful, don't hesitate to subscribe and to like them. It would then appear to other Youtubers to help even more!

I also recommend you to:

Download these files
Fire up your beautiful shell
Try each command you'll see in this article

You won't learn much if you only read (or watch a video) passively.

In this article we'll see together how to:

Colorize grep output
Case insensitive patters
Output lines with pattern matched
Output lines with pattern not matched
Output only the pattern matched
Output lines numbers
Output the match count
Hiding filenames
Always output filenames
Only output filenames
Only output filenames without matches
Output lines after the match
Output lines before the match
Output lines before and after the match
Excluding files
Piping grep
Summary: a mindmap of grep
Using grep in practice

Colors!

I would advise you to colorize the output of grep using the option --color as following:

grep --color=auto

But it's a pain to use this option each time you use grep. You can define an alias to make grep always colorful:

alias grep="grep --color=auto"

It's now a firework machine.

Output Lines With Pattern Matched

Here's the general way to use grep:

grep "<pattern>" <input>

The <pattern> is a regular expression and the <input> can be what you enter in your terminal or a file. For example:

grep "package" pacman.log

You'll see this output:

You can see that the pattern package is part of these two lines; that's why they are output. Incredible! It would be nice if we could do that in our daily life. I already know what words I would filter.

Note that the pattern wouldn't be colorized without the option --color set to auto as we saw above.

Output Lines With Pattern Not Matched

If you want to invert the match, that is, only output the lines which don't match the pattern, you can use the option -v as follows:

grep -v "package" pacman.log

This time, you'll output all the file except the two lines containing the pattern package.

Output Only The Pattern Matched

It can be useful to only output the match and not the whole line. You can do that with the option -o

grep -o "package" pacman.log

Output Line Numbers

If you to output the lines and the line numbers, you need to use the option -n. For example:

grep -n "package" pacman.log

The line numbers will be prefixed to each line of the output.

Only Output Match Count

You can also only output the match count with the option -c:

grep -c "package" pacman.log

Case Insensitive Patterns

By default, to match a pattern, you'll need to know what letters of the pattern are uppercase or lowercase. You can match your pattern without worrying about the case if you use the option -i:

grep -i "pacman" *.log

You can see that grep can filter more than one file. The glob * has a special meaning: it represents 0 or more character. Said differently, we want every file in our working directory ending up with .log. The following is equivalent:

grep -i "pacman" haskell.log pacman.log

Filenames

Hiding Filenames

As you saw, grep will output the filenames of the occurrence matches if you use more than one file as input. To hide them, you can use the option -h. An example:

grep -h "package" *.log

Always Output Filenames

If you want to output the filename where there's a match, even if you gave only one file as input, you can use the option -H. As usual, here's an example you can run in your terminal:

grep -H "package" pacman.log

For some option, the uppercase variant will do the inverse of the lowercase one, like -h and -H.

Only Output Filenames

To only output the filenames where the patterns match, you can use the option -l:

grep -l "package" *.log

Only Output Filenames Without Matches

You might have guessed it: the option -L only output the filename where the pattern doesn't match:

grep -L "package" *.log

Output Lines Around The Match

Outputing the lines before or after your match can be useful to get a bit of context and understand the result of your filtering.

Output Lines After The Match

To output a variable number of lines After the line with a match, you can use the option -A:

grep -A 3 "package" pacman.log

Output Lines Before The Match

Similarly, if you want to output the lines Before the match, you can use the option -B:

grep -B 3 "package" pacman.log

Output Lines Before And After the Match

You can even output some lines before and after the match with the option -C:

grep -C 3 "package" pacman.log

In this example, the second match of package won't output the 3 lines after because it's the end of the file after the line containing the match.

Did you see the pattern? After, Before, Context: it's ABC!

Excluding Files

We always gave the files we wanted to filter to grep's input, but what if we want everything except a couple of files? For that, we can use the option --exclude:

grep "package" --exclude="pacman.log" *.log

Piping grep

You can directly give the output of a command to the input of grep if you want to filter it. Actually, you can do that with every CLI accepting an input using pipes.

For example, if I run ps (a CLI to output the processes running on my machine), I get this:

    PID TTY          TIME CMD
   2026 pts/9    00:11:36 nvim
   2431 pts/11   00:01:11 hugo
   2584 pts/10   00:00:01 tmuxp
   3009 pts/18   00:00:01 nvim
   3241 pts/20   00:00:17 taskell
   3484 pts/22   00:00:11 taskell
   3663 pts/21   00:00:00 nvim

What if I want only the lines containing the pattern nvim? I can do the following:

ps -a | grep "nvim"

It means that I give to the input of grep the output of the command ps -a. Here's the result:

   2026 pts/9    00:11:36 nvim
   3009 pts/18   00:00:01 nvim
   3663 pts/21   00:00:00 nvim

Pipes are one of the reason why the command line is so powerful: you can pipe small CLIs doing one thing and you'll have crazy results!

In a Nutshell: a Mindmap of grep

It's nice to learn all of that, but how can you use grep in real life? Here's a video showing you a problem I had and I solved nicely using grep and other CLIs:

Becoming Mouseless

Do you want to build a Mouseless Development Environment where the Linux shell has a central role?

Switching between the keyboard and the mouse costs cognitive energy. This book will guide you step by step to set up a Linux-based development environment that keeps your hands on your keyboard.

Take the brain power you've been using to juggle input devices and focus it where it belongs: on what you create.