Forem: J. Bobby Lopez

‘hi’ shows it’s colors

J. Bobby Lopez — Sun, 21 Jul 2024 04:39:48 +0000

I previously introduced ‘hi’ – the Host Information tool to the world and described it’s base functionality. Later, I described how hi can be used to craft more complex notifications. In this article, I’ll describe some of the latest features implemented as part of the v0.4.5 pre-release.

Get ‘hi’ on GitHub: https://github.com/jbobbylopez/hi

Logging and State Management

One of the biggest features introduced is the use of the logging module to implement a logging facility to log every status change event detected by the ‘hi’ tool. State is captured as a JSON file, which is then compared against the current state of each check. If there is a difference, a log is written, and the last state is updated with the current state information.

State file example

➤ cat state.json 
{
    "Systems": "FAIL",
    "ExpressVPN": "SUCCESS",
    "KeepassXC": "FAIL",
    "sshd": "SUCCESS",
    "signal-desktop": "FAIL",
    "Dropbox": "SUCCESS",
    "Galaxy S20 (sshfs)": "FAIL",
    "vdc2022nas": "SUCCESS",
    "wdred-8tb-pro-01": "SUCCESS",
    "Apache NiFi": "FAIL",
    "Open WebUI": "SUCCESS",
    "Thunderbird": "SUCCESS",
    "qbittorrent": "SUCCESS",
    "minidlna": "FAIL",
    "Steam": "FAIL",
    "CUPS": "SUCCESS",
    "Data Backup": "SUCCESS",
    "dockerd": "SUCCESS",
    "vi": "SUCCESS",
    "Midnight Commander": "FAIL",
    "tmux": "SUCCESS"
}

‘hi’ Logging Functionality

Log files are written in JSONL (JSON Lines) format, to make it easy to consume by other upstream log and API analytics and monitoring tools.

Tools such as Logstash, Graylog, Nagios, Zabbix, Splunk, Kibana, Datadog, and so on can easily ingest JSONL formatted logs, which makes it easy for these tools to parse and index the log data to generate simple or complex reports and notifications as required.

Log file example

➤ tail hi_log.json 
{"timestamp": "2024-07-20T21:26:15.546248Z", "level": "INFO", "message": "vi state changed from SUCCESS to FAIL", "check_name": "vi", "previous_state": "SUCCESS", "new_state": "FAIL"}
{"timestamp": "2024-07-20T21:27:38.308707Z", "level": "INFO", "message": "sshd state changed from SUCCESS to FAIL", "check_name": "sshd", "previous_state": "SUCCESS", "new_state": "FAIL"}

Logs are only written when there is a status change for any of the checks being monitored.

Logging can be enabled in the config.ini file.

config/config.ini example

 [Paths]    
 checks_file = config/checks.yml    
 log_file = hi_log.json    

 [Report]    
 header_style = bold bright_magenta on black    
 hostname_style = magenta on black    
 ip_style = bold bright_green on black    

 [Tables]    
 number_of_columns = 3     
 default_style = green on black    
 header_style = bright_green    
 border_style = bright_white    
 column_style = bright_magenta     
 text_style = None    

 [OS Bar]    
 os_text_style = bright_red on black    

 [Logging]    
 enable_logging = true

Color Customization

Because, Why Not.

If you are going to use a terminal regularly, you want your tools to look good. Well, at least I do. Some tools offer minimal choices regarding terminal color and styling options. Some offer a lot of options but make it rather difficult to customize quickly.

I know my personal taste in terminal color and styling choices may not be for everyone. So, I decided to make the ‘hi’ tool output easily customizable.

Below are just some of the possibilties for display customization you now have with the ‘hi’ tool:

Magenta on Magenta Color Scheme

Green on Green Color Scheme

Green on Black Color Scheme

Blue and Black Color Scheme

Magenta on Black Color Scheme

Conclusion

With the new logging and state tracking capabilities, along with customizable color schemes, the ‘hi’ tool paves the way for further expansion, opening doors for integration with other command-line or API-oriented tools that utilize standard JSON formatted outputs.

I would love your feedback on this tool! There are always bugs to fix and features to add as outlined in our ROADMAP.md, but I can’t catch everything on my own. Please don’t hesitate to reach out via GitHub if you encounter any issues or have suggestions. Sometimes the fix is as simple as an update or clarification in the README.md, and I’m more than happy to address it.

Thanks for checking out ‘hi,’ the Host Information tool! Let’s make monitoring more efficient together.

Join the discussion and contribute on GitHub

This article is cross-posted from 'hi' shows its colors on thedevopsjoint.com.

Saying "hi" to your Linux Host Information

J. Bobby Lopez — Sun, 07 Jul 2024 19:23:29 +0000

I wanted to share a little tool that I think you'll find to be a joy to use. Especially if you are a command-line enthusiast like myself.

Say hello to "hi", my "Linux Host Information Reporting" tool.

I've written a little bit previously about this tool in Introducing ‘hi’ – the Host Information tool, so in this article, I'll just dive into what this tool does, and how to use it.

Firstly, you'll need to know where to get it. "hi" can be found on GitHub at https://github.com/jbobbylopez/hi. Currently published release is a 0.4.1 pre-release, so this tool is still in it's early development stages. Some tests have been defined in test_host_information.py, and will continue to be improved in order to guide development.

Once you check-out that repository, one of the first things you'll notice is that there is no script, file, or binary called 'hi'. The script is actually called host_information.py. There are instructions in the README.md for how to setup an alias to get the 'hi' command configured correctly.

What is 'hi'?

"hi" is a command-line tool designed to monitor and display the status of various services running on a host. The services and their respective checks are customizable via a YAML configuration file, allowing for flexibility to suit different monitoring needs. This tool leverages the Rich library to provide visually pleasing output that includes UTF8 status icons, so it should be run in a terminal that supports the UTF8 character-set.

Features

Highly customizable checks that can report on any command-line executable status.
Verifies the running status of various services.
Checks the last modified date of backup logs to ensure up-to-date backups.
Monitors the connectivity status of ExpressVPN.
Displays the local IP address and hostname of the host.
Utilizes the Rich library for visually appealing console output with UTF8 status icons.

Here is an example of the output that 'hi' reports in the terminal:

Using 'hi info', you can get some extended information reported from the 'info:' section of the check.yml configuration.

"hi info" output:

Groups can be customized in the config/groups.yml file, and only those system checks configured with the groups defined will be reported.

Example of the groups.yml and checks.yml configuration files:

How System Checks Are Defined

The 'hi' checks are literally any valid bash command line that can return a non-zero value. If the command is successful, then the check is reported as positive. However, because the bash command includes the ability to use pipes, you have an immense amount of control around how your checks are implemented.

You can do a simple check to see if a process is running. Lets say for example you wanted to check if the cups printing service is running on your host. Normally the fastest way to check would be to execute a ps -ef | grep cups on the command-line, like so:

➤ ps -ef | grep cups
root      828443       1  0 Jun27 ?        00:00:00 /bin/sh /snap/cups/1058/scripts/run-cups-browsed
root      828451       1  0 Jun27 ?        00:00:00 /bin/sh /snap/cups/1058/scripts/run-cupsd
root      828648  828451  0 Jun27 ?        00:00:00 cupsd -f -s /var/snap/cups/common/etc/cups/cups-files.conf -c /var/snap/cups/common/etc/cups/cupsd.conf
root      828734  828443  0 Jun27 ?        00:00:00 /bin/sh /snap/cups/1058/scripts/run-cups-browsed
root     2458874       1  0 00:00 ?        00:00:00 /usr/sbin/cupsd -l
lp       2458875 2458874  0 00:00 ?        00:00:00 /usr/lib/cups/notifier/dbus dbus://
root     2458877       1  0 00:00 ?        00:00:00 /usr/sbin/cups-browsed
lp       2458897 2458874  0 00:00 ?        00:00:00 /usr/lib/cups/notifier/dbus dbus://
jbl      2539184 2453821  0 14:38 pts/5    00:00:00 grep --color=auto cups

Looks like it's running! But I don't want to have to issue that command every time I want to know if the service is running. And lets be honest, it's kind of noisy, and not the prettiest thing to look at if all I want to know is if the service is running.

Now with using 'hi', I can just use a similer ps -ef | grep -E "[c]ups" command as part of a newly defined check in config/checks.yml, as follows:

  CUPS:
    info: CUPS print service
    group: Media
    command: |
      ps -ef | grep -E "[c]ups"

Now when I run 'hi', I get a beautiful little indicator that tells me exactly what I want to know, amongst other information that might be important to me in general.

Output showing CUPS service indicator:

For an exmaple of a more complex check, take a look at the one defined for ExpressVPN:

   ExpressVPN:
     info: For general privacy and security.
     group: Security
     command: |
       expressvpn status | grep -i connected | sed "s/\\x1b\\[[0-9;]*[mGK]//g"

In the case of ExpressVPN, I don't just want to check if the process is running, but I'm interested in capturing a portion of the command output to be included in the 'hi' tool reporting.

In order to leverage that output, there is a custom handler defined in host_information.py:

         elif 'expressvpn' in process.lower():    
             if re.search("Connected", output.strip()):    
                 output_messages.append(f"[✅] {process} Status: {output.strip()}")    
             else:    
                 output_messages.append(f"[❌] {process} Status: {output.strip()}")

Instrumenting 'sub-checks'

You can now define multiple sub-checks using the same check declaration pattern.

Here's an example of how sub-checks are configured. We will use the 'Thunderbird' check which comes pre-defined in config/checks.yml.

In the following check defined for Thunderbird, you can see that not only is the main check for the running process in place, but there are also two sub-checks defined for checking Thunderbird's 'Memory Usage' and 'Version' information.

  Thunderbird:
    info: My favorite mail client
    group: Communications
    indicators:
      positive:
        status:
        icon: 📧
      negative:
        status:
        icon: 📧
    command: |
      ps -ef | grep -E "[t]hunderbird"
    sub_checks:
      Memory Usage:
        command: |
          ps --no-headers -o rss -C thunderbird | awk '{sum+=$1} END {printf "%.2f MB (%.2f GB)\n", sum/1024, sum/1048576}'
        indicators:
          positive:
              icon: 💾
          negative:
              icon: 💾
      Version:
        command: |
          thunderbird --version

Custom Indicators

In the above Thunderbird check configuration example, you can also see the custom indicators which can be defined within the config/checks.yml file.

These custom indicators, when defined, will override the default 'hi' check indicators (in most cases).

You can see that custom indicators have been defined not just for Thunderbird, but also for it's 'Memory Usage' sub-check. The 'Version' sub-check does not have any indicators defined, and so that sub-check will use the default indicators.
The flexibility of the 'hi' tool's bash-based checks makes this fairly easy to implement.

Conclusion

I have found using this tool to be an invaluable pleseant suprise, to both develop and to use.

As I mentioned in my previous article, there are many tools out there that can do some excellent reporting on host services, including running processes, CPU utilization, and file system services.

However I've never come across such a tool that allowed me to customize the reporting, to show exactly what I wanted, and ONLY what I wanted - in a nice, organized, terminal-friendly way.

I plan to continue development on this tool as time permits. To keep me motivated and focused, I've added a ROADMAP.md file to the repository that highlights some ideas and upcoming features that I plan to explore and implement.

I hope you find it a joy to use as much as I do. If you want to hear more about 'hi', visit my website The DevOps Joint, or connect with me on LinkedIn, or @jbobbylopez on X.com.

Enjoy!