Forem: Mark Caudill

My Bash Prompt

Mark Caudill — Wed, 27 Feb 2019 16:31:03 +0000

Note: this is republished from my blog and slightly out of date in that I now mostly use Fish but don't have a fun prompt yet.

I recently updated my Bash prompt. It started because I wanted to incorporate git status information into the prompt. But once I got that working it turned out I didn't really like it so I ditched it but kept the other stuff I did.

Here's the relevant portion from my .bashrc:

# Prompt
gen_prompt() {
    ## Looks like:
    ## ╭─┤pty0├─╢mark@bender:~/src╟─┤(✿◠‿◠)├─╮
    ## ╰>
    local __last_exit=$?
    [[ -f ~/.colors ]] && source ~/.colors

    local __line_color=${Red}
    local __prompt_color=${White}
    local __text_color=${White}

    local __success_string="${Green}(✿◠‿◠)"
    local __failure_string="${Red}(◡﹏◡✿)"
    local __exit_string=$(if [[ $__last_exit -ne 0 ]]; then echo $__failure_string; else echo $__success_string; fi)
    local __prompt="> "

    PS1="\n${__line_color}╭─┤${__text_color}\l${__line_color}├─╢${__text_color}\u@\h:\w${__line_color}╟─┤${__exit_string}${__line_color}├─╮\n╰${__prompt}\[$(tput sgr0)\]"
}
export PROMPT_COMMAND=gen_prompt

Here's colors if you want the color commands to work (save to ~/.colors):

# Reset
    Color_Off="\[\033[0m\]"       # Text Reset

    # Regular Colors
    Black="\[\033[0;30m\]"        # Black
    Red="\[\033[0;31m\]"          # Red
    Green="\[\033[0;32m\]"        # Green
    Yellow="\[\033[0;33m\]"       # Yellow
    Blue="\[\033[0;34m\]"         # Blue
    Purple="\[\033[0;35m\]"       # Purple
    Cyan="\[\033[0;36m\]"         # Cyan
    White="\[\033[0;37m\]"        # White

    # Bold
    BBlack="\[\033[1;30m\]"       # Black
    BRed="\[\033[1;31m\]"         # Red
    BGreen="\[\033[1;32m\]"       # Green
    BYellow="\[\033[1;33m\]"      # Yellow
    BBlue="\[\033[1;34m\]"        # Blue
    BPurple="\[\033[1;35m\]"      # Purple
    BCyan="\[\033[1;36m\]"        # Cyan
    BWhite="\[\033[1;37m\]"       # White

    # Underline
    UBlack="\[\033[4;30m\]"       # Black
    URed="\[\033[4;31m\]"         # Red
    UGreen="\[\033[4;32m\]"       # Green
    UYellow="\[\033[4;33m\]"      # Yellow
    UBlue="\[\033[4;34m\]"        # Blue
    UPurple="\[\033[4;35m\]"      # Purple
    UCyan="\[\033[4;36m\]"        # Cyan
    UWhite="\[\033[4;37m\]"       # White

    # Background
    On_Black="\[\033[40m\]"       # Black
    On_Red="\[\033[41m\]"         # Red
    On_Green="\[\033[42m\]"       # Green
    On_Yellow="\[\033[43m\]"      # Yellow
    On_Blue="\[\033[44m\]"        # Blue
    On_Purple="\[\033[45m\]"      # Purple
    On_Cyan="\[\033[46m\]"        # Cyan
    On_White="\[\033[47m\]"       # White

    # High Intensty
    IBlack="\[\033[0;90m\]"       # Black
    IRed="\[\033[0;91m\]"         # Red
    IGreen="\[\033[0;92m\]"       # Green
    IYellow="\[\033[0;93m\]"      # Yellow
    IBlue="\[\033[0;94m\]"        # Blue
    IPurple="\[\033[0;95m\]"      # Purple
    ICyan="\[\033[0;96m\]"        # Cyan
    IWhite="\[\033[0;97m\]"       # White

    # Bold High Intensty
    BIBlack="\[\033[1;90m\]"      # Black
    BIRed="\[\033[1;91m\]"        # Red
    BIGreen="\[\033[1;92m\]"      # Green
    BIYellow="\[\033[1;93m\]"     # Yellow
    BIBlue="\[\033[1;94m\]"       # Blue
    BIPurple="\[\033[1;95m\]"     # Purple
    BICyan="\[\033[1;96m\]"       # Cyan
    BIWhite="\[\033[1;97m\]"      # White

    # High Intensty backgrounds
    On_IBlack="\[\033[0;100m\]"   # Black
    On_IRed="\[\033[0;101m\]"     # Red
    On_IGreen="\[\033[0;102m\]"   # Green
    On_IYellow="\[\033[0;103m\]"  # Yellow
    On_IBlue="\[\033[0;104m\]"    # Blue
    On_IPurple="\[\033[10;95m\]"  # Purple
    On_ICyan="\[\033[0;106m\]"    # Cyan
    On_IWhite="\[\033[0;107m\]"   # White

Cloudflare Dynamic DNS Updater

Mark Caudill — Wed, 27 Feb 2019 16:26:45 +0000

Here's a quick Python script I wrote that, when provided with your Cloudflare account email address and API token, can update a DNS entry for you. It's designed to be used in a cron job but can be run one-off as well.

#!/usr/bin/env python                                      

import argparse                                                                                                       
import CloudFlare                                          
import logging                                             
import requests                                            

def public_ip(service='http://ip.mrkc.me'):                                                                           
    """                                                    
    Get the apparent public IP of this computer. This does not imply that the
    necessary network configurations are in place to allow public access.

    Raises Exceptions for anything but a 200 response.

    :param service: the url to perform a GET against (should return only an IP)
    :type  service: str
    :returns: this machine's IP as seen by a server on the Internet
    :rtype: str
    """
    logging.debug("Looking up public IP")
    r = requests.get(service)
    r.raise_for_status()
    logging.debug("Found " + r.text)
    return r.text

def update_cf_dns(cf, name, ip_address):
    # 'AAAA' for IPv6, 'A' for IPv4 (default)
    if ':' in ip_address:
        ip_address_type = 'AAAA'
    else:
        ip_address_type = 'A'
    logging.debug("Address type " + ip_address_type)

    # Get the Zone ID
    logging.debug("Looking up zone id for " + name)        
        for zone in cf.zones.get():
            if '.'.join(name.split('.')[-2:]) in zone['name']:
                zone_id = zone['id']
                logging.debug("Found " + zone_id)
                break
        else:
            raise Exception("Unable to find zone id for " + name)


        # Get the DNS record
        logging.debug("Fetching DNS record")
        dns_record = cf.zones.dns_records.get(zone_id,
                params={'name': name, 'match': 'all', 'type': ip_address_type})[0]
        logging.debug("Found " + dns_record['id'])

        # Check to see if the IP has changed
        if dns_record['content'] != ip_address:
            logging.info("Performing DSN record update", dns_record['content'],
                    "->", ip_address)
            cf.zones.dns_records.put(zone_id, dns_record['id'],
                    data={'name': name,
                        'type': ip_address_type,
                        'content': ip_address})
        else:
            logging.debug("Old Address:" + dns_record['content'] +
                    "New Address:" + ip_address)
            logging.info("Not updating")




            def main():
                parser = argparse.ArgumentParser()
                parser.add_argument('--email', type=str, required=True)
                parser.add_argument('--token', type=str, required=True)
                parser.add_argument('--domain', type=str, required=True,
                        help='the domain name to update')
                args = parser.parse_args()
                logging.basicConfig(format='%(asctime)s %(message)s',
                        level=logging.INFO)
                cf = CloudFlare.CloudFlare(email=args.email, token=args.token)
                update_cf_dns(cf, args.domain, public_ip())

            if __name__ == '__main__':
                main()

You can then invoke it with a cron job:

*/15 * * * *  /root/updatedns.py --email cfemail@domain.tld --token TOKEN --domain something.com 2>&1 | logger -t "updatedns.py"

Encryption with a Funky Partition Layout

Mark Caudill — Wed, 27 Feb 2019 16:24:04 +0000

Note: to see how I setup LVM with caching, check out my previous post.

I recently ran into some trouble with the system while mucking around and decided to take the opportunity to restructure my partitioning layout and do a full OS reinstall; this time with full disk encryption (except /boot). Suffice it to say, the Fedora installer is fairly flexible, but not nearly enough to support a mix of mdadm, LUKS, and LVM.

Before I tell you–roughly–how I did it, here are the results:

NAME                                              MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda                                                 8:0    0 447.1G  0 disk
├─sda1                                              8:1    0   512M  0 part  /boot
└─sda2                                              8:2    0 446.6G  0 part
  ├─vg_SIIIKE-root                                253:0    0  59.6G  0 lvm
  │ └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE   253:3    0  59.6G  0 crypt /
  ├─vg_SIIIKE-lv_swap                             253:1    0  14.9G  0 lvm
  │ └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE   253:2    0  14.9G  0 crypt [SWAP]
  └─vg_SIIIKE-home                                253:4    0 372.1G  0 lvm
    └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE   253:9    0 372.1G  0 crypt /home
sdb                                                 8:16   0 119.2G  0 disk
└─sdb1                                              8:17   0 119.2G  0 part
  ├─vg_SIIIKE-lv_var_cache_cdata                  253:5    0   115G  0 lvm
  │ └─vg_SIIIKE-lv_var                            253:8    0   1.8T  0 lvm
  │   └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE 253:10   0   1.8T  0 crypt /var
  └─vg_SIIIKE-lv_var_cache_cmeta                  253:6    0     1G  0 lvm
    └─vg_SIIIKE-lv_var                            253:8    0   1.8T  0 lvm
      └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE 253:10   0   1.8T  0 crypt /var
sdc                                                 8:32   0   1.8T  0 disk
└─md0                                               9:0    0   1.8T  0 raid1
  └─vg_SIIIKE-lv_var_corig                        253:7    0   1.8T  0 lvm
    └─vg_SIIIKE-lv_var                            253:8    0   1.8T  0 lvm
      └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE 253:10   0   1.8T  0 crypt /var
sdd                                                 8:48   0   1.8T  0 disk
└─md0                                               9:0    0   1.8T  0 raid1
  └─vg_SIIIKE-lv_var_corig                        253:7    0   1.8T  0 lvm
    └─vg_SIIIKE-lv_var                            253:8    0   1.8T  0 lvm
      └─luks-SIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIIKE 253:10   0   1.8T  0 crypt /var

To clarify, I have an SSD with /, /home, and swap on it and a software RAID-1 set with an SSD as a caching layer in front of it. The partitions (sda2, sdb1, and md0) are all physical volumes in the same volume group. I simply created logical volumes in that volume group and directed that they reside on certain physical volumes (such a handy feature). Then it was simply a matter of encrypting them with LUKS, configuring crypttab, and configuring fstab.

Unfortunately, you can accomplish very few of these tasks in the Fedora Desktop installer. I initially tried to configure everything outside of the installer hoping that it would get the hint and leave everything alone. The problem there is that the Workstation version doesn't know how to handle software RAID sets properly. I tried using the Server version but it did its own munging of things by making certain assumptions about how I wanted to do LVM.

In the end, since really only /var was going to be special, I decided to install everything on the primary SSD then after the installation configure the /var volumes and migrate the newly installed /var over to its new home. This worked better than expected and other than having to boot single to fix a fat-fingered fstab everything booted right up.

While I don't think the Fedora installers need the ability to generate arbitrary partitioning schemes, it would be nice to have some sort of "I know what I'm doing please leave my partitions alone" mode.

Leveraging LVM Caching

Mark Caudill — Wed, 27 Feb 2019 16:22:04 +0000

Note: I'm migrating relevant blog posts I've written over to dev.to. This is the first one.

I recently implemented LVM caching based largely on Richard WM Jones' post and thought I'd document my own experience here.

I started with an SSD containing /, /home, and swap, two 2 TB HDDs (sdb, and sdc) and an additional SSD (sdd).

[root@bender ~]# lsblk
NAME            MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda               8:0    0 447.1G  0 disk
├─sda1            8:1    0   500M  0 part /boot
└─sda2            8:2    0 446.7G  0 part
  ├─fedora-root 253:0    0    50G  0 lvm  /
  ├─fedora-swap 253:1    0  15.7G  0 lvm  [SWAP]
  └─fedora-home 253:2    0   381G  0 lvm  /home
sdb               8:16   0   1.8T  0 disk
sdc               8:32   0   1.8T  0 disk
sdd               8:48   0 119.2G  0 disk

I then used fdisk to create a Linux partition on each HDD and a Linux LVM partition on the SSD. Though, these partition types don't actually matter much for our purposes beyond looking nice.

[root@bender ~]# lsblk
NAME            MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sda               8:0    0 447.1G  0 disk
├─sda1            8:1    0   500M  0 part /boot
└─sda2            8:2    0 446.7G  0 part
  ├─fedora-root 253:0    0    50G  0 lvm  /
  ├─fedora-swap 253:1    0  15.7G  0 lvm  [SWAP]
  └─fedora-home 253:2    0   381G  0 lvm  /home
sdb               8:16   0   1.8T  0 disk
└─sdb1            8:17   0   1.8T  0 part
sdc               8:32   0   1.8T  0 disk
└─sdc1            8:33   0   1.8T  0 part
sdd               8:48   0 119.2G  0 disk
└─sdd1            8:49   0 119.2G  0 part

I then created a RAID 1 set using the two HDD partitions.

[root@bender ~]# mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sdb1 /dev/sdc1
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
[root@bender ~]# echo 'DEVICE /dev/sdb1 /dev/sdc1' > /etc/mdadm.conf
[root@bender ~]# mdadm --detail --scan >> /etc/mdadm.conf
[root@bender ~]# cat /etc/mdadm.conf
DEVICE /dev/sdb1 /dev/sdc1
ARRAY /dev/md0 metadata=1.2 name=bender:0 UUID=2f9fd7a2:cf0d4dd8:579ec7ce:073d0886

Then I created the LVM volumes and an ext4 filesystem on the RAIDed volume.

[root@bender ~]# pvcreate /dev/md0 /dev/sdd1
  Physical volume "/dev/md0" successfully created
  Physical volume "/dev/sdd1" successfully created
[root@bender ~]# vgcreate data /dev/md0
  Volume group "data" successfully created
[root@bender ~]# lvcreate -n vol0 -L 1.5T data /dev/md0
  Logical volume "vol0" created.
[root@bender ~]# lvs
  LV   VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  vol0 data   -wi-a-----   1.50t
  home fedora -wi-ao---- 380.95g
  root fedora -wi-ao----  50.00g
  swap fedora -wi-ao----  15.69g
[root@bender ~]# pvs
  PV         VG     Fmt  Attr PSize   PFree
  /dev/md0   data   lvm2 a--    1.82t 326.89g
  /dev/sda2  fedora lvm2 a--  446.64g   4.00m
  /dev/sdd1         lvm2 ---  119.24g 119.24g
[root@bender ~]# mkfs.ext4 /dev/data/vol0
mke2fs 1.42.13 (17-May-2015)
Creating filesystem with 402653184 4k blocks and 100663296 inodes
Filesystem UUID: 51e8796a-5090-45ce-ae62-4944f2eb9132
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000, 214990848

Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done

Now for the caching configuration. It's important to specify the actual partition (sdd1) in the following commands to ensure that the cache gets created on that device and not arbitrarily within the volume group.

[root@bender ~]# vgextend data /dev/sdd1
  Volume group "data" successfully extended
[root@bender ~]# lvcreate -L 1G -n vol0_cache_meta data /dev/sdd1
  Logical volume "vol0_cache_meta" created.
[root@bender ~]# lvcreate -L 115G -n vol0_cache data /dev/sdd1
  Logical volume "vol0_cache" created.
[root@bender ~]# lvs
  LV              VG     Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
  vol0            data   -wi-a-----   1.50t
  vol0_cache      data   -wi-a----- 115.00g
  vol0_cache_meta data   -wi-a-----   1.00g
  home            fedora -wi-ao---- 380.95g
  root            fedora -wi-ao----  50.00g
  swap            fedora -wi-ao----  15.69g
[root@bender ~]# lvconvert --type cache-pool --poolmetadata data/vol0_cache_meta data/vol0_cache
  WARNING: Converting logical volume data/vol0_cache and data/vol0_cache_meta to pool's data and metadata volumes.
  THIS WILL DESTROY CONTENT OF LOGICAL VOLUME (filesystem etc.)
Do you really want to convert data/vol0_cache and data/vol0_cache_meta? [y/n]: y
  Converted data/vol0_cache to cache pool.
[root@bender ~]# lvconvert --type cache --cachepool data/vol0_cache data/vol0
  Logical volume data/vol0 is now cached.
[root@bender ~]# lvs
  LV   VG     Attr       LSize   Pool         Origin       Data%  Meta%  Move Log Cpy%Sync Convert
  vol0 data   Cwi-a-C---   1.50t [vol0_cache] [vol0_corig] 0.00   1.43            100.00
  home fedora -wi-ao---- 380.95g
  root fedora -wi-ao----  50.00g
  swap fedora -wi-ao----  15.69g

Then I mounted it at /var/data.

[root@bender ~]# echo '/dev/mapper/data-vol0 /var/data                 ext4    defaults        1 2' >> /etc/fstab
[root@bender ~]# mount -a
[root@bender ~]# df -h /var/data
Filesystem             Size  Used Avail Use% Mounted on
/dev/mapper/data-vol0  1.5T   69M  1.5T   1% /var/data