Forem: pr4th4m

Tmux - fast window switching across sessions

pr4th4m — Mon, 06 May 2019 02:46:38 +0000

Tmux window switching

Tmux already makes it easy to switch windows in a session, however, it's hard to switch to windows from another session. To make window switching across sessions fast we can leverage fzf

Using fzf-tmux

  tmux list-windows -a -F "#S:#I-#W" | fzf-tmux | cut -d "-" -f 1 | xargs tmux switch-client -t

Binding it with .tmux.conf

  bind -n ^f run-shell "tmux list-windows -a -F \"##S:##I-##W\" | fzf-tmux | cut -d \"-\" -f 1 | xargs tmux switch-client -t"

Stream cli output over http

pr4th4m — Mon, 06 May 2019 02:43:55 +0000

Runnel

A program to stream any command line output over http. This is useful when we want to display command output on client side. One such use case can be displaying live logs in browser which are running inside a container on server side.

Installation:

Using docker (recommended)

  git clone https://github.com/VeritasOS/runnel.git
  cd runnel

  # build docker image
  docker build -t runnel:latest .

  # start runnel server
  docker run --name runnel -d -p 127.0.0.1:9090:9090 runnel:latest

Using binary (not recommended, because system dependencies are taken care in docker file on behalf of you)

  git clone https://github.com/VeritasOS/runnel.git
  cd runnel

  ./bin/linux_64/runnel_server -p localhost:9090

Usage:

Trigger below commands to get live stream

  # Fire command with curl
  curl -X POST -H 'Content-Type: application/json' -d '{"cmd":"ping -c 3 google.com"}' http://localhost:9090/command

  # Get live output stream
  # replace uuid with one which you get from above command
  while true; do curl -sS -H 'Content-Type: application/json' 'http://localhost:9090/stream/fd4b1a38-94f4-4eba-80e7-50578ac4baae' | jq '.response'; done

Package:

Dependency: Install redis server
Use as golang package

  # Get lib
  go get github.com/veritasos/runnel/runnel

  # Fire command
  client := runnel.NewClient()
  key, err := client.RunCommand("ping", "-c 2 google.com")

  # Get output stream
  client := runnel.NewClient()
  output, err := client.Stream(key, 10)

Detailed documentation:

A much more detailed documentation is provided here runnel
Feel free to Contribute back

OpenEMM python client

pr4th4m — Fri, 03 May 2019 09:36:19 +0000

Py-OpenEMM

OpenEMM is a feature-rich web-based enterprise application for email marketing, newsletters and service mails.

Pre-requisites:

Python 2.7 or higher
Python Suds 0.4 or higher

Installation:

Clone repo

  git clone https://github.com/pratz/py-openemm.git

Configuration:

Move to directory where you have cloned the repo

  cd py-openemm/pyopenemm
  vi config.py

Open config.py file and enter openemm wsdl url, username and password

  OPENEMM_URL = 'http://127.0.0.1:8080/cms_services/urn:agnitas-webservice?wsdl'
  WEBSERVICE_USER = 'test_user'
  WEBSERVICE_PASSWORD = 'test_123'

Usage:

Create client connection

  from pyopenemm.connection.connect import create_connection
  client = create_connection()
  # connect to webservice , returned is suds client

Get OpenEMM client

  from pyopenemm.webservice.service import OpenEMM
  openemm = OpenEMM(client)

Find subscriber

  # get subcriber by email
  subscriber = openemm.find_subscriber(('email','test@gmail.com'))

Get subscriber details

  subscriber_details = openemm.get_subscriber(subscriber)

Add new subcriber

  user_dict = {'email':'openemm@openemm.com','firstname':'openemmfirstname','lastname':'openemmlastname','gender':0}
  new_subscriber = openemm.add_subscriber(user_dict,True,'email',True) # add new subscriber to openemm

  # Four parameters of add_subscriber() method
  # user_dict = Dictionary containing user information
  # double_check - If True, check if subscriber is already in database
  # key_column - column used for double_check
  # overwrite - If True, subscriber gets updated

Feel free to Contribute back

Netrc python client

pr4th4m — Fri, 03 May 2019 09:31:07 +0000

What is Netrc file ?

Netrc file contains user credentials and is used to auto-login. It is usually located in users home directory .netrc but location can be overridden with NETRC environment variable. Netrc also supports macros macdef to automate tasks. Netrc can be used with ftp, curl, git etc.

Netrc file sample:

  ~ $ cat .netrc
  machine product.company.com
  login first.last
  password secret-password

  macdef macro-name1
  command1
  command2

Get client:

  curl -o netrc_client.py  "https://gist.githubusercontent.com/pratz/789dc165c6d9f79be86608547b128c69/raw/41fb7796f6b6d0a09ac5ddeaf8c79de30a2387ed/NetRc%2520-%2520read,%2520write,%2520update"

Client compatibility:

Linux/Mac
Python 2.7.x

Usage:

  # create client
  netrc_instance = NetRc(
          "product.company.com",
          login="first.last",
          password="secret-password",
          account="", # account is optional
          )

  # create or update file content
  netrc_instance.create_or_update()

  # Custom file path (not recommended, as other programs might look for .netrc in users home directory)
  NETRC_FILE_PATH = /custom/file/path
  netrc_instance.create_or_update(path=NETRC_FILE_PATH)

Warning:

Netrc stores credentials in plan text. This is how netrc is meant to be ;)

Use case:

Its a good idea to use netrc when you have token based authentication.
Automation for service accounts with token based authentication.

Artifactory python client

pr4th4m — Fri, 03 May 2019 09:22:34 +0000

Py-Artifactory

Artifactory is a artifact repository manager which supports software packages created by different technologies.
It can also be integrated with major CI/CD and DevOps tools.
This article shows how we can communicate with artifactory using a python api client.

Pre-requisites:

Python 2.7 or higher
libxml2/libxslt (will be deprecated in future releases)

  # Debian based
  sudo apt-get install libxml2-dev libxslt1-dev

  # RedHat based
  sudo yum install libxml2-devel libxslt-devel

Installation:

Fire-up below command in terminal, tag specifies a version number.

  pip install git+https://github.com/veritasos/py-artifactory.git@<tag>

Usage:

Create client instance

  from artifactory import Artifactory
  artifactory = Artifactory(
          url="http://127.0.0.1:8081",
          username="username",
          password="password",
          )

List users

  user_list = artifactory.security.users.list()

Get user

  user = artifactory.security.users.fetch("user.name")

Create user

  user = artifactory.security.users.new()
  user.name = "first.last"
  user.password = "test"
  user.email = "first.last@testartifactory.com"
  user.groups = ["readers"]
  response = user.create()

And much more ......

  Artifactory Users
  Artifactory Groups
  Artifactory Permissions
  Artifactory Repositories
  Artifactory Repository Replication
  Artifactory LDAP
  Artifactory User Api Keys

Detailed documentation:

A much more detailed documentation is provided here py-artifactory
Feel free to Contribute back

Git login with Netrc

pr4th4m — Thu, 02 May 2019 13:18:02 +0000

What is Netrc file ?

Netrc file:

Create file named .netrc in home directory.
Lets consider your git server is hosted on domain git.company.com

  ~ $ cat .netrc
  machine git.company.com
  login first.last
  password secret-password

Secure netrc file:

As .netrc is used to store credentials, lets secure the file.

  ~ $ chmod 0600 ~/.netrc

That's it, next time when you use git for domain git.company.com, git should pick up the credentials on behalf of you ;)

Warning:

Netrc stores credentials in plan text. This is how netrc is meant to be ;)

Use case:

Its a good idea to use netrc when you have token based authentication.
Automation for service accounts with token based authentication.

Custom git credential helper

pr4th4m — Thu, 02 May 2019 13:08:34 +0000

Git credential helper:

Git credential helper is used to save user credentials, so that user does not require to enter credentials on each git operation.
Git provides few default git credential helpers, see how to use them.
This blog will demonstrate on how to build custom git credential helper.

Simple python cli:

Let's write a python cli called auth_helper.py

  #!/usr/bin/env python
  import argparse

  class Credential(object):
      def get(self):
          # logic to get username/password from auth file
          pass
      def store(self):
          # logic to store username/password to auth file
          # its better to encrypt password if its in plain text
          pass
      def erase(self):
          # logic to delete auth file
          pass

  def main():
      parser = argparse.ArgumentParser()
      parser.add_argument('operation', action="store", type=str,
              help="Git action to be performed (get|store|erase)")
      # parser all arguments
      arguments = parser.parse_args()
      # get credentials
      credentials = Credential()

      if arguments.operation == "get":
          creds = credentials.get()
          print("username={0}".format(creds.get("username")))
          print("password={0}".format(creds.get("password")))
      elif arguments.operation == "store":
          credentials.store()
          # if credentials are already stored do not store again
      elif arguments.operation == "erase":
          credentials.erase()
      else:
          print "Invalid git operation"

  if __name__ == "__main__":
      main()

This cli takes three git operations as cli arguments get, store and erase.
These arguments are not coincident, they are used by git. Let's know more about them.

Git credentials store:

Git credentials store looks for three arguments
- get: called when triggered git pull, git fetch, git push etc.
- store: called when triggered git pull, git fetch, git push etc.
- erase: if our provided credentials fail, git will fallback to its own credentials prompt, if this fails as well erase is called.

Git helper configuration:

To configure our cli as git helper, trigger the below command

  git config --global credential.https://git.company.com.helper "./path/to/cli/auth_helper.py"

This will define auth_helper.py under credential section of .gitconfig file (user's home directory).
https://git.company.com is the domain where git is hosted.

Done!! Next time we use git, our auth_helper.py should provide credentials for git authorization.

Custom ansible modules

pr4th4m — Thu, 02 May 2019 12:58:56 +0000

How to write ansible custom modules

Ansible modules are easy way to interact between an existing application and ansible playbook. This blog covers how to write your own ansible module.

Module

Create a python file called greet.py with following content

  def main():
      module = AnsibleModule(
          argument_spec = dict(
              message = dict(required=True, type="str"),
          ),
          supports_check_mode=True
      )

      # get module params
      message = module.params.get("message")

      try:
          module.exit_json(changed=True, msg=message)
      except:
          module.fail_json(msg="It's bad not to greet someone")

  from ansible.module_utils.basic import AnsibleModule
  if __name__ == "__main__":
      main()

File name greet.py is considered as ansible module.
Arguments defined in ansible playbooks will be parsed by AnsibleModule class.
Ansible module always returns json, for convenience ansible provides two methods, one for success and other for failure

  # For success
  module.exit_json(changed=True, msg=message)

  # For failure
  module.fail_json(msg="It's bad not to greet someone")

Usage

Create an ansible playbook called playbook.yml

  touch playbook.yml

Create directory library alongside playbook.yml and copy greet.py to library

  mkdir library
  cp greet.py library

Now our directory structure should look like

  library
      |- greet.py
  playbook.yml

We can now use greet module with our playbook.yml

  ---
  - hosts: localhost
    tasks:
    - name: Lets start greeting
      greet:
          message: "Good morning"

Setup NFS and SMB server on Raspberry Pi

pr4th4m — Thu, 02 May 2019 12:44:16 +0000

NFS server

Install server

  sudo apt install nfs-common nfs-server

Create directory you wanna share

  mkdir /media/storage

Edit file /etc/exports with below content

  /media/storage 192.168.1.0/24(rw,all_squash,insecure,no_subtree_check)
  # <directory> <who_can_access> <options>

Restart services

  sudo service nfs restart
  /etc/init.d/nfs-kernel-server restart

Connecting from client (MacOS)

  mkdir /Users/username/nfs_shared
  sudo mount -v -o "resvport" 192.168.1.9:/media/storage /Users/username/nfs_shared

Samba server

Install server

  sudo apt-get install samba samba-common-bin

Create samba password for system user

  sudo smbpasswd -a pi

Create directory you wanna share

  mkdir /media/storage

Edit file /etc/samba/smb.conf with below content

  [PI]
  comment = Pi workspace
  path = /media/storage
  create mask = 0775
  directory mask = 0775
  read only = no
  browseable = yes
  public = no
  force user = pi
  only guest = no

Restart service

  sudo service samba restart

Connecting from client (MacOS)

  mkdir /Users/username/smb_shared
  mount -t smbfs //pi@192.168.1.9/pi /Users/username/smb_shared

On-prem ansible galaxy

pr4th4m — Thu, 02 May 2019 11:12:51 +0000

Generic ansible roles can be shared through ansible galaxy, however, its hard to access ansible-galaxy from within corporate network. A good alternative is to host something similar to ansible-galaxy on-prem.

On-prem galaxy repository

For this blog stash is considered (https://stash.company.com), however, gitlab, gogs or similar should work as well
Create a new project on stash Ansible Galaxy with key AG
Create repo for each generic ansbile role.
All generic ansbile roles can be browsed at https://stash.company.com/projects/AG
Now these roles can be used with multiple ansible playbooks.

Usage

Create a new file requirements.yml alongside ansible playbook.yml.
For this blog purposes I am considering awscli as an generic ansible role with its own repo, which resides in our newly created stash project Ansible Galaxy (AG)
Copy/paste below content to requirements.yml

  ---
  name: awscli
  src: git+https://stash.company.com/scm/ag/awscli
  version: master
  path: ~/.ansible/roles

  # name - name of role
  # src - location of role
  # version - which branch the role should be installed from
  # path - where we want to install roles

The requirements.yml files acts as a role dependency list for our playbook.yml

Installation

Once we define our ansible playbook dependencies, its time to install them.

  sudo ansible-galaxy install -r requirements.yml
  # NOTE: no need to install ansible-galaxy seperately, it ships with default ansible installation

Done! all the roles are now installed and ready to be used by our playbook.yml

SemVer versioning utility

pr4th4m — Thu, 02 May 2019 10:57:21 +0000

Versioner

Version reader/writer for popular package managers as per SemVer

Installation

  pip install git+https://github.com/VeritasOS/versioner.git

Current support

npm
dep
json
toml
yaml (currently multi doc in single file not supported)

Usage

Read version (run command from project root)

  > versioner read npm  # projects based on npm
  "0.0.1"

  > versioner read dep  # golang projects
  "0.1.1"

Write version (run command from project root)

  # package.json
  ....
  "version": "1.1.1",
  ....

  # Release patch version
  > versioner write npm --inc-patch

  # package.json
  ....
  "version": "1.1.2",
  ....

  # Release minor version
  > versioner write npm --inc-minor

  ....
  "version": "1.2.0",
  ....

  # Release major version
  > versioner write npm --inc-major

  ....
  "version": "2.0.0",
  ....

Read version from custom file

  > versioner read yaml --file /root/app/version.yaml --key-depth metadata,version
  0.0.2

  version.yaml
  ---
  metadata:
    version: 0.0.2

Write version to custom file

  > versioner write yaml --inc-major --file /root/app/version.yaml --key-depth metadata,version

  version.yaml
  ---
  metadata:
    version: 1.0.0

Development

  git clone https://github.com/VeritasOS/versioner.git
  cd versioner
  pip install -e .

NOTE

This utility is tested for npm and dep.
Read/write for custom file may break, please feel free to send a patch.

VeritasOS / versioner

Version reader/writer for popular package managers as per SemVer

Versioner

Version reader/writer for popular package managers as per SemVer

Installation

pip install git+https://github.com/VeritasOS/versioner.git

Current support

npm
dep
json
toml
yaml (currently multi doc in single file not supported)

Usage

Read version (run command from project root)

  > versioner read npm  # projects based on npm
  "0.0.1"
  > versioner read dep  # golang projects
  "0.1.1"

Write version (run command from project root)

  # package.json
  ....
  "version": "1.1.1",
  ....

  # Release patch version
  > versioner write npm --inc-patch

  # package.json
  ....
  "version": "1.1.2",
  ....

  # Release minor version
  > versioner write npm --inc-minor

  ....
  "version": "1.2.0",
  ....

  # Release major version
  > versioner write npm --inc-major

  ....
  "version": "2.0.0",
  ....

Read version from custom file

  > versioner read yaml --file /root/app/version.yaml --key-depth metadata,version
  0.0.2

  version.yaml
  ---
  metadata:
    version: 0.0.2

Write version to custom file

  > versioner write yaml --inc-major --file /root/app/version.yaml --key-depth metadata,version

  version.yaml
  ---
  metadata:
    version: 1.0.0

Development

git clone

…

View on GitHub