Forem: paschal

Pipelining for Memcached

paschal — Sat, 24 May 2025 00:32:00 +0000

Typical Client-Server protocols operate a request - response model.

With the HTTP1 protocol for example, Request N+1 is blocked by Request N on a single TCP connection. With clients like web browsers, the workaround is to establish a pool of connections which allow multiple assets like images, scripts to be requested all at once. This means that requests N and N+1 could be sent all at once.

Pipelining was eventually implemented in HTTP1.1, which allows multiple requests to be sent over the same connection without waiting for the server's response.

Even with this solution, The requests must still be processed in order. This is the head of line (HOL) blocking problem, where the processing of these requests are still as fast as the slowest request.

HTTP2 solved this with multiplexing. Multiple requests which are distinguished by a unique stream ID can be interwoven on a single TCP connection and responses could be delivered out of order!

HTTP2 still suffered the HOL blocking problem at the TCP layer. This is because at the TCP layer, everything is a stream of bytes over a single connection and if any bytes are dropped (packet loss), all the bytes after (other requests) are blocked till TCP detects and retransmits.

With HTTP3, TCP was dropped for UDP which can handle streams, so only a single stream or request is blocked when there’s packet loss.

This is a brief and simplified summary but it helps paint the picture.

Why is Pipelining important ?

Redis added support for pipelining and explains why beautifully. Basically, for every request (without pipelining), the client has to issue a write syscall to copy bytes from the application in user mode to the tcp buffer in kernel mode and send this over the network. The server on the other hand, makes a read syscall to read the tcp buffer and processes the request. Pipelining doesn’t reduce the time the server uses to process these requests but saves us time as we won’t have to context switch multiple times when we issue these syscalls. Even in cases where the client and server are on the same machine and communicate via the loopback address, we still see improvements when we utilise pipelining.

Memcached

Memcached is a key-value store (like redis) written in C. I've been working with it a lot recently and it uses either its ASCII protocol or its binary protocol.

Starting a memcached server in verbose mode, i see these lines which show that the server supports both protocols and its up to the client to decide what to use:

$ memcached -vv
server listening (auto-negotiate)
server listening (auto-negotiate)
new auto-negotiating client connection
……………
Client using the ascii protocol

I noticed that the rust memcached client i was using didn’t have pipelining support for any commands except the “Get” command. The Set, Increment, Decrement commands have to be issued per key.

Then i realised that memcached’s ASCII protocol doesn’t support pipelining, only its binary protocol does. The ASCII protocol implements some form of pipelining for the get command.

get <key1> <key2>....<key N>
value 1, value 2, …. value N

The binary protocol implements pipelining in a different way. It has “quiet” variants of these commands. GetQuiet, SetQuiet, AddQuiet, IncrementQuiet etc.

The “quiet” variants tell the server to not respond. These responses are queued internally and are returned when the client sends a non-quiet command or a no-op. This returns all the queued keys.
Eg:

getq <key1> <key2>.....<key N>
….NO RESPONSE FROM THE SERVER
get key<N + 1>
value 1, value 2, value N, value N + 1.

Many clients are comfortable pipelining Get requests as these are read only operations but pipelining Increment requests can get tricky as these are not atomic. Some increments might fail and it’ll be difficult to track and account for.

bmemcached

I found a python library that implements the binary memcached protocol and the author also wrote a rust flavour.

To ensure we're using the binary protocol, i run the tests and look at memcached logs:

28: Client using the binary protocol
<28 Read binary protocol data:

The next hurdle is that these libraries don’t implement the quiet variant of the memcached commands. I have to fork the repo and implement IncrementQuiet (IncrementQ).

After some ChatGPT research, I learn that IncrementQ implements the same binary message as plain ol Increment. It just has a different binary code. This means this will be relatively easy to implement.

Testing our IncrementQ implementation

Connect to Memcached and set a key called "counter" to an intial value of 1 with a TTL of 5 seconds.

let client = MemcachedClient::new(vec!["127.0.0.1:11211"], 1).unwrap();
let key = "counter";
let value = "1";
client.set(key, value, 5000).unwrap();

Looking at memcached logs:

<27 Read binary protocol data:
<27    0x80 0x01 0x00 0x07
<27    0x08 0x00 0x00 0x00

<27 SET counter Value len is 1

>27 Writing bin response:
>27   0x81 0x01 0x00 0x00
>27   0x00 0x00 0x00 0x00
>27   0x00 0x00 0x00 0x00
>27   0x00 0x00 0x00 0x00
>27   0x00 0x00 0x00 0x00
>27   0x00 0x00 0x00 0x02

We see that the value is set and we get a response back.

Issue four commands to increment this "counter" quietly. We shouldn't see any responses which means that we're pipelining.

for x in 1..5{
    // This means increment key "counter" by 1 from its initial value "x" and set TTL to 7 seconds.
    let _ = client.increment_quiet("counter", 1, x, 7000).unwrap();
}

Taking a look at memcached logs:

incr counter 1, 2, 7000
<27 Read binary protocol data:
<27    0x80 0x15 0x00 0x07
<27    0x14 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x1b
<27    0x00 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x00
incr counter 1, 3, 7000
<27 Read binary protocol data:
<27    0x80 0x15 0x00 0x07
<27    0x14 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x1b
<27    0x00 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x00
incr counter 1, 4, 7000
<27 Read binary protocol data:
<27    0x80 0x00 0x00 0x07
<27    0x00 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x07
<27    0x00 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x00
<27    0x00 0x00 0x00 0x00

No responses!

Validate the counter's new value is 5.

let new_value: String  = client.get(key).unwrap();
assert_eq!(new_value, "5");

It pipelined ✅

Hello World in Go From Rust

paschal — Sun, 07 Jan 2024 20:31:26 +0000

Introduction

Suppose every software in the world was written in just one programming language Eg. C. In that case, there'll be no need for bindings and foreign function interfaces but since we have thousands of programming languages, genuine use cases arise where communicating across programming languages is necessary.

Some programming languages are more performant than others and you'll need to hand off computation-intensive tasks to these languages.
Some programming languages are state of the art in different areas, Eg, Javascript with Browsers, Python with Data, and C with Embedded systems, so it'll be easier to utilize the existing libraries instead of reinventing the wheel.
Some languages are strict and strongly typed; you want to prototype fast but not compromise on performance. An example is Game development or NGINX module extensions with Lua and C.

The Base C Layer

Golang uses cgo to communicate with external C libraries and Rust uses FFI (Foreign function interfaces) to export C functions. Additionally, a library for automatically generating bindings can be used to generate the header files (.h). In our case, we'll only be exporting two C functions, so using cbindgen is overkill but we'll use it regardless because why not?
Firstly, we'll write the rust functions we want to export. We'll start by creating a new rust library:

cargo new r --lib

This means we're creating a new rust library called r.

In our Cargo.toml, we'll specify that this crate is a dynamic library and then include the libc and cbindgen dependencies.

[package]
name = "r"
version = "0.1.0"
edition = "2021"

[lib]
crate-type = ["cdylib"]

[dependencies]
libc = "0.2.151"

[build-dependencies]
cbindgen = "0.26.0"

We'll need libc to access some C types, specifically the c_char type. This is because strings are UTF-32 encoded (4 bytes) in C while they're UTF-8 encoded in Rust, meaning they can have variable sizes (1 to 4 bytes).

Rust Functions

To write our hello_world function which will be called from Go, we'll need the no_mangle annotation so the linker can find the function while linking without the Rust compiler mangling it.

Then we can write our functions hello_world and hello_world_free:

extern crate libc;

use std::ffi::CString;

use libc::c_char;

#[no_mangle]
pub extern "C" fn hello_world(item: *mut c_char) -> *mut c_char {
    unsafe {
        let c_str =
            CString::new(format!("hello world {:?}", CString::from_raw(item))).expect("dont fail");
        c_str.into_raw()
    }
}

#[no_mangle]
pub extern "C" fn hello_world_free(item: *mut c_char) {
    unsafe {
        if item.is_null() {
            return;
        }
        let _ = CString::from_raw(item);
    };
}

hello_world accepts a mutable pointer to a char array as an argument and returns the same. The unsafe keyword is needed as the rust compiler can't guarantee that from_raw which entails dereferencing a null pointer, won't lead to undefined behavior. Finally, we use into_raw to hand ownership back to the caller, in this case, Go.

hello_world_free accepts a mutable pointer, and calls from_raw on it, so we can properly reconstruct it, claim ownership, and then drop it. This is to avoid memory leaks.

cbindgen::Builder::new()
        .with_crate(crate_dir)
        .with_config(config)
        .generate()
        .expect("Unable to generate bindings")
        .write_to_file("bindings.h");

To generate the bindings, we'll add a build script build.rs which uses cbindgen to generate the bindings.h file. Then we can run:

cargo build

If everything goes smoothly, we should have a compiled binary and a newly created bindings.h file with the following methods in it:

char *hello_world(char *item);

void hello_world_free(char *item);

Go

After building the Rust binary, we'll need to tell the linker where to find it and we can do this by passing the linker flags via cgo. We'll also need to include (think import) the bindings.h file so we can call the rust methods.

// #cgo LDFLAGS: -L/Users/pbaba/projects/go-rust/r/target/debug -lr
// #include <bindings.h>
import "C"

Finally, we can use the hello_world function and include the "merry christmas" string as an argument. We'll also defer the hello_world_free method call, which will free the memory referenced by the pointer b afterward.

a := C.CString("merry christmas")
b := C.hello_world(a)
defer C.hello_world_free(b)
fmt.Println(C.GoString(b))

Then we can run go run main.go and it's alive:

hello world. "merry christmas"

In some cases, we can do without the hello_world_free function and replace it with the custom free function provided by cgo

defer C.free(unsafe.Pointer(b))

This should replace the function we already have so we don't end up freeing the same location in memory twice and causing a double free:

malloc: Double free of object 0x7f8c340040a0

But for CStrings, The Rust documentation recommends we free this explicitly by reconstructing and then dropping it instead of using C's free.

The rest of the code is on GitHub.

Failure Detection. The Phi Accrual Failure Detector

paschal — Thu, 21 Oct 2021 10:02:57 +0000

Introduction

In distributed systems, Nodes need to know if their peers are still healthy and update their peer lists accordingly. The simplest way to do this is with heartbeats. If there is a network with nodes A, B, and C, A sends heartbeat messages to its peers B and C saying “I’m still here”. If B and C don't hear from node A after a particular period, they can assume that node A is dead and can no longer handle requests, and nodes B and C will stop sending traffic to node A.
The question is how long should nodes B and C wait for A before marking it as dead. Twenty seconds ?, Thirty seconds ?. If a very short time is chosen as the threshold, there will be lots of false positives on the “dead nodes” peer list. A lot of nodes will be marked as dead when they are actually not. They might have had bad internet connections etc. On the other hand, if a longer time is chosen as the threshold, there will be lots of failed requests because dead nodes will keep getting traffic as though they’re still active and dropping them. So what exactly is the sweet spot? The Phi Accrual failure detector.

The Phi Accrual Failure Detector

The Phi Accrual failure detector, which was originally introduced in this paper uses a suspicion model. For example, when nodes B and C startup, they’ll set the confidence level they have in node A to 0.5, where the scale is from 0 to 1. After a particular period, if node A doesn't respond, the confidence level is dropped by 0.1, which makes the new confidence 0.4. If it does respond, it is increased by 0.1 and the confidence is 0.6. A threshold can then be set, that if node A doesn't respond when it's down to 0.2 then we can mark the node as dead. This is a more compound approach because it recognizes the past response times of the node. Data can also be stored, and used to analyze and decide what to set the threshold for a node to be marked as dead to.
Simulation
I did a small simulation with Docker and Golang, and this is what it looked like.

Leader

The first component is the Leader. I set up the entity in leader.go. The Leader receives the heartbeats, updates the tables, manages the thresholds and the failure points.

var (
   threshold = 15
   failurePoint = 0.2
)

The threshold is set to 15 and is denominated to seconds . This means if we don't hear from a node after fifteen seconds, the confidence level will be decremented by 0.1.
The failurePoint is set to 0.2 and once the node’s confidence level reaches the failurePoint or lower, it will be marked as dead.

Follower

The next entity is the Follower which is setup in
follower.go. The followers will send periodic heartbeats to the leader. A heartbeat looks like this:

type HeartBeat struct {
   IpAddress string
   Pid       int
   Timestamp int64
}

The follower sends its IP address, process id (PID), and timestamp to the leader periodically.
I setup two docker containers, phi_leader to hold the leader executable and phi_follower for the follower executable.
I scaled the phi_follower instance to 5 instances on a docker swarm network.

# docker service scale phi_follower=5  

phi_follower scaled to 5
overall progress: 0 out of 5 tasks 
1/5: ready     [======================================>            ] 
2/5: ready     [======================================>            ] 
3/5: starting  [============================================>      ] 
4/5: preparing [=================================>                 ] 
5/5: ready     [======================================>            ]

After running this process for about five minutes, I tailed the leader’s logs and the peer list looks like this:

# docker service logs phi_leader -f
Received heart beat from process with IP 10.0.4.222 and PID 1
updating Accrual failure detection table...
    *------------*-----*-----------------*------------*
    ╪     IP     ╪ PID ╪   CONFIDENCE    ╪    TIME    ╪
    *------------*-----*-----------------*------------*
    ╪ 10.0.4.219 ╪   1 ╪        0.500000 ╪ 1634756932 ╪
    ╪ 10.0.4.223 ╪   1 ╪        0.500000 ╪ 1634756940 ╪
    ╪ 10.0.4.222 ╪   1 ╪        0.600000 ╪ 1634756942 ╪
    ╪ 10.0.4.224 ╪   1 ╪        0.600000 ╪ 1634756924 ╪
    ╪ 10.0.4.225 ╪   1 ╪        0.900000 ╪ 1634756922 ╪
    *------------*-----*-----------------*------------*
    ╪                    TOTAL PROCESSES ╪     5      ╪
    *------------*-----*-----------------*------------*

The five different nodes have been registered on the leader, with different confidence levels and timestamps. With the confidence levels, all the peers are healthy.
Now I stopped two nodes, and watched to see if the detector decrements their confidence levels.

// Stop first container
# docker stop 14d07e02a1ad            
14d07e02a1ad
// Stop second container
# docker stop 3829bb23fe41
3829bb23fe41

After 17 seconds, which is below the threshold of 15 seconds, a node hadn’t sent a heartbeat and the confidence level was decreased. Node with IP 10.0.4.223 has been dropped to 0.9

# docker service logs phi_leader -f
Checking for inactive processes...
Havent received a heartbeat from process with ip 10.0.4.223 and pid 1 for the past 17 seconds
Decrementing process confidence by 0.1
    *------------*-----*-----------------*------------*
    ╪     IP     ╪ PID ╪   CONFIDENCE    ╪    TIME    ╪
    *------------*-----*-----------------*------------*
    ╪ 10.0.4.219 ╪   1 ╪        1.000000 ╪ 1634757451 ╪
    ╪ 10.0.4.223 ╪   1 ╪        0.900000 ╪ 1634757439 ╪
    ╪ 10.0.4.222 ╪   1 ╪        1.000000 ╪ 1634757453 ╪
    ╪ 10.0.4.224 ╪   1 ╪        1.000000 ╪ 1634757440 ╪
    ╪ 10.0.4.225 ╪   1 ╪        1.000000 ╪ 1634757430 ╪
    *------------*-----*-----------------*------------*
    ╪                    TOTAL PROCESSES ╪     5      ╪
    *------------*-----*-----------------*------------*

After a few minutes of tailing the logs, one of the nodes go below the failure point of 0.2and is marked as dead. The leader removes this node from its peer list, and the total drops to 4

# docker service logs phi_leader -f
This process with ip 10.0.4.224 and pid 1 now has confidence 0.200000 which is below or equal to the failure point 0.200000 and will be marked as dead
    *------------*-----*-----------------*------------*
    ╪     IP     ╪ PID ╪   CONFIDENCE    ╪    TIME    ╪
    *------------*-----*-----------------*------------*
    ╪ 10.0.4.219 ╪   1 ╪        0.900000 ╪ 1634757996 ╪
    ╪ 10.0.4.223 ╪   1 ╪        1.000000 ╪ 1634757998 ╪
    ╪ 10.0.4.222 ╪   1 ╪        1.000000 ╪ 1634758005 ╪
    ╪ 10.0.4.227 ╪   1 ╪        0.700000 ╪ 1634758010 ╪
    *------------*-----*-----------------*------------*
    ╪                    TOTAL PROCESSES ╪     4      ╪
    *------------*-----*-----------------*------------*

So it worked! The code to the full implementation and simulation of the Phi Accrual Detector can be found here.
Thank you for reading!

Originally published here

Notes

The actual phi accrual detector is more complex than this, it uses a better probabilistic model like calculating the mean and standard deviation of previous results to determine the threshold and failure point instead of hardcoding the values as we did. You can find a good read here (phi = -log10(1 - F(timeSinceLastHeartbeat)) )

The billion user table - A Review

paschal — Sat, 24 Jul 2021 01:38:12 +0000

Introduction

The web currently works in silos. Every platform operates as an autonomous system. So every organization, for example, Google, has its database of users and a unique identifier for every user. This identifier can be an email, a phone number, etc.
If a user on Google wants to sign up on Facebook, they'll need to go through another signup process, as Facebook has no idea who the user is within Google's system. Solutions like social login have been helpful, but primarily, every platform requires a signup process. If Facebook wants to implement a feature to allow the user to fetch all their images on Google photos and post them on Facebook, Facebook and Google will have to communicate and this should happen via an API (Application Programmable Interface). This interface is necessary, primarily because the unique identifiers of the user on Facebook and Google databases will be different. Google will demand something that represents a unique identifier on their systems, maybe a Gmail address and Facebook will have to provide this to get meaningful results.

API Rules

So for Facebook to fetch a user's photos from Google, they'll need to access Google's APIs and adhere to certain rules.

Facebook will need Access keys and secrets to obtain third-party access to Google's APIs. These access keys can be revoked by Google at any time.
The requests will be rate-limited, as Google wouldn't want you overloading their infrastructure.
Google will need to keep these photos available via the API else Facebook will lose access to this information.

With these few rules, there are already too many variables, and this isn't a robust solution if we want to achieve maximum interoperability. Also, if Facebook wanted to allow their users to reuse their Twitter bios on Facebook, that will require another third-party request to Twitter, and it is clear how this can quickly spiral into a web of third-party dependencies, albeit hanging on a very small thread.

Solution

One solution can be making the database publicly accessible and verifiable. Unique identifiers are no longer localized and can be globally verifiable hashes that can be used across different autonomous systems. A globally verifiable unique ID can negate the need for multiple email addresses, phone numbers, etc. This database will be a Layer 1 infrastructure and different organizations can build up the stack.
Much is still left to be desired, for one, there'll be a need for private databases, as not all information should be publicly accessible. This privacy is similar to what the Taproot upgrade on the Bitcoin network is trying to solve, but with a publicly accessible and verifiable database, intercommunication will become smoother.

References

1729 - A billion user table

Write better Vue JS code

paschal — Fri, 27 Sep 2019 15:30:37 +0000

Introduction

Architecture might not matter at the start of a project, but the ease at which components can be added or removed without breaking stuff shows how well the codebase was structured. Let's look at ways to make our Vue JS code better.

Use State, Map Getters and Actions.

Using state and the maps (i.e mapGetters, mapActions, mapState, etc) provided by Vuex makes code very reusable. Hard coding state to the data() object in your SFC because its "faster" will raise difficulties if some of those values are needed in the future.

<!-- first.vue -->
<template>
  <h3>{{firstname}}{{lastname}}</h3>
</template>

<script>
  export default {
    data() {
      return {
        firstname: "",
        lastname: ""
      };
    },
    methods: {
      async getFullName() {
        const { firstname, lastname } = await fetchNameFromApi();
        this.firstname = firstname;
        this.lastname = lastname;
      }
    },
    created() {
      this.getFullName();
    }
  };
</script>

Project Manager: We need the firstname and lastname to show up on two more pages.

With that request, you'll keep copying, pasting, importing and exporting from different files.

Better Still,

const state = {
  firstname: "",
  lastname: ""
};

const actions = {
  async getFullName({ commit, dispatch }, data) {
    getFullNameFromApi().then(res => {
      commit(mutate.FULL_NAME, res.body);
    });
  }
};

const mutations = {
  //Set default mutation types in another file
  [mutate.UPDATE_FULL_NAME](state, data) {
    state.firstname = data.firstName;
    state.lastname = data.lastName;
  }
};

const getters = {
  firstName: state => state.firstname,
  lastName: state => state.lastname
};

const FullName = {
  state,
  actions,
  mutations,
  getters
};

export default FullName;

Then on our first.vue component,

<template>
  <h3>{{firstName}}{{lastName}}</h3>
</template>

<script>
  import {mapGetters, mapActions} from 'vuex';

  export default {
   methods:{
   ...mapActions(['getFullName']);
   },
   created(){
   this.getFullName();
   },
   computed:{
   ...mapGetters(['firstName', 'lastName']);
   }
  }
</script>

Now, if we need to include a new component that needs the first and last names of our user, we can easily map the getters and the actions.

This also helps us avoid things like:

const firstname = this.$store.state.fullName.firstName;
const lastname = this.$store.state.fullName.lastName;

We can simply use getters

computed:{
 ...mapGetters(['firstName','lastName'])
}

Finally, this helps us abstract business logic from the SFC and makes testing easier. Allow the Store to handle all the logic, and the SFC should just handle stuff tightly coupled to it, like the state of alert buttons/snack bars, etc.

Filters over Mixins.

Mixins lead to implicit dependencies, namespace clashes, etc. You can find more about that here. Some Mixins can be converted to Filters.

//dateMixin.js
export default {
  methods: {
    formatDate(date) {
      return date.split("T")[0];
    }
  }
};

In our SFC, we have:

<template>
  <h3>{{formatDate(date)}}</h3>
</template>

<script>
  import dateMixin from "./dateMixin";

  export default {
    mixins: [dateMixin],
    data() {
      return {
        date: "2019-08-07T00:00:00"
      };
    }
  };
</script>

With filters,

//main.js
import Vue from "vue";

Vue.filter("formatDate", value => value.split("T")[0]);

In our SFC,

<template>
  <h3>{{date | formatDate}}</h3>
</template>

<script>
  export default {
    data() {
      return {
        date: "2019-08-07T00:00:00"
      };
    }
  };
</script>

Use Modules to separate the different services on your application.

Instead of having everything needed by our state in one object, we can segregate them into modules.

Instead of

const state = {
  token: "",
  amount: "",
  firstname: "",
  lastname: "",
  email: "",
  isLoggedIn: ""
};

We can divide our services into authentication, profile-management and wallet.

Our folder structure would look like

modules
 authentication
    index.js
 profile-management
    index.js
 wallet
    index.js

In the index.js file, we can have the state that matters to that service.

//modules/authentication/index.js

const state = {
 token: '',
 isLoggedIn:''
}

...

Then when we initialize our store, we can add all the modules.

export const store = new Vuex.store({
 state: {
    //something general
    isAppBusy: false
 },
 modules:{
    authentication,
    profile-management,
    wallet
 }
});

Conclusion

These are my thoughts on how to make the structure of Vue code better. If you have extra additions or subtractions, I'll like to see it in the comments 😄.

Understand Master-Slave Relationships using the Election Algorithm

paschal — Fri, 09 Aug 2019 23:26:32 +0000

Introduction

While building distributed systems, Transparency is a very important factor. The Engineer has to consider Access transparency, Concurrency transparency, Location transparency, Replication transparency, etc. Replication transparency answers the question, 'Will my data resources always be consistent ?'.

What is Replication transparency?

With distributed systems, we can access different copies of our resources, which helps with redundancy, backup, speed, etc. Having replicas of a particular resource, also raises the issue of consistency. How do we ensure that all the replicas of a particular resource are consistent at all times? Two-phase commits can help in ensuring that if for any reason, all the replicas of a particular instance don't get updated may be due to timeouts or propagation errors, the instances will be rolled back to their previous state. This means that the update is lost and has to be done again.

Three models help us with handling replicas:

Primary-Backup / Master-Backup Model
Peer to Peer Model
Master-Slave Model

The Primary-Backup model exposes only one instance to all external processes. This instance is the master instance, and it has read and write permissions. All other instances or replicas have only read permissions. So with this model, we are sure that only one instance can be updated, and then the change is propagated. The drawback of this model is that it isn't scalable, because only one instance is exposed and if that instance crashes before propagation happens, we will still encounter inconsistencies.

The Peer to Peer model gives all the instances read and write permissions. With this model, we will observe performance issues, especially when we need to propagate very large chunks of data. Maintaining global consistency will also be difficult. It is best suited for applications that require low data replication. User-specific applications for example.

The Master-Slave model has one instance as the Master model, with read and write permissions. The other instances(slaves) have read permissions, but are "hot-spares" in the sense that immediately they notice that the Master node is down, a slave becomes the Master. It is best used for systems where reading operations are higher than writing. Eg. Databases. This is because to write or update an item on a database, it reads first (read-modify-write).

Which Slave is selected to be the Master?

This is where the Election algorithm comes in. It is used to elect a slave(to be master) after the master node fails.
We have the

Bully Election Algorithm
Ring Election Algorithm
Leader Preelection Algorithm

The Bully election algorithm takes the node with the highest ID as the next master. Once a node realizes that the master node has failed, the election process starts. If the last node to join the conversation is the node with the highest ID then the election process is going to take some time compared to when the node with the highest ID joins first.

The Ring election algorithm implements the Bully election algorithm but the nodes are arranged in a logical ring. This means each node sends messages to its neighboring nodes, and not to every node.

The Leader Preelection algorithm chooses the "backup" master node while the master node is still running. It still implements the election algorithm, but it happens while the master node is still running. This eliminates the overhead that happens with the other methods, but its also a waste of resources because the backup nodes can fail before the master, and then the elections will keep happening.

Simulating the Election Algorithm

We will be simulating the Bully election algorithm, using four docker containers which will represent our nodes. (1 master and 3 slaves) running NodeJS and a message-broker(Rabbitmq). I initially tried using actual VMs, Welp. Good luck with that.

To achieve this simulation, we'll have to:

Create a Docker network, which will host all the containers and the rabbitmq server.
Spin up the rabbitmq server, and bind the port to rabbitmq running on our localhost.
Spin up four docker containers from our Dockerfile.
Use the Pub/Sub pattern, and the fanout method, so that every node sends and receives messages from every node.

Create a Docker network

# The name of this network is election-algorithm_default
$ docker network create election-algorithm_default

# confirm it exists and copy the network id
$ docker network ls

The Rabbitmq Server

The Server will use the management alpine image, so ports 5672 and 15672 will be used. If any processes are running on these ports, you will need to kill them.

# Run the rabbitmq image in detached mode
$ docker run -it -d --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.6-management-alpine

# Confirm its running and copy the container id
$ docker container ls

Now, we can add the rabbitmq server to our network, so it can communicate with the other nodes.

# connect the rabbitmq server to the network
$ docker network connect <NETWORK_ID> <CONTAINER_ID_OF_THE_RABBITMQ_SERVER>
# Confirm its running
$ docker inspect election-alogithm_default
# You should see a "containers" key with the rabbitmq server.

Create Dockerfile

In our present directory, We'll need a server.js file and some dependencies.

$ npm init && npm i --save amqlib node-cron && touch server.js Dockerfile

Then our Dockerfile

FROM alpine:latest

WORKDIR /usr/src/app
# Install Node js and npm
RUN apk add --update nodejs npm

RUN npm install

COPY . .

CMD ["node","server.js"]

Now, we'll need to get the IP address of the Rabbitmq server, because that is what we'll connect our containers to. This will enable all the containers to see all the messages from neighboring containers or nodes.

$ docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' <CONTAINER_ID>
#OR
$ docker inspect <CONTAINER_ID> | grep "IPAddress"

We should be able to see our IP address from any of those results.

Server.js

In this file, every node sends a heartbeat to the rabbitmq server, which is a cron job that runs every 10 seconds. Every node can see all the responses and then sort the information according to the container ids. The container with the highest ID is automatically the master, and if that node fails, the next node takes over! We'll store the messages in a set so that there will only be unique ID's.

The server.js file should look like this

// Require libraries
const amqp = require("amqplib/callback_api");
const cron = require("node-cron");
const os = require("os");

//Connect to the IP address of the Rabbitmq container
const url = `amqp://guest:guest@${IP_ADDRESS_OF_THE_RABBITMQ_SERVER}`;

//The transmitter
const sendContainerIdToOthers = () => {
  /**
   * method for sending containerId to other nodes
   * @param {null}
   * @returns {null}
   *
   */
  // This returns the container id
  console.log(`My id is ${os.hostname()}`);

  //Connect to the server
  amqp.connect(url, (error0, connection) => {
    if (error0) throw error0;
    //Create channel
    connection.createChannel((error1, channel) => {
      if (error1) throw error1;
      //Create exchange
      const exchange = "logs";
      //Send Message indicating your ID
      const msg = `My id is ${os.hostname()}`;
      //Use the fanout mechanism
      channel.assertExchange(exchange, "fanout", { durable: false });
      //Publish this message
      channel.publish(exchange, "", Buffer.from(msg));
    });
  });
};

//The receiver
amqp.connect(url, (error0, connection) => {
  if (error0) throw error0;
  connection.createChannel((error1, channel) => {
    if (error1) throw error1;
    const exchange = "logs";
    channel.assertExchange(exchange, "fanout", { durable: false });

    channel.assertQueue("", { exclusive: true }, (error2, q) => {
      if (error2) throw error2;
      console.log(`Waiting for messages in ${q.queue}`);
      channel.bindQueue(q.queue, exchange, "");
      //Since we want the IDs to be unique, we'll use a set
      let resultSet = new Set();
      //Clear the set every 15 seconds
      setInterval(() => {
        resultSet = new Set();
      }, 15000);

      channel.consume(
        q.queue,
        msg => {
          if (msg.content) {
            console.log(`received: ${msg.content.toString()}`);
            //Split the response to get the ID
            const id = msg.content
              .toString()
              .split("is")[1]
              .trim();
            //Add ID to the set
            resultSet.add(id);

            console.log("Container id's", resultSet);
            //FInd the master node
            const findMaster = Array.from(resultSet).sort();

            console.log(`Our Master Node is ${findMaster[0]}`);
          }
        },
        {
          noAck: true
        }
      );
    });
  });
});

//Run every 10 seconds
cron.schedule("10 * * * * *", () => sendContainerIdToOthers());

Results

Now we can spin up four servers from the Dockerfile and connect them to the network

# build the image
$ docker build --tag=server1 .
# Run this command for three other servers, server2, server3, and server4.

#Run the image and connect the container to the network election-algorithm_default
$ docker run -it -d --network <NETWORK_ID> server1
# Run this command for three other servers, server2, server3, and server4.

#Confirm they are running
$ docker container ls | grep server1

After 10 Seconds, we can check the logs of any of our nodes

$ docker logs --follow <CONTAINER_ID>

Then, we'll see all the nodes join in, and how the master node is changed when a higher node comes in.

If we kill a node, we'll find out the next elected node according to ID, becomes the Master.

Conclusion

I just got started with Docker / Distributed systems, I hope this informs you a little. The repo for this is here.

Handling Pagination with Vue JS and Pouch DB

paschal — Wed, 03 Jul 2019 02:03:10 +0000

Introduction ☕️

When the responses from a server come after just one request, and we have to paginate on the front end, its easier to handle. This is because we have all the data on the client side, so we just display a range of indexes on a particular page. So if the response is an array of 400 items, we can display 20 items per page. But if for some reason, the responses from the server can't come in a single request, we'll need to use pagination or an infinite scroll. This depends on the use case. So what are the performance issues with pagination? and how do we solve them? Let's dive in.

The Problem 🍉

Let's say we have an endpoint that returns a payload like:

{
    totalNumber: 1000,
    data: [0,....,20],
    pageNumber: 1
}

From the object, we can see how much data we have in total, and the particular page we are on, so we know the range of page numbers we have. If each response returns an array with 20 elements, then we have (1000 / 20) i.e 50 pages.

...

With pagination, if we click on '1', it fetches the first twenty elements. Page 2 fetches the next twenty, coming back to page 1 fetches the first twenty elements again. So going back and forth is going to be really slow and nothing short of a hassle.

PouchDB: A solution

For handling storage on the client(browser), we have the local storage, session storage, indexed DB, cookies, etc. Pouch DB actually uses IndexedDB under the hood. Its mostly used on offline apps for automatic synchronization with the Live database, most likely Couch DB.

We'll be using Vue JS to explain how this works. Firstly, we'll install the packages needed.

npm i --save vuejs-paginate pouchdb-browser pouch-vue pouchdb-find pouchdb-live-find axios

vue create pouchdb-app

In our main.js file, we'll install and initialize the database.

// main.js 
import Vue from 'vue';
import Paginate from 'vuejs-paginate';
import PouchDB from 'pouchdb-browser';
import * as pouchVue from 'pouch-vue';

PouchDB.plugin(require('pouchdb-find'));
PouchDB.plugin(require('pouchdb-live-find'));

Vue.component('paginate', Paginate);

Vue.use(pouchVue, {
    pouch: PouchDB, 
    defaultDB: 'users_database' //You can give it any name 
});

// .............

Moving on to the vue file, where we display all our users. Assuming the response from our endpoint is something like this.

{
    total:'',
    pageNumber:'',
    users:[
        {
            firstname: '',
            lastname:''
        },
        {
            firstname: '',
            lastname:''
        },
    ]
}

We can display our first batch of users on the vue file.

<!-- users.vue -->
<div>
    <!--
        getUsers: {
            users: [],
            total: '' ,
            pageNumber: '' 
        }
    -->
    <v-data-table
        :headers="headers"
        :items="getUsers.users"  
        v-if="getUsers.users"
        hide-actions
    >
        <template slot="items" slot-scope="props">
        <td>{{ props.item.firstname}} {{props.item.lastname}}</td>
        </template>
    </v-data-table>
    <paginate
            :page-count="pageCount"
            :click-handler="fetchNext"
            :prev-text="'Prev'"
            :next-text="'Next'"
        />
</div>

Firstly, there is a table to display the first and last names of all the users. This table is from Vuetify.

Secondly, we have the paginate component, which has the page-count prop, this shows the number of paginated pages we have. If the response has a total of 400 users, and we receive 20 users on every request, the page-count will be (400 / 20) 20. The click-handler prop, accepts a function that runs when any page is clicked. The prev-text and next-text props just accept the text to be displayed for the previous and next pages.

Using the diagram above, we will create a getAllUsers action, that fetches the first batch of users, commits them to state and then stores them in PouchDB. We can also access Pouch DB from our single file component(SFC) using this.$pouch . This will be done on the created() lifecycle hook in the users.vue file.

//....
<script>
    //...users.vue
    import {mapActions, mapGetters} from 'vuex'; //Using vuex for our store
    export default {
        data(){
            return {
                headers: [
                    {
                        text: 'Name',
                        value: 'firstname',
                        align: 'left'
                    }
                ]
            }
        },
        created(){
            this.getAllUsers({
                pageNumber: 1, // Fetch first page
                pouch: this.$pouch //pass pouch db reference to action
            })
        },
        methods:{
            ...mapActions(['getAllUsers'])
        },
        computed:{
            ...mapGetters(['getUsers'])
        }
    }
</script>

Now we'll write the getAllUsers action, and other elements needed to complete our store.

//store.js
import Vue from 'vue';

//mutation type
const UPDATE_ALL_USERS = 'UPDATE_ALL_USERS';

//state
const state = {
    allUsers: null,
};

const getters = {
    getUsers: state => state.allUsers
}

const actions = {
    getAllUsers({commit}, data){
        // retrieve the pouch db reference and page number we just sent
        const {pageNumber, pouch} = data;

        //Using axios
        Vue.axios.get(`allUsers/?page=${pageNumber}`)
            .then(res =>{
                console.log('data retrieved');
                /*
                    res: {
                        users:
                        total:
                        pageNumber:
                    }
                */
                // store data in pouch db
                pouch.put({
                    _id: `${pageNumber}`,
                    users: res.data.users
                })
                .then(()=>{
                    console.log('your data has been stored');
                })
                .catch(e => console.log(e))
            });
        commit(UPDATE_ALL_USERS, res.data);
    }
}

const mutations = {
    [UPDATE_ALL_USERS](state, data){
        state.allUsers = data;
    }
}

So after it fetches a particular batch of users from the server, it caches it in pouch db, with the pageNumber variable as the id, and the users as the actual data.

Finally, we'll need to write the fetchNext function that occurs every time a particular page is clicked. So if the page is clicked, we'll check our cache, if it's there, display the result and if not call the getAllUsers action which will fetch from the server and then cache.

<!--users.vue-->
<script>
//.... This is a continuation of the users.vue file above
    methods: {
        fetchNext(event){
            // event is the particular number clicked
            this.$pouch.get(`${event}`)
                .then((doc) => {
                    //if it is found, store in the cachedUsers variable
                    this.cachedUsers = doc.users;
                })
                .catch(e => {
                    //if that page wasn't found, run the getAllUsers function
                    if(e.name === 'not_found'){
                        return this.getAllUsers({
                            pageNumber: event,
                            pouch: this.$pouch
                        })
                    }
                })
        }
    }
</script>

We need the cachedUsers variable because the getUsers variable is just a getter, and if the users are found in our cache, we need to be able to set our variable to that value.

<script>
    //.....
    computed:{
        ...mapGetters(['getUsers']),
        cachedUsers: {
            get(){
                // return the getter
                return this.getUsers
            },
            set(value){
                //use the setter
                this.getUsers.users = value;
            }
        },
        pageCount(){
            // get the total number of users and the amount of users per page to get the number of pages.
            const {total, users} = this.cachedUsers;
            //fix to one decimal space
            const numberOfPages = (Number(total) / users.length).toFixed(1);
            //if there is a decimal value, increase the number of pages by 1.
            return Number(numberOfPages.split('.')[1]) > 0
                ? Math.round(Number(numberOfPages)) + 1
                : Math.round(Number(numberOfPages));
        }
    }
</script>

Conclusion ✂️

This just shows one of the ways we can handle pagination on the front end, it also ensures things are fast. It helped me recently, and it can be written with any framework. Understanding the concept is what is important, and I hope you did. Thanks for reading!

This post was originally published on my blog

Handling Ever Increasing Server Logs

paschal — Sun, 19 May 2019 17:17:26 +0000

Introduction

I thought the game of thrones would be the most stressful thing I came across this week, but searching through logs, handling errors can be very tedious, and knowing what exactly is happening on your server is keen to find solutions. So let's find out how we can handle our server outputs.

PM2

PM2 is an advanced, production process manager for Node.js [1]. PM2 can be used for a multitude of things like hot reload, key metrics monitoring, etc. but it's also very useful for Log management.

So we can install PM2 on our NGINX server for instance, with:

npm i pm2 -g

So if we connect to our server, maybe via SSH on our terminal.

ssh -i "privatekey.pem" username@server-host-name

# Navigate to the pm2 folder
cd .pm2/logs

# List the files 
ls 

>server-out.log
>server-error.log

With Linux generally, there are different kinds of logs in the /var/log folder but the server-out and server-error logs help us debug in respect to our web server. The server-out.log outputs the console.log lines and the server-error.log outputs the errors our server encounters.

Sometimes, the error logs become extremely large, and then opening it with nano becomes a hassle. We can use the zip and wget packages to get the file locally.

sudo apt-get install zip

# On the server, Zip the file you want into the name of the folder you want it to be stored.

zip server-error.log server-error.zip

#On our client, install wget
sudo apt-get install wget

#Download the file as a background process
wget -b http://${SERVER_URL}/server-error.zip

PAPER TRAIL

Paper trail is cloud-hosted log management for faster troubleshooting of infrastructure and application issues [2]. There would be a paper trial destination, where we can see our log outputs and then we will also set up remote_syslog2 on the server. Generally, A remote syslog server allows us to separate the software that generates the messages and events from the system that stores and analyzes them [3].

# Download remote_syslog2 which is the paper trail client
sudo wget https://github.com/papertrial/remote_syslog2/releases/download/${VERSION}/${DEBIAN_FILE}

# Install the deb file with the package manager
sudo dpkg -i <DEBIAN_FILE>

# Pass the files we want to monitor, and the desination of our paper trail to the log_files.yml file.
sudo bash -c "cat > /etc/log_files.yml"  <<EOF
files:
  - /.pm2/logs/server-error.log
  - /.pm2/logs/server-out.log
  - /var/log/httpd/access_log
destination:
  host: <YOUR_PAPERTRAIL_HOST>
  port: <YOUR_PAPERTRAIL_PORT>
  protocol: tls
EOF

JFILE

The JFile module is simply a wrapper of the fs package in an object-oriented paradigm [4].

npm install JFile

//For exmple = new JFile(__dirname + "/file.txt")
const myFile = new JFile(__dirname + `${WHERE_YOU_WANT_THE_LOGS_STORED}`);

async function harryPotter(){
    const x = await fetch('http://example.com');

    // Add what you want to the myFile variable.
    myFile.text = 'Function harry potter has been called';
}

BUNYAN

Bunyan is a simple and fast JSON logging library [5]. Basically, it makes your errors look beautiful and more detailed.

const bunyan = require('bunyan');
const log = bunyan.createLogger({name:'hello'})

const findMyPizza = (err,location) => {
    if(err){
        log.info(err)
    } else {
        log.info('Yas');
    }

}

The default output has keys like name, hostname, pid, level etc.

ZAPIER HOOKS + SLACK

Zapier is a really nice tool for automating processes and workflows. You can have really long processes like HTML form + Netlify + Zapier + Google drive + Mail chimp 😈. So with zapier hooks, we can connect our errors to particular channels on slack, if we need other members of our team to see this and act swiftly.

 const errorObject = {
    Error: "We've been hacked!!!",
    Time: new Date(),
  };
  request.post(
    { url: `${LINK_TO_ZAPIER_HOOK}`, form: errorObject },
    (error, response, body) => {
        ...
    }
  );

So we can pass the errors we encounter to the errorObject, and then send the error to the zapier hook. The rest of the setup is handled on zapier directly.

Conclusion

Dealing with server logs is very important, and if not handled properly is very tedious. I hope this reduces the time you'll spend next time. This post was originally written on my blog

References

[1] PM2 Documentation.

[2] Papertrail documentation.

[3] Cisco Assets Remote Syslog Server Help File

[4] JFile Documentation.

[5] Bunyan Documentation.

Understand Logic and Boolean Algebra with De Morgan's law and Truth tables

paschal — Sat, 20 Apr 2019 05:03:43 +0000

Introduction

With De Morgan's law and truth tables, we will be able to simplify logical expressions and models, find possibilities and even bugs. These processes help us organize, simplify and almost even visualize how things will work. Truth tables are also useful in designing logic circuits and logic gates. Let's dive in.

De Morgan's Law

The complement of the union of two sets is the intersection of their complements and the complement of the intersection of two sets is the union of their complements.

That's a lot of gibberish, I know, but I understood this personally with this example

If U is {1,2,3,4,5,6}, A is {2,3} and B is {3,4,5}

//The union of A and B
A u B = {2,3,4,5}

// What the universal set contains and (A u B) doesn't
(A u B)' = {1,6}

// What the universal set contains and A doesn't
A' = {1,4,5,6} 

//What the universal set contains and B doesn't
B' = {1,2,6}

//The intersection of the complements of A and B
A' n B' = {1,6} 

A u B = A' n B'

In English, It can't be summer and winter at once, so its either not summer or not winter. And it's not summer and not winter if and only if it's not the case that its either summer or winter. Following this reasoning, ANDs can be transformed into ORs and vice versa

This basically means

/*it cannot be summer and winter at once also 
means it is either, not summer or not winter, 
using the template that its either summer or winter that are available*/

!(Summer AND Winter) = !Summer OR !Winter

/*If its either not summer and not winter, that means it can't be summer or winter*/

!Summer AND !Winter = !(Summer OR Winter)

Truth tables

We can use truth tables to analyze the internal variables that our model relies on. The rows represent the possible states or combinations for the variables. Each variable has two possible outcomes, so we use the 2 ^ n formula, where n is the number of variables. Each outcome can either be True or False.

Use Case

class User {
  constructor(firstname, lastname, isValidated, rateCount,isBlocked){
    this.firstname = firstname;
    this.lastname = lastname;
    this.isValidated = isValidated;
    this.rateCount = rateCount;
    this.isBlocked = isBlocked;
  }

  writeToFile() {
    if(!this.isBlocked && this.rateCount < 10 && this.isValidated ){
      console.log('User is writing...');
      this.addToRate();
    }else console.log(`${this.firstname} ${this.lastname} you have issues`)

  }

  addToRate() {
    this.rateCount ++;
  }

  get rate(){
    return this.rateCount;
  }

}

This is a system that grants write permissions to authenticated users and blocks users if they try to write to the system with an unvalidated email address or tries to write to the system after the limit of 10 commits is exceeded.

Using De Morgan's Law

We want to analyze the logical path or process that leads to a user getting blocked.

A: Unvalidated User
B: Writes to the system
C: Exceeding rate limit(10)
D: User gets blocked

/*If an Unvalidated user writes to the system or if a validated user exceeds the limit, the user gets blocked.*/
(A AND B) OR (B AND C) -> D

//We can factorize using distributivity
B AND (A OR C) -> D

//The user remains unblocked at !D
!D -> !(B AND (A OR C)) // The law of contrapositivity

//Using DeMorgan's law (!(A AND B) = !A OR !B)
!D -> !B OR !(A OR C)

//Using DeMorgan's law again
!D -> !B OR (!A AND !C)

The final expression tells us that the user is not blocked if he doesn't write to the system or if he is validated and doesn't exceed the limit.

Using Truth tables

If we have to create a system with the following requirements

If the user hasn't validated his/her email, he/she has only read permissions.
An Unvalidated user cannot have write permissions
The user either has read or write permissions.

A: Unvalidated User
B: read permissions
C: write permissions

A ---> B (Unvalidated User has only read permissions ) This statement is only true, when the output(B) is true, or when they are both(A and B) false.
!(A and C) (An Unvalidated User can't have write permissions)
B or C (It's either a user has read or write permissions)

Since we have three variables, we will have 8 possible outcomes (2 ^ 3) of true or false for each variable. Then we test these possible outcomes with the three statements we have above.

A	B	C	1	2	3	Total
T	T	T	T	F	T	F
T	T	F	T	T	T	T
T	F	T	F	F	T	F
T	F	F	F	T	F	F
F	F	F	T	T	F	F
F	F	T	T	T	T	T
F	T	T	T	T	T	T
F	T	F	T	T	T	T

So we can see that we only have truthy outcomes when at least one of the variables is false, or at least one of the variables is true. They can't be all false or all true. Which makes sense, you can't be invalidated and still have to write permissions.

Conclusion

There are so many other rules and laws that come with analyzing logical models, this is just a slice of the pie, I came across it and I liked it, so I decided to share. Thanks for reading! 🌹 🌹

Originally written on my blog

References

Computer Science Distilled by Wladston Ferreira Filho
http://www.ask-math.com/de-morgans-law.html

How to Automate Dumping for SQL Databases

paschal — Fri, 01 Mar 2019 00:52:02 +0000

Introduction 😸

Due to the ever-increasing probability of things going haywire in production, the need for redundancy cannot be overemphasized. This leads us to create a simple automated flow of backing up our DB. Hosting and Deployment services like AWS already provide this, but we will be doing a little hack for smaller projects.

Pipeline 👽

We will be using a remote psql(postgres) database and a linux terminal for this example. Users on Windows OS can run the windows susbsystem for Linux(WSL).

So we will create a dump of our live DB, and store the dump.sql file and then obviously, we don't want it on our local host for security purposes, so we'll finally transfer our dump to a secure server.

We will then assign this script to a cron job, and then our dumps will be created on the days we choose.

Automate.sh

The bash script that will handle this will be named 'Automate.sh'. Let's write our script.

#!/bin/bash
#Filename: Automate.sh
#Description: Create a dump of my PSQL DB

pg_dump -U 'YOUR USERNAME' -h 'YOUR HOSTNAME' -d 'DATABASE NAME' > dump.sql

With the 'pg_dump' tool, we can easily create a dump of our database. When this code is run, a password prompt will come up and then we can put in the password of the database. Manually inputting the password defeats the purpose of the automation, so we can attach a '--no-password' flag to stop the prompt from coming up and then we can pass the password when running the script.

Our Automate.sh file should look like this now

pg_dump -U 'YOUR USERNAME' -h 'YOUR HOSTNAME' -d 'DATABASE NAME' --no-password > dump.sql

Then we can run the script like this:

root@L: PGPASSWORD='YOUR_PASSWORD' ./Automate.sh

We can also add the password to our environmental variables when scheduling our cron job. After the dump file is created, we want to send this file to a secure server, an AWS Instance for example, and then we can delete the file from our localhost.

We can transfer files to remote servers using the scp (Secure copy) tool, and we can quickly install this if we don't have it

sudo apt-get install openssh-client

Our Automate.sh file should look like this now:

#!/bin/bash
#Filename: Automate.sh
#Description: Create a dump of my PSQL DB

pg_dump -U 'YOUR USERNAME' -h 'YOUR HOSTNAME' -d 'DATABASE NAME' > dump.sql

#Transfer file to server
scp -i "PRIVATE_KEY.pem" dump.sql  'SERVER IP'

#Delete file from localhost
if [ $? -eq 0 ];
then
     echo "Backup successful!" && rm dump.sql
else
    echo "Oops!"
fi

To ensure your private key is not overwritten accidentally, we can set permissions to 400 and then we can also set the permissions of our dump.sql file to make it writable and readable.

root@L: chmod 400 'PRIVATE_KEY.pem'
root@L: chmod 777 dump.sql

Scheduling With Cron ✈️

Finally, we can set the job to run by 12:59 AM everyday.

root@L: crontab <<EOF
> PGPASSWORD='YOUR_DB_PASSWORD'
> 59 12 * * * ./Automate.sh
> EOF

Other ways to accomplish this will be by creating a .cron file or using the crontab -e command.

root@L: crontab -e

This opens up a file that allows you schedule your cron.