Forem: Mo'men Ahmed

Migrate from redis server to a new one

Mo'men Ahmed — Sun, 19 Sep 2021 15:34:17 +0000

If you have some data stored on a redis server, and for any reason, you need to migrate the data to another redis server, then you may use MIGRATE command in a redis-cli terminal.

The MIGRATE command

MIGRATE destination-host destination-port "" destination-db-index timeout [COPY] [REPLACE] [AUTH password] [AUTH2 username password] [KEYS key [key ...]]

Options:

COPY -- Do not remove the key from the local(source) instance.
REPLACE -- Replace existing key on the remote(destination) instance.
KEYS -- If the key argument is an empty string, the command will instead migrate all the keys that follow the KEYS option (see the above section for more info).

For more info check MIGRATE redis docs here.

Example

Suppose you want to migrate data from source-host to destination-host, you should open redis-cli by writing redis-cli in a terminal in the source redis server machine, then run the following command replacing the host and port with the right values:
MIGRATE destination-host destination-port "" 0 5000 COPY KEYS key1 key2

Detecting outliers in a time series using tsmoothie in Python

Mo'men Ahmed — Tue, 25 May 2021 10:28:11 +0000

What are outliers?

Outliers are data points that are far from most of the other data points.

In this example of time series, all the points outside the blue band can be considered as outliers.

How to get a list of outliers in a time series?

Here we will use a library called tsmoothie.
It is a python library for time-series smoothing and outlier detection in a vectorized way.

On the time series in the figure: we can see that we have 4 outliers, we can get them by:



import numpy as np
from tsmoothie.utils_func import sim_randomwalk
from tsmoothie.smoother import LowessSmoother

data = df['value'].values.reshape(1, -1)

# operate smoothing
smoother = LowessSmoother(smooth_fraction=0.1, iterations=1)
smoother.smooth(data)

# generate intervals
low, up = smoother.get_intervals('prediction_interval')

points = smoother.data[0]
up_points = up[0]
low_points = low[0]

for i in range(len(points)-1, 0, -1):
    current_point = points[i]
    current_up = up_points[i]
    current_low = low_points[i]
    if current_point > current_up or current_point < current_low:
        print(f'found an outlier value: {current_point}')

Considering that the data frame df has a column named 'value'.
So, we get this output after running the code:

One way to implement a time-series using Redis

Mo'men Ahmed — Sun, 23 May 2021 16:02:45 +0000

What is Redis?

Redis stands for 'REmote DIctionary Server', so the name itself implies a dictionary (key-value pairs).

Why use Redis?

Redis is used as a database cache to save data in a fast way in the form of key-value pairs. Also, wide use for it is for sending messages between different systems through a shared Redis database.

What types of data can be stored in Redis?

Redis provides data structures such as strings, hashes, lists, sets, and sorted sets.

How to use Redis?

First, you need to install it from their website. Then you can choose to work in redis-cli or any language client, here we use python redis client. You can install redis for python by pip install redis. This python library has all redis-cli commands as functions, This is a cheatsheet for Redis commands.

A great VS Code extension for Redis

I use this extension in VS Code, it can view all the contents in Redis in a nice and easy way.

An Example on Redis string data type

First, we make a Redis instance by r = redis.Redis(redisServer, 6379)
then we set a value to a key and get it:

r.set('name', 'Ali')
r.get('name')

so, now we have a key called 'name' and has a value called 'Ali'.

An Example on Redis hash data type

# r.hset('price', 'time', 'value')
r.hset('price', '12:20:00', '250.0')
r.hset('price', '12:20:15', '250.7')
r.hset('price', '12:20:30', '252.0')

You can get the value of any time by:

r.hget('price', '12:20:15')

so, it will return the value '250.7'.

You can also get all values and keys by:

Redis Time-series using Hash

The idea about making a time-series in Redis is to have a key for each column you have, and use a hash data type for each one, each hash has a field and a value in the format {field: value}. We put the timestamp as the field.

An example will be like this:

How std::cin works in C++?

Mo'men Ahmed — Fri, 09 Apr 2021 20:06:34 +0000

std::cin is used to get user input from external source.

When the user enters input, that data is put in a buffer inside of std::cin. The buffer is used to hold user input while it’s waiting to be extracted to variables.

std::cin extraction steps:

If there is data already in the input buffer, that data is used for extraction.
If the input buffer contains no data, the user is asked to input data for extraction. When the user hits enter, a ‘\n’ character will be placed in the input buffer.
operator>> extracts as much data from the input buffer as it can into the variable (ignoring any leading whitespace characters, such as spaces, tabs, or ‘\n’).
Any data that can not be extracted is left in the input buffer for the next extraction.

Extraction succeeds if at least one character is extracted from the input buffer. Any unextracted input is left in the input buffer for future extractions. For example:

int x {0};
std::cin >> x;

If the user enters: "22m", the variable x is filled with "22" leaving the "m" alone in the buffer.
Extraction fails if the input data does not match the type of the variable being extracted.
For example, if the user enters "m" extraction would fail because "m" can not be extracted to an integer variable. But then what would be the value of x?
Before c++ 11:a failed extraction would not modify the variable being extracted to.
However, as of C++11, a failed extraction due to invalid input will cause the variable to be zero-initialized.

Look at this code

// uint16_t is a typedef for unsigned short
std::uint16_t x {0}; // max is UINT16_MAX (65535)
std::cin >> x;

when a user enters 65537, std::cin goes immediately into failure mode, but also assigns the closest in-range value to the variable. Consequently, x is left with the assigned value of 65535. Additional inputs are skipped.

Before C++11, a failed extraction would not modify the variable being extracted.
However, as of C++11, an out-of-range failed extraction will cause the variable to be set to the closest in-range value.

Also note that when std::cin goes to failure mode, it will skip any other extractions, so you should put it back to the normal mode:

if (std::cin.fail())
{
    std::cin.clear();
}