Forem: MilburnGomes

Getting started with electronjs

MilburnGomes — Thu, 10 Sep 2020 08:14:46 +0000

Objective: This guide will teach you how to develop and distribute an Electron.js application.
Audience: This guide is targeted at Electron beginners. It is strongly recommended that readers have at least fundamental knowledge of HTML, CSS and JavaScript with Node.js as the Electron framework is built around these items.

Introduction

Electron is an open-source software framework developed and maintained by GitHub. It allows for the development of desktop GUI applications using web technologies: it combines the Chromium rendering engine and the Node.js runtime. source

Understanding Electron

If you found this guide by wanting to develop cross platform applications then you probably know that Electron.js does just that. You can easily develop and distribute applications for windows/macOS/linux with the same code (bear in mind this does not include android or iOS).

The question becomes, “How does Electron accomplish this?”. In short; Electron launches a headless chromium browser which has access to the Node.js API via Electron’s own API. Which has plenty of use cases but, probably the biggest being is that your app can theoretically work without an internet connection, unless your code requires an internet connection.

If that was a bouncer that’s okay but, it’s important to understand the Electron combines the browser and Node.js to create this seamless development experience for us.

Electron is the main GUI framework behind several notable open-source projects including Atom, GitHub Desktop, Light Table, Visual Studio Code, and WordPress Desktop.

The above are some of the top apps, however you can go ahead and check out more app built with electron here

Advantages

What you need to get started

Inspired by the getting started page in the Electron Documentation Writing Your First Electron App

To develop desktop apps using Electron, I personally use Visual Studio Code, but most code editors that have a terminal included should work.
You'll also need to install the NodeJS runtime.

Setting up your project

Now that you've got the right tools, let's get started setting up the project. To do so, you'll have to create a folder containing your project, and then open that folder using your code editor (VSCode in my case).
Then open up a terminal window into the newly created folder.

Then type in npm init. This will setup your package.json file.
You'll have to enter the following information:

Package name: your project's name (lowercase and no spaces)
Version: you can just hit enter for this one as it will default to 1.0.0
Description: just enter some basic information about the purpose of your project
Entry point: this one is quite important. It is the javascript file that will be executed when launching the project. By default it will name index.js, but you can change it to any name as it will handle all of our desktop application's windows. Many devs name it as main.js for electron apps.
Test command: this is the command that will be executed when typing npm test in the terminal. you can hit enter to keep as it is, as later I will show you to set a command to run the electron app.

The remaining fields like Git Repository, Keywords, Author, license are just some information for when you'll publish your project on npmjs.

Once you confirm, a file called package.json will be created.
It should resemble something like this:

{
  "name": "sample-app-electron",
  "version": "1.0.0",
  "description": "A sample app using Electronjs",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "Milburn Gomes",
  "license": "ISC"
}

And to finish setting up your project, you'll have to install electron using npm. This is very easy, as all you have to do is type npm install electron --save-dev in the terminal window. Use --save-dev instead of --save so you can work on multiple apps with multiple electron version in the future.
Note that a package-lock.json file is created, but you don't have to worry about it.
Notice that you will also have electron in your package.json file under dependencies.

Also, while we are in package.json file you will need to one more change to run electron when you run npm start command. So inside your scripts tag add start property as "start": "electron .", because we want Electron to launch our application. The dot specifies the directory in which the Electron project is stored, which is just the root directory here, but if you want to store your project files somewhere else, you can adjust the directory. Code as shown below:

{
  "name": "sample-app-electron",
  "version": "1.0.0",
  "description": "A sample app using Electronjs",
  "main": "index.js",
  "scripts": {
    "start": "electron .",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "electron": "^10.1.1"
  }
}

Notice the script object, containing start which has the value of electron ., which means running the command npm start will execute your project. You can also add other scripts there. Read more about it here.

Displaying a window

Now that everything is set up, we can start coding! Let's begin by creating the index.js file, that will handle our app's windows. So go ahead and create a new file and name it as index.js or if you have changed the name while running npm init command create file with that name.

Enter the following line of code:

const { app, BrowserWindow } = require('electron')

I'll begin by referencing the electron package which we installed earlier. In this we are importing app, BrowserWindow from the reference to electron. app object will be used to detect app events, such as when the user launches the app and BrowserWindow lets us display an HTML document in the application window.

I'll create a function createWindow(). It will be triggered when the app is launched, and inside of it I'll set up the win variable with some options defining the size in pixels of the window and since we are using node, set nodeIntegration: true inside webPreferences

Next specify the HTML file when the electron app window loads.

Here is a list of all the available options for BrowserWindow.

function createWindow () {
  // Create the browser window.
  const win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      nodeIntegration: true
    }
  })

  // and load the index.html of the app.
  win.loadFile('index.html')
}

And finally, when the app is ready call the function createWindow, as shown below:

app.whenReady().then(createWindow)

The final code in index.jsshould look like this:

const { app, BrowserWindow } = require('electron')

function createWindow () {
  // Create the browser window.
  const win = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      nodeIntegration: true
    }
  })

  // and load the index.html of the app.
  win.loadFile('index.html')
}

app.whenReady().then(createWindow)

Displaying your HTML file

Now let's create the HTML file that is going to be displayed in the window. Create a new file index.html.

Inside it I'll just write a basic web page with Hello World! inside H1 tag:

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Document</title>
</head>

<body>
    <h1>Hello World!</h1>
</body>

</html>

Now you can just type npm start in the terminal, and you should see the HTML file inside a window as shown below:

Till now we just used only plain HTML inside our webpage. To use CSS and JavaScript in your electron app, you will need to reference the CSS and JavaScript files in your index.html file. So go ahead and create two new files and name it as styles.css and script.js.

I have added bootstrap CDN and referenced the newly created CSS and JavaScript files. I've changed the text of H1 from Hello World! to Welcome Back! and also given an ID for H1 tag. Also I've added input field and a button. Another important thing to notice that I've added the jquery.min.js reference manually instead of CDN. If you try to run the electron app with jquery CDN, it will throw jquery not defined error. However, if you run just the html file it will work. This is because when jquery loads in electron, it loads as a module, it is not available globally hence the BrowserWindow is unable to access it and throws jquery not defined error. The index.html code is as follows:

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css">
    <script>window.$ = window.jQuery = require('./jquery.min.js');</script>
    <!-- <script src="https://ajax.googleapis.com/ajax/libs/jquery/3.5.1/jquery.min.js"></script> -->
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js"></script>
    <link rel="stylesheet" type="text/css" href="./styles.css" />
    <title>Document</title>
</head>

<body>
    <h1 id="welcomeUser">Welcome Back!</h1>
    <input id="user" placeholder="Enter your name here..."></input>
    <button id="submit">Submit</button>

    <script src="./script.js"></script>
</body>

</html>

For styling, I've added 10px margin for the entire document. Also for the html and body tag I've set the font size to 22px and color to dodgerblue. The styles.css code is as follows:

* {
    margin: 10px;
  }

  html,
  body {
    font-size: 22px;
    color: dodgerblue;
  }

For javascript logic, I'm taking the user input for the input field on click of the submit button and displaying in the H1 tag. The script.js code is as follows:

document.getElementById('submit').addEventListener('click', () => {
  if (document.getElementById('user').value) {
    document.getElementById('welcomeUser').innerHTML = `Hello, ${
      document.getElementById('user').value
    }!`;
  } else {
    document.getElementById('welcomeUser').innerHTML = `Hello, Guest!`;
  }
});

Now run the app by running the same command used earlier npm start

The output of the app is as follows:

On entering the name and clicking the submit button, you'll get the following:

And now since you have created an app, you can go ahead and start building by using HTML, CSS and JavaScript. Because we're using NodeJS and Electron, you have the simplicity of creating websites combined with the power of Node. You can install modules from npmjs

Another way to get started is to clone and run the code from the electron GitHub repository "electron-quick-start" by using the electron/electron-quick-start repository.

Clone the repository

$ git clone https://github.com/electron/electron-quick-start

Go into the repository

$ cd electron-quick-start

Install dependencies

$ npm install

Run the app

$ npm start

For a list of boilerplates and tools to kick-start your development process, see the Boilerplates and CLIs documentation.

In order to set an icon, you need an image file. So get an image file and copy it in the project directory and set
icon: 'icon.ico', in the BrowserWindow in index.js file. The name of my icon file is icon with .ico extension. Note that you can use image of any type example: png, jpg but ico is preferred. Mac OS supports ico file for icon.

In order to make your run full screen, you need to specify fullscreen: true, in the BrowserWindow. But I'll be commenting it in the code.

If you want to open Developer Tools when the app runs, include win.webContents.openDevTools(); in your index.js file. But I'll be commenting it in the code.

The index.js code is as follows:

const { app, BrowserWindow, Menu } = require('electron');
const path = require('path');
const url = require('url');

// SET ENV
process.env.NODE_ENV = 'development';

function createWindow() {
  // Create the browser window.
  const win = new BrowserWindow({
    width: 800,
    height: 600,
    icon: 'icon.ico',
    // fullscreen: true,
    webPreferences: {
      nodeIntegration: true,
    },
  });

  // and load the index.html of the app.
  win.loadFile('index.html');

  // Open the DevTools.
  // win.webContents.openDevTools();

  // Quit app when closed
  win.on('closed', function () {
    app.quit();
  });

  const mainMenu = Menu.buildFromTemplate(mainMenuTemplate);

  // Insert menu
  Menu.setApplicationMenu(mainMenu);
}

app.whenReady().then(createWindow);

// Create menu template
const mainMenuTemplate = [
  {
    label: 'File',
    submenu: [
      {
        label: 'New Window',
        accelerator: process.platform == 'darwin' ? 'Command+N' : 'Ctrl+N',
        click() {
          NewWindow();
        },
      },
      {
        label: 'Quit',
        accelerator: process.platform == 'darwin' ? 'Command+Q' : 'Ctrl+Q',
        click() {
          app.quit();
        },
      },
    ],
  },
];

// Handle Open New Window
function NewWindow() {
  console.log(`Create a New Window`);
  let addWindow = new BrowserWindow({
    width: 500,
    height: 500,
    title: 'New Window',
  });
  addWindow.loadURL(
    url.format({
      pathname: path.join(__dirname, 'New.html'),
      protocol: 'file:',
      slashes: true,
    })
  );
  // Handle garbage collection
  addWindow.on('close', function () {
    addWindow = null;
  });
}

// If mac, add empty object to menu
if (process.platform == 'darwin') {
  mainMenuTemplate.unshift({});
}

// Add dev tools, if not in prod
if (process.env.NODE_ENV !== 'production') {
  mainMenuTemplate.push({
    label: 'Developer Tools',
    submenu: [
      {
        label: 'Toggle DevTools',
        accelerator: process.platform == 'darwin' ? 'Command+I' : 'Ctrl+I',
        click(item, focusedWindow) {
          focusedWindow.toggleDevTools();
        },
      },
      {
        role: 'reload',
      },
    ],
  });
}

New.html file code is as follows:

<!DOCTYPE html>
<html lang="en">

<head>
    <meta charset="UTF-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <link rel="stylesheet" href="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.2/css/bootstrap.min.css">
    <script>window.$ = window.jQuery = require('./jquery.min.js');</script>
    <script src="https://maxcdn.bootstrapcdn.com/bootstrap/4.5.2/js/bootstrap.min.js"></script>
    <link rel="stylesheet" type="text/css" href="./styles.css" />
    <title>Document</title>
</head>

<body>
    <h1>Hello World!</h1>
</body>

</html>

Next, I'll be showing how to package the app!

There are several app packagers. I will show two of them electron-packager and electron-builder. The difference between them is that the former creates an simple executable file while the latter creates an installer which prompts user to select install location.
First I will show you using electron-packager. In the terminal, run npm install electron-packager --save-dev. Once installed, add "package-win": "electron-packager .", inside of scripts in package.json.

Your package.json file should look like this:

{
  "name": "sample-app-electron",
  "version": "1.0.0",
  "description": "A sample app using Electronjs",
  "main": "index.js",
  "scripts": {
    "start": "electron .",
    "package-win": "electron-packager .",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "electron": "^10.1.1",
    "electron-packager": "^15.1.0"
  }
}

Now our app is ready to publish. Run the command in the terminal npm run package-win. Once it finishes executing the command you will see a new folder being created into the project directory sample-app-electron-win32-x64 and inside this folder you will see sample-app-electron.exe, which is our electron app which we just developed. But there's a catch, if you navigate to sample-app-electron-win32-x64\resources\app you will see your source code.

This means that the source code is not protected. So in order to protect your code edit the "package-win" to "package-win": "electron-packager . --asar",
Along with this, I have also added some more useful options. One of them is to overwrite the application build folder if it already exists, platform is set to win32, icon is given the icon file, prune set to true gets rid of unwanted JavaScript files, out sets the output folder of the application build folder, and few more.

{
  "name": "sample-app-electron",
  "version": "1.0.0",
  "description": "A sample app using Electronjs",
  "main": "index.js",
  "scripts": {
    "start": "electron .",
    "package-win": "electron-packager . --overwrite --asar --platform=win32 --icon=icon.ico --prune=true --out=release-builds --version-string.CompanyName=Example --version-string.FileDescription=SampleApp --version-string.ProductName=\"SampleApp\"",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "electron": "^10.1.1",
    "electron-packager": "^15.1.0"
  }
}

Run the command in the terminal npm run package-win. Now if you navigate to sample-app-electron\release-builds\SampleApp-win32-x64\resources you will see a single file named app.asar which means that your code is protected.

Next, run the command npm install electron-builder --save-dev. Then add in your package.json file "productName": "SampleApp",, "build-installer": "electron-builder", under scripts and create a text file and name it as license.txt. Add your license in this file. In this example I'll be adding just a sample text. Your license.txt file is as follows:

SampleApp

A sample app using Electronjs

Also add a build property as follows:

 "build": {
    "appId": "sample-app-electron",
    "win": {
      "target": [
        "nsis"
      ],
      "icon": "icon.ico",
      "requestedExecutionLevel": "requireAdministrator"
    },
    "nsis": {
      "installerIcon": "icon.ico",
      "uninstallerIcon": "icon.ico",
      "uninstallDisplayName": "SampleApp",
      "license": "license.txt",
      "oneClick": false,
      "allowToChangeInstallationDirectory": true
    }
  },

So finally your package.json file should look like this:

{
  "name": "sample-app-electron",
  "productName": "SampleApp",
  "version": "1.0.0",
  "description": "A sample app using Electronjs",
  "main": "index.js",
  "scripts": {
    "start": "electron .",
    "package-win": "electron-packager . --overwrite --asar --platform=win32 --icon=icon.ico --prune=true --out=release-builds --version-string.CompanyName=Example --version-string.FileDescription=SampleApp --version-string.ProductName=\"SampleApp\"",
    "build-installer": "electron-builder",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "build": {
    "appId": "sample-app-electron",
    "win": {
      "target": [
        "nsis"
      ],
      "icon": "icon.ico",
      "requestedExecutionLevel": "requireAdministrator"
    },
    "nsis": {
      "installerIcon": "icon.ico",
      "uninstallerIcon": "icon.ico",
      "uninstallDisplayName": "SampleApp",
      "license": "license.txt",
      "oneClick": false,
      "allowToChangeInstallationDirectory": true
    }
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "electron": "^10.1.1",
    "electron-packager": "^15.1.0",
    "electron-builder": "^22.8.0"
  }
}

Run the command npm run build-installer. You will notice a new folder named dist created which will contain the exe file named SampleApp Setup 1.0.0.exe. Also, if you navigate to dist\win-unpacked\resources you will see that your source code is protected as well. Now if you execute the exe file it should open the installer. Following are the screens at each click:

And there you go, you have successfully developed and is now ready to distribute an Electron.js application!

You can find the entire source code for this tutorial here

Also, don't forget to check out my other electron.js projects:
Bitcoin Price Alert App
Note Taking App
System Info App
Tasks List App

In Conclusion:

This guide should have given you a fundamental understanding of how Electron works. If you had trouble following this guide I suggest spending more time learning Node.js before jumping into Electron. If this guide was too simple, I highly suggest checking out the following resources:
Electron Documentation
Electron Packager Documentation
Electron Builder

Thank you!

Elasticsearch Logstash Kibana

MilburnGomes — Sat, 13 Jun 2020 10:07:06 +0000

The goal of this post is to load CSV (Comma Separated Values) file into Logstash which will process the data and give it to Elasticsearch for data storage and use Kibana to query and visualize the data. To achieve this, the process is shown in the image below:

Note: This article assumes that the reader has no prior knowledge of Elasticsearch, Logstash, Kibana and I will try to explain the basics to get it up and running. This post is not for advance working of Elasticsearch, Logstash, Kibana.

So let's get started

For those of you who are unaware of the Elasticsearch, Logstash, Kibana below is a quick intro of all three. When used together it is also known as the ELK stack.

Elasticsearch:

Logstash:

Kibana:

You can read more about it from here

Now having the understanding of the ELK stack lets dive in. First, you will need to download Elasticsearch, Logstash, Kibana

Note that the downloads will be in compressed folder. Extract all three (Elasticsearch, Logstash, Kibana) folders in your system.

Open Command Prompt in Windows or Terminal in Mac, go to the path where the folders are extracted.
(For this tutorial I'm using Windows operating system and I have extracted the folders in "D:\2020\Logstash\". At the time of this post the latest version for all three (Elasticsearch, Logstash, Kibana) is 7.7.1)

To run elasticsearch. Go to the bin folder of elasticsearch and then type elasticsearch, as shown below:

cd D:\2020\Logstash\elasticsearch-7.7.1-windows-x86_64\elasticsearch-7.7.1\bin
D:\2020\Logstash\elasticsearch-7.7.1-windows-x86_64\elasticsearch-7.7.1\bin>elasticsearch

By default elasticsearch runs on port 9200 of localhost. Once the elasticsearch is loaded and running, as shown in below screenshot.

You can now hit the http://localhost:9200 in your browser to check elasticsearch details. The URL returns a JSON file with details like name, cluster_name, cluster_uuid and version details.

To run kibana. Go to the bin folder of kibana and then type kibana, as shown below:

cd D:\2020\Logstash\kibana-7.7.1-windows-x86_64\kibana-7.7.1-windows-x86_64\bin
D:\2020\Logstash\kibana-7.7.1-windows-x86_64\kibana-7.7.1-windows-x86_64\bin>kibana

By default kibana runs on port 5601 of localhost. Once the kibana is loaded and running, as shown in below screenshot.

You can now hit the http://localhost:5601/app/kibana#/home in your browser to check kibana details. The URL returns the homepage of kibana.

Next, to run logstash, we need to supply a config file as a parameter. So we'll need to create the config file.

Structure of a Config File
A Logstash config file has a separate section for each type of plugin you want to add to the event processing pipeline. For example:

# This is a comment. You should use comments to describe
# parts of your configuration.
input {
  ...
}

filter {
  ...
}

output {
  ...
}

Each section specifies the configuration options for one or more plugins. If you specify multiple filters, they are applied in the order of their appearance in the configuration file.

You can read more about it from here

Click here for more configuration examples

For CSV file, you can create a CSV file or download it from Kaggle.com. In this post I've taken a sample file from movies-on-netflix-prime-video-hulu-and-disney.
Note that these files may contain billions of data/ records which makes a perfect example to use it in elasticsearch.

Example of the CSV file which I've used for this example:

I've created a new folder named data and pasted the csv file in it. The name of the CSV file is movies.csv. Then create a new file with .config extension. I have named it logstash_movies.config. Note that these files need not be under the same folder and can have different names as well, as we will be specifying their paths with their file name, but for convenience I will put them under the same folder. Paste the below content in the config file

input {
    file {
        path => "D:/2020/Logstash/data/movies.csv"
        start_position => "beginning"
        sincedb_path => "NUL"
    }
}

filter {
    csv {
        separator => ","
        columns => ["Srno","ID","Title","Year","Age","IMDb","Rotten Tomatoes","Netflix",
        "Hulu","Prime Video","Disney+","Type","Directors","Genres","Country","Language",
        "Runtime"]
    }
    mutate { convert => ["Year", "integer"]} 
    mutate { convert => ["IMDb", "float"]} 
    mutate { convert => ["Runtime", "integer"]} 
}

output {
    elasticsearch {
        hosts => "localhost"
        index => "movies"
        document_type => "streaming_movies"
    }
    stdout {}
}

The input section configures the path to the CSV file, start_position specifies where to begin reading the data example from beginning or end of file (by default if you don't set, it is set to end of file), sincedb_path specifies the path to store the data, set to null if data storing is not required, etc.

The filter section configures the csv and mutate filters. The csv section specifies the separator, columns, etc. By default all columns are set datatype as string but you can mutate it by converting it to the specified datatype like integer, float etc.

The output section configures elasticsearch which specifies
host - set to localhost or the actual I.P address where the ELS stack is hosted,
index - it is the index parameter used to identify the records for the given data. All data is fetched/ retrieved using this index parameter. We'll discuss more about index parameter later of this post,
document_type - each record is a document.

Note that if you are using Mac O.S, change the following line:
Windows: sincedb_path => "NUL"
MAC: sincedb_path => "/dev/null"

Now that our config file is ready, we can run logstash.

To run logstash . Go to the bin folder of logstash and then type logstash followed by -f and the config file path with name, as shown below:

cd D:\2020\Logstash\logstash-7.7.1\logstash-7.7.1\bin
D:\2020\Logstash\logstash-7.7.1\logstash-7.7.1\bin>logstash -f "D:\2020\Logstash\data\logstash_movies.config"

If everything is in order then logstash will process the CSV file and feed it to elasticsearch. Once the CSV file is being read, it will log it to the console, as below:

Note that all three consoles must be running, do not close any of the consoles, as shown in the screenshot below:

In order to visualize data, we need to create the index pattern. This index pattern is the same parameter which we specified in the config file.

From the home page of kibana, head to Management by navigating for the left pane, as shown below:

Under management on the left side, you will have two options Elasticsearch and Kibana. Under Kibana, click on the Index Patterns. You will see as the screenshot below:

As seen above, there are no search patterns created. We need to create a index pattern. Click the + Create index pattern button on the top right corner. Creating an index pattern is a two step process.
In the first step, specify the index pattern name as movies*. You should able to see Success! ... in green text. Then click on Next step.

In the second step, In Configure settingsunder Time Filter field name, you can select I don't want to use the Time Filter and click on Create index pattern.

Now you should be able to see the newly created index pattern under index pattern, as below:

Here you get to see the columns names along with their datatypes, format, searchable, aggregatable etc.

Once the index pattern is create, you are now ready to visualize the data. But before that you can explore few options in kabana like Discover and Dev Tools

Under Dev Tools you can query the following:

movies get all endpoint:

GET /movies

movies search endpoint:

GET /movies/_search
{
  "query": {
    "match_all": {}
  }
}

movies count endpoint:

GET /movies/_count

Under Discover you get to see the data records:

Following is the screenshot with the record expanded:

Now lets take a look at kibana visualization

Head to Visualize page, you should able to see as below:

Click on + Create new visualization. A pop up screen will ask you select a visualization type, lets select Pie.

On selecting Pie, the pop up will ask you to choose a source, it is the index pattern, select the index pattern: movies*

On the right side, you will see Buckets, click on Add and select Split slices

Next, select Aggregation to Terms and select any field of your choice. In this case I've selected Country.keyword. This will populate the pic chart by countries, as shown below:

You can also do some customization under Options tab. Example: you can change the pie from donut, show or hide tooltip, label settings etc.

Finally, after making the desired changes, you can save the visualization, as below:

The saved visualization can then be used in dashboard.

Finally, Dashboard

We can create advanced dashboard like below screenshot from their official site, but we'll create a simple dashboard:

Dashboard contains visualization which we created in the previous step. To create a dashboard, head to Dashboards

Click on the + Create new dashboard button

On the next screen, you will asked to choose panels, which are nothing be the visualization which you have created, or you have an option to create a new one.

Then save the dashboard:

Our final dashboard looks as below:

You can find the config file and the CSV file on my github page MilburnGomes

See also
https://www.elastic.co/

That's it for the ELK stack, hope you have enjoyed and understood it as well.

Thanks for reading!

Exploring Docker Commands

MilburnGomes — Mon, 13 Jan 2020 03:14:13 +0000

In this article I'll talk about what docker is and we'll explore few useful Docker commands. We will look at different commands to create containers, images, docker hub.

Docker is used to run software application environment, programs, services inside an container. Example nginx or apache, mysql, wordpress, mongodb which can run inside a container which are separate from your operating system, separate from your local machine environment.

Instead of having to build an app on one machine and having to try to mimic that environment on another machine specially if it's a different operating system, you might have to install things differently and it becomes tedious in the end

Docker completely gets rid of that because you can simply run it in a container and it's going to be the same, no matter what machine you run it on - Mac, Linux, Windows machine or even the versions of the operating system.

Docker concepts
Docker is a platform for developers and sysadmins to build, share, and run applications with containers. The use of containers to deploy applications is called containerization. Containers are not new, but their use for easily deploying applications is.

Containerization is increasingly popular because containers are:

Flexible: Even the most complex applications can be containerized.

Lightweight: Containers leverage and share the host kernel, making them much more efficient in terms of system resources than virtual machines.

Portable: You can build locally, deploy to the cloud, and run anywhere.

Loosely coupled: Containers are highly self-sufficient and encapsulated, allowing you to replace or upgrade one without disrupting others.

Scalable: You can increase and automatically distribute container replicas across a datacenter.

Secure: Containers apply aggressive constraints and isolations to processes without any configuration required on the part of the user.

Images and containers
Fundamentally, a container is nothing but a running process, with some added encapsulation features applied to it in order to keep it isolated from the host and from other containers. One of the most important aspects of container isolation is that each container interacts with its own, private filesystem; this filesystem is provided by a Docker image. An image includes everything needed to run an application -- the code or binary, runtimes, dependencies, and any other filesystem objects required.
Containers and virtual machines

A container runs natively on Linux and shares the kernel of the host machine with other containers. It runs a discrete process, taking no more memory than any other executable, making it lightweight.

By contrast, a virtual machine (VM) runs a full-blown “guest” operating system with virtual access to host resources through a hypervisor. In general, VMs incur a lot of overhead beyond what is being consumed by your application logic.

Image taken from here

On VMware, you have host machine/ server run multiple VM's and each VM machine has full blown OS and has their own kernel and you run apps on top of each OS. However, with docker you have one host machine/ server, one OS, it could be any OS - Windows, Mac, Linux and run multiple containers which are nothing but processes that run on your system. They all share the same kernel. It's much lighter and less overhead and very easy to port to other machine and very easy to deploy. So docker is very much better solution than doing it the VMware way.

Orchestration
The portability and reproducibility of a containerized process mean we have an opportunity to move and scale our containerized applications across clouds and datacenters; containers effectively guarantee that those applications will run the same way anywhere, allowing us to quickly and easily take advantage of all these environments. Furthermore, as we scale our applications up, we’ll want some tooling to help automate the maintenance of those applications, able to replace failed containers automatically and manage the rollout of updates and reconfigurations of those containers during their lifecycle.

Tools to manage, scale, and maintain containerized applications are called orchestrators, and the most common examples of these are Kubernetes and Docker Swarm. Development environment deployments of both of these orchestrators are provided by Docker Desktop, which we’ll use throughout this guide to create our first orchestrated, containerized application.

Docker Commands
If you have Docker installed on your system and if you open your terminal and type docker, you should be able to see list of docker commands such as below:

List Options and Management Commands
docker

and few more!
Shows list of Options and Management Commands.
Usage: docker [OPTIONS] COMMAND
Run docker COMMAND --help for more information on a command.

Docker Version Info
docker version

Shows the Client and the Server versions.
Server is the docker engine which runs in the background and client is used to communicate with the engine.
It shows the version in most cases.
Docker is built on Go programming language, hence it shows the Go version.
The server OS/ arch is linux/amd64 and the client is what your machine is.
For me its windows/amd64 or if you use mac you might have darwin/amd64

Docker Info
docker info

Shows you how many containers - running, paused, stopped. Also shows the number of Images installed on your system etc.

Creating Containers

Let's create a container. We'll create nginx - it's a very common web server similar to Apache.

Create nginx
docker container run -it -p 80:80 nginx

'docker container' is the 'Management Command' and it has 'run' as the sub command
'-it' is for interactive mode, will run in the foreground rather than background. Running in foreground means it will shows logs in the terminal.
-p or --publish is publish
'80:80' first port 80 is the port that you want to use on your machine, you can set it to any unused port on your machine, since it's locally on your system, the second port 80 is what is exposed from the container, it's default port for nginx. You cannot change it.
'nginx' is the name of the image. You cannot change it, it has to be the same.

You will also notice "Unable to find image 'nginx:latest' locally", it means that it was unable to find the image locally on your system hence it will pull from docker hub will go to Docker Hub which is GitHub of all docker images and pulls it down.
'latest' is just the tag it is using to get the latest build.
Once it is downloaded on your system, it will automatically start running.
Since it is downloaded on my system, now it's running and if I run it localhost on browser
No need to type localhost:80, as port 80 is default.

Voilà, I’m running nginx on my machine!

Its running in a container, it not installed on my machine. Its inside a container.
Since I ran it as interactive mode (-it) in the foreground, every time I reload it going to show logs. it would not show had I run it detached (-d)

All the images are stored on Docker Hub

Create a container in foreground
docker container run -it -p 80:80 nginx

Creates a container which runs in foreground

Create a container in background
docker container run -d -p 80:80 nginx

Creates a container which runs in background

Naming Containers
docker container run -d -p 80:80 --name nginx-server nginx

Names the container as specified in name. In this case it is named as 'nginx-server'. If the Containers are not named, docker will set default names. Names are named as -.

List All Running Containers
docker container ls or docker ps

Shows list of running containers

List All Containers
docker container ls –a

Shows list of all containers, running or not

Stop A Running Container
docker container stop [ID or NAME]
docker container stop mysql

Stops the specified running container .

Stop All Running Containers
docker stop $(docker ps -aq)

Stops all running containers.

Remove A Container
docker container rm [NAME]
docker container rm mysql

docker container rm [ID]
docker container rm 5515

Removes the specified non running container.

Remove A Running Container
docker container rm -f [ID]
docker container rm mysql –f

Removes a running container by force.
By running the command docker container rm mysql will throw error that it cannot remove the container because it is running. So we need to use force (-f).

Remove Multiple Containers
docker container rm [ID] [ID] [ID]

Removes all the container specified with the ID.

Remove All Non Running Containers
docker rm $(docker ps -aq)

Removes all non running containers.

Remove All Containers
docker rm $(docker ps -aq) -f

Removes all containers, running as well as non running.

Get Logs (Use NAME or ID)
docker container logs [NAME]

Shows logs of the container specified either by NAME or ID.

List Processes Running in Container
docker container top [NAME]

Lists the processes running in container by the name specified.

Image Commands

List Docker Images
docker images

Shows list of all the images which are downloaded on your local system. Even though we have removed the container, the image still remains on the system

Remove An Image.
docker image rm ID
docker image rm f7bb

Removes the image specified by ID.
No need to put the entire ID, just 3-4 characters are fine.

Remove All Images
docker rmi $(docker images -a -q)

Removes all images present on your system.

Pull Image
docker pull [IMAGE]
docker pull mongo

Manually pulls down the image specified by the Image Name if exists.

Environment Variables

mysql

docker container run –d –p 3306:3306 –name mysql –env MYSQL_ROOT_PASSWORD=123456 mysql

When you start the mysql image, you can adjust the configuration of the MySQL instance by passing one or more environment variables on the docker run command line. Do note that none of the variables below will have any effect if you start the container with a data directory that already contains a database: any pre-existing database will always be left untouched on container startup.

See also https://dev.mysql.com/doc/refman/5.7/en/environment-variables.html for documentation of environment variables which MySQL itself respects (especially variables like MYSQL_HOST, which is known to cause issues when used with this image).

Docker Login
docker login

Used to login in Docker. If you run the command and you are already logged in, it will try to login with your existing credentials.

JavaScript Arrow Functions Expressions

MilburnGomes — Sun, 12 Jan 2020 11:14:26 +0000

This article is referred from here

An arrow function expression is a syntactically compact alternative to a regular function expression, although without its own bindings to the this, arguments, super, or new.target keywords. Arrow function expressions are ill suited as methods, and they cannot be used as constructors.

Syntax
Basic syntax

(param1, param2, …, paramN) => { statements } 
(param1, param2, …, paramN) => expression
// equivalent to: => { return expression; }

// Parentheses are optional when there's only one parameter name:
(singleParam) => { statements }
singleParam => { statements }

// The parameter list for a function with no parameters should be written with a pair of parentheses.
() => { statements }

Advanced syntax

// Parenthesize the body of a function to return an object literal expression:
params => ({foo: bar})

// Rest parameters and default parameters are supported
(param1, param2, ...rest) => { statements }
(param1 = defaultValue1, param2, …, paramN = defaultValueN) => { 
statements }

// Destructuring within the parameter list is also supported
var f = ([a, b] = [1, 2], {x: c} = {x: a + b}) => a + b + c;
f(); // 6

You can create a simple HTML page and add the below JavaScript code in order to see the output, or you can clone my GitHub project.

Below JavaScript code is used to show how Regular function and Arrow functions looks like. Take a look:

console.log('Welcome to Arrow Functions!');

// no parameters

// no parameter, regular function
function hello1() {
    return 'Hello!';
}
console.log('no parameters, regular function ' + hello1());

// no parameter, arrow function
const hello2 = () => {
    return 'Hello!';
}
console.log('no parameters, arrow function ' + hello2());

// no parameter, arrow function single line 
const hello3 = () => 'Hello!';
console.log('no parameters, arrow function single line  ' + hello3());

// one parameter

// one parameter, regular function
function hello4(x){
    return 'one parameter, regular function ' + 'Hello, ' + x;
}
console.log(hello4('milburn'));

// one parameter, arrow function
const hello5 = x => {
   return  'one parameter, arrow function ' + 'Hello, ' + x;
}
console.log(hello5('milburn'));

// one parameter, arrow function single line 
const hello6 = x => 'Hello, ' + x;
console.log('one parameter, arrow function single line ' + hello6('milburn'));

//two parameters

// two parameters, regular function
function hello7(x,y){
    return 'Hello, ' + x + ' ' + y; 
}
console.log('two parameters, regular function ' + hello7('milburn', 'gomes'));

// two parameters, arrow function
const hello8 = (x,y) =>{
    return 'two parameters, arrow function ' + 'Hello, ' + x + ' ' + y;
}
console.log(hello8('milburn', 'gomes'));

// two parameters, arrow function single line 
const hello9 = (x,y) =>  "Hello, " + x + " " + y;
console.log('two parameters, arrow function single line ' + hello9('milburn', 'gomes'));

Description

Two factors influenced the introduction of arrow functions: the need for shorter functions and the behavior of the this keyword.

Shorter functions

var elements = [
  'Hydrogen',
  'Helium',
  'Lithium',
  'Beryllium'
];

// This statement returns the array: [8, 6, 7, 9]
elements.map(function(element) {
  return element.length;
});

// The regular function above can be written as the arrow function below
elements.map((element) => {
  return element.length;
}); // [8, 6, 7, 9]

// When there is only one parameter, we can remove the surrounding parentheses
elements.map(element => {
  return element.length;
}); // [8, 6, 7, 9]

// When the only statement in an arrow function is `return`, we can remove `return` and remove
// the surrounding curly brackets
elements.map(element => element.length); // [8, 6, 7, 9]

// In this case, because we only need the length property, we can use destructuring parameter:
// Notice that the `length` corresponds to the property we want to get whereas the
// obviously non-special `lengthFooBArX` is just the name of a variable which can be changed
// to any valid variable name you want
elements.map(({ length: lengthFooBArX }) => lengthFooBArX); // [8, 6, 7, 9]

// This destructuring parameter assignment can also be written as seen below. However, note that in
// this example we are not assigning `length` value to the made up property. Instead, the literal name
// itself of the variable `length` is used as the property we want to retrieve from the object.
elements.map(({ length }) => length); // [8, 6, 7, 9]

Function body

Arrow functions can have either a "concise body" or the usual "block body".

In a concise body, only an expression is specified, which becomes the implicit return value. In a block body, you must use an explicit return statement.

var func = x => x * x;                  
// concise body syntax, implied "return"

var func = (x, y) => { return x + y; }; 
// with block body, explicit "return" needed

Returning object literals

Keep in mind that returning object literals using the concise body syntax params => {object:literal} will not work as expected.

var func = () => { foo: 1 };
// Calling func() returns undefined!

var func = () => { foo: function() {} };
// SyntaxError: function statement requires a name

This is because the code inside braces ({}) is parsed as a sequence of statements (i.e. foo is treated like a label, not a key in an object literal).

You must wrap the object literal in parentheses:

var func = () => ({ foo: 1 });

Line breaks

An arrow function cannot contain a line break between its parameters and its arrow.

var func = (a, b, c)
  => 1;
// SyntaxError: expected expression, got '=>'

However, this can be amended by putting the line break after the arrow or using parentheses/braces as seen below to ensure that the code stays pretty and fluffy. You can also put line breaks between arguments.

var func = (a, b, c) =>
  1;

var func = (a, b, c) => (
  1
);

var func = (a, b, c) => {
  return 1
};

var func = (
  a,
  b,
  c
) => 1;

// no SyntaxError thrown

Parsing order

Although the arrow in an arrow function is not an operator, arrow functions have special parsing rules that interact differently with operator precedence compared to regular functions.

let callback;

callback = callback || function() {}; // ok

callback = callback || () => {};
// SyntaxError: invalid arrow-function arguments

callback = callback || (() => {});    // ok

More examples

// An empty arrow function returns undefined
let empty = () => {};

(() => 'foobar')(); 
// Returns "foobar"
// (this is an Immediately Invoked Function Expression)

var simple = a => a > 15 ? 15 : a; 
simple(16); // 15
simple(10); // 10

let max = (a, b) => a > b ? a : b;

// Easy array filtering, mapping, ...

var arr = [5, 6, 13, 0, 1, 18, 23];

var sum = arr.reduce((a, b) => a + b);
// 66

var even = arr.filter(v => v % 2 == 0); 
// [6, 0, 18]

var double = arr.map(v => v * 2);
// [10, 12, 26, 0, 2, 36, 46]

// More concise promise chains
promise.then(a => {
  // ...
}).then(b => {
  // ...
});

// Parameterless arrow functions that are visually easier to parse
setTimeout( () => {
  console.log('I happen sooner');
  setTimeout( () => {
    // deeper code
    console.log('I happen later');
  }, 1);
}, 1);

Browser compatibility

Docker Hub Quickstart

MilburnGomes — Thu, 09 Jan 2020 13:49:45 +0000

This article is referred from Docker Hub Quickstart

Docker Hub is a service provided by Docker for finding and sharing container images with your team. It provides the following major features:

Repositories: Push and pull container images.
Teams & Organizations: Manage access to private repositories of container images.
Official Images: Pull and use high-quality container images provided by Docker.
Publisher Images: Pull and use high- quality container images provided by external vendors. Certified images also include support and guarantee compatibility with Docker Enterprise.
Builds: Automatically build container images from GitHub and Bitbucket and push them to Docker Hub.
Webhooks: Trigger actions after a successful push to a repository to integrate Docker Hub with other services.

Step 1: Sign up for Docker Hub

Start by creating an account.

Step 2: Create your first repository

To create a repo:

Sign in to Docker Hub.
Click on "Create a Repository" on the Docker Hub welcome page:

Name it <your-username>/my-first-repo as shown below. Select "Private":

You’ve created your first repo. You should see:

Step 3: Download and install Docker Desktop

We’ll need to download Docker Desktop to build and push a container image to Docker Hub.

Download and install Docker Desktop. If on Linux, download Docker Engine - Community.

Open the terminal and sign in to Docker Hub on your computer by running docker login

Step 4: Build and push a container image to Docker Hub from your computer

Start by creating a Dockerfile to specify your application as shown below:

cat > Dockerfile <<EOF
FROM busybox
CMD echo "Hello world! This is my first Docker image."
EOF
Run docker build -t <your_username>/my-first-repo . to build your Docker image.
Test your docker image locally by running docker run <your_username>/my-first-repo
Run docker push <your_username>/my-first-repo to push your Docker image to Docker Hub.

You should see output similar to:

And in Docker Hub, your repository should have a new latest tag available under Tags:

Congratulations! You’ve successfully:
Signed up for Docker Hub
Created your first repository
Built a Docker container image on your computer
Pushed it to Docker Hub

Next steps
Create an organization to use Docker Hub with your team.
Automatically build container images from code through builds.
Explore official & publisher images.
Upgrade your plan to push additional private Docker images to Docker Hub.

Introduction To Docker!

MilburnGomes — Thu, 09 Jan 2020 04:15:53 +0000

Happy New Year 2020!

This article is for beginners only. Specially for those who have never used Docker or want to get the hang and feel of Docker. Hence this is a very basic tutorial. In my further posts I will be covering more practical projects.

So what is Docker?
Docker is a set of platform as a service products that use OS-level virtualization to deliver software in packages called containers. Containers are isolated from one another and bundle their own software, libraries and configuration files; they can communicate with each other through well-defined channels. (Source: Wikipedia)

In order to use Docker, you need to create an account. While creating your account, you will be asked to create your Docker ID. Hence this docker ID will be used in future such as to login in Docker Hub, to identify your Repository, upload to your repository etc.

Once your account has been created, you will need to download the Docker software called as 'Docker Desktop' on your system.

So let's get started!

I've just begun using Docker and as I discover, I will upload more articles on the same. This article I've found on the Docker site and followed their step by step procedure and I was able to achieve the result. Following is the same URL for the Quick Start Tutorial

Step 1: Download
Let's get you started: you will need to download and install Docker to use the Docker command line interface (CLI).

Docker Desktop
The preferred choice for millions of developers that are building containerized applications

Download Docker Desktop for Windows or Mac

Step 2: Clone
First, open a Mac terminal or Windows PowerShell and download a fun example.
This repository contains everything you need to create your first container.

git clone https://github.com/docker/doodle.git
(Requires Git)

Step 3: Build
Now let's build and tag a Docker image.
A Docker image is a private filesystem, just for your container. It provides all the files and code your container will need. Running the docker build command creates a Docker image using the Dockerfile. This built image is in your machine's local Docker image registry.

cd doodle/cheers2019 && docker build -t <docker_id>/cheers2019 .

Replace <docker_id> with your Docker ID which you have entered while creating Docker Account.

Step 4: Run
Great! Now let's run your first container.
Running a container launches your software with private resources, securely isolated from the rest of your machine.

docker run -it --rm <docker_id>/cheers2019

You should be able to see the output as below:

Step 5: Ship
Share your image on Docker Hub
Once you're ready to share your container with the world, push the image that describes it to Docker Hub.

docker login && docker push <docker_id>/cheers2019

If everything up to here is done correctly, then

Congratulations

At this point, you have successfully set up Docker Desktop and Docker Hub. You have built, run and shipped your first container. Carry on to master all the power of the Docker platform.

Thanks for reading.
Cheers!