Forem: Arvind Kumar

Automate your git push to remote server

Arvind Kumar — Thu, 17 Mar 2022 17:12:09 +0000

Step by step deployment of my github repo to my server.

I have a ReactJS app setup as a git repo. Every-time I update and push to my currently working dev branch I have to ssh/login to my server and pull it to reflect changes to server.

In addition I have to restart my app by running PM2 restart "myapp". Sometimes I have to run yarn install as well, when I have added or removed a package in my app.

I wish I could automate all this process of pushing, pulling, yarn install and finally restart my app. Is there a way? It seems like. Let's try to set it up.

1. Create the Bare Repo

Login to your server

ssh arvind@devarticles.in

Setup a bare repository

git init --bare /path/to/My_React_App.git

create a `post-receive` hook

nano /path/to/My_React_App.git/hooks/post-receive

And enter into it, this:

#!/bin/sh

# Check out the files
git --work-tree=/path/to/My_React_App --git-dir=/path/to/My_React_App.git/hooks/post-receive checkout -f dev

Note: In my example I am pushing my local dev branch to remote dev branch. You may omit passing branch name if you are working on your default branch, i.e. main or master, whatever is set in your config.

Important Step:

Make this file executable by setting:

chmod +x /path/to/My_React_App.git/hooks/post-receive

Run this within your own local repo:

git remote add Development 'ubuntu@<IP-ADDRESS>:My_React_App.git'

Note: if you use ssh key to login to server use the following to set up your key in Git configuration

GIT_SSH_COMMAND='ssh -i /Users/arvind/mykey.pem' git submodule update --init

And then make the push:

git push --set-upstream Development dev

If everything works fine you should be able see your local app changes on your server.

Running Tasks After Git Hook Push

As I mentioned earlier I was working on a reactapp for which I wanted to setup this post-receive hook. Hence after every successful push I wanted to run some specific commands inside my app folder, the following ones to be specific:

yarn install pm2 restart "myapp"

Thankfully this is as easy to automate running these commands as adding these two or rather three lines of code into our /path/to/ICOV3_React_Admin.git/hooks/post-receive file. That is edit post-receive file to add:

cd /path/to/My_React_App yarn install pm2 restart "myapp"

Note: I assume that I had run pm2 command with same name to run this app first time, from my app folder. That is:

pm2 start yarn --name "myapp" -- start

Deploying ReactJS App on Ubuntu 20.04 LTS on Amazon EC2 instance

Arvind Kumar — Thu, 17 Mar 2022 11:57:14 +0000

React, which is also known as React.js or ReactJS, is a free and open-source front-end JavaScript library for building user interfaces based on UI components. It is maintained by Meta (formerly Facebook) and a community of individual developers and companies.

Some of the popular websites built with or using ReactJS extensively are Facebook, BBC, Netflix, Salesforce, Asana and Dropbox. According a 2021 survey by Stackoverlow React.js (40.14%) has surpassed jQuery(34.43%) as the most commonly used web framework. Support link

While it could be a breeze developing an App in React, it may not be that straight-forward to deploy a React app on a Ubuntu 20.04 server, if you are doing it for first time in particular.

Hosting on Amazon AWS

AWS EC2 is one of the popular options to host a React app. In this article we’ll see how to deploy a react app with ngnix on a Ubuntu 20.04.3 LTS (GNU/Linux 5.11.0-1022-aws x86_64) hosted as an AWS EC2 Instance.

Note: This article assumes that you have a basic knowledge of AWS cloud system and ReactJS. This article also assumes that you have already bought a domain and it is already pointed to your server, the EC2 instance and it is working.

Note: To check whether a domain is pointed correctly to a server's IP check it at https://dnschecker.org/#A/domainxyz.com (replace your domain name)

If you haven't registered a domain yet you may want to checkout official AWS instruction or may want to look at some other cheaper options.

Install and Configure ngnix on EC2 instance

Launch an EC2 instance with latest Ubuntu LTS AMI
Connect to the console from preferred terminal through ssh:

ssh <username>@<server-ip> -i <key-name>

Install nginx

sudo apt update
sudo apt install nginx -y

Once installed it should show the default nginx page when you visit your domain in address bar of browser:

Nodejs

Although Nodejs is not required to run basic ReacJS applications, it is essential if you are using the popular JSX syntax in your react app.

Nodejs makes developers life easier by providing access to Javascript ecosystem a ReactJS uses.

Nodejs is required if you want to use a node.js-based build tool like browserify or webpack to build nity production bundles.

Install Nodejs

In order to install Nodejs we need curl to get Nodejs downloaded to our server. On a EC2 instance curl comes installed by default. So if curl --version does not show you result on your server, install it by running:

sudo apt-get install curl

Next, install Nodejs

curl -sL https://deb.nodesource.com/setup_15.x | sudo -E bash -

Test the NodeJs version and npm version

node -v npm --v

Install Yarn (optional)

Since I, personally like to use Yarn for package management in my React.js apps I would install Yarn, using npm of course.

`sudo npm install yarn -g

Create project

Let's create a react app using create-react-app. If create-react-app is not installed it should ask you to install it so press y when asked to do so.

npx create-react-app react-tutorial

Now go into directory and build and run, using yarn.
yarn install yarn run build yarn start`

(Alternately you could run all command using npm by replacing yarn in above commands.)

After you enter the yarn start, you'll be able to see the react server running and the relevant ports in the shell.

Try to access the react app at public IP for your server. For example http://172.31.3.180:3000

Note: 3000 port should be opened in the security group of your EC2 instance, as shown below. You can add an inbound rule to the security group attached to your EC2 instance.

So now, your server's public IP with 3000 port should display something like this:

The ReactApp is now running but there is a problem. The problem is, if as you exit the ssh console, the ReactApp will stop. In order to fix this, and keep app running even if we closed or exited the ssh console, we need a process manager which would keep app running all the time, unless we ask it to stop it.

PM2 is a daemon process manager that will help you manage and keep your application online 24/7. Let's install it.

sudo yarn global add pm2
or
sudo npm install pm2@latest -g

once pm2 is installed you may want to test some of its basic commands, such as

pm2 status
pm2 list
pm2 restart 
pm2 stop 
pm2 start 
pm2 delete

To run our app run the following while being at the app folder, i.e. react-tutorial

pm2 start yarn --name "React Tutorial" -- start

Once App is started, running would yeild pm2 list

In order to start pm2 on system reboot add the following:

pm2 startup systemd

Now our react application will keep running in the background unless stopped anyhow.

Since we have access to our app at port 3000 we would like it to show it at the default port 80 or our Nginx web server. That also means that it has to show at the very root of our our domain name since we already have pointed our domain to this server's public IP address.

In the next step of this tutorial we are going to see how we can use Nginx as a reverse proxy and divert traffic to port 80 i.e. the default landing page of our domain or public IP.

Create a new site config in /etc/nginx/sites-available

cd /etc/nginx/sites-available
sudo nano react-tutorial

The last command will open a text file to be edited. Paste the following code into it while replacing xxx.xx.. with your IP address, add domain name with space, use any one of two or both.

server {
    listen 80;
    listen [::]:80;
    
    server_name xxx.xxx.xxx.xxx yourdomain.com;
    access_log /var/log/nginx/reat-tutorial.com.access.log;                
    error_log /var/log/nginx/reat-tutorial.com.error.log;       
    location / {
            proxy_pass http://127.0.0.1:3000;
            client_max_body_size 50m;
            client_body_buffer_size 16k;
            proxy_http_version 1.1;                                              
            proxy_set_header Upgrade $http_upgrade;                              
            proxy_set_header Connection 'upgrade';                               
            proxy_set_header Host $host;                                         
            proxy_cache_bypass $http_upgrade;   
    }
}

Save and exit the file with Ctl+x -> Y.

Next, we need to activate this new site by creating a symlink to new site configuration

sudo ln -s /etc/nginx/sites-available/react-tutorial /etc/nginx/sites-enabled/

Make sure that your nginx configuration syntax is error free

sudo nginx -t

Restart Nginx

sudo systemctl restart nginx

You may want to restart your app also:

pm2 restart "React Tutorial"

If everything goes well you should see your app running at the root domain or your Amazon EC2 instance's public IP address.

Validate CSV row content with Custom validation rules with FormRequest in Laravel

Arvind Kumar — Mon, 14 Mar 2022 10:47:35 +0000

In this example I am going to show and brief my implementation of validating CSV row content.

Scenario

I need to create a simple custom CSV importer to import a few rows into my Laravel application database. I want to validate the incoming data in the columns of each row to make sure that data is what we expect to be. For example, some of columns should be string type and some integer etc.

Solution

Since I am using FormRequest validation method I need to implement it to adapt to FormRequest mechanism. I thank to this post for providing me a base for it.

I am not going to create a new Laravel application and I will just show the relevant code only.

Here is the store method of UserController.php


namespace App\Http\Controllers\API;

use App\Http\Requests\UserImportRequest;
use App\Http\Controllers\Controller;
use App\Http\Traits\CsvParser;
use App\Models\User;

class UserController extends Controller
{
    use CsvParser;

public function store( UserImportRequest $request, Merchant $merchant )
{
    $fileContents = request()->file('field_csvfile')->get();
    $csvData = $this->CsvToArray($fileContents);
    $imported = [];
    foreach( $csvData as $row ) {
        $imported[] = User::create(
            $row
        );
    }
    return response( $imported );
}

Here is UserImportRequest.php, the FormRequest file

namespace App\Http\Requests;

use Illuminate\Validation\Factory as ValidationFactory;
use \Illuminate\Contracts\Validation\Validator;
use Illuminate\Validation\ValidationException;
use Illuminate\Foundation\Http\FormRequest;
use App\Rules\CsvContent;

class UserImportRequest extends FormRequest
{
    public function __construct(ValidationFactory $validationFactory)
    {
        $validationFactory->extend(
            'dob_custom_check',
            function ($attribute, $value, $parameters) {
                //If valid date ; write your own "date check" logic
                return true; //Else
                //return false
            },
            ':attribute is not valid'
        );
    }
    /**
     * Determine if the user is authorized to make this request.
     *
     * @return bool
     */
    public function authorize()
    {
        return true;
    }

    /**
     * Get the validation rules that apply to the request.
     *
     * @return array
     */
    public function rules()
    {
        return [
            'field_csvfile' => [
                'bail',
                'file',
                'mimes:csv,txt,xls,xlsx',
                new CsvContent( [
                    'first_name' => 'required|date',
                    'last_name' => 'required|date',
                    'email' => ['required','unique:users'],
                    'date_of_birth' => ['required, 'date', 'dob_custom_check'],
                    'address' => 'required|string',
                    '' => 'required|string',
                ])
            ]
        ];
    }
}

Here is the CsvParser trait App\Http\Traits\CsvParser.php

namespace App\Http\Traits;

trait CsvParser {
    public function CsvToArray( $content )    {
        $rows = array_map('str_getcsv', explode(PHP_EOL, $content));
        $rowKeys = array_shift($rows);
        $formattedData = [];
        foreach ($rows as $row) {
            if( sizeof($row) == sizeof($rowKeys) )  {
                $associatedRowData = array_combine($rowKeys, $row);
                if (empty($keyField)) {
                    $formattedData[] = $associatedRowData;
                } else {
                    $formattedData[$associatedRowData[$keyField]] = $associatedRowData;
                }
            }
        }
        return $formattedData;
    }
}

I hope it helps someone.

React Table Server Side Pagination with Sorting and Search Filters

Arvind Kumar — Tue, 04 Jan 2022 06:53:01 +0000

After following a post on dev.to I was able to setup a basic server side paginated table. However since the post did not have the sorting and search features I had to extend it and hence this one!

You may find full code here at my github repo

Here is what my final table looks like:

Let's start with some initial imports. My example uses react-query so make sure you have it installed. It is a great library anyway. I also use axios library for making ajax calls.



import React, {useState, useEffect, useMemo} from "react"
import { useTable, usePagination, useSortBy } from "react-table"
import { QueryClient, QueryClientProvider, useQuery } from 'react-query'
import axios from 'axios'

Next I import user columns which I place in another file named columns.jsx



import { USERS_COLUMNS } from "./columns"

And here are the contents of columns.jsx file:



export const USERS_COLUMNS = [
    {
        Header: "Email",
        accessor: "email",
    },
    {
        Header: "Name",
        accessor: "name",
    },
    {
        Header: "Phone",
        accessor: "phone",
    },
    {
        Header: "Role",
        accessor: "role",
    },
    {
        Header: "Employee Number",
        accessor: "employee_number"
    },
]

Next imports are:



import SortIcon from 'mdi-react/SortIcon'
import SortAscendingIcon from 'mdi-react/SortAscendingIcon'
import SortDescendingIcon from 'mdi-react/SortDescendingIcon'
import ReactTablePagination from '@/shared/components/table/components/ReactTablePagination'
import UsersFilter  from "./UsersFilter"

Let me explain it a bit. First three imports are icons used for sorting. Next to it is ReactTablePagination component I have created for pagination links and last one UsersFilter is the search area where I place search box with a submit link. I may also want to add more filters later on.

I will post ReactTablePagination and UsersFilter code down the page. Let's first work with our current UsersIndex.jsx file and its main component DataTable but before that let me post some declarations I have made outside of DataTable component.

Okay, once all the imports are done at the top of this page. Let start with structure of rest of this file.

Since I am using react-query, and you should also consider using it if your app is doing ajax requests for data extensively, I will wrap my DataTable component within QueryClientProvider which is exported from react-query library if you noticed it at the top of the page.

So after imports I initialise the queryClient



const queryClient = new QueryClient()

... and wrap my DataTable with QueryClientProvider by passing client to it and export it at the end of the page. You may also consider to wrap you main within this client, I have just added it in my this one page only.

This is the overall structure of UsersIndex.jsx file




... imports at the top of the file

const queryClient = new QueryClient()

... other file code

const DataTable = () => {
   ... component code
}

const TableWrapper = () => {
    return (
        <QueryClientProvider client={queryClient}>
            <DataTable />
        </QueryClientProvider>
    )
}

export default TableWrapper;

Lets dive into the ...other file code first. This is the code which is before the main DataTable component.



const initialState = {
    queryPageIndex: 0,
    queryPageSize: 10,
    totalCount: 0,
    queryPageFilter:"",
    queryPageSortBy: [],
};

const PAGE_CHANGED = 'PAGE_CHANGED'
const PAGE_SIZE_CHANGED = 'PAGE_SIZE_CHANGED'
const PAGE_SORT_CHANGED = 'PAGE_SORT_CHANGED'
const PAGE_FILTER_CHANGED = 'PAGE_FILTER_CHANGED'
const TOTAL_COUNT_CHANGED = 'TOTAL_COUNT_CHANGED'

const reducer = (state, { type, payload }) => {
  switch (type) {
    case PAGE_CHANGED:
        return {
            ...state,
            queryPageIndex: payload,
        };
    case PAGE_SIZE_CHANGED:
        return {
            ...state,
            queryPageSize: payload,
        };
    case PAGE_SORT_CHANGED:
        return {
            ...state,
            queryPageSortBy: payload,
        };
    case PAGE_FILTER_CHANGED:
        return {
            ...state,
            queryPageFilter: payload,
        };
    case TOTAL_COUNT_CHANGED:
        return {
            ...state,
            totalCount: payload,
        };
    default:
      throw new Error(`Unhandled action type: ${type}`)
  }
};

const fetchUsersData = async (page, pageSize, pageFilter, pageSortBy) => {
    let paramStr = ''
    if( pageFilter.trim().length > 1 ) {
        paramStr = `&keyword=${pageFilter}`
    }
    if( pageSortBy.length > 0 ) {
        const sortParams = pageSortBy[0];
        const sortyByDir = sortParams.desc ? 'desc' : 'asc'
        paramStr = `${paramStr}&sortby=${sortParams.id}&direction=${sortyByDir}`
    }
    try {
        const response = await axios.get(
        `/users?page=${page+1}&limit=${pageSize}${paramStr}`
        );
        const results = response.data.data;
        const data = {
            results: results,
            count: response.data.total
        };
        return data;
    } catch (e) {
        throw new Error(`API error:${e?.message}`)
    }
}

New thing to notice in the code above is the use of reducer. If you are not sure how reducers work you should check this post or a simplified post here

Also there is fetchUsersData function which is responsible for fetching user data and most of it is self-explaining.

And finally here is the DataTable component



const DataTable = () => {
    const [keyword, setKeyword] = useState('');
    const [useFilter, setUseFilter] = useState(false);
    const onClickFilterCallback = ( filter ) => {
        if(filter.trim() === "") {
            alert('Please enter a keyword to search!')
            return
        }
        if(filter === keyword)   {
            alert('No change in search')
            return
        }
        setUseFilter(true)
        setKeyword(filter)
    }

    let columns = useMemo( () => USERS_COLUMNS, [])

    const [{ queryPageIndex, queryPageSize, totalCount, queryPageFilter, queryPageSortBy }, dispatch] =
    useReducer(reducer, initialState);

    const { isLoading, error, data, isSuccess } = useQuery(
        ['users', queryPageIndex, queryPageSize, queryPageFilter, queryPageSortBy],
        () => fetchUsersData(queryPageIndex, queryPageSize, queryPageFilter, queryPageSortBy),
        {
            keepPreviousData: false,
            staleTime: Infinity,
        }
    );

    const totalPageCount = Math.ceil(totalCount / queryPageSize)

    const {
        getTableProps,
        getTableBodyProps,
        headerGroups,
        rows,
        prepareRow,
        page,
        pageCount,
        pageOptions,
        gotoPage,
        previousPage,
        canPreviousPage,
        nextPage,
        canNextPage,
        setPageSize,
        state: { pageIndex, pageSize, sortBy }
    } = useTable({
        columns,
        data: data?.results || [],
        initialState: {
            pageIndex: queryPageIndex,
            pageSize: queryPageSize,
            sortBy: queryPageSortBy,
        },
        manualPagination: true,
        pageCount: data ? totalPageCount : null,
        autoResetSortBy: false,
        autoResetExpanded: false,
        autoResetPage: false
    },
    useSortBy,
    usePagination,
    );
    const manualPageSize = []

    useEffect(() => {
        dispatch({ type: PAGE_CHANGED, payload: pageIndex });
    }, [pageIndex]);

    useEffect(() => {
        dispatch({ type: PAGE_SIZE_CHANGED, payload: pageSize });
        gotoPage(0);
    }, [pageSize, gotoPage]);

    useEffect(() => {
        dispatch({ type: PAGE_SORT_CHANGED, payload: sortBy });
        gotoPage(0);
    }, [sortBy, gotoPage]);

    useEffect(() => {
        if ( useFilter ) {
            dispatch({ type: PAGE_FILTER_CHANGED, payload: keyword });
            gotoPage(0);
        }
    }, [keyword, gotoPage, useFilter]);

    useEffect(() => {
        if (data?.count) {
            dispatch({
            type: TOTAL_COUNT_CHANGED,
            payload: data.count,
            });
        }
    }, [data?.count]);

    if (error) {
        return <p>Error</p>;
    }

    if (isLoading) {
        return <p>Loading...</p>;
    }
    if(isSuccess)
    return (
            <>
                <div className='table react-table'>
                    <form className="form form--horizontal">
                        <div className="form__form-group">
                            <div className="col-md-9 col-lg-9">
                                <UsersFilter onClickFilterCallback={onClickFilterCallback} defaultKeyword={keyword} />
                            </div>
                            <div className="col-md-3 col-lg-3 text-right pr-0">
                                <Link style={{maxWidth:'200px'}}
                                className="btn btn-primary account__btn account__btn--small"
                                to="/users/add"
                                >Add new user
                                </Link>
                            </div>
                        </div>
                    </form>
                    {
                        typeof data?.count === 'undefined' && <p>No results found</p>
                    }
                    {data?.count && 
                    <>
                    <table {...getTableProps()} className="table">
                        <thead>
                            {headerGroups.map( (headerGroup) => (
                                <tr {...headerGroup.getHeaderGroupProps()}>
                                    {headerGroup.headers.map( column => (
                                        <th {...column.getHeaderProps(column.getSortByToggleProps())}>
                                            {column.render('Header')}
                                            {column.isSorted ? <Sorting column={column} /> : ''}
                                        </th>
                                    ))}
                                </tr>
                            ))}
                        </thead>
                        <tbody className="table table--bordered" {...getTableBodyProps()}>
                            {page.map( row => {
                                prepareRow(row);
                                return (
                                    <tr {...row.getRowProps()}>
                                        {
                                            row.cells.map( cell => {
                                                return <td {...cell.getCellProps()}><span>{cell.render('Cell')}</span></td>
                                            })
                                        }
                                    </tr>
                                )
                            })}
                        </tbody>
                    </table>
                    </>
                }
                </div>
                {(rows.length > 0) && (
                    <>
                        <ReactTablePagination
                            page={page}
                            gotoPage={gotoPage}
                            previousPage={previousPage}
                            nextPage={nextPage}
                            canPreviousPage={canPreviousPage}
                            canNextPage={canNextPage}
                            pageOptions={pageOptions}
                            pageSize={pageSize}
                            pageIndex={pageIndex}
                            pageCount={pageCount}
                            setPageSize={setPageSize}
                            manualPageSize={manualPageSize}
                            dataLength={totalCount}
                        />
                        <div className="pagination justify-content-end mt-2">
                            <span>
                            Go to page:{' '}
                            <input
                                type="number"
                                value={pageIndex + 1}
                                onChange={(e) => {
                                const page = e.target.value ? Number(e.target.value) - 1 : 0;
                                gotoPage(page);
                                }}
                                style={{ width: '100px' }}
                            />
                            </span>{' '}
                            <select
                            value={pageSize}
                            onChange={(e) => {
                                setPageSize(Number(e.target.value));
                            }}
                            >
                            {[10, 20, 30, 40, 50].map((pageSize) => (
                                <option key={pageSize} value={pageSize}>
                                Show {pageSize}
                                </option>
                            ))}
                            </select>
                        </div>
                    </>
                )}
            </>
    )
}

And there is one helper component which is outside of DataTable component. I just placed it at the bottom, just before the TableWrapper.



const Sorting = ({ column }) => (
    <span className="react-table__column-header sortable">
      {column.isSortedDesc === undefined ? (
        <SortIcon />
      ) : (
        <span>
          {column.isSortedDesc
            ? <SortAscendingIcon />
            : <SortDescendingIcon />}
        </span>
      )}
    </span>
);

It is not possible to explain every line and I hope the code makes sense to you. There is one thing I want to mention though. Notice the last three settings in the block:



manualPagination: true,
pageCount: data ? totalPageCount : null,
autoResetSortBy: false,
autoResetExpanded: false,
autoResetPage: false

I had to set them in order to get rid of "Maximum update depth exceeded" error after I turned manualPagination on and implemented server side pagination with sorting and search in my reactjs application. (See ref here)

Full code here at my github repo.