Forem: Syamkumar

Apache Airflow in Centos 7

Syamkumar — Tue, 08 Feb 2022 07:03:43 +0000

Apache Airflow is an open-source workflow management platform. It started at Airbnb in October 2014 as a solution to manage the company's increasingly complex workflows. Creating Airflow allowed Airbnb to programmatically author and schedule their workflows and monitor them via the built-in Airflow user interface.

This document assumes you have a centos 7 VM with internet connectivity and a Sudo user.

sudo --login

Update Packages

yum update -y

Install Epel release and yum-utils packages

yum -y install epel-release yum-utils

Install Development tools

yum groupinstall "Development tools" -y

Install required npm packages

yum install -y zlib-devel bzip2-devel openssl-devel ncurses-devel sqlite-devel python-devel wget cyrus-sasl-devel.x86_64

Install python and python developer tools Packages

yum install -y python3 python3-devel

Disable SeLinux

setenforce 0

Make the change permanent by editing

vi /etc/selinux/config

change value

SELINUX=enforcing to SELINUX=disabled

Install Postgres Database

Add Postgres repository to centos

yum -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm

View info your Postgres repo by running

rpm -qi pgdg-redhat-repo

Enable and Install Postgresql

yum-config-manager --enable pgdg12

yum install -y postgresql12-server postgresql12 postgresql-devel

Initialize Database

/usr/pgsql-12/bin/postgresql-12-setup initdb

Enable and Start Postgresql Service

systemctl enable --now postgresql-12

Confirm Database is started by Running

systemctl status postgresql-12

Configure a String Administrator Password for Postgres

su - postgres

Open psql shell

psql

Run the alter user query

alter user postgres with password 'StrongPassword';

Edit the Postgres Service to enable MD5 authentication

vi /var/lib/pgsql/12/data/pg_hba.conf

# Accept from anywhere
host all all 0.0.0.0/0 md5

# Accept from trusted subnet
host all all 192.168.18.0/24 md5

Choose according to your requirement.

Optional Step

Enable remote Postgres access ( Not recommended for Production systems

vi /var/lib/pgsql/12/data/postgresql.conf
Edit the above file Find the entry named listen_address
uncomment the entry and modify it as below.

listen_addresses = '192.168.10.10'

or ( Not recommended in Production )

listen_addresses = '*'

*Restart Postgres Service after you are done.

systemctl restart postgresql-12

How to take Hourly RDS snapshots for Disaster Recovery?

Syamkumar — Tue, 08 Feb 2022 06:13:07 +0000

Amazon RDS creates and saves automated backups of your DB instance during the backup window of your DB instance.

RDS creates a storage volume snapshot of your DB instance, backing up the entire DB instance and not just individual databases.
RDS saves the automated backups of your DB instance according to the backup retention period that you specify.
You can recover your database to any point in time during the backup retention period.

Creating a DB snapshot

Amazon RDS creates a storage volume snapshot of your DB instance, backing up the entire DB instance and not just individual databases. * Creating this DB snapshot on a Single-AZ DB instance results in a brief I/O suspension that can last from a few seconds to a few minutes, depending on the size and class of your DB instance.

For MariaDB, MySQL, Oracle, and PostgreSQL, I/O activity is not suspended on your primary during backup for Multi-AZ deployments, because the backup is taken from the standby.
For SQL Server, I/O activity is suspended briefly during backup for Multi-AZ deployments.

Unlike automated backups, manual snapshots aren't subject to the backup retention period. Snapshots don't expire.

Taking Backups using AWS Cli

When you create a DB snapshot using the AWS Cli, you need to identify which DB instance you are going to back up, and then give your DB snapshot a name so you can restore from it later.
You can do this by using the AWS Cli create-db-snapshot command with the following parameters:

--db-instance-identifier
--db-snapshot-identifier

The Action to do this.

The Action requires the following environment variables to be set as secrets in the repository you will be running this action from.

AWS_REGION -> Your AWS Region
AWS_ACCESS_KEY_ID -> Access key ID
AWS_SECRET_ACCESS_KEY -> Access Secret
DB_INSTANCE_IDENTIFIER -> DB Name

The above access key should have the permission to create snapshots.
The action has a cron based trigger that runs every hour and also a manual trigger that you can run if you want to take a snapshot manually(eg snapshot before running a migration).

name: Take Database Snapshots
on:
  schedule:
    - cron: '0 */1 * * *'
  workflow_dispatch:

env:
  AWS_REGION: ${{ secrets.AWS_REGION}}
  AWS_DEFAULT_OUTPUT: json
  AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID}}
  AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
  DB_INSTANCE_IDENTIFIER: ${{ secrets.DB_INSTANCE_IDENTIFIER }}

jobs:
  snapshot:
    runs-on: ubuntu-latest
    name: Take Database Snapshot
    steps:
      - name: Set current date & time as ENV variable
        run: echo "NOW=$(date +'%Y-%m-%d-%H-%M-%S')" >> $GITHUB_ENV

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v1
        with:
          aws-access-key-id: $AWS_ACCESS_KEY_ID
          aws-secret-access-key: $AWS_SECRET_ACCESS_KEY }}
          aws-region: $AWS_REGION
      - name: Take the Snapshot
        run: |
          aws rds create-db-snapshot --db-instance-identifier $DB_INSTANCE_IDENTIFIER --db-snapshot-identifier $DB_INSTANCE_IDENTIFIER-$NOW

The above GitHub action uses the AWS cli to trigger a snapshot creation .
The creation time depends on the actual db size.
PS. AWS charges you $0.095 per GB-Month ( us-east-1) for RDS snapshot storage as part of backup service .

MongoDB Cluster Setup Centos

Syamkumar — Mon, 10 Aug 2020 14:19:12 +0000

We will be setting up Mongo DB for high availability using 3 virtual machines

1) Mongo Primary (t3a.medium)
2) Mongo Secondary (t3a.medium)
3) Mongo Arbiter (t3a.small)

Let's begin the system preparations

The following setup has to be done in all 3 machines

sudo --login

Update centos packages to the latest version Centos 7

yum update -y

Centos 8

dnf update -y

install Nano editor (I am a Nano fan) Centos 7

yum install nano -y

Centos 8

dnf install nano -y

add hosts values edit the nano /etc/hosts file to add the following entries

172.31.28.16 mongo1
172.31.31.194 mongo2
172.31.25.65 mongo3

important Step

Disable SELinux as it has breaking effects on internode connection
edit /etc/selinux/config
change

SELINUX=disabled

do the above step in all 3 machines and make sure to do a reboot by running

reboot

Now Install Mongo

Configure the package management system

Create a nano /etc/yum.repos.d/mongodb-org-4.4.repo file so that you can install MongoDB directly using yum

Add the following content to it.

[mongodb-org-4.4]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/4.4/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-4.4.asc

Install Mongo DB latest Stable Centos 7

yum install -y mongodb-org

Centos 8

dnf install -y mongodb-org

Before we can begin cluster Setup lets prepare mongo

Enable Mongo Authentication

To add authentication to mongo add the following lines to
nano /etc/mongod.conf

security:
    authorization: enabled

Then do a restart

service mongod restart

let's create mongodb user Open mongo shell

mongo

the following commands

use admin

db.createUser({
    user: "tomahawk",
    pwd: "tomahawkPilot",
    roles: [ 
        { 
            role: "userAdminAnyDatabase", 
            db: "admin" 
        },
        {
            role: "clusterAdmin",
             db: "admin"
        },
        {
            role: "root,
            db: "admin
        }
    ]
})

Now let's Begin the Replication Configuration

Add replSet to mongod.conf in all 3 machines edit your nano /etc/mongod.conf file

replication:
  replSetName: rs0

Keyfile Access Control on Replica Set

navigate to your mongo path

/var/lib/mongo in my case

Use the following command to generate a keyfile

openssl rand -base64 756 > keyfile

Copy the key file to all 3 machines
add Appropriate permissions

chmod 400 keyfile
chown mongod:mongod keyfile

Now add the keyfile to your nano /etc/mongod.conf file

security:
    authorization: enabled
    keyFile: /var/lib/mongo/keyfile

do a restart of all mongod services

systemctl restart mongod

for the changes in replication to take effect before initializing the replica

On mongo1

use admin
db.auth("tomahawk","tomahawkPilot");
rs.initiate(
   {
      _id: "rs0",
      version: 1,
      members: [
         { _id: 0, host : "mongo1:27017" }
      ]
   }
)

You can always run

mongo --port 27017 -u tomahawk --authenticationDatabase 'admin' -p tomahawkPilot

to directly login to the cluster

Now let's create the replica set

rs.add("mongo2:27017");

Check connection status by running

rs.status()

On Successful connection, you will get a response

rs.status()
{
        "set" : "rs0",
        "date" : ISODate("2020-08-10T14:03:42.405Z"),
        "myState" : 1,
        "term" : NumberLong(2),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 2,
        "writableVotingMembersCount" : 2,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1597068221, 2),
                        "t" : NumberLong(2)
                },
                "lastCommittedWallTime" : ISODate("2020-08-10T14:03:41.727Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1597068221, 2),
                        "t" : NumberLong(2)
                },
                "readConcernMajorityWallTime" : ISODate("2020-08-10T14:03:41.727Z"),
                "appliedOpTime" : {
                        "ts" : Timestamp(1597068221, 2),
                        "t" : NumberLong(2)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1597068221, 2),
                        "t" : NumberLong(2)
                },
                "lastAppliedWallTime" : ISODate("2020-08-10T14:03:41.727Z"),
                "lastDurableWallTime" : ISODate("2020-08-10T14:03:41.727Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1597066170, 1),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2020-08-10T14:03:41.721Z"),
                "electionTerm" : NumberLong(2),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1597066987, 1),
                        "t" : NumberLong(1)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2020-08-10T14:03:41.727Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2020-08-10T14:03:41.898Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "mongo1:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 454,
                        "optime" : {
                                "ts" : Timestamp(1597068221, 2),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2020-08-10T14:03:41Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1597068221, 1),
                        "electionDate" : ISODate("2020-08-10T14:03:41Z"),
                        "configVersion" : 2,
                        "configTerm" : 2,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "mongo2:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 7,
                        "optime" : {
                                "ts" : Timestamp(1597066987, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1597066987, 1),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2020-08-10T13:43:07Z"),
                        "optimeDurableDate" : ISODate("2020-08-10T13:43:07Z"),
                        "lastHeartbeat" : ISODate("2020-08-10T14:03:41.727Z"),
                        "lastHeartbeatRecv" : ISODate("2020-08-10T14:03:42.261Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 2,
                        "configTerm" : 1
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1597068221, 2),
                "signature" : {
                        "hash" : BinData(0,"17CdXYPWQ6I24TCLhxpQt8nGGPk="),
                        "keyId" : NumberLong("6859346398467325956")
                }
        },
        "operationTime" : Timestamp(1597068221, 2)
}
rs0:PRIMARY>

Setting up the Arbiter

rs.addArb("mongo3:27017");

{
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1597068567, 1),
                "signature" : {
                        "hash" : BinData(0,"q5F6v883q8+1gBfmEJtwINbXAYY="),
                        "keyId" : NumberLong("6859346398467325956")
                }
        },
        "operationTime" : Timestamp(1597068567, 1)
}

Now check connection status by running

rs.status()

rs.status()

rs0:PRIMARY> rs.status()
{
        "set" : "rs0",
        "date" : ISODate("2020-08-10T14:10:18.328Z"),
        "myState" : 1,
        "term" : NumberLong(2),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 3,
        "writableVotingMembersCount" : 2,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1597068611, 1),
                        "t" : NumberLong(2)
                },
                "lastCommittedWallTime" : ISODate("2020-08-10T14:10:11.737Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1597068611, 1),
                        "t" : NumberLong(2)
                },
                "readConcernMajorityWallTime" : ISODate("2020-08-10T14:10:11.737Z"),
                "appliedOpTime" : {
                        "ts" : Timestamp(1597068611, 1),
                        "t" : NumberLong(2)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1597068611, 1),
                        "t" : NumberLong(2)
                },
                "lastAppliedWallTime" : ISODate("2020-08-10T14:10:11.737Z"),
                "lastDurableWallTime" : ISODate("2020-08-10T14:10:11.737Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1597068601, 1),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2020-08-10T14:03:41.721Z"),
                "electionTerm" : NumberLong(2),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1597066987, 1),
                        "t" : NumberLong(1)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2020-08-10T14:03:41.727Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2020-08-10T14:03:41.898Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "mongo1:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 850,
                        "optime" : {
                                "ts" : Timestamp(1597068611, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2020-08-10T14:10:11Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1597068221, 1),
                        "electionDate" : ISODate("2020-08-10T14:03:41Z"),
                        "configVersion" : 3,
                        "configTerm" : 2,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "mongo2:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 403,
                        "optime" : {
                                "ts" : Timestamp(1597068611, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1597068611, 1),
                                "t" : NumberLong(2)
                        },
                        "optimeDate" : ISODate("2020-08-10T14:10:11Z"),
                        "optimeDurableDate" : ISODate("2020-08-10T14:10:11Z"),
                        "lastHeartbeat" : ISODate("2020-08-10T14:10:17.346Z"),
                        "lastHeartbeatRecv" : ISODate("2020-08-10T14:10:17.389Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "mongo1:27017",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 3,
                        "configTerm" : 2
                },
                {
                        "_id" : 2,
                        "name" : "mongo3:27017",
                        "health" : 1,
                        "state" : 7,
                        "stateStr" : "ARBITER",
                        "uptime" : 50,
                        "lastHeartbeat" : ISODate("2020-08-10T14:10:17.351Z"),
                        "lastHeartbeatRecv" : ISODate("2020-08-10T14:10:17.471Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "configVersion" : 3,
                        "configTerm" : 2
                }
        ],
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1597068611, 1),
                "signature" : {
                        "hash" : BinData(0,"Pir6qAUNxIu6/AbFv6fCxpVVwOs="),
                        "keyId" : NumberLong("6859346398467325956")
                }
        },
        "operationTime" : Timestamp(1597068611, 1)
}

now run

rs.conf()

to see your config

rs0:PRIMARY> rs.conf()
{
        "_id" : "rs0",
        "version" : 3,
        "term" : 2,
        "protocolVersion" : NumberLong(1),
        "writeConcernMajorityJournalDefault" : true,
        "members" : [
                {
                        "_id" : 0,
                        "host" : "mongo1:27017",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 1,
                        "tags" : {

                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                },
                {
                        "_id" : 1,
                        "host" : "mongo2:27017",
                        "arbiterOnly" : false,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 1,
                        "tags" : {

                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                },
                {
                        "_id" : 2,
                        "host" : "mongo3:27017",
                        "arbiterOnly" : true,
                        "buildIndexes" : true,
                        "hidden" : false,
                        "priority" : 0,
                        "tags" : {

                        },
                        "slaveDelay" : NumberLong(0),
                        "votes" : 1
                }
        ],
        "settings" : {
                "chainingAllowed" : true,
                "heartbeatIntervalMillis" : 2000,
                "heartbeatTimeoutSecs" : 10,
                "electionTimeoutMillis" : 10000,
                "catchUpTimeoutMillis" : -1,
                "catchUpTakeoverDelayMillis" : 30000,
                "getLastErrorModes" : {

                },
                "getLastErrorDefaults" : {
                        "w" : 1,
                        "wtimeout" : 0
                },
                "replicaSetId" : ObjectId("5f314b35458f8460ab22cda1")
        }
}

Have an awesome day ahead

Setup an OpenLDAP Server in Centos

Syamkumar — Wed, 29 Jul 2020 07:44:09 +0000

OpenLDAP is a free, open-source implementation of the Lightweight Directory Access Protocol developed by the OpenLDAP Project. It is released under its own BSD-style license called the OpenLDAP Public License. LDAP is a platform-independent protocol.

Follow the following steps to setup an OpenLDAP server in centos

login as Sudo. sudo --login
Run package updates yum update -y
Install Epel Release yum install -y epel-release
Install nano editor yum install -y nano
Install OpenLDAP

yum -y install openldap compat-openldap openldap-clients openldap-servers openldap-servers-sql openldap-devel

start the LDAP daemon and enable it on boot

systemctl start slapd
systemctl enable slapd
systemctl status slapd

Now create an OpenLDAP administrative user and assign a password for that user
slappasswd will generate a hashed value for a given password which we can use to configure admin auth
create an LDIF file (ldaprootpasswd.ldif) which is used to add an entry to the LDAP directory.

nano ldaprootpasswd.ldif

Add the following contents in it:

dn: olcDatabase={0}config,cn=config
changetype: modify
add: olcRootPW
olcRootPW: {SSHA}PASSWORD_CREATED

add the corresponding LDAP entry by specifying the URI referring to the LDAP server and the file above.

ldapadd -Y EXTERNAL -H ldapi:/// -f ldaprootpasswd.ldif

copy the sample database configuration file for slapd into the /var/lib/ldap directory, and set the correct permissions on the file.

cp /usr/share/openldap-servers/DB_CONFIG.example /var/lib/ldap/DB_CONFIG
chown -R ldap:ldap /var/lib/ldap/DB_CONFIG
systemctl restart slapd

import some basic LDAP schemas from the /etc/openldap/schema directory

ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/openldap/schema/cosine.ldif 
ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/openldap/schema/nis.ldif
ldapadd -Y EXTERNAL -H ldapi:/// -f /etc/openldap/schema/inetorgperson.ldif

add your domain in the LDAP database and create a file called ldapdomain.ldif for your domain.

dn: olcDatabase={1}monitor,cn=config
changetype: modify
replace: olcAccess
olcAccess: {0}to * by dn.base="gidNumber=0+uidNumber=0,cn=peercred,cn=external,cn=auth"
  read by dn.base="cn=auth,dc=example,dc=com" read by * none

dn: olcDatabase={2}hdb,cn=config
changetype: modify
replace: olcSuffix
olcSuffix: dc=example,dc=com

dn: olcDatabase={2}hdb,cn=config
changetype: modify
replace: olcRootDN
olcRootDN: cn=auth,dc=example,dc=com

dn: olcDatabase={2}hdb,cn=config
changetype: modify
add: olcRootPW
olcRootPW: {SSHA}PASSWORD_CREATED

dn: olcDatabase={2}hdb,cn=config
changetype: modify
add: olcAccess
olcAccess: {0}to attrs=userPassword,shadowLastChange by
  dn="cn=auth,dc=example,dc=com" write by anonymous auth by self write by * none
olcAccess: {1}to dn.base="" by * read
olcAccess: {2}to * by dn="cn=auth,dc=example,dc=com" write by * read

add the above configuration to the LDAP database

ldapmodify -Y EXTERNAL -H ldapi:/// -f ldapdomain.ldif

create baseldapdomain.ldif

dn: dc=example,dc=com
objectClass: top
objectClass: dcObject
objectclass: organization
o: example com
dc: example

dn: cn=auth,dc=example,dc=com
objectClass: organizationalRole
cn: Manager
description: Directory Manager

dn: ou=People,dc=example,dc=com
objectClass: organizationalUnit
ou: People

dn: ou=Group,dc=example,dc=com
objectClass: organizationalUnit
ou: Group

add the entries to the LDAP directory.

ldapadd -Y EXTERNAL -x -D cn=auth,dc=example,dc=com -W -f baseldapdomain.ldif

create an LDAP User

useradd tomahawk
passwd tomahawk

create an LDAP group create a file called ldapgroup.ldif

dn: cn=auth,ou=Group,dc=example,dc=com
objectClass: top
objectClass: posixGroup
gidNumber: 1005

gidNumber is the GID in /etc/group for tomahawk and add it to the OpenLDAP directory.

Add to OpenLDAP directory.

ldapadd -Y EXTERNAL -x  -W -D "cn=auth,dc=example,dc=com" -f ldapgroup.ldif

Create an LDAP user create a file named tomahawk.ldif

dn: uid=tomahawk,ou=People,dc=example,dc=com
objectClass: top
objectClass: account
objectClass: posixAccount
objectClass: shadowAccount
cn: tomahawk
uid: tomahawk
uidNumber: 1005
gidNumber: 1005
homeDirectory: /home/tomahawk
userPassword: {SSHA}PASSWORD_HERE
loginShell: /bin/bash
gecos: tecmint
shadowLastChange: 0
shadowMax: 0
shadowWarning: 0

** Run slappasswd to generate a hashed password **

Add the above file to LDAP directory

ldapadd -Y EXTERNAL  -x -D cn=auth,dc=example,dc=com -W -f  tomahawk.ldif

You can also download Tools like
Apache Directory Studio to manage LDAP after following the above steps to add more user and groups without creating config files.

Optional Steps

LDAPS ( LDAP via SSL)

Now we will generate a certificate and a private key so we can communicate securely with the OpenLDAP server using OpenSSL

openssl req -new -x509 -nodes -out \
/etc/openldap/certs/auth.example.com.cert \
-keyout /etc/openldap/certs/auth.example.com.key \
-days 365

Change the owner and group permissions so OpenLDAP can read the files:

chown -R ldap:ldap /etc/openldap/certs

Now create ssl.ldif to configure OpenLDAP to use the LDAPS protocol:

dn: cn=config
changetype: modify
replace: olcTLSCertificateFile
olcTLSCertificateFile: /etc/openldap/certs/auth.example.com.cert

dn: cn=config
changetype: modify
replace: olcTLSCertificateKeyFile
olcTLSCertificateKeyFile: /etc/openldap/certs/auth.example.com.key

Add the above file to LDAP directory

ldapmodify -Y EXTERNAL -H ldapi:/// -f ssl.ldif

Test SSL configuration using

slaptest -u

You have completed Openldap Setup in Centos 7

Centos 7 to 8 Upgrade

Syamkumar — Wed, 29 Jul 2020 07:09:41 +0000

Red Hat Enterprise Linux 8 was released on 2019-05-07, and everyone is waiting to find out when the CentOS rebuild will occur. This document is meant to cover general questions and a timeline for what is happening.

A CentOS major release takes a lot of planning and changes in tooling as it is based on a much newer version of Fedora than previous versions. This means that everything from the installer, packages, packaging, and build systems need major overhauls to work with the newer OS. This means that there is always a ramp-up period depending on the changes needed to make a rebuild work. The differences between EL-8 and EL-7 are no exception as the kernel has changed drastically, the repository format has added 'modules' and RPMS has grown many features that EL7 and before do not have. About the only item which has not drastically changed between EL7 and EL8 is the init system which is still system.

To upgrade centos 7 to centos 8

Steps :

Install Epel release package

yum install epel-release -y

Install yum utils

yum install yum-utils -y

Resolve RPM packages by executing the command.

yum install rpmconf -y
rpmconf -a

Perform a clean-up of all the packages you don’t require.

 package-cleanup --leaves
 package-cleanup --orphans

Install DNF package manager since Centos 8 uses DNF instead of yum

yum install dnf -y

Remove yum package manager which was part of centos 7

dnf -y remove yum yum-metadata-parser 
rm -Rf /etc/yum

Run dnf upgrade -y
install CentOS 8 release package

dnf install http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/centos-repos-8.2-2.2004.0.1.el8.x86_64.rpm http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/centos-release-8.2-2.2004.0.1.el8.x86_64.rpm http://mirror.centos.org/centos/8/BaseOS/x86_64/os/Packages/centos-gpg-keys-8.2-2.2004.0.1.el8.noarch.rpm

upgrade the EPEL repository.

dnf -y upgrade https://dl.fedoraproject.org/pub/epel/epel-release-latest-8.noarch.rpm

remove all the temporary files.

dnf clean all

Remove the old kernel core for CentOS 7.

 rpm -e `rpm -q kernel`

to remove conflicting packages.

rpm -e --nodeps sysvinit-tools

launch the CentOS 8 system upgrade

 dnf -y --releasever=8 --allowerasing --setopt=deltarpm=false distro-sync

Troubleshooting for common errors during this step.

Python 3 dependency error. fix by

dnf remove python3

Segmentation fault

rm -rf /var/lib/rpm/__db.*
rpm --rebuilddb
dnf clean all
dnf makecache

install a new kernel for CentOS 8

dnf -y install kernel-core

install CentOS 8 minimal package

dnf -y groupupdate "Core" "Minimal Install"

check the version of CentOS installed by running.

cat /etc/redhat-release

You have complete Centos Upgrade