Forem: Dennis Maina

How to log PHP errors and warnings into a file

Dennis Maina — Wed, 03 Feb 2021 08:43:19 +0000

When writing server-side code in PHP, it's always a good practice to consider security first. When dealing with PHP errors that could occur at runtime, it's advisable to hide these error messages from the users and instead log them into a file for developers or system admins to review them. This is a good security practice as the error messages can expose an existing vulnerability or give an attacker an entry point into your system.

Enough of talking, how can we do that effectively and efficiently?

We are going to create a class that's going to be invoked every time an error occurs in the system.

If you are using an MVC place this code at the top of your main index file.
If you are not using an MVC, place this code at the top of every PHP file that you want the errors handled.

include('<REPLACE WITH THE LOCATION OF YOUR FILE>.php');
if (ERR_HANDLER_ENABLED) { 
    register_shutdown_function('sysFatalErr'); 
    set_error_handler('sysErrorhandler'); 
}

This class collects all the errors reported from a PHP script, classifies them as either general errors or fatal errors. After classification, it passes the error to the respective method which then calls the log function that writes them out.

Let's define some constant variables here.

/**
 * if your error class file resides in a folder called classes and the index file is in the base folder use:
 *
 * define('ERR_HANDLER_PATH', substr(dirname(__file__), 0, strpos(dirname(__file__), 'classes') - 1) . '/'); // DO NOT change!!
 * 
 * after using the above line comment or remove the fisrt line down here
**/
define('ERR_HANDLER_PATH', dirname(__file__).'/'); // DO NOT change!!
define('ERR_HANDLER_LOG_FOLDER', 'logs'); // Name of logs folder.. Create if it does not exist.
define('ERR_HANDLER_ENABLED', 1); // Enable custom error handler?
define('ERR_HANDLER_DISPLAY', 1); // Display a message on screen?
define('ERR_APPEND_RAND_STRING', 0); // Adds random string to file name for security. Prevents someone attempting browser access.
define('MASK_FILE_PATH', 0); // Hide file path if error occurs..
define('FILE_ERR_LOG_FILE', 'errors.log'); // File name of error log
define('FILE_FATAL_ERR_LOG_FILE', 'fatal_errors.log'); // File name of fatal error log

We create our class which has four functions and each function calls the log function when invoked:

generalErr - This function is invoked when a general error occurs.
mailErr - This function is invoked when an email error occurs.(if you have an email sending script)
fataErr - This function is invoked when a fatal error occurs and no further execution can happen.
log - calls the write function which then writes the error into a file.

class Errs {

  public function generalErr($error) {
    Errs::log($error, FILE_ERR_LOG_FILE);
  }

  public function mailErr($error) {
    Errs::log($error, FILE_ERR_LOG_FILE);
  }

  public function fatalErr($error) {
    Errs::log($error, FILE_FATAL_ERR_LOG_FILE);
  }

  public function log($error, $file) {
    if (is_dir(ERR_HANDLER_PATH . ERR_HANDLER_LOG_FOLDER)) {
      write(ERR_HANDLER_PATH . ERR_HANDLER_LOG_FOLDER . '/' . Errs::raStr() . $file, trim($error) . linending() . '***** - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -  - - - - - - - - - - - - *****' . linending());
    }
  }

  public function raStr() {
    return (ERR_APPEND_RAND_STRING ? substr(md5(uniqid(rand(),1)), 3, 30) . '-' : '');
  }


}

// Initiate the class..
$DDEH = new Errs();

Now we check to see if the error handler is enabled. If it is enabled, we turn of displaying all the errors to the user.

if (ERR_HANDLER_ENABLED) {
  // Switch off display errors
  @ini_set('display_errors', 0);
  // Set error reporting level..
  error_reporting(E_ALL);
}

On the Error reporting levels, we're going to be using E_ALL. PHP provides lots of them which you can look at here PHP Error Levels.
Here's just a few that I think are useful together with their error codes.

Error	Error Code	Description
E_ERROR	1	Fatal runtime error that can be recovered, The execution of the script is stopped immediately
E_WARNING	2	A runtime warning, not fatal and most errors fall here, execution is not stopped
E_NOTICE	8	Runtime notice indicating something that could possibly be an error was encountered when running a script normally
E_COMPILE_ERROR	64	Fatal error that occurs while the script was being compiled generated by the Zend Scripting engine
E_USER_ERROR	256	A fatal user-generated error, generated by the PHP code using the function trigger_error() rather than the PHP engine
E_USER_WARNING	512	NON FATAL USER GENERATED warning message, generated by the PHP code using the function trigger_error() rather than the PHP engine
E_STRICT	2048	Not strictly an error but it's triggered whenever PHP encounters code that could lead to problems or forward incompatibilities
E_ALL	32767	All errors and warnings except from level E_STRICT

TIP: Passing a value of (-1) to the error_reporting() function will show every possible error when new levels and constants are added in future PHP versions.

Now, we create a function that inserts a newline depending on the OS architecture.

function linending() {
  $newline = "\r\n";
  if (isset($_SERVER["HTTP_USER_AGENT"]) && strstr(strtolower($_SERVER["HTTP_USER_AGENT"]), 'win')) {
    $newline = "\r\n";
  } else if (isset($_SERVER["HTTP_USER_AGENT"]) && strstr(strtolower($_SERVER["HTTP_USER_AGENT"]), 'mac')) {
    $newline = "\r";
  } else {
    $newline = "\n";
  }
  return (defined('PHP_EOL') ? PHP_EOL : $newline);
}

This function will be called whenever a fatal error occurs.

function sysFatalErr() {
  global $DDEH;
  $error = error_get_last();
  if (isset($error['type'])) {
    if ((defined('E_ERROR') && $error['type'] == E_ERROR) || $error['type'] == 4) {
      $string = '[Error Code: ' . $error['type'] . '] ' . $error['message'] . linending();
      $string .= '[Date/Time: ' . date('j F Y @ H:iA') . ']' . linending();
      $string .= '[Fatal error on line ' . $error['line'] . ' in file ' . $error['file'] . ']';
      if (ERR_HANDLER_DISPLAY) {
        echo '<div style="background:#ff9999"><p style="padding:10px;color:#fff">A fatal error has occurred. For more details please view "' . ERR_HANDLER_LOG_FOLDER . '/' . FILE_FATAL_ERR_LOG_FILE . '".</div>';
      }
      $DDEH->fatalErr($string);
    }
  }
}

This other function is called when general errors occur, and can further classify them accordingly. You can add more in the switch block.

function sysErrorhandler($errno, $errstr, $errfile, $errline) {
  global $DDEH;
  if (!(error_reporting() & $errno)) {
    return;
  }
  if (!method_exists($DDEH,'generalErr') || !method_exists($DDEH,'fatalErr')) {
    return;
  }
  switch ($errno) {
    case E_USER_ERROR:
      $string = '[Error Code: ' . $errno . '] ' . $errstr . linending();
      $string .= '[Date/Time: ' . date('j F Y @ H:iA') . ']' . linending();
      $string .= '[Error on line ' . $errline . ' in file ' . $errfile . ']';
      if (ERR_HANDLER_DISPLAY) {
        echo '<div style="background:#ff9999"><p style="padding:10px;color:#fff">A fatal error has occurred. For more details please view "' . ERR_HANDLER_LOG_FOLDER . '/' . FILE_FATAL_ERR_LOG_FILE . '".</div>';
      }
      $DDEH->fatalErr($string);
      exit;
      break;

    case E_USER_WARNING:
      $string = '[Error Code: ' . $errno . '] ' . $errstr;
      $string .= '[Date/Time: ' . date('j F Y @ H:iA') . ']' . linending();
      $string .= '[Error on line ' . $errline . ' in file ' . $errfile . ']';
      if (ERR_HANDLER_DISPLAY) {
        echo '<div style="background:#ff9999"><p style="padding:10px;color:#fff">An error has occurred. For more details please view "' . ERR_HANDLER_LOG_FOLDER . '/' . FILE_ERR_LOG_FILE . '".</div>';
      }
      $DDEH->generalErr($string);
      break;

    case E_USER_NOTICE:
      $string = '[Error Code: ' . $errno . '] ' . $errstr . linending();
      $string .= '[Date/Time: ' . date('j F Y @ H:iA') . ']' . linending();
      $string .= '[Error on line ' . $errline . ' in file ' . $errfile . ']';
      if (ERR_HANDLER_DISPLAY) {
        echo '<div style="background:#ff9999"><p style="padding:10px;color:#fff">An error has occurred. For more details please view "' . ERR_HANDLER_LOG_FOLDER . '/' . FILE_ERR_LOG_FILE . '".</div>';
      }
      $DDEH->generalErr($string);
      break;

    default:
      $string = '[Error Code: ' . $errno . '] ' . $errstr . linending();
      $string .= '[Date/Time: ' . date('j F Y @ H:iA') . ']' . linending();
      $string .= '[Error on line ' . $errline . ' in file ' . $errfile . ']';
      if (ERR_HANDLER_DISPLAY) {
        echo '<div style="background:#ff9999"><p style="padding:10px;color:#fff">An error has occurred. For more details please view "' . ERR_HANDLER_LOG_FOLDER . '/' . FILE_ERR_LOG_FILE . '".</div>';
      }
      $DDEH->generalErr($string);
      break;
  }
  return true;
}

Finally, we write out our errors into a file. Remember to have the logs folder created.

function write($file, $data) {
  file_put_contents($file, $data, FILE_APPEND);
}

You can get the whole code on the GitHub repo Here

In need of a developer for a project? Hit me Up DentriceDev Solutions

Happy Coding.

How to create an AI Chatbot in Python and Flask

Dennis Maina — Wed, 27 Jan 2021 09:29:39 +0000

Perhaps you have heard this term and wondered: what is this chatbot, what is it used for, do I really need one, how can I create one? If you just want to build your own simple chatbot, this article will take you through all the steps in creating one for yourself. Let's dive right in.

Preliminaries:

Python programming knowledge
Flask framework knowledge
Machine learning and algorithm knowledge

What is a chatbot?
Where is a chatbot used?
Do you need a chatbot?
Categories of chatbots
Terms Used
Start Programming

What is a Chatbot?

A Chatbot, also called an Artificial chat agent, is a software program driven by machine learning algorithms that aim at simulating a human-human like conversation with a user by either taking input as text or speech from the user.

Where is it used?

Chatbots have extensive usage, and we can not expound on all the possibilities where it can be of use. But basically, you'll find them in: Help desks, transaction processing, customer support, booking services, and providing 24-7 real-time chat with clients.

Do I need one?

Well, this is a personalized opinion where one has to do a cost-benefit analysis and decide whether it is a worthwhile project.

At the current technology stand, most companies are slowly transitioning to use chatbots for their in-demand day-day services. A good example that everybody uses is the Google Assistant, Apple Siri, Samsung Bixby, and Amazon Alexa.

In this article, we will learn how to create one in Python using TensorFlow to train the model and Natural Language Processing(nltk) to help the machine understand user queries.

There are two broad categories of chatbots:

Rule-Based approach - Here the bot is trained based on some set rules. It is from these rules that the bot can process simple queries but can fail to process complex ones.
Self-Learning approach - Here the bot uses some machine learning algorithms and techniques to chat. It is further subcategorized into two:
- Retrieval-Based models - In this model, the bot retrieves the best response from a list depending on the user input.
- Generative models - This model comes up with an answer rather than searching from a given list. These are the Intelligent Bots.

In this chatbot, we will use the rule-based approach.

Terms to encounter

Natural Language Processing(nltk) - This is a subfield of linguistics, computer science, information engineering, and artificial intelligence concerned with the interactions between computers and human (natural) languages.
Lemmatization - This is the process of grouping together the different inflected forms of a word so they can be analyzed as a single item and is a variation of stemming. For example “feet” and “foot” are both recognized as “foot”.
Stemming - This is the process of reducing inflected words to their word stem, base, or root form. For example, if we were to stem the word “eat”, “eating”, “eats”, the result would be the single word “eat”.
Tokenization - Tokens are individual words and “tokenization” is taking a text or set of text and breaking it up into its individual words or sentences.
Bag of Words - This is an NLP technique of text modeling for representing text data for machine learning algorithms. It is a way of extracting features from the text for use in machine learning algorithms.

Let's start programming now

Directory structure

|--  Static
|    |--  style.css
|--  Templates
|    |--  index.html
|--  app.py
|--  train.py
|--  intents.json
|--  words.pkl
|--  classes.pkl
|--  chatbot_model.h5

First, we need to make sure that we have all the required libraries and modules. Install tensorflow, nltk, and flask using the following command.

pip install tensorflow
pip install tensorflow-gpu
pip install nltk
pip install Flask

Our intents.json file should look something like this: Here I'm sorry it's quite a long file I can't embed it here.
Now that we have everything ready, Let's start by creating the train.py file. We need to import all the packages and libraries that we're going to be using.

# libraries
import random
from tensorflow.keras.optimizers import SGD
from keras.layers import Dense, Dropout
from keras.models import load_model
from keras.models import Sequential
import numpy as np
import pickle
import json
import nltk
from nltk.stem import WordNetLemmatizer

Download punkt, omw-1.4, and wordnet package. Punkt is a pre-trained tokenizer model for the English language that divides the text into a list of sentences.

nltk.download('omw-1.4')
nltk.download("punkt")
nltk.download("wordnet")

We now need to initialize some files and load our training data. Note that we are going to be ignoring "?" and "!". If you have some other symbols or letters that you want the model to ignore you can add them at the ignore_words array.

# init file
words = []
classes = []
documents = []
ignore_words = ["?", "!"]
data_file = open("intents.json").read()
intents = json.loads(data_file)

Then we tokenize our words

for intent in intents["intents"]:
    for pattern in intent["patterns"]:

        # take each word and tokenize it
        w = nltk.word_tokenize(pattern)
        words.extend(w)
        # adding documents
        documents.append((w, intent["tag"]))

        # adding classes to our class list
        if intent["tag"] not in classes:
            classes.append(intent["tag"])

Next, we shall lemmatize the words and dump them in a pickle file

words = [lemmatizer.lemmatize(w.lower()) for w in words if w not in ignore_words]
words = sorted(list(set(words)))

classes = sorted(list(set(classes)))

print(len(documents), "documents")

print(len(classes), "classes", classes)

print(len(words), "unique lemmatized words", words)


pickle.dump(words, open("words.pkl", "wb"))
pickle.dump(classes, open("classes.pkl", "wb"))

Now that we have our training data in place, we initialize the model training

# initializing training data
training = []
output_empty = [0] * len(classes)
for doc in documents:
    # initializing bag of words
    bag = []
    # list of tokenized words for the pattern
    pattern_words = doc[0]
    # lemmatize each word - create base word, in attempt to represent related words
    pattern_words = [lemmatizer.lemmatize(word.lower()) for word in pattern_words]
    # create our bag of words array with 1, if word match found in current pattern
    for w in words:
        bag.append(1) if w in pattern_words else bag.append(0)

    # output is a '0' for each tag and '1' for current tag (for each pattern)
    output_row = list(output_empty)
    output_row[classes.index(doc[1])] = 1

    training.append([bag, output_row])
# shuffle our features and turn into np.array
random.shuffle(training)
training = np.array(training)
# create train and test lists. X - patterns, Y - intents
train_x = list(training[:, 0])
train_y = list(training[:, 1])
print("Training data created")

We shall create a 3-layer output model. The first layer having 128 neurons, the second layer having 64 neurons, and the third layer contains the number of neurons equal to the number of intents to predict output intent with softmax.
We shall be using ReLu activation function as it's easier to train and achieves good perfomance.

model = Sequential()
model.add(Dense(128, input_shape=(len(train_x[0]),), activation="relu"))
model.add(Dropout(0.5))
model.add(Dense(64, activation="relu"))
model.add(Dropout(0.5))
model.add(Dense(len(train_y[0]), activation="softmax"))
model.summary()

Then, we compile the model. Stochastic gradient descent with Nesterov accelerated gradient gives good results for this model. I won't go into details about Stochastic gradient descent as this is a vastly complicated topic on its own.

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss="categorical_crossentropy", optimizer=sgd, metrics=["accuracy"])

OPTIONAL: For choosing an optimal number of training epochs to avoid underfitting or overfitting use an early stopping callback to Keras based on either accuracy or loss monitoring. If the loss is being monitored, training comes to halt when there is an increment observed in loss values. Or, If accuracy is being monitored, training comes to halt when there is a decrement observed in accuracy values.

from Keras import callbacks 
earlystopping = callbacks.EarlyStopping(monitor ="loss", mode ="min", patience = 5, restore_best_weights = True)
callbacks =[earlystopping]

Now we can train our model and save it for fast access from the Flask REST API without the need of retraining.

hist = model.fit(np.array(train_x), np.array(train_y), epochs=200, batch_size=5, verbose=1)
model.save("chatbot_model.h5", hist)
print("model created")

Your train.py file now should look like this:

Run the train.py to create the model.

Now that we are done with training let's create the Flask interface to initialize the chat functionalities.

We load the required libraries and initialize the Flask app

# libraries
import random
import numpy as np
import pickle
import json
from flask import Flask, render_template, request
from flask_ngrok import run_with_ngrok
import nltk
from keras.models import load_model
from nltk.stem import WordNetLemmatizer
lemmatizer = WordNetLemmatizer()


# chat initialization
model = load_model("chatbot_model.h5")
intents = json.loads(open("intents.json").read())
words = pickle.load(open("words.pkl", "rb"))
classes = pickle.load(open("classes.pkl", "rb"))

app = Flask(__name__)
#run_with_ngrok(app) -Use this option if you have ngrok and you want to expose your chatbot to the real world

@app.route("/")
def home():
    return render_template("index.html")

This function will be called every time a user sends a message to the chatbot and returns a corresponding response based on the user query.

@app.route("/get", methods=["POST"])
def chatbot_response():
    msg = request.form["msg"]
    if msg.startswith('my name is'):
        name = msg[11:]
        ints = predict_class(msg, model)
        res1 = getResponse(ints, intents)
        res =res1.replace("{n}",name)
    elif msg.startswith('hi my name is'):
        name = msg[14:]
        ints = predict_class(msg, model)
        res1 = getResponse(ints, intents)
        res =res1.replace("{n}",name)
    else:
        ints = predict_class(msg, model)
        res = getResponse(ints, intents)
    return res

The above function will call the following functions which clean up sentences and return a bag of words based on the user input.

def clean_up_sentence(sentence):
    sentence_words = nltk.word_tokenize(sentence)
    sentence_words = [lemmatizer.lemmatize(word.lower()) for word in sentence_words]
    return sentence_words


# return bag of words array: 0 or 1 for each word in the bag that exists in the sentence
def bow(sentence, words, show_details=True):
    # tokenize the pattern
    sentence_words = clean_up_sentence(sentence)
    # bag of words - matrix of N words, vocabulary matrix
    bag = [0] * len(words)
    for s in sentence_words:
        for i, w in enumerate(words):
            if w == s:
                # assign 1 if current word is in the vocabulary position
                bag[i] = 1
                if show_details:
                    print("found in bag: %s" % w)
    return np.array(bag)

The next functions are for predicting the response to give to the user where they fetch that response from the chatbot_model.h5 file generated after the training.

def predict_class(sentence, model):
    # filter out predictions below a threshold
    p = bow(sentence, words, show_details=False)
    res = model.predict(np.array([p]))[0]
    ERROR_THRESHOLD = 0.25
    results = [[i, r] for i, r in enumerate(res) if r > ERROR_THRESHOLD]
    # sort by strength of probability
    results.sort(key=lambda x: x[1], reverse=True)
    return_list = []
    for r in results:
        return_list.append({"intent": classes[r[0]], "probability": str(r[1])})
    return return_list


def getResponse(ints, intents_json):
    tag = ints[0]["intent"]
    list_of_intents = intents_json["intents"]
    for i in list_of_intents:
        if i["tag"] == tag:
            result = random.choice(i["responses"])
            break
    return result

Generally, our app.py should look like this:

Lastly, we have the index.html file which I won't go into detail about as it's basic HTML and CSS.

And there you have your working chatbot.

For the full code with all the files visit my GitHub repo here. Don't forget to Star if you find it helpful.

Also to access the pre-trained model there in case you don't have enough computing power to train or it takes much time.

In need of a developer for a project? Hit Us Up DentriceDev Solutions

Happy Coding Devs.

How to set up a proxy server in Kali Linux

Dennis Maina — Mon, 25 Jan 2021 08:36:34 +0000

We all hate it knowing that we are leaving our prints on the internet every time we surf. Or maybe you want to access a service that is not allowed in your location and you simply can't. This article will help you become anonymous on the internet and overcome some of those barriers. Let's have some fun.

Disclaimer: This article is purely for educational purposes only. I am not liable for any malicious intent done with this information.

Required tools

tor
proxychains

Proxychains is an open-source software and mostly comes pre-installed in Kali Linux. This tool redirects TCP connections through proxies like TOR, SOCKS4, and SOCKS5 and as the name suggests, it allows us to chain proxy servers. With this tool we can hide the source address of our traffic and evade IDS and Firewalls.

To install tor, run this command in your terminal.

sudo apt install tor

To install proxychains, run this command.

sudo apt install proxychains

Next, we're going to edit the procychains configuration file.
You can edit this using your favorite text editor. In our case, we're going to be using nano editor which runs in the terminal.
On your terminal type,

sudo nano /etc/proxychains.conf

After opening the file, we need to do a few changes;
To enable a functionality, remove the '#' at the beginning of that line, and to disable add a '#' at the beginning of that line.

Look for the line with dynamic_chain and enable it.
Look for the line with strict_chain and disable it.
Look for the line with proxy_dns and enable it.
Look for the line with Proxy DNS requests - no leak for DNS data and enable it.
Add tor socks5 at the end.

Now, your file should look something like this.

# proxychains.conf  VER 3.1
#
# HTTP, SOCKS4, SOCKS5 tunneling proxifier with DNS.
#       

# The option below identifies how the ProxyList is treated.
# only one option should be uncommented at time,
# otherwise the last appearing option will be accepted
#
dynamic_chain
#
# Dynamic - Each connection will be done via chained proxies
# all proxies chained in the order as they appear in the list
# at least one proxy must be online to play in chain
# (dead proxies are skipped)
# otherwise EINTR is returned to the app
#
#strict_chain
#
# Strict - Each connection will be done via chained proxies
# all proxies chained in the order as they appear in the list
# all proxies must be online to play in chain
# otherwise EINTR is returned to the app
#
#random_chain
#
# Random - Each connection will be done via random proxy
# (or proxy chain, see  chain_len) from the list.
# this option is good to test your IDS :)

# Make sense only if random_chain
#chain_len = 2

# Quiet mode (no output from library)
#quiet_mode

Proxy DNS requests - no leak for DNS data
proxy_dns
# Some timeouts in milliseconds
tcp_read_time_out 15000
tcp_connect_time_out 8000

# ProxyList format
#       type  host  port [user pass]
#       (values separated by 'tab' or 'blank')
#
#
#        Examples:
#
#               socks5  192.168.67.78   1080    lamer   secret
#               http    192.168.89.3    8080    justu   hidden
#               socks4  192.168.1.49    1080
#               http    192.168.39.93   8080    
#
#
#       proxy types: http, socks4, socks5
#        ( auth types supported: "basic"-http  "user/pass"-socks )
#
[ProxyList]
# add proxy here ...
# meanwile
# defaults set to "tor"
socks4  127.0.0.1 9050
socks5 127.0.0.1 9050

Now save the changes by pressing CTRL + O then ENTER followed by CTRL + X to exit.

SOCKS is an internet protocol that routes packets between a server and a client using a proxy server.
127.0.0.1 is the loopback IP address.
9050 is the port by default TOR listens on for proxy connections.

Finally, start the tor service by running this command:

service tor start

To check the status of the service, run

service tor status

After starting TOR, run the following command to start proxychains

proxychains firefox www.duckduckgo.com

And in the browser tab opened visit https://dnsleaktest.com/ to do a DNS Leak Test.

The good thing about proxychains is that it dynamically changes your IP address without leaking your DNS

And Boom! I'm in the US.