Forem: Adnan Alam

Containerize your Dev environment! Forget about painful project setup. (Beginner friendly)

Adnan Alam — Sat, 29 May 2021 00:24:31 +0000

After only a few years of working in the industry I am already tired of having to setup my machine each time something happens to it or I want to work on it on my laptop when visiting someone. Setting up DEV environment has irked me for as long as I have been working in this industry. And after some inspiration from a company that is doing many things right, I decided to make this amazing solution easily available to everyone!

NOTE: IDE will not work on Windows. Please read windows section.

Pre-requisite

Install docker
For Windows: install wsl

Disclaimer

Windows

The display of IDE is done by piggybacking off the X11 server that unix systems use for display. This is not available in windows. However, you should still have full read write access to the "devenv" folder which will be your "home" workspace in your container. So this means that you should still be able to load the files in your favorite IDE, just cannot run it directly from the container.
(Note: I am still looking to overcome this challenge.)
Edit: I've found that you can use vcxsrv. Please check the github repo for this and download the executable you trust! Once you have this installed, start the X server via XLaunch. I would recommend "Full Screen". Run ipconfig and grab your IP, then set your DISPLAY environment variable (if you are using mingw64 like me you can run export DISPLAY=<?IP?>:0.0). Then you can run the windocker.sh. You can then start off idea or any other app through this method. (I do understand that this is a lot of workaround, so let's hope Microsoft starts supporting X server natively 😁)

Mac

I have not tested this, please try out the dockerrun.sh script, and let me know if this does not work.

Key Concepts

I don't intend to leave behind the many developers that are just getting started. So, here's the general breakdown that you HAVE TO know.

Container is just as it sounds, a box to separate out resources used by it. This is somewhere between running apps directly in your host vs running in a VM (Virtual Machine). In a VM only the hardware resources are shared. However, in docker the OS kernel is shared. This does mean that the security of your container can be compromised if you let it (That's a whole another beast of its own, so I'll leave that out. Yes, this does mean you should not just pull down some random image and run exactly as the writer tells you to 😭. PLEASE ONLY USE TRUSTED IMAGES).
Docker runs natively on your machine, and in this tutorial you will learn to mount folders as volumes into your container so that the work is easily available (generally, you can access any files the container is using from the host, because of the nature of containerization).
VOLUMES!! Now this is the meat of the entire blog. You will mount your work folders as volumes onto the container which will let you easily work and streamline development. The main idea here is to give the container access to your work environment (for this, I'll assume all of your projects live in a single folder).
Image Layers. Docker images use layering. Where each layer is generated by a statement in your "Dockerfile" (think of this as the configuration for your container). This is great for many reasons but the most important of them all is re-usability. We can pretty much build on top of an existing image, create our own "base" image and layer that based on what kind of project we will be working on. Think of a sandwich. You put the bread down, your partner puts in the meat, then the kids choose the toppings.

That should cover most of the key questions you have regarding this. Please comment on whatever finer details you need and I will be happy to help.

Base Module

Lets take a look at the main Dockerfile which you will use as the starting point for many of your customized images.
Please go to the github repo to find the source code! 😊

Dockerfile:

We will start off with openjdk alpine so that we don't have to install and configure jdk, and set environment variables.
Then let's go ahead and setup for the user. Running as root is sadly not recommended. So we will configure a "dev" user. This definitely does mean that you will not be able to run some of the commands you are used to running as root. However, you can always shell into a container as root, and run sudo commands that way.
Now let's go ahead and update our alpine image.
Let's create the user we setup.
Finally add our base applications and install dev tools.
Let's install the browser and IDE.
Last thing is to make sure we have proper access to our work env.

dockerrun.sh (Shell script)

Now we really want to automate the image creation and container start and attach process. This is because very few developers like to do extra manual work 😛.
Let's cleanup if there's a stopped container.
If there is no container then we create and start a shell session. If there is then we simply start a new shell session within the same container.

That is all! 😊

Setup the environment

Let's start off by cloning the repo. You will build the image yourself so that you have better control over it.

Let's go into where you will be working cd <workdir>
git clone https://github.com/opiteq/dev-environment .
./dockerrun.sh
now if you open up a new terminal you can simply run the same shell script to get inside of your container.
I have setup ports 3000-3003 for dev, so you can easily access them from host if needed. Idea is you would run your server and use one of those ports.
Now, if you are on linux, go ahead and run idea or firefox or geany from the terminal and voilà.

Breakdown

The core concept here is to get you started with a base image then you customize it to your heart's content and get going with your development. I chose alpine linux as the base image because I think less is more. However, I did go with openjdk image so that majority of the java configuration is handled for me. This is also a work-in-progress, so the repository may evolve overtime as I find better fit for my needs.

Conclusion

Your container is what you make of it! Please feel free to customize it to your heart's content and reach out to the amazing community with your questions. I would love to address anything that I missed so definitely let me know. If you think anything I said is incorrect I would be thrilled to learn and correct myself for future readers.
Leave your questions down below and I will get to them as soon as I can.

Make an IoT Device for tracking vehicles

Adnan Alam — Mon, 15 Jun 2020 03:03:41 +0000

Introduction

IoT (Information of Things) devices have become more and more prevalent in everyday life. From home assistants to smart toasters these devices are all over most households and offices. Most of the house appliances utilize WiFi module in order to communicate with the designated API. However, for IoT devices to be able to collect data at a remote location or on the go a SIM module would be needed. In this article we will explore implementing an IoT device to collect and send data on the go.

BOM

Below are the parts that are required for the project. (Note: Though link provided, please purchase at your own risk)

Raspberry pi (Zero W used for demo, but any model should work)
SIM7000 (A used for demo, but select based on location) breakout board (breakout boards simplify the circuit for easier development/testing)
Female-Female jumper wires
Header pins (for the Zero model this is needed)
GSM Antenna
GPS Antenna

Schematic

The wiring is straight forward, breakout board needs 5V so, connect pin 2 (5V) from raspberry pi to the V pin then connect pin 6 (GND) from raspberry pi to G. For UART protocol, connect Receiver pin to Transmitter pin, so R pin connects with pin 8 (Tx), and T pin connects with pin 10 (Rx). Please refer to the below schematic for better guidance.

Fig 1.1: Raspberry pi and SIM7000 breakout board schematic

Setting up Raspberry pi

The raspberry pi zero W has a WiFi module that allows setting up the device headless. To do this, please follow this wiki: http://wiki.lofarolabs.com/index.php/Install_Raspbian
Once the device is setup headless or with monitor, keyboard and mouse, please start off with making sure that everything is updated.
sudo apt-get update
sudo apt-get install upgrade -Y
Once everything is updated and the device is ready, please start validating that UART is enabled.

sudo raspi-config
option 5 - interfacing options
option P6 - serial
Would you like a login shell to be accessible over serial? -> No
Would you like the serial port hardware to be enabled? -> Yes

With UART configured please check whether or not you see the device we will be interfacing with /dev/ttyS0 with command: ls -l /dev | grep tty.
This project utilizes the UART protocol for communicating with the SIM7000 module. In this case there is a python-serial library that will simplify the communication with the module. However, initially there is a possibility that the device might not respond to commands, this is mostly due to a power issue. This is why standard USB port on laptops/desktops might not be enough to power the device. The recommendation is to first give it 10-15 minutes and see if it responds, if not would recommend trying a different power source. The best result was achieved by using a power brick, but YMMV. During this testing phase it is not practical to run the actual python code as there are already tools for sending UART commands.
Please run the following to install minicom.
sudo apt-get install minicom -Y
Once minicom is installed please run it by specifying the device and baudrate you want to use as well.
minicom -D /dev/ttyS0 -b 115200
Once in, for first time usage please note that to exit press [CTRL]+[A] -> [Z] (Note: not all three together, first two combo, then last key). This is crucial because if the device is not working then screen will be blank and you will not be able to get out, this should save you from that panic attack. For better understanding of what is going on, please press [CTRL]+[A] -> [E] to enable echo and this should allow you to see that the commands are being sent, just no response. For basic tests to see that the device is at least responding, please try the command AT in the minicom window/terminal. This should return OK. Once this is verified we can move onto the next step: Running the Code.

Running the Code

Please clone the library here. To verify this please ensure that the python-serial library is installed, as that is the only prerequisite for this script. Then run using python tracker.py and you should the commands and the output of the commands on the screen. Once done, please schedule it to run on boot by scheduling a cron job. Run crontab -l to check existing cron jobs, then run crontab -e to edit. Please add the following:
@reboot python /full/path/to/tracker.py
to the cron script. (Please comment if something is not working as expected)

Logical Breakdown

The basic concept for the project is to upload data from a low power consuming device on the go using internet to an API then utilize those data points to show the travel history of the device. This was then broken down into three components to simplify the project. The components are API, UI, and the IoT device. The API supports GET and POST request for footprints (data points). The UI displays the footprints on a map, and allows user to get direction to the place where the last ping was received from. Though there are libraries for the SIM modules, the most reliable implementation is through serial library, as the user could follow the official documentation for the AT commands by device model.

Fig 1.2: Diagram of the project design

Conclusion

The possibilities with IoT devices are limitless, however, because we see them mostly utilizing WiFi for communication which might be discouraged when thinking of outdoor projects. Whether you are concerned about reliability, cost, or simplicity of implementing your ideas the road towards them have already been paved. Hologram provides free data for beginners, SIMcom provides affordable SIM modules with well documented guides, and Heroku provides free easy to host solution for deploying web apps.

All relevant code:

If you would like more information on the code, please let me know and I will create a second part.

If you would like to see a different kind of project please let me know, and I will do my best to implement it.

Create a Web Server and save form data into MySQL database using PHP (Beginners Guide)

Adnan Alam — Tue, 16 Apr 2019 01:47:47 +0000

Static pages are great for showcasing your projects. However, if you would like to get data from clients and save it into the server or allow clients to retrieve data from your server then you would need a dynamic webpage. In this tutorial I will show you how to setup a web server, collect data and save into the server database.

Key concepts

Before we begin let's review a few HTTP methods. GET requests data (text/file) from the server. POST requests server to accept data (text/file) into the server. PUT requests to modify a specified resource in the server. DELETE requests server to remove a specified resource. We will only be using POST, and PUT in this tutorial.

Environment Setup

We will have to setup the server before we can begin to code for the website. There are many other languages and databases that can be used for this purpose, however this tutorial will only cover PHP, MySQL database and Apache web server. We will go over two methods, Linux, and Windows. I recommend using Linux as it is much simpler to get started. If you already have environment setup then move on to the coding block. If you do not have a Linux environment setup, then I would recommend using virtual-box to setup a basic Linux VM and run Ubuntu on it.

Windows

First in Windows, go ahead and install WAMP server from here. This will take a while as you will have to install a lot of packages. Please ensure all of the Visual C++ packages from the readme.txt file have been installed before moving forward as this can cause a lot of headache later on. Once installed then please open your browser then go to 'localhost' without the quotation marks. If you see a webpage then you are good to go. We will be working in the www/html folder inside of the wamp directory (the installation path). On the right hand corner you will see a green W icon indicating that the server is online and working.

I would recommend checking out this video if you still are not sure how to setup the wamp server.

Linux (Ubuntu)

If you have a different distribution then please use the appropriate package manager to install the tools necessary. Start off by installing Apache2.
sudo apt-get install apache2
Then ensure the service is running.
sudo service apache2 start

Afterwards install mysql and go through the setup. Please remember your root password.
sudo apt-get install mysql-server
sudo /usr/bin/mysql_secure_installation

Then install php and we should be ready to go.
sudo apt-get install php
sudo apt-get install php-mysql
sudo /etc/init.d/apache2 restart

Please ensure you install the php-mysql package as without it you will most likely get driver error.

Our working directory will be /var/www/html.

Apache2

In order to create new projects you will be creating new .conf files per project which will store the virtual host configurations. Please look at the existing config file to get a better understanding of this. For Linux this file can be found in the /etc/apache2/sites-available/ directory. Please check the file named 000-default.conf if available, otherwise just read any .conf file in that directory to get an understanding of the format.

Code

Once everything is setup please create a new file inside of the html folder (or your working directory if you have changed the Apache configurations).
We will start by creating a new file in the html directory.
sudo touch /var/www/html/form.html
Then go ahead and edit the file and populate it with some basic html code.
sudo nano /var/www/html/form.html



<head>
<title>
Test Page
</title>
</head>
<body>
<form action="form_submit.php" class="alt" method="POST">
<div class="row uniform">
<div class="name">
<input name="name" id="" placeholder="Name" type="text">
</div>
<div class="email">
<input name="email" placeholder="Email" type="email">
</div>
<div class="message">
<textarea name="message" placeholder="Message" rows="4"></textarea>
</div>
</div>
<br/>
<input class="alt" value="Submit" name="submit" type="submit">
</form>
</body>

Database

Now let's setup the database to so that we can store the information. I will only cover creating database and table in Linux. First login
mysql -u root -p



create database dev_to;
use dev_to;
create table test(
    id int NOT NULL AUTO_INCREMENT,
    name varchar(255),
    email varchar(255),
    message text,
    PRIMARY KEY (id)
);

Let's make sure this works.
describe test;
output should look like:

Let's exit out.
exit
Now we are finally ready for the PHP code that will actually handle the data and save it into the database.

Form Handling

Start by creating a new file named form_submit.php.
sudo touch /var/www/html/form_submit.php
Now let's populate it with some code.
sudo nano /var/www/html/form_submit.php
Now try the code below which should persist the data into the specified table in the database.



<?php
$host = "localhost";
$db_name = "dev_to";
$username = "root";
$password = "password";
$connection = null;
try{
$connection = new PDO("mysql:host=" . $host . ";dbname=" . $db_name, $username, $password);
$connection->exec("set names utf8");
}catch(PDOException $exception){
echo "Connection error: " . $exception->getMessage();
}

function saveData($name, $email, $message){
global $connection;
$query = "INSERT INTO test(name, email, message) VALUES( :name, :email, :message)";

$callToDb = $connection->prepare( $query );
$name=htmlspecialchars(strip_tags($name));
$email=htmlspecialchars(strip_tags($email));
$message=htmlspecialchars(strip_tags($message));
$callToDb->bindParam(":name",$name);
$callToDb->bindParam(":email",$email);
$callToDb->bindParam(":message",$message);

if($callToDb->execute()){
return '<h3 style="text-align:center;">We will get back to you very shortly!</h3>';
}
}

if( isset($_POST['submit'])){
$name = htmlentities($_POST['name']);
$email = htmlentities($_POST['email']);
$message = htmlentities($_POST['message']);

//then you can use them in a PHP function. 
$result = saveData($name, $email, $message);
echo $result;
}
else{
echo '<h3 style="text-align:center;">A very detailed error message ( ͡° ͜ʖ ͡°)</h3>';
}
?>

Now your website should be able to handle forms and save it onto the local database.

Here's a sample output from my test run:

Note: Please let me know about any issues you are having with this. The tutorial was created to help people get started on their journey to dynamic web development. If anything is unclear, or any information is inaccurate please let me know so I can correct it A.S.A.P.