Forem: Jordan Polaniec

Useful sites and apps I've come across while studying web frontend development and web server configuration

Jordan Polaniec — Wed, 16 Oct 2019 23:27:41 +0000

Web development is hard. There are seemingly infinite ways to make things work, many of them outdated and no longer useful to the developer because of the incredible speed with which the web development ecosystem pushes forward.

This speed has its pros and cons, and one of the drawbacks I've found while learning web development is that sometimes finding up-to-date resources for learning is difficult.

What follows is a smattering of resources I've found incredibly useful while beginning to study web development - both front- and back-end.

I've stayed away from learning frameworks, focusing instead on learning mostly fundamentals. For me personally, just learning a framework like Angular or React without having a solid understanding of the basics means long, frustrating debugging sessions while attempting to interpret what was messed up in the framework. Often times, this issue is exacerbated by the fact that the framework assumes I know the basics of whatever the context is and glosses over certain details when reporting error information.

Please note: I am in no way affiliated with any of the following resources.

Frontend basics

Interneting is Hard - This site offers a beautifully made tutorial about using HTML and CSS to build modern websites. It starts from the perspective of someone who knows nothing about HTML or CSS. If you're looking for a rock solid place to start this is it.

Mozilla Developer Network
This is most likely the best reference for general web development knowledge and has replaced the W3 Schools in my digital library.

CSS Diner
Practice CSS selectors while selecting fruits and veggies! CSS selectors are very important to building clean, maintainable CSS.

Browser feature compatibility

CanIUse
I quickly learned not all browsers are created equal. CanIUse shows just how different they are, feature by feature. Example queries:

CSS Grid

Grid By Example
While I spent a large amount of time learning Flexbox, CSS Grid is just as cool if not a bit more easy to use (in my novice opinion) from a responsive design perspective. This feature in particular is very cool.

Templating engines

I do not consider these to be frameworks akin to Angular or React, but they helped with some of the pain points of writing plain HTML, such as writing list items. I do not have a strong opinion on either yet so I've listed the two I've spent some time learning and using in small projects.

The idea behind these types of tools is helping componentize blocks of HTML by 'templating' out logical groups (like a 'card' of data) and allowing for data to be inserted into specific areas of the template at runtime. This way, a single logical grouping of HTML can be used over and over again while the actual markup only needs to exist once (while avoiding the overhead of larger frameworks that also do this).

It's also worth noting that Express has support for some template engines, although that is outside of the scope of this post.

Browser usage stats

gs.statcounter
Straightforward, no-nonsense data about browser usage, common screen resolutions and device usage. If you're looking to build a web app for a more specific audience this site could help you decide what to focus on first.

Design

I am not very artistic. The resources here mostly helped me learn some ways web design can be done. One crucial concept I learned is that there is nothing wrong with having someone dedicated to designing your site for you as the web developer to then build. This might mean acquiring a pre-built theme and then modifying it as needed or working with a full-time web designer. I found that I intertwined web development and web design as a single profession more than they need to be. Web design is a full-time job and people study it just as hard as programmers study programming. If you want something that looks professional and don't want to attempt to manage learning both good design and web programming yourself, these artists will do the design right, thus freeing your mind to focus on building the vision. That being said, understanding some of their processes and tooling will help you communicate with them which can be a major benefit.

Provisioning

It's one (very important) thing to know how to build the frontend of a website, but hosting it was something rather foreign to me as well. There are a smorgasboard of tools available to do this, many of them offering more or less options based on how much stuff in the server you want to manage yourself.

Here are some options:

Microsoft Azure
Coming from a .NET background I thought I'd be more confortable learning to host sites on Azure. I've hosted a few very small APIs on Azure in the past, but never any actual web sites.

To be honest, the amount of options in Azure was overwhelming to me as a novice. More power to you if you want to go this route or the Amazon Web Services route, but I found the complexity more than I wanted to take on at this stage of learning.

Heroku
At no point in time did I need to remote into a machine and run terminal commands to configure a site while using Heroku. Their web UI is more than enough to get a basic site hosted and they offer some really nice tooling around configuring Continuous Integration among some other great scaling options, and there is a free tier.

Digital Ocean
In constrast to Heroku, Digital Ocean requirements for configuration are much more technical, however, that is not a bad thing. In fact, coming at setting up a web server with almost no knowledge of how to do it led me to using Digital Ocean for several sites I've done so far specifically because I wanted to learn more about how this is done.

Aside from 'empty' VPS's (Virtual Private Servers - they call them Droplets) they have these One-click apps which are VPS's with a bunch of software pre-installed and configured on them. These are amazing, especially for a beginner like me. This reduces the overhead for getting a server up and running drasitically, allowing me to focus on the specifics of my server configuration (like setting up sites in Nginx). I've used the NodeJS and Ghost One-click apps so far and might check out the GitLab one in the future.

In addition to fantastic infrastructure provisioning, DigitalOcean has some of the best tutorials and documentation on web server configurations I've come across. Some examples:

DNSMap
One thing that burned me several times was that making DNS changes is not instant. There were a few times I was repointing a domain to different name servers while adjusting the site config (in Nginx). This was confusing because browsers would show the site was misconfigured (especially if SSL certs were involved) even though in reality DNS propagation of my changes had not yet completed. There are a bunch of variations on the above tool, but it helps to know when these types of changes have 'completed'. These tools aren't perfect, but I haven't found another way to be notified when propagation has been completed.

Performance

Google's PageSpeed Insights
You may have already heard of PageSpeed Insights since 'page speed' is now factored into Google search rankings for mobile. I found this tool useful after a first pass on a site as it helps identify issues that I found fixes easy to learn and implement. It also seperates mobile and desktop scores which can be very helpful if you're trying to tailor your site to a specific audience. Additionally, if you're looking for something more intensive, take a look at Lighthouse .

Nginx

In my brief time learning about how to configure web servers Nginx has been fairly straightforward to use. There are docs and articles galore about how to do very specific things.

Once I got some sites up and running I looked around for some services to monitor them. I didn't want to be writing my own tooling or parsing log files and landed on Nginx Amplify. You can connect a few Nginx instances to Amplify for free and the monitoring tools are great. Included in the free 'tier' is the ability to send Slack messages and emails when metrics go over thresholds you configure. Thanks to this tool I was notified of a bot making thousands of requests per second to one of my sites just a few minutes after the bot started doing this. A quick update to the firewall stopped the issue. Not only does this show monitoring metrics, but will use static analysis on your nginx.conf files to find common issues and show info about SSL configs and security advisories for the running version of Nginx. Again, tools like this are super useful to newbies like me as it helped give an awareness of the system I might not have been able to grasp otherwise.

NginxConfig.io - This tool helps kickstart writing nginx config files and server blocks

Web server configuration tools

Laboratory (Content Security Policy / CSP Toolkit)
While learning about secure server configurations I came across this Firefox addon. It will build your Content Security Policy while recording you using your site. This, of course, assumes your site is stable and serving expected data. It should also be noted this is an experimental addon.

Test your server's SSL implementation
Again, being a newbie I wanted to find tools that could tell me what I was messing up while learning. This site focuses on grading your SSL implementation in detail and is a good jumping point for seeing what makes up a secure SSL implementation.

Image optimization

Of all the frontend concepts I'm learning I find this the most challenging. I didn't discover Cloudinary until sinking a bunch of time into learning how to shape or serve variations of my images depending on the client parameters. In fact, using newer features like sizes and srcset require knowing how all the parameters to configure them work, which is a decent hill to climb by itself.

However, once I started to understand how sizes and srcset work in tandem with media queries I loved Cloudinary all the more. The work it does on the fly for the developer is incredible.

TinyPng is listed because it's important to understand how responsive image sizing (by itself) is different from compression-based image optimization. TinyPng (which supports more than PNGs) focuses on optimization of an existing image rather than only resizing it. While Cloudinary does offer this service as well, it's probably overkill for smaller sites. TinyPng will optimize your images for free up to a certain quota. I found their .NET api super easy to use to send up a folder of images for processing.

That's my set of sites and tools I've come across so far while learning web development. I hope some of them are useful to you!

Featured Image by nihon graphy

Part 7 - Deploying a simple web app with monitoring and analytics without Docker or containers for beginners

Jordan Polaniec — Thu, 10 Oct 2019 21:00:32 +0000

Thanks for reading this tutorial and I hope you enjoyed it! You now have a website, the ability to analyze server logs, hosted storage and analysis of client error logs and are receiving alerts on noteworthy system usage. What's next? Here's a few tools and posts you may want to check out:

NginxConfig.io - This tool provides a UI for building Nginx conf files. It has a dizzying amount of options available with many common presets. If you're looking to increase the capability of your Nginx configuration this might be a good place to start, even if just to see new terms to learn about.

Server hardening

There are a multitude of additional steps that can be taken to secure your server. We have done some of them, but there are always more. It's up to you how much time you want to put into this based on the website(s) you are serving.

OWASP on Nginx

Digital Ocean - How to secure Nginx on Ubuntu 14.04

Blocking malicious requests - In the tutorial we used ufw, which allows us to block specific IP addresses via command line entries. These can be found while analyzing server logs, but a more proactive approach could be using tools like fail2ban.
Nginx Monitoring - In the tutorial, we configuring monitoring for our Droplet's resources, but there tools to monitor Nginx itself. I've used Nginx Amplify before and it works well. There is a free tier. It involves installing an agent on your Droplet which then sends information back to the Nginx Amplify server. You can then view metrics and set alerts from the Nginx Amplify dashboard.
Nginx Log Rotation - It's important to keep your server logs tidy. logrotate is already installed and configured on our Droplet, but if you want to make changes you can do so in the /etc/logrotate.d/nginx file.

Hopefully this tutorial was useful for learning one of the many ways to setup your own website.

Thank you for reading!

Part 6 - Deploying a simple web app with monitoring and analytics without Docker or containers for beginners

Jordan Polaniec — Thu, 10 Oct 2019 21:00:25 +0000

In the previous post we went over using GoAccess to view server logs. In this part, I'll go over using Sentry to automatically report errors on the client-side.

Sentry has a Developer tier that should meet our needs. As of time of writing this, the Developer tier supports Real-time error tracking, which is the main feature we want. Configuring this for our website will enable the following:

Client-side errors will be logged and stored
We will be able to view errors and the details surrounding them (such as browser type)
We will be notified via email when new errors are encountered by users

Do I need a user consent banner to use this?

Sentry gathers some client-side info to use in tracking errors. If you decide to incorporate Sentry into your website I recommend adding a user consent banner with a Privacy Policy detailing what is done with the user's data. Sentry will help you meet GDPR compliance, but you should still notify your users with a Privacy Policy. There are a lot of way to handle this and I am not a lawyer so I will leave it up to you, the reader, to determine what's best for your website.

Installing and configuring Sentry

First, you will need to make an account with Sentry. In this tutorial, we will be using the hosted version of Sentry, but they also have an on-premise version. After making an account, during Sentry's 'Onboarding' step, they should ask you to Select a Platform for your application. If you are using the website provided for this tutorial, choose React. Otherwise, choose the one that best fits your needs.

Next, Sentry should display a short guide to installing and connecting the SDK to Sentry in your project. It boils down to this:

Install the package (e.g., npm i @sentry/browser)
Connect the SDK (Sentry.init({dsn:'[your unique url]'}))

You may notice Sentry is 'waiting' for a first event on this 'Onboarding' page. We'll set that up next.

Go ahead and copy that Sentry.init({dsn:'[your unique url]'}) line above the ReactDOM.render method in our example project's src/index.js file. Don't forget to import Sentry import * as Sentry from '@sentry/browser';.

That's it! Sentry will now handle any uncaught errors in our website. Let's test it out.

In our example website, throw a new Error in the src/App.js file in the first line of componentDidMount(). Example:

throw new Error('Hi Sentry!')

Now reload your website locally. You should see the error on the browser page. If you still have the Sentry 'Onboarding' page open notice how that 'waiting for first event' control has updated and our error event was received.

You should even see a button Take me to my event on the Sentry Onboarding page. Go ahead and click that. Take some time to check out the page showing this error. There's lots of great info!

Disabling storing of IP Addresses

Ultimately, this is up to you, but minimizing the amount of data we collect is usually a good idea. Unless you have a specific need to collect IP addresses we can turn this off. Click the 'Projects' item in the sidebar and then click the title of the Project you created for this tutorial. Scrolling down, you should see a Data Privacy section:

Turn the Prevent Storing of IP Addresses option on. Going forwards, IP Addresses will no longer be stored for events. Unless you really need this data, it's a good idea to turn this setting on. If you want to read more about these settings Sentry has documentation for that.

Configuring Email Alerts for new Issues

Sentry is now receiving events from our website, but it would be nice if we were emailed the details of an event the first time it happens. Support for this is built-in to Sentry. To configure it, click the Settings item in the sidebar. In the Organization sidebar you now see, click the Projects item. Now click your Project in the list of Projects. You should be taken to a Project Settings pane.

From here, click the Alerts item in the General Settings section of the Project sidebar. You should see the following:

Click the Edit Rule button in the upper right. In the Edit Alert Rule page look for the Take these actions section. In the Select dropdown, select the Send a notification via {service} option:

You should see a new action added, using Mail as the default, which is what we want in this case. Now click Save Rule.

Let's go back to our website code and modify our Error so Sentry receives it as a new event. Change the Hi Sentry! string to some other text (e.g., Mail time!) and move it to inside render() in src/App.js so Sentry does not think this event can be grouped with the previous one. Now reload the page locally to cause the error to happen again. You should receive an email with a summary of the error!

It's important to note Sentry supports a variety of services to use for alerts rather than just mail. For example, Slack, if you prefer it to email.

Great! We have covered the absolute basics of adding Sentry to our website to automatically report errors. This gives us a powerful tool to see issues on our site without requiring users to log issues for us so we are aware of them. Between, Sentry and GoAccess, we have some very useful infrastructure with regards to logging and error analysis.

Thanks for reading! In the next and final part, we will go over some next steps for you to take.

Part 5 - Deploying a simple web app with monitoring and analytics without Docker or containers for beginners

Jordan Polaniec — Thu, 10 Oct 2019 21:00:16 +0000

In the previous post, we covered how to configure SSL with Nginx and our site. With that post, general configuration of the site is complete. The remainder of the posts in this tutorial focus on logging and analytics. Specifically server-side logging and error logging. Even if your website is designed to just be a small hobby site it is extremely useful to have the ability to look at access logs and error logs.

Viewing Nginx logs

First, we'll focus on viewing server-side access logs. On our Droplet, Nginx is actually already creating these at a default location for us. On the Droplet, if you open up the /etc/nginx/nginx.conf you should see something around line 38 like this:

##
# Logging Settings
##

access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;

Open up the access.log file:

sudo nano /var/log/nginx/access.log

You should see timestamps and details of requests made. To view the default configuration I recommend checking out the ngx http log_module docs. By default the combined log format is used (you can see this in the module docs linked above). You can also create your own custom format if desired, but for this tutorial we'll stick with the default configuration.

The error.log won't get much use in our simple single-page application for our tutorial, but errors encountered on the server will be logged here. It's worth noting there are several severity levels and a default is used so at least some information is gathered by default.

Looking at these logs individually can be useful, but it's also very granular. In many cases, being able to group and organize this information will give us valuable insight into how our site and web server are being used. GoAccess can provide this type of view with minimal overhead. In fact, we can get a very clean view of this data right in the console.

Using GoAccess

GoAccess is an open-source log analyzer and viewer. We'll focus on two features for now:

Analyzing logs in the terminal
Creating real-time HTML report of logs to view in a browser

First, we need to pull down GoAccess to our Droplet. According to the GoAccess docs since we're using Ubuntu we can run:

sudo apt-get install goaccess

Now, let's try to get a real-time view of our logs in the terminal:

sudo goaccess /var/log/nginx/access.log -c --log-format=COMBINED

You should see something like this appear in the terminal:

You can see goaccess has populated a Log, Date and Time format since we specified the COMBINED log format as a command argument. You can hit ENTER to proceed and should now see a terminal dashboard with lots of great information.

Note the Log Source at the top. This is the file we're currently viewing. Based on our Nginx config, this is the active log file. You can load past log files by simply specifying the old log file you want to see on the command line when using goaccess.

Also at the top of the terminal you can see info like Total Requests, Bandwidth, Unique Visitors, etc. If you try hitting the site in your local browser again you should see some of these values change. Scrolling down in the terminal you can see a wealth of detail such as Request Files in order of most popular, 404s, Geo Location and Browser as well as other information. It's wonderful that goaccess offers all this as baked-in functionality that requires such a small amount of setup.

If you would rather have this info rendered in a web browser, goaccess can generate an HTML report of the same data we just saw in the terminal. This can be handy for having a web page that's easily accessible outside of a terminal session. goaccess can generate a real-time report in HTML as well, but since we're using SSL on our site the setup is more involved that I believe is in the scope of this tutorial. If you want to configure that I recommend reading the following:

GoAccess Getting Started page - Check out the 'Real-Time HTML Output' section
GoAccess HTML Report FAQ - Shows more detail on how to configure this with SSL support
This GoAccess GitHub issue - Has additional info on setting up SSL support
Restricting Access with HTTP Basic Authentication on Nginx - If you decide to make this accessible remotely it should probably not be available to everyone.

I find that using the terminal support is enough for my purposes, but it's great that this HTML output is available as an option. As a way to see this we can generate a snapshot of the current log file as an HTML page with the following command:

sudo goaccess access.log -o report.html --log-format=COMBINED

This will print out a report of the current contents of the access.log file. This file can then be transferred over to our local machine using your favorite SFTP tool and opened up to see the data in the browser.

NOTE: Be sure to delete this report html file from your Droplet when done with it just to keep things tidy.

goaccess is a super accessible way to analyze logs from our website. There a multitude of other features goaccess has that we didn't touch! To find out more about it check out the GoAccess Features page. (I'm in no way affiliated with the tool)

Great! We now have the ability to easily analyze basic server logs.

Securely handling your logs

NOTE: Our server logs will be rotated (archived) with the configured rotate value in /etc/logrotate.d/nginx. You can change this to your desired value if necessary. This default log rotation configuration will not automatically clean up your logs until 14 days have been logged. This is set with the rotate 14 line in the etc/logrotate.d/nginx file. With this setting, the oldest log file will be removed to create room for the newest one after 14 have been created (that matching our file name scheme of 'access.log.[number]'

You should monitor these logs for your website and make sure no sensitive information is exposed. GDPR has brought some changes to the how certain data is stored and processed that are worth reading about. I will leave these as an exercise for the reader as I am not a lawyer and the site you are deploying may have unique requirements.

You may have thought we would be working with Google Analytics or other client-side analytics tools like Fathom or Matomo. There are definitely great uses for client-side analytics. This tutorial focuses on the basics and I believe basic stats include metrics such as top requests, geo location, browser usage, server response errors and the ability to view access logs for potential malicious behavior. All of this data can be acquired via server logs. You can, of course, supplement these types of logs on the server-side as well as the client-side.

In part 6 of the tutorial we will see how to add automated error logging to our website with Sentry. Hope to see you there!

Part 4 - Deploying a simple, modern web app with monitoring and analytics without Docker or containers for beginners

Jordan Polaniec — Thu, 10 Oct 2019 21:00:08 +0000

In the previous post, we configured Nginx and deployed our website. We're now live! However, it's a good idea to add support for accessing our website via HTTPS. In this post, we'll configure an SSL certificate for our website and setup automatic renewal of the certificate using Certbot and Let's Encrypt. After that, we'll adjust some more settings in our general Nginx configuration to make things a bit more secure. At the end we'll submit our website to an SSL test at SSL Labs and see how it performs.

Using Let's Encrypt and Certbot

We will need to add the Ubuntu software repository for Certbot.

sudo add-apt-repository ppa:certbot/certbot

After that, we can install it:

sudo apt install python-certbot-nginx

More details about install can be found on the certbot site;

Certbot uses the server_name in all the loaded server blocks in your Nginx configuration files to find the domain you've specified when generating a certificate. This is because Certbot will actually automatically insert the necessary lines into our server block once it finds our domain. To create a certificate for our website:

sudo certbot --nginx -d yourdomain

The domain you use must match the server_name you put in the Nginx conf file we created earlier. When running certbot it might prompt you to enter an email address and agree to Terms of Service. Adding your email address can be handy as it enables certbot to send you emails when your certificate is coming up for renewal. More information can be found on the Let's Encrypt Expiration Emails page.

Finally, you should see certbot ask you if you want to redirect HTTP traffic to HTTPS. We want this feature, so option number 2 can be selected. certbot will go ahead and add the necessary configuration lines to our Nginx conf file for our website.

Upon success, certbot will now show you information about where your certificate and chain exist on our Droplet as well as their expiry date. Our certificate should be loaded in our website configuration so go ahead and hit your website from your local machine again, this time using https://. You should see the same website as before, this time with https! Additionally, you can try hitting your site with http and verify you are redirected to https.

If you open up your Nginx conf file you'll see several lines have been added regarding our choices to redirect http to https, our ssl_certificate entries, a listen directive for port 443 among other entries.

You'll also notice a line include /etc/letsencrypt/options-ssl-nginx.conf. We'll take a look at that in just a moment.

Testing out certificate renewal

It's very important our certificate not expire and certbot has configured automatic renewal for us. We can test renewal out with the following command:

sudo certbot renew --dry-run

This will run the process for renewing a certificate, but not actually renew our current production certificate. If any failures happen during the renewal process they will be listed in the output of this command.

If want to see more information about the automated renewal you can run a command to print out info about it:

sudo systemctl status certbot.timer

Will show details about the certbot.timer service.

sudo systemctl list-timers

Will show information about scheduled services. You should see certbot.timer listed under the UNIT column on the far right of the printed output.

Analyzing our setup via SSL Labs

SSL Labs is a useful site for getting an over-arching grade on the security of your site. Here, you can enter your site url (e.g., https://yourdomain) and it will give you a grade. Running your domain against our site now should produce a A. There are a few small tweaks we can make to our Nginx configuration to get that up to an A+.

First, in our Droplet terminal open up the file /etc/letsencrypt/options-ssl-nginx.conf that I mentioned earlier. You should see the following line around line 10:

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

The TLSv1 section means our Nginx config supports TLSv1. If you take a look at your SSL Labs report you can see several of the yellow-orange lines are due to supporting TLSv1. Let's remove that setting. So this line should like this now:

ssl_protocols TLSv1.1 TLSv1.2;

Next, we're going to add a line right below the ssl_prefer_server_ciphers line. Add the following line:

add_header Strict-Transport-Security "max-age=63072000; includeSubDomains; preload" always;

This enables HSTS which, according to the MDN docs "lets a web site tell browsers that it should only be accessed using HTTPS, instead of using HTTP." More info can be found in the link.

Now that we've adjusted these two settings, save this file.

Test your Nginx configuration to ensure there are not any errors:

sudo nginx -t

Reload Nginx:

sudo systemctl nginx restart

Once reloaded, go back to your SSL Labs report. Scroll to the top of the page and click the Clear cache link. This will cause SSL Labs to restart testing your site fresh. Otherwise it'll return the previous results since we just had our site analyzed a moment ago.

After it's done testing, you should see an A+. Notice the TLSv1 warnings are gone and TLSv1 is now marked No in the Configuration -> Protocols section of the report. Also, near the bottom you can see that Strict Transport Security (HSTS) is green and marked as enabled. Hooray!

There is definitely more you can do regarding securing your site configuration, which I'll touch on at the end of this tutorial.

Our site is now accessible via HTTPS and our SSL certificate is automatically renewed for us!

Next, we'll see how we can access and view server logs for our site on our Droplet using GoAccess in Part 5.

Thanks for reading so far!

Part 0 - Deploying a simple web app with monitoring and analytics without Docker or containers for beginners

Jordan Polaniec — Thu, 10 Oct 2019 20:59:03 +0000

Over the past year or so I find myself learning more and more of frontend development and ops/deployment of web sites. There are so many different ways to deploy a web site nowadays it can be overwhelming.

Many examples assume prior knowledge of DevOps tooling of some sort. Docker and/or containers are examples that I commonly found. Sure, there are services that abstract large portions of this away, but what if I just wanted learn about how a basic website might be deployed? Reading many of these examples, I would need to first read up on Docker and containers before actually getting to learn how to deploy and maintain a website. My website doesn't need to be a globally scaling production monster and doing this without the use of Docker and/or containers is still viable. I found it rewarding to be able to configure the whole infrastructure, not that there's anything wrong with using any of the amazing services that help with this.

In this tutorial I want to focus on taking a very simple web app from a dev machine to a publically accessible domain on a web server. The tutorial will not use Docker or containers directly to serve or ship the website itself, as I want to focus on the basics of setting a web server up. I do understand there are non-performance benefits related to using containers when setting a web server up, but this tutorial is aimed at someone just dipping their feet into setting up a website who also is interested in configuring it themselves instead of using a service such as Zeit Now, Netlify, etc.

Below is what this tutorial will cover. The idea is to break these out into reusable pieces that you can hopefully take with you to other projects that may be more complex.

Part 1

Using a React app that has been created via `create-react-app`

Feel free to sync the repository or bring your own website! You do not have to use this tool to make your web site. The tutorial will focus on running a tiny Single-Page Application, but the website you use in this tutorial does not need to be a Single-Page Application. If you're comfy using your favorite framework or no framework, by all means go right ahead. Just know the process of getting it running on our web server might be slightly different.

Part 2

Creating a \$5 Digital Ocean Droplet

We'll be configuring our web server on a $5/month Digital Ocean Droplet. These are extremely handy and straightforward to configure and I'll go through the process of doing so. We will be using Ubuntu on our Droplet. Required knowledge of Linux is minimal. If you would like, Digital Ocean offers a referral program. Use this referral link to create your account and Digital Ocean will give you get $50 in credit over 30 days. More details on their referral program can be found on their site.

Also, we will configure Digital Ocean Droplet monitoring to enable alerts when our Droplet's resources are over a threshold for a certain period of time. We want to monitor the Droplet's resource availability itself and Digital Ocean offers tooling to do so with minimal configuration necessary. In this section, we'll setup email alerts based on memory and CPU usage of our Droplet.

Part 3

Configuring a domain to point to Digital Ocean nameservers and deploying our website

In order to serve up a memorable website you'll need a domain name. This is necessary to route users to our Droplet when querying for our website on the interwebs.

We will also configure Nginx on our Droplet. Nginx is an popular, open-source application that has many uses. We will be using it as our web server. This will allow us to route users to our site and serve our content when they hit the Droplet as well as support SSL for our site using a generated certificate.

Finally we will push our built website to the Droplet. We need to actually put the files from our dev machine on our Droplet. Here's where we can first test if our site is externally accessible from our domain!

Part 4

Generating an SSL certificate so our website is served over https

Let's Encrypt and Certbot have made it very easy to so in a reliable way. We will also run our site through Qualys SSL Labs analysis tool to verify our configuration.

Part 5

Install GoAccess and view real-time or historical server logs

I use this to browse Nginx logs on my servers. It's lightweight, free, and console-based. A wonderful tool. It's also not limited to viewing just Nginx logs.

Part 6

Setup Sentry for error logging

Logging errors automatically is very important for debugging and feedback purposes. Sentry is an incredible tool with a useful free tier that we can use. We'll go over adding it to our website and generate some test errors.

Part 7

Conclude with a few potential next steps now that your website is up and running. Server hardening, Nginx config tools, GDPR, Nginx amplify, banning IP addresses.

Whew! There's a lot to deploying a website. It's no wonder there are more and more services dedicated to making it easier for developers. However, I hope you'll find that going from local hobby website to deployed hobby website is not that bad. Additionally, you should be able to take this knowledge with you for other projects that may not be limited to just a basic website.

Will your server be a highly-optimized, globally scaling beast after following this tutorial? No. However, you will be able to go quite far with this setup and get a lot of great insight from the various logging and monitoring services we'll be configuring.

Let's get started!

Part 3 - Deploying a simple web app with monitoring and analytics without Docker or containers for beginners

Jordan Polaniec — Thu, 10 Oct 2019 20:47:17 +0000

In the previous post, we configured our Droplet on Digital Ocean. Now that we have our Droplet up and running we need to configure DNS for our domain and install software that will help us serve our website to users under a recognizable name (our domain name).

In this post, we will be:

Configuring DNS for our domain so visitors are routed to our Droplet
Installing Nginx
Adjusting our Firewall to handle website traffic
Creating an Nginx configuration file for our website
Deploying a production build of our website to the Droplet

Configuring DNS

In order to configure DNS for our domain, you will need a domain name. If you already have one to use for this tutorial, great! If not, you can acquire one from your favorite domain registrar. I tend to use domains.google, but have also used namecheap in the past. Digital Ocean is not a domain name registrar, but they do offer a DNS hosting service that we will be using.

NOTE: Keep in mind, there's more than one way to complete this part of setting up a domain. We'll be leveraging Digital Ocean as much as we can in this tutorial, but there are other ways of setting up DNS. Use whichever you prefer.

After acquiring your domain we want to add a Domain to our Digital Ocean account. This will allow us to manage records for the domain within Digital Ocean (e.g., A, AAAA, CNAME records). In the Manage sidebar section on the Digital Ocean page, click Networking.

You should now see a page that allows you to Add a domain. Add the domain you acquired earlier. For example, if the domain you acquired was frannsoft.com you would enter 'frannsoft.com', no quotes. Next, click Add Domain.

Your Domain entry will be created and you should be taken to a page showcasing records for this domain. By default, three NS records will be automatically created, pointing to Digital Ocean's nameservers. You should also see a Create new Record section:

Let's add one more record for now. We will add an A record which will provide Digital Ocean's Nameservers with an IP Address for our domain. In the Hostname input field, enter @. This means your root domain name will be used (e.g., example.com).

Next, click in the Will Direct To input field. In the list of Droplets that appears, select the one created for this tutorial. This is how we specify where the request should go. The TTL should be defaulted to 3600, which is fine. Click the Create Record button and you should see a new A entry in the DNS records section of the page. Your root domain should be listed with the IP address of your Droplet.

Now that we have configured our Domain on Digital Ocean we need to make sure our DNS is correct on the domain registrar for our domain. Digital Ocean has a guide that offers walkthroughs on pointing to Digital Ocean Nameservers from some of the more popular Domain Registrars. If you're using domains.google you simply need to do the following:

Click the DNS sidebar option for your domain in domains.google
Click the Use custom name servers option in the Name servers pane
Enter Digital Ocean's nameservers in the input fields. These can be found in the Digital Ocean networking overview

And DNS configuration is complete. One important thing to keep in mind is that DNS changes can take a while to propogate. This can be a bit confusing as you might have made these changes and moved forward with the rest of your site configuration only to hit DNS resolution issues down the line. You can track DNS change propogation through sites like dnschecker.org (I am not affiliated with them). There are definitely other sites that perform the same checks. I would recommend waiting until your DNS changes have been propogated in your region before moving forward, just to help avoid any issues once we're ready to test our website later on in the tutorial.

That being said, we're done configuring DNS!

Opening ports for web traffic

Next, we'll be installing and configuring Nginx so our site is actually served to users upon request.

Jump back to our Droplet and switch to the user webadmin we created earlier:

su webadmin

To install Nginx we'll run the following commands:

sudo apt update
sudo apt install nginx

This is the first time we're using apt so updating our packages to have the most recent listings is a good idea. The second command will install Nginx on our Droplet. After installing Nginx we need to update our firewalls so that common web traffic ports are allowed. Nginx has a few profiles that are compatible with ufw for doing this. To see the profile run the command:

sudo ufw app list

Four application profiles should be listed:

Nginx Full
Nginx HTTP
Nginx HTTPS
OpenSSH

We have already added the OpenSSH profile. Now we want to add the Nginx Full profile via the command:

sudo ufw allow 'Nginx Full'

This opens ports 80 and 443. To confirm the expected ports were opened you can run the command:

sudo ufw status

You should see the following:

Now we check to see if Nginx is running:

sudo systemctl status nginx

You should see a response with nginx.service and and Active property that is set to active (running).

Another way we can test if Nginx is working is to hit our Droplet's IP address from our browser on our local machine. Example: http://[droplet IP address]. You should see a page stating:

Welcome to nginx!

If so, nginx is now running! Hit CTRL+C to exit the output in our terminal session from our previous command.

We have nginx running, but it has no configuration details about our website yet. We will configure that next.

Nginx configuration for our site

Nginx uses configuration files sometimes called conf files. These files typically house information such as:

The server name of the site. In our case this is the domain name
How to respond to requests for specific file types (e.g., images, javascript files, html files, etc)
How to respond to requests for specific endpoints (e.g., /admin, /home, etc)
SSL certificate information
gzip configuration

...And more. We'll be setting up a few of these in the conf file for our website.

First, navigate to /etc/nginx/sites-available. Create a new file with a name the same as your domain. For example, if the domain you've been working with so far is example.com, the conf file would be named example.com. If you are using a subdomain the file would be named something like mysubdomain.example.com.

Technically, this naming convention is not required by Nginx, but it helps keep our files informative without needing to open them to look through the contents as much. If you're configuring multiple sites on the same Droplet, the naming of these files gets even more important to help keep things clear. You can also split these files up to avoid repeat configuration blocks when multiple sites use the same configuration.

To create the file run:

sudo nano yourdomain

This will open nano with your new file. In the file we want add a few basic items for now (we'll add more later).

A server block is an Nginx term for a group that uses the configured server_name and listen directives to bind to tcp sockets on the machine. Here's what the file contents should look like for now:

server {
    # We will add https support in just a bit. 
    # For now, this will support http.
    listen 80; 

    # Use your domain here. 
    # This is what Nginx will use when attempting to find 
    # the configuration for the incoming user's request.
    server_name yourdomain; 

    # Use your domain here. 
    # This is the root directory Nginx will serve data when responding to requests.
    root /var/www/html/yourdomain; 

    # If the client is request just the root domain 'e.g., example.com',
    # serve the index.html page for the site.
    location / {
        try_files $uri $uri/ /index.html; 
    }

    # We do not want to cache the index.html file for our React Single-Page Application.
    # If we do, when we update the site, users might still 
    # see old info and links of they have a stale cache.
    # More info here - https://www.nginx.com/blog/nginx-caching-guide/
    location /index.html {
        add_header Cache-Control 'no-store, no-cache';
    }

    # gzip configuration. We want to compress our responses.
    gzip on;
    gzip_types text/plain text/css application/json application/javascript;
}

Once you have added that to the file save the contents and exit the file.

Finally, we'll create a symbolic link from this file to the sites-enabled directory. The sites-enabled directory is the directory Nginx uses as a 'live sites' folder which Nginx reads on start. We want to avoid having to duplicate our physical conf file so a symbolic link is useful here. To create the symbolic link run the command:

sudo ln -s /etc/nginx/sites-available/[your conf file] /etc/nginx/sites-enabled/

You can now test that your configuration is correct in Nginx with the command:

sudo nginx -t

This will report any errors with your configuration. If there are no problems, you can now restart Nginx so it will load your changes:

sudo systemctl restart nginx

Creating the physical directory for our site

Now we will create the folder where our website will be physically deployed. This should match the root value we specified in our Nginx conf file above. Change to /var/www/html:

cd /var/www/html

Create a new directory that will contain our website contents using your domain instead of the example one given below:

sudo mkdir yourdomain

Lastly, change the owner of the directory we just created to the webadmin user:

sudo chown webadmin:webdamin your-folder-name

Deploying our website to our Droplet

It's time to push our website to our Droplet! In this section we will be:

Building a production version of our site locally
Pushing it to our Droplet at the root location in our conf file

If you are not using create-react-app these steps might be slightly different for you. The end result is the same, however. Get the production website files to the root location in our conf file for our website on our Droplet.

To create a production build of our create-react-app project we'll run the command:

npm run build

You should now see a build directory in your local project structure. The contents of this folder are what we want to copy to our Droplet.

In order to copy files from our local machine to our Droplet we'll use SFTP. I'm on Windows and will be using WinSCP to do this, but you can accomplish it using any SFTP client you like. Digital Ocean also has a guide to do this entirely within the terminal if that suits your needs better. If you are using WinSCP they have a guide to configuring SSH key usage.

Next, we can copy the contents of the build folder on our local machine to the root folder we just created on our Droplet. Take everything inside the build folder.

Now, if we hit our domain again from our local browser we should see our website!

Our site is now up and running. Let's configure SSL so that the site can be accessed via HTTPS.

We'll do that in part 4.

Part 2 - Deploying a simple web app with monitoring and analytics without Docker or containers for beginners

Jordan Polaniec — Thu, 10 Oct 2019 20:46:06 +0000

In the previous post, we setup our website using this repository. If you want, you should be able to follow these posts using your own website. Just know the setup on our web server might differ slightly if you do so.

Now we want to configure our web server. Here is what we want to have configured by the end of this post:

Digital Ocean \$5 (for a full month's use) Droplet
Nginx
An Nginx configuration file for our website
The domain for our website pointing to Digital Ocean nameservers

Keep in mind This will create a Droplet that will charge your account a maximum of \$5/month. Charges for the Droplet will stop accruing if you Destroy the Droplet. For more info, check the Digital Ocean pricing FAQ.

Also Keep in mind - If you would like, Digital Ocean offers a referral program. Use my this referral link to create your account and Digital Ocean will give you get \$50 in credit over 30 days if you add a valid payment method with your Digital Ocean account (ah yes, the catch). More details on their referral program can be found on their site.

First off, we'll create a \$5 Digital Ocean Droplet. I like using Digital Ocean because UX for maintaining these Droplets is super easy to use, the prices are good for the specs on what you buy and they have all the features I've wanted for my projects so far. It's personal preference, of course. There are other great platforms that offer a similar featureset.

Go ahead and create a Digital Ocean account. Once done and logged into your account you may need to a create a Project:

Add any Name and purpose and then click Create Project. You should be taken to the 'home page' for that Project. A Project is like a grouping of resources in Digital Ocean.

In the upper right corner of the page you should see a green Create button. Go ahead and click that, then select Droplets and you should be taken to a page with a header of Create Droplets. Here is where we'll configure our machine.

In the Choose an Image section we want to choose an Ubuntu 18.04.3 (LTS) x64 image. At the time of writing, this is the latest available version of Ubuntu. Feel free to take a later version if available:

Our plan will be the Standard plan. After selecting the Standard option under the Starter Plan section, click the arrows to move to the left so you can see the $5/mo option:

For the Choose a datacenter region choose the region that you think the majority of your visitors will reside. I usually just default to either New York or San Francisco since the majority of my projects see visitors in the United States. Choosing a subregion number is just fine-tuning your region selection. For our example app it's fine to leave this at the pre-selected option. For me, this is New York and 1.

Next is Additional Options. Check the Monitoring box. This will come into play later, once our website is up and running. Checking this box will add in a monitoring agent to our Droplet, letting it communicate cpu usage, memory usage, metrics collection and few other system stats back to Digital Ocean, allowing us to easily configure alerts on those stats.

Authentication

For the Authentication section, choose SSH and add your Key. Clicking the New Key button will bring up a helpful dialog to walk you through creating a Key if you're less familiar with SSH. After adding your SSH Key content and clicking the Add SSH Key in the dialog on the screen you should see your Key added on the Droplet configuration page we've been working with so far.

It's important to note that by using an SSH key while configuring our Droplet, Digital Ocean will configure our Droplet to not allow logins via password (only by SSH key).

You can see a bit more info on this by hovering of the ? next to the 'Authentication' section header:

Almost done!

Now you'll see a section dedicated to some final steps in our configuration.

How many Droplets? We just need 1

Choose a hostname? This is not our domain hostname or our website name. This is the name of the Droplet itself according to Digital Ocean. The default generated name will consist of some our Droplet's current configuration. If this works for you, great! Otherwise, feel free to name it what you find to be most helpful. Digital Ocean will show you your configuration for each Droplet even if you use a name here that doesn't contain that info.

Add tags: Here you can add tags to help categorize and/or organize your Droplet. We'll add nginx and web server, but feel free to add your own.

Select Project: We'll just assign our Droplet to the Project we created earlier.

Here's an example of this portion of the Droplet configuration:

Add backups: Since this is just an example Droplet we won't enable any backups.

Keep in mind - This will create a Droplet that will charge your account a maximum of \$5/month. Charges for the Droplet will stop accruing if you Destroy the Droplet. For more info, check the Digital Ocean pricing FAQ.

Now you can click the green Create Droplet button and Digital Ocean will get to work! Navigating to the Project we assigned our Droplet to should show our new Droplet in the list of Droplets for the Project. If you click on the Droplet you'll be taken to details about that Droplet including taking manual backups, adjusting configuration details and/or destroying the Droplet.

Let's SSH into the terminal. If you're newer to SSH, Digital Ocean provides a nice tutorial on How to Connect to your Droplet with OpenSSH. I'm on Windows and tend to use PuTTY. Digital Ocean provides a guide on How to Connect to your Droplet with PuTTY on Windows.

After connecting, you can login as root. We'll now create a new user and perform the rest of our tasks on the Droplet using that user.

Creating a User

This can be done with the following command: adduser [your new username]. Ours will be adduser webadmin. You can populate the Full Name, Room Number info you want, but we'll just leave it blank.

Next, we want to grant administrative privileges to this user so they can perform admin tasks when necessary without us needing to switch back and forth to root.

This can be done with the following command: usermod -aG sudo webadmin. Now, we can perform actions with superuser privileges when we enter the password for this user and prefix our commands with sudo.

Switch to the user:

su webadmin

Let's create an SSH Key for this user. This will allow us to SSH in as this user so we do not have to login as root. We can then disable root access even via SSH.

For creating and configuring the SSH Key for our webadmin user I recommend following Step 4 (using 'Option 2') of this Digital Ocean tutorial. If you're on Windows like me and using PuTTY to login to your Droplet they have a guide to using their SSH key generator.

Disabling root access

We no longer need to login as root to be able to get into our Droplet so we can disable the ability to login as root as a security measure.

NOTE: Doing this will disable root access. If you have misconfigured your access for the webadmin user we created above you might lock yourself out of your Droplet. It's a good idea to test that you can SSH into the Droplet with your webadmin user prior to doing this.

To do this, we will modify the sshd_config file to not allow root login:

sudo nano /etc/ssh/sshd_config

Will open this file. Around line 32 there should be:

PermitRootLogin yes

Change this line to:

PermitRootLogin no

Lastly, we need to restart the ssh service in order for our changes to take effect:

sudo service ssh restart

Great, our server is a bit more secure now with regards to access.

Firewall configuration

Now let's setup basic Firewall rules. On our Droplet, we will be using ufw and we want to allow SSH connections.

First, enable ufw with the command sudo ufw enable. You should receive a response stating Firewall is active and enabled on system startup.
Second, add the OpenSSH profile. This will add common OpenSSH settings to our Firewall and make it less tedious for us to set these up ourselves. This is done using the command sudo ufw allow OpenSSH. You should see a response of:

Rule added
Rule added (v6)

Now enter the command sudo ufw status. You should see the following:

If you do, great! Basic configuration of our server is complete!

Configuring Email Alerts for Droplet Usage

When we checked the Monitoring box earlier, our Droplet was created with the Digital Ocean monitoring agent already installed and configured. It's important to be notified when your Droplet might soon run out of resources or is running under heavy usage for unexpectedly long amounts of time. Digital Ocean allows us to create Alert Policies and integrate those alert notifications with email or Slack In this tutorial we will choose to use Email.

On the Digital Ocean page that lists your Droplets, click on the Droplet you are using for this tutorial and in the center you should see blue text stating Create Alert Policy. Go ahead and click that and you will be taken to a page like this:

Here is where we can configure the specific metrics we want to watch and the threshold we consider worthy of an alert. For now, select Memory Utilization, is above, 70 (percent) and 5 min. This means we will get an alert if the Memory utilization remains above 70% for 5 consecutive minutes.

In the next section, select the Droplet you are using for this tutorial.

Then, in the Send alerts via section, select the checkbox that displays your email address. This is the email address Digital Ocean will send alerts to when necessary.

Finally, you can name your alert policy if you would like. After that, click the Create alert policy button. Done! Alerts are now setup. Feel free to add others if you would like.

Next, we will be installing and configuring tooling more specific to serving up our website.

Onward to Part 3!

Part 1 - Deploying a simple web app with monitoring and analytics without Docker or containers for beginners

Jordan Polaniec — Thu, 10 Oct 2019 20:45:44 +0000

Our website

We will be deploying a simple website that shows a collection of Pokemon from the PokeAPI when the user lands on the home page. Clicking on a specific Pokemon will take the user to a 'card' that shows information about that Pokemon.

I've pushed the project to GitHub so you can clone the website if you would like.

If you are using the website I've created for this tutorial you can run it locally via the following commands:

npm i to install packages locally

Once all the packages are restored you can then run the site locally:

npm start

Once started, the site will be accessible on port 3000. You should see something like this:

Is it the best looking thing? Nah, but it works for our tutorial!

In the next part we'll focus on setting up our web server with Digital Ocean.

Parallel test execution within the context of MSTestv2, NUnit3 and VSTest.Console

Jordan Polaniec — Mon, 15 Oct 2018 18:12:31 +0000

Keeping automated tests within a reasonable performance range is no trivial task and I'm always looking for ways to reduce the total duration of automated tests. Recently, I did some reading on the capabilities of a few different tools with regards to parallel execution of tests. These tools included MSTestv2 and NUnit3 when run with vstest.console.

While building a test case to compare the two I built two projects. One using MSTestv2 (v1.3.2) [DataTestMethod]s, designed to take advantage of DynamicData and another using NUnit3 (v3.10.1). Below is my MSTestv2 test.

using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections.Generic;
using System.Threading;

[assembly: Parallelize(Workers = 0, Scope = ExecutionScope.MethodLevel)] //0 means use as many workers as possible

namespace mstest_parallel
{
    [TestClass]
    public class UnitTest1
    {
        private static IEnumerable<object[]> MyTestData()
        {
            for (int i = 0; i < 10; i++)
            {
                yield return new object[] { i * 1000 };
            }
        }

        [DataTestMethod]
        [DynamicData(nameof(MyTestData), DynamicDataSourceType.Method)]
        public void TestMethod1(int testVal)
        {
            Thread.Sleep(testVal);
        }
    }
}

Now, MSTestv2 has the ability to parallelize tests to some degree. Here are the parallelization options it offers at a high level:

Class level - each thread executes a [TestClass] worth of tests. Within the [TestClass], the test methods execute serially
Method level - each thread executes a [TestMethod]
Custom - users can provide a plugin implementing the required execution semantics (Not yet supported).

These capabilities can be referred to as fine-grained parallelization features and the number of worker threads can be configured via assembly attribute [assembly: Parallelize(Workers = n, Scope = Execution.ClassLevel)] or a .runsettings file. More details here

Keep in mind these are different than MSTest v1's parallel options and also different from vstest.console's parallel options.

Vstest.console's parallel options focus on what its authors refer to as coarse-grained parallelization. This means one can parallelize the running of multiple test containers at once. So if you are running vstest.console from the command line and specify several test containers i.e., mytestlib1.dll mytestlib2.dll vstest.console will run each test container simultaneously up to the maximum possible on the machine or the maximum configured.

Why do we care? I want to squeeze as much performance out of the chosen test framework and runner as possible - ideally without having to write any threading code myself. Unfortunately, MSTestv2 (and v1) lacks a very specific and important parallelization option; The ability to run tests in parallel which are configured with [DynamicData] and/or [DataRow]. Evidence for this not being supported exists here, here and here. For our situation, this means our test method that utilizes [DynamicData] to generate unique cases will have all of the test cases per [TestMethod] run in serial rather than in parallel, thus making the majority of our parallelization efforts moot.

With parallelization turned on at the method level for the assembly our proof of concept MSTestv2 project showed the following results when using vstest.console to execute:

Total tests: 11. Passed: 11. Failed: 0. Skipped: 0.

Test Run Successful.

Test execution time: 47.4547 Seconds

Not that great. The majority of these tests are being run one by one.

NOTE: vstest apparently reports the 'base' test method as a test case when running MSTestv2 tests which is why the test count is 11 instead of 10. My understanding is that this does not actually run, but contains the overall results for all the tests associated with this [TestMethod].

Let's check out NUnit3. This framework offers the feature we want, which is running parameterized tests that have their cases generated at runtime in parallel.

The fundamental difference between the frameworks here is that MSTestv2 considers test cases via [DynamicData] and [DataRow] to all be under a single [TestMethod] whereas NUnit3 considers each test case its own separate [Test].

Here's our test method adjusted to be used with NUnit3:

using NUnit.Framework;
using System.Collections.Generic;
using System.Threading;

namespace nunit_parallel
{
    [TestFixture]
    [Parallelizable(ParallelScope.Children)] //parameterized tests are considered child tests
    public class UnitTest1
    {
        private static IEnumerable<object[]> MyTestData()
        {
            for (int i = 0; i < 10; i++)
            {
                yield return new object[] { i * 1000 };
            }
        }

        [Test]
        [TestCaseSource(nameof(MyTestData))]
        public void TestMethod1(int testVal)
        {
            Thread.Sleep(testVal);
        }
    }
}

The attribute specified at the top of the class [Parallelizable(ParallelScope.Children)] tells NUnit to parallelize every test method in the class and it considers each of our generated test cases a unique test method. Additionally, I am letting NUnit decide my [LevelOfParallism]. By default, NUnit3 uses the processor count or 2, whichever is greater.

Now, running with the same tests using vstest.console and same number of tests cases per [Test]:

Total tests: 10. Passed: 10. Failed: 0. Skipped: 0.

Test Run Successful.

Test execution time: 17.5729 Seconds

Predictably, the time to run the suite of tests is drastically shorter.

One important part of this setup is how these tests are run. Vstest.console handles running tests from each test framework for us as long as we can specify the necessary test adapter. For both NUnit3 and MSTestv2, the adapters are available as NuGet packages. Adding these NuGet packages to your test project will cause the necessary libraries to be copied out upon build and vstest.console will find them for you so long as they are in the same directory as the test containers you specify.

I hope this helps explain some of the options regarding parallelizing tests, especially parameterized ones. I learned a ton while studying this stuff and am I sure this will continue to be the case while writing these types of test suites. If you believe anything I have written above needs to be corrected please reach out to me.

Experimenting with non-tabular ways of showing test results

Jordan Polaniec — Sat, 28 Oct 2017 01:09:29 +0000

Ever heard of FitNesse? If not, imagine what the father of SpecFlow would look like and how it might be used.

If you imagined HTML tables filled with wiki-style syntax to perform assertions on the data placed in them you're on the right track.

A small example from the FitNesse Two Minute Example page:

And the backing markup that creates the table:

This seems relatively simple for straightforward tests. While maybe not an ultra-modern way of writing tests, enabling testers who aren't necessarily able to write code with the ability to write automated tests is a good thing.

These tests can be executed in the browser and are backed by code in a similar fashion to SpecFlow. Users don't need to install anything at all in order to get started; only go to a url.

Recently, I came across a scenario where a small suite of these tables were being used to verify data coming from an external service. These tests were necessary as we need to make sure data arrives in the expected format and expected value.

The problem was some of these FitNesse tables had over 90 columns. This reduces the readability of the tests to almost zero since the only way someone reading the results could find a specific failure is manually searching the results or CTRL+F on the results page. Any web page with a 90+ column table isn't navigable using this format.

Be that as it may, these types of tests have been around for a while and simply porting them to something other than FitNesse wasn't really an option. This poses a problem as these types of tests hold a key role in ensuring the provider services apps integrate with are working properly. The search was on for a better way to visualize these results.

D3.js

Enter d3.js. I encourage you to check out the link, but essentially d3.js specializes in rendering complex graphs of data in html (often as SVGs). After doing some research on non-tabular ways to present tabular data I chose a Sunburst diagram.

Visualizing these large FitNesse results tables as a zoomable sunburst diagram allows readers to view all tables at once without needing to scroll. The sunburst diagram also offers a high-level view of the results ("Did any tests fail?") as well as the ability to zoom in to a specific table for more in-depth analysis. Placing additional details for results at specific layers on the diagram eases readers into the mass amount of info presented with these results.

Ideally, the highest level uses a unique color for each table and red/green colors for each database column to show fail/pass, respectively. Hovering over a 'table' section section reveals database table-specific info and hovering over a specific section for a single test shows fail/pass details, the database table and column under test.

Here's an example of the generated diagram:

First, the FitNesse test results page is scraped for these database column test results and proper JSON structure is built. My technique was nothing magical - just some Regex and selectors to get at the necessary HTML elements with the test result data I needed. According to this example by Mike Bostock the necessary JSON for a sunburst diagram in d3.js would look like this:

{
    "name": "name",
    "children": [
        "name": "name",
        "children": [
            "name": "name",
            "children": [
                ...and so on
            ]
        ]
    ]
}

Each 'layer' inside the children property represents another layer in the sunburst diagram. The first layer is the root and contains overarching information about the system under test. The second layer of children nodes each contain a database table with information about that table. The third layer of children contains database column information specific to the parent table and test result information. Referring to the image shown above, one can see how this JSON structure is rendered.

(It's worth mentioning that JSON is not the only format d3.js accepts, I just found it to be the easiest for this experiment.)

The last step is generating the sunburst diagram using the scraped and parsed FitNesse results data. This ended up being very similar to the previously mentioned example sunburst diagram by Mike Bostock. Here's an simplified example of what that may look like:

svg.selectAll("path") //get all the path elements in the SVG.  At thsi point it's empty so there will only be one.  https://github.com/d3/d3-selection
    .data(nodes) //nodes is the parsed FitNesse results JSON data.  Give the data to d3 and begin the process of generating the SVG.
    .enter().append("path") //iterates nodes, creating a new path element for each.  https://github.com/d3/d3-selection#selection_enter
    .on("click", click) //wire events for each create path element that contains a node.  Click will cause the diagram to zoom in on the clicked node
    .on("mouseover", mouseover) //show info about the cell in question
    .on("mouseleave", mouseleave) //clear shown info
    .attr("d", arc) //determine the measurements of the generated path for this node
    .style("fill", function(d){
        if(d.data.passed != null){ //set the color of the arc based on whether or not the node is a test and passed/fail or the node is another non-test piece of data like a db table
            if(d.data.passed = true){
                return testPassedColor;
            } else{
                return testFailedColor;
            }
        }
        if(d.data.name === 'root'){
            return rootColor;
        }
        return colors((d.children ? d : d.parent).data.name); //ensure unique colors are used for tables
    })

Here's a link to a gif of how it all comes together.

The dashboard pieces shown in the gif like the breadcrumb trail and combobox are designed to additionally ease the process of finding the info readers need. As a result, readers can delve directly into the results instead of having to scroll through them. The comboboxes used are from select2.js and contain autocomplete functionality so users can start typing the name of a database table or column they would like to see and filter results as needed.

Results

All of this is aimed towards making test results easier to digest for consumers. Ultimately, readability is one of the most important features of test suites. If users have a hard time analyzing test results they will start to ignore them as it becomes too tedious to work through them over time.

Expansion

FitNesse uses a fairly generic structure for test tables, so resuing the code to create JSON from results on other test results would theoretically be straightforward.

By way of a disclaimer, I am in no way a FitNesse expert. In the event of a more efficient, alternative method existing that would allow us to obtain the raw data from the FitNesse wiki test results tables as JSON then that should be used to eliminate a step in the process. I did come across a url parameter that can be used to return the test results as XML, but it appeared to return the test results inside a 'table' of sorts rather than just containing the raw data.

It's also worth noting that while this proof of concept code is geared towards creating a sunburst diagram, any type of visualization is possible. If another type of diagram is desired, I highly recommend taking a look at the d3.js gallery to get ideas.

MSTest v2 - New (Old) Kid on the Block

Jordan Polaniec — Tue, 22 Aug 2017 02:06:26 +0000

It's been a long time since MSTest was in discussions of modern testing frameworks. However, here we are in 2017 and MSTest is getting active updates once again, due in part to it being open-sourced. Let's take a look at what some of these new and exciting features are.

Please note, this post will cover features that are currently in pre-release for the MSTest v2 NuGet package.

NOTE: Remember to use the pre-release flag with the NuGet package you will be using:

Visual Studio 2015+ is also recommended; your mileage may vary with 2013.

This beta release is NOT the first public iteration of MSTest v2, but contains some of the features discussed below. Having stronger support around parameterized test cases is very important and those features are in the 1.2.0 beta; version 1.1.13 was the first v2 release from github.

Release notes for each version can be found here.

On a post about the future of MSTest v2 and mstest.exe, user (and presumed employee of Microsoft) 'Abhitej_MSFT' clarifies that:

MSTest V2 tests are not supported with “mstest.exe”. In the TFS build template the Test Runner should be “Visual Studio Test Runner”(https://msdn.microsoft.com/en-us/library/ms253138(v=vs.110).aspx#Runner) . I hope your definition does not require the legacy testsettings. Do let us know on aajohn@microsoft.com if you hit any issues.

Source - https://blogs.msdn.microsoft.com/devops/2017/02/25/mstest-v2-now-and-ahead/#comment-78796

I have yet to come across an MSTest v2 feature that NUnit has not already had for quite some time. However, the features discussed here are still very useful if you decide to use MSTest over another test framework.

For those who want to see it all, the repository for MSTest can be found on github.

Features

DynamicData #141

Parameterized testing has long been available in MSTest, using the DataRowAttribute which look like this:

[TestMethod]
[DataRow (1, 2, 3)]
[DataRow (4, 5, 6)]
public void MyParameterizedTest (int a, int b, int c) {
    //perform assertions on a,b and c
}

The resulting two tests each use one instance of the 'rows' of data. However, what if we want to reuse these same values across tests? As it would be inefficient to have to duplicate these values on every test where they are used, v2 offers DynamicDataAttribute:

private static IEnumerable<object[]> ReusableTestData =>
    new List<object[]> {
        new object[] { 1, 2, 3 },
        new object[] { 4, 5, 6 }
    };

[TestMethod]
[DynamicData (nameof (ReusableTestData))]
public void MyParameterizedTest (int a, int b, int c) {
    //perform the same assertions the same as before.
}

Now this test data can be used on any number of tests without having to duplicate the actual data, making the process cleaner. Those of you who are familiar with NUnit may know this as the TestCaseSourceAttribute.

Methods can also be used as test data. To do so, simply use the overload of the DynamicDataAttribute constructor that takes in a DynamicDataSourceType.Method enum. By default, the framework will assume the name of the dynamic data passed in is a Property.

Custom Test Data for Parameterized Tests #141

Taking DynamicData one step further is useful if the parameters used in your tests are a bit more complex. Using the CustomTestDataSourceAttribute you can now create an attribute that loads up this data for you to consume in your tests while keeping your test nice and clean. Here is the end result:

[TestMethod]
[CarTestData]
public void ATestUsingACar (string model, int year) {
    var carUnderTest = new Car { Model = model, Year = year };
    //perform assertion on the car object.
}

//config for the 'CarTestData' attribute:
class CarTestDataAttribute : Attribute, ITestDataSource {
    public IEnumerable<object[]> GetData (MethodInfo methodInfo) {
        return new List<object[]> {
            new object[] { "Ford", 1990 },
            new object[] { "Nisson", 2017 }
        };
    }

    public string GetDisplayName (MethodInfo methodInfo, object[] data) {
        if (data != null) {
            return string.Format (CultureInfo.CurrentCulture, "{0} ({1})", methodInfo.Name, string.Join (",", data));
        }
        return null;
    }
}

public class Car {
    public string Model { get; set; }
    public int Year { get; set; }
}

Creating a new Attribute that implements the MSTest v2 ITestDataSource interface allows us to set up our test objects outside of the test, provide a variable amount of them for parameterized tests, and reuse them in other tests without duplication.

This can be pretty powerful in some scenarios, however I am not entirely in love with ITestDataSource.GetData() returning an IEnumerable<object[]> instead of returning an IEnumerable<object>, IEnumerable<T> or offering some other interface. This makes it somewhat cumbersome to use this mechanism to return non-primitives.

ITestDataSource.GetDisplayName will be displayed when the test is executed via the target runner (VSTest if running in Visual Studio), allowing us to determine which cases of a test failed when tests are parameterized.

Assert.That #116

With the introduction of a focused extension point for assertion logic, MSTest v2 now offers That, a static property on Assert which returns the instance and allows for an easy jumping point for extension assertion methods.

Here's an example:

public static class MyAssertExtensions {
    public static void CountIsGreaterThan<T> (this Assert assert, IEnumerable<T> objs, int num) {
        int actualCount = objs.Count ();
        if (actualCount > num) {
            return;
        }
        throw new AssertFailedException ($"Expected {nameof(objs)} count to be greater than {num}, but was {actualCount}.");
    }
}

This means tests are much more readable in their assertions:

[TestClass]
public class MyTests {
    [TestMethod]
    public void ATest () {
        var aList = new List<int> { 1, 2, 3, 4 };
        Assert.That.CountIsGreaterThan (aList, 0);
    }
}

Assert.That can be used to replace and/or supplement calls to Assert.AreEqual() and Assert.IsTrue(). By themselves, these are such broad calls that often times they can lead to confusion when debugging or reading tests. In my experience this is especially true with Assert.IsTrue() due to its default failure output of Assert.IsTrue failed. AreEquals() attempts to mitigate any confusion by reporting the actual and expected values.

NOTE: If you are not going the route of extension methods a quick way to help solve this problem is to pass the name of the property/method under test when passing a failure message. An example:

public void MyTest () {
    var aList = new List<int> { 1, 2, 3 };

    //nameof() is a C#6 feature, but is not required to make this work.
    Assert.IsTrue (aList.Count > 3, $"{nameof(aList.Count)}");

    //failure prints: Assert.IsTrue failed. Count
}

And an example of enhancing Assert.IsTrue with this type of approach:

public static class MyAssertExtensions {
    public static void IsTrue<T> (this Assert assert, T instance, Expression<Func<T, bool>> assertionExpression) {
        if (assertionExpression.Compile ().Invoke (instance)) { return; }
        throw new AssertFailedException ($"Assertion failed for expression {assertionExpression}");
    }
}

//Example
public void IsTrue_Extended_Test () {
    var aList = new List<int> { 1, 2, 3, 4 };
    Assert.That.IsTrue (aList, list => list.Count > 4);

    //failure prints: Assertion failed for expression 'a => (a.Count > 4)'.
}

Taking this a bit further we can create a fluent syntax that lends itself to more extensibility in the future:

[TestMethod]
public void IsGreaterThan_Redux () {
    var aList = new List<int> { 1, 2, 3, 4 };
    Assert.That.For(aList).IsTrue (list => list.Count > 4);

    //failure output: Assertion failed for expression 'list => list.Count > 4'
}

public static class MyAssertionExtensions {
    public static For<T> For<T> (this Assert assert, T instance) {
        return new For<T> (instance);
    }
}

public class For<T> {
    private readonly T _instanceUnderTest;

    public For (T instanceUnderTest) {
        _instanceUnderTest = instanceUnderTest;
    }

    public void IsTrue (Expression<Func<T, bool>> assertionExpression) {
        if (assertionExpression.Compile ().Invoke (_instanceUnderTest)) { return; }
        throw new AssertFailedException ($"Assertion failed for expression '{assertionExpression}'.");
    }
}

Clearly, creating extension methods with Assert.That heavily mitigates the readability problem. For an additional point of reference, see the original RFC.

While this is not a comprehensive overview of the new features in the pipeline for MSTest v2, I have covered those that strike me as being particularly useful.

For more information on MSTest v2 check out these links:

MSTest v2 is definitely making MSTest a more relevant testing framework. The concepts here are the aspects I was most interested in as a current user of NUnit.
While I myself have not yet made the switch from NUnit to MSTest v2, these new features have made considering the possibility much more favorable. Hopefully these features combined with more features coming down the pipeline in the future continue to improve developers view of MSTest when compared to other testing frameworks.