Forem: Ken Simeon

20+ years of advice to testers & QAs

Ken Simeon — Mon, 07 Aug 2023 00:20:04 +0000

From my original contribution to Starting Your Software Testing Career by @nicolalindgren

Don't fall into the trap of being in a perpetual cycle of wanting to do or wanting to be something, Instead be in a perpetual cycle of learning something new and applying it [to your career or life].

Throughout my 20+ year career as a QA Professional & hiring manager, I've heard countless times that a candidate wants to learn automation or be an automation engineer.

I immediately ask, what are they doing to get there? And how long have they been on this journey?

The majority of responses have been something like;

I've been learning about [fill in a popular automation technology] cause that is what’s used at work.
I’ve been learning it on & off for the past few years cause I don't get enough time at work.

As a result, hiring managers like myself, could hastily jump to one of the following conclusions:
1) This person really isn’t dedicated to what to their goal.
2) This person doesn't have the aptitude to do automation.
3) This person is not a fast learner.

Being in a perpetual cycle of wanting & not doing will leave any professional in a decelerated career path.

The approach I’ve used to aid those that want to learn automated testing and be successful, is that they need to learn the foundations of the language used in their automation framework of choice and beyond the basics of the technology they want to automate.

For instance, you’re a QA professional wanting to write automated tests for a web application using a JavaScript framework and Selenium.

I suggest the following:

Learn HTML & CSS to the point you can create a working stylized web site.
Learn the advanced foundations of the JavasScript language and apply them to your website.
Learn about the framework & the underlying technology that enabled the automation (i.e. Selenium Webdriver, WebdriverIO, PlayWrite, etc.).
Learn about the fundamentals of unit testing.
Put all the pieces together behind a solid goal of automating a single functional area of a feature before expanding beyond that scope.

When QA professionals hear this suggestion, they can be discouraged by the magnitude of knowledge and the time it could take to consume & apply the knowledge.

But that is the point.

Advancing one's knowledge and career takes time & effort. None of it is just handed to a person. You need to make the time to up-skill.

There are those that will try to find a shortcut. That is ok as long as they are continuing to learn & applying the knowledge they’ve gained.

Now one might ask; How does the breakdown of these various areas of learning apply to both a QA professional’s current career and their goal of becoming an automation engineer?

Learning HTML & CSS gives the individual a foundation of how a website is put together. It will aid in the conversations with developers, UX designers and product managers around the customer experience. It will also help with the explanation of issues with a web experience to a developer and provide the foundational information that accelerates a potential fix.

Learning JavaScript for a web app helps a QA professional understand how interactions can happen on a web page. They will also learn how to troubleshoot and debug issues. This directly applies back to their current role by allowing them to give more detailed information to a developer when an issue is experienced.

Learning beyond the foundations of a programming language is key. Without this, one won’t know or understand how to structure their a clean code base. They won't fully understand why automation frameworks work or behave in a certain way.

Beyond that, it will give a person an overall foundation behind programming in general. This aids in your conversations in your profession. Now your conversations with developers & technical leaders will shift.

You’ll begin understanding how all the inner parts of the companies code bases works together and be able to ask technical questions that could drive deeper conversations & understanding for a better site reliability or user experience.

The key to learning about Unit Testing, is that not all automated tests from a QA organization should be End-to-End test scenarios. End-to-End tests do have their place within an automated test suite, but that’s not the end all be all.

Finally when you put all of the learnings together, the QA professional can leverage this knowledge in their daily job.

By doing this, it will lead to accelerated professional progression within your current company or the next company you take your learnings to.

Web UI Automation Must Haves

Ken Simeon — Wed, 01 Apr 2020 06:24:08 +0000

There has alway been a need to have a better feedback loop a the desktop application or a web application under test. Many times we've had to ask [occasionally beg] a developer to build in some needed feedback functionality. But now that isn't so much the case when it comes to testing web applications. With the accessibility of Webdriver there are easier opportunities to get direct feedback from the web browser itself.

This direct ability to interact gives those writing automated tests the ability to answer some of the some of the most troublesome and nagging questions we all have needed the answer to.

Has the page fully loaded?
Have any JavaScript errors been thrown?

FYI: The examples below are from my WebdriverIO library, but the concept can be adapted to any variant of automated testing using Selenium or another automation framework.

1. Has The Page Fully Loaded?

There are many ways to resolve trouble maker #1. Many time's I've seen and done it myself, write my automation to wait for a particular element in the page to load or exist. This can be useful, but it does not scale.

The first part of the solution is to have the browser tell us its current document loading status. The is accomplished by hooking into the Document.readyState property of the browser. The Document.readyState property returns one of three responses.

loading
interactive
complete

This functionality of the browser document can be queried while the page is loading like the example below that returning TRUE if the document.readyState equals 'complete'. To access to the document property of the browser you have to use WebdriverIO/Selenium's ability to directly execute a JavaScript call.

export default class utilsLib {
    constructor() {}

    checkIfPageLoaded() {
        // console.log( `Done loading? ${browser.executeScript('return document.readyState', []) }` );
        return browser.executeScript('return document.readyState', []) === 'complete';
    }
...

Now that we have the ability to query the document.readyState, let's write a custom wait function called waitForPageToLoad to check if the browser has completed loading. But making sure to timeout if the waiting time is excessive.

...

    waitForPageToLoad() {
        browser.waitUntil(() => {
            // console.log('Waiting for the page to load');
            return this.checkIfPageLoaded() == true;
        }, 15000, 'Page never completed loading');
    }

...

2. Have any JavaScript errors been thrown?

After checking that a page has loaded, we now need to face the silent killer [no I'm not talking about heart disease]. The silent killers to any web application are JavaScript errors. There are times when a JavaScript error is thrown but it doesn't always cause a web application to fail a running test. A JavaScript error could cause a failure on functionality that isn't being tested at the moment or has automated tests for it. In any case, its good practice to always check for JavaScript errors after a web page is loaded.

To access the logging of JavaScript errors in the browser, you have to leverage WebdriverIO/Selenium's access to the JsonWireProtocol for Logging. There are five Log Types that can be accessed, but we are focused on browser so we can capture & parse the logs for potential JavaScript errors.

Log Type	Description
client	Logs from the client.
driver	Logs from the webdriver.
browser	Logs from the browser.
server	Logs from the server.

Along with Log Types, there are also Log Levels to consider [which any good API would provide]. To get down to the nitty gritty we are going to parse for SEVERE errors and just console log any other errors so they can be generally captured.

Log Level	Description
ALL	All log messages. Used for fetching of logs and configuration of logging.
DEBUG	Messages for debugging.
INFO	Messages with user information.
WARNING	Messages corresponding to non-critical problems.
SEVERE	Messages corresponding to critical errors.
OFF	No log messages. Used for configuration of logging.

As you review the checkForJavaScriptErrors function below, you'll see we're leveraging waitForPageToLoad to make sure the page is loaded before checking for JavaScript Errors. This allows for chaining of multiple levels of validation even before an assertion point is even executed.

...
    checkForJavaScriptErrors() {
        this.waitForPageToLoad();
        var logs = browser.getLogs('browser');
        logs.forEach(function (log) {
            if (log.level.toLowerCase() == 'severe') {
                if (log.source.toLowerCase() == 'javascript') {
                    console.error(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                    expect.fail(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                }
                else {
                    console.log(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                }
            }
        });
    }


...

Leveraging the functions

As an example of how I've used by three super helper functions, I created a custom openUrl function as part of my class that will wait for the page to load and check for JavaScript errors.

...

    openUrl(path) {
        browser.url(path);
        this.checkForJavaScriptErrors();
    }
}

If you have any questions about WebdriverIO or Selenium based testing, please don't hesitate to leave a comment or message me directly.

Happy Testing!!!

Full Example Source

export default class utilsLib {
    constructor() {}

    checkIfPageLoaded() {
        // console.log( `Done loading? ${browser.executeScript('return document.readyState', []) }` );
        return browser.executeScript('return document.readyState', []) === 'complete';
    }

    waitForPageToLoad() {
        browser.waitUntil(() => {
            // console.log('Waiting for the page to load');
            return this.checkIfPageLoaded() == true;
        }, 15000, 'Page never completed loading');
    }

    checkForJavaScriptErrors() {
        this.waitForPageToLoad();
        var logs = browser.getLogs('browser');
        logs.forEach(function (log) {
            if (log.level.toLowerCase() == 'severe') {
                if (log.source.toLowerCase() == 'javascript') {
                    console.error(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                    expect.fail(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                }
                else {
                    console.log(`${log.source.toUpperCase()} ERROR: ${log.message}`);
                }
            }
        });
    }

    openUrl(path) {
        browser.url(path);
        this.checkForJavaScriptErrors();
    }
}

Three keys to being successful as a QA/QE/SDET

Ken Simeon — Tue, 10 Mar 2020 05:09:52 +0000

1) Always think about how the small pieces interact with the larger whole
2) Be curious about how & why a developer is implementing a change
3) Understand what's important to the business' success

Bonus via Arlene Andrews
4) Be able to accept that things will change (be it tools, code, and/or requirements).

Do You Use Your Software Automation Beyond Testing?

Ken Simeon — Thu, 16 Jan 2020 10:49:56 +0000

Tonight as I sit here watching my terminal retort data structures while creating digital entities, I wonder what other people use their automated test frameworks for beyond testing?

Cooking Up NAS Pis

Ken Simeon — Wed, 08 Jan 2020 17:07:11 +0000

For the past few weeks I've been doing a lot of research into Raspberry Pi SATA hard drive support. This particular internet combing event was all started with the Rock Pi Dual/Quad SATA HAT announcement back in Dec. Lets just say its been a fruitful adventure. 😜

Disclaimer: I have not been paid or received any compensation for the collection of words & thoughts about the products mentioned below.

Finding the best recipe

I've always liked the idea of using a Raspberry Pi (RPi) as a storage management system. But I've never liked the idea of a bunch of USB enclosures hanging off a tiny RPi. It's always been a messy solution to me, especially when wanting to drive multiple 3.5" hard drives. Which would require a power source for each HD enclosure. I did look into attaching a multi disk JBOD to a RPi, but that still adds an additional power source & another device to maintain. I've considered buying an off the shelf NAS like a QNAP or Synology instead of build out a RPi solution, but I've always felt they were over priced and not power efficient enough. Not to mention these companies have had a hard time staying compliant with the licenses of all the open source software they leverage.

The biggest draw for me with the RockPi QUAD SATA HAT solution is that it would allow me to have a single power source to drive my entire RPi NAS concoction for either 3.5" or 2.5" hard drives. Not to mention that this RPi SATA hat solution supports true hard drive power management from the OS, unlike a basic USB enclosure. The ability to spin down a hard disk array is critical to the longevity of a spinning disk hard drive & the reduction of power consumption of the entire system that isn't accessed 24/7.

Another big benefit for me would be that I'd then be able to decommission my aged DIY Franken-NAS that I feel like I've duct tap repair too often. Not to mention that keeping running 24/7 it out of the question because of fan noise & power consumption.

Buying for the recipe

Finding an adequate solution took some time. It all came down to balancing the "ultimate dream" against cost, my ability to reuse existing hardware, space constraints and estimating the true potential usage. The long pole of my research came down to the RPi version compatibility across the various add-on boards/hats and determining the right way to power the system. From all of my solution searching, I become even more confident in my decision to purchase the Rock Pi Quad SATA HAT solution.

Now with a shopping list in mind [or digital paper] and narrow down the just the essentials. I sought out the true best pricing and availability of the items I was planing to buy.

2.5" NAS Setup

Rock Pi Quad HAT for Pi v4
Purchase Raspberry Pi v4 1GB
Purchase USB Type C power supply
use existing 2.5" drives (Various sizes x4)

3.5" NAS Setup

Rock Pi QUAD HAT for RPI v2/3
Options:
- Use existing RPi v2b
- Purchase a Libre Compute board (Le Potato)
- Purchase a RPI v3+
Purchase supported PSU
use existing 3.5" drives (1TB x4)

Take My Money

NOT...SO...FAST!!!

I was so ready to put my money where my mouths was, but didn't purchase the Rock Pi board for the following reasons.

The two sellers of the product were out of stock. 😭
I'm waiting on version 1.2 of the board to become available so I don't have to use an ATX power supply to power the board & RPi while connecting 3.5" hard drives (it would remove my low power goals).

So for a short time [I hope] the build out of my RPi NAS will be on hold. Once the version 1.2 is available I'll be buying one or two Rock Pi Quad SATA boards and do a write up of the build out process.

Attribution: Banner Image

Are Testers, QAs & QEs relevant in today's software development lifecycles? Asking for a friend.

Ken Simeon — Tue, 03 Dec 2019 07:59:45 +0000

Software Design or Just Winging It?

Ken Simeon — Wed, 25 Sep 2019 06:43:05 +0000

Today I went down the path of designing the initial database schema of a work project I'll be kicking off in the next day or two. And then it had me thinking . . .

Do current Full-Stack and/or Backend developers actually use architecture design tools any more?

Back in my early days of the software development industry, it was common for a senior developer or architect to create a UML design doc of their projected Class & Method interactions for the platform. And in turn they created the necessary UML document for the database structure to support their overall software design. Nowadays, at least in the past 5+ years for me, I haven't seen such a wholistic document created.

So what are "modern" developers & architects doing? Winging it?

Buy or Build a Test Case Management Platform?

Ken Simeon — Tue, 17 Sep 2019 16:25:30 +0000

In today's swift moving software development space there are lots of tools & services for helping a Software QA team manage their test case library. The key needs for one of these systems is to have a way to easily curate the creation, maintenance and organization of the test cases and track their execution progress over time.

Here are my questions:

Did you buy or build a TCM?
What did your company/team do?
What was the driving force behind your decision?

Looking forward to your insights.

What's in my Toolbox

Ken Simeon — Thu, 05 Sep 2019 01:13:52 +0000

As a Hybrid software developer and quality analysis (QA) [aka Quality Engineer (QE)] I use a vast variety of applications & tools to keep myself productive in my daily activities on my Mac.

For Terminal Junkies

iTerm2

Great replacement to the standard MacOS terminal with support for theming and more features then you can shake a stick at.

Xcode Command Line Tools

Its essential for any developer.
Quick Tip- don't click the link. Just open a Terminal and type in git or xcode-select --install or install Xcode from the AppStore. Homewbrew
A must have package manager so you can get all of the missing *nix based utilities not installed by the base operating system.

Tmux

I prefer to use Tmux for managing multiple terminals over the built in functionality iTerm2.
Tmux also allows me to make my configuration more portable across *nix systems.

Vim

Its included with every *nix based system. So it's better to know it then be scared of it.
Install Vim using Homebrew. The version included with MacOS can be outdated.

When you have to GUI

MacVim

I don't leave home without it!

TablePlus

It blows Sequel Pro out the water

MacDown

Its a very simple Markdown editor that includes an inline viewer.

Postman

When you want to test APIs or write API documentation, it doesn't get much easier

Cyberduck

So much more than an FTP client

VisualStudio Code [when i'm being lazy]

Might be the most popular cross platform Editor [next to Vim 😂]

SourceTree

When Git branches gets a bit too messy I turn to SourceTree

Xmind

There are times when I have to visually map out a workflow and Xmind is the right tool for that job.

Utilities

Charles Proxy

CP is it for when you want to see every network communication or manipulate the communication between your device/browser and a server.

VirtualBox

Virtualization is everything these days. Plus VirtualBox pairs well with Vagrant and Docker

Caffeine

When I'm presenting or partnering with someone, the last thing I want to worry about is my Mac going to sleep & locking. SSH Tunnel Manager

Chargeberry

Its just my little nerdy Battery monitor

That's it. What about you? Do you have specific tools or apps you can't live without?

Automated Testing 101 - Swimming with Sharks

Ken Simeon — Wed, 04 Sep 2019 08:09:23 +0000

Preamble

I'm starting a new series called Automated Testing 101 where I will be covering the topic of getting a test automation effort off the ground & making it successful. This has been something I've wanted to write about for a long time and it all kind of kicked off based on my comment to Steven Lemon's post over on Medium about his development teams Test Automation troubles.

There's sharks in the water

Lets say you build cages for shark encounters...You and your engineering team understand the "perceived" dangers. All creation steps of the cages are done by hand. The testing of a cage's durability is conducted by hand also. You've heard some complaints from customers about the lack of durability of your cages but no ones been hurt by a shark as far as you know. But in the same breath from your business leaders you are told your engineering team/org needs to start building more shark encounter cages faster with higher quality. Then someone says aloud that we should automate our engineering processes.

You might be saying, How the hell does this relate to software development? Like my fake shark cage engineering company, we build software products for people and have an idea of how the product should be used. We never fully know how our products are actually used. There are complaints about the quality of the product (bugs or missing features) and our business leaders always want something new or fixed faster.

[Fast forward] To meet the needs of delivering faster your software engineering team has implemented layers of automation around the development processes like continuous builds/deployments and infrastructure scaling. Now the business turns to the QA team and says the testing efforts are taking too long and slowing delivery. We need testing to be quicker and more accurate. So lets implement automated testing.

Getting Started with Test Automation

TIME. The #1 reason given to a QA/Engineering team for implementing automated testing. #2 is consistency of test execution. Both of these bubble down to a belief that the bottle neck in software development lies within the QA team & their abilities to complete the necessary validation efforts [this is a topic for another post. we'll go with it for now].

So here is the deal with shifting a group from manual testing to automated testing that the business or engineering teams need to understand.

Manual testing will never be fully replaced by automated testing.
An engineering team can not become reliant on automated tests over manual testing for a long time until trust is earned by the automated tests by preventing and/or discovering issues.
Know that all current manual tests will not translate to being automated. There will be a need to rewrite or throw away some tests.
Automated testing should not just be UI tests. This could becomes a receipt for disaster.
Automated testing is not just owned by the QA team. The entire development team has just as much responsibility by making the platform developed consistently and accessible for automated testing.
UI tests can be flaky or brittle so they must be focused on testing the UI interactions and use the necessary business logic to drive the UI behaviors.
The kick off of a test automation project should start by defining the scope of test automation project that the engineering & QA teams will create [unit test, integration tests, API tests, UI tests, etc]. Google Test Automation Pyramid if you haven't yet.
The initial focus of UI automated testing should be around getting rid of the low hanging fruit [BVT, common UI interactions, etc] while you flush out your automation framework/platform of choice. And then target more advanced scenarios/tests.
Finally you'll have to figure out a way to show the execution consistency of the automated tests [aka Reporting] so you can show the business/team the value of all the efforts behind the project.

Closing this chapter

Ultimately in software development, the customers and business leaders are the sharks. For each, you have to design and develop your products & testing approaches to keep your engineering team safe from being eaten alive while pleasing the hungriest of each species.

So, now that I've armed you with information and theory, are you ready to jump into your shark encounter cage of test automation? Hahah no you're not! Right now I've made you shark bait. You'll have to wait for Part 2 so I can start turning you into a fisherman & not food for the sharks.

Bonus: 10. I don't do 10 Tips or Top 10s

Credits:

Banner Image: isango.com
Shark GIFs: Finding Nemo