Forem: Cascade Energy

External Library Research Questions and Guidelines

alexsiegberg — Mon, 04 Oct 2021 19:07:31 +0000

External libraries can significantly boost productivity and dramatically reduce turnaround time when developing new applications or when implementing new features. However, researching a new library can be a very time-consuming process. It's important to recognize that there's always a cost associated with introducing a new library to an application. This cost may not necessarily be monetary; there may be a learning curve to use the library, time for setup and integration, and requirements to maintain the dependency. Although, if chosen appropriately, the benefits of a library will very much outweigh the costs. Outlined below is a set of guidelines and questions to help navigate the process of researching a new library.

Alternatives and Necessity

Before going down the potential rabbit hole of researching a new library to add to your application, peer into your current dependencies to see if a solution is already at hand.

Can you employ a library that your application already leverages to accomplish this task?

Inspect the libraries that have already been implemented, would any of these fulfill the task requirements?
If they can, would it require you to make any modifications? Would a new library require less effort to implement?

Would a new library save you time in the short or long term?

It's important to note if a library is intended to be used as a temporary measure that will require intervention soon, or as a long-term solution. This is important to keep in mind when researching how much effort is required to implement this library.

Support and Adoption

When choosing a new library, it's advantageous to select a consistently maintained, well-documented, and widely adopted solution.

How well maintained is this library?

Is it extensively documented and easily navigable?
Are security vulnerabilities patched on a regular basis?
How many monthly downloads does it have? When was the last commit merged?
- While these aren't necessarily correlated with the quality of the library, wide adoption and frequent updates are green flags

Is support offered in any manner?

Is it developed by a team or an individual?
Do they offer support plans?
Are questions and issues addressed in their GitHub?
Is there an active community behind this library?

Performance and Implementation

Any additional libraries that you bring into your ecosystem, should not have any notable, negative impact on your performance. Ideally, you should also pursue libraries that don't require a significant amount of effort to implement, if at all possible.

Performance and size

How large is the package size? How much will it affect the size of your code bundle?
Is this library efficient? Would it negatively affect performance and load times?
Is the software bloated with other features that you don't plan to utilize?

The burden of integration

It takes time to learn how to use new libraries. What is the learning curve for this new package?
How much time would take to implement?

Spike testing

It's highly recommended to implement the library into your application, in its simplest form. This will ensure the library is compatible with other dependencies and that it will be suitable for your use case.

Accessibility

Software should be inclusive and cater to as many individuals as possible. This includes bringing in libraries that accommodate individuals with disabilities, seeing impairments, color blindness, support dated equipment, and slower internet connectivity.

Is this a UI-based library?

Can you verify from examples if this library uses aria attributes, if it’s keyboard accessible, and if it meets any applicable WAI-ARIA Authoring Practices guidelines?

License and Costs

What type of license does this library operate under?

Pay close attention to the license conditions of the library. Ensure the license is compatible with your planned usage and align with your organization's requirements.

Monetary costs

Is there a "Pro" version? Do you need any features from the "Pro" version?
If you do, are the charges reoccurring or is it a one-time payment?
Will this require any additional system architecture to operate? What additional costs would that entail?

If No Solutions Exist

There may be occasions where no library exists that fits your exact needs. In cases like these, it would be best to consult your design team, product team, and/or other developers to see if it's within reason to modify the task at hand in such a way that would circumvent the need for an additional library. Lastly, if all else fails, a custom solution may need to be developed and employed. Consider making it open source, to benefit the community!

Design graphs for color vision deficiencies and everybody reaps the rewards

Loren Guerriero — Wed, 24 Feb 2021 00:58:00 +0000

One of our energy coaches (Pam) shared some customer feedback about a significant share of our customers who have difficulty interpreting data visualizations because of color vision deficiency, a condition which affects 1 in 12 men (8%). When Pam was instructing her class to observe the “red line” on a graph, a participant raised their hand and said “it’s all the same to me”, and when Pam asked the group who else has trouble distinguishing colors, more members of the class spoke up.

The most common form of color vision deficiency is Deuteranomaly, which makes greens appear more weak. Below is an example of how one of our graphs would appear to someone with this condition:

In order to distinguish the multiple data series, one must compare the color of the line, and of the tooltip text, to the legend below, and by that infer which data series they are looking at, a task that requires more effort and difficulty even if you do not have any color vision deficiencies, but merely are looking at a small screen, or a dimly lit screen, or with vision. impairment more generally.

This is why it is important to communicate information in multiple ways in addition to colors, such as using icons and text. When this feedback came in, we happened to be in the middle of designing some new graphs. With color vision deficiencies top-of-mind, we made the following interventions:

Use a high-visibility color palette, and choose colors that avoid the hazardous red-green distinction. Our data visualization library offers one of these out of the box.
Vary scatter series by not only color, but shape as well
Include the color, shape and series name text in all tooltips

This is an insight that isn’t limited to software per se - it is also something you can keep in mind if you are responsible for preparing reports that use data visualizations to share with others. You can even run your graph through this color vision deficiency simulator to see your report through others’ eyes.

Like with most accessibility work, designing for people living with disabilities often benefits a much broader group of people. It's just simply more mental effort to interpret information through an implicit color code - we have to keep the code in our memory as we evaluate new information. The more you explicitly spell out key information, the easier it will be for people affected by color vision deficiencies, and everybody else, to comprehend your data presentations.

Remote Pairing Strategies

Ben Flath — Wed, 08 Apr 2020 22:13:59 +0000

On our team we do a lot of what we call “pair programming”. Many of you may be familiar with this term already, but for those who aren't, pair programming typically involves sitting in a room together and tackling a particular problem or feature, often for a day or more. However, since our office is temporarily closed for safety reasons we’ve had to learn to adapt our practices. During a recent retrospective the team collected some of what has worked for us for remote teamwork since the start of quarantine.

Most of these tips and techniques can be applied to any kind of work, not just writing code! They are geared toward two people working together rather than a full team or larger meeting.

Note that many can be applied to working with another person more generally, but we found that every challenge that comes with 1-on-1 work in person is magnified when doing so remotely and therefore worth highlighting.

One final message before we begin: all of this data is qualitative, with a small sample size. By bringing this post to the community we hope to expand that and create a list that more closely represents universal trends.

Benefits

We’ll start off with the benefits of pairing on work.

The biggest reason we pair on tasks is the classic adage: “two heads are better than one.” It’s true! When two people are working on something together it is delivered more polished and complete than one person working alone, thanks to real-time review and quality-control. The people involved also report that they come out with a better understanding of the final product.

When more than one person is working on a subject this also furthers the dissemination of understanding across the team. Leaving one person to work on a particular problem means that only one person truly understands the solution and we’ll need to hope they’re around if it pops up again.

And finally, a huge benefit to pairing is you get some Real Human Interaction! As we’ve heard frequently over the past few weeks, it is important to balance our physical and mental health with our working lives. Take a few minutes at the start of a call to be social! Show your coworkers that you care about their well-being. It helps to relax your mind and takes off some of the pressure that we’re all feeling.

What has worked

Without further ado, here are some strategies and techniques that have worked for our team.

Start with a goal and milestones

This one is helpful for any project you’re working on, remote or in person. Creating a project goal and smaller milestones allows you to pick which items you would like to focus on completing during each call. Be sure to brainstorm this at the beginning if it’s not something that already exists on your project!

It is also a good idea to define your roles and responsibilities at this point. When each of you is clear as to who is doing what, it makes working independently easier if you can’t find a time to work together. If it makes sense for one person to take on more of the project, they should. There’s no sense in dividing the work equally if it will actually be harder to complete that way.

Set a working schedule

Let’s get real: most of us are working from home right now, we’re not used to it, and that has totally messed up our normal schedules. That can be good in small doses, but also stressful! Setting a daily check-in with your pair programming partner can help regulate your schedule in general, but it will also help you both get in tune with individual progress and what still needs to be done. This doesn’t need to be a video call; it could simply be ten minutes set aside to message back and forth with minimal distraction.

In addition to the short check-in, creating a daily work session will provide a block of time for heads-down work without fear of someone scheduling another meeting for you. This is especially useful when there is a short turnaround time. You don’t always need to work together during the daily session, or even to use the entire time, just keep it on the calendar!

Take turns being “the driver”

Watching someone else write code can feel like watching grass grow. I’m sure if you're reading this you are more interested than most, but after a while it gets old. Try switching off who is sharing their screen with whom regularly. Not only does this keep attention better, but it increases the sense of “ownership” around the feature. It also helps with QC, as the passenger is more likely to spot an error where the driver may be focused elsewhere.

An important part of being the driver is making sure your passenger knows what you’re doing. If they don’t, you’ll both get lost and frustrated. It helps to narrate what you are doing. Speak what you’re typing; say why you created that function the way you did; explain the unit test you’re about to write. Especially when one of you has a poor connection and can’t make out the code, this sort of explanation helps you stay in sync. Creating this shared understanding helps with continuity when you’re working separately, too.

Use a shared digital workspace

Whether it’s a shared IDE, online notes program, or whiteboard app, having a place where both users can type or draw simultaneously while talking on video chat helps describe ideas, remember bits of information for later, or provide examples for something. You may also find that sharing this out later helps other team members gain a deeper understanding of your feature.

Check on process regularly

Just like your agile (or kanban, or scrum, or…) process, your pairing process shouldn’t be set in stone. Check in every so often and ask some of the standard retrospective questions. What is working well and what could be improved? What should we keep doing, stop doing, and start doing? Any of these will work. You don’t need to spend too much time on it but be sure to ask and answer questions without ego. Remember: feedback is a gift! Use it to learn where you can improve and iterate on your pairing process.

Be patient!

Sure, you should always be patient when working with another person, but remote pairing requires an extra level of patience. Lag in video calls or screen shares may suggest that the other person is done speaking when they’re just delayed. It’s hard to read cues in body language over video that we take for granted with in-person interactions, so instead just wait another three seconds before starting to speak.

Pairing with someone remotely also exacerbates differences in the pace at which people work. Don’t get frustrated when your collaborator asks you to slow down or doesn’t know all the shortcut keys that you do. In addition, narrating while working is a new challenge for most of us, so expect that both of you will be going more slowly than usual.

Challenges

There are still many challenges remaining when it comes to remote teamwork. Here are a few we are still consciously working on.

All interactions are “meetings” now

Maybe it’s different at your employer, but with a workforce that is 100% remote, many people have found that they have more meetings on their calendars than before. Coworkers are scheduling time because they can’t just walk over to someone’s desk and ask a question. This can make setting a recurring meeting time more difficult.

Additionally, notifications from chat programs can be a distraction or privacy issue during screen sharing. Be sure to disable any popup or banner alerts before you start your meeting!

Technical issues

There are inherently more technical issues involved in coding over video calls and screen sharing than coding while sitting next to someone. You have to depend on your and your collaborator’s internet connections, the call app, any shared documents, and more. These issues can be particularly frustrating since they’re out of your control. But to reference back to the last section, just be patient! If you can’t work through a particular issue, take a break from pairing and come back to it later.

It is also important to keep in mind that some tasks don’t translate to screen sharing. Scrolling through code or webpages looking for something specific just does not work, even with the best internet connections. Small fonts or particularly detailed screens are another no-go; if you need to lean in toward your monitor to see something, your collaborator won’t be able to distinguish it.

Learning curve

Remote pairing is not easy! It takes time, cooperation, patience, and practice to do it well, so don’t expect your first work session to be flawless. It is particularly challenging for most people to talk and type at the same time. Even just explaining what you’re doing and why could be something you’re not used to.

During this learning curve you will also find the remote pairing sessions more physically and mentally demanding than in-person pairing. This is also normal! Just like the first time you give a presentation to the CEO or go for a run, some things take time to get used to. Practice. Iterate on your process. Don’t assume that the way you pair with one person will be the most effective way to pair with every person. Did I mention “iterate”?

Feedback

Let us know what you think! Have you tried any of these techniques? Do you have any of your own that you find invaluable? Let’s keep the list growing!

Renewing Our Open Source Commitment

alex-at-cascade — Tue, 15 Oct 2019 15:39:49 +0000

Motivation

Cascade Energy is an engineering services company, focused on helping industrial customers increase their energy efficiency. The software team, of which I am a part, is a small group within that company, responsible for the development, operations, and security of energy management software. Our team resources are limited, and so we make extensive use of open source libraries to increase the value we can deliver.

Over the years, the team, through its various incarnations, has also published a number of open source projects (see our past public projects), but the efforts had decreased over time. (As it turns out, many of our earlier contributions are less relevant now as the ecosystem itself has evolved.) Recently, however, we were revisiting the Cascade Energy core values (Do the Right Thing, Put Others First, Seek Shared Success, Learn Constantly, and Be Industrious). We considered how much value we get from the technology community of which we are a part, and we were inspired to redouble our efforts to give back. To crystallize our thinking into action, we initiated an ongoing series of team meetings oriented around "outreach" of all kinds, including meetups and conferences, blog posts, open source, and even hosting interns.

As a result of our deliberate attention, we have tangibly increased our own engagement level in these areas, all of which are worthy of their own write-ups. This article, in particular, is about our concrete efforts toward rekindling our open source efforts.

First Things First

Given how much time had passed since our previous efforts, we did some preliminary groundwork to make sure company leadership understood and supported our renewed drive to publish open source. As we had hoped, they were completely supportive of our efforts, but it was important for us to verify that.

To minimize the chances of accidentally publishing something that shouldn't be, and to emphasize the difference between public and private, we decided to create separate "orgs" in github.com and npmjs.com, distinct from the existing orgs that hold our proprietary work. (This may also make it more straightforward to include outside maintainers, if that were desired in the future.) We picked a name to publish under, "Cascade Energy Labs", and cleared that with Marketing and Legal.

The Set Up

GitHub

Everyone on the team already has a GitHub account tied to their work email. So, once signed in, I found the tiny + drop-down in the top bar, and clicked "New Organization".

For "Organization name" I entered Cascade Energy Labs (which generated an org URL of https://github.com/cascade-energy-labs).
For "Choose a subscription" I selected Team For Open Source: $0 per month (being thankful how supportive of open source these collaboration platforms are).
For "Billing email" I entered the team-lead's email address
For "This organization belongs to" I selected A business or institution and entered Cascade Energy, Inc.

Once created, I invited the rest of the team to the new org (via their email addresses). And after they accepted the invitations, I set each of them to have a role of "Owner" via the "Change role" option, under the gear menu for each person in the "People" tab of the org.

Note: this is also the place to verify that all the members have 2FA enabled for their sign-in. If, as we do, you want to enforce that for everyone in the org, go to the "Settings" page for the org, go to the "Security" section, and select "Require two-factor authentication for everyone".)

With this new org, we saw a great opportunity to explore some of the new features offered by GitHub, namely "GitHub Actions" and "GitHub Package Registry", and so I applied for the public beta of both of those programs. (We received confirmation of acceptance into both of them about a week later.) Finally, I made sure a couple of GitHub Apps were installed for the org, namely "Dependabot" and "Pull Panda".

NPM

Team members also all have an NPM account tied to their work email. I clicked to + next to Organizations in the left bar:

For "Name" I entered cascade-energy-labs
For the "Unlimited public packages / Free" option I clicked the Create button (again being thankful of open source support).

I then invited the rest of the team via their email addresses. After they accepted the invitations, I set each of them as an "owner" of the org.

To enable automated package publishing, I:

(through our IT department) created a special email list alias (that includes the team members as recipients)
used it to create a "bot" account for our org (saving the password in a secure location accessible by the team)
created a "publish" token for the account (also saving the token in a secure location)

The First Project

Given the likelihood of false starts and missteps when establishing new patterns, I decided to start off our new org with a toy project that would let us focus on the infrastructure first. After much consideration, I finally came up with (wait for it...) "Hello World". And, since automation in software deployment has become the norm in our industry, the rest of this section describes exactly how I configured that.

Creating a new project in GitHub is incredibly easy. Under the Repositories page for the org, I just clicked the "New" button, made sure the "Owner" was shown as cascade-energy-labs, and typed in the name hello-world (as well as a helpful "Description"). I made sure the "Public" box was checked (enforced for you when your org is Open Source), and also selected "Initialize this repository with a README". Finally, I opted to "Add .gitignore" for Node, and also opted to "Add a license" of MIT. Then I clicked the big green "Create repository" button, and there it was!

Note: The selection of an open source license has many ramifications, far too many to address here. A web search on "open source licenses overview" will turn up countless resources to guide you.

Since Dependabot is enabled, I went to the "Security" tab for the repo, and enabled "Automated security fixes". (This will automatically generate pull requests for package updates related to known security vulnerabilities.) Also, since I know I want to publish to NPM, I went to the "Settings" tab for the repo, clicked the "Secrets" menu, and added a secret named npm_token (the exact name is important) with the value of the NPM "publish" token that I had saved off up above.

GitHub Actions

A new feature of GitHub (still in beta testing) is GitHub Actions. This is a mechanism that allows creating continuous integration (CI) and other workflows from scripts that will automatically execute on your behalf, based on various events. Although you can write your own workflows from scratch, there is a growing library of canned workflows that cover many common use cases.

In my case, I started with Node CI (which will build and test your project on every push, in every branch). Once our org was accepted into the beta program, adding the first action to a repo was incredibly simple. In the repo page, I went to the "Actions" tab, and under the "Popular continuous integration workflows" found the "Node.js" card. I clicked "Set up this workflow", and instantly had a pre-filled YML file in front of me, ready to commit. (The commit process via the web interface is straightforward - I was presented with the choice to either commit directly to master, or to create a new branch to commit to while automatically starting a PR.)

Including additional workflows is just as simple, by clicking the "New workflow" button in the Actions screen. Using this, I added the Node.js Package workflow to the repo. To get it to work correctly, I did have to make a couple of modifications to the default file:

for the publish-npm job, I needed the --access public option on the npm publish command (otherwise NPM tries to create it as a "private" package, which is unsupported under the "Free" option).
also for the publish-npm job, I verified NODE_AUTH_TOKEN: ${{secrets.npm_token}} - the name should match the secret added when you created the repo above.
for the publish-gpr job, I needed scope: '@cascade-energy-labs' instead of the canned scope: '@your-github-username'.

The "Node.js Package" workflow, by default, is triggered on a release action, and publishes to both GPR (see below) and NPM. So, from the releases page of my repo I clicked the "Draft a new release" button, and watched the actions run. Several minutes later, my package appeared in NPM and also in GPR.

Note: Repo workflows are stored as files in your code repository; any .yml file in .github/workflows will be treated as one. And, while various versions of these files may exist in other branches, only the version in the "default" branch (typically master) will actually be applied to the repo as a whole (including all branches). You can add and edit files here directly as well (as an alternative to using the "New workflow" UI), which allows you to create custom workflows, and also make modifications to canned workflows. Given that the feature is still in beta, editing might be needed from time to time.

GitHub Package Registry (GPR)

Another new feature of GitHub (also in beta testing) is GitHub Package Registry (also known as "GPR"). This functions as a storage-and-delivery service for code packages (similar to, e.g., NPM and Maven Central).

To publish a Node project to GPR, I used the same npm publish command as when publishing to the central NPM repository, but configured with a different registry URL and explicitly specified scope. (Those differences can be seen in the "Node.js Package" workflow referenced above.) Although I used GitHub Actions to do the publishing, executing it from a command line would be very similar (with the changes being made in the .npmrc file directly).

I published to both GPR and NPM, since npm tools make it easier to pull from the default NPM registry than from other places. But if I had only published on GPR, or if GPR was preferred as a source, a consumer of my Node package would simply need to set an entry in their .npmrc file tied to my package's scope, e.g., @cascade-energy-labs:registry=https://npm.pkg.github.com/; then, npm install would download from there instead.

Going Forward

We wanted to make sure we were set up properly for our renewed efforts. (Sometimes the first steps are the hardest.) And now that the groundwork is laid, we, as a team, have identified several real projects that we'd like to share with the community in the coming months. There is some effort needed to make them "publishable" (like removing hard-coded idiosyncrasies that are tied to our specific environment). And that effort has to be squeezed in between our regular work tasks. But it's something we're excited about doing!

So, what about you? What has inspired you to enter (or re-enter) the world of open source? I'd like to hear your story. Did our story make you think of other things? Please let me know in the comments!

Balancing Dev & Ops: What is a Product Owner to do?

Ben Flath — Wed, 25 Sep 2019 22:52:56 +0000

Introduction

What’s the Problem?

As the product owner (PO) for a small software team, I have read countless blogs, training manuals, books, how-to guides, and more about how to order items in my team’s backlog. I’ve gotten tons of great advice and worked with my team to implement strategies that help us decide which features will deliver the most value to the customer. There is a near endless world of information on the subject and for that I am grateful.

However, my team is not just a development team. We are a DevOps team (or DevSecOps, if you prefer). This means our backlog is not just full of new features and improvements, it also contains bug fixes, infrastructure management, security tasks, and more. With such diverse tickets, how is a PO to compare and sort them in order of importance? What’s a higher priority: a patch fix to a production server, or creating that new front-end component a stakeholder asked for? It’s comparing apples and oranges (cookies and muffins? wine and beer?).

Team Background

Before I jump into it, let me give you some quick background on my team. We are a small team (<10 people), embedded in an established engineering consulting company (>150 people), building a product for a small, but rapidly growing, market. We follow the Agile Principles and are leaning towards Scrum, with no illusions that we are a proper Scrum Team. Our largest user base by number are the company’s clients. Our largest user base by time spent in the application are internal to the company.

Note: I am not a software developer! My background is in civil engineering and I have just enough knowledge to say the right words in the right situations and mostly understand what’s going on in the code. Any technical bits in this post are meant for background only, so please forgive any mistakes or unelaborated, non-specific statements.

What Hasn’t Worked

First, let’s talk about what hasn’t worked for us in the past. A couple years ago we embarked on a long project to rebuild our data pipeline. It changed not only how data is processed when it arrives to our system, but how that data gets transformed and delivered to the front end. The entire team worked on this for a full year. The outcome was a nearly 10-fold increase in front-end performance, and dramatic improvements reliability and stability on the back end. It was a huge win!

Except we also delivered exactly zero front end features for a full year. Yes, users were happy with the speed of the application, but they also remember 11 months of silence. What we delivered did not replenish the magnitude of good will that we lost over that time, and it took close to another year before we were in good standing with our core internal users.

Fast forward a few years as we start to replace one of our long-in-the-tooth front-end modules. This was another major project with months of planning, user interviews, design iterations, etc. The importance of delivering our first “new page” since the initial release got us excited and we wanted to knock it out of the park. So again, everyone was working on one thing. You can imagine where this is going: we delivered a great product and our users were happy! But we also started running into the limits of our system architecture as more users switched to the new module. Time for another all-hands-on-deck infrastructure project!

At this point we know that going all-in for big projects doesn’t work. But what is the ideal split between ensuring the system is secure, developing new features, and managing operational tasks?

Strategies for Prioritizing

Here are some of the strategies that have worked for our team to order backlog items. We’ve been refining this system for about a year, and in true Scrum fashion, will continue to refine it every cycle going forward. Some of it may work for you, and some may not.

Security, Requirements, Breaking Bugs

In the words of Amazon Web Services, security is “job zero”. If our system is compromised, that puts our customers’ data at risk. That is a level of trust that we will never win back. Critical security issues are always the priority!

But not every task related to the security of the system is critical. There are some changes that will improve security but are not required for the application to be secure. Others may be associated with a particular compliance or contract rule but are not for security reasons alone. What is important in situations like this is to work with your decision-makers to ensure that any deadlines give you sufficient time to implement the necessary measures without stressing your resources.

It can help to explain what the trade-offs are of setting a shorter project time period. For example, if a proposed contract would require implementing changes within three months, and you need three months of all-in work to complete it, let the decision-makers know that the next few features in the backlog will not be released to the existing user base during this time. If the contract can be negotiated to six months, you can deliver new features and still meet the requirements.

The other type of task in this category is bugs that break functionality. Anything that prevents a user from doing what they need to do is important and should make its way to the top of the backlog. That could mean sliding into the current sprint, if a large number of users are affected, or the next, if there is a reasonable workaround.

Functional Obsolescence

Borrowing a term from my civil engineering days, I’m using “functionally obsolete” to describe a part of the system that typically runs fine, but at times of high traffic may start returning errors or becoming overloaded. There are often band-aid-like solutions, like increasing processing power or available memory for a service, that can be applied easily, however the larger solution would take more development resources. Even if the band-aid could get by for a year, you’ll need to re-build it eventually.

One key for us with projects like this is to spend the time to plan out, roughly, what needs to be done and estimate the size of the undertaking. This doesn’t have to be a full set of tickets, individually sized, with definition of done and acceptance criteria, but more of a bulleted list of how the team would go through the process. The brainstorming session could help determine if it’s a 6-week project or a 6-month project, which lets you fit it into the roadmap with more accuracy. It may also reveal discrete, independent tasks that move toward the proposed solution little by little rather than all at once. From there, you need to determine how other factors, such as the risk of downtime or loss of data, customer frustration, or increased server cost, weigh into the decision on when to move forward.

If a project turns out to be smaller, or if you are close to finishing a larger project, we’ve found that going all-in for a single sprint can actually increase the sense of accomplishment for the team. Coming together and working towards a single goal, with a defined ending, can be a boost, especially when it feels like lots of projects are floating around half-complete. Better yet, end the sprint with a team lunch or happy hour!

Saves Developer Time

Now it’s getting a little tougher! This category now has no direct effect on end users, but the indirect value is hard to ignore. If a task will increase the velocity of the development team, it’s worth doing, that much is clear. But how do you fit it into the backlog?

To start with, it can help to quantify how much time or effort the task would save, and how often it’s done. If you can save an hour every week, that’s a great opportunity! If it’s 5 minutes once per quarter, that has less value. Time isn’t everything, though. A task that reduces the complexity or likelihood of human error in the team’s work should also be higher value.

The most common way we’ve found to weave tasks like this into the backlog is to pick them up when that part of the code base is touched for another reason. For example, we recently implemented automated deployment for all our services. Rather than do this all at once, we went repo-by-repo as other backlog tickets touched the services. While it took half a year to get to each repo, we were able to balance it with continued delivery of new front-end features.

Candy Jar

This is a practice that can be controversial with some in the Agile community. If a task is small, let’s say only about a half-day of work for a developer, but it isn’t going to displace other tasks of higher value, we add it to the candy jar. Opponents argue that the reason the backlog is ordered is so that when a developer is finished with one task, they can move on to the next most important. However, if it’s the last day of the sprint, and all other committed stories are complete, a candy jar ticket fills in that space nicely. It also provides an outlet for a task that may not necessarily be important now but might be nice to finish before it becomes a problem somewhere down the line.

Side Projects & Experiments

While I don’t think any of us have the resources to implement Google’s “20% of time to side projects” policy, it is important to allow some time for experimentation. Especially during large, multi-sprint epics that can get monotonous, developer burn-out is a big concern. Making sure your team is happy and engaged is essential to a healthy workplace!

This is one area in which we have not made great progress. Every few months we sit down as a team and talk about how to start working on a fun team side project, but regular sprint deliverables always seem to get prioritized. One next option would be to meet with management to discuss the importance of “extracurricular” projects and budget time to support them.

A way to make these projects more attractive is to use them as a test bed for new tools or technologies that will eventually become part of your product. That way the team can iron out the kinks and make the newbie mistakes outside of the main code base, then implement them with more refined knowledge later.

Other

Of course, there are items that don’t fit into any of the categories above. If the backlog is always ordered by value added, they may never get done.

Okay, here’s the big secret: I don’t have the answer!

We tend to pick up tasks when they feel convenient, or when a developer gets frustrated enough with something on a given day that they just make the change (or convince me that it’s important enough to add to the sprint).

Sometimes a task can be added to an existing story with a small increase in scope, such as adding half a day to a three-day ticket. The important note for this approach is that you need to make that decision before the start of the sprint, not during the sprint. If it happens during the sprint, then it’s scope creep and your sprint could be in danger of going off track. But if you make the decision before the sprint starts, the team can size the ticket appropriately for the larger scope and still commit an appropriate amount of work.

Conclusion

Let’s face it: for a PO, ordering items in the backlog is hard when it’s a bunch of new features. It’s even harder when it’s new features, security updates, operational tasks, and bug fixes! I provided a few strategies for approaching this challenge that have worked for us, but I can say, with high certainty, that there is no one right way to do this. What has worked for us may not work for your team.

Fortunately, following Scrum practices gives us frequent feedback on what is working and what is not. This is really the key to everything! Just as important is sharing this feedback with decision-makers and stakeholders to ensure that they are bought in to whatever strategies you decide to use.

Lastly, I would like to invite you all to share what strategies you use to order your diverse DevSecOps backlogs. I’m interested to learn what other teams are doing, and hopefully bring new ideas into my repertoire.

What I Learned From My First Year as a Software Developer

eric crowder — Mon, 16 Sep 2019 15:44:35 +0000

Like a lot of individuals in software development I worked in a different industry prior to making the career shift. I recently completed my first year as a software developer at Cascade Energy and thought I would share a few bits of information that may be of use to others who are beginning to embark on a similar path.

After working 10 years in the accounting field, I acquired an array of skills and experiences that ultimately presented software development as a viable alternative career path. It should be noted that these skills are not specific to accounting; in fact, you may find that they are universal skills for all endeavors. Therefore, I have purposely made light comparisons to my previous career experience for illustration reasons only.

In no particular order:

Communication is Key

Being able foster collaboration between team members, external customers, and other stakeholders is a universal skill that is important in all fields.

A concept that I was exposed to a few years ago was the idea of "emotional intelligence". I believe this is a fundamental prerequisite for effective communication and can be summarized as effective communicators should possess traits such as: having empathy for others, being self-aware, and utilizing social skills that inspire others. To read more about emotional intelligence, see Daniel Goleman's article titled What Makes a Leader in the Harvard Business Review.

At Cascade Energy, we are a small team of engineers (fewer than 10 people) who work across the stack on a regular basis. We each have specializations, such as front-end, back-end, infrastructure, etc., but our team structure is such that we have to be able to work in all parts of the product with confidence to ensure that we can meet the milestones in our roadmap.

To facilitate this, we make communication the utmost priority. When picking up a ticket that I may not be familiar with, such as writing an AWS lambda function, I will read the code for similar repos, look over blog posts and consult with more experienced developers on our team before starting. This will ensure that I am able to adequately plan a solution before coding. These processes take a lot time to complete. In fact, I was surprised at how little time we spend actually writing code throughout a typical day - it is a lot less than I thought!

Our stack is written mostly in JavaScript. Luckily, I had focused on front-end development when learning to code and was fairly comfortable writing JavaScript. So, that made working with our Node services slightly less daunting. As a team, we foster an encouraging environment that encourages developers to work on tasks that are outside their comfort zone. So, planning solutions with other team members, pair programming, and other knowledge sharing was instrumental in my personal development.

I learn best by doing. Pairing with other experienced developers allowed me to see how they think and leverage tools to solve problems. Making a habit of doing this early on was key in learning our codebase. Additionally, pair programming is an excellent team builder - solving problems with other people is a lot more rewarding than solving them yourself (and probably takes less time)!

Side note - ultimately, when you land that first programming job, you will not be able to anticipate what stack you will be working in. As long as you are comfortable with one language, you can easily pick up another - the core analytical skills that you acquire while learning your first language remain constant.

Problem Solving - Have a Plan, Implement It and Leverage Your Team

Whether it is implementing a new feature, fixing a bug, researching a topic, or sketching out the technical details of an infrastructure ticket, problem solving is inherent in what software developers do each day. Observing problems at a high level allows one to deeply understand the issue, which sets them up to adequately weigh possible solutions.

In practice, the following has worked well for me:

Summarize the problem in one paragraph.
Using lists, define the following:
- potential causes of the problem.
- potential solutions to the problem.
For each potential solution, outline the pros and cons.
Select a solution and implement it

Once you select a solution, sketch out how you would implement it. This could take the form of words, diagrams, pseudo-code, or whatever best illustrates the particular issue at hand. This document will serve as your roadmap for the code you will eventually write. Writing code in accordance with an outline is much more effective than writing code without one. You will be able to better track your progress and have an easier time debugging issues along the way. Besides, the code is basically already been written - you are just need to translate your outline to your programming language.

Debugging is Awesome

Learning how to properly use debugging tools has had the most profound impact on my programming skills. When I first started working as a software developer, my debugging strategy was limited to placing logging statements throughout the codebase and hoping that I would find clues as to the location of the problem.

Logging statements are fine, but they should not be the only tool in your toolbox. Once I was introduced to the debugger in my browser and text editor, my understanding of our codebase increased by an order of magnitude.

Debugger tools allow you to place breakpoints strategically and see what variables and functions (including arguments) resolve to. Further, in addition to the values that are in scope with respect to your breakpoints, you can see all of the values throughout your program for a more holistic view of how the code is executing on each call. That way you can understand what and why the variables and functions in your program are resolving in the manner they are. The combination of logging statements and debugging tools has enabled me to learn new code faster and therefore allow me to fix bugs and write features a lot more effectively.

Summary

As you can see, the themes of communication and problem solving (debugging included) are key attributes for a software developer to possess. The best part is, however, that these skills are required in many facets in life and are not specific to software development (or accounting!). That means you get to bring your specific life experience to your team to leverage collectively. The diversity of life experience within a team enables it to make better decisions because the team has access to different perspectives. After all, as software engineers, we determine solutions to address problems for our customers, so we better do it the best way we can!

The 5 Dysfunctions of a Software Team

Jinsy Oommen — Sun, 25 Aug 2019 20:03:20 +0000

Over the past fifteen years, I have worked with some amazing teammates, but sometimes, we struggled to ship. Not shipping features is very demoralizing to a software team and it corrodes the product and team cohesity. Even though all of the individuals within a team might be capable and extremely productive individually, there might be some factors that cause us not to ship when we get together as a team.

As part of an assigned reading for a team I was on, I read the Five Dysfunctions of a Team by Patrick Lencioni. He frames the dysfunctions as Absence of Trust, Fear of Conflict, Lack of Commitment, Avoidance of Accountability, and Inattention to Results. From my experience, I found software teams to be more nuanced in their productivity and output and I found it helpful to reflect and define the dysfunctions of a software team with more specificity.

Lack of Technical Leadership

Decision paralysis can set in when there are many things on your plate and you need to pick a direction. A strong technical leader can help facilitate the decision making process and own the decisions that have been made. Teams also need to be constantly aware of the technical vision. Individual contributors can and should have tunnel vision on what they are working on, but it is important for them to have direction when they Need it. An effective technical leader can also help communicate the need for and importance of sustainability and technical maintenance of the product to the stakeholders.

Lack of Product Definition

A default state for developers and individual contributors is bettering the product through working on maintenance or trying out new ways to improve developer experience. This is great, but can demoralize the team eventually if it is not balanced with features that provide business value. When the business path of a product is not well defined we also may have the tendency to fill our backlog with not-essential-to-the-business features. This could cause users to not engage with the product, leading to not enough feedback and eventual disengagement within the dev team. Not having proper product definition could also lead to too many pivots in the development cycle, which in turn can contribute to the feeling of never finishing a project.

Lack of KIND and Deliberate post-mortems and incident reviews

Incidents are inevitable and I would even posit that they are a sign of progress and change. If we are not deliberate on conducting the review without assigning blame, we could go down the slippery slope of not learning from the incident, prevent us from team accountability, and keep us from owning the entire lifecycle of the feature. This can contribute to a culture of fear of failure, thus never trying new things.

Failure to Embrace Automation

Not embracing automation of process, deployment, and release can lead to tedium and release fatigue. Not automating deployment and shipping puts an easy target on the developer’s back leading to blame when something defective gets shipped. Not automating process and standards can lead to time unnecessarily spent on nitpicking and bikeshedding.

Lack of Defined and Documented Process and Rhythm

We are creatures of habit and look for rhythm in our professional lives. A productive shipping team has purposeful process and a rhythm that is sustainable for the team. For some teams this may look like weekly cycle and for other teams it maybe a 6 weeks, but it is important to have that rhythm.

I used to think being productive had mainly to do with our tech stack choices but experience has led me to believe it is more nuanced than that. I think I can say this on behalf all developers that we are content and happy when we ship something that make our customers happy. It takes a village to create and sustain an impactful product.

Tangible software development

Ashlin Aronin — Wed, 17 Jul 2019 22:39:28 +0000

When I first started writing software professionally, I was puzzled by the fixation on tools. A few years later, I’ve come to realize that the tools we use drastically affect both the efficiency of our work and how fulfilling it is. For comparison, imagine a carpenter framing a house without power tools. It can be done, but requires much more time and effort.

At Cascade Energy, we’re a small dev-ops team with a lot of work to do. This makes our choice of tools important. Over the past year, we’ve introduced automated deployments, static analysis and re-formatting (with CodePipeline, eslint and prettier, respectively). This post will focus on one slightly more experimental tool that is redefining our workflow: hot reloading.

Recently I was tasked with adding a new feature to our customer-facing React application. Here is the process I went through, repeatedly:

Decide on a small unit of change I could do at a time
Read through the code and click through the production application to figure out how it currently works
Read tests related to this component
Make a small change to the code itself
Save the file
Switch to my web browser
Refresh the page
Select a sample customer from a dropdown menu to display data on the page
Scroll down the page to the component I'm working on
Click on the component
See if my change worked
If not, repeat

Many of these steps are unavoidable. For instance, most developers will tell you they spend more time reading code than writing it. However, we can consolidate steps 6-10 using hot reloading. With hot reloading configured, every small change I make automatically registers in the web browser, with the surrounding context preserved. There’s a lot of plumbing to make this happen, but once it’s set up, it’s magical.

These days, when creating a new frontend application, you can use a pre-configured starter pack that already has hot reloading and other productivity features out of the box (vue-cli, create-react-app, etc). In this case, we couldn't lean on these tools since this was an existing application with some custom configuration.

Our setup is a Node backend layer which handles connections to our other services and serves up our frontend React application. We use webpack as our frontend build system.

The webpack team maintains the pre-packaged webpack-dev-server module, but it wouldn’t work for our purposes, since the backend and frontend of our application are intertwined. If the backend of our application were built using express, then we could configure the server to use webpack-dev-middleware (used by webpack-dev-server under the hood) directly. However, we’re using hapi, which doesn’t support Express-style middleware.

Only slightly discouraged, I took a deep breath and pushed on. It was still possible to write a wrapper around webpack-dev-middleware. Fortunately, I found an article that got me started-- a tutorial for writing a hapi middleware adaptor for webpack.

I borrowed the basic premise of hooking into hapi’s onRequest and onPreResponse lifecycle extension points to intercept requests and pipe them to webpack so it can handle hot reloading. However, I didn’t find the author’s suggestion of webpack-dashboard to be any more helpful than webpack’s built-in logging capabilities, and it obscured our API logs which normally get routed to the same console.

With a bit more tinkering, I was able to get webpack-dev-middleware hooked up to hapi. Here’s roughly where that got us.

// Install dev middleware
server.ext("onRequest", (request, reply) => {
  if (passThroughRequest(request)) {
    return reply.continue();
  }

  devMiddleware(request.raw.req, request.raw.res, err => {
    if (err) {
      return reply(err);
    }
    return reply.continue();
  });
});

// Install hot middleware (for module reloading without reloading the page)
  server.ext("onPostAuth", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    hotMiddleware(request.raw.req, request.raw.res, err => {
      if (err) {
        return reply(err);
      }
      return reply.continue();
    });
  });

  // Make sure react-router can handle our actual routing
  server.ext("onPreResponse", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    return reply.file("public/index.html");
  });

(passThroughRequest ignores a few paths that need to skip webpack and go straight to the backend.)

With this set up, I tried saving a change to a module. However, instead of a hot reload, I got a warning in the console:

Ignored an update to unaccepted module ./client/components/project/tileView/ProjectTile.js -

…

process-update.js?e135:104 [HMR] The following modules couldn't be hot updated: (Full reload needed)
This is usually because the modules which have changed (and their parents) do not know how to hot reload themselves. See https://webpack.js.org/concepts/hot-module-replacement/ for more details.
…

It turns out that not only do we need to wire up webpack for hot reloading, we also have to teach our frontend components to hot reload themselves. This way, when webpack’s watch process notices the components have changed, it can inject just the changed bit of code and not reload the whole page. Each framework has a different approach to this. React has react-hot-loader, a pet project of Dan Abramov that, despite being quite experimental, is well supported and active. Abramov has written extensively about the concepts behind it, and this article is well worth a read. Essentially, you have to mark your top-level App component as hot-exported:

App.js
import React from "react";
import { hot } from "react-hot-loader/root";

import Routes from "./Routes";
import CascadeTheme from "./theme/Cascade";
import { AppContainer } from "./sharedComponents";

const App = () => (
  <CascadeTheme>
    <>
      <AppContainer>
        <Routes />
      </AppContainer>
    </>
  </CascadeTheme>
);

export default hot(App);

We also had to make some changes to webpack config to load both react-hot-loader and the webpack-hot-middleware client. This is the relevant section:

if (process.env.npm_lifecycle_event
 === "start") {
  config = merge(common, {
    devtool: "cheap-module-eval-source-map",
    plugins: [new webpack.HotModuleReplacementPlugin()],
    module: {
      rules: [
        {
          // Mark everything matching this rule as "cold" (e.g. not hot exported)
          // This will allow hot reloading to work as expected for the rest of the
          // application
          test: /\.js?$/,
          include: /node_modules/,
          exclude: /node_modules\/@sensei\/shared-components\/src/,
          use: ["react-hot-loader/webpack"],
        },
      ],
    },
  });

  config.entry.app = ["webpack-hot-middleware/client", ...common.entry.app];
}

As you can see, this config only applies when the app is run via npm start (aka, in development).

So I got that working, PR’d and merged. Then one of our other frontend developers noticed a bizarre occurrence—logout functionality was broken while using hot reloading in development. The app was still visible to logged-out users but in a broken state, with all calls to the backend failing. I realized that all of our webpack dev/hot middleware calls were getting through, regardless if the user was authenticated or not.

I had nagging feeling that there was a fundamental security flaw in my hot reloading implementation, and that I'd have to just make peace with manually refreshing the page every time I made a change. Regardless, I pressed on.

I tried debugging the extension points and checking the auth object, but it seemed that cookies had not been parsed yet. My first instinct was not a helpful one— I tried importing our hapi authorization plugin and re-injecting it into the new server request extension event methods. This led to numerous forays into the internals of hapi which started to develop a bad code smell.

So I took a deep breath and a step back and re-read hapi’s documentation. The most useful bit was the request lifecycle.

It turns out that in the example posted above, the author relied on the onRequest extension points, which come before authentication in the request lifecycle. The solution was to use the onPostAuth extension point to hook up webpack-dev-middleware and webpack-hot-middleware, so that our normal authentication middleware still processed each request. However, I still needed the onPreResponse handler to serve the HTML file, so we needed to check authorization at that point, too.

Here is what we ended up with:

/* eslint-disable consistent-return */
const config = require("config");
const webpack = require("webpack");
const webpackDevMiddleware = require("webpack-dev-middleware");
const webpackHotMiddleware = require("webpack-hot-middleware");

const webpackConfig = require("./webpack.config");

const IGNORE_PATTERN = /^\/(api|styles|img)\//;

function installWebpackHapiMiddleware(server) {
  // If we're not running under `npm start`, bail
  if (process.env.npm_lifecycle_event !== "start") {
    return server;
  }

  console.log(
    "You appear to be running a development server. Initializing webpack dev/hot middleware..."
  );

  const compiler = webpack(webpackConfig);

  const devMiddleware = webpackDevMiddleware(compiler, {
    port: config.port,
    historyApiFallback: true,
    publicPath: webpackConfig.output.publicPath,
  });

  const hotMiddleware = webpackHotMiddleware(compiler);

  // Install dev middleware
  server.ext("onPostAuth", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    devMiddleware(request.raw.req, request.raw.res, err => {
      if (err) {
        return reply(err);
      }
      return reply.continue();
    });
  });

  // Install hot middleware (for module reloading without reloading the page)
  server.ext("onPostAuth", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    hotMiddleware(request.raw.req, request.raw.res, err => {
      if (err) {
        return reply(err);
      }
      return reply.continue();
    });
  });

  // Make sure react-router can handle our actual routing
  server.ext("onPreResponse", (request, reply) => {
    if (passThroughRequest(request)) {
      return reply.continue();
    }

    return reply.file("public/index.html");
  });

  return server;
}

function passThroughRequest(request) {
  const isNotAuthenticated = request.auth.mode === "required" && !request.auth.isAuthenticated;
  return isNotAuthenticated || IGNORE_PATTERN.test(request.path);
}

module.exports = installWebpackHapiMiddleware;

Software development can often feel intangible, since the code we write looks very different from the direct machine instructions it eventually becomes, through complex processes that few of us fully understand. Ultimately, the immediacy of hot reloading brings our daily workflow closer to that of a carpenter working with physical tools-- make a change, and see it reflected immediately. I celebrate tangible software development and the tools that make it possible!

13 things I learnt in my first year leading a software team

Jinsy Oommen — Sat, 13 Jul 2019 13:51:33 +0000

About a year ago, I moved into the role of a tech lead on my team. For some background, I work on an embedded software team within a larger Energy Efficiency Consultancy organization. Embedded Teams function very differently than teams in larger software organizations. You will be required to wear multiple hats, process will have to be built from ground up, you'll have to be very mission oriented, and you'll have to learn the business. I learnt a lot along the way, and I am grateful to my team for giving me space to grow and learn.

1. Context is everything

People like being part of a greater mission. One of the ways to reiterate your company's mission is by providing context for everything your team will build. Over-communicate your roadmap, bring up your quarterly painted picture regularly. Connect stories back to the bigger picture. Connect stories to problems it could solve in your business domain.

2. Manage up

As a Tech Lead, you have the opportunity of expanding yourself beyond the stories that your team works on. You have a chance to anticipate business needs or see places that need process improvement. Freely share your learning to your peers.

3. Document your decisions

Decision making is tedious, and you may encounter decision fatigue. Document these decisions and make it freely available to your team.

4. Be selective of your advisors

A lot of the advice for software development and management process is from a perspective of a company providing solutions purely through software. Be aware that sometimes it is harder to scale down advice than scale up. Read all the blogs/books you can, but do reflect on what works best for your team.

5. Be your team's biggest advocate

Get to know your team in a professional setting, through regular 1:1s. Get to know their interests and how you can help them grow their career. Be a coach and a mentor. Recognize the value that each member brings to the team and give positive feedback generously.

6. Provide direction but let your team be autonomous

Don't feel like you have to control every part of the software development lifecycle. Provide a framework where decisions can be made without fear of failure. Things can go wrong, but that's OK. We'll move on and learn from it.

7. Read code and participate in code reviews

You may not be able to contribute to sprint cycles regularly anymore. Keep in the know by reading code that is being written and merged to production every sprint.

8. Do some spikes

You do have the opportunity as a team lead to look at the bigger picture. Go ahead and do some spikes. Try various architectures, so you can help make informed decisions.

9. Engage in customer support

Delight the customer. Put yourself in regular customer service rotations to gain customer perspective and understand their pain points.

10. Broaden your skill set

Now is the time to go broader on your T-shaped skill sets, rather than go deeper. Read generously about developments in the industry across your full stack.

11. Learn about your business

Become a business domain "expert". This will come in handy when you have to provide context to your team.

12. Talk less, listen more

Folks who speak the most benefit the least. Foster innovation on your team through a culture of knowledge sharing and brainstorming.

13. Don't be afraid to fail

Go forth and fail, rise up, and learn. Be a kind, empathetic human being.