Forem: Joost Visser

Dev during pandemic, how are you doing?

Joost Visser — Tue, 06 Apr 2021 12:19:49 +0000

Are devs doing better or worse after a year of pandemic programming?

By Joost Visser and Magiel Bruntink

How are software developers doing, while working at home?

Are you missing the office? The bustle, the perks? Your colleagues? Your boss! Are are you happy to sit quietly at home, where you can pound out code without interruptions? Or are you getting disconnected from your team and its mission?

Developer happiness and developer productivity are known to be interconnected. And both have been impacted by the sudden pandemic-induced switch to fully remote work.

But has this impact been negative only? Or sometimes positive? And what are the factors that developer and their employers can influence to alleviate the bad and accentuate the good effects?

Just after the pandemic hit globally, and governments started to instate lock-downs of varying severity, a team of researchers, led by Dr. Paul Ralph and Dr. Sebastian Baltes, decided to investigate the well-being and productivity of software developers around the globe. What did they find?

“Developers’ wellbeing and productivity are suffering”
“Productivity and wellbeing are closely related”
“Better home office ergonomics help wellbeing and productivity”
“Women, parents, and people with disabilities may be disproportionately affected”
“Different people need significantly different kinds of support”

These findings of the pandemic programming study are now a year old. In the meantime, waves and lock-downs have come and gone, and many of us are still working from home. The experience of software developers a couple of weeks into the pandemic versus one year in might have evolved significantly.

To see if anything has changed after a year of pandemic, researchers from the University of Leiden and from the Software Improvement Group, decided to conduct an independent replication of the original pandemic programming study.

If you are a software developer, impacted to some extent by the Covid-19 pandemic, please take the 10-min survey to tell us how your are doing!

Take the survey here

We are donating $500 to an open source project selected by our respondents. Be sure to vote for your favorite cause to donate to!

The results of the replication study will be available before the summer, and we will publish and link them on this forum and elsewhere.

Read more

3 reasons ML disrupts traditional software engineering

Joost Visser — Fri, 19 Mar 2021 17:08:38 +0000

Photo by Hans-Peter Gauster on Unsplash

Traditional software engineering methods have been designed and optimized to help (teams of) developers to build high-quality software in a controlled and cost-effective manner.

When building software systems that include Machine Learning (ML) components, those traditional software engineering method are challenged by three distinctive characteristics:

Inherent uncertainty: ML components insert a new kind of uncertainty into software systems. While software developers and architects are used to design, build, and test their systems to be able to deal with external factors of uncertainty (network latency, unpredictable user behavior, unreliable hardware), they must now deal with internal components that behave in a non-deterministic fashion. ML components map inputs to outputs in a probabilistic fashion. Take for instance an image-recognition component, that categorizes the input as a cat or a dog, with a certain level of probability, rather than having a crisp outcome.
Data-driven behavior: The behavior of ML components is only very partially determined by the logic that a programmer writes. Instead, behavior is learned from data. This huge dependence on (large volumes of) data, already early in the development stage, changes the development and deployment processes in fundamental ways. Data cleaning, versioning, and wrangling become essential parts of the development cycle. Also in the deployment stage, new challenges arise, such as the need to monitor the (statistical) characteristics of production data versus training data.
Rapid experimentation : Development of ML components is strongly experiment based, where different ML models and different sets of parameters are attempted and evaluated in rapid succession and often in parallel, in order to continuously optimize behavior. This puts the iterative nature of agile software development in over-drive. Where sprints may take 2 to 3 weeks, ML experiments are sometimes initiated, evaluated, and then discarded or adopted within hours.

Researchers in the field of software engineering have begun to study the impact of these challenges. In the Software Engineering for Machine Learning (SE4ML ) project, we have taken the approach of creating a catalog of engineering practices employed by ML teams and advocated for by ML engineering practitioners and researchers.

So far, our catalog consists of 45 engineering practices, in 6 different categories, ranging from data management, through model deployment, to governance. We also provide data on how these practices influence team effectiveness, software quality, traceability, and various requirements for trustworthy AI.

Do you want to know how you and your team are doing on those practices?Take our 10-minute survey. The survey is anonymous, but if you leave your contact information in the last question, we’ll get back to you with a team benchmark report.

➽ Take the 10-minute survey: https://se-ml.github.io/survey

Joost Visser

@jstvssr

How do teams that build #MachineLearning applications deal with versioning, deployment, test automation, bias prevention, privacy mechanisms, audit trails, and other engineering and governance practices?

Tell us how your team works.

A 10min survey: se-ml.github.io/survey/

13:37 PM - 01 Oct 2020

Why do ML engineers struggle to build trustworthy ML applications?

Joost Visser — Tue, 16 Mar 2021 17:36:33 +0000

On the gap between theory and practice and how to close it

By Alex Serban and Joost Visser

While research on the Ethics of AI increased significantly in recent years, practitioners tend to neglect the risks coming from inequity or unfairness of their ML components. This is one of the insights of the annual State of AI report for 2020 recently published by the Human-Centered Artificial Intelligence group from Stanford University.

Why does this gap exist between theory and practice, and how can it be closed?

In our own research (SE4ML), we have approached this question from several angles, specifically for the major AI sub-area of machine learning (ML).

On the one hand, we studied ML engineering practices and their level of adoption among ML engineering teams. On the other hand, we studied the requirements for trustworthy ML formulated by policy makers. Finally, we investigated the relative importance of a range of decision drivers for architects of ML systems.

As we discuss in more detail below, our overall conclusion is that ML engineering teams lack concrete courses of action for designing and building their systems specifically to satisfy trustworthiness requirements, such as security, robustness, and fairness.

Also, we can draw the conclusion that practitioners are aware of the ethical and robustness risks raised by improper development of ML components, but are currently constrained by more traditional concerns such as scaling their systems, or maintaining performance between training and testing.

What are ML engineering practices?

ML engineering is a discipline concerned with developing engineering principles for the design, development, operation and maintenance of software systems with ML components.

In order to facilitate the adoption of engineering principles by practitioners, we have systematically distilled actionable engineering practices from interviews with practitioners and a broad review of practitioner blogs and academic literature. The resulting practices descriptions are catalogued on our website.

Moreover, we measure and publish the adoption rate of these practices and their perceived effects , in order to allow assessment and comparison of teams of engineers building software with ML components.

If you are member of a team that builds software systems with ML components, you can contribute to our research by taking our 10-minute survey on ML engineering practices.

Trustworthy ML

Following recent interest by policy makers to address improper use of machine learning (ML), we made efforts to extend our catalogue of practices to include operational practices for trustworthy ML.

In particular, we followed the requirements for trustworthy ML developed by the High level expert group on AI from the European Commission, which defines trustworthy AI as being lawful, ethical and robust, and defines seven key requirements for trustworthy AI.

Inspired by these requirements, we searched the literature for engineering practices that can be directly applied by developers to address ethical and robust development of ML components. In total, we identified 14 new practices, which include topics such as testing for bias, assuring security, or having the application audited by third parties.

For all new practices, we summarized the related work into a body of knowledge that follows the same structure as the previous practices from our catalogue, including detailed description, concise statement of intent, motivation, applicability, related practices, and references.

Adoption of practices for trustworthy ML remains low. These 14 practices for trustworthy ML are defined in detail in our ML engineering practices catalog.

We also extended our survey to measure the adoption of trustworthiness practices by teams developing ML solutions. Unfortunately, we found out that the practices for trustworthy ML have relatively low adoption (as can be seen in the figure above).

In particular, practices related to assuring security of ML components have the lowest adoption. The contributing factors to these results are diverse. For example, most defenses against adversarial examples — a known threat to ML components — have been breached. Moreover, research into data poisoning attacks shows that only a small percentage of training data needs to be altered in order to induce malicious behaviors. We believe the lack of “off the shelf” solution to security issues of ML components is the largest contributor factor to our results.

Architecture of ML systems

In a parallel study, in which we investigated how practitioners define the software architecture of systems with ML components, we asked practitioners to rank the most important decision drivers for their systems.

Once again, we found out that decision drivers related to trustworthy ML, such as Security, Privacy, or Robustness, were considered less important than more traditional drivers such as Performance or Scalability (as can be seen in the figure below).

Architectural decision drivers for systems using ML components. Drivers related to trustworthiness tend to be considered less important.

These results entail that practitioners are still working on solving basic concerns for developing and operationalizing the ML components, and tend to neglect the importance of trustworthy ML requirements.

Outlook

We believe that consistent efforts to address ethics and robustness through the eyes of software engineering will enhance the ability of practitioners to prioritize these requirements and develop trustworthy ML components.

Moreover, we believe that the adoption of trustworthiness-specific and general ML engineering practices is interconnected. For instance, the practice of continuous integration can make the practices for bias testing more effective.

Therefore, we are working on defining sets of traditional engineering and trustworthiness related practices that can be directly applied by practitioners to develop more ethical and robust ML components.

Learn more about the practices for trustworthy ML by reading our publication.

Alex Serban is PhD candidate at Radboud University and guest at the Leiden Institute of Advanced Computer Science.

Joost Visser is professor of Software and Data Science at Leiden University.

In the SE4ML project, we investigate how software engineering principles and practices need to be adapted or complemented for software systems that incorporate ML components.

If you are member of a team that builds software systems with ML components, you can contribute to our research by taking our 10-minute survey on ML engineering practices.

Software engineering best practices for machine learning

Joost Visser — Wed, 05 Feb 2020 11:10:22 +0000

An awesome 😎 reading list

The awesome Livraria Lello in Porto. Photo by Ivo Rainha on Unsplash.

An ever-increasing number of organizations are developing applications that involve machine learning components. The complexity and diversity of these applications calls for software engineering techniques to ensure that they are built in a robust and future-proof manner.

Do traditional software engineering practices apply equally to the development of applications with ML components, or do these practices need to be adapted to cater for particular ML characteristics?

To investigate this question, we have reviewed both scientific literature and popular publications to identify software engineering best practices that are recommended or used by teams that develop machine learning applications.

Awesome literature

Before diving into the best practices that we identified, let me share some early observations about the literature we collected. For more details, you can take a look at the awesome reading list that we compiled.

Even though the surge of interest in machine learning started relatively recently, there is already a large number of articles, blogs, white papers, etc. readily available. Out of everything we encountered, we selected close to 40 articles to include on our recommended reading list.
Very few articles cover the entire lifecycle of developing and deploying machine learning applications. Only 3 qualified for our category of Broad Overviews.

Categories of literature on Software Engineering for Machine Learning applications.

The scientific literature on the topic of software engineering practices for machine learning is still fairly limited. Only about half of our recommended readings are scientific articles.
Among authors, there is a lot of variation in how they describe the development process of applications with ML components. There seems to be no clear consensus on the names of steps in the process and the boundaries between them. That said, some broad categories of activities can be recognized. We divided the selected literature into the categories of (1) Data Management , (2) Model Training , and (3) Deployment and Operation , depending on the activities in their scope.
Additionally, we found valuable articles on how to organize teams and projects involved in development of ML applications. We grouped those into the category of Social Aspects. So far, we identified only one scientific article for that category.

Best practices

Throughout the collected literature, we identified about 40 best practices. These concern for instance data versioning, feature testing, model deployment, and model performance measurement.

We are currently in the course of creating clear descriptions of these best practices, documenting their literature sources, and organizing them into development process stages.

We are also investigating to what extent these best practices are actually adopted by practitioners. For this purpose, we are engaging with practitioners through interviews as well as an online survey. This will allow us to measure adoption, but also to identify common challenges for practitioners and researchers in the field.

Help us

If you are involved in developing a machine learning application, you can help us by taking the survey and inviting your colleagues and friends to do the same.

Take the 10-minute survey!

As our research progresses, we will be sharing our results right here.

Joost Visser is professor of Software and Data Science at Leiden University. Previously, Joost held various leadership positions at the Software Improvement Group. He is the author of numerous publications on software quality and related topics.

Joint work with Alex Serban, Holger Hoos, and Koen van der Blom.

Are we getting better at software development?

Joost Visser — Thu, 03 May 2018 14:06:31 +0000

Our benchmark of development best practices shows piece-meal improvement

Following best practices or typing away blindly?

Deployment automation is quickly gaining ground. Backlog grooming is being perfected further. Code quality control and automated testing are improving, but still not fully adopted by most teams. These are just 3 of the lessons we learned from the yearly re-calibration of our development best practices benchmark.

Read on for some interesting stats and what they tell us about where software development is heading. Bonus at the end: a quick self-assessment to benchmark your own team.

What are development best practices?

For over 15 years, my colleagues and I at the Software Improvement Group (SIG) have been in the business of evaluating the quality of code, architecture, and development practices for our customers.

We started out with just giving an expert judgement, based on what we observed and what our own software-development gut told us.

As we collected more observations and gained more experience, we decided to whip out our scientific skills (🎓) and build a structured evaluation model. The model collects a series of checks on which development practices are applied by a given team, but also sets thresholds for when teams are applying those practices fully, partially, or not at all. We have now used this model for a number of years to provide teams with objective feedback and comparisons against peers.

Ten best practices for effective software development. We described our structured evaluation model for software development best practices at length in the @OReillyMedia book “Building Software Teams”.

Annual updates of the evaluation model

About once per year, we update our benchmark repository with the observations and evaluations we have collected throughout the past year and use this additional data to adjust the model.

This annual update is also an excellent moment to study the data and look for trends. Are we getting better at software development? We just finished the latest calibration, and here is what we learned about trends in development best practices.

Lesson #1: More teams apply deployment automation

We measure the practice of deployment automation by looking at whether teams have a deployment process in place that is quick, repeatable, and preferably fully automated.

For example, a team that deploys each new release with a single push of a button would receive a perfect score on this practice, codified as fully applied. But a team that needs to go through a limited number of well-documented manual steps would be scored as partially applied.

More teams fully apply deployment automation (43%, was 30%) and fewer teams do not apply deployment automation at all (11%, was 26%).

As the chart shows, more teams are applying deployment automation practices fully (43%, was 30%) and fewer teams do not apply deployment automation at all (11%, was 26%).

This is a significant positive trend that is mirrored by the trends in continuous integration (automatic compilation and testing after each change) and continuous delivery (automatic deployment after each change), as shown in the following charts.

Full or partial adoption of continuous integration (currently 68%) has significantly improved, but still lags behind compared to deployment automation (currently 89%). For continuous delivery, adoption has also improved significantly, but still has a long way to go (currently 29%).

Though the trends for these two practices are equally positive, their adoption still lags behind. And especially for continuous delivery, the great majority of teams (and the organisations of which they are part) still have a long way to go.

Lesson #2: Almost all teams groom their backlogs

Nearly all teams (95%, was 92%) maintain product and sprint backlogs and a significantly larger portion of teams applies this best practice fully (80%, was 71%).

The best practice of backlog grooming already enjoyed high adoption, with 71% of the teams diligently maintaining both Product and Sprint backlogs, and 22% doing so at least partially. As teams perfected their backlog grooming, full adoption increased to 80%. Only a small percentage of teams (5%, down from 8%) does not do any backlog grooming at all.

In fact, most Agile-Scrum best practices that we assess showed improvement, or stable high adoption. With one small exception:

More teams do not stick to the discipline of holding all meetings prescribed by Scrum (15%, up from 11%).

As the chart shows, fewer teams seem to stick to the discipline of holding meetings prescribed by Scrum (planning, daily standup, review, retrospective). This may not be a bad thing per se, as more experienced teams are encouraged to adapt their meeting rhythms to their own needs.

Lesson #3: Code quality control and testing are improving

Fewer teams are failing to enforce consistent code quality standards (20%, down from 25%). Fewer teams fail to run automated test at each commit (41%, down from 48%).

Quality control is an essential part of professional software development. Nevertheless, the best practices of code quality control and automated testing are still only fully applied by a minority of the teams.

To assess code quality control, we observe whether a team works with clear coding standards, systematically reviews code (against these standards and against principles of good design), and whether automated code quality checks are performed. Full adoption of code quality control has somewhat increased (31%, up from 23%), but still 20% of teams are producing code without adequate quality control in place.

To assess testing practices, we observe whether teams have an automated regression test suite that is being executed consistently after each change. Full adoption of this practice is increasing (33%, up from 29%). The percentage of teams that change their code without proper regression testing is declining rapidly but is still a staggering 31% (down from 48%).

Getting better?

So our question: Are we getting better at software development? can be answered: Yes, but at a modest pace.

For some practices, the needle doesn’t seem to have moved too much over the past year (e.g. documenting just enough, managing third-party components, using SMART requirements and goal-oriented metrics). I won’t bore you with the flat-lining charts.

We do see significant progress on a number of practices, especially deployment automation, continuous integration, code quality control, and automated testing. This is incredibly good news!

But we’re not there yet. Personally, I’m a bit shocked that less than 1 in 3 software development teams follow quality and testing best practices, since adopting these best practices can bring immediate benefits with limited effort.

Less than 1 in 3 software development teams follow quality and testing best practices

What can you do?

Assess your own team: If you’d like to learn more or would like to have your team assessed by us, drop me a line. To do a quick self-assessment, you can take this survey
Stay up to date on software quality trends : If you are interested in learning more about longer-term trends of more software quality aspects, you are in luck. A 10-year retrospective of software quality by SIG is forthcoming. Follow me here on Medium or on twitter if you want to be notified when it comes out.

Joost Visser is CTO at the Software Improvement Group (SIG), Professor of Large-scale Software Systems at Radboud University, author of O’Reilly books Building Maintainable Software and Building Software Teams , and leading the Better Code Hub team at SIG.

Thanks to Lodewijk Bergmans for crunching and charting data!

Writing quality code under time pressure?

Joost Visser — Thu, 12 Apr 2018 10:59:51 +0000

These hackathon teams prove it can be done

Last weekend 63 teams gathered in the old Sugar Factory in the Dutch city of Groningen for the world’s largest Blockchain hackathon. Here is what we learned from these teams about writing high-quality code under time pressure.

Quality under pressure?

Anyone can code. There are no formal entry criteria. An inquisitive mind and a laptop is all you need.

But writing quality code can be hard. Each programming problem can be solved in many different ways, some clean and straightforward, many muddled and messy. To turn an initial working solution into an understandable, adaptable, testable, reusable nugget of code takes time and dedication.

But most programming happens under time pressure. Pressure to ship a feature, pressure to squash a bug. Users, clients, testers, product owners, and team mates are waiting for you to close your ticket and move on.

And with the clock ticking, most teams find themselves compromising on quality.

Compromising on quality

When compromising on quality is a conscious decision, we may call it “technical debt”: taking out a loan to be paid back through future refactorings.

Conscious or not, when we compromise on code quality, we rarely come back at a later stage to mend our ways. Rather, we double down. Struggling to understand existing code, we add new code that is even messier. The “broken-window syndrome” also applies to software.

And so begins a vicious cycle. Once quality decreases beyond a tipping point, it becomes infeasible to recover. No time, no budget, no support for a prolonged feature-freeze to allow complex, multi-sprint refactorings.

“Yes. I hold that once we slack on quality, we’ll never get it all the way back. We’ll be slower forever by some amount.” — Ron Jeffries

In the end, coding (too) fast is self-defeating. Lower quality code is harder to maintain and extend. Bugs take longer to find and solve, and often re-appear. Adding or changing a feature becomes difficult and error-prone.

Can we maintain speed AND quality?

Yes. There is a way to code fast without compromising quality. In fact, the only way to code at a sustainable speed is by not compromising on quality. Here are the essential ingredients for doing just that:

A definition of done for code quality — a set of clear guidelines that the team embraces to decide when code is of high enough quality to merge and move on. We defined 10 such guidelines in the O’Reilly book Building Maintainable Software.
An immediate feedback mechanism that determines whether a piece of code conforms to the guidelines. And provides us with actionable information on how to quickly regain compliance. We created such a feedback mechanism as a GitHub integration called Better Code Hub.

With these elements in place, quality deficits can be flagged as they are introduced and nipped in the bud. A tiny investment in quality at each commit that pays back 100-fold in all the commits that come after. A drip of oil to keep the machine spinning without friction.

Quality code at the world’s largest Blockchain hackathon

When it comes to coding, the ultimate pressure cooker is a hackathon. Build a working piece of software within a day or two. You are on the clock. You are inventing as you go. Ideas are raw, teams just met, solutions are still unexplored.

There is a huge temptation to just slap some code together without ever looking back.

So, what better opportunity than the world’s largest Blockchain hackathon, to find out whether quality code can be produced under the pressure of time. Last weekend 63 teams gathered in the old Sugar Factory in the Dutch city of Groningen for the Blockchaingers Hackathon 2018. Coding started at noon on Friday and ended 48 hours later at noon on Sunday. A competition among teams, a race against the clock, an intense struggle of each developer with the complexities of mind and machine.

“When coding for the future, write future-proof code”

Supporting quality

Here is what we did to support the teams in producing high-quality code:

Before the Hackathon started, we gave the teams a 1-hour Masterclass on code quality, explaining the relevance, introducing basic concepts, presenting the 10 guidelines for writing future-proof code from Building Maintainable Software, and showing them how to use Better Code Hub to keep an eye on compliance with those guidelines.

Each push and pull request is checked by Better Code Hub against the 10 guidelines for writing future-proof code from Building Maintainable Software. Feedback is provided in the GitHub conversation flow. When quality has decreased, click-through to detailed feedback per guidelines for concrete pointers on how to improve.

During the Hackathon, Better Code Hub provided feedback about the impact on the guidelines each time code was changed. In all, such feedback was given on about 4000 contributions. The current scores of all teams were displayed on a huge screen for all to see. Also, code-quality Jedis were present to assist the teams in person or through Slack with activating Better Code Hub, asking questions about its usage, and sparring on refactorings, design patterns, testing approaches, or any other topic related to software quality.

The current code quality scores for all teams were displayed as GitHub badges on a central scoreboard for all to see.

Quality results

So now the “Stop Coding!” alert has sounded, the winners have been awarded, the Hackathon is over, and the dust has settled. What was achieved in terms of code quality?

19 out of 63 teams delivered a prototype that scored 10 out of 10. Wow! These teams prove that delivering high-quality code under intense time-pressure is possible!

19 out of 63 teams (30%) delivered a prototype that scored 10 out of 10, proving that writing high-quality code under time pressure is possible.

During the hackathon, many teams used the feedback from Better Code Hub, not only to sustain their initial level of quality, but to make quality improvements.

During the 48-hour hackathon, many teams were able to raise their scores above their initial level.

Delivering quality code is fun and rewarding. Look at how these teams show off their scores on their Impact canvasses!

Some of the Impact Canvases showing Better Code Hub scores.

Lessons

So here are a couple of important lessons that I learned from supporting the Blockchaingers Hackathon with Better Code Hub.

Code quality control does not need to be disruptive — The feedback that Better Code Hub gives to developers is concise and to the point, and localised close at hand in the GitHub conversation flow. When a green checkmark appears, feel free to move on to your next contribution. When a red cross appears, dive in to get clear pointers on what to improve before moving on.
Code quality control does not slow you down, but speeds you up — By acting immediately on the piece-meal feedback that Better Code Hub provides, rather than waiting for running into bigger problems later, minimal effort is spent on keeping the code base clean. This effort pays back on every subsequent contribution.
Setting and achieving a clear quality goal is rewarding and stimulating — When the goal is achievable, and progress is observable by all, the team gets loads of positive energy out of quality control.
High-quality code provides a solid basis for post-hackathon acceleration — While some hackathon teams disband and abandon their prototypes, many use the hackathon as a spring-board for a new venture with a new product. High-quality code provides a head start in the right direction.

Accelerate towards funding

Based on code quality scores as well as the overall design quality of the prototypes, our Jedis selected the 7 hackathon teams that did outstanding work on non-functional quality aspects. Security, performance, reliability, maintainability, and whether Blockchain technology was applied in a sensible, future-proof way.

These winning teams were awarded the SIG Investor Readiness Workshop , where we will help the teams prepare their code bases for future funding rounds. Investors are increasingly paying attention to software quality in the due diligence phase of acquisitions. We know, because the SIG Transaction Services team perform so-called IT Due Diligence investigations for investors on a regular basis. In the workshop we help teams to be prepared for that moment.

The 7 teams that built prototypes with outstanding software quality were awarded with the SIG Investor Readiness Workshop. Great job!

Are you organising a hackathon? — Contact me to discuss how to support the teams with code quality.
Are you a startup? — Check out the Better Code Hub Starter Pack .
Are you investing in a software-intensive company? — Get in touch with my colleagues at the SIG Transaction Services team for an IT Due Diligence investigation.
Are you a CTO or CIO who wants his software development teams to go fast, but sustainably so? — Consider empowering them with Better Code Hub.

Constraints that drive creativity

Joost Visser — Wed, 20 Dec 2017 09:39:32 +0000

A different code quality philosophy

Should you measure and control code quality? According to a poll conducted by O’Reilly Media and my colleagues at the Software Improvement Group, virtually all developers say YES! Here’s a number of compelling reasons:

Happiness: Better code and happy developers go together. It’s more fun to work on code that is good. In fact, bad code quality and coding practice were identified as the number 3 cause of developer _un_happiness. In case developer happiness is not sufficient as an intrinsic reason, think of the increased productivity and retention that will follow.

From “On the unhappiness of software developers” by Daniel Graziotin, Fabian Fagerholm, Xiaofeng Wang, and Pekka Abrahamsson published and presented at the EASE 2017 conference.

Speed : Good code is easier to understand, easier to adapt, easier to test, and easier to reuse. By definition. Quickly write up some bad code now, and fixing bugs or adding features will take up more time in future. Or in other words: “Current development speed is a function of past development quality”.

Brian McCallister

@brianm

Current development speed is a function of past development quality.

18:31 PM - 10 Aug 2016

858 685

Agility : Being agile in software development is not just about processes and best practices, but also about making frequent changes to your code. To accommodate and ship the next feature. Or to respond to changing requirements. Or because you learned how to express your intentions better. High-quality code allows you to make these changes frequently and effortlessly.
User satisfaction : When developers are happy, fast, and agile, they can build the best possible products. With the functionality, performance, reliability, and other quality characteristics that you users expect and appreciate. So ultimately, better code means better business.

Most teams do not properly control code quality

So code quality brings compelling benefits. But now the shocker. Not all professional software development teams actually measure and control code quality in a professional manner. In fact, when assessing software development practices across many organisations over the past several years, we found that about 83% of the teams do not control code quality at all or only do so partially.

From “Building Software Teams” (2016)

Why not?

What are the main reasons cited for not applying code quality control? What is wrong with the tools that are available for this?

False positives and lack of actionable recommendations: Unfortunately, many code quality tools flood their users with a large number of violation alerts of which only a small percentage is valid and actionable . As a result, the output of these tools is often ignored.
Too costly and too difficult to configure : To improve the validity and actionability of their output, some tools offer extensive configuration options. They also come with the possibility to extend the standard behaviour by supplying rules sets that are specific for your team or your product. Such configuration and customisation may seem like a cool and advanced feature. In fact, they may be advertised as unique selling points, justifying higher license fees. But in fact, it means you need to invest valuable time of senior developers in learning and maintaining these tools that were not cheap to begin with.
No clear standard, across teams and technologies : Another disadvantage of customisation is that it undermines standardisation. When each team or developer uses his own set of quality rules, or even different sets of rules for different programming languages, then how can we as developers aspire to the same level of quality? Software development is a socio-technical discipline, just like driving. In traffic and in development, we can prevent accidents and travel fast, only when we all stick to the same rules of engagement.

A different code quality philosophy

At the Software Improvement Group, we set out to resolve this apparent conflict between the (economic) necessity of controlling code quality and the poor adoption of code quality tooling. To do so, we needed a different perspective on code quality.

Our key insight was that we need to raise the level of abstraction from individual code quality violations, that are observed in particular code locations, to quality profiles that characterise an entire code base or a significant portion thereof.

The simplest example of such a quality profile is “the percentage of code lines in methods longer than 15 lines of code”. This quality profile can characterise the readability of a single class, a package, or an entire code base. A lower percentage means higher readability. While grounded in quality measurements in specific code locations (methods in this case), the profile does not pinpoint any specific location as a “violation”. Instead, we can formulate quality norms at a higher level of abstraction, for instance by demanding that said percentage does not exceed e.g. 20%.

Quality profiles allow to set crisp, actionable thresholds (e.g. 15) for measurements that are highly relevant to code quality (e.g. method length). But they also come with an intrinsic concept of tolerance where a fair amount of code is allowed to surpass the threshold (e.g. 20%), but the developers retain freedom within those limits to decide which violators to accept and which to refactor.

Ten Guidelines for Future-Proof Code

Based on this philosophy, we formulated Ten Guidelines for Future-Proof Code.

Together, the ten guidelines for writing future-proof code from “Building Maintainable Software” provide a Definition of Done for code quality that is ambitious, yet achievable.

Note that these are guidelines , not rules. A rule is binary, drawing a line between right and wrong. Compliance or violation. But a guideline merely provides direction. A guideline allows choice within a clear boundary. In software development, as in art, well chosen constraints do not limit, but enhance creativity.

“Creativity is driven by constraints” — Dave Grey

We calibrated both thresholds and tolerances in these guidelines against code analysis results from hundreds of software systems. Together the 10 guidelines provide a crisp definition of done for code quality. A definition of done that is ambitious, but not overly restrictive due to intrinsic tolerance. And we built the Better Code Hub to check those guidelines for you on your code base.

Better Code Hub

The purpose of Better Code Hub is to show that a different approach to code quality control is possible. An approach that is less cumbersome, less restrictive, and more fun. Here is what it does for you:

Provide a crisp definition of done , that is independent of technology, team, or product. A single standard for code quality.
Easy adoption and immediate feedback. Through the GitHub Marketplace (free for open source, start-ups, and educational use) you can run Better Code Hub on your code within minutes. And by flipping one toggle, you activate feedback on each push and pull request.
Refactoring assistance : Better Code Hub tells you not only which guidelines your code does and does not comply with. It also provides you with refactoring candidates. And when you make a selection of candidate to refactor, you get a real-time prediction of how your guidelines compliance will be impacted.

So, if your team is not measuring and controlling code quality yet, this is the time to start. Give Better Code Hub a try. It will be fun!

Joost Visser is CTO at the Software Improvement Group, Professor of Large-scale Software Systems at Radboud University, and author of O’Reilly books “Building Maintainable Software” and “Building Software Teams”.

How a Hackathon appreciates quality code

Joost Visser — Wed, 08 Mar 2017 11:27:30 +0000

Some lessons from the world's largest Blockchain hackathon

Image: Robin Knaapen of team DC Cicero shows off their 10 out of 10 score on code quality at the worlds largest Blockchain hackathon. Image courtesy of Drenthe College.

Last weekend, the world's largest Blockchain hackathon took place in The Netherlands. During 54 hours, 350 participants, organised into 55 teams, competed in five tracks, on one grid, and with zero time to lose.

Also, these teams took responsibility for the quality of the code they were hacking together by executing 2000 code quality checks.

Here is how we made those 2000 checks possible, and how we made sure they helped to accelerate the teams.

An appreciative approach to judging

The hackathon organisers decided to take what they call an “appreciative” approach to the judging process. In their words:

we will NOT have a beauty contest and judge you afterwards based on smooth sales pitches or your shiny smile

Rather, they facilitated a holistic approach, throughout the event and beyond, in four stages: (1) support, (2) reward, (3) award, and (4) accelerate. A key instrument in this setup is the “Impact Canvas” on which the teams share the wealth of their projects on multiple dimensions.

The Impact Canvas template. Space for sharing all dimensions of a great hackathon project. Kudos to Ritzo ten Cate for driving this approach.

Using the canvas, the teams share and discuss with supporters, Jedi, accelerators, and reporters, taking them along on their journey, engaging them, co-creating with them.

How about code quality?

One of the dimensions of a great hackathon contribution is the quality of the solution, including the code. But how can teams collaborate on code quality and share their code quality achievements with others?

For this, we gave them the Better Code Hub:

We gave the team 10 guidelines for building maintainable software. The guidelines were presented to them in a pre-hackathon meet-up and they received copies of our O'Reilly book Building Maintainable Software that describes the guidelines. Together, the 10 guidelines provide a Definition of Done for code quality that is challenging, yet achievable and fun.
We gave the teams free access to the Pro version of Better Code Hub, allowing them to check their code against the 10 guidelines and giving them actionable feedback on what changes to make in the code for meeting the guidelines. Teams can head over to www.bettercodehub.com to get feedback within minutes, and they can turn on Push&Pull support to get feedback on every commit.
A team of software quality Jedi were available during the event to help teams use the Better Code Hub and for counseling them on anything related to code quality. Which refactoring should I apply? How can I deal with dependencies on third party code? How can I make my code more testable? The code quality Jedi's have answers.
Before code-freeze on the last day of the Hackathon, all teams were given the opportunity to have their code quality results validated by the Jedi, who provided them with swag and badges and encouraged them to share their code quality score on their Impact Canvas.
After the event, the teams retain access to the Pro version of Better Code Hub for another 12 months, as well as continued support from the Jedi on anything related to code quality. The hackathon is a great learning experience for putting together a complete application, including third-party components and build pipeline, but it's only the first step towards a shippable product.

This is what code quality looks like on some of the Impact Canvases:

Code quality achievements on the Impact Canvas of sevderal teams at the Dutch Blockchain Hackathon.

Digitally, teams showed off their code quality prowess with GitHub badges that were collected on a scoreboard:

The Better Code Hub badges were collected on a central scoreboard.

So what is the impact of code quality during a hackathon?

We chose not to just measure code quality and feed measurement results to the jury. That would run counter to the “appreciative approach” and have the wrong effect. Instead, we empowered the participants to get code quality right, in a uniform manner, enabling them to show off their code quality achievements as one of the many dimension of their achievement. The effects are as follows:

Uniting teams: with a clear definition of done for code quality, teams don't need to waste time quibbling about coding style. Also, they get to celebrate together, each time a code check shows improvement.
Faster coding: teams that build high-quality code from the start can keep up their coding speed throughout the hackathon. Also they can more easily handover code within the team. No spaghetti to get entangled in.
Appreciation for code: Hackathons should be about code, as much as they are about learning, team work, and brilliant ideas. Making code quality easy to measure, easy to share, and easy to reward contributes to give code the visibility it deserves.

So, if you are organising a Hackathon, we can recommend an appreciative approach, including an empowering approach to code quality. If you want to learn from our experiences at the Dutch Blockchain Hackathon, we are happy to help.

Joost Visser is CTO at the Software Improvement Group, Professor of Large-scale Software Systems at Radboud University, and author of O'Reilly books “Building Maintainable Software” and “Building Software Teams”.

Why Blockchain needs Future-Proof Code

Joost Visser — Wed, 18 Jan 2017 10:52:04 +0000

Because our society needs solutions that are innovative and sustainable.

I’m not a great programmer; I’m just a good programmer with great habits. — Kent Beck> Quality of code may not guarantee the success of a project, but it can definitively be its main, though invisible, cause of failure. — Mario Fusco

The Blockchain Hackathon

In the second weekend of February, 50 teams will compete in the Dutch Blockchain Hackathon. Over a 54-hour stretch, they will unleash their creativity to build innovative blockchain-based solutions that may shape the future of our society.

The teams will compete in 5 thematic tracks, vying for praise and recognition by the jury, and for €50,000 in prize money.

The jury will be looking for quality, impact and contribution to the blockchain ecosystem. Teams can earn bonus points by using open standards and sharing their code through open source licenses. And by producing high-quality code.

Why is code quality important?

So why is code quality important in this hackathon? Are hackathons not intended to hack things together until they work? Not to produce something polished and future-proof?

Well, here are some reasons code quality is not a luxury.

Blockchain is based on trust. Or, as Ravish Gopal explains, with Blockchain, “the conventional trusted third party mechanism is being replaced by software”. And trust requires auditability. Better code is more auditable.
Innovative solutions, whether built by startups or established organisations, must be adaptable, extensible, and transferable. Otherwise the solution will not outlive the startup founders and/or the first team that works on it.
In general, better code allows you to build better products. Rather, as argued by Rob van der Leek, great products are not built by great individuals, but by great teams. And good code is both the hallmark of a great team and a necessary precondition for their successful operation as a team.

Well, you may say, these reasons only kick in after some time, not before my hackathon is over. So let’s narrow our focus to the first 54 hours of your team’s project. Why is code quality relevant?

Lunch : Good code starts paying back before lunch on the first day. By that time you are no longer in your first iteration. You are modifying and enhancing the code you wrote in the morning. If it’s messy, hard to read, untested, bloated, and so forth, then you will slow down, inject defects without noticing, and lose your way.
Dinner. Bad code makes your team hate you before dinner. The code you wrote looks weird and alien to your teammate that wants to build on it. Before adding to it, she needs to refactor. You don’t like what she does to your code. Rather than complementing each other you enter into a turf war.
36-hours in. You just had your breakthrough idea that will set you apart from the competition and impress the jury beyond belief. Except that changing your code to implement this idea seems like jumping off a cliff. Your code is a house of cards. Should you refactor, reorganise, make the required changes? Or rather start over. Time is ticking.
Win. You pulled it off. Your solution works, it wows the jury, and you’re are invited for the post-hackathon accelerator track. But wait! Though it works, you don’t know why or how. Some of your team members won’t hang around after the hackathon, but their code is a mystery to the others. In a way, you still come out of the event empty-handed.

Why code quality matters in a Hackathon

Can code quality be measured?

Given the importance of code quality for teams to build great products, we need a shared definition of what code quality is and how it can be measured.

An abstract definition of code quality can be found in the notion of maintainability as provided by the ISO 25010 standard on software product quality:

maintainability = degree of effectiveness and efficiency with which a product or system can be modified by the intended maintainers, where modifications can include corrections, improvements or adaptation — ISO 25010

We have operationalised this abstract definition in a concrete set of measurements, complete with thresholds and aggregation to a simple rating scheme (see “A practical model for measuring maintainability”, IEEE Computer Society Press, 2007).

In turn, this measurement model has been the basis for 10 guidelines for the working programmer, published in “Building Maintainable Software” (O’Reilly, 2016).

The 10 guidelines for writing future-proof code from “Building Maintainable Software”.

At the Software Improvement Group, we have been using and improving our maintainability model over a decade to measure billions of lines of code and provide hundreds of executive clients with fact-based advice on how to manage their software projects and application portfolios.

So, yes, code quality can be measured.

Can you measure code quality yourself?

Plenty of tools are available for measuring code quality. But they come with pitfalls. They often measure the wrong things by default and require intricate configuration, and they flood the developer with false positives, i.e. warnings that don’t reveal any real problem (for a more detailed discussion, see my success criteria for applying software analysis tools effectively).

To avoid these pitfalls for developers, we have developed the Better Code Hub. It provides developers with immediate feedback on the 10 guidelines. We designed Better Code Hub to be extremely easy to use, provide a clear definition of done, and empower teams to improve what matters.

The guidelines set crips thresholds, but they also provide tolerance for the occasional outlier. As a result, the definition of done provided by Better Code Hub is challenging, yet achievable and fun.

Code quality at the Dutch Blockchain Hackathon

The teams participating in the Dutch Blockchain Hackathon are stimulated and supported to innovate with high quality code. Support includes:

Books. Free copies of Building Maintainable Software will be distributed to the participants.
Better Code Hub. The teams will get 12-month Pro access to the Better Code Hub.
Assistance. During the event, a team of SIG consultants will help participants use the tool, and coach them on code quality.
Jury. The measurement results of Better Code Hub will be made available to the jury, allowing them to award bonus points for higher quality.

Is code quality relevant beyond this hackathon?

Software is the DNA of our digital society. So yes, code quality is important for all of us. Competitive power and business agility depend on mastery of software.

If you are not participating in the Hackathon, but you are interested in measuring and managing code quality, here are some options:

If you are a start-up , we have a complimentary Pro license for Better Code Hub for you as well (conditions apply).
If you are an open-source developer, you can use the Free license on any public GitHub repository, within reasonable size limitations.
If you are an IT director, development team lead, or similar , let me know about your challenges regarding code quality, developer competencies, team empowerment, software governance, application portfolio management, etc. At the Software Improvement Group, we know what it takes to be successful in both empowering self-organising teams and applying effective governance over your in-house software development, your suppliers, and your IT landscape.

Software: more and faster, but how?

Joost Visser — Mon, 12 Dec 2016 11:12:14 +0000

Sustainable business requires mastery of software

Software is the DNA of our information society

We rely on software in everything we do. Travel, payment, entertainment, health care, trade, manufacturing, education. Nowadays, nearly every human activity involves software. And, just as DNA predetermines how we interact with our environment, so does software determine how objects move and how information flows.

If software is the DNA of our modern information society, does it evolve autonomously or under our control? See also “How does your software Measure Up? — Mastering the DNA of our information society”

Software needs to evolve in order to survive

The needs of users change. Platforms change. Regulations change. Software needs to adapt. Increasingly, software is the differentiator in a competitive marketplace, or a prerequisite to enter into a market in the first place. Businesses that are not able to rapidly create new software solutions or adapt existing solutions are unable to stay relevant and compete.

Software is eating the world. And competition will eat those businesses that do not master their software.

Automation and ecosystems are key trends

Software developers try to meet this need for producing more software and for evolving software faster. Two important current trends support increased automation and reuse in software development:

Automation : Modern software development teams strive to automate as much of the software development process as possible. Examples are continuous integration, automated testing and deployment, and low-code environments (check with Jeroen Heijmans for a balanced view on Low Code trends). Automation helps to increase development speed, limit knowledge dissipation, and build quality into every step.
Ecosystems : Rather than building software systems from the ground up, modern teams increasingly create new systems by combining partial solutions from third parties. Sophisticated new functionality is created with a limited amount of code, gluing together libraries, frameworks and external web services. Open platforms unleash the creativity of partners, customers, and user communities.

Looming disruption

As development tools and third-party components are widely available against minimal costs, the software marketplace becomes more and more competitive. Start-ups, technology vendors, and community-driven open source initiatives frequently launch new and improved solutions that may disrupt incumbent businesses at any moment. Established businesses risk being slowed down by regulations, security requirements, technical debt, legacy code, outdated technologies, and architectural complexity, and to realise their Digital Transformation ambitions they need to plan their escape from Legacy Mountain.

[Webinar] Escape from Legacy Mountain - Software Improvement Group

Two success factors for mastering software

Organisations striving for sustainable, competitive business must master the creation and evolution of software. Highly productive and predictable software development management relies on two success factors:

Product over process : Successful software management keeps a clear view of what ultimately matters most: high-quality software products. Too many organisations attempt to control software development only through process, including extensive documentation, formal methodology, decision and reporting structures.
Pragmatic measurement : Successful software management makes effective use of meaningful measurements of both products and processes. Too many organisations rely mostly on intuition, hearsay, and untested assumptions to assess the status of their products and the effectiveness of their processes.

When managers and developers have a clear, quantified view of the state of their software products at any moment in the software production lifecycle , they are able to correct defects as they are introduced, modify designs as needed, and adapt through controlled iterations to business needs as they evolve. Measurement of all stages in the software production pipeline allows incremental improvement of the degree of automation, reuse, productivity, and technical quality. Sustainable business requires mastery of software.

Joost Visser is CTO at the Software Improvement Group, Professor of Large-scale Software Systems at Radboud University, and author of O’Reilly books “Building Maintainable Software” and “Building Software Teams”.