Forem: Hopefully Surprising

SonarQube Community Edition: Comprehensive guide for a free personal setup

Hopefully Surprising — Tue, 03 Oct 2023 16:11:03 +0000

When we develop software, we want it to be technically remarkable. It's a common desire to ensure high code quality to benefit both others and your future self. Static code analysers are essential tools that assess your code against predefined rules without running it.

While enterprise-grade solutions for that might be enterprise-level expensive, some of the most advanced solutions - such as SonarQube - can be used for free if all the pieces of the stack are put together by yourself.

In this material, I will walk you through the process of creating personal SonarQube Community Edition instance and setting up the scanner on an example of NodeJS project that I wrote about some time ago (Creating a custom webpack loader for text files, compatible with TypeScript. PlantUML mind map example). In addition to that, I'll share a couple of thoughts about why a particular option is chosen and not an alternative.

Solution

This guide consists of a few parts:

Setting up the stage. Listing system requirements, installing missing dependencies.
Configuring and starting SonarQube CE instance.
Creating a SonarQube project, configuring and running the SonarScanner.
(Bonus). Installing and configuring SonarLint for getting feedback about the code right in IDE.

Setting up the stage

Using Docker as the Runner

We will use Docker for running SonarQube server and SonarScanner.

While it's possible to run the server and SonarScanner from a binary file, I choose the Docker way because of two benefits:

It's much simpler to upgrade SonarQube with Docker: instead of extracting config files, placing binaries somewhere in $PATH again and adding config back, you simply restart the container from a newer image and bind volumes there.
You don't need to have Java in the host machine.

Providing existing database as a storage. Postgres example.

SonarQube needs to store a lot of information about the users, rules, projects, scanning sessions, etc. It uses a connected database instance for that.

SonarQube server supports different databases so we can use this advantage. The key for me is to use the database that you're used to and that is already running in your computer. That helps reduce the footprint of the running SonarQube server - no need to have an entire database instance for a specific use case.

In my case, it's Postgres. I'm not sharing the instructions on how to install Postgres itself but here is the link to official documentation about installing Postgres 12 on Ubuntu. SonarQube's Docker Compose example refers to PostgreSQL 12 so I'm using this version.

When the database is available, we need to create schema, database and the user that SonarQube server will be using for storing data. Below is the Postgres way:

NOTE: Don't forget to replace SONAR_DB_PASSWORD with the real one you'll be using.

CREATE SCHEMA sonarqube;
CREATE USER sonarqube WITH PASSWORD 'SONAR_DB_PASSWORD';
GRANT USAGE ON SCHEMA sonarqube TO sonarqube;
GRANT CREATE ON SCHEMA sonarqube TO sonarqube;
ALTER DEFAULT PRIVILEGES IN SCHEMA sonarqube GRANT ALL ON TABLES TO sonarqube;
ALTER DEFAULT PRIVILEGES IN SCHEMA sonarqube GRANT ALL ON SEQUENCES TO sonarqube;
ALTER DEFAULT PRIVILEGES IN SCHEMA sonarqube GRANT ALL ON FUNCTIONS TO sonarqube;
CREATE DATABASE sonarqube;
ALTER DATABASE sonarqube OWNER TO sonarqube;

Configuring and starting SonarQube CE instance

SonarQube server

Now, when the database and Docker engine are available in the host machine, we can start the SonarQube server container.

Some troubleshooting notes will follow the happy path of the installation.

Create the Docker volumes

sudo docker volume create --name sonarqube_data
sudo docker volume create --name sonarqube_logs
sudo docker volume create --name sonarqube_extensions

Start the SonarQube server container

sudo docker run --detach --name sonarqube \
    --add-host=host.docker.internal:host-gateway \
    --publish 9000:9000 \
    --env SONAR_JDBC_URL=jdbc:postgresql://host.docker.internal:5432/sonarqube \
    --env SONAR_JDBC_USERNAME=sonarqube \
    --env SONAR_JDBC_PASSWORD=SONAR_DB_PASSWORD \
    --volume sonarqube_data:/opt/sonarqube/data \
    --volume sonarqube_extensions:/opt/sonarqube/extensions \
    --volume sonarqube_logs:/opt/sonarqube/logs \
    sonarqube:community

NOTE 1: Don't forget to put the real SONAR_DB_PASSWORD password from the database created earlier.

NOTE 2: If you use different database, put your specific SONAR_JDBC_URL.

Common issues

If the Docker container is not starting, check the logs. The issues I encountered are listed below:

If Elasticsearch complains about the heap size in Linux, update this setting: vm.max_map_count to allow using more memory.
If the Postgres DB is not reachable, you might need to update pg_hba.conf to allow incoming connections for sonarqube user.

Access SonarQube UI

When the container with SonarQube server is running and is available on the port 9000 in your machine (see --publish parameter), you can access https://localhost:9000 in your browser. You'll be invited to log in. The most secure in the world admin - admin are your credentials. You'll need to change it as soon as you log in for the first time.

Running SonarScanner

Generate token for the scanner

Before creating and scanning projects, we need to generate a token that will allow the scanner to upload the results to SonarQube server. For a simple setup, we will create a global token. You can do that via opening the user dropdown in the top right corner of the screen and going to "My Account" -> "Security".

Other options are explained here.

Copy the token and save it in a secure place. You won't be able to see it in the UI ever again.

Create and configure a SonarQube project

In SonarQube dashboard, select "Create Project" and choose "Manually". Input display name and the project key, choose the default branch.

NOTE: For this setup, we will disable versioning so the branch name here doesn't really matter.

For the baseline, you can choose the global settings which is the default SonarQube preset.

In the opened project page, choose "Locally" for scanning setup.

Provide the global token you generated before and click Next.

You will be given the CLI command to run but you can ignore it. We will be running the scanner in a Docker container. Also, we will provide the parameters via .properties file and not CLI arguments.

You can leave the browser page open, it will refresh automatically when the results of scanning were pushed to SonarQube server.

In the root of the project you're going to scan with SonarQube, create sonar-project.properties file with the following content:

sonar.projectKey=plantuml-mindmap-loader

sonar.sources=.
sonar.sourceEncoding=UTF-8
sonar.scm.disabled=true
sonar.exclusions=**/node_modules/**,**/coverage/**,**/lib/**
sonar.javascript.lcov.reportPaths=./coverage/lcov.info

NOTE: This file contains instructions on unit-test coverage because this project has Jest tests. If you don't have it in the project, exclude sonar.javascript.lcov.reportPaths parameter.

Run the scanner

As mentioned above, we will run the scanner in a Docker container. You need to execute the below command with a correct SONAR_GLOBAL_TOKEN and the path to the folder containing the code you scan (this folder should contain sonar-project.properties you created before).

docker run \
    --rm \
    -e SONAR_HOST_URL="http://localhost:9000" \
    -e SONAR_SCANNER_OPTS="-Dproject.settings=sonar-project.properties" \
    -e SONAR_TOKEN="SONAR_GLOBAL_TOKEN" \
    -v "./software/packages/plantuml-mindmap-loader:/usr/src" \
    sonarsource/sonar-scanner-cli:5

It will take a couple of minutes to execute and after it's completed you can enjoy the code quality assessment results in SonarQube

Review the issues, check suggested solutions, apply a patch and run the scan again. The setup is complete.

Bonus. Setup SonarLint in VS Code

While seeing the results of scanning in SonarQube is super helpful, in software development, shortening the feedback loop for code changes is essential for efficient workflow. For that, there is SonarLint extension available for quite a few modern IDEs, including VS Code.

What makes this extension especially impressive is the connected mode when the rules applied to the project are synchronised with SonarQube server.

To install and configure it:

Find and install SonarLint extension in VS Code.
Go to the "SonarLint" tab and click at the plus button in "Connected Mode" section.
Add details: Server URL (http://localhost:9000), User Token (can use SONAR_GLOBAL_TOKEN from before).
Click "Save Connection".
For this connection, click the plus button for "Add project binding".
Select the workspace.
Select the project created in SonarQube server.
Enjoy the binding.

You can see the syntax highlight right in your IDE now and see the underlines going away as you address the quality concerns.

Results & Beyond

With this SonarQube Community Edition setup, you will be able to ensure outstanding quality in your projects with very little time and resource investments. Variety of benefits attracts:

If your project evolves quickly, you can enjoy sturdier stance of your project against changes. Your project will be harder to break, easier to maintain, more challenging to hack.
If it's a weekend project, future self will be extremely happy to start a new feature from a reliable checkpoint - with cleaner code, fewer bugs.

This guide focuses on the most straightforward setup possible and many possible improvements are not included in the scope not to lose focus on the MVP. A few evident things you might work out yourself:

Integrate SonarQube with CI/CD to ensure the quality gate fully protects your shared branches from code of low quality.
Create an automated local runner for SonarQube scanner to have a quality gate not before the code is merge but before the code is committed.

I hope the length of this guide doesn't intimidate you from trying this local setup. In fact, it's pretty straightforward and even lightweight. My objective was to provide information on the aspects of particular steps so that if in your situation a slightly different approach is more suitable, you make this turn and return to the main track with the solution that fits your needs the best.

And last but not the list, getting quality suggestions from SonarQube is a whole load of fun. You'll enjoy it.

Our Experiment with AI-Powered Dev Tools: The GitHub Copilot Experience of One Company

Hopefully Surprising — Tue, 26 Sep 2023 15:33:35 +0000

We provided 5 of our developers with access to GitHub Copilot for 2 months to find out if we need to buy it for everyone. Here are the unexpected results of the experiment

Introduction

As someone who constantly searches for ways to improve personal coding performance and performance of my development teams, I have been experimenting a lot with AI driven utilities that might assist developers in writing code. TabNine, ChatGPT 3 & 4 and, finally, GitHub Copilot that wraps a special version of OpenAI's CodeX in a tool that is easy-to-integrate with modern IDEs.

All these tools have proven useful in many situations and allowed me to create utilities for automating daily routines with ease I had never experienced before. My initial intention was to provide all the developers in my teams with access to this tool. But to better justify this regular spending to my company, I decided to conduct a little experiment: a pilot group had been given paid access to GitHub Copilot with a duty to share feedback regularly.

I have to say that it was a very right decision because the results weren't what I expected to see. And, sorry for the spoiler, we changed our plans - we're not giving it to everyone.

Before we started: Security

Like any company that considers information security a paramount subject, we reviewed the terms of usage of GitHub Copilot with our legal team. We ensured that running GitHub Copilot in a client machine doesn't involve long-term data collection by Microsoft. The prompts are discarded immediately after providing (see details).

We didn't include Copilot Chat in the scope of the pilot project (that is in beta at the moment of writing). Using the Chat might require accepting more conditions for data collection and processing.

Setup

For the pilot group, we have selected 5 software developers of different experience (from 3 years in IT to 10+ years) and different skillset (React.js in the frontend, Ruby on Rails and Node.js in the backend, GitLab CI for CI/CD pipelines). These 5 people were given GitHub Copilot licenses for 10 weeks and were asked to provide feedback every 2 weeks.

The common themes in the feedback have been collected by me (I also was a part of this program. I wanted to stop paying for it from my own wallet) and are presented in this article.

Observations

It's easy to start using it

What everyone has noticed and mentioned is how easy it is to install in VS Code. Basically, after 2 minutes of installing and configuring GitHub Copilot, the tool is ready for use and provides suggestions in all the files you work in. Convenience of installing is extreme.

But I have to mention that some people found it easier to migrate from Sublime to VS Code to start using it. Technically, it should be possible with Sublime though (this one looks interesting but we didn't spend time on it).

When it's wrong, you are (rightfully) not happy

One of the things noticed by more than one person was the effect of a wrong suggestion. When the suggestion is incorrect, you lose time. At best, you gave a second to Copilot to provide you with something you don't want to see. At worst, you have your code broken and your page turns red. And absolutely always, you have a half-a-second pause to see what it has got for you. You wait. For a fraction of a second but you wait. These micro-pauses are not what we liked.

It might be helpful (and dangerous) for beginners

A couple of people have mentioned that they think GitHub Copilot might be more useful to beginners in technologies. While I partially agree with that, I also have concerns. Any complicated tool requires a skilful operator. The suggestions from AI-powered tools must be validated by a person who knows what the code does and what it should do. Without this validation, it might be like an untamed id without ego. Who might want more chaos in their project?

Tricky things (sometimes) don't get less tricky with Copilot

One of more experienced developers tried to use Copilot for tricky things he is not extremely familiar with. The example is writing PostgreSQL query that filters records based on nested parameters stored in JSONB value. The suggestions given by Copilot were always somewhat imperfect. A solution was found with help of good old ChatGPT after specifying context explicitly (schema, version of PostgreSQL, version of ORM and so on).

It might document your function better than you

We spotted that at times GitHub Copilot helps a lot with documenting a method or a class. It's not lazy, it will write what's needed to full extent. It considers the context of the project, all the implications of underlying logic and it produces a good short summary. Probably, you don't want all of your methods to be documented in plain English. Or do you?

Short takes

Sometimes it surprises you with how much you're guessable.
If you have a typo in your code, it inherits it and can suggest code with this typo again.
Try start typing Database tables walk into a bar for a bit of fun.

Summary & Decision

The results are quite surprising and that's why it's in this blog. Every developer in our experiment came to a simple conclusion: if the company doesn't provide the license, they wouldn't purchase it themselves. But that statement might be a bit too generalised - I personally know a few people who pay for this tool for personal use. Including myself. The drawbacks of having wrong suggestions are usually overweighed by ability to create an Apple Script or getting a suggestion on Jira API protocol without leaving IDE.

Copilot Chat seems to be a more promising feature. A conversation-style tool (like ChatGPT) allows adding context that you realise is lacking. Also, it provides a much more natural environment of chatting with an extremely experienced developer who knows programming inside out but probably unaware of what exactly you're working on right now.

We decided to communicate to the developers that they can request a license for themselves. They will need to write a form, to justify the purchase. In simpler terms, we let developers request a tool from the company, just like a specific IDE or GUI for a database. This approach will help us optimise our software expenses while ensuring that those who genuinely want and need the tool have access to it.

GitHub Copilot is not a one-size-fits-all solution, and it was never intended to be.

Searching for the next one. Stay tuned.

Building a TypeScript-Compatible Webpack Loader: A PlantUML Mind Map Example

Hopefully Surprising — Wed, 13 Sep 2023 14:44:38 +0000

Introduction

In the historical book "The Pragmatic Programmer", there has always been an entire chapter dedicated to the importance of the ability to process text files of various formats quickly and efficiently (see "The Basic Tools: Text Manipulation"). The ability to use different text formats in various situations has always been essential and likely will continue to be.

For example, while JSON notation is super useful in many contexts, in some use cases, such as creating mind maps, another format might be much better. For example, PlantUML's mind maps might be a more preferable option for storing serialised trees.

That's what we will implement - store this tree-structured data as text in PlanUML format and use it in a JS app.

Quick links

Custom webpack loader in npm
Source code in GitHub

Vision

Although parsing files at runtime might be a good idea, another option might be even more beneficial. If we can integrate loading with the building process, we will be able to ensure that the data required for running the app is a part of the bundle and doesn't require passing non-common extension files separately.

In this tutorial, we will create a webpack loader that will be taking care of parsing PlantUML mind map file, converting the content to a standard JS object and providing the object to any JS module that imports it by file name.

Solution

We need to do 2 things to achieve the goal: create a loader as an injectable package and integrate this loader with the building process of the project that will be actually importing files with .puml extension.

As a bonus, we will setup basic TypeScript support that will allow importing the text file content with type definitions provided.

Create a custom loader

Project setup

Create a folder:

mkdir plantuml-mindmap-loader && cd plantuml-mindmap-loader

Initialise an npm project (you don't need to change default suggestions as some will be manually updated later in package.json):

npm init

Install dev dependencies:

npm install --save-dev @types/webpack typescript

Add webpack as a peer dependency to the project:

In package.json:

{
    ...
    "peerDependencies": {
        "webpack": "^4.0.0 || ^5.0.0"
    }
    ...
}

Note: It's a webpack loader, hence, webpack is always installed in a "parent" project.

Update main and types properties in package.json to paths that we will be using for the results of building:

{
    ...
    "main": "lib/index.js",
    "types": "index.d.ts",
    ...
}

Create tsconfig.json in the root directory of the project with following content:

{
    "compilerOptions": {
        "target": "es2016",
        "module": "commonjs",
        "allowJs": false,
        "outDir": "./lib",
        "esModuleInterop": true,
        "strict": true
    }
}

Note: If you want to use Jest for testing, consider adding the below config to skip building test files, like this:

{
    ...
    "include": [
        "**/*"
    ],
    "exclude": [
        "**/*.spec.ts",
        "coverage/**/*"
    ]
    ...
}

Create the entry point for the loader - index.ts - with the following content:

import { validate } from 'schema-utils';
import { LoaderContext } from 'webpack';

import { parseMindMap } from './src/parseMindMap';

interface LoaderProperties {
    test: string;
}

const schema = {
    type: 'object' as 'object', // `as` is for making the type match `JSONSchema7`
    properties: {
        test: {
            type: 'string' as 'string', // `as` is for making the type match `JSONSchema7`
        },
    },
};

// Should match the definition in index.d.ts
export interface PlantUMLMindMapNode {
    id: number; // Order number of the line in the file
    label: string;
    tags: string[];
    children: PlantUMLMindMapNode[];
}

export default function (this: LoaderContext<LoaderProperties>, source: string): string {

    // 1. Get options that can be provided to the loader
    // via webpack loader configuration.
    // Also, validate these options.
    const options = this.getOptions();
    validate(schema, options, {
        name: 'plantuml-mindmap-loader',
        baseDataPath: 'options',
    });

    // 2. Do the core thing of your loader.
    // In our case, we take `source` which is the content of a file being imported
    // and convert it to a JS object, recreating the tree structure
    // defined by PlantUML mind map.
    const result = parseMindMap(source);

    // 3. We return a string that is a valid JS code
    // that will be injected as a result of file loading.
    // In our case, we simply stringify the JSON object.
    return `export default ${JSON.stringify(result)};`;
}

Note: The key thing is the content of the default exported function. Here, we have 3 building blocks. Take a look at the comments in the code for information.

The functionality of converting PlantUML mind map string should be placed in ./src/parseMindMap.ts. Since, it's not the core subject of this article, let me only share the link to public GitHub repository where actual content can be found.
Create types for this plugin in index.d.ts so that the consumers of the loader can use it:

declare module '*.mindmap.puml' {

    // Should match the definition in index.ts
    export interface PlantUMLMindMapNode {
        id: number; // Number of the line in the file
        label: string;
        tags: string[];
        children: PlantUMLMindMapNode[];
    }
    const content: PlantUMLMindMapNode;
    export default content;
}

That should be enough for now. If you're interested in how it might be covered with unit-tests, take a look at the GitHub repo.

Integrate it with your target project

This part is pretty straightforward. All you need to do is a few things:

Install the plugin as a dev dependency (it's needed for building only)

npm install --save-dev ../packages/plantuml-mindmap-loader

Note: This installation is from a local directory. For installing from npm or from another repository, change the package path.

Update webpack.config.js to include this loader:

module.exports = {
    ...
    module: {
        rules: [
            ...
            {
                test: /\.mindmap.puml$/,
                use: 'plantuml-mindmap-loader'
            },
            ...
        ]
    },
    ...
}

Note 1: The test value includes RegEx that specifies for files of what extension should the loader be used. In our case, we ask webpack to process any encountered **.mindmap.puml with this new loader.

Note 2: The test value should match the module definition in index.d.ts file of the loader. That will ensure that every import is covered with an accompanying type set from module definition.

If the types from the plugin aren't visible for your IDE or webpack complains about missing types for your new imports, update tsconfig.json in the target project to include the type definition like this:

{
    ...
    "include": [
        "src/**/*",
        "node_modules/plantuml-mindmap-loader/index.d.ts"
    ]
    ...
}

Import your custom format file into your code and work with it as if it was a JS object:

import testMindMap from "./test.mindmap.puml";

export function printMindMap() {
    console.log(JSON.stringify(navigationMindMap, null, 2));
}

Results

If everything is done right, for such content of test.mindmap.puml:

@startmindmap

* Root #tag1 #tag2
** Node 1 #tag3
*** Node 1.1
**** Node 1.1.1
** Node 2
*** Node 2.1

@endmindmap

you will get that output in your target project:

{
  "id": 3,
  "label": "Root",
  "tags": [
    "#tag1",
    "#tag2"
  ],
  "children": [
    {
      "id": 4,
      "label": "Node 1",
      "tags": [
        "#tag3"
      ],
      "children": [
        {
          "id": 5,
          "label": "Node 1.1",
          "tags": [],
          "children": [
            {
              "id": 6,
              "label": "Node 1.1.1",
              "tags": [],
              "children": []
            }
          ]
        }
      ]
    },
    {
      "id": 7,
      "label": "Node 2",
      "tags": [],
      "children": [
        {
          "id": 8,
          "label": "Node 2.1",
          "tags": [],
          "children": []
        }
      ]
    }
  ]
}

Result & Beyond

By using webpack loaders in this manner, we can work with any text format in our JS projects. This approach isolates the specifics of parsing distinct text formats to the place that is the most suitable for that: the build process.

Also, the validation of content embedded in a loader helps ensure that the static assets of this type are compatible with the latest version of your code.

But it has at least one significant drawback that worth a mention:

With that approach, you can work with static assets only. If a file should be located in a shared storage and on purpose fetched and processed in runtime, a different approach should be chosen.

Nevertheless, this approach is helpful in many scenarios. That makes almost any static asset functional.

Share other samples of inventive use of custom webpack loaders!

Conclusion

I hope this can serve as a helpful guide when you'll be creating your own custom loaders. It's a powerful tool and it deserves more frequent and inventive use in your projects.

The loader I created is also published to npm.

In future posts, I'll share more info about why I want to make PlantUML mind maps first-class citizens of my codebase. Adding these details here would inflate the article... There will be a day and there will be another story.

Follow me so you don't miss these updates.

Securing Software Development: Integrating InfoSec and Scrum Teams

Hopefully Surprising — Thu, 24 Aug 2023 08:01:33 +0000

As a manager or a software engineer working with Scrum methodology, you might be very well aware of the underlying principles of an effective workflow that maximises the outcome of efforts by establishing a universal pipeline for feature requests from business to the market and back. But it shouldn't be surprising if not all the parts of your organisation follow the principles of the Agile process and Scrum in particular. Each organisation unit, each part of the overall process should be free to choose the approach that works the best.

In a situation when requirements for a development team using Scrum come from external sources, it's critical to establish correct expectations on input and output of the sub-processes driven by different yet interacting teams.

The key to creating successful process here is understanding that a robust Scrum team operates as a streamlined and effective pipeline, efficiently translating formulated business requests into tangible outcomes. However, when these requests arrive in unfamiliar formats, a disconnect emerges, hindering the realisation of objectives. In the worst cases, that might significantly impact feature delivery (for the whole team) due to prolonged investigation and/or implementation.

In this material, we will consider a specific case of addressing information security-related work in the most effective manner when requests for such work are coming from outside the team. We will set expectations on processing vulnerability reports to seamlessly flow the items to Scrum backlog, facilitating this integration into established operational procedures. Addressing this disparity is essential to enable the team to manage these requests without impeding other items in the backlog.

Vision

The core solution revolves around defining precise communication expectations across multiple stakeholders - InfoSec professionals, IT managers, and the development team itself. This structured approach mirrors the concept of the adapter pattern, transferring its essence from code implementation to real-world procedures.

Clear communication plays a pivotal role, with InfoSec leaders gaining insights into the expected format for business requests, while the development team learns how to communicate changes' outcomes. This mutual understanding empowers InfoSec professionals to give the right initial impulse to effective security remediation work and maintain an updated security state in both internal and external documentation.

We're going to utilise the capabilities of a proficient Scrum team to implement change requests that follow the requirements the team puts to input (in a form of Definition of Ready or a similar principle) by bridging the gap between teams following different approach to work organisation.

Solution

The solution lies in following a comprehensive, step-by-step guide that navigates the journey from a fresh security report to Scrum backlog stories and vice versa. This seamless integration ensures minimal disruption to the team's processes and capitalises on their comfort zone and work habits.

InfoSec team: Initial report analysis. The initial holistic report for one or multiple projects undergoes dissection by an InfoSec specialist, segregating it into independent units of work. This step ensures that vulnerabilities can be prioritised, estimated, and planned independently, fostering several benefits:
- Parallel investigation and resolution of multiple items.
- Prevention of low-criticality issues obstructing more critical tasks.
- Enhanced accuracy in estimating smaller tasks, minimising uncertainty.
InfoSec team: Issue categorisation. Issues are classified into two categories: infrastructure issues and application issues. Infrastructure concerns are directed to DevOps or Ops teams, while application issues progress through the provided guide.
InfoSec team: Story Creation: Drafts for individual vulnerabilities translate into user stories within the chosen task management system. These stories are assigned to a delivery manager or another responsible individual who is familiar with both the system and team processes. Story creation emphasises several key points for the team:
- Background Information: Detailed understanding of the vulnerability's nature, affected components, and contextual conditions.
- Reproducibility: Steps for replicating the issue in a controlled environment, vital for QA and automated testing.
- Description: Recommendations for addressing the problem, suggesting common approaches. When the solution is unclear, the story's objective is to identify a solution.
- Acceptance Criteria: Criteria defining successful vulnerability resolution - what we should see to say that we have achieved the goal. These stories are systematically labeled, tagged, or grouped for streamlined accessibility.
IT Manager: Oversight. A delivery manager reviews the created stories, ensuring they meet the outlined requirements. Additionally, they assess the need for further task breakdowns, particularly for less atomic requests.
IT Manager: Ensuring System Alignment. The recommendations for vulnerability addressing are evaluated against the system's perspective. If necessary, the selected strategy is adjusted to align with the system's values and patterns. Examples include:
- Addressing verbose error messages in a mobile app by updating the backend API.
- Consolidating updates to a dependency by upgrading to the current version during the current process instead of the minimal recommended one. The delivery manager ensures this alignment is communicated effectively to the product owner and team, with information enabling the identification of issues throughout the application.
IT Manager. Assignment and Monitoring. Stories are assigned to releases based on vulnerability criticality, and tasks are allocated to corresponding teams. A dashboard or filter that presents tagged vulnerability remediation tasks is required at this stage, providing visibility on statuses and release timelines. This process allows for real-time monitoring and adjustments for both IT manager and InfoSec team.

Result & Beyond

The structured procedure ensures that stakeholders can operate within their preferred formats and methodologies while seamlessly integrating security concerns. This approach minimises the impact on the team's ongoing plans, as security issues are addressed with minimal disruption.

This methodology also minimises the risk of unexpected challenges when addressing vulnerabilities. By tailoring recommendations to the project situation, the likelihood of encountering unforeseen obstacles is significantly decreased.

The InfoSec team benefits from real-time visibility into project plans and progress, enabling swift reactions to deprioritisation and successful deliveries as their capacity allows.

While the process is comprehensive, avenues for improvement remain:

Educating InfoSec team and other peers about Scrum methodologies fosters better story creation.
Educating the Scrum team on InfoSec team's priorities enhances understanding of the requests and more accurate issue addressing.
Involvement of senior developers in processing stories streamlines the process and helps scale the process when the vulnerability reports cover multiple projects.
Measuring lead time of the security-related requests in your projects must lead to well-informed decision on claimed remediation time for security vulnerabilities that might lead to establishing a vulnerability remediation SLA if it hasn't happened yet.

Conclusion

Within our organisation, this process has been helpful in rapidly addressing security concerns while maintaining focus on project innovation. We welcome your thoughts on this approach, proposed enhancements, and any additional recommendations or queries.

Let's save time for beautiful innovations through using well-optimised processes for routine tasks!

Enabling Local Editing for MkDocs Documentation with IDE

Hopefully Surprising — Sun, 13 Aug 2023 18:22:09 +0000

There are a million good reasons to start writing documentation. Documentation is the key to preserving acquired knowledge indefinitely - for others and for your future self (who might be quite different if things are going well). As essential tool as documentation must promote the idea of keeping it up-to-date by ensuring easy access to editing pages, so that including more details to existing information and putting focus on right things is simple.

While some documentation management systems like Confluence, Notion and others allow in-place content editing, systems with serialisable content stored separately, such as MkDocs, it's not that straightforward. In this article, we will discuss MkDocs that offers an out-of-the-box solution for that.

MkDocs has the ability to enhance pages with an "Edit" button that allows opening the source code of a specific page in Git repository. However, most guides (including official documentation) focus on scenarios involving cloud-based Git repositories. That's a quite cool feature, especially, if documentation maintenance is public but that's not as helpful for maintainers working on pages locally.

In this material, let's explore how to make "Edit" button open pages in modern IDEs using custom URI schemes, allowing for a seamless transition from viewing to editing.

Vision

The objective is simple: equip every page in a MkDocs Material generated documentation site with an "Edit" button that opens the corresponding source markdown file in VS Code.

Solution

Understanding the Configuration: Familiarise yourself with the edit_uri configuration option in the MkDocs documentation.
Unveiling VS Code's Power: Explore the documentation on launching VS Code using custom URLs. This knowledge will empower our "Edit" button to spring into action.
Crafting the Base URL: Create a base URL that directs users to the directory housing your documentation. For instance: vscode://file/${FULL_PATH_TO_CONTENT_DIRECTORY} Note: If you're an VS Code Insiders user, kick off the URL with vscode-insiders://.
Configuring edit_uri: Modify your mkdocs.yml file, setting the edit_uri option to your created URL: edit_uri: vscode-insiders://file/Users/me/Development/my-project/documentation/content/. Note: If you aren't integrating a Git link, you can omit the optional repo_url setting.
Enabling the "Edit" Button: Activate the "Edit" button feature within MkDocs Material by updating your mkdocs.yml as follows:

    theme:
        features:
            - content.action.edit

Final. Final Touch: Now, perform the documentation build process.

Results and Beyond

With these adjustments, every page of your documentation will get an "Edit" button, making the source file open in VS Code. If this solution helps you in your workflow, consider potential expansions:

Conditional URI Setup: Leverage environment variables to configure the URI dynamically. For local use, it fires up VS Code; for hosting, it points to the Git repository.
Beyond VS Code: If your preferred IDE differs from VS Code, explore available URI schemes that align with your choice.
Theme Compatibility: If you're not using the Material theme, verify whether your theme supports the "Edit" button feature.

Conclusion

This approach streamlines local documentation edits while leaving room for enjoying well-organised navigation between built documentation pages. Join the discussion in the comments if you have insights to share about the potential expansions or the overall concept.

Let's document effectively with minimal effort!