Forem: Lukasz Gornicki

Top 3 harmful incidents in open-source in 2021

Lukasz Gornicki — Sun, 06 Feb 2022 17:44:55 +0000

This article was already published in brainfart blog

For starter

I am a full-time open sourcerer by choice. I believe that open-source is the way software should be built, by default. Thus every single crap that happens in open source hurts me personally.

In this 2021 summary, I will not tackle Shitoberfest and all the hate around Hacktoberfest, as now it seems to be a regular October shitstorm.

I think that Shitoberfest initiative is harmful as every radical initiative that sees the world only in black and white colors. That is it. Maybe one day I'll write about it.

1. I don't have time for amateur hour shenanigans

The Tech world is a small world.

GitHub is not Facebook. You can't just throw shit on people without consequences. Hate speech on GitHub is not easily forgotten; your relations can be easily tracked. If you do not control your emotions, you may become a ghost one day.

I'm referring to GitLabPHP Client related issue. I bet that Graham Campbell had some bad moments because of it.

This is the essence of communication issues on social platforms.

People forget they interact with real human beings. Like seriously, I bet this guy would never behave like this if he would speak to the maintainer, like face-to-face.

Is it really so hard to imagine that written words can harm another human? Even more than the ones you say directly, in private. Written words stay online forever.

So to summarize:

Yes, there can be open-source projects that have a bug
Yes, there can be open-source projects that have broken tests

Just help or leave, don't forget to take your frustrations with you.

2. Rust moderation team resignation

The Rust programming language is popular and has a large community.

The larger you get, the more people you need to be actively involved with, especially project moderation. Somebody needs to ensure the code of conduct is followed, and the community is healthy.

Unfortunately, something went wrong, and the moderation team members resigned.

I'm not part of the Rust community. For me, this topic is interesting for two reasons:

I'm an open-source fanatic that simply feels sad when some glitches are showing up in open-source communities,
I'm building community at AsyncAPI and definitely prefer to learn from the mistakes of others

Now, public resignation had to happen for transparency reasons. It was a good call.

The problem is that they tried to be as polite as possible and transparent at the same time. It only caused more confusion.

I learned nothing. Provided Reddit thread did not help much either.

Official message from the Rust team appeared 3 days later. Zero details, but at least acknowledgment. In a slow open-source world, pretty fast response.

About 3 weeks later, we got more details, but because of the sensitiveness of the issue, we didn't learn more.

Sensitive topics require reasonable investigation, I agree. Unfortunately, a lack of transparency leaves many open questions.

Open questions lead to more discussion and conspiracy theories. So you either stand, be transparent and moderate discussion, or censor and let people talk about it elsewhere. I know it only sounds easy, though.

What feels super weird to me is that the blog post about the issue was actually an email. It was sent a week earlier to project members. There seems to be a clear separation between the community and the Rust project members. It is not my community, I don't know the details, but it sounds super unhealthy that the community learns details a week after Project members. Like members are employees and community a customer.

Anyway, there is one thing I like in the blog post:

Needless to say, it is difficult to govern an open-source project which has reached a size larger than most companies4 and yet is composed of volunteers.

A large and sustainable community can't be built only on the shoulders of individual contributors.

You always need a hybrid approach, individual and corporate contributors. Model governance to not favor any specific group.

So to summarize:

Transparency, transparency, and transparency,
Doesn't matter if you are an active maintainer or occasional supporter. Every community member has an equal right to know the state of the project at the same time,
Large communities need lots of regular maintainers

3. Log4J

Oh yes, the recent tsunami of hate and love.

Many hope it will shake companies and wake them up from a dream that they can use open-source software for free without consequences.

I'm not a type of a believer. I don't share these hopes.

For those that hear about it for the first time:

Log4J is a Java package for logging
It is open-source and under Apache Foundation
Folks found out large vulnerability issue

Now, is open-source broken or is it not broken. To be or not to be that is the question.

All the folks complaining at Log4J maintainers seem to miss a crucial point. Mainly because it is open-source, maintained by people with passion, the issue was solved quickly.

Also, many people do not understand one thing. The fact that the project is under a foundation doesn't mean it is well funded. This is not the goal of having a project under a foundation.

You put the project under the foundation to make all these corporations happy, so they do not have to worry about the project license in the long run. People that represent corporations basically admit that they use or plan to use your project for free on a scale. Unfortunately, they do not ask if your rent is due.

There needs to be a way to finance open-source at scale. Have some global budget or something that distributes money. Maybe existing foundations should adjust their profile and collect more to put money back into the projects under their umbrella.

Nobody should expect regular engineers to be good at coding and collecting money simultaneously. Going from door to door asking for money is not cool at all.

Platforms like GitHub and package managers like NPM (in the case of JS ecosystem) should build a kind of alert system. There should be some reporting tool that lists all the projects at risk. There should be a foundation or something that gets money from all tech, asses the risky projects, contact maintainers, and offer help. Yeah, I know, super tricky, but the only way IMHO.

Don't expect maintainers to beg for money!

Is Hacktoberfest Good For Maintainers?

Lukasz Gornicki — Thu, 05 Nov 2020 13:41:38 +0000

tl;dr

In October, we welcomed at AsyncAPI Initiative 26 new contributors with 70 pull requests (PRs) merged. It was an exhausting but also a fascinating experience.

Hacktoberfest Is Ok

Don't be afraid of Hacktoberfest. It is an excellent event for both contributors and maintainers.

There are haters that will tell you something different. My advice, follow truncated mean measure and always discard extreme opinions, especially if they call for boycotting:

Finally, and most importantly, we can remember that this is how DigitalOcean treats the open source maintainer community, and stay away from their products going forward

Cancel culture at its best. The fact that someone is good at programming or works at Google or Facebook doesn't make them experts in everything. Remember that celebrities are not good candidates for a role model.

There are no perfect events; there are no best solutions. There is always a place for improvement, but it should be followed by open, civilized discussion.

Let me conclude by saying that I hope the "harm to the open source made by DigitalOcean" is not as significant as the harm that such hate does to open source by discouraging new open-source contributors. However, this is just speculation. How can I consider any of these things harmful if I did not conduct scientific research? I can only confirm that Hacktoberfest did not harm AsyncAPI Initiative. On the contrary, it was pretty neat.

Spam

During the entire event, we had only two spam PRs. I can imagine that a much more popular and known project might have had more. Nevertheless, adding the invalid label and closing a PR is a super simple operation, three clicks.

The definition of spam heavily depends on maintainers. For example, this is not invalid to me, because I don't think of grammar as "subjective nits".

Why We Participated in Hacktoberfest

Our intentions were pretty clear from the very beginning. As I wrote in the previous post, we wanted to:

Promote AsyncAPI Initiative as a place where we work not only on the AsyncAPI specification, but also lots of tools
Help members of the broader open-source community make their first contributions in a friendly environment

My impression is that sometimes the open-source is perceived as a kind of elite gathering. This is quite often blocking people from joining because they feel they cannot help but rather waste others' time. I was there in the past, I thought the same. It's just another variation of the damn impostor syndrome. You can always help, no matter what your experience is.

Start small. Don't start with tasks that can be overwhelming. Don't throw yourself into the deep end.

We wanted to help others make first baby steps in a secure and inclusive environment, with lots of patience and support.

What It Takes To Have 70 PRs Merged

It is not enough to label 70 issues with the hacktoberfest label, sorry 😃

First of all, you need to be passionate about open source and dedicated to what you do. It can't just be a task that somebody assigned to you. It would help if you were prepared to treat Hacktoberfest participants as a priority. I would compare it to the onboarding process of new hires.

Of course, not all participants join to stay longer, usually they just follow the "one pull request, and I'm out of here" approach.

It doesn't matter.

Please don't make assumptions; assumptions are evil. Be opened, treat every contribution equally, and remember that the onboarding process is a crucial element. If you fail with the onboarding process, you fail big time at the very beginning.

What We Prepared

We prepared the following materials for potential participants:

Blog post about our participation
Onboarding videos that explain how to start
78 issues from more than 30 repositories and put them all in one list with additional information about the difficulty level or the technical area,

It took me around eight workdays to do it all. The most time-consuming part was to go through all the issues, pick candidates, create new ones, and group them all in a Google Sheet.

What We Did During The Event

70 pull requests mean —at least— 70 reviews 😅

If you know your project well, it is not very time-consuming, and anyway, it is the work you have to do as a maintainer. I do not count this time as an extra Hacktoberfest effort. Of course, it can be overwhelming if this is not a standard amount of PRs that you get every month.

We also hosted office hours for participants. This was fun, and we wanted to start doing live streams about AsyncAPI anyway.

Last but not least, once a week, I advertised our project on the official Hacktoberfest Discord server.

It looks like we do not need to do more for the next year.

Was It Worth It

Hell yeah, and I'm already looking forward to Hacktoberfest 2021.

It was great to see so many different people interacting with the project and seeing we reached our goal.

We got some new features, CI/CD cleanup in all repositories, and solved many trivial SonarCloud-reported issues that we would never found time to solve.

What Made It Such a Success

Our success was a typical return on investment.

We asked all contributors to provide feedback on how they learned about us and what was the most helpful resource. 20 out of 26 responded to our request.

Figure 1: Ways in which the contributors learned about AsyncAPI participating in Hacktoberfest

I think it is pretty clear that introducing the official Discord channel was a great idea. I personally do not like Discord because of the lack of support for threads, but better this than nothing.

Figure 2: Resources helpful for contributors

Such a simple thing as a Google Sheet with a list of issues grouped by different factors was, in the end, our best resource for contributors. I encourage you to create such a sheet for your contributors next year.

Open Request to Hacktoberfest Organizers

There is one thing that calls for an improvement for next year. Just one? Yeah, opt-in solution for projects that want to participate in Hacktoberfest was addressed during this event, and I assume it stays on.

The number of projects that people can work on is overwhelming, and finding the right issue seems very difficult. Please have a look at the feedback we got from our contributors. It is enough to develop an official application where potential contributors can adequately filter out issues by technology and difficulty. In the end, it doesn't make sense for all maintainers to work on their own Google Sheets and post it on Discord. A better way would be to introduce an app for all.

Hall Of Fame

Below you can find a list of all contributors that joined AsyncAPI during Hacktoberfest and contributed their time to the project. The list is sorted alphabetically, including the number of PRs created by this contributor.

ab510 (2)	anbreaker (3)	Jesús Miguel Benito Calzada (1)	Barbara Szwarc (8)
Mitchell Sawatzky (2)	Chenemi Zekeri (2)	Charlie Tharas (1)	Christeen Fernando (1)
danielchu (4)	depimomo (4)	Miguel Angel Falcón Muñoz (1)	Gabriel Claudino (1)
Talmiz Ahmed (5)	HUTCHHUTCHHUTCH (11)	Jakub Iwanowski (2)	João Francisco Lino Daniel (1)
Jürgen B. (3)	mbeuil (2)	Moritz Wiesinger (1)	Katrina Knight (2)
Jimmy Kasprzak (3)	Phil Antiporda (1)	Olivier Lechevalier (6)	Sanskar Patro (1)
Sam (1)	GrimPix (1)

Cover photo by Ian Schneider on Unsplash

Join AsyncAPI @Hacktoberfest and Make Your First Contribution in a Friendly Environment

Lukasz Gornicki — Thu, 01 Oct 2020 07:27:01 +0000

What is AsyncAPI

AsyncAPI is a specification for describing your event-driven architecture. You are probably using already OpenAPI/Swagger specification for describing your synchronous RESTful APIs. AsyncAPI is something that supplements OpenAPI. When you should use AsyncAPI, one of the cases is when your services do not talk to each other directly but through a message broker.

In contrast to the OpenAPI Initiative, AsyncAPI Initiative is focused not only on creating and evolving the AsyncAPI specification but also on its tooling. It is a vendor-neutral space for the community to work together on the spec and its tools. We work on tools like specification parsers or docs and code generators.

What Is Hacktoberfest And Why AsyncAPI Initiative Joins It

Hacktoberfest is a well-known event that promotes open source contributions. In short, you have the entire October to submit four pull requests to any project you want, and in exchange, you get a super cool t-shirt. Is that it? Is it just for a t-shirt? Nah, the t-shirt is nice but what you also get is easy access to open source world. Maintainers of many projects open up for contributions, and it is a great chance to make your first step to joining this fantastic world.

AsyncAPI Initiative joins the Hacktoberfest for two main reasons:

Promote AsyncAPI Initiative as a place where we don't work on the specification only but also build a lot of great tools
Make it much easier for the community to make the first contribution to one of the AsyncAPI repositories

In the past, we were also there where you are now, shy and uncertain if we can impact open source community. We want to give you an easy path to take the first baby steps in the world of open source in a welcoming and friendly environment.

How Can You Help

There is always a lot of work waiting out there. For the sake of this special event, we prepared around 75 GitHub issues that you can pick up. They represent different areas (for example, JavaScript or HTML), different difficulty (for example, 50 issues are easy), and different repositories. No matter if they are trivial or demanding, all of them are important for us. Even with trivial ones, where you, for example, need to remove a semicolon, we will still be super happy if you do this work because this will improve the quality of the project (SonarCloud reports). In other words, every single issue from this list is important.

1. Pick The Right Issue

Here you can find a list of all the issues that you can work on. Most of the issues are about code contribution, but not all of them. There are also issues about documentation or CI/CD configuration (we use GitHub Actions). Just pick the issues you want to work on, one at a time, and let us know in the comments section that you want to work on it.

2. Setup Your Environment and Create a First Pull Request

Once you install Git on your machine and get a GitHub account, you need first to decide if you are here just for Hacktoberfest or longer, and make sure if your issue is easy and maybe you can complete it in the GitHub UI.

In case you are here just for the Hacktoberfest, and you picked easy issues that involve changes only to a single file, there is no need to install Git and complicate your life. GitHub UI enables you to make changes to a single file online.

In case you:

want to stay with us longer,
you picked up an issue where you need to make changes to more than just one file,
you also need to run the project locally to check if it works

Then follow this short instruction on how to fork the repository and set it up locally.

Once you are ready with your changes, submit a pull request. Be nice and follow our code of conduct and make sure your pull request is described properly.

Office Hours

Do you feel overwhelmed? No need. You can do it. Just take this blog post seriously.

Trust me when I write that every pull request is crucial for us.
Trust me when I write that we are a welcoming community.
Don't be afraid that you will waste our time. If we would think about it this way, we would not even join the Hacktoberfest.

Still not sure if you can make it? Don't worry. We want to host office hours throughout the event, 2x a week, 1h long, and different timezones. You can join whenever you want and ask us anything, or do pair programming with us. We start with the first meeting on Tuesday 6th, 8AM UTC and then on the following days:

Tuesday 6th, 8AM UTC
Thursday 8th, 4PM UTC
Tuesday 13th, 8AM UTC
Thursday 15th, 4PM UTC
Tuesday 20th, 8AM UTC
Thursday 22nd, 4PM UTC
Tuesday 27th, 8AM UTC
Thursday 29th, 4PM UTC

You can also join us in a more asynchronous discussion on Slack. For updates and latest news, the best is to follow our Twitter account.

Enjoy the Hacktoberfest!

Blooper Reel

Before you jump to your first contribution, have a look a the making of the videos. It was quite fun.

Analyse Sentiments In Issues and PRs And Get Notified

Lukasz Gornicki — Thu, 17 Sep 2020 17:51:27 +0000

My Workflow

I created an action named Code of conduct compliance through sentiments analysis and want to use it in a workflow with Slack integration in the AsyncAPI Initiative. I'm a maintainer and community guardian in this project and want to make sure that people respect each other when working in the project.

Submission Category: Maintainer Must-Haves

Cause you want to keep your community healthy and know when people disrespect each other.

Yaml File and Link to Code

The most basic example of using this action with other actions is when you want to send the information about negative sentiments to some communicator. Below example shows how to use this action with another action for sending messages to Slack:

name: 'Sentiment analysis'

on: 
  issue_comment:
    types: 
      - created
      - edited
  issues:
    types:
      - opened
      - edited
  pull_request:
    types:
      - opened
      - edited
  pull_request_review:
    types:
      - submitted
      - edited
  pull_request_review_comment:
    types:
      - created
      - edited
jobs:
  test:
    name: Checking sentiments
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Check sentiment
        uses: derberg/code-of-conduct-sentiment-analysis-github-action@v1
        # id is needed for a step if you want to access its output in another step
        id: sentiments
        with:
          # you can find an instruction on how to setup Google project with access to Natural Language API here https://github.com/BogDAAAMN/copy-sentiment-analysis#gcp_key-get-your-gcp-api-key
          gcp_key: ${{ secrets.GCP_KEY }} 
      - uses: someimportantcompany/github-actions-slack-message@v1
        # this step runs only if sentiment is a negative numner
        if: steps.sentiments.outputs.sentiment < 0
        with:
          # to get a webhook url of your channel use this documentation https://slack.com/intl/en-pl/help/articles/115005265063-Incoming-webhooks-for-Slack
          # first register new app here (https://api.slack.com/apps
          webhook-url: ${{ secrets.SLACK_WEBHOOK_URL }}
          text: Here ${{steps.sentiments.outputs.source}} you can find a potential negative text that requires your attention as the sentiment analysis score is ${{steps.sentiments.outputs.sentiment}}
          color: orange

This is what you see in Slack once the negative sentiment is spotted:

Not using Slack? then check different actions:

There are plenty of different notifications-like actions that you can use. Check this awesome list.

The rest of the examples of using the action are stored in the readme of the GitHub Action.

derberg / code-of-conduct-sentiment-analysis-github-action

Use this action to analyze sentiments in issues and pull request to identify negative emotions that need to be checked against the Code of Conduct

Code of Conduct Compliance Through Sentiments Analysis

Overview

Use this action to analyze sentiments in issues and pull requests to identify emotions that need to be checked against the Code of Conduct This action only analyzes sentiments. Alone it is not very useful as it is just able to log the sentiment on GitHub Actions log level. It is intended to be used with other Github Actions. An example use case could be run this action and then send the results to Slack with another action.

Why this action requires other actions?

It is because it "only" analyzes the sentiment, but it is up to you to decide in the workflow setup the negative sentiment for you…

View on GitHub

Additional Resources / Info

This action can analyze sentiments using two different solutions: built-in and 3rd party. Out of the box, this action is integrated with sentiment package to do analytics based on AFINN wordlist. Instead of such basic analytics, better use this action to communicate with Google Natural Language API.

Google API is not expensive. It is free up to 5k requests and 1$ up to 1m requests. I recommend this way instead of the dictionary check.

Cover photo by Greg Rakozy on Unsplash

Convert Swagger 2.0 to OpenAPI 3.0

Lukasz Gornicki — Tue, 18 Aug 2020 13:09:34 +0000

Staying with Swagger 2.0 is like staying with ... still working on a good comparison. Anyway, you should migrate to OpenAPI, and my goal here is not to convince you why. I assume you already made this smart decision, and I just want to make it easier for you.

I did not perform any more in-depth investigation of available tools. There are tools provided by Mermade Software. To be more specific, by Mike Ralphson that is a member of Technical Steering Committee. For me, this is a good reason not to research different tools.

Convert in a Browser
Convert in a Terminal
- NPM
- NPX
- Docker
Convert Multiple Files

Convert in a Browser

Conversion in a browser is addressed by Kin Lane aka API Evangelist in this thread and the video so watch it and go to https://mermade.org.uk/openapi-converter

Convert in a Terminal

Use swagger2openapi and have fun:

NPM

Install the tool npm install -g swagger2openapi
Run conversion swagger2openapi --yaml --outfile openapi.yaml https://petstore.swagger.io/v2/swagger.json

And that is it. Check your new openapi.yaml file.

NPX

NPX is useful in CI/CD where you do not want to install swagger2openapi globally.

npx -p swagger2openapi swagger2openapi --yaml --outfile openapi.yaml https://petstore.swagger.io/v2/swagger.json

Docker

NPM and NPX is not your thingy? use the Docker image provided by Mike.



#this part "-v ${PWD}:/usr/src/app" mounts the directory where you started "docker run" inside the container where CLI is triggered, this way generated "openapi.yaml" gets into your local drive
docker run --rm -v ${PWD}:/usr/src/app mermade/swagger2openapi swagger2openapi --yaml --outfile openapi.yaml https://petstore.swagger.io/v2/swagger.json

Convert Multiple Files

You most probably have many services, and you need to convert many Swagger files, and you do not want to do it one by one but all at once with a script. You can use Bash and write some script that runs the CLI, but writing Bash scripts is like ... yeah, one day I'll find a good comparison.

Just use swagger2openapi as a library. Go to this repository and try out the sample project that converts multiple files located in directories and subdirectories.

derberg / convert-swagger-to-openapi-playground

Some instructions and sample how to convert Swagger 2.0 files to OpenAPI 3.0

Convert Swagger to OpenAPI Playground

Convert in a Browser

Conversion in a browser is addressed by Kin Lane aka API Evangelist in this thread and the video so watch it and go to https://mermade.org.uk/openapi-converter

Convert in a Terminal

Use swagger2openapi and…

View on GitHub

(Part 2) Full automation of release with GitHub Actions and Conventional Commits for non-JS projects

Lukasz Gornicki — Tue, 14 Apr 2020 15:01:33 +0000

tl;dr
Here you can find the first blog post about automated releasing. The purpose of this blog post is to show how you can do the same automation in non-JavaScript projects. Even if JavaScript community created tooling, you can still use it in other projects and don't freak out.

This post and the previous one come from our experience we gained when working on full automation for all tools maintained by AsyncaPI Initiative.

AsyncAPI is a specification that you use to create machine-readable definitions of your event-driven APIs.

The previous post focused on JavaScript as the first library that we automated was our generator. It covered publishing to NPM and usage of the JavaScript community ecosystem. Now we have automation rolled out to all our libraries, Go-written too.

What I need to automate release?

To automate a release efficiently, you need two things:

Machine-readable information that allows you to identify if a given commit should trigger a release or not.
Tooling that you can easily plug in and configure without the need to write everything from scratch.

This automation is possible thanks to the following:

The Conventional Commits specification. The purpose of Conventional Commits is to make commits machine-readable but also human-readable. It defines a set of commit prefixes that can be easily parsed and analyzed by tooling and looks good to the human eye too.
The Semantic Release package and related plugins that support Conventional Commits and publishing to different channels like GitHub, NPM, Slack, and others.

Where's the catch?

This blog post is about the automation of releases for non-JavaScript projects. Let me be honest though, solutions I mentioned in the previous chapter come from the JavaScript community.

The problem is, there are people who Hate JavaScript, they truly hate it like it is a living thing. Although, I'm personally proud to be an idiot that has a programming language that I can use.

Conventional Commits specification is heavily inspired by Angular Commit Guidelines. The Semantic Release package and its plugins ecosystem are all Node.js packages.

If you have Java or Go project, you can still use these tools. You do not have to keep package.json in your repository, so don't worry, you can keep your repository clean. The great folks from Semantic Release thought about you too.

Using Semantic Release with GitHub Action in Go project

One of the projects where we use this JavaScript tools is our parser for AsyncAPI documents. It is a Go parser.

Semantic Release configuration

The Semantic Release package supports configuration files in different formats and file types. You are not bound to package.json. We chose to use .releaserc file in YAML format but there are other options too.

---
branches:
- master
plugins:
- - "@semantic-release/commit-analyzer"
  - preset: conventionalcommits
- - "@semantic-release/release-notes-generator"
  - preset: conventionalcommits
- - "@semantic-release/github"
  - assets:
    - path: asyncapi-parser.darwin.amd64
      label: Binary - Darwin AMD64
    - path: asyncapi-parser.linux.amd64
      label: Binary - Linux AMD64
    - path: asyncapi-parser.windows.amd64.exe
      label: Binary - Windows AMD64

Our configuration uses plugins to:

Analyze Git commits with Conventional Commits specification.
Create a Git tag and generate changelog for release notes.
Publish a release with additional assets. We compile our parser as binaries that are compatible with many platforms and we want to have them easily accessible with each release.

We place the configuration under .github/workflows/, next to our GitHub Action release workflow file: release.yml. It indicates that it is for release only, nothing else.

Release workflow

Let us have a look at the differences between this workflow and the workflow I described for a typical JavaScript project here.

First, you define a test job with the Go environment to trigger tests with different versions of Go.

test:
  name: 'Testing'
  runs-on: ubuntu-latest
  strategy:
    matrix:
      go: 
        - '1.14'
        - '1.13'
        - '1.12' 
  steps:
    - name: Checkout repo
      uses: actions/checkout@v2
    - name: Setup Go
      uses: actions/setup-go@v1.1.2
      with:
        go-version: '${{ matrix.go }}'
    - name: Invoking go test
      run: go test ./...

The next step is the release job, where you can differentiate two core steps. The first part is the generation of the binaries that you want to expose in the GitHub release.

- name: Setup Go
  uses: actions/setup-go@v1.1.2
  with:
    go-version: '1.14'
- name: Invoking go vet and binaries generation
  run: |
    go vet ./...
    GOOS=darwin GOARCH=amd64 go build -o=.github/workflows/asyncapi-parser.darwin.amd64 ./cmd/api-parser/main.go
    GOOS=linux GOARCH=amd64 go build -o=.github/workflows/asyncapi-parser.linux.amd64 ./cmd/api-parser/main.go
    GOOS=windows GOARCH=amd64 go build -o=.github/workflows/asyncapi-parser.windows.amd64.exe ./cmd/api-parser/main.go

So far, it is all Go-related operations. How about the release? For the release, you need to set up a Node.js environment to run Semantic Release. Node.js community has this excellent package, npx, that allows you to run a package without installing it, and this is what you can do here in the workflow.

- name: Setup Node.js
  uses: actions/setup-node@v1
  with:
    node-version: 13
- name: Add plugin for conventional commits
  run: npm install conventional-changelog-conventionalcommits
  working-directory: ./.github/workflows
- name: Release to GitHub
  working-directory: ./.github/workflows
  env:
    GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
    GIT_AUTHOR_NAME: asyncapi-bot
    GIT_AUTHOR_EMAIL: info@asyncapi.io
    GIT_COMMITTER_NAME: asyncapi-bot
    GIT_COMMITTER_EMAIL: info@asyncapi.io
  run: npx semantic-release

You only have to install conventional-changelog-conventionalcommits explicitly if you want to use conventionalcommits preset when analyzing Git commits and generating the changelog:

plugins:
- - "@semantic-release/commit-analyzer"
  - preset: conventionalcommits
- - "@semantic-release/release-notes-generator"
  - preset: conventionalcommits

Take a look at full release workflow for reference:

name: Release

on:
  push:
    branches:
      - master

jobs:
  test:
    name: 'Testing'
    runs-on: ubuntu-latest
    strategy:
      matrix:
        go: 
          - '1.14'
          - '1.13'
          - '1.12' 
    steps:
      - name: Checkout repo
        uses: actions/checkout@v2
      - name: Setup Go
        uses: actions/setup-go@v1.1.2
        with:
          go-version: '${{ matrix.go }}'
      - name: Invoking go test
        run: go test ./...

  release:
    name: 'Release to GitHub'
    runs-on: ubuntu-latest
    needs: 
      - test
    steps:
      - name: Checkout repo
        uses: actions/checkout@v2
      - name: Setup Go
        uses: actions/setup-go@v1.1.2
        with:
          go-version: '1.14'
      - name: Invoking go vet and binaries generation
        run: |
          go vet ./...
          GOOS=darwin GOARCH=amd64 go build -o=.github/workflows/asyncapi-parser.darwin.amd64 ./cmd/api-parser/main.go
          GOOS=linux GOARCH=amd64 go build -o=.github/workflows/asyncapi-parser.linux.amd64 ./cmd/api-parser/main.go
          GOOS=windows GOARCH=amd64 go build -o=.github/workflows/asyncapi-parser.windows.amd64.exe ./cmd/api-parser/main.go
      - name: Setup Node.js
        uses: actions/setup-node@v1
        with:
          node-version: 13
      - name: Add plugin for conventional commits
        run: npm install conventional-changelog-conventionalcommits
        working-directory: ./.github/workflows
      - name: Release to GitHub
        working-directory: ./.github/workflows
        env:
          GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
          GIT_AUTHOR_NAME: asyncapi-bot
          GIT_AUTHOR_EMAIL: info@asyncapi.io
          GIT_COMMITTER_NAME: asyncapi-bot
          GIT_COMMITTER_EMAIL: info@asyncapi.io
        run: npx semantic-release

You see, you can still have your project "clean" from any JavaScript-specific files and references. Everything you need for running your release with the JavaScript community tooling is only in the release-related configuration.

Conclusion

I don't think I can ever understand this "hate" towards JavaScript. I think, though, that you can "hate" the language, but if you see some amazing tooling built with it, that can increase your productivity, grit your teeth, put bias aside, and enjoy life. Especially, if in exchange you get this excellent feature, notification about release under the Issue and Pull Request:

In case you want to have more explanation on the release automation subject, I recommend reading the first part of the automation story. You can also join our Slack for further discussion.

Cover photo by Rock'n Roll Monkey on Unsplash

This post was originally published at https://www.asyncapi.com/blog/automated-releases-part-two/

AsyncAPI Conference scheduled for 22.04 - Learn how we did it so you can do it too

Lukasz Gornicki — Thu, 09 Apr 2020 10:14:52 +0000

It is happening, a first-ever AsyncAPI Online conference. We start on 22 of April, at 11 AM UTC. The event is going to take 7 hours with 12 talks from all over the world. We start in sunny Sydney and finish in cloudy Vancouver. The event is for free, online, in your favorite quarantine room.

How do you organize an event like this in such a short time

There are two requirements you should fulfill behind you start organizing a conference:

Gather a few people that like to walk off the beaten tracks. Best would be to involve your whole core team, as we did in AsyncAPI. All three of us got involved 😃
- Fran Méndez - Project Director.
- Eva Morcillo Rodríguez - Marketing and Video Content Manager.
- Łukasz Górnicki - Maintainer and Dev Community Keeper.
Make sure you have a fantastic community behind your back that supports your efforts

First baby steps to kick off the event

Consult your community if what you plan to do makes sense
Set a conference date and call for proposals deadline
Get your community involved

We decided on a tight schedule. Our community had only three days to submit proposals. Oh boy, they exceeded our expectations. In total, we got 18! proposals. We were happy and proud but also afraid that we can't accept all talks.

Talks selection

We wanted to establish as many fair rules as possible in such a short timeframe. We decided to form a committee of people that could join Fran and vote for talks that they found most interesting:

They could give 1-10 points per talk.
They had no idea who is behind the talk. They knew only a title and an abstract of every proposal.

After voting, we applied the following additional rules:

We did not accept more than one talk from the same company. We choose the one that was most popular among voters.
We did not accept talks that were either not related to the theme or other talks accepted for the conference.

In other words, we did not want to favor any company and wanted to have a schedule with talks around a similar area.

As a result, we accepted 11 talks from the community. We did it thanks to the support of this amazing group of people:

Sponsoring

We decided to enable companies and individuals to support the event financially.

How to collect money? Do it transparently with tools that you already know. We decided to use Open Collective because we use it already to collect money for AsyncAPI

We created a dedicated event using Open Collective events. This approach allows us to collect money and share expenses explicitly for the event only. The sponsors in exchange get much more than just being listed in our conference sponsors list, and it is clearly described under our event page.

What is the money for? We want to spend it mainly on:

Prizes for the community involved in the event.
Paid event marketing. We are doing our best to promote event on our own, but we know that we could do much better using paid channels too. More people know about the event and indirectly about AsyncAPI, better profits for the community, and the sponsors.

Logistics aka connectivity issues can kill your event

These are coronavirus times. ISPs are having stress tests. Netflix is probably hitting view records. Kids are running around the house all the time. We took it all into account, and this is how we want to solve those challenges:

First of all, we asked our presenters to record their sessions upfront and send them to us. We want to stream them live to YouTube, and we are exploring platforms that might make it easy to stream not only to YouTube but also Twitter and others. Presenters still join their sessions, and they are going to engage with the audience in conference chat in real-time during the whole presentation. What is so great about it?

The audience doesn't have to wait until the end of the presentation with questions as it works in traditional conferences
Presenters have much more time to engage with their audience, to exchange thoughts, and get more questions. They can also ask their teammates, organization members, or experts in their field to provide support during the talk.

Conclusion

Don't be afraid to organize your event; just find some freaks that follow you to organize it. I'm a proud freak that bought the idea immediately. It is worth it, even if you do not know the outcome yet, the experience you always gain profits in the future.

In the name of the whole AsyncAPI community and the core team, I'd like to invite you to join our event on 22 of April, at 11 AM UTC. The conference line-up suggests that the AsyncAPI Online conference is going to be epic!

#stayhome #staysafe #staycurious

Automate AsyncAPI workflows with Github Actions

Lukasz Gornicki — Fri, 03 Apr 2020 14:52:19 +0000

tl;dr
AsyncAPI community got rich with two GitHub Actions that you can use for validation and generation.

GitHub organized a hackathon for GitHub Actions. There is no better reason to work on a solution if there is a bag of swags waiting for you

The hackathon was only a trigger, the right moment to decide that we should engage. The primary motivation was to write a GitHub Action that can help the AsyncAPI community in specification adoption.

Two AsyncAPI related actions we crafted in March are:

Our community member, Waleed Ashraf created an action to validate AsyncAPI documents with our parser
We also created official AsyncAPI action for our generator.

Writing a GitHub Action

Our actions are both written in JavaScript. The other way of writing action is to do a Docker container action. The best way to start writing your action is to:

Follow this tutorial to create a simple action to understand its components.
Get familiar with the official toolkit that you can use to simplify writing an action.
Create your custom action with this template that has many things plugged in already, like eslint, testing, and most important, distro generation, so you do not have to commit node_modules directory to your repository.

These are all the resources I used to write my first action, and to master it, I only had to read the official docs, like the reference docs for the "action.yml" file. Well done GitHub!

What I can do today with AsyncAPI GitHub Actions

Those two actions can help you a lot already, together or separately. I present in this post only two possible workflows, and you can take it from here and think about your ideas.

Validation of AsyncAPI files in a Pull Request

You can make sure that whenever someone makes a Pull Request to propose a change in the AsyncAPI document, you can validate it automatically using Waleed's action WaleedAshraf/asyncapi-github-action@v0.0.3.

Actions can be triggered by multiple types of events. In this example, we will trigger the action on any pull_request event.

name: Validate AsyncAPI document

on:
  pull_request:

jobs:
  validation:
    runs-on: ubuntu-latest
    - name: asyncapi-github-action
      uses: WaleedAshraf/asyncapi-github-action@v0.0.3
      with:
        filepath: 'my-directory/asyncapi.yaml'

Generating HTML and publishing it to GitHub Pages

One of the AsyncAPI use cases is to define your application and generate docs out of this definition, best in HTML. The typical workflow here would be to have a GitHub Action that your trigger on every push to the master branch.

name: AsyncAPI documentation publishing

on:
  push:
    branches: [ master ]

To generate HTML from your AsyncAPI definition, you need to use asyncapi/github-action-for-generator@v0.2.0 action. You also need to specify a few more things:

The template you want to use for generation. In this example, you can see the official AsyncAPI HTML Template. You can also write your custom template but hosting it on npm is not mandatory.
Path to the AsyncAPI file, in case it is not in the root of the working directory and its name is not asyncapi.yml
The template specific parameters. The crucial part here is the baseHref parameter. When enabling GitHub Pages for a regular repository, the URL of the Web page is https://{GITHUB_PROFILE}.github.io/{REPO_NAME}/. Specifying baseHref parameter helps the browser to properly resolve the URLs of relative links to resources like CSS and JS files. You do not have to hardcode the name of the repo in workflow configuration. Your workflow has access to information about the repository it is running in. You could do this: ${baseHref=/{github.repository}}/
The output directory where the generator creates files. You might access those files in other steps of the workflow.

- name: Generating HTML from my AsyncAPI document
  uses: asyncapi/github-action-for-generator@v0.2.0
  with:
    template: '@asyncapi/html-template@0.3.0'  #In case of template from npm, because of @ it must be in quotes
    filepath: docs/api/my-asyncapi.yml
    parameters: baseHref=/test-experiment/ sidebarOrganization=byTags #space separated list of key/values
    output: generated-html

Now you have a trigger and you can generate a Web page. The next step is to publish the generated HTML documentation to GitHub Pages. For this, you can use one of the actions created by the community, like JamesIves/github-pages-deploy-action@3.4.2. You can also use other hosting solutions than GitHub Pages, like, for example, Netlify and one of their actions.

- name: Deploy GH page
  uses: JamesIves/github-pages-deploy-action@3.4.2
  with:
    ACCESS_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    BRANCH: gh-pages
    FOLDER: generated-html

Here is how a full workflow, with embedded validation, could look like:

name: AsyncAPI documentation publishing

on:
  push:
    branches: [ master ]

jobs:
  generate:
    runs-on: ubuntu-latest
    steps:
    #"standard step" where repo needs to be checked-out first
    - name: Checkout repo
      uses: actions/checkout@v2

    #Using another action for AsyncAPI for validation
    - name: Validating AsyncAPI document
      uses: WaleedAshraf/asyncapi-github-action@v0.0.3
      with:
        filepath: docs/api/my-asyncapi.yml

    #In case you do not want to use defaults, you, for example, want to use a different template
    - name: Generating HTML from my AsyncAPI document
      uses: asyncapi/github-action-for-generator@v0.2.0
      with:
        template: '@asyncapi/html-template@0.3.0'  #In case of template from npm, because of @ it must be in quotes
        filepath: docs/api/my-asyncapi.yml
        parameters: baseHref=/test-experiment/ sidebarOrganization=byTags #space separated list of key/values
        output: generated-html

    #Using another action that takes generated HTML and pushes it to GH Pages
    - name: Deploy GH page
      uses: JamesIves/github-pages-deploy-action@3.4.2
      with:
        ACCESS_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        BRANCH: gh-pages
        FOLDER: generated-html

Conclusion

First of all, huge thank you to Waleed Ashraf for creating an action to validate AsyncAPI documents.

Please try out the above-described actions and let us know what you think. Feel free to leave an issue to suggest improvements or ideas for other actions.

In case you are interested with other GitHub Actions related posts you might have a look at:

Original blog post published at AsyncAPI blog

AsyncAPI Online Conference CFP Ends On Friday

Lukasz Gornicki — Thu, 02 Apr 2020 15:05:44 +0000

After reaching 1000 ⭐ on GitHub, AsyncAPI Initiative community decided to go crazy and quickly organize a first, online AsyncAPI conference.

We just released the conference website

You can submit your talk proposal using this form.
The deadline is Friday, tomorrow, 03.04.2020
We know it is a short time, but the conference is scheduled for 22.04.2020 so not much time left. You do not have to submit a 100% perfect proposal. If we pick you, you will have a chance to prettify it before we publish the agenda next Wednesday.

Proposals we are looking for:

How AsyncAPI helps you
What products you built thanks to the AsyncAPI
AsyncAPI use cases
non-AsyncAPI but still super important for the world of event-driven architectures

#StayHealthy #StayHome #StayStrong

Reddit spam filters are blocking dev.to

Lukasz Gornicki — Fri, 20 Mar 2020 13:56:35 +0000

When I started writing on dev.to I tried to promote my work a few times with Reddit as I did in the past with other resources.

All my posts are always blocked when they have dev.to link, like this one https://www.reddit.com/r/github/comments/fjjiyn/github_actions_when_fascination_turns_into/

Others, not dev.to related, are fine, like these ones:

To me, it seems like whatever comes from dev.to is blocked by Reddit. Is it just me or? What is your experience?

Full automation of release to NPM and Docker Hub with GitHub Actions and Conventional Commits

Lukasz Gornicki — Fri, 20 Mar 2020 07:17:31 +0000

tl;dr
from now on, we release generator in an automated way. We roll-out this setup to the rest when we see it is needed.

Repetitive tasks are tedious. If what you do manually can be automated, then what are you waiting for!

But these tasks take only a couple of minutes from time to time, gimme a break

A couple of minutes here, a couple of minutes there and all of a sudden you do not have time on more important things, on innovation. Automation makes it easier to scale and eliminates errors. Distractions consume time and make you less productive.

We kick ass at AsyncAPI Initiative at the moment. We started to improve our tooling regularly. We are now periodically sharing project status in our newsletter, and host bi-weekly open meetings, but most important is that we just recently updated our roadmap.

Am I just showing off? It sounds like, but that is not my intention. I wish to point out we are productive, and we want to continue this trend and automation helps here a lot. If you have libraries that you want to release regularly and you plan additional ones to come, you need to focus on release automation.

What full automation means

Full automation means that the release process if fully automated with no manual steps. What else did you think?

Your responsibility is just to merge a pull request. The automation handles the rest.

You might say: but I do not want to release on every merge, sometimes I merge changes that are not related to the functionality of the library.

This is a valid point. You need a way to recognize if the given commit should trigger the release and what kind of version, PATCH, or MINOR. The way to do it is to introduce in your project Conventional Commits specification.

Conventional Commits

At AsyncAPI Initiative we use Semantic Versioning. This is why choosing Conventional Commits specification was a natural decision.

Purpose of Conventional Commits is to make commits not only human-readable but also machine-readable. It defines a set of commit prefixes that can be easily parsed and analyzed by tooling.

This is how the version of the library looks like when it follows semantic versioning: MAJOR.MINOR.PATCH. How does the machine know what release you want to bump because of a given commit? Simplest mapping looks like in the following list:

Commit message prefix fix: indicates PATCH release,
Commit message prefix feat: indicates MINOR release,
Commit message prefix {ANY_PREFIX}!: so for example feat!: or even refactor!: indicate MAJOR release.

It other words, assume your version was 1.0.0, and you made a commit like feat: add a new parameter to test endpoint. You can have a script that picks up feat: and triggers release that eventually bumps to version 1.1.0.

Workflow design

At AsyncAPI Initiative where we introduced the release pipeline for the very first time, we had to do the following automatically:

Tag Git repository with a new version
Create GitHub Release
Push new version of the package to NPM
Push new version of Docker image to Docker Hub
Bump the version of the package in package.json file and commit the change to the repository

This is how the design looks like:

There are two workflows designed here.

The first workflow reacts to changes in the release branch (master in this case), decides if release should be triggered, and triggers it. The last step of the workflow is a pull request creation with changes in package.json and package-lock.json. Why are changes not committed directly to the release branch? Because we use branch protection rules and do not allow direct commits to release branches.

You can extend this workflow with additional steps, like:

Integration testing
Deployment
Notifications

The second workflow is just for handling changes in package.json. To fulfill branch protection settings, we had to auto-approve the pull request so we can automatically merge it.

GitHub Actions

Even though I have my opinion about GitHub Actions, I still think it is worth investing in it, especially for the release workflows.

We used the GitHub-provided actions and the following awesome actions built by the community:

Release workflow

Release workflow triggers every time there is something new happening in the release branch. In our case, it is the master branch:

on:
  push:
    branches:
      - master

GitHub and NPM

For releases to GitHub and NPM, the most convenient solution is to integrate semantic release package and related plugins that support Conventional Commits. You can configure plugins in your package.json in the order they should be invoked:

"plugins": [
  [
    "@semantic-release/commit-analyzer",
    {
      "preset": "conventionalcommits"
    }
  ],
  [
    "@semantic-release/release-notes-generator",
    {
      "preset": "conventionalcommits"
    }
  ],
  "@semantic-release/npm",
  "@semantic-release/github"
]

Conveniently, functional automation uses a technical bot rather than a real user. GitHub actions allow you to encrypt the credentials of different systems at the repository level. Referring to them in actions looks as follows:

- name: Release to NPM and GitHub
  id: release
  env:
    GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
    NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
    GIT_AUTHOR_NAME: asyncapi-bot
    GIT_AUTHOR_EMAIL: info@asyncapi.io
    GIT_COMMITTER_NAME: asyncapi-bot
    GIT_COMMITTER_EMAIL: info@asyncapi.io
  run: npm run release

Aside from automation, the bot also comments on every pull request and issue included in the release notifying subscribed participants that the given topic is part of the release. Isn't it awesome?

Docker

For handling Docker, you can use some community-provided GitHub action that abstracts Docker CLI. I don't think it is needed if you know Docker. You might also want to reuse some commands during local development, like image building, and have them behind an npm script like npm run docker-build.

- name: Release to Docker
  if: steps.initversion.outputs.version != steps.extractver.outputs.version
  run: | 
    echo ${{secrets.DOCKER_PASSWORD}} | docker login -u ${{secrets.DOCKER_USERNAME}} --password-stdin
    npm run docker-build
    docker tag asyncapi/generator:latest asyncapi/generator:${{ steps.extractver.outputs.version }}
    docker push asyncapi/generator:${{ steps.extractver.outputs.version }}
    docker push asyncapi/generator:latest

Bump version in package.json

A common practice is to bump the package version in package.json on every release. You should also push the modified file to the release branch. Be aware though that good practices in the project are:

Do not commit directly to the release branch. All changes should go through pull requests with proper peer review.
Branches should have basic protection enabled. There should be simple rules that block pull requests before the merge.

Release workflow, instead of pushing directly to the release branch, should commit to a new branch and create a pull request. Seems like an overhead? No, you can also automate it. Just keep on reading.

- name: Create Pull Request with updated package files
  if: steps.initversion.outputs.version != steps.extractver.outputs.version
  uses: peter-evans/create-pull-request@v2.4.4
  with:
    token: ${{ secrets.GH_TOKEN }}
    commit-message: 'chore(release): ${{ steps.extractver.outputs.version }}'
    committer: asyncapi-bot <info@asyncapi.io>
    author: asyncapi-bot <info@asyncapi.io>
    title: 'chore(release): ${{ steps.extractver.outputs.version }}'
    body: 'Version bump in package.json and package-lock.json for release [${{ steps.extractver.outputs.version }}](https://github.com/${{github.repository}}/releases/tag/v${{ steps.extractver.outputs.version }})'
    branch: version-bump/${{ steps.extractver.outputs.version }}

Conditions and sharing outputs

GitHub Actions has two excellent features:

You can set conditions for specific steps
You can share the output of one step with another

These features are used in the release workflow to check the version of the package, before and after the GitHub/NPM release step.

To share the output, you must assign an id to the step and declare a variable and assign any value to it.

- name: Get version from package.json after release step
  id: extractver
  run: echo "::set-output name=version::$(npm run get-version --silent)"

You can access the shared value by the id and a variable name like steps.extractver.outputs.version. We use it, for example, in the condition that specifies if further steps of the workflow should be triggered or not. If the version in package.json changed after GitHub and NPM step, this means we should proceed with Docker publishing and pull request creation:

if: steps.initversion.outputs.version != steps.extractver.outputs.version

Full workflow

Below you can find the entire workflow file:

name: Release

on:
  push:
    branches:
      - master

jobs:
  release:
    name: 'Release NPM, GitHub, Docker'
    runs-on: ubuntu-latest
    steps:
      - name: Checkout repo
        uses: actions/checkout@v2
      - name: Setup Node.js
        uses: actions/setup-node@v1
        with:
          node-version: 13
      - name: Install dependencies
        run: npm ci
      - name: Get version from package.json before release step
        id: initversion
        run: echo "::set-output name=version::$(npm run get-version --silent)"
      - name: Release to NPM and GitHub
        id: release
        env:
          GITHUB_TOKEN: ${{ secrets.GH_TOKEN }}
          NPM_TOKEN: ${{ secrets.NPM_TOKEN }}
          GIT_AUTHOR_NAME: asyncapi-bot
          GIT_AUTHOR_EMAIL: info@asyncapi.io
          GIT_COMMITTER_NAME: asyncapi-bot
          GIT_COMMITTER_EMAIL: info@asyncapi.io
        run: npm run release
      - name: Get version from package.json after release step
        id: extractver
        run: echo "::set-output name=version::$(npm run get-version --silent)"
      - name: Release to Docker
        if: steps.initversion.outputs.version != steps.extractver.outputs.version
        run: | 
          echo ${{secrets.DOCKER_PASSWORD}} | docker login -u ${{secrets.DOCKER_USERNAME}} --password-stdin
          npm run docker-build
          docker tag asyncapi/generator:latest asyncapi/generator:${{ steps.extractver.outputs.version }}
          docker push asyncapi/generator:${{ steps.extractver.outputs.version }}
          docker push asyncapi/generator:latest
      - name: Create Pull Request with updated package files
        if: steps.initversion.outputs.version != steps.extractver.outputs.version
        uses: peter-evans/create-pull-request@v2.4.4
        with:
          token: ${{ secrets.GH_TOKEN }}
          commit-message: 'chore(release): ${{ steps.extractver.outputs.version }}'
          committer: asyncapi-bot <info@asyncapi.io>
          author: asyncapi-bot <info@asyncapi.io>
          title: 'chore(release): ${{ steps.extractver.outputs.version }}'
          body: 'Version bump in package.json and package-lock.json for release [${{ steps.extractver.outputs.version }}](https://github.com/${{github.repository}}/releases/tag/v${{ steps.extractver.outputs.version }})'
          branch: version-bump/${{ steps.extractver.outputs.version }}

Automated merging workflow

You may be asking yourself:

Why automated approving and merging is handled in a separate workflow and not as part of release workflow

One reason is that the time between pull request creation and its readiness to be merged is hard to define. Pull requests always include some automated checks, like testing, linting, and others. These are long-running checks. You should not make such an asynchronous step a part of your synchronous release workflow.

Another reason is that you can also extend such an automated merging flow to handle not only pull requests coming from the release-handling bot but also other bots, that, for example, update your dependencies for security reasons.

You should divide automation into separate jobs that enable you to define their dependencies. There is no point to run the automerge job until the autoapprove one ends. GitHub Actions allows you to express this with needs: [autoapprove]

Below you can find the entire workflow file:

name: Automerge release bump PR

on:
  pull_request:
    types:
      - labeled
      - unlabeled
      - synchronize
      - opened
      - edited
      - ready_for_review
      - reopened
      - unlocked
  pull_request_review:
    types:
      - submitted
  check_suite: 
    types:
      - completed
  status: {}

jobs:

  autoapprove:
    runs-on: ubuntu-latest
    steps:
      - name: Autoapproving
        uses: hmarr/auto-approve-action@v2.0.0
        if: github.actor == 'asyncapi-bot'
        with:
          github-token: "${{ secrets.GITHUB_TOKEN }}"

  automerge:
    needs: [autoapprove]
    runs-on: ubuntu-latest
    steps:
      - name: Automerging
        uses: pascalgn/automerge-action@v0.7.5
        if: github.actor == 'asyncapi-bot'
        env:
          GITHUB_TOKEN: "${{ secrets.GH_TOKEN }}"
          GITHUB_LOGIN: asyncapi-bot
          MERGE_LABELS: ""
          MERGE_METHOD: "squash"
          MERGE_COMMIT_MESSAGE: "pull-request-title"
          MERGE_RETRIES: "10"
          MERGE_RETRY_SLEEP: "10000"

For a detailed reference, you can look into this pull request that introduces the above-described workflow in the generator.

Conclusions

Automate all the things, don't waste time. Automate releases, even if you are a purist that for years followed a rule of using imperative mood in commit subject and now, after looking on prefixes from Conventional Commits you feel pure disgust.

In the end, you can always use something different, custom approach, like reacting to merges from pull requests with the specific label only. If you have time to reinvent the wheel, go for it.

Cover photo by Franck V. taken from Unsplash.

Originally published at https://www.asyncapi.com/blog/automated-releases/

GitHub Actions - When Fascination Turns Into Disappointment

Lukasz Gornicki — Mon, 16 Mar 2020 08:24:39 +0000

Looks like this blog post is not valid anymore. July 2020 GitHub released proper support for forks https://github.blog/2020-08-03-github-actions-improvements-for-fork-and-pull-request-workflows/ which is super awesome!

This post was supposed to be about something totally different. It was supposed to be a post about how easily you can automate workflows in your project using GitHub Actions. It was...but I was in the middle of my task when I planned this post. Now I'm pretty much done and I'm disappointed with GitHub for the very first time.

GitHub Actions got my ❤️ since it was announced. I could not play with it. I had no use case for it as in my previous project we were bound to a specific CI tool. Then I moved to work on AsyncAPI and I was super happy to introduce GitHub Actions there.

First workflow with GH Actions ❤️

I'm an automation freak. I hate to do the same things twice, and that is why the first thing I wanted to improve was Docker image publishing.

Oh man, I loved it. Starting with GitHub Actions was so easy. Googling out basics was super simple. They also have this super awesome UI for noobs that just started with Actions. Even editing of the workflow files looks different then with other files. There is a dedicated view for workflows with fancy autocomplete and linting. Well done GitHub, well done!!! Using GitHub is as awesome as using TravisCI, but without leaving GitHub UI.

You can imagine that with such a level of excitement when I've learned about GitHub Actions Hackathon I was like "Hell yeah, I'd love to participate".

Now I'm puzzled and lost my motivation to participate in the hackathon. Why? cause I spent some time introducing different workflows and I've learned about some limits that make GitHub Action not as cool as I thought in the beginning.

All things GitHub Actions

After my first task with GitHub Actions and a review of available actions my statement was clear: all automation must go through GitHub Actions.

What use cases I had in mind?

Welcome message

Writing a welcome message to the first-time contributors is a very useful feature. You can not only use it to say usual Hi but also to share important links with new joiners. In the case of AsyncAPI, I wanted to make sure we welcome them with a link to a page that explains all the different ways people can interact with AsyncAPI community.

Auto-labeling

Automated labeling of pull requests depending on what files were modified helps with project maintenance. Not only labeling basing on files modified but also other reasons, like labeling based on the size of the change. It speeds up the maintenance of the project at scale.

Pull request merge

Triggering pull request merge with a specific label is a great step to improve the security of your GitHub organization. To be 100% sure you are secure from vulnerabilities (someone left your organization and you forgot to change his access right) and also you're own mistakes, you should not have write access to repositories. Write access should belong only to a bot that can merge pull requests.

The moment of 💔

Adding all those different actions was super easy, especially thanks to the awesome community of devs that added so many different GitHub Actions to the marketplace. I tested easily all changes on my fork. I created pull requests in a fork and proved the setup works like a charm.

Everything was great until I started using them in a real environment. Real environment meaning a project, in an organization that works with contributions through forks like all the other projects on GitHub.

All those workflows are not supported when pull requests come from forks.

The reason is that the default GITHUB_TOKEN used in a fork-workflow has read-only rights. You can, of course, provide your own secrets, like MY_GITHUB_TOKEN but it is pointless effort as custom secrets are not available in a fork-workflow.

It is like the GitHub team behind the feature never worked in open-source, or their security team is stronger than product management. I don't believe any of these can be true. It is GitHub after all, you kind of trust the brand. I'm truly afraid the reason is money. On enterprise-level, you almost never work with fork-workflow, so basically those that pay, do not really need strong support for forks.

Did we broke up?

There are relationships where you wish to never see your ex again. Not in this case. My heart is broken, I'm disappointed, but I want us to stay friends. I hope this is a temporary issue and that GitHub fixes it, otherwise they will lose a lot on adoption.

I will still use GitHub Actions for release automation and write about it soon.

Now, come on man, you criticize, you're such a smart ass, but this flow has limits for a reason, this is a real security threat. Someone could fork the repo, modify workflow in the way that he can read the secret and exploit the system secured with it. (a very possible comment that I could get in this post)

This is a valid point, but very easy to solve with current GitHub functionality. In my previous project, I used Prow that was built for Kubernetes to test it and maintain the community on a very high scale. Prow also has to support fork-workflow.

How the Kubernetes test-infra team solved the security concern? They do not trigger CI jobs if the pull request was not created by the member of the organization. Organization member, after checking that the pull request is not modifying GitHub Action workflow, must comment on the pull request to trigger CI with a message like /ok-to-test. As simple as that.

GitHub can go a step further here and integrate GitHub Actions with code owners feature. You should be able to specify in CODEOWNER file a more limited group of people that can approve changes in workflows and trigger them with /ok-to-test.

I hope that, by accident, I revealed how the GitHub Actions team is already working on fixing this limitation 😃. I really like this functionality and want to also use it for managing and supporting AsyncAPI community. I'm looking forward to doing all the things with GitHub Actions.