Forem: Wealize

Avoiding IF nesting ✨

Jesús Mejías Leiva — Sun, 25 Oct 2020 18:33:43 +0000

Introduction

Today I come to share a little tip to improve the readability of our code. We will achieve this by using guard clauses which will help us avoid this IF nesting.

Examples

Bad ❌

In this example, we can see how and IF nesting appears, in this case, it is a simple example but this can get complicated, with each nesting that is added the code will become less readable and therefore less friendly and less maintainable.

const calculatePercentageHeight = (width, height) => {
  const result = (height * 100) / width;

  if (!Number.isNaN(result)){
    if (Number.isInteger(result)){
      return result
    }else{
      return result.toFixed(2)
    }
  }else{
    return 0
  }
};

// use function
calculatePercentageHeight(50,50) // result 100
calculatePercentageHeight(50,0) // result 0

Good ✅

Using guard clauses we avoid the nesting of IF conditionals since we will "first control the error cases", and then we will continue with our logic.

const calculatePercentageHeight = (width, height) => {
  const result = (height * 100) / width;

  if (Number.isNaN(result)) return 0; // guard clause
  if (Number.isInteger(result)) return result; // guard clause

  return result.toFixed(2);
};

// use function
calculatePercentageHeight(50,50) // result 100
calculatePercentageHeight(50,0) // result 0

Conclusion

Using guard clauses is good practice to avoid unnecessary ramifications and therefore make your code easier and more readable. If we apply these guidelines we will make our software much more maintainable and friendly.

Thanks for reading me. 😊

Translations for React using i18n hook

Jesús Mejías Leiva — Thu, 25 Jun 2020 17:23:29 +0000

Introduction

Recently in my work, the need to include translations to a project built with React arose. I had to investigate and came to this solution, which I think someone might find helpful.

Installation of necessary packages

For the installation of these packages, we will need to have previously installed and configured npm.

Once we have npm installed we must install the following packages:

npm install --save i18next

npm install --save react-i18next

npm install --save i18next-browser-languagedetector

Create a file where translations will be stored

We will create a folder called translations within src and, within the translations folder, we will create a folder for each language to which we want to add the translations.

./src
    ./translations
        ./es
            ./translations.js

We will create the translation.js file where the translations stored in this case for the Spanish language:


// File: translations.js

export const TRANSLATIONS_ES = {

    "female": "Femenino"

}

Create the configuration file for i18n

./src
    ./translations
        ./i18n.js

with the following content:

// File: i18n.js

import i18n from "i18next";
import { initReactI18next } from "react-i18next";
import LanguageDetector from "i18next-browser-languagedetector";

// import translations file
import { TRANSLATIONS_ES } from '../translations/es/translations'

i18n
    .use(LanguageDetector)
    .use(initReactI18next)
    .init({
    resources: {
        es: {
            // use translations file for spanish resources
            translation: TRANSLATIONS_ES,
        },
    }
});

Using the hook for translations in component

// File: ExampleComponent.js

// import hook for translations and i18n configuration file
import { useTranslation } from "react-i18next";
import '../../translations/i18n.js';

export const ExampleComponent = () => {

    // destructuring t() function for useTranslation()
    const { t } = useTranslation();

    return (
        {// using t() function for translate}
        <p>{ t('female') }</p>
    )
}

Thanks for reading me. 😊

Why we ditched story points to be more value-oriented

Javier Aguirre — Wed, 11 Dec 2019 09:00:00 +0000

At TNP we worked using story points to estimate and control the amount of work everybody on the team was doing, but we decided to ditch it, here is why.

We discovered several problems:

People were stressed to finish their points, and they couldn’t focus on helping each other because they wanted to end their user stories. This issue was pushing individualism rather than quality and ship feature focus.
Having story points wasn’t pointing us to build faster or with more quality. User story points weren’t focusing the work on value but rather on risk, time, and complexity of the user story and features. 1-week sprints were making everything worse due to this fact when we finished features that weren’t good enough to put in production and were coming back to haunt us.

We always like to iterate on processes to improve it and reduce friction at maximum, so everybody is comfortable working and focusing on delivering as much value as possible. With this idea, we decided to focus much more on delivery, following some rules.

We have two-week sprints, and we organize teams by features, so everybody is on board to ship the feature and responsible for it. We don’t have departments, but feature teams with different roles. I haven’t invented this. It’s from The Nature of Software Development book that @stanete recommended some time ago. Every user story or feature has a team behind, design, product, development, and everybody is needed to accomplish it.

We had to improve user story definition, make them more shippable and self-contained so we could deploy each of them with value for the client. That’s why we decided to be much more product-oriented. Make those minimum viable features to adjust the value and time consumption working on them. This idea is probably the most difficult to achieve. Still, we think it is better to work in this direction rather than trying to estimate better, which won’t focus on deliverability but internal needs.

This new way of working makes it more natural to have other advantages for the team. For example, when someone finished their stories early, he/she has to help the rest of the team on the features to complete for this sprint so we can ship as a team. We have time every week to learn in the process because we’re not stressed, and the time is much less tight. Improve accomplishment managing scope rather than improving estimates. We plan to do as much as in the last iteration to do good work at a consistent pace.

In the case of engineering tasks and reducing technical debt, we follow the rule ‘leave the campground better than you found it.’ I have adopted that rule for years since I read Clean Code, but all the team must stay in tune, so every time we do a feature, we can clean up the area where we are going to work. We’re exhaustive on testing and good quality code to avoid technical debt. We also try to move the QA phase of the feature as soon as possible and not only when it’s done. Once we ship, the feature shouldn’t come back with defects in it; otherwise, It disrupts our development pace.

We haven’t invented this way of working. Some of the tips are from the Influence of the Nature of Software Development and Clean Code , among other books. We’re always iterating to improve our processes and learning new things so that these changes will vary in the future.

Cover Photo by Franck V. on Unsplash

Get release tag from GitHub Actions to debug on Heroku

Javier Aguirre — Thu, 28 Nov 2019 09:00:00 +0000

In TNP we’ve moved everything to GitHub Actions and we’re very happy about it. :-)

But… this article is not about how happy we are using it but a small tip on how we could simplify every day debugging with some simple improvements.

As a good metric oriented company, We measure and monitor everything we believe relevant, this is a solution to control which release we have in production and being able to monitor its breaking changes in a simple way.

We decided to add the git release tag to our deploy and our app footers, with the following steps.

First, We push to production only when a git tag (a release) is created on GitHub.

jobs:
  ...

  deploy_production:
    ...

    if: success() && contains(github.ref, 'tags')

We deploy to Heroku where is super easy to set a config variable via API, so we decided to:

Get the release tag from the current release
Update a RELEASE_VERSION variable on Heroku every time a new deploy is successful.
Profit!

steps:
 - name: Push to Heroku
   run: git push -f https://heroku:${{ secrets.HEROKU_API_TOKEN }}@git.heroku.com/${{ secrets.HEROKU_APP_PRODUCTION }}.git origin/master:master
 - name: Update RELEASE_VERSION on Heroku production
   if: success()
   run: |
    curl -n -X PATCH https://api.heroku.com/apps/${{ secrets.HEROKU_APP_PRODUCTION }}/config-vars \
    -d '{
      "RELEASE_VERSION”: "${{ github.ref }}",
    }’ \
    -H "Authorization: Bearer ${{ secrets.GITHUB_API_TOKEN }}"
    -H "Content-Type: application/json" \
    -H "Accept: application/vnd.heroku+json; version=3"

We need to set two variables:

HEROKU_API_TOKEN: Your personal token, you can get it from your account settings.
HEROKU_APP_PRODUCTION: Your production app name on Heroku, we don’t hardcode it, so it’s easier to share these recipes in other projects.

That’s It! You’ll have something like this on your Heroku app if all goes well.

Cover Image Garett Mizunaka - Unsplash

Natural Language Processing with Google Cloud AutoML for safer humans

Francisco Aranda — Wed, 17 Jul 2019 10:21:54 +0000

At The Neon Project, we enjoy participating in projects that require the application of date technologies such as artificial intelligence and, more specifically, machine learning. These technologies help us extract and analyze large amounts of data to make better decisions.

Natural language processing in our last project

In our most recent project for an organization in the USA, we have been given the task to help reduce and ultimately eliminate mistakes in a disaster prevention system through natural language processing (NLP).

The organization currently uses a system that relies on a traditional parser in order to extract relevant data from informative documents. These documents are manually written by humans from trusted organizations. In this scenario, a document with a wrong format that is not processed by the parser is critical for the rest of the infrastructure.

Because of the sensitivity of the system to false negatives (documents that are ignored by the parser but that are actually relevant and critical), the organization requires engineers to check a lot of the documents rejected by the parser at whatever time they come in.

Technically the source of the problem is that data is heterogeneous and reports don’t necessarily match a standard or template. Human typing mistakes can be expected as they are generated in critical and tense contexts. In this scenario, a traditional parser has major technical limitations.

Our approach was to design a natural language processing model that can identify a set of identities within a huge set of documents. After a few training and validation iterations, this model can replace the current parser and be better at identifying relevant documents. This reduces the risk of false negatives and the need for human intervention for corrections.

Adding Google Cloud AutoML in our tech stack

Google launched last April a new set of services for machine learning development under the name Google Cloud AutoML. We were very excited to use them in a real-world use case, so this seemed to be a great chance to do it.

AutoML offers a simple interface where we could easily upload sample documents to create a dataset. We had documents that were shared by our clients. We completed this data set with similar documents we scraped from trusted sources on the Internet.

Once the documents were uploaded, we had to tag with labels the entities we wanted our model to recognize. Only after the dataset had a minimum quantity of samples for each label, we could begin the training of the model. Once finished, our model was able to identify the entities from a new document, not previously known to the system.

As soon as we had validated the model was properly designed and trained, we used the Node.js library for the AutoML API. In addition, we built a proxy API service together with a simple web interface. In the interface, users can upload documents and see how confident the system is of the importance of the report, based on the content and context of the document and not on its format.

Building the model and the user interface took days instead of weeks thanks to Google Cloud AutoML and allowed us to make our clients happy. We found AutoML an exciting platform for creating machine learning solutions. Getting things done was very straight-forward, and the results were pretty good. We will continue using it in the future!

Do you have a conversational project?

Let’s work together

The post Natural Language Processing with Google Cloud AutoML for safer humans was first published at The Neon Project - Agency for the digital transformation.

Improve code reviews using pull request templates

Daniel Luque Quintana — Tue, 09 Jul 2019 10:12:55 +0000

At The Neon Project we care about our internal processes: we believe they’re important, we design them meticulously, we use them, and we improve them as we learn from their performance in the real world.

One of the most important processes when developing software is code review, where at least one teammate will review your code. This process improves the quality of the code since, many times, code that seems easy to understand for the author is not as clear for the rest of the team. Everyone involved in the code review learns: learn how to give feedback, learn how your partners think and write code & learn technical details.

To make the code review process as easy and quick as possible, the more context we give our colleagues about what we have done and why, the better. After reviewing the Dev.to Github repository and seeing some pull requests, I thought there was a way to improve our code review process in The Neon Project: using templates for pull requests.

A pull request template is, basically, a file containing markdown text that is added to your pull request description automatically when it is created.

The file must have the name pull_request_template.md. In our case, this file is in a hidden folder named .github, but you can put it wherever you want (more information about Github pull request templates here)

Our template consists of the following parts:

A reminder: as we use Jira for project management, a very useful feature is to link a PR to a Jira issue by adding the issue code to the PR title. This way we can see its status from Jira
Type of PR: consists of 3 checkboxes, indicating whether the pull request is a bugfix, a new feature or a refactor.
Description: a brief description of what we can expect from PR. This part is essential for the person who reviews the code to know, in one pass and briefly, what to expect in the code.
Screenshots (if there are changes of UI): one of the problems of the PR that have visual changes is to know what has changed or what is the final result. In general, the process to execute the code locally and see it won't be fast, the person will have to download the code changes in their machine, have the environment configured, etc. A few simple photos and/or videos will save a lot of time to see what's new. Personally, I use macOS screenshots and Imgur to upload them.
(optional) a gif that describes how it makes you feel: because there's always a good reason to put gifs, right? 🤓

In this simple way, we improve the code review process: more information and context so that code reviewers can understand quickly what they are going to review and thus are able to deploy code more often 🚀

One more thing...

After the announcement that Github has acquired Pull Panda, we have integrated it into our process and we are very satisfied! For those who don't know it, Pull Panda allows you to remember through Slack when a contributor needs to have a PR checked, can assign a reviewer automatically and offers a series of very interesting analytics to know, for example, the average time needed to make reviews or how many took more than 8 hours. And it's free!

That's it, folks. So, how's your code review process? Any advice on how to improve ours?

Automatize your new developer’s team onboarding with this simple app (in Python)

Javier Aguirre — Sun, 23 Jun 2019 09:00:00 +0000

When a new developer joins our team, everybody wants to give them the best possible experience 🙌.

We’ve created a simple command line app in Python that with minimum configuration helps the new developer, in a geeky way, set up his/her environment and checks if everything works! It also gives information about which services would they have access to so they can check and don’t need to go somewhere else for that info.

We use Click, the library to create Command line interfaces in Python, and YAML for the configuration. The config has a specific structure moreover, we can configure the file path so someone could reuse the project for their own company 🤓.

The documentation is in the README. You can set the company information in the config, as in our own that you could use as an example. To use your config file, you can set ONBOARDING_FILE_PATH.

You can install it with pipenv.

pip3 install pipenv
pipenv --python 3.7
pipenv install

Once this command finishes, we can run it using pipenv.

export ONBOARDING_FILE_PATH=config.yml
pipenv run python cli.py

It has the configuration to launch from VS Code Debug tab too.

You can see here the output of the app with my company configuration.

The project is open source and can you can clone it from GitHub.

Maybe if we have time, we’ll create a Python package and add more functionality 🎸, but for now, it does the trick. :-)

Cover image by Photo by Harley-Davidson on Unsplash

The Journey to control Code Quality

Javier Aguirre — Wed, 05 Jun 2019 22:21:00 +0000

Coding, programming, developing, name your action! Such a funny and exciting amusement, sometimes going wrong under stress or when lacking experience in a certain technology.

The Challenge

We sometimes need to write code we’re not proud of, but nowadays we have mechanisms to quantify that and improve its quality. Like boy scouts, we can go back and clean the camp, make more tests because we didn’t have enough coverage and we could be not catching all the possible bugs.

When I go back and read my code from some time ago, I’m ashamed, and that’s a good thing, It means we’re growing. Otherwise, we wouldn’t notice.

I’ve always been in the improving code quality path, I know projects and startups need to achieve their goals, otherwise, we all could lose our jobs, but that’s why it’s so important to have proper processes and a good CI/CD, control code quality, and measure legacy code so it doesn’t grow too much.

I’ve been a testing advocate for years now. I know what we should test and what is more challenging to test, why should we refactor if our tests are too complex because we have an architecture problem, what parts are more likely to be problematic. Still, there is usually doubt, risk and terror.

However, when you’re in a small agency like mine, you need to have metrics, you need to rely on excellent services, so you save all the time possible to do what it gives value to the company. We need to improve the process every time (otherwise what’s the fun in doing everything always the same way right? :-)).

The Journey

Our journey for a better code starts with reviews, we defined the user story and estimated the cost of development (more on this on the following article), a member of the team got to it, implemented it and created a PR on GitHub.

The first step, in this case, assures two checks are passing within the PR:

Gitlab CI would run the tests and tell us if they passed.
Codacy tells us if there were significant regressions in terms of code complexity, code duplication, standards, and code coverage.

If the checks pass at least a developer in the team reviews the PR code for clarity, good patterns, functionality. If everything goes well, the code gets merged to the release branch and, when the sprint ends, to master. We can never remove the human interaction since code patterns, architecture and naming are so dependant on the domain and context of the application.

Codacy brings us more functionality than other contenders, such as Danger or Code Climate, but if you don’t like it, you might check it out those. We chose Codacy over the others because it gave us better progress feedback, and they have excellent code coverage integration, much easier to set up than Code Climate (for example). It gave us more than Danger since Danger is usually more used as linter but doesn’t get anything regarding code coverage.

When we started using Codacy, something great happened, the Pull Requests were more alive since everybody was trying to minimize the technical debt they were adding, so Codacy gamified our Pull Requests too. :-)

Codacy integrates with many languages, we usually use Python Pylint, Bandit, Prospector, and ESLint for Node. Codacy would read these libraries config in your repositories and use that configuration if it exists. Otherwise, you can check the configuration items you want from a particular tool right from their project settings view.

At that point, our CI gets triggered (because we merge to master) and if the tests pass the code is automatically deployed to our staging environment, usually Heroku, but also AWS/Kubernetes.

When we want to deploy to production, we create a new tag like vYYYY.MM.DD and the CI/CD triggers the deploy to production.

The Destiny

When the code reaches production, we have two services giving us input so we know if something is wrong or an alert is triggered.

Sentry has been my favorite for years in terms of controlling errors in production, it’s straightforward to use, to tag, to separate between different environments (backend, frontend, mobile) and it works so well!
New Relic is mighty, and we use it to detect bottlenecks and performance problems when they occur (or before it is too late if you configure your alerts well!). It’s so nice to be able to monitor your queries and see how are they behaving to be able to prevent a problem in the future, New Relic is excellent for that.

What are your favourites? What are your recommendations? Let me know!

Debug Python and Django easily from Visual Studio Code

Javier Aguirre — Wed, 22 May 2019 22:21:00 +0000

Debugging code, sometimes painful, sometimes funny, but always an exciting task.

If we don’t use the right tools, It’s where the developer can waste more time. Making debugging easy for developers has been sometimes a problem for technology, but fortunately, in Python, there have been good debug options right from the start.

Visual Studio Code has a very powerful and configurable tab where you can define the desired configuration for your project. You can have all the configurations you want! (for testing, debugging, running a specific process…)

TL;DR

From VS Code, we can easily debug Python code locally but also when running inside of a Docker container using ptvsd. I’ll tell you how.

Print

We all have printed code at some point of our career, but the path to becoming a better developer goes beyond that, it’s essential to spend time researching until we find better tools for the task.

Printing has never been cool, but before learning the proper debugging tools, it has always been that quick and dirty option to check variable output.

PDB

PDB and pudb are great tools! I still use them sometimes when I want to debug something tricky, but for debugging speed purposes I find much more convenient using VS Code Debug tab, where I can see all the context on a specific line of code.

Before that, I used to have a snippet for pdb, and that would extend to import pdb; pdb.set_trace() so I could stop the execution at a specific line of code. Then with (n)ext, (c)ontinue, (q)uit… I could control the execution line by line and step into a specific function or method (more on pdb and pudb).

VS Code

After a while using VS Code I started getting deep into different user and workspace configurations on Django and Node/React, this article gives an example on how we could configure a Django project to use the VS Code debugging tab.

Starting services having installed a Django project locally is easy, we could just use runserver. We’d click the Debug tab, select Add configuration... in the selector and follow the wizard selecting Django: runserver.

The configuration would look like this:

{
    "name": "Django: Runserver",
    "type": "python",
    "request": "launch",
    "program": "${workspaceFolder}/manage.py",
    "args": [
        "runserver",
        "--noreload",
        "--nothreading"
    ],
    "django": true
}

If you use pipenv, virtualenv or have selected the pythonPath on your workspace configuration (Command-, on Mac), VS Code knows how to run the command automatically. We could have a command for testing too, having an envvar to select another DJANGO_SETTINGS_MODULE.

{
    "name": "Django: Test",
    "type": "python",
    "request": "launch",
    "program": "${workspaceFolder}/manage.py",
    "env": {"DJANGO_SETTINGS_MODULE": "myproject.settings.test"},
    "args": [
        "test"
    ],
    "django": true
}

We’d click play and have our tests passed. :-)

VS Code debug in Docker

Debugging inside a Docker container is a bit trickier, we need to open a socket so we can debug from VS Code, let’s see how.

Our Django Dockerfile looks something like this if we use pipenv.

FROM python:3.7

RUN mkdir /code

WORKDIR /code

RUN pip install pipenv
ADD Pipfile /code/
RUN pipenv install --dev

COPY . /code/

EXPOSE 8000

We could have a docker-compose.yml file to manage dependencies easier, such as postgres, redis, or just the ports opened for the host.

version: '3.7'

services:
  db:
    ...

  backend:
    build: .
    image: mydjangoproject
    command: pipenv run manage.py runserver 0.0.0.0:8000
    environment:
      DJANGO_DEBUG: 1
      DJANGO_SETTINGS_MODULE: myproject.settings.local
    volumes:
      - .:/code
    ports:
      - 8000:8000 # Serving Django
      - 8888:8888 # debugging service (ptvsd)
    depends_on:
      - db

  frontend:
    ...

In this case, we open the 8000 for Django runserver and the 8888 so VS Code can connect via the Debug tab to Django.

To be able to do this, Microsoft created a library, ptvsd the Python tools for the Visual Studio debugger. This library runs within our Python code and opens a socket we can connect to debug the application. In Django, we can put it on the wsgi.py file and It will only run if settings.DEBUG is True.

import os

from django.core.wsgi import get_wsgi_application

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "myproject.settings")

if os.environ.get('DJANGO_DEBUG', False): # You can use django.conf settings.DEBUG
    import ptvsd
    ptvsd.enable_attach(address=('0.0.0.0', 8888))
    ptvsd.wait_for_attach() # We can remove this line it gives you trouble,
                             # but it's good to know if the debugger started or not
                             # blocking the execution for a while :-)

application = get_wsgi_application()

We open the socket to the outside world 0.0.0.0 in the port 8888 so we can connect from the host, our machine.

We execute our docker-compose script, in my case docker-compose up backend.

In VS Code the configuration would be pretty straight forward, we add the port where ptvsd opened the socket, we tell VS Code, our localRoot is the current project and the remoteRoot which is my /code path, defined on my Dockerfile for this project.

{
    "name": "Remote Django App",
    "type": "python",
    "request": "attach",
    "pathMappings": [
        {
            "localRoot": "${workspaceFolder}",
            "remoteRoot": "/code"
        }
    ],
    "port": 8888,
    "host": "localhost"
}

Now It’s when the magic happens! You should be able to do something like this. :-)

Do you know more tricks debugging on VS Code? Let me know in the comments!

Common Git problems and easy incremental solutions for all the family

Javier Aguirre — Wed, 22 May 2019 21:07:59 +0000

Git is a gentle beast, but beasts are wild, and they require taming, here are some tips to resolve 99% of git problems.

Working with developers of different experience levels made me realize we should have a simple suitable method to resolve all possible issues when working on a git project, that's why I wrote this tutorial in the first place.

At my company, we follow a small variation of git flow, where there are no develop branch, but a release one
that gets merged every sprint with master and gets recreated for the next sprint until we merge again, deploying automatically to production.

We did this variation because we found out that keeping the develop branch neat and clean when there are different levels of git knowledge in the company is challenging.

We use git fetch to update our git database locally, so we have the latest changes. In case of doubt, we always rebase and create branches from origin, never from a local branch because we might mess up.

When developing a feature, we always use rebase to update our current feature from release. We squash the feature branches into one commit, so it's easy to rollback a feature once it is reviewed and ready to merge.

# Being on the feature/my-tasty-feature branch

git fetch origin
git rebase origin/release

Creating a feature branch:

# Being on master or release or any other branch different from the one created

git fetch origin
git checkout -b feature/my-tasty-feature origin/release

First problem and solution

We need to update our code with release because we don't have the latest changes. In our feature branch we can do:

# Being on the feature/my-tasty-feature branch

git fetch origin
git rebase origin/release
git push -f feature/my-tasty-feature

-f can only be used on an unmerged personal branch. It means force and it rewrites the git history, so we need to be very careful when using it.

Second problem and solution

We have to rebase from release, we have many commits, and we need to solve several conflicts on different painfully commits, we need to make our branch changes into one!

First, we need to know how many commits do we want to add to release and squash them.

# Being on the feature/my-tasty-feature branch

git rebase -i HEAD~5 # where five is the number of commits

We replace all the first lines pick for squash except in the first line and save.

We can change the commit name and put something relevant, save the file.

Done!

Now we need to fix the conflict:

# Being on the feature/my-tasty-feature branch

git rebase origin/release

We fix the conflicts on our favorite editor and add the files, then

# Being on the feature/my-tasty-feature branch

git rebase --continue
git push -f feature/my-tasty-feature

Third problem and solution

In our GitHub pull request we have commits that don't belong to our branch, but are from release, this might happen for two reasons:

Because we didn't do the first solution
Because we updated the master branch and some commit hashes have changed (This should seldom happen).

Check the link if you want to know more about how rebase works.

In this case, the best approach (or most straightforward) is usually to rebuild our branch. First, we need to know how many commits do we want to add to release and squash them, so we'll do the second solution entirely.

After doing the second solution, we'll do the following.

# We are on the feature/my-tasty-feature branch

git fetch origin

git checkout -b feature/my-backup-branch origin/feature/my-tasty-branch

# We delete the branch we're going to rebuild locally

# We are on feature/my-backup-branch
git branch -D feature/my-tasty-branch
git checkout -b feature/my-tasty-branch origin/release

git log feature/my-backup-branch

# We are on feature/my-tasty-branch
# We copy the last commit hash (see screenshot below)
git cherry-pick the-hash-we-have-copied
git push -f origin/feature/my-tasty-branch

Done!

These are my tips for people starting on my team when they have problems with Git, what do you think? Would you change anything? Any recommendations?

CI/CD on a Hyperledger Fabric project

Javier Aguirre — Mon, 20 May 2019 18:43:17 +0000

Today we bring you the second post of this series of technical articles regarding development with Hyperledger Fabric.

Last time we talked about how we test our chaincode,
today we go one step further and we’ll show you how we deploy the code automatically.

CI/CD is the combined practice of automating the testing of our application and the deployment of the application to our servers.

Our workflow integrates different platforms in our pipeline, Github, Gitlab CI and IBM Blockchain. We store the code in Github, when we push code to Github it automatically goes to Gitlab CI where the automated tests are executed. If everything goes well we deploy the code to IBM automatically.

Continuous Integration

Continuous Integration is a practice that requires developers to integrate code into a shared repository several times a day. Every time the code is pushed to the code repository, the automated tests are executed.

There are many cool CI services and we’ve used several of them, right now we’re happy with Gitlab CI, very easy to configure and full of possibilities!

There are two phases on our CI/CD process: the testing and the deploy phase. You can have several phases in parallel, in the test phase we run three processes, backend, frontend, and chaincode testing. When the tests pass and if the rules are met, the application and chaincode are deployed. :-)

Here we’re gonna show you the configuration for the chaincode only since the rest of the application is well documented everywhere (we use industry standards such as Django, React, etc…).

In the test phase we run the chaincode tests with this configuration, the steps are documented in the code snippet:

Continuous Delivery

Every change that passes the automated tests is deployed to production automatically, so when the tests pass it’s time to deploy!

This stage is executed only in two specific cases, for the staging and the production environment:

Staging: When we push the changes to master in Git
Production: When we push the changes with a Git tag. We use the following formats for the tags vYYYY.MM.DD (Year.Month.Day)

Each deploy consists of two API calls. The first one is used to install the chaincode in the platform, the second one to instantiate the chaincode in one of our channels.

You can read the IBM Blockchain API documentation to know what can you send on these API calls.

The variables used within Gitlab CI are:

NETWORK_ID: network_id in IBM Network credentials
NETWORK_KEY: key in IBM Network credentials
NETWORK_PASS: secret in IBM Network credentials
CHANNEL_{ENVIRONMENT}_ID: the channel is the name we added when we created the channel for this project (defaultchannel by default on IBM Blockchain)

See the entire example recipe to CI/CD Hyperledger Fabric on IBM Blockchain.

Enter IBM Blockchain

The IBM Blockchain platform is, by far, the easiest way we used to build a Hyperledger Fabric network, it has also a nice API to interact with it remotely, it saves a lot of time and pain not needing to administer any of the typical components within a Fabric network. There are only some details we need to take into account when using it:

When we deploy several smart contracts with the same name it might be difficult to know which is the last one deployed, there is no chaincode grouping by name.
There is no way of archiving old chaincode release, so we end up with many chaincode versions in the chaincode list.
There is no way of telling IBM Blockchain that my smart contract, built in Go, has an external dependency.
Logging is not integrated in the platform, it opens a Kibana dashboard for us to see the logs (I’m not sure if this is good or bad, but the experience is weird).

Conclusion

Once CI/CD is well configured, creating and instantiating chaincode on IBM Blockchain is a breeze, but there are some caveats we need to overcome to get the most of their platform.

This post was written on my blog.

Testing chaincode on Hyperledger Fabric

Javier Aguirre — Sun, 19 May 2019 22:10:20 +0000

Hyperledger is the Blockchain family of tools and frameworks for permissioned networks. We've been working with solutions built with Hyperledger Fabric for a while now.

Hyperledger Fabric is a complex system to deploy and it’s important to be careful when we configure certificates, open ports or assign roles, among other things. Luckily there are some good documentation articles from Hyperledger you could read to extend your knowledge if you are interested, such as how to build your first network or writing your first application.

Our workflow integrates different platforms in our pipeline: It starts on Github, goes to Gitlab CI and finally deploys the smart contract on the IBM Blockchain platform.

Today we want to talk about the first part that makes our development faster, the unit, integration and acceptance tests for the chaincode.

Test all the things!

We have automated testing at our core. We apply Continuous Integration and Continuous Delivery in all our projects,
so it made sense for us to create a more robust test suite for our chaincode.

We decided to go with Go, static typing, Go structs and the examples in the Hyperledger Fabric official repositories made testing easy and straightforward.

To make Hyperledger Fabric chaincode easier to test, the developers have created a MockStub struct. A mock is a service simulating functionality so you don’t call the real one while testing. For example, if we use a service to send SMS, we don’t want our tests to send SMS every time we run them, that’s why we usually use a mock, to simulate we call the method and its response. There are still several commands that aren’t implemented in the mock needed for some of the MockStub.

Run the tests

To run the tests locally, we use a Dockerfile that builds our environment. The Dockerfile would be something like this:

We’re using Go v1.10 and copying our project code to /go/src/app.

We need to install the dependencies, we’ll use hyperledger/fabric-sdk-go, testify/assert to have a nice way of asserting in our tests and optionally if we do acceptance tests, cmd/godog.

We’ll need libltdl-dev and install in the Fabric library manually in the $GOPATH, it didn’t work for us at the time, but this might be easier with the new version of Fabric.

We define the path where our chaincode lives ($GOPATH/src/gitlab.com/TheNeonProject/mychaincode) and run the tests with go test -v ..

To run this locally we use two commands, one to build the Docker container, and another one to run the tests.

We’ll get a response like this if everything went nicely.

Code Example

But talk is cheap, so we’ll show you a code example. :-)

Here you have a chaincode example with a method getAsset. Read through to see how it looks and we’ll explain right afterwards.

This code is for a smart contract with a method getAsset.

The Invoke method is the entry point when calling our smart contract via RPC; if the method is not supported it will return an error.
That would be a very good initial test to check if our test environment is correctly configured. :-)

The getObjectAsBytes method is just used to return a saved object as a response in the method call.

So going back to the getAsset method, it receives a parameter, which is the ID of the resource and, if it exists, it returns the result as JSON.

Now you know what the code does, let’s create some tests for this beast, because, as great chefs do, we always need to taste the recipe before serving.

Tests

We’re going to create a test suite with two main test cases:

Test when the ID sent doesn’t exist, we’ll assert the error.
Test when the ID sent does exist, we’ll assert the response is good and an attribute of the object (the name) is present.

We follow AAA (Arrange Act Asset) principle to see the different parts of the test cases easily. In the different part of the test cases we can see:

In this case, the Arrange would need to initialize the smart contract “connection” and, in case of a valid id, create the Asset with its name.

In the Act part of the test, we’ll do mostly the same, use the method name, the ID, and in case of creation or append(edit), the JSON object
with the payload.

In the Assert we’ll check if the response is OK or Error, and we can also check with the variables res.Message or res.Payload if the content
of the response is what the right one.

That’s it for now! In following articles we’ll talk about how to do BDD for the chaincode, data format manipulation for chaincode methods and Continuous Integration / Continuous Deployment for our chaincode on an Hyperledger Fabric network.

This post was written appeared first on my blog.