Forem: Koichi

What is gRPC

Koichi — Mon, 13 Mar 2023 00:34:14 +0000

Introduction

gRPC is an open-source framework for high-performance remote procedure calls created by Google. It is commonly used for Server-To-Server communication in Microservice architecture because it allows a server to connect with another server even written in a different programming language.

What is RPC

RPC stands for Remote Procedure Call. It is a protocol that allows a client to call a function on a server and get a response back. In this context, A client can be a server that wants to send a request to another server.

A client takes a gRPC Stub, a kind of instance that provides functions/methods of a server, and calls a function from the instance.

Advantages of gRPC

One advantage is data transportation. gRPC is built on an HTTP/2 protocol that enhances data transfer performance and efficiency.
gRPC also uses Protocol Buffers, a data serialization format, to encode data. Protocol Buffers are smaller in size than JSON, which means that the data transfer is faster.

HTTP/2

HTTP/2 is a new version of the HTTP protocol that allows multiple requests over a single connection. It also enables to send of binary data, which is a more efficient way to send than text data.

Protocol Buffers

Protocol Buffers are also called Protobuf. You can define a data schema with a .proto file:

syntax = "proto3";

message Example {
    int32 id = 1;
    string name = 2;
}

The numbers 1 and 2 is the field number, different from their actual values.

This example corresponds with this JSON structure:

{
    "id": ,
    "name":
}

Protobuf supports multiple language platforms and this feature enables gRPC to let servers written in several languages connect.

I also wrote a post about Protocol Buffers, so if you want to know more about it, please check it out.
What are Protocol Buffers

And this is an example of .proto file for gRPC:

message Example {
  int32 id = 1;
  string name = 2;
  string header = 3;
  string body = 4;
}

message ExampleRequest {
  Example example = 1;
}

message ExampleResponse {
  string result = 1;
}
service ExampleService {
  rpc callExample(ExampleRequest) returns (ExampleResponse) {};
}

// function ExampleService
@Override
public void callExample(ExampleRequest request, StreamObserver<ExampleResponse> responseObserver) {
  Example example = request.getExample();
// do Something
}


const client = getClient();

const exampleRequest:ExampleRequest = {/** some props **/}
const response = this.client.callExample(exampleRequest);

This diagram shows how gRPC works:

4 Types of API calls

gRPC has realized several types of API calls:

Unary RPC
Server streaming RPC
Client streaming RPC
Bidirectional streaming RPC

Unary API

Unary API is a basic API call that consists of 1 request and 1 response. It is the most common type of API call.
The definition of the API call is as follows:

message Example {
    int32 id = 1;
    string name = 2;
    string header = 3;
    string body = 4;
}

message ExampleRequest {
  Example example = 1;
}

message ExampleResponse {
  string result = 1;
}
service ExampleService {
  rpc Example(ExampleRequest) returns (ExampleResponse) {};
}

The message keyword is used to define a new message type called Example. The service keyword defines a type of service that is supposed to be used by a client, and the rpc keyword defines a remote procedure call method.

Server streaming RPC

Streaming Server API is an API call that consists of 1 request and multiple responses. The server sends multiple responses to the client. The proto file uses the stream keyword to define a streaming data flow that is multiple data sendings.

message ExampleServerStreamingRequest {
  Example example = 1;
}

message ExampleServerStreamingResponse {
  string result = 1;
}
service ExampleService {
  rpc ExampleServerStreaming(ExampleServerStreamingRequest) returns (stream ExampleServerStreamingResponse) {};
}

Client streaming RPC

Streaming Client API is opposite to the Streaming Server API, which allows clients to send multiple requests. The stream keyword is attached to the request in the rpc declaration.

message ClientStreamingExampleRequest {
    Example example = 1;
}

message ClientStreamingExampleResponse {
    string result = 1;
}

service ExampleService {
    rpc ClientStreamingExample(stream ClientStreamingExampleRequest) returns (ClientStreamingExampleResponse) {};
}

Bidirectional streaming RPC

Bi-directional Streaming API is an API call that allows both the client and the server to send multiple requests and responses.

message ExampleBiDirectionRequest {
    Example example = 1;
}

message ExampleBiDirectionResponse {
    string result = 1;
}
service ExampleService {
    rpc ExampleBiDirection(stream ExampleBiDirectionRequest) returns (stream ExampleBiDirectionResponse) {};
}

gRPC-web

gRPC is mainly used in server-to-server communication, but it can also be used in client-to-server communication. gRPC-web is a gRPC implementation for web browsers. It is a JavaScript library that allows you to call gRPC services from a web browser. It supports Unary and Streaming Server API calls.

Conclusion

gRPC gives advantages to data transfer performance and efficiency and also enables several types of communication styles.
gRPC is a good choice if your servers require to communicate with other servers frequently.

Reference:

https://grpc.io/docs/what-is-grpc/introduction/

What are Protocol Buffers

Koichi — Tue, 21 Feb 2023 02:05:07 +0000

Summary

Protocol Buffers is a method for serializing structured data developed by Google. The primary usage is to facilitate structured data exchange between different applications and systems.

Several data formats are used to exchange data, such as XML, CSV or JSON. They are still commonly used in web development, but each of them has disadvantages in some aspects. Protocol Buffers were developed to solve some of these drawbacks.

This article will mention the newest version, proto3.

Advantages

There are several reasons why using Protocol Buffers is a better choice rather than other options:

Smaller data size
Fast encoding and decoding
Automatically generate code
Support a variety of programming languages

Smaller data size

Protocol Buffers in a binary format, meaning that it uses only 0s and 1s to represent the data and does not include any extra metadata or formatting like XML or JSON. Additionally, Protocol Buffers' compact encoding method allows it to compress data more efficiently, resulting in smaller message sizes.

Fast encoding and decoding

Another benefit of Protocol Buffers is their speed. Because it is a binary format, it can be encoded and decoded much more quickly than text-based formats like XML and JSON, which require additional processing to parse the text and convert it into a usable format. This makes Protocol Buffers a good choice for systems that require fast and efficient data exchange.

Automatically generate code

Protocol Buffers are designed to be language- and platform-independent, meaning that they can be used with any programming language and on any platform that has a code generator for the format. This allows systems and applications written in different languages to communicate with each other using a common data exchange format, regardless of the programming language or platform they were developed on.

Support a variety of programming languages

The Protocol Buffers compiler, protoc, supports several types of programming languages: C++, C♯, Java, Kotlin, Objective-C, PHP, Python, and Ruby.

Workflow

The workflow of Protocol Buffers is straightforward and easy to use. Firstly, you define the schema with a .proto file, and the protocol compiler generates code to serialize and deserialize data in your application. Your application uses the generated code to send or receive the data.

Basic syntax

The schema definition file is called .proto file, and it defines data schema.

syntax="proto3";

message Person {
  int32 id = 1;
  string name = 2
}

In this example, syntax="proto3" specifies that the .proto file is written in version 3 of Protocol Buffers. The message keyword is used to define a new message type called Person. Inside the Person message type, there are two fields: id and name. int32 and string are the field types, and 1 and 2 are field tags. Field tags are used to identify each field in the binary data stream and are required for proper decoding of the data.

Common usage

Protocol Buffers is widely used in various applications and systems. One of its most popular use cases is in the gRPC framework, which is a modern, high-performance, open-source RPC (Remote Procedure Call) framework that uses Protocol Buffers as its default data format. gRPC allows developers to build distributed systems that communicate with each other using a simple and efficient RPC interface, making it a great choice for building microservices and other distributed applications.

Code Example

Finally, quickly check what the generated code looks like.
When the protocol buffer compiler generates a Java class code for this example .proto file,

syntax="proto3";

message Person {
  int32 id = 1;
  string name = 2
}

The generated code will be here, click the collapse mark.

generated Java file

// Generated by the protocol buffer compiler.  DO NOT EDIT!
// source: person.proto

public final class PersonOuterClass {
  private PersonOuterClass() {}
  public static void registerAllExtensions(
      com.google.protobuf.ExtensionRegistryLite registry) {
  }

  public static void registerAllExtensions(
      com.google.protobuf.ExtensionRegistry registry) {
    registerAllExtensions(
        (com.google.protobuf.ExtensionRegistryLite) registry);
  }
  public interface PersonOrBuilder extends
      // @@protoc_insertion_point(interface_extends:Person)
      com.google.protobuf.MessageOrBuilder {

    /**
     * <code>int32 id = 1;</code>
     * @return The id.
     */
    int getId();

    /**
     * <code>string name = 2;</code>
     * @return The name.
     */
    java.lang.String getName();
    /**
     * <code>string name = 2;</code>
     * @return The bytes for name.
     */
    com.google.protobuf.ByteString
        getNameBytes();
  }
  /**
   * Protobuf type {@code Person}
   */
  public static final class Person extends
      com.google.protobuf.GeneratedMessageV3 implements
      // @@protoc_insertion_point(message_implements:Person)
      PersonOrBuilder {
  private static final long serialVersionUID = 0L;
    // Use Person.newBuilder() to construct.
    private Person(com.google.protobuf.GeneratedMessageV3.Builder<?> builder) {
      super(builder);
    }
    private Person() {
      name_ = "";
    }

    @java.lang.Override
    @SuppressWarnings({"unused"})
    protected java.lang.Object newInstance(
        UnusedPrivateParameter unused) {
      return new Person();
    }

    @java.lang.Override
    public final com.google.protobuf.UnknownFieldSet
    getUnknownFields() {
      return this.unknownFields;
    }
    private Person(
        com.google.protobuf.CodedInputStream input,
        com.google.protobuf.ExtensionRegistryLite extensionRegistry)
        throws com.google.protobuf.InvalidProtocolBufferException {
      this();
      if (extensionRegistry == null) {
        throw new java.lang.NullPointerException();
      }
      com.google.protobuf.UnknownFieldSet.Builder unknownFields =
          com.google.protobuf.UnknownFieldSet.newBuilder();
      try {
        boolean done = false;
        while (!done) {
          int tag = input.readTag();
          switch (tag) {
            case 0:
              done = true;
              break;
            case 8: {

              id_ = input.readInt32();
              break;
            }
            case 18: {
              java.lang.String s = input.readStringRequireUtf8();

              name_ = s;
              break;
            }
            default: {
              if (!parseUnknownField(
                  input, unknownFields, extensionRegistry, tag)) {
                done = true;
              }
              break;
            }
          }
        }
      } catch (com.google.protobuf.InvalidProtocolBufferException e) {
        throw e.setUnfinishedMessage(this);
      } catch (com.google.protobuf.UninitializedMessageException e) {
        throw e.asInvalidProtocolBufferException().setUnfinishedMessage(this);
      } catch (java.io.IOException e) {
        throw new com.google.protobuf.InvalidProtocolBufferException(
            e).setUnfinishedMessage(this);
      } finally {
        this.unknownFields = unknownFields.build();
        makeExtensionsImmutable();
      }
    }
    public static final com.google.protobuf.Descriptors.Descriptor
        getDescriptor() {
      return PersonOuterClass.internal_static_Person_descriptor;
    }

    @java.lang.Override
    protected com.google.protobuf.GeneratedMessageV3.FieldAccessorTable
        internalGetFieldAccessorTable() {
      return PersonOuterClass.internal_static_Person_fieldAccessorTable
          .ensureFieldAccessorsInitialized(
              PersonOuterClass.Person.class, PersonOuterClass.Person.Builder.class);
    }

    public static final int ID_FIELD_NUMBER = 1;
    private int id_;
    /**
     * <code>int32 id = 1;</code>
     * @return The id.
     */
    @java.lang.Override
    public int getId() {
      return id_;
    }

    public static final int NAME_FIELD_NUMBER = 2;
    private volatile java.lang.Object name_;
    /**
     * <code>string name = 2;</code>
     * @return The name.
     */
    @java.lang.Override
    public java.lang.String getName() {
      java.lang.Object ref = name_;
      if (ref instanceof java.lang.String) {
        return (java.lang.String) ref;
      } else {
        com.google.protobuf.ByteString bs = 
            (com.google.protobuf.ByteString) ref;
        java.lang.String s = bs.toStringUtf8();
        name_ = s;
        return s;
      }
    }
    /**
     * <code>string name = 2;</code>
     * @return The bytes for name.
     */
    @java.lang.Override
    public com.google.protobuf.ByteString
        getNameBytes() {
      java.lang.Object ref = name_;
      if (ref instanceof java.lang.String) {
        com.google.protobuf.ByteString b = 
            com.google.protobuf.ByteString.copyFromUtf8(
                (java.lang.String) ref);
        name_ = b;
        return b;
      } else {
        return (com.google.protobuf.ByteString) ref;
      }
    }

    private byte memoizedIsInitialized = -1;
    @java.lang.Override
    public final boolean isInitialized() {
      byte isInitialized = memoizedIsInitialized;
      if (isInitialized == 1) return true;
      if (isInitialized == 0) return false;

      memoizedIsInitialized = 1;
      return true;
    }

    @java.lang.Override
    public void writeTo(com.google.protobuf.CodedOutputStream output)
                        throws java.io.IOException {
      if (id_ != 0) {
        output.writeInt32(1, id_);
      }
      if (!com.google.protobuf.GeneratedMessageV3.isStringEmpty(name_)) {
        com.google.protobuf.GeneratedMessageV3.writeString(output, 2, name_);
      }
      unknownFields.writeTo(output);
    }

    @java.lang.Override
    public int getSerializedSize() {
      int size = memoizedSize;
      if (size != -1) return size;

      size = 0;
      if (id_ != 0) {
        size += com.google.protobuf.CodedOutputStream
          .computeInt32Size(1, id_);
      }
      if (!com.google.protobuf.GeneratedMessageV3.isStringEmpty(name_)) {
        size += com.google.protobuf.GeneratedMessageV3.computeStringSize(2, name_);
      }
      size += unknownFields.getSerializedSize();
      memoizedSize = size;
      return size;
    }

    @java.lang.Override
    public boolean equals(final java.lang.Object obj) {
      if (obj == this) {
       return true;
      }
      if (!(obj instanceof PersonOuterClass.Person)) {
        return super.equals(obj);
      }
      PersonOuterClass.Person other = (PersonOuterClass.Person) obj;

      if (getId()
          != other.getId()) return false;
      if (!getName()
          .equals(other.getName())) return false;
      if (!unknownFields.equals(other.unknownFields)) return false;
      return true;
    }

    @java.lang.Override
    public int hashCode() {
      if (memoizedHashCode != 0) {
        return memoizedHashCode;
      }
      int hash = 41;
      hash = (19 * hash) + getDescriptor().hashCode();
      hash = (37 * hash) + ID_FIELD_NUMBER;
      hash = (53 * hash) + getId();
      hash = (37 * hash) + NAME_FIELD_NUMBER;
      hash = (53 * hash) + getName().hashCode();
      hash = (29 * hash) + unknownFields.hashCode();
      memoizedHashCode = hash;
      return hash;
    }

    public static PersonOuterClass.Person parseFrom(
        java.nio.ByteBuffer data)
        throws com.google.protobuf.InvalidProtocolBufferException {
      return PARSER.parseFrom(data);
    }
    public static PersonOuterClass.Person parseFrom(
        java.nio.ByteBuffer data,
        com.google.protobuf.ExtensionRegistryLite extensionRegistry)
        throws com.google.protobuf.InvalidProtocolBufferException {
      return PARSER.parseFrom(data, extensionRegistry);
    }
    public static PersonOuterClass.Person parseFrom(
        com.google.protobuf.ByteString data)
        throws com.google.protobuf.InvalidProtocolBufferException {
      return PARSER.parseFrom(data);
    }
    public static PersonOuterClass.Person parseFrom(
        com.google.protobuf.ByteString data,
        com.google.protobuf.ExtensionRegistryLite extensionRegistry)
        throws com.google.protobuf.InvalidProtocolBufferException {
      return PARSER.parseFrom(data, extensionRegistry);
    }
    public static PersonOuterClass.Person parseFrom(byte[] data)
        throws com.google.protobuf.InvalidProtocolBufferException {
      return PARSER.parseFrom(data);
    }
    public static PersonOuterClass.Person parseFrom(
        byte[] data,
        com.google.protobuf.ExtensionRegistryLite extensionRegistry)
        throws com.google.protobuf.InvalidProtocolBufferException {
      return PARSER.parseFrom(data, extensionRegistry);
    }
    public static PersonOuterClass.Person parseFrom(java.io.InputStream input)
        throws java.io.IOException {
      return com.google.protobuf.GeneratedMessageV3
          .parseWithIOException(PARSER, input);
    }
    public static PersonOuterClass.Person parseFrom(
        java.io.InputStream input,
        com.google.protobuf.ExtensionRegistryLite extensionRegistry)
        throws java.io.IOException {
      return com.google.protobuf.GeneratedMessageV3
          .parseWithIOException(PARSER, input, extensionRegistry);
    }
    public static PersonOuterClass.Person parseDelimitedFrom(java.io.InputStream input)
        throws java.io.IOException {
      return com.google.protobuf.GeneratedMessageV3
          .parseDelimitedWithIOException(PARSER, input);
    }
    public static PersonOuterClass.Person parseDelimitedFrom(
        java.io.InputStream input,
        com.google.protobuf.ExtensionRegistryLite extensionRegistry)
        throws java.io.IOException {
      return com.google.protobuf.GeneratedMessageV3
          .parseDelimitedWithIOException(PARSER, input, extensionRegistry);
    }
    public static PersonOuterClass.Person parseFrom(
        com.google.protobuf.CodedInputStream input)
        throws java.io.IOException {
      return com.google.protobuf.GeneratedMessageV3
          .parseWithIOException(PARSER, input);
    }
    public static PersonOuterClass.Person parseFrom(
        com.google.protobuf.CodedInputStream input,
        com.google.protobuf.ExtensionRegistryLite extensionRegistry)
        throws java.io.IOException {
      return com.google.protobuf.GeneratedMessageV3
          .parseWithIOException(PARSER, input, extensionRegistry);
    }

    @java.lang.Override
    public Builder newBuilderForType() { return newBuilder(); }
    public static Builder newBuilder() {
      return DEFAULT_INSTANCE.toBuilder();
    }
    public static Builder newBuilder(PersonOuterClass.Person prototype) {
      return DEFAULT_INSTANCE.toBuilder().mergeFrom(prototype);
    }
    @java.lang.Override
    public Builder toBuilder() {
      return this == DEFAULT_INSTANCE
          ? new Builder() : new Builder().mergeFrom(this);
    }

    @java.lang.Override
    protected Builder newBuilderForType(
        com.google.protobuf.GeneratedMessageV3.BuilderParent parent) {
      Builder builder = new Builder(parent);
      return builder;
    }
    /**
     * Protobuf type {@code Person}
     */
    public static final class Builder extends
        com.google.protobuf.GeneratedMessageV3.Builder<Builder> implements
        // @@protoc_insertion_point(builder_implements:Person)
        PersonOuterClass.PersonOrBuilder {
      public static final com.google.protobuf.Descriptors.Descriptor
          getDescriptor() {
        return PersonOuterClass.internal_static_Person_descriptor;
      }

      @java.lang.Override
      protected com.google.protobuf.GeneratedMessageV3.FieldAccessorTable
          internalGetFieldAccessorTable() {
        return PersonOuterClass.internal_static_Person_fieldAccessorTable
            .ensureFieldAccessorsInitialized(
                PersonOuterClass.Person.class, PersonOuterClass.Person.Builder.class);
      }

      // Construct using PersonOuterClass.Person.newBuilder()
      private Builder() {
        maybeForceBuilderInitialization();
      }

      private Builder(
          com.google.protobuf.GeneratedMessageV3.BuilderParent parent) {
        super(parent);
        maybeForceBuilderInitialization();
      }
      private void maybeForceBuilderInitialization() {
        if (com.google.protobuf.GeneratedMessageV3
                .alwaysUseFieldBuilders) {
        }
      }
      @java.lang.Override
      public Builder clear() {
        super.clear();
        id_ = 0;

        name_ = "";

        return this;
      }

      @java.lang.Override
      public com.google.protobuf.Descriptors.Descriptor
          getDescriptorForType() {
        return PersonOuterClass.internal_static_Person_descriptor;
      }

      @java.lang.Override
      public PersonOuterClass.Person getDefaultInstanceForType() {
        return PersonOuterClass.Person.getDefaultInstance();
      }

      @java.lang.Override
      public PersonOuterClass.Person build() {
        PersonOuterClass.Person result = buildPartial();
        if (!result.isInitialized()) {
          throw newUninitializedMessageException(result);
        }
        return result;
      }

      @java.lang.Override
      public PersonOuterClass.Person buildPartial() {
        PersonOuterClass.Person result = new PersonOuterClass.Person(this);
        result.id_ = id_;
        result.name_ = name_;
        onBuilt();
        return result;
      }

      @java.lang.Override
      public Builder clone() {
        return super.clone();
      }
      @java.lang.Override
      public Builder setField(
          com.google.protobuf.Descriptors.FieldDescriptor field,
          java.lang.Object value) {
        return super.setField(field, value);
      }
      @java.lang.Override
      public Builder clearField(
          com.google.protobuf.Descriptors.FieldDescriptor field) {
        return super.clearField(field);
      }
      @java.lang.Override
      public Builder clearOneof(
          com.google.protobuf.Descriptors.OneofDescriptor oneof) {
        return super.clearOneof(oneof);
      }
      @java.lang.Override
      public Builder setRepeatedField(
          com.google.protobuf.Descriptors.FieldDescriptor field,
          int index, java.lang.Object value) {
        return super.setRepeatedField(field, index, value);
      }
      @java.lang.Override
      public Builder addRepeatedField(
          com.google.protobuf.Descriptors.FieldDescriptor field,
          java.lang.Object value) {
        return super.addRepeatedField(field, value);
      }
      @java.lang.Override
      public Builder mergeFrom(com.google.protobuf.Message other) {
        if (other instanceof PersonOuterClass.Person) {
          return mergeFrom((PersonOuterClass.Person)other);
        } else {
          super.mergeFrom(other);
          return this;
        }
      }

      public Builder mergeFrom(PersonOuterClass.Person other) {
        if (other == PersonOuterClass.Person.getDefaultInstance()) return this;
        if (other.getId() != 0) {
          setId(other.getId());
        }
        if (!other.getName().isEmpty()) {
          name_ = other.name_;
          onChanged();
        }
        this.mergeUnknownFields(other.unknownFields);
        onChanged();
        return this;
      }

      @java.lang.Override
      public final boolean isInitialized() {
        return true;
      }

      @java.lang.Override
      public Builder mergeFrom(
          com.google.protobuf.CodedInputStream input,
          com.google.protobuf.ExtensionRegistryLite extensionRegistry)
          throws java.io.IOException {
        PersonOuterClass.Person parsedMessage = null;
        try {
          parsedMessage = PARSER.parsePartialFrom(input, extensionRegistry);
        } catch (com.google.protobuf.InvalidProtocolBufferException e) {
          parsedMessage = (PersonOuterClass.Person) e.getUnfinishedMessage();
          throw e.unwrapIOException();
        } finally {
          if (parsedMessage != null) {
            mergeFrom(parsedMessage);
          }
        }
        return this;
      }

      private int id_ ;
      /**
       * <code>int32 id = 1;</code>
       * @return The id.
       */
      @java.lang.Override
      public int getId() {
        return id_;
      }
      /**
       * <code>int32 id = 1;</code>
       * @param value The id to set.
       * @return This builder for chaining.
       */
      public Builder setId(int value) {

        id_ = value;
        onChanged();
        return this;
      }
      /**
       * <code>int32 id = 1;</code>
       * @return This builder for chaining.
       */
      public Builder clearId() {

        id_ = 0;
        onChanged();
        return this;
      }

      private java.lang.Object name_ = "";
      /**
       * <code>string name = 2;</code>
       * @return The name.
       */
      public java.lang.String getName() {
        java.lang.Object ref = name_;
        if (!(ref instanceof java.lang.String)) {
          com.google.protobuf.ByteString bs =
              (com.google.protobuf.ByteString) ref;
          java.lang.String s = bs.toStringUtf8();
          name_ = s;
          return s;
        } else {
          return (java.lang.String) ref;
        }
      }
      /**
       * <code>string name = 2;</code>
       * @return The bytes for name.
       */
      public com.google.protobuf.ByteString
          getNameBytes() {
        java.lang.Object ref = name_;
        if (ref instanceof String) {
          com.google.protobuf.ByteString b = 
              com.google.protobuf.ByteString.copyFromUtf8(
                  (java.lang.String) ref);
          name_ = b;
          return b;
        } else {
          return (com.google.protobuf.ByteString) ref;
        }
      }
      /**
       * <code>string name = 2;</code>
       * @param value The name to set.
       * @return This builder for chaining.
       */
      public Builder setName(
          java.lang.String value) {
        if (value == null) {
    throw new NullPointerException();
  }

        name_ = value;
        onChanged();
        return this;
      }
      /**
       * <code>string name = 2;</code>
       * @return This builder for chaining.
       */
      public Builder clearName() {

        name_ = getDefaultInstance().getName();
        onChanged();
        return this;
      }
      /**
       * <code>string name = 2;</code>
       * @param value The bytes for name to set.
       * @return This builder for chaining.
       */
      public Builder setNameBytes(
          com.google.protobuf.ByteString value) {
        if (value == null) {
    throw new NullPointerException();
  }
  checkByteStringIsUtf8(value);

        name_ = value;
        onChanged();
        return this;
      }
      @java.lang.Override
      public final Builder setUnknownFields(
          final com.google.protobuf.UnknownFieldSet unknownFields) {
        return super.setUnknownFields(unknownFields);
      }

      @java.lang.Override
      public final Builder mergeUnknownFields(
          final com.google.protobuf.UnknownFieldSet unknownFields) {
        return super.mergeUnknownFields(unknownFields);
      }


      // @@protoc_insertion_point(builder_scope:Person)
    }

    // @@protoc_insertion_point(class_scope:Person)
    private static final PersonOuterClass.Person DEFAULT_INSTANCE;
    static {
      DEFAULT_INSTANCE = new PersonOuterClass.Person();
    }

    public static PersonOuterClass.Person getDefaultInstance() {
      return DEFAULT_INSTANCE;
    }

    private static final com.google.protobuf.Parser<Person>
        PARSER = new com.google.protobuf.AbstractParser<Person>() {
      @java.lang.Override
      public Person parsePartialFrom(
          com.google.protobuf.CodedInputStream input,
          com.google.protobuf.ExtensionRegistryLite extensionRegistry)
          throws com.google.protobuf.InvalidProtocolBufferException {
        return new Person(input, extensionRegistry);
      }
    };

    public static com.google.protobuf.Parser<Person> parser() {
      return PARSER;
    }

    @java.lang.Override
    public com.google.protobuf.Parser<Person> getParserForType() {
      return PARSER;
    }

    @java.lang.Override
    public PersonOuterClass.Person getDefaultInstanceForType() {
      return DEFAULT_INSTANCE;
    }

  }

  private static final com.google.protobuf.Descriptors.Descriptor
    internal_static_Person_descriptor;
  private static final 
    com.google.protobuf.GeneratedMessageV3.FieldAccessorTable
      internal_static_Person_fieldAccessorTable;

  public static com.google.protobuf.Descriptors.FileDescriptor
      getDescriptor() {
    return descriptor;
  }
  private static  com.google.protobuf.Descriptors.FileDescriptor
      descriptor;
  static {
    java.lang.String[] descriptorData = {
      "\n\014person.proto\"\"\n\006Person\022\n\n\002id\030\001 \001(\005\022\014\n\004" +
      "name\030\002 \001(\tb\006proto3"
    };
    descriptor = com.google.protobuf.Descriptors.FileDescriptor
      .internalBuildGeneratedFileFrom(descriptorData,
        new com.google.protobuf.Descriptors.FileDescriptor[] {
        });
    internal_static_Person_descriptor =
      getDescriptor().getMessageTypes().get(0);
    internal_static_Person_fieldAccessorTable = new
      com.google.protobuf.GeneratedMessageV3.FieldAccessorTable(
        internal_static_Person_descriptor,
        new java.lang.String[] { "Id", "Name", });
  }

  // @@protoc_insertion_point(outer_class_scope)
}

Conclusion

In conclusion, Protocol Buffers are a powerful method for serializing structured data to facilitate data exchange between applications and systems. Its advantages include smaller data size, fast encoding and decoding, automatic code generation, and support for various programming languages. The basic syntax is straightforward and widely used in multiple applications, especially in the gRPC framework. Using Protocol Buffers, developers can quickly build distributed systems that communicate with each other using a simple and efficient RPC interface, making it an excellent choice for building microservices and other distributed applications.

References

Protocol Buffers Documentation

The difference between primitives and objects in JavaScript

Koichi — Thu, 16 Feb 2023 02:49:42 +0000

What are data types in JavaScript

JavaScript has several data types to represent a variety of values. These types are mainly categorized into 2 types: primitive types and object types.

The understanding of the difference

The fundamental data types are treated as primitives, and the others are considered objects. Objects can contain multiple primitives values as a data set and also have methods.

Primitives:

Fundamental data type
Single value
Immutable - can not be modified
Does not have methods and properties
Stored in stack memory

Objects:

Complex data type
A data collection have multiple values
Mutable - can be modified
Consists of properties and methods
Stored in Heap memory

Primitives

These are primitive types:

number
string
boolean
null
undefined
symbol

Primitive data types are immutable, meaning that their values cannot be changed once they are created.

Objects

An object is a complex data type that represents a collection of properties and methods. A method is a function, a process of logic. Objects are also known as reference types. The actual data is stored in Heap memory, and a variable holds the location information of the reserved area.

If you want to know more about memories, this article must help your understanding:

Wrapper objects

Even though primitives have neither properties nor methods, we can act to access them.
Example:

"string".length; // 5
"lower".toUpperCase() // LOWER

That is because of the implicit support of objects called wrapper objects. Wrapper objects exist globally and help primitive types. Their names are quite similar to the data type, like Number or String. Their spells start in upper-case. When a script attempts to access a property of a primitive value, JavaScript automatically creates the wrapper object around the primitive value to provide access to these properties.

Conclusion

The main difference between primitives and objects is that primitives are simple, immutable values, while objects are complex, mutable entities that can contain properties and methods. To understand the difference seems hard; however, it must help your developer life!

References

How to automate code format on commit

Koichi — Sun, 16 Oct 2022 23:05:40 +0000

Keeping coding conventions is strikingly important for team development; however, being careful is not enough as the team size increases. Automation will solve this problem by fixing different coding styles.

How this automation work

This method uses the npm script to check and fix codes automatically. The best timing is on each git commit.

Tools

Here are the main tools:

ESLint
Husky

ESLint is a code check tool based on custom configurable rules and Husky enables execution of the code fixing command on commit timing.

How to set up

Let's get started from scratch! The process is divided into 3 parts:

Set up the project
Write npm script
Set pre-commit hook

Pre-conditions

First of all, an important thing is the project structure. It requires setting a git repository and the repository should be for one project and includes only either client or server side. In other words, the monorepo structure is not covered in this article.

1. Set up the project

Make a folder and a file

make one file called index.js.

const express = require('express')
const app = express()
const port = 3000

app.get('/', (req, res) => {
  res.send('Hello World!')
})

app.listen(port, () => {
  console.log(`Example app listening on port ${port}`)
})

But actually, naming and the file content is ok whatever you want.

Set up git and npm packages

If your project is not initialized by git and npm, please set it by using these commands.

git init
npm init -y

In this article, the test code uses express.js, so please install express

npm install express

And you need to have ESLint library. There is a convenient command to set up the ESLint configuration

npm init @eslint/config

You can select the lint rules by choosing several options.

For now, your project structure looks like this:

/.git
/node_modules
/.eslintrc.js
/index.js
/package.json
/package-lock.json

Now, you will see some errors in the index.js based on the lint rule.

If you do not see any error, please set the content below in .eslintrc.js

module.exports = {
  env: {
    browser: true,
    commonjs: true,
    es2021: true
  },
  extends: "standard",
  overrides: [
  ],
  parserOptions: {
    ecmaVersion: "latest"
  },
  rules: {
    quotes: [2, "double", "avoid-escape"]
  }
}

2. Write npm script

Add 2 scripts in package.json.

package.json

  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1",
    "lint": "eslint index.js",
    "lint:fix": "eslint --fix index.js"
  },

The lint script just checks and shows errors. Let's try it.

npm run lint

THe output looks like that.

The next command, 'link:fix' will fix the errors.

npm run lint:fix

Now, you can see the errors disappeared!
But, please return the code to check the next process.

3. Set the pre-commit hook

The next setting is to make a hook on commit.
Run this command:

npx husky-init && npm install

You can see that the /.husky folder has been generated!

Open the /.husky/pre-commit, and change the content as below:

#!/usr/bin/env sh
. "$(dirname -- "$0")/_/husky.sh"

npm run lint:fix
# This command should be the same command that you executed the previous process

# Optional
#exit 1

Optional:
If you want to just try the hook and do not want to commit anything, add exit 1 to the last line.

To register the hook, please add & commit ./husky/pre-commit.

git add .husky/pre-commit && git commit -m "Add pre-commit"

Now, time to test the automation. Please execute the command:

git add index.js && git commit -m "test commit"

When you check index.js, you can see the errors were removed again.

Conclusion

Developers will be released to be always careful on coding conventions by this automation. Even if they forget some rules, ESLint helps to keep the rules once the developer commits.

Reference:
https://expressjs.com/en/starter/hello-world.html
https://www.npmjs.com/package/eslint
https://typicode.github.io/husky/

3 good things of OOP approach in Express.js with TypeScript

Koichi — Mon, 10 Oct 2022 02:54:24 +0000

What TypeScript enables

TypeScript provides a strict typing system for JavaScript projects and saves programmers from errors during coding. In addition, it enables to use of Object-Oriented Programming(OOP) style by giving the 'Interface' and 'Abstract' class.

Not only for the client-side

Using TypeScript has become popular on the client side, but it is also applicable to server-side JavaScript. Express.js is a web framework of Node.js, a runtime environment that enables JavaScript codes to work on the server side, and TypeScript is also used with Express.

Common style of Express

Typical Express code is written in the functional programming style, as it takes a callback function at once.

const express = require('express');
const app = express();
app.get('/', (req, res) => {
  res.send('Hello world');
});

But, when you use TypeScript, you can make use of the Class-based style and see several merits of it.

3 advantages of using TypeScript with OOP

These are the positive merits when you take OOP approach:

Reduce the number of type declarations
Utilize design pattern
Use Decorators

Reduce the number of type declarations

When you write controllers, you need to export all variables and type declarations. Having a controller class can put them together.

This is an example:
controller/users.ts

export class UsersController {
  public async createUser(req: Request, res: Response) {
    const user = new User(req.body);
    await user.save();
    res.status(201).send(user);
  }

  public async fetchUsers(req: Request, res: Response) {
    const users = await User.find({});
    res.status(200).send(users);
  }
}

The router should know the controller is UsersController and detect its type.

routes/users.ts

import { Router } from 'express';
import { UsersController } from '../controllers/users';

const router = Router();
const usersController = new UsersController();
router.get('/', usersController.fetchUsers);
router.get('/create', usersController.createUser);

You can also make a router class.

Design patterns

Design patterns are common solution patterns of software development, they go with the OOP.

GoF Design Patterns
DAO Design Pattern

Decorators

A Decorator is a special kind of declaration that can be attached to a class declaration, method, accessor, property, or parameter.
From TypeScript Official Site
Decorators are TypeScript specialty, and it is based on the OOP approach. For example, you can execute some processes before a class instance is generated.

const Something = (constructor: Function) => {
  console.log('Some functions before the class instantiate');
}

@Something
class User {
  constructor(name: string) {
    console.log(`User ${name} is created`);
  }
}

const user1 = new User('user2');
const user2 = new User('user1');

The output is

You can set more than one decorators.

Conclusion

You can make the most of OOP and the unique features of TypeScript in Express.　
You may also like Nest.js or Deno; Both are runtime environments and natively support TypeScript.