Forem: Assyahid Hasan Albana

Microservices Design - API Gateway Pattern

Assyahid Hasan Albana — Fri, 01 Jan 2021 19:38:20 +0000

According to the definition by Gartner: “Microservice is a tightly scoped, strongly encapsulated, loosely coupled, independently deployable, and independently scalable application component.”
The goal of the microservices is to sufficiently decompose/decouple the application into loosely coupled microservices/modules in contrast to monolithic applications where modules are highly coupled and deployed as a single big chunk. This will be helpful due to the following reasons:

Each microservice can be deployed, upgraded, scaled, maintained, and restarted independent of sibling services in the application.
Agile development & agile deployment with an autonomous cross-functional team.
Flexibility in using technologies and scalability.

Different loosely coupled services are deployed based upon their own specific needs where each service has its fine-grained APIs model to serve different clients (Web, Mobile, and 3rd party APIs).
Client to Microservices connections

While thinking of the client directly communicating with each of the deployed microservices, the following challenges should be taken into consideration:

In the case where microservice is exposing fine-grained APIs to the client, the client should request to each microservice. In a typical single page, it may be required for multiple server round trips in order to fulfill the request. This may be even worse for low network operating devices such as mobile.
Diverse communication protocol (such as gRpc, thrift, REST, AMQP e.t.c) existing in the microservices makes it challenging and bulky for the client to adopt all those protocols.
Common gateway functionalities (such as authentication, authorization, logging) have to be implemented in each microservice.
It will be difficult to make changes in microservices without disrupting client connection. For e.g while merging or dividing microservices, it may be required to recode the client section.

API Gateway

To address the above-mentioned challenges, an additional layer is introduced that sits between the client and the server acting as a reverse proxy routing request from the client to the server. Similar to the facade pattern of Object-Oriented Design, it provides a single entry point to the APIs encapsulating the underlying system architecture which is called API Gateway.
In short, it behaves precisely as API management but it is important not to confuse API management with API Gateway.

Functionalities of API Gateway:
Routing
Encapsulating the underlying system and decoupling from the clients, the gateway provides a single entry point for the client to communicate with the microservice system.
Offloading
API gateway consolidates the edge functionalities rather than making every microservices implementing them. Some of the functionalities are:

Authentication and authorization
Service discovery integration
Response caching
Retry policies, circuit breaker, and QoS
Rate limiting and throttling
Load balancing
Logging, tracing, correlation
Headers, query strings, and claims transformation
IP whitelisting
IAM
Centralized Logging (transaction ID across the servers, error logging)
Identity Provider, Authentication and Authorization Backend for Frontend (BFF) pattern

It is a variation of the API Gateway pattern. Rather than a single point of entry for the clients, it provides multiple gateways based upon the client. The purpose is to provide tailored APIs according to the needs of the client, removing a lot of bloats caused by making generic APIs for all the clients.

How many BFFs do you need?
The base concept of BFF is developing niche backends for each user experience. The guideline by Phil Calçado is ‘one experience, one BFF’. If the requirements across clients (IOS client, android client, a web browser e.t.c) vary significantly and the time to market of a single proxy or API becomes problematic, BFFs are a good solution. It should also be noted that the more complex design requires a complex setup.
GraphQL and BFF
GraphQL is a query language for your API. Phil Calçado presents in this article that BFF and GraphQL are related but not mutually exclusive concepts. He adds that BFFs are not about the shape of your endpoints, but about giving your client applications autonomy where you can build your GraphQL APIs as many BFFs or as an OSFA (one-size-fits-all) API.
Notable API Gateways
Netflix API Gateway: Zuul
The Netflix streaming service available on more than 1000 different device types (televisions, set‑top boxes, smartphones, gaming systems, tablets, e.t.c) handing over 50,000 requests per second during peak hours, found substantial limitations in OSFA (one-size-fits-all) REST API approach and used the API Gateway tailored for each device.
Zuul 2 at Netflix is the front door for all requests coming into Netflix’s cloud infrastructure. Zuul 2 significantly improves the architecture and features that allow our gateway to handle, route, and protect Netflix’s cloud systems, and helps provide our 125 million members the best experience possible.

Amazon API Gateway
AWS provides fully managed service for creating, publishing, maintaining, monitoring, and securing REST, HTTP, and WebSocket where developers can create APIs that access AWS or other web services, as well as data stored in the AWS Cloud.

Kong API Gateway
Kong Gateway is an open-source, lightweight API gateway optimized for microservices, delivering unparalleled latency performance and scalability. If you just want the basics, this option will work for you. It is scalable easily horizontally by adding more nodes. It supports large and variable workloads with very low latency.

Other API Gateways
Apigee API Gateway
MuleSoft
Tyk.io
Akana
SwaggerHub
Azure API Gateway
Express API Gateway
Karken D
Choosing the right API gateway
Some of the common baseline for evaluation criteria include simplicity, open-source vs propriety, scalability & flexibility, security, features, community, administrative (support, monitoring & deployment), environment provisioning(installation, configuration, hosting offering), pricing, and documentation.
API Composition / Aggregation
Some API requests in API Gateway map directly to single service API which can be served by routing request to the corresponding microservice. However, in the case of complex API operations that requires results from several microservices can be served by API composition/aggregation (a scatter-gather mechanism). In case of dependency of one another service where synchronous communication is required, the chained composition pattern has to be followed. The composition layer has to support a significant portion of ESB/integration capabilities such as transformations, orchestration, resiliency, and stability patterns.
A root container is deployed with the special distributor and aggregator functionalities (or microservices). The distributor is responsible for breaking down into granular tasks and distributing those tasks to microservice instances. The aggregator is responsible for aggregating the results derived by business workflow from composed microservice.
API Gateway and Aggregation
API gateway with added features results in overambitious gateways that encourage designs that continue to be difficult to test and deploy. It is highly recommended to avoid aggregation and data transformation in the API Gateway. Domain smarts are better suited to be done in application code that follows the defined software development practices. Netflix API Gateway, Zuul 2 removed a lot of the business logic from Gateway that they had in Zuul to origin systems. For more details, refer here.

Service Mesh and API Gateway
Service mesh in microservices is a configurable network infrastructure layer that handles interprocess communication. This is akin to what is often termed as sidecar proxy or sidecar gateway. It provides a lot of functionalities such as:

Load Balancing
Service Discovery
Health Checks
Security On the surface, it appears as though API gateways and service meshes solve the same problem and are therefore redundant. They do solve the same problem but in different contexts. API gateway is deployed as a part of a business solution that is discoverable by the external clients handling north-south traffic(face external client), however, service mesh handles east-west traffic (among different microservices).

Implementing service mesh avoids the resilient communication pattern such as circuit breakers, discovery, health checks, service observability in your own code. For a small number of microservices, alternative strategies for failure management should be considered as service mesh integration may overkill you. For a larger number of microservices, it will be beneficial.

Combining these two technologies can be a powerful way to ensure application uptime and resiliency while ensuring your applications are easily consumable. Viewing two as a contemporary can be a bad idea and it is better to view two as being complementary to one another in deployments that involve both microservices and APIs.

Considerations for API Gateway implementation:

Possible single point of failure or bottleneck.
Increase in response time due to additional network hop through API Gateway and risk of complexity.

References:
https://microservices.io/index.html
https://docs.microsoft.com/en-us/azure/architecture/
https://github.com/wso2/reference-architecture/blob/master/api-driven-microservice-architecture.md
https://tsh.io/blog/design-patterns-in-microservices-api-gateway-bff-and-more/
https://www.infoq.com/articles/service-mesh-ultimate-guide/
https://samnewman.io/patterns/architectural/bff/
https://netflixtechblog.com/

Redis, Kafka or RabbitMQ: Which MicroServices Message Broker To Choose?

Assyahid Hasan Albana — Sat, 26 Dec 2020 17:04:40 +0000

When using asynchronous communication for Microservices, it is common to use a message broker. A broker ensures communication between different microservices is reliable and stable, that the messages are managed and monitored within the system and that messages don’t get lost. There are a few message brokers you can choose from, varying in scale and data capabilities. This blog post will compare the three most popular brokers: RabbitMQ, Kafka and Redis.

Microservices Communication: Synchronous and Asynchronous

There are two common ways Microservices communicate with each other: Synchronous and Asynchronous. In a Synchronous communication, the caller waits for a response before sending the next message, and it operates as a REST protocol on top of HTTP. On the contrary, in an Asynchronous communication the messages are sent without waiting for a response. This is suited for distributed systems, and usually requires a message broker to manage the messages.

The type of communication you choose should consider different parameters, such as how you structure your Microservices, what infrastructure you have in place, latency, scale, dependencies and the purpose of the communication. Asynchronous communication may be more complicated to establish and requires adding more components to stack, but the advantages of using Asynchronous communication for Microservices outweigh the cons.

Asynchronous Communication Advantages

First and foremost, asynchronous communication is non-blocking by definition. It also supports better scaling than Synchronous operations. Third, in the event Microservice crashes, Asynchronous communication mechanisms provide various recovery techniques and is generally better at handling errors pertaining to the crash. In addition, when using brokers instead of a REST protocol, the services receiving communication don’t really need to know each other. A new service can even be introduced after an old one has been running for a long time, i.e better decoupling services.

Finally, when choosing Asynchronous operations, you increase your capability of creating a central discovery, monitoring, load balancing, or even policy enforcer in the future. This will provide you with abilities for flexibility, scalability and more capabilities in your code and system building.

Choosing the Right Message Broker

Asynchronous communication is usually manages through a message broker. There are other ways as well, such as aysncio, but they’re more scarce and limited.

When choosing a broker for executing your asynchronous operations, you should consider a few things:

Broker Scale – The number of messages sent per second in the system.
Data Persistency – The ability to recover messages.
Consumer Capability – Whether the broker is capable of managing one-to-one and/or one-to-many consumers.

One-to-One

One-to-Many

We checked out the latest and greatest services out there in order to find out which provider is the strongest within these three categories.

Comparing Different Message Brokers

RabbitMQ (AMQP)

Scale: based on configuration and resources, the ballpark here is around 50K msg per second.

Persistency: both persistent and transient messages are supported.

One-to-one vs one-to-many consumers: both.
RabbitMQ was released in 2007 and is one of the first common message brokers to be created. It’s an open source that delivers messages through both point-to-point and pub-sub methods by implementing Advanced Message Queuing Protocols (AMQP). It’s designed to support complex routing logic.

There are some managed services that allow you to use it as a SaaS but it’s not part of the native major cloud provider stack. RabbitMQ supports all major languages, including Python, Java, .NET, PHP, Ruby, JavaScript, Go, Swift, and more.

Expect some performance issues when in persistent mode.

Kafka

Scale: can send up to a millions messages per second.

Persistency: yes.

One-to-one vs one-to-many consumers: only one-to-many (seems strange at first glance, right?!).

Kafka was created by Linkedin in 2011 to handle high throughput, low latency processing. As a distributed streaming platform, Kafka replicates a publish-subscribe service. It provides data persistency and stores streams of records that render it capable of exchanging quality messages.

Kafka has managed SaaS on Azure, AWS, and Confluent. They are all the creators and main contributors of the Kafka project. Kafka supports all major languages, including Python, Java, C/C++, Clojure, .NET, PHP, Ruby, JavaScript, Go, Swift and more.

Redis

Scale: can send up to a million messages per second.

Persistency: basically, no – it’s an in-memory datastore.

One-to-one vs one-to-many consumers: both.

Redis is a bit different from the other message brokers. At its core, Redis is an in-memory data store that can be used as either a high-performance key-value store or as a message broker. Another difference is that Redis has no persistency but rather dumps its memory into a Disk/DB. It’s also perfect for real-time data processing.

Originally, Redis was not one-to-one and one-to-many. However, since Redis 5.0 introduced the pub-sub, capabilities boosted and one-to-many became a real option.

Message Brokers per Use Case

We covered some characteristics of RabbitMQ, Kafka, and Redis. All three are beasts in their category, but as described, they operate quite differently. Here is our recommendation for the right message broker to use according to different use cases.

Short-lived Messages: Redis

Redis’s in-memory database is an almost perfect fit for use-cases with short-lived messages where persistence isn’t required. Because it provides extremely fast service and in-memory capabilities, Redis is the perfect candidate for short retention messages where persistence isn’t so important and you can tolerate some loss. With the release of Redis streams in 5.0, it’s also a candidate for one-to-many use cases, which was definitely needed due to limitations and old pub-sub capabilities.

Large Amounts of Data: Kafka

Kafka is a high throughput distributed queue that’s built for storing a large amount of data for long periods of time. Kafka is ideal for one to many use cases where persistency is required.

Complex Routing: RabbitMQ

RabbitMQ is an older, yet mature broker with a lot of features and capabilities that support complex routing. It will even support complex routing communication when the required rate is not high (more than a few tens of thousands msg/sec).

Consider Your Software Stack

The final consideration, of course, is your current software stack. If you’re looking for a relatively easy integration process and you don’t want to maintain different brokers in a stack, you might be more inclined to work with a broker that is already supported by your stack.

For example, if you’re using Celery for Task Queue in your system on top of RabbitMQ, you’ll have an incentive to work with RabbitMQ or Redis as opposed to Kafka who is not supported and would require some rewriting.

We at Otonomo have used all the above through our platform evolution and growth and then some! It’s important to remember that each tool has its own pro & cons and it’s about understanding them and choosing the right tool for the job and that specific moment, situation and requirements.

2020-2021 Technology Outlook: Why Python Is Here to Stay

Assyahid Hasan Albana — Fri, 25 Dec 2020 22:09:27 +0000

ack Overflow

SOURCE: Stack Overflow
In 2020, the Python coding language is chosen by tech gods like Google and Reddit, Facebook, PayPal, Instagram, Netflix, and Dropbox. But achieving this success wasn’t easy. This achievement is explained by the fact that Python is a multi-purpose language that can be used for completing any programming task: mobile and web app development, scientific computing, game development, and system administration.

Whether you’re about to join IT as a coder or interested in Python web development, here’s what you need to know about why Python language has no expiration date.

It’s easy to learn
Less time and resources are required for a beginner to learn and start implementing Python if we compare this programming language to Java or C++. Python is now called an accessible and straightforward language free of intricate guidelines or complex language syntax.

At the time when C and C++ were dominant (and not very user-friendly) languages, Python was created with simplicity in mind. The core philosophy of the programming language’s syntax is based on three principles: readability counts, beautiful prevails over ugly, and simple beats complex. So it’s easy to figure out what the code does when you read it.

It’s a high-level language
What factor makes a programming language high-level? This is the number of layers of code that this language is away from machinery binary code. In other words, high-level languages are close to natural spoken and written languages.

SOURCE: Security Stack Exchange

Python leads the list of high-level programming languages, leaving Ruby, Java, C, and C++ behind. To form instructions, Python uses words that resemble abbreviations of English words. Developers prefer Python to other languages because while using the language to code, they can concentrate on the task itself instead of focusing on how the computer works.

It’s cross-platform
Cross-platform or platform-independent is one of the programming language features that allows a developer to run intermediate code on different OS. Coding in cross-platform languages like Python is rising in popularity.

A Python software written on a Linux system will run on macOS and vice versa. If the Windows PC has the Python interpreter installed, the Python program will run on the Windows PC as well.

It has exceptional scalability
In the context of web applications, scaling stands for the app’s ability to accept more requests and visitors and have long-term support for security patches and bugs. This is why scalability is the key concern when building a new application. Even though the app can start off as rather small, it might grow large with time.

SOURCE: Ncube

To avoid code rewiring in the future, it’s essential to pick the right programming language and its framework. Python’s framework Django scales exceptionally. This is why Django is the choice of Spotify, Instagram, YouTube, Quora, Pinterest, and Disqus. But Django isn’t the only possible choice. Check out more about the top 13 Python frameworks to learn.

It has hundreds of libraries
Any programming language’s library is a set of pre-written reusable codes. Python’s library collection is huge and allows developers to use pre-written code saving time when developing a new program. With a freely-accessible pre-written code, developers working on big projects can significantly cut down the initial development cycle.

In addition to standard libraries, Python has a few specific ones: those for natural language processing and machine learning. Most of the libraries are mature. Many of them have been battle-tested for over ten years.

SOURCE: Learn data science

It offers excellent community support
A Python developer stuck with a piece of code, shouldn’t be concerned about receiving relevant support. To start with, while most languages lack documentation, guides, and video tutorials, Python has this in abundance. And secondly, this is because Python has one of the strongest communities.

Say a Python developer (beginner or expert) runs into a major issue in the middle of crunch time. Receiving timely help from an active community will help him escape delays.

So what does a Python community look like? It includes international members with different experiences and skills who call themselves Python programming enthusiasts. They host meetups and conferences or arrange collaborations on code. The community uses mail lists to keep in touch with developers.

Top 5 Python real-world applications
What are the areas where Python programming plays a crucial role? With so many advanced features, Python has many applications, including:

Web and mobile app development — Top Python frameworks used for web and mobile app development are Django, Pyramid, and Flask. These frameworks are packed with tools that simplify many tasks and help developers build apps at a rapid rate.

Machine Learning and Artificial Intelligence — Development of ML and AI apps requires stable but flexible, secure, and well-equipped programming language. Since these four qualities are what Python is known for, this programming language is favored by ML and AI developers. In the field of AI and ML, Python’s most widely used packages are Seasborn, SciPy, Pandas, Numby, and Keras.

Game development — Python’s extensions help with interactive game development. Basically, there are two Python-based extensions used for game engineering: PyGame and PySoy. They helped developers to build a basis for games like Civilization-IV, World of Tanks, and Battlefield 2.

FinTech and the financial industry — Python currently dominates the pack of top programming languages when it comes to FinTech and the financial industry. World-known financial institutions like Bank of America have already transformed their tech stack to Python.

SOURCE: Skill Crush

Desktop graphical user interface development — Along with its ability to work on various OS, Python is also known for its modular architecture. Combined, these two features make Python a rather frequent choice for GUI app development. The list of best Python-based frameworks for GUI applications includes PyQt, WxPython, PyGtk, Tkinter, PySide, and Kivy.

Python has been around for a while. Today, it is the choice of Netflix, Google, Dropbox, Stripe, Instagram, and Spotify. Indeed, Python is frequently referred to as a simple language. However, it’s also very powerful. And even though a developer starts easily, outgrowing Python isn’t as easy as outgrowing any other programming language. With this in mind, consider Python as our primer choice for software development.