Forem: Ibrahim Cesar

Kafka on the Serverless Shore: Building event-driven applications with Kafka

Ibrahim Cesar — Wed, 25 May 2022 14:08:12 +0000

As I awake one morning from uneasy dreams, I found myself asking why I never tried to use Apache Kafka.

As a event-driven enthusiast and with the recent announcements of Amazon Managed Streaming for Apache Kafka (MSK) Serverless and the Serverless Kafka offering of Upstash I gave another try.

I played a little on my computer in the past, but only see the work to go to production. I confess was beyond what I was up to do in my free time. So the serverless come to rescue! This is one of best-selling point of “not having to think about servers”, you can just experiment with new technologies.

Since the AWS offering is in public preview and not General Available, I went with Upstash's Kafka Serverless offering.

What is Kafka?

I thought that since Kafka was a system optimized for writing, using a writer’s name would make sense. I had taken a lot of lit classes in college and liked Franz Kafka. Plus, the name sounded cool for an open source project.
So basically there is not much of a relationship.
Jay Kreps, Apache Kafka’s co-creator on Quora

Kafka is a publish/subscribe messaging system. Other ways to describe it are “distributed commit log” or nowadays, as a “distributing streaming platform”.

The core entity for Kafka is a message. For all purposes, Kafka doesn't care about the content of the message. It's just an array of bytes. Messages also can optionally have a key. Kafka persistently stores the messages in the order. Is possible replay the stream to get to the state of a system at a point in time. A time machine for your transactions.

Topics are a way to categorize the messages in order to allow other applications to have some visibility of what messages they will access.

And then, the heart of the system: Producers and Consumers. Basically, the first, also called publishers or writers, produces messages, usually a lot of messages, to require Kafka, and other applications act upon those messages. Consumers read messages from a topic and. We can call them subscribers or readers too.

Kafka is a lengthy subject, worth of books, courses and hand-on practice. There are a lot more concepts that I will not cover in this post. Well is even subject for papers, like On Efficiently Partitioning a Topic in Apache Kafka. I highly recommend Apache Kafka for beginners - What is Apache Kafka?.

The benefits of a serverless offering is the overhead in management of brokers, partitions, the Zookepeer and all the other moving parts to allow a fault-tolerant and highly distributed applications is taken care for you. And then, you can focus on write the code you need in your domain, with your tools — this is what I call the joy of serverless.

On the shore

To start, you just need to login, select Kafka and will create our first cluster. Kafka works in a distributed way from the get go.

To create the cluster, we'll need a name and select the region. And also, if we need to be in only one availability zone or span of over one. Choose a single-zone for testing/development, multi-zone for production use cases.

We will the create our first topic and choose our desired number of partitions. Partitions are how Kafka provides redundancy and scalability, and each one can live on a different server. This is a very important concern for performance and scaling.

In the advanced options for topics, there will be several options with sensitive defaults. Some configurations will require an upgrade in your plan. So, plan for the go-live depending on your application needs.

And… that's it. No need to worry about servers.

For this tutorial I will use Node.js on an AWS Lambda. Code is available in this GitHub repository using TypeScript:

ibrahimcesar / lambda-serverless-kafka

🐚 Kafka on the Serverless Shore

Experimenting the Upstash's Kafka Serverless.

Read Kafka on the Serverless Shore: Building event-driven applications with Kafka for context.

Deploying

You will need:

An AWS account
CDK v2 installed (npm install -g aws-cdk)
CDK boostraped in the Account / Region we'll deploy
A user for the CDK deploy with the right permissions

If you need help with all the above, open an Issue to see if there's enough people interested in a tutorial for that.

npm install
cp .env.example .env

Fill .env with your credentials and names.

cdk deploy

Remember if you have multiple AWS profiles in your machine to pass the flag:

cdk deploy --profile namedProfile

Bootstraped with my CDK v2 Simple Lambda HTTP ApiGateway Starter

View on GitHub

I'll use the vendor package (@upstash/kafka) that leverages the REST API. Since we are at the shore, we'll not go open sea and build a ship. Not even a boat. We'll just get our feets wet.

I'm writing an application that will read something, from some author. And post every line to Kafka. Then, I'll consume after that.

Basically is just two lambdas, one for each endpoint: a writer and a reader.

And we'll use my topic ulysses. I could send all lines with page numbers from the book Ulysses from James Joyce. Or analytics data. Or log data. You get the gist.

// This is just a edited. 
// View full file at: 
// https://github.com/ibrahimcesar/lambda-serverless-kafka/blob/main/src/lambda/producer.ts

import { Kafka } from "@upstash/kafka";

const kafka = new Kafka({
  // configuration
});
const writer = kafka.producer();

try {
    const res = await writer.produce("ulysses", writing, {
      key: payload.author ?? "",
    });
    response = res;
  } catch (err) {
    if (err instanceof Error) throw err;
    else throw new Error(`${err}`);
  }

And to consume is simple. Note that I have there two hardcoded values, consumerGroupId and instanceId. You, at the time, basically can send whatever you want in the first request and Upstash will provision for you.

// Try and catch block. 
// View full file at: 
// https://github.com/ibrahimcesar/lambda-serverless-kafka/blob/main/src/lambda/consumer.ts

  try {
    const writing = await reader.consume({
      consumerGroupId: "group_1",
      instanceId: "instance_1",
      topics: payload.topics,
      autoOffsetReset: payload.autoOffsetReset ?? "earliest",
    });
    response = writing;
  } catch (err) {
    console.error(err);
    if (err instanceof Error) throw err;
    else throw new Error(`${err}`);
  }

Little TypeScript goodie. I'm using the flag useUnknownInCatchVariables: true in the tsconfig.json file, so the catch is of type unknown. Hat tip for Mike North to show the way.

And now we can start writing and reading. To write is just a POST.

Eventual consistency

To read, we'll use another POST, but beware: Here comes something important about distributed systems. We have eventual consistency. Which means, your application cannot rely on the data being immediatly available for you just after the write. If you do, you could get... nothing, like a empty array in this case:

But, after very little time, you can see all data available in the topic:

Dive Deep

Checkout Upstash's Docs on Kafka
Be yourself a reader: Kafka: The Definitive Guide: Real-Time Data and Stream Processing at Scale
Checkout the Confluent's content on Kafka
AWS just released MKS Serveless, the Managed Streaming for Apache Kafka (MSK). But please note that, while is a serverless offering in the sense you'll not need to worry about cluster servers as they put: "Easily stream data with Amazon MSK without managing cluster capacity". Emphasis mine. You'll need to put a lot more effort than the Upstash's offering as you can see here: Getting Started - Step 2: Create a Client Machine
All the code used is open to use in TypeScript, using AWS Cloud Developement Kit (CDK). So you'll have no problem at all in provisioning your infrastructure.

Cover image by Alejandro Cartagena on Unsplash

Certificações AWS — Por quê e de que forma

Ibrahim Cesar — Fri, 08 Apr 2022 10:14:20 +0000

O objetivo desta postagem é responder por quê você deve ter entre seus objetivos uma certificação AWS. Também exploro minhas motivações que me levaram a buscar uma certificação — spoiler: eu não havia nenhum incentivo em minha posição e fiz para validar minha atuação. No final, trago algumas respostas a questões comuns.

Por quê você deve tirar uma certificação

Não existe algoritmo de compressão para experiência. — Andy Jassy

É um data-point. Demonstra, para qualquer pessoa, incluindo você, sua capacidade em um nível mínimo (entre 70% a 75%, dependendo do nível), de reconhecer cenários e associá-los a soluções efetivas, atingindo objetivos propostos, seja escalabilidade, performance, excelência operacional, redução de custos, segurança, entre outros.

Todos precisamos estudar qualquer problema complexo. Não vai chegar uma demanda e vamos ter opções para escolher. O exame só demonstra que avaliando um conjunto de opções, que no dia a dia você provavelmente levantaria como opções, escolheu a mais apropriada para aquela determinada situação. E isto, demonstra uma agilidade, solidez dos conhecimentos, associação de domínios e conhecimentos.

No nível Professional, por exemplo, existem questões em que todas respostas são soluções válidas. Não há nada que as elimine por detalhes, como ser logicamente impossível, se tratar de uma categoria de armazenamento diferente, etc. Porém, a questão lhe pede, por exemplo, que escolha a que obtenha o menor custo ou a maior disponibilidade global, por exemplo.

E é necessário ter algum conhecimento prático. Acredito sim que podem existir várias pessoas que estudam sem esse viés e passam com sucesso. Mas eu diria, se concentre em habilidades e não certificações. É bem comum ver 4x AWS... 10x AWS... 100x AWS... Mas nesse caso já se torna um jogo de números. Obviamente não tiro o mérito de nenhum, pelo contrário, devem exibir com orgulho, mas para quem está começando, não se concentre nisso. Se concentre em construir habilidades, não apenas o suficiente para passar em um exame. Não existe algoritmo de compressão para experiência.

Para as empresas, é claro, você vai ser uma pessoa valorizada. No livro Reaching Cloud Velocity: _ A Leader's Guide to Success in the AWS Cloud_ é citado que com a experiência de diversos clientes — além de estudos como Social consensus through the influence of committed minorities; fica clara a necessidade de atingir uma massa crítica de 10% de pessoas advogando uma plataforma antes dos efeitos de rede tomarem efeito. Ou seja, investir no aprendizado e certificação das pessoas na organização, tem efeitos internos de aceleração, um aumento da qualidade de entregas, torna a empresa mais atrativa e motiva outros da equipe a buscarem suas certificações também.

Caso de estudo: Minha própria jornada

Na empresa em que retirei minha primeira certificação e a segunda, eu não tinha nenhum incentivo para retirar uma certificação. Eu paguei do meu próprio bolso (depois me pediram para avisar nesses casos, mas não me importava). Para mim foi um investimento. Ah, estava querendo tirar a certificação e trocar de emprego?. Pelo contrário. Eu estava no que eu chamo de solidão técnica.

Você é a pessoa responsável por uma área, digamos arquitetura. Não tem pares para validar suas decisões, as documentações que está fazendo e muitas vezes, você é a última linha de defesa entre a empresa e o mundo externo, tendo que implementar segurança, a um custo baixo (geralmente solidão técnica existe em empresas de pequeno a médio porte).

Meu time estava crescendo. Eu não disse a ninguém que estava me preparando. Eu queria dar mais segurança para a gestão da minha organização, para o meu time e, talvez mais ainda, para mim mesmo. Síndrome de impostores já virou assunto carimbado. É o fantasma percorrendo as entrelinhas de meus textos e códigos. Não tive formação em ciência da computação ou engenharia.

Comecei com o AWS Certified Cloud Practitioner e então o Architect Associate e agora, recentemente, o Professional. Saí da organização que estava recentemente e ainda que a que eu esteja no momento estimule e pague por certificações, eu ainda fiz este exame com um voucher que recebi por ser parte do programa AWS Community Builder e fiz, novamente para ancorar meus conhecimentos e habilidades.

E estudar para os exames me fez descobrir tanta coisa e tantas possibilidades. Eu sou um profissional diferente desde a primeira certificação. É um acelerador pessoal e profissional. Esta é a forma como a certificação me ajudou.

FAQ

Por onde devo começar?

Criando uma conta, fazendo experimentos. Verifique o que é esperado em cada exame. Existem cursos preparatórios da própria AWS. Existem simulados. Há muitos cursos pagos, e outros, inteiramente gratuitos, em plataformas como o YouTube. Não posso fazer uma ampla revisão dos recursos disponíveis pois, obviamente, eu não fiz todos o suficiente para ter uma opinião formada. Mas, tendo realizado cursos no Udemy e um pago, percebi que a relação custo-benefício do curso pago foi muito maior. Inclusive para o trabalho no dia a dia.

Tem dúvidas, pouca experiência prática com a nuvem? AWS Certified Cloud Practitioner.

Qual caminho devo seguir?

Não há nenhuma obrigatoriedade de progressão na AWS. Você não precisa ter um certificado Associate para fazer a prova Professional. Se acredita possuir as habilidades e conhecimentos necessários para qualquer certificação, basta escolher.

Por isso, existem muitos caminhos necessários. Sou AWS Community Builder e vou compartilhar o caminho que vejo ser bem comum: realizar dois ou todos os Associate antes do AWS Solutions Architecture Professional, pois a quantidade de serviços que é necessário o conhecimento é enorme.

Embora o AWS DevOps Engineer Professional seja igualmente uma tarefa de esforço, ele é limitado a um domínio, o que para os especializados, se torna mais confortável lidar com a carga cognitiva de dezenas, realisticamente, centenas de serviços e recursos que serão exigidos seu conhecimento, ainda que com menos profundidade, mas entendendo seus trade-offs frente a determinadas necessidades.

Não foi o caminho que tomei como escrevi acima. Avalie seu momento e contexto.

É muito caro! Não poderia ser mais barato?

Sim. Para a imensa maioria dos brasileiros o custo de uma certificação é proibitivo. Ainda que determinadas posições sejam associadas com maior poder aquisitivo, mesmo uma certificação de entrada custa 100 doláres.

E sim, acredito que poderia existir uma forma de Purchasing power parity (PPP), ou paridade de poder de compra. Somente foi através de PPP que pude adquirir alguns cursos importantes.

Deveria ser gratuito? Regularmente, a AWS e outras clouds realizam programas para certificação e oferecem algumas de forma totalmente gratuita. Não existem dados oficiais, mas a taxa de uso efetivo desses descontos parece ser baixa.

Certificações expiram. Não se trata de mais uma forma dessas empresas ganharem dinheiro?

Em 2020, o AWS Simple Storage Service (S3) passou a ter consistência forte na leitura após escrita, sendo que era até aquele momento, eventual. O que é uma consideração muito importante e crítica, dependendo do seu tipo de uso.

As tecnologias mudam. Alguém certificado em 2015 encontra hoje um contexto de tecnologias extremamente diferente. Uma certificação não se foca em fornecer meta-habilidades ou grandes teorias que poderá aplicar a diferentes tecnologias e recursos como uma formação acadêmica ou curso, por exemplo. Ela foca em demonstrar capacidade de reconhecer as melhores soluções baseadas em casos particulares de uso com determinados objetivos a partir das tecnologias existentes. É um exame prático. Assim como direção, exige uma constante qualificação.

Preciso saber inglês?

Sim e não. No momento, a maioria das certificações Associate além da Fundamental oferecem versões em português brasileiro. No nível Professional apenas em inglês. Mas eu recomendo, fortemente, se possível que você faça em inglês. Nossa atuação seja utilizando com SDKs, CLIs ou mesmo lendo documentações de arquitetura passarão pelo contato com os termos em inglês.

Os primitivos da nuvem são como os primitivos das linguagens, de sua sintaxe. E estes são em inglês, if, else, then, entre outros. Já fiz alguns cursos disponíveis no AWS Skill Builder e falhei em identificar as opções corretas em perguntas que acreditei que acertaria, pois o termo em português não me pareceu satisfatório para a solução. Existem falsos cognatos, entre outros fatores.

E sim, menos de 5% da população brasileira sabe inglês em qualquer nível. Eu mesmo, tive uma curva de aprendizado bem íngreme. Tenho, em 2022, 36 anos, sempre estudei em escola pública, onde o inglês, quando ensinado, parecia um eterno loop do verbo to be, ao menos em minha época de estudante. Mesmo o acesso a computadores é limitado. Eu usei um computador aos 18 anos. A "sala dos computadores" era um local mítico e sagrado em nossa escola, que ninguém entrava. Lembro de irmos uma vez e em trios, sentarmos na frente do computador e sermos apresentados ao Clippy, do Windows. E eu não estou fazendo piada.

Se você deve saber inglês, minha resposta é sim, deveria.

O que acontece se eu não passar no exame?

Você precisa esperar 14 dias para poder agendar outro exame. Para agendar este novo exame, você deverá adquirir novamente.

Dúvidas? Ponderações? Deixe nos comentários e podemos debater!

As badges acabaram de serem repaginadas, então, aproveitem para estudar e já receber uma badge bem mais bonita como esta 😀:

Revisão: Rafael B. Pires

Imagem de destaque por ThisisEngineering RAEng em Unsplash

SAFe™ Considered Harmful

Ibrahim Cesar — Sun, 17 Oct 2021 13:50:44 +0000

Is safe to say, the framework SAFe™ embodies everything wrong with the so-called Agile Industrial Complex. Is so bloated and focus on the wrong things that are not even remotely agile in the sense of the original manifesto. I’ll argue how SAFe™ do more harm to the community than generates value and is devoid of any agility. Pay for your training, get your certificate, some taps on the back and... whatever. SAFe™ is so bad that makes the case for Waterfall looks like a good idea.

I don’t know who needs to hear this, but is safe to say: SAFe™ is not agile. Jeff Gothelf made a post with exactly this title in May 2021. Since they shove up Lean UX in version 4.5 of the framework, people keep asking Gothelf about how both work together. But the “adoption” of Lean UX was more of “let’s add a couple buzzwords more” with no regard to principles, methodologies or alignments. Gothelf is the author of Lean UX, and about how works within SAFe™ this is his position:

The slightly longer answer is that all the principles we’ve built into Lean UX don’t seem to exist in SAFe.

Ewan Leith

@ewantoo

There's quite a lot of anti-Agile sentiment in software development circles at the moment. I think we've reached the point where a critical mass of developers have never experienced how terrible "Waterfall" was as a planning methodology, and think it sounds like a good plan🤦‍♂️

08:00 AM - 18 Jun 2021

Is less “anti-Agile sentiment” and more “anti-Agile Practices™️”. SAFe™ is this Cthulhuian Azathoth-like cosmic horror framework that is driven the bad rep and the harm is doing to the development environment. This isn’t some kind of ‘homebrew’ agile we see selling — SAFe™ is straightforward a vapor agile framework to sell certifications. ToughtWorks place in their hold category, the Proceed with caution, which is a polite way to say Run.

Our positioning regarding "being agile before doing agile" and our opinions around this topic shouldn't come as a surprise; but since SAFe™ (Scaled Agile Framework®), per Gartner’s May 2019 report, is the most considered and most used enterprise agile framework, and since we're seeing more and more enterprises going through organizational changes, we thought it was time to raise awareness on this topic again. We've come across organizations struggling with SAFe's over-standardized, phase-gated processes. Those processes create friction in the organizational structure and its operating model. It can also promote silos in the organization, preventing platforms from becoming real business capabilities enablers. The top-down control generates waste in the value stream and discourages engineering talent creativity, while limiting autonomy and experimentation in the teams. Rather than measuring effort and focusing on standardized ceremonies, we recommend a leaner, value-driven approach and governance to help eliminate organizational friction such as EDGE, as well as a team cognitive load assessment to identify types of teams and determine how they should better interact with each other.
ToughtWorks Radar 24 Hold technique: SAFe™

Agily McAgileface

This could be a map for SAFe™ but no, is just a slide that was actually used in a briefing to the commanding general of U.S. forces in Afghanistan, Stanley McChrystal. It purported to explain U.S. counterinsurgency strategy in Afghanistan, all on one slide.

The first time I saw this as “agile” on Twitter I laugh. I thought it was just satire. This is actual SAFe™ map. Find the errors:

This is called "SAFe 5 for Lean Enterprises".

Other ways SAFe™ harmful? It violates the very first agile value: “Individuals and interactions over processes and tools”, being an upside-down version of it and is a truly industry (“used by 1,000,000 people in 20,000 organizations around the globe”! So much suffering in the world... gives me a broken heart) of a bloated framework that makes not only Waterfall looks like a good idea but is bad for the lean, agile and DevOps thinking and practices.

Allen Holub

@allenholub

Just curios, but has anybody seen an article endorsing SAFe written by someone who (1) actually understands and has worked in an agile environment (so no non-programmer journalists), and (2) doesn’t profit by selling SAFe training, consulting, books, &c.?

19:27 PM - 21 Jul 2021

Scaled Agile Framework® and SAFe™ are trademarks of Scaled Agile, Inc. When I just copied the title of their map, just "SAFe 5 for Lean Enterprises", of course they inject some text on my clipboard:

© Scaled Agile, Inc.
Include this copyright notice with the copied content.

Read the FAQs on how to use SAFe content and trademarks here:
https://www.scaledagile.com/about/about-us/permissions-faq/
Explore Training at:
https://www.scaledagile.com/training/calendar/"

Event-Driven Architectures on AWS

Ibrahim Cesar — Sun, 12 Sep 2021 01:27:05 +0000

In September 2021, Corey Quinn, Chief Cloud Economist at The Duckbill Group published The Key to Unlock the AWS Billing Puzzle is Architecture and I identified myself a lot with some of his remarks. One, sometimes downplayed is the notion cost is architecture.

For example, if you didn’t use SQS when building your application because it couldn’t handle your throughput needs or it was too expensive, that changed a couple of weeks ago. SQS is to the point where it’s now effectively unlimited throughput at a cost that’s just 3.6% of what it was at launch.

And the 3.6% was because of a typo by Zack Kanter. In fact, the price drop is 99.4% from 2006. Which makes it cost only 0.6% of what was. You will pay less than 1% it was years ago!

Loosing our couplings

There’s general advice that is very well known in modern software architecture, at leat in the web or distribute space that advocates for loose coupling. And sure, is a noble goal but there are several kinds of coupling such as Temporal coupling when we deal wit time-based dependencies such as collaborating system components, sequential operations or operations that needs another to complete such a request or another operation. We have Spatial coupling such as not having to know where your collaborating applications are in the network and providing fail-over mechanisms such as Load Balancers, pub/subs, and other. And of course, this comes a lot even more on the serverless space than any other in the Cloud: Platform coupling. Have proprietary protocols and components from a vendor or platform.

But as pointed, coupling is a function of multiple dimensions, not just binary options like “tight” or “loose”. We are always. Always working with trade-offs.

“The event-driven architecture style is a popular distributed asynchronous architecture style used to produce highly scalable and high-performance applications. It is also highly adaptable and can be used for small applications and as well as large, complex ones. Event-driven architecture is made up of decoupled event processing components that asynchronously receive and process events. It can be used as a standalone architecture style or embedded within other architecture styles (such as an event-driven microservices architecture).”
Fundamentals of Software Architecture, Mark Richards and Neal Ford

Following the inspiration from Domain-Driven Design: Tackling Complexity in the Heart of Software by Eric Evans and later, Domain-Driven Design Distilled and Implementing Domain-Driven Design by Vaughn Vernon one of way to model and work with software involves deal with bounded contexts we can map and create services, applications and systems to better tackle our problems. And to integrate all of this, one pattern is Domain Events.

Event becomes the primary mechanism for sharing information across bounded context integrations about change in state. Domains events are immutable, self-describing. Events give the data they carry meaning by supplying business context. Not data transfer objects or change data captures. These are state / state change representations that are not reflective of a system's behaviour.

An event is a quanta, a unit that describes something in the system. Could be a simple JSON as that:

{
  "eventId": 61452,
  "event": "OrderPlaced",
  "createdAt": "2021-10-07"
}

For example, with this event, different domains could decide to act or not in the event. Maybe we have an Inventory domain, a Package domain, a service that will email the user about the order and each one will handle the event as seem fit, and we could add and apply as many services and applications needed. We will publish all these events to “Bus”, which in the AWS is via Amazon EventBridge. As a CDK enthusiast, even to create your Infrastructure as Code in a couple of lines of TypeScript.

import * as cdk from '@aws-cdk/core';

const stack = new stack();

const bus = new EventBus(stack, 'bus', {
  eventBusName: 'MyCustomEventBus'
});

bus.archive('MyArchive', {
  archiveName: 'MyCustomEventBusArchive',
  description: 'MyCustomerEventBus Archive',
  eventPattern: {
    account: [stack.account],
  },
  retention: cdk.Duration.days(365),
});

With the code above you could provision a custom event bus and create an 365 archives of all events, that if needed, could be used to auditing or event replay event as needed. And even in my example, let’s say the user wants to cancel the order, we will not delete the previous event. Different domains will need to process the new events in order to undo actions or make an analysis about how and why and so on.

{
  "eventId": 61456,
  "event": "OrderCancelled",
  "createdAt": "2021-10-07"
}

Maybe another email will be sent, to confirm the cancel, Inventory Package will have to adjust their line of work and so on. And now maybe a CRM system you be fed to understand why the customer canceled after the conversion.

Coupling will always exist. But we can orchestrate with such services and create highly responsive, available and robust architectures. And you can combine EventBridge with Lambda, SNS, SQS, Step Functions and soon you are dealing with systems that can scale up and down easily and if done correctly, try to optimize a lot your costs because some services and applications we have today sitting at some servers 24/7 sometimes just need to be active when a specific event happens.

Right now I’m in the middle of an event-driven creation and I really like the mental model and the developer experience as the organization experience and soon we used the “Ubiquitous Language” from DDD. And for this set of tools, none other Cloud seems so fit and solid as AWS does.

Books to read

Building Event-Driven Microservices: Leveraging Organizational Data at Scale by Adam Bellemare. Brings much value with both architectural and application patterns to event-driven architecture.
Architecture Patterns with Python: Enabling Test-Driven Development, Domain-Driven Design, and Event-Driven Microservices by Harry Percival and Bob Gregory. Python is not my primary language at work or even my preference, but I get so much from this book. Because is aiming at patterns if you do not work with Python don’t let the title fool you. There’s a lot from here in this book!

Heroes to follow

Yeah, they are actually heroes.

Ben Ellerby

@EllerbyBen
For me one of his post, helped me a lot in my work and opened so many possibilities: EventBridge Storming — How to build state-of-the-art Event-Driven Serverless Architectures. Blogs a lot about the concept of Serverless Transformation, a term I adopted too.

Sheen Brisals

@sheenbrisals
Blog often at Medium. Last post was How To Build Better Orchestrations With AWS Step Functions, Task Tokens, And Amazon EventBridge!

AWS Event-Driven Tools

The hidden bias in Core Web Vitals

Ibrahim Cesar — Thu, 02 Sep 2021 00:56:36 +0000

Google's Core Web Vitals has arrived. And as Google prides itself, has an huge business impact. The data is very well known that even small increases in performance, with a faster website your conversion rates grows. But the set, based on the top performers, has hidden bias embedded in it I want to talk about.

First, more context: the Core Web Vitals are a set of three metrics.

Largest Contentful Paint (LCP): measures loading performance. To provide a good user experience, LCP should occur within 2.5 seconds of when the page first starts loading.
First Input Delay (FID): measures interactivity. To provide a good user experience, pages should have a FID of 100 milliseconds or less.
Cumulative Layout Shift (CLS): measures visual stability. To provide a good user experience, pages should maintain a CLS of 0.1. or less.

Google does a great deal to share a lot of information.

The Three Core Web Vitals

In this post I will not argue against Web Core Vitals. I think is past due the time we take our user metrics, real user metrics (sometimes called by its acronym, RUM) in the center of everything we do. The "rise of user centric" metrics such as CLS (Cumulative Layout Shift) and other are remarkable feats and much needed. But I have some thoughts in the complexity that arise from it. I was lucky enough to attend in person the 2019 Google Chrome Developer Summit¹. It was the first time I saw presentations calling out developers to not only pay attention to their top devices — several of the talks the subject and reality of low end devices. It was amazing to see that. But I was thinking... "I'm working in Brazil for at least 10 years and all my references came from here, this place, and now, 2019, they are talking about the most common scenario I face everyday".

// Detect dark theme var iframe = document.getElementById('tweet-1186647094221824001-302'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1186647094221824001&theme=dark" }

There’s a material side of web and computing in general², that sometimes we think little about. Most of the time, we only take the bits and bytes as part of things. We literally wired together our continents with cables, physical cables, transporting data from one place to another. Even communicating with the computer in the other room takes times. There’s no such thing as instant communication, laws of physics continues to apply.

In Defining the Core Web Vitals metrics thresholds that aims to explore the research and methodology behind Core Web Vitals thresholds. Is stated:

Specifically, we aim to identify a threshold that is consistently achievable at the 75th percentile for top performing sites.

According to Ookla, a company specializing in internet speed testing, reported that the global average download speed on fixed broadband is 85.73 Mbps as of September 2020. The average mobile download speed is 35.96. As technology becomes more advanced and networks are improved, these numbers are expected to continue to grow.

Based on data from Ookla, Singapore is the nation with the fastest broadband download speeds of 226.60 Mbps. Hong Kong follows this with 210.73 Mbps and Romania with 193.47 Mbps. The United States has the 10th-fastest internet download speeds of 161.14 Mbps.

The countries with the fastest mobile internet speeds are different. The fastest mobile download speeds can be found in South Korea at 121.00 Mbps. China follows with 113.35 Mbps and the United Arab Emirates with 109.43 Mbps.
Internet Speeds By Country 2021

The top performing web sites I assume are against the global pool of sites. Mobile speed is not an issue with code itself, but there’s a material issue with this "fair measurement". Because in the Global South we have historically less speed and very low end devices with a minority using top end devices. A site with 1MB payload will load much faster in a 5G connection and in a high end device than in a 3G connection (or sometimes worst as we have here in Brazil and I think I could apply to all great Global South) and low end devices that our audiences use. Using the same data, Brazil has an average of 67.86 in broadband speed and 28.36 in mobile speed. Orders of magnitude lesser than the top velocities. Low end devices also means that we cannot always use the latest improvement in browsers and web apis. We need to use polyfills and other tactics that can hurt even more our overall performance. When Douglas Crockford said that the browser was the most hostile development environment he was talking from some office in Silicon Valley, I wonder what he would say about developing for the wider range of low end devices we have around here.

I asked a representative of Google on the subject. And asked if the realities in connection and devices would be take in account, the answer and I quote, since in the spirit of the Honest Results Policy employed by Google in search, was in an open session, and you can check it out, around 1:05:00 was:

I'm not aware of anything planned in that direction. Regarding on the website and what your audience makes it a bit easier or harder, but essentiality we want this Core Web Vital be as global as possible. In practice, if your users are in very slow connections then you must make sure your site is extra fast for them.

Not only run, but run at least twice as fast! "Now, here, you see, it takes all the running you can do, to keep in the same place. If you want to get somewhere else, you must run at least twice as fast as that!" — Red Queen from Lewis Carroll's Through the Looking-Glass

[...] We have three areas in Core Web Vital metrics ... one is poor, another needs improvement and good. As soon you reach that middle area with the Core Web Vitals fields data, you will see improvements ...So as long you are in that middle range, you are in a good area.

So... is good not being good?

[...] The good thing about Core Web Vitals you are competing with sites that are in the same situation as you. If you have the same users you are kinda competing against them is not that you are measuring your site against the fastest of the fastest other sites out there. For the individual queries we see websites that have good web vitals and sites that don’t have good web vitals [...] And, if none of the sites can reach good Core Web Vitals for specific queries that don’t cause any problems because everyone is on the same level again.

When thinking about the question this was the first thing that I thought because of this I put bold in the sentence. Is reasonable and at surface level looks like resolves the “material question” for good. But not so fast.

But this is a sort of “Invisible Hand” argument. The sites competing for the queries are not born the same. Some have niche audiences (maybe with better connection speeds because is a niche of user with more money and inequality only goes up in South America), some are ultra bare bones blogs created to spread misinformation and “fake news” and therefore will score higher than an outlet that has subscribers, has more content to show, produces graphics and all the things provide better and more contextualized news but has to pay the “tax” to try a broad audience in a reality of several low end devices and poor connections. Is an optimistic and “best-case scenario” answer. Is something that everybody on the internet knows on some level: you are not competing with “your peers”, you are competing with everyone else for individual queries and there’s a whole sort of plurality in the actors.

I think since they operate with a closer look on the top performers they are not paying enough attention on how these "fair" set of metrics are biased, one-sided and might have several side-effects since it has such an impact on business.

In Algorithms of Oppression: How Search Engines Reinforce Racism, Safiya Umoja Noble challenges the idea that search engines like Google offer an equal playing field for all forms of ideas, identities, and activities. Data discrimination is a real social problem; Noble argues that the combination of private interests in promoting certain sites, along with the monopoly status of a relatively small number of Internet search engines, leads to a biased set of search algorithms that privilege whiteness and discriminate against people of color, specifically women of color.

Big Tech always use algorithm as a shield, in order to avoid further scrutiny, and as a means to excuse bad outcomes or deny agency. But as the researcher does, the politics and assumptions about the world are embedded in the tools and even in "fair" metrics that cannot accurately describe and measure with the promised “fairness” from math and algorithm — which besides codes, in the end mean “our choices”.

Google wants to "No site gets preferential treatment". And I believe in them. I do. I don't think there's any Evil Room where some "White Guy 10x Developer" is making the decisions, but as all escalations must use public channels, I'm here, trying to push and shine some light in these particular specifics since the structural biases are by definition invisible because they are part of society and world views, and I and hope to open a more diverse conversation.

It was my first time in the United States, I was motivated to write my first open source component and even got engaged! ↩
For a great paper on the subject, I highly recommed The Environmental History of Computing by Nathan Ensmenger. ↩

Next.js TypeScript SSR and ISR serverless deploy with AWS CDK

Ibrahim Cesar — Thu, 19 Aug 2021 11:32:25 +0000

Incremental Static Regeneration is the Next.js superpower. We’ll deploy a Next.js webapp in AWS using Lambda@Edge and CloudFront with support for SSR and ISG. Both our Next.js webapp and our Infrastructure as Code (IaC) will use TypeScript, using the Cloud Development Kit (CDK).

When you have a highly dynamic webapp with a lot of content, your number of pages will grow and grow. Static generation is great, but more pages, more time to build. Server Side Rendering is great to solve some of these problems but takes its cut on Time To First Byte (TTFB). Also, long builds with unnecessary computation might also incur additional expenses. Ideally, your application is intelligent enough to understand which products changed and incrementally update those pages with no full rebuild. And Incremental Static Regeneration just do that in Next.js. Until now I was just able to deploy functional webapps with ISR in AWS using containers (ECS or Fargate), but now, thanks to new releases for CDK Construct we can make full serverless deployment of rich webapps on AWS with CDK.

What you need to know

Level 200: Intermediate
Have Node installed;
You should have an AWS CLI and setting your credentials;
You should have CDK installed and bootstrapped in your account;
Some TypeScript knowledge needed.

Basic architecture

From docs. Missing in this picture is an extra lambda function to handle images, for optimization.

Gotta Fetch’Em All!

Our application will fetch data from the PokéAPI — will be not much fancy, just a regular page that fetches some data and render a page server-side, and in a specific route, we’ll make a “starter build” with the first 25 Pokémons, and then regenerate on the go new Pokémons as requested.

I generated all the types, Pokedex.ts and Pokemon.ts from quicktype. Even their examples are from PokéAPI! I used their VS Code Paste JSON as Code extension. It takes the API JSON output and returns interfaces and types to better deal with our logic. The autocomplete for the nested attributes on the JSON tree of an object makes the developer experience great, this is what TypeScript was born for.

Next.js

This tutorial assumes you already know how to setup your environment and take your first steps with Next.js (install, running, create some pages). And knows what is TypeScript, which, has an excellent support on Next.js. To generate a brand new project just run your weapon of choice npm or yarn:

npx create-next-app --ts
# or
yarn create next-app --typescript

Or read the great official documentation from Next.js team on TypeScript integration.

We’ll use the pattern to create an /src folder and place our pages and components folder there. I like to create at root level a lib and types folder and a build folder with the code to deploy our stack. After install all dependencies, dev dependencies and deploy it to AWS, your root folder for this project will look like this:

.
 public/
 lib/
 src/
 node_modules/
 build/
 types/
 cdk.out/
 cdk.json
 bin.ts
 stack.ts
 next-env.d.ts
 next.config.js
 package-lock.json
 package.json
 tsconfig.json
 README.md

Since we don’t need build or cdk.out to be on version control, remember to place both on your .gitignore file.

TSConfig

Our tsconfig.json will work both for our CDK and our Next.js. How cool is that?

{
  "compilerOptions": {
    "alwaysStrict": true,
    "downlevelIteration": true,
    "esModuleInterop": true,
    "forceConsistentCasingInFileNames": true,
    "inlineSourceMap": true,
    "lib": [
      "es2020",
      "DOM"
    ],
    "moduleResolution": "node",
    "noEmitOnError": true,
    "noFallthroughCasesInSwitch": true,
    "noImplicitAny": true,
    "noImplicitThis": true,
    "noImplicitReturns": true,
    "noUncheckedIndexedAccess": true,
    "noUnusedLocals": true,
    "noUnusedParameters": true,
    "resolveJsonModule": true,
    "strict": true,
    "strictBindCallApply": true,
    "strictFunctionTypes": true,
    "strictNullChecks": true,
    "strictPropertyInitialization": true,
    "stripInternal": true,
    "target": "ES2020",
    "typeRoots": [
      "node_modules/@types"
    ],
    "useDefineForClassFields": true,
    "allowJs": true,
    "skipLibCheck": true,
    "noEmit": true,
    "module": "commonjs",
    "isolatedModules": true,
    "jsx": "preserve",
        "baseUrl": ".",
    "paths": {
      "@/components/*": [
        "./src/components/*"
      ],
      "@/lib/*": [
        "./lib/*"
      ],
      "@/pages/*": [
        "./src/pages/*"
      ],
      "@/types/*": [
        "./types/*"
      ]
    },
  },
  "exclude": [
    "node_modules"
  ],
  "include": [
    "src",
    "next-env.d.ts",
    "**/*.ts",
    "**/*.tsx",
    "lib"
  ]
}

Some notes about it: compilerOptions.lib must have [‘es2020’, ‘DOM’] at least. Our application will have a life in the lambda functions in Node.js and in the browser as a JavaScript webapp. I always use absolute paths compilerOptions.paths[‘@/components/*’] per example. It makes so clean the imports and is more clear the domains. For this app we’ll follow the strict path.

CDK dependencies and configurations

All CDK dependencies are development dependencies. I will show examples in npm from now on, but feel free to use yarn or whatever please you.

npm install -D aws-cdk @aws-cdk/core @sls-next/cdk-construct @sls-next/lambda-at-edge @aws-cdk/aws-lambda ts-node

As highlight, aside from CDK and its construct libraries, ts-node is a TypeScript execution engine and REPL for Node.js. It JIT transforms TypeScript into JavaScript, enabling you to directly execute TypeScript on Node.js without pre-compiling. This is accomplished by hooking node’s module loading APIs, enabling it to be used seamlessly alongside other Node.js tools and libraries.

And we’ll need some types for our webapp:

npm install -D @types/gtag.js @types/node @types/react

So, we create cdk.json:

{
  "app": "npm run deploy"
}

That will instruct CDK to run a specific script we need to add to the scripts section of the package.json along side Next.js ones:

  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "cdk": "cdk",
    "deploy": "ts-node bin.ts"
  }

So, the deploy command will run the file bin.ts:

import * as cdk from "@aws-cdk/core";
import { Builder } from "@sls-next/lambda-at-edge";
import { NextStack } from "./stack";

const builder = new Builder(".", "./build", {args: ['build']});

builder
  .build(true)
  .then(() => {
    const app = new cdk.App();
    new NextStack(app, "NextJsPokeStack", {
      env: {
        region: 'us-east-1',
      },
      analyticsReporting: true,
      description: "Testing deploying NextJS Serverless Construct"
    });
  })
  .catch((e) => {
    console.error(e);
    process.exit(1);
  });

That is importing the stack.ts file, with our infra declaration for the NextJSLambdaEdge:

import * as cdk from "@aws-cdk/core";
import { Duration } from "@aws-cdk/core";
import { NextJSLambdaEdge } from "@sls-next/cdk-construct";
import { Runtime } from "@aws-cdk/aws-lambda";

export class NextStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props: cdk.StackProps) {
    super(scope, id, props);
    new NextJSLambdaEdge(this, "NextJsApp", {
      serverlessBuildOutDir: "./build",
      runtime: Runtime.NODEJS_12_X,
      memory: 1024,
      timeout: Duration.seconds(30),
      withLogging: true,
      name: {
        apiLambda: `${id}Api`,
        defaultLambda: `Fn${id}`,
        imageLambda: `${id}Image`,
      },
    });
  }
}

Checkout the docs for more available props. I recommend the use of naming each one of three lambda functions created: apiLambda, defaultLambda and imageLambda. Because otherwise will create with a specific name and if you happen to deploy another stack in the same account you will get an error about the a function with that name already exists. So, rename your stacks or even better, add some randomness to names.

We will create two different renders, [ditto].tsx for Server Side Rendering (SSR), that is basically the same of static props, but in server side and because of it, dynamic by nature.

import { GetServerSideProps } from 'next'
import Head from "next/head"
import { getPokemonData } from '@/lib/fetch'
import PokemonForm from '@/components/pokemon'

import type { Pokemon } from '@/types/Pokemon'

interface PokemonApi {
  data: Pokemon
}

const Ditto = (props: PokemonApi) => {
  if (!props?.data?.name) return null;

  const pokeName = props.data.species.name.charAt(0).toUpperCase() + props.data.species.name.slice(1)

  return (
    <section className="container">
      <Head>
        <title>{pokeName} | PokéServeless - AWS Serverless Lambda@Edge</title>
        <meta property="og:title" content={`${pokeName} | PokéServeless - AWS Serverless Lambda@Edge`} key="title" />
      </Head>
      <header>
          <h1>PokéServerless — Server Side Rendering</h1>
      </header>
      <PokemonForm poke={props} />
    </section>
  )

}


export const getServerSideProps: GetServerSideProps = async (context) => {
  let data;

  const { ditto} = context.query;

  if (typeof ditto === 'string') {
    data = await getPokemonData(ditto)
  } else {
    data = {}
  }

  return { props: { data } }
}

export default Ditto

And [porygon].tsx, to do Incremental Static Regeneration (ISR). This is an overview of the flow of ISR:

Its from Lee Robinson’s “A Complete Guide To Incremental Static Regeneration (ISR) With Next.js” pots and its a very fair title.

This is my code:

// Example of ISG
import { GetStaticPaths, GetStaticProps } from 'next';
import Head from "next/head"
import { useRouter } from 'next/router';
import { getPokemons, getPokemonData } from '@/lib/fetch'
import PokemonForm from '@/components/pokemon'

import type { Pokemon } from '@/types/Pokemon'
import type { Pokedex } from '@/types/Pokedex'

interface PokemonApi {
  data: Pokemon,
  date: string
}

const Porygon = (props: PokemonApi) => {
  if (!props?.data?.name) return null;
  const router = useRouter();

  if (router.isFallback) {
    return <div>Loading......I had to fetch incrementally!!</div>;
  }

  const pokeName = props.data.species.name.charAt(0).toUpperCase() + props.data.species.name.slice(1)

  return (
    <>
    <section className="container">
      <Head>
        <title>{pokeName} | PokéServerless - AWS Serverless Lambda@Edge</title>
        <meta property="og:title" content={`${pokeName} | PokéSSR - AWS Serverless Lambda@Edge`} key="title" />
      </Head>
      <header>
          <h1>PokéServerless — Incremental Static Regeneration</h1>
      </header>
      <PokemonForm poke={props} />
    </section>
    <p className="poke-center">{`Generated at ${new Date(props.date).toLocaleString()}`}</p>
    </>
  )
}

export const getStaticProps: GetStaticProps = async (context) => {
  let data

  if (context.params) {

    data = await getPokemonData(context.params.porygon as string)
  } else {
    data = {}
  }

  return {
    props: {
      data,
      date: new Date().toISOString(),
    },
    revalidate: 60 * 5
  }
};

export const getStaticPaths: GetStaticPaths<{ porygon: string }> = async () => {

  const pokemons = await getPokemons(25) as Pokedex

  const paths = pokemons.results.map((pokemon) => {
    return { params: { porygon: pokemon.name.toString() } };
  });

  return {
    fallback: true,
    paths,
  };
};

export default Porygon

To leverage all the power of components we’ll use the same component for each strategy, which we’ll call... pokemon.ts:

import Image from 'next/image'
import Link from 'next/link'
import Button from '@/components/button'
import Spacer from '@/components/spacer'
import { Pokemon, Type } from '@/types/Pokemon'

interface PokemonInfo {
  poke: {
    data: Pokemon
  }
}

const PokemonForm = (props: PokemonInfo) => {

  const pokeImage = props.poke?.data?.sprites?.other?."official-artwork"]?.front_default ?? props.poke?.data?.sprites?.front_default;

  const number = props?.poke?.data?.order;
  const isPositive = number >= 1;
  const pokeNumber = isPositive ? number : "Max version";

 return (
    <>
    <article className="ditto">
        <Image
          src={pokeImage}
          width={240}
          height={240}
          alt={`Pokémon ${props?.poke?.data?.name}`} />

        <h1 className="poke-name">{props?.poke?.data?.name}</h1>
        <Spacer size="12" style={{marginTop: '20px'}}/>
        <p>Number: {pokeNumber}</p>
        <p>Type:</p>
        <ul className="poke-list">
          {props?.poke?.data?.types?.map((info: Type, index: number) => (
            <li key={index}> {info.type.name}</li>
          ))}
        </ul>

        <Spacer size="12" style={{marginTop: ‘20px’}} />
        <Button />
      </article>

      <div className="poke-footer">
        <div></div>
        <div className="poke-options">
            <div>
              <Link href="/ssr">
                <a title="Server Side Rendering">
                  <Image
                    src="https://raw.githubusercontent.com/ibrahimcesar/nextjs-ssr-cdk-aws/main/public/ditto.png"
                    width="125"
                    height="112"
                  />
                  <h2>Server Side Rendering<br/>(SSR)</h2>
                </a>
              </Link>
            </div>

            <div>
              <Link href="/isr">
                <a title="Incremental Static Regeneration">
                  <Image
                    src="https://raw.githubusercontent.com/ibrahimcesar/nextjs-ssr-cdk-aws/main/public/porygon.png"
                    width="125"
                    height="112"
                  />
                  <h2>Incremental Static Regeneration<br/>(ISR)</h2>
                </a>
              </Link>
            </div>
        </div>
        <div></div>
      </div>
    </>
  )
}

export default PokemonForm

It has do some checks, if the number is equal or bigger than one (because Dynamax and Gigamax versions doesn’t return numbers) and which image to use, and we render with the Image component from Next.js for optimization.

I also made two different “home pages” for each strategy, SSR and ISR, and you could checkout the bare-bones and ugly site for demonstration purposes only™.

Then, to deploy you just run in your root folder cdk and you should see and approve the changes and wait for ir to finish:

cdk deploy

If you are like me and have several profiles in your AWS credentials file, as a “advice”, I would argue to always explicit use cdk deploy --profile personal or whatever you named to not mix environments, resources, stacks and accounts!

After done you’ll get a ✅ with the name of your stack, mine being NextJsPokeStack and you can search the address in the CloudFront distribution, or configure a domain in the props or output yourself this value from the process if you needed.

And... done ✅ You successfully deployed Next.js serverless in AWS with the help of AWS CDK. This is the final infra created, made by the awesome tool cdk-dia:

Click here for a full version

Earlier this year AWS launched the Amplify SSR support for Next.js but... is stuck at version 9 at the time — you can’t use the Image component and the ISR at the time. And we just made a isomorphic deployment of code AND infra with TypeScript.

NextJS Developer Extensions Pack: Install the best VSCode extensions with one click

Ibrahim Cesar — Wed, 21 Jul 2021 01:04:47 +0000

It looks that we'll never run out of VS Code best extensions posts and lists, right? As an daily user of VS Code I think I can related with the feeling of share with other people how amazing some tools are and help boost our productivity in some many forms and ways that we wouldn't guess.

As a web developer and mostly focused on NextJS working with style-components and TypeScript, my arsenal of tools may not be everyone's cup of tea but more and more helping others and teaching, I decided to create an extension pack to better share my weapons of choice. After that you can just uninstall that you don't use it or even needed. After all, there's no size fits all, even more with such personal as is the development space in which we program.

This is all the extensions on my greatest hist lists, I'm sure most are very well known, others are more niche but super helpfull.

Auto Close Tag: Automatically add HTML/XML close tag, same as Visual Studio IDE or Sublime Text
Auto Rename Tag: Auto rename paired HTML/XML tag
axe Accessibility Linter: Coding for accessibility made easy.
Better Comments: The Better Comments extension will help you create more human-friendly comments in your code
Blockman - Highlight Nested Code Blocks: Mark/Highlight code blocks
Bookmarks: It helps you to navigate in your code, moving between important positions easily and quickly.
Code Spell Checker: Spelling checker for source code
Color Highlight: Highlight web colors in your editor
DotENV: Support for dotenv file syntax
Git Graph: View a Git Graph of your repository, and perform Git actions from the graph.
GitLens — Git supercharged: Supercharge the Git capabilities built into Visual Studio Code — Visualize code authorship at a glance via Git blame annotations and code lens, seamlessly navigate and explore Git repositories, gain valuable insights via powerful comparison commands, and so much more
ESLint: Integrates ESLint JavaScript into VS Code.
File Utils: A convenient way of creating, duplicating, moving, renaming and deleting files and directories.
indent-rainbow: Makes indentation easier to read
Jest: Use Facebook's Jest With Pleasure.
JSON to TS: Convert JSON object to typescript interfaces
Markdown Preview Enhanced: Markdown Preview Enhanced ported to vscode
markdownlint: Markdown linting and style checking for Visual Studio Code
Material Icon Theme: Material Design Icons for Visual Studio Code
Next.js snippets: snippets for nextjs
Paste JSON as Code: Copy JSON, paste as Go, TypeScript, C#, C++ and more.
Path Intellisense: Visual Studio Code plugin that autocompletes filenames.
Prettier - Code formatter: Code formatter using prettier
Rainbow Brackets: Provide rainbow colors for the round brackets, the square brackets and the squiggly brackets. This is particularly useful for Lisp or Clojure programmers, and of course, JavaScript, and other programmers.
Sort JSON objects: Sorts the keys within JSON objects
Template String Converter: Converts quotes to backticks when a $ and { are entered within quotes in JavaScript and TypeScript files
Thunder Client: Rest API Client for VS Code, GUI based Http Client
Todo Tree: Show TODO, FIXME, etc. comment tags in a tree view
Toggle Quotes: cmd ' (ctrl ' on win/linux) will cycle the first quote pair found
Trailing Spaces: Highlight trailing spaces and delete them in a flash!
TSLint: TSLint support for Visual Studio Code
Visual Studio IntelliCode: AI-assisted development
vscode-styled-components: Syntax highlighting for styled-components

To install, just go to the Visual Studio Marketplace and smash the Install button.

Happy coding everyone 😃

A Trillion Dollar Paradox that never was: the largest bank in Latin America will move half of its infra to AWS Cloud

Ibrahim Cesar — Sat, 05 Jun 2021 21:40:23 +0000

The Cost of Cloud, a Trillion Dollar Paradox by Sarah Wang and Martin Casado makes two main arguments:

Cloud is the “perfect platform to optimize for innovation, agility, and growth, but it comes at a cost; there’s a ‘flexibility tax’ for the nimbleness the cloud provides.” Therefore, it’s critical to make cloud spend a first-class KPI metric, and empower engineering to optimize from day one.
As a company grows, the pressure that cloud puts on its margins “can start to outweigh the benefits.” Therefore, Wang and Casado present the idea of “repatriating” cloud workloads — bringing all or most out of the cloud.

The short version is: You’re crazy if you don’t start in the cloud, but you’re crazy if you stay on it.
Jeremy Daly on off-by-none #144

But I think it paints a picture biased with a series de logical jumps and do so ignoring other and probably bigger trends in motion. I would like to bring the case of a Brazilian banking that is repatriating. But, nothing coming back, but seeking Cloud citizenship.

Itaú Unibanco is the largest bank in Latin America with 407.37 US$ Billions in Assets. Milton Maluhy Filho, CEO of Itaú, highlighted growth as the main driver and technology as a way to do so, in an event in June, 2 called Itaú Day (this links is an official English dubbed version of it). At some point he says that they have 24M MAU in their app (again, a banking app).

Contract with AWS ([sic]AmazonWeb Services)

Ricardo Guerra, the banks's CTO made public a milestone: they want to move half of its capabilities to the public Cloud until the end of 2022, with a ten-year deal with AWS. Last year it was made public the contract only hinting that "a great cut" of its infra would go to the Cloud. Now the number has revealed to be 50%.

50% of the bank in the Cloud until 2022

It was mentioned analytics, machine learning, serverless, containers, managed databases, computing, storage and security – the whole deal. And they also publicly announced they are moving from monoliths to microsservices. The infrastructure for Pix, a new and more cheap payment released by Brazil's Central Bank is already in a microservices architecture, and the Credit Card operation as well.

Microservices

Last year, profits of Itaú because of COVID-19 and Brazilian market, felt strongly 35%. They made only 18.91 BRL Billions in profit. "In times of crises, some people cry — other people sell handkerchiefs", there's a saying. I would add, "...and then there are the banks".

For this they already hired 4 thousand new professionals. Until 5 years ago, the plan was invest in its own infra: the bank own that in 2015 received 3.3 BRL Billions, which if you take the inflation, would around 1 US$ Billion.

We are talking about call-centers, bank web and mobile apps and banking platforms.

Nubank, a Brazilian Unicorn, is also the largest independent digital bank in the world. It started with a credit card offering and now is turning itself in a complete bank, without one single physical agency. And runs on AWS.

I bring attention to these two prominent examples that are close to home to me and from one of the hardest markets in the world: banking. The focus on SaaS also leaves a lot of workloads of a far great Universe of companies using the public Cloud.

To frame the discussion: We estimate the recaptured savings in the extreme case of full repatriation, and use public data to pencil out the impact on share price. We show (using relatively conservative assumptions!) that across 50 of the top public software companies currently utilizing cloud infrastructure, an estimated $100B of market value is being lost among them due to cloud impact on margins — relative to running the infrastructure themselves. And while we focus on software companies in our analysis, the impact of the cloud is by no means limited to software. Extending this analysis to the broader universe of scale public companies that stands to benefit from related savings, we estimate that the total impact is potentially greater than $500B.

You can say I'm just a person random on the Internet, but these numbers are completely made-up. Right after state that the "point of this post isn’t to argue for repatriation", and then drops a bomb that leads to the trillion dollar paradox of the title. The only paradox in this case, I would say is that it was never existed first place.

I also recommend Zack Kanter Stedi's Founder and CEO analysis of the piece:

// Detect dark theme var iframe = document.getElementById('tweet-1399013516107948037-705'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1399013516107948037&theme=dark" }

So, again, I think nothing better than analyses actual use cases to get a better picture of what's going on or even the myths and narratives some people created based on their own limited dataset. The biggest bank in Latin America is moving on-premises to be 50% Cloud at AWS until 2022 and they invested a huge money and they had plan to expand but their strategy changed to embrace Cloud, and one of Brazil's Unicorn, also from Banking industry is Cloud Native since day one. If I was a VC, I would bet my money on Cloud citizenship and not otherwise.

We need to talk more about equality in tech

Ibrahim Cesar — Sun, 23 May 2021 21:02:38 +0000

This post is an invitation by Salma from Unbreak.tech that aims to be “a platform for MEN to educate other MEN about the need for change and equality in tech”. Politics is a taboo in tech circles. I’m out of my base here, but in some camps politics led to a meltdown after some gems of privileged men in power. But we are social animals. Politics is everywhere because is about everything we do. As simple as a Wikipedia definition can be, politics “is the set of activities that are associated with deciding in groups, or other forms of power relations between individuals, such as the distribution of resources or status". And I will talk direct with you, my fellow men.

Even if you cannot see there’s a context, a structural web of power relations codified in the public speech, gender norms, cultural products and in the market itself that are a byproduct of several instances of decision making that made the world what it is. Sexist. Racist. Abusive. And I know, is difficult to see, exactly because of that. It embedded us in it. Because, my friends, this is water.

This video, from the late writer David Foster Wallace, starts with a humorous parable:

There are two young fish swimming along who meet an older fish. The older fish nods at them and says:

‘Morning boys, how’s the water?’

The two young fish swim on for a bit and then eventually one of them looks over at the other and asks:

‘What the hell is water?’

It is easy to not see the environment structures, we are immersed and shaped every aspect of our existence. Structural racism, gender norms. We come out to this world and this doesn’t look like the “natural way of this”. This is water.

We must embrace other perspectives, understand our immersion and bias. We must listen. As David Foster Wallace, understand this has nothing to do with knowledge, that we developers are so proud of, but with “how to think”, and he says: “The alternative is unconsciousness. The gnawing sense of having lost some infinite thing”. Maybe because a highly respected white guy said this, more people can understand and immerse in this call for empathy and think in our context, give gravitas. Do not consider only the lobster. Consider people with disabilities. Consider people of color. Consider people of all genders.

“Even prior to the term “computer” being a name given to a machine, it had been the name given to job. Originally, a computer was a person—almost always a young woman—who computed complex equations with the help of pen and paper or a desktop accounting machine. A common misperception is that women got into computing during World War II simply because men were at the front, but the gendering of computing work existed before the war, and before computers were electronic. The feminization of this work continued through and after the war, with women returning to the civilian workforce to perform computing work with electromechanical and later electronic systems—everything from programming and operation, to systems analysis, to hardware assembly.”

“Your Computer Is on Fire”, “Sexism Is a Feature, Not a Bug” by Mar Hicks

And if you are a man, you don’t feel personally responsible for any of that. Right? You did nothing wrong. We mean, well, except maybe that time or that time. Was a grey area wasn’t? But this is water.

“If you are white in a white supremacist society, you are racist. If you are male in a patriarchy, you are sexist.” — Ijeoma Oluo, So You Want to Talk About Race

Since we are existing in this planet if you is identified with a specific set of characteristics you are, signed or not for this, benefiting from this. What to do?

Mike Monteiro wrote Ruined by Design: How Designers Destroyed the World, and What We Can Do to Fix It. This book talks about our roles in creating – or ruined - the world. Monteiro calls the responsibility in the hands of the designers of digital products, meaning everyone involved in the process to plan, mock, code, ship and support digital products. In one moment of the book he recalls a grim conversation he had in Copenhagen:

[…] the topic of “being good allies” came up. To which my new Danish friend shouted that men our age had committed too many sins and done too many things wrong to ever be good allies in any sense of the world. And the best thing we could to for the planet was to die.

I still think a lot on it. As Monteiro points out:

As uncomfortable as it is to admit, I am both those things. And if you are reading this and you look like me, you are too. Regardless of how well you’ve lived your life, regardless of how good your intentions were, you benefited from a stacked deck. And yet, even with the deck stacked in our favor, we couldn’t do the job. So yes, the best thing we can do for the planet is to die.

I think a lot about this. But I don’t know if suggest you to die would be useful. All human beings are entitled to live. You are entitled to pursue your happiness, but we should be aware of the water.

I’m a male cisgender Brazilian. I doubt that any American or person in Europe would call me “White”—I think this should be a common issue with Latinxs in their homelands. And we even talked about intersectionality. If you are a person of color, the accents (if I was talking to you right now, maybe you would be biased by my way of talk since English is my second language). But my skin falls in the spectrum of whiteness here in Brazil and I have many of the benefits of it. Even having a lot of miscegenation on my family. I rise from a poor family to be a technology director at a small, but innovative, company in São Paulo, our biggest city. I could be the poster child of “meritocracy”. But this is bullshit. Meritocracy appeared first as a parody. There is no such thing as meritocracy.

It was not entirely but design, but at certain time, I was the only male developer on my team. Which is rare in a market in which woman occupy 25% of the roles, and this research included support and QA, areas that we like or not are considered “less” than others. Brillant engineers I have the opportunity to look and simply didn’t get a change in the workplace.

// Detect dark theme var iframe = document.getElementById('tweet-1393201902762954752-253'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1393201902762954752&theme=dark" }

Worst yet if you to be a founder. To automate the work of Paul Graham as VC we would not even need Machine Learning. A few if and else’s would do the job. White. Male. Dropout, but of an Ivy League College. Looks like Mark Zuckerberg.

“As our technological systems become increasingly destructive to our professed social and political ideals, we can no longer afford to collectively fail to understand the layers and decades of intentional decisions that have led to these supposedly unforeseen consequences. The current situation shows us clearly how, as large computing and telecommunications systems have scaled, the power imbalances they foster have altered all of our social institutions, including our political process”

“Your Computer Is on Fire”, “Sexism Is a Feature, Not a Bug” by Mar Hicks

Still, what to do? So many things in so many areas.

Hire women. Promote women. If you see a woman being interrupted, ask for her input right after the interruption. If someone makes a “joke”, ask why that person thinks that is funny. See something weird, ask her if she feels safe. Raise your voice to speak with her if needed. Do not be complacent. Instead of trying to be “10x Developer”, why not first try to be at least 1x human? Do something. Act. Because only posts will not do much. Only actions will mean a thing for the person you helped. Use your power. Yes, you have. You born with this, asking or not for it. With great power comes great responsibility. Is up to us. Do not just reproduce structural sexism. What are you, an android? Are you just programmed? Chances are you are. We are born this way. We must fight it every minute. Keep afloat. Do not get drowned in the abyss of the deep water. The alternative is unconsciousness. The gnawing sense of having lost some infinite thing.

Cover image, detail of photo by cottonbro on Pexels.

How to host a Next.js web apps with server-side rendering (SSR) in AWS Amplify

Ibrahim Cesar — Tue, 18 May 2021 23:31:45 +0000

No more waiting! It’s finally here, AWS Amplify hosting for Next.js server-side rendering (SSR).

Why this is great: It’s no joke, I wait for this feature for more than a year. And is finally here and is fast!

Not so great: Next 10.x.x was a huge leap forward. All things I wanted to try, right off the bat, and backed in my applications...But this will be on hold, as the time of writing, this launch only his support is for Next.js 9.x.x version. Considering version 10.x is from October 2020 I think the pace is a little slow to catchup.

To evaluate what type of rendering your application need, I recommend the post Next.js: Server-side Rendering vs. Static Generation by Lee Robinson—and in the real world sometimes you’ll need both.

The webapp

For all purposes, you could do with your app or create a boilerplate new NextJS with SSR. I created this barebones site that renders some info from the PokéAPI. This is the repository on GitHub:

ibrahimcesar / nextjs-ssr-amplify-aws

👾 ‏‏‎ ‎‏‏‎ ‎‏‏‎ ‎Repository to test NextJS Server Side Rendering with AWS Amplify

👾 AWS Amplify NextJS SSR

Example of NextJS SSR example to deploy with AWS Amplify SSR Hosting using the PokéAPI

Developed in 🇧🇷 Brazil

PokéAPI Status Page

Deploying

If you never used Amplify

You’ll need to have the amplify installed and configured. Just follow the docs and you’ll be ready to go. Or make sure you have the latest version.

Amplify Init

❯ amplify init
Note: It is recommended to run this command from the root of your app directory

? Enter a name for the project pokessr

The following configuration will be applied:

Project information

| Name: pokessr

| Environment: dev

| Default editor: Visual Studio Code

| App type: javascript

| Javascript framework: react

| Source Directory Path: src

| Distribution Directory Path: build

| Build Command: npm run-script build

| Start Command: npm run-script start

? Initialize the project with the above configuration? Yes

Using

…

View on GitHub

You can leave all build options as it is, because Amplify will automatically pick up as SSR and deploy it. For this to happen you also don’t need (or have to) choose a different export folder. If you clone my repo, you could check that I’m indeed using NextJS 10.0.0 but, I cannot use any additional feature like the new Image component.

Deploying

If you never used Amplify

You’ll need to have the amplify installed and configured. Just follow the docs and you’ll be ready to go. Or make sure you have the latest version.

Amplify Init

❯ amplify init
Note: It is recommended to run this command from the root of your app directory
? Enter a name for the project pokessr
The following configuration will be applied:
Project information
| Name: pokessr
| Environment: dev
| Default editor: Visual Studio Code
| App type: javascript
| Javascript framework: react
| Source Directory Path: src
| Distribution Directory Path: build
| Build Command: npm run-script build
| Start Command: npm run-script start

? Initialize the project with the above configuration? Yes
Using default provider  awscloudformation
? Select the authentication method you want to use: AWS profile
For more information on AWS Profiles, see:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html

? Please choose the profile you want to use amplify

Besides the name, pokessr, I only choose my profile to deploy, called amplify but you can deploy in whatever profile you configured. I accepted all defaults. Then amplify will create your environment:

Adding backend environment dev to AWS Amplify Console app: d31r520fbr96mj

⠙ Initializing project in the cloud...

CREATE_IN_PROGRESS amplify-pokessr-dev-185133 AWS::CloudFormation::Stack Tue May 18 2021 18:51:41 GMT-0300 (Horário Padrão de Brasília) User Initiated
CREATE_IN_PROGRESS UnauthRole                 AWS::IAM::Role             Tue May 18 2021 18:51:45 GMT-0300 (Horário Padrão de Brasília)
CREATE_IN_PROGRESS AuthRole                   AWS::IAM::Role             Tue May 18 2021 18:51:45 GMT-0300 (Horário Padrão de Brasília)
CREATE_IN_PROGRESS DeploymentBucket           AWS::S3::Bucket            Tue May 18 2021 18:51:46 GMT-0300 (Horário Padrão de Brasília)
CREATE_IN_PROGRESS UnauthRole                 AWS::IAM::Role             Tue May 18 2021 18:51:46 GMT-0300 (Horário Padrão de Brasília) Resource creation Initiated
CREATE_IN_PROGRESS AuthRole                   AWS::IAM::Role             Tue May 18 2021 18:51:46 GMT-0300 (Horário Padrão de Brasília) Resource creation Initiated

⠇ Initializing project in the cloud...

CREATE_IN_PROGRESS DeploymentBucket AWS::S3::Bucket Tue May 18 2021 18:51:46 GMT-0300 (Horário Padrão de Brasília) Resource creation Initiated

⠸ Initializing project in the cloud...

CREATE_COMPLETE AuthRole   AWS::IAM::Role Tue May 18 2021 18:51:59 GMT-0300 (Horário Padrão de Brasília)
CREATE_COMPLETE UnauthRole AWS::IAM::Role Tue May 18 2021 18:51:59 GMT-0300 (Horário Padrão de Brasília)

⠹ Initializing project in the cloud...

CREATE_COMPLETE DeploymentBucket           AWS::S3::Bucket            Tue May 18 2021 18:52:08 GMT-0300 (Horário Padrão de Brasília)
CREATE_COMPLETE amplify-pokessr-dev-185133 AWS::CloudFormation::Stack Tue May 18 2021 18:52:10 GMT-0300 (Horário Padrão de Brasília)

✔ Successfully created initial AWS cloud resources for deployments.
✔ Initialized provider successfully.

Initialized your environment successfully.

Your project has been successfully initialized and connected to the cloud!

Some next steps:

“amplify status” will show you what you’ve added already and if it’s locally configured or deployed
“amplify add <category>“ will allow you to add features like user login or a backend API
“amplify push” will build all your local backend resources and provision it in the cloud
“amplify console” to open the Amplify Console and view your project status
“amplify publish” will build all your local backend and frontend resources (if you have hosting category added) and provision it in the cloud

Pro tip:
Try “amplify add api” to create a backend API and then “amplify publish” to deploy everything

What we are going to use is hosting.

And by zero configuration, you just need to connect your repository and the building settings will be set.

And you can always have a look at how the build is going accessing the logs in the AWS Amplify console. For our purposes, see a Starting SSR Build in your logs:

2021-05-18T22:35:49.379Z [INFO]: info  - Creating an optimized production build...
2021-05-18T22:35:58.592Z [INFO]: info  - Compiled successfully
                                 info  - Collecting page data...
2021-05-18T22:35:59.098Z [INFO]: info  - Generating static pages (0/28)
2021-05-18T22:35:59.480Z [INFO]: info  - Generating static pages (7/28)
2021-05-18T22:35:59.600Z [INFO]: info  - Generating static pages (14/28)
2021-05-18T22:35:59.706Z [INFO]: info  - Generating static pages (21/28)
2021-05-18T22:35:59.797Z [INFO]: info  - Generating static pages (28/28)
2021-05-18T22:35:59.797Z [INFO]: info  - Finalizing page optimization...
2021-05-18T22:35:59.814Z [INFO]: 
2021-05-18T22:35:59.860Z [INFO]: Page                              Size     First Load JS
                                 ┌ λ /                             1.32 kB        68.7 kB
                                 ├   /_app                         0 B            64.2 kB
                                 ├ λ /[ditto]                      1.39 kB        68.7 kB
                                 ├ ○ /404                          2.76 kB        66.9 kB
                                 ├ ● /pokemons/[name]              1.53 kB        68.9 kB
                                 ├   ├ /pokemons/bulbasaur

                                 ├   ├ /pokemons/ivysaur

                                 ├   ├ /pokemons/venusaur

                                 ├   └ [+22 more paths]

                                 └ λ /random                       1.39 kB        68.7 kB
                                 + First Load JS shared by all     64.2 kB
                                 ├ chunks/commons.b2f5db.js      13.5 kB
                                 ├ chunks/framework.149f13.js    42 kB
                                 ├ chunks/main.e0d560.js         6.8 kB
                                 ├ chunks/pages/_app.9245f7.js   865 B
                                 ├ chunks/webpack.f82c36.js      950 B
                                 └ css/b8e1ed54af27c57535f7.css  897 B

2021-05-18T22:35:59.861Z [INFO]: λ  (Server)  server-side renders at runtime (uses getInitialProps or getServerSideProps)
                                 ○  (Static)  automatically rendered as static HTML (uses no initial props)
                                 ●  (SSG)     automatically generated as static HTML + JSON (uses getStaticProps)
                                 (ISR)     incremental static regeneration (uses revalidate in getStaticProps)

2021-05-18T22:35:59.993Z [INFO]: Starting SSR Build...
2021-05-18T22:37:10.138Z [INFO]: SSR Build Complete.
2021-05-18T22:37:11.159Z [INFO]: # Completed phase: build
2021-05-18T22:37:11.159Z [INFO]: ## Build completed successfully

Then you gave to wait a couple of minutes and your application you be on your custom domain or in the generate domain of Amplify. For this demonstration my web apps is available here.

Pages

The front page is itself server side generated:

export const getServerSideProps: GetServerSideProps = async () => {
  const data = await getPokemons()

  return {
    props: {
      data
    }
  }
}

It will query in the PokéAPI and return all Pokémons until reaching the number or the maximum today. I placed 3000 but as you can check, the actual number today is 1118.

When you click in a Pokémon, I use a dynamic route in the file ditto to generate the Pokémon by the name. Pokémon fans will get the reference. And [ditto].tsx is also SSR.

export const getServerSideProps: GetServerSideProps = async (context) => {

  let data;
  const { ditto} = context.query

  if (typeof ditto === ‘string’) {
    data = await getPokemonData(ditto)
  } else {
    data = {}
  }

  return { props: { data } }

}

But for fun I created a random page... that renders a random Pokémons to test even better the SSR. It get all the possible Pokémons and returns one at random using Math.random():


export const getServerSideProps: GetServerSideProps = async () => {

  const random = await getPokemons() as Pokedex
  const ditto = random.results[random.results.length * Math.random() | 0].name

  let data;
  if (typeof ditto === ‘string’) {
    data = await getPokemonData(ditto)
  } else {
    data = {}
  }

  return {
    props: {
      data
    }
  }
}

And to test ISG (Incremental Static Generation) I created a folder called pokemons. Stable static generation was added to Next 9.3 but my test doesn’t show that works right now with the Amplify SSR hosting, it defaults to the SSR. ISG is a mechanism to update existing pages, by re-rendering them in the background as traffic comes in using the property revalidate. Also, another great use is, per example, you have a specific dataset of pages to generate at build time but you’ll need on dynamic routes to be generated new pages as soon you publish another in your headless CMS or database. ISG generated even a bounty as feature in another project and unlocks a lot of interesting use cases.

export const getStaticProps: GetStaticProps = async (context) => {
  let data
  if (context.params) {
    data = await getPokemonData(context.params.name as string)
  } else {
    data = {}
  }

  return {
    props: {
      data,
      date: new Date().toISOString(),
    },
    revalidate: 60 * 5
  }
};

export const getStaticPaths: GetStaticPaths<{ name: string }> = async () => {

  const pokemons = await getPokemons(25) as Pokedex

  const paths = pokemons.results.map((pokemon) => {
    return { params: { name: pokemon.name.toString() } }

  })

  return {
    fallback: true,
    paths,
  }

};

For my example, I generate at build time the first 25 Pokémons. See that my props are changing, I’m passing a dynamic date. But my first 25 Pokémons, starting with Bulbasaur and going up to Pikachu. They have a text Generated at that will not revalidate at all (right now configured to re-validate at each 5 minutes revalidate: 60 * 5 in which it should change this date. But if you access any other Pokémon than the first 25, like the number 26, Raichu or the 186, Scizor will be server-side generated at the time and you’ll see the date of any time you accessing again, so no ISG. This feature was not advertised but since it was stable at 9.3+ and I did not found work and it happens you are after this, you’ll need to wait or surprise me show-me what I’m getting wrong here because I would love to have ISG already 😀

I also created a private first component for YouTube embeds if you plan to use something like this in your Next app, check it out:

Why I made my open source React component private by default: an Open Source story

Ibrahim Cesar ・ May 18 '21

#react #opensource #typescript #privacy

Please leave your thoughts, takes and insights in the comments! Or problems if you have one or the solution to ISG!

Why I made my open source React component private by default: an Open Source story

Ibrahim Cesar — Tue, 18 May 2021 12:22:44 +0000

I’m a big fan of open source. Because, well, we all owe to this vast collection of code, libraries, frameworks and knowledge. Just like with arts and science, we stand on the shoulders of Giants. Inspired and iterations upon previous works, adding one more chain, keeping us together. And not just in the figurative sense, but literally, as xkcd pointed in the “It’s funny cause is truth”, Dependency:

Last year, one of books I loved was Working in Public from Nadia Eghbal whose subtitle, “The Making and Maintenance of Open Source Software” makes clear about the themes in the book: the para-social relationship between maintainers and community, the different kinds of projects in a very helpful framework and the economics of produce and maintain high quality work for free for almost all projects. If you scan for reviews quickly, you’ll see someone “complaining” is not technical or doesn’t help as a maintainer... I think they just misread the book and its purposes: explore a vast topic, where each actor has its own interest, that much of our infrastructure at production today depended on, and ... works. These notes from Sid Shanker helps to summarize it or a more in-depth guide here.

Nadia Ehgbal’s framework has four classifications: Federations, Clubs, Stadiums and Toys. My story comes from the ranks of the toys – another world from those in the big categories, like the post from the the Babel project, used by millions but running out of funds.

Motivation

I was fortunate enough to take part of the Chrome Developer Summit 2019 in San Francisco (on a personal note, I propose to my wife there, in the Cupid’s Span). At the time I was finishing a project in the works in the last 6 months: a complete rewrite of our frontend. We used a Bootstrap 3.0 in a backed-in framework on our CMS, that we had basically to “shadow” properties we don’t wanted in the JSPs. It was madness. With the new offering of a headless CMS we rewrote everything in React SSR. I work at a digital only news site in Brazil. I was doing what I called “Lighthouse-Driven Development” to achieve better performance.

I notice two things right away: YouTube iframe looked to load not so fast like the rest. And we don’t run ads in our sites. Of any kind. But as soon as our iframe load with our videos, every ad-blocker would point that there was an ad in the page, and traced back to the YouTube embed. Also, our embeds sometimes loaded with several little “recommendations” that don’t matched our themes exactly. But who I was to blame YouTube embed?

But in one of the talks, Paul Irish presented Lite YouTube Embed, a custom element renders just like the real thing but approximately 224× faster.

And there, in the code, I found that I was loading at least two sources of ad for one iframe. But I was more interested in the “224x”. The project at work could not “wait”. for me work on this. But I used my free time and weekends and built 📺 React Lite YouTube Embed that later became part of my project. My benchmarks never got close, but I could do a faster and cleaner way of loading iframes for our project since we would wait for user interaction, applying Adaptive Loading as presented by Addy Osmani—which, many of the ideas were a lot about my every day work since I was dealing with very poor performance mobiles and low-quality networks that we have here in Brazil and not exception. My motivation was part solve a problem for me and part apply those ideas of respect users’s hardware and network.

It’s alive!

The first release I was so afraid. Of criticism. Of put my code out there. I had to do some tweaks because when integrating with my own work I faced problems. I receive feedback, pull requests and then, on the 6th PR a collaborator, Botho added a nice option noCookie, when true, connected to YouTube via the Privacy-Enhanced Mode using https://www.youtube-nocookie.com. I already had in place adNetwork to preconnect or not to the ad networks and it delighted me to get this kind of not only validation for these kind of concerns but active ideas and work towards the goal. But these privacy enhancements were all by opt-in.

I was getting feedback that the component was not playing nice with Next.JS. I was making a hard-coded import of the css and this would make the app not render at all. Since I was taking more and a more close look to Next.JS—and loving it! I did not want to my first open sourced my little project doesn’t give access to this framework and potentially others.

Privacy by default

At the same time in Brazil we got a new law, our own home-baked GDPR, called LGPD (Lei Geral de Proteção de dados or General Law of Data Protection) and all concerns with the massive misuse of our data. I made a course in the area to better prepare our systems (since we deal with billing for subscriptions and other kind of PII). And I was a “TypeScript newborn”, a recent converted and got the challenge to rewrite from the zero (well, is a small lib, it was not a tremendous feat!). And got me thinking about privacy, how sensitive defaults could become the norm in an invisible, but fundamental way and, I as designer of this component, decided I would make a private by default component. I would make the developer responsible for the choices or even better, be possible to give to the user these choices.

How will evolve our society trusting on advertising corporations to be neutral purveyors of information when their profit depends on manipulating that information misunderstands our capitalist marketplace and the value and nature of unbiased information? I as a developer in a news media outlet was thinking on things like that and I know everyone has an approach to this, even if is “I don’t care”. So, I choose privacy by default.

And this will be my choice from now on. And that PR, of a person I never meet or spoke, but that we collaborate in the same piece of code, gave me confidence in my choices.

And this is my first open source project, on 2.0 since the TypeScript migration.

ibrahimcesar / react-lite-youtube-embed

📺 ‏‏‎ ‎‏‏‎ ‎‏‏‎ ‎< A private by default, faster and cleaner YouTube embed component for React applications />

📺 React Lite YouTube Embed

A private by default, faster and cleaner YouTube embed component for React applications

Developed in 🇧🇷 Brazil

Port of Paul Irish's Lite YouTube Embed to a React Component. Provide videos with a supercharged focus on visual performance. The gain is not the same as the web component of the original implementation but saves some requests and gives you more control of the embed visual. An "Adaptive Loading" way to handle iframes for YouTube.

View Demo

🔒 Up 2.0.0 Privacy by Default

The biggest change is, from 2.0.0 this component is private by default. Meaning that will not preconnect with the ad network from Google and connect to YouTube via the Privacy-Enhanced Mode using https://www.youtube-nocookie.com.

🚀 Install

Use your favorite package manager:

yarn add react-lite-youtube-embed

npm install react-lite-youtube-embed -S

🕹️ Basic Usage

import React from "react";
import { render } from "react-dom";
import

…

View on GitHub

Please leave your thoughts, takes and insights in the comments!

Cover photo by Christian Wiediger on Unsplash

Forem: Ibrahim Cesar

Kafka on the Serverless Shore: Building event-driven applications with Kafka

What is Kafka?

On the shore

ibrahimcesar / lambda-serverless-kafka

🐚 Kafka on the Serverless Shore

🐚 Kafka on the Serverless Shore

Deploying

Eventual consistency

Dive Deep

Certificações AWS — Por quê e de que forma

Por quê você deve tirar uma certificação

Caso de estudo: Minha própria jornada

FAQ

Por onde devo começar?

Qual caminho devo seguir?

É muito caro! Não poderia ser mais barato?

Certificações expiram. Não se trata de mais uma forma dessas empresas ganharem dinheiro?

Preciso saber inglês?

O que acontece se eu não passar no exame?

Top 5 technology books I read in 2021

5. Your Computer Is On Fire

4. Sooner Safer Happier

3. Cloud Native Patterns

2. The CDK Book

Honorable mentions 📚

1. Crafting Interpreters

SAFe™ Considered Harmful

Agily McAgileface

Further reading:

Event-Driven Architectures on AWS

Loosing our couplings

Books to read

Heroes to follow

Ben Ellerby

Sheen Brisals

AWS Event-Driven Tools

The hidden bias in Core Web Vitals

Read more

Next.js TypeScript SSR and ISR serverless deploy with AWS CDK

What you need to know

Basic architecture

Gotta Fetch’Em All!

Next.js

TSConfig

CDK dependencies and configurations

Links

NextJS Developer Extensions Pack: Install the best VSCode extensions with one click

A Trillion Dollar Paradox that never was: the largest bank in Latin America will move half of its infra to AWS Cloud

We need to talk more about equality in tech

How to host a Next.js web apps with server-side rendering (SSR) in AWS Amplify

The webapp

ibrahimcesar / nextjs-ssr-amplify-aws

👾 ‏‏‎ ‎‏‏‎ ‎‏‏‎ ‎Repository to test NextJS Server Side Rendering with AWS Amplify

👾 AWS Amplify NextJS SSR

Deploying

If you never used Amplify

Amplify Init

Deploying

If you never used Amplify

Amplify Init

Pages

Why I made my open source React component private by default: an Open Source story

Ibrahim Cesar ・ May 18 '21

Why I made my open source React component private by default: an Open Source story

Motivation

It’s alive!

Privacy by default

ibrahimcesar / react-lite-youtube-embed

📺 ‏‏‎ ‎‏‏‎ ‎‏‏‎ ‎< A private by default, faster and cleaner YouTube embed component for React applications />

📺 React Lite YouTube Embed

View Demo

🔒 Up 2.0.0 Privacy by Default

🚀 Install

🕹️ Basic Usage