Forem: Miguel Müller

Do You Know How to Debug Code?

Miguel Müller — Mon, 05 Aug 2024 12:11:57 +0000

A Portuguese version is available here

Debugging, or troubleshooting, means thoroughly examining code to detect and eliminate potential errors. Debugging is already an exhausting process and a real challenge in itself. As we advance in our careers, we need to debug not only our code but also our colleagues' code. If that weren't enough, we also face challenges like performance debugging, security, race conditions, and other complex debugging tasks.

Recently, I went through a situation that made me reflect on debugging. I was informed that an implementation I had just delivered was not working as expected. The task involved sending data from a specific record, including the creation date, to an external service. I implemented it, wrote functional tests, performed local tests, and "confirmed" that everything was working correctly. However, BANG!, it wasn't. During debugging, I discovered that a method I frequently used did not return the record's creation date when called remotely. This method, which worked in one context, exhibited different behavior during remote execution.

This episode illustrates the importance of questioning our assumptions. When confronted with a bug, it is crucial to recognize that we might be wrong about one or more of our assumptions. We should not ignore a block of code involved in the bug just because we "know" it works. We must test. Test again. And again with different data and conditions, exploring all possible scenarios.

Stay calm. Panicking is a common reaction when an unexpected problem arises. However, it is essential to face the situation calmly and clearly. Tension can cloud your judgment and make it harder to identify the true source of the problem. We must take a deep breath, analyze the situation objectively, and avoid rash decisions that could worsen the situation.

Accept that the problem exists and focus on solving it. Denying the existence of the problem or downplaying it will not make it go away. Keep an open mind to different possibilities, but avoid tunnel vision. We must identify and address the root cause of the problem, not just the side effects.

Act on the root cause. When confronted with a bug, the tendency might be to try to resolve the issue directly where it manifests. However, this approach may not only be ineffective but could also further harm the code.

Here are some helpful tips for effective debugging:

Don't make assumptions. Assuming that part of the code is working correctly without testing it can lead to errors. Check each part of the code involved in the bug.
Reproduce the bug. Consider the inputs and outputs. Try to replicate the problem exactly as it occurs.
Visualize the state of your data. Understand how the data is being manipulated and how it should be at different points in the process.
Trace the data. From visualizing the data, start tracing how it flows through the system. This can help identify where the data is deviating from the expected.
Start somewhere. When unsure where to start, use binary search as a strategy. Split the problem into two parts. If the problem is present in the first half, the bug is between the start and the midpoint; otherwise, it is between the midpoint and the end.
Eliminate logically. The operating system is not malfunctioning. The database engine is processing your query correctly. Remember the saying: If you find hoof prints, think of horses—not zebras.

Conclusion

Debugging is a vital skill for any programmer, requiring not only specific techniques but also the right mindset. Staying calm, accepting the existence of the problem, and starting the analysis from the beginning are fundamental steps for successful debugging. Every bug is an opportunity to learn and improve your skills, both in problem-solving and maintaining mental clarity in the face of challenges.

Você sabe debugar código ?

Miguel Müller — Mon, 05 Aug 2024 12:09:50 +0000

Debugar, ou Depurar, significa submeter o código a um exame detalhado para detectar possíveis falhas e eliminá-las. Debugar já é um processo exaustivo e um verdadeiro desafio por si só. Conforme avançamos em nossas carreiras, precisamos debugar não apenas nossos próprios códigos, mas também debugar o código de colegas. Se não bastasse ainda precisamos enfrentar desafios como a depuração de performance, segurança, race coditions e outras depurações complexas.

Recentemente, passei por uma situação que me fez refletir sobre a depuração. Fui informado de que uma implementação que eu tinha acabado de entregar não estava funcionando como esperado. A tarefa envolvia enviar dados de um determinado registro, incluindo a data de criação, para um serviço externo. Realizei a implementação, escrevi testes funcionais, fiz testes locais e "confirmei" que tudo estava funcionando corretamente. No entanto, BANG!, Não estava. Na depuração acabei descobrindo que um método que usava com frequencia não retornava a data de criação do registro quando chamado remotamente. Esse método, que funcionava em um contexto, apresentava diferenças de comportamento na execução remota.

Esse episódio ilustra bem a importância de questionar nossas suposições. Quando confrontado com um Bug, é crucial reconhecer que podemos estar errados em uma ou mais de nossas suposições. Não devemos ignorar um bloco de código envolvido no bug apenas porque "sabemos" que ele funciona. Temos que testar. Testar novamente. E de novo com diferentes dados e condições, explorando todos os cenários possíveis.

Mantenha a calma. Entrar em pânico é uma reação comum quando um problema inesperado surge. No entanto, é essencial enfrentar a situação com calma e clareza. A tensão pode prejudicar seu julgamento e dificultar a identificação da verdadeira origem do problema. Devemos respirar fundo, analisar o cenário objetivamente e evitar decisões precipitadas que podem agravar a situação.

Aceite que o problema existe e concentre-se em solucioná-lo. Negar a existência do problema ou minimizá-lo não fará com que ele desapareça. Mantenha a mente aberta para diferentes possibilidades, mas evite ser míope. Temos que identificar e tratar a causa raiz do problema, e não apenas os efeitos colaterais.

Haja na raiz. Quando confrontado com um bug, a tendência pode ser tentar resolver o problema diretamente no ponto onde ele se manifesta. No entanto, essa abordagem pode não só ser ineficaz como também prejudicar ainda mais o código.

Aqui estão algumas dicas úteis para uma depuração eficaz:

Não faça suposições. Assumir que uma parte do código está funcionando corretamente sem testá-la pode levar a erros. Verifique cada parte do código envolvida no bug.
Reproduza o bug. Considere as entradas e saídas. Tente replicar o problema exatamente como ele ocorre.
Visualize o estado dos seus dados. Entenda como os dados estão sendo manipulados e como eles deveriam estar em diferentes pontos do processo.
Rastreie os dados. A partir da visualização dos dados, comece a rastrear como eles fluem através do sistema. Isso pode ajudar a identificar onde os dados estão se desviando do esperado.
Comece de algum lugar. Quando não souber por onde começar, utilize a busca binária como estratégia. Divida o problema em duas partes. Então, Se o problema estiver presente na primeira metada, o bug estará entre o ponto inicial e o intermediário; caso contrário, estará entre o ponto intermediário e o final.
Faça uma eliminação consciente. O Sistema operacional não está com defeito. A engine de banco de dados está processando a sua query de maneira adequada. Lembre-se do ditado: Se encontrar pegadas feitas por cascos, pense em cavalos – não em zebras.

Conclusão

A depuração é uma habilidade vital para qualquer programador, exigindo não apenas técnicas específicas, mas também uma abordagem mental adequada. Manter a calma, aceitar a existência do problema e iniciar a análise do começo são passos fundamentais para uma depuração bem-sucedida. Cada bug é uma oportunidade para aprender e aprimorar suas habilidades, tanto na solução de problemas quanto na manutenção da clareza mental diante dos desafios.

Diferenças entre as Funções Serverless da GCP e as Funções Lambda da AWS - Um Guia Comparativo de Migração

Miguel Müller — Tue, 14 May 2024 15:09:06 +0000

Versão em inglês aqui

Já se passaram alguns meses desde que migrei de uma ambiente GCP (Google Cloud Platform) para um ambiente AWS (Amazon Web Services). No começo eu achei que ia estranhar e ter dificuldade de me adaptar pois já ouvi relatos de profissionais no caminho inverso reclamando, mas para minha surpresa, isso não aconteceu.

E nessa transição eu decidi escrever e compartilhar de forma didática as diferenças sutis com as quais me deparei. Em vez de apresentar um confronto direto, optei por um guia de comparação para ajudar outros profissionais que possam estar considerando uma transição semelhante.

Os padrões de cloud que utilizamos hoje são um esforço principalmente do NIST (Instituto Nacional de Padrões e Tecnologia). Esses padrões têm orientado o desenvolvimento e a implementação de tecnologias de nuvem, garantindo uma abordagem consistente e segura em todos os provedores de computação em nuvem. Por causa desses padrões as semelhanças são muitas entre os serviços de FaaS (Function as a Service) da AWS e da GCP.

Dos pontos práticos que são relevantes conhecer:

01. Infraestrutura:

Ambas as plataformas, AWS e GCP, adotam um modelo de divisão de infraestrutura em regiões e zonas, seguindo os padrões estabelecidos pelo NIST (Instituto Nacional de Padrões e Tecnologia).

No contexto da computação em nuvem, uma região refere-se a uma área geográfica específica onde os recursos podem ser implantados. Dentro de cada região, existem as Zonas de Disponibilidade, que representam datacenters físicos distintos, isolados uns dos outros, onde os serviços são executados.

É importante observar que, em ambas as plataformas, alguns serviços podem não estar disponíveis em todas as regiões. No entanto, para serviços de funções serverless, eles estão presentes na região brasileira, conhecida como sa-east-1 na AWS e southamerica-east1 na GCP.

Um ponto que se destaca nas diferenças é a nomenclatura das Zonas de Disponibilidade, onde profissionais da GCP se referem a elas apenas como "zona", enquanto os profissionais da AWS as chamam de "AZ".

02. Nomes diferentes, serviços similares

Mesmo falando aqui basicamente de Funções serverless no dia a dia do desenvolvimento precisamos saber bem mais do arcabouço envolvido para desenvolvimento, integração, implantação e monitoramento dos nossos projetos, dessa forma abaixo apresento um de-para para os serviços mais conhecidos

Type	AWS	GCP
Serverless Functions	Lambda Function	Cloud Functions
API endpoints	API Gateway	Cloud Endpoints
Content Delivery Network (CDN)	Cloudfront	Cloud CDN
Kubernetes (K8s) Service	EKS	GKE
Messaging Queue Service	SQS e SNS	Cloud Pub/Sub
Operation, Log and Monitoring	CloudWatch	Cloud Logging e Cloud Monitoring

O Google oferece uma lista mais completa que inclui outros serviços e fica disponível nessa comparação.

03. Custos

A estrutura de preços é quase idêntica entre a AWS e a GCP. Os principais fatores que impactam no custo em ambos os serviços são o número de solicitações e o tempo de execução. É importante colocar aqui que outras tarifas podem ser cobradas por serviços paralelos, como transferência de dados ou custos de armazenamento, mas essas dependem do contexto do projeto.

Um detalhe importante que pode causar diferença entre os valores é como o tempo de computação é calculado em cada provedor. Na AWS, o tempo de execução é cobrado em incrementos de 1 milissegundo. Isso significa que cada milissegundo adicional é cobrado separadamente.

No Google Cloud Functions, o tempo de execução é arredondado para cima para cada intervalo de 100 milissegundos após o primeiro 100 milissegundos. Isso significa que, se a execução da função durar menos de 100 milissegundos, você não será cobrado por esse tempo. No entanto, se durar mais de 100 milissegundos, você será cobrado pelo tempo total arredondado para o próximo incremento de 100 milissegundos. Exemplo: uma função executada por 260 ms seria faturada como 300 ms.

Além disso, outro detalhe imporante sobre preço é que na GCP, o tempo de computação é medido considerando a alocação de memória (GB por segundo) e de poder de processamento da CPU (GHz por segundo).

Podemos analisar também os níveis gratuitos oferecidos pelos provedores de nuvem para seus serviços de funções serverless.

Provedor	Invocações Gratuitas por Mês	Tempo de Computação Gratuito (GB-segundos) por Mês	Tempo de CPU Gratuito (GHz-segundos) por Mês
Google Cloud Functions	2 milhões	400.000	200.000
AWS Lambda	1 milhão	400.000	-

Como fazer cálculos de processamento:

500.000 invocações / 40 ms duração / 128 MB

40 ms = 0,04 seg
128 MB = 0,125 GB

Tempo de uso: 500.000 invocações/mês * 0,04 seg/invocação = 20.000 seg/mês

Memória Provisionada: 20.000 seg/mês * 0,125 GB = 2,5 GB-seg/mês

3.000.000 invocações / 230 ms duração / 1024 MB

230 ms = 0,23 seg
1024 MB = 1 GB

Tempo de uso: 3.000.000 invocações/mês * 0.23 seg/invocação = 690.000 seg/mês

Memória Provisionada: 690.000 seg/mês * 1 GB = 690 GB-seg/mês

Calculadoras online

AWS: https://calculator.aws/#/createCalculator/Lambda
GCP: https://cloud.google.com/products/calculator/
FAAS Calc: https://faascalc.com/

Ps.: Eu não confio muito na calculadora do Google 🤷

04. Linguagens Suportadas:

Ambos os serviços foram certeiros em incluir uma variedade boa de linguagens suportadas. Enquanto o suporte do Cloud Functions do Google inclui Node.js, Python, Go e Java, Ruby, PHP e .NET, as Lambda Function da AWS tem uma lista um pouco mais extensa, facilitando o uso de TypeScript (usando ambiente de Node) e incluindo Rust e PowerShell.

Além disso, como trabalho principalmente com Python, pude notar o compromisso dos provedores em manter as tecnologias atualizadas. Na GCP, ainda é possível criar uma Lambda com Python 3.7, mas na AWS isso não é mais permitido. Vale mencionar também que tanto na AWS quanto na GCP, o Python 3.8 será descontinuado ainda este ano (2024). Uma tristeza para as empresas que adeptas da cultura do "Se tá funcionando não mexe".

05. Tempo limite de Execução:

O tempo de execução é um fator crucial a ser considerado ao escolher entre o Google Cloud Functions e a AWS Lambda. No Google Cloud Functions, o tempo limite varia de acordo com a geração da função: na primeira geração, o limite é de 9 minutos (540 segundos), enquanto na segunda geração, pode chegar a 60 minutos (3.600 segundos) para funções HTTP e permanece em 9 minutos (540 segundos) para funções orientadas a eventos.

Por outro lado, na AWS Lambda, o tempo limite padrão é de apenas 3 segundos, mas pode ser ajustado para até 15 minutos (900 segundos) conforme necessário. Esses limites podem afetar diretamente o desempenho e a funcionalidade de suas funções serverless, especialmente em casos de requisições externas síncronas.

Abaixo, uma tabela resume os limites de tempo de execução em ambas as plataformas:

Plataforma	Geração	Tempo Limite
Google Cloud Functions	1ª	540s
Google Cloud Functions	2ª (HTTP)	3.600s
Google Cloud Functions	2ª (Evento)	540s
AWS Lambda	-	3s (padrão) a 900s (máximo)

06. Integração com o Ecossistema:

A integração com o ecossistema foi um dos fatores que mais me surpreenderam durante a transição. A AWS Lambda oferece uma integração bem flexível com todo o ecossistema AWS, o que tem sido extremamente útil na arquitetura atual em que trabalho. Desde integrações perfeitas com o Amazon S3, Amazon DynamoDB e Amazon API Gateway.

No entanto, um ponto negativo é que isso pode gerar um alto nível de acoplamento. Embora ambos os provedores ofereçam suporte à integração HTTP, a AWS exige o provisionamento e configuração de um recurso separado, o API Gateway, que também é cobrado separadamente.

07. Desempenho e Could Start

O desempenho e o tempo de inicialização são aspectos críticos a serem considerados ao avaliar as opções de computação serverless entre o Google Cloud Functions e a AWS Lambda. Embora as métricas de tempo de inicialização não sejam divulgadas publicamente pelos provedores, uma análise superficial revela algumas diferenças significativas.

Pesquisas indicam que as Cloud Functions têm um tempo de inicialização nativamente mais rápido em comparação com as Lambdas da AWS. No entanto, é desafiador elaborar um comparativo preciso devido à falta de informações detalhadas.

Na AWS, por exemplo, uma Lambda em Python em estado inativo pode iniciar em média em torno de 200ms, mas vale ressaltar que esse valor não considera o uso do recurso de provisioned concurrency, uma funcionalidade que elimina o Cold Start ao manter instâncias prontas para responder a solicitações.

08. Ferramentas de Linha de Comando (CLI)

As ferramentas de linha de comando desempenham um papel importante no gerenciamento das plataformas de nuvem. Ambos os provedores, AWS e GCP, oferecem suas próprias ferramentas CLI para facilitar a automação, implantação e gerenciamento de recursos na nuvem.

8.1 GCP

gcloud tool: Com o gcloud, os usuários podem criar, modificar e gerenciar recursos como instâncias de máquinas virtuais, bancos de dados e serviços serverless.
gsutil tool: Com o gsutil é possível fazer a transferência de arquivos, a configuração de políticas de acesso e a execução de várias outras operações em buckets do Cloud Storage. Pouco utilizado no contexto do GCP Cloud Functions.
bq tool: O bq é uma ferramenta de linha de comando destinada principalmente para o BigQuery, o serviço de análise de dados do GCP. Com o bq, os usuários podem executar consultas SQL, carregar e exportar dados, além de gerenciar conjuntos de dados e tabelas. Essa ferramenta também é pouco utilizada no contexto do GCP Cloud Functions.

8.2 AWS

AWS tool: A Interface de Linha de Comando da AWS é a principal ferramenta para interagir com os serviços da AWS através da linha de comando. Com a AWS CLI, os usuários podem gerenciar instâncias EC2, buckets S3, funções Lambda e uma variedade de outros serviços AWS.
SAM tool: O AWS SAM (Serverless Application Model) é uma extensão da AWS CLI que facilita a criação, teste e implantação de aplicativos serverless na AWS. Com o AWS SAM, os desenvolvedores podem definir recursos serverless usando uma sintaxe simplificada e implantá-los com facilidade. O AWS SAM se sobrepõe ao AWS CLI para nos fornecer algumas ferramentas extras úteis.

09. Conclusão:

Como dizem, o diabo está nos detalhes. A escolha entre a AWS Lambda e o Google Cloud Functions vai além da comparação de recursos técnicos. Embora a AWS Lambda seja considerada mais madura, o Google Cloud Functions também é uma opção robusta, com sua própria gama de vantagens e funcionalidades. A decisão entre os dois provedores deve ser baseada não apenas nas características técnicas, mas também nos custos associados, na experiência do desenvolvedor e no contexto específico do projeto.

Minha migração da GCP para a AWS foi surpreendentemente tranquila, em contraste com os relatos de dificuldades que ouvi de profissionais que seguiram o caminho inverso. Isso pode ser atribuído às integrações e facilidades oferecidas pela AWS, destacando a importância de escolher uma plataforma que atenda às necessidades específicas do projeto e da equipe de desenvolvimento.

No geral, essa migração não apenas ampliou meu conhecimento sobre cloud computing e cloud native, mas também destacou a importância da independência de plataforma. Desenvolver e implantar software de forma independente dos provedores de nuvem subjacentes não só facilita a portabilidade entre diferentes serviços, mas também maximiza a utilização dos recursos, resultando em economia de custos significativa.

Differences between GCP Cloud Functions and AWS Lambda Functions - A Comparative Guide from Migration

Miguel Müller — Tue, 14 May 2024 15:08:47 +0000

A Portuguese version is available here

It's been a few months since I migrated from a GCP (Google Cloud Platform) environment to an AWS (Amazon Web Services) environment. At first, I thought I would find it strange and have difficulty adapting because I've heard reports of professionals going the other way complaining, but to my surprise, that didn't happen.

And in this transition, I decided to write and share in a didactic way the subtle differences I encountered. Instead of presenting a direct confrontation, I opted for a comparison guide to help other professionals who may be considering a similar transition.

The cloud standards we use today are primarily an effort of NIST (National Institute of Standards and Technology). These standards have guided the development and implementation of cloud technologies, ensuring a consistent and secure approach across all cloud computing providers. Because of these standards, there are many similarities between AWS and GCP's FaaS (Function as a Service) offerings.

From the practical points that are relevant to know:

01. Infrastructure:

Both AWS and GCP platforms adopt a model of dividing infrastructure into regions and zones, following the standards established by NIST (National Institute of Standards and Technology).

In the context of cloud computing, a region refers to a specific geographical area where resources can be deployed. Within each region, there are Availability Zones, which represent distinct physical data centers, isolated from each other, where services are executed.

It is important to note that, on both platforms, some services may not be available in all regions. However, for serverless function services, they are present in the Brazilian region, known as sa-east-1 in AWS and southamerica-east1 in GCP.

One point that stands out in the differences is the nomenclature of Availability Zones, where GCP professionals refer to them simply as "zone," while AWS professionals call them "AZ."

02. Different Names, Similar Services

Even though I'm basically talking about serverless functions in everyday development, we need to know much more about the framework involved for development, integration, deployment, and monitoring of our projects. Thus, below is a mapping for the most well-known services:

Type	AWS	GCP
Serverless Functions	Lambda Function	Cloud Functions
API endpoints	API Gateway	Cloud Endpoints
Content Delivery Network (CDN)	Cloudfront	Cloud CDN
Kubernetes (K8s) Service	EKS	GKE
Messaging Queue Service	SQS and SNS	Cloud Pub/Sub
Operation, Log, and Monitoring	CloudWatch	Cloud Logging and Cloud Monitoring

Google offers a more complete list that includes other services and is available in this comparison.

03. Costs

The pricing structure is almost identical between AWS and GCP. The main factors that impact the cost on both services are the number of requests and the execution time. It is important to note here that other fees may be charged for parallel services, such as data transfer or storage costs, but these depend on the project context.

An important detail that can cause a difference between the values is how the computing time is calculated on each provider. In AWS, the execution time is charged in increments of 1 millisecond. This means that each additional millisecond is charged separately.

In Google Cloud Functions, the execution time is rounded up for each interval of 100 milliseconds after the first 100 milliseconds. This means that if the function execution lasts less than 100 milliseconds, you will not be charged for that time. However, if it lasts more than 100 milliseconds, you will be charged for the total time rounded up to the next 100 milliseconds increment. For example, a function executed for 260 ms would be billed as 300 ms.

Furthermore, another important detail about price is that in GCP, computing time is measured considering memory allocation (GB per second) and CPU processing power (GHz per second).

We can also analyze the free tiers offered by cloud providers for their serverless function services.

Provider	Free Invocations per Month	Free Compute Time (GB-seconds) per Month	Free CPU Time (GHz-seconds) per Month
Google Cloud Functions	2 million	400,000	200,000
AWS Lambda	1 million	400,000	-

How to calculate processing:

500,000 invocations / 40 ms duration / 128 MB

40 ms = 0.04 sec
128 MB = 0.125 GB

Usage Time: 500,000 invocations/month * 0.04 sec/invocation = 20,000 sec/month

Provisioned Memory: 20,000 sec/month * 0.125 GB = 2.5 GB-sec/month

3,000,000 invocations / 230 ms duration / 1024 MB

230 ms = 0.23 sec
1024 MB = 1 GB

Usage Time: 3,000,000 invocations/month * 0.23 sec/invocation = 690,000 sec/month

Provisioned Memory: 690,000 sec/month * 1 GB = 690 GB-sec/month

Online Calculators

AWS: https://calculator.aws/#/createCalculator/Lambda
GCP: https://cloud.google.com/products/calculator/
FAAS Calc: https://faascalc.com/

P.S.: I don't trust Google's calculator much 🤷

04. Supported Languages:

Both services were spot on in including a good variety of supported languages. While Google's Cloud Functions support includes Node.js, Python, Go, and Java, Ruby, PHP, and .NET, AWS Lambda's list is slightly more extensive, facilitating the use of TypeScript (using Node environment) and including Rust and PowerShell.

Additionally, as I mainly work with Python, I noticed the providers' commitment to keeping technologies up to date. In GCP, it is still possible to create a Lambda with Python 3.7, but in AWS, this is no longer allowed. It's worth mentioning that both in AWS and in GCP, Python 3.8 will be discontinued later this year (2024). A sadness for companies adhering to the culture of "If it ain't broke, don't fix it."

05. Execution Timeout:

The execution time is a crucial factor to consider when choosing between Google Cloud Functions and AWS Lambda. In Google Cloud Functions, the timeout varies according to the function's generation: in the first generation, the limit is 9 minutes (540 seconds), while in the second generation, it can reach 60 minutes (3,600 seconds) for HTTP functions and remains at 9 minutes (540 seconds) for event-driven functions.

On the other hand, in AWS Lambda, the default timeout is only 3 seconds but can be adjusted to up to 15 minutes (900 seconds) as needed. These limits can directly affect the performance and functionality of your serverless functions, especially in cases of synchronous external requests.

Below is a table summarizing the execution time limits on both platforms:

Platform	Generation	Timeout
Google Cloud Functions	1st	540s
Google Cloud Functions	2nd (HTTP)	3,600s
Google Cloud Functions	2nd (Event)	540s
AWS Lambda	-	3s (default) to 900s (max)

06. Integration with Ecosystem:

Integration with the ecosystem was one of the factors that surprised me the most during the transition. AWS Lambda offers a very flexible integration with the entire AWS ecosystem, which has been extremely useful in the current architecture I work in. From seamless integrations with Amazon S3, Amazon DynamoDB, and Amazon API Gateway.

However, a downside is that this can lead to a high level of coupling. Although both providers support HTTP integration, AWS requires provisioning and configuring a separate resource, API Gateway, which is also charged separately.

07. Performance and Cold Start

Performance and startup time are critical aspects to consider when evaluating serverless computing options between Google Cloud Functions and AWS Lambda. Although startup time metrics are not publicly disclosed by providers, a superficial analysis reveals some significant differences.

Research indicates that Cloud Functions have a natively faster startup time compared to AWS Lambdas. However, it is challenging to draw an accurate comparison due to the lack of detailed information.

In AWS, for example, an inactive Python Lambda can start on average around 200ms, but it is worth noting that this value does not consider the use of the provisioned concurrency feature, a functionality that eliminates Cold Start by keeping instances ready to respond to requests.

08. Command-Line Interface (CLI) Tools

Command-line tools play an important role in managing cloud platforms. Both providers, AWS and GCP, offer their own CLI tools to facilitate automation, deployment, and resource management in the cloud.

8.1 GCP

gcloud tool: With gcloud, users can create, modify, and manage resources such as virtual machine instances, databases, and serverless services.
gsutil tool: With gsutil, file transfer, access policy configuration, and various other operations can be performed on Cloud Storage buckets. It is less utilized in the context of GCP Cloud Functions.
bq tool: The bq is a command-line tool primarily intended for BigQuery, GCP's data analysis service. With bq, users can run SQL queries, load and export data, and manage datasets and tables. This tool is also less utilized in the context of GCP Cloud Functions.

8.2 AWS

AWS tool: The AWS Command Line Interface is the primary tool for interacting with AWS services through the command line. With the AWS CLI, users can manage EC2 instances, S3 buckets, Lambda functions, and a variety of other AWS services.
SAM tool: The AWS SAM (Serverless Application Model) is an extension of the AWS CLI that makes it easy to create, test, and deploy serverless applications on AWS. With AWS SAM, developers can define serverless resources using a simplified syntax and deploy them with ease. AWS SAM overlays AWS CLI to provide us with some useful extra tools.

09. Conclusion:

As they say, the devil is in the details. The choice between AWS Lambda and Google Cloud Functions goes beyond comparing technical features. Although AWS Lambda is considered more mature, Google Cloud Functions is also a robust option, with its own range of advantages and features. The decision between the two providers should be based not only on technical characteristics but also on associated costs, developer experience, and the specific context of the project.

My migration from GCP to AWS was surprisingly smooth, contrasting with reports of difficulties I heard from professionals who went the other way. This can be attributed to the integrations and facilities offered by AWS, highlighting the importance of choosing a platform that meets the specific needs of the project and development team.

Overall, this migration not only expanded my knowledge of cloud computing and cloud-native but also highlighted the importance of platform independence. Developing and deploying software independently of underlying cloud providers not only facilitates portability between different services but also maximizes resource utilization, resulting in significant cost savings.

Comparing Text Similarity Measurement Methods: Sentence Transformers vs. Fuzzy

Miguel Müller — Wed, 11 Oct 2023 15:01:15 +0000

Introduction:

The measurement of text similarity is a fundamental task in the realm of natural language processing and information retrieval. It underpins a wide array of applications, including information retrieval, text categorization, recommendation systems, and more. Accurately assessing the similarity between textual data can have a profound impact on the performance of these systems. In this article, we delve into text similarity measurement and explore two distinct methodologies for achieving this goal: Sentence Transformers and Fuzzy.

Text similarity measurement involves determining how similar two pieces of text are, often expressed as a numerical score. This score can be used for a variety of purposes, such as identifying similar documents in a large corpus, matching user queries to relevant content, and even detecting plagiarism. The methods we will discuss here cater to different aspects of text similarity, from semantic equivalence to character-level matching.

In real-world applications, the choice between Sentence Transformers and Fuzzy is not a matter of one-size-fits-all, but rather a strategic decision guided by the specific domain and requirements of the task at hand. Sentence Transformers shine when the context and meaning of text play a crucial role, such as in natural language understanding and document retrieval. On the other hand, Fuzzy excels when the focus is on character-level similarities, making it a valuable tool for tasks like deduplication of records, correcting typos, and matching strings with minor variations. Understanding these nuances empowers practitioners to select the most suitable method for their particular use cases, unlocking the potential of text similarity measurement in diverse domains.

Practical Examples:

Comparing Semantic Similarity with Sentence Transformers: Let's consider an example involving the use of Sentence Transformers to measure semantic similarity. Imagine we have two sentences: "The vast ocean is beautiful" and "The immense sea is stunning". We can leverage Sentence Transformers to compute embeddings for these sentences and measure their semantic similarity. The result, obtained using the MiniLM model, reflects their shared semantic meaning and might yield a high similarity score.

"The vast ocean is beautiful" and "The immense sea is stunning"

Sentence Transformers: 0.8006699085235596
Fuzzy: 0.52

Comparing Textual Variations with Fuzzy: Next, let's explore the practical application of Fuzzy. Suppose we have two strings with minor variations: "color" and "colour". Fuzzy can be employed to calculate the similarity between these strings by focusing on character-level differences. In this case, Fuzzy's comparison, based on the Levenshtein distance, would highlight that "color" and "colour" are closely related due to their minor spelling variation.

"color" and "colour"

Sentence Transformers: 0.973908543586731
Fuzzy: 0.91

To bring these concepts to life, let's look at a code example that demonstrates the practical use of both methods to measure text similarity.

This code, along with additional resources and updates, is available on our GitHub repository at https://github.com/miguelsmuller/comparing-text-similarity, where you can further explore the implementation and applications of these text similarity measurement methods.



import sys

from sentence_transformers import SentenceTransformer, util
from thefuzz import fuzz
from torch import Tensor


def compare_with_model(text1: str, text2: str):
    model = SentenceTransformer("sentence-transformers/all-MiniLM-L6-v2")
    embedding_1: Tensor = model.encode(text1, convert_to_tensor=True)
    embedding_2: Tensor = model.encode(text2, convert_to_tensor=True)

    similarity = util.pytorch_cos_sim(embedding_1, embedding_2)[0][0].item()

    return similarity


def compare_with_fuzz(text1: str, text2: str):
    similarity = fuzz.ratio(text1, text2) / 100

    return similarity


if len(sys.argv) != 3:
    print("Please provide exactly two sentences as arguments.")
    sys.exit(1)

sentence1 = sys.argv[1]
sentence2 = sys.argv[2]

model_similarity = compare_with_model(sentence1, sentence2)
fuzz_similarity = compare_with_fuzz(sentence1, sentence2)

print("Usando Modelo:", model_similarity)
print("Usando Fuzz:", fuzz_similarity)

Method 1: Sentence Transformers

Sentence Transformers represent a modern approach to text similarity measurement by leveraging deep learning and contextual embeddings. These models are pretrained on vast corpora of text data, allowing them to capture the intricate semantic relationships between words and phrases. Sentence Transformers generate high-dimensional vector representations of text, where similar phrases are mapped to nearby points in the vector space. The key idea is to encode not just individual words but also the context and meaning of an entire sentence, making them ideal for tasks like semantic search, document retrieval, and paraphrase identification.

At the core of Sentence Transformers is the ability to transform text into dense, semantically meaningful embeddings. The model takes a sentence as input and, through a series of neural network layers, converts it into a fixed-size vector. This vector encodes the sentence's meaning in a high-dimensional space, facilitating comparisons between different sentences.

To learn more about embeddings, you can visit Hariom Gautam's article, from where i also sourced the image below.

Sentence Transformers find applications in various domains, including natural language understanding, information retrieval, and question answering systems. They excel in scenarios where understanding the underlying semantics of text is crucial, such as matching user queries with relevant documents or identifying similar sentences in large text corpora.

To illustrate the method's capability, consider the task of measuring the similarity between two sentences: "The quick brown fox jumps over the lazy dog" and "A fast brown fox leaps over a dozing dog". Sentence Transformers can generate embeddings for these sentences and measure their similarity based on semantic content. In this case, the two sentences are likely to receive a high similarity score, reflecting their shared meaning despite slight wording differences.

"The quick brown fox jumps over the lazy dog" and "A fast brown fox leaps over a dozing dog"

Sentence Transformers: 0.8295611143112183
Fuzzy: 0.63

Sentence Transformers offer the advantage of capturing nuanced semantic information, but they may require substantial computational resources. Their performance shines in tasks requiring an understanding of context and semantics, but their effectiveness can diminish when dealing with character-level similarities or extremely short text snippets.

Method 2: Fuzzy

Fuzzy, with its whimsical name, offers a unique approach to text similarity measurement. Instead of delving into the semantic intricacies of text, Fuzzy focuses on character-level similarities. It employs algorithms like the Levenshtein distance to calculate the difference between two strings in terms of the number of insertions, deletions, and substitutions required to transform one into the other. This method excels when the task involves comparing strings with slight variations, making it a valuable tool for various applications, including record deduplication, spell checking, and string matching.

At the core of Fuzzy is the computation of a similarity ratio, often expressed as a percentage, that quantifies how similar two strings are. This ratio is calculated by comparing the characters in the two strings and determining the number of operations needed to make them identical.

Fuzzy finds extensive utility in domains where small differences in text are critical, such as database management and data cleansing. It is instrumental in detecting and resolving near-duplicate records, ensuring data quality by correcting typos, and matching strings with minor deviations.

Consider a scenario in which a database contains multiple records with slight variations in names, such as "John Smith" and "Jon Smithe". Fuzzy can be employed to calculate a similarity ratio, allowing for the identification and removal of these near-duplicate entries. The result is a cleaner and more accurate database.

"John Smith" and "Jon Smithe"

Sentence Transformers: 0.7414223551750183
Fuzzy: 0.9

Fuzzy is highly efficient at handling character-level similarities, particularly in scenarios where minor text variations matter. However, it is less suited for capturing semantic nuances in text and is best reserved for tasks that prioritize character-based comparisons.

Comparison of Methods:

When to Use Sentence Transformers: Sentence Transformers excel in scenarios where the context and meaning of text are paramount. If your task involves understanding the semantic relationships between words, phrases, or sentences, Sentence Transformers are a valuable choice. They are particularly effective for applications like natural language understanding, document retrieval, and paraphrase identification. When your goal is to match user queries with relevant content or find semantically similar sentences, Sentence Transformers are a robust option.

When to Use Fuzzy: On the other hand, Fuzzy comes into its own when character-level similarities are the primary focus. If your text data contains variations due to typos, minor differences, or slight misspellings, Fuzzy is a dependable choice. It excels in tasks like deduplication of records, spell checking, and string matching, where capturing subtle character-level distinctions is crucial.

The choice between these methods boils down to the nature of your data and the objectives of your task. If your goal is to measure the semantic relatedness of text and capture its meaning, Sentence Transformers are the go-to option. On the other hand, if your primary concern is to deal with textual variations and character-level differences, Fuzzy is the method of choice. Understanding this fundamental distinction empowers practitioners to make informed decisions and choose the method that aligns with their specific needs.

Selecting the right method is crucial for the success of your text similarity measurement task. Consider the nature of your text data, the specific requirements of your application, and the level of similarity you aim to capture. Combining these insights will guide you to choose either Sentence Transformers or Fuzzy, ensuring that your text similarity measurements align with your objectives.

Conclusion:

In this article, we explored two distinct approaches to measuring text similarity: Sentence Transformers and Fuzzy. Each method has its place and utility depending on the context and task requirements. By understanding the differences between them, professionals can make informed decisions about which method to use in their projects. The choice between semantic similarity and character-based similarity depends on the type of text being compared and the specific project needs.

References:

Acknowledgments:

Cover Photo by Jason Dent at Unsplash

The time of Axios and Node-Fetch is coming to an end?

Miguel Müller — Fri, 11 Feb 2022 12:53:46 +0000

It's so beautiful how technologies evolve isn't it?

From yesterday to today (10/02/22 - 11/02-2022) we had an impactful change in something that is in our daily lives - Node.JS.

It was officially started in this v17.5.0 of NodeJS the implementation, at the moment experimental, of the native API for Fetch to work with Request and Response .

It was always natural to turn to external libraries to consume external web APIs. But now it looks like all that is about to change.

Unlike what happens in most current browsers where there is, natively, the Fetch API for requests, NodeJS did not have its own feature for this.

As I said, this feature is experimental, so it still needs to be used with the flag: --experimental-fetch.

I haven't had time to test the functionality yet but just reading it made me salivate. If you try it out there, let us know how the result was and if it met your expectations.

If you want to know more about the changelog of this new version of NodeJS, you can access this link here