Forem: Bruno Noriller

Slop is the new normal

Bruno Noriller — Wed, 30 Apr 2025 00:08:03 +0000

AI has its uses. In some places, it can be great, but in others, it will be lethal and have unintended and not thought-out consequences.

Slop in Time

We are living the what? Second year in the 6 months to be replaced by AI?

Yes, it’s certainly better. Today, you can run on a cheap GPU a model that will outperform the first iterations of what was being used with GitHub Copilot.

I remember being excited about using it and how good the completions were. Did I use it to write full functions and whatnot? NO!

I used it as a “super autocomplete” to finish writing the line of code I was already writing.

Nowadays, it can complete multiple lines, even full files.

Slop in Love

While I did remember it completing lines, and today that and more. Nowadays, I see and get more annoyed at how much it fails to do that.

Why does it keep failing me so much? Wasn’t it supposed to be a better model?

At first, it will look perfect, but with time, more cracks will show and love will fade.

Diversity of Slop

This applies to code, images, songs, and any kind of slop there is. Yes, it seems good, at a distance, without caring too much about the details.

An image might get a pass. How long do people stare at those?

A song might get a pass. Just a quirk, some might say.

But code is a prick of a thing. It either will run or not. And even when it runs, it might not do what you thought it would.

And the more you see, the worse it all gets. Generated images are only as good as you look at a distance, and generated music always seems to be the same music, and code is a disaster away because no one is thinking about the details.

Slop is in the Details

I like this masterclass Uncle Bob gave: https://www.youtube.com/playlist?list=PLo61EKto8ZPHUOld83z0pwpzdlioliu6j

And while I would recommend watching everything, I want to point out first to here where Uncle Bob starts telling that “when you put your foot in the brake of a car, do you believe that there’s a cable going from the pedal to the brake disks? Or do you realize there are ‘if’ statements in the way? Who wrote those ‘if’ statements deciding whether or not to stop the car when you push on the brake?” and then jump a little forward to here where he says that society will wake up to the fact that “we” (programmers) rule everything with code when “a poor programmer does some dumb thing and kills 10_000 people in a shot, then the politicians of the world will rise in indignation and point their finger at us”.

These classes have been on YouTube for over 5 years. I don’t know when it took place, but I’m certain that even before, he probably gave the same speech other times.

In this case, it was about the craftsmanship and even taking jabs at code being written by “some 22-year-old at 3 in the morning”. This was then.

Today, it would be a 16-year-old talking to a possibly hallucinating AI that will throw everything in a word mixer and slop a whole new file with code in every interaction.

Sloping around

Remember when any grocery store wanted a mobile app? (Or maybe you still suffer because you need an app for some dumb thing that didn’t need to be an app?)

Today it’s AI, and be certain that there will be a lot of that being shoved and slopped everywhere.

But more than that… do you trust the custom shopping cart asking for your credit information was “hand-made” or was it “vibe coded”? What about the car you’re driving? Or the one heading your way? Your bank? How about some software that keeps all your memories?

Some people are announcing that the app was “vibe coded” as a mark of pride. I see it as a giant red flag, and I’m sure that many don’t announce it.

I might be an alarmist, but then again, I check both sides of a one-way street twice before crossing.

Remoto é bom e pode melhorar!

Bruno Noriller — Mon, 03 Mar 2025 15:25:38 +0000

Tivemos toda a história de trabalhar em um escritório… mas apenas alguns anos trabalhando remotamente e, no pior dos casos… foi apenas um pouco “pior” (em termos de produtividade) e isso sem contar que tudo começou em uma maldita pandemia!

Trabalho remoto não é para todos os tipos de função e nem para todas as pessoas (algumas já experimentaram trabalhar remotamente em outros contextos e não gostaram), existem razões válidas contra ele e momentos ou lugares onde não é a melhor opção.

No entanto, há muitas razões que não têm nada a ver com o que se possa dizer sobre o trabalho remoto ser bom ou ruim, e que ficarão fora do escopo deste artigo porque por algum motivo, trabalhar remotamente se tornou algo controverso por razões alheias à própria natureza do trabalho remoto.

De qualquer forma, este artigo explora a minha jornada pelo trabalho remoto — desde a transição inicial durante a pandemia, passando pelos desafios e benefícios que experimentei como programador com formação em administração, até como torná-lo ainda melhor.

Vamos Lembrar de Algumas Coisas

Se você está lendo isto próximo da data de publicação, então provavelmente compartilhamos que passamos pela pandemia. E, para muitos, inclusive eu, foi a primeira vez trabalhando remotamente.

Alguns gostaram e não querem voltar a um escritório, outros odiaram e nunca mais querem tentar novamente.

Mas aqui vai o lembrete: não foi tudo ruim ou tudo bom.

Temos que lembrar que foi uma pandemia, mesmo que algumas pessoas tentassem fingir o contrário. As pessoas foram separadas, ficaram estressadas, solitárias, preocupadas… havia muitas emoções em jogo e os lockdowns não ajudaram.

No entanto, hoje, sem uma pandemia, as coisas são um pouco diferentes. Você pode trabalhar de casa e depois sair sem preocupações, pode trabalhar e depois encontrar os amigos e fazer outra coisa.

O trabalho remoto no cenário atual é diferente do que era naquela época. E se as piores estimativas indicavam que o trabalho remoto era apenas um pouco inferior… com tudo o que estava acontecendo, o que dizer hoje, quando está muito melhor?

Por que Estou Escrevendo Isto?

Há muito tempo… ou o que parece ser há muito tempo… eu simplesmente “ia trabalhar” porque isso era considerado “normal” e, então, veio a pandemia e eu passei a trabalhar totalmente remoto.

A pandemia “acabou” ou pelo menos o perigo dela diminuiu graças às vacinas, eu me mudei de cidade para voltar a trabalhar em um escritório, agora longe de casa.

Todos nós tivemos que voltar aos escritórios, “esquecendo” o trabalho remoto, apenas para que essa brilhante ideia chamada “trabalho remoto” fosse introduzida, passo a passo, mas sem ser totalmente remoto.

Quando me mudei, preferi conseguir um lugar perto do escritório, mas ele não tinha espaço suficiente para trabalhar remotamente. Então, continuei indo ao escritório, aproveitando para caminhar um pouco todos os dias.

Há alguns meses, um membro da família teve problemas de saúde e precisaria passar por uma cirurgia, e eu voltei para casa para ajudar.

Finalmente, de volta ao escritório, novamente.

A Pandemia, de Novo?

Ter que voltar ao trabalho remoto por motivos médicos… meio que parecia a pandemia de novo.

Isso me levou a revisitar uma época em que o trabalho remoto era novidade e algo fresco para quase todos.

Desta vez, entretanto, algumas coisas estavam diferentes, mais como deveriam e poderiam ter sido se tivéssemos “alcançado” o trabalho remoto por outros meios.

“Totalmente Remoto”, Mas de um Jeito Estranho

Devido a regras internas, eu não podia realmente trabalhar totalmente remoto durante o mês inteiro. No entanto, eu podia preencher os dias em que deveria estar no escritório queimando meus dias de folga e férias.

Antes de tudo, tanto meu lado de programador quanto o lado formado em administração concordam que ter que parar várias vezes em um curto período é simplesmente ruim.

É impossível ter qualquer tipo de fluxo ou concluir um projeto de longo prazo de forma adequada assim. Esse foi um efeito infeliz das regras que tive que seguir, e foi simplesmente ruim.

Agora podemos chegar a um ponto em que, enquanto internamente os lados administração e de programação concordam, outras pessoas podem discordar.

A primeira pausa que tive que fazer, bem antes de iniciar um novo projeto… essa foi uma boa pausa.

Foi uma forma de “limpar o cache” do que eu estava fazendo antes e recomeçar em preparação para o novo projeto.

É um problema o fato de o “ágil” nos fazer passar por uma maratona de sprint após sprint. Simplesmente errado, mesmo quando se tenta considerar do que se tratava o manifesto original ou até mesmo o que o Scrum e outras “metodologias ágeis” pregam.

Você faz um sprint, revisa… e isso significa que uma pausa precisa ocorrer. Só então você pode iniciar outro sprint.

Esta parte nem é sobre o remoto, mas acredito que esses intervalos podem melhorar a qualidade em qualquer lugar!

A Parte Ruim da Pandemia, de Novo

Lembra quando eu disse que nem tudo era ruim?

Bem, algumas coisas foram, e eu passei por isso novamente. Você deve se lembrar de estar estressado, cansado, com medo… tudo isso aconteceu porque os motivos médicos envolviam cirurgias e, depois, complicações para alguém com mais de 80 anos.

Houve dias em que eu me jovaga no trabalho, enquanto em outros, o trabalho era uma tarefa árdua. Deixarei para a sua imaginação, mas considerando que todos passamos por uma pandemia… foi mais ou menos assim.

Algo que Não Tivemos na Pandemia

Especialmente quando havia lockdowns, e talvez minha família tenha exagerado um pouco nisso, e depois… Mas, neste caso, eu podia simplesmente sair, encontrar amigos, ir ao cinema, sair para correr… Eu não estava em lockdown e podia liberar o estresse de várias maneiras.

Este e o ponto anterior não se aplicam apenas ao remoto, mas, já que o escritório é um “lugar de trabalho” e nossa cultura em geral, tendemos a guardar certas coisas para nós mesmos.

Provavelmente é óbvio apontar, mas algumas demonstrações de emoção provavelmente gerariam muitos problemas no escritório.

A Estranheza do Híbrido

Quando surgiu essa nova modalidade “híbrida”, às vezes eu era o único da equipe no escritório enquanto todos trabalhavam remotamente.

Embora sempre houvesse mais equipes no mesmo andar, houve dias em que eu olhava para a esquerda e para a direita e não via ninguém por perto.

Então, finalmente, vi o oposto: eu era o único remoto enquanto os outros estavam no escritório.

Quando eu ia para o escritório “apenas porque sim”, muitos dos meus dias eram resumidos na expressão “entrou quieto e saiu calado”. Por muito tempo, eu estava sempre lá, mas além de alguns acenos e “bão”/“opa”/“bom dia” e outras comunicações superficiais causadas apenas pelo cruzar no corredor ou por olhar acidentalmente nos olhos das pessoas… eu não criei conexões significativas ali.

Por conta disso, eu ficaria realmente surpreso se alguém realmente sentisse minha falta lá no escritório…

O Trabalho Remoto é Bom

Com a história contada, onde o trabalho remoto se destaca:

Presuma a Comunicação Assíncrona

Você pode simplesmente assumir que as pessoas não estão presentes e deixar uma mensagem.

Quando lerem, responderão. O mesmo para você! Se você só quer se concentrar no que está fazendo, pode simplesmente ignorar qualquer distração e focar no que está fazendo.

Agora… tente fazer isso com alguém olhando por cima do seu ombro!

Uma Rotina Melhor

Lembro de algo que o Uncle Bob disse: ao fazer estimativas, você deve considerar os dias em que tudo dá errado.

Como eu caminhava muito, frequentemente ficava irritado com o quanto a cidade é carrocentrica. Isso deveria ser incluído nas estimativas, elevando elas.

Mas, remotamente, você tem muito menos “partes móveis” que podem atrapalhar o seu dia e bagunçar sua rotina. Obviamente, há muitas mais coisas que podem dar errado, mas quanto mais pudermos eliminar, melhor. (Se você realmente quiser entregar mais rápido.)

Novas Formas de Comunicar

Algumas pessoas gostam de começar a falar para apenas no meio da conversa chegar a um ponto; algumas pessoas prosperam nisso… Contudo, colocar as ideias por escrito faz com que você pense um pouco mais.

Esse tempo extra de reflexão pode ser tudo o que algumas pessoas precisam para resolver seus próprios problemas.

Além disso, é mais fácil colar prints que podem ajudar na comunicação sobre qualquer assunto.

Mesmo em uma reunião remota, fluxogramas e ferramentas de desenho estão sempre disponíveis e podem fazer com que até a pessoa mais tímida sinta vontade de participar, mesmo sem falar.

É Bom, Mas Pode Melhorar!

A Comunicação Assíncrona Não é Obrigatória

Embora as pessoas não estarão por cima do seu ombro, ou você do delas… num ambiente de escritório, você pode sempre aparecer sem avisar e começar a conversar.

Sim, no escritório você notaria se alguém não estivesse em sua cadeira. Online, você pode simplesmente iniciar uma chamada, deixá-la tocar algumas vezes e, se não houver resposta, enviar uma mensagem e continuar de forma assíncrona.

Presumo que a maioria das ferramentas de reunião tenha, pelo menos, indicadores “verde”/“vermelho” para mostrar o status das pessoas. Se estiver verde e for algo que levará menos de 5 minutos, apenas ligue. Se estiver vermelho e não for tão urgente, então deixe uma mensagem.

Por outro lado, use e abuse desses status para informar às pessoas como elas devem interagir com você.

Além disso… eu não sou o digitador mais rápido; poucas pessoas são tão rápidas. Então, embora a comunicação assíncrona seja legal, se ambos estão digitando e esperando o outro digitar, significa que ambos estão presentes ao mesmo tempo… por que não fazer um upgrade da conexão para uma reunião rápida?

Talvez venha com ser brasileiro, mas consigo ver como pode ser cansativo ter toda vez que começar na “formalidade” do “Bom dia, como vai, tudo bem, e isso e aquilo?”, resolver o que era necessário rapidinho e depois entrar naquelas cenas de sitcom onde dois amantes ficam esperando que o outro desligue primeiro.

E quando digo rápido, eu quero dizer rápido. Sem formalidades: entre em uma chamada, comece a conversar, resolva o que precisa ser resolvido e desligue. Simples assim!

Você sempre pode ter chamadas mais longas e guardar toda a conversa fiada para quando não precisar ser rápido.

Conheça Suas Ferramentas

Dependendo do que você utiliza, terá diferentes recursos para explorar. Então, não posso ajudar muito além deste conselho:

Aprenda as ferramentas. Use elas.

Não é mais tudo caneta e papel!

Remoto ou não, sempre haverá recursos prontos para serem usados e explorados para uma melhor comunicação e trabalho em equipe.

Peça Ajuda

Quando temos problemas, mostramos isso, seja na cara ou com suspiros (ou talvez atirando coisas macias na tela ou fazendo gestos rudes para ela…).

No escritório, você pode perceber isso e tentar ajudar, mas remotamente… se você não me disser que tem um problema, não posso ajudar.

Às vezes, será algo pequeno ou bobo. Talvez você acabe caindo no que disse: enquanto escreve o problema, você encontra a solução.

Temos um problema com o pedir ajuda, mas você sempre pode encarar isso como uma forma de facilitar o compartilhamento de conhecimento.

Pair Programming Turbinada!

Aprendi sobre pair programming através da metodologia XP (Extreme Programming), é realmente uma maneira bacana de trabalhar, então, talvez dê uma olhada nos motivos pelos quais o recomendam e experimente em algum momento.

Mas eu gosto demais do meu espaço pessoal para considerar fazê-lo por longos períodos no escritório.

Mas remotamente?

Cada um com seu próprio IDE e esquema de atalhos estranho (funciona bem para mim, ok?) trabalhando no mesmo problema é mais do que possível com as ferramentas de colaboração que temos online.

Além disso, há também o mob programming, ou mais provavelmente, às vezes mob programming para aprendizado e compartilhamento de conhecimento, e depois mob programming para resolução de problemas críticos, como quando algo extraordinário acontece e você convoca uma reunião improvisada onde qualquer um pode participar e ajudar a resolver o problema coletivamente.

Você Não Pode Falar Sobre Remoto Se Você Não Fizer Isso Também no Escritório

Defendo o que disse anteriormente: seria estranho se as pessoas sentissem minha falta no escritório. Posso conversar com a equipe, não importa onde eu esteja, e colaborar de forma ainda mais fácil e fluida com pessoas de toda a organização.

Acredito que isso seja o mesmo para muitas pessoas que tiveram que voltar ao escritório.

Se voltarmos ao início da pandemia, veríamos a alta administração pedindo coisas que nunca haviam sido exigidas antes das pessoas no escritório.

Algumas empresas estão utilizando spyware hardcore para rastrear o mouse e o teclado das pessoas, ou até mesmo para saber se estão olhando para a tela com precisão de segundos… como se essa métrica realmente importasse para a produtividade.

Como pessoas que trabalham remotamente, só podemos fazer até certo ponto. Mas se você tem poder de decisão, outras questões são para você resolver.

Para resumir o que estou prestes a dizer, da forma mais informal e sincera possível:

Ei, se você nem faz isso no escritório, por que quer essa merda no remoto?

Quem Não é Visto, Não é Lembrado

Algumas empresas estão anunciando que trabalhadores remotos serão deixados para trás em promoções e afins. No entanto, muitos bons profissionais já são passados pelos “mais barulhentos” e por aqueles que “jogam o jogo”.

A “política de escritório” é um problema com o qual acredito que todos já tiveram alguma dificuldade.

As pessoas já mudam de emprego e departamento porque não são vistas onde trabalham, e mesmo quando são, os orçamentos para novas contratações geralmente são maiores do que o orçamento para manter os funcionários...

Esse é um problema enraizado. Talvez saber sobre isso ajude de alguma forma.

Ou talvez pense assim: se você tem tarefas críticas para delegar, quem vem à mente primeiro? Ou, para que tipo de tarefa você designaria a pessoa que está avaliando? Comece a questionar isso e você encontrará pessoas que só serão lembradas quando precisarem entregar.

Você Não Interage com Outras Equipes Quando Está Remoto

Além de estar fisicamente perto de algumas “outras equipes” no escritório, que tipo de “interação” existe? Quais conexões são feitas ou qual resolução colaborativa de problemas ocorre?

Eu diria que é mais fácil reunir pessoas de várias equipes em um ambiente remoto do que fisicamente. Não estou dizendo que reunir um grupo de pessoas em uma sala não tem seu valor, mas não seria algo para se fazer todos os dias.

Remotamente, você pode adicionar e remover pessoas conforme necessário e, se o escritório é apenas um lugar para reuniões online… a forma tradicional de organizar pessoas pode nem ser necessária (duas pizzas já não precisam ser o padrão para uma equipe). Você pode ter pessoas de várias equipes trabalhando em um único projeto ou objetivo.

No Escritório, Você Não Tem Ideia do Que as Pessoas Estão Fazendo

Pelo que ouço, acredito que quanto mais alto se está na hierarquia de gestão, maior é a tendência de se ter um viés a favor do trabalho no escritório.

Uma das razões é que eles gostam de pensar que sabem o que as pessoas abaixo deles estão fazendo só porque as estão “vendo”.

Se você pensa assim, não se iluda; você não faz ideia do que as pessoas estão fazendo na maior parte do tempo. Talvez você saiba o que algumas pessoas fazem em alguns momentos, uma visão geral… e é isso.

A menos que você esteja microgerenciando um punhado de pessoas, talvez nem mesmo assim você saberia.

Aqui vai um experimento divertido: pegue uma lista das pessoas que estão sob sua responsabilidade e escolha três nomes. O que elas fizeram durante toda a semana passada?

Você sabe? Você realmente se importa? Isso importa?

Se importa, então vá perguntar a elas! Talvez nem elas se lembrem de tudo o que fizeram.

Todas as maneiras de parecer ocupado não são o importante. Se elas entregaram o que era esperado é. Porque, mais do que usar o tempo de forma inteligente, é mais importante parecer ocupado para não parecer que você não está fazendo nada. Se eu quisesse torturar alguém, obrigar a não fazer nada num escritório “ocupado” estaria no topo da lista.

Acho que no “Trabalhe 4 Horas por Semana” (The Four Hour Workweek*)* havia algo como: “se você quer parecer ocupado, pegue algumas folhas de papel e comece a andar apressadamente de um lugar para outro. Os chefes vão pensar: agora, esse é um funcionário produtivo!”

E lembre-se também, a Lei de Parkinson dita que “o trabalho se expande de modo a preencher o tempo disponível para sua conclusão”.

Novamente, se você precisasse delegar tarefas, tenho quase certeza de que delegaria para as mesmas pessoas para as quais já delega muito. Assim, algumas pessoas recebem mais tarefas, enquanto outras desperdiçam tempo tentando parecer ocupadas. Tudo isso para dizer: não presuma que sabe o que pensa que sabe e use isso como métrica para outra coisa.

Conclusão

Lá e de volta outra vez. Tanto no remoto quanto no escritório.

O trabalho remoto veio para ficar e, se nós acreditamos na “melhor ferramenta para o trabalho”, ele deve sempre ser considerado.

Algumas pessoas estão escolhendo por si mesmas como elas desejam trabalhar, mesmo que isso signifique partir para longe de empresas que resistem à flexibilidade.

Algumas empresas estão em conflito entre o jeito legado e as novas formas de trabalhar.

E enquanto existem coisas fora do nosso controle, algumas coisas nós certamente podemos começar a fazer para melhorar hoje.

Aprendemos muito sobre trabalhar no escritório ou remotamente, e espero que meus insights te incentivem a melhorar no trabalho remoto. Temos feito isso por um período tão curto que levará décadas para encontrarmos um ponto em que possamos dizer “Então… é isso para o trabalho remoto”.

Então, neste novo mundo, não há caminhos pré-estabelecidos, apenas oportunidades. Enquanto alguns podem erguer barreiras, a direção que você toma é inteiramente sua.

Remote is good and it can get better!

Bruno Noriller — Sun, 23 Feb 2025 21:33:15 +0000

We had all the history of working in an office… but only a few years of working remotely and at worst… it was only slightly “worse” (productivity-wise) and that’s not considering we started it all in a fucking pandemic!

Remote work is not for every type of job and not for everyone (some people did experience remote working in other settings and didn’t like it), there are valid reasons against it and moments or places where it’s not the best tool.

However, there are lots of reasons that have nothing to do with whatever anyone might say about remote working being good or bad that will be outside the scope of this article because for some reason, remote working became something controversial and for reasons unrelated to working remotely.

In any case, this article explores my journey through remote work—from the initial pandemic shift to the challenges and benefits I’ve experienced as a programmer with a business background and finally, how to make it even better.

Let's remember some things

If you're reading this anywhere near the publication date, then we probably share that we went through the pandemic. And for many, myself included, that was the first time working remotely.

Some liked it and don’t want to go back to an office, others hated it and never want to try again.

But here comes the reminder: It was not all bad or all good.

We have to remember that it was a pandemic, even though some people tried to pretend otherwise. People got separated, stressed, lonely, worried… there were a lot of emotions going on and the lockdowns didn’t help.

However, today, without a pandemic, things get a little different. You can work from home and then go outside without worries, you can work and then go see your friends and do something else later.

Remote working in today’s setting is something different than it was there. And if the worst estimates were that remote work was only slight worse… with all that that was happening, then what about nowadays when it's so much better?

Why I’m writing this?

A long time ago… or what seems so long ago… I would “just go to work” because that was just “normal” and then came the pandemic and I went full remote.

The pandemic “ended” or at least the danger of it because of vaccines, I moved cities to return to working in an office, now far away from home.

We all had to go back to the offices “forgetting” about remote work only to have this shiny new thing called “remote work” introduced, step by step, but not fully remote.

When I moved, I preferred getting a place near the office, but it didn’t support enough space to work remote. So I just continued going to the office while squeezing some walking every day.

A few months back, a family member had some health issues and would go under surgery and I went back home to help.

Finally back in the office, again.

The pandemic again?

Having to go back to remote for medical reasons… it kinda felt like the pandemic all over again.

This led me to revisit a time when remote work was all new and fresh to almost everyone.

This time, however, some things were different, more like they should and could have been if we had "reached" remote working by other means.

“Full remote”, but in a weird way.

Because of internal rules, I couldn’t actually work fully remotely for the whole month. I could, however, fill the days I should be in the office by burning my PTO and vacation days.

First and foremost, my programmer and business sides will agree that having to stop multiple times in a short period is just bad.

It’s impossible to have any kind of momentum or to have any long-term project properly done like this. This was an unfortunate effect of the rules I had to follow, and it was just plain bad.

Now we can enter a place where while internally business and programmer sides agree, other people might not.

The first pause I had to do, just before starting a new project… now, that one was good.

It was a way to “clear all the cache” from what I was doing before and reset in lieu of the new project.

It IS a problem that “agile” makes us go through a marathon of sprint after sprint. Just plain wrong, even when you try to consider what the original manifesto was about or even what Scrum and other “agile methodologies” preach.

You sprint, you review… and that means some stop has to happen. Then and only then can you have another sprint.

This part is not even about remote, but I believe those downtimes can improve quality anywhere!

The bad part of the pandemic, again.

Remember when I said that not everything was bad?

Well, some things were, and I went through that again. You might remember being stressed, tired, afraid… all that happened because the medical reasons were about surgeries and then complications for someone over 80.

There were days I could throw myself at work while other’s, work was a chore to be done with. I would leave it to your imagination, but considering we all went through a pandemic… it was kinda like that.

Something we didn’t have in the pandemic

Especially when there were lockdowns, and maybe my family did go a little too overboard on those, and afterward… But in this case, I could just go out and see friends, go to the movies, go out for a run… I wasn’t in a lockdown and could let some steam out in more ways.

This and the last point is not something that applies only to remote, but since the office is a “place for working” and culture in general, we tend to keep things to ourselves.

It’s probably obvious to point out, but some displays of emotion would probably generate a lot of problems in the office.

The weirdness of hybrid

When this new “hybrid” thing came, sometimes I would be the only one in the team in the office while everyone would be remote.

While there were always some teams on the same floor, there were days I could look left and right and see no one near.

Then, finally, I saw another side where I would be the only one remote while the others were in the office.

When I was going to the office “just because”, in Portuguese, for many of my days I would say “entrou quieto e saiu calado”, which literally translates to “came in silent, left without a word”. Not sure what kind of similar idiom there is for that, but for a long time, I was always there, but aside from some nods and “sup”/“hey”/“morning” and other non-communications caused just because crossing in the corridor or accidentally looking people in the eye… I didn’t actually have much meaningful connections there.

Because of that, I would be really surprised if anyone actually were to miss me there in the office…

Remote Work is Good

Story out of the way, where does remote work shine:

Assume Async Communication

You can just assume they aren't there and drop a message.

When they read, they will respond. Same for you! When you just want to focus on what you’re doing, you can just ignore anyone and everything else and focus.

Now… try doing that with someone looking over your shoulder!

A Better Routine

I remember something Uncle Bob said, that when estimating, you have to account for the days when everything goes wrong.

Since I walked a lot, I would often get annoyed by how carbrain the city is. This should go into the estimations, and it will bring the estimates up.

But remotely, you have a lot less “moving parts” that can break your day and throw your routine. There are, obviously, a lot more things that can go wrong, but the more we can remove, the better. (If you actually want to deliver faster.)

New Ways to Communicate

Some people like to talk and figure out their point somewhere in the middle; some thrive on that… however, putting it into text makes you think a little more.

This extra time thinking might be all some people need to solve their own problems.

Not only that, it’s easier to paste some prints that can help with whatever the communication is about.

Even in a remote meeting, flowcharts, and drawing tools are always available and can make even the most awkward person feel like participating, even without talking.

It’s Good, but it can get Better!

Async Communication is Not Mandatory

While people won’t be over your shoulder, or you over theirs… in an office environment, you can always drop by unannounced and start chatting.

Yes, in an office you would see if they were not in their seat. Online, you can just pop a call, let it ring a couple of times, and if there’s no response, then just drop a message and continue async.

I’ll assume that most meeting tools have at least “green”/“red” badges to announce people's status. If it’s green and it is something less than 5 minutes, just call. If it’s red and not that urgent, then drop a message.

On the other hand, use and abuse those statuses to tell people how they should interact with you.

Also… I’m not the fastest typist; few people are that fast, so, while async is cool and all, if you’re both having to type and then wait for the other type, both of you are there at the same time… why not upgrade the connection to a quick meeting?

Maybe it comes with being Brazilian, but I can see it getting tiring to have some pleasantries talk for a few minutes (”Good day, how are you, good, what about this and that?”), deal with what was needed really quickly, and then turn on those sitcom sketches where two lovers keep waiting for the other part to disconnect first.

And when I say quick, I mean it. No pleasantries, get on a call, start talking, solve what you need to solve, and disconnect. Simple as that!

You can always have longer meetings and save all the chit-chat for when you don’t need it to be quick.

Learn Your Tools

Depending on what you use you’ll have different features to play with. So, I can’t really help you there except with this advice:

Learn them. Use them.

It’s not all pen and paper anymore!

Remote or not, there will be features ready to be used and abused for better communication and teamwork.

Ask For Help

When we have problems we show it, be it in our faces or our sighs (or maybe throwing soft things at the screen or doing rude gestures toward it…).

In an office, you might pick that up and try to help, but remote… if you don’t tell me you have a problem, then I can’t help you.

Sometimes it will be something small or something dumb. Maybe you’ll fall into what I said above that while you write the problem you will find the solution.

We have a problem with asking for help, but you can always think of that as a way to facilitate the sharing of knowledge.

Pair Programming unleashed!

I learned about pair programming from the XP (Extreme Programming) methodology, it’s actually a nice way of working, so maybe check there why they recommend it and give it a try sometime.

But I like my personal space a bit too much to ever cogitate doing it for long stretches of time in the office.

But remote?

Each to its own IDE and weird shortcut schema (it works for me ok?) working on the same problem is more than possible with the tools for collaborative working we have online.

Then you also have mob programming, or more likely, sometimes mob programming for some learning and knowledge sharing and then mob programming for critical problem solving, as in: something extraordinary happened, and you just pop an impromptu meeting, anyone can join and try to solve the problem collectively.

You Don’t Get to Talk About Remote If You Don’t Also Do It in the Office

I stand behind what I said above that it would be weird if people were to miss me there. I can talk with the team no matter where I am and even more easily and seamlessly collaborate with people from all over the organization.

I believe this is the same for a lot of people who had to go back to the office.

If we go back to the start of the pandemic, we would see upper management asking for things they never asked before of people in the office.

There is some hardcore spyware being used by some companies to track people's mouse and keyboard, or even if they are looking at the screen with seconds precision… as if that metric actually mattered for productivity.

As people who work remotely, there’s only so much we can do. But if you are someone who has decision power, then other things are for you to do.

To sum up what I’m about to say, in the most informal and heart-to-heart way possible:

Hey, if you don’t even do this in the office, why you want this shit when remote?

If You’re Not Seen, You’ll Not Be Remembered

Some companies are announcing that remote workers will get passed over for promotions and such. However, many good workers are already passed by the “loudest” and those who “play the game”.

“Office politics” is a problem that I believe everyone already had some issue with.

People already jump jobs and departments because they are not being seen where they work, and even when they are, budgets for new hires are usually higher than the budget to keep people there...

This is an ingrained problem. Maybe knowing about it might help somewhat.

Or maybe thinking like this: if you have critical tasks to assign, who comes first to mind? Or, which kind of tasks you would assign to the person you’re evaluating? Start asking that around and you’ll find people that might be only remembered when it is to deliver.

You Can’t Network with Other Teams When Remote

Aside from being physically near some “other teams” in the office, what “networking” is there? What connections are made or collaborative problem-solving is there?

I’ll say it’s easier to gather some people from multiple teams in a remote environment than physically. Not saying gathering a bunch of people in a room doesn’t have its value, but it wouldn’t be something to do every day anyway.

Remotely, you can add and remove people as needed, and if the office is only a place for online meetings anyway… the traditional way to organize people might not even be needed (two pizzas don’t need to be a standard for a team anymore). You can have people spanning multiple teams all working on one project or objective.

In The Office, You Have No Idea What People Are Doing

From what I hear, I believe that the higher one is in the management chain, the more one might be biased toward office work.

One of the reasons is that they like to think they know what people under them are doing just because they are “seeing” them.

If you think like this, don’t kid yourself; no, you don’t. You have no idea of what people are doing most of the time. You might know what a few people are doing some of the time, maybe more like an overview… and that’s it.

Unless you’re micromanaging a team of a handful of people, maybe not even then, you wouldn’t know.

Here’s a fun experiment: take a list of people under you, and from there pick three names. What did they do the whole week last week?

Do you know? Do you actually care? Does it matter?

If it does, then go ask them! Maybe not even they will remember all that they did.

All the ways to appear busy are not what is important. If they delivered what was expected is. Because more than using the time you have in a smart way, it’s more important to look busy for the sake of not looking like you’re doing nothing. If I wanted to torture someone, mandating they do nothing in a “busy” office would be top of the list.

I think it was in the Four Hour Workweek something along the lines of “if you want to appear busy, pick some sheets of paper and start walking hastily from one place to another. The bosses will be like: now, that’s one productive employee!”

And also remember, Parkinson’s Law dictates that "Work expands so as to fill the time available for its completion”.

Again, if you needed to delegate some tasks, I’m almost sure you would delegate to the same people you already delegate a lot of them. So, some people get more tasks, while others waste more time trying to look busy. All this to say: don’t assume what you think you know and use that as a metric for something else.

Conclusion

I was there and back again. Both for the remote and in the office.

Remote work is here to stay and if we believe in the “best tool for the job”, then remote working should always be considered.

Some people are choosing for themselves how they wish to work, even if it means parting ways with companies that resist flexibility.

Some companies are conflicted between the legacy way and the new ways of working.

And while there are things outside our control, some things we can certainly start doing to improve today.

We learned a lot about working in the office or remotely, and I hope my insights will push you to get better at remote working. We have been doing this for so short a period of time that it will take decades to find a point where we can say “Yep, that's about it for remote working”.

So, in this new world, there are no set paths, only opportunities. While some may erect barriers, the direction you take is entirely up to you.

How to Brick your NAS and lose your data

Bruno Noriller — Tue, 11 Feb 2025 21:18:30 +0000

This is a horror story, but don’t worry… I just lost a lot of time… this time.

What happened?

For context, I’ve been messing around with self-hosting. I have a Proxmox and inside I’m running TrueNAS as a VM and passing the disks.

Before all that, I spent a lot of time running Badblocks and SMART tests, both of which check the health of the disks and are especially helpful in finding problems in a new disk, and then I started dumping data from my laptop and external drives.

I watched a lot about TrueNAS and ways to speed up it with caches, RAID levels (for data protection even if a disk or two were to stop working), and other important things.

Then, some days after dumping everything something bugged me about the disk layouts and which /dev/sdX they were using… since most were just virtualized disks I was passing to speed up the NAS operations… should be easy right? Just remove and then add again…

But wait! Can’t remove? Why?

You see…

RTFM

RTFM or Read the fucking manual. I was click-happy and just created and added disks willy-nilly. Hell! I even added “mirrored” drives because of “redundancy”.

The problem?

All those were virtualized drives coming from a single SSD.
Some of those drives became an integral part of the NAS Pool.

What is a Pool?

If the part above didn’t scare you… it’s because you don’t know what a pool is.

You gather multiple disks in a pool and they work as a unit. You don’t have multiple disks anymore, you have one Pool that works as such.

With RAID levels you distribute data between multiple disks so that you can lose disks without losing data (there are trade-offs, but you probably want some redundancy).

The horror time

Some of the “extra” disks you can add also became part of the pool.

They were the dedup tables (to speed handling deduplication) and metadata (to handle small files and metadata).

Each of those, when lost WILL brick your data. So if you are going to assign disks to them, you really want to assign multiple to each to have redundancy there.

Meanwhile, what I did was to give it shares of one single SSD.

I set up RAID 6, losing 2 disks worth of space to have enough redundancy only to add one point of failure to absolutely destroy all my data in that case of losing that single SSD. Isn’t this scary?

Test your backup!

More than a couple times I heard about this, you might have also and I’ll be saying here again: TEST YOUR BACKUP!

One part is: if you do lose your data, you don’t want to be stuck trying to figure out how to retrieve your data.

Other part is: can you even retrieve that data?

I removed some of those virtualized drives and bricked the pool. Lesson learned, still had the data, was able to dump everything again.

I also know that I can turn off a couple drives and still have my data. (Also, after RTFM I’m only using actual cache drives for read/write that won’t affect the pool.)

How are your backups?

Are they available? Can you actually restore them in a pinch?

Better give a look when you don’t need it, than having to use it and finding yourself without any.

Starting with Self-Hosting

Bruno Noriller — Sun, 26 Jan 2025 20:27:34 +0000

My new obsession is self-hosting. And I’m not even saying “VPS” (and yes, I know about the OCI one), I’m saying having a physical server at arms reach like the “webmasters” of old.

What I’m using

It’s not the best rig, but I have 5 HDDs of 4 TB each (with an extra slot for later), but using RAID 6, so I have around 11 TB of usable space and I can blow up 2 drives and still lose no data. I also have 128 GB RAM, half of which will go to the NAS to be used for the ZFS cache.

Glossary

I learned a lot, but I also know that I had to learn a lot to understand what I’m able to say now.

VPS - Virtual Private Server (basically a server someone else hosts for you)
OCI - Oracle Cloud Infrastructure (think AWS. The thing I’m referring there is that you can claim some free VPS there. The amd64 are micro instances, but the arm64 is crazy good totaling with 4 cores, 24 GB RAM, and 200 GB disk to use however you want).
RAID Level - It’s a way to distribute data in multiple disks with duplication in a way you can lose what's in one (or more) disks and don’t lose the data.
ZFS - a file system like the more “normal” ext4 for Linux, NTFS for Windows, and FAT for external drives. ZFS has advantages over those for use cases like for a NAS.
NAS - Network-Attached Storage (think Google Drive).
VM - Virtual Machine (basically a machine running on another machine)
KVM - “keyboard, video, and mouse” (there are devices that let you control multiple devices with one set of those, some that work over the internet so you can hard reboot and access BIOS settings without physical access to the hardware)

Why self-host

There’s a lot you can do with a private server, not to mention it will teach you a lot about computers, networks, and how to use your hardware...

You can host utilities and services for you to use (and even share with family and friends) and, of course, host your own services for the world to use.

Cost

Initial investment might be high if you just buy new hardware like I did, but if you already have some old PC/Laptop lying around, you can just start there. Not to mention one or more Raspberry Pi in the mix.

Hardware aside, there will be a time investment (especially if you don’t read the articles to come =P) to make everything work initially.

But when you check the costs for storing TBs of data, plus VPN, plus other services… You can also compare with VPS costs. Few cores with some RAM can easily go into “I could just buy some Pi every year and host myself”.

And, obviously, you can host things you build yourself and easily handle hundreds of requests per second without breaking a sweat (most likely even thousands).

Disclaimer: this might (or not) reflect what I keep repeating myself to justify buying the rig XD. Oh well…

Privacy

I don’t have much expectation of it anymore, but having control of some is nice and I haven’t even scratched the surface.

NAS

One of the things I’m planning with it is for a NAS, primarily for backups, but the more you learn, the more you can use it. I can run VMs with disks from the NAS and depending on how the services running it uses the disk… I say I wouldn’t know it’s running on spinning HDDs.

If you’re planning on sharing only within your home, it’s pretty straightforward to mount and use as a “normal disk” on your computer.

On ethernet cable, if the devices are on 10 GB ports, I understand people can edit 4K video as if the files were on their drives.

My laptop on wifi could handle files fast enough for my needs that I didn’t even notice they weren’t on my machine.

ARR Stack

Some play with the ARR Stack (no glossary for this one =p).

It’s interesting to play and learn how it works and I guess it has its uses to… hmm… sharing Linux distros or something like that.

Network-wide Ad Blocking

One of the first things I “installed” was a Pi-Hole, a DNS resolver that you can use as a DNS sinkhole.

By using lists to have network-level ad blocking, it will work on your computer, phone, and even smart fridge. It works by saying that https://ads.and.more.ads.com is at 0.0.0.0, making ads fail to work. It also makes you use less data since you don’t need to “download” all the assets even if you use another ad blocker on the browser.

There are also lists to block some tracking and metrics… and from what I heard… people with children can use custom parental filters and also have the power to “turn off” the internet for kid's devices.

“Private” VPN

Running a VPN (Virtual Private Network).

Since I’ll be away from the server for long stretches of time, I want to be able to access it from anywhere in the world and also be able to not care about being on public networks.

All traffic is encrypted and will end up going out from your server even if you’re on a sketchy wifi labeled “free wifi” (seriously, even then don’t use that).

I was already “convinced” to buy some VPN, mostly because I knew I would be on public wifi from airports/hotels, and one of the biggest problems with it i that some services block you when using a VPN (since commercial VPN IPs are basically “public” and well known), but with a “private” VPN then it would be like using your home network… and if you don’t have a problem there, you probably won’t have a problem anywhere.

“Replace Google” and other Services

Without paying for more space, you probably have to delete some things or maybe juggle having multiple accounts.

Google Drive and Photos are first on my list. There are awesome alternatives that you can host yourself and be limited only by your own hardware.

There is just so much you can do… I recommend checking https://awesome-selfhosted.net to see if anything piques your interest.

Offload your devices

Instead of keeping everything on my computer or phone, I can just have VMs running things I want to be done there and then just use or download if/when needed on my device.

One example: you have a potato PC lying around but have a good internet. You can connect to a VM in your server and use it instead.

What I’m really thinking is automating some things I do every day/week and just getting the results later. Hell, might as well run some LLM and mix in the pipeline there.

Since I’m planning on checking out some other distros, this would allow me to have a “working PC” if I need to do anything before fixing any problems I have.

Self-Host your projects

I mean… I know it seems a “lost art” to some, but learning to self-host on “bare metal” is teaching me a lot and I plan to do a lot with it.

I’m writing this to my “future-past-self”

I’m sure I’ll forget most of the little fixes I did along the way and it will come to bite me later. I’ll probably rediscover some as I fix or when I set a new node and then I might stumble upon this… so, future-past-self.

My goals with self-hosting

I want things for me (pi-hole, NAS, VPN) but also to self-host services to the internet.

I’m on a timer because I have to do things that might screw up the server now since later I won’t have easy physical access to it. A KVM switch over IP would help with it, but unfortunately, I don’t have it… yet.

But after that critical part, I still need ways to connect from wherever I will be.

And, BTW, within minutes of setting a DNS record, I got a lot of (probably malicious) requests (and I didn’t even have anything there!) so security is a high priority, and then some segregation for “management” and “services” on top if I can manage.

After this base is done I can then work on the “fun stuff”.

Nix at level 0 is still awesome

Bruno Noriller — Sun, 29 Dec 2024 13:57:05 +0000

There’s a recurring joke that once you understand monads you lose the ability to explain monads. Well… I think this weirdly applies to Nix.

What is Nix

There’s nix: the language, nix: the package manager (think apt, dnf…) and related NixOS, the OS based on nix (both language and package manager).

I won’t go further than that and it’s not necessary for now except to say that it is the one package manager with the most packages available.

Which Nix are we talking about here?

The package manager.

Why use Nix?

Recently I was playing with IP ranges, so you could use a package like ipcalc but you have to install it and then it will stay there… even if you do remove it, maybe something will be left behind.

And let’s not think about conflicts on dependences…

Nix solves both problems, but for that, you need to install it.

Installing Nix

While you can use the official installer: https://nixos.org/download/

But the one I recommend and I’m using is: https://github.com/DeterminateSystems/nix-installer

Which handles a lot of problems, including some quicks of some distros (including even MacOS and distros under Windows WSL).

Using Nix:

Before using it, I do need to expand on what you know about Nix here.

All packages are packages that have their own list of dependencies, which also might have their own list.

If you know JS and the package.json, it’s like that.

one-of use

Do you need something only occasionally? Maybe you want to test something or evaluate which option is better?

nix-shell -p plus the name of the package(s)

(which, by the way, you can find here: https://search.nixos.org/packages).

That’s it. All you need to know is that it will create a shell with the packages available, and if it has conflicting dependencies? It will just use the ones it needs and ignore the rest.

Once you exit the shell, it’s all gone (* terms and conditions apply).

actually “install”

nix profile install nixpkgs# package name and if needed, nixpkgs#other-package all in one line.

Now it’s installed and ready to use.

removing things

If you “installed”, then remove with nix profile remove package name, and that's it.

Then for those you’ve removed and those you’ve played around with nix-shell…

nix-store --gc and it will be actually, all gone with disk space freed.

You see… Nix downloads packages and dependencies and uses them from a store. The first time you need something it will take time to download everything (and by everything, I mean everything… even “low-level” utilities you take for granted… all to keep everything pure and working). Again, like in the JS ecosystem.

Outro

I can’t see myself without Nix anymore… it’s just so handy to pull packages as you need them without worrying about having to remove them later or weird problems happening because you, at some point, had installed something that needed some version of some dependency and it never got updated.

If you start checking out nix (language), nix (package manager) and NixOS, then you’ll probably be overwhelmed by the sheer amount of information, but believe me that these 2 (3 if you consider the one to clear unused dependencies) is all you need to solve most of the problems you might have and even just manage your dependencies overall.

And if something didn’t work OOTB? Just install it the way you would normally.

I hope I can manage to keep learning while retaining the ability to explain it… so that I’m confident in what “level 1” would be… then expect more to come.

Chunk-Busters: Don’t cross the Streams!

Bruno Noriller — Mon, 25 Nov 2024 17:14:21 +0000

⚠️ If you have photosensitivity, you probably want to skip this.
See the static image below, those lights will start blinking real fast!

How does the internet work?

Remember the title… we are talking about streams here.

I could talk about protocols, packets, ordering, acks, and nacks… but we are talking about streams here, and as you probably guessed right (I believe in you =D) with streams… it’s either binary or strings.

Yes, strings are zipped before being sent… but for what we usually care about in front and backend development… strings and binary.

In the following examples, I’ll be using JS streams.

While Node has its own legacy implementations, we have ways to deal with streams that are the same code, be it in the front or back.

Other languages have their way of dealing with streams, but as you’ll see… the actual code part of dealing with it was not that complicated (not to say there aren’t complex things happening).

The example problem

You have a frontend that has to consume data from multiple sources.

While you could access each source individually via its IP/port, you put them behind an API Gateway for ease of use and control.

The Repo

Check the repo at the link, there learn how to run it yourself so you can play with it.

https://github.com/Noriller/chunk-busters

The Video

Video version to follow along:

https://youtu.be/QucaOfFI0fM

v0 - the naive implementation

You have the sources, you fetch, wait, and render. Rinse and repeat.

await fetch1();
handleResult(1);
await fetch2();
handleResult(2);
...
await fetch9();
handleResult(9);

You might be thinking that no one will actually do that…

In this example, it's clear that something is wrong, but it’s not that hard to fall into this.

The obvious: it’s slow. You have to fire and wait for each request and if it’s slow… you have to wait.

v1 - the eager version

You know you don’t want to wait for each request individually… so you fire all and then wait for them to complete.

await Promise.all([
  fetch1(),
  fetch2(),
  ...
  fetch9(),
]);
handleAllResults(results);

This is what you probably do, so it’s good right?

I mean, except if you have ONE single request being slow… this would mean that even if all the others are already done… you still would have to wait that one to complete.

v2 - the smarter, eager, version

You know you might have some requests that are slower, so you still fire all and wait, but as they come you already do something with the result when possible, so when the last one arrives, the others are already done.

await Promise.all([
  fetch1().then(handleResult),
  fetch2().then(handleResult),
  ...
  fetch9().then(handleResult),
]);

This HAS to be the best solution right?

Hmm… something weird?

v3 - I was lying to you… this is what v1 should look like

Remember v1? Yeah… this is what it should look like:

Turns out there’s a limit to how many connections you can have with the exact same endpoint in http/1 and not only that… it’s browser dependant and each browser might have different limits.

You might think to just use http/2 and call it a day… but even if this was a good solution, you still have to deal with multiple endpoints in the frontend.

Is there even a good solution for this?

v4 - enter streams!

Let’s revisit v0 but using streams…

You’re smart, so you were probably expecting this since the warning spoiled it a little… but yeah… what you were seeing before was not all the data that the backend was generating.

Anyway… as we fetch we render.

// usually we do this:
await fetch(...).then((res) => {
  // this json call accumulate all the response
  // that later is returned for you to use
  return res.json()
})

If we instead tap on the stream coming, we can do something with the chunks of data as it comes. (Yes! Like Chat GPT and the like do.)

Even if v0 is the worst way of handling this problem, it’s greatly improved by using streams. You can trick the user by showing something, anything, even if the total wait time is the same.

v5 - v1, again, but with streams!

The http/1 problem is still an issue, but again, you can already see things as they come.

Yeah… I can’t stall this anymore… so…

v6 - one API to rule them all!

Or… maybe I can?

You see, the frontend had to manage too much… if we can offload that to the backend, then you can have one endpoint that will handle all the sources.

This solves complexity on the frontend and http/1 issues.

await fetchAll();
handleAllResults(results);

// if you had already offloaded to the backend,
// then you might had data like
{
    data0: [...],
    data1: [...],
    ...
    data9: [...],
}
// since the backend would be doing the fetch
// waiting ALL the data and then sending to the front

// in this case, however...
[
    { from: 'data9', ... },
    { from: 'data1', ... },
    { from: 'data1', ... },
    { from: 'data0', ... },
    ...
    { from: 'data1', ... },
]
// The backend will stream from the sources,
// it will handle, parse, do something with each,
// then send it to the front
// the order is not guaranteed, it will be out of order
// between the sources and you need to add something
// to know which source sent each piece
// but you can receive piece by piece as it's generated
// and handle them accordingly

v7 - and finally… one API, multiple sources, and streaming.

We call one API, that will call all the sources, stream the data, handle it, and pass it to the front that will, in turn, render the data as it comes.

The code used for this is basically the same on both front and back:

// in both frontend and backend we can use the same `fetch` api
fetch(url, {
  // we use the AbortController to cancel the request
  // if the user navigates away (or if connection is closed)
  signal,
}).then(async (res) => {
  // instead of the "normal" `res.json()` or `res.text()`
  // we use the body of the response
  // and get the reader from the body
  // this is a `ReadableStream`
  // (there are other types of streams and ways to consume them)
  const reader = res.body?.getReader();
  if (!reader) {
    return;
  }

  // remember we are "low level" here
  // streams can be strings or binary data
  // for this one we know it's a string
  // so we will accumulate it in a string
  let buffer = '';

  // let's read until the stream is done
  while (true) {
    const { done, value } = await reader.read();

    // if the stream is done (or the user aborted)
    if (done || signal.aborted) {
      // cancel the stream, exit the loop
      reader.cancel(signal.reason);
      break;
    }

    // if we have a value
    if (value) {
      // decode and add to the buffer
      buffer = buffer + new TextDecoder().decode(value);

      // here is where the magic happens!
      // we check if we can consume something from the buffer
      if (canConsumeSomething(buffer)) {
        // value above might be half or double of one "chunk"
        // so, we need to slice what we can consume
        // the if above might be changed to a "while"
        // or the chunk below might be "multiple"
        // consumable values... it all depends on how
        // you will use it.
        const [chunk, remaining] = consumeSomething(buffer);

        // in the frontend, make it render the chunk
        // (in react, we can simply update the state
        // and let react handle the rerendering based on that)
        renderChunk(chunk);
        // in the backend... send the response
        // we can always parse, transform before that
        // but the back will probably either send to the
        // frontend, another service or to a DB.
        sendResponse(chunk);

        // and then we update the buffer
        buffer = remaining;
      }
    }
  }
});

Yes… that's it (as the most basic and easy example goes).

We add the string coming to a buffer, parse it, check if there’s a usable chunk, use it, and forget it. This means you could receive/consume TBs of data… one chunk at a time, with little RAM.

I know what you’re thinking… and it’s stupid… it’s also madness…

MOOOOOOOM I want Websockets!

No sweetheart, we have websockets at home!

Websockets at home: next?

v8 - it’s only stupid if it doesn’t work

You’re smart, you thought that if the source is still generating data… then maybe we could update some variables…

This way you can keep the one connection being used to get more data or change something from what it’s generating.

Yes… I guess you could do that… and I did the example on your insistence. =D

Still… it’s a stupid idea, I don’t know where/if it can be used in a real production environment. Maybe if you travel back in time to that awkward JS phase between MPA and Ajax where you had enough interactivity, but not enough connections to the same server (some browsers had a limit of only 2!) then maybe?

Aside from that, no idea. If you do have… let me know.

In the example above, attention to the center board, especially the "progress border": you can see that one keeps updating. If you opened the network tab you would see that the GET connection is never closed before the end. You would also see multiple other requests that change what that one, still alive, connection was doing… all of that with vanilla http/1.

What’s next?

String vs JSON

This example is the most basic I could make. I’m even using simple strings instead of JSON since it’s easier to parse.

To use JSON, you have to accumulate the string (we do have to JSON.stringify the backend response for a reason).

Then either check where to break it and then parse that value or parse as you go.

For the first one, think NDJSON: instead of a JSON array, you separate the objects with new lines, then you could “more easily” find where to break, then JSON.parse each and use the object.

For the latter, you parse as you go as in: you know you’re in an array, now it’s an object, ok first key, now it’s the value of the key, next key, skip that, next key… and so on and on… it’s not something trivial to make manually, but it’s like the jump from the await then render to the render as you await this is all about… except… on an even smaller scale.

Error Handling

People like to host examples, this one you need to run yourself… I hope the reason for not hosting the example somewhere is clear now, but another one is that we don’t expect any error here, and if you were to add network errors above everything else… well…

Errors should be handled, but they do add another layer of complexity.

Should you be using it?

Maybe… you can say it depends…

There are places where streaming is the answer but in most cases… await json is enough (not to mention easier).

But learning about streams opens up ways to solve some problems, be it in the frontend or the backend.

In the frontend, you can always use this to “trick” the user. Instead of showing spinners everywhere, you can show something as it comes and then show more as it comes, even if it takes a while. As long as you don’t block the user from interacting with it… you can even make something that is “slower” than just showing spinners feel like it’s way faster than anything actually faster.

In the backend, you can save up on RAM since you can just parse each chunk of data as it comes, be it from the front, a database, or anything else in between. Handle the data as needed, and send it without having to wait for the entire payload that would make it throw an OOM (Out of Memory) Error. GBs or even TBs of data… sure, why not?

Outro

Is React slow? This whole example frontend was done with React and aside from the “main” thing going on with all the blinking “lights”, there’s a lot of other stuff going on.

Yes… if you go fast enough the example can’t keep up and start freezing. But since it’s easily thousands of renderings going on per minute… I do think it’s enough for most applications.

And, you can always improve performance: for the “progress border”, I’ve used deferred values to make it smoother if you need to save some in renderings… I could done this and other performance enhancements for the “lights“ and the title, but it would just make the “lights” stop blinking a lot of the time (which wouldn’t make a nice demo), and also the “electric” underline in the title wouldn’t be as fun as it is.

In this example, all those “improvements” wouldn't be ideal, but for normal applications... you can make it handle a lot. And if you do need something more, then in this case use another solution.

Conclusion

Add streams to your arsenal… it might not be a panacea solution, but it surely will come in handy someday.

And if you’re gonna do something with it and want help, well… maybe give me a call. =P

Fully Typed, Easy React Context API Example

Bruno Noriller — Sat, 26 Oct 2024 14:41:05 +0000

The trick to not having to babysit the context types is easy!

If you use the context API, then one problem is the babysitting of its types.

Another one is having to use multiple imports to use it when you need it.

With this example, we solve both problems and make it quick and easy to use the React Context API.

Copy Paste Example

Copy, paste, then just replace all “example” to what you need to name and you're ready to go.

(Afterwards, there will be a fully commented version.)

import {
  createContext,
  useCallback,
  useContext,
  useDeferredValue,
  useMemo,
  useState,
} from 'react';

function useContextValue(init: number = 0) {
  const [state, setState] = useState(init);

  const doubleValue = state * 2;
  const defferedStringValue = useDeferredValue(state.toString());

  const reset = useCallback(() => {
    setState(init);
  }, []);

  const value = useMemo(
    () => ({
      state,
      doubleValue,
      defferedStringValue,
      reset,
    }),
    [
      state,
      doubleValue,
      defferedStringValue,
      reset,
    ],
  );

  return value;
}

type ExampleContext = ReturnType<typeof useContextValue>;

const Context = createContext<ExampleContext>(null!);

Context.displayName = 'ExampleContext';

export function ExampleContextProvider({
  children,
  initValue = 0,
}: {
  children: React.ReactNode;
  initValue?: number;
}) {
  const value = useContextValue(initValue);
  return <Context.Provider value={value}>{children}</Context.Provider>;
}

export function useExample() {
  const value = useContext(Context);

  if (!value) {
    throw new Error('useExample must be used within a ExampleContextProvider');
  }

  return value;
}

Commented Version

import {
  createContext,
  useCallback,
  useContext,
  useDeferredValue,
  useMemo,
  useState,
} from 'react';

/**
 * We create a custom hook that will have everything
 * that would usually be in the context main function
 *
 * this way, we can use the value it returns to infer the
 * type of the context
 */
function useContextValue(init: number = 0) {
  // do whatever you want inside

  const [state, setState] = useState(init);

  const doubleValue = state * 2;
  const defferedStringValue = useDeferredValue(state.toString());

  // remember to memoize functions
  const reset = useCallback(() => {
    setState(init);
  }, []);

  // and also memoize the final value
  const value = useMemo(
    () => ({
      state,
      doubleValue,
      defferedStringValue,
      reset,
    }),
    [
      state,
      doubleValue,
      defferedStringValue,
      reset,
    ],
  );

  return value;
}

/**
 * Since we can infer from the hook,
 * no need to create the context type by hand
 */
type ExampleContext = ReturnType<typeof useContextValue>;

const Context = createContext<ExampleContext>(null!);

Context.displayName = 'ExampleContext';

export function ExampleContextProvider({
  children,
  /**
   * this is optional, but it's always a good to remember
   * that the context is still a react component
   * and can receive values other than just the children
   */
  initValue = 0,
}: {
  children: React.ReactNode;
  initValue?: number;
}) {
  const value = useContextValue(initValue);
  return <Context.Provider value={value}>{children}</Context.Provider>;
}

/**
 * We also export a hook that will use the context
 *
 * this way, we can use it in other components
 * by importing just this one hook
 */
export function useExample() {
  const value = useContext(Context);

  /**
   * this will throw an error if the context
   * is not used within the provider
   *
   * this also avoid the context being "undefined"
   */
  if (!value) {
    throw new Error('useExample must be used within a ExampleProvider');
  }

  return value;
}

Final Words

That's it. Context API is easier and more subtilized than it should, but it’s a powerful tool for the cases where it needs to be used.

Just do remember that the React Context API is not Redux (or other state managers) and you shouldn’t shove the whole application state in it.

Well, you can, but it can cause unnecessary problems.

This was written with React < 19 in mind, with the new compiler coming, memorization and unnecessary rendering might not cause problems.

React: State X Derived State

Bruno Noriller — Mon, 23 Sep 2024 22:56:16 +0000

What’s a derived state? Think one state for text and then another for uppercaseText.

Derived State

function Foo() {
    const [text, setText] = useState('hello, za warudo!');
    const [uppercaseText, setUppercaseText] = useState(text.toUpperCase());

    useEffect(() => {
        setUppercaseText(text.toUpperCase());
    }, [text])

    ...
}

Putting like that it’s crazy to think anyone would do this… right? RIGHT?

Yes, an example like this will make clear that this is wrong.

The Bad of Derived State

Stored separately and out-of-sync with the actual state.
Triggers (depend) on unnecessary re-renders.

How to refactor the derived state?

Say it’s an expensive calculation… the solution is to use useMemo.

function Foo() {
    const [text, setText] = useState('hello, za warudo!');
    const uppercaseText = useMemo(() => text.toUpperCase(), [text]);
    ...
}

How to quickly spot state that can be derived?

I came up with a good way of thinking that should make it easier to KNOW if a state should be “another state” or just a computed property (memorized or not depending on the case).

function Foo({
    text = 'hello, za warudo!',
    uppercaseText = text.toUpperCase(),
}) {
    ...
}

// Forget react for a moment...
// Would you ever call a function like this?
const text = 'hello, za warudo!';
Foo({
    text,
    uppercaseText: text.toUpperCase(),
});

If you think of those states as “props”, then this makes it more blatantly what it should be.

Forget React entirely, think only of functions:
Would you call a function with a variable and then another variable you could just compute inside?

If you have these three things, you will be fine (as a dev)

Bruno Noriller — Sat, 07 Sep 2024 22:08:51 +0000

2024-09-12 is once again Programmers Day (the 256th day of the year), so following the other years ([2023, 2022, 2021]), here I am again with three things.

By the 4th year of the dev life, some people might catch themselves doubting if they made the right choice or are fit to be a programmer.

But some things might tell you that you will do just fine.

You read the documentation (maybe even write!)

Especially nowadays with the AI hype, you catch yourself going after the documentation for something you use over and over.

While some might think they are stupid because they keep going back to read the same things, you know where the docs are and don’t mind defaulting to that over trying to stumble trying to remember how to do something.

Not only that, you may write some docs yourself. It might be as little as a how/when to use a function you made or some piece of code you spent a few minutes figuring out and instead of just leaving it there like that, you translate what you’ve learned into some helpful commentary for the next time someone might need that.

If you still don’t: don’t worry and start small, remember to say not what you’re doing, but why you’re doing something. You know the feeling of going to read some code you’ve made and not remembering why you made it like that? When you wrote, you thought it was clear as day why you did what you did… next time this happens, explain that to your future self by leaving some comments.

You double (and triple) check your work

You already know perfect code doesn’t exist. After all, every time you open some code from a month ago you get a little embarrassed of your past self.

Some people think: “My work is done when the code works”. You, on the other hand, already know that working is just the beginning. You can’t make it perfect, but at least you can make sure you won’t cringe seeing your code in the future.

It’s working? Then you might commit when it’s a tricky piece of code.

You might add and commit some tests, to make sure it will keep working.

Then you refactor a little here and there, make it clear where you can, and comment when you can’t.

You commit again, now a better piece of code, and might do even another pass.

Before making a pull request, you check one more time and only then submit the code.

If you don’t: remember that we read more code than write. Why make your life harder because “it’s already done” or “faster to just code” when you should know that it will come back to hunt you and then it won’t be fresh on your mind anymore and you’ll waste a lot more time to decipher what you did back then.

You write code for the most stupid person you know: you!

You stopped all pretense of trying to appear smart, stopped “clever code” and “code golf” code. You know you’re stupid and you learned to make code you can keep up with even after a long day.

This usually means adding comments and documentation, cleaning, refactoring over and over, and making it as clear as you possibly can… for your future self.

If you ever caught yourself in this situation, you might think you’re stupid and maybe impostor syndrome might kick in. But if you’re doing code that feels so simple… you’re in the right direction.

There are things complicated out there, but for most of the time, for most people… code isn’t the thing that should be complicated. WE complicate things a lot, business rules might be complex, but most of the time: code should be simple and direct.

If you’re not there yet, don’t worry. Remember to look at things you’ve made last month and try to figure out what the hell were you thinking back then. Do this when you’re not pressed for time and keep in mind that a time will come when you’ll need to check old code, under pressure, maybe with someone breathing down your neck, the git blame will say it’s “your” code and you’ll have no idea what is happening… So, before that happens, try to think which type of code you would like to see: the “clever code” or the stupid simple one.

Happy Programmer’s Day!

Cover Photo by ready made: https://www.pexels.com/photo/cute-purebred-dog-resting-on-bed-near-laptop-in-apartment-4297818/

To fold or to double down? That’s a tech question!

Bruno Noriller — Sat, 24 Aug 2024 21:02:23 +0000

We can’t stop now, as we already invested 1x in it, but let’s proceed to spend 100x more over the years! STONKS!

The Javascript example

You might have heard it before, but Javascript was written in 10 days. It had a rapid growth of adoption and they didn’t want to introduce breaking changes even after some years of using the language… so, now the language is almost 30 years old (remember it's from 1995!) and we still have to deal with those decisions.

They knew way back in the early years of JS that there were a lot of changes that would benefit the language, but didn’t want to break the “countless” websites from back then (there were a couple of million total websites worldwide back then, not necessarily using JS at all!).

Hindsight is 20/20, and they might have broken JS in a way that wouldn’t be what it is today.

Then again… Angular did just that when breaking and launching “Angular 2”. Just because many people were using Angular, they knew it was not sustainable to keep going with AngularJS, so they started to phase out from it to Angular2 and Angular got better because of it.

Would you invest in this today?

When investing, we feel like “losing” or “quitting” by jumping ship, but the people who can go farther are those who know when to quit a bad decision (not necessarily bad, but things change over time).

Back to the Angular/JS/2 example. Some companies are still using AngularJS, even though it reached end of life and some more some years ago. Now, they have to support and patch it on its own as they increase their codebase and make the decision to stick to it more and more painful and harder to change because of all the investments in it.

Of course, on the other side of this are people jumping from fad to fad and creating monsters that show whatever tech was more popular at each time, some that just stopped being used once people tried to start using it or that, some reason or another, just died and stopped being maintained.

The name of this is the sunk cost fallacy

You think you’ve invested “too much” already, so you can’t back down. You also think you don’t need the new shiny things if the boring old LAMP stack is enough and working.

But one thing is “it works” and another is: “in the long term, this will cost more than changing”.

In the financial sense, some new thing might let you go faster and farther, or at least, will let you be free to pursue other opportunities you might have missed otherwise.

The COBOL example

Banks run on COBOL, it’s been decades in “decline” but at each turn, they say it’s worth more to keep their COBOL legacy and hire COBOL developers at higher and higher prices than it is to work on migrating from it. TBF, hopefully, they are strangling their legacy. But if they are still actively developing in COBOL, no LLM will help them when it starts costing prohibitively more to hire COBOL devs in a market with fewer and fewer available people.

Meanwhile, new competitors without that legacy can enter the market with other stacks that let them move faster, cheaper, and with a bigger hiring pool available.

Should you? When?

The analogy is that of steering a huge ship. When you turn the ship’s wheel, the change is not immediate. You need to prepare beforehand and there are strategies to make this easier. Then it will still take some time until it’s clear that yes, the ship is turning.

ROI

But coming back to the financial analogies: ROI (return on investment).

You use ROI to calculate which investment is better, usually comparing the one you’re assessing against a “base” investment.

This would mean calculating how much time is “wasted” by keeping the status quo against the estimated gains from the proposed change. Of course, you would need to add the time spent on the change and with that, you would get some number of how much time it would start netting you “profits” on that investment.

If the calculated gains are so small that it would take a long time to take effect, then it might not be worth it.

This one was about time spent on a task, but you can also use other metrics like accessibility, security, and reliability… if you can measure something, then it can be used to justify a change.

Your code is NOT special.

Bruno Noriller — Sat, 27 Jul 2024 18:32:57 +0000

Why aren’t more things open-source? And to remember: open source doesn’t mean free and it doesn’t even need to mean open contributions.

Switzerland mandates open source for the public sector

I saw some news about the mandate issued by Switzerland.

As with anything on the internet, mixed reactions.

For most people outside Switzerland and even there, this doesn’t matter much (sorry Switzerland).

Between that and the launch of Llama 3.1 with the weights and a comprehensive paper on how it was made… it resurfaced a thought that at one point most of us might have had:

Why aren’t more things open-source?

It might be just source-available, closed to contributions and paid. But why not open source? Is your way of doing CRUDs so innovative or special? How about the way you center divs? Is it that good?

Because I’m pretty sure most companies and us “out there” are all using the same libraries for that… which are probably open-source.

Open source already does the “heavy lifting”

The most marvelous pieces of engineering are already made in an open-source manner, some of which make money out of it and some are “unicorns” (companies evaluated at 1B without even being publicly traded on the stock market).

The case for closed-source:

The bad ones

No need to hide credentials from the code if it’s internal.

No need to mind using real people's data in tests and the like.

Easier to obscure shady practices.

The good ones

Not show the “next big thing” and make a surprise. (In this one I would argue that after that, why not open the source?)

Some companies might have something that is their “secret sauce” (intellectual property) that they don’t want others to use or know about. (There are software patents that might cover also this.)

The valid ones

Security concerns about vulnerabilities in the open without enough resources to dedicate to a security team.

Security is hard and closed means a smaller vector of attack, on one hand, you could have people contributing to finding those vectors, but before that, you would probably have someone exploiting then.

The case for open-source

My biggest point is that if we learned to learn from each other, maybe a lot of things could be easier to achieve.

We already joke about “stealing” (copy-pasting) from other sources available, this would just streamline the job. Have a problem? Just show the code, no NDA or its “private code” concerns… it would be all in the open.

The “shoulders of giants” is where all the foundation is built and why we got so far. If more things were closed source back then, then the internet might have been a different place (or maybe it just wouldn’t be, maybe just a niche thing if that).

I know it’s a naive view. And that there are more valid reasons not to open source that I just can’t think of.

A lot of companies probably think they would “lose revenue” because people would just host the solution… coming from a world where people don’t even have “personal pages” anymore and just use whatever centralized solution is more popular.

Or that some company will just steal their code. Not to mention that companies will already steal if they want and just pay a settlement if needed (or just buy it).

Code is just a tool

Think of most social media nowadays: they do the same thing, and they work in the same way.

The code is a detail, and if a “new thing” appears they will just copy it. The code doesn’t matter and the idea is usually simple enough. All that changes is all the shady “how to keep you here for the most time possible” (that they probably don’t want out there).

Even in the “real” tangible world, few things can be truly kept a secret, most are reverse-engineered anyway and copies are out there. But between patents, lawsuits, and “branding” the copies don’t matter and people will still go for the original ones.

Meanwhile… many brands today started out just copying something (some still do), and started adding their own spin to things and improving further than the “originals”. Then they are the copied ones and the cycle repeats.