Forem: Wagner Mattei

The Broken Windows Theory in Software Development: Why You Must Fix Everything, Now

Wagner Mattei — Sat, 08 Mar 2025 04:19:54 +0000

The Broken Windows Theory, originating in criminology, suggests that visible signs of disorder—such as broken windows in a building—lead to further neglect and decay. If one broken window is left unrepaired, soon others will follow, and the entire area will degrade. In software development, the same principle applies: if small problems like code smells or bad processes are ignored, they snowball into larger issues that become significantly harder to fix.

However, there is a dangerous tendency in development teams to assess the urgency of an issue based on how many users it affects or how critical it is to functionality. This is a fundamental mistake. The real impact of ignoring issues is not on users or cost—it is on the mindset of developers and the culture of the team.

The Real Cost of Ignoring Small Issues

A broken window in software isn’t just a minor inconvenience—it changes how developers think about quality. When small issues are ignored, developers begin to internalize a dangerous lesson: "bugs are normal," "sloppy code is acceptable," and "no one really cares about quality." This shift in mindset is far more damaging than any individual bug.

Why This Matters More Than Cost or User Impact

Eroding Developer Standards – When developers see that code smells and small bugs are tolerated, they begin lowering their own standards. Over time, the entire team’s perception of quality deteriorates.
Compounding Technical Debt – The moment you decide an issue is "not urgent," you begin accumulating debt that will cost exponentially more to fix later.
Loss of Code Ownership – When problems are routinely deprioritized, developers lose pride in their work and start seeing the codebase as a decaying structure rather than something worth maintaining.
Cultural Decay – Just like an unmaintained building invites further vandalism, a neglected codebase invites more negligence. Small problems turn into systemic issues, and soon, no one feels responsible for keeping the system clean.

No Issue is Too Small to Fix

It’s common to hear, "this bug affects no users, so we’ll fix it later." But the question isn't about users—it's about the team's standards and how they approach their work.

If a developer sees an issue go unfixed, they assume it’s acceptable to leave other issues unfixed.
If a team tolerates minor errors, they will eventually tolerate major ones.
If a bug sits in the backlog for months, it becomes part of the system’s DNA and is rarely addressed.

Why "Fix Later" is a Lie

One of the most dangerous phrases in software development is “we’ll fix it later.” It almost always means “we will never fix it.” The idea that a team will go back and clean up minor issues after shipping new features is wishful thinking. By the time "later" arrives, priorities have shifted, the backlog has grown, and that small issue has been forgotten—or worse, it has multiplied.

The "Fix Everything Now" Mindset

Instead of classifying issues by their immediate user impact, teams should adopt a mindset where every problem is a priority because its true cost is in how it shapes developer behavior.

How to Enforce This:

Never push code with known issues, no matter how small.
Refuse to accept "low-priority bugs." All bugs are priority bugs.
Do not allow issues to pile up in the backlog. Fix them at the moment they are discovered.
Hold developers accountable for fixing what they break. If an issue is found in code someone wrote, they should be responsible for correcting it immediately.
Be prepared for an initial slowdown.

Transitioning to This Philosophy

If a team has historically allowed broken windows to persist, suddenly enforcing a "fix everything now" approach will feel like a massive shift. Things that used to take a small amount of time will now take longer due to necessary refactors and higher quality standards. This transition phase is critical and must be handled carefully to prevent frustration and pushback.

Expect and Accept an Adjustment Period

At first, development speed will slow down. Fixing longstanding issues and refactoring code that was previously ignored will take time. This is not wasted time; it’s an investment. Teams must acknowledge that while progress may seem slower, the long-term benefits far outweigh the short-term delays.

Shift Mindsets Gradually

Begin by enforcing the rule on new code. Every new feature must meet higher quality standards.
Introduce dedicated cleanup sprints to address existing issues systematically.
Make it clear that quality is now a core engineering principle, not an optional step.

Leadership Buy-In is Essential

This transition cannot succeed unless leadership supports it. If management still pushes for unrealistic deadlines while ignoring quality, developers will feel pressured to cut corners. Leadership must reinforce that fixing issues now is a strategic decision, not an inconvenience.

Early-Stage Startups: The Most Crucial Time for This Philosophy

Once a startup decides to move beyond the POC and build a real product, this is the most crucial moment for enforcing a zero-tolerance policy on broken windows. This is when:

The core team culture is formed.
Development habits are solidified.
The product’s foundation is laid down.

If a startup begins its real product development with a culture that tolerates sloppy work, that attitude will persist and become impossible to change later. This is why early-stage startups must be aggressive in fixing every issue, even when the number of affected users is zero.

Conclusion: No Broken Windows, Ever

There is no such thing as a low-impact bug. The moment an issue is discovered, it must be fixed—not because of its effect on users, but because of its effect on the team.

The best developers don’t just write code—they maintain a culture of zero tolerance for broken windows. The only way to move fast is to never let problems accumulate in the first place. A clean codebase doesn’t just improve performance—it defines the mindset of the team that builds it.

Tudo que voce precisa saber sobre o futuro da engenharia de software

Wagner Mattei — Sat, 27 Jul 2024 19:04:56 +0000

Precisamos encarar o fato de que está difícil conseguir o primeiro emprego na área de programação, e a ansiedade que isso gera em estudantes ou aspirantes a programadores não é saudável. Este artigo é minha tentativa de suavizar essa ansiedade, e sim, eu tenho muita experiência para falar sobre ansiedade (:

Disclaimer: Todo o conteúdo desse artigo pode ou pode não ter sido revelado à mim em um sonho. Levá-lo à serio é responsabilidade única do leitor.

Transformação da carreira de programador

Na segunda metade da década de 2010, a profissão de programador passou por uma transformação significativa, deixando de ser associada exclusivamente ao estereótipo de "nerdola" com poucas habilidades sociais para se tornar uma carreira respeitada e valorizada. O crescente impacto da tecnologia em todas as indústrias fez com que os programadores fossem vistos como profissionais extremamente necessários.

Com a alta demanda por suas habilidades, os salários dos programadores dispararam, o que resultou em uma mudança na percepção da profissão. Continuamos sendo "nerdolas" com poucas habilidades sociais, mas agora com uma remuneração atraente. Essa valorização financeira atraiu uma quantidade absurda de sonhadores que buscavam altos ganhos com menos esforço – um desejo que, embora não deva ser criticado, refletia a nova realidade do mercado de trabalho.

Essa corrida ao ouro atingiu seu ápice durante a pandemia, o mundo mudou, e toda vez que o mundo muda, novos softwares precisam ser desenvolvidos e antigos precisam ser adaptados. As empresas precisaram contratar o maior número possível de programadores para suprir a demanda que esses tempos malucos trouxeram, e a qualidade dessas contratações era um mero detalhe, qualquer programador que soubesse copiar o template de um site seria contratado.

Com o fim da pandemia, as coisas voltaram ao normal e ficamos com os "espólios de guerra". Uma geração de programadores que sabiam escrever código, mas não conheciam as nuances de criar um produto, não tinham noção de gestão de tempo, e frequentemente tinham conflitos com gerentes de projetos porque os memes no twitter os ensinaram a odiá-los, além de uma postura muitas vezes arrogante.

Essas atitudes, juntamente com a queda da demanda, levaram as empresas a evitarem contratar programadores com pouca experiência. Isso tornou cada vez mais difícil iniciar uma carreira na área. Em 2024, a ansiedade entre os estudantes e aspirantes a programador é palpável, o que me desperta um pouco de pena (não muita).

Para aumentar ainda mais essa ansiedade, grandes empresas de tecnologia investiram pesado em ferramentas que utilizam modelos de linguagem para fazer um chat bot que consegue escrever código (não muito bem), ameaçando mais ainda o futuro dos jovens programadores, ou pelo menos é isso que os lideres dessas grandes empresas estão dizendo.

Então eu resolvi escrever essa lista com algumas dicas que eu dou para iniciantes na área de programação, além de alguns insights de como eu vejo o futuro da industria.

Dicas para iniciantes

Não somos pagos para escrever código, e sim resolver problemas

Não sou o primeiro a dizer isso, e certamente não serei o último. Todo guru de tecnologia já mencionou essa ideia, embora possa parecer contra-intuitivo. Afinal, nossa principal habilidade é escrever código, certo? Na verdade, cada linha de código que escrevemos é uma consequência de não termos encontrado uma solução já existente. Escrever código deve ser o último recurso. Caso contrário, estaríamos constantemente reinventando a roda.

Em uma empresa de tecnologia, uma tarefa geralmente envolve um problema a ser resolvido, um motivo para resolvê-lo e uma forma de comprovar que foi resolvido. No entanto, muitos problemas já foram solucionados, e suas soluções estão disponíveis para uso. Por exemplo, digamos que você esteja desenvolvendo um sistema que exige autenticação de usuários com email e senha antes de qualquer operação. Como programador, você poderia passar o dia criando um algoritmo de criptografia para garantir a segurança do sistema. Mas alguém já fez isso antes, e esses algoritmos estão prontos para uso. Seu trabalho não é escrever o algoritmo, mas saber como integrá-lo ao restante do sistema, que também é composto por algoritmos escritos por outras pessoas.

Ninguém liga para o seu esforço

Essa frase também pode parecer contra-intuitiva, já que fomos ensinados que, se nos esforçarmos ao máximo, seremos reconhecidos por isso. Mas a indústria da tecnologia é mais fria do que isso; o que realmente importa são os resultados gerados pelo nosso esforço. Se for para passar a noite escrevendo código, garanta que, às 6:00 da manhã, tudo esteja funcionando.

Isso também vale para quem nunca teve um emprego. A quantidade de horas que você passa estudando sua linguagem de programação preferida ou fazendo exercícios no LeetCode é irrelevante para conseguir um emprego na área. FAÇA ALGUMA COISA. Aprenda Git, crie produtos para você mesmo, coloque-os no seu GitHub e mostre ao mundo o que você fez, independentemente do esforço envolvido. Se você não sabe o que fazer, faça qualquer coisa – não precisa ser algo novo. Refaça o Facebook, crie um aplicativo de calendário, faça um site com fotos do Nicolas Cage, não importa. Você está em busca de uma oportunidade, e os recrutadores querem saber o que você já fez, mesmo que não tenha sido pago por isso.

Nunca confie na palavra de alguém que está sendo pago pela sua confiança

Em uma corrida ao ouro, quem mais ganha dinheiro é quem vende as picaretas! Desde a pandemia, vendedores de cursos surgiram como abutres, prometendo resultados absurdos para seus alunos e explorando o desejo de se tornarem programadores.

Existem bons cursos de programação na internet, e talvez você deva comprar algum, mas se você está no começo e não sabe nem o que estudar, o que compraria? A primeira coisa que você precisa fazer é escolher um caminho dentro da indústria. Talvez seja redes, programação de baixo nível, desenvolvimento web, iOS ou Android, sistemas embarcados, cibersegurança, sistemas operacionais, jogos, computação gráfica, etc. Se você não tem ideia de onde começar, vai comprar um curso de cada área?

Esqueça os cursos por enquanto. Existe uma quantidade infinita de conteúdo gratuito sobre cada uma dessas áreas no YouTube. Você precisa explorar cada uma delas e não ter medo de desistir quando perceber que não gostou. Assim que encontrar uma área que você goste, aí sim pode ser a hora de comprar um curso em sites como Udemy. Mas certifique-se de que você realmente quer investir seu tempo e dinheiro nessa área.

Nunca confie em gurus de tecnologia que querem te vender um estilo de vida. Esses gurus geralmente não têm ideia do que estão falando, e quando têm, podem te alienar de um jeito que será difícil sair. Mantenha sempre uma mente aberta; a tecnologia que você gosta pode não ser ideal para todas as situações. Não tem nada pior que trabalhar com programadores que são apaixonados por uma linguagem ou paradigma de programação.

Destaque-se

Durante a pandemia, não era necessário ser um bom programador para conseguir um emprego. Felizmente, isso mudou, e agora, para ingressar na área, é preciso se destacar. Mas nunca foi tão fácil se destacar como programador júnior. Se você fizer o mínimo, já estará acima da média.

Para se destacar como programador júnior, minha primeira dica é: Assuma sempre que você não sabe de nada. Os cursos que você fez ou seu diploma na faculdade são muito menos relevantes que a experiência dos seus colegas. Você ainda não conquistou o direito de discordar de uma decisão do seu time. Seu papel como iniciante é aceitar as decisões que foram tomadas e fazer o seu melhor, mesmo que isso vá contra aquilo que você aprendeu.

Não confunda "discordar" com "questionar". Sempre questione tudo. Se seu time escolheu Java em vez de C#, você tem o direito de saber o motivo. Independentemente de quem tomou a decisão, você, como funcionário, pode questionar qualquer decisão que impacte seu trabalho, mesmo que venha do time de produto ou gestão. Pergunte, entenda, aceite e faça o necessário.

Aprenda a aprender

Infelizmente, você escolheu uma profissão que exige aprendizado contínuo. Decisões que tomei há seis meses já estão obsoletas, e hoje eu faria escolhas totalmente diferentes, pois muita coisa mudou nesse período.

Se você tentar aprender todas as novas linguagens, frameworks ou paradigmas que surgem, rapidamente descobrirá o significado de burnout. Por isso, é importante aprender a aprender. Por exemplo, eu não sei como funciona o NextJS e nunca o usei, embora seja um dos frameworks mais populares atualmente. Não me preocupo em aprender NextJS agora. Quando eu precisar dele, tirarei duas horas para ler a documentação oficial, entender as diferenças em relação a outros frameworks e decidir se é adequado para o problema que estou tentando resolver.

Consigo fazer isso porque sei como funciona um navegador, entendo o protocolo HTTP e como são feitas as chamadas de API. Sei onde costumam ocorrer gargalos ao servir conteúdo na internet. Não preciso de mais de duas horas lendo a documentação de um framework para entender se ele é ideal para resolver um problema específico. Esse conhecimento adquiri em oito anos como programador, e nada além da experiência poderia ter me proporcionado essa compreensão. No entanto, qualquer programador júnior pode acelerar esse processo estudando os conceitos básicos da computação.

Futuro da Programação

A era de ouro da programação está voltando. Mas, ao contrário da pandemia, quando altos salários e demanda descontrolada prevaleceram, agora estamos retornando a uma era onde o conhecimento profundo e habilidades avançadas serão valorizados.

Desafios Atuais

O hardware não evolui mais tão rapidamente, e a Lei de Moore já não se aplica como antes. A capacidade computacional está chegando a um platô, e as otimizações de hardware estão atingindo seus limites. As empresas de tecnologia estão cientes disso e precisam crescer sem depender apenas dos avanços em hardware.

O Papel dos Programadores

Com a estagnação do hardware, a responsabilidade de otimização está voltando para nós, programadores. Durante a última década, muitos de nós focamos mais em entregar funcionalidades rapidamente do que em escrever código eficiente. Isso era possível devido ao avanço constante do hardware, que permitia que a capacidade computacional aumentasse sem grandes preocupações com a eficiência do código.
No entanto, à medida que o hardware atinge seus limites, essa abordagem se torna insustentável. A otimização de software, que já foi uma habilidade essencial, vai voltar a ser crucial. Programadores terão que:

Escrever códigos eficientes que utilizem bem os recursos disponíveis.
Aproveitar múltiplos núcleos de processamento para melhorar o desempenho.
Gerenciar memória de forma eficaz para evitar desperdícios e gargalos.

Importância da Otimização

A era em que podíamos nos dar ao luxo de ignorar a eficiência do código está acabando. No início dos anos 2000, durante a bolha das .com, o hardware era caro e limitado. Programadores precisavam otimizar cada linha de código para garantir que seus softwares rodassem nas máquinas da época. Com o avanço do hardware e a chegada da cloud, essa necessidade diminuiu. Recursos de computação ficaram mais baratos, e o foco mudou para a entrega rápida de funcionalidades.
Agora, com o hardware atingindo um platô, a otimização volta a ser uma prioridade. Programadores precisarão entender profundamente como seus códigos funcionam em nível de hardware para criar software eficiente.

Preparação para o Futuro

Os programadores que entraram na profissão durante a pandemia muitas vezes não têm a preparação necessária para essa nova era. Portanto, é crucial voltar ao básico:

Estude sistemas operacionais, algoritmos de otimização e notação Big O.
Aprenda sobre gestão de memória e programação multithread.
Entenda que cada decisão de design pode impactar significativamente o desempenho e os custos operacionais.

Conclusão

Prevejo uma nova era da programação, onde meros escrevedores de código perderão espaço na indústria, sendo substituídos por pessoas que realmente sabem o que estão fazendo.
Se você está preocupado com a inteligência artificial tomando seu emprego, entenda que a função do computador sempre foi substituir empregos. Há 100 anos, o termo "computador" se referia a uma pessoa que realizava cálculos. Com o surgimento do computador eletrônico, esses empregos foram substituídos, mas uma infinidade de novos empregos foi criada. Isso acontece há mais de 100 anos e continuará no futuro.

Não posso garantir que sua linguagem de programação favorita será relevante em 10 anos, provavelmente não será. Por isso, não se apegue a ela. Estude os conceitos básicos da computação, da matemática e a teoria por trás de cada algoritmo que usamos no dia a dia.
Acima de tudo, se você está começando na área agora, seja humilde e reconheça sua incapacidade inicial de resolver problemas complexos. Com a atitude certa, sempre haverá alguém disposto a ajudar.

CloudGaming Part 2: That code was terrible lol

Wagner Mattei — Fri, 27 Oct 2023 23:46:00 +0000

A few weeks ago, I wrote my first post about Cloud Gaming. My goal was to learn how cloud gaming works. I made a simple game that takes player moves, makes frames of the game, and sends them to the player with WebRTC video.

I did it! I made a snake game and it worked.

But, I only called it a success because a snake game doesn't need more than 10fps.

I knew that a bigger game would have delay issues and might not run smoothly at 30 or 60fps.

So, I chose to make a new game.

TLDR

https://github.com/wmattei/go-snake/tree/dev-to-10-27-2023

The Latency Check Game

I aimed to create an animation where a red rectangle would appear at the player's current mouse position. This meant every change in the mouse position had to be sent to our server. Then, 60 times a second, I needed to update the game state, render a new frame, encode that frame to h264, and stream it via WebRTC.

Here's the outcome with the architecture I used:

As evident, the results were poor. I was losing many frames as I couldn't deliver them quickly enough.

The bottleneck

I began timing each step of my process to identify the slow parts and see how to speed them up.

The main issue was the encoder, which took about 40ms for each frame. Here's the code I had in place:

func StartEncoder(pixelCh chan []byte, encodedFrameCh chan []byte, windowWidth, windowHeight int) {
    ffmpegCommand = fmt.Sprintf(ffmpegBaseCommand, windowWidth, windowHeight, constants.FPS)
    for {
        rawRGBDataFrame, ok := <-pixelCh
        if !ok {
            break
        }
        encodedData, err := encodeFrame(rawRGBDataFrame, windowWidth, windowHeight)
        logutil.LogFatal(err)

        encodedFrameCh <- encodedData
    }
}

const ffmpegBaseCommand = "ffmpeg -hide_banner -loglevel error -f rawvideo -pixel_format rgb24 -video_size %dx%d -framerate %d -i pipe:0 -c:v libx264 -preset ultrafast -tune zerolatency -f h264 pipe:1"

var ffmpegCommand string

func encodeFrame(rawFrame []byte, windowWidth, windowHeight int) ([]byte, error) {
    cmd := exec.Command("bash", "-c", ffmpegCommand)
    cmd.Stderr = os.Stderr

    inPipe, err := cmd.StdinPipe()
    logutil.LogFatal(err)
    outPipe, err := cmd.StdoutPipe()
    logutil.LogFatal(err)

    if err := cmd.Start(); err != nil {
        logutil.LogFatal(err)
        return nil, err
    }

    _, err = inPipe.Write(rawFrame)
    if err != nil {
        return nil, err
    }

    inPipe.Close()

    encodedData, err := readH264NALUnits(outPipe)
    if err != nil {
        return nil, err
    }

    err = cmd.Wait()
    if err != nil {
        return nil, err
    }

    return encodedData, nil
}

func readH264NALUnits(outPipe io.Reader) ([]byte, error) {
    h264, err := h264reader.NewReader(outPipe)
    if err != nil {
        return nil, fmt.Errorf("failed to create H.264 reader: %v", err)
    }

    var data []byte
    var spsAndPpsCache []byte

    for {
        nal, h264Err := h264.NextNAL()
        if h264Err == io.EOF {
            break
        } else if h264Err != nil {
            return nil, fmt.Errorf("error reading H.264 NAL: %v", h264Err)
        }

        nal.Data = append([]byte{0x00, 0x00, 0x00, 0x01}, nal.Data...)
        if nal.UnitType == h264reader.NalUnitTypeSPS || nal.UnitType == h264reader.NalUnitTypePPS {
            spsAndPpsCache = append(spsAndPpsCache, nal.Data...)
            continue
        } else if nal.UnitType == h264reader.NalUnitTypeCodedSliceIdr {
            nal.Data = append(spsAndPpsCache, nal.Data...)
            spsAndPpsCache = []byte{}
        }

        data = append(data, nal.Data...)
    }

    return data, nil
}

It was lagging because for every frame, we initiated a new ffmpeg process, wrote the raw frame to an input pipe, and read the result from an output pipe. Using that output, we gathered all the H264 NAL units to set up a batch of data for WebRTC transmission.

I basically build the encoder in the worst possible way lol.

The First Solution

The first fix I attempted was encoding several frames at once by starting 4 new go routines to grab raw frames from the pixelCh. While this made the game run quicker, it mainly just hid the problem that the encoder was too slow.

The final solution

After researching how ffmpeg and pipes function, I discovered that I could initiate an ffmpeg process at the start of the function. Then, I could continuously "feed" the command with data by writing raw frames to the input pipe in a separate go routine. Concurrently, I'd launch another goroutine to retrieve the compressed frames from the output pipe and forward them to our streamer, bypassing the need to read the NAL.

Our brand new encoder can now encode our Latency Check™ game frames in less than 2ms.

And here is the code:

func (e *Encoder) Start() {

    debug := "-hide_banner -loglevel error"
    if constants.FFMPEG_BANNER {
        debug = ""
    }

    ffmpegCommand := fmt.Sprintf(ffmpegBaseCommand, debug, e.windowWidth, e.windowHeight, constants.FPS, constants.FPS)
    cmd := exec.Command("bash", "-c", ffmpegCommand)
    cmd.Stderr = os.Stderr
    inPipe, err := cmd.StdinPipe()
    logutil.LogFatal(err)
    outPipe, err := cmd.StdoutPipe()
    logutil.LogFatal(err)
    err = cmd.Start()
    logutil.LogFatal(err)

    e.wg.Add(2)

    go e.writeToFFmpeg(inPipe)
    go e.streamToWebRTCTrack(outPipe)

    go func() {
        _, ok := <-e.closeSignal
        if !ok {
            e.markAsClosed()
            fmt.Println("Closing encoder")

            cmd.Wait()
            e.wg.Wait()
            close(e.encodedFrameCh)
        }
    }()

}

func (e *Encoder) writeToFFmpeg(inPipe io.WriteCloser) {
    defer e.wg.Done()

    for canvas := range e.canvasCh {
        if e.isClosed() {
            return
        }
        select {
        case <-e.encodedFrameCh: // Check if there's a backlog.
            e.debugger.ReportDroppedFrame()
            continue
        default:
            _, err := inPipe.Write(canvas.Data)
            logutil.LogFatal(err)
        }
    }

    inPipe.Close()
}

func (e *Encoder) streamToWebRTCTrack(outPipe io.Reader) {
    defer e.wg.Done()

    buf := make([]byte, 1024*8)
    for {
        if e.isClosed() {
            return
        }
        timestamp := time.Now()
        n, err := outPipe.Read(buf)
        if err == io.EOF {
            break
        } else if err != nil {
            logutil.LogFatal(fmt.Errorf("error reading from FFmpeg: %v", err))
            continue
        }

        e.encodedFrameCh <- &webrtcutil.Streamable{Data: buf[:n], Timestamp: timestamp}
    }
}

Here is the final result:

Some people call this "Blazingly fast".

Never be confident

I was genuinely happy at that point (doesn't happy often). I believed that any game could be developed using this updated structure. Now, the game's speed was only limited by the time it took to render a frame. I even made the frame renderer parallel and arrived at this final setup:

It was time to test it and begin creating actual game elements, like gravity.

I aimed to design a basic game where, every time a player clicked the page, a new ball would appear. This ball would continue bouncing until it ran out of energy.

I build the logic (which is not very complex):


func NewBall(x, y int, radius float64, ground int, color *artemisia.Color) *Ball {
    return &Ball{
        Radius: radius,
        Position: Position{
            X: x,
            Y: y,
        },
        Ground:     ground,
        Elasticity: 0.8,
        Color:      color,
    }
}

const gravity = 1

func (b *Ball) Update(dt int64) {
    b.Velocity.Y += float64(gravity)
    b.Position.X += int(b.Velocity.X)
    b.Position.Y += int(b.Velocity.Y)
    if b.Position.Y >= b.Ground {
        b.Position.Y = b.Ground
        b.Velocity.Y = -(float64(b.Velocity.Y) * b.Elasticity)
    }
    if b.Position.Y == b.Ground && int(b.Velocity.Y) == 0 && b.StoppedAt == nil {
        now := time.Now()
        b.StoppedAt = &now
    }

    if b.StoppedAt != nil && time.Since(*b.StoppedAt) > 3*time.Second {
        b.IsDead = true
    }
}

I would even remove the ball after it was dead!

And this is the result I got:

My "masterpiece" of an engine choked on just two balls, and my spirits plummeted again.

After some research, I found out something pretty basic: the more detailed the frame, the longer it takes to encode. And circles? They're hard for the CPU to handle.

Luckily, there's a simple answer: "Hardware accelerated encoding." ffmpeg can do this, so I just changed my command to use a different method to convert my raw frames to H264.

Here is the update command:

ffmpeg -re -f rawvideo -pixel_format rgb24 -video_size %dx%d -framerate %v -r %v -i pipe:0 -pix_fmt yuv420p -c:v h264_videotoolbox -f h264 pipe:1

The h264_videotoolbox encoder uses the GPU on Mac OS. For other operating systems, you might need to install different libraries.

Here is my Bouncing Balls game with hardware accelerated encoding:

Let them balls bounce!

Yeah, now I am not happy because I learned my lesson, but this is a smooth real time animation driven by user input and its really close to an architecture of a real game.

Conclusion

It works now, it runs at 60fps, it uses hardware accelerated encoding, and I think we can start thinking in an actual game. Stay tuned for the next posts where we will build a real game.

How I think CloudGaming works + Building a Game with GO and Streaming the frames via WebRTC.

Wagner Mattei — Sat, 16 Sep 2023 19:35:49 +0000

Introduction

Cloud gaming has been gaining momentum in recent years, with companies like Google, Microsoft, and Nvidia investing heavily in the technology. But what is cloud gaming, exactly? In short, it's a way to play video games without needing a powerful gaming computer or console. Instead, the game is run on a remote server and streamed to your device over the internet.

You may notice that the title of this post is not particularly assertive, and you would be correct. My objective is not to teach how to create a game that renders on the server, but rather to speculate on how this might be achieved.

While this technology has existed for a long time, there isn't much information available about it. Big players like Google or Microsoft (Xbox Cloud) offer only a high-level description of how it works, and likely use custom protocols and mixed rendering between server and client to achieve their goals.

So, I decided to give it a try!

TLDR

https://github.com/wmattei/go-snake/tree/dev-to-post

Defining the goals

My main goal for this project was to create a simple game that can be played with very few commands, and does not require a high frame-rate. Therefore, I chose to create the Snake Game.

Artifacts of a server side rendered game

After deciding which game to build, I conducted some research to understand how a cloud game should work. As a result, I came up with a few artifacts.

Commands channel

We require a method for sending commands from the client to the server. For instance, if the player presses the "Up Arrow" key, we want to send a command to the server to make the snake go up.

Game loop

Games typically have a game loop, which is an infinite loop that updates the game state based on new commands, or if no new commands are received, the game state still needs to update our snake's position.

Game Renderer

We need to render a new frame to display it back to the client. In our case, we have fixed the FPS to 10 frames per second.

Encoding

The rendered frame will be in a raw RGBA format, which is simply a matrix of pixels, with each pixel containing information about the red, green, and blue colors. However, this format is not suitable for streaming as it is not compressed. To stream the frames, we need to encode them into a new format. In this case, we have chosen h.264.

Streaming

Now that we have the frames in the correct format, we need to stream them back to the client so that they can be displayed in a simple HTML 5 <video> tag.

WebRTC

Two of our game artifacts require communication between the client and the server: the "commands channel" and the "streaming". Fortunately, we have an effective communication protocol that can solve both problems: WebRTC.

WebRTC, which stands for Web Real-Time Communication, is an open-source project that enables real-time communication of audio, video, and data directly between web browsers and mobile applications. It allows for peer-to-peer communication without the need for additional plugins or downloads.

Although WebRTC is mostly used for client-to-client connections, it can also be used as a server-client communication tool. It can be thought of as a websocket on steroids because it has two-way data channels, as well as video and audio tracks that we can use to send our frames to the client.

Encoding

Encoding is one of the most challenging aspects of our game. We need to convert our raw RGBA frames into a compressed format to stream back to the browser as quickly as possible, ensuring low latency.

Let's code

The HTML:

First, we create a basic HTML page that will serve as the starting point for the player. Most importantly, we include the video tag, which will display the game itself.

<!DOCTYPE html>
<html>
  <head>
    <script src="./index.js"></script>
    <link rel="preconnect" href="https://fonts.googleapis.com" />
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
    <link
      href="https://fonts.googleapis.com/css2?family=Press+Start+2P&display=swap"
      rel="stylesheet"
    />
    <style>
      * {
        margin: 0;
        padding: 0;
        background-color: #1a1a1a;
      }
      #welcome_panel {
        display: flex;
        flex-direction: column;
        align-items: center;
        color: white;
        height: 100vh;

        font-family: "Press Start 2P", cursive;
      }

      #welcome_panel h1 {
        font-size: 2rem;
        margin: 2rem;
      }
      #welcome_panel p {
        text-align: center;
        line-height: 2.5rem;
      }

      #start_btn {
        padding: 1rem 2rem;
        border: none;
        border-radius: 0.5rem;
        background-color: #4caf50;
        color: white;
        font-size: 1.5rem;
        cursor: pointer;
        margin-top: 10rem;
      }
      #start_btn:hover {
        background-color: #3e8e41;
      }

      #video {
        display: none;

        width: 100vw;
        height: 100vh;
      }
    </style>
  </head>
  <body>
    <div id="welcome_panel">
      <h1>Welcome to my Snake Game</h1>
      <p>
        The page you are seeing right now is a simple HTML rendered by the
        browser.
      </p>
      <p>
        But as soon as you press the Start button, this whole page will turn
        into a single <b>HTML 5 video</b> tag
      </p>
      <button id="start_btn">Start</button>
    </div>
    <video id="video" autoplay></video>
  </body>
</html>

I also added a bit of CSS to make it look a bit better:

JavaScript

Since we are using WebRTC, we need to establish a connection with our other pair (our Golang server) and capture the player's commands to play the game.

First, let's initialize a few variables that we will use throughout the code. We will explain the meaning of each later in this post.

let dataChannel;
let peerConnection;
let signalingChannel;
let videoElement;
let welcomePanelElement;

window.onload = async function () {
  const startBtn = document.getElementById("start_btn");
  startBtn.addEventListener("click", start);

  videoElement = document.getElementById("video");
  welcomePanelElement = document.getElementById("welcome_panel");

  keyBindings();
};

function keyBindings() {
  document.addEventListener("keydown", (event) => {
    // Send command based on key press
    if (event.key === "ArrowUp") {
      sendCommand("UP");
    } else if (event.key === "ArrowDown") {
      sendCommand("DOWN");
    } else if (event.key === "ArrowLeft") {
      sendCommand("LEFT");
    } else if (event.key === "ArrowRight") {
      sendCommand("RIGHT");
    }
  });
}

In this section, we define several variables and set the HTML element variables with their reference to the DOM. Additionally, when the window loads, we define the key bindings - in our case, we bind the arrow keys - and call the function sendCommand on each key press. We will explore the sendCommand function later on.

Following this, we must manage the WebRTC negotiation involving our counterpart and the signaling server. In our scenario, the counterpart and the signaling server are synonymous since we aren't constructing a standard client-to-client communication system.

Even though all game communication happens through WebRTC, we first need to share some details with the signaling server using websockets to set up the WebRTC connection.

async function start() {
  // Initialize signaling channel
  signalingChannel = new WebSocket("ws://localhost:4000/ws");

  // Event handler for signaling channel open
  signalingChannel.addEventListener("open", async () => {
    // Initialize peer connection and data channel
    peerConnection = new RTCPeerConnection();
    createDataChannel(peerConnection);

    peerConnection.onicecandidate = handleIceCandidateEvent;
    peerConnection.ontrack = handleTrackEvent;

    // Create and send offer
    const offer = await peerConnection.createOffer({
      offerToReceiveVideo: true,
    });
    await peerConnection.setLocalDescription(offer);
    signalingChannel.send(JSON.stringify({ type: "offer", data: offer.sdp }));
  });

  // Event handler for signaling channel messages
  signalingChannel.addEventListener("message", async (event) => {
    const message = JSON.parse(event.data);

    if (message.type === "ice") {
      handleIceMessage(message.data);
    }

    if (message.type === "answer") {
      handleAnswerMessage(message.data);
    }
  });
}

function createDataChannel(peerConnection) {
  // Create a data channel for commands
  dataChannel = peerConnection.createDataChannel("commandsChannel");

  dataChannel.onerror = (error) => {
    console.log("Error on data channel:", error);
  };

  dataChannel.onclose = () => {
    setTimeout(() => {
      videoElement.style.display = "none";
      welcomePanelElement.style.display = "flex";
    }, 1000);
  };
}

As shown in the code above, we set up a websocket connection to our server (which will be built later).

Once the connection is established, we handle various WebRTC-related tasks:

Initialize a new RTCPeerConnection
Establish a data channel for sending commands
Set up a handler for when new ice candidates are gathered
Set up a handler for when a track is added to the connection
Generate and send an offer to the signaling server through our websocket connection

Additionally, we implement a listener for incoming messages from the websocket. We anticipate two message types: one for receiving the WebRTC answer and another for obtaining the ICE candidates of our counterpart.

See below the implementation of our additional functions:

const handleTrackEvent = (event) => {
  if (event.track.kind === "video") {
    // Display video and hide welcome panel
    videoElement.style.display = "block";
    welcomePanelElement.style.display = "none";

    // Set video stream
    videoElement.srcObject = event.streams[0];
  }
};

const handleIceCandidateEvent = (event) => {
  if (event.candidate) {
    // Send ICE candidate
    signalingChannel.send(
      JSON.stringify({ type: "ice", data: event.candidate.candidate })
    );
  }
};

const handleIceMessage = async (iceData) => {
  const iceCandidate = new RTCIceCandidate({
    ...iceData,
    sdpMLineIndex: 0,
    sdpMid: "0",
  });
  try {
    await peerConnection.addIceCandidate(iceCandidate);
  } catch (error) {
    console.log("Error adding ICE candidate:", error);
  }
};

const handleAnswerMessage = async (answerData) => {
  const remoteDescription = new RTCSessionDescription({
    sdp: answerData,
    type: "answer",
  });
  await peerConnection.setRemoteDescription(remoteDescription);
};

The handleTrackEvent function is responsible for updating the HTML page's view, hiding the welcome panel, and displaying the video tag. It also assigns the new track added by our WebRTC counterpart (our Go server) to the video element's source (src).

The handleIceCandidateEvent function transmits the collected ICE candidates from the local WebRTC connection to our counterpart via the WebSocket connection.

The handleIceMessage function receives ICE candidates from our counterpart's WebRTC pair and sets them for the local pair.

The handleAnswerMessage function sets the remoteDescription using the SDP file received as a WebSocket message.

Lastly, we define our function for sending commands through the WebRTC channel once the negotiation is completed.

function sendCommand(command) {
  dataChannel.send(JSON.stringify({ type: "command", data: command }));
}

Let's code our server!

In WebRTC negotiations, using a signaling server to exchange SDP definitions and ICE candidates between pairs is a common practice. And that's exactly what we did using a WebSocket as you saw earlier.

In our specific case, we'll develop a Golang server that serves as both a signaling server and one of the WebRTC pairs.

In this post, I'll omit the detailed implementation of the signaling server and the Golang portion of the negotiation since it closely resembles what we've already covered in JavaScript.

The starting point of our server, starts a simple HTTP server and listens for request on /ws

func main() {
    http.HandleFunc("/ws", handleWebsocketConnection)
    err := http.ListenAndServe(fmt.Sprintf(":%d", port), nil)
    snake_errors.HandleError(err)
}

snake_errors.HandleError is just an utility module that I created to PANIC on any error (lol)

Here is the handleWebsocketConnection code:

func handleWebsocketConnection(w http.ResponseWriter, r *http.Request) {
    peerConnection, err := snake_webrtc.CreateAndNegotiatePeerConnection(w, r)
    snake_errors.HandleError(err)

    track := peerConnection.GetSenders()[0].Track().(*webrtc.TrackLocalStaticSample)
    fmt.Println("Peer connection established")

    handleDataChannel(peerConnection, track)
}

The entire WebRTC signaling and negotiation process is encapsulated in snake_webrtc.CreateAndNegotiatePeerConnection. I mentioned that I wouldn't delve into the specifics in this post, but feel free to explore how it operates in the project's GitHub repository. In fact, the function does way more than just creating and negotiating a peer connection, but best practices are beyond the scope of this post.

Once we have our peerConnection connected to the other pair (our javascript code) and we already have a created track to send our videos, we will start listening to our data channel:

func handleDataChannel(peerConnection *webrtc.PeerConnection, track *webrtc.TrackLocalStaticSample) {
    peerConnection.OnDataChannel(func(dataChannel *webrtc.DataChannel) {
        fmt.Println("Data channel established")
        closeSignal := make(chan bool)

        commandChannel := make(chan string)
        gameStateCh := make(chan *game.GameState)
        pixelCh := make(chan []byte)
        encodedFrameCh := make(chan []byte)

        gameLoop := game.NewGameLoop(&game.GameLoopInit{CommandChannel: commandChannel, GameStateChannel: gameStateCh, CloseSignal: closeSignal})
        go gameLoop.Start()

        go renderer.StartFrameRenderer(gameStateCh, pixelCh)
        go encoder.StartEncoder(pixelCh, encodedFrameCh)
        go stream.StartStreaming(encodedFrameCh, track)

        go handleChannelClose(dataChannel, peerConnection, gameStateCh, commandChannel, pixelCh, encodedFrameCh, closeSignal)

        dataChannel.OnMessage(func(msg webrtc.DataChannelMessage) {
            handleDataChannelMessage(msg, commandChannel)
        })
    })
}

Once a new data channel connection is established, we kickstart our game loop, renderer, encoder, and streamer. We also set up a listener for incoming messages, which will contain players' commands.

func handleDataChannelMessage(msg webrtc.DataChannelMessage, commandChannel chan string) {
    var message snake_webrtc.Message
    err := json.Unmarshal(msg.Data, &message)
    if err != nil {
        fmt.Println("Error unmarshalling message:", err)
        return
    }
    if message.Type != "command" {
        fmt.Println("Channel used for wrong message type:", message.Type)
        return
    }

    fmt.Println("Received command:", message.Data.(string))
    commandChannel <- message.Data.(string)
}

The handleDataChannelMessage decodes new messages coming from our counterpart pair and if the message has the correct Type, it will send the commands to the commandChannel

The game loop.

As depicted earlier, we initiated a game loop responsible for computing the game state in each iteration. This game loop must respond to new commands issued by the player, as well as update the game when no commands are sent. For instance, if the snake is moving left and no new commands are received, the snake should continue in that leftward direction.

type GameLoop struct {
    gameState      *GameState
    commandChannel chan string
    gameStateCh    chan *GameState
    closeSignal    chan bool
    frameTicker    *time.Ticker
}

type GameLoopInit struct {
    CommandChannel   chan string
    GameStateChannel chan *GameState
    CloseSignal      chan bool
}

func NewGameLoop(options *GameLoopInit) *GameLoop {
    return &GameLoop{
        gameState:      NewGameState(constants.ROWS, constants.COLS),
        commandChannel: options.CommandChannel,
        gameStateCh:    options.GameStateChannel,
        closeSignal:    options.CloseSignal,
        frameTicker:    time.NewTicker(time.Second / constants.FPS),
    }
}

func (gl *GameLoop) Start() {
    defer gl.frameTicker.Stop()

    for {
        select {
        case command := <-gl.commandChannel:
            gl.handleCommand(command)

        case <-gl.frameTicker.C:
            gl.updateGameState(nil)
            gl.gameStateCh <- gl.gameState
        }
    }
}

func (gl *GameLoop) Close() {
    gl.closeSignal <- true
}

func (gl *GameLoop) handleCommand(command string) error {
    return gl.updateGameState(&command)
}

func (gl *GameLoop) updateGameState(command *string) error {
    gameOver := !gl.gameState.handleCommand(command)
    if gameOver {
        gl.Close()
    }
    return nil
}

This code manages updates to the game state and sets a flag (closeSignal) to true upon detecting a game over.

I'll bypass the detailed implementation of the game state in this post, as it exceeds the intended scope. Feel free to examine the complete implementation on GitHub.

The game renderer.

The game renderer's role is to receive the game state and generate a frame based on that state. For instance, if the game state indicates that the snake is 5 squares long, the renderer should display the corresponding snake colors in the pixels representing those 5 squares. This rendering process occurs each time the game state is updated.

var (
    snakeHeadColor = color.RGBA{R: 255, G: 0, B: 0, A: 255}
    snakeBodyColor = color.RGBA{R: 255, G: 255, B: 255, A: 255}
    foodColor      = color.RGBA{R: 0, G: 255, B: 0, A: 255}
)

const bytesPerPixel = 3 // RGB: 3 bytes per pixel

func drawRectangle(img *image.RGBA, min, max image.Point, col color.RGBA) {
    for x := min.X; x < max.X; x++ {
        for y := min.Y; y < max.Y; y++ {
            img.Set(x, y, col)
        }
    }
}

func convertRGBAtoRGB(img *image.RGBA) []byte {
    width, height := img.Rect.Dx(), img.Rect.Dy()
    rawRGBData := make([]byte, bytesPerPixel*width*height)

    idx := 0
    for y := 0; y < height; y++ {
        for x := 0; x < width; x++ {
            pixel := img.RGBAAt(x, y)
            rawRGBData[idx] = pixel.R
            rawRGBData[idx+1] = pixel.G
            rawRGBData[idx+2] = pixel.B
            idx += bytesPerPixel
        }
    }

    return rawRGBData
}

func StartFrameRenderer(gameStateCh chan *game.GameState, pixelCh chan []byte) {
    for {
        gameState := <-gameStateCh
        if gameState == nil {
            break
        }

        img := image.NewRGBA(image.Rect(0, 0, constants.FRAME_WIDTH, constants.FRAME_HEIGHT))
        matrix := gameState.GetMatrix()

        for y := 0; y < len(matrix); y++ {
            for x := 0; x < len(matrix[0]); x++ {
                rectMin := image.Point{X: x * constants.CHUNK_SIZE, Y: y * constants.CHUNK_SIZE}
                rectMax := image.Point{X: rectMin.X + constants.CHUNK_SIZE, Y: rectMin.Y + constants.CHUNK_SIZE}
                switch matrix[y][x] {
                case 1:
                    drawRectangle(img, rectMin, rectMax, snakeHeadColor)
                case 2:
                    drawRectangle(img, rectMin, rectMax, snakeBodyColor)
                case 3:
                    drawRectangle(img, rectMin, rectMax, foodColor)
                }
            }
        }

        rawRGBData := convertRGBAtoRGB(img)
        pixelCh <- rawRGBData
    }
}

As observed in the provided code, we transmit raw RGB data to a pixel channel. This RGB data must undergo encoding into h.264, as we previously discussed in this post.

Encoding.

We decided to use ffmpeg to do our encoding:

const ffmpegCommand = "ffmpeg -hide_banner -loglevel error -f rawvideo -pixel_format rgb24 -video_size %dx%d -framerate %d -i pipe:0 -c:v libx264 -preset ultrafast -tune zerolatency -f h264 pipe:1"

func StartEncoder(pixelCh chan []byte, encodedFrameCh chan []byte) {
    for {
        rawRGBDataFrame, ok := <-pixelCh
        if !ok {
            // Channel closed, exit the loop
            break
        }

        cmd := exec.Command("bash", "-c", fmt.Sprintf(ffmpegCommand, constants.FRAME_WIDTH, constants.FRAME_HEIGHT, constants.FPS))
        cmd.Stderr = os.Stderr

        // Create a pipe for input and output
        inPipe, err := cmd.StdinPipe()
        snake_errors.HandleError(err)
        outPipe, err := cmd.StdoutPipe()
        snake_errors.HandleError(err)

        // Start the command
        if err := cmd.Start(); err != nil {
            snake_errors.HandleError(err)
            continue
        }

        // Write raw RGB data to the input pipe
        _, err = inPipe.Write(rawRGBDataFrame)
        snake_errors.HandleError(err)

        // Close the input pipe to indicate no more input
        inPipe.Close()

        // Read H.264 NAL units from the output pipe and send to the channel
        encodedData, err := readH264NALUnits(outPipe)
        snake_errors.HandleError(err)

        // Wait for the command to finish
        err = cmd.Wait()
        snake_errors.HandleError(err)

        // Send the encoded data to the channel
        encodedFrameCh <- encodedData
    }
}

func readH264NALUnits(outPipe io.Reader) ([]byte, error) {
    h264, err := h264reader.NewReader(outPipe)
    if err != nil {
        return nil, fmt.Errorf("failed to create H.264 reader: %v", err)
    }

    var data []byte
    var spsAndPpsCache []byte

    for {
        nal, h264Err := h264.NextNAL()
        if h264Err == io.EOF {
            // Finished sending frames
            break
        } else if h264Err != nil {
            return nil, fmt.Errorf("error reading H.264 NAL: %v", h264Err)
        }

        nal.Data = append([]byte{0x00, 0x00, 0x00, 0x01}, nal.Data...)
        if nal.UnitType == h264reader.NalUnitTypeSPS || nal.UnitType == h264reader.NalUnitTypePPS {
            spsAndPpsCache = append(spsAndPpsCache, nal.Data...)
            continue
        } else if nal.UnitType == h264reader.NalUnitTypeCodedSliceIdr {
            nal.Data = append(spsAndPpsCache, nal.Data...)
            spsAndPpsCache = []byte{}
        }

        // Append NAL unit data to the result
        data = append(data, nal.Data...)
    }

    return data, nil
}

The encoding code leverages FFmpeg, a powerful multimedia processing tool, to encode raw RGB frames into the H.264 video format. The RGB frames, representing individual frames of the game, are continuously fed into a FFmpeg command that utilizes specific settings for ultrafast encoding and zero-latency tuning. The RGB frames are processed through the FFmpeg command, producing H.264-encoded frames that are then sent for further use or streaming. This process is crucial for efficiently compressing and transmitting video frames in real-time applications such as live streaming or video communication, ensuring optimal performance and reduced bandwidth consumption.

Streaming

After all our frames are converted into a streamable format, we will now send those frames to our WebRTC track that we opened before.

func StartStreaming(encodedFrameCh chan []byte, videoTrack *webrtc.TrackLocalStaticSample) {
    for {
        encodedFrame := <-encodedFrameCh
        if encodedFrame == nil {
            break
        }
        videoTrack.WriteSample(media.Sample{Data: encodedFrame, Duration: time.Second / constants.FPS, Timestamp: time.Now()})
    }
}

Every new frame gets written to the track along with information regarding duration and Timestamp.

Conclusion

With all our steps finished, we should have a running Snake Game, rendered in the server and streamed to the client on a very low latency:

Final considerations:

The goal of this post, as mentioned before, is not to be a tutorial on how to build a cloud game, rather, it is a report on how I did it.

In a real time cloud game, I would never mix the game engine with the cloud game server, which is what we are doing here. Instead, I would use some sort of “frame capturing” approach and my server only responsibility would be to stream those frames back to the client.

I deeply encourage criticism on this post, as I don't see much content of Cloud Gaming in the developer community.

Thanks for reading! Bye!!

Javascript and the meta-programming hell

Wagner Mattei — Mon, 26 Jun 2023 21:01:18 +0000

Meta-programming can have multiple definitions, it is often defined with "Code Generation" or "Reflection". But I like to define it simply as "Code you didn't write, but still makes your life miserable".

Now, let’s delve into the technical aspects. In programming, we encounter implicit and explicit languages and frameworks, each offering its own advantages and disadvantages. This article aims to explore these approaches in detail, providing insights into their characteristics.

Explicit Code

Explicit code is an approach where the developer’s intent is clear, and the developer is in control of what the code they write is doing. It provides the developer with more flexibility and freedom over variables, data, and flows.

However…

In the vast expanse of freedom lies the weighty mantle of responsibility, for as the boundaries of choice widen, so too must the conscience and accountability of the liberated soul. — Chat GPT

Explicit code is often more verbose and requires the developer to possess a deeper understanding of the language or framework being used, as well as a deeper understanding of general software engineering concepts. This is why explicit languages or frameworks may not appeal to new programmers, as they typically prefer to see quick results with as few lines of code as possible.

Implicit Code

Implicit code is the exact opposite. The developer only needs to focus on the business logic, while most of the complexity is abstracted by the language or framework itself. Typically, languages and frameworks that employ this approach are highly opinionated when it comes to handling data and managing flows. It also results in less code for the developer to maintain, which can be beneficial in many cases. However, there are instances when this approach may not be ideal.

The Spectrum

Explicit and implicit are not binary definitions; they exist along a spectrum. The image below illustrates where certain languages fall on this spectrum, based on their underlying philosophy rather than the specific implementations within each language or framework. In fact, the majority of languages are flexible enough to accommodate both approaches.

Meta-programming hell

In the image above, “meta-programming hell” is depicted as a point where the code becomes so implicit that the engineer loses control over the engineering process. At this stage, the engineer’s sole responsibility is to fill in functions or files, hoping that the language will handle the underlying tasks.

The Javascript of 2023

The JavaScript language embodies a more implicit approach due to its characteristics such as dynamic typing, loose syntax, and automatic type coercion. However, the evolving ecosystem in 2023 is pushing the language toward the realm of meta-programming hell.

While this article aims to discuss the ecosystem as a whole, it is important to highlight Vercel’s Next.js framework. Why? Because Next.js is currently garnering significant attention and Vercel plays a crucial role in steering the future of the JavaScript ecosystem.

Next.js has propelled JavaScript to a critical juncture, approaching a point of no return. It has almost transformed into a low-code tool, thereby diminishing the developer’s control and contributing to the implicit nature of the language.

Why is Next.js a meta-programming hell?

Imagine you have written a JavaScript function. As the author, you should have a clear understanding of what the function does. However, the actual execution result of YOUR function can depend on several factors:

What is the name of your function?
What is the name of the file where your function was written?
Does the file name include a "(" or a "[" or even a "…([{" ?
What is the name of the folder where the file stays?
What version of Next.js are you using?
Does the file start with 'use client' or 'use server'?

Even if you believe you are in control of all these factors, what happens when someone new joins your team and needs to fix a bug in the function YOU wrote? They can’t simply interpret the code. First, they have to familiarize themselves with the specific version of Next.js being used.

Furthermore, what if Vercel decides to release a new major version and your team decides to update? The routing system might have undergone significant changes (which it had in the past), requiring your entire team to re-learn Next.js.

The appeal of implicit code stems from the desire to code faster, deliver quick results, and create attractive landing pages in a short time. However, when it comes to engineering real products, we must consider consistency and code readability. When code becomes so complex that nobody understands what’s happening, the community often suggests, “Rewrite your entire codebase in .” But is that really the direction we want the ecosystem to go? It’s important to think about the long-term implications and find a balance between speed and maintainability. We need discussions within the community to ensure that we prioritize both productivity and the quality of our code in the long run.

Conclusion

There isn’t much to conclude from this statement. Meta-programming is indeed a valid approach that can be sensible in certain cases. However, I have a sense that the programming community has not thoroughly questioned the trajectory the ecosystem has taken. Consequently, individuals who profit from offering coding courses, often referred to as “code gurus,” are inadvertently promoting the practice of junior developers writing code without having complete control over its functionality.

It is important to foster a culture of critical thinking and accountability within the programming community. While meta-programming has its merits, it is equally crucial for developers, regardless of their level of experience, to possess a comprehensive understanding of the code they write. By maintaining control over their code and actively seeking comprehension of its behavior, developers can ensure the creation of robust and reliable software solutions.

Just think about it! (and maybe give flutter web a chance (?) IDK).

How is the hate of PHP related with sexism and the rise of nationalism in America.

Wagner Mattei — Tue, 31 Aug 2021 10:37:00 +0000

It isn't.