Forem: MiguelMJ

Código humano

MiguelMJ — Thu, 25 Dec 2025 12:04:05 +0000

Una perspectiva poco compartida sobre la naturaleza del código es lo que se puede saber de una persona por su manera de escribirlo. Algo así tenía en mente cuando escribí sobre la programación como conversación, pero en aquel entonces no había leído tanto código escrito por IA. No voy a entrar en cuestiones como la productividad, la calidad del código ni métricas software o buenas prácticas; estas incumben a la industria, y ahora quiero hablar de la programación como actividad humana.

Creo que puedo recordar cada vez que he sentido la impresión de otra persona al leer su código. Y hablo de pasar a relacionarse con el código de manera cotidiana; no leer un fichero, una o dos funciones, sino a leer, día tras días, las líneas escritas por otro; a comprender un sistema de su autoría. Al hablar de textos narrativos, poéticos, diarios, cartas… la presencia de la persona nos llega a través de un lenguaje humano, el mismo lenguaje que usaría para hablarnos por la calle. Sin embargo, en el código se mezclan las imposiciones de un lenguaje formal y la definición de un algoritmo. Aun así, la personalidad del programador asoma en el conocimiento de sus herramientas, la comprensión del problema a solucionar y la forma de expresar estas soluciones, incluso cuando está mediada por convenciones y estándares. A menudo, a los programadores nos gusta que el código tenga cierto aspecto, cierta manera de leerse; eso después se convierte en convenciones de estilo, pero nace como una elección personal que persigue una subjetiva legibilidad. Incluso la ausencia de esta intención construye una identidad. Me pasa incluso cuando leo mi propio código, que tengo una reminiscencia de cómo era yo cuando lo escribí; qué me interesaba probar, con qué técnicas había adquirido soltura, o qué conocimientos no tenía todavía.

Por eso, he tenido la sensación, algunas veces, de que se crea una pequeña intimidad en esa lectura de código. Es una pequeña ventana a cómo funciona la maquinaria mental de otra persona. Cuando se trabaja en proyectos en equipo, llegas a reconocer de un vistazo al autor de trozos de código sin necesidad de preguntar. Y esta forma de conocimiento, a mí, me resulta entrañable incluso cuando maldigo a la persona que escribió semejante espagueti. Porque otras veces, si es un buen código, el autor te comunica cómo programar mejor; y cuando no lo es, llegas a compartir la confusión que pudo tener que soportar, o como mínimo, a empatizar con su nivel de experiencia, porque todos hemos empezado en algún lado.

Cuando estoy programando un sistema un poco más complejo, me da la sensación de estar sosteniendo un pequeño edificio con la mente, y que al pasarlo a código, lo dejo plasmado en él. Como cualquier proceso técnico creativo, en ocasiones tiene un punto solitario. Y al leer el código de otra persona, al ver su elección de arquitectura, qué prácticas usa y cuáles no, voy reconstruyendo el edificio que él/ella sostuvo mentalmente en algún momento. Es como ponerse en sus zapatos. Al hacer eso muchas veces, adquieres familiaridad con su forma de trabajo, reconoces cómo estructura el código, cómo nombra variables y funciones, cómo segrega responsabilidades, cuándo decide dejar de anidar condiciones, qué estructuras de datos prefiere, cómo le gusta procesar colecciones de datos, sus inclinaciones dentro de los paradigmas de programación…

El estilo de cada programador/a se parece a una especie de caligrafía, un acento al hablar o una colección de muletillas e incluso de palabrotas. No es algo que deba importar en la industria, pero sí es algo que, a veces, aparece. Y quería hablar de ello, porque desde fuera, parece que el código está libre de la humanidad de sus programadores; y no hay nada más lejos de la realidad. En la era de los LLMs, esto está cambiando; y no hago un juicio de valor, pero sí hay algo quería visibilizar. Muchas actividades humanas, aunque se deleguen en muchos casos o se automaticen para el consumo, no van a desaparecer de la vida de quienes las practican, por el simple hecho de que son inherentemente humanas; y como tecnólogo, siempre me ha parecido que programar es una de esas actividades.

Human code

MiguelMJ — Thu, 25 Dec 2025 11:57:54 +0000

A little-shared perspective on the nature of code is what you can learn about a person from the way they write it. I had something like this in mind when I wrote about Programming as Conversation, but at that time I hadn’t read as much code written by AI. I’m not going to get into issues such as productivity, code quality, software metrics or best practices; these are matters for the industry, and now I want to talk about programming as a human activity.

I think I can remember every time I have felt the impression of another person when reading their code. And I am talking about interacting with code on a daily basis; not reading a file or a pair of functions, but reading, day after day, the lines written by another person; understanding a system they have created. When we talk about narrative texts, poetry, diaries, letters… the presence of the person reaches us through human language, the same language they would use to talk to us on the street. However, in code, the impositions of a formal language and the definition of an algorithm are mixed together. Even so, the programmer’s personality shines through in their knowledge of their tools, their understanding of the problem to be solved, and the way they express these solutions, even when mediated by conventions and standards. Often, we programmers like the code to have a certain look, a certain way of reading; this then becomes style conventions, but it starts out as a personal choice pursuing some subjective legibility. Even the absence of this intention builds an identity. It happens to me even when I read my own code, that I feel a reminiscence of how I was when I wrote it; what I was interested in trying out, what techniques I had become proficient in, or what knowledge I did not yet have.

That’s why I’ve sometimes felt that reading code creates a small sense of intimacy. It’s a little window into how another person’s mind works. When working on team projects, you can recognise the author of pieces of code at a glance without having to ask. And I find this form of knowledge endearing, even when I curse the person who wrote such spaghetti code. Because other times, if it’s good code, the author tells you how to program better; and when it’s not, you get to share the confusion they may have had to endure, or at least empathise with their level of experience, because we all started somewhere.

When I am programming a slightly more complex system, I feel like I am holding a small building in my mind, and when I translate it into code, I leave it imprinted on it. Like any creative technical process, it can sometimes feel lonely. And when I read someone else’s code, when I see their choice of architecture, what practices they use and which they don’t, I reconstruct the building that they held in their mind at some point. It’s like putting yourself in their shoes. By doing this many times, you become familiar with their way of working, you recognise how they structure the code, how they name variables and functions, how they segregate responsibilities, when they decide to stop nesting conditions, what data structures they prefer, how they like to process data collections, their inclinations within programming paradigms…

Each programmer’s style is like a kind of calligraphy, a speech accent or a collection of catchphrases and even swear words. It’s not something that should matter in the industry, but it is something that sometimes just appears. And I wanted to talk about it, because from the outside, it seems that code is free from the humanity of its programmers; and nothing could be further from the truth. In the era of LLMs, this is changing; and I’m not making a value judgement, but there is something I wanted to highlight. Many human activities, even if they are delegated in many cases or automated for consumption, are not going to disappear from the lives of those who practise them, simply because they are inherently human; and as a technologist, I have always felt that programming is one of those activities.

Programming as a conversation - How the communicative scheme is also applied in the written code itself

MiguelMJ — Sat, 11 May 2024 14:28:09 +0000

Coding conventions, documentation practices and nomenclature styles are commonly subject of strong opinions among developers. In my case, I really thought for years that the best way to write software had to exist, even if it was not mine. And, surely, there are objective criteria that make code better, from the best use of idioms to the readability-performance tradeoffs. However, I've come to think that understanding code as a conversation where subjectivity takes a role might be the most valuable approach to write software, rather than one that assumes universal metrics.

Cover image by Christina Morillo from Pexels

Introduction - Programming as talking to a machine

When I was learning to program, my university teachers used to present programming as a way to communicate with a computer. Compilers are translators between high level programming languages (closer to human languages) and machine language, binary encoded in metal circuits; therefore, to program is to talk with a computer. Obviously, this definition is not wrong, but in the software development industry context, it is clearly reductionist, as it ignores many other factors involved in development. However, this is precisely the approach that most academics I've worked with take, deliberately or not.

Academy and industry

The reason for this difference between industry and academy is the fundamental understanding of what programming is and, as a consequence, what life cycle of software is.

In the academy, programming is a tool to build systems that model a theoretical problem, that solve particular tasks, that embody specific demonstrations or proofs. Projects evolve from simple scripts to bigger systems, usually written collaboratively between professors, doctoral candidates and inexperienced, unsupervised, recently graduated students, neither of whom uses to care much about long term maintainability. This, along with a general lack of supervision, results in partial or absolute absence of tests, absence of style conventions, inconsistent coupling levels across the architecture and other usually considered bad consequences.

On the other hand, industry software development takes programming as a tool to build products that need to meet requisites set by clients and changing markets. Projects might last for many years and will be developed, extended, refactored and maintained by very different developers over the course of time. In the industry, this software is alive and evolving. Therefore, its architecture must be able to adapt and grow by the hand of whole teams that may also change over time.

For this reason, projects that come from the academy to find a place in the market will eventually face a complete readjustment if they want to survive. However, not all software is meant to end up in the market. Not all projects are born to live long lives, to be built collaboratively or to evolve at some point. If a piece of software works and no one is meant to ever touch it again, apart from maybe the original creator, then complex architectures, coding conventions and best-practices can be an unnecessary overhead. It is very important to identify the expected cycle of life for a program, to be objective about how it will need to be maintained.

Programming as a dialogue

The first time I was told my code sucked was in an issue on one of my open source projects, where the developer in question told me, with no offense intended, that the unreadable code made it hard to contribute. I always tried to make my code readable, how could someone not understand it? So I went to the mentioned piece of code and, what a shame, it was ten times more unreadable than I remembered: obscure variable names, deep levels of nesting in control structures, walls of code without either comments or subroutine division... Basically, they were right.

I didn't take time to improve it. In the end, it's an open source library used by very few people and contributed by even less, and I had better things to do. But a year or two later, I landed a new job and my new co-worker told me, during my first code review: "You are not used to writing code in a team, right?". Since he had seen I was willing to learn, he had a very good attitude towards me and showed me the clean code practices that the team followed. However, that particular question stuck with me. It was not my first job, but it was, indeed, the first time I was going to write code, on which my income would depend, that other people would be working with.

I came to the realization that, when I wrote code without supervision (for personal projects or in a solo team) my inner discourse would go on like a monologue. As I internalized at university, I was just talking to the computer. After that code review and more that came after, I was required to have an inner discourse that worked as a dialogue. I had to talk to the machine but also to my teammates. But, how is that done through code?

Langue vs Parole

In linguistics, there's a difference between langue and parole. The first one is the common code use to communicate with others (the languagSince he had seen that I was well disposed, he had a very good attitude towards me and showed me the clean code practices that the team followed.e), and the second one is the particular way it's used by an individual. For this difference, if you happen to be in the bus and listen to a conversation between two close friends, you might hear a lot of things that you don't understand, even though they are speaking the same language as you. Not only because you may lack context, but because (oversimplifying) their specific use of language might be alien to you. The same happens with code.

If you alone are communicating with the computer, there is no problem to write code as a monologue. As far as I know, this is pretty frequent, for example, in indie game development. Some great indie games, like Undertale, are said to have terrible codebases. Given that many indie developers learn to code during the development of the game itself, it is not something strange. But, in the end, it doesn't matter at all. The game is finished, machines run it and people play it. There would be absolutely no point in rewriting the game only for it to follow industry best-practices. The same happens with some university projects. Once you have processed a dataset and extracted what you needed, or once you have tested the algorithm you were developing, a new architecture won't make any improvement on your desired results.

However, if you work in a team and follow this "monologue approach", you will end up with serious problems. And when I say team, I also mean open source contributors or future maintainers for solo projects. Interpret it in a broad sense. The codebase receives input from several channels that, the same way as in the bus example, exclude others from the conversation. Your teammates will need to understand the language and your use of it to comprehend the code. This will be aggravated with each new contributor that joins the project.

To avoid that, conventions are needed. Because, let's be real: even the most self-explanatory code can reach a point of complexity that require outside explanations, at the very least for inexperienced members of the team. Every developer that says otherwise is just assuming a level of experience and knowledge of industry standards that not everybody (starting with me) has reached yet. So, the conventions must include design patterns, code styling, nomenclature and even testing philosophy. Choosing the conventions that adjust the best to both the project requirements and the used technologies, and effectively communicating them to the other developers (usually via documentation or onboarding processes), can make the definitive difference for a project in the long term.

Needless to say, it is not all about choosing the perfect conventions, that, to start with, don't exist. Unclear or changing requirements, too simple or too complex architectures, and many other factors are also determinants of this long term maintainability. Again, software development is a complex activity with many factors involved.

Conclusion - Ensuring conventions

As in any form of communication, the code conventions chosen for a team and a project may be sometimes interpreted in different ways or with varying levels of rigor by each programmer. Subjectivity is like that. Therefore, code reviews become an inseparable part of the said conventions' proper application. What is legible for you may (and will) not be legible for others. So having more eyes on the code is a good way to make it more accessible. Both giving and receiving feedback on written code must be handled with humanity and an open mind.

When numbers don't make sense

MiguelMJ — Thu, 09 Mar 2023 16:30:00 +0000

Introduction

Most of the time, basic arithmetic is pretty simple and intuitive. Even working with powers and roots isn't that much of a headache. However, the IEEE standard for floating-point numbers includes special values that can mess things up: NaN (Not a Number), positive infinity and negative infinity. In fact, most algebraic properties of real numbers are no longer valid. The only purpose of this post is to show some cases I found interesting, where these values produce counterintuitive results.

Cover image by Armand Khoury from Unsplash

Standard behavior of NaN and Infinity

For all x: NaN > x is false.
For all x: NaN < x is false.
For all x: NaN == x is false.
For all x: NaN != x is true.
For all x except NaN and +Infinity: +Infinity > x is true.
For all x except Nan and -Infinity: -Infinity < x is true.
Infinity / Infinity = NaN
Infinity * 0 == NaN
Infinity - Infinity == NaN
Any other arithmetic operation between a number and Infinity results in Infinity (with a possible change of sign).
Any other arithmetic operation between a number (or Infinity) and NaN results in NaN.

Everything I will tell you from now is trivially deduced from understanding the list above; in other words, if you understand all its implications, I won't tell you much new. Anyway, I think it is interesting to review some side effects that may escape from us when we don't keep them in mind.

Broken properties

The following is a non-exhaustive list of properties that we use to expect while programming, but are no longer true due to the existence of NaN and Infinity. I will provide the statement of the property (which doesn't hold) and then a piece of code in JavaScript and Python that implements a check for it, followed by a call that returns false.

Note: JavaScript produces NaN and Infinity values easily, not only because they are native values but because they result from regular operations (Math.sqrt(-1) or 1/0). In the case of Python, they are implemented in the math library (math.nan and math.inf) and operations that would produce them (math.sqrt(-1) or 1/0) will raise an exception. For this reason, assume the following header in each Python snippet.

from math import nan, inf

Disclaimer: The snippets of JavaScript will also fail with values like strings or objects. I'm not trying to make any point about errors in JavaScript. The point is about the NaN and Infinity values, which belong to numeric types in any language with full support for the IEEE standard.

Now, behold the fall of the tautologies.

Identity

Let's start with the most obvious and painful.

function identity(x){
    return x == x
}
identity(NaN)

def identity(x):
    return x == x

identity(nan)

Some of the following are direct results of this.

A list can't contain elements that aren't equal to any element

Sorry for the apparent redundancy in the statement: check the examples to see what I mean.

function checkIncludesEqual(arr, x){
    if(arr.includes(x)){
        return arr.some(e => e == x)
    }else{
        return true
    }
}

checkIncludesEqual([1, 2, NaN], NaN)

def checkIncludesEqual(arr, x):
    if x in arr:
        return any(x == e for e in arr)
    else:
        return True

checkIncludesEqual([1, 2, nan], nan)

The max value of a list is greater than or equal to the rest

function checkMaxIsGE(arr){
    let maxVal = Math.max(...arr)
    return arr.every(x => maxVal >= x)
}
checkMaxIsGE([-1, 0, 1, NaN])

def checkMaxIsGE(arr):
    maxVal = max(arr)
    return all(maxVal >= x for x in arr)

checkMaxIsGE([-1, 0, 1, nan])

The min value of a list is lesser than or equal to the rest

function checkMinIsLE(arr){
    let minVal = Math.min(...arr)
    return arr.every(x <= minVal >= x)
}
checkMinIsLE([-1, 0, 1, NaN])

def checkMinIsLE(arr):
    minVal = min(arr)
    return all(minVal <= x for x in arr)

checkMinIsLE([-1, 0, 1, nan])

Dividend equals divisor times quotient

This is the first we learn about division. D / d = Q implies D = Q * d. Well... no more. This example can only be shown in JavaScript because Python prevents division by zero.

function checkDdQ(dividend, divisor){
    quotient = dividend / divisor
    return dividend == quotient * divisor
}
checkDdQ(1, 0)
checkDdQ(1, Infinity)

The square of a value is either greater than or equal to itself or less than one

function checkSquareIsGE(x){
    return x*x >= x ||  x*x < 1
}
checkSquareIsGE(NaN)

def checkSquareIsGE(x):
    return x*x >= x || x*x < 1

checkSquareIsGE(nan)
checkSquareIsGE(inf)

"Less than or equal to" is equivalent to "Not greater than"

I find this particularly disturbing, not sure why.

function checkLEQisNGT(x){
    return (x <= 10) == !(x > 10)
}
checkLEQisNGT(NaN)

def checkLEQisNGT(x):
    return x <= 10 and not x > 10

checkLEQisNGT(nan)

A value divided by itself equals one

function checkBetweenSelfIsOne(x){
    return x/x == 1
}
checkBetweenSelfIsOne(NaN)
checkBetweenSelfIsOne(Infinity)

def checkBetweenSelfIsOne(x):
    return x/x == 1

checkBetweenSelfIsOne(nan)
checkBetweenSelfIsOne(inf)

A value minus itself equals zero

function checkMinusSelfIsZero(x){
    return x-x == 0
}
checkMinusSelfIsZero(NaN)
checkMinusSelfIsZero(Infinity)

def checkMinusSelfIsZero(x):
    return x-x == 0

checkMinusSelfIsZero(nan)
checkMinusSelfIsZero(inf)

A value times zero equals zero

function checkTimesZero(x):
    return x * 0 == 0

checkTimesZero(NaN)
checkTimesZero(Infinity)

def checkTimesZero(x):
    return x * 0 == 0

checkTimesZero(nan)
checkTimesZero(inf)

These last two have an interesting side effect too. One would expect that 1 / (x-x) and 1 / (x * 0) to result in Infinity. However, when x is Infinity, they result in NaN. Don't you feel dizzy yet?

Bonus fun fact!

Hey, while working on this article I found something interesting about square roots in Python. As I stated earlier, Python raises an exception if you try to compute the square root of a negative number. But Python implements complex numbers natively so if you use the power operator (**) instead, it returns a valid result!

>>> from math import sqrt
>>> sqrt(-1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: math domain error
>>> (-1) ** 0.5
(6.123233995736766e-17+1j)

The perfect result would have been (0+1j) but due to floating-point precision issues, instead of 0 you have a very, very small number. I hope you found it interesting too!

Conclusion

Semantically sound code is much less prone to errors and reduces the need to be constantly catching exceptions. I hope these examples have served as a reminder that we cannot neglect special numeric values or take for granted many seemingly intuitive properties that, if overlooked, can cause apparently correct code to exhibit undesired behavior.

Programming mathematical objects

MiguelMJ — Thu, 26 Jan 2023 17:31:42 +0000

Introduction

This post is divided into two main parts. In the first one, I'm going to introduce mathematical objects and mathematical applications. It is programmer-oriented, but I assume some level of math knowledge. In the second part, I will tell my personal experience discovering and researching previous work on this same topic.

Note: In case you know it, the Wolfram Language may come to your mind while reading this post. It is probably the most powerful tool out there for programming mathematics, as it is the result of years of continuous work by a very talented team. However, I won't talk about it here because even though it is more accessible than I thought at first, it imposes restrictions that I didn't want to deal with. I was not going to expect my reader to do so if I didn't, in the first place.

What are mathematical objects?

Anything we use to make math is a mathematical object. For example, numbers ( $3$ , $27$ , $- 80$ , $3.15$ , ...), symbols ( $+$ , $-$ , $=$ , $π\pi$ , $ϵ\epsilon$ , $\sqrt{}$ , ...) and variables ( $x$ , $y$ , ...). From simple mathematical objects, we can build other more complex, like operations ( $3 + 5$ , $2^x$ , $\cup B$ ...), relationships ( $\lt 3$ , $\ne 10$ , $1 = 1$ ) and even full theorems ( $∃z∈N:x+z=y\forall x, y \in \mathbb{N}, x<y \iff \exists z \in \mathbb{N}:x+z=y$ ).

Modern programming languages provide all kinds of interfaces to represent and manipulate mathematical objects. Numbers, for example, are present in any non-esoteric language you will stumble upon. But what about more complex objects?

Mathematical objects in programming languages

Mathematical objects as data structures

While mathematical objects are relatively abstract concepts, their implementation in a computer is directly related to data structures. Different languages define different native data structures, and each compiler/interpreter implements the operations on said data structures, determining the mathematical objects that can be manipulated by that programming language without additional programming.

I say "without additional programming" because, in the end, all programming languages (the Turing-complete ones) are equivalent. Since we live in the future, you will find that popular languages have libraries that implement anything you can think of.

Languages for mathematical applications

MatLab works natively with matrices and their operations. Do you want to work with matrices in Python? With NumPy, you can, but it will be a little bit harder than with MatLab. Do you want to work with matrices in C? You also can, but it will be harder than with Python.

This is just a crude case of how different languages handle better some mathematical objects. And while matrices are a relatively common thing to use in programming languages, other math concepts are not. Remember that we are talking about very different fields of mathematics. You would be surprised how cleaner or even faster can be a program written in less conventional languages like Haskell or Prolog, depending on the problem.

So imagine you need an application capable of handling a lot of different math concepts: arithmetic, algebra, logic, geometry... What are your options as a programmer?

You can choose a single multi-purpose language, like Python, R or Julia, that have immense repositories of libraries of all sorts. You may find redundancies that you would need to contemplate and probably translate to common data structures. Also, some concepts might end up being difficult to use in an idiomatic, clean way. Or, on the contrary, to make them idiomatic, they turn out to be poorly optimized.
You could use more than one language and glue them somehow or even embed one into another. This way, you could take advantage of the benefits of each. Have you ever wondered how powerful could be an application made with JavaScript + Python (PyScript) + Prolog (TauProlog)? That sounds like the third wave of AI to me! Just kidding, sounds very interesting but just reading it already gives me a headache.
Another option is to turn your application into different distributed services. Collaboration is key to growth, isn't it? What if we implemented specialized services for different math fields and tasks and then make them talk to each other? It has the advantages of both previous points. You have a modular architecture, you can reuse code and you can use different languages, the most suited for each task.

Distributed math applications

Shared formats as middleware

When you connect several applications, you need formats. We could let each application communicate the way it wanted and enable the main system to interpret every possible format, but that's not scalable.

The Web is the best example of this. Everything you display on a webpage is HTML by convention. There are countless markup languages, document formats... but if you want something rendered in your browser, you better translate it to HTML. Look at Mathjax or KaTeX. We didn't want to write mathematical equations directly in HTML, because LaTeX was better for that. So we made applications that translated it into HTML.

Formats for humans vs Semantic formats for machines

Standard formats are powerful. When machines are able to communicate, everything is better. But HTML and CSS, for example, are a special case for one reason: their final target are human eyes. An HTML page might be a complete mess, but if it renders into a human-readable page, it's alright.

In fact, when we decided that we wanted the web to be machine-readable, the concept of the semantic web was born. Because machines do not care about presentation, they care about semantics: the meaningful information they can parse from the used formats.

Semantic math formats

And here's the problem with a lot of formats for mathematical object representation; they are made to present information to humans, not to machines. LaTeX, for example, doesn't hold much semantic information.

$x y + s in (e)$

The expression above is written in valid LaTeX, but it is pretty ambiguous. Is $x y$ a single variable, or the product of $x$ and $y$ ? Is $e$ the Euler constant or an independent variable? We don't know, because LaTeX is not a semantic format. A human would know the answers thanks to the context in which we use the expression, but probably that context is not machine-readable.

Wikipedia points out three main formats for computer interchange: MathML, OpenMath and OMDoc. The people involved in the development of each one worked on the others too, so they are sort of complementary.

MathML has two variants: Presentation MathML, which doesn't hold semantic information, and Content MathML, which does. The mechanism for describing the semantics of its symbols is not present in the format itself. For that, it relies on the OpenMath standard (or similar alternatives).
OpenMath provides two encodings to represent mathematical objects and also defines an extensible mechanism to describe the semantics of its symbols: the Content Dictionaries. There are official dictionaries that describe various symbols for different fields of mathematics, but any person or organization could define more dictionaries to extend the number of objects representable by the format. This way, an application that uses follows this standard only has to specify which dictionaries it supports.
OMDoc, on the other hand, is some kind of extension to both MathML and OpenMath, designed to represent mathematical documents for humans and machines alike.

Note: I shouldn't finish this section without mentioning the existence of SCSCP (Symbolic Computation Software Composability Protocol). It is a protocol developed to communicate distributed systems for mathematical computation, or CAS (Computer Algebra System). It is built on top of the OpenMath standard to describe a server-client communication oriented to mathematical objects.

My experience with the OpenMath standard

First encounter - Apparent death

OpenMath is the best suited standard to communicate distributed math applications.

I was really excited when I found this standard. As long as an application supported the OpenMath encoding (OM), it could be communicated with lots of other interesting applications easily, with a common format. A whole new world of possibilities for collaboration! I was sure I would find applications to transform OM into renderable formats like LaTeX or HTML, reduce or evaluate complex mathematical expressions, transform function definitions into code, etc.

However, disappointment came when I continued to research. Reading the openmath.org page I found a list of projects that used the standard. The first time I clicked on one: broken link. But, hey! the next link works! ... and the linked project is obsolete; it has unresolved and unfindable dependencies. The official page is full of broken links and obsolete, incomplete and undocumented projects.

But it did not feel like abandonware from decades ago; there were things dated less than a few years ago. It was the same feeling as being late for a party that had just ended.

Finding the trail

I only found a Python library and a JS package implementing the standard, that seemed to be active to some extent... but neither of them contemplated the full specification.

The one in JS is a bit basic. And when I tried to use the Python one, I didn't find it easy to do anything with it apart from parsing and serializing XML; it felt like the mathematical objects were not programmed to be manipulated easily. Of course, that can be (and probably is) my fault, because the library is used in another SCSCP library, where it proves to be useful.

At first, I didn't give too much attention to the SCSCP library, because I thought it was not related to what I was looking for. I started my own Python implementation of the standard (if you visit it on my GitHub keep in mind it is very unstable and mostly experimental for now) and forgot about any official implementation.

However, I recently returned to that library, trying to understand it better with the knowledge acquired by working with OpenMath on my own. The SCSCP library caught my attention this time, and I followed one link after another until I understood what was going on. Part of these projects was recently funded by the EU in a program to foster the development of a European research infrastructure called OpenDreamKit that happened to finish in 2020. In fact, there are two more projects built on top of both the OpenMath and SCSCP Python libraries that belong to said infrastructure and seem to have little attention from other developers, even though they seem to be the only current functioning projects based on that technologies.

My interpretation

Disclaimer: I don't talk on behalf of anyone except myself and I might have incomplete or wrong information about what's been going with OpenMath on in the last decades.

I've spent many hours browsing the pages related to OpenMath and I've come to a conclusion. It looks like a lot of this work started long ago, in the nineties, which explains why there is so much that's become obsolete (so it is in fact like abandonware from decades ago). To me, it looked like back then their work was more similar to projects of software development. But their society has always been closely related to universities and public funds (similar to the OpenDreamKit one). So their current available work is very academic! They have papers instead of documentation and research groups instead of communities.

They made their last workshop a year and a half ago (Jul 2021) so they were active recently... But then, why don't they update their website to reflect the actual state of the project? Why isn't there any work to integrate the standard into modern, popular frameworks used for mathematics? I've come to think that it's just that, if they are actually still active, I'm out of the appropriate communication channels. I don't know if I'm in an empty room or just in the wrong room.

Conclusion

Is the project just close to a death of disinterest? Or maybe the circles using and developing it are not the circles of software developers, but of researchers and academics? I honestly don't know. But, also, I don't care! I found the project to be fascinating and full of potential. The OM Society might not have the same philosophy as other modern open source projects, but their work is there for us to use it.

I think the idea of distributed mathematical systems is wonderful. But not only for researchers, who are the people using them right now; also for developers! My intention with this post was to make these concepts and possibilities known to people interested in working on open source projects like the ones we do know, in the hope that new, open and active communities can be formed.

If you liked this topic, let me know and I will write more about it! ✍🏻 I haven't explained anything about the format, encodings and content dictionaries of the OpenMath standard, so I'll probably write another post explaining how to use and implement them.

References

My blogging journey until now - I'm moving to Hashnode

MiguelMJ — Thu, 19 Jan 2023 16:31:02 +0000

Cover photo by Jess from Pexels

2020 - Starting a blog

It's been more than two years since I began blogging. It all started with a development journal for one of my latest projects at the second half of 2020. I knew I wouldn't be graduating that year, so I decided that I could try to gain some online presence to compensate my lack of professional experience.

Writing a Conversational User Interface Library - Full Series

MiguelMJ ・ Dec 10 '20

#closed

DEV.to was in a boom, I was seeing a lot of developers start blogs here, so I decided to give it a try. I found the other platforms a bit intimidating, but this one was very approachable and beginner friendly. I wrote every week or two and found some interesting people to follow. I didn't get much attention, but it didn't matter! I was here for the experience of writing and the growth that comes along.

I wrote more than a development journal. I had a lot of opinions I had formed during my years in university and advice to give, so a lot of what I wrote at that moment was the kind of things I would've love to know as a freshman. It was a true pleasure to have a space to write it all down and share it.

Why and how to use pen and paper - Designing algorithms with no code

MiguelMJ ・ Dec 13 '20

#beginners #codenewbie #productivity #codequality

Before the year ended, the DEV.to community did a lot of work around the moderation of the platform. At that time I was made a trusted member and to say that I found it rewarding, for the short time I had been here, is an understatement.

2021 - A comfortable plateau and the seed of dissatisfaction

With the new year, I slowed down my publication pace and started writing monthly. There was a lot going on in my life, so I felt comfortable with little activity; enough to continue the habit and keep my presence out there. You can tell it was a weird year for me because I have several unfinished series and a failed attempt to translate all my posts to Spanish (my native language).

Despite all this, it was a great year in terms of growing my personal brand. My open source projects started to thrive (I was even featured in an incipient newsletter, now dead) and I started writing for my current self, more than for my past self.

Data structures in Prolog - Where to start

MiguelMJ ・ Jun 15 '21

#prolog #datastructures #resources

Also, a very strange episode happened that year.

I was invited to write a featured post for another platform. As it was my first post written for someone else, I wanted to make it about a popular topic: JavaScript, something I had never written about before.

Unfortunately, that site turned out to be a complete scam. Most of its content is stolen, their complete Q&A section is a literal scrap of the StackOverflow page and each and every user I interacted with was a bot (they even used watermarked stock photos as profile pics).

I got out of there as quickly as possible and published my article here in DEV.to. In less than 24 hours, it had more traffic than all my previous posts together. Today (January 2023) it is still my third most read post.

Take full advantage of high order functions - Examples in Javascript

MiguelMJ ・ Jun 22 '21

#tutorial #javascript #functional

I remember reading several posts here talking about the spam, the clickbaity low-effort listicles, the daily copy-pasted git tutorials, etc. If you have been here for a while, you will know that the quality of the DEV.to content is still an unsolved issue. I didn't fully care, I was comfortable here and that was all that mattered. As a trusted user, I did my best to mark low-quality posts and boost well written, original content.

However, dissatisfaction started to grow inside me, and a feeling that something was not fair here. Not only because of all my beloved posts eclipsed by my own single JavaScript tutorial, but also for the people I follow that publish interesting, advanced content, that get 1% of attention and feedback received by the mass of low quality posts focused on beginners.

2022 - Last year in DEV

The following year I did very little blogging. Most of my posts in 2022 were for promoting a project or another (looking back, I coded some interesting stuff). The two biggest exceptions were very significant posts to me.

How to get answers on StackOverflow

MiguelMJ ・ Feb 6 '22

#beginners #codenewbie #productivity

This one came after I saw some people complaining about the StackOverflow community. I wanted to do an attempt on bringing closer two of the communities I love, from which beginners benefit specially. I expected some backslash in the comments (given the hate SO receives from some people) but instead I got a lot of comments from veteran devs, liking my post. It also gave more insight about who reads what.

JavaScript is not an untyped language

MiguelMJ ・ Jun 24 '22

#javascript #plt #computerscience #beginners

And then, there's this one. This one was an explosion. I know it was a controversial opinion on a perma-hot topic like JS, but damn, this post has five times as many views as my next most viewed post, and it's my top reacted and commented post of all time. It got featured in the social networks of DEV, tweeted several times and even improperly published as material for an online course.

// Detect dark theme var iframe = document.getElementById('tweet-1541501371962253314-196'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1541501371962253314&theme=dark" }

And it was a rant written in a single afternoon! This is perfectly one of the posts I've put the least effort into. I stopped writing, discouraged. I know effort doesn't correlate with impact when we talk about writing on the Internet. But I just didn't like the idea of going back to write high effort posts for no one, always tempted to write about JavaScript again only for reads.

After some time, I was contacted yet by another platform, asking me to write a course for them. Another dev that had worked with them recommended me writing my own book, instead of just a course. Don't get me wrong, that platform is legit and has great content; in fact, I tried to make the course they wanted. But I didn't have enough time.

Writing an article for a blog is something, but writing a course or a book is another dimension. It is something I want to do, but I probably won't anytime soon.

The big hiatus and the decision to move

It's been months since I published anything. There are several things I want to write about, but these months have been very busy, plus I don't feel like I have found my audience yet, which disencourages me even more. I don't even read DEV anymore, as the content quality has only gone down, in spite of the efforts to implement tag filters and experience level for posts.

So, to find new motivation, I have decided to move on. I need to keep looking for an audience that I feel I connect with. I love DEV and everything I've achieved thanks to it, but I need to keep looking for a better space for my blog.

I'm going to try my luck with Hashnode. I don't have any particular reason, so I don't know if it will be my definitive platform. I want to see what kind of audience I can build there and explore the possibilities of a different tool.

Conclusion

miguelmj.hashnode.dev

My first step has been to migrate there a curated list of my posts and polish them a bit. The canonical URLs will still point to the original posts, nonetheless. I will keep publishing in DEV everything, for the readers I already have here, but I will make Hashnode my main target.

I guess that, as a lifelong student and an eventual teacher I will always have something to write for beginners. However, I want to try a different level of writing and even different types of content. I don't know how my blog might evolve, but, of course, that's the exciting part!

Make It Pixel! - Make pixel art from any image

MiguelMJ — Sun, 17 Jul 2022 09:59:09 +0000

There are several options out there for generating pixel art from images. However, none has all the things I would like to have at the same time, so I decided to make my own.

Cover generated by MakeItPixel from the Mona Lisa by Leonardo Da Vinci

A little disclaimer: I have great respect for pixel artists and I know that "pixel art" is not the most proper name for what programs like mine generate. But lacking a better term, and because everyone uses it, I'll do it too, being aware that generated generated pixel art has flaws that any artist could point out and correct.

Motivation

What would I expect from a pixel art generator?

Re-scaling of an image to a given resolution.
Optional use of fully customizable palettes.
Parameterizable color quantization and dithering.
Background removal.
Optional outlines.
Control over image preprocessing options.
Fast to use over large amounts of images.

There are tools out there pretty close to what I want, but, as I said, none has them all. Some have very basic and rigid algorithms others some are GUI based and thus must process the files one by one... and even worse, some don't work on Linux! Most of them discard the option of trying out different configurations on several files at once. In short, if I want to meet my needs, I must do it myself.

With this motivation, I created MakeItPixel.

MiguelMJ / MakeItPixel

Image processing tool to make images look like pixel art

Make images look like pixel art

Preview

Original:

Example results:

Table of contents

Features

Intelligent and customizable handling of color and palettes.
Configure color selection during downscaling.
Configure color quantization and dithering.
Fully customizable both by file and console.
Process dozens of files with in a single run.
Fully documented.

Future features

Parameters to generate outlines using border detection.
Add options to blur before downscaling to reduce noice.
Smart palette extraction from images.

Dependencies

SFML v2.5.1 (release)

Build

g++ src/* -Iinclude -lsfml-graphics -o makeitpixel

Contributing

Read the contributing guidelines if you want to contribute to the code.
Open a new issue to make a request or report a bug.
Alternatively, comment it on the SFML forum.
If you use it in a project, you don't have to give…

View on GitHub

The ideal design I had in mind would offer as much customization as possible, implementing different algorithms to get the best of each possibility. But with great flexibility, comes a big headache trying to find the perfect configuration; so the system would have to be intelligent enough to consider autonomously the best alternatives to get a quality pixel-art-like image. I wouldn't say that this version achieves this, but it's definitely on the right path.

Implementation

The prototype only implemented two basic scaling and color quantization algorithms, based on RGB distance and gray values distance, and automatic palette generation, based on what I learnt of color schemes from W3school and Wikipedia. Then, I saw the video from javidx9 on dithering and I immediately added options to apply both the Floyd-Steinberg algorithm explained in the video and ordered dithering, that I read again on Wikipedia.

The steps of a MakeItPixel run for a loaded image are the following:

Generate a palette if necessary.
Downscale the image.
Apply color quantization.
Apply dithering.

Each and every step can be configured in different ways or skipped.


Original	Scaled
Quantization	Dithering

Steps 3 and 4 are actually performed at the same time but I think it's more visual to see it like this.

A more detailed description of the process can be found in the wiki.

Plans for the future

This first version already has some interesant features that differentiate MakeItPixel from others: the full customizability of the palette, automatic palette generation, selection of different quantization and dithering algorithms or the independence from a GUI, that comes with the possibility to process hundreds of images quickly.

However, it still lacks some of the features I first envisioned. Mainly:

Smart palette extraction from an image.
More preprocessing options, like blurring to reduce noice before downscaling.
Implementation of border detection to include outlines in the generated pixel art.
Customization for matrices used in ordered dithering. Right now MakeItPixel only offers hard coded matrices.

Don't have a doubt that all these will exist in a future version.

Also, I used SFML to work with the images because is a library I've used for a long time and I feel really comfortable with it. However, the last year I've been using ImageMagick a lot and I found a great, great advantage: ImageMagick has support for GIFs, while SFML doesn't. Unless SFML 2.3 or SFML 3 changes that, it is possible that I would change the graphic library... but there is still time before that happens.

MakeItPixel is open source!

The project is open to contributions, comments, feature requests... Star it if you like it to give it visibility and feel free to share your pixelated art online with the #MakeItPixel hashtag!

I'll be happy to read your thoughts in the comments.

High level programming - Explaining abstraction

MiguelMJ — Tue, 05 Jul 2022 11:41:09 +0000

When designing our programs, we make use of abstraction while also providing it. In this post I want to analyze the concept of abstraction in the context of programming and software development.

Cover photo by Vanessa Garcia

What are "high level" and "low level" languages? When we talk about level, we are not talking about performance or power, but about the level of abstraction. Let's see what this means.

Abstraction as thinking about "what" and not "how"

Anything that provides an interface to simplify a complex structure or algorithm behind it provides a form of abstraction. We find this constantly while programming, using functions, subroutines, APIs, libraries...

The same algorithm can be expressed in different levels of abstraction:

Level of abstraction	Algorithm
Highest	1. Prepare a cup of coffee
High	1. Boil some water. 2. Place the filter with coffee over a cup. 3. Pour slowly the water into the filter.
Low	1. Fill a cup of water. 2. Pour it into a pot. 3. Heat the pot until the water boils. 4. Place a coffee filter over a cup. 5. Put a spoon of coffee in the filter. 6. Pour slowly the water into the filter.
Very low	Steps that could include how to traverse your kitchen, how to look for each tool in your drawers and even the movements of your wrist and fingers for each task.

The higher the level, the more information is taken for granted. The lower, less complexity is hidden.

Why is it useful to understand this?

For design. When you define an algorithm to solve a problem, it's usually better to start with a high-level approach, and then break each step down into smaller steps, solving each one at a time (this is related to dynamic programming). Also, this top-down design process usually results in more modular code, which is easier to work with to a certain extent for larger programs.
For documentation. A common beginner mistake is to write low level documentation, when a high level one would provide more useful information. If you have written a piece of code I'm going to use, just let me know how to use it. Give me low level information in case I'm going to review or modify the code, or if I need to understand technical details for any reason.
For optimization. You might have heard that low level languages run faster than high level languages. This is only half true. Every high level language performs behind-the-scenes optimizations that a low level program should match to be as fast. One of my favorite quotes, whose source I don't remember, is: "Badly written C is slower than Python". So don't assume that just because something is low level is faster than a high level alternative, unless you understand the objective complexity of both cases.

Abstraction as generalization of specific cases

Any process that operates with independence from the nature of one or more of its elements provides a form of abstraction.

A structure or algorithm may have more specific instances, and be, at the same time, a specific instance of a more general case:

Data structures, for example, are identified at high level by the insertion, access and deletion rules, while in the low level are identified by the type and arrangement of their values.

Level of abstraction	Structure
General	Set of values identified by a key of any type
Less general	Set of values identified by any number
Specific	A list of values
Very specific	A linked list of strings

The abstraction level of an algorithm depends on the abstraction level of the structures involved and how much you can determine the result for any specific input.

Level of abstraction	Algorithm
General	Traverse any iterable structure and apply an operation to its values
Less general	Iterate a list and apply an operation to its values
Specific	Iterate a list of numbers and apply an operation to its values
Very Specific	Iterate a list of numbers and duplicate each value

Why is it useful to understand this?

Using abstractions already present in programming languages. Most high level languages offer tools like templates, high order functions, interfaces, etc, that usually spare you a lot of work, when you are able to identify the best opportunities to use them.
Reusing logic to improve your code. Taking recurrent logic in a program and making it abstract results in shorter, more flexible and scalable code.

Good practices and conclusion

Now, you must be careful. Too abstract code might be sometimes more difficult to read and difficult to optimize, because there exist more dependencies and procedures tend to take more parameters. The complete opposite is also true: code with zero abstraction is going to be a pain in the neck to write, review or maintain.

Always try to find a balance between ingenious and maintainable solutions.

Recommended reading

Hackathon submission - An audio search engine powered by Deepgram

MiguelMJ ・ Mar 28 '22

#hackwithdg #telegram #showdev #python

The Turing Machine made understandable

MiguelMJ ・ Dec 8 '20

#computerscience #algorithms #learn

You can follow me on Twitter! 🐦

JavaScript is not an untyped language

MiguelMJ — Fri, 24 Jun 2022 10:40:22 +0000

JavaScript's popularity

At the time of writing this post (Jun 2022), JavaScript is #3 in the PYPL index, only beaten by Java and Python in popularity. In the latest Stack Overflow Developer survey, it came out as the most used language for the tenth year in a row. Being the main language for programming web applications in a connected world, it's not that of a surprise that programmers of all levels of experience work with it, both for learning and professionally.

Cover by Fiona Art from Pexels

But JavaScript is a peculiar language. Designed for running on web browsers, some people wanted to use it out of them so much that they even created NodeJS for that. JavaScript has contributed to create things like AJAX and JSON (they liked greek myths) that would still be used even if another scripting language appeared someday to replace it, something that won't happen soon. So it's clear that people love it.

Love it? When you listen and read about it, you'll quickly notice that people tend to have very strong opinions (maybe it's a common trait in developers) on this language. They either love it or hate it. And a lot of people have an opinion, just because depending on what part of the industry you work, it's just inevitable to use it daily. Yet, it is so error prone and unsafe, unless you have really consolidated good practices, that it's unbelievable that it's still a standard.

Tech-influencers

What calls my attention the most is the fact that lots beginners are so centered around it. In fact, the same phenomenon goes on with Python. Their high level, dynamic nature, makes them easy to start with. Dictionaries are primitive types, functions are first class citizens... they are just so flexible! And it's so easy to write listicles about JavaScript, 30 secs tutorials or make 3 slides explaining how great it is compared to other languages to get traffic on your posts, videos, etc.

Some tech-influencers even build their brands specifically on this kind of JavaScript content. The damage that these tech-influencers do in general is not the matter of this post, but there's something very specific that's wrong and that I've read a lot that I would like to explain.

Is JavaScript an untyped language?

If there's something new programmers hate is error messages. I guess that's why I've seen listed as one of its advantages over other languages that JavaScript produces no type errors because it's an untyped language.

This is just wrong

Yes, JavaScript raises less errors than other languages in the same context. Is this good? If the infamous "ba" + +"a" + "a" == "baNaNa" is not a red flag for you... Anyways, that's not the matter of this post either. The point I really want to make is: JavaScript is a typed language.

There was a discussion on Stack Overflow in 2015 on this topic (and I'm sure in many other sites over the years) where the top answer summarizes the problem very well. The key is what you mean as "untyped". Well, Brendan Eich himself used this term to describe JavaScript, meaning that it's not statically typed. However, one thing lots of programmers missed is that even he pointed out that the term is used differently in academia and that JavaScript, in fact, had typed values.

It's important for professionals to use standard terminology, and like it or not, "untyped" is not appropiate to describe a non statically typed language. If "untyped" already has another use: a programming language with no types; and what you are saying also has an unambiguous term associated: "dynamically typed", then the more clear and distinct option should be preferred.

Let's consider the following points:

A programming language with types is a typed language. No matter if statically-typed or dynamically-typed: it is typed.
Types impose semantic restrictions: type errors occur.
JavaScript might perform almost every possible casting (this means weakly typed) to prevent this errors, but the can still raise. Examples:
- null[0]
- undefined[0]
- null(1)
- undefined(1)
- 1(1)

All of these operations could silence their errors with more castings, like turning null and undefined into empty dictionaries, or any non-function value to a bottom function that always returns a nullish value. But even for JavaScript that would be too much of a bad decision. Because type errors are not there to annoy you, but to help you. Type safety is a good thing.

Conclusion

JavaScript is a typed language that can raise type errors and the fact that a lot of implicit castings may be happening all the time is just a feature of the language that demands extra attention from the programmer.

Let me know your thoughts on the comments!

EDIT

There have already been several people asking the same things in the comments so I'm going to write here the answers for anyone going to ask the same things again:

Where have you read that JavaScript is untyped?. Scroll up a bit. As I say in the post, Brendan Eich (creator of JS) uses that word to describe JS, in the SO discussion linked above there people saying the same thing, tech-influencers I don't want to promote do so too and finally some users in the comment section.
There are no untyped languages, that makes no sense. Using the description of untyped languages as languages without a type system (a definition used by authors like L. Cardelli or R. Harper very different from dynamically or weakly typed), untyped languages do exist.
- A lot of Turing complete formal languages are untyped, from the Turing machines themselves to cellular automata.
- Assembly languages are untyped. Some may provide syntax sugar to work with primitive typed-like values, but there is not a type system that imposes restrictions on operations based on types (in most of them).
- Some shell languages are untyped. The type-like restrictions come from subprocesses, not from a native type system.
- Pure Prolog is untyped. Without the IO stream entities, everything is a Horn clause that can be evaluated to true or false, without type restrictions.
- Some esoteric languages are untyped. Brainf*ck being the most popular, lots of languages, usually minimalistic, don't have a type system.
Yes, I know typed languages are more present in the industry (except for assembly and shell languages), but that doesn't make the distinction pointless. Remember the software industry came way much later than computer science itself and both still develop in separate streams.
That's not how type systems work. If you read the thread in the comments with @darkwiiplayer, you'll find that there doesn't seem to be a consensus on whether types are syntactic or semantic information, so depending on the sources you use, all this will make sense or not. The paradigm I'm adjusting to is that types are semantic information, not syntactic. It's a complex topic so instead of elaborating further I'll link some references.
- L. Cardelli & P. Wegner (1985) On Understanding Types, Data Abstraction, and Polymorphism (PDF)
- R. Harper (2016) Practical Foundations for Programming Languages (2nd ed.) (PDF)
- Is type-checking "syntactic" or "semantic"? Computer Science Stack Exchange (link)
To be fair with the opposite paradigm, here's their reference too.
- H. Geuvers (2019) Introduction to type theory (PDF)

Thank you to all those who have participated in the comments!

Recommended reading

Take full advantage of high order functions - Examples in Javascript

MiguelMJ ・ Jun 22 '21

#tutorial #javascript #functional

Introduction to Markov chains - AI for text generation - Part I

MiguelMJ ・ Jul 9 '21

#ai #machinelearning #nlp

Let's connect!

How to build a basic REST API in Lua - Milua micro framework

MiguelMJ — Tue, 17 May 2022 06:13:47 +0000

Prelude

I recently started a new project, for which I wanted to build a REST API. Here are the alternatives I considered and why I rejected them:

Using a framework I'm more familiar with, in a different language, like Flask for Python. My project already required Lua and used a bunch of different technologies, so I'd prefer to keep it simple and not add a whole new language and framework to the list.
Using an existing framework in Lua, like Lapis. The problem here was that Lapis relies on third-party software, like Nginx, so the dependencies list would keep growing. This wouldn't be a problem in a more heavy project, but the API I need here is really small, so I would consider overkill anything more than a single technology for it.

Cover image by aloiswohlfahrt from Pixabay

Milua

I felt clearly that what I really wanted was a pure Lua solution. After a bit of research I didn't find any de-facto standard (I might be wrong), but I did find a http library by duarnimator that provided an excellent set of functionalities to launch a simple server in a few lines.

So inspired by the experience of using Flask I made Milua

MiguelMJ / Milua

Lua micro framework for web development

Milua

Lua micro framework for web development

Milua is inspired by frameworks like Flask or Express, so it just aims to be quick to install and simple to use, enough to prototype any idea you have in mind without needing to worry too much about third-party software.

Preview

Here's a minimal example of how to use Milua:

local app = require("milua")
app.get(
    "/",
    function()
        return "<h1>Welcome to the <i>handsome</i> server!</h1>", {
            ["Content-Type"] = "text/html"
        }
    end
)

-- Example capturing a path variable
app.get(
    "/user/{username}/tell",
    function(path_params, query)
        local times = query.times or 1
        return "The user " .. path_params.username .. " is" .. (" very

…

View on GitHub

Installation

I learned how the luarocks, a lua package manager, works and published the first version, so you can install it with it's dependencies only with:

luarocks install milua

Examples

Right after, you can try the example that comes in the repository.

local app = require "milua"

-- Basic example
app.add_handler(
    "GET",
    "/",
    function()
        return "<h1>Welcome to the handsome server!</h1>"
    end
)

-- Example capturing a path variable
app.add_handler(
    "GET",
    "/user/...", 
    function (captures, query, headers)

        local username = captures[1]
        local times = query.times or 1
        return "The user " .. username .. " is" ..
               (" very"):rep(times) .. " handsome"

    end
)

app.start()

Launching this would just require you to execute the script after installing Milua. Then, you can test it with curl.

$ curl localhost:8800/
<h1>Welcome to the handsome server!</h1> 

$ curl localhost:8800/user/foo
The user foo is very handsome

$ curl localhost:8800/user/foo?times=3
The user foo is very very very handsome

Features

For now, version 0.1 as I write, the milua module only offers two functions:

add_handler(method, pattern, handler) to associate a method and a path to a handler.
- The handler function must accept the following arguments:
  - captures: An array with the variables fields of the path, specified with three dots (...) in the pattern string. In fact, this pattern is just a regular Lua pattern with some syntactic sugar, so you can capture anything you like in your path and be as specific as you want: /([0-9]+) would capture a path to a numeric value; /admin_... would capture the destination without the prefix admin_.
  - query: A table with the key-value pairs of the query in the URL (the options that come after the ?).
  - headers: The headers of the HTTP request.
  - body: The body of the HTTP request.
- and must return the following values:
  - The body of the repsonse.
  - (Optional) A table with the headers of the response.
start(config) where config contains the host and the port to run the application.

Conclusion

There is still room for a couple of modifications that would keep the complexity at the same level, like adding error handlers (to allow a 404.html page, for example) or specifying directories for static files. For example, it has no templating functionalities (maybe in a future), but you can require any of your preference and use it in your app. But for now it servers its purpose and it's compatible with anything you want to throw at it, thanks to its minimal nature.

I'm still learning about Lua and its tools and might write more about this language in the future. What's your experience with Lua? Have you used before for web development? I'll be happy to read any comment you have!

Recommended reading

Data structures in Prolog - Where to start

MiguelMJ ・ Jun 15 '21

#prolog #datastructures #resources

How to get answers on StackOverflow

MiguelMJ ・ Feb 6 '22

#beginners #codenewbie #productivity

You can follow me on Twitter! 🐦

Hackathon submission - Idea: Group dynamics analysis

MiguelMJ — Fri, 08 Apr 2022 10:45:55 +0000

Introduction

Well, this is my third submission for this hackathon. I guess Automatic Speech Recognition (ASR) really inspired me. This time I'm participating in the Innovative Ideas challenge because I don't know if I'll have time to program anything else and I prefer just to write.

Natural Language Processing (NLP) and conversational interfaces are one of my favorite topics in technology. That's why I've tried to find interesting use cases for ASR, more than just plain transcription functionalities, which is just a little part of what Deepgram provides. I was surprised to discover how easily you could get other useful data like timing (timestamps of start and end of words) and source recognition (differentiate speakers). We know there's a lot of useful information in speech and I think that's the main reason I've felt so inspired. I'll write more in depth about my thought process behind each submission in the next and final post of this series.

My Deepgram Use-Case

For a long time, I've been interested in group dynamics. We all know from experience that each person feels more motivated to participate in different groups or to talk about certain topics. I think it's important to observe the behavior of each speaker and their reactions in different situations in order to create more comfortable and productive conversations in the future.

So one possible use case for Deepgram and ASR technology could be to analyze recordings of meetings to extract useful information about the dynamics of a group.

What topic makes the quiet one speak more? What kind of words prevent interruptions? What topics are more polemic to certain people?

This kind of analysis would be useful in cases like work meetings, community moderation and even psychology experiments.

Dive into Details

Technical approach

Get the data

First of all, the program would use Deepgram to get the punctuated and diarized transcription, with additional information about the utterances.

Enrich the data

From that information, it would reconstruct the conversation, separating the intervention of each speaker, counting the number of utterances and also the time separating each one. It might be interesting also to mark profanity.
Now we could find anomalies in the time separating interventions to detect possible interruptions or long silences.
Then, it might be necessary to make several runs of LDA (Latent Dirichlet Distribution) to find the topics treated in the conversation, when they change, who brings them, etc.
Similarly, it would run several sentiment analyses by topic, by person, by intervention and of course a general one.

Summarize and present the data

With some probabilistic analysis, it could be possible to estimate correlation with interruptions, silences, topics, topic changes, people, sentiments, etc. I would propose either a Bayesian network or a Rule discovery algorithm, like Apriori.
Finally, the program would make a summary of the interventions of each person. Some possible data to extract would be:
- Time spent speaking
- Topics where they intervene more/less.
- Who speaks usually before or after them.
- Sentiment correlation with topics and other people.
- Most repeated words.
- Exceptional words. By this I mean relevant words that nobody else said or that has little relation with the topics of the conversation... You never know if you'll learn something important about that speaker.

Application

One of the first things I realized was that this wouldn't be of much use in controlled environments, like interviews. The interest comes from natural dynamics that arise from the interaction between individuals when discussing everyday topics.

So one clear context where this could be useful would be work meetings, that can be recorded and analyzed. However, I feel like recording conversations in other settings is somewhat intrusive, so I can't actually propose specific, additional use cases. As I said before, maybe use it with event recordings for community moderation and somehow in psychology experiments... I'm interested in what's your opinion and what case can you come up with.

Edit: I've been told that there's an interesting use case, related to studying cultural differences. There are cultures where an interruption meant to express agreement is considered polite, while in others the convention is to leave seconds of silence between interventions. I guess that another application would be to study group dynamics in multicultural groups.

Conclusion

I've had real fun thinking and working on this hackathon, so thanks a lot to Deepgram and DEV for organizing it. As I said at the beginning, I've felt really inspired and I'm sure I'll work further with the projects born from it. I'll soon post the last post of this series,

Hackathon submission - Interactive conversation scripts for language learning

MiguelMJ — Sun, 03 Apr 2022 10:35:18 +0000

Overview of My Submission

My second submission consists on a tool designed for languages teachers and students. Audios with recorded conversations are a common and good resource in language lessons, so I thought it would be useful not only to have a transcription to read while you listen, but also make them interactive to show phrase by phrase translations and to quickly navigate the audio to listen again certain part of the conversation.

Cover image by Julia Filirovska from Pexels

Submission Category

Accessibility Advocates
Wacky Wildcards

Link to Code on GitHub

MiguelMJ / Scripter

Create interactive scripts from recordings for language learning. Deepgram + DEV hackathon submission.

Scripter

Create interactive scripts from recordings for language learning

Create interactive HTML5 pages with a script from a recorded conversation. Powered by Deepgram. Requires API key from Deepgram. Submission for Deepgram+DEV hackathon, 2022.

Yout might want to read the submission post, where you will also find a demo.

Get the API key

Deepgram (required): Create an account in deepgram.com and get an API key.

Store it in a file named deepgramApiKey in the root folder or pass it directly in the CLI using the --deepgram-api-key argument.

Features

Tune the voice recognition process with the Deepgram query parameters for transcriptions pre-recorded audio with -P|--param KEY=VALUE arguments.
Downloads and results are cached to reduce redundant traffic, but you can force it using the -F|--force flag or directly removing the _cache folder.
All log information outputs through stderr and the search output through stdout (or a file, with the -o|--output-file FILE…

View on GitHub

Additional Resources / Info

This time I reused most of the code from my previous submission. In that one I forgot to tell a bit of background, so as most of the code is the same, I will do it here.

In spite of using Python for the program, I didn't want to use the Deepgram SDK. I know other participants have done the same; in my case, it's just because I'm used to making HTTP requests for lots of things, so I chose not to add more dependencies to the application. I felt that the Deepgram API is accesible enough for me.

I've once more used another API: LibreTranslate. This time, the difference is that LibreTranslate is a open source API, so there are different available mirrors, and you can even set up one yourself, so I allowed the user to specify which one to use with a -H|--host parameter and made a quick guide on it on the README.

I'm not actually a webdev person, so the resultant HTML file might not be very polished, but I thought is more than enough for a prototype. I learned to use the <audio> element of HTML5 and manipulate it via JS, which is nice. Thanks to it and the rich information returned by the Deepgram API, the interactive scripts have the following features:

The audio is embedded in the HTML file, so it can be played directly from there (as long as the path of the source audio doesn't change).
Each sentence of the audio is printed in a different color, according to who is the speaker.
If you hover over a sentence, the translation to your language (if supported) appears.
If you click on a sentence, the audio plays only the sentence you clicked on, making it more easy to replay specific parts of the audio.

Here's a little demonstration on this features.

I recorded a short, sample conversation with my sister, who also provided the voice recorder. Some notes:

The translation on hover is not immediate because it uses the title HTML attribute, that has a little delay.
If you speak Spanish (or have a good ear) you'll notice that for the 4th intervention, it doesn't notice that it's a question, so both the transcription and the translation are wrong. That's what happens with automation!

In any case, I think it's neat and even those little mistakes made by the audio recognition and translation can be manually fixed by the teacher, and most of the work would be automatic.

Possible future improvements

Allow usage of output templates.
Add some tool for easy manual fixes.
Enable more flexibility on the options to use different services, if the user wants to.

I hope you like it! And if you are a language teacher or student, feel free to use it and please, give me some feedback!