Forem: Pedro Madrid

Stopping a Java memory leak

Pedro Madrid — Mon, 21 May 2018 22:57:24 +0000

Java objects are stored in a memory area called the heap. This area increases and decreases in size, but has a limited space defined by several factors, depending on the Java version your application is running on. No matter how you reach that limit, you'll get an OutOfMemoryErrortype of error. If you increase the maximum memory allocation by setting the jvm parameter -Xmx and your application keeps hitting the limit and throwing this errors, it is very likely you have a memory leak.

A memory leak happens when your application starts storing more objects in memory than the Garbage Collector can get rid of. An object can be removed from memory by the Garbage Collector when there are no references to it, so that means that your application is keeping references to objects it no longer needs, or that need to be handled in a different way to prevent filling up the heap.

Taking a heap dump

The first thing you need to have in order to properly know what's going on in your application's memory management is a heap dump. There are several ways in which you can accomplish that. The first one is by enabling automatic heap dump when your app reaches an OutOfMemoryError. You can enable that via command line options, like this, when starting the application:
java -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=my/favorite/path MyApp

-XX:+HeapDumpOnOutOfMemoryError makes the JVM take a heap dump when the app runs out of memory.
-XX:HeapDumpPath=my/favorite/path specifies the location where you want the dump to be saved.

The other way to obtain a heap dump is using the jmap jdk tool. If your application is running only with the JRE, you will have to install the JDK so you can use jmap. Usage of the command is as follows:
jmap -dump:format=b,file=heapdump.bin <process_id>

The options below are useful to perform analysis on a production environment, since you can extract the heap dump and copy the file over to your machine and load it into a tool. Another way to extract a heap dump is by attaching a tool to a running java application remotely. That involves using JMX, which in turn requires an open port, but if the application is running locally, in the same machine as your tool, you will see it listed:

Once you are connected to the application, you will be able to load the heap dump. Refer to documentation and tutorials for each tool to learn how to take the heap dump.

Analyzing the data

After you have the heap dump, it's time to look into the data and determine what's causing the memory leak. The dump file can be analyzed with many tools, commercial, free and open source. I would recommend two tools: jvisualvm, and eclipse memory analyzer. If you have installed the JDK in your environment, you already have jvisualvm (it's in the bin directory). The eclipse memory analyzer (MAT) is an eclipse.org project, and can be downloaded from here.

Jvisualvm will give you a list of the most used objects in the heap dump loaded, this can help identifying the memory leak by inspecting on the objects with more instances in the heap or those that are occupying the most amount of bytes.

If you decide to use MAT, I would highly suggest follow the process described in this blog post, in which the author describes the way the tool organizes the data and gives instructions on how to point down the culprit.

Trabajando con software legacy

Pedro Madrid — Fri, 02 Jun 2017 13:45:13 +0000

Es preocupante que últimamente los ingenieros de software estén condicionando ser contratados o transferidos de un proyecto a otro aduciendo que no quieren trabajar con software legacy (también conocido como trabajo en proyectos de mantenimiento).
En el mundo de la ingeniería de software el hecho de tener que trabajar con software existente y en operación es bastante probable. Si no fuera así, entonces nunca saldríamos a producción, y no se cumpliría el objetivo principal del proyecto: brindar una solución tecnológica a una necesidad de negocio. Y como en todo negocio, el retorno de la inversión (ROI) es importante. Pagar una gran suma de dinero a un grupo de desarrolladores para que construyan un sistema que soluciona una necesidad de negocio, y que ese producto no vea la luz del sol, es una gran pérdida de tiempo y dinero. Así que es inevitable que en algún momento nosotros como ingenieros de software tengamos que dar soporte al correcto funcionamiento de un producto en operación.
Qué sentido tiene trabajar en un producto utilizando las últimas tendencias en diseño y tecnología, si finalmente no nos preocupamos por lo que en realidad es importante, que es el negocio? La finalidad de un proyecto de software no es poner servidores, frameworks y líneas de código a trabajar juntas. La verdadera finalidad de un proyecto de software es solucionar una necesidad, y esa debería ser el principal objetivo de cada uno de los integrantes de un grupo de trabajo.
Una de las cosas más gratificantes de ser un ingeniero de software es ver cómo las líneas de código que uno escribió y todo el esfuerzo que se puso en un proyecto están siendo puestas en ejecución y aportando a que la solución que requería el cliente efectivamente está mejorando su negocio y satisfaciendo sus necesidades. Pero no todo llega hasta ahí. Tenemos que asegurar que ese software cumple a cabalidad con las exigencias del entorno sin perturbar el negocio.
Utilizando técnicas de refactoring, automatización, deja el “lugar mejor de lo que lo encontramos y finalmente lograr la satisfacción de los usuarios finales le da un mayor sentido a nuestra labor. Disfrutar el camino, aunque este sea empedrado, a veces da más satisfacción que cuando tienes todo servido en bandeja de plata. Y trata de que tu nuevo código sea limpio, expresivo y legible, para que a tus futuros colegas el camino esté menos empedrado que el que tú tuviste que sortear.

Originally published in Medium

Boosting the skills of your team

Pedro Madrid — Tue, 02 May 2017 13:19:41 +0000

When a new software project is starting, a new team needs to be put up together according to the client and the project’s needs. Ideally, the team should be composed of highly skilled members with the adequate knowledge in the technology stack as required by the project’s architecture. Taking into account the availability of the developers within the different projects in the company and the recruiting process in place to on-board new hires and the costs associated, teams end up having a mix of highly experienced and junior developers. What usually occurs is that a few experienced engineers with different background join the team while some others less experienced are given the chance to acquire some new skills and knowledge as the project unfolds. The trick here is: how to make those less experienced developers improve their skills and motivate them to become the future leaders in this and other projects? I will try to elaborate on some techniques that in my experience have given good results:

On-boarding project

To be able to demonstrate how the company works, the best way is to have the people work on a real project right away. But working on a real project is risky, so we setup a fake project where the people get familiar with the methodology and the environment. This training makes emphasis on the methodology, which is important across all projects no matter the technology skills needed. The fake project help us on-board new hires, they work on user stories, get familiar with general agile and scrum methodology concepts and they actually develop their own user story, it gets peer reviewed, tested, deployed and demoed to product owners.

On demand training

As the initial requirements of the project start to come up and the technology stack is identified as detailed as possible at that stage, we need to match the skills that the team has and the skills the project needs. In order to start the first sprints/iterations with a team that can handle the client’s expectations, team members need to be trained according to those needs. What we usually try to do is find experts inside the company that know the technology the new team needs to know about. It is a good idea to keep track of people’s skill in your company, so everybody know who can they reach if they want to know about something or get help. A small training plan comes up and these experts take care of the training and we arrange the needed sessions to train the team, using different on-hands teaching techniques, such as workshops or coding dojos, among others. If the knowledge is not found inside the company, outsiders are hired to provide the training.

Peer reviews

When the desired skills are present in some of the team members, one of the most powerful techniques to spread the knowledge is to do peer reviews. This technique allows less experienced developers get feedback on their work and also allows the reviewers to strength their coaching skills. A good practice to accompany the peer reviews is to do a periodic development meeting, where the reviewers present to the whole team the most common mistakes and/or key concepts that they think are worth sharing with the whole team.

Pair programming

This technique is basically the peer reviews taken to the extreme. Teammates work together in the same task (and on the same workstation), one of them having more knowledge than the other on the technologies and/or the business, so the less experienced developer gets up to speed more quickly than doing it by him/herself. This technique is very helpful specially when we need to on-board people to an existing team very quickly, or when we need to acquire knowledge from the client: we set up pair programming sessions that allow a full immersion in the development methodology, the technology stack and the business model, according to the team’s needs.

Encourage knowledge sharing

So you think you know Docker? or TDD? or even basic stuff like OOP? Think twice! When you want to share what you know with others and start structuring all the information in your head and you are preparing a presentation, you realize maybe there’s a few things you need to clarify first. Giving a talk in your local community or having a scheduled tech meeting, like a lunch & learn, open space, etc., makes you study the subject more deeply and have all bases covered. Encouraging your teammates to give short talks about a topic is challenging and stimulates the person to clarify all concepts they want to talk about.

Conclusion

Making a team acquire new skills is a combination of providing them with appropriate tools and techniques, and their personal will to improve as software developers. You need to provide an adequate and stimulating environment that can be leveraged by all types of developers with any kind of backgrounds.
What else has worked for you?

Post originally posted on medium.

Hi, I'm Pedro Madrid

Pedro Madrid — Wed, 22 Feb 2017 17:00:41 +0000

You can find me on Twitter as @pmadridb

I live in Halifax, NS.

I work for Lixar

I mostly program in these languages: java, javascript, python.

I am currently learning more about being a good developer.

I hate tea.

Nice to meet you. ;)