Forem: Kevin Gilpin

Data is coming in - from Microsoft, CircleCI, and others. AI coding assistants can't make reliable commits. Let's talk about the RIGHT way to use AI for software development.

Kevin Gilpin — Fri, 30 May 2025 14:29:54 +0000

Kevin Gilpin

May 30 '25

AI coding assistants "drive developers crazy" and "submit broken code"

#ai #programming #webdev #productivity

Comments 1

2 min read

AI coding assistants "drive developers crazy" and "submit broken code"

Kevin Gilpin — Fri, 30 May 2025 14:27:28 +0000

The last twelve months have seen an explosion in AI coding assistants. The hype about these tools has focused on developer replacement - "developers will lose their jobs to AI".

How's that going? If you're following the news and metrics, a very different reality is becoming apparent.

🐛 Does code review come too late to be useful? Get AI-assisted code review suggestions automatically in real-time: bugs, security flaws, and performance problems.

💡 Check out my open source VSCode extension - OpenTips

Here are two specific ways in which AI software assistants are falling short of the hype.

1. (reddit) AI is driving developers "insane"

(reddit) My new hobby: watching AI slowly drive Microsoft employees insane

Microsoft developers are playing whack-a-mole with AI coding assistants while they run around in circles, fixing one bug while introducing another and paying no attention to the design of the project.

"No real understanding of what it's doing, it's just guessing"

2. (CircleCI) Real data about the impact of AI code on code velocity

(CircleCI) Hype cycle: The real impact of AI code in software delivery outcomes

41% rise in bug rates
40% lower success rate of CI jobs
50% increase in mean time to recovery (MTTR)

"developers using AI assistants ... more likely to introduce vulnerabilities while overestimating their code's security"

"while AI speeds up coding tasks, it also leads to more frequent failures"

Summary

My take - while AI coding assistants can be very useful, they are frequently being used in completely the wrong way, and by the wrong people.

First, don't expect AI coders to take the place of developers; even junior developers. Read the whole Reddit thread above if you want to see exactly why AI is not even capable of providing quality bug fixes for well-defined issues.

Second, don't use AI to generate code commits. Use it to:

1) Generate code design ideas
2) Help with troubleshooting
3) Get code review suggestions

In all of these cases, it's incumbent on the developer to curate and ultimately approve or reject the AI coding output.

I see no indicate that this fundamental partnership between AI and software developers is going to change any time soon.

Embrace this partnership in your work - value both the AI and the person, and let's stop with hype about AI as a human-replacing technology in software development.

🐛 Does code review come too late to be useful? Get AI-assisted code review suggestions automatically in real-time: bugs, security flaws, and performance problems.

💡 Check out my open source VSCode extension - OpenTips

Should you accept an AI coding suggestion?

Kevin Gilpin — Fri, 21 Mar 2025 16:32:08 +0000

How do you know if you should accept an AI code suggestion? I’m a software engineer with 30 years of experience and I’ve built AI coding tools myself, including AppMap and OpenTips. In this vide, I break down how to analyze AI code suggestions to get the best results. I have a lot of experience doing this and I can quickly simplify this process down to its essential steps.

This video uses an example of an OpenTips AI suggestion for an iOS app. I discuss the two criteria that I use to decide if I want to accept the tip:

Is it needed?

1) Does it fill an essential gap in the user experience?
2) Does it fix a security problem?

Is the implementation correct?

1) Do I understand it? If not, is it worth the research?
2) Is it adding a new tool or library? It has to add something that would be hard to do myself.
3) Does it follow the conventions of the existing code?
4) Will it make the code simpler without breaking a lot of tests?

Caption this image! (Azure & CrowdStrike outages)

Kevin Gilpin — Fri, 19 Jul 2024 13:36:11 +0000

Context: https://www.theguardian.com/technology/article/2024/jul/19/crowdstrike-microsoft-outage

I admit that this is just cheap fun at their expense but hey - if I take down the entire global infrastructure of hospitals, government, and airlines someday, you can feel free to razz me too.

How AppMap Navie solved the SWE bench AI coding challenge

Kevin Gilpin — Tue, 25 Jun 2024 16:11:31 +0000

AppMap Navie is an AI coding assistant that you can use directly in your VSCode or JetBrains code editor.

SWE Bench is a benchmark from Princeton University that assesses AI language models and agents on their ability to solve real-world software engineering issues. It's made up of 2,294 issues from 12 popular Python repositories, along with their human-coded solutions and test cases. It is considered to be the most difficult of the well-known coding benchmarks.

AppMap Navie recently posted 14.6% on SWE Bench, ahead of Amazon Q and 8 other tools. We were able to process the entire benchmark in under 4 hours, and at the lowest recorded cost of operation - up to 95% less expensive than other solvers.

How did we do it? Read on for useful techniques that you can apply to your own AI programming.

Why basic solvers fail

The easiest way to use an LLM to solve a code issue is simply to send the issue description to the LLM along with all the code files and prompt the LLM to generate a patch file. This is basically what the first generation of SWE bench solvers attempted to do. However, the solution rate of this approach is very low (single digit percents). Why?

1) Wrong context Most LLMs have a context limit which is too small to load the entire codebase. So, some guesses have to be made about which files to give the LLM. And when those guesses are wrong, the LLM fails to generate the right solution.

2) Failed patching LLMs are not good at creating patch files. Most LLM-generated patch files will fail to apply.

3) Broken code LLMs will generate code that is malformed. It won't even run cleanly, never mind pass the test cases.

4) Wrong code design The LLM does not understand the project design and architecture. So, it tries to solve the problem in the wrong place; or it fixes one problem while creating another.

You can see some of these early solvers on the SWE bench leaderboard:

Generation 2 - Agentic architecture

The next generation of solvers adopted a more complex architecture, in an effort to solve the problems above.

Basically, the idea of an "agent" is to give the LLM a wider variety of tools, and then run a loop in which the LLM chooses a tool and examines the results of using it.

Tools do things like:

Search the code for a keyword
Inspect a file
Run a program and examine the console output
Edit a file

Agents do substantially better on the benchmark:

However, most of the "agentic" solutions only appear on the SWE Bench "Lite" leaderboard. Why is that?

1) 💸 Cost Every tool an agent uses consumes tokens. Tokens cost money. Agentic loops use tokens over and over.
2) 🐢 Speed By design, agentic solvers can take a lot of time to explore the problem space. They can backtrack and repeat things they've already done. They can get stuck.

AppMap Navie architecture - Semi-agentic

Agents have a higher pass rate than Basic solvers, but they are slow and expensive. AppMap Navie takes an intermediate architecture, in which the solver is provided with powerful capabilities:

Rich, but selective, code context to enable retrieval-augmented generation (RAG) architecture.
A powerful and reliable tool for making file edits.
Self-healing feedback for fixing code.

Code context

It's inefficient and expensive to send huge amounts of code context to the LLM. Embeddings are slow and expensive to generate. But without the right code context, the LLM-generated solution will fail.

Navie uses a technique called "client-side RAG", in which the code is organized and searched locally. Client-side compute is fast, cheap, and much more efficient than sending huge token payloads to the LLM or building expensive embeddings.

Planning

With the right context selected, it's time to get right to code generation - right?

Wrong. Before code generation comes Planning. Human developers don't dive into coding without some kind of plan. Building an understanding of the system architecture is an essential step, it can't be skipped over by humans, and it shouldn't be skipped by AI coders either.

So, Navie performs an explicit planning step, in which the the issue description is combined with the context to produce a detailed plan. The plan includes:

A restatement of the problem.
A high level solution description.
A list of files to be modified.
For each file, a description (no code, yet), of how that file will be changed.

Here's an example of a Navie-generated plan.

File editing

Now, with the plan in hand, the LLM is ready to change code files.

Navie doesn't ask the LLM to generate patch files; it doesn't work. Instead, the LLM generates a "search / replace" pair of code snippets. This works most of the time, and a simple retry loop fixes up most of the occasions when it doesn't.

Here are the Navie-generated code changes that implement the Plan.

Lint repair

The LLM still might get something wrong. Common cases include:

Mistakes with indenting (particularly with Python).
Missing imports.

The Navie solver runs a linter, then feeds the linter errors back into the AI code editor. Most lint errors can be fixed this way.

An example of lint errors fixed by Navie.

Test repair

Still not done! If the solution generated by Navie breaks existing tests, it's probably not going to fix the issue properly. So the Navie solver runs the application test cases to try and catch and fix any incompatibilities that may have been introduced.

Now, it's ready

Now a patch file is created by simply diff-ing the AI-edited code with the Git base revision. This patch file is submitted to the SWE Bench harness for evaluation.

How is this so efficient?

The Navie solver runs for about 1/3 the cost of most other solvers; and it's 95% cheaper than some of the most intensive agentic solvers on the benchmark (of those that post their costs; many don't 🙁).

Efficient client-side RAG context saves $$ on embeddings and LLM tokens.
Lint repair and test repair fixes solutions that might be almost, but not quite, correct.
A smaller "tool" suite and a linear approach to solving the problem prevents the LLM from wandering down dead ends or getting lost in pointless loops.

Try Navie yourself!

Navie is available today, with no wait list. Here's how to get Navie, or learn more about AppMap:

⬇️ Download AppMap Navie for VSCode and JetBrains: https://appmap.io/get-appmap
⭐ Star AppMap on GitHub: https://github.com/getappmap
📰 Follow on LinkedIn: https://www.linkedin.com/company/appmap
💬 Join AppMap Slack: https://appmap.io/slack
ℹ️ Read the AppMap docs: https://appmap.io/docs

Announcing AppMap for GitHub - Runtime Code Reviews for Every Pull Request

Kevin Gilpin — Thu, 09 Nov 2023 01:17:20 +0000

Most of us have a love-hate relationship with code reviews. We rely on code reviews to ensure that the code we send to production is clean, performs well, and is free of security flaws. But code reviews can also add a lot of overhead and delays to shipping code.

Some people suggest that code reviews should be eliminated altogether. Well, I'm not in that camp, and here's why: Unexpected code behavior and deep runtime defects are responsible for a staggering 40% of performance problems¹ and 50% of security concerns² in software development. But, I do think that it's important to make code reviews easier for developers, and at AppMap we've created AppMap for CI to help do that.

“Security and performance reviews are part of the process on any engineering team, and ours is no different. It can take months to find issues and weeks to fix problems using static analysis and code reads. With AppMap, that time is reduced to minutes.”

-Padraig Coffey, CEO at Zartis.

If you know AppMap already, you know us for our VS Code and JetBrains extensions that automatically generate complete, accurate, interactive diagrams of your code. So now, with AppMap for CI, we've built on the solid foundation of AppMap to make code reviews easier, better, and faster.

The first CI platform we support is GitHub Actions. Our solution combines an AppMap GitHub App with GitHub Actions. When you build your project, the AppMap Analysis Action analyzes AppMaps from before and after each code change. It's able to identify critical performance, security, and stability issues and annotate specific lines of code in the pull request. It also performs deep analysis of test case failures and web service API changes, giving you a big head start on code review. And by the way, the performance and security flaws that AppMap finds are not discoverable by static analysis tools - because modern code is too dynamic for static analysis tools to understand.

“When we initially adopted AppMap, we were only using it for visualizing N+1 queries locally. We later integrated it into our CI run, and we are finding the GitHub comments posted by AppMap to be very useful. The OpenAPI integration, in particular, was a surprise: it’s nice to see an automatically generated summary of what endpoints have changed in a PR."

-Paul Kuruvilla, CTO at CodeCrafters.

Your data stays with you

We know that your code is sensitive, and you don't want it to be transmitted to 3rd parties. All AppMaps are stored as artifacts in your GitHub build system and are sent directly to your browser when you view them. The data only resides in two places - the GitHub Actions environment, and your machine. Learn more at appmap.io/security.

A tour of AppMap's "runtime code review"

✅ AppMap issues a summary report of its runtime analysis as a comment on every PR. The PR comment includes four key categories:

Failed tests

Identifies the root cause of test failures with detailed insights and direct links to the line of code which caused the error. AppMap also includes source diffs and behavioral sequence diffs for efficient debugging.

API Changes

Detects API route differences, including new, deleted, or modified routes, as well as alterations in response attributes like body, content, and descriptions.

Security flaws

Uncovers issues such as broken or missing access control, deprecated cryptography, improper session management, and missing authentication. AppMap focuses on the runtime security flaws that dominate the OWASP Top 10, and can't be reliably detected by static analysis.

Performance problems

Identifies performance problems like N+1 SQL queries, very complex queries, RPC anti-patterns, and plain old slow code.

Example - Performance problems

Here's an example of a code change that introduces four new performance problems.

The AppMap runtime code review includes details about the flaws detected, a stack trace, and links to the offending source code.

AppMap highlights specific code changes that are contributing to the flaw. It's able to focus on the specific code changes that are relevant to the newly introduced problem.

AppMap also links to an interactive "diff" diagram showing how the code change has introduced the flaw.

Example - Security flaw

Here is an example of AppMap describing a security flaw and the code change that produced it.

Included in the comment is a stack trace showing what led to the security flaw, including links to the source code.

AppMap also provides a full set of interactive visualizations that describe the code behavior that resulted in the security flaw.

Wrap-up

To get started, install the AppMap GitHub App from the GitHub Marketplace.

Not ready to try it for yourself? We have example projects on our GitHub so you can check out AppMap for GitHub Actions at work.

Links

🤖 AppMap for GitHub Actions: https://getappmap.com
⬇️ Download AppMap for VSCode and JetBrains: https://appmap.io/download
⭐ Star AppMap on GitHub: https://github.com/getappmap
📹 Follow on: https://youtube.com/@appmap
💬 Join AppMap Slack: https://appmap.io/slack
ℹ️ Read the AppMap docs: https://appmap.io/docs

References

Unlock developer creativity with GitHub Actions

Kevin Gilpin — Tue, 24 Oct 2023 16:27:48 +0000

Since its release in November 2019, GitHub Actions has continued to gain in popularity. Fully 72% of AppMap users tell us that their team is using GitHub Actions! I've been personally using GitHub Actions intensively for about six months, and it feels to me like a substantial leap forward in automation technology. Why? As CI/CD has gained importance, it has increasingly become disconnected from the productivity needs of developers, primarily due to security concerns. I find that GitHub Actions allows me to automate code in innovative ways that are not possible within the confines of traditional CI/CD pipelines.

How GitHub Actions works

In case you're new to GitHub Actions, here's a quick overview:

A GitHub Action is a code snippet that runs on GitHub-hosted infrastructure (or you can provide your own compute).
GitHub Actions can be combined together into sequences called Workflows.
A GitHub action can be defined in a GitHub repository, and imported by other projects. So they are highly sharable and reusable.
A Workflow is a YAML file that lives in the .github folder of the code itself.
Because a workflow is a git-managed file, each version of the code can have its own workflows. That makes it easy to try out new workflows on a branch, and then merge them to main when they are debugged.
There are lots of different ways to trigger workflows. Common triggers include "run on pull request", "run on push", and "run on schedule".

With this combination of features, GitHub Actions gives developers the power to easily create just about any kind of code automation.

How we got here

I'm, um, old enough that when I started my career, manual builds were the norm. Build automation tools like Jenkins came along pretty soon afterwards, and brought repeatability to the build + test + package cycle.

Deployment was still almost always a manual step. The next big step was CI/CD, which moved build automation to cloud-hosted infrastructure and also automated the deploy steps - this, of course, was made possible by cloud computing infrastructure: Amazon EC2 and all the services that derived from it.

Trouble in paradise

CI/CD became the core of DevOps, which expanded automation into many new areas. Most of this growth and expansion has been very positive, but the rigor of the automation has actually had some drawbacks for developers. Why? Because CI/CD has such so much power to package and deploy code, it's also become an area of high security risk.

Hackers quickly identified and exploited multiple avenues of attack:

Inject malicious code directly into the target software as it's being built.
Use a compromised CI job is a stepping-stone for access to production systems.
Use inside knowledge of the code to build the perfect exploit or backdoor.
Attack the CI/CD vendor directly to obtain credentials in bulk. (Four well-known vendors in the CI/CD space have all been breached in the last decade).

Security is a real concern with CI/CD systems. As a result, they are now highly protected. This is necessary - but it's also meant that CI/CD is not a developer-friendly environment anymore. CI/CD is too tightly controlled to be a good place for developers to try out new experiments in code automation.

GitHub Actions to the rescue

This is where GitHub Actions comes in. A GitHub Action does not need to be part of the "blessed" CI/CD process. It can perform any kind of ad-hoc job that a developer might want it to. For example, there are GitHub Actions to report test status, inject comments into commits and PRs, synchronize with other systems, generate code, send notifications, look for bugs, etc. And it can do these things "off to the side", out of the way of the official pipeline job. It's no longer necessary to modify the master build file (think: travis.yml, Jenkinsfile, .circleci) in order to try out some new code automation.

Since Actions are controlled by YAML files that any developer can create and modify - and they run in the cloud with direct and easy access to project source code - GitHub Actions combine power and safety in a way that's not found either in local developer environments or in traditional CI/CD tools.

Powerful, tune-able performance

As a developer, you have a good understanding of performance considerations such as CPU capabilities, memory use, and parallelization. So, you'll be glad to know that GitHub Actions is powerful enough to let you control and optimize exactly how your workflow uses and consumes compute and storage.

Each Workflow runs on a pre-defined runner instance, which is basically a container running with a pre-defined allocation of CPU cores and amount of memory. So you can run small, lightweight jobs with little cost.
Is your Workflow slow and sluggish? You've got options; several options in fact. First, you can bump the workflow up to a larger runner instance. How does 64 cores and 256GB of RAM sound? You can get that. What if the code you are running doesn't scale that well with cores? Well, you can apply matrix build strategies as well.
You can store and retrieve data within and across workflows. Both a cache store and artifact store are available. The cache is really fast, and it can only be accessed from within Workflows. Artifacts are slower, but they can be accessed from anywhere using the GitHub REST API. Cache expiration times and artifact retention times are both customizable, so you can tune the amount of money you are spending on these storage resources.

Proceed! (With caution)

While it is separate from the official CI/CD pipeline, the privileged operation of GitHub Actions does still pose a security risk. GitHub Actions have access to the code; can commit and push code; can be given access to secrets that may be mishandled. Recognizing this, GitHub provides repository and organization settings that can be used to control which GitHub Actions are allowed to run, and even whether the feature is enabled at all.

Furthermore, each GitHub Action Workflow runs with its own explicit permissions. So, the security capabilities of each Workflow are clearly stated, and less-trusted Actions can be run with lower privilege settings than more-trusted Actions. This type of "least-privilege" access management is a highly effective security strategy, and it's not possible in monolithic and/or legacy CI/CD jobs.

As always, freedom comes with responsibility and we've heard of at least one organization that has locked its developers out of GitHub Actions after incidents of careless, insecure coding. So, be aware and mindful of the security controls that are available, and use them!

How we're using GitHub Actions and AppMap together to improve code quality

AppMap is a tool that records code traces into JSON data files, which can be visualized and analyzed to improve code understanding and quality. GitHub Actions offers a way to create a comprehensive AppMap data set for every pull request, providing a complete profile of the runtime architecture, performance, and security characteristics of your application. It also offers a way to compare the AppMap data sets of two different code versions.

Comparing AppMaps of two code versions (a baseline and a proposed change set) is an amazingly powerful technique for analyzing code for architecture changes, security flaws and performance problems. With AppMap for GitHub Actions, we are offering exactly this capability. Our Early Access program is wrapping up now, and we are transitioning to general availability. Come check it out!

Because AppMap runs in your GitHub account, the data is processed within your account and it stays within the boundaries of it. You don't have to share source code or sensitive data with any third party. This is the type of win/win/win capability that GitHub Actions offers. A high level of developer control, good performance, and good security characteristics.

How are you using GitHub Actions?

Has your organization adopted GitHub Actions? How enabled are developers to add and change GitHub Actions? Did you encounter any problems or mishaps along the way that others could benefit from hearing about? We would love to hear about it in the comments below.

Links

🤖 Get Early Access to AppMap for GitHub Actions: https://getappmap.com
⬇️ Download AppMap for VSCode and JetBrains: https://appmap.io/download
⭐ Star AppMap on GitHub: https://github.com/getappmap
📹 Follow on: https://youtube.com/@appmap
💬 Join AppMap Slack: https://appmap.io/slack
ℹ️ Read the AppMap docs: https://appmap.io/docs

How to interpret a sequence diagram

Kevin Gilpin — Tue, 23 May 2023 14:03:56 +0000

Follow along on this tour of a sequence diagram, covering the following concepts:

Lifeline: A vertical line that represents an object or component in the system. The lifeline shows the lifetime of the object/component and its existence over time.
Object header: The rectangular box attached to the top of the lifeline that contains the name of the component (package, class, etc.) that the lifeline represents.
Activation bar: A horizontal line that represents the duration of time when an object is performing an action or processing a message. It is shown as a solid line on top of the lifeline.
Message: A horizontal arrow that represents the communication between objects or components. It is shown as a line with an arrowhead pointing from the sender to the receiver. Messages can be synchronous, asynchronous, or self-referential.
Return message: A horizontal arrow with a dashed line that represents the return message from the receiver to the sender.
Self-message: A message that is sent by an object to itself.
Combined fragment: A diagram fragment representing a specific flow of control. It is shown as a box with a label that indicates the condition under which it is executed.

Quickly learn how new-to-you code works using sequence diagrams

Kevin Gilpin — Mon, 22 May 2023 16:27:10 +0000

You may have heard about or experienced the value of sequence diagrams. In this article, I’ll introduce sequence diagrams and describe what they are good for. I’ll also give you a tour through the features of sequence diagrams, and explain how to use each feature to interpret the behavior of your code. Then I’ll show how AppMap makes sequence diagrams even better, by (a) generating diagrams from your running code and (b) making diagrams fully interactive.

What is a sequence diagram?

Sequence diagrams are a type of Unified Modeling Language (UML) diagram that shows the interactions between different components or objects in a system over time. They show the flow of messages and the order in which these messages are exchanged. They are great for understanding how code works because they show enough detail to communicate the key elements of code behavior while being compact enough to fit a lot of information onto a single page.

Uses of sequence diagrams

Sequence diagrams are particularly useful for designing and testing software systems that involve multiple components, asynchronous events, or complex control flows. Here are some common use cases for sequence diagrams:

Understanding system behavior: Sequence diagrams can help you understand how different parts of a system interact with each other, and how the system behaves in response to different inputs.
Designing new systems: Sequence diagrams can be used as a tool to design new systems, by helping to identify the components needed and how they should interact.
Communication: Sequence diagrams can be used as a visual aid to communicate system behavior and design to stakeholders, such as developers, project managers, and clients.
Documentation: Sequence diagrams can be used to document the design and behavior of a system, providing a clear and concise description of how the system works.

How to interpret a sequence diagram

Let’s talk about the basic features of a sequence diagram. These features are present in any sequence diagram; we’ll look at interactive AppMap capabilities in a later section.

Lifeline: A vertical line that represents an object or component in the system. The lifeline shows the lifetime of the object/component and its existence over time.
Object header: The rectangular box attached to the top of the lifeline that contains the name of the component (package, class, etc.) that the lifeline represents.
Activation bar: A horizontal line that represents the duration of time when an object is performing an action or processing a message. It is shown as a solid line on top of the lifeline.
Message: A horizontal arrow that represents the communication between objects or components. It is shown as a line with an arrowhead pointing from the sender to the receiver. Messages can be synchronous, asynchronous, or self-referential.
Return message: A horizontal arrow with a dashed line that represents the return message from the receiver to the sender.
Self-message: A message that is sent by an object to itself.
Combined fragment: A diagram fragment representing a specific flow of control. It is shown as a box with a label that indicates the condition under which it is executed.

Here’s what this particular sequence diagram can tell us about the code that it models. The first message begins from outside of the diagram, indicating that it originates from a source external to the system being diagrammed. In the case of this diagram, the external source is whatever entity made the POST request. The next message is the function call to process_action in the actionpack package, which then “activates” the actionpack lifeline, as can be seen from the lifeline changing from a dashed line to a rectangular box, which is the activation bar. This means that the package is now “active” in the system. Next, as part of the process_action message, we see a loop with two calls to the write function in the activesupport package. The loop is depicted as a combined fragment, as depicted by the grey box labeled “loop”. Within this loop box, we see our first example of a return message, which is the dashed arrow with integer above it, indicating the return type. This indicates that the write function executes and returns before we proceed to the next function call. If we skip a couple of function calls to update, we see an example of a self-message, where the actionpack package makes a call from one part of itself to another, which is depicted by a message arrow originating from the actionpack lifeline and then returning to the same lifeline. You can see that this increases the width of the activation bar temporarily. There are more function calls after these, but that covers the basics.

Making sequence diagrams interactive

As useful as sequence diagrams are, there can be made even better. AppMap is a tool that adds two great features to sequence diagrams - automatic diagram generation, and interactivity.

Let’s get right into an example. Here’s a sequence diagram that was generated by running a test case for a Twitter-like application. The name of the AppMap is:

Following should unfollow a user with Ajax

☝️ Click here to open a live diagram! ☝️

When you click on this diagram, you’ll see immediately that the diagram is “alive.” You can expand, collapse, refine, and drill down for detailed trace information.

How to interact with AppMap sequence diagrams

Here are several tips for getting the most out of AppMap sequence diagrams:

Show event parameters, return values, and a link to source code: If you click on the name of the message arrow, then that event will be selected, and you can see more information about it in the sidebar.

Hide a lifeline: If you click on the small x icon in the top right corner of an Object header, then you will hide that lifeline. This can be useful if the sequence diagram is too large and contains code that is not useful for the task at hand. For example, you might suspect that there’s a bug in the code, and you have a hypothesis that the logger package is not relevant. In that case, you can hide the logger lifeline to make it easier to find the relevant information in the diagram.

Collapse a section of the sequence diagram: You can click on the [-] button on a message arrow, next to the function name, and it will collapse that section of the diagram. This is useful in a similar way that hiding a lifeline is useful. Sometimes there is extraneous information in a diagram that is not relevant to your current task. Temporarily hiding that information is a great way to quickly find what you’re looking for.

Expand a package into classes: If an object header is a package and it contains two or more classes, then you can click on the icon in the top left corner of the object header and expand the package. This lets you get a more detailed view of the interactions between objects in the system and can help you better understand the code.

The power of interactive sequence diagrams

Sequence diagrams are a valuable tool in software engineering for modeling and visualizing complex interactions between objects in a system. Interactive sequence diagrams take it to the next level, enabling you to:

Select specific events for more details.
Expand and collapse object headers.
Hide lifelines to remove clutter and focus on the most relevant information.
Collapse sections to focus on the high level information.

How have sequence diagrams helped (or hurt!) you?

Have you used sequence diagrams on a project? Please share your personal experience by adding a Comment.

Links

⬇️ Get AppMap for VSCode and JetBrains: https://appmap.io/get-appmap
⭐ Star AppMap on GitHub: https://github.com/getappmap
🐦 Follow on Twitter: https://twitter.com/getappmap
💬 Join AppMap Slack: https://appmap.io/slack
ℹ️ Read the AppMap docs: https://appmap.io/docs

Acknowledgements

Thanks to Adam Trotta for the first draft of this post, as well as for the images.

Automatically generate interactive sequence diagrams of your Java code's runtime behavior

Kevin Gilpin — Tue, 11 Apr 2023 19:14:58 +0000

The AppMap team is excited to announce that automatic, interactive sequence diagrams for Java applications in the IntelliJ IDE is live!

(Also available in VS Code)

Sequence diagrams are incredibly powerful for visualizing and understanding runtime behaviors. Their inherent structure - objects flowing across the page, time flowing down the page - is intuitive and easy to learn. And they are one of the best tools for developers and technical non-developers (like engineering managers and product managers) to use when discussing code design.

AppMap can now generate sequence diagrams automatically from any running Java application.

Before, sequence diagrams can only be created by manually inspecting code listings or debugging a system. With AppMap, you simply run your application and accurate sequence diagrams are generated instantly.

They're interactive 🖱!

Once you install AppMap into IntelliJ and generate your own sequence diagrams, you can click on any function call in the sequence to inspect the code.

Each vertical "lifeline" in the diagram floats as you scroll down long interactions, making it easy to keep track of which lifeline is involved in any function call.

Lifelines can also be hidden to make the sequence diagram even more readable. This is especially useful if you want to hide calls to things which are less important (for example, a logging class).

Examine loops and call timings ➰

Loops are automatically detected and rendered in a summarized fashion to prevent the sequence diagram from becoming too long. All function call durations are shown, so you can quickly see where your code is spending the most time doing its processing.

Always accurate sequence diagrams ✅

The biggest challenge with manually-created sequence diagrams is that they quickly go out of date compared to the current behavior of the system. Because AppMap records the runtime behavior of your application, the sequence diagrams it creates are always guaranteed to be accurate and complete.

Easily view database queries - even auto-generated ones 👀

AppMap captures all SQL queries made to a database while your application runs, even if your database queries are automatically generated by an ORM/JPA framework like Hibernate.

3 simple steps

Install the AppMap plugin into JetBrains
Run your application (or your tests) with the AppMap library loaded
Open the generated AppMap to view the sequence diagram

Available now for IntelliJ and JetBrains

AppMap with sequence diagram support is now available for installation from the JetBrains marketplace, and it is free for personal use and for open source projects. Get it now!

(Also available in VS Code)

Auto-magically generate sequence diagrams of your code's runtime behavior

Kevin Gilpin — Tue, 29 Nov 2022 19:43:46 +0000

Sequence diagrams are frequently described as "the best part of UML". Their inherent structure - objects flowing across the page, time flowing down the page - is intuitive and easy to learn. And they are one of the best tools for developers and technical non-developers (like engineering managers and product managers) to use when discussing code design.

Update! April, 2023 We now have developed a new version of this feature, which is fully interactive and doesn't require Java or PlantUML. Read more about AppMap interactive sequence diagrams for JetBrains.

Or, keep reading to see how we first developed this feature using AppMap and PlantUML 👇 👇 👇

Historically, creating a sequence diagram required specialized software. But accordance with the “everything as code” movement, tools like PlantUML and Mermaid have kept sequence diagrams relevant by providing a fairly basic text language that can be used to generate sequence diagrams. Think: Markdown for diagrams.

But while it’s fairly easy (and rewarding!) to create sequence diagrams using text files, it still takes effort. And, being documentation, diagrams-as-code still suffer from the achilles heel of documentation, namely - getting out of date.

What if you could have the best of both worlds - sequence diagrams, with no effort? Read on, to learn how you can generate sequence diagrams just by running your code.

Note Would you prefer to watch rather than read? Check out "Auto-magically generate sequence diagrams of your Rails App using VSCode" on YouTube

How it works

A sequence diagram illustrates a particular code flow, like an HTTP request or the processing of a job. So the first step in generating a sequence diagram is to run your code through a particular flow. I suggest two ways you can do this:

Run a test case
Run your app and interact with it, either through its UI or by sending API requests from a tool like Postman.

In order to generate a diagram of the code behavior, we need to record exactly what happens as the code runs. For this, we’ll use AppMap, a free and open source runtime code analysis tool that I created. You can use AppMap with Ruby, Python, Java and JavaScript - you can find AppMap setup instructions here. AppMap can record both test cases and live application API requests, so you can use either technique to acquire the diagram data.

Once you’ve recorded a test case or an interactive session, you can use either the AppMap CLI or the AppMap extension for VSCode to generate a sequence diagram. I’ll cover both options here.

In both cases, the AppMap tools will generate a PlantUML file, which will then be rendered as SVG. So, before proceeding, download the PlantUML JAR file; the latest version at the time of this writing is plantuml-1.2022.13.jar. The rest of this post will assume that you’ve saved this file as ~/Downloads/plantuml.jar. Adjust as necessary.

Note You'll also need to have java available. On MacOS, you can simply run brew install java.

Note You can find copies of all the files used in this blog post in this Gist

VSCode extension

First, using the VSCode extension - because it’s a bit easier.

Start by installing the AppMap extension from the VSCode Marketplace. You’ll be prompted to configure your project for AppMap, if you haven’t done so already. Once you’ve set up your project, commit the AppMap configuration changes to Git so that other members of your team can use AppMap without having to go through the setup process themselves.

Now, open VSCode Settings:

Then search for “AppMap Sequence Diagram” and configure the location of the PlantUML JAR.

You’re ready to generate a sequence diagram! Open the AppMap view by clicking the icon in the extensions sidebar, then right-click an AppMap to open the context menu. Choose “AppMap View: Generate Sequence Diagram”.

Now you’ll get prompted twice. The first time through, just hit Enter to accept the defaults. Once you've seen the generated diagram, you can customize its behavior using these options.

You’ll see a brief progress indicator, and then your diagram will open in the browser!

Interpreting the diagram

Here’s a sequence diagram that I generated from the Rails Sample App 6th Edition. (Are you feeling adventurous? Try generating AppMaps of Mastodon!).

From left to right, you see the inbound HTTP server request POST /login. From there, the code flows to the controllers package, then through helpers, models, to the database, and through views.

Each left-to-right arrow is an HTTP server request, function call, SQL query, or HTTP client request. The right-to-left arrows are the return values.

When code runs in a loop, you’ll see a Loop box enclosing the repeated behavior.

Here’s an example of a Loop, also from the Rails Sample App, in which a sequence of two SQL queries is repeated 29 times! This is a common performance flaw called N+1 query - here’s a blog post all about N+1 queries in Rails.

N+1 queries can be tricky to find, because all these queries are issued automatically by an ORM system like ActiveRecord (Rails), Hibernate (Java), or Django ORM (Python). In a sequence diagram, they are really obvious! (You can also use AppMap to automatically identify performance flaws like this one.)

Exporting and sharing

If you want to see the generated the PlantUML code, click an AppMap to open it and then go to the Explorer view. Next to the appmap.json file, you’ll see the PlantUML file (.uml) and SVG (.svg).

You can take this UML file or SVG and drop it into any other tool, like a GitHub Issue, Jira ticket, pull request, or Slack message.

CLI

You can also generate sequence diagrams using the AppMap CLI. The AppMap CLI is an NPM package called @appland/appmap, so there are three ways to invoke it:

Install it with npm (npm install --save-dev @appland/appmap; npm run appmap)
Install it with yarn (yarn add --dev @appland/appmap; yarn run appmap)
Run it with npx (npx @appland/appmap@latest)

I’ll use option (3), because it’s easy to document. But to avoid downloading the package from NPM over and over, you’ll probably want to use (1) or (2).

First, let’s look at the command help.

$ npx @appland/appmap@latest sequence-diagram --help
appmap sequence-diagram appmap

Generate a sequence diagram for an AppMap

Positionals:
  appmap                                                     [string] [required]

Options:
      --version     Show version number                                [boolean]
  -v, --verbose     Run with verbose logging                           [boolean]
      --help        Show help                                          [boolean]
  -d, --directory   program working directory                           [string]
      --output-dir  directory in which to save the sequence diagrams
      --format      output format
                             [choices: "plantuml", "json"] [default: "plantuml"]
      --exclude     code objects to exclude from the diagram

For basic usage, the only option you really need is the appmap argument. Here’s an example:

$ npx @appland/appmap sequence-diagram tmp/appmap/minitest/Following_followers_page.appmap.json
Printed diagram tmp/appmap/minitest/Following_followers_page.sequence.uml

Note that the diagram is printed as a *.sequence.uml file in the same directory as the AppMap file. Now, we’ll use PlantUML to convert this to an SVG.

$ java -jar ~/Downloads/plantuml-1.2022.8.jar -tsvg tmp/appmap/minitest/Following_followers_page.sequence.uml

Check that the file exists:

$ ls tmp/appmap/minitest/Following_followers_page.sequence.svg       
tmp/appmap/minitest/Following_followers_page.sequence.svg

Open it in the browser:

$ open tmp/appmap/minitest/Following_followers_page.sequence.svg

There you have it! Of course, if you want to you can edit the UML by hand, and you can upload and share both the UML and SVG files anywhere you like.

Wrap-up

The AppMap CLI @appland/appmap includes the sequence-diagram command which generates a PlantUML text file from AppMap data (which, by the way, is JSON). The PlantUML JAR file can transform the text file into SVG (or other formats like PNG). The AppMap extension ties all these tools together into an integrated UI experience that makes it easy to continuously write code, run your app, write tests, create AppMaps, generate sequence diagrams and share them with your team.

Now, try it yourself! If you have questions, thoughts or comments, join us in the AppMap Slack. We’d love to hear from you.

Visualize the architecture of your Node.js app in under 5 minutes

Kevin Gilpin — Tue, 01 Feb 2022 13:48:10 +0000

AppMap is an open source VSCode extension which helps you improve your code by showing you how it really behaves, feature by feature, as you work. It does this by automatically generating beautiful, interactive maps of architecture and code behavior right in your editor.

This less-than-5-minute video shows how install the AppMap extension, how to configure it to map your Node.js code, and how to navigate your code using the interactive diagrams.

It uses an Express.js app as an example program.

For full usage instructions, check out the AppMap documentation.