Forem: Sakir Temel

How to Communicate Your Process Visually using BPMN as Code

Sakir Temel — Mon, 23 Jan 2023 17:50:41 +0000

When you have multiple participants in a process, and we often do, you realize that you're dealing with a lot of miscommunication / misalignment issues. It's so hard to remember who was doing what and when, furthermore it's even harder to make some optimization in the process itself as you lose the control over it. Besides you'll need to group all the participants together for them tell their opinions.

Processes are long and complex, but they're there even if you don't formalize them. So, why don't we find a way to approach it in an easy way? First thing each of us would do in such a situation is drawing some boxes and lines, a.k.a. diagrams. Today, we're going to talk about ways of drawing some diagrams, in a comprehensive yet simple way.

Business Process Model and Notation (BPMN)

BPMN diagram is like Flowchart, but the tasks (activities) are grouped by the participants. It can be drawn / read by both business or technical people as the main purpose is to communicate a process in a simple yet complete way.

BPMN is actually a set of standards has been used for years for complex enterprise processes, and nowadays it's becoming more accessible thanks to the development of the new techniques. Web based tooling (like Camunda, BPMN.io), more platforms supporting integrating diagrams into the flows, and remote work culture all helps us to use BPMN easier. Besides all of that, we drive/lead more and more initiatives together and we're keep looking for the efficiency and improvement.

I started to see more and more teams are formalizing their development / pull request flows. Many of them are going with a very long, not up-to-date flowchart diagrams just to keep the things "simple" and not require learning any better suited specifications like BPMN. It's a fair tradeoff, but at the end, it makes these diagrams not serving to their purposes. We should rather prefer solutions that makes BPMN diagrams more friendly, and super easy to learn/draw.

Diagram as Code

Diagram as Code is a trending concept that automatically generates diagrams from a simple text/code. Some of the pioneers are PlantUML, Mermaid and Kroki. It has several great benefits like having your architecture diagrams close to your code and your development flow. If you want to learn more about it, there's a presentation that covers main benefits in action.

BPMN as Code with BPMN Sketch Miner

BPMN Sketch Miner is a rapid sketching tool for BPMN process models, you just write it as you're telling a story. They also have a great BPMN tutorial to make understanding of BPMN easier.

Waiter, Chef, Customer are the roles defined as Lanes, the yellow rectangles are Tasks, the circles are Events, the arrows are the Flows.

Since we learnt the basics, we can just start drawing of one of our processes ! Sounds a bit strange, right? BPMN as a set of standards seems complex to learn because it has hundreds of elements that allows us to map almost any situation in the real world, but do we need all of them to start? To me, it's very similar to when we learn programming languages, you don't need to know / apply the best of the bests to be able to create an application, you know that you can continuously improve your codebase later on. Since now we're storing our diagrams as code in our codebases, the same rule applies! Just start mapping a process that you believe it can be helpful for your team to have it visualized.

Using more features of BPMN

Here you can see how you can evolve the diagram above by using a few more elements of BPMN.

Start and End events are labeled, used Timer event to represent delay, Exclusive Gateway is used to branch for alternative paths, Loop is used for repeated actions.

Another great thing I use in BPMN is the Data and Message Flow, it clearly describes who needs what and when, and where it's generated at. This is one of the biggest ambiguity in the processes and often it's missed because of the assumptions. BPMN helps us to explicitly talk about that.

Message Events are used to represent what's received or sent with the tasks, Data objects used to show inputs and outputs of the tasks. Data Flow is used to describe how the data is processed throughout the process.

Mapping our processes

Imagine you'd like to describe what's the journey of your feature from being developed to customers using that. You may want this to cooperate with marketing department for them to understand better which steps are there and they can align, you can use it for your new team members to onboard them faster, or you can use it for a brainstorming session with different experts to see if there's a way to optimize the process. Either way, just start drafting something and live-edit it during your meeting.

BPMN Sketch Miner VSCode Extension

There's a VSCode Extension to integrate BPMN Sketch Miner to your development flow easily. I often store them under docs/diagrams/source/process_name.sketchminer and the png versions at docs/diagrams/out/process_name.png.

Further thoughts

Visually expressing our thoughts or new processes engages people more and allows going more efficient in our discussions. BPMN is the language of the processes and it's good to speak this language, and even daily basis. This article covered how to start doing that, of course there'll be a lot of development opportunities on the go, a few of them are; how to use it in our git flow, using BPMN files as an outcome to auto generate forms etc, allowing teams to describe their internal or external facing processes in an efficient manner. Note that the diagrams help with having shorter and interesting meetings, async work, less regression of the established practices, and less resistance of a change. Feel free to contact us if you encounter any of the opportunities, always happy to share our practices.

Linode, Firewall, Cloudflare Full (Strict) with Terraform Cloud

Sakir Temel — Wed, 21 Dec 2022 20:38:19 +0000

You built your application and now looking for a way to deploy it with a managed security? You'd like to have a simple yet powerful enough solution that can extend later on? We have an example web server setup in this article that provides you that. At KelebekLabs, whenever we choose to go with a basic server setup, not PaaS or Kubernetes, we follow these principles.

Cloudflare Application Security gives us a lot of services for free, such as DDoS protection, managing bad bots, rate limiting, free SSL certificate and many others. What we need to do is to make sure all the connections to our servers are going through Cloudflare and only. We're going to configure a firewall at our server provider side to make sure we only allow connection from Cloudflare so that the people who knows our IP address won't be able to bypass our security. You can think this as almost hiding your servers from the internet as Cloudflare will be the only door to your servers. In our example, we use Linode, a developer friendly, low cost cloud provider.

Instead of setting up the things manually at Linode, we're using Terraform, and to manage that we're using Terraform Cloud. In this article, you'll learn to set it up easily.

Preparing for Infrastructure as Code

We need a place where we're going to define our infrastructure, how many servers we have, what are their sizes, their connections in between etc. You can just create a new private repository on Github, or you can create a folder in your existing project. I prefer the second option as I like the monorepo more, so just create a readme file under terraform directory in your codebase.

Terraform Cloud

Terraform Cloud is an amazing service by HashiCorp with a free pricing option that helps you to ease your Terraform journey. It automatically connects to your repository, keeps the terraform state(the last state of your infrastructure, how many serves were created etc) for you, helps you to have your environment variables for you in their UI. Just create an account (you can use login via GitHub if you click on Continue via HCP) and then create a workspace, which is basically your system.

Remember the option we talked about earlier, creating a folder in a repo, you can specify it while you're creating a workspace. Congratulations, you now have an automation, doing any changes under terraform directory will generate a new terraform plan.

Generating a Linode Token for Terraform Cloud

We need to give permission to Terraform Cloud to create servers for us on Linode. We're going to do that via personal API tokens.

WARNING: Terraform will create servers, make sure that you're aware of what's created and how much does it cost for you.

You can create an API token by going to API tokens page under My Profile.

Once you get the token, add this as an environment variable in our Workspace at Terraform Cloud. Great, now Terraform Cloud can do operations on Linode.

Cloudflare SSL

As mentioned at the beginning, Cloudflare protects our system from various problems. We're now going to focus on the connection between Cloudflare -- Linode. Cloudflare generates an SSL certificate that is called Origin Server (Linode is our Origin Server in this case) certificates that ensures the connection between Cloudflare and Linode is fully secure.

Assuming that your DNS is already managed by Cloudflare, we can begin to make our configuration there. Create an Origin Server certificates by providing your domain name. The generated certificate will only be used between Cloudflare to Linode, real users won't be aware of it at all.

After creating the certificates, copy them as terraform variables in your Workspace variables pages at Terraform Cloud. Copy Private Key as SSL_CERTIFICATE_KEY and the other as SSL_CERTIFICATE.

Now you have all the necessary configuration ready, you'll be able to access these keys in your server and use them in your web server configuration.

Provision with Terraform

It's time for coding our infrastructure, you should put these IaC (Infrastruce as Code) files in your terraform folder. Once you commit them to your repository, you'll see a plan in your Terraform Cloud waiting for your approval. Once you approve your plan, Terraform is going to create an Ubuntu server for you with a simple init script, configure the firewall to only allow connections through Cloudflare and injects the SSL certificate keys as files.

Deploy your application

Now you have your server up and running, go to Linode console to access to your server via Lish Console, it's a secure way to access to your server. Clone your application server and configure your nginx. There's an example nginx configuration attached as nginx.conf. This part is left out on purpose as the deployment can be different for everyone.

Configure your DNS with your server's IP address

This part is a bit hacky for my case, if I find a better option, I'll update the article.

So, since we have our Linode server up and running, get its IP Address and add this as an A record with Proxy on option on Cloudflare. Now you should be able to send a request to your server in a secure way.

Conclusion

We learnt how to programmatically provision our infrastructure with a basis of production grade security in a high level perspective. This gives us a peace of mind as we know we're protected from most of the basic attacks. Now we can learn more how to configure our Cloudflare better, how to improve our Application level security, go through OWASP Top 10, organizational security principles etc. Building Secure applications is key for Reliability, Reliable applications boosts customers' and the developers' happiness. We'd like to be a part of this message by sharing our knowledge, so please reach out to us if you find any difficultly or anything wrong in this article.

Load Testing with Ruby-JMeter

Sakir Temel — Wed, 21 Dec 2022 19:01:12 +0000

You're ready for your product's launch, you'd like to show your customers finally there's a solution for their pain. You expect 1000 people using your service per hour, you made all the calculations and somehow you're confident. But, are you? Load Testing is here to simulate the traffic, 1000 people using your service per hour, and increase your confidence on launch day(and after).

The story for FolderCode, a temporary file uploading service, was exactly like above, and just to show how it was addressed in its backlog, see the picture below.

Designing your JMeter Test Plan

JMeter is a well-known tool for load or stress testing, it basically sends requests to your application within the given time, according to the behaviour plan you provided. The plans are designed to be detailedly configurable and focused on load-testing jargon. That makes the JMeter a comprehensive solution, however makes it a bit hard to understand & focus on the well written tests.

Here comes the ruby-jmeter! An easy to use tool that helps you to write readable test plans, which leads to focusing on your simulator scenarios to become closer to real customer behaviours. Under the hood, it uses JMeter. The code below simulates 10 customers keep visiting Google while its being run.

Preparing our Dashboards to observe our Product's behaviour

Before we start sending some traffic to simulate the launch day, do you already monitor your system? If not, you should! Some basic things like traffic, CPU, memory, and also some functional metrics, for example our case was how many files are being uploaded. Create a dashboard where you can easily watch the vital signs of your system.

You'll find so many things that weren't ready in your dashboard as you were focusing on the developing particular features not overall system's behaviour. This is actually a great benefit of preparing for a load testing.

We used New Relic One and custom metrics, it has a free plan, complete feature-set and easy to use interface.

You should design some alerts for non-desired values such as storage usage should never reach to 70%.

Let's send some traffic!!

That's the most fun part, sending the traffic with using JMeter and watching the vitals of our system. It's as exciting as the product launch, if not more. While the script is being run, for 2 minutes as configured, you can watch the metrics are changing on your dashboard.

Analyzing the results

Did you have receive any errors during the simulation?
Customers' latency was increasing? Increased latency gives worse customer experience.
Did any request fail?
Did you find any additional vital sign that should be monitored?
Do you need to make some infrastructure modifications?
Are you now confident that your system can handle the pre-calculated load without any increased latency?

Show your team your results! I was so happy to see that uploading ~1000 files in 2 minutes caused no issue to our system. Now you should have a validated confidence over your system. Make sure that you save your JMeter code in you repositories for reuse and improving the test plan!

You can run the load tests periodically to stay confident over your system especially for the critical parts.

Reference links

https://www.blazemeter.com/blog/jmeter-performance-testing

https://www.blazemeter.com/blog/performance-testing-vs-load-testing-vs-stress-testing

Using Notion as a Content Management System (CMS) for your applications

Sakir Temel — Wed, 21 Dec 2022 18:50:39 +0000

Recently while developing GarajApp, a car magazine app, we had a need to use a CMS to add content such as News, Campaigns and static pages like Guides. We considered several options like Strapi, Contentful or even implementing a simple enough markdown editor with WYSIWYG editor.

Choosing the right CMS

Founder of GarajApp tried using Strapi earlier and it ended up a maintenance burden with updates, errors, and not-so-easy development flow. We needed something simple enough, yet production ready. We just needed to serve some content from our database to the mobile app, how hard it could be?

We were leaning towards implementing a very simple page with a good WYSIWYG editor, perhaps with markdown support, that simply retrieves a page record and updates when we click save. It really looked easy but we had some concerns like adding a login, uploading the photos, managing the pages etc. Ok, so it wasn't so easy, then should we go with a full-scale content management system that supports a looot of things?

During a weekly brainstorming session, we threw the idea of using Notion! Notion has a great user experience when you edit or organize content, it has great permissions and login capabilities, and uploading images is so easy on that. We immediately started doing an end-to-end proof of concept, editing a page in Notion should be reflected in our mobile app within 10 minutes with using GraphQL(with Hasura engine).

Proof of Concept

We first created a page in Notion for News and an example item in it with adding a few columns.

Notion has an API that gives the list of pages and the page content. We used Notion client for JavaScript.

Now we need to run this script and call the API every 10 minutes with cron, can be more frequent if needed, and update the records in the database. Simply upserting the objects in fetchedNews to our news table in our database would be enough. We used Hasura as a backend in this project, so we just called the API with an upsert request with conflict on id field.

Making a GraphQL call to our Hasura engine, and we return the data! Now you can implement a code for transforming this Notion blocks to be displayable elements. You can implement a simple markdown generator or can just directly use the blocks in your code. We moved with implementing a module in iOS that converts these blocks into a display.

Handling the Images

Here's the tricky part, image blocks are returning us URLs that expires. So, if we copy the blocks from Notion directly into our database, our users won't be able to see the images after 60 minutes. To solve this issue, we simply cloned the images to Cloudinary during importing our Notion pages and saving the Cloudinary URL into our database. This gave also a power over the images for us, so that we could implement different versions of the same images easily.

Production ready implementation

To make this implementation production ready, we created 2 main pages like GarajDataStaging and GarajDataProduction. GarajDataProduction's data structure(columns and so on) is changed less frequently and controlled while doing any modification not to have any surprises. We also added Sentry to our script to make sure that our code is not failing. We haven't implemented any monitoring system for the code as the scope was too small, but feel free to include that, too.

Conclusion

It has been a great experience to use Notion as CMS, it's free and it has a great user experience. Simplifying the content management enabled us to develop the application smoothly, we were able to get real data during the development thanks to ease of using Notion. Not to forget to mention, Notion's blocks system gives a powerful yet organizable content with a great UI for modifying the content, which makes it a default choice over using HTML, markdown or a custom implementation.

Finding spikes in your plans

Sakir Temel — Wed, 21 Dec 2022 18:39:07 +0000

Spikes are for research, design, investigation, exploration. The purpose is to gain necessary knowledge to better understand a task and to work on that.

Word definition

The word spike comes from rock climbing activities. While climbing, we might stop to force a spike into the rock face, which is not actual climbing but by doing this we are making sure that future climbing will be smooth and easy.

Spikes as the Solution

Problem: We have a lot of uncertainties in our plans

It's demotivating when we're doing our tasks and we don't even understand why we're demotivated before doing it
We can't see the risks earlier, we can't say how long something will take
We don't know if we should spend time on something, what happens if we won't need it at the end?

Solution: Let's use Spikes!

We do the initial research early and our expectation is actually to "Learn" and "Understand" before we "Do" something
Spikes allow us to focus on analyzing the risks, do exploration. The purpose of a spike is not "Doing" but Enabling doing
Work for researching is never lost, outcome of a research is knowledge 🧠. After the research we can say that we shouldn't do the task now, and we're still staying with knowledge.

Let's say there's a task Make a slideshow with taking 10 photos, how you can plan this? How long this will take? You can say taking photos will take ~2 hours, making slideshow ~5 / ~10 / ~15 hours? If you don't know which tool to use, if you know the tool but you never used it, how you can plan working with that tool? In these cases, we create a Spike and we call this as an Enabler. You first do your Spike: "Research and make a slideshow with a few photos to decide which tool to use" and you timebox it for 2 hours. After 2 hours, you have the enough knowledge of slideshow tools and enough understanding of your task. After your spike, it's better to refine your task once again as you know more now.

KelebekLabs is doing R&D, it's likely to happen that we'll have a lot of Spikes.

If we knew what we were doing, it wouldn’t be called research. — Albert Einstein

Best practices of a Spike

Spike to represent uncertainty
Planning for both the spike and the resulting stories in the same iteration is sometimes risky
Better to plan the spike first and then take the decisions on actual story
Treat spike as a problem to be fixed, ideally shouldn't stay in the backlog for long
Spikes are only used with research(or exploration and other activities above) and documentation of it

Spikes can be technical or functional;

Functional spikes – They are used to analyze overall solution behavior and determine:

How to break it down
How to organize the work
Where risk and complexity exist
How to use insights to influence implementation decisions

Technical spikes – They are used to research various approaches in the solution domain. For example:

Determine a build-versus-buy decision
Evaluate the potential performance or load impact of a new user story
Evaluate specific technical implementation approaches
Develop confidence about the desired solution path

Reference links

https://www.slideshare.net/GoAtlassian/spiking-your-way-to-improved-agile-development-anatoli-kazatchkov

https://www.scaledagileframework.com/spikes/

https://www.c-sharpcorner.com/article/what-is-spike-in-agile-software-development2/

https://en.wikipedia.org/wiki/Spike_(software_development)

https://www.leadingagile.com/2014/04/dont-estimate-spikes/

https://t2informatik.de/en/smartpedia/spike-story/

https://youtu.be/zOIQjluFQZ4?t=203

https://blogs.starcio.com/2019/11/spikes-agile-poc.html

https://rgalen.com/agile-training-news/2018/12/7/12-considerations-for-user-story-spikes

https://spin.atomicobject.com/2017/05/23/spikes-for-problem-solving/

https://community.atlassian.com/t5/Jira-questions/How-do-you-use-Spikes-in-Jira/qaq-p/492960

https://www.leadingagile.com/2016/09/whats-a-spike-who-should-enter-it-how-to-word-it/

Auth0, Hasura, Social Media Login

Sakir Temel — Tue, 20 Dec 2022 02:18:14 +0000

We all want a super simple login and sign up system, so that our customers can become our members in a second and keep logging in without any hassle. Today we're going to talk about how to do it in the projects when we're using Hasura and Auth0.

Setting Up a Table in Hasura

First we need to have a users table in our Hasura, our unique key on email helps our users to login with different providers to the same account.

Auth0 Configuration

There's not much to add except following any Quickstart, I prefer disabling email&password login and only allowing login via Google and Apple (because for our case it was an iOS project). It gives a really simple experience to our customers.

Auth0 Rule for Syncing Users to Hasura

This is the most important part, it should be added as a Rule under Auth Pipeline. Whenever a successful login happens at Auth0 end, this code below runs. This code communicates with our Hasura via API and injects the just logged in user to our table that we created. It also retrieves back a token from Hasura and returns to our Auth0 client(library that you're using on mobile/web) as idToken. Now you can use Hasura with "Authorization: Bearer token" header.

Note: don't forget to add HASURA_GRAPHQL_JWT_SECRET environment variable to Hasura with the value generated here. This is necessary for Hasura to recognize our Auth0 as a provider, so that Hasura can generate a user token.

Conclusion

We just went through an end-to-end login/sign-up + Authentication experience with using Auth0 and Hasura. This became our default choice at KelebekLabs whenever we develop a new application. It gives a peace of mind both for the team and the customers.