Forem: Eugene Tolbakov

trino istio

Eugene Tolbakov — Mon, 16 Jun 2025 23:00:26 +0000

Eventbridge S3

Eugene Tolbakov — Wed, 22 Jan 2025 12:26:42 +0000

Enhancing the SQL Interval syntax: A story of Open Source contribution

Eugene Tolbakov — Tue, 09 Jul 2024 21:54:11 +0000

There are many reasons why developers dive into the world of Open Source contributions.

Contributing can be a way to give back to the community and use your skills for the greater good. It's a fantastic environment that allows you to network with talented developers, build relationships, and potentially find mentors or collaborators. For those seeking career advancement, contributions become a public portfolio showcasing your skills and experience. Sometimes, it's about a personal itch! You might encounter a bug or limitation in a project you rely on, and contributing a fix not only solves your frustration but benefits everyone who uses that software.

The recipe for a successful Open Source contribution goes beyond just code. A strong desire to learn fuels the journey, as you navigate unfamiliar codebases and tackle new challenges. But learning flourishes best within a supportive environment.

A responsive community acts as a safety net, offering guidance and feedback to help you refine your contributions. Ideally, you can also "dogfood" the tool you contribute to, using it in your work or personal projects. This firsthand experience provides valuable context for your contributions and ensures they address real-world needs. With these elements in place, you're well on your way to making a lasting impact on the Open Source community.

While I've been building my software development skills for a while now, Rust still feels like a whole new world to explore. This “newbie” feeling might make some shy away from contribution, fearing their code won't be good enough. I use silly mistakes as stepping stones to improve my skills, not a reason to feel discouraged.

Over a year of contributing to GreptimeDB has been a rewarding journey filled with learning experiences. Today, I'd like to walk you through one of those. Let's get our hands dirty (or should I say, claws? 🦀)
This time we will be enhancing the Interval syntax by allowing the [shortened version]

Motivation

This time I chose a task to enhance the Interval syntax by allowing a shortened version. The SQL standard specifies a syntax format such as:

select interval '1 hour';

My objective was to ensure the functionality of the following alternative(shortened) format:

select interval '1h';

Diving right into the code, I discovered that the core functionality for handling transformations already exists. The scope of a change boils down to adding a new rule specifically for the Interval data type: intervals with abbreviated time formats will be automatically expanded to their full versions. Let's take a closer look at a specific section of the code that does the logic

fn visit_expr(&self, expr: &mut Expr) -> ControlFlow<()> {
    match expr {
        Expr::Interval(interval) => match *interval.value.clone() {
            Expr::Value(Value::SingleQuotedString(value)) => {
                if let Some(data) = expand_interval_name(&value) {
                    *expr = create_interval_with_expanded_name(
                        interval,
                        single_quoted_string_expr(data),
                    );
                }
            }
            Expr::BinaryOp { left, op, right } => match *left {
                Expr::Value(Value::SingleQuotedString(value))=> {
                    if let Some(data) = expand_interval_name(&value) {
                        let new_value = Box::new(Expr::BinaryOp {
                            left: single_quoted_string_expr(data),
                            op,
                            right,
                        });
                        *expr = create_interval_with_expanded_name(interval, new_value);
                    }
............
}

Code Review

An experienced Rust developer, Dennis (CEO & Co-Founder @ Greptime), quickly identified an area for improvement and suggested the fix:

Code review shines as a learning tool.

Beyond simply accepting the suggested change (though the rationale for “efficiency” was clear!), I decided to take a deep dive. Analyzing the proposed improvement and explaining it to myself(in the form of this post), helped me better understand the Rust code and its recommended practices.

Avoiding unnecessary cloning and ownership transfers

Originally I used .clone() on interval.value:

match *interval.value.clone() { ... }

Cloning here creates a new instance of the data each time, which can be inefficient if the data structure is large or cloning is expensive. The suggested version avoids this by using references:

match &*interval.value { ... }

Matching on a reference (&*interval.value) eliminates the cost of cloning. The same improvement is applied to the match on left in the binary operation case:

match &**left { ... }

This one is slightly more involved: it uses a double dereference to get a reference to the value inside a Box, which is more efficient than cloning.

Clearer Pattern Matching

Using references in pattern matching emphasizes the intention of only reading data, not transferring ownership:

match &*interval.value { ... }

This shows explicitly that the matched value is not being moved. It helps with reasoning about the code, especially in a context with complex ownership and borrowing rules.

Reduced Cloning Inside the Binary Operation

In the original code, the op and right fields of the Expr::BinaryOp variant are cloned unconditionally.

let new_value = Box::new(Expr::BinaryOp {
    left: single_quoted_string_expr(data),
    op,
    right,
});

However, they only need to be cloned if the left field is an Expr::Value variant with a string value. The suggested enhancement moves the cloning inside the if let block, so it only happens when necessary.

let new_value = Box::new(Expr::BinaryOp {
    left: single_quoted_string_expr(data),
    op: op.clone(),
    right: right.clone(),
});

Using references instead of cloning:

In the original code, expand_interval_name(&value) is used, which borrows value. ~~However, value is of type String, which implements the AsRef<str> trait. This means that it can be automatically dereferenced to a &str when necessary~~. However, value is of type String, which implements the Deref<Target=str> trait (more information could be found here). This means that it can be automatically dereferenced to a &str when necessary. The improved version uses expand_interval_name(value), which avoids the need for explicit reference.

Summary

So in the context of this change “efficiency” stands for:
• Avoiding unnecessary cloning, reducing overhead.
• Making the borrowing and ownership patterns clearer and safer.
• Enhancing overall readability and maintainability.

This is how the visit_expr function looks like after all suggestions have been applied:

fn visit_expr(&self, expr: &mut Expr) -> ControlFlow<()> {
        match expr {
            Expr::Interval(interval) => match &*interval.value {
                Expr::Value(Value::SingleQuotedString(value))
                | Expr::Value(Value::DoubleQuotedString(value)) => {
                    if let Some(expanded_name) = expand_interval_name(value) {
                        *expr = update_existing_interval_with_value(
                            interval,
                            single_quoted_string_expr(expanded_name),
                        );
                    }
                }
                Expr::BinaryOp { left, op, right } => match &**left {
                    Expr::Value(Value::SingleQuotedString(value))
                    | Expr::Value(Value::DoubleQuotedString(value)) => {
                        if let Some(expanded_name) = expand_interval_name(value) {
                            let new_expr_value = Box::new(Expr::BinaryOp {
                                left: single_quoted_string_expr(expanded_name),
                                op: op.clone(),
                                right: right.clone(),
                            });
                            *expr = update_existing_interval_with_value(interval, new_expr_value);
                        }
                    }
                    _ => {}
                },
    .....................
    }

Open Source contribution has been an incredible way to accelerate my Rust learning curve. My efforts on this #GoodFirstIssue serve as an illustration of how to improve skills through collaborations. Depending on your feedback, I'm excited to share more of these learning experiences in future posts!

A huge thanks to the entire Greptime team, especially Dennis, for their support and guidance! Let’s keep the contributing/learning going!

UPD: Thanks to @tisonkun for the thorough review!

Multi-state terraform project

Eugene Tolbakov — Fri, 20 Jan 2023 08:31:36 +0000

For the past several days I’ve been terraforming Opensearch resources with the Elasticsearch/OpenSearch Provider. Phill Baker did a really great work, so take a minute of your time and star this repo!

The problem statement

Let's assume that our infrastructure has three environments(Development, UAT and Prod) and three clusters per environment - A, B and C.

How would you organise the project structure so each cluster in each environment will have a separate state file?
But why one might need such setup? Isn't it enough to have a state file per environment? Of course, there are scenarios when that is absolutely fine, nonetheles, it's worth considering:

💥 the "blast radius". Potential misconfiguration that may happen will impact only a single cluster, while others will be safe and sound when their states are managed independently.
⏱ plan/apply step time. In very large projects it can be even a few hours. On top of that an update for a big number of resources may lead to the API requests throttling by cloud providers. Not a nice situation to be in.

In this post I'd like to describe a solution that I came up with. Hopefully, this could be useful for anyone solving anything similar.

First approximation

Usually a multi-cluster setup means that each cluster has it's own purpose, however, structure-wise clusters have the same set of resources. We can encapsulate the logic of creationindex_templates,ism_policies, etc in the opensearch module by providing a number of tftpl template files with Opensearch resources definitions.

The content of the clusters.tf file can look like:

module "cluster_a" {
  source               = "./opensearch"
  index_templates      = fileset(path.module, "/path/to/cluster_a/index_templates/*.tftpl")
  ism_policy_templates = fileset(path.module, "/path/to/cluster_a/ism_policy_templates/*.tftpl")
  ...
}
module "cluster_b" {
  source               = "./opensearch"
  index_templates      = fileset(path.module, "/path/to/cluster_b/index_templates/*.tftpl")
  ism_policy_templates = fileset(path.module, "/path/to/cluster_b/ism_policy_templates/*.tftpl")
  ...
}
module "cluster_c" {
  source               = "./opensearch"
  index_templates      = fileset(path.module, "/path/to/cluster_c/index_templates/*.tftpl")
  ism_policy_templates = fileset(path.module, "/path/to/cluster_c/ism_policy_templates/*.tftpl")  ...
}

An abstract CI/CD tool can execute the following commands (from the project root level) to initialize and deploy the project:

export ENVIRONMENT=dev
terraform init -backend-config=./environment/$ENVIRONMENT/config.s3.tfbackend
terraform plan -var-file=./environment/$ENVIRONMENT/variables.tfvars
terraform apply -var-file=./environment/$ENVIRONMENT/variables.tfvars

A couple of notes:

config.s3.tfbackend file reflects the fact that I use S3 remote backend, but it's not the requirement. You are good to go with any other backend.
variables.tfvars file makes it easier to pass values required for the opensearch module to create resources.

Next step

The easiest way to covert the existing project to a multi state file one is to initialize each opensearch module(for cluster_a, cluster_b, cluster_c) separately. The picture below shows how the new structure might look like:

This solution is fit for purpose. It gives us multi state terraform project.-chdir setting is used here to be able to launch commands from the project root level:

export ENVIRONMENT=dev
export CLUSTER=cluster_a
terraform -chdir=./$CLUSTER/ init -backend-config=./../environment/$ENVIRONMENT/config.s3.tfbackend

terraform -chdir=./$CLUSTER/ plan -var-file=./../environment/$ENVIRONMENT/variables.tfvars

terraform -chdir=./$CLUSTER/ apply -var-file=./../environment/$ENVIRONMENT/variables.tfvars
...
# a similar set of commands for 'cluster_b' and 'cluster_c'

The obvious downside with this solution - multiple cluster_<id> folders which have almost identical content.

Can we do better?

It would be great if we could initialize the same cluster folder for different backend files without creating multiple copies of it. TF_DATA_DIR to the rescue. As per documentation:

TF_DATA_DIR changes the location where Terraform keeps its per-working-directory data, such as the current backend configuration.

This is exactly what we need: by using TF_DATA_DIR we've written the cluster initalization code only once but "reused" it with all our tfbackend and tfvars files thus being able to obtain a dedicated state file per cluster and per environment. Here is how the final project structure looks like:

And terraform commands:

mkdir {cluster_a,cluster_b,cluster_c} # create folders for cluster initialization 
export ENVIRONMENT=dev
export CLUSTER=cluster_a
export TF_DATA_DIR=$CLUSTER

terraform -chdir=./$CLUSTER/ init -backend-config=./environment/$ENVIRONMENT/config.s3.tfbackend

terraform -chdir=./$CLUSTER/ plan -var-file=./environment/$ENVIRONMENT/variables.tfvars

terraform -chdir=./$CLUSTER/ apply -var-file=./environment/$ENVIRONMENT/variables.tfvars

export CLUSTER=cluster_b
...

This set of commands could be refactored as a loop and wrapped in a utility script.

Final thoughts

It is always nice to write the least amount of code/configuration as possible without violating functionality and readability.
The approach that has been shown in this post works fine. However, if there are no limitations on the tools you can choose, I would suggest looking at the terragrunt for solving a similar issue.

Thank you for your time! If you have any suggestions or concerns you are more than welcome to approach me in comments or via twitter.

P.S. Thanks to Eugene Rubanov for the review and feedback 🤝.

nbb and lambda function URL: turn a boring task into a piece of cake.

Eugene Tolbakov — Mon, 11 Apr 2022 17:02:50 +0000

Hi 👋, Eugene is here.
A few weeks ago I ran into the talk given by Michiel Borkent about nbb - a tool for ad-hoc CLJS scripting on Node.js.
I truly liked his presentation, so I went away inspired and ready to transition new knowledge from theory to practice. So if you are curious to know what I’ve ended up with, let’s jump in.

In my team at Hyde-Housing we promote the culture of experimentation. It creates a great opportunity for trying out new things in non-critical project areas with further internal demos and discussions about whether we should adopt innovations wider or call it a day.
For such experiments, I often try to choose either something with a clear outcome or something that I've implemented many times. One of our internal services uses a Node.js lambda that generates presigned URLs for file uploads. Quite ~~boring~~ standard thing. Sounds like a perfect candidate for rewriting, doesn't it?

The project uses AWS SDK for JavaScript v3 that has a new modular architecture. So the package.json includes two libraries @aws-sdk/client-s3 and @aws-sdk/s3-presigned-post to import an S3 client and createPresignedPost method that generates a presigned url. Also we need to add nbb as a dependency. The full package.json file looks like:

{
   "dependencies": {
       "@aws-sdk/client-s3": "^3.67.0",
       "@aws-sdk/s3-presigned-post": "^3.67.0",
       "nbb": "^0.3.4"
   }
}

nbb allows you to develop in the same REPL-driven manner which is a usual workflow for clojure programmers. That’s fantastic, isn’t it?
Though the lambda code is relatively straightforward, I still would like to give some explanations. The link to the full project can be found at the end of the post.

(ns handler
 (:require ["@aws-sdk/client-s3" :refer [S3Client]]
           ["@aws-sdk/s3-presigned-post" :refer [createPresignedPost]]
           [clojure.string :as s]
           [goog.string.format]
           [applied-science.js-interop :as j]
           [promesa.core :as p]))

(def s3 (S3Client. #js{:region "eu-west-1"}))

(def bucket-name-template "%s-docs-upload-bucket")

(defn handler [event _ctx]
     (p/let [{:keys [env folderName fileName]} (-> event
                                                   (j/get :pathParameters)
                                                   (j/lookup))
             key (str (s/replace folderName #"_" "/") "/" fileName)
             bucket-name (goog.string/format bucket-name-template env)
             response (createPresignedPost s3 (clj->js {:Bucket     bucket-name
                                                        :Key        key
                                                        :Expires    300
                                                        :Fields     {:key key}
                                                        :Conditions [["eq", "$key", key]
                                                                     ["content-length-range", 0, 10485760]
                                                                     ["starts-with", "$Content-Type", "text/"]]}))]
            (clj->js {:statusCode 200
                      :headers    {"Access-Control-Allow-Origin"  "*"
                                   "Access-Control-Allow-Headers" "Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token,X-Amz-User-Agent,Access-Control-Allow-Origin",
                                   "Access-Control-Allow-Methods" "OPTIONS,GET"}
                      :body       (js/JSON.stringify response nil 2)})))

#js {:handler handler}

careful reader might ask why some of the the dependencies in the :require have double quotes while others do not? This convention came from the shadow-clj and adopted by nbb to denote an npm library;
the p/let - a neat way of chaining promises: each expression will be treated as a promise expression and will be executed sequentially;
the lambda gets the environment, folder and filename parameters from the incoming API Gateway event. Those are needed to understand which bucket and folder the upload is going to happen to. j/lookup method from the js-interop library is a quick and handy way that allows you to use the clojure destructuring;
according to the business logic S3 folders may be nested. The folderName parameter uses underscore to reflect the nesting nature, f.e. "folder1_folder2_folder3" should be converted to "folder1/folder2/folder3";
when all required parameters are prepared the signing bit comes into play: the createPresignedPost method requires the s3 client instance and the options object;
clj->js does exactly what it says on the tin - recursively transforms cljs values to javascript;
the returned response consists of url and fields that we need to pass on to the caller. They will use it for the actual upload;
again using clj->js to assemble a lambda function response that consists of a status, headers (for the sake of simplicity "Access-Control-Allow-Origin" is configured to "allow all", however it's recommended to specify the exact origin) and body;

There's one file that needs to be added - index.mjs - an ES6 module that’s required for Node.js applications.

Please note that promesa and js-interop are built-in libraries so you can use them straightaway!
The whole project and deployment instructions can be found at this github repo.

Recently AWS announced Lambda Function URLs support. This feature allows configuring an https endpoint to the AWS lambda. It is a huge improvement for simple use cases where you don’t need the advanced API Gateway functionality.
In order to benefit from it we need slightly tweak our stack declaration and the handler because the incoming event structure has changed to something like:

{
 "version": "2.0",
 "routeKey": "$default",
 "rawPath": "/dev/folder1/report1.csv",
 "rawQueryString": "",
 "headers": {
   "x-amzn-trace-id": "Root=1-4a40e828-b893-47b2-937c-19623c4f88e4",
   "x-forwarded-proto": "https",
   "host": "kafts7qpofkzbbvxbxxzavlv6i0aelqr.lambda-url.eu-west-1.on.aws",
   "x-forwarded-port": "443",
   "x-forwarded-for": "aeaf:c432:4e41:50ee:060d:fc8d:5d42:df65",
   "accept": "*/*",
   "user-agent": "curl/7.64.1"
 },
 "requestContext": {
   "routeKey": "$default",
   "stage": "$default",
   "time": "11/Apr/2022:07:54:29 +0000",
   "domainPrefix": "kafts7qpofkzbbvxbxxzavlv6i0aelqr",
   "requestId": "cd14fcd8-ff00-4fd9-a13a-3bc772f038ea",
   "domainName": "kafts7qpofkzbbvxbxxzavlv6i0aelqr.lambda-url.eu-west-1.on.aws",
   "http": {
     "method": "GET",
     "path": "/dev/folder1/report1.csv",
     "protocol": "HTTP/1.1",
     "sourceIp": "aeaf:c432:4e41:50ee:060d:fc8d:5d42:df65",
     "userAgent": "curl/7.64.1"
   },
   "accountId": "anonymous",
   "apiId": "kafts7qpofkzbbvxbxxzavlv6i0aelqr",
   "timeEpoch": 1649663669499
 },
 "isBase64Encoded": false
}

This is how we obtain environment, folder and filename now:

(defn handler [event _ctx]
     (p/let [[env folderName fileName] (-> event
                                           (j/get-in [:requestContext :http :path])
                                           (s/split #"/")
                                           (rest))
...

The rest of the handler's implementation remained the same.

Thank you for reading 🙏, I hope you found it useful.
Please let me know if you have any questions, suggestions and feedback.