Forem: burak

Pipelink

burak — Mon, 28 Apr 2025 09:38:59 +0000

Hello everyone, I will be sharing my experiences with Vibe Coding, which has recently become popular. By practicing Vibe Coding, you can increase your development speed and even build products in areas you are not familiar with. In the .NET world, the MediatR library, which has maintained its popularity for a long time, announced that it will become a paid product in the future. You can see the details on this site: https://www.reddit.com/r/dotnet/comments/1jul7yy/mediatr_and_masstransit_going_commercial_what_are/. After hearing about this, I asked myself, 'Why don't I develop my own library?' After providing ChatGPT with some details about the project and having a Q&A session about what the project name could be, the name 'Pipelink' emerged."

While creating the Pipelink NuGet package, I started by giving the first prompt about the foundational structure of the project. After that, I specified that the project should primarily use CQRS design patterns. In general, I also prepared prompts asking for both unit tests and integration tests to be written for the generated code, and for the tests to be executed to verify the outputs.

After these steps, my first task was to create an account on the NuGet server and generate an API Key from that account. Using this API Key, I created .nuspec files with the help of Cursor AI and pushed the package versions to NuGet. However, the NuGet package didn't have a logo. To solve this, I used ChatGPT to iterate 2-3 times and finally generated a logo.

I made the project code and the package available for free. If you would like to check it out, you can find it here: https://github.com/bburaksseyhan/pipelink/tree/production.

After completing these steps, I used Cursor AI to generate the README.md files for both the library and the integration tests. Additionally, I created all the code comments within the library using JetBrains AI. Thanks to this approach, I was able to produce the usage guide within minutes.

After completing these steps, I focused a bit on performance by generating prompts related to optimization and repeatedly running tests. I compared the results I obtained with MediatR. Whenever I identified areas where the performance was slower, I created new prompts and specifically updated the code around reflection usage. I repeated this cycle at least five times. With this post, I am now running one final round of benchmarks.

To briefly summarize the steps I completed:

Designed the project infrastructure
Developed the entire project code
Added comments and documentation to methods
Created and deployed the NuGet package
Wrote unit and integration tests
Performed performance optimizations
Designed a logo

I developed the Pipelink package in a total of 6–8 hours of work. I believe this amount of effort would normally take considerably longer for a developer. My goal was not to create competition, but rather to explore what I could build, how well, and how quickly by applying the Vibe Coding approach. As .NET developers, you can freely download and use the Pipelink package in your projects. I'll discuss second project that is call Pipelink.Outbox but still avaiable.

Nuget : Pipelink

Happy coding...

Design interservice communication for microservice

burak — Mon, 04 Dec 2023 13:09:01 +0000

What Are Microservices?

Microservices are independently deployable services modeled around a business domain. They communicate with each other via networks, and as an architecture choice offer many options for solving the problems you may face. They are a type of service-oriented architecture, albeit one that is opinionated about how service boundaries should be drawn and that independent deployability is key. From a technology viewpoint, microservices expose the business capabilities that they encapsulate via one or more network endpoints. They also encapsulate data storage and retrieval, exposing data, via well-defined interfaces.

Communication between microservices must be efficient and robust. With lots of small services interacting to complete a single business activity, this can be a challenge. Speaking about difficulty, I would like to write about microservice patterns that I'm gonna write publisher subscriptions. These days I find have time to develop technically. While watching a training video on Linkedin, I remembered the work I had done at the company. I said why didn't you write a short article? Today's content is a pattern that we know publisher-subscriber. The reason is we need multiple subscribers. You will find them in this article.

Details of pattern
Definition of Entities
Cross-Account Integration with Amazon SNS

"Pub/Sub" stands for "Publish/Subscribe," and it's a messaging pattern used in distributed systems and event-driven architectures. In a Pub/Sub system, the communication between different components (or services) is facilitated through the concept of channels or topics. The publisher-subscriber design architecture is below.

What are the use cases:

Multiple tasks
Multiple responders

Publishers: Entities that generate and send messages without knowing who or what might be listening. Publishers "publish" messages to specific channels or topics.
Subscribers: Entities that express interest in receiving messages on specific channels or topics. Subscribers "subscribe" to these channels and receive any messages published to those channels.
Broker (Message Queue or Middleware): This is an intermediary component that manages the distribution of messages. It takes care of routing messages from publishers to subscribers based on the channels or topics to which they are subscribed.

The key idea is that publishers and subscribers are decoupled. Publishers don't need to know anything about the subscribers, and vice versa. This decoupling allows for a more flexible and scalable system, where components can be added or removed without affecting the others.

I needed to pub&sub implementation in the last 2 companies that I worked for. Specifically, it was Tiko. Before the business changed at Tiko, my team was responsible for web scraping and web crawling. The main area is collecting, storing, and sharing data with cross domains. The challenge is one crawl operation collects an average of 5 million data and needs to be shared as fast and without losing data. Each team represents a different AWS account. The problem is how to send a message to an SQS in a different account. In the following section, I will describe how to develop steps and explain the subscription access policy rule. In 2023 AWS released the Document db streaming which means every CRUD operation can be a trigger event. I used the streaming process because it is fully managed, with no downtime, and is consistent, and reliable.

Steps:

Set up the document db streaming settings.
Create a new Serverless application
Create an SQS with the name of the router queue
Add trigger Serverless to SQS
Create 3 SNS Topics for listing separation by country code (use filter policy).

The topics have already subscribed to the router-service lambda application. That architecture works as event-driven when listing data and writing the database that throws the event.

{
  "Version": "2008-10-17",
  "Id": "__default_policy_ID",
  "Statement": [
    {
      "Sid": "__default_statement_ID",
      "Effect": "Allow",
      "Principal": {
        "AWS": "*"
      },
      "Action": [
        "SNS:GetTopicAttributes",
        "SNS:SetTopicAttributes",
        "SNS:AddPermission",
        "SNS:RemovePermission",
        "SNS:DeleteTopic",
        "SNS:Subscribe",
        "SNS:ListSubscriptionsByTopic",
        "SNS:Publish"
      ],
      "Resource": "arn:aws:sns:eu-central-1:owner-account-id:distributer-topic",
      "Condition": {
        "StringEquals": {
          "AWS:SourceOwner": "owner-account-id"
        }
      }
    },
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": [
          "arn:aws:iam::team-1:root",
          "arn:aws:iam::team-2:root"
        ]
      },
      "Action": [
        "SNS:Subscribe",
        "SNS:Publish"
      ],
      "Resource": "arn:aws:sns:eu-central-1:owner-account-id:distributer-topic"
    }
  ]
}

Advantages:

Loose Coupling: Publishers and subscribers are independent of each other, promoting a more modular and maintainable architecture.
Scalability: New subscribers or publishers can be added easily without affecting the existing components.
Asynchronous Communication: Pub/Sub systems often enable asynchronous communication, allowing components to operate independently and potentially improving system responsiveness.

Last words

Every team manages own applications which means no dependency
Almost fully managed and event-driven system that was created.
Manage the data send speed with lambda concurrency.
Any team subscribes to any Topic. That provides more flexibility
Every team works own SQS and SQS DLQ.

Reference:

How to use Go With MongoDB

burak — Wed, 01 Dec 2021 11:36:02 +0000

https://medium.com/yemeksepeti-teknoloji/how-to-use-go-with-mongodb-72963c8cebbc

Working MongoDB with Golang

burak — Thu, 11 Nov 2021 14:23:47 +0000

Every tutorial has a story. In that tutorial you'll find out different contents that is related to MongoDB, GoLang and working with mock data and deployment. Here is my content.

Project structure

PS: Here is the one different folder that name is dummy_api. That folder has own main file. What does it mean? When I run the main.go file before I'll add mock data. If you didn't catch up Working with the marshal and unmarshal tutorial you should read it. Another important topics is "context". Essential Go

context.TODO()
TODO returns a non-nil, empty Context. Code should use context.TODO when it’s unclear which Context to use or it is not yet available (because the surrounding function has not yet been extended to accept a Context parameter). TODO is recognized by static analysis tools that determine whether Contexts are propagated correctly in a program.

context.Background()
Background returns a non-nil, empty Context. It is never canceled, has no values, and has no deadline. It is typically used by the main function, initialization, and tests, and as the top-level Context for incoming requests.

Another different package is about MongoDb Client however I'll talk about below.

package main

import (
    "context"
    "encoding/json"
    "io/ioutil"
    "os"

    "github.com/bburaksseyhan/contact-api/src/pkg/client/mongodb"
    "github.com/bburaksseyhan/contact-api/src/pkg/model"
    "github.com/sirupsen/logrus"
)

func main() {

    contactsJson, err := os.Open("contacts.json")

    if err != nil {
        logrus.Error("contact.json an error occurred", err)
    }

    defer contactsJson.Close()

    var contacts []model.Contact

    byteValue, _ := ioutil.ReadAll(contactsJson)

    //unmarshall data
    if err := json.Unmarshal(byteValue, &contacts); err != nil {
        logrus.Error("unmarshall an error occurred", err)
    }

    logrus.Info("Data\n", len(contacts))

    //import mongo client
    client := mongodb.ConnectMongoDb("mongodb://localhost:27017")
    logrus.Info(client)

    defer client.Disconnect(context.TODO())

    collection := client.Database("ContactDb").Collection("contacts")

    logrus.Warn("Total data count:", &contacts)

    for _, item := range contacts {
        collection.InsertOne(context.TODO(), item)
    }

    logrus.Info("Data import finished...")
}

Firstly let's open the terminal and goes to dummy_api directory. Another important thing, is database running? Let's have look.

docker compose up

docker container ls

Working with mock data

I was creating mock data from Mockaroo

Working with MongoDB queries

docker exec -it ad2d44477f28 mongo //connect to mongodb cli

help

show dbs // return all database names

use ContactDb 

show collections // return collection name

db.contacts.find() //return all collections

db.contacts.find({}).count() // return row count

db.contacts.find({}).pretty({}) // return rows with a format

db.contacts.find({"email":""})
db.dropDatabase() // remove database

Using packages

"go get -u go.mongodb.org/mongo-driver/bson"
"go get -u go.mongodb.org/mongo-driver/mongo"
"go get -u go.mongodb.org/mongo-driver/mongo/options"
"go get -u github.com/gin-gonic/gin"
"go get -u github.com/sirupsen/logrus"
"go get -u github.com/spf13/viper"

Implementation

main.go file

That file read config.yml or .env file after that call the Init function.

package main

import (
    "os"

    "github.com/bburaksseyhan/contact-api/src/cmd/utils"
    "github.com/bburaksseyhan/contact-api/src/pkg/server"
    log "github.com/sirupsen/logrus"
    "github.com/spf13/viper"
)

func main() {

    config := read()
    log.Info("Config.yml", config.Database.Url)

    mongoUri := os.Getenv("MONGODB_URL")

    if mongoUri != "" {
        config.Database.Url = mongoUri
    }

    log.Info("MONGODB_URL", mongoUri)

    server.Init(config.Database.Url)
}

func read() utils.Configuration {
    //Set the file name of the configurations file
    viper.SetConfigName("config")

    // Set the path to look for the configurations file
    viper.AddConfigPath(".")

    // Enable VIPER to read Environment Variables
    viper.AutomaticEnv()

    viper.SetConfigType("yml")
    var config utils.Configuration

    if err := viper.ReadInConfig(); err != nil {
        log.Error("Error reading config file, %s", err)
    }

    err := viper.Unmarshal(&config)
    if err != nil {
        log.Error("Unable to decode into struct, %v", err)
    }

    return config
}

ConnectMongoDb function takes the MongoDB URL parameter so this function opens the connection and check the status. Related Documentation

package mongodb

import (
    "context"

    log "github.com/sirupsen/logrus"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

func ConnectMongoDb(url string) *mongo.Client {

    clientOptions := options.Client().ApplyURI(url)

    // Connect to MongoDB
    client, err := mongo.Connect(context.TODO(), clientOptions)

    if err != nil {
        log.Fatal(err)
    }

    // Check the connection
    err = client.Ping(context.TODO(), nil)

    if err != nil {
        log.Fatal(err)
    }

    log.Info("MongoClient connected")

    return client
}

unfinished contact_handler.go file. HealthCheck function is not only health check. That function checks the MongoDb database status with a timeout. If any cancellation comes from the server, context will be triggered and response will be un-health. Let's think the opposite result will be pong.

package handler

import (
    "context"
    "net/http"
    "time"

    "github.com/gin-gonic/gin"
    "github.com/sirupsen/logrus"
    "go.mongodb.org/mongo-driver/mongo"
)

type ContactHandler interface {
    GetAllContacts(*gin.Context)
    GetContactByCity(*gin.Context)
    HealthCheck(*gin.Context)
}

type contactHandler struct {
    client *mongo.Client
}

func NewContactHandler(client *mongo.Client) ContactHandler {
    return &contactHandler{client: client}
}

func (ch *contactHandler) GetAllContacts(c *gin.Context) {
    _, ctxErr := context.WithTimeout(c.Request.Context(), 30*time.Second)
    defer ctxErr()

    //request on repository

    c.JSON(http.StatusOK, gin.H{"contacts": "pong"})
}

func (ch *contactHandler) GetContactByCity(c *gin.Context) {
    _, ctxErr := context.WithTimeout(c.Request.Context(), 30*time.Second)
    defer ctxErr()

    //request on repository

    c.JSON(http.StatusOK, gin.H{"contacts": "pong"})
}

func (ch *contactHandler) HealthCheck(c *gin.Context) {

    ctx, ctxErr := context.WithTimeout(c.Request.Context(), 30*time.Second)
    defer ctxErr()

    if ctxErr != nil {
        logrus.Error("somethig wrong!!!", ctxErr)
    }

    if err := ch.client.Ping(ctx, nil); err != nil {
        c.JSON(http.StatusOK, gin.H{"status": "unhealty"})
    }

    c.JSON(http.StatusOK, gin.H{"status": "pong"})
}

Completed codes

main

package main

import (
    "os"

    "github.com/bburaksseyhan/contact-api/src/cmd/utils"
    "github.com/bburaksseyhan/contact-api/src/pkg/server"
    log "github.com/sirupsen/logrus"
    "github.com/spf13/viper"
)

func main() {

    config := read()
    log.Info("Config.yml", config.Database.Url)

    mongoUri := os.Getenv("MONGODB_URL")
    serverPort := os.Getenv("SERVER_PORT")
    dbName := os.Getenv("DBNAME")
    collection := os.Getenv("COLLECTION")

    if mongoUri != "" {
        config.Database.Url = mongoUri
        config.Server.Port = serverPort
        config.Database.DbName = dbName
        config.Database.Collection = collection
    }

    log.Info("MONGODB_URL", mongoUri)

    server.Init(config)
}

func read() utils.Configuration {
    //Set the file name of the configurations file
    viper.SetConfigName("config")

    // Set the path to look for the configurations file
    viper.AddConfigPath(".")

    // Enable VIPER to read Environment Variables
    viper.AutomaticEnv()

    viper.SetConfigType("yml")
    var config utils.Configuration

    if err := viper.ReadInConfig(); err != nil {
        log.Error("Error reading config file, %s", err)
    }

    err := viper.Unmarshal(&config)
    if err != nil {
        log.Error("Unable to decode into struct, %v", err)
    }

    return config
}

config

package utils

type Configuration struct {
    Database DatabaseSetting
    Server   ServerSettings
}

type DatabaseSetting struct {
    Url        string
    DbName     string
    Collection string
}

type ServerSettings struct {
    Port string
}

server

package server

import (
    "github.com/bburaksseyhan/contact-api/src/cmd/utils"
    "github.com/bburaksseyhan/contact-api/src/pkg/client/mongodb"
    "github.com/bburaksseyhan/contact-api/src/pkg/handler"
    repository "github.com/bburaksseyhan/contact-api/src/pkg/repository/mongodb"

    "github.com/gin-gonic/gin"
    log "github.com/sirupsen/logrus"
)

func Init(config utils.Configuration) {

    // Creates a gin router with default middleware:
    // logger and recovery (crash-free) middleware
    router := gin.Default()

    client := mongodb.ConnectMongoDb(config.Database.Url)

    repo := repository.NewContactRepository(&config, client)
    handler := handler.NewContactHandler(client, repo)

    router.GET("/", handler.GetAllContacts)
    router.GET("/contacts/:email", handler.GetContactByEmail)
    router.POST("/contact/delete/:id", handler.DeleteContact)

    router.GET("/health", handler.HealthCheck)

    log.Info("port is :8080\n", config.Database.Url)

    // PORT environment variable was defined.
    router.Run(":" + config.Server.Port + "")
}

handler

package handler

import (
    "context"
    "net/http"
    "strconv"
    "time"

    "github.com/bburaksseyhan/contact-api/src/pkg/model"
    db "github.com/bburaksseyhan/contact-api/src/pkg/repository/mongodb"

    "github.com/gin-gonic/gin"
    "github.com/sirupsen/logrus"
    "go.mongodb.org/mongo-driver/mongo"
)

type ContactHandler interface {
    GetAllContacts(*gin.Context)
    GetContactByEmail(*gin.Context)
    DeleteContact(*gin.Context)

    HealthCheck(*gin.Context)
}

type contactHandler struct {
    client     *mongo.Client
    repository db.ContactRepository
}

func NewContactHandler(client *mongo.Client, repo db.ContactRepository) ContactHandler {
    return &contactHandler{client: client, repository: repo}
}

func (ch *contactHandler) GetAllContacts(c *gin.Context) {

    ctx, ctxErr := context.WithTimeout(c.Request.Context(), 30*time.Second)
    defer ctxErr()

    var contactList []*model.Contact

    //request on repository
    if result, err := ch.repository.Get(ctx); err != nil {
        logrus.Error(err)
    } else {
        contactList = result
    }

    c.JSON(http.StatusOK, gin.H{"contacts": &contactList})
}

func (ch *contactHandler) GetContactByEmail(c *gin.Context) {

    ctx, ctxErr := context.WithTimeout(c.Request.Context(), 30*time.Second)
    defer ctxErr()

    var contactList *model.Contact

    //get parameter
    email := c.Param("email")

    //request on repository
    if result, err := ch.repository.GetContactByEmail(email, ctx); err != nil {
        logrus.Error(err)
    } else {
        contactList = result
    }

    c.JSON(http.StatusOK, gin.H{"contacts": contactList})
}

func (ch *contactHandler) HealthCheck(c *gin.Context) {

    ctx, ctxErr := context.WithTimeout(c.Request.Context(), 30*time.Second)
    defer ctxErr()

    if ctxErr != nil {
        logrus.Error("somethig wrong!!!", ctxErr)
    }

    if err := ch.client.Ping(ctx, nil); err != nil {
        c.JSON(http.StatusOK, gin.H{"status": "unhealty"})
    }

    c.JSON(http.StatusOK, gin.H{"status": "pong"})
}

func (ch *contactHandler) DeleteContact(c *gin.Context) {

    ctx, ctxErr := context.WithTimeout(c.Request.Context(), 30*time.Second)
    defer ctxErr()

    //get parameter
    id, err := strconv.Atoi(c.Param("id"))
    if err != nil {
        logrus.Error("Can not convert to id", err)
    }

    //request on repository
    result, err := ch.repository.Delete(id, ctx)
    if err != nil {
        logrus.Error(err)
    }

    c.JSON(http.StatusOK, gin.H{"deleteResult.DeletedCount": result})
}

repository

package repository

import (
    "context"

    "github.com/bburaksseyhan/contact-api/src/cmd/utils"
    "github.com/bburaksseyhan/contact-api/src/pkg/model"
    "github.com/sirupsen/logrus"
    log "github.com/sirupsen/logrus"
    "go.mongodb.org/mongo-driver/bson"
    "go.mongodb.org/mongo-driver/bson/primitive"
    "go.mongodb.org/mongo-driver/mongo"
    "go.mongodb.org/mongo-driver/mongo/options"
)

type ContactRepository interface {
    Get(ctx context.Context) ([]*model.Contact, error)
    GetContactByEmail(email string, ctx context.Context) (*model.Contact, error)
    Delete(id int, ctx context.Context) (int64, error)
}

type contactRepository struct {
    client *mongo.Client
    config *utils.Configuration
}

func NewContactRepository(config *utils.Configuration, client *mongo.Client) ContactRepository {
    return &contactRepository{config: config, client: client}
}

func (c *contactRepository) Get(ctx context.Context) ([]*model.Contact, error) {

    findOptions := options.Find()
    findOptions.SetLimit(100)

    var contacts []*model.Contact

    collection := c.client.Database(c.config.Database.DbName).Collection(c.config.Database.Collection)

    // Passing bson.D{{}} as the filter matches all documents in the collection
    cur, err := collection.Find(ctx, bson.D{{}}, findOptions)
    if err != nil {
        log.Fatal(err)
        return nil, err
    }

    // Finding multiple documents returns a cursor
    // Iterating through the cursor allows us to decode documents one at a time
    for cur.Next(context.TODO()) {
        // create a value into which the single document can be decoded
        var elem model.Contact
        if err := cur.Decode(&elem); err != nil {
            log.Fatal(err)
            return nil, err
        }

        contacts = append(contacts, &elem)
    }

    cur.Close(ctx)

    return contacts, nil
}

func (c *contactRepository) GetContactByEmail(email string, ctx context.Context) (*model.Contact, error) {

    findOptions := options.Find()
    findOptions.SetLimit(100)

    var contacts *model.Contact

    collection := c.client.Database(c.config.Database.DbName).Collection(c.config.Database.Collection)

    filter := bson.D{primitive.E{Key: "email", Value: email}}

    logrus.Info("Filter", filter)

    collection.FindOne(ctx, filter).Decode(&contacts)

    return contacts, nil
}

func (c *contactRepository) Delete(id int, ctx context.Context) (int64, error) {

    collection := c.client.Database(c.config.Database.DbName).Collection(c.config.Database.Collection)

    filter := bson.D{primitive.E{Key: "id", Value: id}}

    deleteResult, err := collection.DeleteOne(ctx, filter)
    if err != nil {
        log.Fatal(err)

        return 0, err
    }

    return deleteResult.DeletedCount, nil
}

Deployment

docker compose up

Repository

Thank you,

Cassandra & GoLang

burak — Tue, 09 Nov 2021 12:43:40 +0000

Hi there,

Did you read my latest implementation of PostgreSQL with GoLang? PostgresWithGo If you said of course :) go head... Do you know have any knowledge about Cassandra? Don't worry neither am I've not. I like to share about the subjects I work on. So be relax. Are you curious about NOSQl and GoLang? In this post I'll share my Cassandra Db and GoLang knowledge. You'll ask these questions ask yourself, What is Cassandra Db then how to install Cassandra DB to your local machine, which package should I use for it. All of these are in here.

Step 1 : What is Cassandra DB
Step 2 : How to install it
Step 3 : Important commands
Step 4 : Connect to database
Step 5 : Implementation
Step 6 : Postman Tests

Let's go to work and I hope you'll enjoy it.

Step 1: What is Cassandra DB

NoSQL databases are called “Not Only SQL” or “Non-relational” databases. NoSQL databases store and retrieve data other than tabular relations such as relation databases.

Elastic scalability − Cassandra is highly scalable; it allows to add more hardware to accommodate more customers and more data as per requirement.

Always on architecture − Cassandra has no single point of failure and it is continuously available for business-critical applications that cannot afford a failure.

Fast linear-scale performance − Cassandra is linearly scalable, i.e., it increases your throughput as you increase the number of nodes in the cluster. Therefore it maintains a quick response time.

Flexible data storage − Cassandra accommodates all possible data formats including: structured, semi-structured, and unstructured. It can dynamically accommodate changes to your data structures according to your need.

Easy data distribution − Cassandra provides the flexibility to distribute data where you need by replicating data across multiple data centers.

Transaction support − Cassandra supports properties like Atomicity, Consistency, Isolation, and Durability (ACID).

Fast writes − Cassandra was designed to run on cheap commodity hardware. It performs blazingly fast writes and can store hundreds of terabytes of data, without sacrificing the read efficiency.

Data Types

Step 2: How to install

I'm using mac os. I've already installed Docker application on my machine. So may be you want to install only docker container. Install Docker

docker pull cassandra
docker run -d --name cassandra -p 9042:9042 cassandra

In this repository I created docker-compose.yml file that file contains with required images.


  cassandra:
    image: cassandra:latest
    container_name: cassandra
    ports: 
      - 7000:7000
      - 7001:7001
      - 7199:7199
      - 9042:9042
      - 9160:9160
    restart: always
    environment:
       - CASSANDRA_BROADCAST_ADDRESS=host.docker.internal
       - CASSANDRA_SEEDS=host.docker.internal 
    volumes:
        - ./out/cassandra_data:/var/lib/cassandra
    healthcheck:
        test: ["CMD", "cqlsh", "-u cassandra", "-p cassandra" ,"-e describe keyspaces"]
        interval: 15s
        timeout: 10s
        retries: 10

7199 - JMX (was 8080 pre Cassandra 0.8.xx)
7000 - Internode communication (not used if TLS enabled)
7001 - TLS Internode communication (used if TLS enabled)
9160 - Thrift client API
9042 - CQL native transport port

Step 3: Important Commands

After the installation since I want to create a database, I have to run this command. What is cqlsh? It is the interface. cqlsh is a command-line interface for interacting with Cassandra using CQL (the Cassandra Query Language). Documentation

docker exec -it cassandra cqlsh

Let's create a todo table.

Step 4: Connect to database

Install the VSCode extension for Cassandra.

Step 5 : Implementation

Project structure:

--> cmd
--> api(main.go) this file deal with read config.yml file
--> util(config.go & errors.go) config.go is related with application configurations, errors.go is a custom response implementation.

--> pkg
--> api (api.go) this file integrated to initialize


func Initialize(config utils.Configuration) {

    // Creates a gin router with default middleware:
    router := gin.Default()

    fmt.Printf("%+v\n", config)

    //register database ,repositories and handlers
    session := cassandra.ConnectDatabase(config.Database.Url, config.Database.Keyspace)

    repository := db.NewTodoRepository(session)
    orderHandler := handler.NewTodoHandler(&repository)

    router.GET("/ping", orderHandler.HealthCheck)

    router.POST("/", orderHandler.CreateTodo)
    router.GET("api/v1/todo/:id", orderHandler.GetTodoById)

    //run the server :8080
    router.Run(":8080")
}

--> client/cassandra

The ConnectDatabase function take 2 parameters from api.go file. After creation that function response session. This session

func ConnectDatabase(url string, keyspace string) *gocql.Session {

    cluster := gocql.NewCluster(url)
    cluster.Keyspace = keyspace
    cluster.Consistency = gocql.Quorum

    session, _ := cluster.CreateSession()

    return session
}

Click fn+12 CreateSession

// CreateSession initializes the cluster based on this config and returns a
// session object that can be used to interact with the database.
func (cfg *ClusterConfig) CreateSession() (*Session, error) {
    return NewSession(*cfg)
}

-->handler
TodoHandler related with request database and response JSON data also error handling is done for that level. All function take c *gin.Context parameter also these functions has receiver function (t *todoHandler) which means that you can access the repository with t.Save or t.GetById.

func (t *todoHandler) CreateTodo(c *gin.Context) {
    var todo model.Todo

    c.BindJSON(&todo)

    todo.Id = uuid.New().String()
    log.Info("Data is ", todo)
    data, err := t.repo.Save(todo)

    if err != nil {
        c.JSON(http.StatusBadRequest, utils.BadRequestError("insert operation failed!", err))
    }

    c.JSON(http.StatusCreated, gin.H{"todo": data})
}

func (t *todoHandler) GetTodoById(c *gin.Context) {

    id := c.Param("id")

    todo, err := t.repo.GetById(id)
    if err != nil {
        c.JSON(http.StatusBadRequest, utils.BadRequestError("todo not found", err))
    }

    c.JSON(http.StatusOK, gin.H{"todo": todo})
}

GIN : c.JSON(http.StatusOK, gin.H{"todo": todo})
Echo: return c.JSON(http.StatusOK, data)

// JSON serializes the given struct as JSON into the response body.
// It also sets the Content-Type as "application/json".
func (c *Context) JSON(code int, obj interface{}) {
    c.Render(code, render.JSON{Data: obj})
}

Binding JSON data in GIN

// BindJSON is a shortcut for c.MustBindWith(obj, binding.JSON).
func (c *Context) BindJSON(obj interface{}) error {
    return c.MustBindWith(obj, binding.JSON)
}

In repository layer has "github.com/gocql/gocql" package that's is related the queries and executions. Cassandra DB queries similar to PostgreSQL, MsSQL or MySQL queries.

func (t *todoRepository) Save(todo model.Todo) (*model.Todo, error) {

    var query string = "INSERT INTO todo(id,title,content) VALUES(?,?,?)"

    if err := t.session.Query(query, todo.Id, todo.Title, todo.Content).Exec(); err != nil {
        return nil, err
    }

    return &todo, nil
}

func (t *todoRepository) GetById(id string) (*model.Todo, error) {

    var todo model.Todo

    var query string = `SELECT id,title,content FROM todo where id=?`

    if err := t.session.Query(query, id).Scan(&todo.Id, &todo.Title, &todo.Content); err != nil {

        if err == gocql.ErrNotFound {
            return nil, err
        }

        return nil, err
    }

    return &todo, nil
}

Now Let's run the docker-compose file. Cassandra DB has own health check configuration.
docker-compose up

    healthcheck:
        test: ["CMD", "cqlsh", "-u cassandra", "-p cassandra" ,"-e describe keyspaces"]
        interval: 15s
        timeout: 10s
        retries: 10

When database is running you can see the status is health

Step 6: Postman Test

run the program

References

TutorialPoint
Guru99

Github Repo

Thank you.

Hey Marshal

burak — Mon, 01 Nov 2021 12:59:38 +0000

Hi everyone,

Before creating a project, let's simply talk about other issue. What is Json, Struct, Marshall & UnMarshall. In this document, I'll introduce Json to Struct and apposite of it. How to make a request and how to convert it. What are these required packages? A struct is a user-defined type that represent a collection of field. If you write C# code before you might remember it. Well let's start. First in first how to define a simple struct.

type Users struct{}

type Customers struct{}

type Students struct{
    FirstName string
    LastName  string
}

Let's create a new GoLang project and run it.

Create Student Struct

type Student struct{
    FirstName string 
    LastName  string
}

type Company struct{
    Name string `json:"name"`
    Address  string `json:"address"`
}

//initialize
student := Student{FirstName:"Burak", LastName:"Seyhan"}

Your application has already it. You don't run go get -u command.

Marshal()

Converting Go objects into JSON is called Marshal() in GoLang. As the JSON is a language-independent data format.

package main

import (
    "fmt"
    "encoding/json"
)

type Student struct{
    FirstName string
    LastName  string
}

func main()  {
    fmt.Println("Welcome to Json and Struct Implementation")

    //initialize
    student := Student{FirstName:"Burak", LastName:"Seyhan"}

    data, err := json.Marshal(student)
    if(err != nil){
        fmt.Println(err)
    }

    fmt.Println(data) // data is byteArray
    fmt.Println(string(data)) convert to byteArray to string
}

UnMarshal()

Unmarshalling just opposite of Marshal.

var unmarshalStudent Student
unmarshal := json.Unmarshal([]byte(data), &unmarshalStudent)

if(unmarshal != nil){
    fmt.Println(unmarshal)
}

fmt.Printf("%v+\n", data)

Full code

package main

import (
    "fmt"
    "encoding/json"
)

type Student struct{
    FirstName string `json:"firtname"`
    LastName  string `json:"lastnamename"`
}


func main()  {
    fmt.Println("Welcome to Json and Struct Implementation")


    //initialize
    student := Student{FirstName:"Burak", LastName:"Seyhan"}

    data, err := json.Marshal(student)
    if(err != nil){
        fmt.Println(err)
    }

    fmt.Println(data)
    fmt.Println(string(data))


    var unmarshalStudent Student
    unmarshal := json.Unmarshal([]byte(data), &unmarshalStudent)
    if(unmarshal != nil){
        fmt.Println(unmarshal)
    }
    fmt.Printf("%v+\n", data)

}

Real world example

In this example I'll request a public Web API. PublicAPI This end point related with some user informations. At the beginning I'm gonna create a Struct.

type Users struct{
    Id int `json:"id"`
    Name  string `json:"name"`
    UserName  string `json:"username"`
    Email  string `json:"email"`
    Address  Address `json:"address"`
    Phone  string `json:"phone"`
    Website  string `json:"website"`
    Company  Company `json:"company"`
}

type Address struct{
    Street  string `json:"street"`
    Suite  string `json:"suite"`
    City  string `json:"city"`
    Zipcode  string `json:"zipcode"`
}

type Company struct{
    Name  string `json:"name"`
    CatchPhrase  string `json:"catchPhrase"`
    Bs  string `json:"bs"`
}

Next step is creating a web request. About net/http

    //endpoint
    url := "https://jsonplaceholder.typicode.com/users"

    //create a client
    client := http.Client{
        Timeout: time.Second * 2, // Timeout after 2 seconds
    }

    //prepera new request s
    req, err := http.NewRequest(http.MethodGet, url, nil)
    if(err != nil){
        fmt.Println(err)
    }

    result , getErr:= client.Do(req)
    if(getErr != nil){
        fmt.Println(getErr)
    }
    fmt.Println(result)

The defer keyword which executes result.Body.Close() at the end of the function is used to close the response body.

if(result.Body != nil){
        defer result.Body.Close()
    }

    body, _ := ioutil.ReadAll(result.Body)

    users := []Users{} // declare array cause Json data is array

    jsonErr := json.Unmarshal(body, &users)

    if(jsonErr != nil){
        fmt.Println(jsonErr)
    }

    for _, data := range users{
        fmt.Println(data)
    }

packages

These packages are loaded by default you don't need a go get command

"fmt"
"net/http"
"time"
"io/ioutil"
"encoding/json"

"fmt" package using for display some information or get any inputs from client but if you don't wanna use "fmt" code is Println("") because GoLang come with default packages. "net/http" packages is Web Request packages same as "fmt" package. "time" packages related to time information like DateTime. In this example doesn't need to this. You don't have to set Timeout actually it depends on. To get the response we are interested in, we have to access the Body property why we use "io/ioutil" packages. "encoding/json" packages related with marshal and unmarshal functions.

Another usage

url := "https://jsonplaceholder.typicode.com/users"

resp, err := http.Get(url)
if err != nil {
   fmt.Println(err)
}

body, err := ioutil.ReadAll(resp.Body)
if err != nil {
    fmt.Println(err)
}

fmt.Println(string(body))

Now run go run main.go see results:

[
  {
    "id": 1,
    "name": "Leanne Graham",
    "username": "Bret",
    "email": "Sincere@april.biz",
    "address": {
      "street": "Kulas Light",
      "suite": "Apt. 556",
      "city": "Gwenborough",
      "zipcode": "92998-3874",
      "geo": {
        "lat": "-37.3159",
        "lng": "81.1496"
      }
    },
    "phone": "1-770-736-8031 x56442",
    "website": "hildegard.org",
    "company": {
      "name": "Romaguera-Crona",
      "catchPhrase": "Multi-layered client-server neural-net",
      "bs": "harness real-time e-markets"
    }
  },
  {
    "id": 2,
    "name": "Ervin Howell",
    "username": "Antonette",
    "email": "Shanna@melissa.tv",
    "address": {
      "street": "Victor Plains",
      "suite": "Suite 879",
      "city": "Wisokyburgh",
      "zipcode": "90566-7771",
      "geo": {
        "lat": "-43.9509",
        "lng": "-34.4618"
      }
    },
    "phone": "010-692-6593 x09125",
    "website": "anastasia.net",
    "company": {
      "name": "Deckow-Crist",
      "catchPhrase": "Proactive didactic contingency",
      "bs": "synergize scalable supply-chains"
    }
  },
  {
    "id": 3,
    "name": "Clementine Bauch",
    "username": "Samantha",
    "email": "Nathan@yesenia.net",
    "address": {
      "street": "Douglas Extension",
      "suite": "Suite 847",
      "city": "McKenziehaven",
      "zipcode": "59590-4157",
      "geo": {
        "lat": "-68.6102",
        "lng": "-47.0653"
      }
    },
    "phone": "1-463-123-4447",
    "website": "ramiro.info",
    "company": {
      "name": "Romaguera-Jacobson",
      "catchPhrase": "Face to face bifurcated interface",
      "bs": "e-enable strategic applications"
    }
  },
  {
    "id": 4,
    "name": "Patricia Lebsack",
    "username": "Karianne",
    "email": "Julianne.OConner@kory.org",
    "address": {
      "street": "Hoeger Mall",
      "suite": "Apt. 692",
      "city": "South Elvis",
      "zipcode": "53919-4257",
      "geo": {
        "lat": "29.4572",
        "lng": "-164.2990"
      }
    },
    "phone": "493-170-9623 x156",
    "website": "kale.biz",
    "company": {
      "name": "Robel-Corkery",
      "catchPhrase": "Multi-tiered zero tolerance productivity",
      "bs": "transition cutting-edge web services"
    }
  },
  {
    "id": 5,
    "name": "Chelsey Dietrich",
    "username": "Kamren",
    "email": "Lucio_Hettinger@annie.ca",
    "address": {
      "street": "Skiles Walks",
      "suite": "Suite 351",
      "city": "Roscoeview",
      "zipcode": "33263",
      "geo": {
        "lat": "-31.8129",
        "lng": "62.5342"
      }
    },
    "phone": "(254)954-1289",
    "website": "demarco.info",
    "company": {
      "name": "Keebler LLC",
      "catchPhrase": "User-centric fault-tolerant solution",
      "bs": "revolutionize end-to-end systems"
    }
  },
  {
    "id": 6,
    "name": "Mrs. Dennis Schulist",
    "username": "Leopoldo_Corkery",
    "email": "Karley_Dach@jasper.info",
    "address": {
      "street": "Norberto Crossing",
      "suite": "Apt. 950",
      "city": "South Christy",
      "zipcode": "23505-1337",
      "geo": {
        "lat": "-71.4197",
        "lng": "71.7478"
      }
    },
    "phone": "1-477-935-8478 x6430",
    "website": "ola.org",
    "company": {
      "name": "Considine-Lockman",
      "catchPhrase": "Synchronised bottom-line interface",
      "bs": "e-enable innovative applications"
    }
  },
  {
    "id": 7,
    "name": "Kurtis Weissnat",
    "username": "Elwyn.Skiles",
    "email": "Telly.Hoeger@billy.biz",
    "address": {
      "street": "Rex Trail",
      "suite": "Suite 280",
      "city": "Howemouth",
      "zipcode": "58804-1099",
      "geo": {
        "lat": "24.8918",
        "lng": "21.8984"
      }
    },
    "phone": "210.067.6132",
    "website": "elvis.io",
    "company": {
      "name": "Johns Group",
      "catchPhrase": "Configurable multimedia task-force",
      "bs": "generate enterprise e-tailers"
    }
  },
  {
    "id": 8,
    "name": "Nicholas Runolfsdottir V",
    "username": "Maxime_Nienow",
    "email": "Sherwood@rosamond.me",
    "address": {
      "street": "Ellsworth Summit",
      "suite": "Suite 729",
      "city": "Aliyaview",
      "zipcode": "45169",
      "geo": {
        "lat": "-14.3990",
        "lng": "-120.7677"
      }
    },
    "phone": "586.493.6943 x140",
    "website": "jacynthe.com",
    "company": {
      "name": "Abernathy Group",
      "catchPhrase": "Implemented secondary concept",
      "bs": "e-enable extensible e-tailers"
    }
  },
  {
    "id": 9,
    "name": "Glenna Reichert",
    "username": "Delphine",
    "email": "Chaim_McDermott@dana.io",
    "address": {
      "street": "Dayna Park",
      "suite": "Suite 449",
      "city": "Bartholomebury",
      "zipcode": "76495-3109",
      "geo": {
        "lat": "24.6463",
        "lng": "-168.8889"
      }
    },
    "phone": "(775)976-6794 x41206",
    "website": "conrad.com",
    "company": {
      "name": "Yost and Sons",
      "catchPhrase": "Switchable contextually-based project",
      "bs": "aggregate real-time technologies"
    }
  },
  {
    "id": 10,
    "name": "Clementina DuBuque",
    "username": "Moriah.Stanton",
    "email": "Rey.Padberg@karina.biz",
    "address": {
      "street": "Kattie Turnpike",
      "suite": "Suite 198",
      "city": "Lebsackbury",
      "zipcode": "31428-2261",
      "geo": {
        "lat": "-38.2386",
        "lng": "57.2232"
      }
    },
    "phone": "024-648-3804",
    "website": "ambrose.net",
    "company": {
      "name": "Hoeger LLC",
      "catchPhrase": "Centralized empowering task-force",
      "bs": "target end-to-end models"
    }
  }
]

Repository

Thank you.

Golang with database operations

burak — Sun, 31 Oct 2021 18:27:41 +0000

Hi all,

In this post I'm going to show you how to use PostgreSQL with GoLang. Briefly I'll introduce history of GoLang. Actually you can find more than one different document about history of GoLang however I found it on a nicely explained site. History

I've experienced .Net Core stack however I've been learning GoLang for 3 months. I believed that GoLang is very powerful and smooth language. I think we'll go head with steps.

Step 1. Install Docker

Step 2. Run PostgreSQL container

docker run --name postgresql-container -p 5432:5432 -e POSTGRES_PASSWORD=Password! -d postgres;

Step 3. Run Database Script

-- Drop table

-- DROP TABLE public.users;

CREATE TABLE public.users (
    firstname varchar NULL,
    lastname varchar NULL,
    id serial NOT NULL
);

Insert Into public.Users(firstname,lastname) values("FirstName-1","LastName-1")
Insert Into public.Users(firstname,lastname) values("FirstName-2","LastName-2")
Insert Into public.Users(firstname,lastname) values("FirstName-3","LastName-3")

Step 4. Create new GoLang Solution

There are different IDE's development code.

VSCode
Vim-go
Atom
Sublime
LiteIDE For more information GoLang Official

According to my habit, Visual Studio Code is quite suitable for me. Now Install the extension of GoLang. Go Extension

Packages:

Web framework: go get -u github.com/labstack/echo/v4
Read Configuration file : go get -u github.com/spf13/viper
Logging : go get -u github.com/sirupsen/logrus
PostgreSQL : go get -u github.com/lib/pq

Let's look at the project structure

cmd>api : include with config file and initialization like Startup.cs
cmd>utils: configuration struct
pkg>api : include with middleware and routing also register with handlers
pkg>entities: database entities
pkg>handlers: related with request&response type also interact with repository layer
pkg>models : api response datas
pkg>repository: related with database operations

When I run a go run main.go this project starts with a cmd>api layer. This layer read a configuration parameters after that registered middleware and routings.

Let's connect to database. This function take postgresCustomerRepository parameter and return *sql.DB pointer. This method related with Open connection and check the connection status with Ping command.

func openDatabaseConn(r *postgresCustomerRepository) *sql.DB {
    db, err := sql.Open("postgres", r.conn)
    if err != nil {
        log.Error("Connection failed")
    }

    pingError := db.Ping()
    if pingError != nil {
        log.Error("Ping != pong")
    }

    log.Info("Postgres connection success!!!")

    return db
}

Add method

$1 and $2 hold the argument of struct. In the older numeric approach, arguments are referenced using the syntax $n: $1 refers to the first input argument, $2 to the second, and so on. This will work whether or not the particular argument was declared with a name. [PostgreSQL][https://www.postgresql.org/docs/9.5/xfunc-sql.html]

    db := openDatabaseConn(r)
    defer db.Close()

    //add data
    query := "Insert into Users(FirstName, LastName) values($1,$2)"
    if _, err := db.ExecContext(ctx, query, customer.FirstName, customer.LastName); err != nil {
        return customer, err
    }

List method

    //connect database
    db := openDatabaseConn(r)
    defer db.Close()

    //read data from server
    rows, _ := db.QueryContext(ctx, "Select id,firstname,lastname from Users")
    defer rows.Close()

Delete method

db := openDatabaseConn(r)
    defer db.Close()

    query := "Delete From Users Where Id=$1"
    affectedRow, err := db.ExecContext(ctx, query, id)
    if err != nil {
        return false, nil
    }

Get one

db := openDatabaseConn(r)
    defer db.Close()

    data := db.QueryRowContext(ctx, "Select id,firstname,lastname from Users Where Id=$1", id)

customerRepository.go file

package repository

import (
    "context"
    "database/sql"
    "fmt"
    "time"

    "github.com/bburaksseyhan/ctmapp/src/cmd/utils"
    "github.com/bburaksseyhan/ctmapp/src/pkg/entities"

    _ "github.com/lib/pq"
    log "github.com/sirupsen/logrus"
)

type CustomerRepository interface {
    List(cntxt context.Context, timeout int) ([]entities.CustomerEntity, error)
    Add(customer entities.CustomerEntity, cntxt context.Context, timeout int) (entities.CustomerEntity, error)
    Delete(id int, cntxt context.Context, timeout int) (bool, error)
    Get(id int, cntxt context.Context, timeout int) (entities.CustomerEntity, error)
}

type postgresCustomerRepository struct {
    dbSetting *utils.DbSettings
    conn      string
}

func NewPostgresCustomerRepository(dbSettings *utils.DbSettings) CustomerRepository {
    //initial log formatter
    log.SetFormatter(&log.JSONFormatter{})

    repo := &postgresCustomerRepository{
        dbSetting: dbSettings,
        conn: fmt.Sprintf("host=%s port=%d user=%s password=%s dbname=%s sslmode=disable",
            dbSettings.Host, dbSettings.Port, dbSettings.User, dbSettings.Password, dbSettings.DbName),
    }

    return repo
}

func (r *postgresCustomerRepository) List(cntxt context.Context, timeout int) ([]entities.CustomerEntity, error) {

    //context
    ctx, cancel := context.WithTimeout(cntxt, time.Duration(timeout)*time.Second)
    defer cancel()

    //connect database
    db := openDatabaseConn(r)
    defer db.Close()

    //read data from server
    rows, _ := db.QueryContext(ctx, "Select id,firstname,lastname from Users")
    defer rows.Close()

    //define slice for store customer information
    var customerEntity []entities.CustomerEntity

    //read data row by row
    for rows.Next() {
        var userId int
        var firstName string
        var lastName string

        _ = rows.Scan(&userId, &firstName, &lastName)

        customerEntity = append(customerEntity, entities.CustomerEntity{Id: userId, FirstName: firstName, LastName: lastName})
    }

    return customerEntity, nil
}

func (r *postgresCustomerRepository) Add(customer entities.CustomerEntity, cntxt context.Context, timeout int) (entities.CustomerEntity, error) {

    ctx, cancel := context.WithTimeout(cntxt, time.Duration(timeout)*time.Second)
    defer cancel()

    db := openDatabaseConn(r)
    defer db.Close()

    //add data
    query := "Insert into Users(FirstName, LastName) values($1,$2)"
    if _, err := db.ExecContext(ctx, query, customer.FirstName, customer.LastName); err != nil {
        return customer, err
    }

    return customer, nil
}

func (r *postgresCustomerRepository) Delete(id int, cntxt context.Context, timeout int) (bool, error) {

    ctx, cancel := context.WithTimeout(cntxt, time.Duration(timeout)*time.Second)
    defer cancel()

    db := openDatabaseConn(r)
    defer db.Close()

    query := "Delete From Users Where Id=$1"
    affectedRow, err := db.ExecContext(ctx, query, id)
    if err != nil {
        return false, nil
    }

    fmt.Println(affectedRow.LastInsertId())
    fmt.Println(affectedRow.RowsAffected())

    return true, nil
}

func (r *postgresCustomerRepository) Get(id int, cntxt context.Context, timeout int) (entities.CustomerEntity, error) {

    ctx, cancel := context.WithTimeout(cntxt, time.Duration(timeout)*time.Second)
    defer cancel()

    db := openDatabaseConn(r)
    defer db.Close()

    data := db.QueryRowContext(ctx, "Select id,firstname,lastname from Users Where Id=$1", id)

    var userId int
    var firstName string
    var lastName string

    _ = data.Scan(&userId, &firstName, &lastName)

    return entities.CustomerEntity{Id: userId, FirstName: firstName, LastName: lastName}, nil
}

func openDatabaseConn(r *postgresCustomerRepository) *sql.DB {
    db, err := sql.Open("postgres", r.conn)
    if err != nil {
        log.Error("Connection failed")
    }

    pingError := db.Ping()
    if pingError != nil {
        log.Error("Ping != pong")
    }

    log.Info("Postgres connection success!!!")

    return db
}

Test

go run main.go

Routings

    e.GET("/health", customerHandler.Health)

    e.GET("/api/v1/customer", customerHandler.List)
    e.POST("/api/v1/customer", customerHandler.Add)
    e.DELETE("/api/v1/customer/:id", customerHandler.Delete)
    e.GET("api/v1/customer/:id", customerHandler.Get)

Create User

List Users

Get by id

Delete

Repository

Thank you.

Pdf Converter

burak — Thu, 07 Jan 2021 19:08:36 +0000

If you need a nice pdf convertor NuGet packages here is the good one. You can use with .net framework and .net core

Select.HtmlToPdf.NetCore

Sample code:

HtmlToPdf converter = new HtmlToPdf();
PdfDocument doc = converter.ConvertUrl(url);
doc.Save("Book" + "_" + page + ".pdf");
doc.Close();

Happy coding days

PostgreSQL with EF Part II

burak — Wed, 06 Jan 2021 17:24:49 +0000

Before you read this article, I advise you to read the first part

Sometimes some developer needs multiple contexts of our project. This is a little different than a single context. Multiple contexts mean multiple database context. That means is direct to the constructor. In this project, I need authentication and JWT token implementation. So I would like to use Microsoft Identity Server Document and I want to store a registered user in a database. Here is the problem I need a new context without CommunityContext. But how? At the same time, I ask myself what is the migration commands for every context. Let me explain.

Create new context and call "AppIdentityDbContext". Every single context include own options like DbContextOptions

using Core.Entities.Identity;
using Microsoft.AspNetCore.Identity.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore;

namespace Infrastructure.Identity
{
    public class AppIdentityDbContext : IdentityDbContext<AppUser>
    {
        public AppIdentityDbContext(DbContextOptions<AppIdentityDbContext> options) :
            base(options)
        {
        }
    }
}

AppUser.cs

public string DisplayName { get; set; }

public bool IsActive { get; set; } 

public bool IsWantToAuthor { get; set; }

public bool IsAuthor { get; set; }

public UserType UserType { get; set; }

Here is the second database context DbContextOptions options

using Core.Entities;
using Core.Entities.Identity;
using Microsoft.EntityFrameworkCore;

namespace Infrastructure
{
    public class CommunityContext : DbContext
    {
        public CommunityContext(DbContextOptions<CommunityContext> options) : base(options)
        {
        }

        public DbSet<Article> Articles { get; set; }

        public DbSet<Users> Users { get; set; }

        protected override void OnModelCreating(ModelBuilder modelBuilder)
        {
            base.OnModelCreating(modelBuilder);
        }
    }
}

DbContext

Unlike single context migration commands, multiple contexts command a little complicated. You have to specify the context name.

Create Migration(example of CommunityContext)
dotnet ef migrations add Initial -p Infrastructure/ -s API -o Data/Migrations -c CommunityContext
Update Migration(example of CommunityContext)
dotnet ef database update -p Infrastructure/ -s API -c CommunityContext

Create Migration(example of AppIdentityDbContext)
dotnet ef migrations add IdentityInitial -p Infrastructure -s API -o Identity/Migrations -c AppIdentityDbContext
Update Migration(example of AppIdentityDbContext)
dotnet ef database update -p Infrastructure -s API -c AppIdentityDbContext

Open the query editor.

Thank you 🧑🏻‍💻

PostgreSQL with EF

burak — Fri, 01 Jan 2021 18:10:04 +0000

Happy new year ladies and gentlemen. Today is the first day of the 2021 year and vacation day. I prepare new study topics for myself every year and plan to write projects. In this post, you will learn how to use PostgreSQL with entity framework code first. Also, you will learn how to get docker image PostgreSQL and install it. At the same time, you'll learn Entity Framework migrations after that write some basic queries in Visual Studio Code. Let's start.

Important: Please install Visual Studio Code and .Net Core 3.1 or the latest version.

Create Project

Let's create a new Web API project. Here is the project structure. Also please install these packages.

Also here is the appsettings.Development.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "ConnectionStrings": {
    "Postgre": "UserID=postgres;Password=123;Server=localhost;Port=5432;Database=Community;Integrated Security=true;Pooling=true;"
  }
}

Migrations

Create Initial Migration.

dotnet ef migrations add InitialCreate -p Infrastructure/ -s API -o Data/Migrations

API :

Microsoft.EntityFrameworkCore.Design

Infrastructure:

Microsoft.EntityFrameworkCore.Design
Microsoft.EntityFrameworkCore
Microsoft.EntityFrameworkCore.Tools
Npqsql.EntityFrameworkCore.PostgreSQL

Let's update the database.

dotnet ef database update -p Infrastructure/ -s API

If you want to remove migration you can use this command.

dotnet ef migrations remove -p Infrastructure -s API

Open the terminal go to the project location

If you do not install "Microsoft.EntityFrameworkCore.Design" package you'll get this error.

Let's migrate our entity

After this command, InitialMigration will be created

then update database

Pull Image

docker pull postgres

docker run --name postgres -e POSTGRES_PASSWORD="123" -d -p 5432:5432 -v /var/lib/postgresql/data postgres

check docker image status

Install Query Editor

I found a very useful tool for PostgreSQL

Query Editor

{
  "label": "postgre",
  "host": "localhost",
  "user": "postgres",
  "port": 5432,
  "ssl": false,
  "database": "Community",
  "password": "123"
}

Basic Queries

SELECT * FROM "Authors" LIMIT 1000;

Part II
Happy new year 🎄

The middleware

burak — Mon, 21 Dec 2020 19:30:36 +0000

Hi everybody,

I hope, you are feeling good today. Also, I'm good thanks. In this post, I'm gonna talk about the importance of system tracing. The main subject, I want to tell you in this post will be outside of the classical methods. Maybe you ask me what are you talking about? I'm talking about designing a custom log framework using different tech stacks. Today, I will create a new library, after that implemented a basic API. Here is the most important content is meaning of architecture

Here is the I used technology stacks

Dotnet Core More Information
MediatR
MassTransit
RabbitMq

Why do we need tracing of the system?

It does not make sense that I do say specific reasons. If we have domain size does not matter, we should trace them. I would like to specify, as the complexity of the domain increases, monitoring and interpreting the system decrease.

I have seen that in small and medium-sized systems, logs are monitored without being written to databases and even kept in text files. It might seem to like normal but here is the biggest problem are it will be hard to identify the problem. Let's give an example. Just think about a domain.

This project about customer information. I created 2 API and 3 worker service. Each worker services applications provide a different service. Like register, login, notification, etc ...
Also, each service has its own service log like warning, error, or information. Let's assume, every API and worker service write request and response logs write to disk. Any service stopped working after that I have to check the service logs. Firstly open the txt log files. After that search it with ctrl+f. This is a very bad situation. You do not have to spend a lot of effort.

What should I do?

Some companies buy software or some companies write their own solutions after that to implement. Let's give an example Nlog, Elmah, Graylog, SeriLog, DataDog.

In this post, I would like to explain the following titles.

What is Middleware?

More Information

What is MediatR?

MediatR is an open-source project and an implementation of the mediator design pattern. The mediator design pattern controls how a set of objects communicate and helps to reduce the number of dependencies among these objects that you must manage. In the mediator design pattern, objects don’t communicate with one another directly, but through a mediator. This article presents a discussion of how we can use MediatR in ASP.Net Core applications.
More Information

What is MassTransit?

More Information

Let's open Visual Studio after that let's write some code.

Project Design and Packages

I would like to create a framework that name is 'RequestTracer'. I designed 2 layers. Here is the folder structure.

RequestTracer.Core layer

Install the below nugget packages.

Let's examine the codes in the file.

Commands: It's about the MediatR request parameter and the return type is bool.

using MediatR;
using RequestTracer.Core.Dtos;

namespace RequestTracer.Core.Commands
{
    public class LogCommand : IRequest<bool>
    {
        public LogDto LogDtos { get; set; }
    }
}

Dtos: It's about request and response parameters. Referenced to Commands.

using System;

namespace RequestTracer.Core.Dtos
{
    public class LogDto
    {
        public Guid Id { get { return Guid.NewGuid(); } }

        public DateTime Date { get { return DateTime.UtcNow; } }

        public string ApplicationName { get; set; }

        public string Environment { get; set; }

        public LogDetail LogDetail { get; set; }
    }
}

namespace RequestTracer.Core.Dtos
{
    public class LogDetail
    {
        public string Host { get; set; }

        public string Method { get; set; }

        public string Protocol { get; set; }

        public string Schema { get; set; }

        public string Path { get; set; }

        public string QueryStringParameter { get; set; }

        public int StatusCode { get; set; }

        public string ContentType { get; set; }

        public long? ContentLength { get; set; }

        public string ExecutionTime { get; set; }
    }
}

Handlers: It's about the MediatR. Every command has a handler. Every environment creates its own exchange.

using System;
using System.Threading;
using System.Threading.Tasks;
using MassTransit;
using MediatR;
using Microsoft.Extensions.Configuration;
using RequestTracer.Core.Commands;

namespace RequestTracer.Core.Handlers
{
    public class LogCommandHandler : IRequestHandler<LogCommand, bool>
    {
        private readonly IBus _bus;
        private readonly IConfiguration _configuration;
        private readonly ISendEndpoint _sendEndpoint;

        public LogCommandHandler(IBus bus, IConfiguration configuration)
        {
            _bus = bus;
            _configuration = configuration;

            var environment = Environment.GetEnvironmentVariables()["ASPNETCORE_ENVIRONMENT"].ToString();

            _sendEndpoint = _bus.GetSendEndpoint(
                new Uri($"{_configuration.GetSection("RabbitMqSettings:server").Value}/{configuration.GetSection("ApplicationName").Value}-{environment}-Logs")).GetAwaiter().GetResult();
        }

        public async Task<bool> Handle(LogCommand request, CancellationToken cancellationToken)
        {
            await _sendEndpoint.Send(request);

            return true;
        }
    }
}

Middlewares: It's about request and response custom pipeline. Every single request goes through here without errors cause I only trace information logs.

using System;
using System.Diagnostics;
using System.Threading.Tasks;
using MediatR;
using Microsoft.AspNetCore.Http;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;

namespace RequestTracer.Core.Middlewares
{
    public class LogMiddleware
    {
        private readonly RequestDelegate _next;
        private readonly IMediator _mediator;
        private readonly IConfiguration _configuration;
        private readonly ILogger<LogMiddleware> _logger;

        public LogMiddleware(RequestDelegate next,
                             IMediator mediator,
                             IConfiguration configuration,
                             ILogger<LogMiddleware> logger)
        {
            _next = next ?? throw new ArgumentNullException(nameof(next));
            _mediator = mediator ?? throw new ArgumentNullException(nameof(next));
            _configuration = configuration ?? throw new ArgumentNullException(nameof(_configuration));
            _logger = logger ?? throw new ArgumentNullException(nameof(logger));
        }

        public async Task Invoke(HttpContext httpContext)
        {
            try
            {
                var stopwatch = new Stopwatch();
                stopwatch.Start();
                await _next(httpContext);
                stopwatch.Stop();

                await SendRequest(httpContext, stopwatch);
            }
            catch (Exception ex)
            {
                _logger.LogError($"LogMiddleware {ex}");

                if (httpContext.Response.HasStarted)
                {
                    _logger.LogWarning("The response has already started");
                    throw;
                }
            }
        }

        private async Task SendRequest(HttpContext httpContext,Stopwatch stopwatch)
        {
            var request = httpContext.Request;
            var response = httpContext.Response;

            await _mediator.Send(new Commands.LogCommand()
            {
                LogDtos = new Dtos.LogDto()
                {
                    ApplicationName = _configuration.GetSection("ApplicationName").Value,
                    Environment = Environment.GetEnvironmentVariables()["ASPNETCORE_ENVIRONMENT"].ToString(),
                    LogDetail = new Dtos.LogDetail()
                    {
                        Host = request.Host.HasValue ? request.Host.ToString() : string.Empty,
                        Method = request.Method,
                        Protocol = request.Protocol,
                        Schema = request.Scheme,
                        Path = request.Path,
                        QueryStringParameter = request.QueryString.HasValue == true ? request.QueryString.Value : string.Empty,
                        StatusCode = response.StatusCode,
                        ContentType = response.ContentType,
                        ContentLength = request.ContentLength,
                        ExecutionTime = $"Execution time of {stopwatch.ElapsedMilliseconds} ms"
                    }
                }
            });
        }
    }
}

Another one is an extension class here is the code.

using Microsoft.AspNetCore.Builder;
using RequestTracer.Core.Middlewares;

namespace RequestTracer.Core
{
    public static class MiddlewareExtensions
    {
        public static IApplicationBuilder UseCustomMiddleware(this IApplicationBuilder builder)
        {
            return builder.UseMiddleware<LogMiddleware>();
        }
    }
}

RequestTracer.Infrastructure.IoC:

Install the below nugget packages.

DependencyContainer.cs

This class referenced to API. At the same time, this class related to setup information like dependency injections.

using System;
using MassTransit;
using MediatR;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using RequestTracer.Core.Commands;

namespace RequestTracer.Infrastructure.IoC
{
    public static class DependencyContainer
    {
        public static void RegisterService(IServiceCollection services, IConfiguration configuration)
        {
            #region IoC layer
            services.AddMediatR(typeof(LogCommand));

            //mass transit configuration
            services.AddMassTransit(x =>
            {
                x.AddBus(provider => Bus.Factory.CreateUsingRabbitMq(config =>
                {
                    config.UseHealthCheck(provider);
                    config.Host(new Uri(configuration.GetSection("RabbitMqSettings:host").Value), "/",
                    h =>
                    {
                        h.Heartbeat(TimeSpan.FromSeconds(30));
                        h.Username(configuration.GetSection("RabbitMqSettings:username").Value);
                        h.Password(configuration.GetSection("RabbitMqSettings:password").Value);
                    });

                }));
            });

            services.AddMassTransitHostedService();

            #endregion

            #region Database Layer
            #endregion

            #region Application Layer
            #endregion
        }
    }
}

API configuration

I designed a Web API that name is "Customer.API". The important is, settings must be defined in the API layer for RabbitMq to work. Be careful! We have 3 different worker services and 2 web API. Each service has its own log queue. The distinctive thing is the name of the apps. If you define worker1,worker2, and worker3, you have 3 different queues. Evert worker service has its own environment variables like development, pre-production, or production. It means a different exchange.

"ApplicationName" :  "Customer.Api",
  "RabbitMqSettings": {
    "host": "rabbitmq://localhost",
    "server": "rabbitmq://localhost:15672",
    "username": "burki",
    "password": "123"
  }

Please reference all layers to CustomerAPI application then open the Startup.cs. Let's define configurations.

  public void ConfigureServices(IServiceCollection services)
        {
            services.AddControllers();
            services.AddSwaggerGen(c =>
            {
                c.SwaggerDoc("v1", new OpenApiInfo { Title = "CustomerApi", Version = "v1" });
            });

            DependencyContainer.RegisterService(services, Configuration);
        }

  public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
        {
            if (env.IsDevelopment())
            {
                app.UseDeveloperExceptionPage();
                app.UseSwagger();
                app.UseSwaggerUI(c => c.SwaggerEndpoint("/swagger/v1/swagger.json", "MonitorApi.Demo v1"));
            }

            app.UseHttpsRedirection();

            app.UseRouting();

            app.UseAuthorization();

            _ = app.UseCustomMiddleware(); 

            app.UseEndpoints(endpoints =>
            {
                endpoints.MapControllers();
            });
        }

Install RabbitMQ

Run the command

docker run -d --hostname my-rabbit --name myrabbit -e RABBITMQ_DEFAULT_USER=admin -e RABBITMQ_DEFAULT_PASS=123456 -p 5672:5672 -p 15672:15672 rabbitmq:3-management

After that start the RabbitMQ docker file with this command

docker run imageid

Check the status

docker ps

Open the browser go to the

http://localhost:15672/
username: admin
password: 123456

I would like to create a new account. I'll set up username: burki and password: 123 and give permission. For this information, please open the Admin tab.

Here is the config result.

PS: If you do not declare a virtual host you'll get The agent is stopped or has been stopped, no additional provocateurs can be created error

Looks good. Let's test it.

First of all, open the run the CustomerAPI and make some requests. Here is my controller class code

using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading;
using System.Threading.Tasks;
using CustomerApi;
using Microsoft.AspNetCore.Mvc;
using Microsoft.Extensions.Logging;

namespace MonitorApi.Demo.Controllers
{
    [ApiController]
    [Route("[controller]")]
    public class CustomerController : ControllerBase
    {
        private readonly ILogger<CustomerController> _logger;

        public CustomerController(ILogger<CustomerController> logger)
        {
            _logger = logger;
        }

        [HttpGet]
        public IEnumerable<Customer> GetCustomers()
        {
            var customers = new List<Customer>();
            customers.Add(new Customer()
            {
                Id = 1,
                FirsName = "Burak",
                LastName = "Seyhan",
                City = "Ankara",
                EmailAddress = "burakseyhan8@gmail.com",
                Fax = string.Empty,
                Phone = "444"
            });
            customers.Add(new Customer()
            {
                Id = 2,
                FirsName = "User_FirstName",
                LastName = "User_LastName",
                City = "User_City",
                EmailAddress = "user@gmail.com",
                Fax = string.Empty,
                Phone = "User_PhoneNumber"
            });

            return customers;
        }
    }
}

Did you see it? I do not care about RabbitMQ configurations or another thing. I just only created a Web API method. I only care about appsettings.Development.json file that's all.

Run the CustomerAPI then request GetCustomers() method.

When you request any method, our middleware catches all information and connects with the RabbitMQ here is the connection status.

Looks good go head 🤟🏻

Please open the Exchange tab.

Click the "Customer.Api-Development-Logs" then you'll see a new window after that I'll configure it to the binding so choose to queue but make sure the queue is defined. Open Queue tab then creates a new queue and I called 'Customer.Api-Development-Logs'. Here is my queue.

After that bind Exchange to Queue.

Now let's make a new request.

My request was reflected here

I haven't written a consumer yet. So let's look at the output you want, made manually. Again open the Queue tab then click the Get Messages button.

Add new Order API then configure the appsettings.Development.json file same with previous.

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "ApplicationName": "Order.Api",
  "RabbitMqSettings": {
    "host": "rabbitmq://localhost",
    "server": "rabbitmq://localhost:15672",
    "username": "burki",
    "password": "123"
  }
}

Let's get test it.

When you make a request Order API exchange created.

Then create a queue then bind it.

Here is the payload.

Every service has its own middleware. So Every middleware creates a single exchange. If I run the all workers you can see the worker exchanges.

As a result,

Logging mechanisms are separated by the application-based queue. In this way, the metrics of the requests and responses of each service can be easily analyzed within itself.
Consumer applications to be written in addition to these applications and payload information can be easily displayed on the web screen interface.

Repository

Thank you for reading.

Benchmark Strings

burak — Wed, 09 Dec 2020 10:27:58 +0000

How much do you know string?

Since when I started day developing code, generally use the string data type. They always ask questions about strings in every job interview. Especially the first times. Today I'll talk about how much do you know strings?

Microsoft says: a string is an object of type String whole value is text. 👉 Microsoft

Microsoft says: the string keyword is an alias for System.String; 👉 Microsoft

Let's declare and initialize strings

// Declare without initializing.
string username;

// Initialize to null.
string username = null;

// Initialize as an empty string.
string username = System.String.Empty;

// Initialize as an empty string with ""
string username = "";

// Initialize with a regular string literal.
string oldPath = "c:\\documents";

// Initialize with a verbatim string literal.
string newPath = @"c:\documents;

// Use System.String if you prefer.
System.String email = "burakseyhan8@gmail.com";

//string concanate
var title = "Hi";
var body = "Thank" + "you";
body = $"{body} for read";

MessageBox.Show("body");

If I need text data I have to use String. Or I got the current date if I wanna store the current date in the database I'll use ToString().

Strings objects are immutable: Strings are can not be changed after they have been created. So let's write basic code.

string firstName = "burak";
string lastName = "seyhan";

firstName += lastName;

Console.WriteLine(firstName);

Escape Sequence Characters

Benchmark

Player 1 : + operator
Player 2 : String.Concat()
Player 3 : StringBuilder

+ operator :

Concatenate using with + operator. With + operator create a new string.

text = "Burak" + i.ToString();

String.Concat() :

String.Concat() function also allows to concatenate strings. You can pass all strings as function parameters.

StringBuilder :

StringBuilder is a class that is useful to represent a mutable string of characters and it is an object of System.Text namespace. Strings are immutable and StringBuilder is mutable. Mutable means once an instance of the class is created, then the same instance will be used to perform any operations. Although the StringBuilder is a dynamic object that allows you to expand the number of characters in the string that it encapsulates, you can specify a value for the maximum number of characters that it can hold. This value is called the capacity of the StringBuilder object.

Method and Definitions:

StringBuilder.Append: This method will append the given string value to the end of the current StringBuilder.
StringBuilder.AppendFormat: It will replace a format specifier passed in a string with formatted text.
StringBuilder.Insert: It inserts a string at the specified index of the current StringBuilder.
StringBuilder.Remove: It removes a specified number of characters from the current StringBuilder.
StringBuilder.Replace: It replaces a specified character at a specified index.

StringBuilder and String.Concat differs in one critical point. When the String.Concat function is used, all strings to concatenate must be given as parameters. By using the StringBuilder class, these values can be added step by step by calling the add function.

Here is the rule. I create 3 players. Every player has their own specific role. Here is the rule. I create 3 players. Every player has their own specific role. Every player iterates 100000 times then all player's results writes on the screen.

Let's write some code here 👨🏻‍💻

using System;
using System.Diagnostics;
using System.Text;

namespace StringvsStringBuilder
{
    class MainClass
    {
        public static void Main(string[] args)
        {
            var count = 100000;
            string text = string.Empty;

            var stopWatch = Stopwatch.StartNew();

            /*
            Console.WriteLine("+ operator");
            for (int i = 0; i <= count; i++)
            {
                text = "burak " + string.Empty + i.ToString();
                Console.WriteLine(text);
            }

            Console.WriteLine($"Content: {text} Execution Time Of : {stopWatch.Elapsed.TotalMilliseconds}");
            stopWatch.Stop();
            Console.WriteLine("*********************");
            */

            /*
            Console.WriteLine("String.Concat");

            for (int i = 0; i <= count; i++)
            {
                String.Concat("burak", string.Empty, "seyhan", i.ToString());
            }
            stopWatch.Stop();
            Console.WriteLine($"Content: {text} Execution Time Of : {stopWatch.Elapsed.TotalMilliseconds}");
            Console.WriteLine("*********************");
            */


            Console.WriteLine("String Builder");
            var builder = new StringBuilder();
            for (int i = 0; i <= count; i++)
            {
                builder.Append("burak");
                builder.Append("");
                builder.AppendLine("seyhan");
                builder.AppendLine(i.ToString());
            }

            stopWatch.Stop();
            Console.WriteLine($"Content: {text} Execution Time Of : {stopWatch.Elapsed.TotalMilliseconds}");
            Console.WriteLine("*********************");


            Console.ReadKey();
            Console.WriteLine("Program finished");
        }
    }
}

Results:

- operator Content: burak 100000 Execution Time Of: 3307.8574
String Concat
Content: Execution Time Of: 222.8163
string builder
Content: Execution Time Of: 178.1494

🥇 🍾 is string builder 👏🏻 👏🏻 👏🏻

Thank you for reading 🧑🏻‍💻