Forem: Dayananda

Overview of Lock API in java

Dayananda — Thu, 26 Dec 2024 07:21:13 +0000

Managing access to shared resources is important in concurrent programming to ensure data consistency. Traditional synchronized keyword lacks flexibility for complex scenarios like fairness, acquire lock immediately, wait for a specific time to acquire lock, etc. Lock, ReadWriteLock, and StampedLock APIs are introduced in Java 1.5 to addresses these limitation and provides great control to developers to handle shared resources access. They are part of java.util.concurrent package.

Lock API

Lock API is an interface and provides below methods to handle thread synchronization.

void lock() It is used to acquire the lock. If the lock is not available, the thread will be blocked until the lock is acquired.
void lockInterruptibly() It is same as lock(), but this thread can be interrupted.
boolean tryLock() It returns true if the lock is acquired; otherwise, false. The thread will not be blocked if the lock is not granted.
boolean tryLock(long timeout, TimeUnit unit) It is same as tryLock() but waits for a specified amount of time only.
void unlock()Releases the lock.
Condition newCondition() Provides the ability for a thread to wait for some condition to occur while executing the critical part.

ReentrantLock

ReentrantLock class implements the Lock interface.
A reentrant lock allows a thread to acquire a lock multiple times and release it multiple times.
If you want fairness policy, use parameterized constructor of ReentrantLock.
If fairness is enabled, It makes sure that the longest waiting thread is given access to the lock.
Use finally block to release the lock to avoid deadlock situation.

ReentrantLock lock = new ReentrantLock(true); //fair lock
public void methodExecution() {
    lock.lock();
    try {
        // Critical section here
    } finally {
        lock.unlock();
    }
}

ReadWriteLock API

ReadWriteLock API is an interface and maintains a pair of locks to maintain read and write scenarios. The read lock can be held simultaneously by multiple threads if there are no writers. The write lock is exclusive.

Lock readLock() - Returns the lock used for reading.
Lock writeLock() - Returns the lock used for writing.

Key Rules

If you have a read lock, you can't acquire a write lock.
If you have a write lock, you can't acquire a read lock in any other thread.
If you have a write lock, you can acquire another write lock in the same thread.
A write lock is allowed when no other read or write lock is active.
While waiting for a write lock, no new read locks from other threads are allowed.

ReentrantReadWriteLock

ReentrantReadWriteLock class implements the ReadWriteLock interface.
Useful if we have more read compare to write.

Example: In micro-service communication, assume Service B expects a JWT from Service A. JWT is generated by Service A and can be cacheable for couple of minutes. In this scenarios ReentrantReadWriteLock will be more useful. We can renew the token if it is expired or about to expire. I am not acquiring read lock here to avoid read starvation.

String jwt = JwtUtil.generateJwt();
ReadWriteLock lock = new ReentrantReadWriteLock();
Lock writeLock = lock.writeLock();
Lock readLock = lock.readLock();

public String getJwt(String key, String value) {
    if (StringUtils.hasLength(jwt)) {
        if (JwtUtil.isJwtEligibleToRenew(jwt)) {
            generateJwt();
        }
    } else {
        generateJwt();
    }
    return this.jwt;
}

public void generateJwt() {
    this.writeLock.lock(); //write lock
    try {
        if (JwtUtil.isJwtEligibleToRenew(jwt)) {
            this.jwt = JwtUtil.generateJwt();
        }
    } finally {
        this.writeLock.unlock(); //release write lock
    }
}

StampedLock API

StampedLock was introduced in Java 8 and it is a class. It supports three modes of locking and returns a stamp that is used to release the lock. It allows lock upgradation.

Read Lock Allows multiple threads to read concurrently.
Write Lock Returns a lock used for writing.
Optimistic Read Lock Allows a thread to read without acquiring a traditional read lock, which improves performance as it avoids contention.

long writeLock() Acquires the exclusive lock, blocking if necessary until available.
void unlockWrite(long stamp) Releases the write lock.
long readLock() Acquires the read lock, blocks if a write lock is in progress.
void unlockRead(long stamp) Releases the read lock.
long tryOptimisticRead() Returns a stamp that can later be validated, or zero if exclusively locked.
boolean validate(long stamp) Returns true if the lock has not been exclusively acquired (write lock) since the issuance of the given stamp; otherwise, false.
long tryConvertToWriteLock(long stamp) Upgrades a read or optimistic lock to a write lock. Blocks until it is available.

Key Points:

It is not reentrant, meaning a thread holding a lock cannot acquire it again.
It doesn’t support conditions.
It doesn’t support fairness.

public void readAndWrite() {
    long l = this.stampedLock.readLock();
    try {
        //critical section
    } finally {
        this.stampedLock.unlock(l);
    }

    long w = this.stampedLock.writeLock();
    try {
        //critical section
    } finally {
        this.stampedLock.unlock(w);
    }
}

Optimistic read example. This allows a thread to read without acquiring a traditional read lock, which improves performance as it avoids lock contention. If any write lock acquired after acquiring optimistic read lock time, validate() will return false otherwise it will be true.

public class StampedLockExample {
    private StampedLock stampedLock = new StampedLock();
    private int cachedValue = 0;
    private long optimisticLock = stampedLock.tryOptimisticRead();

    public int readWithOptimisticLock() {
        if (!stampedLock.validate(optimisticLock)) {
            System.out.println(" Optimistic lock is invalid");
            optimisticLock = stampedLock.readLock();
            try {
            } finally {
                stampedLock.unlock(optimisticLock);
            }
            this.optimisticLock = stampedLock.tryOptimisticRead();
        } else {
            System.out.println("Optimistic lock is still valid");
        }
        return cachedValue;
    }

    public static void main(String[] args) {
        StampedLockExample threadsExample = new StampedLockExample();
        threadsExample.readWithOptimisticLock();
        long l = threadsExample.stampedLock.writeLock();
        threadsExample.stampedLock.unlock(l);
        threadsExample.readWithOptimisticLock();
    }
}

Happy Coding and Learning !!!

Please drop a comment if you have any question.

Overview of Executor Service in Java

Dayananda — Tue, 24 Dec 2024 12:46:07 +0000

Executor and ExecutorService API is a crucial tool for managing and controlling thread execution. They are part of java.util.concurrent package. They simplifies the process of concurrent programming by abstracting the complexities of thread creation, management, and synchronization.

Executors is a utility class from java.util.concurrent package provides factory methods for creating and managing different types of ExecutorService instances. It simplifies the process of creating thread pools and allows us to easily create and manage executor instances with different configurations.

Executor API It is an interface available since Java 1.5. It provides execute(Runnable command) method. This is base interface and ExecutorService extends this interface. The given command will be executed in future time by new thread or thread from thread pool or same thread and doesn't returns void.

ExecutorService API It is an interface available since Java 1.5. It provides multiple methods to control the execution of tasks in concurrent programming. It support both Runnable and Callable tasks. It returns Future for task status. Below are the most frequent used methods.

submit() accepts a Callable or a Runnable task and returns Future type result.
invokeAny() accepts a collection of tasks to run, and returns the result of a successful execution of any one task.
invokeAll() accepts a collection of tasks to run, and returns the result of all tasks in the form of a list of Future objects type.
shutdown() it doesn't stop executor service immediately but doesn't accept new tasks. Once all current running tasks finished, it shutdown the executor service.
shutdownNow() it tries to stop the executor service immediately, but it doesn't guarantee that all the running tasks will be stopped at the same time.
awaitTermination(long timeout, TimeUnit unit) blocks/waits until all tasks are completed or timeout occurs or current thread is interrupted, whichever happens first. The current thread will be blocked.

Types of ExecutorService

FixedThreadPool It crates fixed size thread pool with specified number of threads. Tasks submitted are executed concurrently. If there are no tasks, threads will be idle until task is arrived. If threads are busy, tasks will be added queue.

ExecutorService fixedThreadPool = Executors.newScheduledThreadPool(5);
Future<String> submit = fixedThreadPool.submit(() -> {
    System.out.println("Task executed by " + Thread.currentThread().getName());
    return Thread.currentThread().getName();
});
fixedThreadPool.shutdown();

CachedThreadPool Creates pool of threads and automatically adjust number of threads required in the pool based on workload. If thread is idle, for more than 60 seconds, will be terminated. This works well for dynamic loads. As threads will be killed after idle timeout, resources are better utilized here.

ExecutorService fixedThreadPool = Executors.newCachedThreadPool();
Future<String> submit = fixedThreadPool.submit(() -> {
    System.out.println("Task executed by " + Thread.currentThread().getName());
    return Thread.currentThread().getName();
});
fixedThreadPool.shutdown();

SingleThreadExecutor Creates single thread and tasks are executed sequentially. There is no parallel processing here.

ExecutorService fixedThreadPool = Executors.newSingleThreadExecutor();
Future<String> submit = fixedThreadPool.submit(() -> {
    System.out.println("Task executed by " + Thread.currentThread().getName());
    return Thread.currentThread().getName();
});
fixedThreadPool.shutdown()

ScheduledThreadPool/ScheduledExecutor It creates a thread or trhead pool which will have a capability to run the tasks at regular interval or run after certain delay.

ScheduledExecutorService scheduler = Executors.newSingleThreadScheduledExecutor(); // Single-threaded scheduler
ScheduledExecutorService scheduler = Executors.newScheduledThreadPool(5); // Multi-threaded scheduler

scheduler.schedule(task, 10, TimeUnit.SECONDS); // Schedule task to run after 10 seconds.

scheduler.scheduleAtFixedRate(task, 5, 10, TimeUnit.SECONDS);
    //It schedules a task to run every 10 seconds with an initial delay of 5 seconds.
scheduler.scheduleWithFixedDelay(task, 5, 10, TimeUnit.SECONDS);
   //It schedules a task to run with a fixed delay of 10 seconds between the end of one execution and the start of the next, with an initial delay of 5 seconds.
scheduler.schedule(() -> scheduler.shutdown(), 20, TimeUnit.SECONDS);
   //It schedules a shutdown of the scheduler after 20 seconds to stop the example.

Submitting Tasks to ExecutorService
Tasks can be submitted to the ExecutorService using the execute() and submit() methods. The execute() method is used for Runnable tasks, while submit() can handle both Runnable and Callable tasks."

 executor.execute(new RunnableTask()); //fire-and-forgot
 executor.submit(new CallableTask()); //returns the status of task

Shutting Down ExecutorService
It's important to shut down the ExecutorService to release resources. You can do this using the shutdown() and shutdownNow() methods.

executor.shutdown(); // Initiates an orderly shutdown"
executor.shutdownNow(); // Attempts to stop all actively executing tasks.
executor.awaitTermination(long timeout, TimeUnit unit); //blocks the thread until all tasks are completed or timeout occurs or current thread is interrupted, whichever happens first. Returns `true `is tasks completed, otherwise `false`.

Recommended approach to shutdown

executor.shutdown();
try {
    // Wait for tasks to complete or timeout
    if (!executor.awaitTermination(120, TimeUnit.SECONDS)) {
        // If the timeout occurs, force shutdown
        executor.shutdownNow();      
    }
} catch (InterruptedException ex) {
    executor.shutdownNow();
    Thread.currentThread().interrupt();
}

About Runnable

Runnable is an interface and represents a task that can run by threads.
We can run Runnable tasks by using Threads or Executor service.
Runnable is having run() method and doesn't return any data.
We can't throw checked exception.

About Callable

It is introduced in 1.5
It is having call() method and returns type V.
It contains throws Exception meaning, we can throw checked exception.

About Future

It represents a future result of any task.
The result will eventually appear in the Future after request processing is completed.
boolean isDone() Returns the status of request processing. True if completed otherwise false.
boolean cancel(boolean mayInterruptIfRunning) Cancels the submitted task. if we pass mayInterruptIfRunning as false, then it won't cancel already started task.
boolean isCancelled() returns task is cancelled or not.
V get() returns result of task. Blocks the thread if task is not completed.
V get(long timeout, TimeUnit unit) Waits if necessary for at most the given time for the computation to complete, and then retrieves its result. TimeoutException will be thrown after specified time if computation is not completed.

Happy Coding and Learning !!!

Please drop a comment if you have any question.

How to estimate Java object size

Dayananda — Mon, 25 Nov 2024 07:14:12 +0000

In Java, objects are the basic building blocks of any applications. When an object is created, memory is allocated from the heap to store its instance variables. Understanding how much memory an object consumes is important for optimizing memory usage and preventing OutOfMemoryErrors. It is important to optimize the memory consumption especially in cloud solutions.

Object size is calculated as Object Header size + variable size(for primitives) or refence size(for objects). But it is not that simple. Actual memory size consumed depends on data type, architecture, variable object refereed by other object or not(P1 & P2 refers to same country object), String/primitive pools etc...

In Java, primitive member variables are recommended and it consumes less space compare to wrappers. We will do the comparison and see how much memory can be saved with primitive data types.

Object memory size is always multiple of 8. JVM adds padding bytes to make it multiple of 8 if required. If object memory size comes as 12 byte length, then JVM adds 4 padding byte to make it multiple of 8. 12 byte + 4 byte = 16 byte. In this I will be using HotSpot JVM specification for measuring the sizes.

Size Metrics

To analyze size we use three metrics. Shallow size, Deep size, Retained size.

Shallow size In this metrics, we consider object size itself. If it is having reference to other objects, we consider reference size while calculating size metric.

Deep Size In this metrics, we consider object size and size of reference objects.

Retained Size In this metrics, we consider object's shallow size and the objects are only referenced from this object. In other words, the retained size is the amount of memory that can be reclaimed by garbage-collecting this object. We don't consider objects if it is referred by more than one object.

In real, Retained Size metric is very important and it is purely depends on data and object reference structure. It is tricky to calculate and complicate tasks if object(s) are referred by different instances.

Elements influences object size

Object Header

Every object will have object header and it is based on oopDesc data structure. Object header will contain mark word and klass pointer.

Mark Word will contain information related to hash code, locking and GC information. The size of this is 4 byte and 8 byte in 32 and 62 bit architecture respectively.

Klass pointer will contain information related to class, modifiers and super class. The size of this is 4 byte in both architecture.

Object Header size would be 8 byte and 16 byte in 32-bit and 62-bit architecture respectively.

Member Variables

Primitive data types

Primitive data types consumes fixed amount of memory size.

Primitive Wrappers

Primitive Wrappers will consumes additional memory size. For primitive wrappers memory size calculation is Primitive data type size + Object Header size + Padding bytes if required.

Arrays

Arrays can be primitive or non primitive arrays. Memory size for arrays is sum of Object Header + (number of elements * reference for objects /primitive size for primitives) + length field (4 byte always). Reference is 4 byte size always. Length field will store the length of the array.

If we have Person[] with 36 length, then size is 12 + (36 * 4 (reference size) ) + 4 = 160 byte.

If we have byte[] with 36 length, then size is 12 + (36 * 1 (byte size)) + 4 = 52 + 4 padding = 56 byte.

String

The size (shallow) of this String instance is 16 byte and 24 byte in 32 and 62 bit architecture respectively. It includes the 4 byte of cached hash code, 4 byte of char[] reference, and object header.

The actual string is stored as byte array and size of it is (1 (byte size) * number of characters) + 12 ( Object header) + 4 (length). For 36 length string will have (1*36) + 8 + 4 + 4 = 52 + 4 padding = 56 byte.

Total Retained/Deep size of String (of 36 length) is 24 byte + 56 byte = 80 byte.

Lets look into an example

We will first look into class without variables and later get into with variable. We will also look into member variables with wrappers and primitives. We will estimate the size and see how much memory we can save with primitive data types. I will be using VisualVM for profiling the data and 64-bit architecture system.

1. With out any member variables

Consider simple Person class without any variable.

public class Person{ 
}

Now we will create 1000 objects of this and put it array.

public class MemoryMgmtController {
    int count = 1000;
    Person[] persons = new Person[count]; 
    public MemoryMgmtController() {
        for (int i = 0; i < count; i++) {
            persons[i] = new Person();
        }
    }
}

As we know array memory size is sum of Object Header + (array size * 4 byte), So size of Person[] would be 16 byte (with 4 padding byte) + (1000 * 4 byte) = 4016 byte.

In addition to this, we have created 1000 Person objects which will have Object Header information (16 byte with 4 padding byte) by default. So it would be 16 * 1000 = 16000 byte.

So without any data, for simply creating 1000 objects itself has occupied 4016 + 16000 = 20016 byte (~20KB) memory.

2. With wrappers as member variables

Consider simple Person class with wrappers as member variable.

public class Person{ 
    public Boolean active;
    public Short country; 
    public Character gender; 
    public Integer id; 
    public Float salary;
    public Long pk; 
    public Double bonus; 
    public String name; 
}

Now we will create 1000 objects with data and assign it to array.

public class MemoryMgmtController {
    int count = 1000;
    Person[] persons = new Person[count]; 
    public MemoryMgmtController() {
        for (int i = 0; i < count; i++) {
            Person person = new Person();
            person.setActive(Boolean.TRUE);
            person.setCountry((short) 12);
            person.setGender(GENDER);
            person.setId((int) i);
            person.setSalary((float) i);
            person.setPk((long) i);
            person.setBonus((double) i);
            person.setName(UUID.randomUUID().toString());
            persons[i] = person;
        }
    }
}

Size of Person[] would be same as earlier i.e 16 byte (with 4 padding byte) + (1000 * 4 byte) = 4016 byte.

But if we look at size of 1000 objects, it would increase. The reason we have added member variables which will occupy the size. We have added wrappers as member variable so Person object will hold the reference to wrapper objects.

Even though we say object header is 16 byte in 64-bit architecture, that is with 4 padding byte. While estimating, we consider 12 byte for object header, if required padding byte are added.

Total Memory estimation

3. With primitives as member variables

When we use primitives they will consumer fixed size and directly part of Person. Person will not have any reference except String in our scenario.

Using primitives instead of wrappers has saved nearly 45% memory in this scenario. This will vary based on variables and object structure.

Actual memory consumed will depend on so many other factors like retained size, duplicate strings, JVM implementation and many others. Having idea on high level estimation will help us to develop memory optimized applications.

Tips

Use primitives instead of wrappers where ever possible.
Use Trove library to have collections with primitives.
Consider UseCompressedOops java flag to improve the performance in 64-bit architecture.
Use Java Object Layout (JOL) tool to inspect memory layouts of object.

Happy Coding and Learning !!!

Please drop a comment if you have any question.

Mask logs using logstash logback in java with regex

Dayananda — Mon, 11 Nov 2024 10:02:57 +0000

In today's data-driven world, data security is most important. Logging frameworks play a crucial role in application monitoring and debugging, but they can inadvertently expose sensitive information which should not have been. Log masking is a technique that effectively obfuscates sensitive data in log messages, safeguarding confidential information.

Understanding Logback

Logback is a powerful and mostly used logging framework in Java applications. It offers flexible configuration options, including the ability to format log events as JSON objects. It is a successor to the Log4j framework, has quickly gained popularity due to its features and ease of use. It is comprised of Logger, Encoders, Layout, Appender, Encoder.

Logger: Logger is a context for log messages. Applications will interact with this class to create log messages.

Encoders: Encoders are introduced in logback 0.9.91 and responsible for transforming an event into a byte array as well as writing out that byte array into an OutputStream. Encoders introduced as Layouts are only capable to transform an event into a String which restricts their scope to non-binary output.

Layout: The layout is responsible for formatting the logging request according to the user's wishes, whereas an appender takes care of sending the formatted output to its destination.

Appenders: In logback speak, an output destination is called an appender. This places log messages in their final destinations. A Logger can have more than one Appender. Currently, appenders exist for the console, files, remote socket servers, to MySQL, PostgreSQL, Oracle and other databases, JMS, and remote UNIX Syslog daemons.

About Logstash Logback Encoder

The logstash-logback-encoder library is a valuable tool for enhancing your Spring Boot applications' logging capabilities. It provides a convenient way to format log messages in a structured JSON format, making them readily consumable by log aggregation and analysis tools like Logstash. JSON format provides a structured and machine-readable way to log information, making it ideal for advanced log analysis and security measures. Benefits of logstash

JSON Customization Logstash allows you to customize the JSON output to include specific fields and metadata.
Dynamically Fields It also allows Dynamically add fields to log events based on application context.
Improved Readability JSON format offers a clear and human-readable structure for log events.
Enhanced Search and Analysis Log aggregation tools can easily parse and query JSON logs.
Machine Parsing JSON logs are ideal for automated analysis and alerting systems.

Solution to mask the data in logs

Here main goal is to provide a solution to mask the data which is customizable and configurable at runtime.

Here is our simple requirement to

mask password completely in log.
mask phone number and login name except last 5 in log.

Step 1
Create spring boot application. This solution will work with any java based application with little customization.

Step 2
Configure all regular expressions to mask the data. Please remember regex are costly in terms of resource utilization. Make sure you are tuning your regex. Regex groups will allows us to select required substring from string.

Step 3
Create a class and implement MessageJsonProvider. This interface is from logstash and allows us to customize the message before printing to the appender. writeTo method in this interface will be called for every log message.

in start() method read all regex and prepare LogMasker which contains all MaskingRule. This method is from AbstractJsonProvider and simply marks the process started as true.
MaskingRule will hold regex pattern and a function. This function replaces the identified string from in the log.

@Data
public class MaskingMessagingProvider extends MessageJsonProvider {

    public static final String DEFAULT_RULES_DELIMITER = ",";
    private LogMasker logMasker;
    private String rules;

    public MaskingMessagingProvider() {
        super();
    }

    @Override
    public void start() {
        super.start();
        this.logMasker = LogMasker.create(StringUtils.tokenizeToStringArray(rules, DEFAULT_RULES_DELIMITER));
    }

    @Override
    public void writeTo(JsonGenerator generator, ILoggingEvent event) throws IOException {

        if (isStarted()) {
            JsonWritingUtils.writeStringField(generator, getFieldName(), logMasker.mask(event.getFormattedMessage()));
        }
    }
}

class LogMasker {

    private MaskingRule[] masks;

    public LogMasker(MaskingRule[] masks) {
        super();
        this.masks = masks.clone();
    }

    public static LogMasker create(String[] rules) {

        return new LogMasker(Arrays.stream(rules).map(rule -> MaskingRule.create(rule)).toArray(MaskingRule[]::new));
    }

    public String mask(String input) {
        String transformed = input;
        for (MaskingRule m : masks) {
            transformed = m.mask(transformed);
        }
        return transformed;
    }
}

class MaskingRule {
    public static final int REG_EX_DEFAULT_GROUP_SELECTOR = 2;
    public static final String DEFAULT_REPLACEMENT = "*";

    private Pattern pattern;
    private UnaryOperator<String> replacement;

    public MaskingRule(Pattern maskPattern, UnaryOperator<String> replacement) {
        super();
        this.pattern = maskPattern;
        this.replacement = replacement;
    }

    public static MaskingRule create(String rule) {
        return new MaskingRule(Pattern.compile(rule), (in) -> MaskingRule.maskDataWithReplacement(in, DEFAULT_REPLACEMENT));
    }

    public String mask(String transformed) {
        Matcher matcher = pattern.matcher(transformed);
        StringBuffer sb = new StringBuffer();
        while (matcher.find()) {
            matcher.appendReplacement(sb, replacement.apply(getDataToBeMasked(matcher)));
        }
        matcher.appendTail(sb);
        return sb.toString();
    }

    private static String maskDataWithReplacement(String input, String replacement) {
        int repetition = !StringUtils.hasLength(input) ? 0 : input.length();
        return String.join("", Collections.nCopies(repetition, replacement));
    }

    private static String getDataToBeMasked(Matcher matcher) {
        if (matcher.groupCount() > 1) {
            return matcher.group(REG_EX_DEFAULT_GROUP_SELECTOR);
        }
        return matcher.groupCount() > 0 ? matcher.group(1) : "";
    }
}

Step 4
Configure class in logback-spring.xml file.

<configuration>
    <springProperty scope="context" name="rules" source="app.logging.masking.rules"
                    defaultValue=""/>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
            <providers>
                <provider class="com.daya.logging.logstash.MaskingMessagingProvider">
                    <rules>${rules}</rules>
                    <rulesDelimiter>${rulesDelimiter}</rulesDelimiter>
                    <ruleDelimiter>${ruleDelimiter}</ruleDelimiter>
                </provider>
                <threadName/>
                <timestamp/>
                <logLevel/>
                <loggerName/>
                <mdc/>
                <version/>
                <stackTrace/>
            </providers>
        </encoder>
    </appender>
    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
    </root>
</configuration>

Steps 5
Run the application. For simplicity, i have taken a string which is holding data and printing it at application start up.

@SpringBootApplication
@Slf4j
public class LogDataMaskingApplication {

    public static void main(String[] args) {
        SpringApplication.run(LogDataMaskingApplication.class, args);
        LogDataMaskingApplication.maskingTest();
    }

    public static void maskingTest() {
        String data = "{\"loginName\":\"maskingtest\",\"phoneNumber\":\"9898981212\",\"password\":\"Masking@123\"}";
        log.info(data);
    }

}

This is very basics solution and having lot of scope for enhancement as per requirement like message digest etc...

You can find code at GitHub.

Please drop a comment if you have any question.

Content projection in Angular

Dayananda — Mon, 04 Nov 2024 12:39:06 +0000

Content projection, a powerful technique in Angular, allows developers to create highly reusable and customizable components. This approach enhances component reusability, flexibility, leading to cleaner, and more maintainable Angular applications.

In this article, we will delve into what content projection is, types, how to use it, with below real-world scenario for multi-slot content projection.

What is Content Projection

Content projection is a pattern enables us to insert or project content from a parent component into a child(s) component. This approach makes it possible to design flexible components that can accommodate a wide range of content, making them more reusable.

Content Projection types

Single-Slot Content Projection Single-slot content projection uses a single <ng-content> directive, allowing the insertion of content in a specific place.

Multi-Slot Content Projection Multi-slot content projection allows projecting different sections of content into different parts of the template. Multi-slot content projection uses the select attribute (<ng-content select="[card-subtitle]"></ng-content>) to specify which content need to go where.

Few key concepts to be aware

<ng-content> in a component template defines a placeholder area(s). When a parent component uses this component, it pass content between the tags, customizing the child component's behavior.

<ng-container> is an directive that allows to group elements in a template that doesn’t interfere with styles or layout and not rendered into Angular DOM.

<ng-template> is a template element that Angular uses with structural directives. These template elements work in the presence of structural directives and help us to define a template that doesn’t render anything by itself, but conditionally renders them to the DOM.

Example of Single-Slot Content Projection

Step 1: Create child/reusable component

@Component({
  selector: 'app-single-slot-projection',
  template: `<mat-toolbar>
        <ng-content> </ng-content>
         </mat-toolbar>`,
  styles: ``
})
export class SingleSlotProjectionComponent {

}

Step 2: Use Single-slot component in a Parent Component

All the content inside <app-single-slot-projection> will be projected at <ng-content> place of child component.

@Component({
  selector: 'app-root',
  template: `
  <app-single-slot-projection>
      <span>Content Projection Example</span>
    </app-single-slot-projection>
  `,
  styleUrl: './app.component.scss'
})
export class AppComponent {
  title = 'angular-multi-select-picker';  
}

Example of Multi-Slot Content Projection

Let's look at real time application scenario. Consider you are working on dynamic reports generation application which allows user to select required fields in the report. To select required fields in report, we have a screen like above where all available column names will appear in left box, selected column names will appear in the right box. Actions will appear in the middle. You can select required columns from left menu and move it to right side. You will have an option to move all towards right and left.

As part of core team we decide to define standard sections and consuming application can decide what has to go inside of those sections.

Making it more reusable component with all conditional projections will be published soon in another post.

Step 1: Create child/reusable component

As part of this example i am considering 5 places for content project.

<mat-card appearance="outlined">
    <mat-card-header class="mb-2">
        <mat-card-title>
            <ng-content select="[card-title]"></ng-content>
        </mat-card-title>
        <mat-card-subtitle class="mt-2">
            <ng-content select="[card-subtitle]"></ng-content>
        </mat-card-subtitle>
    </mat-card-header>
    <mat-card-content>
        <div class="list_builder">
            <ng-content select="[source-items]"></ng-content>
            <ng-content select="[action-items]"> </ng-content>
            <ng-content select="[target-items]"></ng-content>
        </div>
    </mat-card-content>
</mat-card>

Step 2: Use Multi-slot component in a Parent Component

card-title: displays some static content.
card-subtitle: displays static content wit number of columns available in right side table.
source-items: displays all the available columns available for our requirement.
action-items: displays all actions that can be performed on left and right side tables.
target-items: displays all the selected columns to present in out report.

<main class="main">
  <div class="content">
    <!-- Single Slot Projection -->
    <app-single-slot-projection>
      <span>Content Projection Example</span>
    </app-single-slot-projection>

    <!-- Multi Slot Projection -->
    <app-multi-select-picker>

      <!-- card-title section -->
      <ng-container card-title>Multi Select Picker</ng-container>

      <!-- card-subtitle section -->
      <ng-container card-subtitle>Selected Items Count: {{targetItems.length}}</ng-container>

      <!-- source-items section -->
      <ng-container source-items>
        <mat-card class="source_items">
          <mat-card-content>
            <mat-list role="list">
              @for (data of sourceItems; track $index) {
              <mat-list-item role="listitem" class="selectable_item" [ngClass]="{selected_item: data.selected}"
                (click)="itemSelected(data)">
                {{data.value}}
              </mat-list-item>
              }
            </mat-list>
          </mat-card-content>
        </mat-card>
      </ng-container>

      <!-- action-items section -->
      <ng-container action-items>
        <mat-card class="action_items">
          <mat-card-content class="action_buttons_container">
            <button mat-icon-button class="action_buttons" (click)="moveAll('right')">
              <mat-icon aria-hidden="false" aria-label="Move All Right"
                fontIcon="keyboard_double_arrow_right"></mat-icon>
            </button>
            <button mat-icon-button class="action_buttons" (click)="moveSelected('right')">
              <mat-icon aria-hidden="false" aria-label="Move Right" fontIcon="chevron_right"></mat-icon>
            </button>
            <button mat-icon-button class="action_buttons" (click)="moveSelected('left')">
              <mat-icon aria-hidden="false" aria-label="Move Left" fontIcon="chevron_left"></mat-icon>
            </button>
            <button mat-icon-button class="action_buttons" (click)="moveAll('left')">
              <mat-icon aria-hidden="false" aria-label="Move All Left" fontIcon="keyboard_double_arrow_left"></mat-icon>
            </button>
          </mat-card-content>
        </mat-card>
      </ng-container>

      <!-- target-items section -->
      <ng-container target-items>
        <mat-card class="target_items">
          <mat-card-content>
            <mat-list role="list">
              @for (data of targetItems; track $index) {
              <mat-list-item role="listitem" class="selectable_item" [ngClass]="{selected_item: data.selected}"
                (click)="itemSelected(data)">
                {{data.value}}
              </mat-list-item>
              }
            </mat-list>
          </mat-card-content>
        </mat-card>
      </ng-container>
    </app-multi-select-picker>
  </div>
</main>

Best Practices

Use consistent naming conventions for select attributes.
ngProjectAs supports only static values and not the dynamic expressions.
We need to avoid overcomplicating component with too many slots.

You can find code at GitHub

Please drop a comment if you have any question.

Cross Field validation with Reactive Forms in Angular

Dayananda — Thu, 31 Oct 2024 14:46:35 +0000

Cross-field validation is a important aspect of form validation in Angular applications. It involves checking the relationship between multiple fields to ensure data consistency and accuracy. By implementing strong cross-field validation, we can prevent invalid data from being submitted thus improving the overall user experience.

Implementing Cross-Field Validation in Angular

Lets see how we can add dynamic validation on postal code length based on selected country in customer details page.

Create custom validator

Validator takes country field name, postal code field name and data which is holding postal code length of countries.

function validatePostCodeLengh(firstControl: string, matcherControl: string, postalCodeLength: PostCodeLength[]): ValidatorFn {

  return (control: AbstractControl): ValidationErrors | null => {

    const sourceValue = control.get(firstControl)?.value;
    const targetValue = control.get(matcherControl)?.value;
    if (sourceValue && targetValue) {
      const matchedRecord = postalCodeLength.find(el => el.key === sourceValue);
      if (matchedRecord?.value !== targetValue.length) {
        control.get(matcherControl)?.setErrors({ invalidLength: "Postal code length must be " + matchedRecord?.value });
      }
    }
    return null
  }
}

Create component using reactive form

Use the FormBuilder service to create a reactive form. Apply custom validator to the form.

@Component({
  selector: 'app-address',
  templateUrl: './address.component.html',
  styleUrl: './address.component.scss'
})
export class AddressComponent implements OnInit {

  private formBuilder = inject(FormBuilder);
  profileForm!: FormGroup;
  countries!: Country[];
  postalCodeLenght!: PostCodeLength[];
  ngOnInit(): void {
    //You get this from API
    this.countries = JSON.parse('[{"name":"United States","code":"US"},{"name":"India","code":"IN"}]');
    //You get this from API
    this.postalCodeLenght = JSON.parse('[{"value":5,"key":"US"},{"value":6,"key":"IN"}]');

    this.profileForm = this.formBuilder.group({
      firstName: ['', Validators.required],
      lastName: ['', Validators.required],
      address: this.formBuilder.group({
        country: ['', Validators.required],
        postalCode: ['', [Validators.required]],
      }, { validators: validatePostCodeLengh('country', 'postalCode', this.postalCodeLenght) }),
    });
  }

  profileFormSubmit() {
    if (this.profileForm.valid) {
      alert('Data validation is successful');
    } else {
      console.log(this.profileForm.get('address.postalCode')?.getError('invalidLength'));
      alert('Please fill all required details');
    }
  }

}

Create angular template

Angular Material is used for styling purpose.

<mat-card appearance="outlined">
    <mat-card-content>
        <form [formGroup]="profileForm" class="profile-form" (submit)="profileFormSubmit()">
            <mat-form-field class="profile-full-width">
                <mat-label>First Name</mat-label>
                <input matInput placeholder="First Name" formControlName="firstName">
                @if (profileForm.get('firstName')?.hasError('required')) {
                <mat-error>Required</mat-error>
                }
            </mat-form-field>
            <mat-form-field class="profile-full-width">
                <mat-label>Last Name</mat-label>
                <input matInput placeholder="Last Name" formControlName="lastName">
                @if (profileForm.get('lastName')?.hasError('required')) {
                <mat-error>Required</mat-error>
                }
            </mat-form-field>
            <div formGroupName="address">
                <mat-form-field class="profile-full-width">
                    <mat-label>Country</mat-label>
                    <mat-select formControlName="country">
                        @for (country of countries; track country) {
                        <mat-option [value]="country.code">{{country.name}}</mat-option>
                        }

                    </mat-select>
                    @if (profileForm.get('address.country')?.hasError('required')) {
                    <mat-error>Required</mat-error>
                    }
                </mat-form-field>
                <mat-form-field class="profile-full-width">
                    <mat-label>Postal Code</mat-label>
                    <input matInput placeholder="Postal Code" formControlName="postalCode">
                    @if (profileForm.get('address.postalCode')?.hasError('required')) {
                    <mat-error>Required</mat-error>
                    }
                    @if (profileForm.get('address.postalCode')?.hasError('invalidLength')) {
                    <mat-error>{{profileForm.get('address.postalCode')?.getError('invalidLength')}}</mat-error>
                    }
                </mat-form-field>
            </div>
            <button mat-raised-button style="float: right; ">Submit</button>
        </form>
    </mat-card-content>
</mat-card>

Lets see how it works

Additional Considerations

Use debounceTime and distinctUntilChanged operators to optimize real-time validation and prevent excessive API calls.
Use asynchronous validators for complex checks that require external data or API calls.
Provide clear and concise error messages to guide the user in correcting invalid input.

You can find code at GitHub

Please drop a comment if you have any question.

How to detect blocking calls in Spring Webflux

Dayananda — Mon, 28 Oct 2024 17:27:31 +0000

Spring WebFlux, a reactive framework built on Project Reactor, is a powerful tool for building non-blocking, asynchronous applications. However blocking operations can inadvertently creep in, degrading performance and responsiveness. BlockHound is a library designed to detect and potentially mitigate blocking operations in the application, making it a valuable tool for optimizing Spring WebFlux applications.

Understanding Blocking Operations in Reactive Systems

In a reactive programming, blocking operations are those that halt the thread of execution until a task is completed. This can significantly impact performance, especially in high-throughput scenarios. Common blocking operations in Spring WebFlux application includes:

Database Access Synchronous database operations will block threads, especially while using traditional JDBC or ORM frameworks.
Network I/O Blocking network calls, such as traditional HTTP requests, can block the thread and downgrade performance.
File I/O Synchronous file operations also leads to blocking the thread.
Thread sleep If you have added thread sleep in your code, it will also result to blocking the thread.
External library Some of your external libraries also may add blocking nature

How BlockHound Works

BlockHound operates by instrumenting the JVM at runtime to detect blocking operations. It can be configured to either throw an exception when a blocking operation is detected. By identifying and addressing these blocking points, developers can significantly improve the performance and scalability of their Spring WebFlux applications.

Enabling BlockHound

Step 1 Add maven dependency.

<dependency>
    <groupId>io.projectreactor.tools</groupId>
    <artifactId>blockhound</artifactId>
    <version>MENTION_LATEST_VERSION</version>
</dependency>

Step 2 Pass -XX:+AllowRedefinitionToAddDeleteMethods JVM option if using Java 13+. Here i passing args through file.

Step 3 Instrument BlockHound to the application

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import reactor.blockhound.BlockHound;

@SpringBootApplication
public class WebfluxExApplication {

    static {
        BlockHound.install();
    }

    public static void main(String[] args) {
        SpringApplication.run(WebfluxExApplication.class, args);
    }

}

Now you will see an exception logged into console and request will be stopped further processing if any blocking calls present in your request flow.

Points to remember

It is not a code scan tool which scans and tell you where blocking call present.
It will throw exception if it find any blocking call in your request flow.
It is advised to not to deploy the code with BlockHound to production as it will result into throwing an error if blocking call is found.

By understanding the potential pitfalls of blocking operations and leveraging tools like BlockHound, you can build highly responsive and efficient Spring WebFlux applications.

Please leave a comment if you have any question.

data masking and disable cut copy paste in angular using directives

Dayananda — Wed, 23 Oct 2024 10:14:10 +0000

Masking is a technique that is used to hide sensitive information, such as card numbers, account numbers, phone numbers, ssn etc. This can help to protect against fraud and identity theft. In addition to this we may be asked to disable the cut-copy-paste options too.

In this article, I will show you how to create an Angular directives that can be used to mask data and disable cut-copy-paste.

Step 1: Create a disable cut-copy-paste directive

import { Directive, HostListener } from '@angular/core';

@Directive({
  selector: '[ftp-disable-ccp]'
})
export class DisableCcpDirective {

  constructor() { }

  @HostListener("copy", ["$event"])
  blockCopy(e: KeyboardEvent) {
    e.preventDefault();
  }

  @HostListener("paste", ["$event"])
  blockPaste(e: KeyboardEvent) {
    e.preventDefault();
  }

  @HostListener("cut", ["$event"]) blockCut(e: KeyboardEvent) {
    e.preventDefault();
  }

}

Step 2: Create data masking directive

I am masking the except last 4 characters data with 'X'. If you want to change this behavior to particular case you can pass those as input props.

import { Directive, ElementRef, HostListener, Input } from '@angular/core';

@Directive({
  selector: '[ftp-masking]'
})
export class MaskingDirective {

  @Input() maskwith = "X";
  @Input() maskRegex = /.(?=.{4})/g;

  private value: string = '';

  private element!: HTMLInputElement;

  constructor(private el: ElementRef) {
    this.element = el.nativeElement;
  }

  ngonInit(): void {
    this.value = this.element.value;
  }

  @HostListener('input')
  @HostListener('change')
  onValueChange() {
    this.value = this.element.value;
  }

  @HostListener('blur')
  onBlur() {
    this.maskData()
  }

  @HostListener('focus')
  onFocus() {
    this.unmaskData();
  }

  maskData() {
    //masking the data except last 4 digits using X. 
    //You can customize this as it is input to the directive
    this.element.value = this.element.value.replace(this.maskRegex, this.maskwith);
  }
  unmaskData() {
    this.element.value = this.value;
  }

}

Step 3: Use the Directive

Now, we can use our directive in our Angular template. Please refer Account Number field

<mat-card appearance="outlined">
            <mat-card-content>
                <form [formGroup]="profileForm" (ngSubmit)="onSubmit()">
                    <mat-form-field class="example-full-width">
                        <mat-label>First Name</mat-label>
                        <input matInput placeholder="First Name" value="" formControlName="firstName">
                    </mat-form-field>
                    <mat-form-field class="example-full-width">
                        <mat-label>Last Name</mat-label>
                        <input matInput placeholder="Last Name" value="" formControlName="lastName">
                    </mat-form-field>
                    <mat-form-field class="example-full-width">
                        <mat-label>Account Number</mat-label>
                        <input matInput placeholder="Account Number" value="" formControlName="accountNumber" ftp-disable-ccp
                            ftp-masking>
                    </mat-form-field>
                    <button mat-raised-button [disabled]="profileForm.invalid" style="float: right; ">Submit</button>
                </form>
            </mat-card-content>
        </mat-card>

Conclusion

Data masking is an important security measure. By using an Angular directives, we can easily implement masking in our applications.

Please leave a comment if you have any question.

Strategy Design Pattern in Spring

Dayananda — Mon, 21 Oct 2024 11:57:27 +0000

In enterprise applications, it is common to have different implementations to our business requirements. At run time we need to execute one of those based on conditions. Common examples are sorting algorithms, data encryption, notification services, authentication services, payment services and much more.

Instead of using if-else conditions, we can use a strategy design pattern to implement it. Strategy design pattern will help us to run/use different implementations at run time. Strategy approach provides code reusability, flexibility, separation of concerns and extensibility.

Consider you have developed a unified authentication service which will receive requests from different channels/clients. At run time, based on configuration, you need to decide the implementation used for authentication purposes (This could be request forward too). Your implementations can be okta, Azure Ad, or Your Own IAM.

Note: We need to implement providers as independent modules and add it to the main project as dependency.

Strategy Interface

public interface Authentication {

    AuthenticationRes authentication(AuthenticationReq authenticationReq);
    OtpDevicesRes otpDevices(OtpDevicesReq otpDevicesReq);
    SendOtpRes sendOtp(SendOtpReq sendOtpReq);
    VerifyOtpRes verifyOtp(VerifyOtpReq verifyOtpReq);

}

Concrete Strategies

@Component("oktaAuthentication")
@Slf4j
public class OktaAuthentication implements Authentication {
    --------------
    --------------
    --------------
}

@Component("ferAuthentication")
@Slf4j
public class FerAuthentication implements Authentication {
      --------------
      --------------
      --------------
}

@Component("eapAuthentication")
@Slf4j
public class EapAuthentication implements Authentication {
      --------------
      --------------
      --------------
}

Service

@Service
@Slf4j
public class AuthenticationService {

    public Map<String, Authentication> authenticationProvidersMap;
    public Set<String> availableAuthProviders;

    public AuthenticationService(Map<String, Authentication> authenticationProvidersMap) {
        this.authenticationProvidersMap = authenticationProvidersMap;
        this.availableAuthProviders = this.authenticationProvidersMap.keySet();
        log.info("Available Auth providers:{}", this.availableAuthProviders);
    }

    public AuthenticationRes getAuthentication(AuthenticationReq authenticationReq, ClientDetails clientDetails) {

        //This method will identify authentication provider based on client details
        // and returns oktaAuthentication/eapAuthentication/ferAuthentication
        String authProvider = getAuthProviderDetails(clientDetails);

        if (this.availableAuthProviders.contains(authProvider)) {
            return this.authenticationProvidersMap.get(authProvider)
                    .authentication(authenticationReq);
        } else {
            throw new AuthProviderUnavailable(authProvider);
        }

    }


    public String getAuthProviderDetails(ClientDetails clientDetails) {
        // implement your business logic to return the provider that need to be used. 
    }
}

If you have any question please leave a question in comments section.

Large JSON file processing in Java

Dayananda — Mon, 21 Oct 2024 06:33:06 +0000

Efficient JSON data processing is important in modern applications. If JSON data is large in size, we need to handle it properly to avoid heap out of memory error and heavy resource consumption.

GSON is one of the best library available in Java to process JSON data.

It provides thread safe instance to reuse the same object to process multiple requests.
It doesn't expect any annotations to be available to serialization and deserialization unless there is a special need.
It provides toJson() and fromJson() methods to serialize and deserialize.
It provides support to read the content using stream readers.

Below is the code snippet to read the file using java stream readers and convert to Java object. I am applying my business logic for every 100 records.

Assumption JSON file is having array of data like below.
[ { "id": 1, "firstName": "Emily", "lastName": "Johnson" } ]
Java Code

private Optional<Boolean> readFile(String path) {
    try (InputStream inputStream = Files.newInputStream(Path.of(path));
         JsonReader reader = new JsonReader(new InputStreamReader(inputStream))) {
        Instant start = Instant.now();
        reader.beginArray();
        List<User> users = new ArrayList<>();
        while (reader.hasNext()) {
            users.add(gson.fromJson(reader, User.class));
            if (users.size() >= 100) {
                //Initiate your records processing
                //You can consider processing it in async with properly handling failure scenarios 
                users.clear();
            }
        }
        reader.endArray();
        LOGGER.info("Total time taken to process file:{}, Duration:{} (millis)", path, Duration.between(start, Instant.now()).toMillis());
    } catch (Exception e) {
        LOGGER.error("", e);
        return Optional.of(false);
    }
    return Optional.of(true);
}

If you have any question please leave a question in comments section.