Forem: Ernesto Herrera Salinas

Empower Your Workflow with Aliases: Save Time by Being Lazy

Ernesto Herrera Salinas — Sun, 27 Jul 2025 01:43:20 +0000

In software development, every keystroke counts. That’s where aliases come in—tiny shortcuts that let you run long or repetitive commands with minimal typing. The Git and shell aliases shared in your Gist are a powerful example of how developers can streamline their daily work.

The Gist entitled “BASH/ZSH ALIASES FOR THE LAZY DEVELOPER” provides a curated set of aliases tailored to:

Git workflows
Docker Compose
NPM commands
General system productivity tools (Hacker News, DEV Community, Gist)

🚀 Why Use Aliases?

Speed & Efficiency
Commands like git status -s become simply s, or git push origin $(git rev‑parse --abbrev‑ref HEAD) becomes push. This cuts down typing and mental context switching.
Consistency
Reuse familiar shortcuts across machines by storing them in your shell startup file (.bashrc, .zshrc, etc.).
Error Reduction
Avoid typos in complex commands by using shorter, predefined aliases.

📦 Highlighted Alias Categories

🔧 Git Workflow Aliases

Alias	Action
`push`	Push current branch to origin: `git push origin $(git rev‑parse --abbrev-ref HEAD)`
`pull`	Pull current branch and submodules
`s`	`git status -s`
`a`	`git add . && git status -s`
`l`	`git log --oneline --all --graph --decorate`
`gb`	`git fetch && git checkout <branch>`
`undo`	`git checkout -- <file>`
`reset`	`git reset --hard HEAD~1`
`clean`	Dry‑run `git clean`
`branch`	`git checkout -b`
`friday`	End-of-week branch setup + push with date-based naming

‍

These handle daily Git tasks faster, and even some creative eco‑friendly shortcuts like friday combine multiple steps into one single alias. (Gist, DEV Community)

🐳 Docker / Compose Aliases

dc, dcu, dcb, dcd, etc. Simplify running and managing Docker Compose commands, often with sudo. A big win for developers frequently working with containers. (Gist)

📦 NPM Aliases

nr, nrs, nrb, npi, nru, npo, npa, and more Quickly run npm scripts like npm run or common tasks like build, lint, test, audit—all with just two or three characters. (Gist)

💻 System Aliases

ls='ls -lah --color=auto' A colorized, human-readable version of ls that’s easier on the eyes. (Gist)

🧠 Productivity Best Practices

Based on articles and expert guidance on command aliasing, here are some best practices to multiply your aliases effectively:

Alias your most-used commands, not everything. Keep aliases memorable & limited. (DEV Community)
Avoid single‑letter overrides of essential shell built‑ins like r or x.
Document your aliases with comments—both for your future self and collaborators. (DEV Community)
Use Git aliases via ~/.gitconfig when possible for consistency across shells and machines. They also integrate with Git’s auto-completion and autocorrect features. (Gist, Opensource.com)

✅ Sample Alias Setup (excerpt from `.bashrc` or `.zshrc`)

# Git shortcuts
alias s="git status -s"
alias a='git add . && git status -s'
alias l='git log --oneline --all --graph --decorate'
alias branch='git checkout -b'
alias undo='git checkout --'

# Push current branch
alias push='git push origin $(git rev-parse --abbrev-ref HEAD)'
alias pull='git pull --recurse-submodules origin $(git rev-parse --abbrev-ref HEAD)'

# YAML end-of-week workflow
alias friday='pull && git checkout -b "friday-$(date +%Y-%m-%d)" && git commit -a -m "[WIP] Friday $(date +%Y-%m-%d)" && push'

🧰 Tips to Customize Further

Use shell history to identify frequently typed commands that deserve an alias.
Keep alias names intuitive—e.g. d for docker, nr for npm run.
Combine commands: a adds + status, l logs + graph formatting.

🔚 Wrap-Up

Aliases may look small, but they deliver huge productivity gains:

Fewer keystrokes means faster workflows and less wrist fatigue.
Simplified mental load—no need to remember verbose flags or commands.
Reliable, error‑free execution of routine tasks.

Use your Gist as a template—customize it to your projects and style. Whether you're managing Git, Docker, NPM, or system commands, well-crafted aliases let you work smarter, not harder.

Let me know if you'd like help tailoring these for your team or expanding them with more advanced patterns like chained commands or conditional logic!

Data-Driven Testing in Jest (Parameterized Tests)

Ernesto Herrera Salinas — Wed, 02 Apr 2025 11:37:48 +0000

What is Data-Driven Testing?

Data-driven testing (also known as parameterized or table-driven testing) is a technique where the same test logic is run against multiple sets of input data. Instead of writing separate test functions for each input scenario, you externalize the test cases (inputs and expected outputs) and feed them into a single, generic test. In other words, the test behavior is driven by a collection of data values. This approach allows one test script to execute with different inputs by separating the data from the test code (What is Data-Driven Testing? Enhancing Accuracy Through Data). The result is more thorough coverage with less repetitive code: you avoid duplicating test code for each case and make it easy to add or modify test scenarios as needed (Parameterized tests in JavaScript with Jest).

Key idea: Design a test once, then run it for a variety of data values. The data sets can come from an in-memory array, an external file (like JSON or CSV), or even a database query (Parameterized tests in JavaScript with Jest). For each data set, the test will execute and verify that the code under test produces the expected outcome for that input. This method is particularly useful for pure functions or algorithms where you want to validate numerous input-output combinations (including edge cases) without writing dozens of nearly identical test functions.

Benefits of Data-Driven Testing

Adopting a data-driven approach in your test suite provides several important benefits:

Avoids Duplication and Reduces Boilerplate: You write the test logic once and reuse it for all data variations. This prevents copy-pasting similar test code multiple times (Parameterized tests in JavaScript with Jest) (Simplify repetitive Jest test cases with test.each - DEV Community). Fewer repeated test functions mean a cleaner, more maintainable test file.
Easier Maintainability: Adding or updating test cases is as simple as modifying the data source. You can insert new input/expected pairs into the data set without touching the test logic at all (Simplify repetitive Jest test cases with test.each - DEV Community). This isolation of data makes it less likely to introduce errors when extending your tests.
Improved Test Coverage: By feeding many different inputs into the same test, you can cover more scenarios with minimal effort. Simply changing or expanding the data set increases coverage without additional test code (What is Data-Driven Testing? Enhancing Accuracy Through Data). This encourages testing normal cases, edge cases, and invalid inputs alike.
Scalability for Large Data Sets: Data-driven tests handle large collections of test cases gracefully. Whether you have 5 cases or 500, the structure of the test remains the same. This scales well for situations like mathematical functions or algorithms that need verification against lots of values.
Enhanced Readability of Test Intent: When structured well, data-driven tests make it clear what is being tested with each input. By using descriptive names or placeholders in test titles, each data-driven test's purpose is evident (e.g. “153 should be an Armstrong number” for a particular input). The list of test cases itself acts as documentation of expected behavior for various inputs.
Consistency and Lower Risk of Human Error: Because the same code path is used for all test cases, you ensure consistent execution. There’s less risk that one of many copy-pasted tests has a typo or mistake – one logic covers all cases. This also makes it easier to update the assertion or test steps in one place if requirements change.

In summary, data-driven testing separates test data from test scripts, which enhances reusability and maintainability of tests (Guide to Data-Driven Testing - BugBug.io). It leads to more thorough testing with less effort by letting simple data variations exercise the code in depth (What is Data-Driven Testing? Enhancing Accuracy Through Data).

Implementing Data-Driven Tests in Jest

Jest – a popular JavaScript testing framework – provides built-in support for data-driven (parameterized) testing. Rather than manually writing loops, you can use Jest’s utilities to supply a table of inputs to a single test definition. This is typically done with the test.each or it.each methods (in Jest, it is just an alias for test). The official Jest documentation recommends using test.each when you find yourself duplicating the same test with different data (Globals · Jest). By using this feature, you “write the test once and pass data in” for each case (Globals · Jest).

Jest offers two primary ways to define data-driven tests:

Using an Array of Cases: You can call test.each(table)(name, fn) where table is an array of arrays (or an array of objects) representing the test cases (Globals · Jest) (Globals · Jest). Each inner array contains the arguments for one test invocation, and each element in the array of objects can be destructured in the test function. For example:
```
const cases = [
  [2, 2, 4],       // a, b, expected
  [-2, -2, -4],    // a, b, expected
  [2, -2, 0]       // a, b, expected
];
test.each(cases)("given %p and %p, returns %p", (a, b, expected) => {
  expect(add(a, b)).toBe(expected);
});
```
In this example, three sets of inputs (a and b) with their expected output are provided. Jest will generate three sub-tests from this one definition, substituting each set of values. The %p placeholders in the test title are tokens that Jest replaces with the actual parameter values for easier identification of each case (Simplify repetitive Jest test cases with test.each - DEV Community). The test output would list “given 2 and 2, returns 4” etc., making it clear which case passed or failed.

Jest also supports using an array of objects instead of arrays. In that case, each object’s properties can be used in the test function via destructuring. For example, you might have [{a: 2, b: 2, expected: 4}, {...}, ...] and use a test title placeholder like %o to print the object, or individually interpolate values in the title. This can sometimes improve clarity if you label the data fields. Under the hood, each element (array or object) in the cases array will result in a separate test execution.
Using a Tagged Template (Table Syntax): Jest allows a more readable table format using template literals. This is invoked as test.each\`table\`(name, fn). The first row of the template defines column names, and subsequent rows define values. For example:
```
test.each`
  a    | b    | expected
  ${2} | ${2} | ${4}
  ${2} | ${-2} | ${0}
  ${-2} | ${-2} | ${-4}
`("add(${'$a'}, ${'$b'}) = ${'$expected'}", ({a, b, expected}) => {
  expect(add(a, b)).toBe(expected);
});
```
Here we use a table layout for readability. Jest will convert each row into a test case, and we can reference the named variables (a, b, expected) in both the test name and the test function. This format can be very clear, though it’s essentially syntactic sugar on top of test.each functionality (Globals · Jest).

Both approaches accomplish the same goal: parameterizing a test. Each row of data creates a new test with the full lifecycle (setup, execution, teardown) just like any normal test case (Parameterized tests in JavaScript with Jest). In the test report, each case is listed separately, which helps pinpoint which input failed if something goes wrong.

Alternative: Manual iteration. In addition to Jest's built-in test.each, one can also simply loop through data within a test suite. For example, you might load an array of test cases and call test() or expect() in a .forEach loop. This achieves a similar effect – dynamically generating tests – and is sometimes used if custom processing of data is needed. However, using Jest’s native parameterization is generally cleaner and provides better reporting. With test.each, you avoid potential pitfalls of asynchronous loops and ensure each case is treated as a separate test by Jest automatically. As a best practice, prefer test.each or it.each for simplicity, unless you have a specific reason to manually generate tests.

Using External Data (JSON) for Test Cases in Jest

One powerful pattern in data-driven testing is to keep the test cases in external files (like JSON) for even cleaner separation of data from code. Jest, running on Node.js, can directly import JSON files using require() or ES import. This means you can store an array of inputs and expected outputs in a .json file, load that file in your test, and feed it into test.each or a loop.

Using an external JSON brings a few advantages: if you have a very large number of test cases or want non-developers to be able to review/edit test data, a separate file is convenient. It also declutters the test file, making the logic more readable. The Jest team and testing experts encourage this approach when dealing with extensive datasets (it.each function in Jest | BrowserStack).

Example of loading JSON test data:

Suppose we have a JSON file testCases.json containing an array of test inputs and outputs for a function (for instance, a set of number pairs to add). It might look like this:

[
  { "input1": 2, "input2": 2, "expected": 4 },
  { "input1": 5, "input2": 7, "expected": 12 },
  { "input1": -3, "input2": 3, "expected": 0 }
]

In the Jest test suite, you can load and use this data as follows:

const testData = require('./testCases.json');

describe('Addition function', () => {
  it.each(testData)('adds $input1 and $input2, expecting $expected', ({ input1, input2, expected }) => {
    const result = add(input1, input2);
    expect(result).toBe(expected);
  });
});

In this snippet, it.each(testData) will iterate over each object in the JSON array and run the test. We use $input1, $input2, and $expected in the test name string to interpolate those values for each case’s name. The test body destructures the object to get input1, input2, and expected, performs the addition, and asserts the result. This pattern of loading external data is directly supported by Jest and keeps tests simple (it.each function in Jest | BrowserStack). The output will list a separate test for each set of inputs, making it clear which data set passed or failed.

Jest even allows other formats (like CSV or other modules) to be used as data sources, as long as you can import or read them in Node. This aligns with the idea that the data could come from any source (CSV file, database, etc.), though JSON is most straightforward in JavaScript. When using large data sets, storing them externally is recommended because it keeps the test file shorter and easier to read, and avoids cluttering your code with lengthy inline arrays (it.each function in Jest | BrowserStack).

Maintainability, Scalability, and Readability Improvements

Data-driven tests significantly improve maintainability of the test code. Since all scenarios use a single implementation, there is one place to update if the function under test changes behavior. For example, if the formula or logic in a function changes, you might only need to adjust the expected values in the data file or adjust the assertion in one spot, rather than editing many individual test functions. Adding new test scenarios is as easy as adding a new entry to the cases array or JSON – no new test() boilerplate needed (Simplify repetitive Jest test cases with test.each - DEV Community). This modularity means the test suite can grow in coverage without a corresponding explosion in code size.

For scalability, consider that some algorithms may need to be verified against dozens or hundreds of inputs (think of a prime number checker, or our Armstrong number example below). Writing 100 separate tests is error-prone and hard to manage; a data-driven approach can handle this by design. Each new data row is effectively a new test, and Jest will handle running them all. There isn’t a practical limit (within reason) to how many cases you can add in a data-driven test – you could even generate them programmatically if needed. The test output will simply show each case result, and you can use grouping (describe) to organize if there are logical subsets of cases.

In terms of readability, a well-structured data-driven test can actually make tests easier to understand at a glance. The reason is that the core logic is written once, clearly, and the list of test cases (data) reads like a specification of expected outcomes. By using descriptive test names with placeholders, anyone reading the test results or code can infer the purpose of each case. Moreover, keeping the data separate (in an array or file) means the test file isn’t bogged down with repetitive code – it focuses on the behavior being verified. This separation of concerns (logic vs data) is a hallmark of clean testing practices (Parameterized tests in JavaScript with Jest).

It's worth noting that each data-driven test case is still isolated and reported separately by Jest. If one case fails, it doesn't stop the others from running, and you get a precise report of which input failed. This granular reporting, combined with dynamic test names, improves debuggability as well – you immediately know which input caused an issue.

To maximize readability and maintainability, follow these tips (many of which apply to any test, but especially parameterized ones):

Keep the data structures simple and representative of the inputs. If needed, include comments or use object keys that make the meaning of each value obvious (Simplify repetitive Jest test cases with test.each - DEV Community).
Use placeholders or interpolation in test names to clearly identify each test case in output. For example, use tokens like %i, %s, %o in the test title (as supported by Jest) to print numbers, strings, or objects (Data-driven Unit Tests with Jest - DEV Community) (Simplify repetitive Jest test cases with test.each - DEV Community). This way, a failing test message immediately shows which data set failed.
Group related data-driven tests using describe() blocks if it makes sense to separate contexts or functionalities. Jest also offers describe.each to run an entire group of tests under multiple conditions (though that is more advanced usage).
Ensure the data sets cover not just typical cases but edge cases (e.g., for Armstrong: 0, 1, largest n-digit Armstrong, etc.). The ease of adding cases means it's feasible to include edge scenarios that might be overlooked in manual one-off tests.
If a particular data-driven test grows too large or complex, consider splitting it into smaller ones for different categories of inputs (for example, valid vs invalid inputs in separate tests) to maintain clarity.

By following these patterns, the test suite remains readable and easy to extend, even as new requirements emerge. Developers can scan a data file or array to see everything that's being validated, and the single test definition remains uncluttered and focused.

Best Practices for Data-Driven Tests in Jest

When implementing data-driven (parameterized) tests in Jest, keep in mind some best practices to get the most out of this technique:

Use Data-Driven Tests Intentionally: Only use test.each (or similar) when you truly have the same logic being tested with varying inputs. If each case requires different setup or fundamentally different assertions, separate test blocks might be clearer. In short, parameterize tests when it removes duplication, but don't force unrelated scenarios into one loop (it.each function in Jest | BrowserStack).
Leverage Descriptive Test Names: Make sure each generated test has a unique and descriptive name. Utilize format strings or template literals to include input values in the name (Data-driven Unit Tests with Jest - DEV Community). This practice makes it much easier to identify failing cases. For example, "isArmstrong(%i) should return %s" is clearer than a generic name for all cases.
Use Placeholders and Template Literals for Clarity: Jest’s %p (pretty print), %i (integer), %s (string), and %o (object) placeholders in test titles can automatically format values (Data-driven Unit Tests with Jest - DEV Community). Alternatively, the tagged template form allows you to reference variables directly in the title. Use these to your advantage so that your test output is self-explanatory and each case can be distinguished (it.each function in Jest | BrowserStack).
Organize Test Data Logically: If the data set is large, consider splitting it into multiple smaller sets or using multiple describe.each blocks for different categories. This can prevent one huge table from becoming unwieldy. Also, keep the data sorted or structured (for example, all expected-true cases first, then false cases) if that helps readability.
Externalize Large Data Sets: As noted, move large case tables to external files (JSON/CSV) for clarity (it.each function in Jest | BrowserStack) (it.each function in Jest | BrowserStack). This also allows non-developers or testers to review the test cases easily. Ensure that your test runner is configured to include these files (with Jest, simply requiring the JSON is sufficient).
Stay Consistent: Use a consistent approach across your test suite. For instance, if you use test.each in one place, use similar patterns elsewhere when appropriate so that other contributors quickly understand the style. Consistency in how data-driven tests are written will make your test codebase more uniform and predictable.
Watch Out for Async Data Sources: If your data is coming from an asynchronous source (like a database or API call), make sure to retrieve it before calling test.each. For example, use a beforeAll to fetch data, then inside it define or run the parameterized tests. The data needs to be available at the time the tests are defined. (If using a static JSON file, this isn’t an issue – it’s loaded immediately.)
Validate Test Data: It can be useful to ensure your test data itself is correct (especially if it's large or hand-crafted). Simple sanity checks (like ensuring no duplicate cases, or that expected outputs match a known formula for known inputs) can save debugging time later. This isn't specific to Jest, but a general good practice when you rely on external test vectors.

Following these best practices helps ensure that your parameterized tests remain efficient and understandable rather than turning into a confusing abstraction (it.each function in Jest | BrowserStack). The goal is to simplify testing, and with a thoughtful approach, data-driven tests can greatly streamline your test suite.

Example: Armstrong Number Tests with Data-Driven Approach

To cement the concepts, let's consider an Armstrong number checker function and see how data-driven tests apply. An Armstrong number (also known as a narcissistic number) is an n-digit number that is equal to the sum of each of its digits raised to the power of n. For example, 153 is a 3-digit Armstrong number because 13+53+33=1531^3 + 5^3 + 3^3 = 153. We want to test a function isArmstrong(num) that returns true if num is an Armstrong number, and false otherwise.

Traditional approach (for comparison): Without data-driven testing, you might write separate tests like:

test('153 is an Armstrong number', () => {
  expect(isArmstrong(153)).toBe(true);
});
test('9474 is an Armstrong number', () => {
  expect(isArmstrong(9474)).toBe(true);
});
test('123 is not an Armstrong number', () => {
  expect(isArmstrong(123)).toBe(false);
});
⋮

If you have many such numbers to test, writing individual tests becomes tedious and repetitive. Instead, we can list all our interesting test cases (both Armstrong numbers and non-Armstrong numbers with expected false) and iterate through them in one sweep.

Data-driven approach with Jest: We create a JSON file (or simply an array in the test file) of cases. For example, armstrong-cases.json might contain:

[
  { "number": 0,    "expected": true },
  { "number": 1,    "expected": true },
  { "number": 153,  "expected": true },
  { "number": 9474, "expected": true },
  { "number": 9475, "expected": false },
  { "number": 123,  "expected": false }
]

Here we include some known Armstrong numbers (0, 1, 153, 9474 are Armstrong numbers) and some that are not (9475, 123, etc.). Now our Jest test can load this data and use it.each to create a test for each entry:

const cases = require('./armstrong-cases.json');

describe('isArmstrong()', () => {
  it.each(cases)('returns $expected for $number', ({ number, expected }) => {
    expect(isArmstrong(number)).toBe(expected);
  });
});

When this test suite runs, Jest will generate a separate sub-test for each object in cases. The test name will interpolate the values, so you’ll see output like:

✓ returns true for 0
✓ returns true for 1
✓ returns true for 153
✓ returns true for 9474
✓ returns false for 9475
✓ returns false for 123

Each of those corresponds to one row in our JSON. We have effectively written one template test and supplied it with multiple inputs. This makes it very easy to verify a bunch of Armstrong numbers and non-Armstrong numbers in one go. If tomorrow we want to test another number (say 370, which is also an Armstrong number), we just add { "number": 370, "expected": true } to the JSON file – no other code changes required.

This Armstrong number test case demonstrates how data-driven testing improves readability (the test cases are clearly enumerated), maintainability (new cases can be added without modifying test logic), and coverage (we can include many examples, including edge cases like 0 or 1). The structure follows the same pattern described earlier for any data-driven test in Jest. In fact, this approach is identical to the addition example shown previously, just applied to a different problem domain (it.each function in Jest | BrowserStack). Whether it's Armstrong numbers, mathematical functions, or any scenario with input-output pairs, the methodology remains the same.

Conclusion

Data-driven testing in Jest is a powerful pattern that simplifies how we write repetitive tests. By separating the data from the test code, we gain cleaner tests that are easy to extend and hard to accidentally break when requirements change. Jest’s built-in support via test.each/it.each makes implementing parameterized tests straightforward – turning an array of cases into a suite of individual tests with one concise definition. This approach yields a more maintainable, scalable, and readable test suite, as demonstrated with the Armstrong number example and others.

By leveraging official Jest features and following best practices (like descriptive test names and externalizing large datasets), you can improve your test quality and coverage with minimal overhead. In summary, data-driven tests allow you to “write once, test many”, ensuring your code is verified against a wide range of inputs while keeping the test code DRY (Don’t Repeat Yourself) (Parameterized tests in JavaScript with Jest). It’s a technique well worth using whenever you have numerous similar test scenarios to cover in your JavaScript projects.

Sources:

Jest Official Documentation on Parameterized Tests (test.each) (Globals · Jest) (Globals · Jest)
Community tutorials on data-driven testing with Jest (Simplify repetitive Jest test cases with test.each - DEV Community) (Simplify repetitive Jest test cases with test.each - DEV Community)
Best practice guides for using it.each with external data (it.each function in Jest | BrowserStack) (it.each function in Jest | BrowserStack)
Explanation of parameterized tests and their benefits in unit testing (Parameterized tests in JavaScript with Jest)

The Lazy Engineer’s Guide to Automating Timesheets: Part 1

Ernesto Herrera Salinas — Tue, 28 Jan 2025 23:34:47 +0000

Ah, timesheets. The Lex Luthor of every software engineer’s existence. If you’re like me, you’d rather debug a race condition at 3 a.m. than spend even five minutes documenting what you did all day. But alas, as freelancers or full-time employees, we’re often stuck with this tedious chore.

This year, I hit my breaking point. After a whirlwind of projects—some canceled, some pivoted, and others shelved indefinitely—I found myself staring down the barrel of a year-end timesheet deadline. The thought of manually reconstructing my entire year’s work made me want to scream. So, I did what any self-respecting engineer would do: I decided to automate the problem away.

This is the story of how I turned a day of dread into a coding adventure. Buckle up—it’s going to be a lazy, efficient ride.

The Problem: Timesheets Are the Worst

Let’s set the scene:

The Task: Document every hour spent on every task for the entire year.
The Obstacle: My memory is about as reliable as a flaky integration test.
The Deadline: One day. Yes, one day.

I knew I couldn’t do this manually. So, I hatched a plan to pull data from all the tools I use daily—JIRA, Git, Slack, and Outlook—and stitch it together into a coherent timesheet.

The Tools of the Trade

Here’s what I used:

JIRA: For tracking tasks and tickets.
Git: For commit history (because every good engineer ties commits to tickets, right?).
Slack: For team communication (because meetings and messages count too).
Outlook: For calendar events (because apparently, meetings are work too).

Step 1: Pulling JIRA Tickets

My first stop was JIRA. I needed to pull all the tickets assigned to me within a specific date range. Thankfully, JIRA has a robust API, and with a little Python magic, I was able to automate this process.

The Script

Here’s the Python script I wrote to fetch JIRA tickets:

import os
from jira import JIRA
import pandas as pd
from datetime import datetime
import logging
import sys
from typing import List, Dict, Any
import argparse

# Configure logging
logging.basicConfig(
    level=logging.INFO,
    format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)

class JiraTicketPuller:
    def __init__(self, server: str, email: str, api_token: str):
        """
        Initialize JIRA client with authentication credentials.

        Args:
            server: JIRA server URL
            email: User's email for authentication
            api_token: JIRA API token
        """
        try:
            self.jira = JIRA(server=server, basic_auth=(email, api_token))
            logger.info("Successfully connected to JIRA")
        except Exception as e:
            logger.error(f"Failed to connect to JIRA: {str(e)}")
            sys.exit(1)

    def get_user_tickets(
        self, 
        username: str, 
        start_date: str, 
        end_date: str
    ) -> List[Dict[str, Any]]:
        """
        Retrieve all tickets assigned to a user within a date range.

        Args:
            username: JIRA username to search for
            start_date: Start date in YYYY-MM-DD format
            end_date: End date in YYYY-MM-DD format

        Returns:
            List of dictionaries containing ticket information
        """
        try:
            # Construct JQL query
            jql_query = (
                f'assignee = {username} AND '
                f'created >= {start_date} AND '
                f'created <= {end_date} '
                'ORDER BY created DESC'
            )

            # Fields to retrieve
            fields = [
                'summary',
                'status',
                'priority',
                'issuetype',
                'created',
                'updated',
                'project',
                'components',
                'description'
            ]

            # Get issues using JQL
            issues = self.jira.search_issues(
                jql_query,
                maxResults=1000,  # Adjust based on your needs
                fields=fields
            )

            tickets_data = []
            for issue in issues:
                ticket = {
                    'Key': issue.key,
                    'Summary': issue.fields.summary,
                    'Status': str(issue.fields.status),
                    'Priority': str(issue.fields.priority),
                    'Issue Type': str(issue.fields.issuetype),
                    'Created': issue.fields.created[:10],  # Get date only
                    'Updated': issue.fields.updated[:10],  # Get date only
                    'Project': str(issue.fields.project),
                    'Components': ', '.join([str(c) for c in issue.fields.components]),
                    'Description': issue.fields.description or ''
                }
                tickets_data.append(ticket)

            logger.info(f"Retrieved {len(tickets_data)} tickets for user {username}")
            return tickets_data

        except Exception as e:
            logger.error(f"Error retrieving tickets: {str(e)}")
            return []

    def export_to_csv(self, tickets: List[Dict[str, Any]], output_file: str):
        """
        Export tickets data to CSV file.

        Args:
            tickets: List of ticket dictionaries
            output_file: Path to output CSV file
        """
        try:
            if not tickets:
                logger.warning("No tickets to export")
                return

            df = pd.DataFrame(tickets)
            df.to_csv(output_file, index=False, encoding='utf-8')
            logger.info(f"Successfully exported tickets to {output_file}")

        except Exception as e:
            logger.error(f"Error exporting to CSV: {str(e)}")

def validate_date(date_str: str) -> bool:
    """Validate date string format (YYYY-MM-DD)."""
    try:
        datetime.strptime(date_str, '%Y-%m-%d')
        return True
    except ValueError:
        return False

def main():
    # Set up argument parser
    parser = argparse.ArgumentParser(
        description='Pull JIRA tickets for a user within a date range'
    )
    parser.add_argument('--username', required=True, help='JIRA username')
    parser.add_argument('--start-date', required=True, help='Start date (YYYY-MM-DD)')
    parser.add_argument('--end-date', required=True, help='End date (YYYY-MM-DD)')
    parser.add_argument(
        '--output', 
        default='jira_tickets.csv', 
        help='Output CSV file path'
    )

    args = parser.parse_args()

    # Validate dates
    if not all(validate_date(date) for date in [args.start_date, args.end_date]):
        logger.error("Invalid date format. Please use YYYY-MM-DD")
        sys.exit(1)

    # Configuration
    JIRA_SERVER = os.getenv("JIRA_SERVER")
    JIRA_EMAIL = os.getenv("JIRA_EMAIL")
    JIRA_API_TOKEN = os.getenv("JIRA_API_TOKEN")

    # Initialize ticket puller
    puller = JiraTicketPuller(JIRA_SERVER, JIRA_EMAIL, JIRA_API_TOKEN)

    # Get and export tickets
    tickets = puller.get_user_tickets(
        args.username,
        args.start_date,
        args.end_date
    )
    puller.export_to_csv(tickets, args.output)

if __name__ == "__main__":
    # Ensure JIRA API token is set
    if not os.getenv("JIRA_API_TOKEN"):
        logging.error("JIRA_API_TOKEN environment variable not set")
        return
    main()

How It Works

Authentication: The script uses your JIRA email and API token to authenticate.
JQL Query: It constructs a JQL query to fetch tickets assigned to you within a date range.
Data Export: The results are exported to a CSV file for easy analysis.

Step 2: Fetching Git Commits

Next, I turned to Git. Since our team follows a practice of including JIRA ticket IDs in commit messages, I wrote a script to extract commit data and map it to tickets.

The Script

import subprocess
import csv
import re

def get_git_commits(since_date=None, author=None):
    # Get commit data with full commit message
    cmd = ['git', 'log', '--pretty=format:%h|%ad|%s', '--date=iso']

    if since_date:
        cmd.extend(['--since', since_date])
    if author:
        cmd.extend(['--author', author])

    result = subprocess.run(cmd, capture_output=True, text=True)
    commits = result.stdout.strip().split('\n')

    # Process commits to extract JIRA IDs
    processed_commits = []
    jira_pattern = re.compile(r'JIRA:\s*([A-Z]+-\d+)')

    for commit in commits:
        if commit:  # Skip empty lines
            hash_id, date, message = commit.split('|', 2)
            jira_match = jira_pattern.search(message)
            jira_id = jira_match.group(1) if jira_match else ''

            processed_commits.append({
                'date': date,
                'commit': hash_id,
                'jira_id': jira_id
            })

    # Write to CSV
    csv_filename = 'git_commits_jira.csv'
    fieldnames = ['date', 'commit', 'jira_id']

    with open(csv_filename, 'w', newline='') as csvfile:
        writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
        writer.writeheader()
        writer.writerows(processed_commits)

    return processed_commits

How It Works

Git Log: The script uses git log to fetch commit history.
JIRA ID Extraction: It uses regex to extract JIRA ticket IDs from commit messages.
CSV Export: The results are saved to a CSV file for later use.

Step 3: Tackling Slack Messages

Slack was trickier. Messages are context-heavy, and mapping them to specific tasks isn’t straightforward. I briefly considered using AI to parse the data but decided against it due to cost and complexity. Instead, I created a generic ticket to capture communication time and wrote a script to fetch Slack messages.

The Script

import os
from datetime import datetime
from slack_sdk import WebClient
from slack_sdk.errors import SlackApiError
import pandas as pd

def get_all_conversations(client):
    """
    Get all conversations (channels, private channels, and DMs) the bot has access to.

    Args:
        client: Slack WebClient instance

    Returns:
        list: List of conversation objects
    """
    conversations = []

    try:
        # Get all conversations (channels, private channels, and DMs)
        result = client.conversations_list(
            types="public_channel,private_channel,im,mpim",
            limit=1000
        )
        conversations.extend(result["channels"])

        # Handle pagination
        while result.get("response_metadata", {}).get("next_cursor"):
            cursor = result["response_metadata"]["next_cursor"]
            result = client.conversations_list(
                types="public_channel,private_channel,im,mpim",
                cursor=cursor,
                limit=1000
            )
            conversations.extend(result["channels"])

    except SlackApiError as e:
        print(f"Error fetching conversations: {e.response['error']}")

    return conversations

def get_user_info(client, user_id):
    """
    Get user information from their ID.

    Args:
        client: Slack WebClient instance
        user_id: User's Slack ID

    Returns:
        dict: User information including real name and email
    """
    try:
        result = client.users_info(user=user_id)
        user = result["user"]
        return {
            "real_name": user.get("real_name", "Unknown"),
            "email": user.get("profile", {}).get("email", "Unknown"),
            "display_name": user.get("profile", {}).get("display_name", "Unknown")
        }
    except SlackApiError:
        return {"real_name": "Unknown", "email": "Unknown", "display_name": "Unknown"}

def get_messages_from_conversation(client, channel_id, channel_name, start_date, end_date):
    """
    Retrieve messages from a specific conversation within a date range.

    Args:
        client: Slack WebClient instance
        channel_id (str): The ID of the Slack channel
        channel_name (str): The name of the Slack channel
        start_date (str): Start date in format 'YYYY-MM-DD'
        end_date (str): End date in format 'YYYY-MM-DD'

    Returns:
        list: List of messages within the date range
    """
    messages = []
    start_timestamp = int(datetime.strptime(start_date, '%Y-%m-%d').timestamp())
    end_timestamp = int(datetime.strptime(end_date, '%Y-%m-%d').timestamp())

    try:
        # Initialize the cursor for pagination
        cursor = None
        while True:
            # Get messages using conversations.history
            result = client.conversations_history(
                channel=channel_id,
                oldest=start_timestamp,
                latest=end_timestamp,
                limit=100,
                cursor=cursor
            )

            # Process messages
            for msg in result['messages']:
                user_id = msg.get('user')
                user_info = get_user_info(client, user_id) if user_id else {
                    "real_name": "Unknown",
                    "email": "Unknown",
                    "display_name": "Unknown"
                }

                message_data = {
                    'channel_name': channel_name,
                    'channel_id': channel_id,
                    'text': msg.get('text', ''),
                    'user_id': user_id,
                    'user_name': user_info['real_name'],
                    'user_email': user_info['email'],
                    'timestamp': datetime.fromtimestamp(float(msg['ts'])).strftime('%Y-%m-%d %H:%M:%S'),
                    'thread_ts': msg.get('thread_ts', None)
                }
                messages.append(message_data)

            # Check if there are more messages to fetch
            cursor = result.get('response_metadata', {}).get('next_cursor')
            if not cursor:
                break

    except SlackApiError as e:
        print(f"Error fetching messages from {channel_name}: {e.response['error']}")

    return messages

def main():
    # Set your date range
    start_date = "2024-01-01"
    end_date = "2024-01-31"

    # Initialize the Slack client
    slack_token = os.environ.get('SLACK_BOT_TOKEN')
    if not slack_token:
        raise ValueError("Please set the SLACK_BOT_TOKEN environment variable")

    client = WebClient(token=slack_token)

    # Get all conversations
    print("Fetching all conversations...")
    conversations = get_all_conversations(client)

    # Fetch messages from all conversations
    all_messages = []
    total_conversations = len(conversations)

    print(f"Found {total_conversations} conversations. Fetching messages...")

    for idx, conv in enumerate(conversations, 1):
        channel_id = conv["id"]
        channel_name = conv.get("name", "DM")  # DMs don't have names

        print(f"Processing {idx}/{total_conversations}: {channel_name}")

        messages = get_messages_from_conversation(
            client,
            channel_id,
            channel_name,
            start_date,
            end_date
        )
        all_messages.extend(messages)

    # Convert to DataFrame for easier handling
    df = pd.DataFrame(all_messages)

    # Save to CSV
    output_file = f"slack_messages_{start_date}_to_{end_date}.csv"
    df.to_csv(output_file, index=False, encoding='utf-8')

    print(f"\nTotal messages retrieved: {len(all_messages)}")
    print(f"Messages saved to: {output_file}")

    # Print summary
    print("\nMessages per channel:")
    channel_summary = df.groupby('channel_name').size().sort_values(ascending=False)
    print(channel_summary)

if __name__ == "__main__":
    main()

How It Works

Conversation List: The script fetches all channels and DMs accessible to the bot.
Message Retrieval: It retrieves messages within a specified date range.
CSV Export: The messages are saved to a CSV file for further analysis.

Step 4: Capturing Outlook Meetings

Finally, I needed to account for meetings. Using the exchangelib Python library, I wrote a script to pull calendar events and export them to a CSV.

The Script

from exchangelib import Credentials, Account
import pandas as pd
from datetime import datetime, timedelta

def get_outlook_meetings():
    # Replace with your email and password
    email = os.environ.get("JIRA_EMAIL")
    password = os.environ.get('OUTLOOK_PASSWORD')

    # Connect to Exchange
    credentials = Credentials(email, password)
    account = Account(email, credentials=credentials, autodiscover=True)

    # Get calendar items for the last 30 days (adjust as needed)
    start_date = datetime.now() - timedelta(days=30)
    end_date = datetime.now() + timedelta(days=30)

    # Get calendar items
    calendar_items = account.calendar.view(
        start=start_date,
        end=end_date
    )

    # Prepare data structure
    meetings = []

    # Process each appointment
    for item in calendar_items:
        # Calculate duration in minutes
        duration = int((item.end - item.start).total_seconds() / 60)

        meetings.append({
            'date': item.start.strftime('%Y-%m-%d'),
            'time': item.start.strftime('%H:%M'),
            'duration': duration,
            'title': item.subject
        })

    # Create DataFrame and save to CSV
    df = pd.DataFrame(meetings)
    df.to_csv('outlook_meetings.csv', index=False)

    return df

# Run the function
if __name__ == "__main__":
    meetings_df = get_outlook_meetings()
    print(f"Exported {len(meetings_df)} meetings to outlook_meetings.csv")

How It Works

Authentication: The script uses your Outlook email and password to authenticate.
Calendar Query: It fetches calendar events within a specified date range.
CSV Export: The events are saved to a CSV file.

What’s Next?

At this point, I had four CSV files:

JIRA Tickets: All the tasks I worked on.
Git Commits: All the code I wrote.
Slack Messages: All the communication I participated in.
Outlook Meetings: All the meetings I attended.

In Part 2, I’ll show you how I stitched these datasets together to create a comprehensive timesheet. Spoiler alert: It involves more Python, some data wrangling, and a little bit of magic.

Stay tuned, and remember: Laziness is the mother of invention.

What’s your least favorite chore as a software engineer? Have you automated it yet? Share your stories in the comments!

Private npm Repositories

Ernesto Herrera Salinas — Thu, 02 Jan 2025 23:01:41 +0000

Private npm Repositories: Taking Control of Your Code

As software craftsmen, we understand that code isn’t just a tool—it’s a legacy. And like any craftsman, we guard our tools fiercely. What happens, though, when your team’s shared libraries or proprietary modules need to be distributed securely, efficiently, and without exposing your intellectual treasure to the world? Enter the private npm repository—a vault for your code artifacts. Let’s explore how to build, configure, and wield this essential tool.

Why Private npm Repositories Matter

Imagine a workshop where every tool is labeled, organized, and accessible only to those who’ve earned the right to use them. That’s what a private npm registry offers:

Security Without Compromise

Keep sensitive code behind your firewall. No more leaking API keys or proprietary logic into public domains.
Speed as a Virtue

Reduce dependency on external networks. Your builds run faster when packages are hosted locally, cached, and always available.
Governance You Can Enforce

Not every developer needs access to every package. Control who publishes, who installs, and who audits.
Versioning Without Chaos

Manage internal dependencies cleanly. No more “works on my machine” surprises when public packages shift beneath your feet.

Choosing Your Registry: A Craftsman’s Guide

Private registries come in many forms. Your choice depends on scale, infrastructure, and philosophy. Let’s weigh the options:

1. Self-Hosted Solutions

For teams who own their infrastructure:

Verdaccio: Lightweight, open-source, and perfect for small teams. It proxies public npm, caches dependencies, and requires minimal setup.
Sonatype Nexus: The Swiss Army knife of repositories. Supports npm, Maven, Docker—ideal for polyglot environments.
JFrog Artifactory: Enterprise-grade, with advanced security and scalability.

2. Git-Integrated Registries

For those already immersed in Git workflows:

GitHub Packages: Tightly integrated with GitHub. Publish once, and your code and packages live side-by-side.
GitLab Packages: Built into GitLab’s DevOps pipeline. CI/CD and package management in one breath.
Bitbucket: Less native, but achievable through add-ons like npm Registry or custom scripts.

3. npm Enterprise

For large organizations needing turnkey solutions:

SSO integration, compliance audits, and dedicated support. It’s npm, but on your terms.

Crafting Your Registry: Hands-On

1. Verdaccio: The Artisan’s Choice

Verdaccio is to npm registries what a well-honed chisel is to a woodworker: simple, effective, and precise.

Step 1: Install

npm install -g verdaccio  # Sharpening the tool

Step 2: Configure

Fire up Verdaccio:

verdaccio  # The registry awakens at http://localhost:4873

Tweak ~/.config/verdaccio/config.yaml to suit your needs:

# Like a key to your workshop 

auth:  
  htpasswd:  
    file: ./htpasswd  
    max_users: 100  

# Proxy public packages to avoid external delays 
uplinks:  
  npmjs:  
    url: https://registry.npmjs.org/

# Define access rules—guard your tools 
packages:  
  '@myteam/*':  
    access: $authenticated  
    publish: $authenticated

Step 3: Publish

npm adduser --registry http://localhost:4873  # Claim your space 
npm publish --registry http://localhost:4873  # Forge the artifact

Step 4: Consume

Either use --registry for one-off installs or lock it in .npmrc:

registry=http://localhost:4873

2. GitHub Packages: The Collaborator’s Haven

When your code lives on GitHub, why scatter packages elsewhere?

Step 1: Authenticate

Generate a Personal Access Token (PAT) with read:packages and write:packages scopes. Then, in .npmrc:

//npm.pkg.github.com/:_authToken=ghp_YourTokenHere  
@your-org:registry=https://npm.pkg.github.com

Step 2: Publish with Precision

Update package.json to reflect your org’s scope:

{  
  "name": "@your-org/secret-sauce",  
  "publishConfig": {  
    "registry": "https://npm.pkg.github.com"  
  }  
}

Then:

npm publish  # Your secret sauce is now sealed

Step 3: Install Seamlessly

Any teammate with access can:

npm install @your-org/secret-sauce  # Serve it hot

3. GitLab Packages: The CI/CD Power Play

GitLab’s registry is a natural fit for teams automating their pipelines.

Step 1: Configure Access

Add to .npmrc:

@your-group:registry=https://gitlab.com/api/v4/packages/npm/  
//gitlab.com/api/v4/packages/npm/:_authToken=YourPAT

Step 2: Publish from the Pipeline

In your .gitlab-ci.yml:

publish:  
  script:  
    - echo "//gitlab.com/api/v4/packages/npm/:_authToken=${CI_JOB_TOKEN}" > .npmrc  
    - npm publish

Enterprise Forges: Nexus and Artifactory

For large-scale operations, these tools are the industrial presses of package management.

Nexus Quickstart:

Install Nexus, create an npm hosted repository.
Point your .npmrc to http://nexus.yourcorp.com/repository/npm-internal/.
npm login --registry=http://nexus.yourcorp.com and publish.

Artifactory in a Nutshell:

Spin up Artifactory, create a local npm repo.
Configure .npmrc with registry=http://artifactory.yourcorp.com/artifactory/api/npm/npm-local/.
Publish, and let Artifactory handle replication, caching, and security.

Principles for the Prudent Craftsman

Scope Your Packages

Prefix internal packages with @your-org/. It’s namespacing, not bureaucracy.
Secure Your .npmrc

Keep tokens out of Git. Use environment variables or encrypted secrets.
Proxy Public Packages

Let your registry cache public dependencies. Faster installs, fewer outages.
Automate Relentlessly

Bake publishing into CI/CD. No manual npm publish—humans err, pipelines endure.

Conclusion: Own Your Artifacts

A private npm registry isn’t just a technical choice—it’s a statement of ownership. Whether you’re a solo developer safeguarding side projects or an enterprise architecting a global pipeline, the principles remain:

Control your dependencies.
Secure your code.
Streamline your workflow.

Choose the tool that fits your shop. Set it up with care. Then go forth, craft brilliantly, and let your code thrive—safely, efficiently, and entirely on your terms.

After all, isn’t that what clean craftsmanship is all about?

A Technical Introduction to LocalStack

Ernesto Herrera Salinas — Thu, 02 Jan 2025 06:32:52 +0000

LocalStack is an excellent tool that allows you to emulate AWS services on your local machine. By spinning up a local environment that replicates various AWS services, developers can speed up their development and testing cycles without incurring cloud costs or requiring an active AWS account. In this post, we’ll discuss what LocalStack is, how to install it, and how you can use it with various AWS services in your local development workflow.

What is LocalStack?

LocalStack is a fully functional local AWS cloud stack. It provides mock implementations of many popular AWS services, including S3, DynamoDB, Lambda, and more. Using LocalStack, you can:

Simulate AWS services locally for development and testing.
Avoid AWS billing charges for creating and destroying resources repeatedly.
Speed up the development cycle by removing the dependency on external AWS infrastructure.
Test AWS infrastructure and application logic in a controlled environment.

LocalStack is built using Python, Docker, and various mocks and emulators for the different AWS services. Once you have LocalStack running, you can use your existing AWS CLI or SDK commands, provided you point them to your local endpoints.

Installation

Prerequisites

Docker: LocalStack uses Docker containers to spin up AWS mock services. Make sure you have Docker installed and running.
Python 3+ (if you prefer to install the LocalStack CLI via pip).

Installing LocalStack

The recommended way to run LocalStack is by using Docker Compose. You can either:

Use the LocalStack Docker Image, or
Use docker-compose.yml to customize your environment.

Below is an example docker-compose.yml configuration:

version: "3.8"
services:
  localstack:
    image: localstack/localstack:latest
    container_name: localstack
    ports:
      - "4566:4566"   # LocalStack edge port
      - "4571:4571"
    environment:
      - SERVICES=s3,dynamodb,lambda   # Which services to start
      - DEBUG=1
      - DATA_DIR=/tmp/localstack/data
      - LAMBDA_EXECUTOR=docker
      - HOST_TMP_FOLDER=/tmp/localstack
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock"
      - "./localstack-data:/tmp/localstack"

Note: LocalStack exposes port 4566 (the “edge” port) to accept requests for all services. This can simplify your endpoint configurations.

After saving this file, run:

docker-compose up

LocalStack will now start in your terminal, showing logs about each service being initialized.

Using LocalStack

Once LocalStack is running, you can interact with it using:

AWS CLI (by configuring endpoints)
AWS SDKs (e.g., boto3 in Python)
LocalStack CLI (optional, if installed locally)

Configuring the AWS CLI

To interact with LocalStack using the AWS CLI, you need to specify the LocalStack endpoint and credentials. For example:

aws configure set aws_access_key_id test
aws configure set aws_secret_access_key test
aws configure set default.region us-east-1

Then, you can specify the --endpoint-url for each service call. For instance, to create an S3 bucket:

aws s3api create-bucket \
    --bucket local-bucket \
    --endpoint-url http://localhost:4566

To list S3 buckets:

aws s3api list-buckets \
    --endpoint-url http://localhost:4566

You should see your locally created buckets in the output.

Using LocalStack with `boto3` (Python)

Below is a short Python snippet showing how to connect to LocalStack’s S3 service using boto3.

import boto3

# Create an S3 client pointing to LocalStack
s3 = boto3.client(
    's3',
    endpoint_url='http://localhost:4566',
    aws_access_key_id='test',
    aws_secret_access_key='test',
    region_name='us-east-1'
)

# Create a bucket
bucket_name = 'local-bucket'
s3.create_bucket(Bucket=bucket_name)

# List buckets
response = s3.list_buckets()
print("Buckets:")
for bucket in response['Buckets']:
    print(f"  - {bucket['Name']}")

# Upload a file
file_content = b"Hello from LocalStack!"
s3.put_object(Bucket=bucket_name, Key='hello.txt', Body=file_content)

# Download the file
res = s3.get_object(Bucket=bucket_name, Key='hello.txt')
downloaded_content = res['Body'].read()
print("Downloaded file content:", downloaded_content.decode('utf-8'))

Running this script will interact with your local S3 service hosted by LocalStack.

Example: DynamoDB

Let’s create a DynamoDB table and add a few items to it:

aws dynamodb create-table \
    --table-name MyLocalTable \
    --attribute-definitions AttributeName=Id,AttributeType=S \
    --key-schema AttributeName=Id,KeyType=HASH \
    --billing-mode PAY_PER_REQUEST \
    --endpoint-url http://localhost:4566

Now, list all tables:

aws dynamodb list-tables \
    --endpoint-url http://localhost:4566

And insert data:

aws dynamodb put-item \
    --table-name MyLocalTable \
    --item '{"Id": {"S": "1001"}, "Name": {"S": "Test Item"}}' \
    --endpoint-url http://localhost:4566

Finally, query to see your data:

aws dynamodb scan \
    --table-name MyLocalTable \
    --endpoint-url http://localhost:4566

Example: Lambda

To test Lambda functions locally, you can:

Use SAM CLI to build and deploy your Lambda code into LocalStack.
Or directly create your Lambda function using a zip package and uploading via AWS CLI.

Creating a zip for a simple Python function:

# main.py
def handler(event, context):
    return {
        "statusCode": 200,
        "body": "Hello from LocalStack!"
    }

# Zip the file
zip function.zip main.py

Create the Lambda function in LocalStack:

aws lambda create-function \
    --function-name LocalLambda \
    --handler main.handler \
    --runtime python3.9 \
    --zip-file fileb://function.zip \
    --role arn:aws:iam::123456:role/irrelevant \
    --endpoint-url http://localhost:4566

Test the function:

aws lambda invoke \
    --function-name LocalLambda \
    --payload '{}' \
    --endpoint-url http://localhost:4566 \
    output.json

Check the results in output.json:

cat output.json
# {
#    "statusCode": 200,
#    "body": "Hello from LocalStack!"
# }

Tips and Best Practices

Service-Specific Ports vs. Edge Port: By default, all AWS services in LocalStack are available through the edge port (4566). If you need to, you can enable service-specific ports (e.g., 4572 for S3). However, it’s recommended to stick to the edge port for simplicity.
Persisting Data: By default, LocalStack does not persist data across restarts. To persist data, set the DATA_DIR environment variable or mount a volume in Docker (as in the example docker-compose.yml).
Integration Testing: LocalStack can be used for end-to-end integration tests without deploying to AWS. You can run your CI/CD workflows using LocalStack in Docker containers and test your infrastructure changes reliably.
Upgrading: LocalStack is actively developed, so new features and bug fixes are frequently released. Keep your Docker image updated to take advantage of the latest changes.

Conclusion

LocalStack is a powerful tool that brings AWS infrastructure right to your local machine. Whether you’re developing new services, testing your IaC (Infrastructure as Code) templates, or running integration tests, LocalStack provides a quick, cost-effective, and convenient solution.

By following the examples above, you’ll be able to spin up essential AWS services like S3, DynamoDB, and Lambda in your local environment. This approach not only accelerates your development cycle but also ensures that you can verify your application logic thoroughly before deploying to real AWS infrastructure.

Give it a try and see how it transforms your local AWS development experience!

References:

Navigating the World of Event-Driven Process Orchestration for Technical Leaders

Ernesto Herrera Salinas — Sat, 28 Dec 2024 10:04:15 +0000

Event-Driven Process Orchestration

Introduction

Imagine you're at a large, bustling party with many activities—games, snacks, music, and more. Event-driven process orchestration is like having an exceptionally skilled party planner who ensures everything happens at the right moment and in the most efficient way possible. This planner monitors the flow of activities (or “events”) and dynamically reacts to changes, ensuring seamless coordination across the board.

In technical terms, event-driven process orchestration is designed to respond to significant changes in state—business events—in real-time or near real-time. This approach is especially beneficial when processes are complex, rapidly evolving, and highly responsive to new information or changing conditions.

Event-driven orchestration involves:

Event Detection: Monitoring for events from sensors, user interfaces, or external applications.
Process Orchestration: Coordinating workflows and business logic responding to these events.
Dynamic Response: Adapting to new data and conditions in real-time or near real-time.
Integration of Services: Seamlessly tying together various services and applications to automate processes. Decentralization and Scalability: These allow individual components to be loosely coupled and scale independently as the event load varies.
Real-Time Analytics and Monitoring: Continuously tracking orchestration performance and adjusting operations as needed.

Event-driven orchestration enables businesses to be more agile and competitive in everything from finance and e-commerce to supply chain management and IoT. Below, we delve deeper into how event-driven process orchestration works, its benefits, and how you can begin implementing it in your organization.

Understanding Event-Driven Process Orchestration

A Basic Python Example

Let’s start with a simple code snippet illustrating the basics of event-driven orchestration. The scenario involves processing an order as soon as an event (e.g., a new order request) is detected:

import json

# Event Listener
def on_new_order_received(event_data):
    order = json.loads(event_data)
    process_order(order)

# Process/Service
def process_order(order):
    check_inventory(order)
    if inventory_available(order):
        confirm_order(order)
        ship_order(order)
    else:
        notify_out_of_stock(order)

# Example event data
event_data = '{"orderId": 123, "item": "Widget", "quantity": 10}'

# Simulate event occurrence
on_new_order_received(event_data)

on_new_order_received: Listens for new orders.
process_order: Orchestrates the next steps—inventory checks, order confirmation, shipping, or notification of stock unavailability.

While this example is straightforward, it demonstrates the essence of event-driven thinking: we detect an event (“new order”) and initiate processes accordingly.

A Distributed Example with Apache Kafka

In real-world scenarios, event-driven orchestration spans multiple services, data sources, and message brokers. One of the most popular distributed event streaming platforms is Apache Kafka. Kafka provides:

Topics: Named related data streams where producers publish, and consumers subscribe.
Producers: Applications or services that write data to Kafka topics.
Consumers: Applications or services that read data from Kafka topics.
Kafka Streams: A powerful library for processing and analyzing event streams in real-time.

Consider an online retail system that tracks user activities (such as viewing products, adding items to the cart, and making purchases) for recommendations, inventory management, and order processing. Below is a simplified pseudocode example in Python:

from kafka import KafkaProducer, KafkaConsumer
import json

# Kafka Producer for publishing user activity events
producer = KafkaProducer(bootstrap_servers='kafka-server:9092')

def publish_user_activity(user_id, activity):
    event = {
        "user_id": user_id,
        "activity": activity
    }
    producer.send('user_activities', json.dumps(event).encode('utf-8'))

# Kafka Consumer for processing user activity events
consumer = KafkaConsumer('user_activities', bootstrap_servers='kafka-server:9092')

def process_user_activity():
    for message in consumer:
        event = json.loads(message.value.decode('utf-8'))
        if event['activity'] == 'purchase':
            process_purchase(event['user_id'])

# Simple orchestration for purchase events
def process_purchase(user_id):
    # Here you could implement logic for inventory check, order processing, etc.
    print(f"Processing purchase for user {user_id}")

# Example usage
publish_user_activity(123, 'view')
publish_user_activity(123, 'add_to_cart')
publish_user_activity(123, 'purchase')

process_user_activity()  # Typically runs as a service or background process

publish_user_activity: Sends user activity events to a Kafka topic named user_activities.
process_user_activity: Listens to the topic and triggers relevant business processes (like process_purchase).
Kafka: Helps decouple the event producers from event consumers, providing scalability, fault tolerance, and high throughput.

Comparison with Traditional Orchestration Methods

In traditional (often synchronous) orchestration models, a central workflow engine directly invokes services step-by-step—think of it a tightly controlled assembly line. Event-driven orchestration, by contrast, decouples producers and consumers, relying on events to trigger asynchronous responses. This model is more flexible and scalable because:

Loose Coupling: Each component can evolve independently if the event contracts (message formats) remain stable.
Scalability: More consumers or producers can be added independently to handle peaks in demand.
Resilience: If one component fails, it doesn’t necessarily bring down the entire system. Events can be cached in brokers like Kafka until the consumer returns online.

Key Principles and Benefits

Principles

Asynchronicity

Components operate independently and communicate through events rather than direct synchronous calls.
Decentralization

Responsibility is distributed among multiple services or microservices, reducing single points of failure.
Event-First Design

System interactions are driven by creating, detecting, and handling events, emphasizing data flow rather than direct commands.
Schema Evolution and Versioning

Events typically follow a standardized schema. The evolution of these schemas must be managed carefully to ensure backward compatibility.

Benefits

Real-time Responsiveness

Systems can immediately react to user actions, sensor data, or market events, giving businesses an edge in competitive environments.
Improved Efficiency and Automation

By automating processes in reaction to events, companies reduce manual work, minimize delays, and boost overall efficiency.
Enhanced Scalability

Event-driven architectures can gracefully handle fluctuating workloads, making them well-suited for applications with variable traffic.
Better Integration of Disparate Systems

Data flows smoothly between multiple services or microservices. This integration fosters better cross-team collaboration and more coherent operations.
Facilitation of Digital Transformation

Event-driven orchestration underpins broader digital initiatives by promoting flexibility, adaptability, and modern integration patterns.
Increased Customer Satisfaction

Rapid responses and seamless processes translate into better user experiences, driving customer loyalty and revenue growth.

Challenges and Considerations

Complexity of Distributed Systems

Managing multiple microservices, ensuring reliable message delivery, and handling eventual consistency can be complex.
Data Consistency and Transactions

Transactional guarantees can be difficult to obtain in an asynchronous environment. Ensuring data integrity may require patterns like Saga or two-phase commit (where applicable).
Monitoring and Observability

Tracing event flows across distributed systems demands robust logging, monitoring, and alerting tools. Systems like Jaeger or OpenTelemetry can help provide visibility.
Error Handling and Retry Mechanisms

Transient issues may cause events to fail to process. Handling retries (e.g., dead-letter queues) is crucial without accidentally reprocessing events.
Security and Governance

Proper authentication, authorization, and encryption must be in place for events, especially in multi-tenant or highly regulated environments.
Schema Evolution

Changes to event structures or topics must be carefully orchestrated to avoid breaking downstream consumers.

Real-World Examples

E-Commerce Personalization

Major e-commerce platforms use event-driven orchestration to update recommendations in real-time. Whenever a user views or purchases a product, that event triggers recommendation services to recalculate relevant product suggestions.
Financial Trading Systems

Event-driven architectures power trading platforms to handle market data feeds, rapidly update order books, and execute trades. Real-time event processing helps analysts respond to market fluctuations instantly.
IoT and Smart Devices

In supply chain and logistics, sensors emit events related to inventory, temperature conditions, and location. These events trigger immediate orchestration steps, such as rerouting shipments or sending maintenance alerts.
Ride-Sharing Applications

Services such as dispatch, payments, driver location tracking, and customer notifications rely on event-driven orchestration to stay synchronized. Each trip request, acceptance, or cancellation triggers workflows in separate microservices.

Lessons Learned

Start Small: Adopt a pilot project to familiarize teams with event-based patterns.
Leverage Managed Services: Cloud providers offer managed Kafka services (e.g., Confluent Cloud, AWS MSK, Azure Event Hubs) to reduce infrastructure overhead.
Invest in Observability: Conduct logging and monitoring early to diagnose event-processing pipelines effectively.

Looking Ahead: Future Trends in Orchestration

Serverless Architectures

Platforms like AWS Lambda, Azure Functions, and Google Cloud Functions enable highly elastic, pay-as-you-go event processing pipelines.
Event Mesh and Edge Computing

As IoT grows, events may be processed close to the data source (edge) for real-time analytics. Event mesh patterns will help route events efficiently across regions.
AI-Driven Orchestration

Machine Learning (ML) models can help optimize event flow or predict future events, further automating and refining the orchestration process.
Global-Scale Implementations

With globalization, event-driven architectures will continue to expand across geographies and data centers, necessitating advanced strategies for data replication, multi-region failover, and compliance.

Conclusion

Event-driven process orchestration represents a significant paradigm shift, enabling systems to become more responsive, scalable, and resilient. By detecting and reacting to events in real-time, companies gain operational efficiencies, improve customer satisfaction, and unlock new insights from data streams.

For technical leaders and developers, now is the time to explore event-driven orchestration. Pilot it for small projects, integrate robust event brokers like Kafka, and ensure you have the right monitoring and observability tools. As organizations adopt microservices, IoT, and AI, event-driven architectures will only grow in importance—both as a strategic differentiator and a foundation for future innovation.

Call to Action

Evaluate your current architecture for potential event-driven opportunities.
Experiment with a proof-of-concept using a lightweight event broker or managed Kafka service.
Expand your knowledge by exploring advanced event processing tools like Kafka Streams, KSQL, or serverless platforms.

By embracing event-driven orchestration, you position your organization to tackle modern engineering challenges with agility, scalability, and resilience.

AGI, Are We There Yet?

Ernesto Herrera Salinas — Tue, 24 Dec 2024 19:02:22 +0000

Exploring the Latest Milestones and Ongoing Quest for Human-Level Machine Intelligence

Introduction

Picture this: you’re chatting with a digital assistant who’s as smart as a star engineer, as creative as your favorite novelist, and as empathetic as your closest friend—all rolled into one. That’s Artificial General Intelligence's (AGI) dream: a machine that can understand, learn, and apply knowledge across practically any domain, just like a human. Sounds a bit like a sci-fi blockbuster, right?

But wait—didn’t we hear that OpenAI’s newest model, o3, aced the Abstraction and Reasoning Corpus (ARC) with a score of 87.5%, beating the average human performance of 84%? Should we pop the confetti and declare AGI has arrived? Not so fast. As it turns out, it’s more of a pitstop on the road to AGI than the final destination. Let’s dive into our milestones, the scientific studies shaping the field, and why experts still say, “We’re not there yet.”

What Is AGI, Really?

At its core, AGI means a machine with the broad adaptability of a human mind. It can switch between tasks without a hitch—solving math problems one minute, giving cooking advice the next, and maybe explaining quantum mechanics if you want to go there. This contrasts with narrow AI, which is laser-focused on a single task, like recognizing cats in images or recommending your next Netflix binge.

AGI isn’t just about raw computational power. It’s about understanding context, nuance, and learning strategies that span multiple domains. In other words, if your AI personal trainer can seamlessly teach you chess and then whip up a solid investment portfolio, that’s the hallmark of AGI. While impressive, today’s best AI systems still fall short of this multi-purpose intelligence.

The State of the Art: Spotlight on OpenAI’s o3

In the quest for AGI, OpenAI’s o3 model is making headlines for scoring 87.5% on the ARC test—a benchmark that measures an AI’s ability to handle novel and complex puzzles. This is no small feat, given that the average human score is around 84%. That might be enough to declare a new champion if we were talking about regular IQ tests.

But here’s the catch: ARC primarily tests abstract reasoning skills, not the wide-ranging adaptability required for true AGI. While o3 shows remarkable performance within that specific domain, it remains, in essence, a specialized system. It’s like a chess grandmaster who might still struggle to fry an egg without burning down the kitchen. Experts caution that o3 doesn’t quite qualify as “general” intelligence despite this milestone.

Breaking Down the “Thinking Process”: Chain-of-Thought Research

As interesting as these performance milestones is new research on how large language models (LLMs) reason, particularly focusing on Chain-of-Thought (CoT) techniques. In the paper “LLMs Do Not Think Step-by-step In Implicit Reasoning” by Yijiong Yu, researchers discovered that even massive models (like the 72-billion-parameter Qwen2.5-72B-Instruct) often rely on intuition or direct experience, rather than systematically computing intermediate steps, when reasoning implicitly.

Why does this matter for AGI? Because for truly general intelligence, it’s not enough to stumble onto the correct answer; how the answer is derived is crucial. If the AI takes shortcuts or lacks solid reasoning steps, it’s more prone to making bizarre mistakes. This is one reason experts are experimenting with explicit CoT, which forces models to articulate their intermediate reasoning, leading to more reliable and transparent outputs—albeit at a higher computational cost. Think of it like showing your work in math class: it’s tedious but prevents many errors and guesswork.

Scaling Laws and the Bigger Picture

Another pivotal piece of the AGI puzzle is understanding how to scale up models for better performance. The paper “Scaling Laws for Autoregressive Generative Modeling” by Tom Henighan et al. spotlights some key discoveries:

Consistent Scaling Laws: As the model size and compute power increase, performance follows a predictable power-law relationship—bigger models generally perform better.
Optimal Model Size: There’s a sweet spot between model size and available compute resources, and it’s strikingly similar across domains like images, video, text, and even math problems.
Information-Theoretic Lens: By viewing cross-entropy loss as a combination of data entropy and KL divergence, the study lays a framework for quantifying data complexity and model inefficiency.
Domain-Specific Insights: From image modeling and multimodal tasks to mathematical problem solving and image classification, the benefits of scaling hold up—even if tasks differ wildly.

In short, “scaling laws” assure us that if we keep throwing resources (and money) at bigger models, we’ll continue to see breakthroughs. However, scaling alone doesn’t guarantee general intelligence; it’s just one piece in a very complex puzzle.

Conclusion

So, are we there yet? No—but we’re inching closer. The journey toward AGI is a marathon, and while we’ve seen some thrilling sprints (like OpenAI’s o3 beating human averages on ARC), we still have a ways to go. Researchers are grappling with how AI models reason, how to make them more transparent, and how to ensure that bigger is better when scaling.

AGI might remain on the horizon for now, but these advancements—strong test performances, nuanced research into chain-of-thought, and robust scaling laws—show that the finish line is at least coming into view. In the meantime, we can celebrate each milestone and keep asking, “AGI, are we there yet?” And maybe, just maybe, we’ll get an AI that can whip up a perfect omelet while explaining quantum mechanics before you know it.

Why Use AWS App Runner?

Ernesto Herrera Salinas — Wed, 11 Sep 2024 19:37:50 +0000

Why Use AWS App Runner?

In today’s fast-paced tech environment, developers and DevOps teams are constantly looking for ways to streamline application deployment and reduce the complexity of managing infrastructure. AWS offers a wide array of services to help achieve this, but with so many options, it can be challenging to decide which one is best for your needs. AWS App Runner is a relatively new service designed to simplify the deployment of containerized and web applications. But what makes it stand out, and why should you consider it over other options like Elastic Beanstalk, AWS Lambda, or ECS?

1. Introduction: The Struggle with Infrastructure Complexity

If you're feeling overwhelmed by the plethora of AWS services available for application deployment, you're not alone. Developers and DevOps professionals often face the frustration of managing complex infrastructures, ensuring scalability, and constantly monitoring their services. AWS App Runner aims to eliminate these pain points by offering a managed service that simplifies the deployment process while handling the infrastructure for you.

2. What is AWS App Runner?

AWS App Runner is a fully managed service that allows you to build, deploy, and run containerized applications and web services at scale without worrying about the underlying infrastructure. It supports deploying applications directly from a container registry like ECR or a GitHub repository, making it incredibly easy to set up and manage.

Key Features:

Ease of Use: With App Runner, you don’t need to manage infrastructure. AWS automatically provisions, scales, and monitors your application.
Scaling: Your application automatically scales based on incoming traffic, and it can scale down to zero when there’s no traffic, saving costs.
Deployment Automation: App Runner integrates easily with CI/CD pipelines, enabling automatic deployments when new code or container images are pushed.

3. How AWS App Runner Simplifies App Deployment

App Runner removes much of the complexity associated with traditional infrastructure management. For example, when deploying an application with AWS ECS, you need to configure task definitions, clusters, load balancers, autoscaling policies, and VPC configurations. With App Runner, you can skip all of these steps. The service automatically provisions these resources and manages them for you.

Additionally, App Runner offers:

Built-in Load Balancing: No need to manually configure ALB/NLB for your web applications.
Health Monitoring and Auto Recovery: App Runner ensures your app stays healthy and automatically recovers it if something goes wrong.

For small teams or startups without a dedicated DevOps team, this can be a game changer.

4. AWS App Runner vs. Other AWS Services

When deciding between AWS App Runner and other AWS services like Elastic Beanstalk, AWS Lambda, or ECS, it’s important to understand the trade-offs.

AWS App Runner vs. ECS/Fargate

App Runner: Great for simple, containerized apps with minimal operational overhead. AWS handles the scaling and infrastructure, making it easy for teams with limited AWS expertise.
ECS/Fargate: Offers more control and flexibility, especially for complex, multi-container applications. However, it requires more expertise and effort to set up and manage infrastructure.

AWS App Runner vs. AWS Lambda

App Runner: Ideal for containerized applications and microservices with stateful components.
AWS Lambda: Perfect for event-driven, stateless applications. It’s also serverless but doesn’t handle long-running processes or stateful applications as well as App Runner.

AWS App Runner vs. Elastic Beanstalk

App Runner: More specialized for containerized apps, with simplified scaling and deployment.
Elastic Beanstalk: Better for applications that need a more customized environment, supporting multiple programming languages and frameworks.

5. The Benefits of AWS App Runner

Here are the key benefits of AWS App Runner:

Simplicity: App Runner abstracts the complexity of infrastructure management, enabling developers to focus on coding and building features.
Automatic Scaling: The service scales your application based on demand, and scales to zero when there’s no traffic, reducing unnecessary costs.
Cost-Efficiency: You only pay for the compute and memory your app uses, with no additional costs for load balancers or infrastructure management.
Security: App Runner provides built-in security features, including automatic SSL certificates and integrations with IAM roles.

6. Potential Drawbacks and Limitations

While AWS App Runner is a powerful service, it does come with some limitations:

Limited Control: Compared to ECS or EKS, App Runner gives you less control over the underlying infrastructure. This might not be ideal for applications that require custom networking configurations or advanced scaling policies.
Cost for High-Traffic Applications: App Runner’s simplicity might come with a higher price tag for large-scale applications that require optimized resource usage.

7. Is AWS App Runner Right for You?

If you're a small team, startup, or business looking for a simple, scalable, and low-maintenance solution for deploying containerized applications, AWS App Runner is an excellent choice. It allows you to focus on your application without worrying about the underlying infrastructure.

However, if you have more complex needs, such as custom scaling, service meshes, or advanced VPC configurations, ECS or EKS might be a better fit.

8. Example: Deploying a Simple Containerized Application with AWS App Runner

Let’s walk through a step-by-step example of how to deploy a simple containerized web application using AWS App Runner.

Prerequisites:

You should have an AWS account.
AWS CLI should be installed and configured with appropriate permissions.
Docker installed on your local machine.
A code repository (like GitHub) or a container image stored in AWS Elastic Container Registry (ECR).

Step 1: Create a Simple Web Application

For this example, we will create a simple Node.js web application.

// app.js
const express = require('express');
const app = express();
const port = process.env.PORT || 3000;

app.get('/', (req, res) => {
  res.send('Hello from AWS App Runner!');
});

app.listen(port, () => {
  console.log(`App is running on port ${port}`);
});

Create a Dockerfile to containerize this application:

# Use the official Node.js image from Docker Hub
FROM node:14

# Create and set the working directory
WORKDIR /usr/src/app

# Copy the package.json and install dependencies
COPY package*.json ./
RUN npm install

# Copy the source code
COPY . .

# Expose the application's port
EXPOSE 3000

# Start the application
CMD ["node", "app.js"]

Step 2: Build and Push the Docker Image to ECR

You need to push the Docker image of your application to AWS Elastic Container Registry (ECR). Let’s create a repository and push the image:

Create a repository in ECR:

aws ecr create-repository --repository-name my-app-runner-repo --region us-west-2

Login to ECR:

aws ecr get-login-password --region us-west-2 | docker login --username AWS --password-stdin <AWS_ACCOUNT_ID>.dkr.ecr.us-west-2.amazonaws.com

Build and tag your Docker image:

docker build -t my-app-runner-app .
docker tag my-app-runner-app:latest <AWS_ACCOUNT_ID>.dkr.ecr.us-west-2.amazonaws.com/my-app-runner-repo:latest

Push the image to ECR:

docker push <AWS_ACCOUNT_ID>.dkr.ecr.us-west-2.amazonaws.com/my-app-runner-repo:latest

Step 3: Deploy the Application with AWS App Runner

Now, let’s deploy the container using AWS App Runner.

Create an App Runner service using the AWS CLI:

aws apprunner create-service \
  --service-name my-app-runner-service \
  --source-configuration ImageRepository={ImageIdentifier=<AWS_ACCOUNT_ID>.dkr.ecr.us-west-2.amazonaws.com/my-app-runner-repo:latest,ImageConfiguration={Port=3000},ImageRepositoryType=ECR} \
  --region us-west-2

This command does the following:

Specifies the service name (my-app-runner-service).
Points to the Docker image stored in your ECR repository.
Defines the port on which your application will run (Port=3000).

Verify your deployment:

You can check the status of your App Runner service by running:

aws apprunner describe-service --service-arn <SERVICE_ARN> --region us-west-2

Once the status shows as ACTIVE, AWS App Runner will automatically handle the scaling, load balancing, and monitoring of your containerized app.

Step 4: Access Your Application

Once deployed, AWS App Runner assigns a unique URL to your application. You can find this in the output of the describe-service command or from the AWS Management Console. Simply open the URL in your browser to access your application:

https://<your-app-runner-url>.awsapprunner.com

You should see:

Hello from AWS App Runner!

Step 5: Continuous Deployment (Optional)

To enable continuous deployment, you can integrate App Runner with your CI/CD pipeline or directly from GitHub or GitLab. This ensures that any code changes are automatically built, containerized, and deployed without manual intervention.

To set this up via GitHub, modify the create-service command to use a GitHub repository instead of an ECR image.

aws apprunner create-service \
  --service-name my-app-runner-service \
  --source-configuration CodeRepository={RepositoryUrl=https://github.com/username/my-app-repo,SourceCodeVersion={Type=BRANCH,Value=main},CodeConfiguration={ConfigurationSource=API,CodeConfigurationValues={Runtime=NODEJS_14,Port=3000}}} \
  --region us-west-2

This will automatically deploy the code from the main branch of your GitHub repository.

9. Conclusion

AWS App Runner offers a simplified, scalable solution for deploying containerized applications without the overhead of managing infrastructure. It's ideal for teams that want to focus on development and minimize operational complexity. However, for those needing more control and flexibility, other AWS services like ECS or EKS might be more suitable.

If you’re looking for a managed service that streamlines deployment while offering scalability, security, and ease of use, AWS App Runner is worth exploring.

Book Review: "A Philosophy of Software Design" by John Ousterhout – A Masterclass in Crafting Maintainable Code

Ernesto Herrera Salinas — Wed, 11 Sep 2024 19:33:13 +0000

Book Review: "A Philosophy of Software Design" by John Ousterhout – A Masterclass in Crafting Maintainable Code

John Ousterhout’s A Philosophy of Software Design isn’t just another programming book. It’s a rallying cry for developers to stop chasing quick fixes and start building software that stands the test of time. In this review, we’ll unpack Ousterhout’s groundbreaking ideas—from “deep modules” to slaying the dragon of complexity—and explore why this book might change how you write code forever.

The Big Idea: Software Design Isn’t Optional

Let’s cut to the chase: Ousterhout argues that most software projects fail not because of bad code but because of bad design. The book flips the script on conventional coding wisdom, making a compelling case that great software isn’t just about making it work—it’s about making it last.

Whether you’re a junior developer or a battle-scarred architect, Ousterhout serves up equal parts theory and hard-won practicality. The result? A playbook for building systems that are maintainable, adaptable, and—dare we say—enjoyable to work with.

Deep Modules: Where Simplicity Meets Power

Imagine a magic black box. You feed it simple inputs, and it delivers incredible results—no need to understand the 10,000 gears whirring inside. That’s Ousterhout’s vision of a deep module, the crown jewel of his design philosophy.

✅ Deep modules = Simple interfaces masking complex functionality
❌ Shallow modules = Fancy interfaces doing almost nothing

“Most frameworks get this backwards,” Ousterhout observes. He challenges developers to create modules that act like icebergs—90% hidden complexity beneath a sleek, minimal interface. The payoff? Codebases where changes don’t trigger domino effects of bugs.

The Myth of “Working Code”

Here’s the uncomfortable truth: Your code works. Now what?

Ousterhout drops a truth bomb: “Working code is just the starting line.” The real race is against complexity—the silent killer lurking in every codebase. Through war stories from his Stanford courses and industry experience, he shows how:

Tactical programming (code-first/think-later) creates “landmines for future you”
Strategic programming (design-first) saves 10x the effort down the line
Complexity compounds—bad design decisions today become existential crises tomorrow

Declaring War on Complexity

Ousterhout identifies complexity as public enemy #1. His battle plan?

Kill shallow modules (they’re complexity in disguise)
Embrace encapsulation like your code’s life depends on it (because it does)
Choose abstractions that map to real-world concepts

The book’s pièce de résistance? A ruthless framework for spotting and eliminating complexity hotspots. Pro tip: If your documentation is longer than your code, you’re probably doing it wrong.

From Theory to Trenches

Don’t mistake this for academic fluff. Ousterhout arms readers with:

🔍 Red flag checklists for spotting design rot
🛠️ Refactoring playbooks for legacy code rescues
🧩 API design patterns that prevent “dependency hell”

Real-world examples hit hard, like his takedown of Java’s InputStream class (Spoiler: It’s the poster child for shallow modules gone wrong).

Code in Action: Ousterhout’s Principles at Work

Let’s make this concrete. Here’s how Ousterhout’s design philosophy translates to real code (language-agnostic examples):

1. Deep Module Showdown

Principle: "Modules should be deep—lots of functionality behind simple interfaces."

✅ Deep Module (Good):

# Configuration Loader  
config = ConfigLoader.load("app_settings.yaml")

What’s hidden:

File I/O
YAML/JSON/XML parsing
Environment variable overrides
Validation rules
Caching mechanism

❌ Shallow Module (Bad):

file = open("app_settings.yaml")  
raw_text = file.read()  
yaml_parser = YamlParser(raw_text)  
config_dict = yaml_parser.parse()  
validator = ConfigValidator(config_dict)  
if not validator.is_valid():  
    raise Exception("Invalid config")  
config = ConfigCache.wrap(config_dict)

Ousterhout’s verdict:

"Every exposed interface is a liability. The best modules act like LEGO bricks—simple connectors, powerful possibilities."

2. Complexity Slayer

Principle: "Complexity is incremental—attack it early."

Before (Complex):

class ReportGenerator {  
  generate(userId, startDate, endDate) {  
    // 200 lines mixing:  
    // - Database queries  
    // - CSV/PDF formatting  
    // - Error handling  
    // - Email delivery  
  }  
}

After (Simple):

// Strategic decomposition  
dataClient = new ReportDataClient(DATABASE)  
formatter = new ReportFormatter("PDF")  
deliveryService = new EmailDeliveryService(SMTP_CONFIG)  

data = dataClient.fetch(userId, startDate, endDate)  
formattedReport = formatter.render(data)  
deliveryService.send(userEmail, formattedReport)

Why it works:

Each class becomes:

Testable in isolation
Reusable across projects
Resilient to format/delivery changes

3. Tactical vs. Strategic

Principle: "Strategic programming requires delaying gratification."

Tactical Trap (Short-Term Win):

public class PaymentProcessor {  
  public void process(Order order) {  
    // Hardcoded tax rate  
    double tax = order.total * 0.08; // 🚨 What if tax rules change?  
    // ...  
  }  
}

Strategic Solution (Long-Term Gain):

public class TaxCalculator {  
  private final TaxStrategy strategy;  

  public TaxCalculator(TaxStrategy strategy) {  
    this.strategy = strategy;  
  }  

  public double calculate(Order order) {  
    return strategy.compute(order);  
  }  
}  

// Implementations:  
// - LocalSalesTaxStrategy()  
// - VATStrategy()  
// - ZeroTaxStrategy()

Ousterhout’s wisdom:

"A tactical programmer asks ‘Does it work?’ A strategic programmer asks ‘Will it still work in 2 years?’"

The Pattern Emerges

These examples embody Ousterhout’s core thesis:

Hide the Volcano (Deep Modules)
- Expose simple endpoints, bury complex magma chambers.
Divide and Conquer (Complexity Reduction)
- When a class/file makes you scroll vertically and horizontally, split it.
Code for the Next Developer (Strategic Thinking)
- Pretend your successor knows where you live.

As Ousterhout quips:

"The most dangerous code is the kind that looks like it works."

These patterns turn that danger into deliberate design.

Why This Book Matters Now

In an era of “move fast and break things,” Ousterhout’s philosophy feels radical. He’s not just teaching design—he’s advocating for software craftsmanship.

Teams that adopt these principles report:

🚀 60% fewer “WTF moments” during code reviews
📈 40% faster onboarding for new developers
💥 80% reduction in “fear-driven development”

The Verdict: Essential Reading for Code Survivors

A Philosophy of Software Design isn’t just a book—it’s a mindset shift. Ousterhout delivers a masterclass in:

Building systems that age like fine wine, not milk
Writing code your future self won’t curse
Transforming complexity from foe to ally

Who needs this book?

🔥 Developers tired of rewriting “working” code
🧩 Tech leads battling legacy system hydras
🚀 Engineers aiming for promotions (clean code gets noticed)

Final Thought: This isn’t just about writing better software. It’s about reclaiming the joy of programming. Because when you slay complexity and embrace deep design, coding stops feeling like combat—and starts feeling like creation.

Ready to level up? This book’s your ticket. 🚀

Balancing High-Quality Code with Rapid Development

Ernesto Herrera Salinas — Thu, 11 Jan 2024 01:47:26 +0000

Introduction

In the fast-paced world of software development, the balance between writing high-quality code and meeting rapid development schedules is a constant challenge. As a Software Architect, I've navigated this balance through various projects and want to share insights and strategies that can help fellow developers.

The Importance of High-Quality Code

High-quality code is clean, understandable, maintainable, and testable. It ensures long-term project health, reduces technical debt, and minimizes bugs.

Key Components of High-Quality Code:

Readability: Code should be easy to read and understand.
Modularity: Breaking down code into logical, reusable components.
Testing: Comprehensive tests to validate code against expected outcomes.
Documentation: Well-documented code to aid future development efforts.

The Need for Rapid Development

The tech industry is highly competitive, with constant pressure to release new features and updates. Rapid development helps in:

Staying ahead in the market.
Quickly responding to user needs.
Iterating and improving products efficiently.

Strategies for Balancing Quality and Speed

Balancing these two aspects isn't straightforward and requires a strategic approach:

1. Implement Agile Methodologies

Iterative Development: Break down projects into smaller, manageable iterations.
Continuous Integration/Continuous Deployment (CI/CD): Automate testing and deployment to streamline the development process.

2. Code Reviews and Pair Programming

Peer Reviews: Encourage learning and catching errors early.
Pair Programming: Combines two perspectives, reducing the likelihood of mistakes.

3. Emphasize on Automated Testing

Unit Tests: Ensure individual components perform as expected.
Integration Tests: Check if different parts of the application work together correctly.
End-to-End Tests: Validate the workflow of the application as a whole.

4. Refactoring and Technical Debt Management

Regularly refactor code to maintain quality.
Prioritize technical debt to prevent it from accumulating.

5. Invest in Developer Training

Continuous learning keeps developers updated with best practices.

6. Leverage Advanced Tools and Frameworks

Utilize tools that increase development speed without compromising on quality.

Conclusion

Balancing high-quality code with rapid development is achievable with the right strategies and mindset. It requires a culture that values both quality and efficiency, supported by practices that integrate these values into the development lifecycle. Remember, in software development, speed should never compromise quality.

Understanding Webhooks: The Slack Notification System

Ernesto Herrera Salinas — Wed, 10 Jan 2024 02:20:56 +0000

Introduction

In today's fast-paced digital world, efficient communication and real-time updates are crucial for effective collaboration. This is where the concept of webhooks becomes invaluable, especially in popular collaboration tools like Slack. This blog post aims to demystify webhooks, using Slack’s notification system as a practical example.

What are Webhooks?

At its core, a webhook is a user-defined HTTP callback. It's a mechanism where an app provides other applications with real-time information. A webhook delivers data to other applications as it happens, meaning you get data immediately. Unlike typical APIs where you need to poll for data frequently, webhooks are more efficient for both the provider and consumer.

How Do Webhooks Work?

Imagine a scenario where an application needs to know when a specific event happens in real time. Traditionally, this would involve regularly checking (polling) the service to see if the event has occurred. This process is not only inefficient but can also lead to delays in response. Webhooks solve this by acting like a phone call to notify when specific events occur.

Webhooks in Action: Slack Notifications

Slack, a popular team collaboration tool, uses webhooks extensively for its notification system. Let's break down how this works:

Setting up a Webhook in Slack: Users or administrators can create a 'webhook URL' through Slack's interface. This URL is unique to a Slack channel or workspace.
Event Trigger: An event occurs in an external service that needs to be communicated to Slack users. This could be anything from a new customer ticket, a source code push, or a sales lead.
Sending Data to Slack: The external service makes an HTTP request to the webhook URL provided by Slack. This request includes the data or information about the event.
Slack Processes the Request: Upon receiving the HTTP request, Slack processes the data and converts it into a message format suitable for the channel.
Notification Delivery: The message is then posted to the specified Slack channel, instantly notifying team members.

Advantages of Using Webhooks

Real-time Data Transmission: Webhooks provide immediate information, making them ideal for time-sensitive operations.
Efficiency: They eliminate the need for constant polling, reducing the load on both the server and the network.
Customization and Flexibility: Webhooks can be customized to transmit specific information based on the event, making them versatile.

Conclusion

Webhooks offer a powerful way to integrate different software and services, providing real-time updates and efficient data transmission. Slack's use of webhooks for notifications is a prime example of how this technology can enhance communication and productivity in a collaborative environment.

By embracing webhooks, developers can build more interconnected and responsive applications, fostering a more dynamic and efficient digital ecosystem.

Web Workers API

Ernesto Herrera Salinas — Wed, 10 Jan 2024 02:15:22 +0000

The Web Workers API, as detailed on Mozilla Developer Network (MDN), offers a way for web content to execute scripts in background threads, separate from the main execution thread of a web application. This separation allows for more intensive processing without blocking or slowing down the user interface thread.

Types of Web Workers:

Dedicated Workers: These are utilized by a single script and operate within a DedicatedWorkerGlobalScope.
Shared Workers: These can be used by multiple scripts, even if they run in different windows, IFrames, or workers, as long as they are in the same domain. They operate within a SharedWorkerGlobalScope.
Service Workers: These act like proxy servers between web applications, the browser, and the network. They are designed to enable offline experiences, intercept network requests, update server assets, and provide access to push notifications and background sync APIs.

Key Features and Functions:

Communication with Workers: Data is exchanged between workers and the main thread through a system of messages using the postMessage() method and the onmessage event handler. The data is copied, not shared, between them.
Error Handling: Workers have an onerror event handler for runtime errors, providing details like the error message, filename, and line number where the error occurred.
Subworkers: Workers can spawn new workers (sub-workers), as long as these sub-workers are hosted within the same origin as the parent page.
Script Importing: Workers use importScripts() to import scripts, which are executed synchronously and in the order they are passed.

Global Contexts and Functions:

Workers run in a different global context than the window and have access to functions like atob(), btoa(), fetch(), and setInterval() through their own WorkerGlobalScope-derived contexts.
However, workers cannot directly manipulate the DOM or use some default methods and properties of the Window object.

Supported Web APIs in Workers:
Workers have access to various Web APIs, including but not limited to the Canvas API, Fetch API, File API, IndexedDB API, and WebSockets API. It's important to check browser compatibility for specific worker types and functions.

Constructing and Using Workers:

To create a dedicated worker, the Worker("path/to/worker/script") constructor is used. For a shared worker, SharedWorker("scriptURL") is employed.
These workers handle messages and errors through event handlers and methods like postMessage() and terminate(). The shared worker also involves a port object for communication.

Shared Worker Specifics:
Shared workers differ from dedicated workers in their ability to be accessed from several browsing contexts. They require the use of a MessagePort object for communication, and the worker's lifetime is tied to its global scope's owner set.

Thread Safety and Security:

Web workers spawn real OS-level threads but are designed to avoid typical concurrency issues as they have controlled communication points and no access to non-threadsafe components or the DOM.
Workers have their own execution context and are generally not governed by the content security policy of the document that created them.

The Web Workers API is a powerful tool for web developers, enabling more efficient and responsive web applications by offloading tasks to background threads. For more detailed information and examples, you can refer directly to the MDN Web Docs on Using Web Workers, Web Workers API, Worker, and SharedWorker.

Forem: Ernesto Herrera Salinas

Empower Your Workflow with Aliases: Save Time by Being Lazy

🚀 Why Use Aliases?

📦 Highlighted Alias Categories

🔧 Git Workflow Aliases

🐳 Docker / Compose Aliases

📦 NPM Aliases

💻 System Aliases

🧠 Productivity Best Practices

✅ Sample Alias Setup (excerpt from .bashrc or .zshrc)

🧰 Tips to Customize Further

🔚 Wrap-Up

Data-Driven Testing in Jest (Parameterized Tests)

What is Data-Driven Testing?

Benefits of Data-Driven Testing

Implementing Data-Driven Tests in Jest

Using External Data (JSON) for Test Cases in Jest

Maintainability, Scalability, and Readability Improvements

Best Practices for Data-Driven Tests in Jest

Example: Armstrong Number Tests with Data-Driven Approach

Conclusion

The Lazy Engineer’s Guide to Automating Timesheets: Part 1

The Problem: Timesheets Are the Worst

The Tools of the Trade

Step 1: Pulling JIRA Tickets

The Script

How It Works

Step 2: Fetching Git Commits

The Script

How It Works

Step 3: Tackling Slack Messages

The Script

How It Works

Step 4: Capturing Outlook Meetings

The Script

How It Works

What’s Next?

Private npm Repositories

Private npm Repositories: Taking Control of Your Code

Why Private npm Repositories Matter

Choosing Your Registry: A Craftsman’s Guide

1. Self-Hosted Solutions

2. Git-Integrated Registries

3. npm Enterprise

Crafting Your Registry: Hands-On

1. Verdaccio: The Artisan’s Choice

2. GitHub Packages: The Collaborator’s Haven

3. GitLab Packages: The CI/CD Power Play

Enterprise Forges: Nexus and Artifactory

Principles for the Prudent Craftsman

Conclusion: Own Your Artifacts

A Technical Introduction to LocalStack

What is LocalStack?

Installation

Prerequisites

Installing LocalStack

Using LocalStack

Configuring the AWS CLI

Using LocalStack with boto3 (Python)

Example: DynamoDB

Example: Lambda

Tips and Best Practices

Conclusion

Navigating the World of Event-Driven Process Orchestration for Technical Leaders

Event-Driven Process Orchestration

Introduction

Understanding Event-Driven Process Orchestration

A Basic Python Example

A Distributed Example with Apache Kafka

Comparison with Traditional Orchestration Methods

Key Principles and Benefits

Principles

Benefits

Challenges and Considerations

Real-World Examples

Looking Ahead: Future Trends in Orchestration

Conclusion

AGI, Are We There Yet?

Introduction

What Is AGI, Really?

✅ Sample Alias Setup (excerpt from `.bashrc` or `.zshrc`)

Using LocalStack with `boto3` (Python)