Forem: cristopher Njuguna

Ridge Regression vs Lasso Regression

cristopher Njuguna — Sun, 25 Jan 2026 13:40:09 +0000

House Price Prediction Context

Linear regression is a common method for predicting continuous values such as house prices.
While Ordinary Least Squares (OLS) is the standard approach for fitting linear models, it often struggles with real-world data that contains many features, correlated variables, or noise. Regularization techniques such as Ridge and Lasso regression improve model performance by controlling complexity and reducing overfitting.

Ordinary Least Squares (OLS) What is OLS and what objective does it minimize?

Ordinary Least Squares (OLS) is a method used to estimate the coefficients of a linear regression model by minimizing the sum of squared residuals—the squared differences between actual and predicted values.

Loss=i=1∑n(yi−y^i)2

Here, yi represents the actual house price and y^i represents the predicted price.

Why can OLS lead to overfitting in real-world datasets?

OLS can lead to overfitting when the model becomes too complex, especially in datasets with many features or when features are highly correlated. In such cases, the model captures noise rather than the underlying data patterns, resulting in poor generalization to unseen data.

Concept of Regularization What problem does regularization solve in linear regression?

Regularization reduces overfitting and model instability by discouraging large coefficient values. It helps the model generalize better to new data, especially when features are numerous or correlated.

Why is adding a penalty term helpful?

Adding a penalty term helps to constrain the magnitude of the coefficients. This keeps them small and reduces the model complexity, leading to better generalization to new data. Regularization techniques like Lasso and Ridge help in stabilizing the coefficient estimates.

Regularized Loss = OLS Loss + 𝜆 × Penalty

The parameter λ controls how strong the penalty is.

Ridge Regression (L2 Regularization) Ridge regression loss function Loss=∑(yi−y^i)2+λ∑βj2

How the L2 penalty affects coefficients

The L2 penalty shrinks all coefficients toward zero by penalizing their squared values. Larger coefficients receive stronger penalties, which stabilizes the model—especially when features are correlated.

Why Ridge regression does not perform feature selection

Ridge regression does not set coefficients exactly to zero. Instead, it reduces their magnitude. As a result, all features remain in the model, meaning Ridge does not perform feature selection.

Lasso Regression (L1 Regularization) Lasso regression loss function Loss=∑(yi−y^i)2+λ∑∣βj∣

How the L1 penalty differs from L2

The L1 penalty uses the absolute value of coefficients rather than squared values. This creates a stronger pull toward zero compared to L2.

Why Lasso can set coefficients exactly to zero

Lasso regression can set some coefficients exactly to zero because the L1 penalty creates a situation where the optimization problem can yield coefficients that do not contribute at all to the prediction, effectively excluding those features from the model.

Comparison: Ridge vs. Lasso Regression

Feature Selection:
Ridge: Retains all features, making it less suitable for dimensionality reduction.
Lasso: Performs feature selection by driving some coefficients to zero, making it more suitable for simpler models.

Coefficient Behavior:
Ridge: Coefficients are shrunk but not zeroed out, which means all variables remain in the model.
Lasso: Coefficients can be zeroed out, allowing for the model to focus on a subset of features.

Model Interpretability:
Ridge: Models are more complex and can be harder to interpret due to all features being included.
Lasso: Models are simpler and more interpretable since irrelevant features can be excluded entirely.

Application: House Price Prediction

Assume features include:

House size

Number of bedrooms

Distance to city

Number of nearby schools

Several noisy or weak features

a. Which regression technique would you choose if all features are believed to contribute to the price? Explain why.

If all features are believed to contribute to house prices, I would choose Ridge Regression. Since Ridge retains all features and simply shrinks their coefficients, it would provide a robust prediction without excluding any potentially valuable information.

b. Which technique would you choose if only a few features are truly important and others add noise? Explain why.

I would opt for Lasso Regression. Lasso's ability to reduce some coefficients to zero allows us to eliminate the irrelevant features, thus simplifying the model and improving its interpretability.

Model Evaluation Detecting Overfitting Using Training and Testing Data

To detect overfitting, the dataset is divided into two parts: a training set and a testing set. The model is trained using the training data and then evaluated on the testing data, which the model has not seen before. If the model shows very low error on the training data but significantly higher error on the testing data, this indicates overfitting. In contrast, similar performance on both datasets suggests that the model generalizes well. Evaluation metrics such as Mean Squared Error (MSE) or R² can be used to compare performance across the two datasets.

Role of Residuals in Evaluating Model Performance

Residuals are the differences between the actual house prices and the prices predicted by the model. Analyzing residuals helps determine how well the model fits the data. A good model produces residuals that are randomly scattered around zero, indicating that the model has captured the underlying relationship. If residuals show patterns, trends, or large outliers, this suggests issues such as missing important variables, non-linear relationships, or the presence of noise that the model has not handled well.

OLS is effective for simple datasets but struggles with noise and high dimensionality. Ridge regression improves stability by shrinking coefficients, while Lasso regression improves simplicity by removing irrelevant features. In house price prediction, Ridge is ideal when all features matter, while Lasso is better when only a few key predictors drive prices.

Introduction to Classes in Object-Oriented Programming

cristopher Njuguna — Thu, 11 Dec 2025 18:37:31 +0000

Understanding Classes in Object-Oriented Programming (OOP)

Object-Oriented Programming (OOP) is a programming paradigm that helps developers structure code in a more organized and modular way. One of its core features is the class, a blueprint used to create objects that represent real-world entities. Understanding how classes work is essential for building scalable and maintainable software.

What Is a Class?

A class is a user-defined blueprint that describes the structure and behavior of an object.
It defines:

Attributes → the data or characteristics of the object
Methods → the actions or behaviors the object can perform

You can think of a class like a template for making multiple identical objects, each containing its own data but sharing the same structure.

Why Are Classes Useful?

Classes help solve programming problems in a structured way by:

Modeling real-world entities
For example, a bank account, a student, or a car.
Grouping related data and functions together
This makes the program cleaner and easier to understand.
Reusing code
Once a class is created, you can make as many objects from it as needed.
Encapsulating data
Classes keep related operations together, making it harder to accidentally misuse data.

Attributes and Methods

Attributes

Attributes are variables contained inside a class.
They store information about the object.

Example attributes in a bank account:

account_number
owner
balance

Methods

Methods are functions defined inside a class.
They define the behaviors that the objects created from the class can perform.

Example methods:

deposit()
withdraw()
check_balance()

Example: A Simple BankAccount Class

Below is an easy implementation of a BankAccount class in Python:

class BankAccount:
    def __init__(self, account_number, owner, balance=0):
        self.account_number = account_number
        self.owner = owner
        self.balance = balance

    def deposit(self, amount):
        self.balance += amount
        return f"Deposited {amount}. New balance is {self.balance}."

    def withdraw(self, amount):
        if amount > self.balance:
            return "Insufficient funds."
        self.balance -= amount
        return f"Withdrew {amount}. New balance is {self.balance}."

    def check_balance(self):
        return f"Account Balance: {self.balance}"

How the Class Works

1. The `init` Method

This special method initializes a new object.
It sets the starting values for the attributes:

self.account_number = account_number
self.owner = owner
self.balance = balance

Every object created from this class will store these values independently.

2. Operations Through Methods

Deposit

Adds money to the balance.

Withdraw

Checks if the account has enough funds before withdrawing.

Check Balance

Returns the current balance.

Creating an Object From the Class

Here is how you can create a BankAccount object and perform actions:

# Create a new bank account
account1 = BankAccount("ACC123", "John Doe", 500)

# Deposit money
print(account1.deposit(300))

# Withdraw money
print(account1.withdraw(200))

# Check balance
print(account1.check_balance())

Output example

Deposited 300. New balance is 800.
Withdrew 200. New balance is 600.
Account Balance: 600

Each method changes or accesses the data stored in the object.

Classes allow you to:

Structure code based on real-world concepts
Keep data and behavior packaged together
Build programs that are easier to expand and maintain

In real applications, the BankAccount class could be expanded with security checks, transaction histories, or links to user profiles. This demonstrates how classes form the foundation of more complex systems.

Connecting Power BI to PostgreSQL (Aiven & Localhost): A Simple Step-by-Step Guide

cristopher Njuguna — Sat, 22 Nov 2025 20:54:01 +0000

Power BI connects smoothly to both cloud-hosted and local PostgreSQL databases once you know where to enter the right details. This guide explains the simplest possible process for connecting Power BI to:

PostgreSQL hosted on Aiven, and PostgreSQL running locally on your machine

Connecting Power BI to PostgreSQL on Aiven Step 1: Open Power BI and Start a Blank Report

Launch Power BI Desktop.
Click Blank Report → Get Data → More → search for PostgreSQL Database.

Step 2: Enter Aiven Connection Details

In Aiven:

Open the Aiven Console

Click Services

Select your PostgreSQL service

View your connection information (host, port, database)

Use these values in Power BI:

Server: host:port
Example: pg-12345.aivencloud.com:28643

Database: your Aiven PostgreSQL database name

Data Connectivity Mode: Import

Step 3: Enter Aiven Credentials

Power BI now prompts for authentication.

Use:

Username: Aiven database user

Password: the associated password

Step 4: Choose Tables in the Navigator

Power BI opens the Navigator panel.
Select the tables you want from your Aiven PostgreSQL database and click Load.

The data imports and appears in the Power BI report interface.

Connecting Power BI to PostgreSQL on Localhost

The process is almost identical — only the connection details change.

Step 1: Open Power BI → Blank Report → Get Data → PostgreSQL

Step 2: Enter Localhost Database Details

Fill the fields with your local PostgreSQL configuration:

Server: localhost:5432

Database: usually postgres or your custom database name

Data Connectivity Mode: Import

Step 3: Enter Local Credentials

Typical local PostgreSQL credentials:

Username: postgres

Password: your local password

Step 4: Select Tables → Load

Choose your tables in the Navigator → click Load → Power BI imports the data.

Conclusion

Connecting Power BI to PostgreSQL is straightforward once you know which values to enter. Aiven uses a cloud host and port, while your local setup uses localhost. After that, both connections behave exactly the same inside Power BI.

Star vs Snowflake Schema: key difference, pros & cons, and when to use each.

cristopher Njuguna — Mon, 13 Oct 2025 05:55:07 +0000

The star schema and snowflake schema are fundamental dimensional modeling techniques used in data warehousing to optimize data retrieval for analytical queries. Both models comprise a central fact table and associated dimension tables, but they differ primarily in the normalization level of their dimension tables.

A star schema keeps dimensions in one big, denormalised table; a snowflake breaks that table into several normalised, related tables—trading query speed for storage efficiency and stricter data integrity.

*The star schema *
The star schema is characterized by a central fact table directly connected to multiple denormalized dimension tables.

This structure resembles a star, with the fact table at the center and dimension tables radiating outwards. In a star schema, each dimension is represented by a single table, which typically contains both descriptive attributes and a primary key for joining with the fact table.

*The advantages of the star schema; *
Its simplicity, which makes it easier to understand and implement.

Queries are often faster because they involve fewer joins compared to highly normalized schemas, typically joining the fact table with only one level of dimension tables.

This streamlined joining process enhances query performance for online analytical processing (OLAP) applications.

The straightforward design also makes it more intuitive for business users to navigate and analyze data. Furthermore, star schemas simplify the development of OLAP cubes and reporting tools.

The primary disadvantage of the star schema is data redundancy due to denormalization.

This redundancy can lead to increased storage requirements and potential data integrity issues if updates are not meticulously managed.

The lack of normalization can also make it more challenging to handle complex hierarchical relationships within dimensions.

*The snowflake schema *
The snowflake schema is an extension of the star schema where dimension tables are normalized into multiple related tables. This means that a dimension table can have sub-dimension tables, forming a hierarchical structure. For example, a Product dimension might be normalized into a Product table and a separate Category table, with the Product table linking to the Category table. This normalization reduces data redundancy by storing commonly repeated attributes in separate tables.

The main advantage of the snowflake schema is its reduced data redundancy and improved data integrity due to normalization.

This makes it more storage-efficient, especially for large datasets with many repeating attributes within dimensions.

It also simplifies data maintenance by ensuring that changes to a descriptive attribute only need to be made in one place.

The snowflake schema is particularly suitable for handling complex hierarchical dimensions, as the structure directly supports these relationships.

The snowflake schema's primary disadvantage is increased query complexity and potentially slower query performance.

Because dimensions are normalized, queries often require more joins across multiple tables to retrieve the necessary data, which can increase execution time.

This complexity can also make it more difficult for business users to understand and interact with the data model.

The design and maintenance of snowflake schemas can be more intricate compared to star schemas.

The star schema is appropriate when:

Query performance is a top priority: Its denormalized structure and fewer joins typically lead to faster query execution, which is crucial for real-time analytical reporting and decision-making systems.
Simplicity and ease of understanding are valued: The straightforward design makes it easier for developers to build and for business users to query and interpret the data.
Dimensions are relatively stable and not deeply hierarchical: If dimension attributes do not change frequently and do not have complex multi-level hierarchies, the benefits of normalization in a snowflake schema are less pronounced.
Storage costs are not a primary concern: The potential increase in storage due to redundancy is acceptable if query speed is paramount.

*The snowflake schema is appropriate when: *

Data integrity and storage efficiency are critical: Normalization minimizes redundancy and ensures data consistency, which is beneficial for very large datasets and environments with strict data quality requirements.
Dimensions have complex, multi-level hierarchies: The snowflake structure naturally accommodates these hierarchies by breaking down dimensions into sub-dimensions, offering a more organized and maintainable model for intricate relationships.
Data changes frequently within dimension attributes: Normalization reduces the impact of updates by isolating changes to specific, smaller tables, thereby simplifying maintenance.
Ad-hoc queries requiring detailed dimension exploration are common: While more complex, the normalized structure can support intricate analytical queries that delve deeply into dimensional attributes.

Overall, the choice between star and snowflake schema involves a trade-off between query performance, data redundancy, storage efficiency, and design complexity.

References

Iqbal, A. et al. 2020. Comparative study of star & snowflake schemas.

Levene, M. & Loizou, G. 2003. Why is the snowflake schema a good data-warehouse design?

Wijaya, A. et al. 2017. Community complaints snowflake implementation.