Forem

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Introduction to Data Engineering Concepts |16| Data Lakehouse Architecture Explained

Introduction to Data Engineering Concepts |16| Data Lakehouse Architecture Explained

Comments
4 min read
Introduction to Data Engineering Concepts |15| Cloud Data Platforms and the Modern Stack

Introduction to Data Engineering Concepts |15| Cloud Data Platforms and the Modern Stack

Comments
4 min read
Introduction to Data Engineering Concepts |12| Scheduling and Workflow Orchestration

Introduction to Data Engineering Concepts |12| Scheduling and Workflow Orchestration

Comments
4 min read
Introduction to Data Engineering Concepts |8| Data Lakes Explained

Introduction to Data Engineering Concepts |8| Data Lakes Explained

Comments
4 min read
Introduction to Data Engineering Concepts |10| Data Quality and Validation

Introduction to Data Engineering Concepts |10| Data Quality and Validation

Comments
4 min read
Introduction to Data Engineering Concepts |2| Understanding Data Sources and Ingestion

Introduction to Data Engineering Concepts |2| Understanding Data Sources and Ingestion

1
Comments
4 min read
When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)

When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)

17
Comments 2
5 min read
Big Data Processing - Case Study 4 (Hadoop) 02:36

Big Data Processing - Case Study 4 (Hadoop)

1
Comments
1 min read
Building and Deploying My First Python ETL Package to PyPI

Building and Deploying My First Python ETL Package to PyPI

1
Comments 1
3 min read
Weather Monitoring System Using IoT

Weather Monitoring System Using IoT

1
Comments
2 min read
The Complete Guide to Setting Up Postgresql on Windows 11 and WSL2

The Complete Guide to Setting Up Postgresql on Windows 11 and WSL2

7
Comments 4
6 min read
Big Data Processing - Case Study 3 (Databricks) 01:53

Big Data Processing - Case Study 3 (Databricks)

Comments 2
1 min read
How I Automated Crypto Price Tracking with Apache Airflow & CoinGecko

How I Automated Crypto Price Tracking with Apache Airflow & CoinGecko

4
Comments 3
2 min read
Personal Picks: Data Product News (March 19, 2025)

Personal Picks: Data Product News (March 19, 2025)

Comments
5 min read
Big Data Processing - Case Study 3 (Spark) 02:35

Big Data Processing - Case Study 3 (Spark)

Comments
1 min read
Data Warehouses and Data Lakes: Understanding Modern Data Storage Paradigms 📦

Data Warehouses and Data Lakes: Understanding Modern Data Storage Paradigms 📦

Comments
2 min read
Python 101 For Data Engineering

Python 101 For Data Engineering

Comments 2
4 min read
Introduction to Presto: Open Source SQL Query Engine that's changing Big Data Analytics

Introduction to Presto: Open Source SQL Query Engine that's changing Big Data Analytics

Comments
2 min read
Get the Records after and before the Searched One — From SQL to SPL #18

Get the Records after and before the Searched One — From SQL to SPL #18

8
Comments 2
2 min read
NoSQL Fighters Arena: The Battle of Data Titans

NoSQL Fighters Arena: The Battle of Data Titans

Comments
3 min read
Kafka Consumers Explained: Pull, Offsets, and Parallelism

Kafka Consumers Explained: Pull, Offsets, and Parallelism

1
Comments
4 min read
Why Pi-Shaped Teams Matter in This AI Era

Why Pi-Shaped Teams Matter in This AI Era

Comments
5 min read
How to Integrate AWS Lambda With TimescaleDB for Scalable IoT Data Pipelines

How to Integrate AWS Lambda With TimescaleDB for Scalable IoT Data Pipelines

2
Comments
6 min read
The 5 Most Common ETL Patterns — When and Why to Use Them

The 5 Most Common ETL Patterns — When and Why to Use Them

Comments
3 min read
Data architecture stack

Data architecture stack

Comments
1 min read
loading...