Skip to content

Forem

# dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Cover image for Building a Modern Data Platform to Track Kenya’s Food Prices — A Data Engineering Case Study

Nov 13 '25

Building a Modern Data Platform to Track Kenya’s Food Prices — A Data Engineering Case Study

#spark #pyspark #grafana #dataengineering

5 min read

Cover image for Part 1: Database Concepts & Architecture

Data Tech Bridge

Dec 18 '25

Part 1: Database Concepts & Architecture

#architecture #database #dataengineering

14 min read

Apache SeaTunnel

Nov 14 '25

Final Project Report 1: Schema Evolution Support on Apache SeaTunnel Flink Engine

#opensource #apacheseatunnel #bigdata #dataengineering

4 min read

Cover image for I Built an ETL Pipeline That Actually Thinks & And Cut Token Costs by 52% (And Here's What I Learned)

Seenivasa Ramadurai

Dec 17 '25

I Built an ETL Pipeline That Actually Thinks & And Cut Token Costs by 52% (And Here's What I Learned)

#ai #dataengineering #performance #llm

17 min read

Cover image for Firehose and Iceberg Tables

Evan

Dec 17 '25

Firehose and Iceberg Tables

#programming #dataengineering #aws #awsdatalake

4 min read

Judy

Dec 17 '25

Beyond SQL: Solving Data Warehouse Performance Bottlenecks with Smart Algorithms, Not Just Bigger Clusters

#algorithms #database #dataengineering #performance

13 min read

David Aronchick

Nov 12 '25

From Pandas to Upstream Control: The Evolution PyData Needs Next

#dataengineering #python #distributedsystems #machinelearning

6 min read

Cover image for Statistics Day 2: Correlation Isn’t Causation — Here’s Why It Matters!

Nov 13 '25

Statistics Day 2: Correlation Isn’t Causation — Here’s Why It Matters!

#statistics #datascience #machinelearning #dataengineering

4 min read

rajeevrajeshuni

Dec 15 '25

Unpacking the Google File System Paper: A Simple Breakdown

#distributedsystems #dataengineering #systemdesign #pwl

3 min read

Cover image for Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Dec 9th - Dec15th, 2025)

Dec 15 '25

Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Dec 9th - Dec15th, 2025)

#news #dataengineering #opensource

7 min read

Aimé Bangirahe

Nov 10 '25

Kafka consumer lag—Measure and reduce

#devops #performance #dataengineering #monitoring

5 min read

Cover image for Understanding Kafka Consumer Lag: Causes, Risks, and How to Fix It

Nov 10 '25

Understanding Kafka Consumer Lag: Causes, Risks, and How to Fix It

#systemdesign #performance #dataengineering #monitoring

3 min read

Dec 15 '25

Building a dbt-UI I Wish Existed

#dbt #data #dataengineering #dataanalytics

3 min read

Cover image for Building a Real-Time Crypto Data Pipeline with Debezium CDC

Nov 10 '25

Building a Real-Time Crypto Data Pipeline with Debezium CDC

#python #kafka #cdc #dataengineering

5 min read

Cover image for Undestanding Kafka Lag, Why It Happens and How To Fix It.

John Kioko

Nov 10 '25

Undestanding Kafka Lag, Why It Happens and How To Fix It.

#kafka #programming #datascience #dataengineering

4 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.