Skip to content

Forem

# dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Eliana Lam

Nov 29 '25

Building Streaming Iceberg Tables for Real-Time Logistics Analytics

#analytics #dataengineering #architecture #opensource

4 min read

Cover image for Building a Scalable Community Health Worker Analytics Platform: My Journey with dbt and Snowflake

Amos Augo

Nov 26 '25

Building a Scalable Community Health Worker Analytics Platform: My Journey with dbt and Snowflake

#dbt #snowflake #dataengineering

4 min read

Cover image for The Great Table Format Debate: A Deep Dive into Apache Iceberg, Delta Lake, and Apache Hudi

Data Tech Bridge

Dec 31 '25

The Great Table Format Debate: A Deep Dive into Apache Iceberg, Delta Lake, and Apache Hudi

#database #dataengineering #iceberg #apachehudi

18 min read

Cover image for Amazon Kinesis vs Amazon MSK: The Complete Guide for Stream Processing on AWS

Data Tech Bridge

Dec 30 '25

Amazon Kinesis vs Amazon MSK: The Complete Guide for Stream Processing on AWS

#architecture #aws #dataengineering

29 min read

Cover image for Day 8: Accelerating Spark Joins - Broadcast, Shuffle Optimization & Skew Handling

Sandeep

Dec 9 '25

Day 8: Accelerating Spark Joins - Broadcast, Shuffle Optimization & Skew Handling

#dataengineering #python #spark #bigdata

2 min read

Cover image for Mastering Serverless Data Pipelines: AWS Step Functions Best Practices for 2026

Jubin Soni

Dec 30 '25

Mastering Serverless Data Pipelines: AWS Step Functions Best Practices for 2026

#aws #serverless #stepfunctions #dataengineering

5 min read

Cover image for 2025 Year in Review: Apache Iceberg, Polaris, Parquet, and Arrow

Dec 29 '25

2025 Year in Review: Apache Iceberg, Polaris, Parquet, and Arrow

#architecture #bigdata #opensource #dataengineering

6 min read

Cover image for A Stranger In a New Town: CsvPath metadata fields

Nov 25 '25

A Stranger In a New Town: CsvPath metadata fields

#metadata #dataengineering #csv #datascience

6 min read

Dec 17 '25

Interesting links - November 2025

#data #dataengineering #kafka #flink

19 min read

Dec 29 '25

💀 RIP Copy-Paste: Google NotebookLM Just Killed Manual Data Entry

#ai #productivity #google #dataengineering

3 min read

Dec 29 '25

Unified Data Fabric: Serverless Spark on ROSA Integrating with AWS Glue Catalog

#serverless #kubernetes #aws #dataengineering

39 min read

Nov 25 '25

dupl

#sql #dataengineering #backend #database

1 min read

Cover image for Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Nov 18–24, 2025)

Nov 24 '25

Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Nov 18–24, 2025)

#data #dataengineering #opensource #resources

5 min read

Benjamin Ibrulj

Dec 29 '25

Building a Realistic Banking Dummy Data Generator with Bad-Data Simulation

#dataengineering #python #opensource #sql

1 min read

maghsood esmaeili

Dec 16 '25

How to Sync Data from an Oracle Table to Elasticsearch using Kafka Connect

#database #kubernetes #dataengineering #tutorial

5 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.