Forem

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Data Engineering 101: Understanding Databases, Storage, and Security
Cover image for Data Engineering 101: Understanding Databases, Storage, and Security

Data Engineering 101: Understanding Databases, Storage, and Security

5
Comments
6 min read
10 Best Platforms to Learn Data Engineering in 2026
Cover image for 10 Best Platforms to Learn Data Engineering in 2026

10 Best Platforms to Learn Data Engineering in 2026

Comments
4 min read
Simulating An Event-Driven Python Shopping App with Kafka on AWS For Real-Time Processing.
Cover image for Simulating An Event-Driven Python Shopping App with Kafka on AWS For Real-Time Processing.

Simulating An Event-Driven Python Shopping App with Kafka on AWS For Real-Time Processing.

5
Comments 2
8 min read
Real-Time Data Streaming Platform: How We Built a Self-Hosted Platform with 90% Cost Reduction vs AWS Managed Services

Real-Time Data Streaming Platform: How We Built a Self-Hosted Platform with 90% Cost Reduction vs AWS Managed Services

1
Comments
6 min read
Building a Real-Time Data Platform with Kubernetes (Kind) - A Complete Local Setup Guide

Building a Real-Time Data Platform with Kubernetes (Kind) - A Complete Local Setup Guide

2
Comments
8 min read
Building a Task Manager with Apache NiFi: From Custom Scheduler to Distributed Workflows

Building a Task Manager with Apache NiFi: From Custom Scheduler to Distributed Workflows

Comments
9 min read
Azure Data Factory (ADF) - A Beginner's Guide to Cloud Data Integration
Cover image for Azure Data Factory (ADF) - A Beginner's Guide to Cloud Data Integration

Azure Data Factory (ADF) - A Beginner's Guide to Cloud Data Integration

2
Comments
4 min read
Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Apache Kafka Deep Dive: Core Concepts, Data Engineering Applications, and Real-World Production Practices

Comments
4 min read
Another Data Nerd Guide to re:Invent 2025
Cover image for Another Data Nerd Guide to re:Invent 2025

Another Data Nerd Guide to re:Invent 2025

2
Comments
3 min read
“𝗘𝗧𝗟 𝗶𝘀 𝗘𝘃𝗼𝗹𝘃𝗶𝗻𝗴 — 𝗔𝗿𝗲 𝗬𝗼𝘂?”

“𝗘𝗧𝗟 𝗶𝘀 𝗘𝘃𝗼𝗹𝘃𝗶𝗻𝗴 — 𝗔𝗿𝗲 𝗬𝗼𝘂?”

5
Comments
1 min read
From OLTP to OLAP: Streaming Databases into MotherDuck with Estuary
Cover image for From OLTP to OLAP: Streaming Databases into MotherDuck with Estuary

From OLTP to OLAP: Streaming Databases into MotherDuck with Estuary

1
Comments
7 min read
Apache Kafka & Amazon MSK: The Beating Heart of Real-Time Data
Cover image for Apache Kafka & Amazon MSK: The Beating Heart of Real-Time Data

Apache Kafka & Amazon MSK: The Beating Heart of Real-Time Data

1
Comments
4 min read
Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide
Cover image for Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide

Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide

Comments
3 min read
Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide
Cover image for Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide

Migrating Oracle Fusion Cloud Data to Azure Fabric: A Practical Guide

Comments
3 min read
Handling Distributed Transactions with Orchestrator Pattern (Withdrawal & Deposit Example)

Handling Distributed Transactions with Orchestrator Pattern (Withdrawal & Deposit Example)

Comments
2 min read
Streaming Data Using Apache Kafka
Cover image for Streaming Data Using Apache Kafka

Streaming Data Using Apache Kafka

1
Comments
2 min read
SUPCON Uses SeaTunnel to Build an Efficient Data Collection Framework, Achieving 0 Failures in Core Data Synchronization Tasks!

SUPCON Uses SeaTunnel to Build an Efficient Data Collection Framework, Achieving 0 Failures in Core Data Synchronization Tasks!

Comments
14 min read
Apache Doris 4.0: One Engine for Analytics, Full-Text Search, and Vector Search

Apache Doris 4.0: One Engine for Analytics, Full-Text Search, and Vector Search

5
Comments
7 min read
Building a Streaming Data Pipeline with Kafka and Spark: Real-Time Analytics Implementation Guide

Building a Streaming Data Pipeline with Kafka and Spark: Real-Time Analytics Implementation Guide

1
Comments
10 min read
🚀 How I Learned That Thinking “Small” Can Save Hours of SQL Headaches
Cover image for 🚀 How I Learned That Thinking “Small” Can Save Hours of SQL Headaches

🚀 How I Learned That Thinking “Small” Can Save Hours of SQL Headaches

1
Comments
2 min read
Elusion v8.0.0 is the best END-TO-END Data Engineering library writen in RUST
Cover image for Elusion v8.0.0 is the best END-TO-END Data Engineering library writen in RUST

Elusion v8.0.0 is the best END-TO-END Data Engineering library writen in RUST

Comments
2 min read
Hands-On ACID in PostgreSQL : Part-1

Hands-On ACID in PostgreSQL : Part-1

1
Comments
3 min read
GETTING TO KNOW:STAR AND SNOWFLAKE SCHEMA.
Cover image for GETTING TO KNOW:STAR AND SNOWFLAKE SCHEMA.

GETTING TO KNOW:STAR AND SNOWFLAKE SCHEMA.

1
Comments
3 min read
Hackeando o Data Engineering: Os Padrões que Todo Engenheiro Precisa Conhecer
Cover image for Hackeando o Data Engineering: Os Padrões que Todo Engenheiro Precisa Conhecer

Hackeando o Data Engineering: Os Padrões que Todo Engenheiro Precisa Conhecer

Comments
1 min read
YouTube Analytics Pipeline Using Delta Lake and PySpark.
Cover image for YouTube Analytics Pipeline Using Delta Lake and PySpark.

YouTube Analytics Pipeline Using Delta Lake and PySpark.

Comments
8 min read
loading...