Skip to content

Forem

# dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Cover image for Core Microsoft Fabric Concepts

Nov 29 '25

Core Microsoft Fabric Concepts

#dp600 #microsoft #dataengineering #microsoftfabric

3 min read

Cover image for Implementing a CDC pipeline with Debezium

Nov 29 '25

Implementing a CDC pipeline with Debezium

#python #postgres #dataengineering #architecture

8 min read

Cover image for End-to-End Real-Time Data Engineering on Databricks Using Spark Structured Streaming and Delta Lake

Nithyalakshmi Kamalakkannan

Jan 2

End-to-End Real-Time Data Engineering on Databricks Using Spark Structured Streaming and Delta Lake

#dataengineering #handson #realtimeproject #spark

1 min read

Cover image for Building Bulletproof Data Pipelines: Orchestration, Testing, and Monitoring (Part 3 of 3)

Karthikeyan Rajasekaran

Jan 2

Building Bulletproof Data Pipelines: Orchestration, Testing, and Monitoring (Part 3 of 3)

#dataengineering #dagster #dataquality #testing

9 min read

Cover image for LogInSight: A Lightweight CloudWatch Log Analytics Tool for Faster Debugging and Real-Time Insights

Nov 29 '25

LogInSight: A Lightweight CloudWatch Log Analytics Tool for Faster Debugging and Real-Time Insights

#fastapi #python #aws #dataengineering

3 min read

Cover image for The Proxy Economy: Residential, Datacenter, and ISP Rotation

Jan 1

The Proxy Economy: Residential, Datacenter, and ISP Rotation

#architecture #dataengineering #networking

5 min read

Jan 1

Automating Serverless Data Ingestion: How to Connect External APIs to BigQuery using Python and Cloud Functions

#googlecloud #bigquery #marketing #dataengineering

12 min read

Jan 1

The Database Query That Could Cost a Company Millions(And Why Data Engineers Exist)

#technology #dataengineering #techhistory

5 min read

Vinicius Fagundes

Dec 21 '25

RAG Evaluation Metrics: Measuring What Actually Matters

#llm #rag #dataengineering #ai

10 min read

Baharath Bathula

Jan 1

Why Data SLAs Fail — and How to Enforce Them with a Unified Reliability Framework

#dataengineering #aws #machinelearning #analytics

2 min read

Eliana Lam

Nov 29 '25

Building Streaming Iceberg Tables for Real-Time Logistics Analytics

#analytics #dataengineering #architecture #opensource

4 min read

Cover image for Building a Scalable Community Health Worker Analytics Platform: My Journey with dbt and Snowflake

Amos Augo

Nov 26 '25

Building a Scalable Community Health Worker Analytics Platform: My Journey with dbt and Snowflake

#dbt #snowflake #dataengineering

4 min read

Cover image for The Great Table Format Debate: A Deep Dive into Apache Iceberg, Delta Lake, and Apache Hudi

Data Tech Bridge

Dec 31 '25

The Great Table Format Debate: A Deep Dive into Apache Iceberg, Delta Lake, and Apache Hudi

#database #dataengineering #iceberg #apachehudi

18 min read

Cover image for Amazon Kinesis vs Amazon MSK: The Complete Guide for Stream Processing on AWS

Data Tech Bridge

Dec 30 '25

Amazon Kinesis vs Amazon MSK: The Complete Guide for Stream Processing on AWS

#architecture #aws #dataengineering

29 min read

Cover image for Day 8: Accelerating Spark Joins - Broadcast, Shuffle Optimization & Skew Handling

Sandeep

Dec 9 '25

Day 8: Accelerating Spark Joins - Broadcast, Shuffle Optimization & Skew Handling

#dataengineering #python #spark #bigdata

2 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.