Forem

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane
Cover image for The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane
markolekic profile

The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane

#python #dataengineering #api #sql
Comments
5 min read
Understanding Kafka Architecture, Schema Registry, ksqlDB, PostgreSQL, Couchbase, and Microservices
jumptotech profile

Understanding Kafka Architecture, Schema Registry, ksqlDB, PostgreSQL, Couchbase, and Microservices

#architecture #dataengineering #microservices #beginners
2
Comments
3 min read
Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025
bolewara profile

Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025

#machinelearning #deeplearning #dataengineering #pca
1
Comments
4 min read
Why We Need Schema Registry in Kafka
jumptotech profile

Why We Need Schema Registry in Kafka

#systemdesign #distributedsystems #dataengineering #architecture
2
Comments
17 min read
Introduction to the Confluent REST Proxy
jumptotech profile

Introduction to the Confluent REST Proxy

#api #architecture #dataengineering
2
Comments
4 min read
An Exploration of the Commercial Iceberg Catalog Ecosystem
Cover image for An Exploration of the Commercial Iceberg Catalog Ecosystem
alexmercedcoder profile

An Exploration of the Commercial Iceberg Catalog Ecosystem

#dataengineering
Comments
14 min read
🧠 ClickHouse LEFT JOINs: Why join_use_nulls Matters
Cover image for 🧠 ClickHouse LEFT JOINs: Why join_use_nulls Matters
ranjbaryshahab profile

🧠 ClickHouse LEFT JOINs: Why join_use_nulls Matters

#clickhouse #sql #dataengineering #join
5
Comments
2 min read
Getting Started Building a Data Platform
jorinvo profile

Getting Started Building a Data Platform

#dataengineering #database #sql
Comments
3 min read
Real-time Data Analytics at Scale: Integrating Apache Flink and Apache Doris with Flink Doris Connector and Flink CDC
apachedoris profile

Real-time Data Analytics at Scale: Integrating Apache Flink and Apache Doris with Flink Doris Connector and Flink CDC

#olap #apacheflink #dataengineering #database
Comments
10 min read
Optimizing Kafka Performance: Best Practices for High Throughput and Low Latency
Cover image for Optimizing Kafka Performance: Best Practices for High Throughput and Low Latency
zoolatech profile

Optimizing Kafka Performance: Best Practices for High Throughput and Low Latency

#architecture #dataengineering #performance
Comments
7 min read
Fixing Type Hints for Callable Objects with Custom Signatures in Dagster
jongwan93 profile

Fixing Type Hints for Callable Objects with Custom Signatures in Dagster

#machinelearning #python #dataengineering #opensource
1
Comments
3 min read
Apache Spark সহজভাবে জানি
sajjadrahman56 profile

Apache Spark সহজভাবে জানি

#spark #dataengineering #bigdata #tutorial
1
Comments
1 min read
Building a Test Data Platform After Watching Teams Secretly Use Production for Years
Cover image for Building a Test Data Platform After Watching Teams Secretly Use Production for Years
alex_hayward_ff2d88ff331e profile

Building a Test Data Platform After Watching Teams Secretly Use Production for Years

#devops #testing #cicd #dataengineering
Comments
3 min read
Chinese DBA's Story: Sui Haifeng - Grasp the two most important five-year periods of your career
tripcat profile

Chinese DBA's Story: Sui Haifeng - Grasp the two most important five-year periods of your career

#database #dataengineering #data
Comments
5 min read
Kafka
jumptotech profile

Kafka

#dataengineering #beginners #opensource #architecture
3
Comments
10 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.