Forem

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
Join OSA CON 2025: Two Days of Open‑Source Analytics and AI (Nov. 4–5)
Cover image for Join OSA CON 2025: Two Days of Open‑Source Analytics and AI (Nov. 4–5)
altinity profile

Join OSA CON 2025: Two Days of Open‑Source Analytics and AI (Nov. 4–5)

#events #opensource #ai #dataengineering
Comments
3 min read
AWS Glue for ETL
godofgeeks profile

AWS Glue for ETL

#aws #dataengineering #serverless
Comments
5 min read
What to use for data preparation in report, query or analysis business?
esproc_spl profile

What to use for data preparation in report, query or analysis business?

#analytics #dataengineering #datascience
5
Comments
10 min read
Optimizing Data Processing on AWS with Data Compaction
datapenguin profile

Optimizing Data Processing on AWS with Data Compaction

#aws #dataengineering #iceberg #lambda
4
Comments
7 min read
Real-Time Earthquake CDC Pipeline
oliver_samuel_028c6f65ad6 profile

Real-Time Earthquake CDC Pipeline

#architecture #database #dataengineering
Comments
5 min read
Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS
datapenguin profile

Designing a Cost-Efficient Parallel Data Pipeline on AWS Using Lambda and SQS

#aws #dataengineering #lambda #sqs
3
Comments
6 min read
The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane
Cover image for The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane
markolekic profile

The Offline Data Engineer: Building Resilient API Pipelines that Work on an Airplane

#python #dataengineering #api #sql
4
Comments
5 min read
Understanding Kafka Architecture, Schema Registry, ksqlDB, PostgreSQL, Couchbase, and Microservices
jumptotech profile

Understanding Kafka Architecture, Schema Registry, ksqlDB, PostgreSQL, Couchbase, and Microservices

#architecture #dataengineering #microservices #beginners
2
Comments
3 min read
Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025
bolewara profile

Building a 75,000-Product Image Feature Dataset for the Amazon ML Challenge 2025

#machinelearning #deeplearning #dataengineering #pca
1
Comments
4 min read
An Exploration of the Commercial Iceberg Catalog Ecosystem
Cover image for An Exploration of the Commercial Iceberg Catalog Ecosystem
alexmercedcoder profile

An Exploration of the Commercial Iceberg Catalog Ecosystem

#dataengineering
Comments
14 min read
🧠 ClickHouse LEFT JOINs: Why join_use_nulls Matters
Cover image for 🧠 ClickHouse LEFT JOINs: Why join_use_nulls Matters
ranjbaryshahab profile

🧠 ClickHouse LEFT JOINs: Why join_use_nulls Matters

#clickhouse #sql #dataengineering #join
5
Comments
2 min read
Getting Started Building a Data Platform
jorinvo profile

Getting Started Building a Data Platform

#dataengineering #database #sql
Comments
3 min read
Building a Universal Lakehouse Catalog: Beyond Iceberg Tables
Cover image for Building a Universal Lakehouse Catalog: Beyond Iceberg Tables
alexmercedcoder profile

Building a Universal Lakehouse Catalog: Beyond Iceberg Tables

#bigdata #dataengineering #architecture #database
Comments
10 min read
Real-time Data Analytics at Scale: Integrating Apache Flink and Apache Doris with Flink Doris Connector and Flink CDC
apachedoris profile

Real-time Data Analytics at Scale: Integrating Apache Flink and Apache Doris with Flink Doris Connector and Flink CDC

#olap #apacheflink #dataengineering #database
Comments
10 min read
Optimizing Kafka Performance: Best Practices for High Throughput and Low Latency
Cover image for Optimizing Kafka Performance: Best Practices for High Throughput and Low Latency
zoolatech profile

Optimizing Kafka Performance: Best Practices for High Throughput and Low Latency

#architecture #dataengineering #performance
Comments
7 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.