Forem

# dataengineering

Posts

đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.
When an AI Suggests DataFrame.append: Missing Pandas Deprecations in Generated Code
gabrieal845 profile

When an AI Suggests DataFrame.append: Missing Pandas Deprecations in Generated Code

#codequality #dataengineering #llm #python
Comments 1
3 min read
The Great Table Format Debate: A Deep Dive into Apache Iceberg, Delta Lake, and Apache Hudi
Cover image for The Great Table Format Debate: A Deep Dive into Apache Iceberg, Delta Lake, and Apache Hudi
datatechbridge profile

The Great Table Format Debate: A Deep Dive into Apache Iceberg, Delta Lake, and Apache Hudi

#database #dataengineering #iceberg #apachehudi
1
Comments
18 min read
Amazon Kinesis vs Amazon MSK: The Complete Guide for Stream Processing on AWS
Cover image for Amazon Kinesis vs Amazon MSK: The Complete Guide for Stream Processing on AWS
datatechbridge profile

Amazon Kinesis vs Amazon MSK: The Complete Guide for Stream Processing on AWS

#architecture #aws #dataengineering
Comments
29 min read
Day 8: Accelerating Spark Joins - Broadcast, Shuffle Optimization & Skew Handling
Cover image for Day 8: Accelerating Spark Joins - Broadcast, Shuffle Optimization & Skew Handling
sandeepk27 profile

Day 8: Accelerating Spark Joins - Broadcast, Shuffle Optimization & Skew Handling

#dataengineering #python #spark #bigdata
Comments
2 min read
Mastering Serverless Data Pipelines: AWS Step Functions Best Practices for 2026
Cover image for Mastering Serverless Data Pipelines: AWS Step Functions Best Practices for 2026
jubinsoni profile

Mastering Serverless Data Pipelines: AWS Step Functions Best Practices for 2026

#aws #serverless #stepfunctions #dataengineering
Comments
5 min read
A Stranger In a New Town: CsvPath metadata fields
Cover image for A Stranger In a New Town: CsvPath metadata fields
david_kershaw_b6916404da6 profile

A Stranger In a New Town: CsvPath metadata fields

#metadata #dataengineering #csv #datascience
Comments
6 min read
Interesting links - November 2025
rmoff profile

Interesting links - November 2025

#data #dataengineering #kafka #flink
Comments
19 min read
đź’€ RIP Copy-Paste: Google NotebookLM Just Killed Manual Data Entry
siddhesh_surve profile

đź’€ RIP Copy-Paste: Google NotebookLM Just Killed Manual Data Entry

#ai #productivity #google #dataengineering
Comments
3 min read
Unified Data Fabric: Serverless Spark on ROSA Integrating with AWS Glue Catalog
mgonzalezo profile

Unified Data Fabric: Serverless Spark on ROSA Integrating with AWS Glue Catalog

#serverless #kubernetes #aws #dataengineering
8
Comments 1
39 min read
dupl
query_filter_591122b53770 profile

dupl

#sql #dataengineering #backend #database
Comments
1 min read
Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Nov 18–24, 2025)
Cover image for Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Nov 18–24, 2025)
alexmercedcoder profile

Apache Dev List Digest: Iceberg, Polaris, Arrow & Parquet (Nov 18–24, 2025)

#data #dataengineering #opensource #resources
Comments
5 min read
How to Sync Data from an Oracle Table to Elasticsearch using Kafka Connect
maghsood_esmaeili profile

How to Sync Data from an Oracle Table to Elasticsearch using Kafka Connect

#database #kubernetes #dataengineering #tutorial
1
Comments 1
5 min read
From Raw to Refined: Data Pipeline Architecture at Scale
pradeep_kaalluri profile

From Raw to Refined: Data Pipeline Architecture at Scale

#dataengineering #bigdata #python #dataquality
Comments
12 min read
Agent Cost Optimization: A Data Engineer's Guide
hoshang_mehta profile

Agent Cost Optimization: A Data Engineer's Guide

#dataengineering #ai
Comments
13 min read
INTRODUCTION TO DBT(Data Build Tool)
john_analytics profile

INTRODUCTION TO DBT(Data Build Tool)

#dataengineering #data #analytics
1
Comments
2 min read
đź‘‹ Sign in for the ability to sort posts by relevant, latest, or top.