Forem

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
dremioframe & iceberg: Pythonic interfaces for Dremio and Apache Iceberg
Cover image for dremioframe & iceberg: Pythonic interfaces for Dremio and Apache Iceberg
alexmercedcoder profile

dremioframe & iceberg: Pythonic interfaces for Dremio and Apache Iceberg

#python #tooling #database #dataengineering
Comments
8 min read
AWS Lambda and AWS Glue Python Shell in the Context of Lightweight ETL
datapenguin profile

AWS Lambda and AWS Glue Python Shell in the Context of Lightweight ETL

#aws #dataengineering #etl
3
Comments
7 min read
SQL: Doing GROUP BY in CsvPath
Cover image for SQL: Doing GROUP BY in CsvPath
david_kershaw_b6916404da6 profile

SQL: Doing GROUP BY in CsvPath

#sql #dataengineering #csv #csvpath
Comments
5 min read
🔥 Day 3: RDDs - The Foundation of Spark
Cover image for 🔥 Day 3: RDDs - The Foundation of Spark
sandeepk27 profile

🔥 Day 3: RDDs - The Foundation of Spark

#distributedsystems #interview #dataengineering #tutorial
Comments
2 min read
🔥 Day 4: RDD Internals - Partitions, Shuffles & Repartitioning Demystified
Cover image for 🔥 Day 4: RDD Internals - Partitions, Shuffles & Repartitioning Demystified
sandeepk27 profile

🔥 Day 4: RDD Internals - Partitions, Shuffles & Repartitioning Demystified

#spark #dataengineering #bigdata #python
Comments
2 min read
The Developer's Guide to Normalizing Historical Airline Flight Data for Machine Learning
Cover image for The Developer's Guide to Normalizing Historical Airline Flight Data for Machine Learning
rameshchauhan profile

The Developer's Guide to Normalizing Historical Airline Flight Data for Machine Learning

#dataengineering #datascience #machinelearning
Comments
6 min read
Overview of Real-Time Data Synchronization from MySQL to VeloDB
apachedoris profile

Overview of Real-Time Data Synchronization from MySQL to VeloDB

#database #aws #dataengineering #mysql
5
Comments
5 min read
Stop Writing df.describe(): Automate EDA with D-Tale (The Lazy Engineer's Way)
theaidataengineer1010 profile

Stop Writing df.describe(): Automate EDA with D-Tale (The Lazy Engineer's Way)

#python #dataengineering #productivity #beginners
Comments
3 min read
CHW Monthly Activity Aggregation: Turning Visit Logs into Insight
oliver_samuel_028c6f65ad6 profile

CHW Monthly Activity Aggregation: Turning Visit Logs into Insight

#analytics #sql #dataengineering #tooling
Comments
5 min read
🔥 Day 2: Understanding Spark Architecture - How Spark Executes Your Code Internally
Cover image for 🔥 Day 2: Understanding Spark Architecture - How Spark Executes Your Code Internally
sandeepk27 profile

🔥 Day 2: Understanding Spark Architecture - How Spark Executes Your Code Internally

#spark #python #dataengineering #bigdata
Comments
2 min read
When models suggest deprecated Pandas APIs: a small mistake that cascades
olivia_perell_ profile

When models suggest deprecated Pandas APIs: a small mistake that cascades

#codequality #dataengineering #llm #python
Comments
3 min read
Marmot: Data catalog without the complex infrastructure
Cover image for Marmot: Data catalog without the complex infrastructure
charliehaley profile

Marmot: Data catalog without the complex infrastructure

#showdev #dataengineering #opensource #go
1
Comments
3 min read
TDD for dbt: unit testing the way it should be
niclasolofsson profile

TDD for dbt: unit testing the way it should be

#codequality #dataengineering #softwareengineering #testing
2
Comments
12 min read
When code-gen suggests deprecated Pandas APIs: a case study in subtle breakage
gabrieal845 profile

When code-gen suggests deprecated Pandas APIs: a case study in subtle breakage

#codequality #dataengineering #llm #python
Comments
3 min read
Schema, COPY, MERGE, and Immutability — A First-Principles Guide for Data Engineers
shrinivasv73 profile

Schema, COPY, MERGE, and Immutability — A First-Principles Guide for Data Engineers

#dataengineering
Comments
5 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.