Forem

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
Scraping the Schema of NetSuite

Scraping the Schema of NetSuite

Comments
2 min read
You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees
Cover image for You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees

You Can't Trust COUNT and SUM: Scalable Data Validation with Merkle Trees

2
Comments 1
8 min read
Docker for Data Engineers: The Complete Beginner’s Guide
Cover image for Docker for Data Engineers: The Complete Beginner’s Guide

Docker for Data Engineers: The Complete Beginner’s Guide

4
Comments
6 min read
Lightweight ETL with AWS Lambda, DuckDB, and delta-rs

Lightweight ETL with AWS Lambda, DuckDB, and delta-rs

3
Comments
5 min read
Dynamic Routing Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

Dynamic Routing Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

3
Comments 1
5 min read
Career Opportunities After Completing AI & Data Science Degree

Career Opportunities After Completing AI & Data Science Degree

Comments
3 min read
Big Data Fundamentals: real-time analytics project

Big Data Fundamentals: real-time analytics project

Comments
6 min read
The Case for Apache Airflow and Kafka in Data Engineering
Cover image for The Case for Apache Airflow and Kafka in Data Engineering

The Case for Apache Airflow and Kafka in Data Engineering

1
Comments
2 min read
🛠️ SiliconPrimeX – Healing AWS Glue Jobs Autonomously with Gemini & Lambda 🚑✨

🛠️ SiliconPrimeX – Healing AWS Glue Jobs Autonomously with Gemini & Lambda 🚑✨

Comments
2 min read
Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

Lightweight ETL with AWS Lambda, DuckDB, and PyIceberg

4
Comments
4 min read
Real-Time Fraud Detection Using Kafka and Machine Learning
Cover image for Real-Time Fraud Detection Using Kafka and Machine Learning

Real-Time Fraud Detection Using Kafka and Machine Learning

Comments
5 min read
🕒 Cumulative Data Without the Pain: PostgreSQL Rollups with Time Buckets
Cover image for 🕒 Cumulative Data Without the Pain: PostgreSQL Rollups with Time Buckets

🕒 Cumulative Data Without the Pain: PostgreSQL Rollups with Time Buckets

Comments
3 min read
Check Out 3 Awesome Open Source Tabular Data Wrangling Apps

Check Out 3 Awesome Open Source Tabular Data Wrangling Apps

2
Comments
3 min read
15 Core Concepts of Data Engineering

15 Core Concepts of Data Engineering

Comments
9 min read
Building a Food Price & Inflation Analysis Pipeline in Kenya (2006–2024)
Cover image for Building a Food Price & Inflation Analysis Pipeline in Kenya (2006–2024)

Building a Food Price & Inflation Analysis Pipeline in Kenya (2006–2024)

Comments
3 min read
Understanding Data Warehousing for Retail Analytics: A Comprehensive Guide
Cover image for Understanding Data Warehousing for Retail Analytics: A Comprehensive Guide

Understanding Data Warehousing for Retail Analytics: A Comprehensive Guide

1
Comments
3 min read
Building a Modern Data Warehouse in SQL Server with Medallion Architecture
Cover image for Building a Modern Data Warehouse in SQL Server with Medallion Architecture

Building a Modern Data Warehouse in SQL Server with Medallion Architecture

Comments
11 min read
Benefits of OLAP and OLTP in Data Management.
Cover image for Benefits of OLAP and OLTP in Data Management.

Benefits of OLAP and OLTP in Data Management.

Comments
2 min read
Building High-Load API Services in Go: From Design to Production
Cover image for Building High-Load API Services in Go: From Design to Production

Building High-Load API Services in Go: From Design to Production

2
Comments
23 min read
Why We Built Confidence Scoring Into Our Date Parser (And Why Every API Should)
Cover image for Why We Built Confidence Scoring Into Our Date Parser (And Why Every API Should)

Why We Built Confidence Scoring Into Our Date Parser (And Why Every API Should)

Comments 1
3 min read
🧭 Data Mesh vs Data Fabric (Part 1) – Rethinking How We Scale Data

🧭 Data Mesh vs Data Fabric (Part 1) – Rethinking How We Scale Data

Comments
2 min read
Is your Vector Database Really Fast?
Cover image for Is your Vector Database Really Fast?

Is your Vector Database Really Fast?

Comments
9 min read
Kubernetes in Depth - Storage, Security, and Advanced Features
Cover image for Kubernetes in Depth - Storage, Security, and Advanced Features

Kubernetes in Depth - Storage, Security, and Advanced Features

1
Comments
6 min read
Building a Resilient Exception Strategy with Apache Beam and DLQ

Building a Resilient Exception Strategy with Apache Beam and DLQ

Comments
3 min read
Classes in Python, a beginner's pov

Classes in Python, a beginner's pov

1
Comments
2 min read
loading...