Forem

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
ClickHouse Has a Free Column-Oriented Database — Query Billions of Rows in Milliseconds
0012303 profile

ClickHouse Has a Free Column-Oriented Database — Query Billions of Rows in Milliseconds

#database #dataengineering #analytics #opensource
Comments
2 min read
How Linux is Used in Real-World Data Engineering
Cover image for How Linux is Used in Real-World Data Engineering
fredrickm profile

How Linux is Used in Real-World Data Engineering

#cli #dataengineering #linux #tutorial
Comments
3 min read
100 Spark Interview Questions for Data Engineer
Cover image for 100 Spark Interview Questions for Data Engineer
hannah_usmedynska profile

100 Spark Interview Questions for Data Engineer

#career #dataengineering #interview #resources
1
Comments
11 min read
Flowfile v0.8.0 — Your Flows Can Run Themselves Now
edwardvaneechoud profile

Flowfile v0.8.0 — Your Flows Can Run Themselves Now

#showdev #automation #dataengineering #tooling
Comments
4 min read
# Apache Data Lakehouse Weekly: March 20–27, 2026
alexmercedcoder profile

# Apache Data Lakehouse Weekly: March 20–27, 2026

#news #community #dataengineering #opensource
Comments
7 min read
Frosty : 150 + AI Open Source Sub- Agents to Automate Snowflake
Cover image for Frosty : 150 + AI Open Source Sub- Agents to Automate Snowflake
malviyapriyank profile

Frosty : 150 + AI Open Source Sub- Agents to Automate Snowflake

#ai #agents #opensource #dataengineering
Comments
2 min read
When Synthetic Data Lies: A Hidden Correlation Problem I Didn’t Expect
Cover image for When Synthetic Data Lies: A Hidden Correlation Problem I Didn’t Expect
mohhddhassan profile

When Synthetic Data Lies: A Hidden Correlation Problem I Didn’t Expect

#dataengineering #clickhouse #analytics #debugging
3
Comments
3 min read
Building & Monitoring Data Backends: Tools, Architecture, and Observability
soytuber profile
soy

Building & Monitoring Data Backends: Tools, Architecture, and Observability

#database #sql #dataengineering
Comments
4 min read
Issues of Multi-GB Spreadsheets in Data Lakes
toby-patrick profile

Issues of Multi-GB Spreadsheets in Data Lakes

#data #dataengineering #performance
Comments
4 min read
Asset-Based Data Orchestration: Lessons from Building a Multi-State Social Data Platform
Cover image for Asset-Based Data Orchestration: Lessons from Building a Multi-State Social Data Platform
engineering profile

Asset-Based Data Orchestration: Lessons from Building a Multi-State Social Data Platform

#dagster #dataorchestration #dataengineering
1
Comments
6 min read
The Backyard Quarry, Part 2: Designing a Schema for Physical Objects
kenwalger profile

The Backyard Quarry, Part 2: Designing a Schema for Physical Objects

#dataengineering #softwareengineering #backendengineering #datamodeling
2
Comments
5 min read
The Vinted Arbitrage War: Building a Scraper That Doesn't Get IP-Banned
Cover image for The Vinted Arbitrage War: Building a Scraper That Doesn't Get IP-Banned
datakaz profile

The Vinted Arbitrage War: Building a Scraper That Doesn't Get IP-Banned

#webdev #python #dataengineering #tutorial
Comments
9 min read
Is AWS Glue Data Catalog Sufficient as a Data Catalog? Organizing Its Design, Limitations, and Complementary Strategies
datapenguin profile

Is AWS Glue Data Catalog Sufficient as a Data Catalog? Organizing Its Design, Limitations, and Complementary Strategies

#aws #dataengineering
6
Comments
10 min read
I built pq - the jq of Parquet. Here's why data engineers need a better CLI
eorlov profile

I built pq - the jq of Parquet. Here's why data engineers need a better CLI

#rust #cli #dataengineering #opensource
1
Comments
1 min read
How to Build a Scalable Serverless Social Media Ingestion & Analytics Pipeline on AWS
Cover image for How to Build a Scalable Serverless Social Media Ingestion & Analytics Pipeline on AWS
capestart profile

How to Build a Scalable Serverless Social Media Ingestion & Analytics Pipeline on AWS

#aws #serverless #dataengineering
1
Comments
4 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.