Forem

# dataengineering

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
[Snowflake's New Feature] Introducing Generation 2 Standard Warehouses: A Performance Comparison

[Snowflake's New Feature] Introducing Generation 2 Standard Warehouses: A Performance Comparison

5
Comments
5 min read
Building a Self-Optimizing Data Pipeline

Building a Self-Optimizing Data Pipeline

Comments
3 min read
Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL

Automating Cryptocurrency Data with Python, Apache Airflow and PostgreSQL

Comments
3 min read
A Quick Developer’s Guide to Effective Data Engineering

A Quick Developer’s Guide to Effective Data Engineering

1
Comments
3 min read
Creating a new Airbyte connector from scratch

Creating a new Airbyte connector from scratch

Comments
4 min read
Introduction to Data Engineering Concepts |18| The Power of Dremio in the Modern Lakehouse

Introduction to Data Engineering Concepts |18| The Power of Dremio in the Modern Lakehouse

Comments
5 min read
Introduction to Data Engineering Concepts |1| What is Data Engineering?

Introduction to Data Engineering Concepts |1| What is Data Engineering?

2
Comments
3 min read
Introduction to Data Engineering Concepts |15| Cloud Data Platforms and the Modern Stack

Introduction to Data Engineering Concepts |15| Cloud Data Platforms and the Modern Stack

Comments
4 min read
Introduction to Data Engineering Concepts |17| Apache Iceberg, Arrow, and Polaris

Introduction to Data Engineering Concepts |17| Apache Iceberg, Arrow, and Polaris

Comments
4 min read
Introduction to Data Engineering Concepts |16| Data Lakehouse Architecture Explained

Introduction to Data Engineering Concepts |16| Data Lakehouse Architecture Explained

Comments
4 min read
Introduction to Data Engineering Concepts |13| Building Scalable Pipelines

Introduction to Data Engineering Concepts |13| Building Scalable Pipelines

Comments
4 min read
Introduction to Data Engineering Concepts |14| DevOps for Data Engineering

Introduction to Data Engineering Concepts |14| DevOps for Data Engineering

Comments
4 min read
Introduction to Data Engineering Concepts |12| Scheduling and Workflow Orchestration

Introduction to Data Engineering Concepts |12| Scheduling and Workflow Orchestration

Comments
4 min read
Introduction to Data Engineering Concepts |8| Data Lakes Explained

Introduction to Data Engineering Concepts |8| Data Lakes Explained

Comments
4 min read
Introduction to Data Engineering Concepts |10| Data Quality and Validation

Introduction to Data Engineering Concepts |10| Data Quality and Validation

Comments
4 min read
Introduction to Data Engineering Concepts |7| Data Warehousing Fundamentals

Introduction to Data Engineering Concepts |7| Data Warehousing Fundamentals

Comments
4 min read
Introduction to Data Engineering Concepts |9| Storage Formats and Compression

Introduction to Data Engineering Concepts |9| Storage Formats and Compression

Comments
4 min read
Introduction to Data Engineering Concepts |6| Data Modeling Basics

Introduction to Data Engineering Concepts |6| Data Modeling Basics

Comments
4 min read
Introduction to Data Engineering Concepts |5| Streaming Data Fundamentals

Introduction to Data Engineering Concepts |5| Streaming Data Fundamentals

Comments
4 min read
Introduction to Data Engineering Concepts |4| Batch Processing Fundamentals

Introduction to Data Engineering Concepts |4| Batch Processing Fundamentals

Comments
4 min read
Introduction to Data Engineering Concepts |2| Understanding Data Sources and Ingestion

Introduction to Data Engineering Concepts |2| Understanding Data Sources and Ingestion

1
Comments
4 min read
Introduction to Data Engineering Concepts |3| ETL vs ELT – Understanding Data Pipelines

Introduction to Data Engineering Concepts |3| ETL vs ELT – Understanding Data Pipelines

Comments
4 min read
When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)

When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)

17
Comments 2
5 min read
Big Data Processing - Case Study 4 (Hadoop) 02:36

Big Data Processing - Case Study 4 (Hadoop)

1
Comments
1 min read
[Snowflake's New Feature]Snowflake Programmatic Access Tokens: Easy Authentication for BI Tools like Tableau & Power BI

[Snowflake's New Feature]Snowflake Programmatic Access Tokens: Easy Authentication for BI Tools like Tableau & Power BI

1
Comments
4 min read
loading...