Skip to content
Navigation menu
Search
Powered by
Search
Algolia
Log in
Create account
Forem
Close
#
dataengineering
Follow
Hide
Posts
Left menu
👋
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
Introduction to Data Engineering Concepts |16| Data Lakehouse Architecture Explained
Alex Merced
Alex Merced
Alex Merced
Follow
May 2
Introduction to Data Engineering Concepts |16| Data Lakehouse Architecture Explained
#
data
#
datascience
#
database
#
dataengineering
Comments
Add Comment
4 min read
Introduction to Data Engineering Concepts |15| Cloud Data Platforms and the Modern Stack
Alex Merced
Alex Merced
Alex Merced
Follow
May 2
Introduction to Data Engineering Concepts |15| Cloud Data Platforms and the Modern Stack
#
data
#
datascience
#
database
#
dataengineering
Comments
Add Comment
4 min read
Introduction to Data Engineering Concepts |12| Scheduling and Workflow Orchestration
Alex Merced
Alex Merced
Alex Merced
Follow
May 2
Introduction to Data Engineering Concepts |12| Scheduling and Workflow Orchestration
#
data
#
datascience
#
database
#
dataengineering
Comments
Add Comment
4 min read
Introduction to Data Engineering Concepts |8| Data Lakes Explained
Alex Merced
Alex Merced
Alex Merced
Follow
May 2
Introduction to Data Engineering Concepts |8| Data Lakes Explained
#
data
#
datascience
#
dataengineering
#
database
Comments
Add Comment
4 min read
Introduction to Data Engineering Concepts |10| Data Quality and Validation
Alex Merced
Alex Merced
Alex Merced
Follow
May 2
Introduction to Data Engineering Concepts |10| Data Quality and Validation
#
data
#
datascience
#
database
#
dataengineering
Comments
Add Comment
4 min read
Introduction to Data Engineering Concepts |2| Understanding Data Sources and Ingestion
Alex Merced
Alex Merced
Alex Merced
Follow
May 2
Introduction to Data Engineering Concepts |2| Understanding Data Sources and Ingestion
#
database
#
dataengineering
#
datascience
#
data
1
reaction
Comments
Add Comment
4 min read
When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)
Olga Braginskaya
Olga Braginskaya
Olga Braginskaya
Follow
May 9
When Small Parquet Files Become a Big Problem (and How I Ended Up Writing a Compactor in PyArrow)
#
dataengineering
#
python
#
data
#
tutorial
17
reactions
Comments
2
comments
5 min read
Big Data Processing - Case Study 4 (Hadoop)
02:36
Afrar Malakooth
Afrar Malakooth
Afrar Malakooth
Follow
May 1
Big Data Processing - Case Study 4 (Hadoop)
#
bigdata
#
hadoop
#
dataengineering
1
reaction
Comments
Add Comment
1 min read
Building and Deploying My First Python ETL Package to PyPI
Denzel Kanyeki
Denzel Kanyeki
Denzel Kanyeki
Follow
Apr 30
Building and Deploying My First Python ETL Package to PyPI
#
python
#
dataengineering
#
programming
1
reaction
Comments
1
comment
3 min read
Weather Monitoring System Using IoT
Raj Malhotra
Raj Malhotra
Raj Malhotra
Follow
Apr 28
Weather Monitoring System Using IoT
#
iot
#
webdev
#
python
#
dataengineering
1
reaction
Comments
Add Comment
2 min read
The Complete Guide to Setting Up Postgresql on Windows 11 and WSL2
kubona Martin Yafesi
kubona Martin Yafesi
kubona Martin Yafesi
Follow
Apr 26
The Complete Guide to Setting Up Postgresql on Windows 11 and WSL2
#
dataengineering
#
beginners
#
opensource
#
learning
7
reactions
Comments
4
comments
6 min read
Big Data Processing - Case Study 3 (Databricks)
01:53
Afrar Malakooth
Afrar Malakooth
Afrar Malakooth
Follow
Apr 27
Big Data Processing - Case Study 3 (Databricks)
#
bigdata
#
databricks
#
cloud
#
dataengineering
Comments
2
comments
1 min read
How I Automated Crypto Price Tracking with Apache Airflow & CoinGecko
Rotich Kelly
Rotich Kelly
Rotich Kelly
Follow
Jun 7
How I Automated Crypto Price Tracking with Apache Airflow & CoinGecko
#
dataengineering
#
data
#
cryptocurrency
4
reactions
Comments
3
comments
2 min read
Personal Picks: Data Product News (March 19, 2025)
Sagara
Sagara
Sagara
Follow
Mar 23
Personal Picks: Data Product News (March 19, 2025)
#
dataengineering
Comments
Add Comment
5 min read
Big Data Processing - Case Study 3 (Spark)
02:35
Afrar Malakooth
Afrar Malakooth
Afrar Malakooth
Follow
Apr 26
Big Data Processing - Case Study 3 (Spark)
#
bigdata
#
spark
#
dataengineering
#
tutorial
Comments
Add Comment
1 min read
Data Warehouses and Data Lakes: Understanding Modern Data Storage Paradigms 📦
Saurabh Mahawar
Saurabh Mahawar
Saurabh Mahawar
Follow
Apr 25
Data Warehouses and Data Lakes: Understanding Modern Data Storage Paradigms 📦
#
opensource
#
presto
#
datascience
#
dataengineering
Comments
Add Comment
2 min read
Python 101 For Data Engineering
Maiyo
Maiyo
Maiyo
Follow
Apr 25
Python 101 For Data Engineering
#
dataengineering
#
python
#
ubuntu
#
cmd
Comments
2
comments
4 min read
Introduction to Presto: Open Source SQL Query Engine that's changing Big Data Analytics
Saurabh Mahawar
Saurabh Mahawar
Saurabh Mahawar
Follow
Apr 25
Introduction to Presto: Open Source SQL Query Engine that's changing Big Data Analytics
#
opensource
#
presto
#
dataengineering
#
datascience
Comments
Add Comment
2 min read
Get the Records after and before the Searched One — From SQL to SPL #18
Judith-Data-Processing-Hacks
Judith-Data-Processing-Hacks
Judith-Data-Processing-Hacks
Follow
Apr 14
Get the Records after and before the Searched One — From SQL to SPL #18
#
programming
#
sql
#
esprocspl
#
dataengineering
8
reactions
Comments
2
comments
2 min read
NoSQL Fighters Arena: The Battle of Data Titans
Wallace Espindola
Wallace Espindola
Wallace Espindola
Follow
Apr 24
NoSQL Fighters Arena: The Battle of Data Titans
#
database
#
developers
#
dataengineering
#
performance
Comments
Add Comment
3 min read
Kafka Consumers Explained: Pull, Offsets, and Parallelism
Konstantinas Mamonas
Konstantinas Mamonas
Konstantinas Mamonas
Follow
Apr 23
Kafka Consumers Explained: Pull, Offsets, and Parallelism
#
eventdriven
#
architecture
#
streaming
#
dataengineering
1
reaction
Comments
Add Comment
4 min read
Why Pi-Shaped Teams Matter in This AI Era
Kannan Kalidasan
Kannan Kalidasan
Kannan Kalidasan
Follow
Mar 19
Why Pi-Shaped Teams Matter in This AI Era
#
ai
#
softwareengineering
#
leadership
#
dataengineering
Comments
Add Comment
5 min read
How to Integrate AWS Lambda With TimescaleDB for Scalable IoT Data Pipelines
Team Timescale
Team Timescale
Team Timescale
Follow
for
TigerData (Creators of TimescaleDB)
Apr 22
How to Integrate AWS Lambda With TimescaleDB for Scalable IoT Data Pipelines
#
aws
#
database
#
dataengineering
#
programming
2
reactions
Comments
Add Comment
6 min read
The 5 Most Common ETL Patterns — When and Why to Use Them
Gervais Yao Amoah
Gervais Yao Amoah
Gervais Yao Amoah
Follow
Apr 22
The 5 Most Common ETL Patterns — When and Why to Use Them
#
dataengineering
#
programming
#
learning
Comments
Add Comment
3 min read
Data architecture stack
Max Verstappen
Max Verstappen
Max Verstappen
Follow
Mar 19
Data architecture stack
#
dataengineering
#
programming
#
softwareengineering
#
database
Comments
Add Comment
1 min read
loading...
We're a blogging-forward open source social network where we learn from one another
Log in
Create account