Skip to content
Navigation menu
Search
Powered by
Search
Algolia
Log in
Create account
Forem
Close
#
dataengineering
Follow
Hide
Posts
Left menu
👋
Sign in
for the ability to sort posts by
relevant
,
latest
, or
top
.
Right menu
There is no Data Engineering roadmap
Al Brown
Al Brown
Al Brown
Follow
Jul 17 '23
There is no Data Engineering roadmap
#
dataengineering
#
beginners
#
career
#
database
2
reactions
Comments
Add Comment
5 min read
Workflow of Data Engineering Project on AWS
Ronal Niraula
Ronal Niraula
Ronal Niraula
Follow
Jul 5 '23
Workflow of Data Engineering Project on AWS
#
aws
#
dataengineering
#
data
#
jupyter
1
reaction
Comments
Add Comment
4 min read
Feature Engineering Has a Language Problem
Tun
Tun
Tun
Follow
Jul 4 '23
Feature Engineering Has a Language Problem
#
machinelearning
#
datascience
#
dataengineering
#
mlops
1
reaction
Comments
Add Comment
15 min read
What is data engineering and a B.I architecture
Pedro H Goncalves
Pedro H Goncalves
Pedro H Goncalves
Follow
Jun 30 '23
What is data engineering and a B.I architecture
#
dataengineering
#
datascience
#
beginners
#
architecture
5
reactions
Comments
Add Comment
6 min read
How To Create Dataflow Job with Scio
Edward Huang
Edward Huang
Edward Huang
Follow
Jun 24 '23
How To Create Dataflow Job with Scio
#
dataengineering
#
programming
#
code
#
scala
2
reactions
Comments
Add Comment
8 min read
Using pyspark to stream data from coingecko API and visualise using dash
James
James
James
Follow
Jun 18 '23
Using pyspark to stream data from coingecko API and visualise using dash
#
spark
#
python
#
dataengineering
#
codenewbie
3
reactions
Comments
Add Comment
6 min read
AWS Redshift: Robust and Scalable Data Warehousing
Mage AI
Mage AI
Mage AI
Follow
Jun 14 '23
AWS Redshift: Robust and Scalable Data Warehousing
#
aws
#
redshift
#
datawarehousing
#
dataengineering
3
reactions
Comments
Add Comment
6 min read
A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes
Mage AI
Mage AI
Mage AI
Follow
Jun 12 '23
A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes
#
startup
#
dataengineering
#
datapipelines
#
etl
6
reactions
Comments
Add Comment
9 min read
Stream data processing with Mage
Mage AI
Mage AI
Mage AI
Follow
Jun 13 '23
Stream data processing with Mage
#
streaming
#
data
#
dataengineering
#
kafka
6
reactions
Comments
Add Comment
8 min read
How to pivot data using Dynamic SQL in SQL Server
Olesia Dudareva
Olesia Dudareva
Olesia Dudareva
Follow
May 22 '23
How to pivot data using Dynamic SQL in SQL Server
#
sql
#
database
#
dataengineering
#
mssql
5
reactions
Comments
4
comments
3 min read
How to clone tables in BigQuery
Marcelo Costa
Marcelo Costa
Marcelo Costa
Follow
May 18 '23
How to clone tables in BigQuery
#
bigquery
#
bigdata
#
dataengineering
2
reactions
Comments
Add Comment
1 min read
kafka: event driven microservices
James
James
James
Follow
May 18 '23
kafka: event driven microservices
#
python
#
kafka
#
dataengineering
4
reactions
Comments
Add Comment
6 min read
Getting started with Apache Flink: A guide to stream processing
Mage AI
Mage AI
Mage AI
Follow
May 15 '23
Getting started with Apache Flink: A guide to stream processing
#
flink
#
streamprocessing
#
dataprocessing
#
dataengineering
33
reactions
Comments
Add Comment
8 min read
How to rotate data using Pivot & Unpivot operators
Olesia Dudareva
Olesia Dudareva
Olesia Dudareva
Follow
May 15 '23
How to rotate data using Pivot & Unpivot operators
#
sql
#
database
#
dataengineering
#
mssql
3
reactions
Comments
2
comments
3 min read
Apply CDC From MySQL To Clickhouse on local environment
Mohammad Arab Anvari
Mohammad Arab Anvari
Mohammad Arab Anvari
Follow
May 15 '23
Apply CDC From MySQL To Clickhouse on local environment
#
dataengineering
#
cdc
#
debezium
#
tutorial
7
reactions
Comments
Add Comment
6 min read
PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
Rafael Pierre
Rafael Pierre
Rafael Pierre
Follow
May 12 '23
PyJaws: A Pythonic Way to Define Databricks Jobs and Workflows
#
spark
#
dataengineering
#
airflow
#
orchestration
4
reactions
Comments
Add Comment
1 min read
Apache Flink vs Apache Spark: A detailed comparison for data processing
Mage AI
Mage AI
Mage AI
Follow
May 8 '23
Apache Flink vs Apache Spark: A detailed comparison for data processing
#
apacheflink
#
apachespark
#
dataprocessing
#
dataengineering
15
reactions
Comments
1
comment
5 min read
Abstract Configurations
CincyBC
CincyBC
CincyBC
Follow
Apr 24 '23
Abstract Configurations
#
programming
#
python
#
dataengineering
#
tutorial
1
reaction
Comments
Add Comment
3 min read
Apache Flink episode 1: A comprehensive introduction
Mage AI
Mage AI
Mage AI
Follow
May 5 '23
Apache Flink episode 1: A comprehensive introduction
#
dataengineering
#
apacheflink
#
data
#
opensource
2
reactions
Comments
Add Comment
6 min read
Data sources episode 2: AWS S3 to Postgres Data Sync using Singer
Mage AI
Mage AI
Mage AI
Follow
May 5 '23
Data sources episode 2: AWS S3 to Postgres Data Sync using Singer
#
dataengineering
#
community
#
awss3
#
singer
3
reactions
Comments
Add Comment
4 min read
Data sources episode 1: Common data sources in modern pipelines
Mage AI
Mage AI
Mage AI
Follow
May 4 '23
Data sources episode 1: Common data sources in modern pipelines
#
dataengineering
#
datasources
#
datapipelines
#
moderndatasources
1
reaction
Comments
Add Comment
6 min read
Handling NULL in the DBs
IrinaBezgina
IrinaBezgina
IrinaBezgina
Follow
May 3 '23
Handling NULL in the DBs
#
database
#
dataengineering
#
sql
5
reactions
Comments
1
comment
2 min read
Class to Airflow Custom Operator
CincyBC
CincyBC
CincyBC
Follow
May 1 '23
Class to Airflow Custom Operator
#
python
#
programming
#
tutorial
#
dataengineering
Comments
Add Comment
3 min read
Scraper Function to Airflow DAG
CincyBC
CincyBC
CincyBC
Follow
Apr 27 '23
Scraper Function to Airflow DAG
#
python
#
programming
#
tutorial
#
dataengineering
5
reactions
Comments
1
comment
3 min read
From Class to Abstract Classes
CincyBC
CincyBC
CincyBC
Follow
Apr 19 '23
From Class to Abstract Classes
#
python
#
beginners
#
programming
#
dataengineering
1
reaction
Comments
Add Comment
3 min read
loading...
We're a blogging-forward open source social network where we learn from one another
Log in
Create account