Skip to content

Forem

# dataengineering

👋 Sign in for the ability to sort posts by relevant, latest, or top.

Jan 5

Streaming SQL Engine: Lightweight Cross-Data Source Integration for Resource-Constrained Environments.

#architecture #dataengineering #python #sql

1 min read

Christian Wagner

Jan 2

Stop Manually Tracing Azure Synapse Dependencies

#database #sql #opensource #dataengineering

1 min read

Cover image for Building Pangolin: My Holiday Break, an AI IDE, and a Lakehouse Catalog for the Curious

Jan 2

Building Pangolin: My Holiday Break, an AI IDE, and a Lakehouse Catalog for the Curious

#ai #dataengineering #devjournal

6 min read

Cover image for Part 8: Databricks Pipeline & Dashboard

Nithyalakshmi Kamalakkannan

Jan 2

Part 8: Databricks Pipeline & Dashboard

#analytics #dataengineering #tutorial #sql

2 min read

Cover image for Part 1: Creating Databricks Workspace and Enabling Unity Catalog

Nithyalakshmi Kamalakkannan

Jan 2

Part 1: Creating Databricks Workspace and Enabling Unity Catalog

#beginners #azure #dataengineering #tutorial

2 min read

Cover image for Part 2: Project Architecture

Nithyalakshmi Kamalakkannan

Jan 2

Part 2: Project Architecture

#systemdesign #data #architecture #dataengineering

2 min read

Cover image for Part 4: Building the Bronze Layer with Auto Loader and Delta Lake

Nithyalakshmi Kamalakkannan

Jan 2

Part 4: Building the Bronze Layer with Auto Loader and Delta Lake

#architecture #dataengineering #tutorial #sql

2 min read

Cover image for Part 5: Building a ZIP Code Dimension Table

Nithyalakshmi Kamalakkannan

Jan 2

Part 5: Building a ZIP Code Dimension Table

#analytics #database #dataengineering #tutorial

2 min read

Cover image for Part 3: Simulating Real-Time Streaming Data Using Databricks Sample Datasets

Nithyalakshmi Kamalakkannan

Jan 2

Part 3: Simulating Real-Time Streaming Data Using Databricks Sample Datasets

#dataengineering #python #tutorial

1 min read

Jan 1

Automating Serverless Data Ingestion: How to Connect External APIs to BigQuery using Python and Cloud Functions

#googlecloud #bigquery #marketing #dataengineering

12 min read

Cover image for The Data Liberation: Amazon Athena and the Architecting of a Serverless Future

Neo

Jan 1

The Data Liberation: Amazon Athena and the Architecting of a Serverless Future

#aws #dataengineering #serverless #cloudcomputing

3 min read

Cover image for Why Apache Ozone is the Preferred Object Store for Big Data

Tayfun Yalcinkaya

Jan 5

Why Apache Ozone is the Preferred Object Store for Big Data

#dataengineering #bigdata #datalakehouse #apacheozone

3 min read

Jan 5

Be Essential or Be Optional: A Reality Check for Data Teams

#leadership #growth #dataengineering #datateams

1 min read

Cover image for Markdown Is Not The Future of LLM Data Infrastructure

Yash Dubey

Feb 5

Markdown Is Not The Future of LLM Data Infrastructure

#ai #llm #webscraping #dataengineering

7 min read

Jan 3

Event-Driven Data Pipelines - Real-Time Orchestration on AWS

#dataengineering #aws #eventdriven

4 min read

👋 Sign in for the ability to sort posts by relevant, latest, or top.