DEV Community

UnplugCharger
UnplugCharger

Posted on

2 1 1 1 1

🚀 Building a Serverless Platform: Systems Programming 🚀

If you've ever felt stuck building yet another todo app and craved a project that dives into the guts of how systems work, you're in the right place. Let me take you through my journey of building a lightweight serverless platform from scratch—a project that taught me more about containerization, process management, and API design than any tutorial ever could.

Heads up: This is an educational project, not production-ready! Check out the full code here.

Why Build a Serverless Platform?

Serverless computing is everywhere, but how does it actually work under the hood? I wanted to peel back the layers and explore:

🛠️ Systems Programming Challenges: File handling, process isolation, security.

🧩 Practical Utility: Build something usable, not just theoretical.

📦 Modern Tech Stack: Docker, REST APIs, multi-language support.

Spoiler: It was harder—and more rewarding—than I expected.


How It Works: Breaking Down the Core Components

1. Handling User Code Safely

When users upload a zip file, the platform must extract it without risking security flaws like the infamous zip slip vulnerability:

// Simplified code snippet for secure zip extraction
filePath := filepath.Join(destDir, file.Name)
if !strings.HasPrefix(filePath, filepath.Clean(destDir)+string(os.PathSeparator)) {
    return fmt.Errorf("illegal file path: %s", file.Name)
} 
Enter fullscreen mode Exit fullscreen mode

Key Takeaways:

  • Validate every file path to prevent directory escapes.
  • Use Go's os and filepath libraries for safe file operations.

2. Detecting Programming Languages

The platform supports Python and Go (for now!). A simple but effective approach:

// Check for Python files
pythonFiles, _ := filepath.Glob(filepath.Join(dir, "*.py"))
if len(pythonFiles) > 0 {
    return "python", nil
}
Enter fullscreen mode Exit fullscreen mode

Why This Matters:

  • Introduces pattern matching and filesystem inspection.
  • Lays groundwork for multi-language support.

3. Docker: The Heart of Isolation

Running untrusted code safely meant leaning into Docker. Here’s the workflow:

Build a Docker image from the user’s code.

Run containers with strict resource limits (CPU, memory, network).

// Simplified Docker execution with security constraints
cmd := exec.Command("docker", "run", 
    "--read-only",     // No writes to filesystem
    "--network=none",  // No internet access
    "--memory=128m",   // Memory cap
    imageTag,
)
Enter fullscreen mode Exit fullscreen mode

Lessons Learned:

Process management in Go using exec.Command.

Security through isolation: Containers run without privileges or network access.

4. The API Layer

A REST API ties everything together. For example, submitting a function:

func submitFunctionHandler(w http.ResponseWriter, r *http.Request) {
    // Handle file upload, extract, build, and store metadata
    respondWithJSON(w, http.StatusCreated, map[string]interface{}{
        "message": fmt.Sprintf("Function '%s' deployed!", function.Name),
    })
}
Enter fullscreen mode Exit fullscreen mode

Design Choices:

  • Used Go standard library net package for routing.
  • Middleware for logging, timeout handling, and error recovery.

The Hardest Parts (And What I Learned)

🔒 Security Challenges

  • Zip Slip Vulnerability: A single path validation oversight could let attackers overwrite system files
  • Resource Limits: Implementing strict CPU/memory constraints via Docker to prevent abuse
  • Input Sanitization: Learned to never trust user input - always validate and sanitize

🐛 Debugging Nightmares

  • Exit Code 124: Spent hours diagnosing Docker's mysterious timeout error
  • Orphaned Containers: The costly lesson of forgetting --rm flag (thousands of dead containers later...)

What's Missing? (Future Improvements)

While functional, this isn't production-grade like AWS Lambda. Here's the roadmap:

  • Persistent Storage: Track function metadata and execution history
  • Authentication Layer: Implement OAuth or API key security
  • Auto-Scaling: Dynamic container provisioning based on workload
  • Extended Language Support: Add JavaScript, Rust, and more!

Why You Should Try a Project Like This

Building a serverless platform from scratch taught me:

  1. Systems Thinking: How file I/O, containers, and APIs work together
  2. The Hidden Cost of Serverless: Appreciation for the complexity abstracted away
  3. Production-Grade Resilience: Why robust error handling makes or breaks systems

Final Thoughts

This wasn't about building a better Lambda - it was about understanding the machinery behind the magic. If you're tired of surface-level tutorials, I challenge you to:

  • Pick a complex systems problem
  • Dive deeper than any tutorial would
  • Embrace the struggle (that's where real learning happens)

You'll gain:

  • Skills most tutorials never cover
  • Deep respect for the tools you use daily
  • Confidence to tackle any technical challenge

Ready to tinker? ➡️ Explore the code on GitHub

What's the most ambitious project you've built to level up your skills? Share your war stories below! 💬

MongoDB Atlas runs apps anywhere. Try it now.

MongoDB Atlas runs apps anywhere. Try it now.

MongoDB Atlas lets you build and run modern apps anywhere—across AWS, Azure, and Google Cloud. With availability in 115+ regions, deploy near users, meet compliance, and scale confidently worldwide.

Start Free

Top comments (0)

Gen AI apps are built with MongoDB Atlas

Gen AI apps are built with MongoDB Atlas

MongoDB Atlas is the developer-friendly database for building, scaling, and running gen AI & LLM apps—no separate vector DB needed. Enjoy native vector search, 115+ regions, and flexible document modeling. Build AI faster, all in one place.

Start Free

👋 Kindness is contagious

Sign in to DEV to enjoy its full potential.

Unlock a customized interface with dark mode, personal reading preferences, and more.

Okay