If you've ever felt stuck building yet another todo app and craved a project that dives into the guts of how systems work, you're in the right place. Let me take you through my journey of building a lightweight serverless platform from scratch—a project that taught me more about containerization, process management, and API design than any tutorial ever could.
Heads up: This is an educational project, not production-ready! Check out the full code here.
Why Build a Serverless Platform?
Serverless computing is everywhere, but how does it actually work under the hood? I wanted to peel back the layers and explore:
🛠️ Systems Programming Challenges: File handling, process isolation, security.
🧩 Practical Utility: Build something usable, not just theoretical.
📦 Modern Tech Stack: Docker, REST APIs, multi-language support.
Spoiler: It was harder—and more rewarding—than I expected.
How It Works: Breaking Down the Core Components
1. Handling User Code Safely
When users upload a zip file, the platform must extract it without risking security flaws like the infamous zip slip vulnerability:
// Simplified code snippet for secure zip extraction
filePath := filepath.Join(destDir, file.Name)
if !strings.HasPrefix(filePath, filepath.Clean(destDir)+string(os.PathSeparator)) {
return fmt.Errorf("illegal file path: %s", file.Name)
}
Key Takeaways:
- Validate every file path to prevent directory escapes.
- Use Go's
os
andfilepath
libraries for safe file operations.
2. Detecting Programming Languages
The platform supports Python and Go (for now!). A simple but effective approach:
// Check for Python files
pythonFiles, _ := filepath.Glob(filepath.Join(dir, "*.py"))
if len(pythonFiles) > 0 {
return "python", nil
}
Why This Matters:
- Introduces pattern matching and filesystem inspection.
- Lays groundwork for multi-language support.
3. Docker: The Heart of Isolation
Running untrusted code safely meant leaning into Docker. Here’s the workflow:
Build a Docker image from the user’s code.
Run containers with strict resource limits (CPU, memory, network).
// Simplified Docker execution with security constraints
cmd := exec.Command("docker", "run",
"--read-only", // No writes to filesystem
"--network=none", // No internet access
"--memory=128m", // Memory cap
imageTag,
)
Lessons Learned:
Process management in Go using exec.Command.
Security through isolation: Containers run without privileges or network access.
4. The API Layer
A REST API ties everything together. For example, submitting a function:
func submitFunctionHandler(w http.ResponseWriter, r *http.Request) {
// Handle file upload, extract, build, and store metadata
respondWithJSON(w, http.StatusCreated, map[string]interface{}{
"message": fmt.Sprintf("Function '%s' deployed!", function.Name),
})
}
Design Choices:
- Used Go standard library net package for routing.
- Middleware for logging, timeout handling, and error recovery.
The Hardest Parts (And What I Learned)
🔒 Security Challenges
- Zip Slip Vulnerability: A single path validation oversight could let attackers overwrite system files
- Resource Limits: Implementing strict CPU/memory constraints via Docker to prevent abuse
- Input Sanitization: Learned to never trust user input - always validate and sanitize
🐛 Debugging Nightmares
- Exit Code 124: Spent hours diagnosing Docker's mysterious timeout error
-
Orphaned Containers: The costly lesson of forgetting
--rm
flag (thousands of dead containers later...)
What's Missing? (Future Improvements)
While functional, this isn't production-grade like AWS Lambda. Here's the roadmap:
- Persistent Storage: Track function metadata and execution history
- Authentication Layer: Implement OAuth or API key security
- Auto-Scaling: Dynamic container provisioning based on workload
- Extended Language Support: Add JavaScript, Rust, and more!
Why You Should Try a Project Like This
Building a serverless platform from scratch taught me:
- Systems Thinking: How file I/O, containers, and APIs work together
- The Hidden Cost of Serverless: Appreciation for the complexity abstracted away
- Production-Grade Resilience: Why robust error handling makes or breaks systems
Final Thoughts
This wasn't about building a better Lambda - it was about understanding the machinery behind the magic. If you're tired of surface-level tutorials, I challenge you to:
- Pick a complex systems problem
- Dive deeper than any tutorial would
- Embrace the struggle (that's where real learning happens)
You'll gain:
- Skills most tutorials never cover
- Deep respect for the tools you use daily
- Confidence to tackle any technical challenge
Ready to tinker? ➡️ Explore the code on GitHub
What's the most ambitious project you've built to level up your skills? Share your war stories below! 💬
Top comments (0)