DeepDiveInfra – Self-Hosted CI/CD & Kubernetes Delivery Lab

Problem / Context

The problem I wanted to solve wasn't about domain logic. It was about understanding what happens after code is written: how a backend goes from source code to a tested build, to a container, to a deployed workload in Kubernetes, to something you can actually monitor. I built this project to work through each layer myself instead of just reading about it.

Solution / Architecture

The solution is a self-hosted local lab: a Windows host with WSL/Linux tooling, Docker Desktop, a kind cluster, GitHub Actions for build and test, a self-hosted runner for deployment, and Prometheus + Grafana for monitoring. The ASP.NET Core API uses SQLite and EF Core with automatic migrations on startup. The app itself remains intentionally small so the infrastructure and delivery chain stay in focus.

How It Works – Architecture Flow

1. 1Build a simple ASP.NET Core API with SQLite and EF Core
2. 2Push changes to GitHub
3. 3GitHub Actions restores, builds, and tests the solution
4. 4A self-hosted Linux runner picks up the deployment job
5. 5Docker packages the API as a deployable image
6. 6The image is deployed to a local kind Kubernetes cluster
7. 7Kubernetes runs the workload as pods/services
8. 8Prometheus scrapes app and cluster metrics
9. 9Grafana visualizes system health and runtime behavior

Goals

• Understand the full delivery chain end to end
• Compare GitHub-hosted and self-hosted runners
• Deploy a real containerized backend to Kubernetes
• Add monitoring and make the system observable
• Document architecture decisions and trade-offs

Challenges

• Understanding the boundaries between development, build, deploy, and runtime environments
• Configuring the self-hosted runner correctly on Linux/WSL
• Handling the Windows + WSL + Docker Desktop + kind stack
• Structuring Kubernetes manifests and deployment flow cleanly
• Exposing useful Prometheus and Grafana metrics without overcomplicating the project

Key Technical Decisions

• Keep the backend intentionally simple – the value is the delivery chain, not the domain
• Use SQLite because the goal is delivery-chain learning, not distributed data design
• Use kind for a local Kubernetes learning environment
• Use Prometheus/Grafana for practical observability
• Keep the project self-hosted and local instead of managed cloud

Results / Impact

• Working CI/CD flow from GitHub push to deployed Kubernetes workload on my local machine
• Self-hosted deployment runner configured in a Linux/WSL environment
• ASP.NET Core API running inside a kind Kubernetes cluster
• Metrics visible in Prometheus and Grafana dashboards
• A concrete infrastructure case showing that I understand the delivery chain beyond just writing code

What I Learned

This project made the difference between development, build, deploy, and runtime environments concrete for me. I learned why containers matter as deployment artifacts, what Kubernetes actually manages at the pod and service level, and how Prometheus and Grafana make even a simple system easier to understand. The biggest takeaway: infrastructure understanding is its own skill, separate from writing application code.

What I Would Improve Next

• Add CPU/memory usage metrics and alerting rules
• Introduce ingress for external access to the cluster
• Move from SQLite to a more production-like data setup
• Package manifests more cleanly with Helm or a similar approach