Deploying to Kubernetes

Aspire provides first-party support for deploying applications to Kubernetes through the Aspire Kubernetes hosting integration. This integration allows you to define your application's infrastructure using the familiar Aspire AppHost and then generate Kubernetes YAML manifests for deployment to any Kubernetes cluster.

Prerequisites

Before deploying to Kubernetes, ensure you have:

• Docker: Docker Desktop or Podman for building container images
• kubectl: The Kubernetes command-line tool
• Kubernetes cluster: Access to a Kubernetes cluster (minikube)

For local development with minikube:

minikube start

Understanding Kubernetes Deployment

When you deploy an Aspire application to Kubernetes, each component becomes a Kubernetes resource:

• Projects: Become Deployments with container specifications
• Container resources: PostgreSQL, Redis, RabbitMQ become StatefulSets or Deployments
• Connections: Become environment variables and ConfigMaps
• Secrets: Become Kubernetes Secrets

Step 1: Install the Kubernetes Hosting Package

Add the Kubernetes hosting package to your AppHost project:

dotnet add package Aspire.Hosting.Kubernetes

This package provides the Kubernetes publisher that transforms your Aspire manifest into Kubernetes YAML manifests.

Step 2: Add Kubernetes Environment to AppHost

In your AppHost's Program.cs, add a call to AddKubernetesEnvironment:

var builder = DistributedApplication.CreateBuilder(args);

// Enable Kubernetes publishing
builder.AddKubernetesEnvironment("cloud-native-workshop");

// Your existing resources
var mainDb = builder.AddPostgres("main-db")
    .WithDataVolume()
    .WithLifetime(ResourceLifetime.Persistent)
    .AddDatabase("dometrain");

var carts = builder.AddAzureCosmosDB("cosmosdb")
    .RunAsPreviewEmulator()
    .WithDataExplorer()
    .AddDatabase("cartdb")
    .AddContainer("carts", partitionKeyPath: "/pk");

var redis = builder.AddRedis("redis")
    .WithLifetime(ResourceLifetime.Persistent)
    .WithRedisInsight();

var rabbitmq = builder.AddRabbitMQ("rabbitmq")
    .WithLifetime(ResourceLifetime.Persistent)
    .WithManagementPlugin();

builder.AddProject<Projects.Dometrain_Monolith_Api>("dometrain-api")
    .WithReplicas(5)
    .WithReference(redis)
    .WithReference(mainDb)
    .WithReference(carts)
    .WithReference(rabbitmq);

builder.Build().Run();

Step 3: Generate Kubernetes Manifests

Generate Kubernetes YAML manifests using the aspire publish command:

aspire publish --publisher kubernetes -o ./k8s-manifests

Or use the interactive mode:

aspire publish

Then select "kubernetes" when prompted for the publisher type.

This command:

1. Analyzes your AppHost project
2. Generates the Aspire manifest
3. Transforms the manifest into Kubernetes YAML files
4. Outputs the files to the ./k8s-manifests directory

What Gets Generated

The generated directory contains:

k8s-manifests/
├── deployment-dometrain-api.yaml
├── deployment-main-db.yaml
├── deployment-redis.yaml
├── deployment-rabbitmq.yaml
├── deployment-cosmosdb.yaml
├── service-dometrain-api.yaml
├── service-main-db.yaml
├── service-redis.yaml
├── service-rabbitmq.yaml
├── service-cosmosdb.yaml
├── configmap-*.yaml
└── secret-*.yaml

Each resource includes:

• Deployments/StatefulSets: Pod specifications with container images and resource limits
• Services: Network endpoints for service-to-service communication
• ConfigMaps: Application configuration and connection strings
• Secrets: Sensitive data like passwords and API keys

Step 4: Build and Push Container Images

Before deploying, you need to build container images for your .NET projects and make them available to your Kubernetes cluster.

Option A: Using Minikube (Local Development)

If using minikube, point your Docker CLI to minikube's Docker daemon:

eval $(minikube docker-env)

Then build your container images:

docker build -t dometrain-api:latest -f src/Dometrain.Monolith.Api/Dockerfile .

Alternatively, load pre-built images into minikube:

minikube image load dometrain-api:latest

Option B: Using a Container Registry

For production or cloud Kubernetes clusters, push images to a container registry:

# Tag your image
docker tag dometrain-api:latest myregistry.azurecr.io/dometrain-api:latest

# Push to registry
docker push myregistry.azurecr.io/dometrain-api:latest

Update the image references in your generated YAML files to match the registry path.

Step 5: Deploy to Kubernetes

Deploy all resources to your Kubernetes cluster:

kubectl apply -f ./k8s-manifests/

Or deploy individual resources:

kubectl apply -f ./k8s-manifests/deployment-dometrain-api.yaml
kubectl apply -f ./k8s-manifests/service-dometrain-api.yaml

Verify Deployment

Check the status of your deployments:

kubectl get deployments
kubectl get pods
kubectl get services

View logs for a specific pod:

kubectl logs -l app=dometrain-api

Follow logs in real-time:

kubectl logs -l app=dometrain-api -f

Step 6: Access Your Application

Port Forwarding (Development)

For local testing, use port forwarding to access your services:

kubectl port-forward service/dometrain-api 8080:80

Then access your API at http://localhost:8080.

LoadBalancer Service (Production)

For cloud Kubernetes clusters, change the service type to LoadBalancer in the generated YAML:

apiVersion: v1
kind: Service
metadata:
  name: dometrain-api
spec:
  type: LoadBalancer  # Change from ClusterIP
  ports:
    - port: 80
      targetPort: 8080
  selector:
    app: dometrain-api

Then get the external IP:

kubectl get service dometrain-api

Ingress Controller (Recommended)

For production, use an Ingress controller to expose your services:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: dometrain-ingress
spec:
  rules:
    - host: api.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: dometrain-api
                port:
                  number: 80

Managing Your Deployment

Updating Your Application

After making code changes:

1. Rebuild your container images
2. Push updated images to the registry (or reload to minikube)
3. Restart the deployment:

kubectl rollout restart deployment/dometrain-api

Scaling Services

Scale your API to more replicas:

kubectl scale deployment/dometrain-api --replicas=10

Or edit the YAML file and reapply:

spec:
  replicas: 10

kubectl apply -f ./k8s-manifests/deployment-dometrain-api.yaml

Viewing Resource Usage

Check resource consumption:

kubectl top pods
kubectl top nodes

Cleaning Up Resources

Delete all resources:

kubectl delete -f ./k8s-manifests/

Or delete the entire namespace:

kubectl delete namespace default

For minikube, stop the cluster:

minikube stop
minikube delete

Limitations and Considerations

Cloud-managed resources: Azure-specific resources need special handling:

• Cosmos DB with Preview Emulator: Will be deployed as a container in Kubernetes
• Azure Cosmos DB (production): You'll need to provide connection strings to actual Azure resources via Kubernetes Secrets
• Azure Service Bus: Either use RabbitMQ in Kubernetes or connect to Azure Service Bus

Persistent storage: Ensure your cluster has a StorageClass configured for PersistentVolumeClaims (used by PostgreSQL, etc.).

Service replicas: The .WithReplicas(5) configuration translates to replicas: 5 in the Kubernetes Deployment.

Resource limits: Consider adding resource requests and limits to prevent resource starvation:

resources:
  requests:
    memory: "512Mi"
    cpu: "250m"
  limits:
    memory: "1Gi"
    cpu: "500m"

Advanced: Helm Charts

For more sophisticated deployments, consider using Helm. You can customize the generated manifests or create Helm charts from them:

helm create cloud-native-workshop
# Copy generated manifests to templates/
helm install my-app ./cloud-native-workshop

Alternative: Using Aspir8

Aspir8 is a community tool that simplifies Aspire-to-Kubernetes deployments with additional features like automatic Helm chart generation. While not covered in this workshop, it's worth exploring for production scenarios.

Best Practices

1. Use namespaces: Isolate different environments (dev, staging, prod) using Kubernetes namespaces

2. Configure health checks: Add liveness and readiness probes to your Deployments

3. Secure secrets: Use Kubernetes Secrets or external secret managers (Azure Key Vault, HashiCorp Vault)

4. Set resource limits: Always define resource requests and limits

5. Use Ingress: Don't expose services directly; use an Ingress controller

6. Monitor your cluster: Set up Prometheus and Grafana for observability

7. GitOps: Store your manifests in Git and use tools like ArgoCD or Flux for deployment

Troubleshooting

Pods not starting: Check pod events and logs

kubectl describe pod <pod-name>
kubectl logs <pod-name>

ImagePullBackOff: Ensure images are available to the cluster (loaded to minikube or pushed to registry)

CrashLoopBackOff: Application is crashing on startup. Check logs and environment configuration

Service not accessible: Verify service selectors match pod labels

kubectl get endpoints

Next Steps

Once deployed to Kubernetes, you can:

• Set up Horizontal Pod Autoscaling (HPA)
• Configure persistent storage with PersistentVolumes
• Implement network policies for security
• Set up monitoring with Prometheus and Grafana
• Deploy to managed Kubernetes services (AKS, EKS, GKE)
• Implement CI/CD pipelines for automated deployments

For more information, see the Aspire Kubernetes hosting integration documentation.