KB: Kubernetes runs once and exits POD

In Kubernetes, the usual pattern for a container that “runs once and exits” is not a Deployment but rather a Job (or a CronJob for scheduled runs). A Deployment is designed to keep a given number of replicas running continuously, restarting pods if they exit. By contrast, a Job is specifically designed for run-to-completion workloads. Here’s why this matters and how you can set it up:

---

Why not a Deployment for “run once, then exit”?

• Deployments expect long-running pods. If your container exits successfully, the Deployment’s ReplicaSet controller will see that the number of running pods is below the desired replica count (e.g., 1), so it will spin up a new pod. You’ll end up in an endless cycle of your “job” container repeatedly starting and exiting.
• Deployments do not retain “completed” pods. When a pod in a Deployment terminates, Kubernetes treats it as “unavailable” and immediately attempts to create a new one.

---

The right approach: a Job (or CronJob)

Jobs create pods that run until they exit (succeed or fail). You can configure how often to retry, whether to keep pods after they succeed, and so on. That’s the standard approach for “one-off” tasks.

CronJobs are essentially “Jobs on a schedule.” You can configure them to run once a day, every hour, etc. But if you only want to start them on-demand, a plain Job is usually simpler.

Example Job Manifest

Below is a minimal example of a Job that runs a single pod to completion and does not restart on success. It also retains the pod so you can inspect its logs/state as long as you do not clean it up:

apiVersion: batch/v1
kind: Job
metadata:
  name: my-run-once-job
spec:
  template:
    spec:
      restartPolicy: Never
      containers:
      - name: my-run-once-container
        image: alpine:3.18
        command: ["sh", "-c", "echo 'Hello, run once.' && sleep 5 && exit 0"]
  backoffLimit: 0   # Do not retry on failure

• restartPolicy: Never: Ensures that Kubernetes will not restart the pod within the Job if it completes.
• backoffLimit: 0: Prevents retries if the container exits with a non-zero status (you can adjust if you do want retries).

When you run kubectl apply -f my-job.yaml, this job will create a single pod, run echo 'Hello, run once.' && sleep 5 && exit 0, and then exit. The status of that pod should become Completed.

Keeping Completed Pods Around

If you want to ensure the pod isn’t garbage collected quickly, make sure you do not set the ttlSecondsAfterFinished field in your Job spec (or set it to null). By default, Kubernetes 1.21+ supports cleaning up old completed pods if ttlSecondsAfterFinished is configured. If it’s unset, the completed pod will remain until you manually delete the Job (or you have some cluster-level cleanup).

Manual Reruns

After the pod completes, you can choose to:

1. Delete the Job and re-create it when you want to run the container again:

   kubectl delete job my-run-once-job
   kubectl apply -f my-job.yaml
   

2. Create a new Job each time (with a unique name). For example:

   kubectl create job my-run-once-job-2 --from=job/my-run-once-job
   

3. Or use a CronJob that sets a schedule for repeats, though you can also manually trigger CronJobs outside of their schedule (less common for purely on-demand runs).

---

Summary

• You can hack a Deployment to do “run once then exit,” but it goes against how Deployments are intended to work. You’d be fighting the ReplicaSet controller that constantly tries to keep a certain number of pods running.
• The best practice for a “run once” or ephemeral job is to use a Job (or a CronJob for scheduled runs), set restartPolicy to Never, and tune backoffLimit and ttlSecondsAfterFinished as needed.
• That way, once the pod completes, it won’t get restarted automatically, and you’ll be able to keep or clean up the completed pod as you see fit. When you need to run it again, you can just create or re-run the Job.

This approach cleanly fits the “once and done” model and allows you to inspect logs, see the exit status, and re-run on-demand without having to fight the Deployment’s logic.