Kubernetes: How to manually run a schedulated job on demand

Introduction

As the DevOps landscape continuously evolves, Kubernetes remains a front-runner in orchestrating containerized applications. Not just a tool for automating deployment and scaling, Kubernetes also offers robust job scheduling capabilities through CronJobs. However, a frequent requirement that surfaces for DevOps engineers and Kubernetes operators is the ability to run these scheduled jobs manually on demand, without waiting for the next scheduled interval. This tutorial provides detailed instructions on how to achieve such functionality.

Understanding CronJobs in Kubernetes

Before diving into the manual execution of scheduled jobs, it’s essential to understand what a CronJob is. In Kubernetes, a CronJob manages time-based jobs, akin to cron in UNIX-like systems. Here’s a basic example of a CronJob definition:

apiVersion: batch/v1beta1
kind: CronJob
metadata:
  name: example-cronjob
spec:
  schedule: "0 1 * * *"
  jobTemplate:
    spec:
      template:
        spec:
          containers:
          - name: hello
            image: busybox
            args:
            - /bin/sh
            - -c
            - date; echo Hello from the Kubernetes cluster
          restartPolicy: OnFailure

This CronJob will execute the ‘hello’ container at 1AM every day. However, if we need to run this job right now, we’ll need to create a Job resource based on this CronJob’s template.

Manually Running a Scheduled Job

To manually run a CronJob, we’ll create a Job from its definition. Consider the following steps:

1. First, obtain the YAML definition of the CronJob’s template:

kubectl get cronjob example-cronjob -o jsonpath='{.spec.jobTemplate}' > manual-job.yaml

The command extracts the job template and writes it to a file manual-job.yaml. Next, we will use this file to create an ad-hoc Job.

2. Create a Job in Kubernetes using the template we just retrieved. Modify the file manual-job.yaml to change the kind from CronJob to Job, and then apply it:

sed -i 's/CronJob/Job/' manual-job.yaml
kubectl apply -f manual-job.yaml

If you’re not using a Unix-like system that supports sed, you’ll need to manually edit manual-job.yaml and change the kind field.

Once the Job is created, Kubernetes will start running the specified container immediately.

Creating a Job from a CronJob with a Unique Name

If you run the same CronJob manually more than once, you need to specify distinct names for each Job. You can achieve this by editing manual-job.yaml to set a unique metadata.name for each Job instance.

kubectl get cronjob example-cronjob -o jsonpath='{.spec.jobTemplate}' > manual-job.yaml
sed -i 's/CronJob/Job/' manual-job.yaml
sed -i 's/name: example-cronjob/name: example-cronjob-manual-$(date +%s)/' manual-job.yaml
kubectl apply -f manual-job.yaml

This snippet appends the current Unix timestamp to the job’s name, ensuring uniqueness,

Handling Job Concurrency

In situations where the CronJob has concurrency policies, running a manual job can cause conflicts. To handle this gracefully, edit the concurrency policy in your YAML file:

spec:
  concurrencyPolicy: Replace

This setting will ensure that if the CronJob is due to run while our manual Job is still executing, Kubernetes will terminate the manual Job and start the scheduled one, ensuring no overlap as per the Replace policy.

Advanced Usage with Kubectl-Create Job

For more advanced users, Kubernetes provides the kubectl create job command which streamlines the process of running a CronJob manually without having to extract the job template. The basic syntax is as follows:

kubectl create job --from=cronjob/example-cronjob example-manual-job

Here, --from is used to specify the name of the CronJob followed by the desired name for the manual Job. Kubernetes takes care of the rest, creating and starting the job immediately.

Error Handling & Job Logs

Error handling and reviewing logs are critical during Job execution, more so while manually triggering jobs. To fetch the log of the most recent job, you can use:

kubectl get pods --selector=job-name=example-manual-job --output=jsonpath='{.items[*].metadata.name}' | xargs kubectl logs

Make sure to replace example-manual-job with the name of your manual job. This command will return the logs of any pods spun up by the Job, providing insight into the Job’s execution.

Conclusion

In this tutorial, we’ve covered the essentials of running Kubernetes CronJobs manually at any given time. The capability to trigger time-based jobs on demand provides Kubernetes users with flexibility and control, ensuring that critical tasks are executed promptly without being confined to the schedule. Mastering this technique is a valuable tool in any Kubernetes operator’s arsenal.