Skip to main content

Non-Web Workloads

The Walkthrough builds up a web service, but very little of the platform is web-specific. This page covers the three shapes that aren't: a long-running process that serves no traffic, something that runs to completion on a schedule or an event, and a pod that needs more than one container.

Choosing a Kind​

What you're runningKindWho creates the workload
Long-running process serving HTTP/gRPCPlatformApplicationOperator (Deployment / StatefulSet)
Long-running process serving no traffic — queue consumer, poller, stream processorPlatformApplication (worker profile)Operator (Deployment)
Runs to completion on a schedule — cron, nightly ETL, reportPlatformTaskYou (CronJob)
Runs to completion on an event — queue message, webhookPlatformTaskYou (Argo Workflow, via a Sensor)
Needs more than one container in the podPlatformTaskYou (Deployment)

The rule of thumb: PlatformApplication if the process runs forever; PlatformTask if it runs to completion, or if you need a pod spec the CRD doesn't model.

Both kinds live in the same API group, meta.p6m.dev/v1alpha1, and both are reconciled by the Platform Application Operator. Both give you the same namespace envelope — config, secrets, cloud identity, cloud resources, and networking policy. The difference is whether the operator also runs your container.

Background Services (the Worker Profile)​

PlatformApplication does not require a web server. A background service leaves out readinessProbe and networking.ingress, but — counterintuitively — still declares a port:

.platform/kubernetes/base/application.yaml
apiVersion: meta.p6m.dev/v1alpha1
kind: PlatformApplication
metadata:
  name: order-consumer
  namespace: order-consumer
  labels:
    p6m.dev/app: order-consumer
spec:
  config:
    LOG_LEVEL: info
    QUEUE_NAME: orders
  secrets:
    - name: order-consumer-secrets
  deployment:
    image: order-consumer-server:latest
    # Declare a port even though nothing listens on it — see the warning below.
    ports:
      - port: 8080
        protocol: http
    resources:
      requests:
        memory: "256Mi"
        cpu: "100m"
      limits:
        memory: "512Mi"
        cpu: "500m"

What this profile gives you:

  • No readinessProbe — the operator injects no probe at all, and the pod is Ready as soon as it starts. This is the correct behavior for a headless process: you do not need to add an HTTP server just to satisfy a health check.
  • No networking.ingress — the Service objects are ClusterIP only and nothing is reachable from outside the cluster.
Declare a port even when nothing listens on it

This is the one piece of ceremony a worker can't skip. The operator always reconciles three Service objects (main, canary, and stable). A Kubernetes Service requires at least one port, so if you omit deployment.ports the API server rejects all three with spec.ports: Required value, and the PlatformApplication reports ResourcesNotReady forever — even though your Deployment is healthy and the pod is running. In ArgoCD the app never converges.

Declaring a nominal port fixes it. Nothing has to be listening on that port: the port makes the Service valid, and because there's no readinessProbe the pod's readiness doesn't depend on anything answering there.

readinessProbe is required when ingress is enabled

The probe doubles as the load balancer health check. If you turn on networking.ingress, you must also set deployment.readinessProbe — which in turn means your container needs a real HTTP endpoint.

Scaling a Worker​

By default the operator creates a HorizontalPodAutoscaler that scales on CPU, which is rarely the right signal for a queue consumer — a backlog of a million messages doesn't necessarily move CPU. Declaring spec.autoscaling.triggers switches the workload to KEDA, so you can scale on the metric that actually matters: queue depth, topic lag, or a time window.

spec:
  autoscaling:
    maxReplicas: 20
    triggers:
      - type: aws-sqs-queue
        metadata:
          queueURL: https://sqs.us-east-2.amazonaws.com/000000000000/orders
          queueLength: "50"
          awsRegion: us-east-2

The example above leaves minReplicas unset, which matters more for a worker than for anything else: on the trigger path it defaults to 0, so an idle consumer scales all the way down to no pods at all. That is usually what you want for a queue drainer — set a floor if it isn't.

A queue scaler needs a credential the manifest cannot carry

KEDA reads the queue depth from its own pod, not from inside your worker, so the cloud identity attached to your pods does not reach it. KEDA authenticates through a TriggerAuthentication object, and spec.autoscaling.triggers has nowhere to name one. Treat the YAML above as the shape a queue trigger takes, and contact Ybor support before depending on one — see Trigger Credentials.

Triggers also replace the CPU autoscaler rather than supplementing it, every metadata value has to be a quoted string, and the available scalers depend on your cluster. Autoscaling covers all of that, along with the ScaledObject the operator creates on your behalf.

Scheduled and Event-Driven Tasks​

Use PlatformTask (shortname pat) for anything that runs to completion.

PlatformTask creates no workload

This is the single most important thing to know about it. PlatformTask provisions the namespace envelope and stops there. It does not create a Deployment, Service, CronJob, Job, HorizontalPodAutoscaler, or Ingress. You supply the workload yourself as a plain Kubernetes manifest alongside it.

What the operator does reconcile from a PlatformTask:

FromResources created
alwaysServiceAccount named after the task, plus a RoleBinding
spec.configConfigMap named <task-name>-task-config
spec.secretsSecretStore, ExternalSecret (and on Azure, the Key Vault)
cloud identityAWS IRSA role, or Azure WorkloadIdentity
spec.resourcesAccess grants only — IAM policy attachments (AWS) or role assignments (Azure)
spec.networkingIstio Sidecar, PeerAuthentication, ServiceEntry
spec.argoArgo Events EventSource + Sensor, plus IRSA for the event path
spec.resources does not provision anything on a PlatformTask

On a PlatformApplication, spec.resources creates the database, queue, or bucket. On a PlatformTask it does not. The task controller reads the block only to work out what its identity needs access to, and then grants exactly that — the resource itself has to already exist.

So resources.crdb on a task will not get you a database. Provision the resource from the PlatformApplication that owns it, then grant your task access to it by name.

Fields That Aren't What They Look Like​

Two more things to know before you write the manifest:

  • There is no scheduling field. PlatformTask has exactly six spec fields: config, secrets, resources, networking, argo, and metaflow. No schedule, type, trigger, runtime, or image. The schedule belongs on your own CronJob. Invent a field and the apply is rejected — kubectl apply returns strict decoding error: unknown field "spec.schedule", and ArgoCD's server-side apply returns field not declared in schema. The error names every offending field, so trust it and delete them rather than reaching for --validate=false, which makes the apply succeed by silently dropping them and leaves you with a task that provisions nothing.
  • networking.ingress has no effect. The schema carries the same networking block as PlatformApplication, so the reference page lists ingress fields, but the task controller doesn't act on them. No Ingress, HTTPRoute, or certificate is created. If you need an HTTP endpoint, you need a PlatformApplication.

Worked Example: a Scheduled CronJob​

The PlatformTask and the CronJob live side by side in the same Kustomize base, so ArgoCD applies them together:

.platform/kubernetes/base/
├── application.yaml      ← PlatformTask (identity, secrets, config)
├── cronjob.yaml          ← the actual workload
└── kustomization.yaml
.platform/kubernetes/base/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
  - application.yaml
  - cronjob.yaml
.platform/kubernetes/base/application.yaml
apiVersion: meta.p6m.dev/v1alpha1
kind: PlatformTask
metadata:
  name: nightly-reconciliation
  labels:
    p6m.dev/app: nightly-reconciliation
spec:
  secrets:
    - name: nightly-reconciliation-secrets
.platform/kubernetes/base/cronjob.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-reconciliation
  labels:
    p6m.dev/app: nightly-reconciliation
spec:
  schedule: "0 2 * * *" # patched per environment in the overlays
  concurrencyPolicy: Forbid
  successfulJobsHistoryLimit: 3
  failedJobsHistoryLimit: 3
  jobTemplate:
    spec:
      backoffLimit: 0
      activeDeadlineSeconds: 10800
      template:
        spec:
          # Must match metadata.name of the PlatformTask — this is the
          # ServiceAccount the operator created, carrying the cloud identity.
          serviceAccountName: nightly-reconciliation
          imagePullSecrets:
            - name: dockerconfig
          restartPolicy: Never
          containers:
            - name: reconciler
              # Short name; the Kustomize images transformer rewrites this to the
              # full registry path + digest from the .platform repo.
              image: nightly-reconciliation-server:latest
              command: ["python", "-m", "reconciler.main"]
              env:
                - name: DATABASE_URL
                  valueFrom:
                    secretKeyRef:
                      name: nightly-reconciliation-secrets
                      key: DATABASE_URL
              resources:
                requests:
                  cpu: 100m
                  memory: 256Mi
                limits:
                  cpu: 500m
                  memory: 512Mi

Override the schedule per environment the same way you patch a PlatformApplication:

.platform/kubernetes/dev/cronjob_patch.yaml
apiVersion: batch/v1
kind: CronJob
metadata:
  name: nightly-reconciliation
spec:
  schedule: "0 */6 * * *" # run more often in dev

Common Pitfalls​

Because you own the pod spec, a few things the operator would normally handle for a PlatformApplication are now yours to get right.

  • serviceAccountName must match the PlatformTask name. The operator names the ServiceAccount after the task's metadata.name. If it doesn't match, the pod silently falls back to the namespace default ServiceAccount, which carries no cloud identity — secret and database access then fail in a way that looks like a credentials problem rather than a manifest problem.

  • Add imagePullSecrets: dockerconfig. Registry credentials live in a namespace-level secret. A PlatformApplication pod gets this reference added automatically; a hand-written pod spec does not, and the symptom is ImagePullBackOff.

  • Mind the ConfigMap name. spec.config becomes a ConfigMap named <task-name>-task-config — note the -task- infix. Reference it explicitly if you want those values in your container:

    envFrom:
      - configMapRef:
          name: nightly-reconciliation-task-config

  • Bound the run. backoffLimit defaults to 6, so a job that fails immediately still retries six times. Set backoffLimit: 0 when a retry cannot help, and set activeDeadlineSeconds so a hung run can't hold the schedule slot indefinitely.

  • Set concurrencyPolicy: Forbid unless overlapping runs are genuinely safe. The default is Allow, which will start a second run on top of a slow first one.

  • Rule out mesh interference early. The platform runs Istio in ambient mode, so no sidecar is injected into your pod and the well-known "sidecar keeps the Job alive forever" problem does not apply here. Ambient mode does route traffic through ztunnel, and outbound HTTPS from short-lived pods has needed attention before. If a task's egress fails where a PlatformApplication in the same namespace succeeds, take it to the platform team rather than working around it in your manifest.

Operating a Task​

# Trigger a run now, off the existing schedule
kubectl create job nightly-reconciliation-manual \
  --from=cronjob/nightly-reconciliation -n nightly-reconciliation

# What ran, and when
kubectl get cronjob nightly-reconciliation -n nightly-reconciliation
kubectl get jobs -n nightly-reconciliation --sort-by=.status.startTime

# Logs from a run (JOB_NAME comes from the previous command)
kubectl logs -n nightly-reconciliation -l job-name=JOB_NAME --tail=200

Event-Driven Tasks​

To trigger work from a queue rather than a clock, declare the source and the trigger under spec.argo and the operator wires up Argo Events for you — an EventSource that watches the queue, a Sensor that submits your workflow, and the IRSA role the event path needs.

The queue itself must already exist — as above, spec.resources here grants the task access to it, it does not create it:

apiVersion: meta.p6m.dev/v1alpha1
kind: PlatformTask
metadata:
  name: order-processor
spec:
  resources:
    sqs:
      - name: orders
        accesses: [all]
  argo:
    sources:
      sqs:
        - name: orders
          queueName: orders
          region: us-west-2
    triggers:
      - template:
          name: sqs
          conditions: "sqs-orders"
          argoWorkflow:
            operation: submit
            source:
              resource:
                apiVersion: argoproj.io/v1alpha1
                kind: Workflow
                metadata:
                  generateName: order-
                spec:
                  entrypoint: process
                  # ... workflow templates

SQS is the only source type modeled today. Trigger bodies are passed through to Argo Events unvalidated, so consult the Argo Events trigger docs for the shape. Field details are in spec.argo.

Multiple Containers and Sidecars​

spec.deployment on a PlatformApplication models exactly one container. The complete set of fields is kind, image, ports, readinessProbe, resources, readOnlyRootFilesystem, volumeMounts, args, volumeClaimTemplates, podManagementPolicy, and fsGroup. There is no containers[], no initContainers, no command, and no per-container env (environment comes from spec.config and spec.secrets).

If you need a second container in the pod, you have two options:

  1. Combine the processes into one image. Preferred where it's feasible — you keep rollouts, autoscaling, probes, Service, and ingress all managed by the operator.
  2. Use a PlatformTask plus your own Deployment. The task provisions identity, config, and secrets; you write the full pod spec and own it. This is the same pattern as the CronJob example above, with a Deployment in place of the CronJob.
Two different meanings of "sidecar"

Both CRDs create an Istio Sidecar resource from spec.networking. That is a mesh configuration object that scopes egress for the workload — it is not an application container, and it does not give you a second container in your pod.

What the Operator No Longer Manages​

When you bring your own workload manifest, the operator stops managing that part of the stack. You take over:

  • Rollout strategy — no canary or stable Service, no spec.rollouts
  • HorizontalPodAutoscaler and VerticalPodAutoscaler
  • Default probes, security context, and node/architecture selectors
  • Service, Ingress / HTTPRoute, and certificates

Config, secrets, cloud identity, and cloud resources still come from the CRD, so this is a partial handoff rather than an all-or-nothing one.