Operation

This is an alpha feature. Crossplane may change or drop this feature at any time.

This feature was introduced in v2.

For more information read the Crossplane feature lifecycle.

This document is for an unreleased version of Crossplane.

This document applies to the Crossplane master branch and not to the latest release v1.20.

An Operation runs a function pipeline once to completion to perform operational tasks that don’t fit the typical resource creation pattern. Unlike compositions that continuously reconcile desired state, Operations focus on tasks like backups, rolling upgrades, configuration validation, and scheduled maintenance.

How operations work

Operations are like Kubernetes Jobs - they run once to completion rather than continuously reconciling. Like compositions, Operations use function pipelines to implement their logic, but they’re designed for operational workflows instead of resource composition.

 1apiVersion: ops.crossplane.io/v1alpha1
 2kind: Operation
 3metadata:
 4  name: backup-database
 5spec:
 6  mode: Pipeline
 7  pipeline:
 8  - step: create-backup
 9    functionRef:
10      name: function-database-backup
11    input:
12      apiVersion: fn.crossplane.io/v1beta1
13      kind: DatabaseBackupInput
14      database: production-db
15      retentionDays: 30

When you create this Operation, Crossplane:

Validates the operation and its function dependencies
Executes the function pipeline step by step
Applies any resources the functions create or change
Updates the Operation status with results and completion state

Important

Operations are an alpha feature. You must enable them by adding --enable-operations to Crossplane’s arguments.

Key characteristics

Runs once to completion (like Kubernetes Jobs)
Uses function pipelines (like Compositions)
Can create or change any Kubernetes resources
Provides detailed status and output from each step
Supports retry on failure with configurable limits

Operation functions vs composition functions

Operations and compositions both use function pipelines, but with important differences:

Composition Functions:

Purpose: Create and maintain resources
Lifecycle: Continuous reconciliation
Input: Observed composite resources
Output: Desired composed resources
Ownership: Creates owner references

Operation Functions:

Purpose: Perform operational tasks
Lifecycle: Run once to completion
Input: Required resources only
Output: Any Kubernetes resources
Ownership: Force applies without owners

Functions can support both modes by declaring the appropriate capabilities in their package metadata. Function authors declare this in the crossplane.yaml file when building the function package:

1apiVersion: meta.pkg.crossplane.io/v1
2kind: Function
3metadata:
4  name: my-function
5spec:
6  capabilities:
7  - composition
8  - operation

This allows Crossplane to know which modes the function supports and avoid trying to use a composition-only function for operations.

Common use cases

Note

The following examples use hypothetical functions for illustration. At launch, only function-python supports operations.

Rolling upgrades

Use Operations for controlled rolling upgrades:

 1apiVersion: ops.crossplane.io/v1alpha1
 2kind: Operation
 3metadata:
 4  name: cluster-upgrade
 5spec:
 6  mode: Pipeline
 7  pipeline:
 8  - step: rolling-upgrade
 9    functionRef:
10      name: function-cluster-upgrade
11    input:
12      apiVersion: fn.crossplane.io/v1beta1
13      kind: ClusterUpgradeInput
14      targetVersion: "1.28"
15      batches: [0.25, 0.5, 1.0]  # 25%, 50%, then 100%
16      healthChecks: [Synced, Ready]

One-time maintenance

Use Operations for specific maintenance tasks:

 1apiVersion: ops.crossplane.io/v1alpha1
 2kind: Operation
 3metadata:
 4  name: certificate-rotation
 5spec:
 6  mode: Pipeline
 7  pipeline:
 8  - step: rotate-certificates
 9    functionRef:
10      name: function-cert-rotation
11    input:
12      apiVersion: fn.crossplane.io/v1beta1
13      kind: CertRotationInput
14      targetCertificates:
15        matchLabels:
16          rotate: "true"

Advanced configuration

Retry behavior

Operations automatically retry when they fail. Configure the retry limit to control how often attempts occur:

 1apiVersion: ops.crossplane.io/v1alpha1
 2kind: Operation
 3metadata:
 4  name: resilient-operation
 5spec:
 6  retryLimit: 10  # Try up to 10 times before giving up (default: 5)
 7  mode: Pipeline
 8  pipeline:
 9  - step: flaky-task
10    functionRef:
11      name: function-flaky-task
12    input:
13      apiVersion: fn.crossplane.io/v1beta1
14      kind: FlakyTaskInput
15      # Task that might fail due to temporary issues
16      timeout: "30s"

Retry behavior:

Each retry resets the entire pipeline - if step 2 of 3 fails, the retry starts from step 1
Operations use exponential backoff: 1 s, 2 s, 4 s, 8 s, 16 s, 32 s, then 60 s max
Operations track the number of failures in status.failures
After reaching retryLimit, the Operation becomes Succeeded=False

Credentials

Operations can provide credentials to functions through Secrets:

 1apiVersion: ops.crossplane.io/v1alpha1
 2kind: Operation
 3metadata:
 4  name: secure-backup
 5spec:
 6  mode: Pipeline
 7  pipeline:
 8  - step: backup-with-credentials
 9    functionRef:
10      name: function-backup
11    credentials:
12    - name: backup-creds
13      source: Secret
14      secretRef:
15        namespace: crossplane-system
16        name: backup-credentials
17        key: api-key
18    - name: database-creds
19      source: Secret
20      secretRef:
21        namespace: crossplane-system
22        name: database-credentials
23        key: connection-string
24    input:
25      apiVersion: fn.crossplane.io/v1beta1
26      kind: BackupInput
27      destination: s3://my-backup-bucket

Multiple pipeline steps

Complex operations can use multiple pipeline steps:

 1apiVersion: ops.crossplane.io/v1alpha1
 2kind: Operation
 3metadata:
 4  name: multi-step-deployment
 5spec:
 6  mode: Pipeline
 7  pipeline:
 8  - step: validate-config
 9    functionRef:
10      name: function-validator
11    input:
12      apiVersion: fn.crossplane.io/v1beta1
13      kind: ValidatorInput
14      configName: app-config
15  - step: backup-current
16    functionRef:
17      name: function-backup
18    input:
19      apiVersion: fn.crossplane.io/v1beta1
20      kind: BackupInput
21      target: current-deployment
22  - step: deploy-new-version
23    functionRef:
24      name: function-deploy
25    input:
26      apiVersion: fn.crossplane.io/v1beta1
27      kind: DeployInput
28      image: myapp:v2.0.0
29      strategy: rollingUpdate
30  - step: verify-health
31    functionRef:
32      name: function-health-check
33    input:
34      apiVersion: fn.crossplane.io/v1beta1
35      kind: HealthCheckInput
36      timeout: 300s
37      healthEndpoint: /health

RBAC permissions

If your Operation needs to access resources that Crossplane doesn’t have permissions for by default, create a ClusterRole that aggregates to Crossplane:

 1apiVersion: rbac.authorization.k8s.io/v1
 2kind: ClusterRole
 3metadata:
 4  name: operation-additional-permissions
 5  labels:
 6    rbac.crossplane.io/aggregate-to-crossplane: "true"
 7rules:
 8# Additional permissions for Operations
 9- apiGroups: ["networking.k8s.io"]
10  resources: ["ingresses"]
11  verbs: ["get", "list", "patch", "update"]
12- apiGroups: [""]
13  resources: ["persistentvolumes"]
14  verbs: ["get", "list"]
15# Add other resources your Operations need to access

This ClusterRole is automatically aggregated to Crossplane’s main ClusterRole, giving the Crossplane service account the permissions needed for your Operations.

Note

The RBAC manager automatically grants Crossplane access to Crossplane resources (MRs, XRs, etc.). You only need to create more ClusterRoles for other Kubernetes resources that your Operations need to access.

For more details on RBAC configuration, see the Compositions RBAC documentation.

Required resources

Operations can preload resources for functions to access:

 1apiVersion: ops.crossplane.io/v1alpha1
 2kind: Operation
 3metadata:
 4  name: resource-aware-operation
 5spec:
 6  mode: Pipeline
 7  pipeline:
 8  - step: process-deployment
 9    functionRef:
10      name: function-processor
11    requirements:
12      requiredResources:
13      - requirementName: app-deployment
14        apiVersion: apps/v1
15        kind: Deployment
16        name: my-app
17        namespace: production
18      - requirementName: app-service
19        apiVersion: v1
20        kind: Service
21        name: my-app-service
22        namespace: production
23    input:
24      apiVersion: fn.crossplane.io/v1beta1
25      kind: ProcessorInput
26      action: upgrade

Functions access these resources through the standard request structure:

 1from crossplane.function import request, response
 2
 3def operate(req, rsp):
 4    # Access required resources
 5    deployment = request.get_required_resource(req, "app-deployment")
 6    service = request.get_required_resource(req, "app-service")
 7    
 8    if not deployment or not service:
 9        response.set_output(rsp, {"error": "Required resources not found"})
10        return
11    
12    # Process the resources
13    new_replicas = deployment["spec"]["replicas"] * 2
14    
15    # Return updated resources with full GVK and metadata for server-side apply
16    rsp.desired.resources["app-deployment"].resource.update({
17        "apiVersion": "apps/v1",
18        "kind": "Deployment",
19        "metadata": {
20            "name": deployment["metadata"]["name"],
21            "namespace": deployment["metadata"]["namespace"]
22        },
23        "spec": {"replicas": new_replicas}
24    })

Status and monitoring

Operations provide rich status information:

 1status:
 2  conditions:
 3  - type: Synced
 4    status: "True"
 5    reason: ReconcileSuccess
 6  - type: Succeeded
 7    status: "True"
 8    reason: PipelineSuccess
 9  - type: ValidPipeline
10    status: "True"
11    reason: ValidPipeline
12  failures: 1  # Number of retry attempts
13  pipeline:
14  - step: create-backup
15    output:
16      backupId: "backup-20240115-103000"
17      size: "2.3GB"
18  appliedResourceRefs:
19  - apiVersion: "v1"
20    kind: "Secret"
21    namespace: "production"
22    name: "backup-secret"
23  - apiVersion: "apps/v1" 
24    kind: "Deployment"
25    name: "updated-deployment"

Key status fields:

conditions: Standard Crossplane conditions (Synced) and Operation-specific conditions:
- Succeeded: True when the operation completed successfully, False when it failed
- ValidPipeline: True when all functions have the required operation capability
failures: Number of times the operation has failed and retried
pipeline: Output from each function step for tracking progress
appliedResourceRefs: References to all resources the Operation created or modified

Events

Operations emit Kubernetes events for important activities:

Function run results and warnings
Resource apply failures
Operation lifecycle events (creation, completion, failure)

Troubleshooting operations

Check operation status:

1kubectl get operation my-operation -o wide

View detailed information:

1kubectl describe operation my-operation

Common failure scenarios:

Operations do nothing - Operations feature not enabled:
```
1# Operation exists but has no status conditions and never progresses
2status: {}
```
Solution: enable Operations by adding --enable-operations to Crossplane’s startup arguments.

ValidPipeline condition is False - Function doesn’t support operations:

1conditions:
2- type: ValidPipeline
3  status: "False"
4  reason: InvalidFunctionCapability
5  message: "Function function-name doesn't support operations"

Solution: use a function that declares operation capability.

Succeeded condition is False - Function run failed:

1conditions:
2- type: Succeeded
3  status: "False" 
4  reason: PipelineFailure
5  message: "Function returned error: connection timeout"

Solution: view function logs and fix the underlying issue.

Resource apply failures - View events for details:

1kubectl get events --field-selector involvedObject.name=my-operation

Debug function runs:

1# View function logs
2kubectl logs -n crossplane-system deployment/function-python
3
4# Check operation events
5kubectl get events --field-selector involvedObject.kind=Operation
6
7# Inspect operation status in detail
8kubectl get operation my-operation -o jsonpath='{.status.pipeline}' | jq '.'

Resource management

Operations can create or change any Kubernetes resources using server-side apply with force ownership. This means:

What Operations can do:

Create new resources of any kind
Change existing resources by taking ownership of specific fields
Apply changes that may conflict with other controllers

What Operations can’t do:

Delete resources (current limitation of alpha implementation)
Establish owner references (resources aren’t garbage collected)
Continuously maintain desired state (they run once)

Important

Use caution with Operations that change resources managed by other controllers. Operations force ownership when applying changes, which can cause conflicts.

Test an operation

You can preview the output of any Operation using the Crossplane CLI. You don’t need a Crossplane control plane to do this. The Crossplane CLI uses Docker Engine to run functions.

Tip

See the Crossplane CLI docs to learn how to install and use the Crossplane CLI.

Important

Running crossplane alpha render op requires Docker.

Provide an operation, composition functions, and any required resources to render the output locally.

1crossplane alpha render op operation.yaml functions.yaml --required-resources=ingress.yaml

crossplane alpha render op prints the Operation status and any resources the operation functions created or modified. It shows what would happen if you applied the Operation to a cluster.

 1---
 2# Operation status showing function results
 3apiVersion: ops.crossplane.io/v1alpha1
 4kind: Operation
 5metadata:
 6  name: ingress-cert-monitor
 7status:
 8  conditions:
 9  - type: Succeeded
10    status: "True"
11    reason: PipelineSuccess
12  pipeline:
13  - step: check-ingress-certificate
14    output:
15      certificateExpires: "Sep 29 08:34:02 2025 GMT"
16      daysUntilExpiry: 53
17      hostname: google.com
18      ingressName: example-app
19      status: ok
20---
21# Modified Ingress resource with certificate annotations
22apiVersion: networking.k8s.io/v1
23kind: Ingress
24metadata:
25  annotations:
26    cert-monitor.crossplane.io/expires: Sep 29 08:34:02 2025 GMT
27    cert-monitor.crossplane.io/days-until-expiry: "53"
28    cert-monitor.crossplane.io/status: ok
29  name: example-app
30  namespace: default
31spec:
32  # ... ingress spec unchanged

Use --required-resources to provide resources that your operation functions need access to. You can specify multiple files or use glob patterns:

1# Multiple specific files
2crossplane alpha render op operation.yaml functions.yaml \
3  --required-resources=deployment.yaml,service.yaml,configmap.yaml
4
5# Glob pattern for all YAML files in a directory
6crossplane alpha render op operation.yaml functions.yaml \
7  --required-resources="resources/*.yaml"

Tip

Use the crossplane alpha render op command to test your Operations locally before deploying them to a cluster. The command helps validate function logic and required resource access patterns.

Best practices

Operation-specific practices

Plan for rollback - Design operations to be reversible when possible, because Operations don’t auto rollback like Compositions
Make operations idempotent - Operations should be safe to retry if they fail partway through
Use required resources - Prepopulate functions with needed resources for efficiency rather than requesting them during running

Function development

Declare capabilities - Explicitly declare operation capability in function metadata to enable Operations support
Return meaningful output - Use the output field to track what the operation accomplished for monitoring and debugging

Next steps

Get started with Operations to create your first Operation
Learn about CronOperation for scheduled operations
Learn about WatchOperation for reactive operations