This document is for an unreleased version of Crossplane.

This document applies to the Crossplane master branch and not to the latest release v1.17.

The base Crossplane installation consists of two pods, the crossplane pod and the crossplane-rbac-manager pod. Both pods install in the crossplane-system namespace by default.

Crossplane pod

Init container

Before starting the core Crossplane container an init container runs. The init container installs the core Crossplane Custom Resource Definitions (CRDs), configures Crossplane webhooks and installs any supplied Providers or Configurations.

Tip
The Kubernetes documentation contains more information about init containers.

The settings the init container sets include installing Provider or Configuration packages with Crossplane, customizing the namespace Crossplane installs in and defining webhook configurations.

The core CRDs installed by the init container include:

  • CompositeResourceDefinitions, Compositions, Configurations and Providers
  • Locks to manage package dependencies
  • DeploymentRuntimeConfigs to apply settings to installed Providers and Functions
  • StoreConfigs for connecting external secret stores like HashiCorp Vault
Note
The Install Crossplane section has more information about customizing the Crossplane install.

The status Init on the Crossplane pod is the init container running.

1kubectl get pods -n crossplane-system
2NAME                                       READY   STATUS     RESTARTS   AGE
3crossplane-9f6d5cd7b-r9j8w                 0/1     Init:0/1   0          6s

The init container completes and starts the Crossplane core container automatically.

1kubectl get pods -n crossplane-system
2NAME                                       READY   STATUS    RESTARTS   AGE
3crossplane-9f6d5cd7b-r9j8w                 1/1     Running   0          15s

Core container

The main Crossplane container, called the core container, enforces the desired state of Crossplane resources, manages leader elections and process webhooks.

Note
The Crossplane pod only reconciles core Crossplane components, including Claims and composite resources. Providers are responsible for reconciling their managed resources.

Reconcile loop

The core container operates on a reconcile loop, constantly checking the status of deployed resources and correcting any “drift.” After checking a resource Crossplane waits some time and checks again.

Crossplane monitors resources through a Kubernetes watch or through periodic polling. Some resources may be both watched and polled.

Crossplane requests that the API server notifies Crossplane of any changes on objects. This notification tool is a watch.

Watched objects include Providers, managed resources and CompositeResourceDefinitions.

For objects that Kubernetes can’t provide a watch for, Crossplane periodically poll the resource to find it’s state. The default polling rate is one minute. Change the polling rate with the --poll-interval pod argument.

Reducing the poll-interval value causes Crossplane to poll resources more frequently. This increases the load of the Crossplane pod and results in more frequent provider API calls.

Increasing the poll-interval causes Crossplane to poll resources less frequently. This increases the maximum time until Crossplane discovers changes in the cloud provider that require updating.

Managed resources use polling.

Note
Managed resources watch for Kubernetes events like deletion or changes to their spec. Managed resources rely on polling to detect changes in the external system.

Crossplane double-checks all resources to confirm they’re in the desired state. Crossplane does this every one hour by default. Use the --sync-interval Crossplane pod argument to change this interval.

The --max-reconcile-rate rate defines the rate, in times per second, Crossplane reconciles resources.

Reducing the --max-reconcile-rate, or making it smaller, reduces CPU resources Crossplane uses, but increases the amount of time until changed resources are fully synced.

Increasing the --max-reconcile-rate, or making it larger, increases the CPU resources Crossplane uses but allows Crossplane to reconcile all resources faster.

Important
Most Providers use their own --max-reconcile-rate. This determines the same settings for Providers and their managed resources. Applying the --max-reconcile-rate to Crossplane only controls the rate for core Crossplane resources.
Enable real time Compositions

With real time compositions enabled Crossplane watches every composed resource with a Kubernetes watch. Crossplane receives events from the Kubernetes API server when a composed resource changes. For example, when a provider sets the Ready condition to true.

Important
Real time compositions are an alpha feature. Alpha features aren’t enabled by default.

With real time compositions enabled, Crossplane doesn’t use the --poll-interval settings.

Enable real time compositions support by changing the Crossplane pod setting and enabling
--enable-realtime-compositions argument.

 1$ kubectl edit deployment crossplane --namespace crossplane-system
 2apiVersion: apps/v1
 3kind: Deployment
 4spec:
 5# Removed for brevity
 6  template:
 7    spec:
 8      containers:
 9      - args:
10        - core
11        - start
12        - --enable-realtime-compositions
Tip
The Crossplane install guide describes enabling feature flags like –enable-realtime-compositions with Helm.
Reconcile retry rate

The --max-reconcile-rate setting configures the number of times per second Crossplane or a provider attempts to correct a resource. The default value is 10 times per second.

All core Crossplane components share the reconcile rate. Each Provider implements their own max reconcile rate setting.

Number of reconcilers

The second value --max-reconcile-rate defines is the number of resources that Crossplane can reconcile at once. If there are more resources than the configured --max-reconcile-rate the remaining resources must wait until Crossplane reconciles a an existing resource.

Read the Change Pod Settings section for instructions on applying these settings.

RBAC manager pod

The Crossplane RBAC manager pod automates required Kubernetes RBAC permissions for Crossplane and Crossplane Providers.

Note

Crossplane installs and enables the RBAC manager by default. Disabling the RBAC manager requires manual Kubernetes permissions definitions for proper Crossplane operations.

The RBAC manager design document provides more comprehensive details on the Crossplane RBAC requirements.

Disable the RBAC manager

Disable the RBAC manager after installation by deleting the crossplane-rbac-manager deployment from the crossplane-system namespace.

Disable the RBAC manager before installation by editing the Helm values.yaml file, setting rbacManager.deploy to false.

Note
Instructions for changing Crossplane pod settings during installation are in the Crossplane Install section.

RBAC init container

The RBAC manager requires the CompositeResourceDefinition and ProviderRevision resources to be available before starting.

The RBAC manager init container waits for these resources before starting the main RBAC manager container.

RBAC manager container

The RBAC manager container preforms the following tasks:

  • creating and binding RBAC roles to Provider ServiceAccounts, allowing them to control their managed resources
  • allowing the crossplane ServiceAccount to create managed resources
  • creating ClusterRoles to access Crossplane resources in all namespaces

Use the ClusterRoles to grant access to all Crossplane resources in the cluster.

Crossplane ClusterRoles

The RBAC manager creates four Kubernetes ClusterRoles. These Roles grant permissions over cluster wide Crossplane resources.

crossplane-admin

The crossplane-admin ClusterRole has the following permissions:

  • full access to all Crossplane types
  • full access to all secrets and namespaces (even those unrelated to Crossplane)
  • read-only access to all cluster RBAC roles, CustomResourceDefinitions and events
  • ability to bind RBAC roles to other entities.

View the full RBAC policy with

1kubectl describe clusterrole crossplane-admin
crossplane-edit

The crossplane-edit ClusterRole has the following permissions:

  • full access to all Crossplane types
  • full access to all secrets (even those unrelated to Crossplane)
  • read-only access to all namespaces and events (even those unrelated to Crossplane).

View the full RBAC policy with

1kubectl describe clusterrole crossplane-edit
crossplane-view

The crossplane-view ClusterRole has the following permissions:

  • read-only access to all Crossplane types
  • read-only access to all namespaces and events (even those unrelated to Crossplane).

View the full RBAC policy with

1kubectl describe clusterrole crossplane-view
crossplane-browse

The crossplane-browse ClusterRole has the following permissions:

  • read-only access to Crossplane compositions and XRDs. This allows resource claim creators to discover and select an appropriate composition.

View the full RBAC policy with

1kubectl describe clusterrole crossplane-browse

Leader election

By default only a single Crossplane pod runs in a cluster. If more than one Crossplane pod runs both pods try to manage Crossplane resources. To prevent conflicts Crossplane uses a leader election to have a single pod in control at a time. Other Crossplane pods standby until the leader fails.

Note

It’s possible to run more than one Crossplane or RBAC manager pods for redundancy.

Kubernetes restarts any failed Crossplane or RBAC manager pods. Redundant pods aren’t required in most deployments.

Both the Crossplane pod and the RBAC manager pods support leader elections.

Enable leader elections with the --leader-election pod argument.

Warning

Running multiple Crossplane pods without leader election is unsupported.

Change pod settings

Change Crossplane pod settings either before installing Crossplane by editing the Helm values.yml file or after installation by editing the Deployment.

The full list of configuration options and feature flags are available in the Crossplane Install section.

Note
Instructions for changing Crossplane pod settings during installation are in the Crossplane Install section.

Edit the deployment

Note
These settings apply to both the crossplane and rbac-manager pods and Deployments.

To change the settings of an installed Crossplane pod, edit the crossplane deployment in the crossplane-system namespace with the command

kubectl edit deployment crossplane --namespace crossplane-system

Warning
Updating the Crossplane deployment restarts the Crossplane pod.

Add Crossplane pod arguments to the spec.template.spec.containers[].args section of the deployment.

For example, to change the sync-interval add --sync-interval=30m.

 1kubectl edit deployment crossplane --namespace crossplane-system
 2apiVersion: apps/v1
 3kind: Deployment
 4spec:
 5# Removed for brevity
 6  template:
 7    spec:
 8      containers:
 9      - args:
10        - core
11        - start
12        - --sync-interval=30m

Use environmental variables

The core Crossplane pod checks for configured environmental variables at startup to change default settings.

The full list of configurable environmental variables are available in the Crossplane Install section.