Write a Composition Function in Go

This is a beta feature.

This feature was introduced in v1.11.
This feature graduated to beta status in v1.14.

For more information read the Crossplane feature lifecycle.

Composition functions (or just functions, for short) are custom programs that template Crossplane resources. Crossplane calls composition functions to determine what resources it should create when you create a composite resource (XR). Read the concepts page to learn more about composition functions.

You can write a function to template resources using a general purpose programming language. Using a general purpose programming language allows a function to use advanced logic to template resources, like loops and conditionals. This guide explains how to write a composition function in Go.

It helps to be familiar with how composition functions work before following this guide.

Understand the steps

This guide covers writing a composition function for an XBuckets composite resource (XR).

 1apiVersion: example.crossplane.io/v1
 2kind: XBuckets
 4  name: example-buckets
 6  region: us-east-2
 7  names:
 8  - crossplane-functions-example-a
 9  - crossplane-functions-example-b
10  - crossplane-functions-example-c

An XBuckets XR has a region and an array of bucket names. The function will create an Amazon Web Services (AWS) S3 bucket for each entry in the names array.

To write a function in Go:

  1. Install the tools you need to write the function
  2. Initialize the function from a template
  3. Edit the template to add the function’s logic
  4. Test the function end-to-end
  5. Build and push the function to a package repository

This guide covers each of these steps in detail.

Install the tools you need to write the function

To write a function in Go you need:

  • Go v1.21 or newer. The guide uses Go v1.21.
  • Docker Engine. This guide uses Engine v24.
  • The Crossplane CLI v1.14 or newer. This guide uses Crossplane CLI v1.14.
You don’t need access to a Kubernetes cluster or a Crossplane control plane to build or test a composition function.

Initialize the function from a template

Use the crossplane beta xpkg init command to initialize a new function. When you run this command it initializes your function using a GitHub repository as a template.

1crossplane beta xpkg init function-xbuckets function-template-go -d function-xbuckets 
2Initialized package "function-xbuckets" in directory "/home/negz/control/negz/function-xbuckets" from https://github.com/crossplane/function-template-go/tree/91a1a5eed21964ff98966d72cc6db6f089ad63f4 (main)

The crossplane beta init xpkg command creates a directory named function-xbuckets. When you run the command the new directory should look like this:

1ls function-xbuckets
2Dockerfile  fn.go  fn_test.go  go.mod  go.sum  input/  LICENSE  main.go  package/  README.md  renovate.json

The fn.go file is where you add the function’s code. It’s useful to know about some other files in the template:

  • main.go runs the function. You don’t need to edit main.go.
  • Dockerfile builds the function runtime. You don’t need to edit Dockerfile.
  • The input directory defines the function’s input type.
  • The package directory contains metadata used to build the function package.

In v1.14 of the Crossplane CLI crossplane beta xpkg init just clones a template GitHub repository. A future CLI release will automate tasks like replacing the template name with the new function’s name. See Crossplane issue #4941 for details.

You must make some changes before you start adding code:

  • Edit package/crossplane.yaml to change the package’s name.
  • Edit go.mod to change the Go module’s name.

Name your package function-xbuckets.

The name of your module depends on where you want to keep your function code. If you push Go code to GitHub, you can use your GitHub username. For example module github.com/negz/function-xbuckets.

The function in this guide doesn’t use an input type. For this function you should delete the input and package/input directories.

The input directory defines a Go struct that a function can use to take input, using the input field from a Composition. The composition functions documentation explains how to pass an input to a composition function.

The package/input directory contains an OpenAPI schema generated from the structs in the input directory.


If you’re writing a function that uses an input, edit the input to meet your function’s requirements.

Change the input’s kind and API group. Don’t use Input and template.fn.crossplane.io. Instead use something meaningful to your function.

When you edit files under the input directory you must update some generated files by running go generate. See input/generate.go for details.

1go generate ./...

Edit the template to add the function’s logic

You add your function’s logic to the RunFunction method in fn.go. When you first open the file it contains a “hello world” function.

 1func (f *Function) RunFunction(_ context.Context, req *fnv1beta1.RunFunctionRequest) (*fnv1beta1.RunFunctionResponse, error) {
 2	f.log.Info("Running Function", "tag", req.GetMeta().GetTag())
 4	rsp := response.To(req, response.DefaultTTL)
 6	in := &v1beta1.Input{}
 7	if err := request.GetInput(req, in); err != nil {
 8		response.Fatal(rsp, errors.Wrapf(err, "cannot get Function input from %T", req))
 9		return rsp, nil
10	}
12	response.Normalf(rsp, "I was run with input %q", in.Example)
13	return rsp, nil

All Go composition functions have a RunFunction method. Crossplane passes everything the function needs to run in a RunFunctionRequest struct.

The function tells Crossplane what resources it should compose by returning a RunFunctionResponse struct.

Crossplane generates the RunFunctionRequest and RunFunctionResponse structs using Protocol Buffers. You can find detailed schemas for RunFunctionRequest and RunFunctionResponse in the Buf Schema Registry.

Edit the RunFunction method to replace it with this code.

 1func (f *Function) RunFunction(_ context.Context, req *fnv1beta1.RunFunctionRequest) (*fnv1beta1.RunFunctionResponse, error) {
 2	rsp := response.To(req, response.DefaultTTL)
 4	xr, err := request.GetObservedCompositeResource(req)
 5	if err != nil {
 6		response.Fatal(rsp, errors.Wrapf(err, "cannot get observed composite resource from %T", req))
 7		return rsp, nil
 8	}
10	region, err := xr.Resource.GetString("spec.region")
11	if err != nil {
12		response.Fatal(rsp, errors.Wrapf(err, "cannot read spec.region field of %s", xr.Resource.GetKind()))
13		return rsp, nil
14	}
16	names, err := xr.Resource.GetStringArray("spec.names")
17	if err != nil {
18		response.Fatal(rsp, errors.Wrapf(err, "cannot read spec.names field of %s", xr.Resource.GetKind()))
19		return rsp, nil
20	}
22	desired, err := request.GetDesiredComposedResources(req)
23	if err != nil {
24		response.Fatal(rsp, errors.Wrapf(err, "cannot get desired resources from %T", req))
25		return rsp, nil
26	}
28	_ = v1beta1.AddToScheme(composed.Scheme)
30	for _, name := range names {
31		b := &v1beta1.Bucket{
32			ObjectMeta: metav1.ObjectMeta{
33				Annotations: map[string]string{
34					"crossplane.io/external-name": name,
35				},
36			},
37			Spec: v1beta1.BucketSpec{
38				ForProvider: v1beta1.BucketParameters{
39					Region: ptr.To[string](region),
40				},
41			},
42		}
44		cd, err := composed.From(b)
45		if err != nil {
46			response.Fatal(rsp, errors.Wrapf(err, "cannot convert %T to %T", b, &composed.Unstructured{}))
47			return rsp, nil
48		}
50		desired[resource.Name("xbuckets-"+name)] = &resource.DesiredComposed{Resource: cd}
51	}
53	if err := response.SetDesiredComposedResources(rsp, desired); err != nil {
54		response.Fatal(rsp, errors.Wrapf(err, "cannot set desired composed resources in %T", rsp))
55		return rsp, nil
56	}
58	return rsp, nil

Expand the below block to view the full fn.go, including imports and commentary explaining the function’s logic.

  1package main
  3import (
  4	"context"
  6	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
  7	"k8s.io/utils/ptr"
  9	"github.com/upbound/provider-aws/apis/s3/v1beta1"
 11	"github.com/crossplane/function-sdk-go/errors"
 12	"github.com/crossplane/function-sdk-go/logging"
 13	fnv1beta1 "github.com/crossplane/function-sdk-go/proto/v1beta1"
 14	"github.com/crossplane/function-sdk-go/request"
 15	"github.com/crossplane/function-sdk-go/resource"
 16	"github.com/crossplane/function-sdk-go/resource/composed"
 17	"github.com/crossplane/function-sdk-go/response"
 20// Function returns whatever response you ask it to.
 21type Function struct {
 22	fnv1beta1.UnimplementedFunctionRunnerServiceServer
 24	log logging.Logger
 27// RunFunction observes an XBuckets composite resource (XR). It adds an S3
 28// bucket to the desired state for every entry in the XR's spec.names array.
 29func (f *Function) RunFunction(_ context.Context, req *fnv1beta1.RunFunctionRequest) (*fnv1beta1.RunFunctionResponse, error) {
 30	f.log.Info("Running Function", "tag", req.GetMeta().GetTag())
 32	// Create a response to the request. This copies the desired state and
 33	// pipeline context from the request to the response.
 34	rsp := response.To(req, response.DefaultTTL)
 36	// Read the observed XR from the request. Most functions use the observed XR
 37	// to add desired managed resources.
 38	xr, err := request.GetObservedCompositeResource(req)
 39	if err != nil {
 40		// If the function can't read the XR, the request is malformed. This
 41		// should never happen. The function returns a fatal result. This tells
 42		// Crossplane to stop running functions and return an error.
 43		response.Fatal(rsp, errors.Wrapf(err, "cannot get observed composite resource from %T", req))
 44		return rsp, nil
 45	}
 47	// Create an updated logger with useful information about the XR.
 48	log := f.log.WithValues(
 49		"xr-version", xr.Resource.GetAPIVersion(),
 50		"xr-kind", xr.Resource.GetKind(),
 51		"xr-name", xr.Resource.GetName(),
 52	)
 54	// Get the region from the XR. The XR has getter methods like GetString,
 55	// GetBool, etc. You can use them to get values by their field path.
 56	region, err := xr.Resource.GetString("spec.region")
 57	if err != nil {
 58		response.Fatal(rsp, errors.Wrapf(err, "cannot read spec.region field of %s", xr.Resource.GetKind()))
 59		return rsp, nil
 60	}
 62	// Get the array of bucket names from the XR.
 63	names, err := xr.Resource.GetStringArray("spec.names")
 64	if err != nil {
 65		response.Fatal(rsp, errors.Wrapf(err, "cannot read spec.names field of %s", xr.Resource.GetKind()))
 66		return rsp, nil
 67	}
 69	// Get all desired composed resources from the request. The function will
 70	// update this map of resources, then save it. This get, update, set pattern
 71	// ensures the function keeps any resources added by other functions.
 72	desired, err := request.GetDesiredComposedResources(req)
 73	if err != nil {
 74		response.Fatal(rsp, errors.Wrapf(err, "cannot get desired resources from %T", req))
 75		return rsp, nil
 76	}
 78	// Add v1beta1 types (including Bucket) to the composed resource scheme.
 79	// composed.From uses this to automatically set apiVersion and kind.
 80	_ = v1beta1.AddToScheme(composed.Scheme)
 82	// Add a desired S3 bucket for each name.
 83	for _, name := range names {
 84		// One advantage of writing a function in Go is strong typing. The
 85		// function can import and use managed resource types from the provider.
 86		b := &v1beta1.Bucket{
 87			ObjectMeta: metav1.ObjectMeta{
 88				// Set the external name annotation to the desired bucket name.
 89				// This controls what the bucket will be named in AWS.
 90				Annotations: map[string]string{
 91					"crossplane.io/external-name": name,
 92				},
 93			},
 94			Spec: v1beta1.BucketSpec{
 95				ForProvider: v1beta1.BucketParameters{
 96					// Set the bucket's region to the value read from the XR.
 97					Region: ptr.To[string](region),
 98				},
 99			},
100		}
102		// Convert the bucket to the unstructured resource data format the SDK
103		// uses to store desired composed resources.
104		cd, err := composed.From(b)
105		if err != nil {
106			response.Fatal(rsp, errors.Wrapf(err, "cannot convert %T to %T", b, &composed.Unstructured{}))
107			return rsp, nil
108		}
110		// Add the bucket to the map of desired composed resources. It's
111		// important that the function adds the same bucket every time it's
112		// called. It's also important that the bucket is added with the same
113		// resource.Name every time it's called. The function prefixes the name
114		// with "xbuckets-" to avoid collisions with any other composed
115		// resources that might be in the desired resources map.
116		desired[resource.Name("xbuckets-"+name)] = &resource.DesiredComposed{Resource: cd}
117	}
119	// Finally, save the updated desired composed resources to the response.
120	if err := response.SetDesiredComposedResources(rsp, desired); err != nil {
121		response.Fatal(rsp, errors.Wrapf(err, "cannot set desired composed resources in %T", rsp))
122		return rsp, nil
123	}
125	// Log what the function did. This will only appear in the function's pod
126	// logs. A function can use response.Normal and response.Warning to emit
127	// Kubernetes events associated with the XR it's operating on.
128	log.Info("Added desired buckets", "region", region, "count", len(names))
130	return rsp, nil

This code:

  1. Gets the observed composite resource from the RunFunctionRequest.
  2. Gets the region and bucket names from the observed composite resource.
  3. Adds one desired S3 bucket for each bucket name.
  4. Returns the desired S3 buckets in a RunFunctionResponse.

The code uses the v1beta1.Bucket type from Upbound’s AWS S3 provider. One advantage of writing a function in Go is that you can compose resources using the same strongly typed structs Crossplane uses in its providers.

You must get the AWS Provider Go module to use this type:

1go get github.com/upbound/provider-aws@v0.43.0

Crossplane provides a software development kit (SDK) for writing composition functions in Go. This function uses utilities from the SDK. In particular the request and response packages make working with the RunFunctionRequest and RunFunctionResponse types easier.

Read the Go package documentation for the SDK.

Test the function end-to-end

Test your function by adding unit tests, and by using the crossplane beta render command.

Go has rich support for unit testing. When you initialize a function from the template it adds some unit tests to fn_test.go. These tests follow Go’s recommendations. They use only pkg/testing from the Go standard library and google/go-cmp.

To add test cases, update the cases map in TestRunFunction. Expand the below block to view the full fn_test.go file for the function.

  1package main
  3import (
  4	"context"
  5	"testing"
  6	"time"
  8	"github.com/google/go-cmp/cmp"
  9	"github.com/google/go-cmp/cmp/cmpopts"
 10	"google.golang.org/protobuf/testing/protocmp"
 11	"google.golang.org/protobuf/types/known/durationpb"
 13	"github.com/crossplane/crossplane-runtime/pkg/logging"
 15	fnv1beta1 "github.com/crossplane/function-sdk-go/proto/v1beta1"
 16	"github.com/crossplane/function-sdk-go/resource"
 19func TestRunFunction(t *testing.T) {
 20	type args struct {
 21		ctx context.Context
 22		req *fnv1beta1.RunFunctionRequest
 23	}
 24	type want struct {
 25		rsp *fnv1beta1.RunFunctionResponse
 26		err error
 27	}
 29	cases := map[string]struct {
 30		reason string
 31		args   args
 32		want   want
 33	}{
 34		"AddTwoBuckets": {
 35			reason: "The Function should add two buckets to the desired composed resources",
 36			args: args{
 37				req: &fnv1beta1.RunFunctionRequest{
 38					Observed: &fnv1beta1.State{
 39						Composite: &fnv1beta1.Resource{
 40							// MustStructJSON is a handy way to provide mock
 41							// resources.
 42							Resource: resource.MustStructJSON(`{
 43								"apiVersion": "example.crossplane.io/v1alpha1",
 44								"kind": "XBuckets",
 45								"metadata": {
 46									"name": "test"
 47								},
 48								"spec": {
 49									"region": "us-east-2",
 50									"names": [
 51										"test-bucket-a",
 52										"test-bucket-b"
 53									]
 54								}
 55							}`),
 56						},
 57					},
 58				},
 59			},
 60			want: want{
 61				rsp: &fnv1beta1.RunFunctionResponse{
 62					Meta: &fnv1beta1.ResponseMeta{Ttl: durationpb.New(60 * time.Second)},
 63					Desired: &fnv1beta1.State{
 64						Resources: map[string]*fnv1beta1.Resource{
 65							"xbuckets-test-bucket-a": {Resource: resource.MustStructJSON(`{
 66								"apiVersion": "s3.aws.upbound.io/v1beta1",
 67								"kind": "Bucket",
 68								"metadata": {
 69									"annotations": {
 70										"crossplane.io/external-name": "test-bucket-a"
 71									}
 72								},
 73								"spec": {
 74									"forProvider": {
 75										"region": "us-east-2"
 76									}
 77								}
 78							}`)},
 79							"xbuckets-test-bucket-b": {Resource: resource.MustStructJSON(`{
 80								"apiVersion": "s3.aws.upbound.io/v1beta1",
 81								"kind": "Bucket",
 82								"metadata": {
 83									"annotations": {
 84										"crossplane.io/external-name": "test-bucket-b"
 85									}
 86								},
 87								"spec": {
 88									"forProvider": {
 89										"region": "us-east-2"
 90									}
 91								}
 92							}`)},
 93						},
 94					},
 95				},
 96			},
 97		},
 98	}
100	for name, tc := range cases {
101		t.Run(name, func(t *testing.T) {
102			f := &Function{log: logging.NewNopLogger()}
103			rsp, err := f.RunFunction(tc.args.ctx, tc.args.req)
105			if diff := cmp.Diff(tc.want.rsp, rsp, protocmp.Transform()); diff != "" {
106				t.Errorf("%s\nf.RunFunction(...): -want rsp, +got rsp:\n%s", tc.reason, diff)
107			}
109			if diff := cmp.Diff(tc.want.err, err, cmpopts.EquateErrors()); diff != "" {
110				t.Errorf("%s\nf.RunFunction(...): -want err, +got err:\n%s", tc.reason, diff)
111			}
112		})
113	}

Run the unit tests using the go test command:

1go test -v -cover .
2=== RUN   TestRunFunction
3=== RUN   TestRunFunction/AddTwoBuckets
4--- PASS: TestRunFunction (0.00s)
5    --- PASS: TestRunFunction/AddTwoBuckets (0.00s)
7coverage: 52.6% of statements
8ok      github.com/negz/function-xbuckets       0.016s  coverage: 52.6% of statements

You can preview the output of a Composition that uses this function using the Crossplane CLI. You don’t need a Crossplane control plane to do this.

Create a directory under function-xbuckets named example and create Composite Resource, Composition and Function YAML files.

Expand the following block to see example files.

You can recreate the output below using by running crossplane beta render with these files.

The xr.yaml file contains the composite resource to render:

 1apiVersion: example.crossplane.io/v1
 2kind: XBuckets
 4  name: example-buckets
 6  region: us-east-2
 7  names:
 8  - crossplane-functions-example-a
 9  - crossplane-functions-example-b
10  - crossplane-functions-example-c

The composition.yaml file contains the Composition to use to render the composite resource:

 1apiVersion: apiextensions.crossplane.io/v1
 2kind: Composition
 4  name: create-buckets
 6  compositeTypeRef:
 7    apiVersion: example.crossplane.io/v1
 8    kind: XBuckets
 9  mode: Pipeline
10  pipeline:
11  - step: create-buckets
12    functionRef:
13      name: function-xbuckets

The functions.yaml file contains the Functions the Composition references in its pipeline steps:

 1apiVersion: pkg.crossplane.io/v1beta1
 2kind: Function
 4  name: function-xbuckets
 5  annotations:
 6    render.crossplane.io/runtime: Development
 8  # The CLI ignores this package when using the Development runtime.
 9  # You can set it to any value.
10  package: xpkg.upbound.io/negz/function-xbuckets:v0.1.0

The Function in functions.yaml uses the Development runtime. This tells crossplane beta render that your function is running locally. It connects to your locally running function instead of using Docker to pull and run the function.

1apiVersion: pkg.crossplane.io/v1beta1
2kind: Function
4  name: function-xbuckets
5  annotations:
6    render.crossplane.io/runtime: Development

Use go run to run your function locally.

1go run . --insecure --debug
The insecure flag tells the function to run without encryption or authentication. Only use it during testing and development.

In a separate terminal, run crossplane beta render.

1crossplane beta render xr.yaml composition.yaml functions.yaml

This command calls your function. In the terminal where your function is running you should now see log output:

1go run . --insecure --debug
22023-10-31T16:17:32.158-0700    INFO    function-xbuckets/fn.go:29      Running Function        {"tag": ""}
32023-10-31T16:17:32.159-0700    INFO    function-xbuckets/fn.go:125     Added desired buckets   {"xr-version": "example.crossplane.io/v1", "xr-kind": "XBuckets", "xr-name": "example-buckets", "region": "us-east-2", "count": 3}

The crossplane beta render command prints the desired resources the function returns.

 2apiVersion: example.crossplane.io/v1
 3kind: XBuckets
 5  name: example-buckets
 7apiVersion: s3.aws.upbound.io/v1beta1
 8kind: Bucket
10  annotations:
11    crossplane.io/composition-resource-name: xbuckets-crossplane-functions-example-b
12    crossplane.io/external-name: crossplane-functions-example-b
13  generateName: example-buckets-
14  labels:
15    crossplane.io/composite: example-buckets
16  ownerReferences:
17    # Omitted for brevity
19  forProvider:
20    region: us-east-2
22apiVersion: s3.aws.upbound.io/v1beta1
23kind: Bucket
25  annotations:
26    crossplane.io/composition-resource-name: xbuckets-crossplane-functions-example-c
27    crossplane.io/external-name: crossplane-functions-example-c
28  generateName: example-buckets-
29  labels:
30    crossplane.io/composite: example-buckets
31  ownerReferences:
32    # Omitted for brevity
34  forProvider:
35    region: us-east-2
37apiVersion: s3.aws.upbound.io/v1beta1
38kind: Bucket
40  annotations:
41    crossplane.io/composition-resource-name: xbuckets-crossplane-functions-example-a
42    crossplane.io/external-name: crossplane-functions-example-a
43  generateName: example-buckets-
44  labels:
45    crossplane.io/composite: example-buckets
46  ownerReferences:
47    # Omitted for brevity
49  forProvider:
50    region: us-east-2
Read the composition functions documentation to learn more about testing composition functions.

Build and push the function to a package registry

You build a function in two stages. First you build the function’s runtime. This is the Open Container Initiative (OCI) image Crossplane uses to run your function. You then embed that runtime in a package, and push it to a package registry. The Crossplane CLI uses xpkg.upbound.io as its default package registry.

A function supports a single platform, like linux/amd64, by default. You can support multiple platforms by building a runtime and package for each platform, then pushing all the packages to a single tag in the registry.

Pushing your function to a registry allows you to use your function in a Crossplane control plane. See the composition functions documentation. to learn how to use a function in a control plane.

Use Docker to build a runtime for each platform.

1docker build . --quiet --platform=linux/amd64 --tag runtime-amd64
1docker build . --quiet --platform=linux/arm64 --tag runtime-arm64
You can use whatever tag you want. There’s no need to push the runtime images to a registry. The tag is only used to tell crossplane xpkg build what runtime to embed.

Use the Crossplane CLI to build a package for each platform. Each package embeds a runtime image.

The --package-root flag specifies the package directory, which contains crossplane.yaml. This includes metadata about the package.

The --embed-runtime-image flag specifies the runtime image tag built using Docker.

The --package-file flag specifies specifies where to write the package file to disk. Crossplane package files use the extension .xpkg.

1crossplane xpkg build \
2    --package-root=package \
3    --embed-runtime-image=runtime-amd64 \
4    --package-file=function-amd64.xpkg
1crossplane xpkg build \
2    --package-root=package \
3    --embed-runtime-image=runtime-arm64 \
4    --package-file=function-arm64.xpkg
Crossplane packages are special OCI images. Read more about packages in the packages documentation.

Push both package files to a registry. Pushing both files to one tag in the registry creates a multi-platform package that runs on both linux/arm64 and linux/amd64 hosts.

1crossplane xpkg push \
2  --package-files=function-amd64.xpkg,function-arm64.xpkg \
3  negz/function-xbuckets:v0.1.0

If you push the function to a GitHub repository the template automatically sets up continuous integration (CI) using GitHub Actions. The CI workflow will lint, test, and build your function. You can see how the template configures CI by reading .github/workflows/ci.yaml.

The CI workflow can automatically push packages to xpkg.upbound.io. For this to work you must create a repository at https://marketplace.upbound.io. Give the CI workflow access to push to the Marketplace by creating an API token and adding it to your repository. Save your API token access ID as a secret named XPKG_ACCESS_ID and your API token as a secret named XPKG_TOKEN.