GCP Quickstart Part 2

This guide walks you through building and accessing a custom API with Crossplane.

Prerequisites

• Complete quickstart part 1 connecting Kubernetes to GCP.
• a GCP account with permissions to create a GCP storage bucket and a Pub/Sub topic.

1helm repo add \
2crossplane-stable https://charts.crossplane.io/stable
3helm repo update
4&&
5helm install crossplane \
6crossplane-stable/crossplane \
7--namespace crossplane-system \
8--create-namespace

1cat <<EOF | kubectl apply -f -
2apiVersion: pkg.crossplane.io/v1
3kind: Provider
4metadata:
5  name: provider-gcp-storage
6spec:
7  package: xpkg.upbound.io/upbound/provider-gcp-storage:v0.41.0
8EOF

1kubectl create secret \
2generic gcp-secret \
3-n crossplane-system \
4--from-file=creds=./gcp-credentials.json

 1cat <<EOF | kubectl apply -f -
 2apiVersion: gcp.upbound.io/v1beta1
 3kind: ProviderConfig
 4metadata:
 5  name: default
 6spec:
 7  projectID: 
 8  credentials:
 9    source: Secret
10    secretRef:
11      namespace: crossplane-system
12      name: gcp-secret
13      key: creds
14EOF

Install the PubSub Provider

Part 1 only installed the GCP Storage Provider. This section deploys a PubSub Topic along with a GCP storage bucket.
First install the GCP PubSub Provider.

Add the new Provider to the cluster.

1cat <<EOF | kubectl apply -f -
2apiVersion: pkg.crossplane.io/v1
3kind: Provider
4metadata:
5  name: provider-gcp-pubsub
6spec:
7  package: xpkg.upbound.io/upbound/provider-gcp-pubsub:v0.41.0
8EOF

View the new PubSub provider with kubectl get providers.

1kubectl get providers
2NAME                          INSTALLED   HEALTHY   PACKAGE                                                AGE
3provider-gcp-pubsub           True        True      xpkg.upbound.io/upbound/provider-gcp-pubsub:v0.41.0    39s
4provider-gcp-storage          True        True      xpkg.upbound.io/upbound/provider-gcp-storage:v0.41.0   13m
5upbound-provider-family-gcp   True        True      xpkg.upbound.io/upbound/provider-family-gcp:v0.41.0    12m

Create a custom API

Crossplane allows you to build your own custom APIs for your users, abstracting away details about the cloud provider and their resources. You can make your API as complex or simple as you wish.

The custom API is a Kubernetes object.
Here is an example custom API.

1apiVersion: database.example.com/v1alpha1
2kind: NoSQL
3metadata:
4  name: my-nosql-database
5spec: 
6  location: "US"

Like any Kubernetes object the API has a version, kind and spec.

Define a group and version

To create your own API start by defining an API group and version.

The group can be any value, but common convention is to map to a fully qualified domain name.

The version shows how mature or stable the API is and increments when changing, adding or removing fields in the API.

Crossplane doesn’t require specific versions or a specific version naming convention, but following Kubernetes API versioning guidelines is strongly recommended.

• v1alpha1 - A new API that may change at any time.
• v1beta1 - An existing API that’s considered stable. Breaking changes are strongly discouraged.
• v1 - A stable API that doesn’t have breaking changes.

This guide uses the group database.example.com.

Because this is the first version of the API, this guide uses the version v1alpha1.

1apiVersion: database.example.com/v1alpha1

Define a kind

The API group is a logical collection of related APIs. In a group are individual kinds representing different resources.

For example a queue group may have a PubSub and CloudTask kinds.

The kind can be anything, but it must be UpperCamelCased.

This API’s kind is PubSub

1apiVersion: queue.example.com/v1alpha1
2kind: PubSub

Define a spec

The most important part of an API is the schema. The schema defines the inputs accepted from users.

This API allows users to provide a location of where to run their cloud resources.

All other resource settings can’t be configurable by the users. This allows Crossplane to enforce any policies and standards without worrying about user errors.

1apiVersion: queue.example.com/v1alpha1
2kind: PubSub
3spec: 
4  location: "US"

Apply the API

Crossplane uses Composite Resource Definitions (also called an XRD) to install your custom API in Kubernetes.

The XRD spec contains all the information about the API including the group, version, kind and schema.

The XRD’s name must be the combination of the plural and group.

The schema uses the OpenAPIv3 specification to define the API spec.

The API defines a location that must be oneOf either EU or US.

Apply this XRD to create the custom API in your Kubernetes cluster.

 1cat <<EOF | kubectl apply -f -
 2apiVersion: apiextensions.crossplane.io/v1
 3kind: CompositeResourceDefinition
 4metadata:
 5  name: pubsubs.queue.example.com
 6spec:
 7  group: queue.example.com
 8  names:
 9    kind: PubSub
10    plural: pubsubs
11  versions:
12  - name: v1alpha1
13    schema:
14      openAPIV3Schema:
15        type: object
16        properties:
17          spec:
18            type: object
19            properties:
20              location:
21                type: string
22                oneOf:
23                  - pattern: '^EU$'
24                  - pattern: '^US$'
25            required:
26              - location
27    served: true
28    referenceable: true
29  claimNames:
30    kind: PubSubClaim
31    plural: pubsubclaims
32EOF

Adding the claimNames allows users to access this API either at the cluster level with the pubsub endpoint or in a namespace with the pubsubclaim endpoint.

The namespace scoped API is a Crossplane Claim.

View the installed XRD with kubectl get xrd.

1kubectl get xrd
2NAME                        ESTABLISHED   OFFERED   AGE
3pubsubs.queue.example.com   True          True      7s

View the new custom API endpoints with kubectl api-resources | grep pubsub

1kubectl api-resources | grep queue.example
2pubsubclaims                 queue.example.com/v1alpha1             true         PubSubClaim
3pubsubs                      queue.example.com/v1alpha1             false        PubSub

Create a deployment template

When users access the custom API Crossplane takes their inputs and combines them with a template describing what infrastructure to deploy. Crossplane calls this template a Composition.

The Composition defines all the cloud resources to deploy. Each entry in the template is a full resource definitions, defining all the resource settings and metadata like labels and annotations.

This template creates a GCP Storage Bucket and a PubSub Topic.

Crossplane uses patches to apply the user’s input to the resource template.
This Composition takes the user’s location input and uses it as the location used in the individual resource.

Apply this Composition to your cluster.

 1cat <<EOF | kubectl apply -f -
 2apiVersion: apiextensions.crossplane.io/v1
 3kind: Composition
 4metadata:
 5  name: topic-with-bucket
 6spec:
 7  resources:
 8    - name: crossplane-quickstart-bucket
 9      base:
10        apiVersion: storage.gcp.upbound.io/v1beta1
11        kind: Bucket
12        spec:
13          forProvider:
14            location: "US"
15      patches:
16        - fromFieldPath: "spec.location"
17          toFieldPath: "spec.forProvider.location"
18          transforms:
19            - type: map
20              map: 
21                EU: "EU"
22                US: "US"
23    - name: crossplane-quickstart-topic
24      base:
25        apiVersion: pubsub.gcp.upbound.io/v1beta1
26        kind: Topic
27        spec:
28          forProvider:
29            messageStoragePolicy:
30              - allowedPersistenceRegions: 
31                - "us-central1"
32      patches:
33        - fromFieldPath: "spec.location"
34          toFieldPath: "spec.forProvider.messageStoragePolicy[0].allowedPersistenceRegions[0]"
35          transforms:
36            - type: map
37              map: 
38                EU: "europe-central2"
39                US: "us-central1"
40  compositeTypeRef:
41    apiVersion: queue.example.com/v1alpha1
42    kind: PubSub
43EOF

The compositeTypeRef defines which custom APIs can use this template to create resources.

View the Composition with kubectl get composition

1kubectl get composition
2NAME                XR-KIND   XR-APIVERSION       AGE
3topic-with-bucket   PubSub    queue.example.com   3s

Access the custom API

With the custom API (XRD) installed and associated to a resource template (Composition) users can access the API to create resources.

Create a PubSub object to create the cloud resources.

1cat <<EOF | kubectl apply -f -
2apiVersion: queue.example.com/v1alpha1
3kind: PubSub
4metadata:
5  name: my-pubsub-queue
6spec: 
7  location: "US"
8EOF

View the resource with kubectl get pubsub.

1kubectl get pubsub
2NAME              SYNCED   READY   COMPOSITION         AGE
3my-pubsub-queue   True     True    topic-with-bucket   2m12s

This object is a Crossplane composite resource (also called an XR).
It’s a single object representing the collection of resources created from the Composition template.

View the individual resources with kubectl get managed

1kubectl get managed
2NAME                                                READY   SYNCED   EXTERNAL-NAME           AGE
3topic.pubsub.gcp.upbound.io/my-pubsub-queue-cjswx   True    True     my-pubsub-queue-cjswx   3m4s
4
5NAME                                                  READY   SYNCED   EXTERNAL-NAME           AGE
6bucket.storage.gcp.upbound.io/my-pubsub-queue-vljg9   True    True     my-pubsub-queue-vljg9   3m4s

Delete the resources with kubectl delete pubsub.

1kubectl delete pubsub my-pubsub-queue
2pubsub.queue.example.com "my-pubsub-queue" deleted

Verify Crossplane deleted the resources with kubectl get managed

1kubectl get managed
2No resources found

Using the API with namespaces

Accessing the API pubsub happens at the cluster scope.
Most organizations isolate their users into namespaces.

A Crossplane Claim is the custom API in a namespace.

Creating a Claim is just like accessing the custom API endpoint, but with the kind from the custom API’s claimNames.

Create a new namespace to test create a Claim in.

1kubectl create namespace crossplane-test

Then create a Claim in the crossplane-test namespace.

1cat <<EOF | kubectl apply -f -
2apiVersion: queue.example.com/v1alpha1
3kind: PubSubClaim
4metadata:  
5  name: my-pubsub-queue
6  namespace: crossplane-test
7spec: 
8  location: "US"
9EOF

View the Claim with kubectl get claim -n crossplane-test.

1kubectl get claim -n crossplane-test
2NAME                SYNCED   READY   CONNECTION-SECRET   AGE
3my-pubsub-queue   True     True                        2m10s

The Claim automatically creates a composite resource, which creates the managed resources.

View the Crossplane created composite resource with kubectl get composite.

1kubectl get composite
2NAME                    SYNCED   READY   COMPOSITION         AGE
3my-pubsub-queue-7bm9n   True     True    topic-with-bucket   3m10s

Again, view the managed resources with kubectl get managed.

1kubectl get managed
2NAME                                                      READY   SYNCED   EXTERNAL-NAME                 AGE
3topic.pubsub.gcp.upbound.io/my-pubsub-queue-7bm9n-6kdq4   True    True     my-pubsub-queue-7bm9n-6kdq4   3m22s
4
5NAME                                                        READY   SYNCED   EXTERNAL-NAME                 AGE
6bucket.storage.gcp.upbound.io/my-pubsub-queue-7bm9n-hhwx8   True    True     my-pubsub-queue-7bm9n-hhwx8   3m22s

Deleting the Claim deletes all the Crossplane generated resources.

kubectl delete claim -n crossplane-test my-pubsub-queue

1kubectl delete pubsubclaim my-pubsub-queue -n crossplane-test
2pubsubclaim.queue.example.com "my-pubsub-queue" deleted

Verify Crossplane deleted the composite resource with kubectl get composite.

1kubectl get composite
2No resources found

Verify Crossplane deleted the managed resources with kubectl get managed.

1kubectl get managed
2No resources found

Next steps

• Explore AWS resources that Crossplane can configure in the Provider CRD reference.
• Join the Crossplane Slack and connect with Crossplane users and contributors.
• Read more about the Crossplane concepts to find out what else you can do with Crossplane.