AWS Quickstart Part 2
This document applies to the Crossplane master
branch and not to the latest release v1.19.
This guide is part 2 of a series.
Part 1 covers to installing Crossplane and connect your Kubernetes cluster to AWS.
This guide walks you through building and accessing a custom API with Crossplane.
Prerequisites
- Complete quickstart part 1 connecting Kubernetes to AWS.
- an AWS account with permissions to create an AWS S3 storage bucket and a DynamoDB instance
- Add the Crossplane Helm repository and install Crossplane
1helm repo add \
2crossplane-stable https://charts.crossplane.io/stable
3helm repo update
4
5helm install crossplane \
6crossplane-stable/crossplane \
7--namespace crossplane-system \
8--create-namespace
- When the Crossplane pods finish installing and are ready, apply the AWS Provider
1cat <<EOF | kubectl apply -f -
2apiVersion: pkg.crossplane.io/v1
3kind: Provider
4metadata:
5 name: provider-aws-s3
6spec:
7 package: xpkg.crossplane.io/crossplane-contrib/provider-aws-s3:v1.21.1
8EOF
- Create a file with your AWS keys
- Create a Kubernetes secret from the AWS keys
1kubectl create secret \
2generic aws-secret \
3-n crossplane-system \
4--from-file=creds=./aws-credentials.txt
- Create a ProviderConfig
Install the DynamoDB Provider
Part 1 only installed the AWS S3 Provider. This section deploys an S3 bucket along with a DynamoDB Table. Deploying a DynamoDB Table requires the DynamoDB Provider as well.
Add the new Provider to the cluster.
1cat <<EOF | kubectl apply -f -
2apiVersion: pkg.crossplane.io/v1
3kind: Provider
4metadata:
5 name: provider-aws-dynamodb
6spec:
7 package: xpkg.crossplane.io/crossplane-contrib/provider-aws-dynamodb:v1.21.1
8EOF
View the new DynamoDB provider with kubectl get providers
.
1kubectl get providers
2NAME INSTALLED HEALTHY PACKAGE AGE
3crossplane-contrib-provider-family-aws True True xpkg.crossplane.io/crossplane-contrib/provider-family-aws:v1.21.1 15m
4provider-aws-dynamodb True True xpkg.crossplane.io/crossplane-contrib/provider-aws-dynamodb:v1.21.1 22s
5provider-aws-s3 True True xpkg.crossplane.io/crossplane-contrib/provider-aws-s3:v1.21.1 15m
Create a custom API
Crossplane allows you to build your own custom APIs for your users, abstracting away details about the cloud provider and their resources. You can make your API as complex or simple as you wish.
The custom API is a Kubernetes object. Here is an example custom API.
1apiVersion: database.example.com/v1alpha1
2kind: NoSQL
3metadata:
4 name: my-nosql-database
5spec:
6 location: "US"
Like any Kubernetes object the API has a
,
and
.
Define a group and version
To create your own API start by defining an API group and version.
The group can be any value, but common convention is to map to a fully qualified domain name.
The version shows how mature or stable the API is and increments when changing, adding or removing fields in the API.
Crossplane doesn’t require specific versions or a specific version naming convention, but following Kubernetes API versioning guidelines is strongly recommended.
v1alpha1
- A new API that may change at any time.v1beta1
- An existing API that’s considered stable. Breaking changes are strongly discouraged.v1
- A stable API that doesn’t have breaking changes.
This guide uses the group
.
Because this is the first version of the API, this guide uses the version
.
1apiVersion: database.example.com/v1alpha1
Define a kind
The API group is a logical collection of related APIs. In a group are individual kinds representing different resources.
For example a database
group may have a Relational
and NoSQL
kinds.
The kind
can be anything, but it must be
UpperCamelCased.
This API’s kind is
Define a spec
The most important part of an API is the schema. The schema defines the inputs accepted from users.
This API allows users to provide a
of where to run their
cloud resources.
All other resource settings can’t be configurable by the users. This allows Crossplane to enforce any policies and standards without worrying about user errors.
Apply the API
Crossplane uses
(also called an XRD
) to install your custom API in
Kubernetes.
The XRD
contains all the
information about the API including the
,
,
and
.
The XRD’s
must be the
combination of the
and
.
The
uses the
specification to define
the API
.
The API defines a
that
must be
either
or
.
Apply this XRD to create the custom API in your Kubernetes cluster.
1cat <<EOF | kubectl apply -f -
2apiVersion: apiextensions.crossplane.io/v1
3kind: CompositeResourceDefinition
4metadata:
5 name: nosqls.database.example.com
6spec:
7 group: database.example.com
8 names:
9 kind: NoSQL
10 plural: nosqls
11 versions:
12 - name: v1alpha1
13 schema:
14 openAPIV3Schema:
15 type: object
16 properties:
17 spec:
18 type: object
19 properties:
20 location:
21 type: string
22 oneOf:
23 - pattern: '^EU$'
24 - pattern: '^US$'
25 required:
26 - location
27 served: true
28 referenceable: true
29 claimNames:
30 kind: NoSQLClaim
31 plural: nosqlclaim
32EOF
Adding the
allows users
to access this API either at the cluster level with the
endpoint or in a namespace
with the
endpoint.
The namespace scoped API is a Crossplane Claim.
View the installed XRD with kubectl get xrd
.
View the new custom API endpoints with kubectl api-resources | grep nosql
1kubectl api-resources | grep nosql
2nosqlclaim database.example.com/v1alpha1 true NoSQLClaim
3nosqls database.example.com/v1alpha1 false NoSQL
Create a deployment template
When users access the custom API Crossplane takes their inputs and combines them with a template describing what infrastructure to deploy. Crossplane calls this template a Composition.
The
defines all the
cloud resources to deploy. Each entry in the template is a full resource
definition, defining all the resource settings and metadata like labels and
annotations.
This template creates an AWS
and a
.
This Composition takes the user’s
input and uses it as the
used in the individual
resource.
This Composition uses an array of resource templates. You can patch each template with data copied from the custom API. Crossplane calls this a Patch and Transform Composition.
You don’t have to use Patch and Transform. Crossplane supports a variety of alternatives, including Go Templating and CUE. You can also write a function in Go or Python to template your resources.
Read the Composition documentation for more information on configuring Compositions and all the available options.
Apply this Composition to your cluster.
1cat <<EOF | kubectl apply -f -
2apiVersion: apiextensions.crossplane.io/v1
3kind: Composition
4metadata:
5 name: dynamo-with-bucket
6spec:
7 mode: Pipeline
8 pipeline:
9 - step: patch-and-transform
10 functionRef:
11 name: function-patch-and-transform
12 input:
13 apiVersion: pt.fn.crossplane.io/v1beta1
14 kind: Resources
15 resources:
16 - name: s3Bucket
17 base:
18 apiVersion: s3.aws.upbound.io/v1beta1
19 kind: Bucket
20 spec:
21 forProvider:
22 region: us-east-2
23 providerConfigRef:
24 name: default
25 patches:
26 - type: FromCompositeFieldPath
27 fromFieldPath: "spec.location"
28 toFieldPath: "spec.forProvider.region"
29 transforms:
30 - type: map
31 map:
32 EU: "eu-north-1"
33 US: "us-east-2"
34 - name: dynamoDB
35 base:
36 apiVersion: dynamodb.aws.upbound.io/v1beta1
37 kind: Table
38 spec:
39 forProvider:
40 region: "us-east-2"
41 writeCapacity: 1
42 readCapacity: 1
43 attribute:
44 - name: S3ID
45 type: S
46 hashKey: S3ID
47 patches:
48 - type: FromCompositeFieldPath
49 fromFieldPath: "spec.location"
50 toFieldPath: "spec.forProvider.region"
51 transforms:
52 - type: map
53 map:
54 EU: "eu-north-1"
55 US: "us-east-2"
56 compositeTypeRef:
57 apiVersion: database.example.com/v1alpha1
58 kind: NoSQL
59EOF
The
defines
which custom APIs can use this template to create resources.
A Composition uses a pipeline of composition functions to define the cloud
resources to deploy. This template uses
.
You must install the function before you can use it in a Composition.
Apply this Function to install function-patch-and-transform
:
1cat <<EOF | kubectl apply -f -
2apiVersion: pkg.crossplane.io/v1
3kind: Function
4metadata:
5 name: function-patch-and-transform
6spec:
7 package: xpkg.crossplane.io/crossplane-contrib/function-patch-and-transform:v0.8.2
8EOF
Read the Composition documentation for more information on configuring Compositions and all the available options.
Read the Patch and Transform function documentation for more information on how it uses patches to map user inputs to Composition resource templates.
View the Composition with kubectl get composition
1kubectl get composition
2NAME XR-KIND XR-APIVERSION AGE
3dynamo-with-bucket NoSQL database.example.com/v1alpha1 3s
Access the custom API
With the custom API (XRD) installed and associated to a resource template (Composition) users can access the API to create resources.
Create a
object to create the
cloud resources.
1cat <<EOF | kubectl apply -f -
2apiVersion: database.example.com/v1alpha1
3kind: NoSQL
4metadata:
5 name: my-nosql-database
6spec:
7 location: "US"
8EOF
View the resource with kubectl get nosql
.
1kubectl get nosql
2NAME SYNCED READY COMPOSITION AGE
3my-nosql-database True True dynamo-with-bucket 14s
This object is a Crossplane composite resource (also called an XR
).
It’s a
single object representing the collection of resources created from the
Composition template.
View the individual resources with kubectl get managed
1kubectl get managed
2NAME READY SYNCED EXTERNAL-NAME AGE
3table.dynamodb.aws.upbound.io/my-nosql-database-t5wtx True True my-nosql-database-t5wtx 33s
4
5NAME READY SYNCED EXTERNAL-NAME AGE
6bucket.s3.aws.upbound.io/my-nosql-database-xtzph True True my-nosql-database-xtzph 33s
Delete the resources with kubectl delete nosql
.
Verify Crossplane deleted the resources with kubectl get managed
Using the API with namespaces
Accessing the API nosql
happens at the cluster scope.
Most organizations
isolate their users into namespaces.
A Crossplane Claim is the custom API in a namespace.
Creating a Claim is just like accessing the custom API endpoint, but with the
from the custom API’s claimNames
.
Create a new namespace to test create a Claim in.
1kubectl create namespace crossplane-test
Then create a Claim in the crossplane-test
namespace.
1cat <<EOF | kubectl apply -f -
2apiVersion: database.example.com/v1alpha1
3kind: NoSQLClaim
4metadata:
5 name: my-nosql-database
6 namespace: crossplane-test
7spec:
8 location: "US"
9EOF
View the Claim with kubectl get claim -n crossplane-test
.
1kubectl get claim -n crossplane-test
2NAME SYNCED READY CONNECTION-SECRET AGE
3my-nosql-database True True 17s
The Claim automatically creates a composite resource, which creates the managed resources.
View the Crossplane created composite resource with kubectl get composite
.
1kubectl get composite
2NAME SYNCED READY COMPOSITION AGE
3my-nosql-database-t9qrw True True dynamo-with-bucket 77s
Again, view the managed resources with kubectl get managed
.
1kubectl get managed
2NAME READY SYNCED EXTERNAL-NAME AGE
3table.dynamodb.aws.upbound.io/my-nosql-database-t9qrw-dcpwv True True my-nosql-database-t9qrw-dcpwv 116s
4
5NAME READY SYNCED EXTERNAL-NAME AGE
6bucket.s3.aws.upbound.io/my-nosql-database-t9qrw-g98lv True True my-nosql-database-t9qrw-g98lv 117s
Deleting the Claim deletes all the Crossplane generated resources.
kubectl delete claim -n crossplane-test my-nosql-database
1kubectl delete claim -n crossplane-test my-nosql-database
2nosqlclaim.database.example.com "my-nosql-database" deleted
Verify Crossplane deleted the composite resource with kubectl get composite
.
Verify Crossplane deleted the managed resources with kubectl get managed
.
Next steps
- Explore AWS resources that Crossplane can configure in the provider CRD reference.
- Join the Crossplane Slack and connect with Crossplane users and contributors.
- Read more about the Crossplane concepts to find out what else you can do with Crossplane.