{"id":1703,"date":"2021-11-05T16:42:18","date_gmt":"2021-11-05T23:42:18","guid":{"rendered":"http:\/\/blog.nillsf.com\/?p=1703"},"modified":"2021-11-05T16:42:25","modified_gmt":"2021-11-05T23:42:25","slug":"creating-kubernetes-clusters-on-azure-using-cluster-api","status":"publish","type":"post","link":"https:\/\/nillsf.com\/index.php\/2021\/11\/05\/creating-kubernetes-clusters-on-azure-using-cluster-api\/","title":{"rendered":"Creating Kubernetes clusters on Azure using cluster API"},"content":{"rendered":"\n

Last week, I wrote a post about how to create a Kubernetes cluster on Azure using kubeadm. It wasn’t as hard as Kubernetes the hard way, but it took some time to get everything setup and get all the infrastructure spun up.<\/p>\n\n\n\n

The goal of this post is to show an easier path to get a non-AKS cluster running on Azure using the cluster API project<\/a>. The goal of the cluster API project is to provide a declarative API and tooling to create and manage Kubernetes clusters on different providers. <\/p>\n\n\n\n

I’ll be using the Cluster API Provider Azure<\/a> for the purpose of this post. This is an implementation of Cluster API for Azure, that spins up infrastructure on Azure to create that cluster.<\/p>\n\n\n\n

Let’s start!<\/p>\n\n\n\n

Chicken and the egg: Using Kubernetes to create more Kubernetes<\/h2>\n\n\n\n

The Cluster API project leverages the Kubernetes API to create new Kubernetes clusters. This essentially means that you need an existing Kubernetes cluster to run the Cluster API. <\/p>\n\n\n\n

To that purpose, I spun up a minikube cluster on my local laptop. This is a M1 MacBook pro, and I was very positively surprised that spinning up a minikube cluster worked from the get-go without any issues. <\/p>\n\n\n\n

brew install minikube\nminikube start\nkubectl get nodes<\/code><\/pre>\n\n\n\n
This gave me a local Kubernetes cluster to work with. <\/p>\n\n\n\n
\"\"
Minikube up and running<\/figcaption><\/figure>\n\n\n\n
You can use any Kubernetes cluster to run the cluster API project. In production use cases, I wouldn’t use minikube for creating your clusters; but would rely on a production Kubernetes cluster (for example an AKS cluster).<\/p>\n\n\n\n
Installing cluster API tooling<\/h2>\n\n\n\n
Next up, you need the cluster API command line tool. Make sure to check out the cluster API documentation<\/a> so you can download the right tool for your environment (which in my case is M1 Mac):<\/p>\n\n\n\n
curl -L https:\/\/github.com\/kubernetes-sigs\/cluster-api\/releases\/download\/v1.0.0\/clusterctl-darwin-arm64 -o clusterctl\nchmod +x .\/clusterctl\nsudo mv .\/clusterctl \/usr\/local\/bin\/clusterctl\nclusterctl version<\/code><\/pre>\n\n\n\n
Which also worked great (this is getting suspicious, isnt it?)<\/em>:<\/p>\n\n\n\n
\"\"
clusterctl tooling available on my workstation<\/figcaption><\/figure>\n\n\n\n
And with that done, we can now go ahead and bootstrap cluster API for Azure on minikube:<\/p>\n\n\n\n
Setting up cluster API for Azure <\/h2>\n\n\n\n
To setup cluster API for Azure, you need a service principal. The secret of this service principal will be stored in a Kubernetes secret. To do this, you can use the following commands:<\/p>\n\n\n\n
export AZURE_SUBSCRIPTION_ID=$(az account show | jq .id | tr -d '\"')\n\n# Create an Azure Service Principal\nexport SERVICE_PRINCIPAL=$(az ad sp create-for-rbac --name nfclusterapiblog --role Contributor -o json)\nexport AZURE_TENANT_ID=$(echo $SERVICE_PRINCIPAL | jq .tenant | tr -d '\"')\nexport AZURE_CLIENT_ID=$(echo $SERVICE_PRINCIPAL | jq .appId | tr -d '\"')\nexport AZURE_CLIENT_SECRET=$(echo $SERVICE_PRINCIPAL | jq .password | tr -d '\"')\n\n# Base64 encode the variables\nexport AZURE_SUBSCRIPTION_ID_B64=\"$(echo -n \"$AZURE_SUBSCRIPTION_ID\" | base64 | tr -d '\\n')\"\nexport AZURE_TENANT_ID_B64=\"$(echo -n \"$AZURE_TENANT_ID\" | base64 | tr -d '\\n')\"\nexport AZURE_CLIENT_ID_B64=\"$(echo -n \"$AZURE_CLIENT_ID\" | base64 | tr -d '\\n')\"\nexport AZURE_CLIENT_SECRET_B64=\"$(echo -n \"$AZURE_CLIENT_SECRET\" | base64 | tr -d '\\n')\"\n\n# Settings needed for AzureClusterIdentity used by the AzureCluster\nexport AZURE_CLUSTER_IDENTITY_SECRET_NAME=\"cluster-identity-secret\"\nexport CLUSTER_IDENTITY_NAME=\"cluster-identity\"\nexport AZURE_CLUSTER_IDENTITY_SECRET_NAMESPACE=\"default\"\n\n# Create a secret to include the password of the Service Principal identity created in Azure\n# This secret will be referenced by the AzureClusterIdentity used by the AzureCluster\nkubectl create secret generic \"${AZURE_CLUSTER_IDENTITY_SECRET_NAME}\" --from-literal=clientSecret=\"${AZURE_CLIENT_SECRET}\"\n<\/code><\/pre>\n\n\n\n
And then finally, you can setup cluster API (for Azure) on the cluster:<\/p>\n\n\n\n
clusterctl init --infrastructure azure<\/code><\/pre>\n\n\n\n
Which will setup cluster API on the minikube cluster:<\/p>\n\n\n\n
\"\"
Cluster API being setup on the minikube cluster<\/figcaption><\/figure>\n\n\n\n
Creating a Kubernetes cluster using Cluster API<\/h2>\n\n\n\n
We are now ready to create a new cluster. First, we’ll setup some Azure environment variables:<\/p>\n\n\n\n
export AZURE_LOCATION=\"westus2\"\nexport AZURE_RESOURCE_GROUP=\"capz-trial\"\n\n# Select VM types.\nexport AZURE_CONTROL_PLANE_MACHINE_TYPE=\"Standard_D2s_v4\"\nexport AZURE_NODE_MACHINE_TYPE=\"Standard_D2s_v4\"<\/code><\/pre>\n\n\n\n
And then we can create the cluster definition:<\/p>\n\n\n\n
clusterctl generate cluster capz-trial \\\n  --kubernetes-version v1.21.0 \\\n  --control-plane-machine-count=3 \\\n  --worker-machine-count=3 \\\n  > capz-trial.yaml<\/code><\/pre>\n\n\n\n
This creates a little over 200 lines of YAML, which you can find here.<\/a><\/p>\n\n\n\n
We can now create this cluster using:<\/p>\n\n\n\n
kubectl apply -f capz-trial.yaml<\/code><\/pre>\n\n\n\n
Which will create the necessary objects in our minikube cluster, to create an Azure cluster:<\/p>\n\n\n\n
\"\"
Creating the Kubernetes cluster on Azure<\/figcaption><\/figure>\n\n\n\n
I then monitored the cluster using kubectl, and apparently after 70 seconds it was provisioned. I had stepped away, assuming it would take a couple minutes so can’t confirm this cluster was operational after 70 seconds. If so, that would be FAST.<\/p>\n\n\n\n
\"\"
Cluster was provisioned after 70 seconds. That was quick.<\/figcaption><\/figure>\n\n\n\n
We can get the kubeconfig to interact with this cluster using:<\/p>\n\n\n\n
clusterctl get kubeconfig capz-trial > capz-trial.kubeconfig<\/code><\/pre>\n\n\n\n
And then do for instance a get nodes:<\/p>\n\n\n\n
\"\"
Using the kubeconfig to get the nodes<\/figcaption><\/figure>\n\n\n\n
The nodes aren’t ready yet, because there’s no CNI setup yet. We can setup a CNI now, and the CAPI project defaults to Calico, which we’ll setup (if you’re following the CAPI docs, please ensure to install the Azure calico CNI, because the default doesn’t work):<\/p>\n\n\n\n
kubectl --kubeconfig=.\/capz-trial.kubeconfig \\\n  apply -f https:\/\/raw.githubusercontent.com\/kubernetes-sigs\/cluster-api-provider-azure\/main\/templates\/addons\/calico.yaml<\/code><\/pre>\n\n\n\n
After that is applied, the nodes should be ready:<\/p>\n\n\n\n
\"\"
With CNI setup, the nodes are ready<\/figcaption><\/figure>\n\n\n\n
And we can now create a sample application, to verify everything is working. I like to use Azure-vote for this purpose. Given CAPZ sets up the Azure load balancer integration, we can use the public version of azure-vote, including an Azure load balancer with public IP:<\/p>\n\n\n\n
kubectl apply -f https:\/\/raw.githubusercontent.com\/Azure-Samples\/azure-voting-app-redis\/master\/azure-vote-all-in-one-redis.yaml --kubeconfig=.\/capz-trial.kubeconfig <\/code><\/pre>\n\n\n\n
This will create the workload and a service. To get the service details, you can use:<\/p>\n\n\n\n
kubectl get svc -w --kubeconfig=.\/capz-trial.kubeconfig <\/code><\/pre>\n\n\n\n
Which will return  you the service’s public IP:<\/p>\n\n\n\n
\"\"
Getting the Azure-vote public IP<\/figcaption><\/figure>\n\n\n\n
And by browsing to that public IP, you should see the Azure vote application:<\/p>\n\n\n\n
\"\"
Azure vote running on a CAPZ cluster<\/figcaption><\/figure>\n\n\n\n
And that’s how you create a CAPZ cluster!<\/p>\n\n\n\n
Let’s also clean up after ourselves. We can do this with:<\/p>\n\n\n\n
kubectl delete cluster capz-trial<\/code><\/pre>\n\n\n\n
And we can also delete the minikube cluster:<\/p>\n\n\n\n
minikube delete<\/code><\/pre>\n\n\n\n
Summary<\/h2>\n\n\n\n
In this post we explored creating a non-AKS Kubernetes cluster on Azure using the cluster API project. In my opinion, it’s a lot easier creating a cluster with cluster API than it is with kubeadm. Less infrastructure to setup, and less interactions with individual machines. Also, cluster API is available for a multitude of infrastructure providers, meaning you can get consistent tooling across cloud providers.<\/p>\n\n\n\n
The downside of using cluster API is that you need an existing Kubernetes API to deploy the infrastructure with. Minikube and kind can solve this for my demo use cases, but I wouldn’t use that in production.<\/p>\n\n\n\n
One very interesting pattern that cluster API unlocks in integrating your Kubernetes cluster definition (i.e. Infrastructure-as-code) with GitOps. This is something I’m exploring at the moment, and will very likely write about soon. Stay tuned!<\/p>\n","protected":false},"excerpt":{"rendered":"
Last week, I wrote a post about how to create a Kubernetes cluster on Azure using kubeadm. It wasn’t as hard as Kubernetes the hard way, but it took some time to get everything setup and get all the infrastructure spun up. The goal of this post is to show an easier path to get […]<\/p>\n","protected":false},"author":1,"featured_media":1714,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[2,58,5],"tags":[189,18],"class_list":["post-1703","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-azure","category-kubernetes","category-open-source","tag-capz","tag-kubernetes"],"jetpack_featured_media_url":"https:\/\/nillsfblog.blob.core.windows.net\/media\/2021\/11\/Screen-Shot-2021-11-05-at-4.39.03-PM.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1703","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/comments?post=1703"}],"version-history":[{"count":1,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1703\/revisions"}],"predecessor-version":[{"id":1713,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/posts\/1703\/revisions\/1713"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/media\/1714"}],"wp:attachment":[{"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/media?parent=1703"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/categories?post=1703"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/nillsf.com\/index.php\/wp-json\/wp\/v2\/tags?post=1703"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}