If you have ever tried to deploy a service in the last three to four years, there's a good chance someone has suggested using Kubernetes; however, if this is your first time hearing of Kubernetes, welcome. Kubernetes is an orchestrator that makes it easy to run large-scale containerized applications.
What makes Kubernetes really great is its extensibility, by using the API users are able to create custom applications that don't ship with Kubernetes out of the box, It's how we have great tools like ArgoCD.
If you clicked on this post, you are likely familiar with Kubernetes and looking to take things a step further, regardless of why you are here. In this article, we will look at a bunch of useful operations for interacting with Kubernetes using the planet's most loved crab language, Rust.
Getting Started
To create a local Kubernetes cluster, we will use KinD, a tool that allows you to run Kubernetes in Docker, hence the name KinD. Don't forget to have Docker installed!
If you're on Mac, you can install kind using brew:
For Linux users, if you have Go installed and path set correctly, you can install KinD using:
With that installed, you can create a new cluster by running:
Pre-requisites
Along with KinD, you will also need to install Kubectl. If you are feeling adventurous, you can install that right away using:
And for ARM machines:
Otherwise, take a look at this section of the Kubernetes docs for more specific installation instructions.
Kube-rs?
To interact with the Kubernetes API we will need a client library. As described in the project's readme, kube-rs
is:
| A Rust client for Kubernetes in the style of a more generic client-go, a runtime abstraction inspired by controller-runtime, and a derive macro for CRDs inspired by Kubebuilder
Project Setup
Create a new cargo project:
Add the following crates to your Cargo.toml
Connecting to Your Cluster
We'll begin by looking at how to connect to the cluster we created earlier. By default, your connection information is stored in ~/.kube/config
. To use this in Rust, we can use the following code:
We'll begin by looking at how to connect to the cluster we created earlier:
Using Config::infer()
kube-rs
will attempt to connect to the cluster using your default kubeconfig.
Creating a Client
Using the config is great for printing out cluster information, but we can't use it to interact with it. To create a new cluster, we can use Client::try_default()
, which calls Client::infer()
under the hood and returns a new client:
Working with Pods
Pods are a fundamental resource in Kubernetes. To create one using kube-rs
we can use the following code:
Create a pod
This works, but as we start to work with larger objects, it can quickly get confusing, and who really likes writing json?
Alternatively, you can create a pod by using a struct; we will be using this method going forward.
Listing pods
Another common operation is listing pods. With the code snippet below, we can list the pods in the kube-system
namespace.
We can also filter what pods by using labels. Labels are customizable key/value tags attached to objects, used for organization.
In this example, only pods with the label component=kube-apiserver
will be returned.
Updating Pods
While creating and listing pods are everyday tasks, you'll often need to update existing pods. Let's look at an example of adding labels to a pod.
First, let's add a label to our existing pong-pod
:
Deleting Pods
Finally, we can delete pods using the following code:
Interacting with running Pods
Switching gears a little, let's explore how to interact with running pods.
Fetching logs from a Pod
Often, you'll want to see what's happening inside a pod to debug issues or monitor application behavior. We can mimic the functionality of kubectl logs
using the kube-rs. Here's a snippet showing how to fetch logs from the API server:
This example demonstrates a slightly more realistic scenario of working with Pods in a Kubernetes cluster. We create a Pod running Nginx, wait for it to be ready, and then fetch its logs.
After creating the Pod, we use the status
field to determine if the Pod is running:
Port forwarding to a Pod
Sometimes, you need to interact directly with a service running in a pod. We can replicate the functionality of kubectl port-forward
. Here's a stripped-down version of port forwarding, adapted from the kube-rs repository:
First, let's set up the Kubernetes client and wait for our pod to be running:
Next, we'll set up the local address to forward traffic to:
Using a TcpListenerStream
, we set up a local TCP server that listens for incoming connections:
Finally, let's look at the function that forwards the connection:
Working with Custom Resources
Kubernetes has an interesting concept called custom resources. These allow you to extend the Kubernetes API and add new resources. This by itself isn't very descriptive, so let's take things a step further with an example.
Say you wanted to inform Kubernetes about a new resource called crustacean
. By default if you run kubectl get crustacean
, you will get an error saying error: the server doesn't have a resource type "crustacean."
. To create the resource, start out by defining a new custom resource:
We can generate a manifest using the following code:
Running the above code will produce the following manifest:
Great, but Kubernetes still has no idea how to handle this CRD. To do this, we need to apply the custom resource definition to the cluster:
There are a couple of things going on in the snippet above, the important parts:
- Using the
fs
crate, we load the yaml file from the current directory. - We introduce a helper method called
new_crustacean
to help with some of the setup for the CRD definition. After extracting the values from the yaml file, we pass the parameters to thenew_crustacean
function and apply the manifest. - The
ensure_crd
function checks if the CRD exists in the cluster before applying the manifest, this is essential because kubernetes will try and validated the manifest we apply against the CRD schema.
At this point, you should be able to run:
The output is similar to:
Watching for Resource Changes
So we've got our Crustacean
CRD up and running. But creating resources is only half the battle. More often you want to react to events in the cluster this is where watchers come in, as the name suggests, watchers are used to monitor specific events.
In our case, we'll monitor for new Crustacean
resources and create a pod for each one.
Using the following code (as from the above code snippet):
We set up a watcher to monitor the Crustacean
custom resource, using the applied_objects()
: method ensures that we only care about the resources that have been applied (created or updated). It filters out other events like deletions.
In the while loop, we watch for events in the stream provided by the crustacean_watcher
. For each Crustacean
resource that the watcher detects, we create a Pod.
Using await_condition(pods.clone(), &pod_name, is_pod_running())
creates a future that waits for the condition where the Pod is running. is_pod_running()
is a predefined condition function that checks the Pod's status to see if it's in the "Running" state.
Running the example should result in the following output:
And if we run kubectl get pods
:
Sure enough, Larry should be there.
Summary
If you made it to this part, you probably are really interested in kube-rs
. While the library is pretty neat, Kubernetes has a LOT of API's and can quickly get confusing. When working with kube-rs
or Kubernetes in general, here are some things to keep in mind.
Know when to use a controller or an operator.
These concepts have some overlap and can be very confusing, which is why I avoided mentioning so as not to blow this out of scope. Luckily, people far smarter than me have written on this topic. Check out Ivan Velichko's guide on Kubernetes operators. Looking for something with more Rust? The folks at Metalbear have an entire guide on creating an operator using kube-rs.
Examples don't bite
The maintainers of kube-rs
did an amazing job of creating several examples of using the crate. This guide is not an exhaustive list of all its features; some of the examples used here were adapted from the repo.
Using Rust to interact with Kubernetes can be quite refreshing, especially if you come from the Kubebuilder world. Creating a CRD by adding a macro is extremely powerful, and in my experience thus far, I have written less code to achieve common functionality. If you're curious about admission controllers, check out the guide I wrote here.
Conclusion
Of course, you don't need Kubernetes for everything. If you're interested in something more simple, Shuttle is your one-stop shop for easy Rust deployments. Create (or migrate) your project, use shuttle deploy
, and watch the magic happen!