Kubernetes 1.31: Read Only Volumes Based On OCI Artifacts (alpha)

By Sascha Grunert |Friday, August 16, 2024

The Kubernetes community is moving towards fulfilling more ArtificialIntelligence (AI) and Machine Learning (ML) use cases in the future. While theproject has been designed to fulfill microservice architectures in the past,it’s now time to listen to the end users and introduce features which have astronger focus on AI/ML.

One of these requirements is to support Open Container Initiative (OCI)compatible images and artifacts (referred as OCI objects) directly as a nativevolume source. This allows users to focus on OCI standards as well as enablesthem to store and distribute any content using OCI registries. A feature likethis gives the Kubernetes project a chance to grow into use cases which gobeyond running particular images.

Given that, the Kubernetes community is proud to present a new alpha featureintroduced in v1.31: The Image Volume Source(KEP-4639). This feature allows users to specify animage reference as volume in a pod while reusing it as volume mount withincontainers:

…kind: Podspec:  containers:    - …      volumeMounts:        - name: my-volume          mountPath: /path/to/directory  volumes:    - name: my-volume      image:        reference: my-image:tag

The above example would result in mounting my-image:tag to/path/to/directory in the pod’s container.

Use cases

The goal of this enhancement is to stick as close as possible to the existingcontainer image implementation within thekubelet, while introducing a new API surface to allow more extended use cases.

For example, users could share a configuration file among multiple containers ina pod without including the file in the main image, so that they can minimizesecurity risks and the overall image size. They can also package and distributebinary artifacts using OCI images and mount them directly into Kubernetes pods,so that they can streamline their CI/CD pipeline as an example.

Data scientists, MLOps engineers, or AI developers, can mount large languagemodel weights or machine learning model weights in a pod alongside amodel-server, so that they can efficiently serve them without including them inthe model-server container image. They can package these in an OCI object totake advantage of OCI distribution and ensure efficient model deployment. Thisallows them to separate the model specifications/content from the executablesthat process them.

Another use case is that security engineers can use a public image for a malwarescanner and mount in a volume of private (commercial) malware signatures, sothat they can load those signatures without baking their own combined image(which might not be allowed by the copyright on the public image). Those fileswork regardless of the OS or version of the scanner software.

But in the long term it will be up to you as an end user of this project tooutline further important use cases for the new feature.SIG Nodeis happy to retrieve any feedback or suggestions for further enhancements toallow more advanced usage scenarios. Feel free to provide feedback by eitherusing the Kubernetes Slack (#sig-node)channel or the SIG Node mailinglist.

Detailed example

The Kubernetes alpha feature gate ImageVolumeneeds to be enabled on the API Serveras well as the kubeletto make it functional. If that’s the case and the container runtimehas support for the feature (like CRI-O ≥ v1.31), then an example pod.yamllike this can be created:

apiVersion: v1kind: Podmetadata:  name: podspec:  containers:    - name: test      image: registry.k8s.io/e2e-test-images/echoserver:2.3      volumeMounts:        - name: volume          mountPath: /volume  volumes:    - name: volume      image:        reference: quay.io/crio/artifact:v1        pullPolicy: IfNotPresent

The pod declares a new volume using the image.reference ofquay.io/crio/artifact:v1, which refers to an OCI object containing two files.The pullPolicy behaves in the same way as for container images and allows thefollowing values:

• Always: the kubelet always attempts to pull the reference and the containercreation will fail if the pull fails.
• Never: the kubelet never pulls the reference and only uses a local image orartifact. The container creation will fail if the reference isn’t present.
• IfNotPresent: the kubelet pulls if the reference isn’t already present ondisk. The container creation will fail if the reference isn’t present and thepull fails.

The volumeMounts field is indicating that the container with the name testshould mount the volume under the path /volume.

If you now create the pod:

kubectl apply -f pod.yaml

And exec into it:

kubectl exec -it pod -- sh

Then you’re able to investigate what has been mounted:

/ # ls /volumedir   file/ # cat /volume/file2/ # ls /volume/dirfile/ # cat /volume/dir/file1

You managed to consume an OCI artifact using Kubernetes!

The container runtime pulls the image (or artifact), mounts it to thecontainer and makes it finally available for direct usage. There are a bunch ofdetails in the implementation, which closely align to the existing image pullbehavior of the kubelet. For example:

• If a :latest tag as reference is provided, then the pullPolicy willdefault to Always, while in any other case it will default to IfNotPresentif unset.
• The volume gets re-resolved if the pod gets deleted and recreated, which meansthat new remote content will become available on pod recreation. A failure toresolve or pull the image during pod startup will block containers fromstarting and may add significant latency. Failures will be retried usingnormal volume backoff and will be reported on the pod reason and message.
• Pull secrets will be assembled in the same way as for the container image bylooking up node credentials, service account image pull secrets, and pod specimage pull secrets.
• The OCI object gets mounted in a single directory by merging the manifestlayers in the same way as for container images.
• The volume is mounted as read-only (ro) and non-executable files(noexec).
• Sub-path mounts for containers are not supported(spec.containers[*].volumeMounts.subpath).
• The field spec.securityContext.fsGroupChangePolicy has no effect on thisvolume type.
• The feature will also work with the AlwaysPullImages admission pluginif enabled.

Thank you for reading through the end of this blog post! SIG Node is proud andhappy to deliver this feature as part of Kubernetes v1.31.

As writer of this blog post, I would like to emphasize my special thanks toall involved individuals out there! You all rock, let’s keep on hacking!

Further reading