OCI Artifacts and a View of the Future

Cross Repo & Registry Signing, Catalog Search and Eventing APIs are some of the next things to come – with your help

This past week, the OCI TOB approved the creation of OCI Artifacts. A means to leverage OCI distribution spec compliant registries for serving additional content. For more info on artifacts, see this post: Artifact Registries evolve from Docker Container Registries and the PRs for OCI Artifacts in flight.

Being able to deliver any artifact, with the same secured content delivery solution is just the beginning. By consolidating content delivery on distribution, we have the basis to continue to improve.

To put some context the impact Artifacts can have, here’s one view of what I hope we can achieve.

Container Registry & Distribution Components

Registry Content Delivery can be broken down into 3 distinct components:

Registry

A registry, as an implementation of the OCI Distribution Spec, serves a few primary roles:

Persistence of Content, Stored as Blobs
Aggregation of Blobs, Defined by Manifests
A REST endpoint for Content Discovery
Content Delivery
Authentication & Authorization

Manifest Schemas

For a registry to store collections of content, it must have well known schemas to uniquely describe each content addressable object. The OCI Manifest and OCI Index are two well known schemas that implementations of the OCI distribution spec MUST support.

While it’s possible for registries to implement additional manifest schemas, the authors of the OCI schemas built in a lot of flexibility that enables the majority of scenarios we know today.

Artifacts

Artifacts, like docker and OCI images are well known types of artifacts. Using the same schemas and persistence, additional artifacts can be defined, persisted and served with OCI distribution spec compliant registries.

OCI Artifacts decouples registries from uniquely storing OCI Images, providing a generic means to define and store additional artifact types.

WIth a generalization of images to artifacts, what else can we add on?

True Content Signing

One of the challenges we face with containers is the ability to truly sign the content we store in a registry. As customers pull images from Docker Hub, the Microsoft Container Registry (MCR), Quay, ecr, gcr and other registries, they likely build and push to their own network-close registry, within their control. (See Choosing a Docker Container Registry). When the images are built, do consumers know the lineage of the image? Are they sure the layers they’re built upon are the same as the original layers they pulled?

When customers move built or acquired images from their development registry to a production registry, or even move repos within the same registry, the signature should be maintained.

When evaluating the content on a given machine, does the source by which the content was retrieved really matter? Or, can the content be verified as that by which it was initially produced?

It turns out we don’t actually have a solution for this today. Docker provided Content Trust, which does solve the verification of content, as delivered from the registry to its destination. However, images singed with content trust can not be moved to another registry, or even to another repository within the same registry. Content trust signs the full content addressable signature, including the registry and repository path.

Vendor Specific Signing Solutions

A number of efforts have been tossed around. We’ve even considered a solution we can deliver within Azure. However, Microsoft is a software company, in addition to a cloud vendor. Delivering a solution that only works within Azure, would not have the same impact. Customers that pull Windows, SQL, and the hundreds of other images available from mcr must be able to run them on-prem and other clouds.

Just as Artifacts can only be truly successful if it’s supported across all registries, content signing will need a community driven, open solution as well.

There’s some interesting conversations happening around TUF and in-toto. If we can find a good balance of usability, verification, and content movement, we might have a contender.

Singing All Artifact Content

The main point, is the solution we identify to sign and verify content should be all artifact types, not just images. Regardless of what the Artifact type pulled from an OCI based registry, regardless of where it wound up, from the container host in your cloud, to the firmware update in your car, the content should be verifiable it originated from where you thought, and it’s still in-tact.

Common Registry Search and Eventing APIs

If you work for a container vulnerability scanning company, or work across multiple clouds, you already painfully know that all registries are not created equally. For instance, there’s no common way to query a registry for its contents. You’ve also probably noticed there’s no common registry tools, or catalog viewers of registries either. Wouldn’t it be nice if you could run a container that could present a docker hub like experience for browsing all the artifacts in your registry? What if you could see the different artifact types, not just container images? A while back, we published the ACR Web Manager as a means to browse content in ACR. It turns out it was difficult to maintain, partially because there’s no standard APIs.

Docker had initially designed the _catalog API as a means to query the list of repositories, and a tag listing api to get the list of tags for a given repo. While the vast majority of the distribution spec has exceeded expectations, there’s always some things that crop up. It turns out the distribution-spec was intentionally open for vendors and cloud operators to build in their implementation. There was enough variance in auth and how registries are considered unique for each customer (login url, vs. root namespace), that _catalog didn’t work out.

Now that registries will be able to start supporting all types of artifacts, it will be even more important for a set of common APIs, so all artifact tools can consistently search and find the artifacts they care about across all implementations of the OCI distribution-spec.

Common Eventing for Artifacts

As developers continually automate their workflows, we have steps along the workflow that will take time to complete. Content may need to be scanned, and we may want to Quarantine the content before it is released, or the next step continues. The content may get geo-replicated between regions, enabling the hundreds of hosts on another continent the ability to pull the artifacts locally. You’ll want to get notified when a regional webhook event fires, letting you know “you’ve got artifacts”.

This is another area where some work was done for registry web-hooks, but it spawned out to unique implementations per registry operator.

Search and Eventing Requirements

Within the OCI working group, a number of folks have started discussing what we could do. As engineers, we were quick to propose solutions, but we decided to step back and gather requirements, starting with: OCI Artifact Search Requirements. At KubeCon 2018 EU, a number of us gathered to discuss the topic in person: KubeCon 2019 Notes: OCI Catalog Listing APIs

We Need Your Help

I thought about making an Uncle Sam version of OCI Needs You, but, well, I figured this is a different climate, and we’ll stick the tech and geeky stuff.

While we’ve had some great ideas for signing, search, eventing and some other interesting thoughts, the same core people working on artifacts were working on these as well. As these are being developed in the open-not everyone can dedicate their work hours to invest here. Or, we simply have a lot already on our plate.

We really need people that want to run with the ball, take this to its fruition, and inspire with new thoughts we hadn’t yet thought of.

If you’ve got a great job, but interested in helping the community, think about joining the OCI discussions on slack or the weekly OCI calls.

If you’re still interested in helping the community but would like to join us in the Azure Container Registry team, look here or DM me.

There’s a lot of exciting work happening in the container, and now artifact space. It’s exciting to see the potential, and the ideas and passion people in the community have. I hope you’ll considering joining, and helping out.

Steve