Module container

Expand description

§APIs bridging OSTree and container images

This module provides the core infrastructure for bidirectionally mapping between OCI/Docker container images and OSTree repositories. It enables bootable container images to be fetched from registries, stored efficiently, and deployed as ostree commits.

§Overview

Container images are fundamentally layers of tarballs. This module leverages the crate::tar module to import container layers as ostree content, and exports ostree commits back to container images. The key insight is that ostree’s content-addressed object storage maps naturally to OCI layer deduplication.

When a container image is imported (“pulled”), each layer becomes an ostree commit. These layer commits are then merged into a single “merge commit” that represents the complete filesystem state. This merge commit is what gets deployed as a bootable system.

§On-Disk Storage Structure

Container images are stored in the ostree repository (typically /sysroot/ostree/repo/) using a structured reference (ref) namespace:

§Reference Namespace

ostree/container/blob/<escaped-digest>: Each OCI layer is stored as a separate ostree commit. The digest (e.g., sha256:abc123...) is escaped using crate::refescape to be valid as an ostree ref. For example: ostree/container/blob/sha256_3A_abc123...
ostree/container/image/<escaped-image-reference>: Points to the “merge commit” for a pulled image. The image reference (e.g., docker://quay.io/org/image:tag) is escaped similarly. This is the ref that deployments point to.
ostree/container/baseimage/<project>/<index>: Used to protect base images from garbage collection. Tooling that builds derived images locally should write refs under this prefix to prevent the base layers from being pruned.

§Layer Storage

Each container layer is stored as an ostree commit with a special structure:

OSTree “chunk” layers: Layers that are part of the base ostree commit use the “object set” format - the filenames in the commit are the object checksums. This enables efficient reconstruction of the original ostree commit.
Derived layers: Non-ostree layers (e.g., from RUN commands in a Containerfile) are imported as regular filesystem trees and stored as standard ostree commits.

§The Merge Commit

The merge commit (ostree/container/image/...) combines all layers into a single filesystem tree. It contains critical metadata in its commit metadata:

ostree.manifest-digest: The OCI manifest digest (e.g., sha256:...)
ostree.manifest: The complete OCI manifest as JSON
ostree.container.image-config: The OCI image configuration as JSON

This metadata enables round-tripping: an imported image can be re-exported with its original manifest structure preserved.

§Import Flow

The import process (implemented in store::ImageImporter) follows these steps:

Manifest fetch: Contact the registry via containers-image-proxy (skopeo) to retrieve the image manifest and configuration.
Layout parsing: Analyze the manifest to identify:
- The base ostree layer (identified by the ostree.final-diffid label)
- Component/chunk layers (split object sets)
- Derived layers (non-ostree content)
Layer caching check: For each layer, check if an ostree ref already exists for that digest. Cached layers are skipped, enabling efficient incremental updates.
Layer import: For uncached layers:
- Fetch the compressed tarball from the registry
- Decompress and parse the tar stream
- Import content into ostree (handling xattrs via bare-split-xattrs format)
- Create an ostree commit and write the layer ref
Merge commit creation: Overlay all layers (processing OCI whiteout files) to create a unified filesystem tree. Apply SELinux labeling if needed. Store manifest/config metadata and write the image ref.
Garbage collection: Prune layer refs that are no longer referenced by any image or deployment.

§Tar Stream Format

The tar format used for ostree layers is documented in crate::tar. Key points:

Uses bare-split-xattrs repository mode to handle extended attributes
XAttrs are stored in separate .file-xattrs objects, avoiding tar xattr complexity
/etc in container images maps to /usr/etc in ostree (the “3-way merge” location)
Hardlinks are used for deduplication within layers

§Connection to Deployments

When bootc deploys an image, it creates an ostree deployment whose “origin” file references the container image. The origin contains:

The OstreeImageReference specifying the image and signature verification method
The merge commit checksum

On subsequent boots, bootc can compare the deployed commit against the registry manifest to detect available updates.

§Signatures

OSTree supports GPG and ed25519 signatures natively. When fetching container images, signature verification can be configured via SignatureSource:

OstreeRemote(name): Verify using the named ostree remote’s keyring
ContainerPolicy: Defer to containers-policy.json (requires explicit allow)
ContainerPolicyAllowInsecure: Use containers-policy.json defaults (not recommended)

This library defines a URL-like schema to combine signature verification with image references:

ostree-remote-registry:<remotename>:<containerimage> - Verify via ostree remote
ostree-image-signed:<transport>:<image> - Use container policy
ostree-unverified-registry:<image> - No verification (not recommended)

Example: ostree-remote-registry:fedora:quay.io/fedora/fedora-bootc:latest

See OstreeImageReference for parsing and generating these strings.

§Layering and Derived Images

Container image layering is fully supported. A typical bootable image structure:

Base ostree layer: Contains the core OS as an ostree commit
Chunk layers: Split objects for efficient updates (optional)
Derived layers: Additional content from Containerfile RUN commands

The ostree.final-diffid label in the image configuration marks where the ostree content ends and derived content begins. This enables:

Efficient layer sharing between images with the same base
Proper SELinux labeling of derived content using the base policy
Round-trip export preserving the layer structure

§Key Types

Transport: OCI/Docker transport (registry, oci-dir, containers-storage, etc.)
ImageReference: Container image reference with transport
OstreeImageReference: Image reference plus signature verification method
SignatureSource: How to verify image signatures
store::ImageImporter: Main import orchestrator
store::PreparedImport: Analysis of layers to fetch
store::LayeredImageState: State of a pulled image
ManifestDiff: Comparison between two image manifests

§Submodules

store: Core storage and import logic
deploy: Integration with ostree deployments
skopeo: Skopeo subprocess management for registry operations

Modules§

deploy: Perform initial setup for a container image based system root
encapsulate 🔒: APIs for creating container images from OSTree commits
skopeo: Fork skopeo as a subprocess
store: Storing layered container images as OSTree commits
unencapsulate 🔒: APIs for “unencapsulating” OSTree commits from container images
update_detachedmeta 🔒

Structs§

Config: Configuration for the generated container.
ExportOpts: Options controlling commit export into OCI
ImageReference: Combination of a remote image reference and transport.
Import: The result of an import operation
ManifestDiff: Represents the difference in layer/blob content between two OCI image manifests.
OstreeImageReference: Combination of a signature verification mechanism, and a standard container image reference.

Enums§

SignatureSource: Policy for signature verification.
Transport: A backend/transport for OCI/Docker images.

Constants§

BOOTC_LABEL: The label for bootc.
COMPONENT_SEPARATOR 🔒: The character we use to separate values in CONTENT_ANNOTATION.
CONTENT_ANNOTATION 🔒: The name of an annotation attached to a layer which names the packages/components which are part of it.
DIFFID_LABEL: The label which indicates where the ostree layers stop, and the derived ones start.
LABEL_VERSION: A commonly used pre-OCI label for versions.
LEGACY_VERSION_LABEL: The label which may be used in addition to the standard OCI label.
OSTREE_COMMIT_LABEL: The label injected into a container image that contains the ostree commit SHA-256.

Functions§

encapsulate: Given an OSTree repository and ref, generate a container image.
fetch_manifest: Download the manifest for a target image and its sha256 digest.
fetch_manifest_and_config: Download the manifest for a target image and its sha256 digest, as well as the image configuration.
labels_of 🔒: Convenience helper to return the labels, if present.
merge_default_container_proxy_opts: Apply default configuration for container image pulls to an existing configuration. For example, if authfile is not set, and auth_anonymous is false, and a global configuration file exists, it will be used.
merge_default_container_proxy_opts_with_isolation: Apply default configuration for container image pulls, with optional support for isolation as an unprivileged user.
unencapsulate: Fetch a container image and import its embedded OSTree commit.
update_detached_metadata: Given an OSTree container image reference, update the detached metadata (e.g. GPG signature) while preserving all other container image metadata.
version_for_config: Retrieve the version number from an image configuration.

Type Aliases§

Result 🔒: Our generic catchall fatal error, expected to be converted to a string to output to a terminal or logs.