Migrating to rules_oci

This document contains some of the lessons we've learned at https://aspect.dev from doing consulting work, migrating some large client repos from rules_docker to rules_oci.

This guide remains a work-in-progress as we find new patterns.

Update WORKSPACE

The WORKSPACE file contains Bazel module dependency fetching and installation.

Add install steps from a release of rules_oci, along with related rulesets you plan to use.

Note that the container_test rule from rules_docker is now in a separate repo. See container_structure_test

Replacements by rule

rules_docker	rules_oci	How to migrate
container_push	oci_push	See below
container_image	oci_image	See below
container_pull	oci_pull	See below
container_test	N/A	Use container_structure_test
container_import	N/A
container_load	N/A
go_image	N/A	Wrap `go_binary`, see go docs
java_image	N/A	Wrap `java_binary`, see java docs
py3_image	N/A	Wrap `py_binary`, see python docs
nodejs_image	N/A	Wrap `js_binary`, see javascript docs
rust_image	N/A	Wrap `rust_binary`, see rust docs

container_pull

puller_darwin, puller_linux_* are unsupported. Unlike container_pull, oci_pull uses Bazel's downloader instead of a custom puller binary.

import_tags is unsupported. no plans to add support for it.

cred_helpers is unsupported. credential helpers should be installed on the host where bazel runs.

docker_client_config is unsupported. instead use DOCKER_CONFIG environment variable to override the config.

os_version is unsupported.

os_features is unsupported.

platform_features is unsupported.

os, architecture, cpu_variant is supported, however, are combined into single string that can be passed to platforms attributes.

-load("@io_bazel_rules_docker//container:container.bzl", "container_pull")
+load("@rules_oci//oci:pull.bzl", "oci_pull")
-container_pull(
+oci_pull(
- os = "linux",
- architecture = "arm64"
- cpu_variant = "v8"
+ platforms = [
+     "linux/arm64/v8"
+ ]
)

timeout is unsupported. Bazel's --http_timeout_scaling can be used to increase/decrease the default timeout.

Environment variables

DOCKER_REPO_CACHE is not supported. Since oci_pull uses Bazel's downloader, remote manifests/blobs are cached via Bazel's repository cache as long as a digest is provided.

PULLER_TIMEOUT is not supported.

container_push

Update BUILD files with: buildozer 'set kind oci_push' //...:%container_push

registry and repository are merged into one attribute; repository.

-registry = "registry.io",
-repository = "org/image"
+repository = "registry.io/org/image"

tag attribute is now remote_tags and allows multiple tags.

-tag = "latest",
+remote_tags = ["latest"]

insecure_repository is not needed anymore as it's automatically detected for most cases. However, in case it can't be detected --insecure can be added to the args attribute.

-insecure_repository = true,
+args = ["--insecure"]

tag_file is now remote_tags and expects one tag per line.

-tag_file = ":tag_file",
+remote_tags = ":tag_file",

repository_file is now repository and accepts a file

-repository_file = ":repository_file",
+repository = ":repository_file",

stamp is not supported directly. We encourage users to stamp in a separate target and pass the stamped file to repository or remote_tags attribute respectively.

# an example from rules_oci stamping tags
stamp_tags(
    name = "stamped",
    remote_tags = [
        # With --stamp, use the --embed_label value, otherwise use 0.0.0
        """($stamp.BUILD_EMBED_LABEL // "0.0.0")""",
        "nightly",
    ],
)

oci_push(
    name = "push",
    image = ":oci_image_target",
    repository = "index.docker.io/<ORG>/image",
    remote_tags = ":stamped",
)

format is unsupported. the default format is oci and legacy docker format isn't supported.

skip_unchanged_digest is unsupported. oci_push skips existing blobs by default.

extension is unsupported. the default extension is sh. no plans to support it.

tag_tpl and windows_paths attributes are unsupported. no plans to support it.

container_image

Update BUILD files with: buildozer 'set kind oci_image' //...:%container_image

oci_image does not support following attributes directly, but they can be passed to pkg_tar for layer creation.

files -> pkg_tar#srcs
compression -> pkg_tar#compressor
compression_options -> pkg_tar#compressor_args
data_path -> pkg_tar#strip_prefix
directory -> pkg_tar#package_dir
empty_dirs -> pkg_tar#empty_dirs
empty_files -> pkg_tar#empty_files
tars -> pkg_tar#deps
mode -> pkg_tar#mode
mtime -> pkg_tar#mtime
symlinks -> pkg_tar#symlinks

An example demonstrating migrating tars, directory, symlinks from container_image to oci_image

@@ -1,6 +1,13 @@
-container_image(
+oci_image(
     name = "image",
+    tars = [
+        ":new_layer"
+    ]
+)
+
+pkg_tar(
+    name = "new_layer",
     symlinks = { "/usr/bin/app": "/app"},
     tars = [":app"],
-    directory = "/org/image"
+    package_dir = "/org/image"
 )

debs is not supported directly as an attribute. However, since .deb files consist of two tar files data.tar.xz and control.tar.xz, these can be passed to oci_image#tars attribute after extracted from a deb file.

If the debian files are downloaded via http_archive, bazel recently landed support for extracting these files on the fly.

An example genrule extracting .deb files before passing down to oci_image

genrule(
    name = "new_layer",
    srcs = [":deb"],
    outs = ["data.tar.xz"],
    cmd = "tar -xvf -c $@/ $(location :deb)",
)

oci_image(
    tars = [":new_layer"]
)

enable_mtime_preservation is unsupported. pkg_tar doesn't support this feature either. Generally, this is a bad idea since it leads to non-reproducible builds.

layers is now tars. it accepts list of arbitrary .tar or .tar.gz files.

Note that oci_image#tars attribute behaves differently than container_image#tars. While container_image#tars squashes multiple tars into single tar, oci_image#tars preserves tars and creates a layer per tar respecting the order of the tars given at oci_image#tars attribute.

-container_image(
+oci_image(
     name = "image",
-    layers = [
+    tars = [
         ":layer"
     ]
 )

launcher and launcher_args are unsupported. As a replacement cmd and entrypoint can be used instead. However, since entrypoint attribute overrides the entrypoint, it wouldn't be possible to inherit the entrypoint from the base.

-container_image(
-container_image(
+pkg_tar(
+    name = "launcher",
+    srcs = [":launcher"]
+)
+oci_image(
     name = "image",
-    launcher = ":launcher",
-    launcher_args = ["--arg1", "--arg2"],
-    entrypoint = ["/app"],
-    cmd = ["--apparg1"]
+    entrypoint = ["/path/to/launcher", "--arg1", "--arg2"]
+    cmd = ["/app", "--apparg1"],
+    tars = [
+        ":launcher"
+    ]
 )

legacy_run_behavior, legacy_repository_naming, and docker_run_flags attributes are unsupported. Instead oci_tarball rule should be used for loading the oci_image into a docker daemon.

the oci_image target can be loaded into the daemon by running bazel build :tarball, docker load -i bazel-bin/tarball/tarball.tar respectively.

docker_run_flags can be passed to docker directly when running docker run gcr.io/test:latest

-container_image(
+oci_image(
     name = "image",
+)
+
+oci_tarball(
+    name = "tarball",
+    image = ":image",
+    repo_tags = ["gcr.io/test:latest"]
 )

repository and tag_name is unsupported. use oci_tarball#repo_tags as a replacement.

-container_image(
+oci_image(
     name = "image",
     tag_name = "latest",
-    repository = "gcr.io"
+    repository = "gcr.io/image"
+)
+
+oci_tarball(
+    name = "tarball",
+    repo_tags = ["gcr.io/image:latest"]
 )

compression and compression_options are unsupported. use pkg_tar#compressor and pkg_tar#compressor_args when creating layers instead.

experimental_tarball_format is unsupported. oci_image does not produce tarballs. oci_tarball, which produces tarballs out of oci_image, should be used instead.

stamp is not supported directly. oci_image allows attributes such as labels, annotations to be stamped. If the intent is to stamp layers, pkg_tar#stamp is the preferred way to stamp layers.

creation_time is unsupported at the moment. by default creation time for the image is static. follow rules_oci#49 for updates.

os_version is unsupported at the moment. follow rules_oci#48 for updates.

ports is unsupported. most runtimes support --port|-p flag which can be passed at container startup. follow rules_oci#220 for updates.

volumes is unsupported. most runtimes support --volume|-v|--mount flag which can be passed at container startup.

Migrating to rules_oci

Update WORKSPACE​

Replacements by rule​

container_pull​

Environment variables​

container_push​

container_image​