docker-buildx/docs.md
David Kovari ef56bac838 Improve support for cache_to and cache_from & add cache_images (#129)
### 📖 Summary

Originally, the cache_to and cache_from were being converted into string arrays. Then their contents were looped over to add to the build command. This has the side affect of stopping the user from setting additional options for the cache-to and cache-from args since their command input format uses commas. i.e. `type=registry,ref=imagepath,mode=max` would result in `--cache-to type=registry --cache-to ref=imagepath --cache-to mode=max`. The command was not designed to be used that way.

The one reason I can think of for cache_to and cache_from to be arrays is so you could have multiple cache registries. But I can't confirm that the build command even works like this.

### 📑 Build PR Images?

PR images are not needed

### 💬 Details

_No response_

Reviewed-on: https://codeberg.org/woodpecker-plugins/docker-buildx/pulls/129
Reviewed-by: Patrick Schratz <pat-s@noreply.codeberg.org>
Co-authored-by: David Kovari <dakovari@gmail.com>
Co-committed-by: David Kovari <dakovari@gmail.com>
2024-02-02 20:25:30 +00:00

17 KiB

name icon description author tags containerImage containerImageUrl url
Docker Buildx https://codeberg.org/woodpecker-plugins/docker-buildx/raw/branch/main/docker.svg plugin to build multiarch Docker images with buildx Woodpecker Authors
docker
image
container
build
woodpeckerci/plugin-docker-buildx https://hub.docker.com/r/woodpeckerci/plugin-docker-buildx https://codeberg.org/woodpecker-plugins/docker-buildx

Woodpecker CI plugin to build multiarch Docker images with buildx. This plugin is a fork of thegeeklab/drone-docker-buildx which itself is a fork of drone-plugins/drone-docker.

Features

  • Build without push
  • Use custom registries
  • Build based on existing tags when needed
  • Push to multiple registries/repos

It will automatically generate buildkit configuration to use custom CA certificate if following conditions are met:

  • Setting buildkit_config is not set
  • Custom registry/logins value is provided
  • File exists /etc/docker/certs.d/<registry-value>/ca.crt

NB! To mount custom CA you can use Woodpecker CI runner configuration environment WOODPECKER_BACKEND_DOCKER_VOLUMES with value /etc/ssl/certs:/etc/ssl/certs:ro,/etc/docker/certs.d:/etc/docker/certs.d:ro. And have created file /etc/docker/certs.d/<registry-value>/ca.crt with CA certificate on runner server host.

Settings

Settings Name Default Description
dry-run false disables docker push
repo none sets repository name for the image (can be a list)
username none sets username to authenticates with
password none sets password / token to authenticates with
aws_access_key_id none sets AWS_ACCESS_KEY_ID for AWS ECR auth
aws_secret_access_key none sets AWS_SECRET_ACCESS_KEY for AWS ECR auth
aws_region us-east-1 sets AWS_DEFAULT_REGION for AWS ECR auth
password none sets password / token to authenticates with
email none sets email address to authenticates with
registry https://index.docker.io/v1/ sets docker registry to authenticate with
dockerfile Dockerfile sets dockerfile to use for the image build
tag/tags none sets repository tags to use for the image
platforms none sets target platform for build

auto_tag

If set to true, it will use the default_tag ("latest") on tag event or default branch. If it's a tag event it will also assume sem versioning and add tags accordingly (x, x.x and x.x.x). If it's not a tag event, and no default branch, automated tags are skipped.

Examples

publish-next-agent:
  image: woodpeckerci/plugin-docker-buildx
  secrets: [docker_username, docker_password]
  settings:
    repo: woodpeckerci/woodpecker-agent
    dockerfile: docker/Dockerfile.agent.multiarch
    platforms: windows/amd64,darwin/amd64,darwin/arm64,freebsd/amd64,linux/amd64,linux/arm64/v8
    tag: next
  when:
    branch: ${CI_REPO_DEFAULT_BRANCH}
    event: push
publish:
  image: woodpeckerci/plugin-docker-buildx
  settings:
    platforms: linux/386,linux/amd64,linux/arm/v6,linux/arm64/v8,linux/ppc64le,linux/riscv64,linux/s390x
    repo: codeberg.org/${CI_REPO_OWNER}/hello
    registry: codeberg.org
    tags: latest
    username: ${CI_REPO_OWNER}
    password:
      from_secret: cb_token
docker-build:
  image: woodpeckerci/plugin-docker-buildx
  settings:
    repo: codeberg.org/${CI_REPO_OWNER}/hello
    registry: codeberg.org
    dry-run: true
    output: type=oci,dest=${CI_REPO_OWNER}-hello.tar

Advanced Settings

Settings Name Default Description
mirror none sets a registry mirror to pull images
storage_driver none sets the docker daemon storage driver
storage_path /var/lib/docker sets the docker daemon storage path
bip none allows the docker daemon to bride ip address
mtu none sets docker daemon custom mtu setting
custom_dns none sets custom docker daemon dns server
custom_dns_search none sets custom docker daemon dns search domain
insecure false allows the docker daemon to use insecure registries
ipv6 false enables docker daemon IPv6 support
experimental false enables docker daemon experimental mode
debug false enables verbose debug mode for the docker daemon
daemon_off false disables the startup of the docker daemon
buildkit_debug false enables debug output of buildkit
buildkit_config none sets content of the dockerbuildkit TOML config
buildkit_driveropt none adds one or multiple--driver-opt buildx arguments for the default buildkit builder instance
tags_file none overrides thetags option with values in a file named .tags; multiple tags can be specified separated by a newline
context . sets the path of the build context to use
auto_tag false generates tag names automatically based on git branch and git tag, tags supplied viatags are additionally added to the auto_tags without suffix
default_suffix"/auto_tag_suffix none generates tag names with the given suffix
default_tag latest overrides the default tag name used when generating withauto_tag enabled
label/labels none sets labels to use for the image in format<name>=<value>
default_labels/auto_labels true sets docker image labels based on git information
build_args none sets custom build arguments for the build
build_args_from_env none forwards environment variables as custom arguments to the build
quiet false enables suppression of the build output
target none sets the build target to use
cache_from none sets configuration for cache source
cache_to none sets configuration for cache export
cache_images none a list of images to use as cache.
pull_image true enforces to pull base image at build time
compress false enables compression of the build context using gzip
config none sets content of the docker daemon json config
purge true enables cleanup of the docker environment at the end of a build
no_cache false disables the usage of cached intermediate containers
add_host none sets additional host:ip mapping
output none sets build output in formattype=<type>[,<key>=<value>]
logins none option to log into multiple registries
env_file none load env vars from specified file
ecr_create_repository false creates the ECR repository if it does not exist
ecr_lifecycle_policy none AWS ECR lifecycle policy
ecr_repository_policy none AWS ECR repository policy
ecr_scan_on_push none AWS: whether to enable image scanning on push

Multi registry push example

Only supported with woodpecker >= 1.0.0 (next-da997fa3).

settings:
  repo: a6543/tmp,codeberg.org/6543/tmp
  tag: demo
  logins:
    - registry: https://index.docker.io/v1/
      username: a6543
      password:
        from_secret: docker_token
      mirrors:
        - "my-docker-mirror-host.local"
    - registry: https://codeberg.org
      username: "6543"
      password:
        from_secret: cb_token
    - registry: https://<account-id>.dkr.ecr.<region>.amazonaws.com
      aws_region: <region>
      aws_access_key_id:
        from_secret: aws_access_key_id
      aws_secret_access_key:
        from_secret: aws_secret_access_key

Using plugin-docker-buildx behind a proxy

When performing a docker build behind a corporate proxy one needs to pass through the proxy settings to the plugin.

variables:
  # proxy config
  - proxy_conf: &proxy_conf
      - http_proxy: "http://X.Y.Z.Z:3128"
      - https_proxy: "http://X.Y.Z.Z:3128"
      - no_proxy: ".my-subdomain.com"
  # deployment targets
  - &publish_repos "codeberg.org/test"
  # logins for deployment targets
  - publish_logins: &publish_logins
      - registry: https://codeberg.org
        username:
          from_secret: CODEBERG_USER
        password:
          from_secret: CODEBERG_TOKEN

steps:
  test:
    image: woodpeckerci/plugin-docker-buildx:2
    environment:
      # adding proxy in env for the plugin runtime itself.
      - <<: *proxy_conf
    privileged: true
    settings:
      dry-run: true
      repo: *publish_repos
      dockerfile: Dockerfile.multi
      platforms: linux/amd64
      auto_tag: true
      logins: *publish_logins
      # Adding custom dns server to lookup internal Docker Hub mirror.
      # custom_dns:
      #   - 192.168.55.31
      #   - 192.168.55.32
      # Adding an optional Docker Hub mirror for the nested dockerd.
      # mirror: https://my-mirror.example.com
      build_args:
        # passthrough proxy config to the build process and Dockerfile CMDs itself.
        - <<: *proxy_conf
      # add driver-opt http config to tell buildkit + buildx to resolve external checksums through a proxy.
      buildkit_driveropt:
        - "env.http_proxy=http://X.Y.Z.Z:3128"
        - "env.https_proxy=http://X.Y.Z.Z:3128"
        - "env.no_proxy=.my-subdomain.com"

Using cache images

You can provide a list of images to use for cache. These cache images are built with mode=max, image-manifest=true, and oci-mediatypes=true. This is to provide better usage of cache and better compatibility with image stores like Harbor.

steps:
  build:
    image: woodpeckerci/plugin-docker-buildx
    settings:
      repo: hari/radiant
      cache_images:
        - hari/radiant:cache
        - harbor.example.com/hari/radiant:cache
      logins:
        - registry: https://index.docker.io/v1/
          username: hari
          password:
            from_secret: docker_password
        - registry: https://harbor.example.com
          username: hari
          password:
            from_secret: harbor_password

Using other cache types

You can specify cache_to and cache_from to use specific settings. For example you can configure an s3 object as cache.

More details can be found in the docker docs.

steps:
  build:
    image: woodpeckerci/plugin-docker-buildx
    settings:
      repo: hari/radiant
      cache_to: type=s3,region=east,bucket=mystuff,name=radiant-cache
      cache_from: type=s3,region=east,bucket=mystuff,name=radiant-cache