Notice: Our URL has changed! Please update your bookmarks. Dismiss

External Docker Registry Integration with OpenShift Enterprise v3

Overview

OpenShift is a platform for running containerized applications. Containers are instances of images and these images are stored in registries. OpenShift has the ability to leverage images stored in its own integrated registry, images stored on DockerHub or images stored in an enterprise or third party registry. This document describes the steps necessary to run containers from images stored in an external docker registry and to use an external registry to store images produced within the platform.

Docker Configuration

Regardless of the source, the underlying docker engine is used to retrieve images stored in remote repositories. There are certain considerations that must be made so that the docker engine can successfully communicate with the remote registry.

Certificates

Communication between the docker engine and the remote registry is facilitated through secure transport. A trust must be established between both parties prior to transmitting content securely. SSL certificates associated with the remote registry must be configured on on each OpenShift node that will be used to transport images. These certificates can be configured in the Red Hat Enterprise Linux (RHEL) platform truststore or for use by Docker exclusively.

To configure docker to trust the certificate of a remote repository that is not included in the platform trust store, create a folder within the /etc/docker/certs.d folder with the name and optional port value. If the remote registry had the address of external-registry.example.com:5000 the folder path would be /etc/docker/certs.d/external-registry.example.com.

Copy the certificate into this folder and rename the file ca.crt

Insecure Registry

Even though Docker and OpenShift emphasize secure communication, the Docker daemon can be configured to retrieve images from insecure registries. Insecure registries can refer to ones that are protected with self signed or untrusted SSL certificates or ones that do not communicate over secure channels (HTTP).

To configure the docker daemon to trust content from an insecure registry, add the following to the OPTIONS property in the /etc/sysconfig/docker file. An example is shown below:

OPTIONS=' --selinux-enabled --insecure-registry=external-registry.example.com:5000
Note
The hostname or IP address and optional port value must match the remote registry value configured in OpenShift

Restart the docker daemon for the new configurations to take effect

systemctl restart docker

Accessing Secure Registries

Secure external registries can be integrated into the OpenShift ecosystem. Through the use of secrets, users can provide the platform with the necessary credentials to access secured resources.

Storing Credentials in Secrets

Docker stores the details for authenticating to remote repositories in one of two formats: a config.json file or a legacy .dockercfg file. OpenShift makes use of the .dockercfg file format for authenticating against remote registries and stores them as secrets. This file can be generated by older versions of docker using the docker login command or the parameters supplied to OpenShift who in turn will produce a new .dockercfg file in the secret.

Existing .dockercfg files can be added as secrets by executing the following command:

oc secrets new external-registry ~/.dockercfg

To create a new .dockercfg file by supplying the necessary parameters, execute the following command:

oc secrets new-dockercfg external-registry \
    --docker-username=mastermind \
    --docker-password=12345 \
    --docker-email=mastermind@example.com \
    --docker-server=external-registry.example.com:5000
Note
When logging into a remote OpenShift cluster, You must use your oc login token instead of your user’s password. You can see more here. Accessing The Registry Directly

Alternatively, if the docker client was used to login to a remote registry and generate a configuration file containing the authentication details, this file can be stored as a secret within OpenShift. Two docker authentication formats are available: config.json and the legacy .dockercfg. Both are compatible for storage within OpenShift.

To store a file in the config.json format as a secret within OpenShift, execute the following command

oc secrets new external-registry .dockerconfigjson=config.json

If the file is in the .dockercfg format, execute the following command to add the secret to OpenShift

oc secrets new external-registry .dockercfg=.dockercfg

External registry integration in Builds

An external registry can be integrated into the OpenShift build process as both a location to store images built in the platform and as an image source for builds.

Storing images as a result of a build

Note
It is recommended that images produced as a result of a build within OpenShift be stored within the docker registry integrated into the platform.

Images that are produced by an image build within OpenShift can be stored in a remote registry instead of the integrated docker registry provided by the platform. As with any OpenShift build, the details are contained in the BuildConfig API object.

The output section of the Build specification defines the location of where the resulting docker image will be stored. Many of the examples found in OpenShift templates and quickstarts use the ImageStreamTag as the output kind. This will publish the image to the integrated docker registry. To publish the image to an external location, where it be to DockerHub or an enterprise registry, the DockerImage type must be used.

The following is a portion of a BuildConfig that pushes an image called sample-app to a remote registry located at external-registry.example.com:5000 in a repository called publicrepo with tag latest

"output": {
    "to": {
        "kind": "DockerImage",
        "name": "external-registry.example.com:5000/publicrepo/sample-app:latest"
    }
},

If the registry is protected by authentication, a secret must be added to the builder service account which is included in each OpenShift project and is used to execute builds on the platform. Once the secret containing the credentials to the secure registry has been created as described in an earlier section, add the newly created secret to the builder service account as shown below:

oc secrets add serviceaccount/builder secrets/external-registry

Confirm the secret for the external registry has been added successfully to the builder service account:

oc describe serviceaccount builder

Name:		builder
Labels:		<none>
Annotations:	<none>

Image pull secrets:	builder-dockercfg-sk6bc

Mountable secrets: 	builder-token-3ppg8
                   	builder-dockercfg-sk6bc
                   	external-registry (1)

Tokens:            	builder-token-3ppg8
                   	builder-token-4pxkl
  1. Secret containing credentials for storing images in the protected external registry

To push resulting images produced by the build process to an authenticated registry, a new section called pushSecret must be added to the BuildConfig containing the name of the secret:

"output": {
    "to": {
        "kind": "DockerImage",
        "name": "external-registry.example.com:5000/publicrepo/sample-app:latest"
    },
    "pushSecret": {
        "name": "external-registry" (1)
    },
  1. The name of the secret configured within the project containing the credentials to access the secure registry

Using images from a secured registry as part of a build

Builder images stored in an external registry can also be used as input for builds. In the strategy section of the build type, declare the location of the image as shown below:

 "type": "Source",
 "sourceStrategy": {
     "from": {
         "kind": "DockerImage",
         "name": "external-registry.example.com:5000/publicrepo/sample-base:1.0"
     },

If the registry is protected by authentication, add a secret containing the credentials to the secure registry as described in an earlier section and add the secret to the builder service account as shown below:

oc secrets add serviceaccount/builder secrets/external-registry

Finally, add the name of the pull secret to the BuildConfig as shown below:

"type": "Source",
"sourceStrategy": {
    "from": {
        "kind": "DockerImage",
        "name": "external-registry.example.com:5000/privaterepo/sample-base:1.0"
    },
    "pullSecret": {
        "name": "external-registry" (1)
    },
  1. The name of the secret configured within the project containing the credentials to access the secure registry

Running images stored in an external registry

Images residing in external registries can be run within OpenShift. This section describes the steps necessary to configure an OpenShift project to utilize images stored in external registries as well as the various methods for managing external images.

Deploying an image using the oc new-app command

The simplest method for deploying an image from an external registry is to use the oc new-app CLI command. This command will resolve the image and set up the necessary API objects based on metadata contained in the image.

Execute the following command to sample-app built in an earlier section

oc new-app --docker-image=external-registry.example.com:5000/publicrepo/sample-app:latest

The CLI will configure services based on exposed ports on the image and set up a DeploymentConfig to deploy the image to the project.

Deploying an image using an ImageStream

An ImageStream represents a virtual repository of related docker images and can be used to abstract the location of image from applications. Images stored in an external repository can be referenced using an ImageStream to relieve the burden from the underlying application.

To create a new ImageStream for the existing image, execute the following command:

oc import-image sample-app --from=external-registry.example.com:5000/publicrepo/sample-app --confirm

A new ImageStream called sample-app will be created and can be used to deploy a new application. Execute the following command to deploy create the application from the imported ImageStream:

oc new-app --image-stream=sample-app

Deploy an Image using the location in the DeploymentConfig

Images from external registries can be deployed in an application by directly specifying its' location in the DeploymentConfig API object in the containers' image property as shown below.

"containers": [
                    {
                        "name": "sample-app",
                        "image": "external-registry.example.com:5000/publicrepo/sample-app:latest",  (1)
                        "ports": [
                            {
                                "containerPort": 8080,
                                "protocol": "TCP"
                            }
                        ],
                        "resources": {},
                        "terminationMessagePath": "/dev/termination-log",
                        "imagePullPolicy": "Always"
                    }
                ],
  1. The image to deploy

Note
It is recommended that an ImageStream be used instead of a direct reference

Deploying Images from an Insecure Registry

Images that are located in external registries that either may not have their SSL certificates trusted on the OpenShift node or communicate using the HTTP protocol can still be used by applications. Both the oc new-app command and ImageStream import processes support the management of images from these locations.

The --insecure-registry option of the oc new-app command can be used to allow new applications to be referenced from insecure registries.

The oc import-image command provides a --insecure option when importing images from insecure registries. Specific tags originating from insecure registries can be defined within an ImageStream by configuring the importPolicy section of the tag as shown below:

kind: ImageStream
apiVersion: v1
metadata:
  name: sample-app
  tags:
  - from:
      kind: DockerImage
      name: external-registry.example.com:5000/publicrepo/sample-app
    name: latest
    importPolicy:
      insecure: "true" (1)
  1. Set insecure to true to signify the image originates in an insecure registry

Deploying Images from an Authenticated Registry

Images that are stored in a registry protected by authentication can by deployed to OpenShift. As described earlier, a Secret must be configured with the details necessary to access the registry.

Once the details of the .dockercfg have been added to a secret as described in the section above, the secret must be added to the service account that will be used to run the pod. In most cases, this will be the default service account, but be sure to confirm these details. Execute the following command to add a secret called external-registry to the default service account:

oc secrets add serviceaccount/default secrets/external-registry --for=pull

The --for=pull option signifies that the secret will be added as a pull secret within the service account

Confirm the secret for the external registry has been added successfully to the default service account:

oc describe serviceaccount default

Name:		default
Labels:		<none>
Annotations:	<none>

Tokens:            	default-token-kssb4
                   	default-token-xqxcf

Image pull secrets:	default-dockercfg-4rq5x
                   	external-registry (1)

Mountable secrets: 	default-token-kssb4
                   	default-dockercfg-4rq5x
  1. Secret containing credentials for pulling images from the protected external registry

Importing ImageStreams from an Authenticated Registry

To import images found within an ImageStream from an authenticated registry, add the necessary credentials to a secret. OpenShift will iterate through all available authentication options in an attempt to access the registry and import the ImageStream

Troubleshooting

This section provides resolution to common issues when working with images located in an external registry

Unable to deploy images from an external registry

When attempting to deploy an application from an external registry, the following error may be observed from a pod:

image pull failed for external-registry.example.com:5000/privaterepo/sample-app@sha256:45fbf8004abd1a4b9a983db7bf91b4934b3d0329399e275840bfe09e246643c4, this may be because there are no credentials on this request.

The pod will typically be observed with a ImagePullBackOff status indicating the underlying image cannot be retrieved.

While this error can be used to diagnose incorrectly configured authentication, it is used by OpenShift internally as a last effort to denote that it is unable to retrieve the configured the image and may not be caused by an authentication issue at all. Be sure to confirm the location of the image before continuing with further diagnosis.