Troubleshoot a wrapped container

Note: Guarded Entrypoint is in beta. To use it, contact Chainguard customer support to enable it for your organization.

This page covers two kinds of problems. A container that is built with Guarded Entrypoint can fail to start. A build can also fail because Chainguard refuses to wrap an image.

First move: set GUARDED_DISABLE

When a wrapped container fails to start, set the GUARDED_DISABLE environment variable on the container and redeploy. The wrapper then starts the image’s original entrypoint and arguments without doing anything else. You don’t need to rebuild the image.

With Kubernetes, set the variable on the deployment:

kubectl set env deployment/$DEPLOYMENT GUARDED_DISABLE=1

With Docker, pass it to docker run:

docker run -e GUARDED_DISABLE=1 cgr.dev/$ORGANIZATION/$REPO:latest

GUARDED_DISABLE counts as set unless its value, lowercased and trimmed, is empty, 0, false, no, or off. A value of 1 or true turns the wrapper off. A value of 0 or false leaves it on. The wrapper checks GUARDED_DISABLE before it reads any other setting. This means that a malformed setting in the image doesn’t stop it from working.

When GUARDED_DISABLE is set, the wrapper does the following:

• It doesn’t resolve secret references. Your application sees the literal cg+... values.
• It doesn’t run preflight checks.
• It doesn’t apply command_override. The image’s original ENTRYPOINT and CMD run.
• It makes no network connections.

If the container then starts, the wrapper or its settings might have caused the failure. Remove the variable after you fix the configuration and Chainguard rebuilds the image.

If the container still fails, the cause isn’t necessarily the application or the deployment. With GUARDED_DISABLE set, your application receives the literal cg+... values and skips its preflight checks. An application that needs its secrets can fail for that reason.

With GUARDED_DISABLE set, an image whose only command comes from command_override exits with code 124 instead of starting.

Anyone who can set environment variables on a container can set GUARDED_DISABLE. This is true for a repo with fail_mode: closed too. See Guarded Entrypoint trust boundary.

Find out why the container stopped

Start with the container’s exit code. For a Kubernetes pod, kubectl describe pod shows it under Last State. For Docker, run the following command:

docker inspect --format '{{.State.ExitCode}}' $CONTAINER

Then read the container’s logs. The wrapper writes JSON messages to standard error, one object per line, and each message names the setting or variable that failed. It never logs a secret value:

kubectl logs $POD

For a pod in a restart loop, add --previous to read the logs of the container that stopped.

To see more detail, set GUARDED_ENTRYPOINT_LOG on the container to debug. The levels are debug, info, warn, and quiet.

Exit codes before the application starts

The wrapper uses the following exit codes when it stops the container before your application runs. Once the application starts, the container exits with the application’s exit code.

Your own application can also exit with any code from 120 to 127. To tell the two apart, look in the logs for a wrapper error line just before the container exited.

With fail_mode: open, only the configuration errors described in Fail mode stop the container with exit code 121.

A container whose entrypoint is an init system

An image whose entrypoint is an init system that must run as PID 1, such as systemd or s6-overlay, doesn’t work under the wrapper. The container behaves in one of two ways:

• It exits with code 1 at start, and systemd prints Explicit --user argument required to run as user manager.
• It exits with code 129 when you stop it.

To fix it, set GUARDED_DISABLE on the deployment, or turn off Guarded Entrypoint for the repo. See Init systems aren’t supported.

Entrypoints the wrapper refuses

When Chainguard can’t wrap an image, the rebuild of that image fails. Chainguard doesn’t ship a broken image. Instead, the build records a failure and its reason.

To see the failure, list the repo’s builds:

chainctl images repos build list --repo $REPO --parent $ORGANIZATION

The Result column shows the failure, and the Reason column shows why. The Reason column is filled only for failures that Chainguard records outside a build, such as these refusals and binding conflicts. It is empty for an ordinary build failure. The failed build has no tags in the Tags column. For the full text, run chainctl images repos build logs --repo $REPO --parent $ORGANIZATION and select the failed build. Without a terminal, for example in a pipeline, pass --build-id with the build’s ID. The output has this form:

guarded entrypoint refused for tags [latest] (digest sha256:...): the environment sets GUARDED_DISABLE; the tags are not rebuilt and are dropped from the repo's active tag list until the refusal is resolved
detail: set in the base image environment

The optional second line, detail: ..., gives more information. If the repo sets command_override, the detail also says that command_override is not applied.

The text after the digest is the reason. It is one of the following:

The tags named in the message aren’t rebuilt until you resolve the refusal. A tag that isn’t rebuilt doesn’t receive package updates, including CVE fixes, until then. To resolve it, do one of the following:

• For the wrapper version reason, remove the guarded-entrypoint or guarded-entrypoint-fips pin from the repo’s contents.packages list.
• Turn off Guarded Entrypoint for the repo. See Turn off Guarded Entrypoint.
• With tag-based Custom Assembly, bind the overlay that sets guarded_entrypoint only to the tags that Chainguard doesn’t refuse.

To check ahead of time whether an image is supported, see the lists of supported and refused entrypoints.

Learn more

• How Guarded Entrypoint works
• Guarded Entrypoint examples
• Guarded Entrypoint trust boundary