Skip to content

Docker images not published for versions 0.62.0 and 0.63.0 - Helm chart deployment fails with ImagePullBackOff #6409

@furkanbegen

Description

@furkanbegen

Docker images not published for versions 0.62.0 and 0.63.0 - Helm chart deployment fails with ImagePullBackOff

Labels

  • bug
  • helm
  • docker
  • ci/cd

Issue Body


Description

The Feast Helm charts for versions 0.62.0 and 0.63.0 reference Docker images that do not exist in the container registries, causing deployment failures with ImagePullBackOff errors.

Affected Versions

  • feast chart: 0.62.0
  • feast chart: 0.63.0

Missing Images

  1. Feature Server (Java)

    • Referenced by chart: quay.io/feastdev/feature-server-java:0.62.0
    • Referenced by chart: quay.io/feastdev/feature-server-java:0.63.0
    • Status: ❌ Does not exist in quay.io
    • Alternative that works: quay.io/feastdev/feature-server:0.62.0 (without -java suffix)
  2. Redis

    • Referenced by chart: docker.io/bitnami/redis:5.0.7-debian-10-r32
    • Status: ❌ Old image removed from Bitnami registry

Root Cause

GitHub Actions workflow for publishing Docker images failed:
Failed Build: https://github.com/feast-dev/feast/actions/runs/25299957140/job/74168167270

The build appears to have failed during the Docker image publishing step, resulting in the feature-server-java images never being pushed to the container registry.

Steps to Reproduce

  1. Add Feast Helm repository:

    helm repo add feast-charts https://feast-helm-charts.storage.googleapis.com
    helm repo update
  2. Install Feast chart:

    helm install feast feast-charts/feast --version 0.63.0
  3. Check pod status:

    kubectl get pods -n feast

Expected Behavior

Pods should start successfully with all images pulled from the registries.

Actual Behavior

Pods fail with ImagePullBackOff:

NAME                                            READY   STATUS             RESTARTS   AGE
feast-feature-server-6f6cd9cbb6-r92lb           0/1     ErrImagePull       0          1m
feast-redis-master-0                            0/1     ImagePullBackOff   0          1m
feast-redis-slave-0                             0/1     ImagePullBackOff   0          1m
feast-transformation-service-86849757d5-vsnln   1/1     Running            0          1m

Error messages:

Failed to pull image "quay.io/feastdev/feature-server-java:0.63.0": 
rpc error: code = NotFound desc = failed to pull and unpack image 
"quay.io/feastdev/feature-server-java:0.63.0": failed to resolve reference 
"quay.io/feastdev/feature-server-java:0.63.0": 
quay.io/feastdev/feature-server-java:0.63.0: not found
Failed to pull image "docker.io/bitnami/redis:5.0.7-debian-10-r32": 
rpc error: code = NotFound desc = failed to pull and unpack image 
"docker.io/bitnami/redis:5.0.7-debian-10-r32": failed to resolve reference 
"docker.io/bitnami/redis:5.0.7-debian-10-r32": 
docker.io/bitnami/redis:5.0.7-debian-10-r32: not found

Verification

Testing image availability:

# feature-server-java does NOT exist:
docker pull quay.io/feastdev/feature-server-java:0.63.0
# Error: manifest unknown: manifest unknown

# feature-server (without -java) DOES exist:
docker pull quay.io/feastdev/feature-server:0.63.0
# Success: image pulled

# Transformation server DOES exist:
docker pull quay.io/feastdev/feature-transformation-server:0.63.0
# Success: image pulled

Environment

  • Kubernetes Version: 1.28+
  • Helm Version: 3.x
  • Feast Chart Version: 0.62.0 / 0.63.0
  • Container Runtime: containerd

Impact

  • Severity: High - Helm chart deployments are completely broken
  • Users Affected: Anyone attempting to deploy Feast 0.62.0 or 0.63.0 via Helm
  • Workaround Exists: Yes, but requires manual image overrides in values.yaml

Suggested Fix

  1. Immediate: Re-run the failed GitHub Actions workflow to publish missing images
  2. Short-term: Update Helm charts to reference correct image names (without -java suffix)
  3. Long-term:
    • Add automated tests to verify published images exist before releasing charts
    • Update Redis chart dependency to use current Bitnami Redis images
    • Consider adding image verification to CI/CD pipeline

Workaround

Users can override images in values.yaml:

feature-server:
  image:
    repository: quay.io/feastdev/feature-server  # without -java suffix
    tag: "0.62.0"

redis:
  image:
    registry: docker.io
    repository: bitnami/redis
    tag: "7.2.4-debian-12-r9"  # current stable version

Related Issues

Additional Context

This issue blocks production deployments using the official Helm charts. The transformation service works because its image was successfully published, but the feature server (the core component) cannot start due to the missing image.


Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions