Skip to content

lakeFS Mount CSI Driver v1.4.0

Changelog

  • Bumped the bundled everest mount binary from 0.12.0 to 0.14.0. For the CSI driver this means:
  • Mount Pods exit after their volume is unmounted and stop promptly on SIGTERM. Before, every unmounted volume left a Running Mount Pod behind, which accumulated in the everest-lakefs-csi namespace and blocked node scale-down. Deleting one by hand waited out the full 600s grace period.
  • Directories are group-writable when the mount has a --mount-gid, which completes the fsGroup support below.
  • See the everest changelog for the full list, including 0.13.0.

Warning: the ServiceAccount names change in this release. If the driver uses driver-level IRSA or EKS Pod Identity, it loses its AWS credentials on upgrade unless you migrate the IAM binding first. See the upgrade notes.

Features

  • Added the node.updateStrategy chart value, rendered as the node DaemonSet's spec.updateStrategy. Kubernetes' default rolls one node at a time, which makes upgrades on clusters with thousands of nodes impractically slow; set, for example, rollingUpdate.maxUnavailable: 10%, or use type: OnDelete. The default {} omits the field, so existing installs are unaffected.
  • A workload Pod's fsGroup is passed to everest as --mount-gid, so the mount is owned by root:<fsGroup> and directories are 0775. A non-root workload with an fsGroup can now create files in a write-mode mount. A --mount-gid set in mountOptions still takes precedence. Mount Pods no longer default to --mount-gid=0.

Fixes

  • Renamed the chart's ServiceAccounts, ClusterRoles, ClusterRoleBindings, Roles and RoleBindings from s3-csi-driver-* to everest-lakefs-csi-*, so the chart installs alongside aws-mountpoint-s3-csi-driver, including in the same namespace. The ServiceAccounts default to everest-lakefs-csi-sa (node) and everest-lakefs-csi-controller-sa (controller).
  • Helm chart bumped to 1.4.0 with matching appVersion and image.tag.

Upgrade notes

  • Driver-level IRSA or EKS Pod Identity: move the IAM binding to the new node ServiceAccount before upgrading, or the driver loses its AWS credentials. Pod-level (authenticationSource: pod) and secret credentials are unaffected.
  • IRSA: add system:serviceaccount:<namespace>:everest-lakefs-csi-sa to the IAM role trust policy's sub condition. Remove the s3-csi-driver-sa entry after the upgrade.
  • IRSA annotation: the chart creates the new ServiceAccount with only the annotations in node.serviceAccount.annotations. If you set eks.amazonaws.com/role-arn there, it carries over. If you added it to s3-csi-driver-sa directly (for example with kubectl annotate or eksctl create iamserviceaccount without --role-only), it does not: set node.serviceAccount.annotations."eks\.amazonaws\.com/role-arn" in your values before upgrading.
  • EKS Pod Identity: create an association for everest-lakefs-csi-sa in the release namespace with the same IAM role. Delete the s3-csi-driver-sa association after the upgrade.
  • To skip the migration, keep the previous names with --set node.serviceAccount.name=s3-csi-driver-sa --set controller.serviceAccount.name=s3-csi-driver-controller-sa. The chart then cannot share a namespace with aws-mountpoint-s3-csi-driver.
  • Rollout window: helm upgrade deletes the previous ServiceAccounts and RBAC objects before it waits for the rollout, even with --wait. Node and controller Pods still running as the previous ServiceAccount lose Kubernetes API access until they are replaced, so volume mounts and unmounts on those nodes can fail during the rollout; kubelet retries them.
  • Running alongside aws-mountpoint-s3-csi-driver: Helm deletes the previous objects even when another release owns them. If the AWS release owns s3-csi-driver-cluster-role, s3-csi-driver-controller-cluster-role, s3-csi-driver-controller-cluster-role-binding or a ServiceAccount of the same name, run helm upgrade on that release afterwards to recreate them.
  • Kustomize: kubectl apply -k deploy/kubernetes/... does not prune. Delete the s3-csi-driver-* ServiceAccounts, ClusterRoles, ClusterRoleBindings, Roles and RoleBindings by hand, unless aws-mountpoint-s3-csi-driver uses them.

Docker

docker pull treeverse/everest-lakefs-csi-driver:1.4.0

Verify the signature (optional)

Requires Cosign:

cosign verify treeverse/everest-lakefs-csi-driver:1.4.0 \
    --certificate-identity-regexp='^https://github\.com/treeverse/everest\-lakefs\-csi\-driver/\.github/workflows/' \
    --certificate-oidc-issuer='https://token.actions.githubusercontent.com'
Expected output 
Verification for index.docker.io/treeverse/everest-lakefs-csi-driver:1.4.0 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The code-signing certificate was verified using trusted certificate authority certificates