lakectl (lakeFS command-line tool)¶
Note
This file (cli.md) is automatically generated from the Go code files under cmd/lakectl.
Any changes made directly to the Markdown file will be overwritten, and should instead be made to the
relevant Go files.
Installing lakectl locally¶
lakectl is available for Linux, macOS, and Windows. You can also run it using Docker.
Or using Homebrew for Linux/macOS:
Configuring credentials and API endpoint¶
Once you've installed the lakectl command, run:
lakectl config
# output:
# Config file /home/janedoe/.lakectl.yaml will be used
# Access key ID: AKIAIOSFODNN7EXAMPLE
# Secret access key: ****************************************
# Server endpoint URL: http://localhost:8000
This will setup a $HOME/.lakectl.yaml file with the credentials and API endpoint you've supplied.
When setting up a new installation and creating initial credentials (see Quickstart), the UI
will provide a link to download a preconfigured configuration file for you.
lakectl configuration items can each be controlled by an environment variable. The variable name will have a prefix of
LAKECTL_, followed by the name of the configuration, replacing every '.' with a '_'. Example: LAKECTL_SERVER_ENDPOINT_URL
controls server.endpoint_url.
Running lakectl from Docker¶
If you'd rather run lakectl from a Docker container you can do so by passing configuration elements as environment variables.
Here is an example:
docker run --rm --pull always \
-e LAKECTL_CREDENTIALS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE \
-e LAKECTL_CREDENTIALS_SECRET_ACCESS_KEY=xxxxx
-e LAKECTL_SERVER_ENDPOINT_URL=https://host.us-east-2.lakefscloud.io/ \
--entrypoint lakectl treeverse/lakefs \
repo list
Bear in mind that if you are running lakeFS itself locally you will need to account for this in your networking configuration of
the Docker container. That is to say, localhost to a Docker container is itself, not the host machine on which it is running.
Command Reference¶
lakectl¶
A cli tool to explore manage and work with lakeFS
Synopsis
lakectl is a CLI tool allowing exploration and manipulation of a lakeFS environment
Options
--base-uri string base URI used for lakeFS address parse
-c, --config string config file (default is $HOME/.lakectl.yaml)
-h, --help help for lakectl
--log-format string set logging output format
--log-level string set logging level (default "none")
--log-output strings set logging output(s)
--no-color don't use fancy output colors (default value can be set by NO_COLOR environment variable)
--verbose run in verbose mode
-v, --version version for lakectl
Note
The base-uri option can be controlled with the LAKECTL_BASE_URI environment variable.
Example usage
$ export LAKECTL_BASE_URI="lakefs://my-repo/my-branch"
# Once set, use relative lakefs uri's:
$ lakectl fs ls /path
lakectl actions¶
Manage Actions commands
Options
lakectl actions help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type actions help [path to command] for full details.
Options
lakectl actions runs¶
Explore runs information
Options
lakectl actions runs describe¶
Describe run results
Synopsis
Show information about the run and all the hooks that were executed as part of the run
Examples
Options
--after string show results after this value (used for pagination)
--amount int number of results to return. By default, all results are returned.
-h, --help help for describe
lakectl actions runs help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type runs help [path to command] for full details.
Options
lakectl actions runs list¶
List runs
Synopsis
List all runs on a repository optional filter by branch or commit
Examples
Options
--branch string show results for specific branch
--commit string show results for specific commit ID
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
-h, --help help for list
lakectl actions validate¶
Validate action file
Synopsis
Tries to parse the input action file as lakeFS action file
Examples
Options
lakectl annotate¶
List entries under a given path, annotating each with the latest modifying commit
Options
--first-parent follow only the first parent commit upon seeing a merge commit
-h, --help help for annotate
-r, --recursive recursively annotate all entries under a given path or prefix
lakectl auth¶
Manage authentication and authorization
Synopsis
Manage authentication and authorization including users, groups and ACLs This functionality is supported with an external auth service only.
Options
lakectl auth groups¶
Manage groups
Options
lakectl auth groups acl¶
Manage ACLs
Synopsis
manage ACLs of groups
Options
lakectl auth groups acl get¶
Get ACL of group
Options
lakectl auth groups acl help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type acl help [path to command] for full details.
Options
lakectl auth groups acl set¶
Set ACL of group
Synopsis
Set ACL of group. permission will be attached to all repositories.
Options
-h, --help help for set
--id string Group identifier
--permission string Permission, typically one of "Read", "Write", "Super" or "Admin"
lakectl auth groups create¶
Create a group
Options
lakectl auth groups delete¶
Delete a group
Options
lakectl auth groups help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type groups help [path to command] for full details.
Options
lakectl auth groups list¶
List groups
Options
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl auth groups members¶
Manage group user memberships
Options
lakectl auth groups members add¶
Add a user to a group
Options
-h, --help help for add
--id string Group identifier
--user string Username (email for password-based users, default: current user)
lakectl auth groups members help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type members help [path to command] for full details.
Options
lakectl auth groups members list¶
List users in a group
Options
--id string Group identifier
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl auth groups members remove¶
Remove a user from a group
Options
-h, --help help for remove
--id string Group identifier
--user string Username (email for password-based users, default: current user)
lakectl auth groups policies¶
Manage group policies
Synopsis
Manage group policies. Requires an external authorization server with matching support.
Options
lakectl auth groups policies attach¶
Attach a policy to a group
Options
lakectl auth groups policies detach¶
Detach a policy from a group
Options
lakectl auth groups policies help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type policies help [path to command] for full details.
Options
lakectl auth groups policies list¶
List policies for the given group
Options
--id string Group identifier
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl auth help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type auth help [path to command] for full details.
Options
lakectl auth policies¶
Manage policies
Options
lakectl auth policies create¶
Create a policy
Options
-h, --help help for create
--id string Policy identifier
--statement-document string JSON statement document path (or "-" for stdin)
lakectl auth policies delete¶
Delete a policy
Options
lakectl auth policies help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type policies help [path to command] for full details.
Options
lakectl auth policies list¶
List policies
Options
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl auth policies show¶
Show a policy
Options
lakectl auth users¶
Manage users
Options
lakectl auth users create¶
Create a user
Options
lakectl auth users credentials¶
Manage user credentials
Options
lakectl auth users credentials create¶
Create user credentials
Options
-h, --help help for create
--id string Username (email for password-based users, default: current user)
lakectl auth users credentials delete¶
Delete user credentials
Options
--access-key-id string Access key ID to delete
-h, --help help for delete
--id string Username (email for password-based users, default: current user)
lakectl auth users credentials help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type credentials help [path to command] for full details.
Options
lakectl auth users credentials list¶
List user credentials
Options
--id string Username (email for password-based users, default: current user)
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl auth users delete¶
Delete a user
Options
lakectl auth users groups¶
Manage user groups
Options
lakectl auth users groups help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type groups help [path to command] for full details.
Options
lakectl auth users groups list¶
List groups for the given user
Options
--id string Username (email for password-based users)
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl auth users help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type users help [path to command] for full details.
Options
lakectl auth users list¶
List users
Options
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl auth users policies¶
Manage user policies
Synopsis
Manage user policies. Requires an external authorization server with matching support.
Options
lakectl auth users policies attach¶
Attach a policy to a user
Options
-h, --help help for attach
--id string Username (email for password-based users)
--policy string Policy identifier
lakectl auth users policies detach¶
Detach a policy from a user
Options
-h, --help help for detach
--id string Username (email for password-based users)
--policy string Policy identifier
lakectl auth users policies help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type policies help [path to command] for full details.
Options
lakectl auth users policies list¶
List policies for the given user
Options
--effective List all distinct policies attached to the user, including by group memberships
--id string Username (email for password-based users)
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl branch¶
Create and manage branches within a repository
Synopsis
Create delete and list branches within a lakeFS repository
Options
lakectl branch create¶
Create a new branch in a repository
Examples
Options
lakectl branch delete¶
Delete a branch in a repository, along with its uncommitted changes (CAREFUL)
Examples
Options
lakectl branch help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type branch help [path to command] for full details.
Options
lakectl branch list¶
List branches in a repository
Examples
Options
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl branch reset¶
Reset uncommitted changes - all of them, or by path
Synopsis
reset changes. There are four different ways to reset changes: 1. reset all uncommitted changes - reset lakefs://myrepo/main 2. reset uncommitted changes under specific path - reset lakefs://myrepo/main --prefix path 3. reset uncommitted changes for specific object - reset lakefs://myrepo/main --object path
Examples
Options
-h, --help help for reset
--object string path to object to be reset
--prefix string prefix of the objects to be reset
-y, --yes Automatically say yes to all confirmations
lakectl branch revert¶
Given a commit, record a new commit to reverse the effect of this commit
Synopsis
The commits will be reverted in left-to-right order
Examples
lakectl branch revert lakefs://example-repo/example-branch commitA
Revert the changes done by commitA in example-branch
branch revert lakefs://example-repo/example-branch HEAD~1 HEAD~2 HEAD~3
Revert the changes done by the second last commit to the fourth last commit in example-branch
Options
--allow-empty-commit allow empty commit (revert without changes)
-h, --help help for revert
-m, --parent-number int the parent number (starting from 1) of the mainline. The revert will reverse the change relative to the specified parent.
-y, --yes Automatically say yes to all confirmations
lakectl branch show¶
Show branch latest commit reference
Examples
Options
lakectl branch-protect¶
Create and manage branch protection rules
Synopsis
Define branch protection rules to prevent direct changes. Changes to protected branches can only be done by merging from other branches.
Options
lakectl branch-protect add¶
Add a branch protection rule
Synopsis
Add a branch protection rule for a given branch name pattern
Examples
Options
lakectl branch-protect delete¶
Delete a branch protection rule
Synopsis
Delete a branch protection rule for a given branch name pattern
Examples
Options
lakectl branch-protect help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type branch-protect help [path to command] for full details.
Options
lakectl branch-protect list¶
List all branch protection rules
Examples
Options
lakectl cherry-pick¶
Apply the changes introduced by an existing commit
Synopsis
Apply the changes from the given commit to the tip of the branch. The changes will be added as a new commit.
Examples
Options
-h, --help help for cherry-pick
-m, --parent-number int the parent number (starting from 1) of the cherry-picked commit. The cherry-pick will apply the change relative to the specified parent.
lakectl commit¶
Commit changes on a given branch
Options
--allow-empty-commit allow a commit with no changes
--allow-empty-message allow an empty commit message
-h, --help help for commit
-m, --message string commit message
--meta strings key value pair in the form of key=value
lakectl completion¶
Generate completion script
Synopsis
To load completions:
Bash:
To load completions for each session, execute once: Linux:
MacOS:
Zsh:
If shell completion is not already enabled in your environment you will need to enable it. You can execute the following once:
To load completions for each session, execute once:
You will need to start a new shell for this setup to take effect.
Fish:
To load completions for each session, execute once:
Options
lakectl config¶
Create/update local lakeFS configuration
Options
lakectl diff¶
Show changes between two commits, or the currently uncommitted changes
Examples
lakectl diff lakefs://example-repo/example-branch
Show uncommitted changes in example-branch.
lakectl diff lakefs://example-repo/main lakefs://example-repo/dev
This shows the differences between master and dev starting at the last common commit.
This is similar to the three-dot (...) syntax in git.
Uncommitted changes are not shown.
lakectl diff --two-way lakefs://example-repo/main lakefs://example-repo/dev
Show changes between the tips of the main and dev branches.
This is similar to the two-dot (..) syntax in git.
Uncommitted changes are not shown.
lakectl diff --two-way lakefs://example-repo/main lakefs://example-repo/dev$
Show changes between the tip of the main and the dev branch, including uncommitted changes on dev.
lakectl diff --prefix some/path lakefs://example-repo/main lakefs://example-repo/dev
Show changes of objects prefixed with 'some/path' between the tips of the main and dev branches.
Options
-h, --help help for diff
--prefix string Show only changes in the given prefix.
--two-way Use two-way diff: show difference between the given refs, regardless of a common ancestor.
lakectl doctor¶
Run a basic diagnosis of the LakeFS configuration
Options
lakectl fs¶
View and manipulate objects
Options
lakectl fs cat¶
Dump content of object to stdout
Options
-h, --help help for cat
--pre-sign Use pre-signed URLs when downloading/uploading data (recommended) (default true)
lakectl fs download¶
Download object(s) from a given repository path
Options
-h, --help help for download
--no-progress Disable progress bar animation for IO operations
-p, --parallelism int Max concurrent operations to perform (default 25)
--part-size int part size in bytes for multipart download (default 8388608)
--pre-sign Use pre-signed URLs when downloading/uploading data (recommended) (default true)
-r, --recursive recursively download all objects under path
lakectl fs help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type fs help [path to command] for full details.
Options
lakectl fs ls¶
List entries under a given tree
Options
lakectl fs presign¶
return a pre-signed URL for reading the specified object
Options
lakectl fs rm¶
Delete object
Options
-C, --concurrency int max concurrent single delete operations to send to the lakeFS server (default 50)
-h, --help help for rm
-r, --recursive recursively delete all objects under the specified path
lakectl fs stat¶
View object metadata
Options
-h, --help help for stat
--pre-sign Use pre-signed URLs when downloading/uploading data (recommended) (default true)
lakectl fs upload¶
Upload a local file to the specified URI
Options
--content-type string MIME type of contents
-h, --help help for upload
--no-progress Disable progress bar animation for IO operations
-p, --parallelism int Max concurrent operations to perform (default 25)
--pre-sign Use pre-signed URLs when downloading/uploading data (recommended) (default true)
-r, --recursive recursively copy all files under local source
-s, --source string local file to upload, or "-" for stdin
lakectl fs stage¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Link an external object with a path in a repository
Synopsis
Link an external object with a path in a repository, creating an uncommitted change. The object location must be outside the repository's storage namespace
Options
--checksum string Object MD5 checksum as a hexadecimal string
--content-type string MIME type of contents
-h, --help help for stage
--location string fully qualified storage location (i.e. "s3://bucket/path/to/object")
--meta strings key value pairs in the form of key=value
--mtime int Object modified time (Unix Epoch in seconds). Defaults to current time
--size int Object size in bytes
lakectl fs update-metadata¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Update user metadata on the specified URI
Options
-h, --help help for update-metadata
--metadata strings Metadata to set, in the form key1=value1,key2=value2
lakectl gc¶
Manage the garbage collection policy
Options
lakectl gc delete-config¶
Deletes the garbage collection policy for the repository
Examples
Options
lakectl gc get-config¶
Show the garbage collection policy for this repository
Examples
Options
lakectl gc help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type gc help [path to command] for full details.
Options
lakectl gc set-config¶
Set garbage collection policy JSON
Synopsis
Sets the garbage collection policy JSON. Example configuration file: { "default_retention_days": 21, "branches": [ { "branch_id": "main", "retention_days": 28 }, { "branch_id": "dev", "retention_days": 14 } ] }
Examples
Options
lakectl help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type lakectl help [path to command] for full details.
Options
lakectl identity¶
Show identity info
Synopsis
Show the info of the configured user in lakectl
Examples
Options
lakectl import¶
Import data from external source to a destination branch
Options
--allow-empty-message allow an empty commit message (default true)
--from string prefix to read from (e.g. "s3://bucket/sub/path/"). must not be in a storage namespace
-h, --help help for import
-m, --message string commit message
--meta strings key value pair in the form of key=value
--no-progress switch off the progress output
--to string lakeFS path to load objects into (e.g. "lakefs://repo/branch/sub/path/")
lakectl local¶
Sync local directories with lakeFS paths
Options
lakectl local checkout¶
Sync local directory with the remote state.
Options
--all Checkout given source branch or reference for all linked directories
-h, --help help for checkout
--no-progress Disable progress bar animation for IO operations
-p, --parallelism int Max concurrent operations to perform (default 25)
--pre-sign Use pre-signed URLs when downloading/uploading data (recommended) (default true)
-r, --ref string Checkout the given reference
-y, --yes Automatically say yes to all confirmations
lakectl local clone¶
Clone a path from a lakeFS repository into a new directory.
Options
--gitignore Update .gitignore file when working in a git repository context (default true)
-h, --help help for clone
--no-progress Disable progress bar animation for IO operations
-p, --parallelism int Max concurrent operations to perform (default 25)
--pre-sign Use pre-signed URLs when downloading/uploading data (recommended) (default true)
lakectl local commit¶
Commit changes from local directory to the lakeFS branch it tracks.
Options
--allow-empty-message allow an empty commit message
--force Commit changes even if remote branch includes uncommitted changes external to the synced path
-h, --help help for commit
-m, --message string commit message
--meta strings key value pair in the form of key=value
--no-progress Disable progress bar animation for IO operations
-p, --parallelism int Max concurrent operations to perform (default 25)
--pre-sign Use pre-signed URLs when downloading/uploading data (recommended) (default true)
lakectl local help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type local help [path to command] for full details.
Options
lakectl local init¶
set a local directory to sync with a lakeFS path.
Options
--force Overwrites if directory already linked to a lakeFS path
--gitignore Update .gitignore file when working in a git repository context (default true)
-h, --help help for init
lakectl local list¶
find and list directories that are synced with lakeFS.
Options
lakectl local pull¶
Fetch latest changes from lakeFS.
Options
--force Reset any uncommitted local change
-h, --help help for pull
--no-progress Disable progress bar animation for IO operations
-p, --parallelism int Max concurrent operations to perform (default 25)
--pre-sign Use pre-signed URLs when downloading/uploading data (recommended) (default true)
lakectl local status¶
show modifications (both remote and local) to the directory and the remote location it tracks
Options
lakectl log¶
Show log of commits
Synopsis
Show log of commits for a given branch
Examples
Options
--after string show results after this value (used for pagination)
--amount int number of results to return. By default, all results are returned
--dot return results in a dotgraph format
--first-parent follow only the first parent commit upon seeing a merge commit
-h, --help help for log
--limit limit result just to amount. By default, returns whether more items are available.
--no-merges skip merge commits
--objects strings show results that contains changes to at least one path in that list of objects. Use comma separator to pass all objects together
--prefixes strings show results that contains changes to at least one path in that list of prefixes. Use comma separator to pass all prefixes together
--show-meta-range-id also show meta range ID
--since string show results since this date-time (RFC3339 format)
--stop-at string a Ref to stop at (included in results)
lakectl merge¶
Merge & commit changes from source branch into destination branch
Synopsis
Merge & commit changes from source branch into destination branch
Options
--allow-empty Allow merge when the branches have the same content
--allow-empty-message allow an empty commit message (default true)
--force Allow merge into a read-only branch or into a branch with the same content
-h, --help help for merge
-m, --message string commit message
--meta strings key value pair in the form of key=value
--squash Squash all changes from source into a single commit on destination
--strategy string In case of a merge conflict, this option will force the merge process to automatically favor changes from the dest branch ("dest-wins") or from the source branch("source-wins"). In case no selection is made, the merge process will fail in case of a conflict
lakectl metastore¶
Manage metastore commands
Options
lakectl metastore copy¶
Copy or merge table
Synopsis
Copy or merge table. the destination table will point to the selected branch
Options
--catalog-id string Glue catalog ID
--dbfs-root dbfs:/ dbfs location root will replace dbfs:/ in the location before transforming
--from-client-type string metastore type [hive, glue]
--from-schema string source schema name
--from-table string source table name
-h, --help help for copy
--metastore-uri string Hive metastore URI
-p, --partition strings partition to copy
--serde string serde to set copy to [default is to-table]
--to-branch string lakeFS branch name
--to-client-type string metastore type [hive, glue]
--to-schema string destination schema name [default is from-branch]
--to-table string destination table name [default is from-table]
lakectl metastore copy-all¶
Copy from one metastore to another
Synopsis
copy or merge requested tables between hive metastores. the destination tables will point to the selected branch
Options
--branch string lakeFS branch name
--continue-on-error prevent copy-all from failing when a single table fails
--dbfs-root dbfs:/ dbfs location root will replace dbfs:/ in the location before transforming
--from-address string source metastore address
--from-client-type string metastore type [hive, glue]
-h, --help help for copy-all
--schema-filter string filter for schemas to copy in metastore pattern (default ".*")
--table-filter string filter for tables to copy in metastore pattern (default ".*")
--to-address string destination metastore address
--to-client-type string metastore type [hive, glue]
lakectl metastore copy-schema¶
Copy schema
Synopsis
Copy schema (without tables). the destination schema will point to the selected branch
Options
--catalog-id string Glue catalog ID
--dbfs-root dbfs:/ dbfs location root will replace dbfs:/ in the location before transforming
--from-client-type string metastore type [hive, glue]
--from-schema string source schema name
-h, --help help for copy-schema
--metastore-uri string Hive metastore URI
--to-branch string lakeFS branch name
--to-client-type string metastore type [hive, glue]
--to-schema string destination schema name [default is from-branch]
lakectl metastore create-symlink¶
Create symlink table and data
Synopsis
create table with symlinks, and create the symlinks in s3 in order to access from external services that could only access s3 directly (e.g athena)
Options
--branch string lakeFS branch name
--catalog-id string Glue catalog ID
--from-client-type string metastore type [hive, glue]
--from-schema string source schema name
--from-table string source table name
-h, --help help for create-symlink
--path string path to table on lakeFS
--repo string lakeFS repository name
--to-schema string destination schema name
--to-table string destination table name
lakectl metastore diff¶
Show column and partition differences between two tables
Options
--catalog-id string Glue catalog ID
--from-address string source metastore address
--from-client-type string metastore type [hive, glue]
--from-schema string source schema name
--from-table string source table name
-h, --help help for diff
--metastore-uri string Hive metastore URI
--to-address string destination metastore address
--to-client-type string metastore type [hive, glue]
--to-schema string destination schema name
--to-table string destination table name [default is from-table]
lakectl metastore help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type metastore help [path to command] for full details.
Options
lakectl metastore import-all¶
Import from one metastore to another
Synopsis
import requested tables between hive metastores. the destination tables will point to the selected repository and branch table with location s3://my-s3-bucket/path/to/table will be transformed to location s3://repo-param/bucket-param/path/to/table
Options
--branch string lakeFS branch name
--continue-on-error prevent import-all from failing when a single table fails
--dbfs-root dbfs:/ dbfs location root will replace dbfs:/ in the location before transforming
--from-address string source metastore address
--from-client-type string metastore type [hive, glue]
-h, --help help for import-all
--repo string lakeFS repo name
--schema-filter string filter for schemas to copy in metastore pattern (default ".*")
--table-filter string filter for tables to copy in metastore pattern (default ".*")
--to-address string destination metastore address
--to-client-type string metastore type [hive, glue]
lakectl repo¶
Manage and explore repos
Options
lakectl repo create¶
Create a new repository
Examples
Options
-d, --default-branch string the default branch of this repository (default "main")
-h, --help help for create
--sample-data create sample data in the repository
lakectl repo delete¶
Delete existing repository
Examples
Options
lakectl repo help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type repo help [path to command] for full details.
Options
lakectl repo list¶
List repositories
Options
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl repo create-bare¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Create a new repository with no initial branch or commit
Examples
Options
-d, --default-branch string the default branch name of this repository (will not be created) (default "main")
-h, --help help for create-bare
lakectl show¶
See detailed information about an entity
Options
lakectl show commit¶
See detailed information about a commit
Options
lakectl show help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type show help [path to command] for full details.
Options
lakectl tag¶
Create and manage tags within a repository
Synopsis
Create delete and list tags within a lakeFS repository
Options
lakectl tag create¶
Create a new tag in a repository
Examples
lakectl tag create lakefs://example-repo/example-tag lakefs://example-repo/2397cc9a9d04c20a4e5739b42c1dd3d8ba655c0b3a3b974850895a13d8bf9917
Options
lakectl tag delete¶
Delete a tag from a repository
Options
lakectl tag help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type tag help [path to command] for full details.
Options
lakectl tag list¶
List tags in a repository
Examples
Options
--amount int how many results to return (default 100)
--after string show results after this value (used for pagination)
--prefix string filter results by prefix (used for pagination)
-h, --help help for list
lakectl tag show¶
Show tag's commit reference
Options
Undocumented commands¶
Warning
These commands are plumbing commands and for internal use only. Avoid using them unless you're really sure you know what you're doing, or have been in contact with lakeFS support!
lakectl abuse¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Abuse a running lakeFS instance. See sub commands for more info.
Options
lakectl abuse commit¶
Commits to the source branch repeatedly
Options
--amount int amount of commits to do (default 100)
--gap duration duration to wait between commits (default 2s)
-h, --help help for commit
lakectl abuse create-branches¶
Create a lot of branches very quickly.
Options
--amount int amount of things to do (default 1000000)
--branch-prefix string prefix to create branches under (default "abuse-")
--clean-only only clean up past runs
-h, --help help for create-branches
--parallelism int amount of things to do in parallel (default 100)
lakectl abuse help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type abuse help [path to command] for full details.
Options
lakectl abuse link-same-object¶
Link the same object in parallel.
Options
--amount int amount of link object to do (default 1000000)
-h, --help help for link-same-object
--key string key used for the test (default "linked-object")
--parallelism int amount of link object to do in parallel (default 100)
lakectl abuse list¶
List from the source ref
Options
--amount int amount of lists to do (default 1000000)
-h, --help help for list
--parallelism int amount of lists to do in parallel (default 100)
--prefix string prefix to list under (default "abuse/")
lakectl abuse merge¶
Merge non-conflicting objects to the source branch in parallel
Options
--amount int amount of merges to perform (default 1000)
-h, --help help for merge
--parallelism int number of merges to perform in parallel (default 100)
lakectl abuse random-delete¶
Delete keys from a file and generate random delete from the source ref for those keys.
Options
--amount int amount of reads to do (default 1000000)
--from-file string read keys from this file ("-" for stdin)
-h, --help help for random-delete
--parallelism int amount of reads to do in parallel (default 100)
lakectl abuse random-read¶
Read keys from a file and generate random reads from the source ref for those keys.
Options
--amount int amount of reads to do (default 1000000)
--from-file string read keys from this file ("-" for stdin)
-h, --help help for random-read
--parallelism int amount of reads to do in parallel (default 100)
lakectl abuse random-write¶
Generate random writes to the source branch
Options
--amount int amount of writes to do (default 1000000)
-h, --help help for random-write
--parallelism int amount of writes to do in parallel (default 100)
--prefix string prefix to create paths under (default "abuse/")
lakectl bisect¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Binary search to find the commit that introduced a bug
Options
lakectl bisect bad¶
Set 'bad' commit that is known to contain the bug
Options
lakectl bisect good¶
Set current commit as 'good' commit that is known to be before the bug was introduced
Options
lakectl bisect help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type bisect help [path to command] for full details.
Options
lakectl bisect log¶
Print out the current bisect state
Options
lakectl bisect reset¶
Clean up the bisection state
Options
lakectl bisect run¶
Bisecting based on command status code
Options
lakectl bisect start¶
Start a bisect session
Options
lakectl bisect view¶
Current bisect commits
Options
lakectl cat-hook-output¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Cat actions hook output
Examples
Options
lakectl cat-sst¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Explore lakeFS .sst files
Options
--amount int how many records to return, or -1 for all records (default -1)
-f, --file string path to an sstable file, or "-" for stdin
-h, --help help for cat-sst
lakectl docs¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Options
lakectl find-merge-base¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Find the commits for the merge operation
Options
lakectl refs-dump¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Dumps refs (branches, commits, tags) to the underlying object store
Options
-h, --help help for refs-dump
-o, --output string output filename (default stdout)
--poll-interval duration poll status check interval (default 3s)
--timeout duration timeout for polling status checks (default 1h0m0s)
lakectl refs-restore¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Restores refs (branches, commits, tags) from the underlying object store to a bare repository
Synopsis
restores refs (branches, commits, tags) from the underlying object store to a bare repository.
This command is expected to run on a bare repository (i.e. one created with 'lakectl repo create-bare'). Since a bare repo is expected, in case of transient failure, delete the repository and recreate it as bare and retry.
Examples
aws s3 cp s3://bucket/_lakefs/refs_manifest.json - | lakectl refs-restore lakefs://my-bare-repository --manifest -
Options
-h, --help help for refs-restore
--manifest refs-dump path to a refs manifest json file (as generated by refs-dump). Alternatively, use "-" to read from stdin
--poll-interval duration poll status check interval (default 3s)
--timeout duration timeout for polling status checks (default 1h0m0s)
lakectl usage¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Usage reports from lakeFS
Options
lakectl usage help¶
Help about any command
Synopsis
Help provides help for any command in the application. Simply type usage help [path to command] for full details.
Options
lakectl usage summary¶
Warning
lakeFS plumbing command. Don't use unless you're really sure you know what you're doing.
Summary reports from lakeFS