Link Search Menu Expand Document

Copying files into lakeFS using AWS CLI

  1. If you don’t have the AWS CLI installed, follow the instructions here.
  2. Configure a new connection profile using the credentials we generated earlier:

    aws configure --profile local
    # output:
    # AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
    # AWS Secret Access Key [None]: ****************************************
    # Default region name [None]:
    # Default output format [None]:
    
  3. Let’s test to see that it works. We’ll do that by calling s3 ls which should list our repositories for us:

    aws --endpoint-url=http://s3.local.lakefs.io:8000 --profile local s3 ls
    # output:
    # 2020-05-18 17:47:03 example
    

    Note: We’re using s3.local.lakefs.io - a special DNS record which always resolves to localhost, subdomains included.
    Since S3’s API uses subdomains for bucket addressing, simply using localhost:8000 as an endpoint URL will not work.

  4. Great, now let’s copy some files. We’ll write to the master branch. This is done by prefixing our path with the name of the branch we’d like to read/write from:

    aws --endpoint-url=http://s3.local.lakefs.io:8000 --profile local s3 cp ./foo.txt s3://example/master/
    # output:
    # upload: ./foo.txt to s3://example/master/foo.txt
    
  5. Back in the lakeFS UI, we should be able to see our file added to the master branch!

    Object Added

Next steps

Now that your repository contains some data, what about using the lakeFS CLI for committing, branching and viewing the history?