Boto s3 download file example wait exists

MinIO Client (mc) provides a modern alternative to UNIX commands like ls, cat, cp, mirror, diff etc. It supports filesystems and Amazon S3 compatible cloud storage service (AWS Signature v2 and v4).

1. Download MinIO Client

Docker Stable

Docker Edge

Note: Above examples run against MinIO play environment by default. To run against other S3 compatible servers, start the container this way:

then use the command.

Homebrew (macOS)

Install mc packages using Homebrew

Binary Download (GNU/Linux)

Binary Download (Microsoft Windows)

Install from Source

Source installation is intended only for developers and advanced users. command does not support update notifications for source based installations. Please download official releases from https://min.io/download/#minio-client.

If you do not have a working Golang environment, please follow How to install Golang.

2. Run MinIO Client

GNU/Linux

macOS

Microsoft Windows

3. Add a Cloud Storage Service

Note: If you are planning to use only on POSIX compatible filesystems, you may skip this step and proceed to Step 4.

To add one or more Amazon S3 compatible hosts, please follow the instructions below. stores all its configuration information in file.

Usage

Keys must be supplied by argument or standard input.

Alias is simply a short name to your cloud storage service. S3 end-point, access and secret keys are supplied by your cloud storage provider. API signature is an optional argument. By default, it is set to "S3v4".

Example - MinIO Cloud Storage

MinIO server displays URL, access and secret keys.

Example - Amazon S3 Cloud Storage

Get your AccessKeyID and SecretAccessKey by following AWS Credentials Guide.

Example - Google Cloud Storage

Get your AccessKeyID and SecretAccessKey by following Google Credentials Guide

Example - Specify keys using standard input

Prompt

Pipe from STDIN

Specify temporary host configuration through environment variable

Static credentials

Example:

Rotating credentials

Example:

4. Test Your Setup

is pre-configured with https://play.min.io, aliased as "play". It is a hosted MinIO server for testing and development purpose. To test Amazon S3, simply replace "play" with "s3" or the alias you used at the time of setup.

Example:

List all buckets from https://play.min.io

5. Everyday Use

You may add shell aliases to override your common Unix tools.

6. Global Options

Option [--debug]

Debug option enables debug output to console.

Example: Display verbose debug output for command.

Option [--json]

JSON option enables parseable output in JSON lines, also called as NDJSON format.

Example: List all buckets from MinIO play service.

Option [--no-color]

This option disables the color theme. It is useful for dumb terminals.

Option [--quiet]

Quiet option suppress chatty console output.

Option [--config-dir]

Use this option to set a custom config path.

Option [ --insecure]

Skip SSL certificate verification.

Option [--version]

Display the current version of installed

Example: Print version of mc.

7. Commands

Command

command lists files, buckets and objects. Use flag to list partially copied content.

Example: List all buckets on https://play.min.io.

Command

command lists buckets and directories in a tree format. Use flag to include files/objects in listing.

Example: List all buckets on play/test-bucket in a tree format.

Command

command creates a new bucket on an object storage. On a filesystem, it behaves like command. Bucket is equivalent of a drive or mount point in filesystems and should not be treated as folders. MinIO does not place any limits on the number of buckets created per user.
On Amazon S3, each account is limited to 100 buckets. Please refer to Buckets Restrictions and Limitations on S3 for more information.

Example: Create a new bucket named "mybucket" on https://play.min.io.

Example: Create a new bucket named "mybucket" on https://s3.amazonaws.com.

Command

command removes a bucket and all its contents on an object storage. On a filesystem, it behaves like command.

NOTE: When a bucket is removed all bucket configurations associated with the bucket will also be removed. All objects and their versions will be removed as well. If you need to preserve bucket and its configuration - only empty the objects and versions in a bucket use instead.

Example: Remove a bucket named "mybucket" on https://play.min.io.

Command

command concatenates contents of a file or object to another. You may also use it to simply display the contents to stdout

Example: Display the contents of a text file

Example: Display the contents of a server encrypted object

Example: Display the contents of a server encrypted object . Pass base64 encoded string if encryption key contains non-printable character like tab

*Example: Display the content of an object 10 days earlier *

*Example: Display the content of an object at a specific date/time in the past *

Command

run sql queries on objects.

Example: Select all columns on a set of objects recursively on AWS S3

Example: Run an aggregation query on an object on MinIO

Example: Run an aggregation query on an encrypted object with customer provided keys

For more query examples refer to official AWS S3 documentation here

Command

display first 'n' lines of an object

Example: Display the first line of a text file

Example: Display the first line of a server encrypted object

Command

sets and gets object lock configuration

deprecates and removes the command.
The retention command fully replaces functionality.

Command

sets object retention for objects with a given prefix or the default
retention settings for a bucket.

Example: Set governance for 30 days for object on bucket

Objects created with prefix in the above bucket cannot be deleted until the compliance period is over

Example: Set compliance for 30 days as default retention setting on bucket

Objects created in the above bucket cannot be deleted until the compliance period is over

Command

sets object legal hold for objects

Example: Enable legal hold for objects with prefix on bucket

Objects created with prefix in the above bucket cannot be deleted until the legal hold is lifted

Command

command copies contents of stdin to a target. When no target is specified, it writes to stdout.

Example: Stream MySQL database dump to Amazon S3 directly.

Command

command copies data from one or more sources to a target. All copy operations to object storage are verified with MD5SUM checksums. Interrupted or failed copy operations can be resumed from the point of failure.

Example: Copy a text file to an object storage.

Example: Copy a text file to an object storage with specified metadata.

Example: Copy a folder recursively from MinIO cloud storage to Amazon S3 cloud storage with specified metadata.

Example: Copy a text file to an object storage and assign storage-class to the uploaded object.

Example: Copy a server-side encrypted file to an object storage.

Example: Perform key-rotation on a server-side encrypted object

Notice that two different aliases myminio1 and myminio2 are used for the same endpoint to provide the old secretkey and the newly rotated key.

Example: Copy a javascript file to object storage and assign Cache-Control header to the uploaded object

Example: Copy a text file to an object storage and preserve the filesyatem attributes.

Command

command moves data from one or more sources to a target. All move operations to object storage are verified with MD5SUM checksums. Interrupted or failed move operations can be resumed from the point of failure.

Example: Move a text file to an object storage.

Example: Move a text file to an object storage with specified metadata.

Example: Move a folder recursively from MinIO cloud storage to Amazon S3 cloud storage with specified metadata.

Example: Move a text file to an object storage and assign storage-class to the uploaded object.

Example: Move a server-side encrypted file to an object storage.

Example: Perform key-rotation on a server-side encrypted object

Notice that two different aliases myminio1 and myminio2 are used for the same endpoint to provide the old secretkey and the newly rotated key.

Example: Move a javascript file to object storage and assign Cache-Control header to the uploaded object

Example: Move a text file to an object storage and preserve the filesyatem attributes.

Command

Use command to remove file or object

Example: Remove a single object.

Example: Remove an encrypted object.

Example: Recursively remove a bucket's contents. Since this is a dangerous operation, you must explicitly pass option.

Example: Remove all uploaded incomplete files for an object.

Example: Remove object and output a message only if the object is created older than 1 day, 2 hours and 30 minutes. Otherwise, the command stays quiet and nothing is printed out.

Example: Remove a particular version ID.

Example: Remove all object versions older than one year.

Command

command securely grants upload or download access to object storage. This access is only temporary and it is safe to share with remote users and applications. If you want to grant permanent access, you may look at command instead.

Generated URL has access credentials encoded in it. Any attempt to tamper the URL will invalidate the access. To understand how this mechanism works, please follow Pre-Signed URL technique.

Sub-command - Share Download

command generates URLs to download objects without requiring access and secret keys. Expiry option sets the maximum validity period (no more than 7 days), beyond which the access is revoked automatically.

Example: Grant temporary access to an object with 4 hours expiry limit.

Sub-command - Share Upload

command generates a ‘curl’ command to upload objects without requiring access/secret keys. Expiry option sets the maximum validity period (no more than 7 days), beyond which the access is revoked automatically. Content-type option restricts uploads to only certain type of files.

Example: Generate a command to enable upload access to . User replaces with the actual filename to upload

Sub-command - Share List

command lists unexpired URLs that were previously shared

Command

command is similar to , except it synchronizes contents between filesystems and object storage.

Example: Mirror a local directory to 'mybucket' on https://play.min.io.

Example: Continuously watch for changes on a local directory and mirror the changes to 'mybucket' on https://play.min.io.

Command

command finds files which match the given set of parameters. It only lists the contents which match the given set of criteria.

Example: Find all jpeg images from s3 bucket and copy to MinIO "play/bucket" bucket continuously.

Command

command computes the differences between the two directories. It only lists the contents which are missing or which differ in size.

It DOES NOT compare the contents, so it is possible that the objects which are of same name and of the same size, but have difference in contents are not detected. This way, it can perform high speed comparison on large volumes or between sites

Example: Compare a local directory and a remote object storage.

Option [--json]

JSON option enables parseable output in JSON lines format.

Example: diff json output.

Diff values in json output

Command

provides a convenient way to watch on various types of event notifications on object
storage and filesystem.

Example: Watch for all events on object storage

Example: Watch for all events on local directory

Command

provides a convenient way to manage various types of event notifications on a bucket. MinIO event notification can be configured to use AMQP, Redis, ElasticSearch, NATS and PostgreSQL services. MinIO configuration provides more details on how these services can be configured.

Example: List all configured bucket notifications

Example: Add a new 'sqs' notification resource only to notify on ObjectCreated event

Example: Add a new 'sqs' notification resource with filters

Add and filtering rules for notification resource.

Example: Remove a 'sqs' notification resource

Command

- A convenient way to manage bucket lifecycle configuration.

Example: List the lifecycle management rules

For more details about the lifecycle configuration, refer to official AWS S3 documentation here

Example: Add rule for prefix "dev" in bucket "testbucket" on play and transition after 2 days to transition tier specified by storage class label "hdd_tier". "hdd_tier" is the label specified when setting up MinIO transition target via command

Example: Edit the lifecycle management configuration rule given by ID "btd6pdot8748n94elvl0" to set tags

Example: Remove the lifecycle management configuration rule given by ID "Documents"

Command

Manage anonymous bucket policies to a bucket and its contents

Example: Show current anonymous bucket policy

Show current anonymous bucket policy for sub-directory

Example : Set anonymous bucket policy to download only

Set anonymous bucket policy for sub-directory and its objects to only. Now, objects under the sub-directory are publicly accessible. e.g is available at https://play.min.io:9000/mybucket/myphotos/2020/yourobjectname

Example : Set anonymous bucket policy from a JSON file

Configure bucket policy for with a policy JSON file.

Example : Remove current anonymous bucket policy

Remove any bucket policy for mybucket/myphotos/2020/ sub-directory.

Command

command provides a convenient way to set, remove, and list bucket/object tags. Tags are defined as key-value pairs.

Example : List tags assigned to an object

List tags for in in alias

Example : Set tags for an object

Set tags for in in alias

Example : Remove tags assigned to an object

Remove tags assigned to in in alias

Command

Please visit here for a more comprehensive admin guide.

Command

command provides a convenient way to manage aliases entries in your config file . It is also OK to edit the config file manually using a text editor.

Example: Manage Config File

Add MinIO server access and secret keys to config file alias entry. Note that, the history feature of your shell may record these keys and pose a security risk. On shell, use and to disable and enable history feature momentarily.

Remove the alias from the config file.

List all configured aliases

Command

Check for new software updates from https://dl.min.io. Experimental flag checks for unstable experimental releases primarily meant for testing purposes.

Example: Check for an update.

Command

command displays information on objects (with optional prefix) contained in the specified bucket on an object storage. On a filesystem, it behaves like command.

Example: Display information on a bucket named "mybucket" on https://play.min.io.

Example: Display information on an encrypted object "myobject" in "mybucket" on https://play.min.io.

Example: Display information on objects contained in the bucket named "mybucket" on https://play.min.io.

Command

manage bucket versioning

Example: Enable versioning on bucket

Example: Display the version configuration for bucket

Example: Suspend versioning for bucket

Command

reverts latest PUT/DELETE operations

Example: Undo the last 3 uploads and/or removals of a particular object

Command

manages bucket encryption config

Example: Display bucket encryption status for bucket

Example: Set SSE-S3 auto encryption for bucket on alias

Example: Set SSE-KMS auto encryption for bucket on alias with KMS Key Id "arn:aws:kms:us-east-1:xxx:key/xxx"

Example: Clear auto encryption config for bucket on alias

Command

manages bucket server side replication

Example: Add replication configuration rule on on alias .Enable delete marker replication and replication of versioned deletes for the configuration

Example: Disable replication configuration rule with rule Id "bsibgh8t874dnjst8hkg" on bucket "mybucket" with prefix "prefix" for alias

Example: Change priority of rule with rule ID "bsibgh8t874dnjst8hkg" on bucket "mybucket" for alias .

Example: Clear tags on rule ID "bsibgh8t874dnjst8hkg" for target myminio/bucket which has a replication configuration rule with prefix "prefix"

*Example: Enable delete marker replication and versioned delete replication on rule ID "bsibgh8t874dnjst8hkg" for target myminio/bucket which has a replication configuration rule with prefix "prefix"

*Example: Disable delete marker and versioned delete replication on rule ID "bsibgh8t874dnjst8hkg" for target myminio/bucket which has a replication configuration rule with prefix "prefix"

Example: List replication configuration rules set on on alias

Example: Clear replication configuration for bucket on alias

Example: Remove replication configuration rule with id for bucket on alias

Example: Import replication configuration for bucket on alias from

Example: Export replication configuration for bucket on alias to