In this article, we will guide you through the process on how to download the entire the S3 bucket. Read this full article to understand the topic better and follow the steps properly. 

The AWS CLI sync command when passed in the S3 bucket like a source and a directory on the file system as a destination, for instance aws s3 sync s3://YOUR_BUCKET , it will help you download the entire bucket. 

More importantly, the sync command repeatedly copies all the content from the source to the destination. 

First and foremost, set the --dryrun parameter to run the sync command in the test mode. Once the command is run successfully, it will download S3 bucket directories, files and folders to the local file system. 

Plus, ‘.’ character describes the current directory. Here we will create a directory to store the S3 bucket folders and files. After that navigate it and then run the sync command in the real mode. 

Let us first run the sync command in order to download entire folders and files of the S3 bucket to the current directory on your system. 

Download S3 BUcket with Filters

How to download a folder from S3? You can filters downloaded files from the bucket to your local system by using two parameters - -include and - -exclude while you run the sync command, for example aws s3 sync s3://YOUR_BUCKET . --quiet --exclude "*.png". 

Here all the files are included by default. The below command will help you exclude all the .png images from the downloaded files. 

This command will download all the files excluding the ‘.png’ files.

Additionally, we have used the different parameter called - -quite which will suppress the sync command output. 

We can also exclude the whole directory from downloading. 

We have also deleted all the files that are in the image/ folder, for example images/cat.jpg, images/dog.png, etc. 

On the other hand, if you want to download the .pdf files to your local system, you can eliminate everything and include files with just .pdf extension. 

 Image 

Keep in mind that when specifying the --exclude and --include arguments, the order matters.

Higher precedence is given to filters that are listed later in the command. For instance, because the --exclude argument overrides --include, the following command excludes all files.

However, if we added the second --include parameter, all.png images would be downloaded to our local file system:

The --exclude and --include arguments can also be set as many times as necessary. For instance, the following command downloads entire .txt and.pdf files and folders to the local file system from the s3 bucket.