Finding Publicly Readable Files in your Amazon S3 Account

Amazon S3 is a cloud storage service which is used by thousands of Enterprises worldwide. The most common use case is data backups. Before going into the specifics, lets try and understand how Amazon S3 organizes data. A registered Amazon S3 user can organize his data (files) into Buckets. A file can be fetched using its unique key. The user can access his file using the full path   http://Bucket_Name.s3.amazonaws.com/Key 

e.g. http://code.securitytube.net.s3.amazonaws.com/IP-Packet-Injection.c

In the above example code.securitytube.net is the bucket name and IP-Packet-Injection.c is the Key.

The service is great but is prone to easy misconfiguration by Sys Admins who may be new to the cloud. As lame as it might sound, the most common mistake seems to be to make your S3 data publicly readable. Robin Wood (@digininja) was the first to point this out and wrote a tool called Bucket Finder. Buckets with public-read have directory indexing enabled as an XML listing. The tool can try and download files from the bucket using the listing. Below is an example:

If a Bucket is private, we cannot fetch the listing:

However, interestingly files inside a private bucket could be mistakenly have public-read enabled on them.

What this means is that you could use tools which try to find "Hidden" files and directories on Web Servers in this scenario once you are aware that the Bucket exists but is private.

Recently the Metasploit team did some analysis on a larger sample of S3 buckets and published an article which confirmed that many Enterprises have misconfigured their S3 buckets to be publicly readable.

Now coming to the goal of this post: If you use Amazon S3 what should you do? Immediately check your S3 buckets for files and make any world readable files private, if you accidentally have them public right now.

If you have thousands of files, how would you check and do this? Definitely not manually :) In this post, I will show you how to trivially automate the process using a Python library called Boto.

Let us first try and understand how the permission sets look like from a programmer's perspective. Below is a simple script to learn about permissions:

Let us set the permission set to "private" on the bucket "hackoftheday" and see the response:

Let us now set the permission to "public-read" on the bucket "hackoftheday":

Fantastic! So we basically note that "READ" is set whenever "public-read" is there on the bucket. Please note that if you go through the documentation in detail, READ can also be set when you do not make the bucket publicly readable, but readable to an authenticated user on Amazon S3. This is unsafe as well as this could pretty much be ANY other S3 user.

Now, all we have to do is check for READ on every file our bucket to check if any of them were supposed to be private. Here is the code to do it:



Let us run this against "hackoftheday" now. The bucket itself is private but a file inside is public readable:

Lets check js.securitytube.net now:

 

Awesome! So you see how just a few lines of Python can allow you to create your own S3 scanner to ensure your S3 files are safe :) 

If you like Video Demos, please checkout my explanation here. 

 

If you are interested in learning how to use Python for Pentesting then please have a look at our SecurityTube Python Scripting Expert course. This course is ideal for penetration testers, security enthusiasts and network administrators who want to learn to automate tasks or go beyond just using ready made tools. We will be covering topics in system security, network security, attacking web applications and services, exploitation techniques, malware and binary analysis and task automation.We have students from 73+ countries taking this course already!