Saving Time and Money with Fake S3

Online storage has become a necessity to many people. This is because of the security associated with online storage. One of the most common online storage is the Simple Storage Service, popularly known as S3, offered by Amazon Web Services. In as much as it is a good service, we at spool believe that the Amazon guys are exploiting people.

Amazon S3 is a extremely powerful service at the core of Amazon Web Services. However, outside of a production environment, S3 can be challenging to work with. It involves passing keys around, provisioning user accounts, and a reliable network connection — not to mention it costs money.

At Spool, we built Fake S3 to make working with S3 in development and testing environments much easier. Our goal was to make a self contained executable that can mimic the majority of S3 Rest API with few external dependencies.

For that reason, we have developed our own fake S3 that will work to help online users get value for their money. What we have developed is similar to S3 just that it will be offered by different people. There are greater benefits of using our services than seeking them from Amazon Webs services.

First, we are known for charging less compared to our competitors, the Amazon counterparts. We see no need in overcharging a service that does not require much technicality. Therefore, based on saving money, we should have more clients to serve.

For development, each engineer runs her own instance of Fake S3 where she can put gigabytes of images and video to develop and test against, and her setup will work offline because it is all local. We also have a continuous integration setup that is running tests 24/7 (often against large video files). Fake S3 saves us $1000 a month in bandwidth alone for our tests. In both development and testing, the time saved in not waiting for assets to go back and forth into AWS, especially our larger media files, makes Fake S3 very useful.

We’re releasing Fake S3 as a gem on github. It’s an early release and we’ll keep improving it. If you have ideas or issues, please contribute to the project!

Installation

gem install fakes3

Running

fakes3 -r ~/fakes3_root -p 10001

Example Client Code

require 'rubygems'
require 'aws/s3'

include AWS::S3
AWS::S3::Base.establish_connection!(:access_key_id => "123",
                                    :secret_access_key => "abc",
                                    :server => "localhost",
                                    :port => "10001")

Bucket.create('mystuff')

('a'..'z').each do |filename|
  S3Object.store(filename, 'Hello World', 'mystuff')
end

bucket = Bucket.find('mystuff')
bucket.objects.each do |s3_obj|
  puts "#{s3_obj.key}:#{s3_obj.value}"
end

Bucket.delete("mystuff",:force => true) # Delete your bucket and all its keys

In general clients can work if you can specify a host and port to connect them, as well as forcing path style requests (instead of subdomains.) Subdomain style S3 requests can work, but it involves adding your bucket names into /etc/hosts (ie. s3.localhost or mybucket.localhost) or using dnsmasq if you have a large number of buckets.

Simulating Network Conditions

Another useful feature is the support for simulating network conditions. You can run fakes3 with bandwidth limiting enabled like so:

fakes3 -r ~/fakes3_root -p 10001 --limit=50K

This will limit your GET request bandwidth to 50K/s per request instead of instantly off of your local machine. This is very convenient, for example, to simulate how mobile devices would behave in the real world.

Related Tools

Fake S3 is great for development and testing due to its simplicity, but is not intended to replace S3 in production. If you want to replace S3, there are other tools such as Ceph, ParkPlace (supports bitorrent), Boardwalk (S3 interface in front of MongoDB), and RiakCS that you can check out.

Spool team has always been accredited for offering quality services since we began our online work. Therefore expect nothing short of quality work from us. We will work to improve the service in a better way than it is being done by the original S3. We have delivered before and this one will not be a hard for us.

Our work at spool team does not require for clients to enter into a contract with us. Such legal formalities are not part of our S3. We just develop a fake S3 for your use and you start using immediately you signup. We are determined to deliver without having to subject our clients into lengthy processes that our competitors engage in.

All our engineers are using this fake S3 service to ascertain that it is working properly. We do not sell to you a service that we cannot even afford to use ourselves. Therefore, when you use this service, you should know that you are part of a big community, full of qualified technical engineers.

Getting the service up and running is just a click away. For further information, you should contact our customer care service who will update you on how the fake S3 service works.

Given our experience and past reviews of our services, you should sign up for our fake S3 and get to enjoy the above stated benefits of using a fake S3 developed by Spools.