Use S3 Storage from Amazon EC2 Instance

To mount S3 storage to EC2 instance, you can use either s3fs or s3backer, Both of which use FUSE.

S3fs is FUSE-based file system backed by Amazon S3. Mount a bucket as a local file system read/write. Store files/folders natively and transparently.

• Files stored using this system will show up in s3 bucket, you can manage/download these files through web interface;
• If the bucket is mounted onto multiple instances, any change in one instance will show up in all the instances;
• This system uses local cache to boost performance and save bandwidth. Whenever you read/write files, this system will download the whole file from s3 bucket and work on the local copy. If you set use_cache, the cache will be stored there, otherwise it will be discard. This could be a problem! Since you can not set a cache limit, if you do not clean the cache, it will occupy lots of disk space. If you have a limited space, and you are working with a large file, larger than your disk space left, even if you do not use local cache, you will get a out of space error.

S3backer  is a filesystem that contains a single file backed by the Amazon Simple Storage Service (Amazon S3). As a filesystem, it is very simple: it provides a single normal file having a fixed size. Underneath, the file is divided up into blocks, and the content of each block is stored in a unique Amazon S3 object.

• This system is efficient especially for random read/write since it stores files as blocks in s3;
• You can not see the real files in s3, this is good for security and bad for web management;
• If the bucket is mounted onto multiple instances, changes from one instance will not apply to the others automatically. You have to un-mount and re-mount the system (is there a way to refresh?);
• You can end up lots lots of blocks in s3, it is important to set up proper blockSize. Most systems' default block size is 4k, and this is definitely not suitable for s3backer. 1M was recommended by the developer;
• It is not easy to delete bucket from web, you have to do it with s3backer.

None is perfect, and nothing can be perfect for every job. You have to choose wisely depends on your task.