English 中文(简体)
Should I persist images on EBS or S3?
原标题:

I am migrating my Java,Tomcat, Mysql server to AWS EC2.

I have already attached EBS volume for storing MySql data. In my web application people may upload images. So I should persist them. There are 2 alternatives in my mind:

  1. Save uploaded images to EBS volume.
  2. Use the S3 service.

The followings are my notes, please be skeptic about them, as my expertise is not on servers, but software development.

  • EBS plus: S3 storage is more expensive. (0.15 $/Gb > 0.1$/Gb)

  • S3 plus: Serving statics from EBS may influence my web server s performance negatively. Is this true? Does Serving images affect server performance notably? For S3 my server will not be responsible for serving statics.

  • S3 plus: Serving statics from EBS may result I/O cost, probably it will be minor.

  • EBS plus: People say EBS is faster.

  • S3 plus: People say S3 is more safe for persistence.

  • EBS plus: No need to learn API, it is straight forward to save the images to EBS volume.

Namely I can not decide, will be happy if you guide.

Thanks

最佳回答

I m currently using S3 for a project and it s working extremely well.

EBS means you need to manage a volume + machines to attach it to. You need to add space as it s filling up and perform backups (not saying you shouldn t back up your S3 data, just that it s not as critical).

It also makes it harder to scale: when you want to add additional machines, you either need to pull off the images to a separate machine or clone the images across all. This also means you re adding a bottleneck: you ll have to manage your own upload process that will either upload to all machines or have a single machine managing it.

I recommend S3: it s set and forget. Any number of machines can be performing uploads in parallel and you don t really need to notify other machines about the upload.

In addition, you can use Amazon Cloudfront as a cheap CDN in front of the images instead of directly downloading from S3.

问题回答

The price comparison is not quite right: S3 charges are $0.14 per GB USED, whereas EBS charges are $0.10 per GB PROVISIONED (the size of your EBS volume), whether you use it or not. As a result, S3 may or may not be cheaper than EBS.

I have architected solutions on AWS for Stock photography sites which stores millions of images spanning TB s of data, I would like to share some of the best practice in AWS for your requirement:

P1) Store the Original Image file in S3 Standard option

P2) Store the reproducible images like thumbs etc in the S3 Reduced Redundancy option (RRS) to save costs

P3) Meta data about images including the S3 URL can be stored in Amazon RDS or Amazon DynamoDB depending upon the query complexity. Query the entries from Amazon RDS. If your query is complex it is also common practice to Store the meta data in Amazon CloudSearch or Apache Solr.

P4) Deliver your thumbs to users with low latency using Amazon CloudFront.

P5) Queue your image conversion either thru SQS or RabbitMQ on Amazon EC2

P6) If you are planning to use EBS, then they are not scalable with your EC2. So ideally you can use GlusterFS as your common storage pool for all your images. Multiple Amazon EC2 in Auto Scaled mode can still connect to it and access/write images.

You already outlined the advantages and disadvantages of both.

If you are planning to store terabytes of images, with storage requirements increasing day after day, S3 will probably be your best bet as it is built especially for these kinds of situations. You get unlimited storage space, without having to worry about sharding your data over many EBS volumes.

The recurrent cost of S3 is that it comes 50% more expensive than EBS. You will also have to learn the API and implement it in your application, but that is a one-off expense which I think you should be able to absorb very quickly.

Do you expect the images to last indefinitely?

The Amazon EBS FAQ is pretty clear; the annual failure rate is not "essentially zero"; they quote 0.1% to 0.5%. It s better than the disk under your desk, but it would need some kind of backup.





相关问题
Mount windows shared drive to MWAA in bootscript

In MWAA startup script sudo yum install samba-client cifs-utils -y sudo mount.cifs //dev/test/drop /mnt/dev/test-o username=testuser,password= pwd ,domain=XX Executing above commonds giving error - ...

How to get Amazon Seller Central orders programmatically?

We have been manually been keying Amazon orders into our system and would like to automate it. However, I can t seem to figure out how to go about it. Their documentation is barely there. There is: ...

Using a CDN like Amazon S3 to control access to media

I want to use Amazon S3/CloudFront to store flash files. These files must be private as they will be accessed by members. This will be done by storing each file with a link to Amazon using a mysql ...

unable to connect to database on AWS

actually I have my website build with Joomla hosted on hostmonster but all Joomla website need a database support to run this database is on AWS configuration files need to be updated for that I ...

Using EC2 Load Balancing with Existing Wordpress Blog

I currently have a virtual dedicated server through Media Temple that I use to run several high traffic Wordpress blogs. Both tend to receive sudden StumbleUpon traffic surges that (I m assuming) ...

SSL slowness in EC2

We ve deployed our rails app to EC2. In our setup, we have two proxies on small instances behind round-robin DNS. These run nginx load balancers for a dynamically growing and shrinking farm of web ...

热门标签