English 中文(简体)
如何根据标签获取亚马逊 S3 上的使用量?
原标题:How to retrieve usage on Amazon S3 based on tag?
We are currently implementing a single bucket for all data stored in S3, which contains data from various organizations. Can we retrieve usage data for the foo or bar organizations based on their respective foo or bar tags? Use case: We deploy a mobile app for our clients, called foo and bar . This app is used to upload files (images and videos) tagged with foo and bar based on the organization. We use the same API for the app and the same bucket in S3. I want to get metrics like the total storage used by the foo or bar tags, so I can monitor that foo or bar is using N GB of storage.
最佳回答
Tagging at the object-level for Amazon S3 was introduced in March 2017. It allows: Lifecycle Management by tag (eg move to Amazon Glacier) Access control policies It does not provide metrics based on tags at the object level. You would need to write your own script to retrieve a list of objects and calculate storage based on tag. However, it appears that the only way to retrieve tags on an object is to request for each individual object. This means you would be making large quantities of API calls. An alternative is to use object metadata against each object. Metadata is returned by the list-objects API call, so it would only require one API call per 1000 objects (which is the paging size of returned data). Finally, you could store the objects in separate buckets, which would make it possible to use Amazon CloudWatch metrics. Amazon S3 sends metrics to Amazon CloudWatch for the number of objects and the amount of storage space.
问题回答
According to the current documentation you can only use bucket tags for cost allocation reports. http://docs.aws.amazon.com/AmazonS3/latest/dev/BucketBilling.html However with CloudWatch you can use a metric configuration with an object tag filter which returns metrics for only those tagged S3 objects. It may be limited to request metrics. You will need to use the new S3 Console and instructions are here: http://docs.aws.amazon.com/AmazonS3/latest/user-guide/configure-metrics.html See http://docs.aws.amazon.com/AmazonS3/latest/dev/metrics-configurations.html Also http://docs.aws.amazon.com/AmazonS3/latest/dev/cloudwatch-monitoring.html
You could create a Lambda function that executes on upload events in s3 that can records the metadata about the object into a database like Dynamodb or mysql. And then a separate lambda function on delete events in the s3 bucket. Then you can run analytics on the database instead of trying to use a lot of API calls to s3. http://docs.aws.amazon.com/lambda/latest/dg/with-s3.html




相关问题
Mount windows shared drive to MWAA in bootscript

In MWAA startup script sudo yum install samba-client cifs-utils -y sudo mount.cifs //dev/test/drop /mnt/dev/test-o username=testuser,password= pwd ,domain=XX Executing above commonds giving error - ...

How to get Amazon Seller Central orders programmatically?

We have been manually been keying Amazon orders into our system and would like to automate it. However, I can t seem to figure out how to go about it. Their documentation is barely there. There is: ...

Using a CDN like Amazon S3 to control access to media

I want to use Amazon S3/CloudFront to store flash files. These files must be private as they will be accessed by members. This will be done by storing each file with a link to Amazon using a mysql ...

unable to connect to database on AWS

actually I have my website build with Joomla hosted on hostmonster but all Joomla website need a database support to run this database is on AWS configuration files need to be updated for that I ...

Using EC2 Load Balancing with Existing Wordpress Blog

I currently have a virtual dedicated server through Media Temple that I use to run several high traffic Wordpress blogs. Both tend to receive sudden StumbleUpon traffic surges that (I m assuming) ...

SSL slowness in EC2

We ve deployed our rails app to EC2. In our setup, we have two proxies on small instances behind round-robin DNS. These run nginx load balancers for a dynamically growing and shrinking farm of web ...

热门标签