It is possible to automatically remove objects from a bucket after a predefined amount of time. This is done by setting a so-called life cycle policy on the bucket. Below an example is given of how you can do this using the command line tools or in a programmatic way using the boto3 python library.


Using the command line (s3cmd)

s3cmd expire --expiry-days 1 s3://lifecycled_bucket

This will delete every object from the bucket after one day. It is also possible to get the life cycle placed on a bucket by:

s3cmd getlifecycle s3://lifecycled_bucket

which would return output like:

<?xml version="1.0" ?>
<LifecycleConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
	<Rule>
		<ID>3klaj7gumyejzfuaz0kw85fgonhvjdq50ty7uce6phl1gs0c</ID>
		<Prefix/>
		<Status>Enabled</Status>
		<Expiration>
			<Days>1</Days>
		</Expiration>
	</Rule>
</LifecycleConfiguration>

A life cycle policy can be deleted by:

s3cmd dellifecycle s3://lifecycled_bucket

Using the command line (aws cli)

aws s3api put-bucket-lifecycle --bucket lifecycled_bucket --lifecycle-configuration file://lifecycle.json

This creates a life cycle policy on the bucket. Here lifecycle.json could look like:

{
    "Rules": [
        {
            "Expiration": {
                "Days": 1
            },
            "ID": "evhl0qo7q6p4f22h8y25jwrswyu2c5ijf1jclayfopw2exaa",
            "Prefix": "",
            "Status": "Enabled"
        }
    ]
}

aws s3api get-bucket-lifecycle --bucket lifecycled_bucket

returns the life cycle policy in json format and 

aws s3api delete-bucket-lifecycle --bucket lifecycled_bucket

deletes the life cycle.


Using the boto3 library

A boto3 s3 client can be created by:

    client=boto3.get_client('s3',
                            endpoint_url,
                            region_name,
                            config=Config(s3={'addressing_style': addressing_style}),
                            aws_access_key_id,
                            aws_secret_access_key)


Setting a life cycle policy can be done by:

resp = client.put_bucket_lifecycle_configuration(
                                     Bucket=mys3bucket,
                                     LifecycleConfiguration={
                                         'Rules': [
                                             {
                                                 'Prefix': '',
                                                 'Expiration': {
                                                     'Days': 1,
                                                     },
                                                 'ID': '0',
                                                 'Status': 'Enabled',
                                                 },
                                             ]
                                         }
                                     )

Getting the life cycle policy on a bucket can be done by:

resp = client.get_bucket_lifecycle_configuration(
                                     Bucket=mys3bucket,
                                     )

Deleting a life cycle policy is done by:

   resp = client.delete_bucket_lifecycle(
                                     Bucket=mys3bucket,
                                     )


  • No labels