If you want to delete a large number of object from a given bucket, there is a bulk delete option available. You need to know the bucket name and the names of the objects you want to delete. You can delete up to 10000 objects at a time.

Bulk delete using the S3 protocol

This example uses the aws-cli tool. Ensure you can run the aws s3  and aws s3api commands properly with your credentials. 

You need to create a json file listing the objects, without the bucket name:

{
    "Objects": [
        {"Key": "delete-object-1"},
        {"Key": "delete-object-2"},
        {"Key": "delete-object-3"}
    ]
}

Save this as for instance 'bulk_delete.json'

Assuming you want to delete these objects from a bucket named 'bulk':

aws s3api delete-objects --bucket bulk --delete file://bulk_delete.json

The response:

{
    "Deleted": [
        {
            "Key": "delete-object-1"
        },
        {
            "Key": "delete-object-2"
        },
        {
            "Key": "delete-object-3"
        }
    ]
}

Note: If you enter non-existing keys, the response will still seem to indicate deletion has taken place, while in reality nothing happens.

Bulk delete using the Swift protocol

This example uses the python-swiftclient and cURL. Ensure you can access the service with for instance swift stat .

You need to create a text file with each object listed as bucket/object:

bulk/delete-object-1
bulk/delete-object-2
bulk/delete-object-3

Save the file as for instance bulk_delete.txt

Retrieve a token using swift auth :

$ swift auth
export OS_STORAGE_URL=https://objectstore.surf.nl/swift/v1/AUTH_<project_id>
export OS_AUTH_TOKEN=<token>

Export these variables into your environment.

Delete the objects using cURL:

$ curl -i -X DELETE -H "X-Auth-Token: $OS_AUTH_TOKEN" --data-binary @bulk_delete.txt "$OS_STORAGE_URL?bulk-delete"
...
Number Deleted: 3
Number Not Found: 0
Response Body: 
Response Status: 200 OK

Note: In this case the statistics returned do reflect what happened.

  • No labels