If you use s3cmd and encountered [Errno 32] Broken pipe as you try to put an object in a bucket, understand that this is a very very bad error message.

Jeremy blogged about this and attributed the error to a typo in the bucket name.

Others attributed it to no permission, file too big, etc..

I attributed it to incorrect permission policy.

I was trying to create a new security group, and adding a new user, and restrict his access to only 1 of my S3 bucket. When you create a new security group, you can edit the policy.

You might change to this, thinking it allows all action on example_bucket:

1
2
3
4
5
6
7

{
  "Statement":[{
     "Effect":"Allow",
     "Action":"*",
     "Resource":"arn:aws:s3:::example_bucket"
   }]
}

You are wrong (though I say Amazon and it’s documentation to blame).

The correct way is to have multiple statements like this:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation",
        "s3:ListBucketMultipartUploads"
      ],
      "Resource": "arn:aws:s3:::example_bucket",
      "Condition": {}
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:AbortMultipartUpload",
        "s3:DeleteObject",
        "s3:DeleteObjectVersion",
        "s3:GetObject",
        "s3:GetObjectAcl",
        "s3:GetObjectVersion",
        "s3:GetObjectVersionAcl",
        "s3:PutObject",
        "s3:PutObjectAcl",
        "s3:PutObjectAclVersion"
      ],
      "Resource": "arn:aws:s3:::example_bucket/*",
      "Condition": {}
    }
  ]
}

You need to split into 2 resources.

1. arn:aws:s3:::example_bucket – allow to list objects in the bucket

2. arn:aws:s3:::example_bucket/* – allow to Get/Put/etc the objects in the bucket

It’s subtle..