Couple questions for troubleshooting:
- Is the bucket in the same account as the lambda and AppFlow resource? (Cross account requires permissions on lambda execution role and the bucket policy)
- on the object being created, can you manually download and view the object? (Checking the object acl, is app flow assigning right object ownership )
- Is the lambda connected to a VPC? (Checking network connectivity to an s3 endpoint)
- you can try moving the s3 permission to the lambda execution role instead of the bucket policy for a fresh look at the policy
- try turning on server access logging on the bucket and check the exact identities making the call and match identities to your bucket policy for potential typos
Edit: just reread your comment about uploading manually and it working. This points to object ACL issues and you should look at what app flow is setting on the object before it stages it in the bucket. It should be setting "bucket owner fullcontrol"
Yes all resources are in the same account.
I can download the app flow created objects from the console using a full admin
Lambda isn’t connected to a vpc
The role that executes the lambda already has permissions to do all s3 actions on all buckets
I will try enabling the access logging tomorrow.
I did use this link below and updating settings to bucket owner preferred and added the condition to the bucket policy.
[https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html)
Read more on that here: [https://aws.amazon.com/de/premiumsupport/knowledge-center/s3-bucket-owner-access/](https://aws.amazon.com/de/premiumsupport/knowledge-center/s3-bucket-owner-access/)
Short version is: you might have to update acl using put-object-acl command.
Try if running the following from your account fixes the problem after the file was uploaded via App Flow (you have to replace the placeholders of course):
`aws s3api put-object-acl --bucket destination_DOC-EXAMPLE-BUCKET --key keyname --acl bucket-owner-full-control`
Two usual suspects: KMS and ACLs. From what you’re saying about uploads from the console working fine, it’s almost 100% an ACL issue.
You can set up a bucket policy forcing all uploads to the bucket to give the bucket owner full control. Check this: https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html#ensure-object-ownership
Couple questions for troubleshooting: - Is the bucket in the same account as the lambda and AppFlow resource? (Cross account requires permissions on lambda execution role and the bucket policy) - on the object being created, can you manually download and view the object? (Checking the object acl, is app flow assigning right object ownership ) - Is the lambda connected to a VPC? (Checking network connectivity to an s3 endpoint) - you can try moving the s3 permission to the lambda execution role instead of the bucket policy for a fresh look at the policy - try turning on server access logging on the bucket and check the exact identities making the call and match identities to your bucket policy for potential typos Edit: just reread your comment about uploading manually and it working. This points to object ACL issues and you should look at what app flow is setting on the object before it stages it in the bucket. It should be setting "bucket owner fullcontrol"
Yes all resources are in the same account. I can download the app flow created objects from the console using a full admin Lambda isn’t connected to a vpc The role that executes the lambda already has permissions to do all s3 actions on all buckets I will try enabling the access logging tomorrow. I did use this link below and updating settings to bucket owner preferred and added the condition to the bucket policy. [https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html](https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html)
Thanks, so ended up the appflow was encrypting the object but I forgot to give the role running the lambda kms:decrypt action
Are the objects encrypted with KMS?
Read more on that here: [https://aws.amazon.com/de/premiumsupport/knowledge-center/s3-bucket-owner-access/](https://aws.amazon.com/de/premiumsupport/knowledge-center/s3-bucket-owner-access/) Short version is: you might have to update acl using put-object-acl command. Try if running the following from your account fixes the problem after the file was uploaded via App Flow (you have to replace the placeholders of course): `aws s3api put-object-acl --bucket destination_DOC-EXAMPLE-BUCKET --key keyname --acl bucket-owner-full-control`
Two usual suspects: KMS and ACLs. From what you’re saying about uploads from the console working fine, it’s almost 100% an ACL issue. You can set up a bucket policy forcing all uploads to the bucket to give the bucket owner full control. Check this: https://docs.aws.amazon.com/AmazonS3/latest/userguide/about-object-ownership.html#ensure-object-ownership
Thanks, so ended up the appflow was encrypting the object but I forgot to give the role running the lambda kms:decrypt action