How can I copy S3 objects from another AWS account?

Ref: https://aws.amazon.com/premiumsupport/knowledge-center/copy-s3-objects-account/

Well, you can do it manually following the notes in the AWS KB article above linked. I am always confused by those, and I have to read them a few times before I’m able to apply the steps correctly. Recently however I’ve been going the other way around: whenever there are steps to follow, I try to translate them into an Ansible playbook

Here’s the playbook for syncing files between S3 buckets in different accounts:

---
# copy-bucket.yml
- hosts: localhost
  vars:
    source_bucket: my-source-bucket  
    dest_bucket: my-dest-bucket
    dest_user_arn: arn:aws:iam::ACCOUNTID:user/USERNAME
    dest_user_name: USERNAME
    source_profile: src_profile
    dest_profile: dest_profile
  tasks:
    - name: Attach bucket policy to source bucket
      s3_bucket:
        name: "{{ source_bucket }}"
        policy: "{{ lookup('template','bucket-policy.json.j2') }}"
        profile: "{{ source_profile }}"

    - name: Attach an IAM policy to a user in dest account
      iam_policy:
        iam_type: user
        iam_name: "{{ dest_user_name }}"
        policy_name: "s3_move_access"
        state: present
        policy_json: "{{ lookup( 'template', 'user-policy.json.j2') }}"
        profile: "{{ dest_profile }}"
      register: user_policy

    - name: Sync the files 
      shell: aws s3 sync s3://{{ source_bucket }}/ s3://{{ dest_bucket }}/ --profile {{ dest_profile }}

You will also need the following json templates

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:GetObject"
            ],
            "Resource": [
                "arn:aws:s3:::{{ source_bucket }}",
                "arn:aws:s3:::{{ source_bucket }}/*"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::{{ dest_bucket }}",
                "arn:aws:s3:::{{ dest_bucket }}/*"
            ]
        }
    ]
}
{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "DelegateS3Access",
            "Effect": "Allow",
            "Principal": {"AWS": "{{ dest_user_arn }}"},
            "Action": ["s3:ListBucket","s3:GetObject"],
            "Resource": [
                "arn:aws:s3:::{{ source_bucket }}/*",
                "arn:aws:s3:::{{ source_bucket }}"
            ]
        }
    ]
}

run: ansible-playbook -v copy-bucket.yml

make sure the profile names are setup on your machine correctly, and the IAM user is there.

Similar Posts:




No Comments


You can leave the first : )



Leave a Reply

Your email address will not be published. Required fields are marked *