Testing S3 Replication Jobs Locally with LocalStack

Introduction

S3 Replication sits in a strange spot in most infrastructure codebases. Teams rely on it for disaster recovery and compliance, yet the configuration is usually written once, deployed, and verified by uploading a file and waiting for it to appear in the other bucket. If it does, nobody touches the setup again.

There are practical reasons for that. On AWS, every iteration on a replication rule costs money (cross-Region transfer and duplicate storage) and time. When replication breaks, the object’s replication status changes to FAILED, but that status alone does not explain why. You can configure S3 Event Notifications to report detailed replication failure reasons, but that requires additional setup. One common cause is an IAM role that is missing a required permission.

LocalStack for AWS 2026.06.0 added emulation for S3 Replication. In this tutorial, we’ll configure cross-Region replication, break it in a way that commonly occurs in production (by using a replication role that can read but not write), read the exact reason from the logs, fix it, check how deletions behave, and verify that the whole flow works as expected.

How S3 replication works

Replication is a bucket-level feature that copies object versions from a source bucket to one or more destination buckets. The buckets can be in different Regions or even different accounts; the mechanics remain the same. To configure replication, you create a rule with a priority, a filter, and a destination bucket.

It has two prerequisites:

1. Versioning must be enabled on both buckets.
2. The replication role needs read permissions on the source and write permissions on the destination.

Replication runs asynchronously. HeadObject reports PENDING, COMPLETED, or FAILED on the source. The destination object reports REPLICA and retains the source version ID.

Objects created before the rule are not copied. Delete markers can replicate when enabled, but deleting a specific version does not affect its replica.

LocalStack processes matching objects with a background worker. With IAM enforcement enabled, the worker assumes the replication role and evaluates its policies before copying an object.

Prerequisites

• lstk
• Docker

The commands use lstk aws, which runs the AWS CLI against LocalStack with test credentials. You can also use awslocal if you prefer.

Step 1: Start LocalStack with IAM enforcement

By default, LocalStack does not enable strict enforcement of IAM policies. It accepts a replication configuration with any role ARN and never checks it. To enable IAM enforcement, set LOCALSTACK_ENFORCE_IAM=1 in the environment.

With enforcement enabled, API calls are evaluated against IAM policies. The replication worker also assumes the configured role and checks its permissions before every copy, as S3 does.

Start LocalStack with IAM enforcement:

LOCALSTACK_ENFORCE_IAM=1 lstk start

Confirm the emulator is up:

lstk status

✔︎ LocalStack AWS Emulator is running
• Endpoint: localhost.localstack.cloud:4566
• Container: localstack-aws
• Version: 2026.6.3
• Uptime: 14s

Step 2: Create the source and destination buckets

Our scenario: a document platform stores customer records in meridian-records in us-east-1, and every object must be mirrored to meridian-records-dr in eu-central-1.

Create a working directory for the files we’ll write along the way:

mkdir -p s3-replication-demo/{policies,config}
cd s3-replication-demo

The finished directory will have this structure:

s3-replication-demo/
├── policies/
│   ├── replication-trust.json
│   ├── replication-policy-v1.json
│   └── replication-policy-v2.json
└── config/
    └── records-replication.json

Create both buckets. LocalStack emulates all Regions in a single container, so cross-Region setups need no extra plumbing:

lstk aws s3api create-bucket --bucket meridian-records

lstk aws s3api create-bucket --bucket meridian-records-dr \
  --region eu-central-1 \
  --create-bucket-configuration LocationConstraint=eu-central-1

Enable versioning on both buckets because replication requires it:

lstk aws s3api put-bucket-versioning --bucket meridian-records \
  --versioning-configuration Status=Enabled

lstk aws s3api put-bucket-versioning --bucket meridian-records-dr \
  --region eu-central-1 \
  --versioning-configuration Status=Enabled

If you skip this and apply a replication configuration anyway, the emulator rejects it with the same error AWS returns:

An error occurred (InvalidRequest) when calling the PutBucketReplication operation:
Versioning must be 'Enabled' on the bucket to apply a replication configuration

Step 3: Create the replication role

S3 assumes an IAM role to perform the copying, so the role’s trust policy has to allow the S3 service principal to assume it. Save this as policies/replication-trust.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {"Service": "s3.amazonaws.com"},
      "Action": "sts:AssumeRole"
    }
  ]
}

Now the permissions. As covered earlier, a replication role needs two halves: read access on the source and write access on the destination. Our first version grants only the read half.

This is deliberate. When the role is created through the S3 console, AWS generates the policy for you. When it is managed as code, someone writes it by hand, and forgetting the destination half (or scoping it to the wrong bucket) is a common mistake. We want to see what that looks like before we fix it.

Save this as policies/replication-policy-v1.json:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadSourceConfig",
      "Effect": "Allow",
      "Action": ["s3:GetReplicationConfiguration", "s3:ListBucket"],
      "Resource": "arn:aws:s3:::meridian-records"
    },
    {
      "Sid": "ReadSourceVersions",
      "Effect": "Allow",
      "Action": [
        "s3:GetObjectVersionForReplication",
        "s3:GetObjectVersionAcl",
        "s3:GetObjectVersionTagging"
      ],
      "Resource": "arn:aws:s3:::meridian-records/*"
    }
  ]
}

Create the role and attach the policy:

lstk aws iam create-role --role-name meridian-replication \
  --assume-role-policy-document file://policies/replication-trust.json \
  --query Role.Arn --output text

lstk aws iam put-role-policy --role-name meridian-replication \
  --policy-name replication \
  --policy-document file://policies/replication-policy-v1.json

The role ARN is used in the replication configuration:

arn:aws:iam::000000000000:role/meridian-replication

Step 4: Configure replication on the source bucket

The replication configuration ties everything together: the role, the rule, and the destination.

Save this as config/records-replication.json:

{
  "Role": "arn:aws:iam::000000000000:role/meridian-replication",
  "Rules": [
    {
      "ID": "disaster-recovery",
      "Status": "Enabled",
      "Priority": 1,
      "Filter": {"Prefix": ""},
      "DeleteMarkerReplication": {"Status": "Enabled"},
      "Destination": {
        "Bucket": "arn:aws:s3:::meridian-records-dr",
        "StorageClass": "STANDARD_IA"
      }
    }
  ]
}

A few things to note:

• Filter with an empty prefix matches every object.
• DeleteMarkerReplication copies delete markers to the destination.
• StorageClass stores replicas as STANDARD_IA.

Apply the configuration and read it back:

lstk aws s3api put-bucket-replication --bucket meridian-records \
  --replication-configuration file://config/records-replication.json

lstk aws s3api get-bucket-replication --bucket meridian-records

Step 5: Upload a document and watch it fail to replicate

Time to move some data:

printf 'INVOICE #042 - year-end audit copy\n' > invoice-042.pdf

lstk aws s3api put-object --bucket meridian-records \
  --key records/2026/invoice-042.pdf --body invoice-042.pdf

The upload succeeds and returns a version ID because replication does not determine whether the source write is accepted.

The copy is queued in the background. Wait briefly for the worker, then inspect the replication status on the source object:

lstk aws s3api head-object --bucket meridian-records \
  --key records/2026/invoice-042.pdf --query ReplicationStatus

"FAILED"

The status can briefly be PENDING before becoming FAILED. The destination remains empty:

lstk aws s3api list-objects-v2 --bucket meridian-records-dr --region eu-central-1

On AWS, this is roughly where the investigation stalls. The status says FAILED, the OperationFailedReplication metric ticks up, and neither of them names the reason. The emulator does:

lstk logs | grep -i "replication error"

ERROR --- [s3_repl_0] l.p.c.s.s.replication_basi : Replication error while replicating
Object Version (version AZ9p9FURGmFzGlkOCA8XEwhl7ZNCGFyQ): Replication failed due to IAM:
User: arn:aws:sts::000000000000:assumed-role/meridian-replication/session-4e98eb0e is not
authorized to perform: s3:ReplicateObject because no identity-based policy allows the
s3:ReplicateObject action

The worker assumed meridian-replication, tried to write the replica, and was denied permission to call s3:ReplicateObject, which is exactly the half of the policy we left out. The role can read everything it needs from the source, but it has no permissions on the destination.

Step 6: Fix the role and retry

policies/replication-policy-v2.json is the v1 policy plus the missing statement:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ReadSourceConfig",
      "Effect": "Allow",
      "Action": ["s3:GetReplicationConfiguration", "s3:ListBucket"],
      "Resource": "arn:aws:s3:::meridian-records"
    },
    {
      "Sid": "ReadSourceVersions",
      "Effect": "Allow",
      "Action": [
        "s3:GetObjectVersionForReplication",
        "s3:GetObjectVersionAcl",
        "s3:GetObjectVersionTagging"
      ],
      "Resource": "arn:aws:s3:::meridian-records/*"
    },
    {
      "Sid": "WriteReplicas",
      "Effect": "Allow",
      "Action": ["s3:ReplicateObject", "s3:ReplicateDelete", "s3:ReplicateTags"],
      "Resource": "arn:aws:s3:::meridian-records-dr/*"
    }
  ]
}

Update the role and upload the invoice again:

lstk aws iam put-role-policy --role-name meridian-replication \
  --policy-name replication \
  --policy-document file://policies/replication-policy-v2.json

lstk aws s3api put-object --bucket meridian-records \
  --key records/2026/invoice-042.pdf --body invoice-042.pdf

lstk aws s3api head-object --bucket meridian-records \
  --key records/2026/invoice-042.pdf --query ReplicationStatus

"COMPLETED"

The first version of the invoice remains FAILED because fixing the role does not automatically retry failed replication. Re-uploading creates a new version and queues a new replication attempt. On AWS, you can also use S3 Batch Replication to replicate versions that previously failed. The new version should replicate within a few seconds.

Inspect the replica:

lstk aws s3api head-object --bucket meridian-records-dr \
  --key records/2026/invoice-042.pdf --region eu-central-1

{
  "ReplicationStatus": "REPLICA",
  "StorageClass": "STANDARD_IA",
  "VersionId": "AZ9p9FUS66D8RjGXgRFRaVrmMwV7l9j9",
  "ContentLength": 35
}

Some details match what you would see on AWS:

• The destination copy reports REPLICA rather than COMPLETED.
• It landed in STANDARD_IA as the rule requested.
• Its version ID is identical to the source version’s.

That last one matters because it keeps the two buckets referring to the same version history.

Step 7: Check what happens on delete

Deletion behavior is often audited only after a replication setup is in use, so it is worth testing up front. There are two types of deletion, and they behave very differently.

A plain DeleteObject without a version ID doesn’t remove data in a versioned bucket; it writes a delete marker on top of the version stack. Because our rule enabled DeleteMarkerReplication, the marker itself replicates:

lstk aws s3api delete-object --bucket meridian-records \
  --key records/2026/invoice-042.pdf

sleep 3

lstk aws s3api list-object-versions --bucket meridian-records-dr \
  --prefix records/ --region eu-central-1

{
  "Versions": [
    {
      "VersionId": "AZ9p9FUS66D8RjGXgRFRaVrmMwV7l9j9",
      "IsLatest": false
    }
  ],
  "DeleteMarkers": [
    {
      "IsLatest": true
    }
  ]
}

The latest object is hidden in both buckets, while the data version remains available in their version histories.

Deleting a specific version permanently removes it from the source, but does not remove its replica:

printf 'meeting notes, nothing sensitive\n' > meeting-notes.txt

lstk aws s3api put-object --bucket meridian-records \
  --key notes/meeting-notes.txt --body meeting-notes.txt --query VersionId --output text

After replication completes, delete the source version using the returned version ID:

lstk aws s3api delete-object --bucket meridian-records \
  --key notes/meeting-notes.txt --version-id <version-id>

lstk aws s3api list-object-versions --bucket meridian-records-dr \
  --prefix notes/ --region eu-central-1

The source no longer contains that version, while the destination still does. AWS works the same way: deleting a specific source version does not replicate the deletion, so the replica remains in the destination.

Summary

We configured S3 cross-Region replication with versioned buckets, IAM enforcement, delete-marker replication, and a destination storage class. The tests covered successful replication, delete markers, and a role missing s3:ReplicateObject.

LocalStack also supports prefix and tag filters, multiple destinations, bidirectional replication, and CloudFormation’s ReplicationConfiguration property. See the release notes and S3 service documentation for supported features and limitations.