S3 Tables: Should You Migrate Your Data Lake?

When AWS announced S3 Tables at re:Invent 2024, several of our clients running self-managed Apache Iceberg on S3 immediately asked: “should we migrate?”

The honest answer, for most of them, was “probably yes, but not yet, and only if you understand what you’re trading.”

This post walks through the decision.

What S3 Tables actually is

S3 Tables is a new bucket type, distinct from general-purpose S3 buckets, with first-class support for storing Iceberg tables. AWS manages:

• The Iceberg catalog (no more Glue Data Catalog dance for tables, though Glue can integrate)
• Compaction (small file merging that you’d otherwise schedule yourself)
• Snapshot expiration and unreferenced file cleanup
• Performance optimization tuned for Iceberg workloads

You still own the schema, the table design, and what queries you point at it. AWS owns the operational toil that makes Iceberg actually pleasant in production.

Pricing has a per-table-per-month component plus storage, requests, and compaction throughput. For a moderately active data lake, the math depends heavily on how much compaction you’d otherwise be running.

Who should migrate

You’re a strong candidate for S3 Tables if:

• You’re already running Iceberg on S3 with hand-rolled or Spark-based compaction.
• You’ve had performance regressions or cost spikes from small files accumulating.
• Your team’s bandwidth for data platform work is the bottleneck, not raw infrastructure cost.

In these cases, S3 Tables removes work that wasn’t really differentiating for your team.

Who shouldn’t migrate (yet)

Skip the migration if:

• Your data lake is happily served by Glue tables in Parquet without table-format features.
• Your queries are simple enough that you don’t actually need Iceberg’s time travel, schema evolution, or row-level operations.
• You have tightly coupled custom tooling around your existing Iceberg catalog.

In these cases, you’re either paying for capabilities you don’t need or paying migration cost without operational relief.

Migration considerations

Three things tend to surprise teams during planning:

Engine support is uneven. Athena and EMR work natively. Other engines (Snowflake, Databricks, Trino-on-EC2) require integration work that’s still maturing. Confirm your engine before committing.

Catalog migration is the actual work. The data layout doesn’t change much; the catalog references and IAM do. Plan for a phased cutover where new writes go to S3 Tables while reads happen from both.

IAM is different. S3 Tables uses a distinct resource model and policy structure from general-purpose buckets. Existing bucket policies don’t translate directly.

How we’d approach it

For a client moving today, we’d recommend:

1. Identify one self-managed Iceberg table that’s been a maintenance burden, probably one with high write volume and accumulating small files.
2. Create a parallel S3 Tables table and dual-write for a couple of weeks.
3. Cut reads over once validation passes; retire the old table.
4. Repeat for the next table; build a pattern.

Don’t try a big-bang migration. The operational risk isn’t worth the calendar time saved.

Closing

S3 Tables is genuinely useful for teams already in the Iceberg world. If you’re in that boat and your platform engineers are spending real time on compaction, it’s worth a serious look. If you’re using Glue tables in Parquet and you’re happy, the cost-benefit is harder to justify.

Either way, the broader trend matters: AWS is steadily moving differentiated operational work into managed services. The right question for any data team is which pieces of your platform are still worth running yourself.