This page requires JavaScript to display.
R
Unpacking...
← Cloud Platforms
Cloud Platforms

Amazon RDS & Aurora for Data Engineers

Engines, Multi-AZ and replicas, backups, Aurora architecture, security, extracting data for the lake (JDBC, DMS, snapshot export), CDC prerequisites and operations.

Contents
1. Amazon RDS Overview 2. High Availability, Replicas & Backups 3. Aurora Architecture 4. Security 5. Extracting Data for the Lake 6. Operations & Choosing a Database 7. Common Interview Questions

1. Amazon RDS Overview

Amazon RDS runs managed relational databases: AWS handles provisioning, patching, backups, failover and monitoring, while you manage schema, queries and access. For data engineers, RDS databases are usually sources (operational data to extract or replicate) and sometimes targets (serving tables, metadata, small marts).

EngineNotes
PostgreSQLMost common open-source choice; rich SQL, JSONB, extensions (pgvector, PostGIS)
MySQL / MariaDBPopular web application databases
Oracle / SQL Server / Db2Commercial engines (license included or bring your own)
Aurora MySQL / Aurora PostgreSQLAWS-built, compatible engines with distributed storage (next section)

2. High Availability, Replicas & Backups

FeatureWhat it gives you
Multi-AZ (instance)A synchronous standby in another AZ with automatic failover (about 1–2 minutes); the standby does not serve reads
Multi-AZ DB clusterOne writer plus two readable standbys in different AZs; faster failover
Read replicasAsynchronous copies (same or other regions) to offload reads, e.g. point extraction jobs at a replica
Automated backupsDaily snapshots plus transaction logs: point-in-time restore to any second within the retention period (up to 35 days)
Manual snapshotsKept until deleted; can be copied across regions and accounts

A restore always creates a new DB instance with a new endpoint; applications must switch to it.

3. Aurora Architecture

              writer instance            reader instances (up to 15)
                    │                          │
   cluster endpoint ┘        reader endpoint ──┘   (load-balanced across readers)
                    ▼                          ▼
   ┌───────────── shared cluster volume: 6 copies across 3 AZs, auto-grows ──────────────┐
FeatureBenefit
Shared distributed storageSix copies across three AZs; survives losing an AZ; replicas share storage so replica lag is typically milliseconds
Aurora Serverless v2Capacity scales in fine-grained units with load; good for variable or dev workloads
Global DatabaseCross-region replication with typically under a second of lag; fast regional failover
Fast cloningCopy-on-write clone of a production database in minutes, for testing or analytics
I/O-OptimizedPricing option with no per-I/O charges for I/O-heavy workloads
Zero-ETL to RedshiftContinuous replication into Redshift for analytics without pipelines

4. Security

5. Extracting Data for the Lake

ApproachWhen
Glue / Spark JDBC with parallel readsBatch extracts of tables, ideally from a read replica
AWS DMS (full load + CDC)Continuous change capture with deletes, low source impact
Zero-ETL to RedshiftAnalytics in Redshift without building pipelines
Snapshot export to S3Export a snapshot as Parquet for bulk historical loads, with no load on the live database
# export a snapshot to S3 as Parquet (runs on AWS's side, not the live database)
aws rds start-export-task \
  --export-task-identifier orders-export-2026-10-26 \
  --source-arn arn:aws:rds:ap-south-1:111122223333:snapshot:orders-db-2026-10-26 \
  --s3-bucket-name acme-lake-prod --s3-prefix raw/rds-exports/orders \
  --iam-role-arn arn:aws:iam::111122223333:role/rds-s3-export \
  --kms-key-id arn:aws:kms:ap-south-1:111122223333:key/1111-2222
# incremental extract with a watermark, from a read replica
import psycopg2, boto3, json

secret = json.loads(boto3.client("secretsmanager").get_secret_value(SecretId="orders-db/reader")["SecretString"])
with psycopg2.connect(host=secret["host"], dbname="orders", user=secret["username"],
                      password=secret["password"], sslmode="require") as conn:
    with conn.cursor(name="extract") as cur:  # server-side cursor streams rows
        cur.itersize = 50_000
        cur.execute("SELECT * FROM public.orders WHERE updated_at > %s AND updated_at <= %s",
                    (last_watermark, new_watermark))
        for row in cur:
            write_row(row)
CDC prerequisiteSetting
PostgreSQLrds.logical_replication = 1 in the parameter group (enables logical decoding)
MySQL / MariaDBAutomated backups on, binlog_format = ROW, and a binlog retention period long enough for the CDC tool
SQL Server / OracleMS-CDC enabled on tables / supplemental logging and archive logs

6. Operations & Choosing a Database

NeedChoose
Standard relational OLTP, commercial engineRDS (Oracle / SQL Server)
High availability and read scale for MySQL/PostgreSQLAurora
Spiky or unpredictable relational loadAurora Serverless v2
Key-value at massive scale, single-digit msDynamoDB
Analytics over large dataRedshift / Athena, not RDS

7. Common Interview Questions

QuestionShort answer
Multi-AZ vs read replica?Multi-AZ is for availability (synchronous standby, automatic failover, no reads on the classic standby); read replicas are asynchronous copies for scaling reads and cross-region DR
How is Aurora different from RDS MySQL/PostgreSQL?Distributed storage with six copies across three AZs, up to 15 low-lag replicas, faster failover, Serverless v2, Global Database and cloning
How do you extract from a production database without hurting it?Read from a replica, use parallel bounded reads off-peak, or log-based CDC (DMS) or snapshot export
How do you encrypt an existing unencrypted RDS database?Snapshot it, copy the snapshot with encryption enabled, and restore from the encrypted copy
Why use RDS Proxy with Lambda?It pools and reuses connections so thousands of concurrent functions do not exhaust database connections