Running an RDS Database in AWS - Four Ways to Authenticate Like a Pro

If you’ve ever built a cloud-native application, you know the struggle: testing database interactions without racking up a hefty AWS bill or waiting forever for cloud resources to spin up.

Setting up a full-fledged AWS RDS instance just to debug your Lambda function? That’s not ideal—not only because of the cost and hassle, but also due to potential latency issues and the risk of over-provisioning resources that far exceed the needs of simple debugging tasks. That’s like renting an entire dog park just to walk one puppy. Not ideal and the hassle might outweigh the benefits.

This is where LocalStack swoops in like a superhero, offering a fully local AWS-like environment, including RDS support. Instead of relying on a remote cloud database, you can spin up an RDS instance locally, connect it to your Lambda functions, and debug everything in minutes instead of hours. This means:

No costly AWS infrastructure just for testing.
Near-instant database setup and teardown.
Full compatibility with AWS SDKs and tools.

The AWS documentation is so extensive when it comes to any service configuration, that it’s easy to get lost in the sea of options. In this post, I’d like to dive into running an RDS database using AWS (and LocalStack) and connecting to it in four different ways:

Direct user and password authentication
IAM authentication token
Using Secrets Manager with an SDK client
Using a username and password from a Secrets Manager secret

To make it fun, let’s imagine we run a dog shelter. Dogs come in, they get adopted, and we need to track them in a database. Our RDS instance will store dog details, and we have four Java-based Lambda functions for managing them:

AddDogHandler (adds a dog’s info)
GetDogHandler (fetches a dog’s info)
DeleteDogHandler (removes an adopted dog)
UpdateDogHandler (updates dog details)

To follow along, with the full setup you can find the code in this GitHub sample repository.

Setting Up LocalStack with RDS

To get a clear understanding of what the AWS stack looks like, here’s a quick overview of the components we’ll be using:

RDS Database: A PostgreSQL database to store dog details.
Secrets Manager: To store database credentials securely.
Lambda Functions: To interact with the database.
API Gateway: To trigger Lambda functions.

LocalStack RDS Setup

Before we start authenticating, let’s ensure we have LocalStack running with RDS. This Terraform configuration sets up a complete serverless API infrastructure in AWS, centered around a PostgreSQL RDS database with dual access patterns through RDS Proxy - one using traditional username/password authentication and another using IAM authentication. The infrastructure includes a VPC with two private subnets, an RDS PostgreSQL cluster, two RDS Proxies (one for each authentication method), four Java Lambda functions for CRUD operations, and an HTTP API Gateway to expose these functions. Security groups are configured to allow Lambda-to-RDS Proxy communication within the VPC. An initialization Lambda function is also included to handle database setup tasks, but not featured in the diagram. To set up your LocalStack environment, please follow the instructions in the GitHub repository.

1. Networking and Security Groups

We define a VPC and private subnets to ensure our RDS and Lambda functions communicate securely.

When reviewing the code, pay attention to the RDS security group because it’s important to understand the relationship between RDS, RDS Proxy, and Lambdas Lambda functions can scale massively and unpredictably, which creates a challenge when connecting to RDS databases. Each Lambda function invocation creates a new database connection, and when dealing with thousands of concurrent executions, this can overwhelm the database with too many open connections. RDS Proxy acts as a connection pooler between AWS Lambda and RDS. Instead of each Lambda function establishing a direct connection to RDS, they connect to RDS Proxy, which manages database connections efficiently by:

Reducing connection churn: By reusing existing database connections, RDS Proxy minimizes the overhead of creating new connections.
Enhancing security: It integrates with IAM authentication, ensuring secure access to the database without exposing credentials in the Lambda function.
Improving scalability: The proxy can handle thousands of connections efficiently, making Lambda-RDS interaction more reliable.

# VPC
resource "aws_vpc" "dog_vpc" {
  cidr_block = "10.0.0.0/16"

  tags = {
    Name = "dog-vpc"
  }
}

# Private subnet 1 (for RDS, RDS Proxy, Lambda)
resource "aws_subnet" "private_subnet_1" {
  vpc_id                  = aws_vpc.dog_vpc.id
  cidr_block              = "10.0.2.0/24"
  availability_zone       = "us-east-1b"

  tags = {
    Name = "private-subnet-1"
  }
}

# Private subnet 2 (for RDS, RDS Proxy, Lambda in another AZ)
resource "aws_subnet" "private_subnet_2" {
  vpc_id                  = aws_vpc.dog_vpc.id
  cidr_block              = "10.0.3.0/24"
  availability_zone       = "us-east-1a"

  tags = {
    Name = "private-subnet-2"
  }
}

# RDS Subnet Group (to allow RDS to use private subnets in multiple AZs)
resource "aws_db_subnet_group" "rds_subnet_group" {
  name       = "rds-subnet-group"
  subnet_ids = [aws_subnet.private_subnet_1.id, aws_subnet.private_subnet_2.id]

  tags = {
    Name = "RDS Subnet Group"
  }
}

# Security group for RDS Proxy
resource "aws_security_group" "rds_proxy_sg" {
  vpc_id = aws_vpc.dog_vpc.id

  ingress {
    from_port       = 5432
    to_port         = 5432
    protocol        = "tcp"
    security_groups = [aws_security_group.lambda_sg.id]
    cidr_blocks = ["10.0.2.0/24", "10.0.3.0/24"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "rds-proxy-sg"
  }
}

# Security group for Lambda (allows access to RDS Proxy)
resource "aws_security_group" "lambda_sg" {
  vpc_id = aws_vpc.dog_vpc.id

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }

  tags = {
    Name = "lambda-sg"
  }
}

For more details, check out the official AWS blog on Using Amazon RDS Proxy with AWS Lambda. While these scenarios are more relevant for production workloads, it’s good to understand the benefits of RDS Proxy and how it can enhance your serverless applications. Additionally it’s a good idea to plan ahead and implement best practices from the start.

2. Setting Up RDS and Secrets Manager

We configure an RDS PostgreSQL cluster for our dog shelter and store credentials securely in AWS Secrets Manager. All the secrets are in the main.tf file.

# Store database credentials securely in AWS Secrets Manager
resource "aws_secretsmanager_secret" "super_secret" {
name = "super-secret"

tags = {
Name = "super-secret"
}
}

# Store the actual secret value
resource "aws_secretsmanager_secret_version" "db_secret_value" {
secret_id     = aws_secretsmanager_secret.super_secret.id
secret_string = jsonencode({
username = var.db_username
password = var.db_password
})
}

3. Deploying Lambda Functions with RDS Connection

Each Lambda function is configured to connect to the RDS instance using the appropriate authentication method. This means some environment variables are set to pass the necessary information to the Lambda function. This is what one of our Lambdas looks like, with all the necessary configurations:

resource "aws_lambda_function" "post_dog" {
  function_name = "post-dog"
  runtime       = "java21"
  role          = aws_iam_role.lambda_role.arn
  handler       = "cloud.localstack.postdog.AddDogHandler"
  filename      = "../api-lambdas/post-dog-lambda/target/post-dog-lambda-1.0.0.jar"
  timeout       = 15
  memory_size   = 512

  vpc_config {
    subnet_ids         = [
      aws_subnet.private_subnet_1.id,
      aws_subnet.private_subnet_2.id,
    ]
    security_group_ids = [aws_security_group.lambda_sg.id]
  }

  environment {
    variables = {
      HOST          = aws_db_proxy.dogdb_secret_proxy.endpoint
      DATABASE_NAME = var.db_name
      DB_USER     = var.db_username
      USER_PASSWORD = var.db_password
    }
  }
}

With this setup, we’re ready to authenticate in four different ways!

1. Authenticating with Direct User & Password

Some things in life are simple: a dog wags its tail, and direct authentication just works. Here’s how our AddDogHandler authenticates using username and password.

Most of the database connection code is abstracted awsay in a DatabaseUtil class, making it easy to switch between different authentication methods.

connection = DatabaseUtil.getConnectionWithUserPassword(HOST, DATABASE_NAME, DB_USER, USER_PASSWORD);
[...]

String query = "INSERT INTO dogs (name, age, category) VALUES (?, ?, ?)";
PreparedStatement stmt = connection.prepareStatement(query);
                stmt.setString(1, name);
                stmt.setDouble(2, age);
                stmt.setString(3, category);

int rowsAffected = stmt.executeUpdate();

                stmt.close();

                System.out.println("Dog added successfully! Rows affected: " + rowsAffected);

This method hides the complex part, which takes a few extra steps to set up the connection:

public static Connection getConnectionWithUserPassword(String host, String databaseName, String username, String password) {
    String jdbcUrl = String.format("jdbc:postgresql://%s/%s",
            host, databaseName);

    try {
        return DriverManager.getConnection(jdbcUrl, username, password);
    } catch (SQLException e) {
        e.printStackTrace();
        throw new RuntimeException("Database connection failed using username/password", e);
    }
}

That’s it, simple and straightforward. The Lambda function uses environment variables to pass the database host, name, username, and password, and this method will handle the connection. Is it the most secure method? Not really, but it’s a good starting point for testing and development. Just don’t go to production with it.

How It Works: Establishing the Database Connection & Executing Queries

The getConnectionWithUserPassword method uses the credentials to construct a JDBC URL. The application connects to the RDS Proxy endpoint (host) using regular database credentials. These credentials are actually stored in AWS Secrets Manager, and the RDS Proxy has an IAM role that allows it to retrieve these credentials. When the Lambda attempts to connect, the proxy retrieves the credentials from Secrets Manager, validates them, and then uses them to establish and manage a connection to the actual RDS database. The proxy maintains a pool of these connections, which means it can reuse existing connections instead of creating new ones for each request, significantly reducing the connection management overhead on the database. From your code’s perspective, we’re just connecting to a database using standard connection parameters, but behind the scenes, the proxy is handling all the connection pooling, credential management, and actual database connections. The Proxy should have the following configuration:

auth {
  description = "RDS Proxy Secret Auth"
  iam_auth    = "DISABLED"
  auth_scheme = "SECRETS"
  secret_arn  = aws_secretsmanager_secret.super_secret.arn
}

Once the connection is established, AddDogHandler can execute SQL queries. The function adds a new dog entry to the database.

2. Authenticating with IAM Auth Token

Now, let’s say we want an extra security layer. Instead of storing credentials, we generate an IAM auth token dynamically.

Analogy: This is like a dog park that requires an entry pass, that’s valid for 15 minutes. Only valid passes (IAM tokens) let you in, keeping the park safe from, well… unauthorized cats.

As before, the DatabaseUtil class abstracts the connection logic, making it easy to switch between different authentication methods.

connection = DatabaseUtil.getConnectionWithIamAuth(DB_USER, HOST, PORT, REGION, DATABASE_NAME);

[...]

String updateQuery = "UPDATE dogs SET name = ?, age = ?, category = ? WHERE id = ?";
PreparedStatement updateStmt = connection.prepareStatement(updateQuery);
                updateStmt.setString(1, name);
                updateStmt.setDouble(2, age);
                updateStmt.setString(3, category);
                updateStmt.setInt(4, id);

int rowsAffected = updateStmt.executeUpdate();
                System.out.println("Update successful! Rows affected: " + rowsAffected);

This will translate, behind the scenes, to the following code:

public static Connection getConnectionWithIamAuth(String username, String host, Integer port, String region, String databaseName) {

    String authToken = generateAuthToken(host, port, region, username);
    String jdbcUrl = String.format("jdbc:postgresql://%s/%s", host, databaseName);

    try {
        return DriverManager.getConnection(jdbcUrl, username, authToken);
    } catch (SQLException e) {
        e.printStackTrace();
        throw new RuntimeException("Database connection failed using IAM authentication", e);
    }
}

private static String generateAuthToken(String hostName, Integer port, String region,String username) {
    RdsUtilities utilities = RdsUtilities.builder()
            .credentialsProvider(DefaultCredentialsProvider.create())
            .region(Region.of(region))
            .build();

    GenerateAuthenticationTokenRequest authTokenRequest = GenerateAuthenticationTokenRequest.builder()
            .username(username)
            .hostname(hostName)
            .port(port)
            .build();

    return utilities.generateAuthenticationToken(authTokenRequest);
    }

How It Works: Generate an IAM Authentication Token & Establish Connection:

When using IAM authentication, instead of sending a password, the Lambda function first calls the RDS Proxy to generate a special authentication token (this is why the Lambda needs the rds:GenerateDbAuthToken permission). This token is temporary and valid for 15 minutes. The app includes this token in place of a password when it connects to the RDS Proxy endpoint. When the proxy receives this connection attempt, it validates the token with IAM to ensure the Lambda has permission to connect (this is why the Lambda needs the rds-db:connect permission). If the token is valid, the proxy then uses its own set of credentials (stored in Secrets Manager - that is iam_secret) to establish the actual connection to the RDS database.

This creates a secure chain of trust: AWS validates the Lambda’s identity through IAM, and the proxy manages the actual database connections. In the Terraform configuration, this is implemented through the dogdb_iam_proxy resource, which has iam_auth = REQUIRED and still uses a secret from Secrets Manager for the proxy’s own connection to the database. What makes this different from username/password auth is that our Lambda never sees or handles the actual database credentials - it proves its identity through AWS’s IAM service instead. The cluster configuration must have iam_database_authentication_enabled = true to allow IAM authentication, and the Proxy must have iam_auth = REQUIRED to enforce IAM authentication for all connections.

Once the connection is established, the Lambda can execute SQL queries. In this case, the PutDogHandler function updates a dog in the database, with details like name, age, and category.

3. Authenticating with Secrets Manager & SDK

Let’s get a little fancy. We don’t want to explicitly fetch the credentials from Secrets Manager. Instead, we use the RDS Data Client to fetch them dynamically, using only the secret’s ARN. This way, we can keep our code clean and secure, without worrying about managing credentials.

String dogId = pathParameters.get("id").toString();

String sql = "SELECT id, name, age, category FROM dogs WHERE id = :id";

ExecuteStatementRequest request = ExecuteStatementRequest.builder()
        .database(DATABASE_NAME)
        .resourceArn(DB_CLUSTER_ARN)
        .secretArn(SECRET_ARN)
        .sql(sql)
        .parameters(
                SqlParameter.builder().name("id").value(Field.builder().stringValue(dogId).build()).build()
        )
        .build();

ExecuteStatementResponse response = rdsDataClient.executeStatement(request);

            if (response.records().isEmpty()) {
        return Map.of("statusCode", 404, "body", "Dog not found");
            }

How It Works: The RDS Data API & Processing the Query Response

The RDS Data API provides a serverless way to interact with Aurora databases without managing direct connections. When using the Data API through the AWS SDK for Java, the Lambda function doesn’t establish a traditional database connection at all. Instead, it makes HTTPS calls to AWS’s Data API service.

Here’s the flow:

Our Lambda function creates an ExecuteStatementRequest object that specifies the database cluster ARN, secret ARN (for credentials), database name, and our SQL statement with parameters.
When executeStatement() is called, the AWS SDK transforms this into an HTTPS request to the Data API service. This is why in the LocalStack web application, we can see a new item appear:

LocalStack RDS Data API

AWS then handles everything about the actual database connection - it establishes a secure connection to the database using the credentials from Secrets Manager. The Data API manages connection pooling and authentication behind the scenes. This approach is particularly well-suited for serverless architectures because the Lambda function doesn’t need to handle connection management or deal with connection pools - it just makes stateless API calls.

In our Terraform configuration, this is why the Lambda role needs the rds-data:ExecuteStatement permission, and why we’re providing the DB_CLUSTER_ARN and SECRET_ARN as environment variables for the Lambda functions. This is fundamentally different from both username/password and IAM authentication methods because the code never directly connects to the database - it’s all handled through AWS’s API interface.

If no records are found, returns an HTTP 404 response with “Dog not found”.

Easy, right? This ensures secure and efficient data retrieval while preventing SQL injection attacks.

However, every query runs over HTTP, which has higher latency than a persistent TCP connection, and it does not support connection pooling (every request starts a new session). This was once a good option for Aurora Serverless v1, but is no longer a viable solution for Aurora Serverless v2 or provisioned RDS clusters, which support JDBC and RDS Proxy connections.

4. Authenticating User & Password from Secrets Manager

Lastly, this method is mostly direct authentication with extra steps: we fetch the credentials from Secrets Manager. Here’s how our DeleteDogHandler authenticates using username and password.

Most of the database connection code is abstracted awsay in a DatabaseUtil class, making it easy to switch between different authentication methods.

connection = DatabaseUtil.getConnectionWithUserPassword(REGION, SECRET_ARN, HOST, DATABASE_NAME);
[...]

String query = "DELETE FROM dogs WHERE id = ?";
PreparedStatement stmt = connection.prepareStatement(query);
            stmt.setInt(1, dogId);

            int rowsAffected = stmt.executeUpdate();

            stmt.close();

            System.out.println("Item deleted successfully! Rows affected: " + rowsAffected);

This method hides the complex part, which takes a few extra steps to set up the connection:

public static Connection getConnectionWithUserPassword(String region, String dbSecretArn, String host, String databaseName) {
    Map<String, String> creds = getDatabaseCredentials(region, dbSecretArn);
    String jdbcUrl = String.format("jdbc:postgresql://%s/%s",
            host, databaseName);

    try {
        return DriverManager.getConnection(jdbcUrl, creds.get("username"), creds.get("password"));
    } catch (SQLException e) {
        e.printStackTrace();
        throw new RuntimeException("Database connection failed using username/password", e);
    }
}

private static Map<String, String> getDatabaseCredentials(String region, String dbSecretArn) {
    try (SecretsManagerClient secretsClient = SecretsManagerClient.builder()
            .region(Region.of(region))
            .credentialsProvider(DefaultCredentialsProvider.create())
            .build()) {

        GetSecretValueRequest secretRequest = GetSecretValueRequest.builder()
                .secretId(dbSecretArn)
                .build();

        GetSecretValueResponse secretResponse = secretsClient.getSecretValue(secretRequest);
        String secretJson = secretResponse.secretString();
        JSONObject secretObj = new JSONObject(secretJson);

        return Map.of(
                "username", secretObj.getString("username"),
                "password", secretObj.getString("password")
        );

    } catch (Exception e) {
        throw new RuntimeException("Error retrieving secrets from Secrets Manager", e);
    }
}

How It Works: Fetching the Credentials for the Database Connection & Executing Queries

This snippet shows a traditional JDBC connection to our PostgreSQL database through RDS Proxy using username/password authentication, but with an extra security layer through AWS Secrets Manager. First, the code creates a SecretsManager client to fetch the database credentials securely from AWS Secrets Manager. When getDatabaseCredentials() is called, it makes an API call to Secrets Manager, retrieves the JSON-formatted secret containing the username and password, and parses them into a Map. Then, getConnectionWithUserPassword() constructs a standard JDBC URL pointing to your RDS Proxy endpoint and uses the retrieved credentials to establish a connection through the usual DriverManager.getConnection() method. Once the connection is established, the code uses standard JDBC PreparedStatement to safely execute parameterized SQL queries. The RDS Proxy intercepts this connection request, validates the credentials, and maintains the actual database connection pool behind the scenes. This method is a good balance between security and simplicity, as it keeps the database credentials out of your codebase while still allowing you to use standard JDBC connections.

Wrapping It Up

And there you have it—four ways to authenticate with an RDS database in AWS (and LocalStack), each with its own strengths depending on your security needs and architecture. Understanding all the options available and how they work is no easy task, but there’s no way around it when building secure and reliable cloud applications. Whether you prefer the simplicity of direct credentials, the security of IAM tokens, the convenience of the Data API, or the flexibility of Secrets Manager, LocalStack makes it easy to test and debug all these methods locally. This not only saves time and costs but also helps you build robust, production-ready applications with confidence. So, the next time you’re setting up RDS authentication, you’ll know exactly which method fits your project like a well-trained pup. 🐾 Happy coding!