Skip to content

adapters: pg-in: TLS support for postgres input#5655

Merged
abhizer merged 2 commits intomainfrom
pg-in-tls
Feb 25, 2026
Merged

adapters: pg-in: TLS support for postgres input#5655
abhizer merged 2 commits intomainfrom
pg-in-tls

Conversation

@abhizer
Copy link
Contributor

@abhizer abhizer commented Feb 18, 2026

Support TLS connectors for the PostgreSQL input connector.

  • In the test-adapters ci workflow, we add a new postgres instance that runs with TLS to test this connector. However, these tests do not cover the verify_hostname flag.

Describe Manual Test Plan

Tested locally with two PostgreSQL instances, one with TLS one without.

# 1. Start non-TLS postgres on port 5432
docker run -d --name postgres-plain \
  -e POSTGRES_USER=postgres \
  -e POSTGRES_PASSWORD=password \
  -e POSTGRES_DB=postgres \
  -p 5432:5432 \
  postgres:16

# 2. Start TLS postgres on port 5433 
# (generates certs, builds image, starts container)

# 3. Run the postgres tests
POSTGRES_URL="postgres://postgres:password@localhost:5432" \
POSTGRES_SSL_URL="postgres://postgres:password@localhost:5433/postgres?sslmode=require" \
POSTGRES_SSL_CA_LOCATION="$PGCERTS/ca.crt" \
POSTGRES_SSL_CLIENT_LOCATION="$PGCERTS/client.crt" \
POSTGRES_SSL_CLIENT_KEY_LOCATION="$PGCERTS/client.key" \
POSTGRES_SSL_VERIFY_HOSTNAME="true" \
cargo test -p dbsp_adapters -- pg

# 4. Cleanup when done
docker rm -f postgres-plain postgres-tls

Checklist

  • Unit tests added/updated
  • Integration tests added/updated
  • Documentation updated
  • Changelog updated

Support TLS connectors for the PostgreSQL input connector.

Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>
Copy link
Collaborator

@mythical-fred mythical-fred left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good work overall. The refactoring is clean — extracting PostgresTlsConfig and make_tls_connector into shared types/logic is exactly the right move. The manual test plan is detailed, integration tests cover both paths, and the commit message is solid.

Step 2 of the manual test plan is missing its command. The comment says "generates certs, builds image, starts container" but the actual shell command is not shown. Future readers and CI debugging will want that.

Everything else is code quality observations — inline below.

file.flush()
.context("failed to flush CA certificate to tempfile")?;

let (_, path) = file.keep()?;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pre-existing from output.rs, but worth fixing now that this is shared code: file.keep() promotes the temp file to a permanent path that is never cleaned up. Every call to make_tls_connector (including reconnects in the output connector) leaks one file on disk.

A cleaner alternative that avoids the file entirely — parse the PEM in memory and add to the cert store directly:

let ca_cert = X509::from_pem(pem.as_bytes())
    .context("failed to parse CA certificate")?;
builder.cert_store_mut()
    .add_cert(ca_cert)
    .context("failed to add CA certificate to store")?;

More symmetric with how client certs are handled below (builder_set_client_from_pem) and eliminates the temp file lifetime concern entirely.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a problem we can fix?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried something like this when I first implemented TLS support but there was some issue with it that required pulling this trick, although I don't really remember what the problem was right now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

});
client
} else {
debug!("postgres {endpoint_name}: no CA certificates provided, connecting without TLS");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This map_err closure is byte-for-byte identical to the one in the TLS branch above. Extract it before the if let to eliminate the duplication:

let make_conn_err = |e: tokio_postgres::Error| {
    ControllerError::invalid_transport_configuration(
        &endpoint_name,
        &format!(
            "Unable to connect to postgres instance '{}': {e}",
            self.config.uri
        ),
    )
};

Both branches then use .map_err(make_conn_err)?.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this

&format!(
"Unable to connect to postgres instance '{}': {e}",
self.config.uri
),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The tokio::spawn block here is identical to the one in the TLS branch. Because Connection<T> differs between TLS and NoTls variants the client assignment itself is hard to unify, but the spawn logic can be extracted into a small helper:

fn spawn_pg_connection_monitor(
    name: String,
    conn: impl Future<Output = Result<(), tokio_postgres::Error>> + Send + 'static,
) {
    tokio::spawn(async move {
        if let Err(e) = conn.await {
            error!("postgres {name}: connection error: {e}");
        }
    });
}

Eliminates the literal duplication and gives the pattern a name.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this too

}

impl PostgresTlsConfig {
pub fn has_tls(&self) -> bool {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

has_tls() only checks for a CA cert. If a user sets ssl_client_pem or ssl_client_key but forgets ssl_ca_pem/ssl_ca_location, they silently get a plaintext connection with no indication anything is wrong. Silent fallback to plaintext when the user clearly intended TLS is the kind of misconfiguration that takes a while to diagnose.

Suggest adding a warning in make_tls_connector:

if !tls.has_tls()
    && (tls.ssl_client_pem.is_some()
        || tls.ssl_client_location.is_some()
        || tls.ssl_client_key.is_some()
        || tls.ssl_client_key_location.is_some())
{
    tracing::warn!(
        "postgres: TLS client certificate fields are set but no CA certificate \
         was provided; connecting without TLS. Set `ssl_ca_pem` or \
         `ssl_ca_location` to enable TLS."
    );
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with this too

@abhizer abhizer requested a review from ryzhyk February 25, 2026 16:28
Copy link
Contributor

@gz gz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we fix the flagged comments otherwise lgtm

(I assume we already had tests for this but they now use SSL?)

Signed-off-by: Abhinav Gyawali <22275402+abhizer@users.noreply.github.com>
@abhizer abhizer enabled auto-merge February 25, 2026 19:06
@abhizer abhizer added this pull request to the merge queue Feb 25, 2026
Merged via the queue into main with commit 14b6e04 Feb 25, 2026
3 checks passed
@abhizer abhizer deleted the pg-in-tls branch February 25, 2026 23:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants