CDC — Change Data Capture
Replicate database changes in real time using NATIS CDC, powered by Debezium-compatible log-based capture.
Change Data Capture (CDC) allows NATIS to replicate INSERT, UPDATE, and DELETE operations from source databases in near real-time by reading the database transaction log (WAL, binlog, redo log) instead of polling tables. This provides millisecond-latency replication with minimal impact on the source database.
Supported CDC Sources
- PostgreSQL (pgoutput, wal2json logical replication plugins)
- MySQL / MariaDB (binary log / binlog-based)
- Oracle Database (LogMiner, XStream)
- Microsoft SQL Server (SQL Server CDC feature)
- MongoDB (Change Streams)
- Cassandra (CDC commit log reader)
Configuring PostgreSQL CDC
-- Step 1: Enable logical replication on PostgreSQL
-- Edit postgresql.conf:
-- wal_level = logical
-- max_replication_slots = 4
-- max_wal_senders = 4
-- Step 2: Create a replication slot
SELECT pg_create_logical_replication_slot(
'natis_cdc_slot',
'pgoutput'
);
-- Step 3: Create a publication for tables to replicate
CREATE PUBLICATION natis_pub
FOR TABLE orders, customers, products
WITH (publish = 'insert,update,delete');
-- Step 4: Grant replication privileges
GRANT REPLICATION SLAVE ON *.* TO 'natis_cdc_user'@'%';CDC Event Types in Delta Lake
NATIS CDC writes all captured events to a Delta Lake staging table with an additional _cdc_op column indicating the operation type. The platform then applies these changes to the target table using MERGE operations.
Soft-delete semantics: by default, DELETE operations result in the target record being marked with _deleted=true and _deleted_at=<timestamp> rather than physically removing the row, preserving historical data for auditing.
- _cdc_op = 'I' — INSERT — new record created in source
- _cdc_op = 'U' — UPDATE — existing record modified in source
- _cdc_op = 'D' — DELETE — record deleted in source (tombstone)
- _cdc_op = 'T' — TRUNCATE — entire table truncated in source
Was this page helpful?
Thanks for your feedback!