Skip to content

Commit

Permalink
add demo illustrations and minor fixes
Browse files Browse the repository at this point in the history
  • Loading branch information
hpgrahsl committed Jul 6, 2023
1 parent f6ca925 commit e6c2d1d
Show file tree
Hide file tree
Showing 4 changed files with 8 additions and 4 deletions.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
4 changes: 2 additions & 2 deletions documentation/modules/ROOT/pages/20-dbz-kafka-connect.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,9 @@ image::dbz-connect-deployment.png["Debezium Deployment Mode with Kafka Connect"]

The following exercise shows and explains how to configure a Debezium Source Connector for postgreSQL. The resulting CDC pipeline will capture all data change events that are occurring in a postgreSQL database table and propagate these changes into an Apache Kafka topic. The following illustration shows the overview and data flow of the demo scenario.

**TODO: add picture here...**
image::dbz-tutorial-elementary-kafka-connect-example-basic.png["Debezium with Kafka Connect Demo Scenario"]

The example will be based on running all required data infrastructure components as containers locally, either with Podman or Docker. **It's important to note, that all containers are configured as _single ephemeral instances_ which means that all state and data is lost after shutting them down!**
For the example all required data infrastructure components are run as containers locally, either with Podman or Docker. **It's important to note, that all containers are configured as _single ephemeral instances_ which means that all state and data is lost after shutting them down!**

== Start required services as containers

Expand Down
8 changes: 6 additions & 2 deletions documentation/modules/ROOT/pages/21-dbz-server.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,10 @@ There are use cases for which you might want to make use of CDC and Debezium wit

image::dbz-server-deployment.png["Debezium Server Mode with different messaging infra"]

The following exercise shows and explains how to configure a Debezium Server for postgreSQL (source) and an HTTP endpoint (sink). The resulting CDC pipeline will capture all data change events that are occurring in a postgreSQL database table and propagate these changes to the specified HTTP endpoint as POST requests. The following illustration shows the overview and data flow of the demo scenario.

image::dbz-tutorial-elementary-debezium-server-example-basic.png["Debezium Server Demo Scenario"]

== Debezium Server Installation

To install Debezium Server download and unpack the https://repo1.maven.org/maven2/io/debezium/debezium-server-dist/2.2.1.Final/debezium-server-dist-2.2.1.Final.tar.gz[server distribution archive].
Expand Down Expand Up @@ -53,7 +57,7 @@ The following configuration for Debezium Server will create a CDC pipeline betwe

NOTE: This HTTP sink just serves an example target for sending Debezium's CDC events to and can be swapped out for other https://debezium.io/documentation/reference/2.2/operations/debezium-server.html#_sink_configuration[sink systems] that Debezium Server currently supports.

Copy your unique `webhook.site` URL which follows this pattern `https://webhook.site/<YOUR_INDIVIDUAL_UUID_HERE>` and ends with an individual UUID. You'll need for the configuration file below.
Copy your unique `webhook.site` URL which follows this pattern `https://webhook.site/<YOUR_INDIVIDUAL_UUID_HERE>` and ends with an individual UUID. You'll need this URL for the configuration file below.

The first step is to **provide a configuration file for Debezium Server** to specify all necessary **settings for the postgreSQL source** (i.e. where to capture CDC events from) **and the HTTP sink** (i.e. where to write CDC events to) to be used. Create a file called `application.properties` in the `conf/` sub-folder of the `debezium-server` folder with the following contents and make sure to use your individual `webhook.site` URL.

Expand All @@ -70,7 +74,7 @@ debezium.format.value=json

debezium.source.connector.class=io.debezium.connector.postgresql.PostgresConnector
debezium.source.offset.storage.file.filename=offsets.dat
#debezium.source.offset.flush.interval.ms=0
debezium.source.offset.flush.interval.ms=0
debezium.source.database.hostname=localhost
debezium.source.database.port=5432
debezium.source.database.user=postgres
Expand Down

0 comments on commit e6c2d1d

Please sign in to comment.