Skip to content

An Apache Pulsar® sink for transferring events/messages from Pulsar topics to Apache Cassandra®, DataStax Astra or DataStax Enterprise (DSE) tables.

License

Notifications You must be signed in to change notification settings

cognitree/pulsar-sink

 
 

Repository files navigation

DataStax Apache Pulsar Connector

An Apache Pulsar® sink for transferring events/messages from Pulsar topics to Apache Cassandra®, DataStax Astra or DataStax Enterprise (DSE) tables.

Installation

To download and install this connector please follow the procedure detailed here.

Documentation

All documentation is available online here.

Building from the sources

If you want to develop and test the connector you need to build the jar from sources. To do so please follow those steps:

  1. First build the package and locate the cassandra-enhanced-pulsar-sink .nar file (in pulsar-dist/target):

    mvn clean package
    
  2. Copy the .nar file into the 'connectors' directory of your Pulsar broker

  3. Open the Pulsar Sink config file config/pulsar-sink.yml and configure your topic and mapping:

    - in "topics" put the name of the topic
    - create a mapping from your topic to your Cassandra table
    
  4. Run Pulsar Sink LocalRun and specify the path to the that config file:

    bin/pulsar-admin sinks localrun \
       --sink-config-file /path/to/config/pulsar-sink.yaml \
       -t cassandra-enhanced \
       -i persistent://public/default/topic
    

    With this command you are reading data from topic 'persistent://public/default/topic', the destination Cassandra cluster (address, authentication...), keyspace and table are defined in pulsar-sink.yml file

    You can map multiple topics to multiple tables.

Mapping specification

To see practical examples and usages of mapping, see: https://docs.datastax.com/en/pulsar-connector/docs/cfgPulsarMapTopicTable.html

About

An Apache Pulsar® sink for transferring events/messages from Pulsar topics to Apache Cassandra®, DataStax Astra or DataStax Enterprise (DSE) tables.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Java 99.9%
  • Other 0.1%