The aim of the Dyalog-Kafka project is to provide a binding to part of the Confluent librdkafka library such that we can access Kafka from Dyalog APL.
Note KAFKA is a registered trademark of The Apache Software Foundation and has been licensed for use by Dyalog Ltd. Dyalog Ltd has no affiliation with and is not endorsed by The Apache Software Foundation.
Compiled libraries for supported platforms — AIX, Linux, macOS (arm64 and x64), and Windows — are available on the Releases page.
You can also build the library from source. To do so, clone this repository and ensure you have a C++ development toolchain installed.
Open kafka.sln in Visual Studio 2022, and build the project.
Copy all the dlls from the OutDir (kafka\x64\Debug for 64 bit debug version) to a directory and Start Apl in that directory.
To build on linux for 64 bit
PLATFORM=linux ./mk_kafka.sh 64
The output files are in distribution/linux/x64
TODO
Install the librdkafka via Homebrew:
brew install librdkafka
Build the wrapper:
cd <path>/kafka
c++ -shared -fpic -oMACbin/kafka.dylib -DFORmac kafka/kafka.cpp -lrdkafka -L/opt/homebrew/opt/librdkafka/lib -I/opt/homebrew/opt/librdkafka/include/librdkafka
or like linux but this requires dotnet to be installed to get the librdkafka package
PLATFORM=mac ./mk_kafka.sh 64
Now start Dyalog. You need a running Kafka instance. In the session, type
]cd path/to/kafka/repo
]link.create # aplsourceFor users who can not use ]link, it is possible to create the kafka.dws workspace.
The workspace can be built using the ]DBuild tool and saving it
]DBuild path/to/kafka.dyalogbuild
)WSID
)SAVEand then )LOAD kafka.dws.
Finally, initialise the library, passing as the argument the path where the kafka.[so|a|dylib] shared library was installed
Init 'path/to/dir/housing/shared/lib' ⍝ to load the dllFor further instructions, see the file SPEC.md, and sample code examples: