An output adapter is a destination for results produced by model execution. Examples are Kafka, MQ, and HDFS.

In order to understand output adapters, be familiar with concepts such as Kafka topics, partitions, and offsets. Read http://kafka.apache.org/documentation.html carefully.

Introduction to Output Adapters

Implementing an output adapter is a relatively easy task, at least compared to implementing a custom consumer. The custom code needs to implement the com.ligadata.KamanjaBase.OutputAdapter Scala trait. The two send methods and the Shutdown method in the following example must be overwritten by the customer implementation.

The send method needs to insert KeyedMessage into a specified message destination object (See Kafka documentation for instructions on how to instantiate and use Kafka producer objects).

The constructor for this producer must have two arguments:

  • AdapterConfiguration
  • CountersAdapter

The AdapterConfiguration and the CountersAdapter objects are described in the Kafka consumer section.

Example Output Adapter Configuration

The following defines each field specified in the output adapter configuration.

Example:

  • Name – unique logical name given to this adapter by the developer.
  • TypeString – either Input (used for the input adapter), Output (used for the output adapter), Status (used for the status adapter), or Validate (used for the validation adapter). The validation adapter is used once to validate and should point to the output adapter where the data is written.
  • InputAdapterToVerify – points to the validation adapter. This is used once.
  • ClassName – adapter class where the adapter is implemented. It should be a static class in Java and a singleton object in Scala.
  • JarName – JAR where the adapter class is implemented. This is loaded before resolving the class name.
  • DependencyJars – JARs required for the adapters and these are loaded before loading JarName.
  • AdapterSpecificCfg – string passed to the adapter. It is the adapter’s responsibility to understand the given string and get whatever is required for it.

Smart File Output Adapter

A smart file output adapter is a particular type of output adapter.