March 18, 2016 at 6:30 am #13160
1) Can we associate a model specifically, to a input adaptor and then the output of the model to an output adaptor.
2) For MQ to file adaptor, Can we have a model in between which will dump the data from the input topic to a file? To have this approach to work we need have yes as the answer for the first 🙂
3) If point 1 is not an option, can we have the input and output adaptors specified in the model?
March 18, 2016 at 6:31 am #13161
There is no way now to associate an output adapter to a model, but this feature is being worked on.
For input adapter, it is possible to do, although not through a clear configuration way. Each Model can be associated (most of the time is associated) to a specific message. In the say way, you can specify an InputAdapter’s “Associated Message” property to be that same message. In this way, technically, each input adapter becomes an “implicitly” associated to any model that acts on a specific message.
If you are using Native Model (java,scala), your model can write your own logic to externalize message to a file (not sure how to do it with a PMML model). Currently, the engine will not do it for you.
Not sure if #1 answers your question
March 18, 2016 at 6:32 am #13162
There is no reason that the input adapter can’t mark the incoming messages with its “id” as it is accepted from its source if that is desired.
With regard to writing such marked messages to a corresponding file, this too can be accomplished by either the native models or the kamanja pmml models. The native models might simply write some function that does this and the pmml model would some custom udf that has this behavior.
March 18, 2016 at 6:32 am #13163
Currently we have 5 fiels which we need to write to a file. If we follow the above way, we will need to have 5 input adaptors configured to 5 messages each having the different format so that it can read the file.If i am correct could you please answer below.
1) Do we need 5 version of same models accepting each message? or can we have all the message linked to the model as below.
<DataField name=”msg” displayName=”msg” optype=”categorical” dataType=”message name”/>
2)in PMML you will need a function which willdo the changes as per my req for each case and return the data along with flag to do teh decsion. For me whatever data it gets via this specific queue has to be written to the file as a CSV. No calculation do be done. But still there need to be a function defined as PMML cant do it without a udf.ie it dont have an in built function.
3) Currently model is writing a file as an output but if we are using a single model and specifying all the messages in side it can write the messages specifi to a queue to a specific file. we will need seperate file as the loading to be done in HBase using different containers.
March 18, 2016 at 6:33 am #13164
Waiting for a response on this.
March 18, 2016 at 6:34 am #13167
So, I am trying to understand. Do you have 5 different messages from which you need to pull the fields into a single file?
March 18, 2016 at 6:35 am #13170
Had a discussion with Dan and Will – we should be able to create separate messages with the various files and push them to a single queue. The MQ input adapter can subscribe to various message headers and present them to the model as separate messages – the model can then decide whether to persist the messages (for reference data) or to take a decision (for transactional data). There are various ways in which the functionality can be implemented, but you shouldn’t need multiple models from what I can see.
Please take a look at the MQ input adapter code for the message creation functionality.