Home Forums Kamanja Forums Problems & Solutions Streaming data into Kafka

This topic contains 2 replies, has 2 voices, and was last updated by  Archived_User28 1 year, 8 months ago.

  • Author
    Posts
  • #13428 Reply

    Archived_User57
    Participant

    Hello,

    we have a job that creates data at the rate of 4 records/sec and writes it to several files.
    each file will contain 1 hour worth of data.
    is there a way to stream this data into kafka continously ? instead of pushing the whole file one time after all the writes ?

    Regards,
    Haitham

  • #13429 Reply

    Archived_User57
    Participant

    just to explain , the question is : is there a CURRENT way to do this ? other than the fileAdapter that takes the file as it is ?

  • #13430 Reply

    Archived_User28
    Participant

    Hi Haitham,

    You can push the data into the kafka queue yourself.

Reply To: Streaming data into Kafka
Your information: