Home Forums Kamanja Forums Data Science & Models Processing during twitter streaming API

This topic contains 0 replies, has 1 voice, and was last updated by  Archived_User19 1 year, 6 months ago.

  • Author
    Posts
  • #13373 Reply

    Archived_User19
    Participant

    Team

    While processing the twitter streaming API after processing a lot of messages it gives the below errors. All these cases the messages are not copied to output queue even though we have input data in input topic.

    1)
    2015-10-16 14:05:53,641 – com.ligadata.KamanjaManager.LearningEngine [pool-6-thread-1] – ERROR – Model Failed => System.KamanjaTwitterDemo1. Reason: null. Message: org.json4s.native.JsonMethods$.render$default$2(Lorg/json4s/JsonAST$JValue;)Lorg/json4s/Formats;
    2)
    2015-10-16 14:07:23,158 – com.ligadata.InputAdapters.KafkaSimpleConsumer [pool-2-thread-1] – INFO – KAFKA-ADAPTER – Querying kafka for Topic pronto_input_queue metadata(partitions)
    2015-10-16 14:07:23,160 – kafka.network.BlockingChannel [pool-2-thread-1] – DEBUG – Created socket with SO_TIMEOUT = 3000 (requested 3000), SO_RCVBUF = 65536 (requested 65536), SO_SNDBUF = 32768 (requested -1).
    2015-10-16 14:07:23,162 – kafka.consumer.SimpleConsumer [pool-2-thread-1] – DEBUG – Disconnecting from brokername:port
    2015-10-16 14:07:23,222 – org.apache.zookeeper.ClientCnxn [main-SendThread(brokername:port)] – DEBUG – Got ping response for sessionid: 0x35040aa56054423 after 0ms
    2015-10-16 14:07:23,310 – org.apache.zookeeper.ClientCnxn [main-SendThread(brokername:port)] – DEBUG – Got ping response for sessionid: 0x15040aa54c44362 after 0ms
    2015-10-16 14:07:23,404 – kafka.producer.async.ProducerSendThread [ProducerSendThread-topic name_1809157631] – DEBUG – 5000 ms elapsed. Queue time reached. Sending..
    2015-10-16 14:07:23,404 – kafka.producer.async.ProducerSendThread [ProducerSendThread-topic name_1809157631] – DEBUG – Handling 0 events
    2015-10-16 14:07:23,540 – org.apache.zookeeper.ClientCnxn [main-SendThread(zookeepernode:port)] – DEBUG – Got ping response for sessionid: 0x25040aa54a142ec after 0ms
    2015-10-16 14:07:23,828 – org.apache.zookeeper.ClientCnxn [main-SendThread(zookeepernode:port)] – DEBUG – Got ping response for sessionid: 0x35040aa56054420 after 0ms

    3) ALso there are a lot of messages like below and even if messages are sent we dont get any messages in the output even though we have messages waiting in the input file.

    2015-10-16 14:59:58,338 – org.apache.zookeeper.ClientCnxn [main-SendThread(zookeeperservername:port)] – DEBUG – Got ping response for sessionid: 0x35040aa5605440e after 0ms
    2015-10-16 14:59:58,510 – org.apache.zookeeper.ClientCnxn [main-SendThread(zookeeperservername:port)] – DEBUG – Got ping response for sessionid: 0x15040aa54c44354 after 0ms
    2015-10-16 14:59:58,580 – kafka.producer.async.ProducerSendThread [ProducerSendThread-fc_fatafat_output_1809157631] – DEBUG – 5000 ms elapsed. Queue time reached. Sending..
    2015-10-16 14:59:58,580 – kafka.producer.async.ProducerSendThread [ProducerSendThread-fc_fatafat_output_1809157631] – DEBUG – Handling 0 events
    2015-10-16 14:59:58,735 – org.apache.zookeeper.ClientCnxn [main-SendThread(zookeeperservername:port)] – DEBUG – Got ping response for sessionid: 0x35040aa56054423 after 0ms
    2015-10-16 14:59:58,773 – org.apache.zookeeper.ClientCnxn [main-SendThread(zookeeperservername:port)] – DEBUG – Got ping response for sessionid: 0x15040aa54c44362 after 0ms
    2015-10-16 14:59:59,053 – org.apache.zookeeper.ClientCnxn [main-SendThread(zookeeperservername:port)] – DEBUG – Got ping response for sessionid: 0x25040aa54a142ec after 0ms

Reply To: Processing during twitter streaming API
Your information: