azkaban-aplcache

Change Kafka message syncing to false (#896) Kafka message …

2/3/2017 8:52:19 PM

syncing ensures that the messages are in order. However, the problem is that the syncing also slows down the transaction rate of these messages drastically. This affects the service significantly and is a blocking call. Reverting this change which was added to #852

Suvodeep Pyne

Commit: 3a346d3

Tree: b70e39a

Parents: 93c1597

Changes

azkaban-exec-server/src/main/java/azkaban/execapp/JobRunner.java 2(+1 -1)

Details

azkaban-exec-server/src/main/java/azkaban/execapp/JobRunner.java 2(+1 -1)

diff --git a/azkaban-exec-server/src/main/java/azkaban/execapp/JobRunner.java b/azkaban-exec-server/src/main/java/azkaban/execapp/JobRunner.java
index 41a34fc..5dbfee1 100644
--- a/azkaban-exec-server/src/main/java/azkaban/execapp/JobRunner.java
+++ b/azkaban-exec-server/src/main/java/azkaban/execapp/JobRunner.java
@@ -299,7 +299,7 @@ public class JobRunner extends EventHandler implements Runnable {
 
   private KafkaLog4jAppender createKafkaAppender() throws UndefinedPropertyException {
     KafkaLog4jAppender kafkaProducer = new KafkaLog4jAppender();
-    kafkaProducer.setSyncSend(true);
+    kafkaProducer.setSyncSend(false);
     kafkaProducer.setBrokerList(azkabanProps.getString(ServerProperties.AZKABAN_SERVER_LOGGING_KAFKA_BROKERLIST));
     kafkaProducer.setTopic(azkabanProps.getString(ServerProperties.AZKABAN_SERVER_LOGGING_KAFKA_TOPIC));