9/14/2017 3:58:51 PM
(#1442)
- Now instead of fixed 1s sleep we wait for max 10s for the node update count to be 2.
- This lets us use the luxury of additional sleep time (1s->2s) before killing the flow, so that there's a better chance for job getting inside its delayExecution() block.
Test failure was:
azkaban.execapp.JobRunnerTest > testDelayedExecutionCancelledJob FAILED java.lang.AssertionError: expected:<2> but was:<1>
It was faithfully reproduced when I removed the 1s sleep before assert on getNodeUpdateCount().
|
9/14/2017 3:25:40 PM
to move metrics foundation class to a common place, I am creating az-core sub module. Since Metrics relies on some utils class in azkaban-common, the first step of this series of refactor is to clean up some methods in `azkaban-common', and it facilitate me to do those refactor in the following PRs.
In this code patch:
* I move addCommonFlowProperties out of PropsUtils, so that we can move PropsUtil to az-core in the next PR easily.
* Also, this method is not suitable to stay in PropsUtils.
|
9/13/2017 8:21:51 PM
[Refactor] (#1459)
constructing Quartz Test, which needs to create tables in h2-in-memory. This PR does:
* Remove unknown code
* simplify logic
|
9/13/2017 4:07:05 PM
(#1458)
build: Enable gradle scans on travis CI to debug #1448
|
9/13/2017 12:20:00 PM
to azkaban.properties to be able to override mail
hostname and port number links. // User facing web server configurations used to construct the user facing server URLs. They are useful when there is a reverse proxy between Azkaban web servers and users. // enduser -> myazkabanhost:443 -> proxy -> localhost:8081 // when this parameters set then these parameters are used to generate email links. // if these parameters are not set then jetty.hostname, and jetty.port(if ssl configured jetty.ssl.port) are used. public static final String AZKABAN_WEBSERVER_EXTERNAL_HOSTNAME = "azkaban.webserver.external_hostname"; public static final String AZKABAN_WEBSERVER_EXTERNAL_SSL_PORT = "azkaban.webserver.external_ssl_port"; public static final String AZKABAN_WEBSERVER_EXTERNAL_PORT = "azkaban.webserver.external_port";
|
9/12/2017 5:05:57 PM
(#1454)
This method is only used in tests.
Make the name more descriptive.
This is particularly helpful when doing code reviews.
|
9/12/2017 1:29:02 AM
options in schedule page
|
9/11/2017 10:41:26 PM
this change, support for reporting events of interest from azkaban has been provided.
The default implementation uses a kafka event reporter. Users can provide alternate
implementations of the reporter. To begin with, the following events are reported. 1. FLOW_STARTED 2. FLOW_FINISHED 3. JOB_STARTED 4. JOB_FINISHED
In future, this can be easily extended to report other events. The default event reporter
implementation uses gobblin-metrics, which provides conevenient methods for event creation
and submission. Gobblin-metrics also provides a schema for the events. The default
implementation uses the kafka async producer.
Configuration changes:
Note: All changes must be applied to the executor server or the solo-server if in solo mode.
// Property is used to enable/disable event reporting with the default being false.
event.reporter.enabled=false
// Alternate implementations of the reporter can be specified using this property.
event.reporter.class=com.foo.EventReporterImpl
// Kafka topic name for the default implementation.
event.reporting.kafka.topic=TestTopicName
// Kafka broker list for the default implementation.
event.reporting.kafka.brokers=hostname.com:port_num
// Schema registry server for the default kafka implementation.
event.reporting.kafka.schema.registry.url=schemaRegistryUrl.com:port/schema
Testing Done:
End-to-end tests were performed and the following scenarios were verified. 1. Consumed the emitted kafka events and messages received were as expected. 2. Event reporter enabled/disabled scenario. 3. Kafka broker down scenario. 4. Reporter enabled, but kafka brokers undefined in the config scenario.
|
9/11/2017 9:41:53 PM
Wait that FAILED job event has been handled before succeeding the other jobs.
- Extra: make InteractiveTestJob more reliable with volatile fields
|
9/11/2017 7:11:11 PM
(#1444)
|
