9/12/2017 1:29:02 AM
options in schedule page
|
9/11/2017 10:41:26 PM
this change, support for reporting events of interest from azkaban has been provided.
The default implementation uses a kafka event reporter. Users can provide alternate
implementations of the reporter. To begin with, the following events are reported. 1. FLOW_STARTED 2. FLOW_FINISHED 3. JOB_STARTED 4. JOB_FINISHED
In future, this can be easily extended to report other events. The default event reporter
implementation uses gobblin-metrics, which provides conevenient methods for event creation
and submission. Gobblin-metrics also provides a schema for the events. The default
implementation uses the kafka async producer.
Configuration changes:
Note: All changes must be applied to the executor server or the solo-server if in solo mode.
// Property is used to enable/disable event reporting with the default being false.
event.reporter.enabled=false
// Alternate implementations of the reporter can be specified using this property.
event.reporter.class=com.foo.EventReporterImpl
// Kafka topic name for the default implementation.
event.reporting.kafka.topic=TestTopicName
// Kafka broker list for the default implementation.
event.reporting.kafka.brokers=hostname.com:port_num
// Schema registry server for the default kafka implementation.
event.reporting.kafka.schema.registry.url=schemaRegistryUrl.com:port/schema
Testing Done:
End-to-end tests were performed and the following scenarios were verified. 1. Consumed the emitted kafka events and messages received were as expected. 2. Event reporter enabled/disabled scenario. 3. Kafka broker down scenario. 4. Reporter enabled, but kafka brokers undefined in the config scenario.
|
9/11/2017 9:41:53 PM
Wait that FAILED job event has been handled before succeeding the other jobs.
- Extra: make InteractiveTestJob more reliable with volatile fields
|
9/11/2017 7:11:11 PM
(#1444)
|
9/11/2017 3:00:46 PM
module doesn't belong to the common module.
It also helps speeding up the build a bit by taking better advantage of
the gradle cache and project dependency management.
Fixes #1227
|
9/11/2017 2:13:12 PM
the job6 status was SUCCEEDED if the flow killing took longer than usual, because it only slept for max 1 second.
Failure from Travis logs:
azkaban.execapp.FlowRunnerTest > exec1FailedKillAll FAILED java.lang.AssertionError: Wrong status for [job6] expected:<KILLED> but was:<SUCCEEDED>
|
9/9/2017 9:51:00 PM
used by the internal build system.
|
|
|
|
|
9/7/2017 5:53:38 PM
the API calling code from ExecutorManager to a new class ExecutorApiGateway
This way ExecutorManager doesn't handle raw JSON, the gateway always returns some higher level object (once again, this will allow making cleaner unit tests for ExecutorManager).
|
