12/3/2018 9:44:53 PM
(#2039)
* AZNewDispatchingLogic_CreateNewModule
* address comment
|
12/3/2018 3:58:15 PM
was introduced in #1833
|
11/29/2018 8:17:40 PM
in CommonJobProperties.
* Fix mismatch in name of job log files when trying to kill Hadoop/Spark jobs. Issue: when a user tries to kill a Hadoop/Spark job within an embedded flow the application looks for a log file using the following name pattern "_job.<exec_id>.<job_id>.log" no matter whether the job was embedded or not. But actually if a job is inside an embedded flow the name format used to create its log file is: "job.<exec_id>.<parent_flow>..<job_id>.log". So trying to kill a job inside an embedded flow before would fail because of the log file name mismatch.
* Fix according to review comments
|
|
|
11/27/2018 11:36:05 PM
provide the ability to call HTTP resources as described in 'HTTP Job Callback' https://azkaban.github.io/azkaban/docs/latest/#common-configurations
job.notification.started.1.url=.. and etc
In the implementation https://github.com/azkaban/azkaban/blob/master/azkaban-exec-server/src/main/java/azkaban/execapp/event/JobCallbackManager.java#L263 'azkabanHostName' used as the value of 'Host' header. But the value should be the hostname of requested HTTP resource, not Azkaban's host.
For example for https://hooks.slack.com/services should be Host:hooks.slack.com
|
|
|
|
|
|
|
11/16/2018 11:27:44 PM
execution (#2023)
* Extract method submitFlowRunner
* Extract method createFlowRunner
* Improve dispatch request handling of a previously submitted execution
- If the execution is indeed running, return OK so that dispatcher knows that it was successfully dispatched
- If the execution was left in some intermediate state, return an error so that dispatcher knows to retry or finalize the execution as failed
|
11/15/2018 11:45:28 PM
PR:
1. adds more logging and standardize existing logging for each token prefetching methods so that we know prefetching from which service is stuck.
2. removes "synchronized" for doPrefetch(HadoopSecurityManager_H_2_0#doPrefetch). Current design makes it hard to debug which token service the job is stuck with fetching token. Since HadoopSecurityManager_H_2_0 is shared by all jobs in the executor, if one job is stuck with fetching token with a problematic token service, all other jobs will be blocked from entering into this synchronized method. It's impossible to infer which token service jobs are stuck with from job logs as they are just waiting for one job to finish fetching token.
|
