9/16/2017 11:38:06 AM
failures. It's quick, too.
* Fix bug where pipelined execution gets stuck
If you had executed a flow with some subflows disabled and then started another execution in pipeline mode, the next execution remained blocked forever by the jobs inside sub-flows, because they remained in READY state instead of being set to SKIPPED.
This was a pretty nasty bug because for example next morning's scheduled run can get stuck after you've launched an execution with some jobs disabled on the previous day.
I think it would make sense to clean this up also on the flow saving side: when subflow status is changed from DISABLED to SKIPPED, all jobs in the subflows would also be marked as SKIPPED.
* Fix bug in FlowWatcher.peekStatus
* Test pipeline block fix with a sub-flow
This test proves (or proved) how it happens. Without the fix this new test testBasicPipelineLevel1RunDisabledJobs gets stuck.
|
9/16/2017 1:07:03 AM
FlowRunnerPipelineTest
* Fast FlowRunnerPipelineTest
Removed all sleep.
* Fix NPE in printStatuses
* Use TemporaryFolder rule for workingDir
* Use ExecutionsTestUtil to get flow files
|
9/15/2017 11:48:28 PM
instead of log4j.
- Add or update the copyright notice for most files.
- Add a link to the dev tips wiki.
- Clarify the standard form of slf4j logger
- Mention the usage of the images repo.
|
9/15/2017 2:22:20 PM
2
* commit 8
* commit 9
* fix format
This PR is a follow-up of #1463 . Moving Props and Metrics Registry to az-core module.
|
9/15/2017 2:13:54 PM
brought a feature that AZ admins can publish messages in banner bar on the top of AZ UI. There exists an issue reported by the user:
I cannot click on the "Job List", "Flow Log", and "Stats" tabs until the orange message banner disappears after about 30 seconds.
The cause is that container-fill (css class) uses absolute position, when the top div elements moves down, it will override the div beneath. I figured out there are a number of places using container-fill. So I'd like not change the original place. The proposed fix is that we just let flow view page's div to be static, such that it will move downward along with top div.
|
|
|
9/15/2017 1:22:48 PM
code patch is to create az-core sub module and move some utils class to here. The discussion of this proposal is at https://github.com/azkaban/azkaban/issues/1462.
Used Intellij to do the refactor.
|
|
|
9/14/2017 3:58:51 PM
(#1442)
- Now instead of fixed 1s sleep we wait for max 10s for the node update count to be 2.
- This lets us use the luxury of additional sleep time (1s->2s) before killing the flow, so that there's a better chance for job getting inside its delayExecution() block.
Test failure was:
azkaban.execapp.JobRunnerTest > testDelayedExecutionCancelledJob FAILED java.lang.AssertionError: expected:<2> but was:<1>
It was faithfully reproduced when I removed the 1s sleep before assert on getNodeUpdateCount().
|
9/14/2017 3:25:40 PM
to move metrics foundation class to a common place, I am creating az-core sub module. Since Metrics relies on some utils class in azkaban-common, the first step of this series of refactor is to clean up some methods in `azkaban-common', and it facilitate me to do those refactor in the following PRs.
In this code patch:
* I move addCommonFlowProperties out of PropsUtils, so that we can move PropsUtil to az-core in the next PR easily.
* Also, this method is not suitable to stay in PropsUtils.
|
