8/17/2017 6:14:33 PM
azkaban db module (#1345)
The goal of this code patch is to gradually migrate to new azkaban DB module. I construct a new class ExecutorFlowDBManager to store Table execution_flow related DB operations. Basically, I move a few methods out from JdbcExecutorLoader to new class and leverage new DB API. There are 3 benefits for the new class:
* Split #1291 into a few commits, and will make reviews easier.
* Put Query Handler class and its caller together. In this way, make code clean and easy to read.
* Also easy to manage unit Tests.
I'm considering using ExecutorFlowDBManager directly to replace JdbcExecutorLoader, when it comes to execution_flows related operations. However, Since some classes like FlowRunner has not been guificied, it is not easy to add another dependency. We will do the follow-up refactor after executor loader is completed migrated.
|
8/17/2017 2:08:50 PM
now we're seeing noise in the number of errors being generated in our logs in part because the current working directory and the output properties file are both being recorded with System.err.println().
I changed these two messages to use this.info() instead. I also refactored an error message in createOutputPropsFiles to leverage the logger error method instead of using System.err.println().
|
8/17/2017 1:35:54 PM
output in the out directory.
Gradle uses build directory.
When running unit tests or solo server in intellij, build from intellij
is still required.
See
https://youtrack.jetbrains.com/issue/IDEA-175172
|
8/17/2017 11:43:05 AM
plugin is no longer needed since the Idea IDE has better support for directly importing a gradle build file. e.g. when using the gradle idea task, the resulting project doesn't have the gradle view. And having the cleanIdea task introduces a small risk when it is invoked by mistake. This will delete all personalization to a project.
|
8/17/2017 12:53:17 AM
the new JdbcProjectImpl class to leverage new DB API. It has been in production for a while, so it is safe to remove deprecated code.
|
8/16/2017 9:26:56 PM
production issues (e.g., hadoop crash, DB down) hits (Or, is schedule to hit) Azkaban, we are thinking about creating a simple UI Note system to inform AZ users the production issues as soon as we can. In this code patch, a UI Note system is proposed to resolve this issue.
AZ admins are able to compose the error/warning message and publish the contents into AZ memory. Then AZ navigation banner will display the message/error to every Azkaban user who's accessing AZ page. The message will disappear after 30 seconds.
|
|
|
8/16/2017 1:28:58 PM
"centerNode" function calls getTransformToElement(), which was removed from the SVG spec and also from browsers such as Chrome and Firefox, so the "Center" context menu button that appears when right-clicking in the graph view doesn't work. Fortunately it doesn't seem to use this call for anything so simply removing the line makes the button work again.
Tested by injecting the modified svg-graph.js as a Greasemonkey script in Firefox 55.0.1, and locally modifying the script in the debugger in Chrome 60.0.3112.90.
|
8/15/2017 7:07:34 PM
API implementation was missing from LocalStorage. Added it. Updated test.
|
8/15/2017 6:53:02 PM
(#1336)
activeJobRunners maintains all active job runners, so we should remove inactive job runners when job finishes. As a consequence, activeJobRunners will keep queuing up and querying activeJobRunners by job name will return invalid job runner. This PR fixes this issue.
|
