4/24/2017 10:39:33 PM
Azkaban doesn't ship the hadoop-security-plugin with the executor.
Although this is clean since the executor doesn't depend on the plugin it
doesn't make Azkaban any usable since most use cases of Azkaban requires
hadoop. For folks who aren't using Azkaban for hadoop, this would be sitting
idle.
However, the strong assumption to use this plugin is that Hadoop classes must
be on the classpath. Else those jobs using the plugin will fail.
Tested this on the solo server and this works fine.
|
4/24/2017 10:30:22 PM
code (#1013)
This change refactors the initialization of the server instances. The current code in these classes is quite entangled. Breaking this up into multiple commits. This is a prerequisite to the upcoming Guice changes. - removed duplicate methods from Azkaban*Server classes - moved some of the references constants to `Constants` class - config was getting updated during server initialization. Fixed that. The same change is now applied when we create the config object `Props` - refactored IntelliJ suggested warnings which simplify the code - extracted methods and created smaller methods to make it easier to navigate and isolate code
Tested on solo server as well as web and executor separately on dev instances.
|
4/24/2017 10:16:27 PM
error (logging inside CpuStatusFilter should use CPUSTATUS_FILTER_NAME, not MINIMUMFREEMEMORY_FILTER_NAME)
|
4/24/2017 6:41:05 PM
simply puts the DB calls behind the newly created storage layer. Currently the flow of the code is as follows
WEB API (REST) -> ProjectManager -> ProjectLoader (JdbcProjectLoader) -> DB
This has been modified to WEB API (REST) -> ProjectManager -> StorageManager -> Storage(DatabaseStorage) -> ProjectLoader (JdbcProjectLoader) -> DB
This has only been done for the put call. The get implementation is untouched yet.
|
4/24/2017 5:25:14 PM
update resolved a build warning:
"The Task.leftShift(Closure) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use Task.doLast(Action) instead."
see
https://github.com/cinnober/semver-git/releases
|
4/24/2017 3:09:33 PM
some jobs, eg hadoop related, 5 sec is not enough for the jobs to
finish their clean up and exit. Increasing the time to 30 sec in code
to temporarily mitigate issue.
|
4/24/2017 3:01:38 PM
Fix ClassCanBeStatic warnings in common sub-project
See http://errorprone.info/bugpattern/ClassCanBeStatic
e.g.
/Users/ruyang/oss/azkaban/azkaban-common/src/test/java/azkaban/utils/RestfulApiClientTest.java:47:
warning: [ClassCanBeStatic] Inner class is non-static but does not
reference enclosing class protected class MockRestfulApiClient extends RestfulApiClient<String> { ^ (see http://errorprone.info/bugpattern/ClassCanBeStatic) Did you mean 'protected static class MockRestfulApiClient extends RestfulApiClient<String> {'?
Fix:
Made these inner classes static inner classes.
* Removed protected from a few static classes introduced
See http://stackoverflow.com/questions/24289070/why-we-should-not-use-protected-static-in-java
These classes only need package private access.
Testing:
Run all tests.
* Removed protected from MockRestfulApiClient
See http://stackoverflow.com/questions/24289070/why-we-should-not-use-protected-static-in-java
These classes only need package private access.
Testing:
Run all tests.
|
4/24/2017 2:28:28 PM
(#1008)
E.g.
/Users/ruyang/oss/azkaban/azkaban-exec-server/src/test/java/azkaban/execapp/JobRunnerTest.java:332: warning: [WaitNotInLoop] Because of spurious wakeups, wait(long) must always be called in a loop wait(2000); ^ (see http://errorprone.info/bugpattern/WaitNotInLoop)
Fix:
Change to Thread.sleep which is not subject to this issue.
|
4/24/2017 2:19:59 PM
work on finding a better solution.
|
4/24/2017 1:52:04 PM
deprecated (#1005)
The update resolved a build warning in the web server sub-project:
"The Task.leftShift(Closure) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use Task.doLast(Action) instead."
see
http://mrhaki.blogspot.com/2016/11/gradle-goodness-replacing-operator-for.html
|
