7/9/2018 6:12:59 PM
dependency order. Otherwise it can happen that web server tries to launch an execution, but it fails as executor server hasn't started yet.
Also start ExecutorManager's threads (that periodically call executors to get updates) not sooner than when the web server is started. Those threads used to be started already as soon as Guice injection creates the ExecutorManager instance (which happens before executor server is started – even now that the order is fixed).
This is one step towards having everything run in "multi-executor" mode.
As you may know, currently there are two possible modes, controlled by azkaban.use.multiple.executors=true|false:
multi-executor mode, based on the executors DB table
single executor mode, based on fixed props executor.host & executor.port
That adds extra complexity and different branches of code are executed depending on the mode. However, everything can just as well be run as "multi-executor" (but just one executor if that's what you want, and naturally so in case of solo-server). It makes it also easier to reach a better test coverage when testing is always done in the same "mode" that's used in production configurations.
I also noticed that ExecutorManager loads executors from the DB at the time of Guice injection (before executor server is started). This may need to be changed as well to properly support multi-executor mode on a single server.
|
6/29/2018 6:54:09 PM
invalid address from users. (#1827)
|
|
|
|
|
6/22/2018 10:00:47 PM
endtime was showing current time when trigger instance is running.
|
6/21/2018 5:24:38 PM
computation of a flow
level++ returns the level variable BEFORE it is incremented. This means
the level of a node is never incremented and is always zero.
Hence, we need to increment the value by one before passing it to the
function.
Fixes #1793.
* followup: add tests for node level computation
* followup: refactor test according to review comments
|
6/20/2018 9:15:46 PM
test failed today on trunk
azkaban.jobExecutor.ProcessJobTest > testCancelAfterJobProcessCreation FAILED org.junit.ComparisonFailure: expected:<[fals]e> but was:<[tru]e> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at azkaban.jobExecutor.ProcessJobTest.testCancelAfterJobProcessCreation(ProcessJobTest.java:228)
Cause:
I suspect it is because of the travis CI machine being too busy to cancel
the job before it finishes.
Solution:
Increase the running time of the job from 1 second to 5 seconds.
This will not increase the test running time when the test is successful
since the job will be canceled as soon as possible.
However it will increase the test running time if the cancel logic
doesn't interrupt the job as expected.
Also removed the printout of the stacktrace from the job. This information
doesn't deliver much value and only makes the log noisier.
|
6/20/2018 9:00:02 PM
mentioned in issue #1778, having a tooltip on the table would let the user know that it can be sorted when the header is clicked. This pull request will add the tooltip on scheduling page and history page which are being mostly used.
Sample screenshot:
image
This is a duplicate PR (#1780) but from a different branch, I moved these changes from master->sort-text in my repo.
|
6/20/2018 5:33:56 PM
3.48.0
change configurable key name
change default value for max cache size to 128GB which is a more standard number.
change default value for stop cleanup threshold to 60 to allow more space to be freed.
change data type of stop cleanup threshold from double to int.
rename the parameter name(projectDirMaxSizeInMB - > projectDirMaxSizeInMb) to align with coding standard.
|
6/19/2018 6:16:35 PM
This PR implements LRU project purging to prevent project files eating up disk space on executor.
We will encapsulate following logic inside the background cleanup thread:
if(disk space consumed by projects >= predefined threshold) List all project files and sort them by creation time in ascending order Iterate over the project file list, delete the file if the project is not running until disk size of projects drops down below a predefined lower threshold or no more projects to delete.
Why using creation time not last access time:
Directory last access time is not maintained by most file systems so we cannot rely on file system API to get last access time. An alternative considered is to modify the project dir every time the code reads it, then last modification time would be equivalent of last access time. The associate cons are code complexity and overhead of disk IO. We will use last creation time as the indicator of oldness of the project. Although creation time might not be as indicative as last access time to determine the hotness of a project, we are still ok with using it based on the assumption that the older the project is, more likely it’s not being used.
This PR also changes execution dir retention from 1 day to 2 hours. Since execution dir is hard linked to project dir, so disk space will be released only when no reference to project dir exists. That's why we want to shorten the execution dir retention time so that disk pressure can be alleviated sooner.
Project cleanup is executed by active executor only to prevent ensure only one thread will be performing deletion. Currently it relies on FlowRunnerManager#isExecutorActive to determine whether itself is active. But isExecutorActive is reliable only when azkaban admin uses activate API to active the executor. If admin manually updates the executors table in database and call reloadExecutors API, then the flag won't be set which inactive executor to perform deletion. So improvement need to be done here to make sure isExecutorActive is authentic.
|
