3/2/2018 9:08:36 PM
3.44.0
the name consistent with the old names to minimize changes in other places that still reference the old names.
|
3/1/2018 6:48:59 PM
service (#1670)
Flow trigger service currently maintains an in memory list of running trigger instances for fast retrieval of the list of currently running trigger instances. However, it also introduces some issues like race condition and threads being created per request of getting running trigger instances. Removing the dependency on running trigger instances and retrieving directly from DB is a simpler and less error prone option. If retrieval of running triggers starts to have performance concern, we will reconsider it.
But this PR doesn't remove the running trigger entirely from flow trigger service since the mapping between running trigger instance and and its DependencyInstanceContext still needs to be kept in memory in case where users want to cancel a running trigger instance.
follow up:
Add index to execution_dependencies table for faster retrieval.
As execution_dependencies table keeps growing, we will need to implement purging mechanism to keep reasonable amount of historical executions.
|
3/1/2018 12:59:24 AM
(#1669)
Issue:
It's confusing which script to use.
If the wrong script is used, AZ admins are confused as to why the
servers are not running.
This type of mistakes have happened to multiple team members.
Solution:
Move the script that is designed to be called by the user facing scripts
to a new directory called internal.
Renamed scripts.
The console logs will now go to the current working directory instead of assuming that there is a logs directory under the current working directory.
Use the same startup/shutdown approach for the solo server.
Did some minor code clean up as well to improve readability and confirm
to the Google coding style.
Reduced code duplication in the shutdown scripts.
Todo: need to update docs to reflect the new script names.
Todo: Remove assumptions about where the AZ installation root directory
is and allow the start up and shutdown scripts to be invoked from a
directory other than the installation root. Also makes the conf file
configurations more portable i.e. in dependent of the working directory.
Testing:
Manually tested starting and stopping web, executor and solo servers.
e.g.
[~/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/start-web.sh
[~/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/shutdown-web.sh
Killing web-server. [pid: 32249], attempt: 1
shutdown succeeded
Testing executor:
start both an executor and a web server locally.
copy a test conf directory to the extracted binary directory.
Hit
http://localhost:8081/status
{
version: "3.43.0-9-g9545a072",
pid: "32950",
installationPath: "/Users/ruyang/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072/lib/azkaban-web-server-3.43.0-9-g9545a072.jar",
usedMemory: 122591464,
xmx: 3817865216,
isDatabaseUp: true,
executorStatusMap: {
1: {
id: 1,
host: "localhost",
port: 12321,
isActive: true
}
}
}
[~/oss/azkaban/azkaban-exec-server/build/distributions/azkaban-exec-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/shutdown-executor.sh
Killing executor. [pid: 32739], attempt: 1
shutdown succeeded
Made sure the process is no longer running.
---
shell script unit tests passed
[~/oss/azkaban/azkaban-common/src/test/bash (move-start-stop-scripts)]$ ./test_util.shunit2
test_is_process_running
test_kill_process_with_retry
Ran 2 tests.
OK
|
2/27/2018 11:10:42 PM
instance in db. (#1666)
This PR switched column mysql data type from datetime to bigint to keep the start/end time of dependency instance. The reason is datetime is not timezone agnostic(https://dev.mysql.com/doc/refman/5.5/en/datetime.html). It's less error-prone to keep UTC timestamp(System.currentTimeMillis()) in db.
|
2/27/2018 5:51:37 PM
3.43.0
(#1667)
changed error message from "max wait min must be longer than X min"
to "max wait min must be at least X min(s)"
|
2/22/2018 10:40:29 PM
finished trigger instances should be retrieved based on end time. Corrected SQL statement and added associated unit test.
|
2/22/2018 6:56:27 PM
(#1656)
Today toProperties in Props API doesn't include parent properties. In this PR, we add an API to import both current and parent properties.
Tested locally.
|
2/22/2018 3:19:52 AM
to allow users to debug credential issues easily, this PR proposes adding job logger into the credential instance, so that related log messages will be shown in Azkaban web UI.
Notice that this PR is backward compatible. In the credential client code side, we have both 2-parameter and 3-parameter constructor. If we need to roll back this commit, it will not fail.
|
2/21/2018 10:43:25 PM
finished flows (#1643)
This PR fixes the bug which includes trigger Instances in cancelling state into recently finished flows.
|
2/21/2018 4:44:39 PM
scheduled flow trigger
AZ_WEB_SERVER_URL:AZ_WEB_SERVER_PORT/flowtrigger
|
