3/1/2018 12:59:24 AM
(#1669)
Issue:
It's confusing which script to use.
If the wrong script is used, AZ admins are confused as to why the
servers are not running.
This type of mistakes have happened to multiple team members.
Solution:
Move the script that is designed to be called by the user facing scripts
to a new directory called internal.
Renamed scripts.
The console logs will now go to the current working directory instead of assuming that there is a logs directory under the current working directory.
Use the same startup/shutdown approach for the solo server.
Did some minor code clean up as well to improve readability and confirm
to the Google coding style.
Reduced code duplication in the shutdown scripts.
Todo: need to update docs to reflect the new script names.
Todo: Remove assumptions about where the AZ installation root directory
is and allow the start up and shutdown scripts to be invoked from a
directory other than the installation root. Also makes the conf file
configurations more portable i.e. in dependent of the working directory.
Testing:
Manually tested starting and stopping web, executor and solo servers.
e.g.
[~/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/start-web.sh
[~/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/shutdown-web.sh
Killing web-server. [pid: 32249], attempt: 1
shutdown succeeded
Testing executor:
start both an executor and a web server locally.
copy a test conf directory to the extracted binary directory.
Hit
http://localhost:8081/status
{
version: "3.43.0-9-g9545a072",
pid: "32950",
installationPath: "/Users/ruyang/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072/lib/azkaban-web-server-3.43.0-9-g9545a072.jar",
usedMemory: 122591464,
xmx: 3817865216,
isDatabaseUp: true,
executorStatusMap: {
1: {
id: 1,
host: "localhost",
port: 12321,
isActive: true
}
}
}
[~/oss/azkaban/azkaban-exec-server/build/distributions/azkaban-exec-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/shutdown-executor.sh
Killing executor. [pid: 32739], attempt: 1
shutdown succeeded
Made sure the process is no longer running.
---
shell script unit tests passed
[~/oss/azkaban/azkaban-common/src/test/bash (move-start-stop-scripts)]$ ./test_util.shunit2
test_is_process_running
test_kill_process_with_retry
Ran 2 tests.
OK
|
2/27/2018 11:10:42 PM
instance in db. (#1666)
This PR switched column mysql data type from datetime to bigint to keep the start/end time of dependency instance. The reason is datetime is not timezone agnostic(https://dev.mysql.com/doc/refman/5.5/en/datetime.html). It's less error-prone to keep UTC timestamp(System.currentTimeMillis()) in db.
|
2/27/2018 5:51:37 PM
3.43.0
(#1667)
changed error message from "max wait min must be longer than X min"
to "max wait min must be at least X min(s)"
|
2/22/2018 10:40:29 PM
finished trigger instances should be retrieved based on end time. Corrected SQL statement and added associated unit test.
|
2/22/2018 6:56:27 PM
(#1656)
Today toProperties in Props API doesn't include parent properties. In this PR, we add an API to import both current and parent properties.
Tested locally.
|
2/22/2018 3:19:52 AM
to allow users to debug credential issues easily, this PR proposes adding job logger into the credential instance, so that related log messages will be shown in Azkaban web UI.
Notice that this PR is backward compatible. In the credential client code side, we have both 2-parameter and 3-parameter constructor. If we need to roll back this commit, it will not fail.
|
2/21/2018 10:43:25 PM
finished flows (#1643)
This PR fixes the bug which includes trigger Instances in cancelling state into recently finished flows.
|
2/21/2018 4:44:39 PM
scheduled flow trigger
AZ_WEB_SERVER_URL:AZ_WEB_SERVER_PORT/flowtrigger
|
2/20/2018 4:57:06 PM
showing flow trigger properties in flow trigger instance UI.
|
2/16/2018 9:16:14 PM
(#1646)
flow trigger service needs to be started first before scheduler starts to schedule existing flow triggers. This PR puts starting flow trigger service before starting flow trigger scheduler and adds more logging when starting flow trigger service/scheduler.
|
