3/1/2018 12:59:24 AM
(#1669)
Issue:
It's confusing which script to use.
If the wrong script is used, AZ admins are confused as to why the
servers are not running.
This type of mistakes have happened to multiple team members.
Solution:
Move the script that is designed to be called by the user facing scripts
to a new directory called internal.
Renamed scripts.
The console logs will now go to the current working directory instead of assuming that there is a logs directory under the current working directory.
Use the same startup/shutdown approach for the solo server.
Did some minor code clean up as well to improve readability and confirm
to the Google coding style.
Reduced code duplication in the shutdown scripts.
Todo: need to update docs to reflect the new script names.
Todo: Remove assumptions about where the AZ installation root directory
is and allow the start up and shutdown scripts to be invoked from a
directory other than the installation root. Also makes the conf file
configurations more portable i.e. in dependent of the working directory.
Testing:
Manually tested starting and stopping web, executor and solo servers.
e.g.
[~/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/start-web.sh
[~/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/shutdown-web.sh
Killing web-server. [pid: 32249], attempt: 1
shutdown succeeded
Testing executor:
start both an executor and a web server locally.
copy a test conf directory to the extracted binary directory.
Hit
http://localhost:8081/status
{
version: "3.43.0-9-g9545a072",
pid: "32950",
installationPath: "/Users/ruyang/oss/azkaban/azkaban-web-server/build/distributions/azkaban-web-server-3.43.0-9-g9545a072/lib/azkaban-web-server-3.43.0-9-g9545a072.jar",
usedMemory: 122591464,
xmx: 3817865216,
isDatabaseUp: true,
executorStatusMap: {
1: {
id: 1,
host: "localhost",
port: 12321,
isActive: true
}
}
}
[~/oss/azkaban/azkaban-exec-server/build/distributions/azkaban-exec-server-3.43.0-9-g9545a072 (move-start-stop-scripts)]$ ./bin/shutdown-executor.sh
Killing executor. [pid: 32739], attempt: 1
shutdown succeeded
Made sure the process is no longer running.
---
shell script unit tests passed
[~/oss/azkaban/azkaban-common/src/test/bash (move-start-stop-scripts)]$ ./test_util.shunit2
test_is_process_running
test_kill_process_with_retry
Ran 2 tests.
OK
Details
diff --git a/azkaban-common/src/test/bash/test_util.shunit2 b/azkaban-common/src/test/bash/test_util.shunit2
index 4db057c..db99113 100755
--- a/azkaban-common/src/test/bash/test_util.shunit2
+++ b/azkaban-common/src/test/bash/test_util.shunit2
@@ -1,9 +1,11 @@
#!/usr/bin/env shunit2
# Make sure to "brew install shunit2" or use your favorite package manager to install
# before running tests
+# Run the test in the directory that contains this test.
+# see https://github.com/kward/shunit2
function oneTimeSetUp() {
- source ../../main/bash/util.sh
+ source ../../main/bash/internal/util.sh
}
function test_is_process_running() {
@@ -16,4 +18,3 @@ function test_kill_process_with_retry() {
kill_process_with_retry $! sleep 3 2>&1 >/dev/null #$! is pid for sleep process, sleep name, 3 times to loop
assertEquals "$?" "0" #$? is exit code of function kill_process_with_retry
}
-
diff --git a/azkaban-exec-server/src/main/bash/shutdown-executor.sh b/azkaban-exec-server/src/main/bash/shutdown-executor.sh
new file mode 100755
index 0000000..a93675e
--- /dev/null
+++ b/azkaban-exec-server/src/main/bash/shutdown-executor.sh
@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# Shutdown script for azkaban executor server
+set -o nounset
+
+script_dir=$(dirname $0)
+base_dir="${script_dir}/.."
+source "${script_dir}/internal/util.sh"
+common_shutdown "executor" ${base_dir}
diff --git a/azkaban-exec-server/src/main/bash/start-executor.sh b/azkaban-exec-server/src/main/bash/start-executor.sh
new file mode 100755
index 0000000..1c1e2b6
--- /dev/null
+++ b/azkaban-exec-server/src/main/bash/start-executor.sh
@@ -0,0 +1,7 @@
+#!/bin/bash
+
+script_dir=$(dirname $0)
+
+# pass along command line arguments to the internal launch script.
+${script_dir}/internal/internal-start-executor.sh "$@" >executorServerLog__`date +%F+%T`.out 2>&1 &
+
diff --git a/azkaban-solo-server/src/main/bash/shutdown-solo.sh b/azkaban-solo-server/src/main/bash/shutdown-solo.sh
new file mode 100755
index 0000000..ab0c215
--- /dev/null
+++ b/azkaban-solo-server/src/main/bash/shutdown-solo.sh
@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# Shutdown script for the azkaban solo server
+set -o nounset
+
+script_dir=$(dirname $0)
+base_dir="${script_dir}/.."
+source "${script_dir}/internal/util.sh"
+common_shutdown "solo-server" ${base_dir}
diff --git a/azkaban-solo-server/src/main/bash/start-solo.sh b/azkaban-solo-server/src/main/bash/start-solo.sh
new file mode 100755
index 0000000..3ca2142
--- /dev/null
+++ b/azkaban-solo-server/src/main/bash/start-solo.sh
@@ -0,0 +1,5 @@
+#!/bin/bash
+
+script_dir=$(dirname $0)
+
+${script_dir}/internal/internal-start-solo-server.sh "$@" > soloServerLog__`date +%F+%T`.out 2>&1 &
diff --git a/azkaban-web-server/src/main/bash/shutdown-web.sh b/azkaban-web-server/src/main/bash/shutdown-web.sh
new file mode 100755
index 0000000..35c773b
--- /dev/null
+++ b/azkaban-web-server/src/main/bash/shutdown-web.sh
@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+# Shutdown script for azkaban web server
+set -o nounset
+
+script_dir=$(dirname $0)
+base_dir="${script_dir}/.."
+source "${script_dir}/internal/util.sh"
+common_shutdown "web-server" ${base_dir}
diff --git a/azkaban-web-server/src/main/bash/start-web.sh b/azkaban-web-server/src/main/bash/start-web.sh
index b02a5b5..cca1781 100755
--- a/azkaban-web-server/src/main/bash/start-web.sh
+++ b/azkaban-web-server/src/main/bash/start-web.sh
@@ -1,5 +1,5 @@
#!/bin/bash
-base_dir=$(dirname $0)/..
+script_dir=$(dirname $0)
-$base_dir/bin/azkaban-web-start.sh $base_dir >$base_dir/logs/webServerLog_`date +%F+%T`.out 2>&1 &
+${script_dir}/internal/internal-start-web.sh >webServerLog_`date +%F+%T`.out 2>&1 &
