10/31/2018 9:25:46 PM
(#2012)
To avoid possible race condition caused by two threads (FlowRunnerManager#Cleaner and FlowPreparer#setup), and to make project directory clean up logic simpler, cleaning up projects of old version method is removed. Project dir deletion will be handled in a single thread in FlowPreparer#setup.
a race condition example:
Old Executor(initial state: active) New Executor(initial state: inactive) start to delete the project of old version
set to inactive set to active, start to load the project dirs into in-memory list load project dir 1.1 into memory
delete project dir 1.1
deletion completes loading completes then new executor's in-memory active project list could contain a project dir deleted by the old executor.
|
10/31/2018 12:43:52 AM
the assumptions
|
10/31/2018 12:41:52 AM
but not both (#2006)
We ran into a bug in the production environment. Users' jars cannot override default pig libraries. It was found that AZ always adds pig additional jar if pig job type defines it under plugin.properties. The fix is straightforward. If a user defines the property, we use it directly by not including system default settings.
|
|
|
10/25/2018 11:40:03 PM
3.60.0
in shutting down mode when its shutdown API is called. Executor will wait for all the ongoing flows to complete then shut itself down. Web server talks to executor periodically to fetch the status of each flow in its running flow cache. This PR accommodates a grace period between completion of all flows and killing the executor process, in order to allow web server to finalize flow status updating.
|
|
|
|
|
10/24/2018 12:40:07 PM
server first starts executor server, then web server.
If web server startup failed, the single server process was left running, making it harder to realize that there was an issue in startup.
This PR makes it easier to spot a startup error by stopping JVM in case of a startup error.
Without this change, the ERROR from web startup is buried in an earlier phase of the logs. The JVM keeps running despite an exception thrown by web start call because executor server's thread remains alive. As the executor server has already successfully started, it will keep logging some periodical stuff into the log file, making it harder to find out what was the last thing that happened on the web server's side.
|
|
|
|
|
