2/7/2018 9:32:36 PM
(#1632)
The motivation of moving dependency plugin interface code(#1609, #1610) out from azkaban web server module to a new module is to expose minimum set of code to external dependency plugin avoiding unnecessary introduction of other classes in the development of dependency plugin. If those interface sit in the azkaban web server module, dependency plugin developers will have to depend on jar of that module containing a lot of other classes more than needed. This might potentially cause dependency conflict.
This PR also includes bettering renaming on test dependency type and consolidating different test dependency types into one package.
|
2/7/2018 9:06:13 PM
PR injects flow trigger service(#1627) into azkaban web server. Flow trigger service will start if enabled in the config file. Upon start, flow trigger service initializes all dependency plugins and recover running trigger instance from last run status.
|
2/6/2018 5:05:25 PM
section in flow yaml file (#1630)
The following validation steps are performed against flow trigger when parsing flow yaml file:
if specified max wait min must >= 1, if not, allowed max wait min(10 days) is applied.
flow trigger schedule cannot be null
flow trigger schedule type must be cron
flow trigger schedule value must be a valid cron expression
flow trigger schedule section contains type and value only
dependency name must be unique at flow trigger level
dependency config(type + params) must be unique at flow trigger level.
if max wait min >= allowed max wait min, it will be automatically set to allowed one(10 days)
dependency name/type are required.
|
2/6/2018 4:33:41 PM
mainly a refactor, in order to fix JDBC intermittent test failure reported in #1614. Looks like JdbcTriggerImplTest uses a deprecated class to construct h2 test db. The fix is to replace it with the one which all XXXDaoTest currently are depending on.
Tested by ./gradlew :azkaban-common:test
|
2/6/2018 3:06:33 AM
a singleton class in the AZ web server to process all flow trigger-related operations. Externally it provides following operations -
Create a trigger instance based on trigger definition.
Cancel a trigger instance.
Query running and historic trigger instances.
Recover incomplete trigger instances.
Internally, it
maintains the list of running trigger instance in memory.
updates status, starttime/endtime of trigger instance.
persists trigger instance to DB.
FlowTriggerService will be leveraged by Quartz scheduler, our new AZ scheduler to schedule starting triggers.
Flow Trigger Service internally contains a single-threaded executor service to process all trigger instance related operation to avoid complicated cases of race condition.
|
1/31/2018 10:44:39 PM
always get confused when they read
>See logs at: https://blahblahXXX://
This PR adds another log message before this message, in order to instruct users how to set job ELK logger.
|
1/31/2018 9:15:44 PM
of the plugin management module for flow trigger dependency type.
We implemented data dependency interface and keep interface code inside main Azkaban open source repo and allow both internal and OSS developers to implement the interface for their own dependency type as plugin.
Plugin management module loads plugin property files from plugin directory and created corresponding DependencyCheck objects based on dependency plugin properties upon plugin management initialization or upon request(when doing plugin deployment).
|
1/29/2018 10:24:23 PM
3.41.0
(#1619)
This PR added two classes to handle status change of trigger instance and dependency instance:
DependencyInstanceProcess/TriggerInstanceProcessor, handling the work post status update. E.x, execute a flow when trigger instance becomes successful, shoot an email when trigger instance is cancelled, persist status update into database.
|
1/25/2018 10:34:38 PM
for flow trigger. The loader provides all DB related operations for flow trigger instance and dependency instance(#1611) to interact with dependency instance table(#1612).
The PR also added toString(), hashCode() and equals() to several existing classes for complementary purpose.
|
1/23/2018 10:01:23 PM
to create table to keep execution of all dependency instances. There will be no table specifically for trigger instance since trigger instance is nothing but the collection of dependency instances. So trigger instance can be easily constructed from dependency instances. The table will be used to keep the historic executions for users to query in the UI and to recover the running triggers from web server restart.
|
