2/16/2018 9:15:28 PM
changes:
* Originally we use getFifteenMinuteRate to get transform meter to gauge. 15 mins was too large, and we'd better change it to 1 minute window to reduce smoothing effect
* Adding database transaction metrics if database throws an exception during the query phase.
Tested in the staging environment.
|
2/16/2018 8:49:32 PM
a series of enhancement to FlowTriggerService:
handle the case where flow trigger without dependencies is a regular schedule without waiting for data, so the trigger instance will be processed as succeed once created.
removes some extra log to be conciser.
replace removeRunningTriggerInstById method with this.runningTriggers.remove()
increase the unit test coverage of flow trigger service.
|
2/16/2018 7:28:44 PM
class around QuartzScheduler - FlowTriggerScheduler.
The class provides
schedule all flows in a project with flow trigger defined with quarz scheduler.
unscheduleAll all flows in a project with flow trigger defined with quartz scheduler.
Retrieve the list of current flow schedules from quartz database.
FlowTriggerScheduler will be used to schedule/unschedule flows if schedule is part of flow configuration file(flow 2.0) when project is uploaded/deleted.
Also when flow trigger is defined, thus schedule is part of flow config, scheduling those flows from UI/API is disallowed.
Tested via manual uploading a project with flow trigger but unit test of this part is a bit tricky, will add unit test if possible or verify in integration test.
|
|
|
2/12/2018 4:38:53 PM
3.42.0
instance servlet and associated template.
The page consists of currently running trigger instances and recently finished one.
|
2/8/2018 4:25:38 PM
schedules. (#1623)
* Fixed the UI display issue of same executionOptions for all schedules.
* Use gson library instead and simplify the code.
* Simplify code.
|
2/7/2018 9:32:36 PM
(#1632)
The motivation of moving dependency plugin interface code(#1609, #1610) out from azkaban web server module to a new module is to expose minimum set of code to external dependency plugin avoiding unnecessary introduction of other classes in the development of dependency plugin. If those interface sit in the azkaban web server module, dependency plugin developers will have to depend on jar of that module containing a lot of other classes more than needed. This might potentially cause dependency conflict.
This PR also includes bettering renaming on test dependency type and consolidating different test dependency types into one package.
|
2/7/2018 9:06:13 PM
PR injects flow trigger service(#1627) into azkaban web server. Flow trigger service will start if enabled in the config file. Upon start, flow trigger service initializes all dependency plugins and recover running trigger instance from last run status.
|
2/6/2018 5:05:25 PM
section in flow yaml file (#1630)
The following validation steps are performed against flow trigger when parsing flow yaml file:
if specified max wait min must >= 1, if not, allowed max wait min(10 days) is applied.
flow trigger schedule cannot be null
flow trigger schedule type must be cron
flow trigger schedule value must be a valid cron expression
flow trigger schedule section contains type and value only
dependency name must be unique at flow trigger level
dependency config(type + params) must be unique at flow trigger level.
if max wait min >= allowed max wait min, it will be automatically set to allowed one(10 days)
dependency name/type are required.
|
2/6/2018 4:33:41 PM
mainly a refactor, in order to fix JDBC intermittent test failure reported in #1614. Looks like JdbcTriggerImplTest uses a deprecated class to construct h2 test db. The fix is to replace it with the one which all XXXDaoTest currently are depending on.
Tested by ./gradlew :azkaban-common:test
|
