5/9/2017 7:32:08 PM
for Azkaban.
Setting up requires the following configuration.
```
# Set the storage type to HDFS
azkaban.storage.type=HDFS
# Tell Azkaban where to find Hadoop Configuration
hadoop.conf.dir.path=/path/to/hadoop/etc/hadoop
# All projects are stored inside the root directory in the format
# <project_id>/<project_id>_<hex(md5-hash-of-file)>.zip
azkaban.storage.hdfs.root.uri=hdfs://hadoop-name-node:9000/tmp/azkaban/root
# For Kerberized Hadoop Cluster
azkaban.keytab.path=/path/to/keytab.file
azkaban.kerberos.principal=user/host@REALM
```
Local Storage bug fix.
Duplicate file in storage just logs the info and not raise an exception.
|
5/9/2017 1:15:46 AM
to [Google style](https://github.com/google/styleguide). No code has been reformatted yet.
Instructions for setup are available here:
https://github.com/HPI-Information-Systems/Metanome/wiki/Installing-the-google-styleguide-settings-in-intellij-and-eclipse
|
|
|
5/8/2017 8:35:23 PM
string for H2 DB needs to be an absolute path.
Has to make the change in two different places since the data source
is currently configured in two places.
Testing:
Tested manually the solo server with the following config:
```
# h2 db
database.type=h2
h2.path=data/azkaban
h2.create.tables=true
```
|
5/8/2017 6:18:21 PM
3.23.0
finis… (#1062)
* Fix bugs in resubmitting flow and fetching flow logs after flow finishes.
After removing runningFlows cache from web server, running flow info will be fetched from DB directly by ExecutorManager. This includes the flow updateTime which is updated by FlowRunnerManager when the flow finishes. Currently the UpdaterThread in ExecutorManager will send update request to executor to get updated flow info. UpdateTime now is synced between web server and executor since they all update and get from DB instead of cache, the previous logic of comparing updateTime should be changed.
TODO: updaterThread in executorManager should be removed in the future to simplify the logic. handleAjaxUpdateRequest() should be deprecated as well.
|
5/8/2017 6:01:34 PM
warning:
```
The Task.leftShift(Closure) method has been deprecated and is scheduled to be removed in Gradle 5.0. Please use Task.doLast(Action) instead. at build_8nu1cdi19xmcy93ijoksjwh7o.run(/Users/ruyang/oss/azkaban/azkaban-db/build.gradle:38)
```
|
5/8/2017 4:54:58 PM
specified the all distribution type.
The other files are generated by ./gradlew wrapper
|
5/8/2017 4:50:51 PM
library brings in an older version of junit which causes
Intellij to pick this version somehow and caused compilation error
when building using Intellij.
|
5/8/2017 3:16:01 PM
(#1057)
Changes: - refactored Storage API to throw IOException - extracted hash computation code to StorageManager - Changed ProjectLoader API to accept hash as parameter
|
5/5/2017 6:05:05 PM
patch is make a replacement to TriggerJDBCLoader class, in order to
try out new Azkaban-db interface/code. Fix the broken JDBC Test class
, and add in-memory database to test. Fix getLstInsertId method inside
akzban-db module.
Also, some other db related refactor is commited in order to allow test
work.
|
