8/15/2017 6:45:11 PM
the refactor before the actual changes.
|
8/15/2017 6:39:31 PM
local storage uses hash to create a filename. With the current code it generates weird characters in the file name. Updating the logic to the one similar with `HdfsStorage`. This fixes the filename issue.
Example old filename: 1/1-���ΰ�LJ�.����*�.zip
Example new filename: 1/1-c5e616ceb09bc787e52eb99bab8d2a8c.zip
|
8/15/2017 6:08:19 PM
API info. The pid is read from the pid file which is configurable via `azkaban.pid.filename`. By default this value is `currentpid`.
Example:
```
{ "version": "unknown", "pid": "69041", ...
}
```
|
8/14/2017 6:57:08 PM
PR #1325.
Instead of simply displaying an error message when assignUserDirOwnership fails, now it throws an Exception using the handleError method.
This is better because if the flow fails later due to unexpected permissions for the user then it would be hard to detect that the failure of assignUserDirOwnership is the root cause. Now the flow will fail quickly and be easier to diagnose.
|
8/11/2017 6:11:31 PM
now, Azkaban leverages the execute-as-user script so that users' jobs are run via a process of the user's creation instead of as the Azkaban process. This is really important because it restricts users from interfering with Azkaban or reading secure information that Azkaban holds.
While this script works well, it has a bug in which it doesn't remove the "azkaban" group from the users list of groups. This is not secure. It did have a side-effect though that should be kept: it allowed users to write to their /execution/<exec_id>/src directory. This is desired because it gives users the ability to cache temporary data on the executor between jobs in a flow.
This PR does two things:
It fixes the security bug in execute-as-user.c by calling the initgroups() function for the effective user.
It uses the execute-as-user executable to set up the /execution/<exec_id>/src directory so that its permissions allow the user to work within it . It also makes sure that this is done in a way that can be cleaned up later by the Azkaban thread. It does this by executing the command: chown <user>:azkaban
This particular change will be hard to test with unit testing, but it will easy to test within the integration testing suite (I discussed this with @jamiesjc). It can be done completely with two test:
The first one tries to write to the current working directory. If the flow concludes successfully (i.e. the file is successfully created), the test passes.
The second tries to perform an "improper" action reserved for members of the "azkaban" group. If the flow fails (i.e. the action cannot be performed), the test passes.
Together I believe this will give sufficient test coverage for this part of the code and each test should be quick (one job flows that should terminate in < 1 second)
|
8/11/2017 4:33:11 PM
plugin
2. make duplicate log construction logic an util method
|
8/11/2017 11:22:08 AM
delete API based on a key. The key is the relative path of the project file and is stored as `resourceId` in the DB.
|
8/10/2017 9:43:54 PM
ExecutorManager class (#1324)
|
8/10/2017 8:04:13 PM
trigger (#1328)
Previously TriggerManager won't persist updated schedule when schedule gets updated(e.x. add new SLA rule to a schedule). It gets persisted only when schedule gets triggered. So if we add a SLA rule to a schedule or do a reschedule, and restart the web server after it, these updates will be lost.
|
|
|
