azkaban-aplcache

Changes

docs/jobTypes.rst 75(+26 -49)

Details

docs/jobTypes.rst 75(+26 -49)

diff --git a/docs/jobTypes.rst b/docs/jobTypes.rst
index d4fdda9..754b212 100644
--- a/docs/jobTypes.rst
+++ b/docs/jobTypes.rst
@@ -24,10 +24,8 @@ The command job type is one of the basic built-in types. It runs
 multiple UNIX commands using java processbuilder. Upon execution,
 Azkaban spawns off a process to run the command.
 
-*****
 How To Use
-*****
-
+~~~~~~~~~~
 One can run one or multiple commands within one command job. Here is
 what is needed:
 
@@ -43,9 +41,8 @@ For multiple commands, do it like ``command.1, command.2``, etc.
 
    <div class="bs-callout bs-callout-info">
 
-*****
 Sample Job Package
-*****
+~~~~~~~~~~~~~~~~~~
 
 Here is a sample job package, just to show how it works:
 
@@ -74,9 +71,8 @@ multiple UNIX commands using java processbuilder. Upon execution,
 Azkaban spawns off a process to run the command.
 
 
-*****
 How To Use
-*****
+~~~~~~~~~~
 
 The ``HadoopShell`` job type talks to a secure cluster via Hadoop
 tokens. The admin should specify ``obtain.binary.token=true`` if the
@@ -170,9 +166,8 @@ talks to Hadoop clusters. That usage should be replaced by
 ``hadoopJava`` type now, which is secure. But we still keep ``java``
 type in the plugins for backwards compatibility.
 
-*****
 How to Use
-*****
+~~~~~~~~~~
 
 Azkaban spawns a local process for the java job type that runs user
 programs. It is different from the "javaprocess" job type in that
@@ -189,9 +184,8 @@ For the most part, using ``java`` type should be no different from
 
    <div class="bs-callout bs-callout-info">
 
-*****
 Sample Job
-*****
+~~~~~~~~~~
 
 Please refer to the  `hadoopJava type <#hadoopjava-type>`_.
 
@@ -211,9 +205,8 @@ ability to talk to a Hadoop cluster securely, via Hadoop tokens. Most
 Hadoop job types can be created by running a hadoopJava job, such as
 Pig, Hive, etc.
 
-*****
 How To Use
-*****
+~~~~~~~~~~
 
 
 The ``hadoopJava`` type runs user java program after all. Upon
@@ -338,9 +331,8 @@ should also make those naming public and consistent.
 .. raw:: html
 
    <div class="bs-callout bs-callout-info">
-*****
 Sample Job Package
-*****
+~~~~~~~~~~~~~~~~~~
 
 Here is a sample job package that does a word count. It relies on a Pig
 job to first upload the text file onto HDFS. One can also manually
@@ -379,9 +371,8 @@ company specific udfs registered and name space imported, so that the
 users don't need to provide all the jars and do the configurations in
 their specific Pig job conf files.
 
-*****
 How to Use
-*****
+~~~~~~~~~~
 
 
 The Pig job runs user Pig scripts. It is important to remember, however,
@@ -516,9 +507,8 @@ most cases will only need Pig scripts in their Azkaban job packages.
 
    <div class="bs-callout bs-callout-info">
 
-*****
 Sample Job Package
-*****
+~~~~~~~~~~~~~~~~~~
 
 
 Here is a sample job package that does word count. It assumes you have
@@ -551,9 +541,8 @@ important that individual MR step inside a single Pig script doesn't
 cancel the tokens upon its completion. Otherwise, all following steps
 will fail on authentication with the JobTracker or NameNode.
 
-*****
 How to Use
-*****
+~~~~~~~~~~
 
 The Hive job runs user Hive queries. The Hive job type talks to a secure
 cluster via Hadoop tokens. The admin should specify
@@ -833,9 +822,8 @@ Common Configurations
 This section lists out the configurations that are common to all job
 types
 
-*****
 other_namenodes
-*****
+~~~~~~~~~~~~~~~
 
 
 This job property is useful for jobs that need to read data from or
@@ -850,9 +838,8 @@ NameNode URLs.
 
 For example: **other_namenodes=webhdfs://host1:50070,hdfs://host2:9000**
 
-*****
 HTTP Job Callback
-*****
+~~~~~~~~~~~~~~~~~
 
 
 The purpose of this feature to allow Azkaban to notify external systems
@@ -864,9 +851,8 @@ following format:
 -  **job.notification.<status>.<sequence number>.body**
 -  **job.notification.<status>.<sequence number>.headers**
 
-*****
 Supported values for **status**
-*****
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 
 -  **started**: when a job is started
@@ -874,26 +860,23 @@ Supported values for **status**
 -  **failure**: when a job failed
 -  **completed**: when a job is either successfully completed or failed
 
-*****
 Number of callback URLs
-*****
+~~~~~~~~~~~~~~~~~~~~~~~
 
 
 The maximum # of callback URLs per job is 3. So the <sequence number>
 can go up from 1 to 3. If a gap is detected, only the ones before the
 gap is used.
 
-*****
 HTTP Method
-*****
+~~~~~~~~~~~
 
 
 The supported method are **GET** and **POST**. The default method is
 **GET**
 
-*****
 Headers
-*****
+~~~~~~~
 
 
 Each job callback URL can optional specify headers in the following
@@ -906,9 +889,8 @@ name and value is ':'
 
 The headers are applicable for both GET and POST job callback URLs.
 
-*****
 Job Context Information
-*****
+~~~~~~~~~~~~~~~~~~~~~~~
 
 
 It is often desirable to include some dynamic context information about
@@ -927,9 +909,9 @@ value of each token will be HTTP encoded.
 
 The value of these tokens will be HTTP encoded if they are on the URL,
 but will not be encoded when they are in the HTTP body.
-*****
+
 Examples
-*****
+~~~~~~~~
 
 
 GET HTTP Method
@@ -960,9 +942,8 @@ of jars and dependencies and keep them up-to-date. We created the
 dependencies are now managed by admins; absolutely no jars or java code
 are required from users.
 
-*****
 How to Use
-*****
+~~~~~~~~~~
 
 
 This is essentially a hadoopJava job, with all jars controlled by the
@@ -1062,9 +1043,8 @@ These go into ``plugin.properties``:
 
    <div class="bs-callout bs-callout-info">
 
-*****
 For more information
-*****
+~~~~~~~~~~~~~~~~~~~~
 
 
 Please refer to `Voldemort project
@@ -1088,9 +1068,8 @@ any job under the same Azkaban work flow management and scheduling.
 Creating new job types is often times very easy. Here are several ways
 one can do it:
 
-*****
 New Types with only Configuration Changes
-*****
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 
 One doesn't always need to write java code to create job types for end
@@ -1109,9 +1088,8 @@ pig/Azkaban users.
 The same practice applies to most other job types. Admins should create
 or tailor job types to their specific company needs or clusters.
 
-*****
 New Types Using Existing Job Types
-*****
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 
 If one needs to create a different job type, a good starting point is to
@@ -1131,9 +1109,8 @@ voldemort stores.
 
 The same applies to the hive type.
 
-*****
 New Types by Extending Existing Ones
-*****
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
 For the most flexibility, one can always build new types by extending
 the existing ones. Azkaban uses reflection to load job types that
@@ -1169,9 +1146,8 @@ To enable this feature, add the following property
 Property "executor.metric.milisecinterval.default" controls the interval
 at which the metrics are collected at
 
-*****
 Statistic Types
-*****
+~~~~~~~~~~~~~~~
 
 
 +----------------------+------------------------------+
@@ -1202,6 +1178,7 @@ are available
 -  To disable the statistic collection - /stats?action=disableMetrics
 
 --------------
+
 *****
 Reload Jobtypes
 *****