<chapter id="applicationClustering">
    <title>Application Clustering</title>

    <para>This chapter is focused on clustering support for your own AS7, EAP6 or Wildfly applications, which are secured by Keycloak.
        We support various deployment scenarios according if your application is:
        <itemizedlist>
            <listitem>
                <para>
                    stateless or stateful
                </para>
            </listitem>
            <listitem>
                <para>
                    distributable (replicated http session) or non-distributable and just relying on sticky sessions provided by loadbalancer
                </para>
            </listitem>
            <listitem>
                <para>
                    deployed on same or different cluster hosts where keycloak servers are deployed
                </para>
            </listitem>
        </itemizedlist>
    </para>

    <para>
        The situation is a bit tricky as application communicates with Keycloak directly within user's browser (for example redirecting to login screen),
        but there is also backend (out-of-bound) communication between keycloak and application, which is hidden from end-user
        and his browser and hence can't rely on sticky sessions.
    </para>

    <note>
        <para>
            To enable distributable (replicated) HTTP Sessions in your application, you may need to do some additional steps. Usually you need to put <![CDATA[<distributable />]]>
            tag into <literal>WEB-INF/web.xml</literal> file of your application and possibly do some additional steps to configure underlying cluster cache (In case of
            Wildfly, the implementation of cluster cache is based on Infinispan). These steps are server specific, so consult documentation of your application server for more details.
        </para>
    </note>

    <section id="stateless-token-store">
        <title>Stateless token store</title>
        <para>
            By default, the servlet web application secured by Keycloak uses HTTP session to store information about authenticated
            user account. This means that this info could be replicated across cluster and your application will safely survive
            failover of some cluster node.
        </para>
        <para>
            However if you don't need or don't want to use HTTP Session, you may alternatively save all info about authenticated
            account into cookie. This is useful especially if your application is:
            <itemizedlist>
                <listitem>
                    <para>
                        stateless application without need of HTTP Session, but with requirement to be safe to failover of some cluster node
                    </para>
                </listitem>
                <listitem>
                    <para>
                        stateful application, but you don't want sensitive token data to be saved in HTTP session
                    </para>
                </listitem>
                <listitem>
                    <para>
                        stateless application relying on loadbalancer, which is not aware of sticky sessions (in this case cookie is your only way)
                    </para>
                </listitem>
            </itemizedlist>
        </para>
        <para>
            To configure this, you can add this line to configuration of your adapter in <literal>WEB-INF/keycloak.json</literal> of your application:
<programlisting>
<![CDATA[
"token-store": "cookie"
]]>
</programlisting>
        </para>
        <para>
            Default value of <literal>token-store</literal> is <literal>session</literal>, hence saving data in HTTP session.
        </para>
        <para>
            One limitation of cookie store is, that whole info about account is passed in cookie KEYCLOAK_ADAPTER_STATE in each HTTP request.
            Hence it's not the best for network performance.
            Another small limitation is limited support for Single-Sign out. It works without issues if you init servlet logout (HttpServletRequest.logout)
            from this application itself as the adapter will delete the KEYCLOAK_ADAPTER_STATE cookie. But back-channel logout initialized from different application can't be
            propagated by Keycloak to this application with cookie store. Hence it's recommended to use very short value of access token
            timeout (1 minute for example).
        </para>
    </section>

    <section id="relative-uri-optimization">
        <title>Relative URI optimization</title>
        <para>
            In many deployment scenarios will be Keycloak and secured applications deployed on same cluster hosts. For this case Keycloak
            already provides option to use relative URI as value of option <emphasis>auth-server-url</emphasis> in <literal>WEB-INF/keycloak.json</literal> .
            In this case, the URI of Keycloak server is resolved from the URI of current request.
        </para>
        <para>
            For example if your loadbalancer is on <emphasis>https://loadbalancer.com/myapp</emphasis> and auth-server-url is <emphasis>/auth</emphasis>,
            then relative URI of Keycloak is resolved to be  <emphasis>https://loadbalancer.com/auth</emphasis> .
        </para>
        <para>
            For cluster setup, it may be even better to use option <emphasis>auth-server-url-for-backend-request</emphasis> . This allows to configure
            that backend requests between Keycloak and your application will be sent directly to same cluster host without additional
            round-trip through loadbalancer. So for this, it's good to configure values in <literal>WEB-INF/keycloak.json</literal> like this:
<programlisting>
<![CDATA[
"auth-server-url": "/auth",
"auth-server-url-for-backend-requests": "http://${jboss.host.name}:8080/auth"
]]>
</programlisting>
        </para>
        <para>
            This would mean that browser requests (like redirecting to Keycloak login screen) will be still resolved relatively
            to current request URI like <emphasis>https://loadbalancer.com/myapp</emphasis>, but backend (out-of-bound) requests between keycloak
            and your app are sent always to same cluster host with application .
        </para>
        <para>
            Note that additionally to network optimization,
            you may not need "https" in this case as application and keycloak are communicating directly within same cluster host.
        </para>
    </section>

    <section id="admin-url-configuration">
        <title>Admin URL configuration</title>
            <para>
                Admin URL for particular application can be configured in Keycloak admin console. It's used by Keycloak server to
                send backend requests to application for various tasks, like logout users or push revocation policies.
            </para>
            <para>
                For example logout of user from Keycloak works like this:
                <orderedlist>
                    <listitem>
                        <para>
                            User sends logout request from one of applications where he is logged.
                        </para>
                    </listitem>
                    <listitem>
                        <para>
                            Then application will send logout request to Keycloak
                        </para>
                    </listitem>
                    <listitem>
                        <para>
                            Keycloak server logout user in itself, and then it re-sends logout request by backend channel to all
                            applications where user is logged. Keycloak is using admin URL for this. So logout is propagated to all apps.
                        </para>
                    </listitem>
                </orderedlist>
            </para>
            <para>
                You may again use relative values for admin URL, but in cluster it may not be the best similarly like in <link linkend='relative-uri-optimization'>previous section</link> .
            </para>
            <para>
                Some examples of possible values of admin URL are:
                <variablelist>
                    <varlistentry>
                        <term>http://${jboss.host.name}:8080/myapp</term>
                        <listitem>
                            <para>
                                This is best choice if "myapp" is deployed on same cluster hosts like Keycloak and is distributable.
                                In this case Keycloak server sends logout request to itself, hence no communication with loadbalancer
                                or other cluster nodes and no additional network traffic.
                            </para>
                            <para>
                                Note that since the application is distributable,
                                the backend request sent by Keycloak could be served on any application cluster node as invalidation
                                of HTTP Session on <emphasis>node1</emphasis> will propagate the invalidation to other cluster nodes due to replicated HTTP sessions.
                            </para>
                        </listitem>
                    </varlistentry>
                    <varlistentry>
                        <term>http://${application.session.host}:8080/myapp</term>
                        <listitem>
                            <para>
                                Keycloak will track hosts where is particular HTTP Session served and it will send session
                                invalidation message to proper cluster node.
                            </para>
                            <para>
                                For example application is deployed on <emphasis>http://node1:8080/myapp</emphasis> and <emphasis>http://node2:8080/myapp</emphasis> .
                                Now HTTP Session <emphasis>session1</emphasis> is sticky-session served on cluster node <emphasis>node2</emphasis> .
                                When keycloak invalidates this session, it will send request directly to <emphasis>http://node2:8080/myapp</emphasis> .
                            </para>
                            <para>
                                This is ideal configuration for distributable applications deployed on different host than keycloak
                                or for non-distributable applications deployed either on same or different nodes than keycloak.
                                Good thing is that it doesn't send requests through load-balancer and hence helps to reduce network traffic.
                            </para>
                        </listitem>
                    </varlistentry>
                </variablelist>
            </para>
    </section>

    <section id="registration-app-nodes">
        <title>Registration of application nodes to Keycloak</title>
        <para>
            Previous section describes how can Keycloak send logout request to proper application node. However in some cases admin
            may want to propagate admin tasks to all registered cluster nodes, not just one of them. For example push new notBefore
            for realm or application, or logout all users from all applications on all cluster nodes.
        </para>
        <para>
            In this case Keycloak should
            be aware of all application cluster nodes, so it could send event to all of them. To achieve this, we support auto-discovery mechanism:
            <orderedlist>
                <listitem>
                    <para>
                        Once new application node joins cluster, it sends registration request to Keycloak server
                    </para>
                </listitem>
                <listitem>
                    <para>
                        The request may be re-sent to Keycloak in configured periodic intervals
                    </para>
                </listitem>
                <listitem>
                    <para>
                        If Keycloak won't receive re-registration request within specified timeout (should be greater than period from point 2)
                        then it automatically unregister particular node
                    </para>
                </listitem>
                <listitem>
                    <para>
                        Node is also unregistered in Keycloak when it sends unregistration request, which is usually during node
                        shutdown or application undeployment. This may not work properly for forced shutdown when
                        undeployment listeners are not invoked, so here you need to rely on automatic unregistration from point 3 .
                    </para>
                </listitem>
            </orderedlist>
        </para>
        <para>
                Sending startup registrations and periodic re-registration is disabled by default, as it's main usecase is just
                cluster deployment. In <literal>WEB-INF/keycloak.json</literal> of your application, you can specify:
<programlisting>
<![CDATA[
"register-node-at-startup": true,
"register-node-period": 600,
]]>
</programlisting>
                which means that registration is sent at startup (accurately when 1st request is served by the application node)
                and then it's resent each 10 minutes.
        </para>
        <para>
            In Keycloak admin console you can specify the maximum node re-registration timeout (makes sense to have it
            bigger than <emphasis>register-node-period</emphasis> from adapter configuration for particular application). Also you
            can manually add and remove cluster nodes in admin console, which is useful if you don't want to rely on adapter's
            automatic registration or if you want to remove stale application nodes, which weren't unregistered
            (for example due to forced shutdown).
        </para>
    </section>

    <section id="refresh-token-each-req">
        <title>Refresh token in each request</title>
        <para>
            By default, application adapter tries to refresh access token when it's expired (period can be specified as <link linkend='token-timeouts'>Access Token Lifespan</link>) .
            However if you don't want to rely on the fact, that Keycloak is able to successfully propagate admin events like logout
            to your application nodes, then you have possibility to configure adapter to refresh access token in each HTTP request.
        </para>
        <para>
            In <literal>WEB-INF/keycloak.json</literal> you can configure:
<programlisting>
<![CDATA[
"always-refresh-token": true
]]>
</programlisting>
        </para>
        <para>
            Note that this has big performance impact. It's useful just if performance is not priority, but security is critical
            and you can't rely on logout and push notBefore propagation from Keycloak to applications.
        </para>
    </section>
</chapter>