Skip to content

ipfailover - control preempt strategy #4674

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 12, 2017
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 75 additions & 17 deletions admin_guide/high_availability.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ unavailable or if the service that *Keepalived* is watching does not respond,
the VIP is switched to another host from the set. Thus, a VIP is always serviced
as long as a host is available.

When a host running *Keepalived* passes the *check* script it becomes available to
become the *MASTER* based on its priority and the priority of the current *MASTER*.
The configurable xref:vrrp-preemption[preemption strategy] determines what to do.

The administrator can provide a script via the `--notify-script=` option, which
is called whenever the state changes. *Keepalived* is in *MASTER* state when it
is servicing the VIP, in *BACKUP* state when another node is servicing the VIP,
Expand All @@ -41,7 +45,7 @@ xref:check-notify[*notify* script] is called with the new state whenever the
state changes.

{product-title} supports creation of IP failover deployment configuration, by
running the `oadm ipfailover` command. The IP failover deployment configuration
running the `oc adm ipfailover` command. The IP failover deployment configuration
specifies the set of VIP addresses, and the set of nodes on which to service
them. A cluster can have multiple IP failover deployment configurations, with
each managing its own set of unique VIP addresses. Each node in the IP failover
Expand Down Expand Up @@ -204,7 +208,7 @@ $ ping hello-openshift.shard1.v3.rhcloud.com
[[configuring-ip-failover]]
== Configuring IP Failover

Use the `oadm ipfailover` command with suitable xref:../admin_guide/high_availability.adoc#options-environment-variables[options], to create ipfailover deployment configuration.
Use the `oc adm ipfailover` command with suitable xref:../admin_guide/high_availability.adoc#options-environment-variables[options], to create ipfailover deployment configuration.

[IMPORTANT]
====
Expand All @@ -214,13 +218,13 @@ link:http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/elb-g
the AWS console].
====

As an administrator, you can configure ipfailover on an entire cluster, or on a subset of nodes, as defined by the label selector. You can also configure multiple IP failover deployment configurations in your cluster, where each one is independent of the others. The `oadm ipfailover` command creates a ipfailover deployment confiugration which ensures that a failover pod runs on each of the nodes matching the constraints or the label used. This pod runs link:http://www.keepalived.org/[*Keepalived*] which uses VRRP (Virtual Router Redundancy Protocol) among all the *Keepalived* daemons to ensure that the service on the watched port is available, and if it is not, *Keepalived* will automatically float the VIPs.
As an administrator, you can configure ipfailover on an entire cluster, or on a subset of nodes, as defined by the label selector. You can also configure multiple IP failover deployment configurations in your cluster, where each one is independent of the others. The `oc adm ipfailover` command creates a ipfailover deployment confiugration which ensures that a failover pod runs on each of the nodes matching the constraints or the label used. This pod runs link:http://www.keepalived.org/[*Keepalived*] which uses VRRP (Virtual Router Redundancy Protocol) among all the *Keepalived* daemons to ensure that the service on the watched port is available, and if it is not, *Keepalived* will automatically float the VIPs.

For production use, make sure to use a `--selector=<label>` with at least two nodes to select
the nodes. Also, set a `--replicas=<n>` value that matches the
number of nodes for the given labeled selector.

The `oadm ipfailover` command includes command line options that set environment
The `oc adm ipfailover` command includes command line options that set environment
variables that control *Keepalived*. The
xref:../admin_guide/high_availability.adoc#options-environment-variables[environment variables]
start with `OPENSHIFT_HA_*` and they can be changed as needed.
Expand All @@ -229,7 +233,9 @@ For example, the command below will create an IP failover configuration on a sel
listening on port 80, such as the router process).

----
$ oadm ipfailover --selector="router=us-west-ha" --virtual-ips="1.2.3.4,10.1.1.100-104,5.6.7.8" --watch-port=80 --replicas=4 --create
$ oc adm ipfailover --selector="router=us-west-ha" \
--virtual-ips="1.2.3.4,10.1.1.100-104,5.6.7.8" \
--watch-port=80 --replicas=4 --create
----


Expand All @@ -238,7 +244,7 @@ You can view what the configuration configuration that would look like
using one of the supported formats (the example below uses the JSON format):

----
$ oadm ipfailover [<Ip_failover_config_name>] <options> -o json
$ oc adm ipfailover [<Ip_failover_config_name>] <options> -o json
----

==== ipfailover command options (subset)
Expand Down Expand Up @@ -284,7 +290,7 @@ Each VIP in the set may end up being served by a different node.
an optional user supplied check script. For example, the script can test a
web server by issuing a request and verifying the response.

The script is provided through the `--check-script=<script>` option to the `oadm
The script is provided through the `--check-script=<script>` option to the `oc adm
ipfailover` command. The script must exit with `0` for *PASS* or `1` for *FAIL*.

By default, the check is done every two seconds, but can be changed using the
Expand Down Expand Up @@ -395,6 +401,54 @@ displayed in decimal (`493`).
+
... Save the changes and exit the editor. This restarts *ipf-ha-router*.

[[vrrp-preemption]]
=== VRRP Preemption

When a host leaves the *FAULT* state by passing the check script it can become *MASTER*.
If the new host has lower priority than the current *MASTER* it becomes a *BACKUP*. If
it has a higher priority the preemption strategy determines what to do. The *nopreempt*
strategy does not move *MASTER* from the lower priority host to the higher priority host.
With *preempt 300*, the default, *keepalived* waits the specified 300 seconds and moves
*MASTER* to the higher priority host.

Preemption can be specified when ipfailover is created through the --preemption-strategy
option or by editing the *OPENSHIFT_HA_PREEMPTION* environment variable.

.. Using `oc adm ipfailover` command:
+
[source,bash]
----
$ oc adm ipfailover --preempt-strategy=nopreempt \
...
----
+

.. Using `oc` commands:
+
[source,bash]
----
$ oc set env dc/ipf-ha-router \
--overwrite=true \
OPENSHIFT_HA_PREEMPTION=nopreempt
----
+
.. Editing the *ipf-ha-router* deployment configuration:
+
... Use `oc edit dc ipf-ha-router` to edit the router deployment configuration
with a text editor.
+
[source,yaml]
----
...
spec:
containers:
- env:
- name: OPENSHIFT_HA_PREEMPTION <1>
value: nopreempt
...
----


[[kepalived-multicast]]
=== Keepalived Multicast

Expand Down Expand Up @@ -501,6 +555,10 @@ See xref:../admin_guide/high_availability.adoc#ha-vrrp-id-offset[this discussion
|
|Full path name in the pod file system of a script that is run whenever the state changes. See xref:../admin_guide/high_availability.adoc#check-notify[this discussion] for more details.

|`--preemption-strategy`
|`OPENSHIFT_HA_PREEMPTION`
|preempt 300
| Strategy for handling a new higher priority host. See xref:../admin_guide/high_availability.adoc#vrrp-preemption[this discussion] for more details.
|===

[[ha-vrrp-id-offset]]
Expand Down Expand Up @@ -568,7 +626,7 @@ The following example runs five instances using the ipfailover service account:
ifdef::openshift-enterprise[]
====
----
$ oadm router ha-router-us-west --replicas=5 \
$ oc adm router ha-router-us-west --replicas=5 \
--selector="ha-svc-nodes=geo-us-west" \
--labels="ha-svc-nodes=geo-us-west" \
--service-account=ipfailover
Expand All @@ -579,7 +637,7 @@ endif::[]
ifdef::openshift-origin[]
====
----
$ oadm router ha-router-us-west --replicas=5 \
$ oc adm router ha-router-us-west --replicas=5 \
--selector="ha-svc-nodes=geo-us-west" \
--labels="ha-svc-nodes=geo-us-west" \
--service-account=ipfailover
Expand Down Expand Up @@ -622,7 +680,7 @@ The ipfailover command for the *router*:
ifdef::openshift-enterprise[]
====
----
$ oadm ipfailover ipf-ha-router-us-west \
$ oc adm ipfailover ipf-ha-router-us-west \
--replicas=5 --watch-port=80 \
--selector="ha-svc-nodes=geo-us-west" \
--virtual-ips="10.245.2.101-105" \
Expand All @@ -634,7 +692,7 @@ endif::[]
ifdef::openshift-origin[]
====
----
$ oadm ipfailover ipf-ha-router-us-west \
$ oc adm ipfailover ipf-ha-router-us-west \
--replicas=5 --watch-port=80 \
--selector="ha-svc-nodes=geo-us-west" \
--virtual-ips="10.245.2.101-105" \
Expand All @@ -645,7 +703,7 @@ $ oadm ipfailover ipf-ha-router-us-west \
endif::[]

+
The following is the `oadm ipfailover` command for the *geo-cache* service that is
The following is the `oc adm ipfailover` command for the *geo-cache* service that is
listening on port 9736. Since there are two `ipfailover` deployment
configurations, the `--vrrp-id-offset` must be set so that each VIP gets its own
offset. In this case, setting a value of `10` means that the
Expand All @@ -655,7 +713,7 @@ offset. In this case, setting a value of `10` means that the
ifdef::openshift-enterprise[]
====
----
$ oadm ipfailover ipf-ha-geo-cache \
$ oc adm ipfailover ipf-ha-geo-cache \
--replicas=5 --watch-port=9736 \
--selector="ha-svc-nodes=geo-us-west" \
--virtual-ips=10.245.3.101-105 \
Expand All @@ -667,7 +725,7 @@ endif::[]
ifdef::openshift-origin[]
====
----
$ oadm ipfailover ipf-ha-geo-cache \
$ oc adm ipfailover ipf-ha-geo-cache \
--replicas=5 --watch-port=9736 \
--selector="ha-svc-nodes=geo-us-west" \
--virtual-ips=10.245.3.101-105 \
Expand Down Expand Up @@ -698,7 +756,7 @@ port 32439 and the external IP address, as defined in the *postgresql-ingress*
service:
====
----
$ oadm ipfailover ipf-ha-postgresql \
$ oc adm ipfailover ipf-ha-postgresql \
--replicas=1 <1> --selector="app-type=postgresql" <2> \
--virtual-ips=10.9.54.100 <3> --watch-port=32439 <4> \
--service-account=ipfailover --create
Expand All @@ -724,7 +782,7 @@ The following example shows how to dynamically update the deployment strategy an
ifdef::openshift-enterprise[]
====
----
$ oadm ipfailover ipf-ha-router-us-west \
$ oc adm ipfailover ipf-ha-router-us-west \
--replicas=5 --watch-port=80 \
--selector="ha-svc-nodes=geo-us-west" \
--virtual-ips="10.245.2.101-105" \
Expand All @@ -735,7 +793,7 @@ endif::[]
ifdef::openshift-origin[]
====
----
$ oadm ipfailover ipf-ha-router-us-west \
$ oc adm ipfailover ipf-ha-router-us-west \
--replicas=5 --watch-port=80 \
--selector="ha-svc-nodes=geo-us-west" \
--virtual-ips="10.245.2.101-105" \
Expand Down