:compat-mode: legacy
= Rules =
////
We prefer [[ch-rules]], but older versions of asciidoc don't deal well
with that construct for chapter headings
////
anchor:ch-rules[Chapter 8, Rules]
indexterm:[Constraint,Rule]
Rules can be used to make your configuration more dynamic, allowing values to
change depending on the time or the value of a node attribute. Examples of
things rules are useful for:
* Set a higher value for <<s-resource-options,+resource-stickiness+>> during
working hours, to minimize downtime, and a lower value on weekends, to allow
resources to move to their most preferred locations when people aren't around
to notice.
* Automatically place the cluster into maintenance mode during a scheduled
maintenance window.
* Assign certain nodes and resources to a particular department via custom
node attributes and meta-attributes, and add a single location constraint
that restricts the department's resources to run only on those nodes.
Each constraint type or property set that supports rules may contain one or more
+rule+ elements specifying conditions under which the constraint or properties
take effect. Examples later in this chapter will make this clearer.
== Rule Properties ==
indexterm:[XML element,rule element]
.Attributes of a rule Element
[width="95%",cols="2m,1,<5",options="header",align="center"]
|=========================================================
|Attribute
|Default
|Description
|id
|
|A unique name for the rule (required)
indexterm:[XML attribute,id attribute,rule element]
indexterm:[XML element,rule element,id attribute]
|role
|+Started+
|The rule is in effect only when the resource is in the specified
role. Allowed values are +Started+, +Slave+, and +Master+. A rule
with +role="Master"+ cannot determine the initial location of a
clone instance and will only affect which of the active instances
will be promoted.
indexterm:[XML attribute,role attribute,rule element]
indexterm:[XML element,rule element,role attribute]
|score
|
|If this rule is used in a location constraint and evaluates to true, apply
this score to the constraint. Only one of +score+ and +score-attribute+ may be
used.
indexterm:[XML attribute,score attribute,rule element]
indexterm:[XML element,rule element,score attribute]
|score-attribute
|
|If this rule is used in a location constraint and evaluates to true, use the
value of this node attribute as the score to apply to the constraint. Only one
of +score+ and +score-attribute+ may be used.
indexterm:[XML attribute,score-attribute attribute,rule element]
indexterm:[XML element,rule element,score-attribute attribute]
|boolean-op
|+and+
|If this rule contains more than one condition, a value of +and+ specifies that
the rule evaluates to true only if all conditions are true, and a value of
+or+ specifies that the rule evaluates to true if any condition is true.
indexterm:[XML attribute,boolean-op attribute,rule element]
indexterm:[XML element,rule element,boolean-op attribute]
|=========================================================
A +rule+ element must contain one or more conditions. A condition may be an
+expression+ element, a +date_expression+ element, or another +rule+ element.
== Node Attribute Expressions ==
[[node-attribute-expressions]]
indexterm:[Rule,Node Attribute Expression]
indexterm:[XML element,expression element]
Expressions are rule conditions based on the values of node attributes.
.Attributes of an expression Element
[width="95%",cols="2m,1,<5",options="header",align="center"]
|=========================================================
|Field
|Default
|Description
|id
|
|A unique name for the expression (required)
indexterm:[XML attribute,id attribute,expression element]
indexterm:[XML element,expression element,id attribute]
|attribute
|
|The node attribute to test (required)
indexterm:[XML attribute,attribute attribute,expression element]
indexterm:[XML element,expression element,attribute attribute]
|type
|+string+
|How the node attributes should be compared. Allowed values are
+string+, +integer+, and +version+.
indexterm:[XML attribute,type attribute,expression element]
indexterm:[XML element,expression element,type attribute]
|operation
|
a|The comparison to perform (required). Allowed values:
* +lt:+ True if the node attribute value is less than the comparison value
* +gt:+ True if the node attribute value is greater than the comparison value
* +lte:+ True if the node attribute value is less than or equal to the comparison value
* +gte:+ True if the node attribute value is greater than or equal to the comparison value
* +eq:+ True if the node attribute value is equal to the comparison value
* +ne:+ True if the node attribute value is not equal to the comparison value
* +defined:+ True if the node has the named attribute
* +not_defined:+ True if the node does not have the named attribute
indexterm:[XML attribute,operation attribute,expression element]
indexterm:[XML element,expression element,operation attribute]
|value
|
|User-supplied value for comparison (required for operations other than
+defined+ and +not_defined+)
indexterm:[XML attribute,value attribute,expression element]
indexterm:[XML element,expression element,value attribute]
|value-source
|+literal+
a|How the +value+ is derived. Allowed values:
* +literal+: +value+ is a literal string to compare against
* +param+: +value+ is the name of a resource parameter to compare against (only
valid in location constraints)
* +meta+: +value+ is the name of a resource meta-attribute to compare against
(only valid in location constraints)
indexterm:[XML attribute,value-source attribute,expression element]
indexterm:[XML element,expression element,value-source attribute]
|=========================================================
[[node-attribute-expressions-special]]
In addition to custom node attributes defined by the administrator, the cluster
defines special, built-in node attributes for each node that can also be used
in rule expressions.
.Built-in Node Attributes
[width="95%",cols="1m,<5",options="header",align="center"]
|=========================================================
|Name
|Value
|#uname
|Node <<s-node-name,name>>
|#id
|Node ID
|#kind
|Node type. Possible values are +cluster+, +remote+, and +container+. Kind is
+remote+ for Pacemaker Remote nodes created with the +ocf:pacemaker:remote+
resource, and +container+ for Pacemaker Remote guest nodes and bundle nodes
|#is_dc
|"true" if this node is a Designated Controller (DC), "false" otherwise
|#cluster-name
|The value of the +cluster-name+ cluster property, if set
|#site-name
|The value of the +site-name+ node attribute, if set, otherwise identical to
+#cluster-name+
|#role
a|The role the relevant promotable clone resource has on this node. Valid only within
a rule for a location constraint for a promotable clone resource.
////
// if uncommenting, put a pipe in front of first two lines
#ra-version
The installed version of the resource agent on the node, as defined
by the +version+ attribute of the +resource-agent+ tag in the agent's
metadata. Valid only within rules controlling resource options. This can be
useful during rolling upgrades of a backward-incompatible resource agent.
'(coming in x.x.x)'
////
|=========================================================
== Date/Time Expressions ==
indexterm:[Rule,Date/Time Expression]
indexterm:[XML element,date_expression element]
Date/time expressions are rule conditions based (as the name suggests) on the
current date and time.
A +date_expression+ element may optionally contain a +date_spec+ or +duration+
element depending on the context.
.Attributes of a date_expression Element
[width="95%",cols="2m,<5",options="header",align="center"]
|=========================================================
|Field
|Description
|id
|A unique name for the expression (required)
indexterm:[XML attribute,id attribute,date_expression element]
indexterm:[XML element,date_expression element,id attribute]
|start
|A date/time conforming to the http://en.wikipedia.org/wiki/ISO_8601[ISO8601]
specification. May be used when +operation+ is +in_range+ (in which case at
least one of +start+ or +end+ must be specified) or +gt+ (in which case
+start+ is required).
indexterm:[XML attribute,start attribute,date_expression element]
indexterm:[XML element,date_expression element,start attribute]
|end
|A date/time conforming to the http://en.wikipedia.org/wiki/ISO_8601[ISO8601]
specification. May be used when +operation+ is +in_range+ (in which case at
least one of +start+ or +end+ must be specified) or +lt+ (in which case
+end+ is required).
indexterm:[XML attribute,end attribute,date_expression element]
indexterm:[XML element,date_expression element,end attribute]
|operation
a|Compares the current date/time with the start and/or end date,
depending on the context. Allowed values:
* +gt:+ True if the current date/time is after +start+
* +lt:+ True if the current date/time is before +end+
* +in_range:+ True if the current date/time is after +start+ (if specified)
and before either +end+ (if specified) or +start+ plus the value of the
+duration+ element (if one is contained in the +date_expression+)
* +date_spec:+ True if the current date/time matches the specification
given in the contained +date_spec+ element (described below)
indexterm:[XML attribute,operation attribute,date_expression element]
indexterm:[XML element,date_expression element,operation attribute]
|=========================================================
[NOTE]
======
There is no +eq+, +neq+, +gte+, or +lte+ operation, since they would be valid
only for a single second.
======
=== Date Specifications ===
indexterm:[Rule,Date/Time Expression,Date Specification]
indexterm:[XML element,date_spec element]
A +date_spec+ element is used to create a cron-like expression relating
to time. Each field can contain a single number or range. Any field not
supplied is ignored.
.Attributes of a date_spec Element
[width="95%",cols="2m,<5",options="header",align="center"]
|=========================================================
|Field
|Description
|id
|A unique name for the object (required)
indexterm:[XML attribute,id attribute,date_spec element]
indexterm:[XML element,date_spec element,id attribute]
|hours
|Allowed values: 0-23 (where 0 is midnight and 23 is 11 p.m.)
indexterm:[XML attribute,hours attribute,date_spec element]
indexterm:[XML element,date_spec element,hours attribute]
|monthdays
|Allowed values: 1-31 (depending on month and year)
indexterm:[XML attribute,monthdays attribute,date_spec element]
indexterm:[XML element,date_spec element,monthdays attribute]
|weekdays
|Allowed values: 1-7 (where 1 is Monday and 7 is Sunday)
indexterm:[XML attribute,weekdays attribute,date_spec element]
indexterm:[XML element,date_spec element,weekdays attribute]
|yeardays
|Allowed values: 1-366 (depending on the year)
indexterm:[XML attribute,yeardays attribute,date_spec element]
indexterm:[XML element,date_spec element,yeardays attribute]
|months
|Allowed values: 1-12
indexterm:[XML attribute,months attribute,date_spec element]
indexterm:[XML element,date_spec element,months attribute]
|weeks
|Allowed values: 1-53 (depending on weekyear)
indexterm:[XML attribute,weeks attribute,date_spec element]
indexterm:[XML element,date_spec element,weeks attribute]
|years
|Year according to the Gregorian calendar
indexterm:[XML attribute,years attribute,date_spec element]
indexterm:[XML element,date_spec element,years attribute]
|weekyears
|Year in which the week started; for example, 1 January 2005 can be specified
in ISO 8601 as '2005-001 Ordinal', '2005-01-01 Gregorian' or
'2004-W53-6 Weekly' and thus would match +years="2005"+ or +weekyears="2004"+
indexterm:[XML attribute,weekyears attribute,date_spec element]
indexterm:[XML element,date_spec element,weekyears attribute]
|moon
|Allowed values are 0-7 (where 0 is the new moon and 4 is full moon).
Seriously, you can use this. This was implemented to demonstrate the ease with
which new comparisons could be added.
indexterm:[XML attribute,moon attribute,date_spec element]
indexterm:[XML element,date_spec element,moon attribute]
|=========================================================
For example, +monthdays="1"+ matches the first day of every month, and
+hours="09-17"+ matches the hours between 9 a.m. and 5 p.m. (inclusive).
At this time, multiple ranges (e.g. +weekdays="1,2"+ or +weekdays="1-2,5-6"+)
are not supported.
[NOTE]
====
Pacemaker can calculate when evaluation of a +date_expression+ with an
+operation+ of +gt+, +lt+, or +in_range+ will next change, and schedule a
cluster re-check for that time. However, it does not do this for +date_spec+.
Instead, it evaluates the +date_spec+ whenever a cluster re-check naturally
happens via a cluster event or the +cluster-recheck-interval+ cluster option.
For example, if you have a +date_spec+ enabling a resource from 9 a.m. to 5 p.m.,
and +cluster-recheck-interval+ has been set to 5 minutes, then sometime between
9 a.m. and 9:05 a.m. the cluster would notice that it needs to start the
resource, and sometime between 5 p.m. and 5:05 p.m. it would realize that it
needs to stop the resource. The timing of the actual start and stop actions
will further depend on factors such as any other actions the cluster may need
to perform first, and the load of the machine.
====
=== Durations ===
indexterm:[Rule,Date/Time Expression,Duration]
indexterm:[XML element,duration element]
A +duration+ is used to calculate a value for +end+ when one is not supplied to
+in_range+ operations. It contains one or more attributes each containing a
single number. Any attribute not supplied is ignored.
.Attributes of a duration Element
[width="95%",cols="2m,<5",options="header",align="center"]
|=========================================================
|Field
|Description
|id
|A unique name for this duration element (required)
indexterm:[XML attribute,id attribute,duration element]
indexterm:[XML element,duration element,id attribute]
|seconds
|This many seconds will be added to the total duration
indexterm:[XML attribute,seconds attribute,duration element]
indexterm:[XML element,duration element,seconds attribute]
|minutes
|This many minutes will be added to the total duration
indexterm:[XML attribute,minutes attribute,duration element]
indexterm:[XML element,duration element,minutes attribute]
|hours
|This many hours will be added to the total duration
indexterm:[XML attribute,hours attribute,duration element]
indexterm:[XML element,duration element,hours attribute]
|weeks
|This many weeks will be added to the total duration
indexterm:[XML attribute,weeks attribute,duration element]
indexterm:[XML element,duration element,weeks attribute]
|months
|This many months will be added to the total duration
indexterm:[XML attribute,months attribute,duration element]
indexterm:[XML element,duration element,months attribute]
|years
|This many years will be added to the total duration
indexterm:[XML attribute,years attribute,duration element]
indexterm:[XML element,duration element,years attribute]
|=========================================================
=== Example Time-Based Expressions ===
A small sample of how time-based expressions can be used:
.True if now is any time in the year 2005
====
[source,XML]
----
<rule id="rule1" score="INFINITY">
<date_expression id="date_expr1" start="2005-001" operation="in_range">
<duration id="duration1" years="1"/>
</date_expression>
</rule>
----
====
.Equivalent expression
====
[source,XML]
----
<rule id="rule2" score="INFINITY">
<date_expression id="date_expr2" operation="date_spec">
<date_spec id="date_spec2" years="2005"/>
</date_expression>
</rule>
----
====
.9am-5pm Monday-Friday
====
[source,XML]
-------
<rule id="rule3" score="INFINITY">
<date_expression id="date_expr3" operation="date_spec">
<date_spec id="date_spec3" hours="9-16" weekdays="1-5"/>
</date_expression>
</rule>
-------
====
Please note that the +16+ matches up to +16:59:59+, as the numeric
value (hour) still matches!
.9am-6pm Monday through Friday or anytime Saturday
====
[source,XML]
-------
<rule id="rule4" score="INFINITY" boolean-op="or">
<date_expression id="date_expr4-1" operation="date_spec">
<date_spec id="date_spec4-1" hours="9-16" weekdays="1-5"/>
</date_expression>
<date_expression id="date_expr4-2" operation="date_spec">
<date_spec id="date_spec4-2" weekdays="6"/>
</date_expression>
</rule>
-------
====
.9am-5pm or 9pm-12am Monday through Friday
====
[source,XML]
-------
<rule id="rule5" score="INFINITY" boolean-op="and">
<rule id="rule5-nested1" score="INFINITY" boolean-op="or">
<date_expression id="date_expr5-1" operation="date_spec">
<date_spec id="date_spec5-1" hours="9-16"/>
</date_expression>
<date_expression id="date_expr5-2" operation="date_spec">
<date_spec id="date_spec5-2" hours="21-23"/>
</date_expression>
</rule>
<date_expression id="date_expr5-3" operation="date_spec">
<date_spec id="date_spec5-3" weekdays="1-5"/>
</date_expression>
</rule>
-------
====
.Mondays in March 2005
====
[source,XML]
-------
<rule id="rule6" score="INFINITY" boolean-op="and">
<date_expression id="date_expr6-1" operation="date_spec">
<date_spec id="date_spec6" weekdays="1"/>
</date_expression>
<date_expression id="date_expr6-2" operation="in_range"
start="2005-03-01" end="2005-04-01"/>
</rule>
-------
====
[NOTE]
======
Because no time is specified with the above dates, 00:00:00 is implied. This
means that the range includes all of 2005-03-01 but none of 2005-04-01.
You may wish to write +end="2005-03-31T23:59:59"+ to avoid confusion.
======
.A full moon on Friday the 13th
=====
[source,XML]
-------
<rule id="rule7" score="INFINITY" boolean-op="and">
<date_expression id="date_expr7" operation="date_spec">
<date_spec id="date_spec7" weekdays="5" monthdays="13" moon="4"/>
</date_expression>
</rule>
-------
=====
== Using Rules to Determine Resource Location ==
indexterm:[Rule,Determine Resource Location]
indexterm:[Resource,Location,Determine by Rules]
A location constraint may contain one or more top-level rules. The cluster
will act as if there is a separate location constraint for each rule that
evaluates as true.
Consider the following simple location constraint:
.Prevent resource "webserver" from running on node3
=====
[source,XML]
-------
<rsc_location id="ban-apache-on-node3" rsc="webserver"
score="-INFINITY" node="node3"/>
-------
=====
The constraint can be more verbosely written using a rule:
.Prevent resource "webserver" from running on node3 using rule
=====
[source,XML]
-------
<rsc_location id="ban-apache-on-node3" rsc="webserver">
<rule id="ban-apache-rule" score="-INFINITY">
<expression id="ban-apache-expr" attribute="#uname"
operation="eq" value="node3"/>
</rule>
</rsc_location>
-------
=====
The advantage of using the expanded form is that one could add more expressions
(for example, limiting the constraint to certain days of the week), or activate
the constraint by some node attribute other than node name.
=== Location Rules Based on Other Node Properties ===
The expanded form allows us to match on node properties other than its name.
If we rated each machine's CPU power such that the cluster had the
following nodes section:
.A sample nodes section for use with score-attribute
=====
[source,XML]
-------
<nodes>
<node id="uuid1" uname="c001n01" type="normal">
<instance_attributes id="uuid1-custom_attrs">
<nvpair id="uuid1-cpu_mips" name="cpu_mips" value="1234"/>
</instance_attributes>
</node>
<node id="uuid2" uname="c001n02" type="normal">
<instance_attributes id="uuid2-custom_attrs">
<nvpair id="uuid2-cpu_mips" name="cpu_mips" value="5678"/>
</instance_attributes>
</node>
</nodes>
-------
=====
then we could prevent resources from running on underpowered machines with this rule:
[source,XML]
-------
<rule id="need-more-power-rule" score="-INFINITY">
<expression id="need-more-power-expr" attribute="cpu_mips"
operation="lt" value="3000"/>
</rule>
-------
=== Using +score-attribute+ Instead of +score+ ===
When using +score-attribute+ instead of +score+, each node matched by
the rule has its score adjusted differently, according to its value
for the named node attribute. Thus, in the previous example, if a
rule used +score-attribute="cpu_mips"+, +c001n01+ would have its
preference to run the resource increased by +1234+ whereas +c001n02+
would have its preference increased by +5678+.
== Using Rules to Define Options ==
Rules may be used to control a variety of options:
* <<s-cluster-options,Cluster options>> (+cluster_property_set+ elements)
* <<s-node-attributes,Node attributes>> (as +instance_attributes+ or
+utilization+ elements inside a +node+ element)
* <<s-resource-options,Resource options>> (as +utilization+, +meta_attributes+,
or +instance_attributes+ elements inside a resource definition element or
+op+ , +rsc_defaults+, +op_defaults+, or +template+ element)
* <<s-operation-properties,Operation properties>> (+meta_attributes+
inside an +op+ or +op_defaults+ element)
=== Using Rules to Control Resource Options ===
Often some cluster nodes will be different from their peers. Sometimes,
these differences -- e.g. the location of a binary or the names of network
interfaces -- require resources to be configured differently depending
on the machine they're hosted on.
By defining multiple +instance_attributes+ objects for the resource
and adding a rule to each, we can easily handle these special cases.
In the example below, +mySpecialRsc+ will use eth1 and port 9999 when
run on +node1+, eth2 and port 8888 on +node2+ and default to eth0 and
port 9999 for all other nodes.
.Defining different resource options based on the node name
=====
[source,XML]
-------
<primitive id="mySpecialRsc" class="ocf" type="Special" provider="me">
<instance_attributes id="special-node1" score="3">
<rule id="node1-special-case" score="INFINITY" >
<expression id="node1-special-case-expr" attribute="#uname"
operation="eq" value="node1"/>
</rule>
<nvpair id="node1-interface" name="interface" value="eth1"/>
</instance_attributes>
<instance_attributes id="special-node2" score="2" >
<rule id="node2-special-case" score="INFINITY">
<expression id="node2-special-case-expr" attribute="#uname"
operation="eq" value="node2"/>
</rule>
<nvpair id="node2-interface" name="interface" value="eth2"/>
<nvpair id="node2-port" name="port" value="8888"/>
</instance_attributes>
<instance_attributes id="defaults" score="1" >
<nvpair id="default-interface" name="interface" value="eth0"/>
<nvpair id="default-port" name="port" value="9999"/>
</instance_attributes>
</primitive>
-------
=====
The order in which +instance_attributes+ objects are evaluated is
determined by their score (highest to lowest). If not supplied, score
defaults to zero, and objects with an equal score are processed in
listed order. If the +instance_attributes+ object has no rule
or a +rule+ that evaluates to +true+, then for any parameter the resource does
not yet have a value for, the resource will use the parameter values defined by
the +instance_attributes+.
For example, given the configuration above, if the resource is placed on node1:
. +special-node1+ has the highest score (3) and so is evaluated first;
its rule evaluates to +true+, so +interface+ is set to +eth1+.
. +special-node2+ is evaluated next with score 2, but its rule evaluates to +false+,
so it is ignored.
. +defaults+ is evaluated last with score 1, and has no rule, so its values
are examined; +interface+ is already defined, so the value here is not used,
but +port+ is not yet defined, so +port+ is set to +9999+.
=== Using Rules to Control Resource Defaults ===
Rules can be used for resource and operation defaults. The following example
illustrates how to set a different +resource-stickiness+ value during and
outside work hours. This allows resources to automatically move back to their
most preferred hosts, but at a time that (in theory) does not interfere with
business activities.
.Change +resource-stickiness+ during working hours
=====
[source,XML]
-------
<rsc_defaults>
<meta_attributes id="core-hours" score="2">
<rule id="core-hour-rule" score="0">
<date_expression id="nine-to-five-Mon-to-Fri" operation="date_spec">
<date_spec id="nine-to-five-Mon-to-Fri-spec" hours="9-16" weekdays="1-5"/>
</date_expression>
</rule>
<nvpair id="core-stickiness" name="resource-stickiness" value="INFINITY"/>
</meta_attributes>
<meta_attributes id="after-hours" score="1" >
<nvpair id="after-stickiness" name="resource-stickiness" value="0"/>
</meta_attributes>
</rsc_defaults>
-------
=====
Rules may be used similarly in +instance_attributes+ or +utilization+ blocks.
Any single block may directly contain only a single rule, but that rule may
itself contain any number of rules.
=== Using Rules to Control Cluster Options ===
indexterm:[Rule,Controlling Cluster Options]
indexterm:[Cluster,Setting Options with Rules]
Controlling cluster options is achieved in much the same manner as
specifying different resource options on different nodes.
The difference is that because they are cluster options, one cannot (or should
not, because they won't work) use attribute-based expressions. The following
example illustrates how to set +maintenance_mode+ during a scheduled
maintenance window. This will keep the cluster running but not monitor, start,
or stop resources during this time.
.Schedule a maintenance window for 9 to 11 p.m. CDT Sept. 20, 2019
=====
[source,XML]
-------
<crm_config>
<cluster_property_set id="cib-bootstrap-options">
<nvpair id="bootstrap-stonith-enabled" name="stonith-enabled" value="1"/>
</cluster_property_set>
<cluster_property_set id="normal-set" score="10">
<nvpair id="normal-maintenance-mode" name="maintenance-mode" value="false"/>
</cluster_property_set>
<cluster_property_set id="maintenance-window-set" score="1000">
<nvpair id="maintenance-nvpair1" name="maintenance-mode" value="true"/>
<rule id="maintenance-rule1" score="INFINITY">
<date_expression id="maintenance-date1" operation="in_range"
start="2019-09-20 21:00:00 -05:00" end="2019-09-20 23:00:00 -05:00"/>
</rule>
</cluster_property_set>
</crm_config>
-------
=====
[IMPORTANT]
====
The +cluster_property_set+ with an +id+ set to "cib-bootstrap-options" will
'always' have the highest priority, regardless of any scores. Therefore,
rules in another +cluster_property_set+ can never take effect for any
properties listed in the bootstrap set.
====