Linux Ethernet Bonding Driver HOWTO

		Latest update: 27 April 2011

Initial release : Thomas Davis <tadavis at lbl.gov>

Corrections, HA extensions : 2000/10/03-15 :

  - Willy Tarreau <willy at meta-x.org>

  - Constantine Gavrilov <const-g at xpert.com>

  - Chad N. Tindel <ctindel at ieee dot org>

  - Janice Girouard <girouard at us dot ibm dot com>

  - Jay Vosburgh <fubar at us dot ibm dot com>

Reorganized and updated Feb 2005 by Jay Vosburgh

Added Sysfs information: 2006/04/24

  - Mitch Williams <mitch.a.williams at intel.com>

Introduction

============

	The Linux bonding driver provides a method for aggregating

multiple network interfaces into a single logical "bonded" interface.

The behavior of the bonded interfaces depends upon the mode; generally

speaking, modes provide either hot standby or load balancing services.

Additionally, link integrity monitoring may be performed.

	The bonding driver originally came from Donald Becker's

beowulf patches for kernel 2.0. It has changed quite a bit since, and

the original tools from extreme-linux and beowulf sites will not work

with this version of the driver.

	For new versions of the driver, updated userspace tools, and

who to ask for help, please follow the links at the end of this file.

Table of Contents

=================

1. Bonding Driver Installation

2. Bonding Driver Options

3. Configuring Bonding Devices

3.1	Configuration with Sysconfig Support

3.1.1		Using DHCP with Sysconfig

3.1.2		Configuring Multiple Bonds with Sysconfig

3.2	Configuration with Initscripts Support

3.2.1		Using DHCP with Initscripts

3.2.2		Configuring Multiple Bonds with Initscripts

3.3	Configuring Bonding Manually with Ifenslave

3.3.1		Configuring Multiple Bonds Manually

3.4	Configuring Bonding Manually via Sysfs

3.5	Configuration with Interfaces Support

3.6	Overriding Configuration for Special Cases

4. Querying Bonding Configuration

4.1	Bonding Configuration

4.2	Network Configuration

5. Switch Configuration

6. 802.1q VLAN Support

7. Link Monitoring

7.1	ARP Monitor Operation

7.2	Configuring Multiple ARP Targets

7.3	MII Monitor Operation

8. Potential Trouble Sources

8.1	Adventures in Routing

8.2	Ethernet Device Renaming

8.3	Painfully Slow Or No Failed Link Detection By Miimon

9. SNMP agents

10. Promiscuous mode

11. Configuring Bonding for High Availability

11.1	High Availability in a Single Switch Topology

11.2	High Availability in a Multiple Switch Topology

11.2.1		HA Bonding Mode Selection for Multiple Switch Topology

11.2.2		HA Link Monitoring for Multiple Switch Topology

12. Configuring Bonding for Maximum Throughput

12.1	Maximum Throughput in a Single Switch Topology

12.1.1		MT Bonding Mode Selection for Single Switch Topology

12.1.2		MT Link Monitoring for Single Switch Topology

12.2	Maximum Throughput in a Multiple Switch Topology

12.2.1		MT Bonding Mode Selection for Multiple Switch Topology

12.2.2		MT Link Monitoring for Multiple Switch Topology

13. Switch Behavior Issues

13.1	Link Establishment and Failover Delays

13.2	Duplicated Incoming Packets

14. Hardware Specific Considerations

14.1	IBM BladeCenter

15. Frequently Asked Questions

16. Resources and Links

1. Bonding Driver Installation

==============================

	Most popular distro kernels ship with the bonding driver

already available as a module and the ifenslave user level control

program installed and ready for use. If your distro does not, or you

have need to compile bonding from source (e.g., configuring and

installing a mainline kernel from kernel.org), you'll need to perform

the following steps:

1.1 Configure and build the kernel with bonding

-----------------------------------------------

	The current version of the bonding driver is available in the

drivers/net/bonding subdirectory of the most recent kernel source

(which is available on http://kernel.org).  Most users "rolling their

own" will want to use the most recent kernel from kernel.org.

	Configure kernel with "make menuconfig" (or "make xconfig" or

"make config"), then select "Bonding driver support" in the "Network

device support" section.  It is recommended that you configure the

driver as module since it is currently the only way to pass parameters

to the driver or configure more than one bonding device.

	Build and install the new kernel and modules, then continue

below to install ifenslave.

1.2 Install ifenslave Control Utility

-------------------------------------

	The ifenslave user level control program is included in the

kernel source tree, in the file Documentation/networking/ifenslave.c.

It is generally recommended that you use the ifenslave that

corresponds to the kernel that you are using (either from the same

source tree or supplied with the distro), however, ifenslave

executables from older kernels should function (but features newer

than the ifenslave release are not supported).  Running an ifenslave

that is newer than the kernel is not supported, and may or may not

work.

	To install ifenslave, do the following:

# gcc -Wall -O -I/usr/src/linux/include ifenslave.c -o ifenslave

# cp ifenslave /sbin/ifenslave

	If your kernel source is not in "/usr/src/linux," then replace

"/usr/src/linux/include" in the above with the location of your kernel

source include directory.

	You may wish to back up any existing /sbin/ifenslave, or, for

testing or informal use, tag the ifenslave to the kernel version

(e.g., name the ifenslave executable /sbin/ifenslave-2.6.10).

IMPORTANT NOTE:

	If you omit the "-I" or specify an incorrect directory, you

may end up with an ifenslave that is incompatible with the kernel

you're trying to build it for.  Some distros (e.g., Red Hat from 7.1

onwards) do not have /usr/include/linux symbolically linked to the

default kernel source include directory.

SECOND IMPORTANT NOTE:

	If you plan to configure bonding using sysfs or using the

/etc/network/interfaces file, you do not need to use ifenslave.

2. Bonding Driver Options

=========================

	Options for the bonding driver are supplied as parameters to the

bonding module at load time, or are specified via sysfs.

	Module options may be given as command line arguments to the

insmod or modprobe command, but are usually specified in either the

/etc/modrobe.d/*.conf configuration files, or in a distro-specific

configuration file (some of which are detailed in the next section).

	Details on bonding support for sysfs is provided in the

"Configuring Bonding Manually via Sysfs" section, below.

	The available bonding driver parameters are listed below. If a

parameter is not specified the default value is used.  When initially

configuring a bond, it is recommended "tail -f /var/log/messages" be

run in a separate window to watch for bonding driver error messages.

	It is critical that either the miimon or arp_interval and

arp_ip_target parameters be specified, otherwise serious network

degradation will occur during link failures.  Very few devices do not

support at least miimon, so there is really no reason not to use it.

	Options with textual values will accept either the text name

or, for backwards compatibility, the option value.  E.g.,

"mode=802.3ad" and "mode=4" set the same mode.

	The parameters are as follows:

active_slave

	Specifies the new active slave for modes that support it

	(active-backup, balance-alb and balance-tlb).  Possible values

	are the name of any currently enslaved interface, or an empty

	string.  If a name is given, the slave and its link must be up in order

	to be selected as the new active slave.  If an empty string is

	specified, the current active slave is cleared, and a new active

	slave is selected automatically.

	Note that this is only available through the sysfs interface. No module

	parameter by this name exists.

	The normal value of this option is the name of the currently

	active slave, or the empty string if there is no active slave or

	the current mode does not use an active slave.

ad_select

	Specifies the 802.3ad aggregation selection logic to use.  The

	possible values and their effects are:

	stable or 0

		The active aggregator is chosen by largest aggregate

		bandwidth.

		Reselection of the active aggregator occurs only when all

		slaves of the active aggregator are down or the active

		aggregator has no slaves.

		This is the default value.

	bandwidth or 1

		The active aggregator is chosen by largest aggregate

		bandwidth.  Reselection occurs if:

		- A slave is added to or removed from the bond

		- Any slave's link state changes

		- Any slave's 802.3ad association state changes

		- The bond's administrative state changes to up

	count or 2

		The active aggregator is chosen by the largest number of

		ports (slaves).  Reselection occurs as described under the

		"bandwidth" setting, above.

	The bandwidth and count selection policies permit failover of

	802.3ad aggregations when partial failure of the active aggregator

	occurs.  This keeps the aggregator with the highest availability

	(either in bandwidth or in number of ports) active at all times.

	This option was added in bonding version 3.4.0.

all_slaves_active

	Specifies that duplicate frames (received on inactive ports) should be

	dropped (0) or delivered (1).

	Normally, bonding will drop duplicate frames (received on inactive

	ports), which is desirable for most users. But there are some times

	it is nice to allow duplicate frames to be delivered.

	The default value is 0 (drop duplicate frames received on inactive

	ports).

arp_interval

	Specifies the ARP link monitoring frequency in milliseconds.

	The ARP monitor works by periodically checking the slave

	devices to determine whether they have sent or received

	traffic recently (the precise criteria depends upon the

	bonding mode, and the state of the slave).  Regular traffic is

	generated via ARP probes issued for the addresses specified by

	the arp_ip_target option.

	This behavior can be modified by the arp_validate option,

	below.

	If ARP monitoring is used in an etherchannel compatible mode

	(modes 0 and 2), the switch should be configured in a mode

	that evenly distributes packets across all links. If the

	switch is configured to distribute the packets in an XOR

	fashion, all replies from the ARP targets will be received on

	the same link which could cause the other team members to

	fail.  ARP monitoring should not be used in conjunction with

	miimon.  A value of 0 disables ARP monitoring.  The default

	value is 0.

arp_ip_target

	Specifies the IP addresses to use as ARP monitoring peers when

	arp_interval is > 0.  These are the targets of the ARP request

	sent to determine the health of the link to the targets.

	Specify these values in ddd.ddd.ddd.ddd format.  Multiple IP

	addresses must be separated by a comma.  At least one IP

	address must be given for ARP monitoring to function.  The

	maximum number of targets that can be specified is 16.  The

	default value is no IP addresses.

arp_validate

	Specifies whether or not ARP probes and replies should be

	validated in the active-backup mode.  This causes the ARP

	monitor to examine the incoming ARP requests and replies, and

	only consider a slave to be up if it is receiving the

	appropriate ARP traffic.

	Possible values are:

	none or 0

		No validation is performed.  This is the default.

	active or 1

		Validation is performed only for the active slave.

	backup or 2

		Validation is performed only for backup slaves.

	all or 3

		Validation is performed for all slaves.

	For the active slave, the validation checks ARP replies to

	confirm that they were generated by an arp_ip_target.  Since

	backup slaves do not typically receive these replies, the

	validation performed for backup slaves is on the ARP request

	sent out via the active slave.  It is possible that some

	switch or network configurations may result in situations

	wherein the backup slaves do not receive the ARP requests; in

	such a situation, validation of backup slaves must be

	disabled.

	This option is useful in network configurations in which

	multiple bonding hosts are concurrently issuing ARPs to one or

	more targets beyond a common switch.  Should the link between

	the switch and target fail (but not the switch itself), the

	probe traffic generated by the multiple bonding instances will

	fool the standard ARP monitor into considering the links as

	still up.  Use of the arp_validate option can resolve this, as

	the ARP monitor will only consider ARP requests and replies

	associated with its own instance of bonding.

	This option was added in bonding version 3.1.0.

downdelay

	Specifies the time, in milliseconds, to wait before disabling

	a slave after a link failure has been detected.  This option

	is only valid for the miimon link monitor.  The downdelay

	value should be a multiple of the miimon value; if not, it

	will be rounded down to the nearest multiple.  The default

	value is 0.

fail_over_mac

	Specifies whether active-backup mode should set all slaves to

	the same MAC address at enslavement (the traditional

	behavior), or, when enabled, perform special handling of the

	bond's MAC address in accordance with the selected policy.

	Possible values are:

	none or 0

		This setting disables fail_over_mac, and causes

		bonding to set all slaves of an active-backup bond to

		the same MAC address at enslavement time.  This is the

		default.

	active or 1

		The "active" fail_over_mac policy indicates that the

		MAC address of the bond should always be the MAC

		address of the currently active slave.  The MAC

		address of the slaves is not changed; instead, the MAC

		address of the bond changes during a failover.

		This policy is useful for devices that cannot ever

		alter their MAC address, or for devices that refuse

		incoming broadcasts with their own source MAC (which

		interferes with the ARP monitor).

		The down side of this policy is that every device on

		the network must be updated via gratuitous ARP,

		vs. just updating a switch or set of switches (which

		often takes place for any traffic, not just ARP

		traffic, if the switch snoops incoming traffic to

		update its tables) for the traditional method.  If the

		gratuitous ARP is lost, communication may be

		disrupted.

		When this policy is used in conjunction with the mii

		monitor, devices which assert link up prior to being

		able to actually transmit and receive are particularly

		susceptible to loss of the gratuitous ARP, and an

		appropriate updelay setting may be required.

	follow or 2

		The "follow" fail_over_mac policy causes the MAC

		address of the bond to be selected normally (normally

		the MAC address of the first slave added to the bond).

		However, the second and subsequent slaves are not set

		to this MAC address while they are in a backup role; a

		slave is programmed with the bond's MAC address at

		failover time (and the formerly active slave receives

		the newly active slave's MAC address).

		This policy is useful for multiport devices that

		either become confused or incur a performance penalty

		when multiple ports are programmed with the same MAC

		address.

	The default policy is none, unless the first slave cannot

	change its MAC address, in which case the active policy is

	selected by default.

	This option may be modified via sysfs only when no slaves are

	present in the bond.

	This option was added in bonding version 3.2.0.  The "follow"

	policy was added in bonding version 3.3.0.

lacp_rate

	Option specifying the rate in which we'll ask our link partner

	to transmit LACPDU packets in 802.3ad mode.  Possible values

	are:

	slow or 0

		Request partner to transmit LACPDUs every 30 seconds

	fast or 1

		Request partner to transmit LACPDUs every 1 second

	The default is slow.

max_bonds

	Specifies the number of bonding devices to create for this

	instance of the bonding driver.  E.g., if max_bonds is 3, and

	the bonding driver is not already loaded, then bond0, bond1

	and bond2 will be created.  The default value is 1.  Specifying

	a value of 0 will load bonding, but will not create any devices.

miimon

	Specifies the MII link monitoring frequency in milliseconds.

	This determines how often the link state of each slave is

	inspected for link failures.  A value of zero disables MII

	link monitoring.  A value of 100 is a good starting point.

	The use_carrier option, below, affects how the link state is

	determined.  See the High Availability section for additional

	information.  The default value is 0.

min_links

	Specifies the minimum number of links that must be active before

	asserting carrier. It is similar to the Cisco EtherChannel min-links

	feature. This allows setting the minimum number of member ports that

	must be up (link-up state) before marking the bond device as up

	(carrier on). This is useful for situations where higher level services

	such as clustering want to ensure a minimum number of low bandwidth

	links are active before switchover. This option only affect 802.3ad

	mode.

	The default value is 0. This will cause carrier to be asserted (for

	802.3ad mode) whenever there is an active aggregator, regardless of the

	number of available links in that aggregator. Note that, because an

	aggregator cannot be active without at least one available link,

	setting this option to 0 or to 1 has the exact same effect.

mode

	Specifies one of the bonding policies. The default is

	balance-rr (round robin).  Possible values are:

	balance-rr or 0

		Round-robin policy: Transmit packets in sequential

		order from the first available slave through the

		last.  This mode provides load balancing and fault

		tolerance.

	active-backup or 1

		Active-backup policy: Only one slave in the bond is

		active.  A different slave becomes active if, and only

		if, the active slave fails.  The bond's MAC address is

		externally visible on only one port (network adapter)

		to avoid confusing the switch.

		In bonding version 2.6.2 or later, when a failover

		occurs in active-backup mode, bonding will issue one

		or more gratuitous ARPs on the newly active slave.

		One gratuitous ARP is issued for the bonding master

		interface and each VLAN interfaces configured above

		it, provided that the interface has at least one IP

		address configured.  Gratuitous ARPs issued for VLAN

		interfaces are tagged with the appropriate VLAN id.

		This mode provides fault tolerance.  The primary

		option, documented below, affects the behavior of this

		mode.

	balance-xor or 2

		XOR policy: Transmit based on the selected transmit

		hash policy.  The default policy is a simple [(source

		MAC address XOR'd with destination MAC address) modulo

		slave count].  Alternate transmit policies may be

		selected via the xmit_hash_policy option, described

		below.

		This mode provides load balancing and fault tolerance.

	broadcast or 3

		Broadcast policy: transmits everything on all slave

		interfaces.  This mode provides fault tolerance.

	802.3ad or 4

		IEEE 802.3ad Dynamic link aggregation.  Creates

		aggregation groups that share the same speed and

		duplex settings.  Utilizes all slaves in the active

		aggregator according to the 802.3ad specification.

		Slave selection for outgoing traffic is done according

		to the transmit hash policy, which may be changed from

		the default simple XOR policy via the xmit_hash_policy

		option, documented below.  Note that not all transmit

		policies may be 802.3ad compliant, particularly in

		regards to the packet mis-ordering requirements of

		section 43.2.4 of the 802.3ad standard.  Differing

		peer implementations will have varying tolerances for

		noncompliance.

		Prerequisites:

		1. Ethtool support in the base drivers for retrieving

		the speed and duplex of each slave.

		2. A switch that supports IEEE 802.3ad Dynamic link

		aggregation.

		Most switches will require some type of configuration

		to enable 802.3ad mode.

	balance-tlb or 5

		Adaptive transmit load balancing: channel bonding that

		does not require any special switch support.  The

		outgoing traffic is distributed according to the

		current load (computed relative to the speed) on each

		slave.  Incoming traffic is received by the current

		slave.  If the receiving slave fails, another slave

		takes over the MAC address of the failed receiving

		slave.

		Prerequisite:

		Ethtool support in the base drivers for retrieving the

		speed of each slave.

	balance-alb or 6

		Adaptive load balancing: includes balance-tlb plus

		receive load balancing (rlb) for IPV4 traffic, and

		does not require any special switch support.  The

		receive load balancing is achieved by ARP negotiation.

		The bonding driver intercepts the ARP Replies sent by

		the local system on their way out and overwrites the

		source hardware address with the unique hardware

		address of one of the slaves in the bond such that

		different peers use different hardware addresses for

		the server.

		Receive traffic from connections created by the server

		is also balanced.  When the local system sends an ARP

		Request the bonding driver copies and saves the peer's

		IP information from the ARP packet.  When the ARP

		Reply arrives from the peer, its hardware address is

		retrieved and the bonding driver initiates an ARP

		reply to this peer assigning it to one of the slaves

		in the bond.  A problematic outcome of using ARP

		negotiation for balancing is that each time that an

		ARP request is broadcast it uses the hardware address

		of the bond.  Hence, peers learn the hardware address

		of the bond and the balancing of receive traffic

		collapses to the current slave.  This is handled by

		sending updates (ARP Replies) to all the peers with

		their individually assigned hardware address such that

		the traffic is redistributed.  Receive traffic is also

		redistributed when a new slave is added to the bond

		and when an inactive slave is re-activated.  The

		receive load is distributed sequentially (round robin)

		among the group of highest speed slaves in the bond.

		When a link is reconnected or a new slave joins the

		bond the receive traffic is redistributed among all

		active slaves in the bond by initiating ARP Replies

		with the selected MAC address to each of the

		clients. The updelay parameter (detailed below) must

		be set to a value equal or greater than the switch's

		forwarding delay so that the ARP Replies sent to the

		peers will not be blocked by the switch.

		Prerequisites:

		1. Ethtool support in the base drivers for retrieving

		the speed of each slave.

		2. Base driver support for setting the hardware

		address of a device while it is open.  This is

		required so that there will always be one slave in the

		team using the bond hardware address (the

		curr_active_slave) while having a unique hardware

		address for each slave in the bond.  If the

		curr_active_slave fails its hardware address is

		swapped with the new curr_active_slave that was

		chosen.

num_grat_arp

num_unsol_na

	Specify the number of peer notifications (gratuitous ARPs and

	unsolicited IPv6 Neighbor Advertisements) to be issued after a

	failover event.  As soon as the link is up on the new slave

	(possibly immediately) a peer notification is sent on the

	bonding device and each VLAN sub-device.  This is repeated at

	each link monitor interval (arp_interval or miimon, whichever

	is active) if the number is greater than 1.

	The valid range is 0 - 255; the default value is 1.  These options

	affect only the active-backup mode.  These options were added for

	bonding versions 3.3.0 and 3.4.0 respectively.

	From Linux 3.0 and bonding version 3.7.1, these notifications

	are generated by the ipv4 and ipv6 code and the numbers of

	repetitions cannot be set independently.

primary

	A string (eth0, eth2, etc) specifying which slave is the

	primary device.  The specified device will always be the

	active slave while it is available.  Only when the primary is

	off-line will alternate devices be used.  This is useful when

	one slave is preferred over another, e.g., when one slave has

	higher throughput than another.

	The primary option is only valid for active-backup mode.

primary_reselect

	Specifies the reselection policy for the primary slave.  This

	affects how the primary slave is chosen to become the active slave

	when failure of the active slave or recovery of the primary slave

	occurs.  This option is designed to prevent flip-flopping between

	the primary slave and other slaves.  Possible values are:

	always or 0 (default)

		The primary slave becomes the active slave whenever it

		comes back up.

	better or 1

		The primary slave becomes the active slave when it comes

		back up, if the speed and duplex of the primary slave is

		better than the speed and duplex of the current active

		slave.

	failure or 2

		The primary slave becomes the active slave only if the

		current active slave fails and the primary slave is up.

	The primary_reselect setting is ignored in two cases:

		If no slaves are active, the first slave to recover is

		made the active slave.

		When initially enslaved, the primary slave is always made

		the active slave.

	Changing the primary_reselect policy via sysfs will cause an

	immediate selection of the best active slave according to the new

	policy.  This may or may not result in a change of the active

	slave, depending upon the circumstances.

	This option was added for bonding version 3.6.0.

updelay

	Specifies the time, in milliseconds, to wait before enabling a

	slave after a link recovery has been detected.  This option is

	only valid for the miimon link monitor.  The updelay value

	should be a multiple of the miimon value; if not, it will be

	rounded down to the nearest multiple.  The default value is 0.

use_carrier

	Specifies whether or not miimon should use MII or ETHTOOL

	ioctls vs. netif_carrier_ok() to determine the link

	status. The MII or ETHTOOL ioctls are less efficient and

	utilize a deprecated calling sequence within the kernel.  The

	netif_carrier_ok() relies on the device driver to maintain its

	state with netif_carrier_on/off; at this writing, most, but

	not all, device drivers support this facility.

	If bonding insists that the link is up when it should not be,

	it may be that your network device driver does not support

	netif_carrier_on/off.  The default state for netif_carrier is

	"carrier on," so if a driver does not support netif_carrier,

	it will appear as if the link is always up.  In this case,

	setting use_carrier to 0 will cause bonding to revert to the

	MII / ETHTOOL ioctl method to determine the link state.

	A value of 1 enables the use of netif_carrier_ok(), a value of

	0 will use the deprecated MII / ETHTOOL ioctls.  The default

	value is 1.

xmit_hash_policy

	Selects the transmit hash policy to use for slave selection in

	balance-xor and 802.3ad modes.  Possible values are:

	layer2

		Uses XOR of hardware MAC addresses to generate the

		hash.  The formula is

		(source MAC XOR destination MAC) modulo slave count

		This algorithm will place all traffic to a particular

		network peer on the same slave.

		This algorithm is 802.3ad compliant.

	layer2+3

		This policy uses a combination of layer2 and layer3

		protocol information to generate the hash.

		Uses XOR of hardware MAC addresses and IP addresses to

		generate the hash.  The formula is

		(((source IP XOR dest IP) AND 0xffff) XOR

			( source MAC XOR destination MAC ))

				modulo slave count

		This algorithm will place all traffic to a particular

		network peer on the same slave.  For non-IP traffic,

		the formula is the same as for the layer2 transmit

		hash policy.

		This policy is intended to provide a more balanced

		distribution of traffic than layer2 alone, especially

		in environments where a layer3 gateway device is

		required to reach most destinations.

		This algorithm is 802.3ad compliant.

	layer3+4

		This policy uses upper layer protocol information,

		when available, to generate the hash.  This allows for

		traffic to a particular network peer to span multiple

		slaves, although a single connection will not span

		multiple slaves.

		The formula for unfragmented TCP and UDP packets is

		((source port XOR dest port) XOR

			 ((source IP XOR dest IP) AND 0xffff)

				modulo slave count

		For fragmented TCP or UDP packets and all other IP

		protocol traffic, the source and destination port

		information is omitted.  For non-IP traffic, the

		formula is the same as for the layer2 transmit hash

		policy.

		This policy is intended to mimic the behavior of

		certain switches, notably Cisco switches with PFC2 as

		well as some Foundry and IBM products.

		This algorithm is not fully 802.3ad compliant.  A

		single TCP or UDP conversation containing both

		fragmented and unfragmented packets will see packets

		striped across two interfaces.  This may result in out

		of order delivery.  Most traffic types will not meet

		this criteria, as TCP rarely fragments traffic, and

		most UDP traffic is not involved in extended

		conversations.  Other implementations of 802.3ad may

		or may not tolerate this noncompliance.

	The default value is layer2.  This option was added in bonding

	version 2.6.3.  In earlier versions of bonding, this parameter

	does not exist, and the layer2 policy is the only policy.  The

	layer2+3 value was added for bonding version 3.2.2.

resend_igmp

	Specifies the number of IGMP membership reports to be issued after

	a failover event. One membership report is issued immediately after

	the failover, subsequent packets are sent in each 200ms interval.

	The valid range is 0 - 255; the default value is 1. A value of 0

	prevents the IGMP membership report from being issued in response

	to the failover event.

	This option is useful for bonding modes balance-rr (0), active-backup

	(1), balance-tlb (5) and balance-alb (6), in which a failover can

	switch the IGMP traffic from one slave to another.  Therefore a fresh

	IGMP report must be issued to cause the switch to forward the incoming

	IGMP traffic over the newly selected slave.

	This option was added for bonding version 3.7.0.

3. Configuring Bonding Devices

==============================

	You can configure bonding using either your distro's network

initialization scripts, or manually using either ifenslave or the

sysfs interface.  Distros generally use one of three packages for the

network initialization scripts: initscripts, sysconfig or interfaces.

Recent versions of these packages have support for bonding, while older

versions do not.

	We will first describe the options for configuring bonding for

distros using versions of initscripts, sysconfig and interfaces with full

or partial support for bonding, then provide information on enabling

bonding without support from the network initialization scripts (i.e.,

older versions of initscripts or sysconfig).

	If you're unsure whether your distro uses sysconfig,

initscripts or interfaces, or don't know if it's new enough, have no fear.

Determining this is fairly straightforward.

	First, look for a file called interfaces in /etc/network directory.

If this file is present in your system, then your system use interfaces. See

Configuration with Interfaces Support.

	Else, issue the command:

$ rpm -qf /sbin/ifup

	It will respond with a line of text starting with either

"initscripts" or "sysconfig," followed by some numbers.  This is the

package that provides your network initialization scripts.

	Next, to determine if your installation supports bonding,

issue the command:

$ grep ifenslave /sbin/ifup

	If this returns any matches, then your initscripts or

sysconfig has support for bonding.

3.1 Configuration with Sysconfig Support

----------------------------------------

	This section applies to distros using a version of sysconfig

with bonding support, for example, SuSE Linux Enterprise Server 9.

	SuSE SLES 9's networking configuration system does support

bonding, however, at this writing, the YaST system configuration

front end does not provide any means to work with bonding devices.

Bonding devices can be managed by hand, however, as follows.