		   Linux ͥåȷɥ饤 ߥ Howto

 : Thomas Davis <tadavis at lbl.gov>

,  (HA) ĥ : 2000/10/03-15 :

  - Willy Tarreau <willy at meta-x.org>
  - Constantine Gavrilov <const-g at xpert.com>
  - Chad N. Tindel <ctindel at ieee dot org>
  - Janice Girouard <girouard at us dot ibm dot com>
  - Jay Vosburgh <fubar at us dot ibm dot com>

 :
----
ɥ饤Фϸ Donald Becker  Linux ͥ 2.0  Beowulf ѥåΰ
ǤΥɥ饤ФϤθ徯ѹ줿ΤǡExtreme-Linux  Beowulf 
ΥȤˤ륪ꥸʥΥġϡΥСΥɥ饤ФǤưʤ
礦

СΥɥ饤СŤͥѤΥѥå줿桼֤Υġ
ϡΥեˤ򻲾ȤƲ

桼 (userspace) ϥͥ (kernelspace) Ȥǡ
  Ԥ䥳ޥɤʤɤΥ桼̥ץԤǥ
  ɥ饤Ф䥹塼ʤɤΥ桼̥ץǤʤͥ
  ̣ޤ

ʸ˲ñ줬ˤ˽ФƤޤ
	ɥ饤 (bonding driver)
	祤󥿡ե (bonding interface)
	ǥХ (bonding device)
	 (bond)
	 (bonding)
  ɥ饤ФϥͥΥǥХɥ饤ФưפʬȻפޤƱͥ
  ȥ󥿡ե bond? ̣ñ줬¿ʬˤ
  פޤΤǡǤϡbonding interfaceסbonding deviceסbond
  bondingפɽƤʪơַ祤󥿡եפȤ
  

ܼ
====
 
󥹥ȡ
祤󥿡ե
⥸塼ѥ᡼
ʣη祤󥿡ե
å
祤󥿡եγǧ
ˤ (FAQ)
 (HA)
˥åե󥰤
»
ȥ

󥹥ȡ
============

1) ɥ饤դΥͥι
-----------------------------------
ǿСηɥ饤Фΰ٤ˡLinux ͥ 2.4.12 ʹߤѤƲ
 (ʤСѥåƤɬפǤ礦)

make menuconfig/xconfig/config ǥͥԤNetwork device
supportץˤBonding driver supportפ򤷤ޤɥ饤Ф
⥸塼Ȥꤹ򤪴ᤷޤΤʤ顢ߤ줬ɥ饤Ф˥
᡼Ϥʣη祤󥿡եꤹͣˡǤ

ͥȥ⥸塼ۤƥ󥹥ȡ뤷ޤ

2) 桼֤Υġȥ󥹥ȡ
-----------------------------------------
ΥСηɥ饤ФϹ줿 ifenslave ץ׵ᤷޤ
Extreme-linux  Beowulf ˤ륪ꥸʥΥġϻȤޤ󡣥ͥ 
2.4.12 ʹߤˤ Documentation/network ǥ쥯ȥ˹С 
ifenslave.c ϿƤޤŤͥˤϡΥեΥ
򻲾ȤƲ

!! ⤷ʤ Red Hat 7.1 ʹߤѤƤ硢/usr/include/linux 
Ϥ /usr/src/linux/include/linux ؤΥܥå󥯤Ǥʤդ
ɬפޤδĶ ifenslave ۤ硢ifenslave Ϥ
ưƤ褦˸Ƥ祤󥿡եưޤifenslave 
ѥԤ -I ץŪϡ/usr/include/linux/if_bonding.h ǤϤʤ 
/usr/src/linux/include/linux/if_bonding.h μ¤˻ѤǤ

Ť Linux ǥȥӥ塼Ǥϡ
	ln -s /usr/src/linux/include/linux /usr/include/linux
	ln -s /usr/src/linux/include/asm /usr/include/asm
	ln -s /usr/src/linux/include/scsi /usr/include/scsi
   /usr/src/linux ˤ륫ͥ륽ΥإåեѤΥܥå
  󥯤ΤŪǤͥϢΥġǤϤΥܥå
  󥯤ȤƤʪޤ

ifenslave.c Υ󥹥ȡϲ̤Ǥ

    # gcc -Wall -Wstrict-prototypes -O -I/usr/src/linux/include ifenslave.c -o ifenslave 
    # cp ifenslave /sbin/ifenslave

祤󥿡ե
==========================

bond0 󥿡եꤵ줿˷ɥ饤ФưŪ˥ɤ褦
/etc/modules.conf ˼ιԤɲäɬפޤ 
modules.conf ν񼰤ξܺ٤ˤĤƤ modules.conf Υޥ˥奢ڡ򻲾Ȥ
ƲʸΡ֥⥸塼ѥ᡼פξϤǷɥ饤Фγƥѥ᡼
ˤĤޤ

	alias bond0 bonding

bond0 ͥåȥ󥿡եˤϡǥȥӥ塼ɸ
˥åѤƲ㤨СǶ Red Hat ǥȥӥ塼Ǥ
С/etc/sysconfig/network-scripts ǥ쥯ȥ˲Τ褦 ifcfg-bond0 
եäƲ

DEVICE=bond0
IPADDR=192.168.1.1
NETMASK=255.255.255.0
NETWORK=192.168.1.0
BROADCAST=192.168.1.255
ONBOOT=yes
BOOTPROTO=none
USERCTL=no

(嵭ˤʤΥͥåȥŬڤͤѤƲ)

祤󥿡ե륤󥿡եˤơSLAVE  MASTER 
ޤ㤨СRed Hat ξ硢eth0  eth1 祤󥿡ե 
bond0 ΰˤ硢Υ󥿡եե (ifcfg-eth0 
 ifcfg-eth1) ϰʲΤ褦ʷˤʤޤ

DEVICE=eth0
USERCTL=no
ONBOOT=yes
MASTER=bond0
SLAVE=yes
BOOTPROTO=none

ifcfg-eth1 եǤ DEVICE=eth1 ѤƲܤη祤
ե (bond1) ꤹϡbond1 Υ졼֤Ȥʤͥåȥ
󥿡եե MASTER=bond1 ѤƲ

ͥåȥ֥ƥƥȤ뤫ȤδġǽΤǤ
з祤󥿡եñ˵ưƲ뤤ϡƥƵư
Red Hat ǥȥӥ塼Ǥϡifup bond0פޤ
/etc/rc.d/init.d/network restartפ¹Ԥޤ

ǥȥӥ塼δġ뤬ͥåȥ󥿡եǥޥ
졼ֵҤ򥵥ݡȤƤʤ硢ΥޥɤȤäƼưǷ
󥿡եꤹɬפޤ

    # /sbin/ifconfig bond0 192.168.1.1 netmask 255.255.255.0 \
      broadcast 192.168.1.255 up

    # /sbin/ifenslave bond0 eth0
    # /sbin/ifenslave bond0 eth1

(嵭ˤʤΥͥåȥŬڤͤѤƲ)

줫顢ΥޥɤޤॹץȤäơŬڤ rc ǥ쥯ȥ
֤Ʋ

äˡɥ饤ФƤΥͥåȥɥ饤ФɤƤɬפ
硢ιԤmodules.conf ɲäȡɥ饤Ф eth0  eth1 
Υͥåȥɥ饤Фɤ褦ˤʤޤ

probeall bond0 eth0 eth1 bonding

ι bond0 Τ򻲾Ȥʤ褦դƲʤ modprobe 
̵¥롼פ˴٤äư۾ｪλƤޤޤ

ǥХ (MTUʤ) 򥹥졼֥ǥХˤŬѤ褦ˤˤϡ
ǥХΥ졼ֲλ˷祤󥿡եꤷƲ
ϥ졼ֲβǥ졼֥ǥХŬѤޤ

SNMP ȤưƤ硢ɥ饤ФϤɤΥͥåȥɥ饤
祤󥿡ե˻ä˥ɤ٤Ǥϡ󥿡
եǥå (ipAdEntIfIndex) Ϳ줿 IP ɥ쥹ĺǽ˸
Ĥä󥿡եȴϢդ뤿ɬפȤʤޤĤޤꡢ IP 
ɥ쥹ФơipAdEntIfIndex ϰĤǤޤ㤨Сeth0  
eth1  bond0 Υ졼֤Ǥꡢeth0 ѥɥ饤Фɥ饤Ф˥
줿硢 IP ɥ쥹Υ󥿡ե eth0 󥿡եȴϢ
ŤޤϲͤˤʤꡢIP ɥ쥹 192.168.1.1 ifDescr ơ
֥ǥ󥿡եֹ 2 Ȥʤꡢϥ󥿡եǥåơ
֥Ǥ (ifDescr.2) eth0 ؤƤޤäƤޤ

     interfaces.ifTable.ifEntry.ifDescr.1 = lo
     interfaces.ifTable.ifEntry.ifDescr.2 = eth0
     interfaces.ifTable.ifEntry.ifDescr.3 = eth1
     interfaces.ifTable.ifEntry.ifDescr.4 = eth2
     interfaces.ifTable.ifEntry.ifDescr.5 = eth3
     interfaces.ifTable.ifEntry.ifDescr.6 = bond0
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.10.10.10 = 5
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.192.168.1.1 = 2
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.74.20.94 = 4
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.127.0.0.1 = 1

Ϸɥ饤Ф (äˤη祤󥿡ե) ¾Υͥåȥ
ɥ饤Ф˥ɤǲǤޤϷɥ饤Фǽ˥
ǡIP ɥ쥹 192.168.1.1.  ifDescr.2 ȴϢŤƤޤ

     interfaces.ifTable.ifEntry.ifDescr.1 = lo
     interfaces.ifTable.ifEntry.ifDescr.2 = bond0
     interfaces.ifTable.ifEntry.ifDescr.3 = eth0
     interfaces.ifTable.ifEntry.ifDescr.4 = eth1
     interfaces.ifTable.ifEntry.ifDescr.5 = eth2
     interfaces.ifTable.ifEntry.ifDescr.6 = eth3
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.10.10.10 = 6
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.192.168.1.1 = 2
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.10.74.20.94 = 5
     ip.ipAddrTable.ipAddrEntry.ipAdEntIfIndex.127.0.0.1 = 1

ĤΥǥȥӥ塼Ǥ ifDescr Υ󥿡ե̾ݡ
ʤΤޤ󤬡IP ɥ쥹 ifIndex δطƱǡ
Interface_Scan_Next  SNMP ǽϤδϢŤݡȤǤ礦

⥸塼ѥ᡼
====================

ɥ饤ФΥץѥ᡼ insmod ޥɤΥޥɥ饤Ȥ
Ϳޤ̾Υѥ᡼ /etc/modules.conf ե
ǻꤹ뤳Ȥˤʤޤ (modules.conf Υޥ˥奢ڡ򻲾ȤƲ) 
ѥ᡼ͿʤϡǥեͤѤޤ祤󥿡
եǽꤹݤˤϡɥ饤ФΥ顼åƻ뤹뤿
ˡ̤Υɥ "tail -f /var/log/messages" ¹ԤƤ򤪴ᤷ
ޤ

miimon ѥ᡼arp_interval  arp_ip_target ѥ᡼ꤹΤ
ɬܤǤɤꤷƤʤ硢󥯤Ԥ˿ʥͥåȥ
ǽ㲼Ǥ

arp_interval
 
	ARP ˥٤ߥñ̤ǻꤷޤɥХ󥷥󥰥⡼
	 (⡼ 0 ޤ 2) ARP ˥󥰤Ѥ줿硢å
	ϥ饦ɥӥ (緫) Τ褦ƤΥ󥯤ФƥѥåȤ
	ʬۤ⡼ɤꤵ٤Ǥå XOR ǥѥ
	Ȥʬۤ硢ARP åȤФƤֿƱ󥯤Ǽ
	Ƥޤ¾ΥСμԤޤARP ˥
	 miimon Ʊ˻Ѥ٤ǤϤޤ󡣥ǥեͤ 0 Ǥ
 
arp_ip_target
 
	arp_interval ξ IP ɥ쥹ꤷޤϡ
	åȤؤΥ󥯤η () ¬ꤹ٤ ARP ꥯ
	ΥåȤǤddd.ddd.ddd.ddd ǤͤꤷƲ
	ʣ IP ɥ쥹ϥޤǶڤɬפޤARP ˥󥰤
	ư뤿ˤϡʤƤ⣱Ĥ IP ɥ쥹ɬפǤǽ
	ʥåȿκͤ 16 Ǥ

downdelay
 
	󥯼ԤΤƤ󥯤̵ˤޤǤٱ֤ߥä
	ꤷޤϡmiimon ͤܿǤʤФʤޤ󡣤ʤ
	ͤƤޤޤǥեͤ 0Ǥ

lacp_rate

	802.3ad ⡼ɤǤ LACPDU ѥåȤž٤Υ󥯥ѡȥʡ
	碌ݤΥ졼Ȥꤹ륪ץ󡣺Τͤϲ̤Ǥ

	slow or 0
		LACPDUѡȥʡ 30ñ̤׵᤹ (ǥե)

	fast or 1
		LACPDU ѡȥʡ 1ñ̤׵᤹

max_bonds

	ɥ饤ФΣѤ˺祤󥿡եοꤷޤ
	Ĥޤꡢmiimon maxbonds  3 ǡɥ饤ФޤɤƤ
	硢bond0, bond1, bond2 ޤǥեͤ 1 Ǥ

miimon
 
	MII 󥯴ƻԤ٤ߥñ̤ǻꤷޤ0  MII 󥯴
	̵ˤޤ100 ϺǽͤȤƤɤǤ礦ɲþϡֹ
	פξϤ򻲾ȤƲǥեͤ 0 Ǥ

mode

	ݥꥷΣĤꤷޤǥեȤϥ饦ɥӥ
	(balance-rr ; 緫) ǤΤͤϲ̤ (ƥȤȿ
	ץΤɤǤѲ)

	balance-rr or 0

		饦ɥӥݥꥷѲǽʥ졼֤ǽ餫Ǹޤ
		֤˻ѤޤΥ⡼ɤʬо㳲
		ޤ

	active-backup or 1

		ƥ֥Хååץݥꥷ祤󥿡եΣ
		졼֤Τߥƥ֤ǤΥƥ֥졼֤Ԥ
		˸¤ꡢ¾Υ졼֤ƥ֤ˤʤޤåκ
		򤱤뤿ᡢη祤󥿡ե MAX ɥ쥹ϳ
		ϣݡ (ͥåȥץ) Τ߸ޤΥ⡼
		ɤо㳲󶡤ޤ
 
	balance-xor or 2

		XOR ݥꥷ( MAC ɥ쥹 XOR 줿 MAC ɥ
		) ⥸塼饹졼֥ȡפˤޤ
		Ʊ MAX ɥ쥹ФƱ졼֤Ȥޤ
		Υ⡼ɤʬо㳲󶡤ޤ

	broadcast or 3

		֥ɥ㥹ȥݥꥷ졼֥󥿡եƤ
		ѥåȤޤΥ⡼ɤо㳲󶡤ޤ

	802.3ad or 4

		IEEE 802.3ad ưŪ󥯽硣ԡɤȾ꤬Ʊ
		祰롼פޤƥֽǤƤΥ졼֤
		ޤ
 
		ɬ׾

		1. ƥ졼֤ΥԡɤȾŤ٤Ρ١
                   饤Фˤ Ethtool ݡ

		2. IEEE 802.3ad ưŪ󥯽򥵥ݡȤå

	balance-tlb or 5

		Ŭʬ̤ʥåݡȤɬפȤʤ
		硣̿ϸߤ٤˽äƳƥ졼֤ʬۤ
		̿ϸߤΥ졼֤Ǽޤ졼֤
		Ԥ硢̤Υ졼֤Ԥѥ졼֤ MAC 
		ɥ쥹Ѥޤ

		ɬ׾

		ƥ졼֤Υԡɤѹ٤Ρ١ɥ饤Фˤ 
		Ethtool ݡȡ

	balance-alb or 6 

		ŬʬIPV4 ̿Ѥ balance-tlb ȼʬ (rlb)
		ޤߡ̤ʥåݡȤɬפȤޤ󡣼ʬ
		 ARP ĤˤäƼ¸ޤɥ饤ФϥФФ
		줿 ARP ֿǲꤷФؤ̿Ǹġ
		饤Ȥ̤Υϡɥɥ쥹Ѥ褦ˡ
		ϡɥɥ쥹祤󥿡եñΥϡ
		ɥ쥹Ǿ񤭤ޤ
		
		Фˤäƺ줿³μ̿ʬޤ
		Ф ARP ׵硢ɥ饤Ф ARP Υ
		饤Ȥ IP 򥳥ԡ¸ޤ饤Ȥ 
		ARP äݡΥϡɥɥ쥹Ĵ٤졢
		ɥ饤ФϷ祤󥿡եΥ졼֤ΣĤ
		ƤơΥ졼֤ΰĤФƤΥ饤Ȥα
		֤ޤʬΰ٤ ARP ĤԤᡢ
		祤󥿡եΥϡɥɥ쥹Ѥ ARP 
		׵᤬֥ɥ㥹Ȥ뤳Ȥ꤬ȯޤΤ
		ʥ֥ɥ㥹Ȥη̡饤ȤϷ祤󥿡ե
		Υϡɥɥ쥹ؽߤΥ졼֤ؤμ̿
		ʬԤޤϡ̿ʬۤ褦ʡ
		Ƥ줿ϡɥɥ쥹ƤΥ饤Ȥ
		ƥåץǡ (ARP ) ˤäưޤ
		졼֤祤󥿡եɲä줿䡢
		ƥ֤Ǥʤ졼֤٥ƥ֤ˤʤäˤ⡢
		դޤ٤Ϥη祤󥿡ե
		Ǥ⤤®٤Υ졼֤Υ롼״֤ǽ֤ (饦ɥӥ) 
		ʬۤޤ

		󥯤³줿ꡢ졼֤祤󥿡ե
		˻äꤷ硢򤵤줿 MAC ɥ쥹 ARP 
		ƥ饤ȤФƳϤˤ̿³θ
		ߥƥ֤졼˺դޤ饤Ȥ
		 ARP åǥ֥å뤳ȤΤʤ褦updelay
		modprobe ѥ᡼򥹥åΥեǥ󥰻֤Ʊ
		礭ꤷʤФޤ

		

		1. ƥ졼֤ΥԡɤĴ٤Υ١ɥ饤Ф 
                Ethtool ݡȡ

		2. ǥХץ󤵤ƤǤ⡢ǥХΥϡɥ
		ɥ쥹ǽȤ١ɥ饤Хݡȡϡ
		祤󥿡եΥϡɥɥ쥹 (current_slave) 
		ȤäƤǡ祤󥿡եγƥ졼֤ñ
		Υϡɥɥ쥹äƤ֡˥졼֤ñǤ
		٤ɬפǤcurrent_slave Ԥ硢ϡɥ
		ɥ쥹򤵤줿 current_slave ؤޤ

multicast

	ޥ㥹ȥݡȤΰ٤⡼ɤꤹ륪ץ
	Τͤϲ̤ꡣ

	disabled or 0
		̵ (ޥ㥹ȥݡȤʤ)

        active or 1
		ƥ֥Хååץ⡼ɤʡƥ֤ʥ졼֤
		ͭ

	all or 2
		ƤΥ졼֤ͭ줬ǥեȡ

primary

	ܤΥǥХȤưʸ (eth0, eth2ʤ)⤷ͤ
	Ϥ졢ǥХ饤ǤСΥǥХϽϥǥȤ
	ƺǽ˻Ѥ褦ˤʤޤΥǥХե饤ξΤߡ
	̤ΥǥХѤޤʤȡ٥ե륪СΤ
	졢ǥեȽϤ򤵤ȡ¦˾㳲ʤ
	¤ꤽ餬Ȥ³뤫Ǥϡ륹졼֤̤Υ졼
	ͥƤǤĤޤꡢ륹졼֤ 1000Mbps ¾
	졼֤ 100Mbps ξǤ1000Mbps 졼֤ԤƸ夫
	硢ΰդ 100Mbps Υ졼֤Ԥʤ®졼
	֤ʤ˥ƥ֤ʥ졼֤ˤʤˡͥ褵Ǥ礦primary 
	λ active-backup ⡼ɤξΤͭǤ

updelay

	󥯥å׾֤Τ˥󥯤ͭˤʤޤǤٱ֤ߥ
	äǻꤷޤ miimon ͤܿǤ٤Ǥʤͤ
	ݤƤޤޤǥեͤ 0 Ǥ

use_carrier

	󥯾֤ΤΤ miimon  MII 뤤 ETHTOOL ioctl
	netif_carrier_ok() ȤȤʤꤷޤMII ޤ 
	ETHTOOL ioctl ϤޤΨɤʤͥ侩Υƥॳ
	ȯԤԤޤnetif_carrier_ok() ϡȤξ֤ 
	netif_carrier_on/off Ȥäƴ뤿˥ǥХɥ饤Ф
	ʸϤ񤤤ƤǤϡۤȤɡƤǤϤʤǥХ
	ɥ饤Фεǽ򥵥ݡȤƤޤ

	󥯤夬٤Ǥʤ˥󥯤夬äƤ祤󥿡ե
	׵ᤷĤŤ硢ϥͥåȥǥХɥ饤Ф 
	netif_carrier_on/off 򥵥ݡȤʤᤫ⤷ޤ󡣤 
	netif_carrier ΥǥեȾ֤carrier onפǤ ξ硢
	user_carrier ̵ǡ󥯾֤θΤ MIIETHTOOL ioctl 
	ˡȤ褦祤󥿡ե˻ؼǤޤ

	1  netif_carrier_ok() λѤͭˤ0 Ϥޤ˾ޤʤ MII
	ETHTOOL ioctl Ѥޤǥեͤ 1 Ǥ



ʣη祤󥿡ե
================================

Ĥη祤󥿡եɬפʾ硢max_bonds ѥ᡼ (嵭) 
ꤹ뤫ǥХɥ饤ФʣɤƲmax_bonds ѥ᡼
Ѥˤꡢ㴳ʣǤϤʤʤޤѰդ줿Ƥη祤󥿡
եƱץĤȤ¤ޤɥ饤Фʣɤϡ
ɥ饤Фγƥ󥹥󥹤̤ΥץǤޤ

㤨СĤη祤󥿡եꤹݡ MII 󥯴ƻ 100
ߥñ̤Ǽ¹Ԥơ⤦ ARP 󥯴ƻ 200ߥñ̤Ǽ¹Ԥ
硢/etc/modules.conf ϲΤ褦ʷˤʤޤ

alias bond0 bonding
alias bond1 bonding

options bond0 miimon=100
options bond1 -o bonding1 arp_interval=200 arp_ip_target=10.0.0.1

ʣ ARP åȤ
===========================

ñΥåȤФ ARP ƻԤǤޤ (HA) å
åפǤϥ˥륿åȤʣĻͭѤǤñ쥿åȤξ硢
ΥåȼΤ󤷤ƤޤΤޤ󤷡ARP ׵˱Ǥʤ
ʤ꤬ȯ뤫⤷ޤɲäΥåȤĻ ARP ƻο
夵ޤ

ʣ ARP åȤϲΤ褦˥ޤǶڤʤФʤޤ

# ĤΥåȤˤ ARP ƻΰ٤Υץ

alias bond0 bonding
options bond0 arp_interval=60 arp_ip_target=192.168.0.1,192.168.0.3,192.168.0.9

ñ쥿åȤǤ⥪ץϻƤޤ

# example options for ARP monitoring with one target
alias bond0 bonding
options bond0 arp_interval=60 arp_ip_target=192.168.0.100

ARP ƻѤŪ
======================================

1. ɥ饤ФΥݡ

ARP ƻϣĤ () 뤿˥ͥåȥǥХɥ饤Ф
¸Ƥޤ
Ǹ˼ (dev->last_rx)
Ǹ (dev->trans_start)
ͥåȥǥХɥ饤Фɤ餫ͤ򹹿ʤ硢ŵŪʷ̤Ȥ
Ƥϥȥå׻祤󥿡ե󥯤󤷤¨
Τˤʤޤͥåȥƻġ (tcpdump ) Ϸ祤󥿡ե
 ARP ׵᡿ɽǤ礦

Τ뤳βȤƤϡ(a)ǥХɥ饤Ф(b)ARP
ƻߤ (㤨Сmiimon Ѥ) Ǥ

2. 롼ƥ󥰤Ǥ

ARPƻǷ祤󥿡եꤵ줿硢졼֥ǥХޥ
Х񤭤褦ʥ롼ƥ󥰤ʤ (뤤ϡŪˡ롼
ƥ󥰤ʤ) ϽפǤ㤨С祤󥿡ե bond0 
Υ졼 (eth0,eth1) äƤꡢ롼ƥ󥰥ơ֥ϼΤ褦ˤʤä
硣

Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
10.0.0.0        0.0.0.0         255.255.0.0     U        40 0          0 eth0
10.0.0.0        0.0.0.0         255.255.0.0     U        40 0          0 eth1
10.0.0.0        0.0.0.0         255.255.0.0     U        40 0          0 bond0
127.0.0.0       0.0.0.0         255.0.0.0       U        40 0          0 lo

ξ硢ARP ƻ ( ARP) Ͼͤ뤫Τޤ󡣲Τʤ顢ARP ׵
Ϥ륤󥿡ե (bond0) 줿Τˡαۤʤ륤󥿡ե
 (eth0) ϤΤʤǤα ARP ˤȤäơ׵ᤷƤ
 ARP ׵˸Ƥޤ (ARP פϥ󥿡եȤƱ
Τ)˴ƤޤޤϤ餯ɥ饤Ǽ֤򹹿
Ǥ礦ѥåȤ򼺤ޤ

βˡϴñǡ졼֤ȤΥ롼ƥ󥰤ʤݾڤƤ
ɤޤĤͳ (㤨С졼֤ޥ¦Υ롼ƥ󥰤
ʤ褦ˤ) ˤꤳɬܤʻǤ̾Ͼ嵭ݾڤޤ
üꡢäޥ˥奢롢ưŪ롼굡ǽʤɤ꤬뤳
Ȥޤ

å
==============

active-backupbalance-tlbbalance-alb ݥꥷ (⡼=156) Ѥ
ϥåꤹɬפϤޤ󤬡round-robinXORbroadcast
802.3ad ݥꥷ (⡼=0234) ѤϥåԤɬ
פޤ


祤󥿡եθ
================================

1) 祤󥿡եե
-----------------------------------
ɥ饤Фξե /proc/net/bond* ǥ쥯ȥˤޤ

mode=0  miimon=1000 ץդǷɥ饤Фɤ줿 
/proc/net/bond0/info ƤΥץϲ̤Ǥ
 
        Bonding Mode: load balancing (round-robin)
        Currently Active Slave: eth0
        MII Status: up
        MII Polling Interval (ms): 1000
        Up Delay (ms): 0
        Down Delay (ms): 0
 
        Slave Interface: eth1
        MII Status: up
        Link Failure Count: 1
 
        Slave Interface: eth0
        MII Status: up
        Link Failure Count: 1

2) ͥåȥθ
---------------------
ͥåȥ ifconfig ޥɤѤƳǧ뤳ȤǤޤ
ˤơeth0eth1 󥿡ե졼 (SLAVE) ǡbond0 󥿡
եޥ (MASTER) Ǥbond0 Υ졼֤٤Ƴƥ졼֤ͣ 
MAC ׵᤹ TLB  ALB ʳΤ٤ƤΥ⡼Ѥ bond0 Ʊ MAC ɥ
 (HWaddr) ĻդƲ


[root]# /sbin/ifconfig
bond0     Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4  
          inet addr:XXX.XXX.XXX.YYY  Bcast:XXX.XXX.XXX.255  Mask:255.255.252.0
          UP BROADCAST RUNNING MASTER MULTICAST  MTU:1500  Metric:1
          RX packets:7224794 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3286647 errors:1 dropped:0 overruns:1 carrier:0
          collisions:0 txqueuelen:0 

eth0      Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4  
          inet addr:XXX.XXX.XXX.YYY  Bcast:XXX.XXX.XXX.255  Mask:255.255.252.0
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3573025 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1643167 errors:1 dropped:0 overruns:1 carrier:0
          collisions:0 txqueuelen:100 
          Interrupt:10 Base address:0x1080 

eth1      Link encap:Ethernet  HWaddr 00:C0:F0:1F:37:B4  
          inet addr:XXX.XXX.XXX.YYY  Bcast:XXX.XXX.XXX.255  Mask:255.255.252.0
          UP BROADCAST RUNNING SLAVE MULTICAST  MTU:1500  Metric:1
          RX packets:3651769 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1643480 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          Interrupt:9 Base address:0x1400 


ˤ (FAQ)
========================

1.  ޥץåޥ (SMP) бƤޤ

	ϤŤ 2.0.xx ͥɥ饤Хѥå SMP бƤޤ
	Ǥɥ饤ФϽᤫ SMP бȤ߷פƤޤ

2.  ɤμΥɤɥ饤ФȰưޤ


	ƤΥͥåȥץɤǤ (ɤ򺮤Ǥޤ
	㤨СIntel EtherExpress PRO/100  3com 3c905b ʤ)ӥå
	ͥåȥɤ礹ޤǤޤ

3.  ɤ줰餤¿η祤󥿡եޤ

	¤Ϥޤ

4.祤󥿡եꡢɤ줰餤¿Υ졼֤Ƥޤ

	Linux ݡȤ¤Υͥåȥ󥿡եȡʤ
	ޥ˥åȤ¤Υͥåȥɿ¤ޤ

5.  ĤΥ졼֥󥯤˲ޤ

	ʤΥͥåȥɤ MII 뤤 ETHTOOL 󥯾ִƻ
	ݡȤɥ饤С MII ƻ뤬ͭˤƤ (⥸塼롦
	ѥ᡼εҤ򻲾)ԹϤޤܥ꡼ηɥ
	СϡMII ˡΤäƤꡢΥ󥯾֤˱
	졼̵֤ͭˤޤ­˴ؤƤϡˤĤƤ
	򻲾ȤƤ

	MII 򥵥ݡȤʤͥåȥɤˤĤƤϡ祤󥿡
	եư٤ˡarp_interval  arp_ip_target Υѥ
	᡼ꤷʤФʤޤ󡣥ѥåȤꤵ줿 
	arp_interval ³ʤä硢̿
	뤿 ARP ׵᤬åȤФޤδֳ֤θˡ
	Τɤ餫äʤ硢Ǽ˻ꤵƤ
	졼֤ƥ֤ʥ졼֤ˤʤǤ礦

	mii_monitor  arp_interval Τɤꤵʤ硢ɥ
	СϤξ򤢤ޤ˰ʤǤ礦ɥ饤Сϥѥå
	³Ǥ礦ĤΥѥåȤϼˤʤޤ
	Ͽǽ㲼ˤʤޤ (ĤΥ졼֥󥯤
	ԤѥåȤȾʬϼˤʤꡢ TCPUDP 
	˿Ȥʤޤ)

6.  祤󥿡եϹ (HA) ѤǤޤ

	ϤMII ƻѤƤΥͥåȥɤ MII 󥯾֥
	ݡȤ򥵥ݡȤƤС¿ξϡֹ (HA)פξϤ
	ȤƲ

7.ɤΥåƥब祤󥿡եȰ˻Ȥޤ

	饦ɥӥ⡼ɤ XOR ⡼ɤǤϡȥ󥭥󥰤򥵥ݡȤ륷
	ƥȰưޤ
	
	* ¿ Cisco åȥ롼 (EtherChannel ݡȤõƲ
	  )
	* SunTrunking եȥ
	* Alteon AceDirector åWebOS(Trunks )
	* BayStack å (trunks ΤꤵʤФʤޤ)
          å֥ǥ (450) ϰۤʤʪ˥åȤΥݡȴ֤
          trunks Ǥޤ
	* Linux 祤󥿡ե

	802.3ad ⡼ɤǤϡIEEE 802.3ad ưŪ󥯽򥵥ݡȤ륷
        Ȱưޤ

	* Extreme ͥåȥ Summit 7i (link-aggregation õƲ)
	* ¿ Cisco åȥ롼 (LACP ݡȤõƲ
	   IOS եȥι׵᤹뤫ΤޤLACP ݡȤ 
	  2002 ǯȾ Cisco ˤɲäƤޤ)
	* Foundry Big Iron 4000

	active-backupbalance-tlbbalance-alb ⡼ɤǤϡɤʥ쥤䡼II 
	åȤưޤ

8.  祤󥿡եϼȤ MAC ɥ쥹ɤƤޤ

	ifconfig Ūꤷʤ¤ꡢ祤󥿡ե MAC ɥ
	ϣܤΥ졼֥ǥХޤ MAC ɥ쥹ϣ
	ܰʹߤƤΥ졼֥ǥХˤϤ졢(ǽΥ졼֤
	줿ȤƤ) 祤󥿡ե󤷤ꡢꤵޤǱ
	³Ū˻Ĥޤ
	
	⤷ (祤󥿡ե) MAC ɥ쥹ѹΤǤС
	ifconfig ѤꤹǤޤ

	  # ifconfig bond0 hw ether 00:11:22:33:44:55

	MAC ɥ쥹Ϸ祤󥿡եΥ󡿥åפȥ졼 (뤤
	Ϥν) ѹˤäƤˤäѹޤ
	
	  # ifconfig bond0 down ; modprobe -r bonding
	  # ifconfig bond0 .... up
	  # ifenslave bond0 eth...

	ˡǤϡɲä뼡Υ졼֤鼫ưŪ˥ɥ쥹ޤ

	졼֤ MAC ɥ쥹ΤǤСΥ졼֤祤
	󥿡ե鳰(ifenslave -d bond0 eth0)
	֤ˤ (ifconfig eth0 down)ǥХɥ饤Ф򥢥ɤ (
	 rmmod 3c59x) MAC ɥ쥹 EEPROM ٤˥ɥ
	Ф٥ɤƲɥ饤ФĤΥǥХǶͭƤ
	硢Ƥ󤵤ɬפޤ¾βˡȤơ
	ư MAC ɥ쥹õƤ (dmesg ޤ tail
	/var/log/messages)ifconfig ѤƼưǥꥻåȤˡ
	ޤ

	  # ifconfig eth0 down
	  # ifconfig eth0 hw ether 00:20:40:60:80:A0

9.  ɤݥꥷȤޤ

	饦ɡӥ󡧥졼ֲν֤˴ŤϥǥХϼͭ
	졼֤Ȥ򤵤ޤѥåȤȤϴط
	ޤ

	ƥ֡ХååסƱˤñ줫ͣΥǥХǤž
	ʤݾڤޤƥ֡ХååץݥꥷϣĤΥϥ֤
	Ѥ (HA) 塼Ǥ(ֹ(HA)פξϤ
	)

	XOR(Υϡɥɥ쥹¾Ū) 򥹥졼
	ǳä;ˤƤޤΥݥꥷγƥϡɥ
	ɥ쥹ФơƱ졼֤򤷤ޤ

	֥ɥ㥹ȡ졼֥󥿡եѥåȤ
	

	802.3adXOR ١Ǥƥֽ󥿡ե֤
        ̿ʬޤ

	ʬ (balance-tlb)ߤγƥ졼֤٤˽äơ̿
	ʬޤʬϥ饤ȥ١ǡǸ٤줿졼
	֤줾쿷饤Ȥ򤵤ޤƥ졼֤٤Ϥ
	ž®٤˴ϢƷ׻졢ž®٤Υʬͭ
	ʤޤ

	Ŭʬ (balance-alb)ΰ٤ʬѤޤ
	٤Ϸ祤󥿡եκǤ®ʥƥ֥졼֤Υ롼
	ǤΤʬޤ٤ϥ饦ɥӥʬޤĤޤꡢ
	®ʥƥ֤ʥ졼֤Υ롼״֤ǼѲǽʥ졼֤ˤ
	ޤ

 (HA)
=============

ɥ饤ФѤƹ٤ˤϡɥ饤Фϥ⥸塼Ȥ
ƥѥ뤵ɬפޤΤʤ顢ΤȤ⥸塼벽ɥ饤
˥ѥ᡼ϤͣμʤǤϾѹ뤫Τޤ

 MII ޤ ETHTOOL ֥ݡȤѤƼ¸Ƥޤ桼
󥿡ե MII  ETHTOOL 󥯾֥ݡȤ򥵥ݡȤƤ
ǧɬפޤLinux ͥ 2.2.17 ǤϡƤ 100MBps б
ͥåȥɥ饤Ф Yellowfin ӥåȥͥåȥɥ饤Ф MII 򥵥ݡ
ȤƤޤETHTOOL 󥯥ݡȤ eth0 󥿡եѤǤ뤫
ȽǤ٤ˤϡethtool eth0פϤƲѤǤLink
detected:׹Ԥ󥯾֤ޤǤȦǤ⤷ʤΥƥ 
MII  ETHTOOL 򥵥ݡȤƤʤ󥿡ե硢Υ󥿡
եΥ󥯼ԤϸΤʤǤ礦ʳ miimon ͤǷɥ饤
ɤ줿ݤˡͥåȥɥ饤Фˤä MII  ETHTOOL ݡȤ
ƤʤˤϥåϿޤ

ɥ饤Ф ETHTOOL IOCTL (ETHTOOL_GLINKޥ) λѤޤ MII ơ
쥸Υåˤꡢ졼֥󥯤Ū˥å뤳
Ǥޤåֳ֤ϡ⥸塼miimon(MII ƻ) ˤäƻ
Ǥޤͤϡߥñ̤ǥå֤ɽȤޤ
α㲼ͤʤΤǡͤ (1000/HZ) (i386  10ߥ) ˶
ͤǤäƤϤޤ100 ϽͤȤɤȻפޤͤϡ
󤷤Ƥ餻 100 ߥäǥ󥯤μԤΤ̣
ޤ

㡧

   # modprobe bonding miimon=100

ޤϡ/etc/modules.conf ˼ιԤɲäޤ

   alias bond0 bonding
   options bond0 miimon=100

ߡΰ٤ˣĤΥݥꥷ¸ߤޤϼΤ줫ˤ


   a) ƥۥȤϥȥ󥭥󥰤򥵥ݡȤñۥȤޤϥå³
      Ƥ

   b) ۥȤϴĤΰۤʤ륹åޤϥȥ󥭥󥰤򥵥ݡȤƤʤ
      ñ쥹å³Ƥ롣


1) ñΥåޤϥۥȾι - ʬ
----------------------------------------------------
ϰִñǤñˡĤΥݡ (TrunkEtherChannel ʤ) 
̿ޤȤ٤˥⡼ȤδĶ (ۥȤޤϥå) ꤷ祤
󥿡եꤷƲŬڤ MII ץդǥ⥸塼
ɤ硢祤󥿡եϼưŪưǤ礦줫顢ۤʤ
󥯤Ƥߤơɥ饤ФΤ뤫Ĵ٤Ƥߤޤ
ƥȤκݡTrunk ݡȤ硢Ĺ Trunk ̵ˤХ
äĤΥåǤ뤫⤷ޤ󡣤 Linux Τ
ǤϤʤºݤϥåʤΤǤ (ǧΤ᥹åƵưƲ)

㣱ܤž®٤Υۥȡݥۥ

          +----------+                          +----------+
          |          |eth0                  eth0|          |
          | Host A   +--------------------------+  Host B  |
          |          +--------------------------+          |
          |          |eth1                  eth1|          |
          +----------+                          +----------+

줾ΥۥȾǡ

     # modprobe bonding miimon=100
     # ifconfig bond0 addr
     # ifenslave bond0 eth0 eth1

㣲ܤž®٤Υۥȡݥå

          +----------+                          +----------+
          |          |eth0                 port1|          |
          | Host A   +--------------------------+  switch  |
          |          +--------------------------+          |
          |          |eth1                 port2|          |
          +----------+                          +----------+

ۥAǡ                             åǡ

     # modprobe bonding miimon=100           # set up a trunk on port1
     # ifconfig bond0 addr                     and port2
     # ifenslave bond0 eth0 eth1             ݡȣ trunk 


2) ʣΥåˤ (ޤ trunk ݡȤʤñ쥹å)
-------------------------------------------------------------------------
Υ⡼ɤϡå𤹤Τ򤱤뤿ˡʣΥݡȤ뤦
 1 ݡȾǤΤߡΥۥȤ MAC ɥ쥹򸫤ʤȤʤȤ
ԤʤФʤʤᡢä񤷤ʤޤ

ɤΥ󥿡եƥ֤ʥ󥿡եޤɤ줬Хåå
ʤΤΤɬפϡifconfig ѤƤХååץ
եơNOARP ե饰åȤäƤޤ

Υ⡼ɤѤ٤ˤϡ⥸塼Υɻ mode=1 ѥ᡼Ϥ


    # modprobe bonding miimon=100 mode=active-backup

	ޤ

    # modprobe bonding miimon=100 mode=1

뤤ϡ/etc/modules.conf ˲򵭽ҤƲ

    alias bond0 bonding
    options bond0 miimon=100 mode=active-backup

㣱ְξǾ㳲ˤʤʤץ塼٤ΡʣΥۥ
      ʣΥåλ

                |                                     |
                |port3                           port3|
          +-----+----+                          +-----+----+
          |          |port7       ISL      port7|          |
          | switch A +--------------------------+ switch B |
          |          +--------------------------+          |
          |          |port8                port8|          |
          +----++----+                          +-----++---+
          port2||port1                           port1||port2
               ||             +-------+               ||
               |+-------------+ host1 +---------------+|
               |         eth0 +-------+ eth1           |
               |                                       |
               |              +-------+                |
               +--------------+ host2 +----------------+
                         eth0 +-------+ eth1

֤ǤϡISL = Inter Switch Link (trunk ˤʤ)ξåˤ줾
³줿ʣΥ (host1host2...)Ĥޤʣγ³ݡ
(port3) ޤƱ˳ƥۥȤñ줫ͣΥ졼֤ƥ֤Ǥꡢ
ޤƤΥ󥯤ƻ뤵Ƥޤ (ƥϥƥ֤ȥХååץ
μԤΤޤ)

ۥȤΥƥ֤ʥ󥿡եѹˡƥ֤ʥ󥿡
ե󤹤ޤǡϿ󥿡եαƤޤ
Ǥϡåžơ֥ͭ¤αƶۤȤɼޤ

host1  host2 ƱǽäƤꡢ餬̤γᥫ˥ˤ
ʬǻѤ硢Υå³줿 host1 Υƥ֥
󥿡եȡ⤦Υå³줿 host2 Υƥ֥󥿡
եΤɤǤΤ褦ʥƥϡñΥۥȡ֥롢
åμԤǤĤǤ礦åԤΥ˵ǰλ
֤ϡ̤Υåžơ֥ͭ¤ڤޤǡۥȤȾʬŪ
̿ǤʤʤȤȤǤ

㣲NIC Ԥ򤹤١ĤΥåؤ³ʣΥͥåȥ
      ɤѤ (å trunk ݡȤ׵ᤵʤ)

          +----------+                          +----------+
          |          |eth0                 port1|          |
          | Host A   +--------------------------+  switch  |
          |          +--------------------------+          |
          |          |eth1                 port2|          |
          +----------+                          +----------+

  ۥAǡ                               åǡ

     # modprobe bonding miimon=100 mode=1     # (optional) minimize the time
     # ifconfig bond0 addr                    # for table expiration
     # ifenslave bond0 eth0 eth1              (ץ)ơ֥ͭ¤ûˤ

ۥȤΥƥ֤ʥ󥿡եѹˡƥ֤ʥ󥿡
ե󤹤ޤǡ󥿡եαƤޤǤϡ
ۥȤϡåžơ֥ͭ¤˶ƶޤ


3)åΥߥ󥰤碌
--------------------------------
åХååץ⡼ɤΤĹ֤硢󥯤
˥Хååץ󥿡ե򥢥ƥ֤ˤʤۤ˾ޤ⤷
ޤ󡣥⥸塼ѥ᡼downdelay(ߥñ̡miimon ͤܿǤ
Фʤޤ) Ϥˤꡢ󥯤̵ˤ륿ߥ󥰤٤
뤳ȤǤޤ

å֡ȤåΥݡȤѲǽˤʤˡ֥󥯥åס
֤ݡȤ뤳ȤϤꤨޤϡޤǤƤʤĤ
ΥݡȤ祤󥿡ե˻ѤǷ祤󥿡ե򶸤碌
ͤޤ󡣥⥸塼롦ѥ᡼updelay(ߥñ̡miimon ܿǤ
Фʤޤ) ϤȤˤꡢƥ֤ʥ󥯤ƻѤ륿ߥ
٤餻뤳ȤǤޤ

ۥȤåȤδ֤Ǽ줿󥯤٥ͥ󤹤Ʊ
ͤξޤ (֥򴹤)

̤ʥȤơ祤󥿡եƤΥ졼֥󥯤򼺤ä礬
ޤξˤϡupdelay ѥ᡼ꤵƤȤƤ⡢ɥ饤
ФϥåפǽΥ󥯤򤹤˺ٻѤޤ (updelay׾֤Υ졼
󥿡ե硢ǽˤξ֤ä󥿡եľ˺
Ѥޤ)ϡupdelay ͤ¿硢ٻ߻֤︺뤳Ȥ
ǽˤޤ

㡧

    # modprobe bonding miimon=100 mode=1 downdelay=2000 updelay=5000
    # modprobe bonding miimon=100 mode=balance-rr downdelay=0 updelay=5000


˥åե󥰤
====================

ͥåȥİѤ˥ͥĤ˷礷 (ɥ饤ФѤʣ
Υ󥿡եν礵줿Ϥ tcpdump  ethereal뤤 
snort Τ褦 IDS ¹Ԥ)ưꤷ Promiscuous 󥿡ե
򰷤ʤФʤޤäˡifconfig bond0 upפ 
promisc ե饰ɲäʤФʤޤ󡣤 ifenslave ˥졼
󥿡եˤŬѤǤ礦ϤʴǤ

   grep bond0 /etc/modules.conf || echo alias bond0 bonding >/etc/modules.conf
   ifconfig bond0 promisc up
   for if in eth1 eth2 ...;do
       ifconfig $if up
       ifenslave bond0 $if
   done
   snort ... -i bond0 ...

 (HA) ȥ̤ͥ̿򽸤뤿ᡢifenslave ߷׵ǽ̤ꥤ󥿡
ե饤󥿡եإɥ쥹򹭤褦Ȥޤ; ϥ󥿡ե
ꤷʤǤưޤ; 줬Ϥٹ򤹤٤Ƥ̵
뤷Ƥ



====
»ϡ

  - 󥯾֤ƻ뤵ޤ¦ΥåʬŪ˥󤷤Ƥ
     (㤨СϤžʤɡ󥯤OK)󥯤̵ˤʤ
    Ǥ礦󥯤å̤ˡȤơ٤ι⤤ۥȾ
    ե졼򥫥ȤˡޤϾʥСˤ
    ŬѤǤޤ󤬡̤ΥåΥ (㤨VRRP) Υޥ
    Ⱦ䡢Фξ֥åޤǤԤͭѤ⤷
    ޤե졼򥫥Ȥ٤ˡ
    arp_interval/arp_ip_target ѥ᡼ѤƤ

ȥ
============

ߤܥɥ饤ФγȯϲƤޤ
 - http://www.sourceforge.net/projects/bonding/

Donald Becker Υͥåȥɥ饤ФȿǥץϲǸĤޤ
 - http://www.scyld.com/network/

ͥåȡNWayMII ˴Ϣξޤ www.scyld.com Ǹ
Ǥ礦

Υɥ饤ФοС󡢸ŤͥؤΥѥå줿桼
ΥġϡWilly Tarreau ΥȤ򸫤Ʋ
 - http://wtarreau.free.fr/pub/bonding/
 - http://www-miaif.lip6.fr/willy/pub/bonding/

Linux ͥ볫ȯ˴ؤǿꤹ뤿ˤϡLinux ͥ᡼
ꥹȥ֤ȤäƲ
   http://boudicca.tux.org/hypermail/linux-kernel/latest/

--  --
