Prev: Can't use /dev/crypto with cryptosoft driver
Next: [7zip] Can't allocate required memory
From: ryanjfarley on 27 Aug 2008 15:18
I am having trouble setting up a fresh install of FreeBSD as an
ethernet bridge.
I have two boxes, call them source, 10.2.0.1, and sink 10.2.0.2. When
they are on the same network they can ping each other fine. However,
when I insert the FreeBSD box, call it bridge, between source and
sink, they can no longer reach each other.

source <--net1-->(le1)bridge0(le2)<--net2--> sink

In particular, if I ping sink from source, then I see:
On net1, as seen from both source's nic and le1: arp who-has 10.2.0.2
(sink)
On net2, as seen from sink's nic: arp who-has 10.2.0.2 and arp reply
10.2.0.2 is-at 00:0c:29:f3:64:fb
On net2, as seen from a fourth machine monitoring traffic for
debugging this: arp who-has 10.2.0.2 and arp reply 10.2.0.2 is-at
00:0c:29:f3:64:fb
On net2, as seen from le2: arp who-has 10.2.0.2

Why doesn't the arp reply 1) show up on le2 and 2) make it back to
net1 so source can see it? Is even the problem?

For some sanity checks:
If I assign le1 10.2.0.3, then bridge can ping 10.2.0.1 (source).
If I assign le2 10.2.0.3 then bridge can ping 10.2.0.2 (sink).
If I assign bridge0 10.2.0.3, then I can not ping either source or
sink.

If manually set the MAC address of 10.2.0.2 on 10.2.0.1, then I see:
On net2, nothing.
On net1, as seen from le1, nothing.
On net1, as seen from source's nic and a sniffer on the network: IP
10.2.0.1 > 10.2.0.2: icmp 64: echo request seq

I'm new to FreeBSD (previously long time Linux user), so there could
be something basic I'm missing. If it matters, all three machines are
VMWare virtual machines and the networks are virtual networks.

I've been banging my head against a wall all day trying to figure this
out, please help. Thank you,
Ryan

[user(a)bridge]$ uname -a
FreeBSD bridge.localdomain 7.0-RELEASE FreeBSD 7.0-RELEASE #0: Sun Feb
24 19:59:52 UTC 2008 root(a)logan.cse.buffalo.edu:/usr/obj/usr/src/
sys/GENERIC i386

[user(a)bridge]$ ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x5
inet6 ::1 prefixlen 128
inet 127.0.0.1 netmask 0xff000000
le0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu
1500
options=8<VLAN_MTU>
ether 00:0c:29:1c:c0:a8
inet 172.16.0.129 netmask 0xffffff00 broadcast 172.16.0.255
media: Ethernet autoselect
status: active
le1: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
0 mtu 1500
options=8<VLAN_MTU>
ether 00:0c:29:1c:c0:b2
media: Ethernet autoselect
status: active
le2: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric
0 mtu 1500
options=8<VLAN_MTU>
ether 00:0c:29:1c:c0:bc
media: Ethernet autoselect
status: active
bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0
mtu 1500
ether 66:0c:0e:5f:57:56
id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15
maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200
root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0
member: le2 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>
member: le1 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP>


[user(a)bridge]$ sysctl -a | grep bridge
kern.hostname: voipbridge.localdomain
net.link.bridge.ipfw: 0
net.link.bridge.log_stp: 0
net.link.bridge.pfil_local_phys: 0
net.link.bridge.pfil_member: 1
net.link.bridge.pfil_bridge: 1
net.link.bridge.ipfw_arp: 0
net.link.bridge.pfil_onlyip: 1
dev.pcib.0.%desc: ACPI Host-PCI bridge
dev.pcib.1.%desc: ACPI PCI-PCI bridge
dev.hostb.0.%desc: Host to PCI bridge
dev.agp.0.%desc: Intel 82443BX (440 BX) host to PCI bridge
dev.isab.0.%desc: PCI-ISA bridge

[user(a)bridge]$ sysctl -a | grep ether
kern.random.sys.harvest.ethernet: 1
net.link.ether.inet.log_arp_permanent_modify: 1
net.link.ether.inet.log_arp_movements: 1
net.link.ether.inet.log_arp_wrong_iface: 1
net.link.ether.inet.proxyall: 0
net.link.ether.inet.useloopback: 1
net.link.ether.inet.maxtries: 5
net.link.ether.inet.max_age: 1200
net.link.ether.ipfw: 0

[user(a)bridge]$ sysctl -a | grep icmp
net.inet.icmp.maskrepl: 0
net.inet.icmp.icmplim: 200
net.inet.icmp.bmcastecho: 0
net.inet.icmp.quotelen: 8
net.inet.icmp.reply_from_interface: 0
net.inet.icmp.reply_src:
net.inet.icmp.icmplim_output: 1
net.inet.icmp.log_redirect: 0
net.inet.icmp.drop_redirect: 0
net.inet.icmp.maskfake: 0
net.inet.tcp.icmp_may_rst: 1
net.inet6.icmp6.rediraccept: 1
net.inet6.icmp6.redirtimeout: 600
net.inet6.icmp6.nd6_prune: 1
net.inet6.icmp6.nd6_delay: 5
net.inet6.icmp6.nd6_umaxtries: 3
net.inet6.icmp6.nd6_mmaxtries: 3
net.inet6.icmp6.nd6_useloopback: 1
net.inet6.icmp6.nodeinfo: 3
net.inet6.icmp6.errppslimit: 100
net.inet6.icmp6.nd6_maxnudhint: 0
net.inet6.icmp6.nd6_debug: 0
net.inet6.icmp6.nd6_maxqueuelen: 1

[user(a)bridge]$ sysctl -a | grep arp
net.inet.ip.same_prefix_carp_only: 0
net.link.ether.inet.log_arp_permanent_modify: 1
net.link.ether.inet.log_arp_movements: 1
net.link.ether.inet.log_arp_wrong_iface: 1
net.link.bridge.ipfw_arp: 0

[user(a)bridge]$ cat /boot/loader.conf
#bridge_load="YES"
#ipfw_load="YES"
ipdivert_load="YES"
if_bridge_load="YES"
bridgestp_load="YES"

[user(a)bridge]$ cat /etc/rc.conf
# -- sysinstall generated deltas -- # Mon Aug 25 17:23:46 2008
# Created: Mon Aug 25 17:23:46 2008
# Enable network daemons for user convenience.
# Please make all changes to this file, not to /etc/defaults/rc.conf.
# This file now contains just the overrides from /etc/defaults/
rc.conf.
gateway_enable="YES"
hostname="voipbridge.localdomain"
ifconfig_le0="DHCP"
inetd_enable="NO"
linux_enable="YES"
sshd_enable="YES"

gateway_enable="YES"

firewall_enable="YES"
firewall_type="open"
firewall_quiet="YES"
firewall_logging="YES"

cloned_interfaces="bridge0"
ifconfig_bridge0="addm le1 addm le2 up"
ifconfig_le1="up"
ifconfig_le2="up"

[user(a)bridge]$ ipfw show
00100 0 0 allow ip from any to any via lo0
00200 0 0 deny ip from any to 127.0.0.0/8
00300 0 0 deny ip from 127.0.0.0/8 to any
65000 0 0 allow ip from any to any
65535 0 0 deny ip from any to any

From: Jason Bourne on 27 Aug 2008 20:24
ryanjfarley(a)gmail.com wrote:

> I am having trouble setting up a fresh install of FreeBSD as an
> ethernet bridge.
> I have two boxes, call them source, 10.2.0.1, and sink 10.2.0.2. When
> they are on the same network they can ping each other fine. However,
> when I insert the FreeBSD box, call it bridge, between source and
> sink, they can no longer reach each other.
>
> source <--net1-->(le1)bridge0(le2)<--net2--> sink
>
> In particular, if I ping sink from source, then I see:
> On net1, as seen from both source's nic and le1: arp who-has 10.2.0.2
> (sink)
> On net2, as seen from sink's nic: arp who-has 10.2.0.2 and arp reply
> 10.2.0.2 is-at 00:0c:29:f3:64:fb
> On net2, as seen from a fourth machine monitoring traffic for
> debugging this: arp who-has 10.2.0.2 and arp reply 10.2.0.2 is-at
> 00:0c:29:f3:64:fb
> On net2, as seen from le2: arp who-has 10.2.0.2
>
> Why doesn't the arp reply 1) show up on le2 and 2) make it back to
> net1 so source can see it? Is even the problem?
>
[snippage]

It has been since a very long time that I've played with a bridge. I don't
have the magic bullet answer and my memory is very hazy at this point, but
I believe at first glance what you may be missing is arp-proxy. It's built
into the arp command, so man arp for more details.

Example from googling:

% arp -s 1.2.3.4 AA:BB:CC:11:22:33 pub

To verify it worked:

% arp -a

(1.2.3.100) at aa:bb:cc:11:22:33 on ed0 [ethernet]
(1.2.3.4.) at aa:bb:cc:11:22:33 on ed0 permanent published [ethernet]

The pub or "published" switch is what invokes the ARP proxy.

Your situation is more complicated and will take some figuring and
configuring, but this may be the missing piece. Try googling for more stuff
on arp-proxy for a handle. YMMV

-Mike

From: rfarley on 28 Aug 2008 22:25
On Aug 27, 8:24 pm, Jason Bourne <j_bourne_treadst...(a)hotmail.com>
wrote:
> ryanjfar...(a)gmail.com wrote:
> > I am having trouble setting up a fresh install of FreeBSD as an
> > ethernet bridge.
> > I have two boxes, call them source, 10.2.0.1, and sink 10.2.0.2. When
> > they are on the same network they can ping each other fine. However,
> > when I insert the FreeBSD box, call it bridge, between source and
> > sink, they can no longer reach each other.
>
> > source <--net1-->(le1)bridge0(le2)<--net2--> sink
>
> > In particular, if I ping sink from source, then I see:
> > On net1, as seen from both source's nic and le1: arp who-has 10.2.0.2
> > (sink)
> > On net2, as seen from sink's nic: arp who-has 10.2.0.2 and arp reply
> > 10.2.0.2 is-at 00:0c:29:f3:64:fb
> > On net2, as seen from a fourth machine monitoring traffic for
> > debugging this: arp who-has 10.2.0.2 and arp reply 10.2.0.2 is-at
> > 00:0c:29:f3:64:fb
> > On net2, as seen from le2: arp who-has 10.2.0.2
>
> > Why doesn't the arp reply 1) show up on le2 and 2) make it back to
> > net1 so source can see it? Is even the problem?
>
> [snippage]
>
> It has been since a very long time that I've played with a bridge. I don't
> have the magic bullet answer and my memory is very hazy at this point, but
> I believe at first glance what you may be missing is arp-proxy. It's built
> into the arp command, so man arp for more details.
>
> Example from googling:
>
> % arp -s 1.2.3.4 AA:BB:CC:11:22:33 pub
>
> To verify it worked:
>
> % arp -a
>
> (1.2.3.100) at aa:bb:cc:11:22:33 on ed0 [ethernet]
> (1.2.3.4.) at aa:bb:cc:11:22:33 on ed0 permanent published [ethernet]
>
> The pub or "published" switch is what invokes the ARP proxy.
>
> Your situation is more complicated and will take some figuring and
> configuring, but this may be the missing piece. Try googling for more stuff
> on arp-proxy for a handle. YMMV
>
> -Mike

I've since looked into arp-proxy and it unfortunately doesn't work. I
may look into netgraph, but am unsure. I've tried a Linux ethernet
bridge in place of the FreeBSD bridge and have gotten the same
results. This seems like the most trivial bridge example, I feel dumb.

The strange thing is that if I make net1 and net2 different subnets
and make bridge an IP router between them, then source can ping sink
so it's not a physical connectivity issue. I'm looking into whether
this is a problem due to running on vmnets -- I'll use the same config
for bridge but on a physical machine. I'm wondering if this isn't a
config or even FreeBSD thing, but possibly bc vmware handles something
funky.

-Ryan

For completeness, here's an email I sent a colleague trying to figure
this out:
source:eth0(00:0c:29:be:f3:7f|10.2.0.1) -> le1:bridge:le2 -
>sink:eth0(00:0c:29:45:d1:77|10.2.0.2)
bridge0 on host bridge has members le1 and le2.

[root(a)bridge /usr/home/gmusip]# ifconfig bridge0 addr
00:0c:29:be:f3:7f Vlan1 le1 1084 flags=0<>
00:0c:29:1c:c0:a8 Vlan1 le2 810 flags=0<>

Here is a sample trace of a frame when source runs 'ping 10.2.0.2'
As seen from source:eth0
00:0c:29:be:f3:7f (oui Unknown) > Broadcast, ethertype ARP (0x0806),
length 60: arp who-has 10.2.0.2 tell 10.2.0.1

As seen from bridge:le1
00:0c:29:be:f3:7f (oui Unknown) > Broadcast, ethertype ARP (0x0806),
length 60: arp who-has 10.2.0.2 tell 10.2.0.1

As seen from the vmware host on the vmnet that contains source:eth0
and bridge:le1
00:0c:29:be:f3:7f > Broadcast, ethertype ARP (0x0806), length 60: arp
who-has 10.2.0.2 tell 10.2.0.1

As seen from bridge:bridge0
00:0c:29:be:f3:7f > Broadcast, ethertype ARP (0x0806), length 60: arp
who-has 10.2.0.2 tell 10.2.0.1

As seen from bridge:le2
00:0c:29:be:f3:7f (oui Unknown) > Broadcast, ethertype ARP (0x0806),
length 60: arp who-has 10.2.0.2 tell 10.2.0.1

As seen from sink:eth0
00:0c:29:be:f3:7f (oui Unknown) > Broadcast, ethertype ARP (0x0806),
length 60: arp who-has 10.2.0.2 tell 10.2.0.1
00:0c:29:45:d1:77 (oui Unknown) > 00:0c:29:be:f3:7f (oui Unknown),
ethertype ARP (0x0806), length 42: arp reply 10.2.0.2 is-at 00:0c:
29:45:d1:77 (oui Unknown)

As seen from the vmware host on the vmnet that contains bridge:le2 and
sink:eth0
00:0c:29:be:f3:7f > Broadcast, ethertype ARP (0x0806), length 60: arp
who-has 10.2.0.2 tell 10.2.0.1
00:0c:29:45:d1:77 > 00:0c:29:be:f3:7f, ethertype ARP (0x0806),
length42: arp reply 10.2.0.2 is-at 00:0c:29:45:d1:77

Note that bridge:le2 and sink:eth0 are be on the same network, yet
they are seeing different things.

I've tried the setup with NAT'ed vmnets and the results are the same
as hostonly vmnets. I've tried a Linux bridge and the results are the
same as a FreeBSD bridge.

Here is the trace when source runs 'arp -s 10.2.0.2 00:0c:29:45:d1:77
&& ping 10.2.0.2'
As seen from source:eth0
00:0c:29:be:f3:7f (oui Unknown) > 00:0c:29:45:d1:77 (oui Unknown),
ethertype IPv4 (0x0800), length 98: 10.2.0.1 > 10.2.0.2: ICMP echo
request, id 15624, seq 1603, length 64

As seen from bridge:le1
<nothing>

As seen from the vmware host on the vmnet that contains source:eth0
and bridge:le1
00:0c:29:be:f3:7f > 00:0c:29:45:d1:77, ethertype IPv4 (0x0800), length
98: IP 10.2.0.1 > 10.2.0.2: icmp 64: echo request seq 1603

As seen from bridge:bridge0
<nothing>

As seen from bridge:le2
<nothing>

As seen from sink:eth0
<nothing>

As seen from the vmware host on the vmnet that contains bridge:le2 and
sink:eth0
<nothing>

From: Christoph Weber-Fahr on 29 Aug 2008 17:20
rfarley wrote:
> I've since looked into arp-proxy and it unfortunately doesn't work.

I doubted it anyway. You don't need proxy arp, you need a bridge.

> This seems like the most trivial bridge example, I feel dumb.

Can you tell us what you actually configured on your BSD box?

And, just for curiosity's sake - why not just take a small
switch for 20 bucks and let that one do the bridging? Is this
a "I want to get this working" thing or a "I need special
features the vanilla switch doesn't give me" issue?

If the latter, which exactly?

Regards

Christop
From: Dead Paul on 30 Aug 2008 07:21
On Wed, 27 Aug 2008 12:18:49 -0700, ryanjfarley(a)gmail.com wrote:

> I am having trouble setting up a fresh install of FreeBSD as an ethernet
> bridge.
> I have two boxes, call them source, 10.2.0.1, and sink 10.2.0.2. When they
> are on the same network they can ping each other fine. However, when I
> insert the FreeBSD box, call it bridge, between source and sink, they can
> no longer reach each other.
>
> source <--net1-->(le1)bridge0(le2)<--net2--> sink
>
> In particular, if I ping sink from source, then I see: On net1, as seen
> from both source's nic and le1: arp who-has 10.2.0.2 (sink)
> On net2, as seen from sink's nic: arp who-has 10.2.0.2 and arp reply
> 10.2.0.2 is-at 00:0c:29:f3:64:fb
> On net2, as seen from a fourth machine monitoring traffic for debugging
> this: arp who-has 10.2.0.2 and arp reply 10.2.0.2 is-at 00:0c:29:f3:64:fb
> On net2, as seen from le2: arp who-has 10.2.0.2
>
> Why doesn't the arp reply 1) show up on le2 and 2) make it back to net1 so
> source can see it? Is even the problem?
>
> For some sanity checks:
> If I assign le1 10.2.0.3, then bridge can ping 10.2.0.1 (source). If I
> assign le2 10.2.0.3 then bridge can ping 10.2.0.2 (sink). If I assign
> bridge0 10.2.0.3, then I can not ping either source or sink.
>
> If manually set the MAC address of 10.2.0.2 on 10.2.0.1, then I see: On
> net2, nothing.
> On net1, as seen from le1, nothing.
> On net1, as seen from source's nic and a sniffer on the network: IP
> 10.2.0.1 > 10.2.0.2: icmp 64: echo request seq
>
> I'm new to FreeBSD (previously long time Linux user), so there could be
> something basic I'm missing. If it matters, all three machines are VMWare
> virtual machines and the networks are virtual networks.

When you configure your virtual machines you have the opportunity to
configure networking. Perhaps you should look at vmware help regarding
networking between boxes. Getting vmware to network realboxes is trivial
but perhaps there is some extra info required when networking virtual
boxes. I dunno. never done it myself. Just my 2 cents.

--
___ _______ ___ ___ ___ __ ____
/ _ \/ __/ _ | / _ \ / _ \/ _ |/ / / / /
/ // / _// __ |/ // / / ___/ __ / /_/ / /__
/____/___/_/ |_/____/ /_/ /_/ |_\____/____/

 |  Next  |  Last
Pages: 1 2
Prev: Can't use /dev/crypto with cryptosoft driver
Next: [7zip] Can't allocate required memory