Discussion:
Cannot create a macvlan private bridge on lx
(too old to reply)
Anjali Kulkarni
2014-08-13 15:57:07 UTC
Permalink
Hi,

We are trying to have a VM and a container ping each other via a private
bridge (not going through host) via macvlan interface. A bridge, lxcbr1 is
already created and contains a link from VM, and we want to add container
to it as well.
To do that, on adding the foll. config to a container, the error shown
below is seen, and tips about how to fix this issue?

Config:
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = down
lxc.network.name = eth0
lxc.network.link = lxcbr1
lxc.network.ipv4 = 1.1.1.1/24


Error seen:
lxc-start: failed to move 'lxcbr1' to the container : Invalid argument
lxc-start: failed to create the configured network
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile
and --log-priority options.

Thanks
Anjali
Serge Hallyn
2014-08-13 16:35:39 UTC
Permalink
You can't do macvlan on a bridge. It has to be done on an real
physical nic.
Post by Anjali Kulkarni
Hi,
We are trying to have a VM and a container ping each other via a private
bridge (not going through host) via macvlan interface. A bridge, lxcbr1 is
already created and contains a link from VM, and we want to add container
to it as well.
To do that, on adding the foll. config to a container, the error shown
below is seen, and tips about how to fix this issue?
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = down
lxc.network.name = eth0
lxc.network.link = lxcbr1
lxc.network.ipv4 = 1.1.1.1/24
lxc-start: failed to move 'lxcbr1' to the container : Invalid argument
lxc-start: failed to create the configured network
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile
and --log-priority options.
Thanks
Anjali
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Anjali Kulkarni
2014-08-13 17:25:22 UTC
Permalink
Thanks - is there any way to do a private bridge between a VM and a
container, so that they can communicate? What's the use case of using
macvlan on a real nic?

Anjali
Post by Serge Hallyn
You can't do macvlan on a bridge. It has to be done on an real
physical nic.
Post by Anjali Kulkarni
Hi,
We are trying to have a VM and a container ping each other via a private
bridge (not going through host) via macvlan interface. A bridge, lxcbr1 is
already created and contains a link from VM, and we want to add container
to it as well.
To do that, on adding the foll. config to a container, the error shown
below is seen, and tips about how to fix this issue?
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = down
lxc.network.name = eth0
lxc.network.link = lxcbr1
lxc.network.ipv4 = 1.1.1.1/24
lxc-start: failed to move 'lxcbr1' to the container : Invalid argument
lxc-start: failed to create the configured network
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile
and --log-priority options.
Thanks
Anjali
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
CDR
2014-08-13 17:36:56 UTC
Permalink
Do a real bridge on the host and use it on both the VM and the
container. I do it all the time.
Post by Anjali Kulkarni
Thanks - is there any way to do a private bridge between a VM and a
container, so that they can communicate? What's the use case of using
macvlan on a real nic?
Anjali
Post by Serge Hallyn
You can't do macvlan on a bridge. It has to be done on an real
physical nic.
Post by Anjali Kulkarni
Hi,
We are trying to have a VM and a container ping each other via a private
bridge (not going through host) via macvlan interface. A bridge, lxcbr1 is
already created and contains a link from VM, and we want to add container
to it as well.
To do that, on adding the foll. config to a container, the error shown
below is seen, and tips about how to fix this issue?
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = down
lxc.network.name = eth0
lxc.network.link = lxcbr1
lxc.network.ipv4 = 1.1.1.1/24
lxc-start: failed to move 'lxcbr1' to the container : Invalid argument
lxc-start: failed to create the configured network
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile
and --log-priority options.
Thanks
Anjali
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Anjali Kulkarni
2014-08-13 17:39:52 UTC
Permalink
Yes, but does this not go through the host? That is, the host's
eth0(management) has to be in this bridge? I want to be able to create
multiple such bridges, so I cannot add the eth0 of host to every such
bridge..
This works already, I want a "private" bridge between VM and container,
which does not go through the host.

Anjali
Post by CDR
Do a real bridge on the host and use it on both the VM and the
container. I do it all the time.
Post by Anjali Kulkarni
Thanks - is there any way to do a private bridge between a VM and a
container, so that they can communicate? What's the use case of using
macvlan on a real nic?
Anjali
Post by Serge Hallyn
You can't do macvlan on a bridge. It has to be done on an real
physical nic.
Post by Anjali Kulkarni
Hi,
We are trying to have a VM and a container ping each other via a private
bridge (not going through host) via macvlan interface. A bridge,
lxcbr1
is
already created and contains a link from VM, and we want to add container
to it as well.
To do that, on adding the foll. config to a container, the error shown
below is seen, and tips about how to fix this issue?
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = down
lxc.network.name = eth0
lxc.network.link = lxcbr1
lxc.network.ipv4 = 1.1.1.1/24
lxc-start: failed to move 'lxcbr1' to the container : Invalid argument
lxc-start: failed to create the configured network
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile
and --log-priority options.
Thanks
Anjali
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Michael H. Warfield
2014-08-13 17:48:06 UTC
Permalink
Post by Anjali Kulkarni
Yes, but does this not go through the host? That is, the host's
eth0(management) has to be in this bridge?
No it does not. If you're using a NATed bridge, the hosts eth0 is NOT
part of the bridge.
Post by Anjali Kulkarni
I want to be able to create
multiple such bridges, so I cannot add the eth0 of host to every such
bridge..
Then don't add eth0 to the bridge. You just create and empty bridge and
go from there.
Post by Anjali Kulkarni
This works already, I want a "private" bridge between VM and container,
which does not go through the host.
Anjali
Post by CDR
Do a real bridge on the host and use it on both the VM and the
container. I do it all the time.
Post by Anjali Kulkarni
Thanks - is there any way to do a private bridge between a VM and a
container, so that they can communicate? What's the use case of using
macvlan on a real nic?
Anjali
Post by Serge Hallyn
You can't do macvlan on a bridge. It has to be done on an real
physical nic.
Post by Anjali Kulkarni
Hi,
We are trying to have a VM and a container ping each other via a private
bridge (not going through host) via macvlan interface. A bridge,
lxcbr1
is
already created and contains a link from VM, and we want to add container
to it as well.
To do that, on adding the foll. config to a container, the error shown
below is seen, and tips about how to fix this issue?
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = down
lxc.network.name = eth0
lxc.network.link = lxcbr1
lxc.network.ipv4 = 1.1.1.1/24
lxc-start: failed to move 'lxcbr1' to the container : Invalid argument
lxc-start: failed to create the configured network
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile
and --log-priority options.
Thanks
Anjali
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
--
Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 465 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20140813/a917b449/attachment.sig>
Jäkel, Guido
2014-08-14 12:41:42 UTC
Permalink
Dear Anjali,

you'll know that a bridge acts at network layer 2, i.e. dealing just with the MACs.

In the typical usecase you want to bridge the hosts outside network to the containers. To archive this, you attach the hostside of the containers virtual NICs (which you can imagine as a "short wire" between the namespaces) to the bridge and also the hosts real NIC. While attaching to the bridge, the NICs are switches to "promiscuous mode", i.e. they don't care of matching IP addresses at layer 3 and accept any packet.

But now, how to connect the host with the outerworld, where to place the hosts layer 3 config? That's why you put this parameter set to the Linux software bridge: To act as an outgoing device of the hosts "IP stack".

If you leave this empty, the bridge is isolated from the host. If you don't attach a physical NIC to the bridge, it's isolated from the outer world.

From that Serge suggested to instanciate a bridge, attach the parties to it (layer 2) and choose some adequate layer3 network configuration to route IP traffic between them.


BTW: If you're dealing with VLANs, you may "first" attach vlan devices to your physical NIC on a trunk and "then" attach a couple of bridges to this vlan devices. This will allow you to host isolated sets of containers in different VLAN's, e.g. for staging purposes.

Guido
-----Original Message-----
From: lxc-users [mailto:lxc-users-bounces at lists.linuxcontainers.org] On Behalf Of Anjali Kulkarni
Sent: Wednesday, August 13, 2014 7:40 PM
To: LXC users mailing-list
Subject: Re: [lxc-users] Cannot create a macvlan private bridge on lx
Yes, but does this not go through the host? That is, the host's
eth0(management) has to be in this bridge? I want to be able to create
multiple such bridges, so I cannot add the eth0 of host to every such
bridge..
This works already, I want a "private" bridge between VM and container,
which does not go through the host.
Anjali
Anjali Kulkarni
2014-08-14 15:37:29 UTC
Permalink
Thanks for the great explanation.!
Anjali
Post by Jäkel, Guido
Dear Anjali,
you'll know that a bridge acts at network layer 2, i.e. dealing just with the MACs.
In the typical usecase you want to bridge the hosts outside network to
the containers. To archive this, you attach the hostside of the
containers virtual NICs (which you can imagine as a "short wire" between
the namespaces) to the bridge and also the hosts real NIC. While
attaching to the bridge, the NICs are switches to "promiscuous mode",
i.e. they don't care of matching IP addresses at layer 3 and accept any
packet.
But now, how to connect the host with the outerworld, where to place the
hosts layer 3 config? That's why you put this parameter set to the Linux
software bridge: To act as an outgoing device of the hosts "IP stack".
If you leave this empty, the bridge is isolated from the host. If you
don't attach a physical NIC to the bridge, it's isolated from the outer
world.
From that Serge suggested to instanciate a bridge, attach the parties to
it (layer 2) and choose some adequate layer3 network configuration to
route IP traffic between them.
BTW: If you're dealing with VLANs, you may "first" attach vlan devices to
your physical NIC on a trunk and "then" attach a couple of bridges to
this vlan devices. This will allow you to host isolated sets of
containers in different VLAN's, e.g. for staging purposes.
Guido
-----Original Message-----
From: lxc-users [mailto:lxc-users-bounces at lists.linuxcontainers.org] On
Behalf Of Anjali Kulkarni
Sent: Wednesday, August 13, 2014 7:40 PM
To: LXC users mailing-list
Subject: Re: [lxc-users] Cannot create a macvlan private bridge on lx
Yes, but does this not go through the host? That is, the host's
eth0(management) has to be in this bridge? I want to be able to create
multiple such bridges, so I cannot add the eth0 of host to every such
bridge..
This works already, I want a "private" bridge between VM and container,
which does not go through the host.
Anjali
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Kevin LaTona
2014-08-16 21:56:55 UTC
Permalink
I've been testing LXC's with Ubuntu 14.4 setting the containers up with static ip's connected via a bridge called br0.

These all run on my local LAN using a 192.168.x.x address behind a DSL router.

So far the container comes up fine.

Once I go inside, I can ping out to the internet.

I can not ping between containers.

But I can ping to the host machine.

Any idea's what I missed in the configs that is causing this?

Thanks
-Kevin
Kevin LaTona
2014-08-17 00:03:38 UTC
Permalink
Post by Kevin LaTona
I can not ping between containers.
I figured it out my script for setting up the container's config file was not creating a correct unique ethernet hardware address.

Fixed that and all is good.



Other than right now though when I login via the console to do the ping test.

It gives me this response and sits there a good long while before the login prompt shows up
Post by Kevin LaTona
Connected to tty 1
Type <Ctrl+a q> to exit the console, <Ctrl+a Ctrl+a> to enter Ctrl+a itself
I don't recall this happening while I was testing other LXC creation idea styles.


Anyone have any ideas what might be going on?


-Kevin
Serge Hallyn
2014-08-13 18:05:11 UTC
Permalink
Depends on what your VM looks like. You could simply create a private
bridge br1, create a tap device, bridge it to br1, pass that tap device
to kvm, then tell lxc to stick a veth endpoing onto br1.

Use case of macvlan is to get faster networking.
Post by Anjali Kulkarni
Thanks - is there any way to do a private bridge between a VM and a
container, so that they can communicate? What's the use case of using
macvlan on a real nic?
Anjali
Post by Serge Hallyn
You can't do macvlan on a bridge. It has to be done on an real
physical nic.
Post by Anjali Kulkarni
Hi,
We are trying to have a VM and a container ping each other via a private
bridge (not going through host) via macvlan interface. A bridge, lxcbr1 is
already created and contains a link from VM, and we want to add container
to it as well.
To do that, on adding the foll. config to a container, the error shown
below is seen, and tips about how to fix this issue?
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = down
lxc.network.name = eth0
lxc.network.link = lxcbr1
lxc.network.ipv4 = 1.1.1.1/24
lxc-start: failed to move 'lxcbr1' to the container : Invalid argument
lxc-start: failed to create the configured network
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the --logfile
and --log-priority options.
Thanks
Anjali
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Kike
2014-10-07 10:48:19 UTC
Permalink
Hello, I'm new with this issue and I'm having problems with containers using
lxc technology.

I'm trying to set a communication between containers but isolating them from
host network. I think is more or less the same than a previous user's
intention but I couldn't make it right.

The point is that I've set a new empty bridge but when I'm starting the
container, it fails!! I should have done something wrong but I haven't found
out it.

Could someone help me?

Commands to set the empty bridge in host:
- sudo brctl addbr lxcbr1
- sudo ip link set lxcbr1 up
- sudo brctl show


Network configuration on any container:
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = up
lxc.network.link = lxcbr1
lxc.network.hwaddr = 00:16:3e:3e:f8:7d

Error reported is: failed to move 'lxcbr1' to the container: Invalid argument

Setting as a bridge the real one (lxcbr0) doesn't work as well.
Any suggestion?




thanks in advance
Fajar A. Nugraha
2014-10-07 11:42:57 UTC
Permalink
Post by Kike
Hello, I'm new with this issue and I'm having problems with containers using
lxc technology.
I'm trying to set a communication between containers but isolating them from
host network. I think is more or less the same than a previous user's
intention but I couldn't make it right.
The point is that I've set a new empty bridge but when I'm starting the
container, it fails!! I should have done something wrong but I haven't found
out it.
Could someone help me?
- sudo brctl addbr lxcbr1
- sudo ip link set lxcbr1 up
- sudo brctl show
lxc.network.type = macvlan
lxc.network.macvlan.mode = bridge
lxc.network.flags = up
lxc.network.link = lxcbr1
Did you read the answers on this thread?

"
You can't do macvlan on a bridge. It has to be done on an real
physical nic.
"

Read the rest of this thread for recommended solution (short
version:lxc.network.type = veth) .
--
Fajar
Post by Kike
lxc.network.hwaddr = 00:16:3e:3e:f8:7d
Error reported is: failed to move 'lxcbr1' to the container: Invalid argument
Setting as a bridge the real one (lxcbr0) doesn't work as well.
Any suggestion?
thanks in advance
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Loading...