Discussion:
Limit file descriptors
(too old to reply)
Alan Hoffmeister
2016-03-16 23:40:31 UTC
Permalink
Hello folks,

I'm having some troubles where one container can drain the amount of file
descriptors available in the host system. Does somebody knows how to limit
file descriptors per container?

I'm running lxd v2.0.0.rc3 and lxc v2.0.0.rc10

Cheers

--
Alan Hoffmeister
https://twitter.com/alan_hoff
https://github.com/alanhoff
https://keybase.io/alanhoff
Guido Jäkel
2016-03-18 06:08:57 UTC
Permalink
Dear Alan,

adjust the limits per Container in /etc/security/limits.conf , e.g. add something like


* hard nofile 8192
* soft nofile 8000
* hard nproc 1024
* soft nproc 1000

and maybe adjust the values at the host, too.


Greetings

Guido
Post by Alan Hoffmeister
Hello folks,
I'm having some troubles where one container can drain the amount of file
descriptors available in the host system. Does somebody knows how to limit
file descriptors per container?
I'm running lxd v2.0.0.rc3 and lxc v2.0.0.rc10
Cheers
--
Alan Hoffmeister
https://twitter.com/alan_hoff
https://github.com/alanhoff
https://keybase.io/alanhoff
Hans Deragon
2016-03-18 13:10:13 UTC
Permalink
Greetings,

Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw
and setup details at the end of this email.

I got the container running and with DHCP configured, it has its own IP
which the host can address with.

Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I
followed instructions from https://wiki.debian.org/LXC/SimpleBridge.
But turns out that I am probably running a different version of LXC and
that this page is now obsolete.

I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container).
This seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.

I found 'man lxc.container.conf'. Seams promising. However, I fail to
find within the manual the path where this file should be saved! If you
write documentation, please always provide the path where configuration
files are supposed to be stored.

I created a profile named 'bridged' using commands, but I have not found
any option/instruction on how to apply that profile on my existing
image. 'lxc start server2' does not provide any option to start the
container with a particular profile. BTW, where are profile
configuration files stored?

I need clear step by step instructions, with full paths on how to set
things up and I fail to find any on the web. Anybody has a useful link
to suggest?

I have a KVM image running (server1) and it works flawlessly with a
static IP on my bridge. And it wasn't hard to find instructions on how
to set it up. But LXD/LXc is another story.

The setup:

Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1

Best regards and thanks in advance,
Hans Deragon
Sean McNamara
2016-03-18 14:18:33 UTC
Permalink
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.

I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.

raw.lxc: "lxc.network.ipv4=1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
devices:
eth0:
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0

On Ubuntu, you can then set up your bridge as follows in
/etc/network/interfaces:

auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off


This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.

It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.

Thanks,

Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw and
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own IP
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I followed
instructions from https://wiki.debian.org/LXC/SimpleBridge. But turns out
that I am probably running a different version of LXC and that this page is
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container). This
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail to find
within the manual the path where this file should be saved! If you write
documentation, please always provide the path where configuration files are
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not found any
option/instruction on how to apply that profile on my existing image. 'lxc
start server2' does not provide any option to start the container with a
particular profile. BTW, where are profile configuration files stored?
I need clear step by step instructions, with full paths on how to set things
up and I fail to find any on the web. Anybody has a useful link to suggest?
I have a KVM image running (server1) and it works flawlessly with a static
IP on my bridge. And it wasn't hard to find instructions on how to set it
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Stéphane Graber
2016-03-18 15:43:24 UTC
Permalink
Our stance hasn't changed. LXD doesn't know nor care about layer-3
networking, all it does is setup your layer-2.

Having LXD pre-initialize your network namespace confuses the heck out
of a bunch of distros which expect all network to be unconfigured by the
time they apply their own config (they don't clean things up so
duplicate entries lead to failure).


Nevertheless, we have recently allowed the following key through raw.lxc:
- lxc.network.X.ipv4
- lxc.network.X.ipv4.gateway
- lxc.network.X.ipv6
- lxc.network.X.ipv6.gateway

Note that we require you set the interface index (X above) as mixing
those raw entris with the LXD generated config would otherwise randomly
cause an invalid config and container startup failure.


The recommended way to manage IPs with LXD is to do it exactly the same
way you would do it for your VMs or physical machines, so either
configure your DHCP server to give a static lease or configure the
container to use a static IP (you can use lxc file pull/push/edit to do
it on a stopped container).
Post by Sean McNamara
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.
I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.
raw.lxc: "lxc.network.ipv4=1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0
On Ubuntu, you can then set up your bridge as follows in
auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off
This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.
It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.
Thanks,
Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw and
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own IP
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I followed
instructions from https://wiki.debian.org/LXC/SimpleBridge. But turns out
that I am probably running a different version of LXC and that this page is
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container). This
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail to find
within the manual the path where this file should be saved! If you write
documentation, please always provide the path where configuration files are
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not found any
option/instruction on how to apply that profile on my existing image. 'lxc
start server2' does not provide any option to start the container with a
particular profile. BTW, where are profile configuration files stored?
I need clear step by step instructions, with full paths on how to set things
up and I fail to find any on the web. Anybody has a useful link to suggest?
I have a KVM image running (server1) and it works flawlessly with a static
IP on my bridge. And it wasn't hard to find instructions on how to set it
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
Sean McNamara
2016-03-18 16:09:40 UTC
Permalink
Post by Stéphane Graber
Our stance hasn't changed. LXD doesn't know nor care about layer-3
networking, all it does is setup your layer-2.
Having LXD pre-initialize your network namespace confuses the heck out
of a bunch of distros which expect all network to be unconfigured by the
time they apply their own config (they don't clean things up so
duplicate entries lead to failure).
Okay.

As someone migrating from OpenVZ (and before that, VMware), one
important use case I was expecting of LXD is that of multi-tenant
boxes, where you need to give root access to a container to the
"tenant", and expect them to adhere to a Terms of Service agreement,
but need to have technical mitigations in place, so that even if they
decide to violate the ToS (or innocently have their box hacked by a
malicious third-party who decides to violate the ToS), access to other
containers and the physical box (host OS) is very difficult to
impossible (pending any undiscovered vulnerabilities or host-side
misconfiguration).

As part of that, I was expecting some way to tell LXD to restrict the
IP addresses that can be claimed/used by a given container. For
instance, if I have a public Internet IPv4 /26 allocated to a physical
host by a hosting provider, I'll want to assign only one or two IP
addresses to each container. Currently, I can have an LXD container
just spuriously decide to use any arbitrary IP, and I haven't found a
way to prevent it from doing that if an untrusted user has root access
in the container. They can just run ifconfig and specify the IP
address they want to use.

How can I configure the host environment (LXD or something else on the
host, assuming I'm running a very recent Ubuntu 16.04 Beta nightly) so
that no packets can be transmitted to/from the guest unless the guest
is using a specific IP or set of IPs? I also want to make sure that no
broadcasting is occurring; i.e., the root user in the container should
not be able to sniff layer 2 and see all the packets going to all the
other containers.

...Or is LXD not suitable for this use case? If it isn't, will it ever be?

Thanks,

Sean
Post by Stéphane Graber
- lxc.network.X.ipv4
- lxc.network.X.ipv4.gateway
- lxc.network.X.ipv6
- lxc.network.X.ipv6.gateway
Note that we require you set the interface index (X above) as mixing
those raw entris with the LXD generated config would otherwise randomly
cause an invalid config and container startup failure.
The recommended way to manage IPs with LXD is to do it exactly the same
way you would do it for your VMs or physical machines, so either
configure your DHCP server to give a static lease or configure the
container to use a static IP (you can use lxc file pull/push/edit to do
it on a stopped container).
Post by Sean McNamara
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.
I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.
raw.lxc: "lxc.network.ipv4=1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0
On Ubuntu, you can then set up your bridge as follows in
auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off
This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.
It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.
Thanks,
Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw and
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own IP
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I followed
instructions from https://wiki.debian.org/LXC/SimpleBridge. But turns out
that I am probably running a different version of LXC and that this page is
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container). This
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail to find
within the manual the path where this file should be saved! If you write
documentation, please always provide the path where configuration files are
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not found any
option/instruction on how to apply that profile on my existing image. 'lxc
start server2' does not provide any option to start the container with a
particular profile. BTW, where are profile configuration files stored?
I need clear step by step instructions, with full paths on how to set things
up and I fail to find any on the web. Anybody has a useful link to suggest?
I have a KVM image running (server1) and it works flawlessly with a static
IP on my bridge. And it wasn't hard to find instructions on how to set it
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Sean McNamara
2016-03-18 16:15:35 UTC
Permalink
Post by Sean McNamara
Post by Stéphane Graber
Our stance hasn't changed. LXD doesn't know nor care about layer-3
networking, all it does is setup your layer-2.
Having LXD pre-initialize your network namespace confuses the heck out
of a bunch of distros which expect all network to be unconfigured by the
time they apply their own config (they don't clean things up so
duplicate entries lead to failure).
Okay.
As someone migrating from OpenVZ (and before that, VMware), one
important use case I was expecting of LXD is that of multi-tenant
boxes, where you need to give root access to a container to the
"tenant", and expect them to adhere to a Terms of Service agreement,
but need to have technical mitigations in place, so that even if they
decide to violate the ToS (or innocently have their box hacked by a
malicious third-party who decides to violate the ToS), access to other
containers and the physical box (host OS) is very difficult to
impossible (pending any undiscovered vulnerabilities or host-side
misconfiguration).
As part of that, I was expecting some way to tell LXD to restrict the
IP addresses that can be claimed/used by a given container. For
instance, if I have a public Internet IPv4 /26 allocated to a physical
host by a hosting provider, I'll want to assign only one or two IP
addresses to each container. Currently, I can have an LXD container
just spuriously decide to use any arbitrary IP, and I haven't found a
way to prevent it from doing that if an untrusted user has root access
in the container. They can just run ifconfig and specify the IP
address they want to use.
How can I configure the host environment (LXD or something else on the
host, assuming I'm running a very recent Ubuntu 16.04 Beta nightly) so
Just wanted to clarify that I am *not* using or intending to use a
pre-release of 16.04 in a production environment. I'm currently
satisfied with LXD 0.24 on Ubuntu Server 14.04.4 LTS. I'm not
currently in a situation where I have untrusted root users with access
to containers, but I am planning to open up that type of usage in the
future if LXD turns out to be able to support it. And of course that
would be using the final release of Ubuntu Server 16.04 LTS.

Thanks,

Sean
Post by Sean McNamara
that no packets can be transmitted to/from the guest unless the guest
is using a specific IP or set of IPs? I also want to make sure that no
broadcasting is occurring; i.e., the root user in the container should
not be able to sniff layer 2 and see all the packets going to all the
other containers.
...Or is LXD not suitable for this use case? If it isn't, will it ever be?
Thanks,
Sean
Post by Stéphane Graber
- lxc.network.X.ipv4
- lxc.network.X.ipv4.gateway
- lxc.network.X.ipv6
- lxc.network.X.ipv6.gateway
Note that we require you set the interface index (X above) as mixing
those raw entris with the LXD generated config would otherwise randomly
cause an invalid config and container startup failure.
The recommended way to manage IPs with LXD is to do it exactly the same
way you would do it for your VMs or physical machines, so either
configure your DHCP server to give a static lease or configure the
container to use a static IP (you can use lxc file pull/push/edit to do
it on a stopped container).
Post by Sean McNamara
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.
I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.
raw.lxc: "lxc.network.ipv4=1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0
On Ubuntu, you can then set up your bridge as follows in
auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off
This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.
It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.
Thanks,
Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw and
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own IP
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I followed
instructions from https://wiki.debian.org/LXC/SimpleBridge. But turns out
that I am probably running a different version of LXC and that this page is
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container). This
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail to find
within the manual the path where this file should be saved! If you write
documentation, please always provide the path where configuration files are
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not found any
option/instruction on how to apply that profile on my existing image. 'lxc
start server2' does not provide any option to start the container with a
particular profile. BTW, where are profile configuration files stored?
I need clear step by step instructions, with full paths on how to set things
up and I fail to find any on the web. Anybody has a useful link to suggest?
I have a KVM image running (server1) and it works flawlessly with a static
IP on my bridge. And it wasn't hard to find instructions on how to set it
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Stéphane Graber
2016-03-18 16:29:13 UTC
Permalink
Post by Sean McNamara
Post by Sean McNamara
Post by Stéphane Graber
Our stance hasn't changed. LXD doesn't know nor care about layer-3
networking, all it does is setup your layer-2.
Having LXD pre-initialize your network namespace confuses the heck out
of a bunch of distros which expect all network to be unconfigured by the
time they apply their own config (they don't clean things up so
duplicate entries lead to failure).
Okay.
As someone migrating from OpenVZ (and before that, VMware), one
important use case I was expecting of LXD is that of multi-tenant
boxes, where you need to give root access to a container to the
"tenant", and expect them to adhere to a Terms of Service agreement,
but need to have technical mitigations in place, so that even if they
decide to violate the ToS (or innocently have their box hacked by a
malicious third-party who decides to violate the ToS), access to other
containers and the physical box (host OS) is very difficult to
impossible (pending any undiscovered vulnerabilities or host-side
misconfiguration).
As part of that, I was expecting some way to tell LXD to restrict the
IP addresses that can be claimed/used by a given container. For
instance, if I have a public Internet IPv4 /26 allocated to a physical
host by a hosting provider, I'll want to assign only one or two IP
addresses to each container. Currently, I can have an LXD container
just spuriously decide to use any arbitrary IP, and I haven't found a
way to prevent it from doing that if an untrusted user has root access
in the container. They can just run ifconfig and specify the IP
address they want to use.
How can I configure the host environment (LXD or something else on the
host, assuming I'm running a very recent Ubuntu 16.04 Beta nightly) so
Just wanted to clarify that I am *not* using or intending to use a
pre-release of 16.04 in a production environment. I'm currently
satisfied with LXD 0.24 on Ubuntu Server 14.04.4 LTS. I'm not
currently in a situation where I have untrusted root users with access
to containers, but I am planning to open up that type of usage in the
future if LXD turns out to be able to support it. And of course that
would be using the final release of Ubuntu Server 16.04 LTS.
Thanks,
Sean
Note that the latest 2.0 snapshot is currently available in
trusty-backports so you don't need to be using pre-release 16.04.

We usually update trusty-backports just a couple of hours after pushing
the new version to 16.04.
Post by Sean McNamara
Post by Sean McNamara
that no packets can be transmitted to/from the guest unless the guest
is using a specific IP or set of IPs? I also want to make sure that no
broadcasting is occurring; i.e., the root user in the container should
not be able to sniff layer 2 and see all the packets going to all the
other containers.
...Or is LXD not suitable for this use case? If it isn't, will it ever be?
Thanks,
Sean
Post by Stéphane Graber
- lxc.network.X.ipv4
- lxc.network.X.ipv4.gateway
- lxc.network.X.ipv6
- lxc.network.X.ipv6.gateway
Note that we require you set the interface index (X above) as mixing
those raw entris with the LXD generated config would otherwise randomly
cause an invalid config and container startup failure.
The recommended way to manage IPs with LXD is to do it exactly the same
way you would do it for your VMs or physical machines, so either
configure your DHCP server to give a static lease or configure the
container to use a static IP (you can use lxc file pull/push/edit to do
it on a stopped container).
Post by Sean McNamara
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.
I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.
raw.lxc: "lxc.network.ipv4=1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0
On Ubuntu, you can then set up your bridge as follows in
auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off
This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.
It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.
Thanks,
Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw and
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own IP
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I followed
instructions from https://wiki.debian.org/LXC/SimpleBridge. But turns out
that I am probably running a different version of LXC and that this page is
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container). This
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail to find
within the manual the path where this file should be saved! If you write
documentation, please always provide the path where configuration files are
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not found any
option/instruction on how to apply that profile on my existing image. 'lxc
start server2' does not provide any option to start the container with a
particular profile. BTW, where are profile configuration files stored?
I need clear step by step instructions, with full paths on how to set things
up and I fail to find any on the web. Anybody has a useful link to suggest?
I have a KVM image running (server1) and it works flawlessly with a static
IP on my bridge. And it wasn't hard to find instructions on how to set it
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
Stéphane Graber
2016-03-18 16:33:46 UTC
Permalink
Post by Sean McNamara
Post by Sean McNamara
Post by Stéphane Graber
Our stance hasn't changed. LXD doesn't know nor care about layer-3
networking, all it does is setup your layer-2.
Having LXD pre-initialize your network namespace confuses the heck out
of a bunch of distros which expect all network to be unconfigured by the
time they apply their own config (they don't clean things up so
duplicate entries lead to failure).
Okay.
As someone migrating from OpenVZ (and before that, VMware), one
important use case I was expecting of LXD is that of multi-tenant
boxes, where you need to give root access to a container to the
"tenant", and expect them to adhere to a Terms of Service agreement,
but need to have technical mitigations in place, so that even if they
decide to violate the ToS (or innocently have their box hacked by a
malicious third-party who decides to violate the ToS), access to other
containers and the physical box (host OS) is very difficult to
impossible (pending any undiscovered vulnerabilities or host-side
misconfiguration).
As part of that, I was expecting some way to tell LXD to restrict the
IP addresses that can be claimed/used by a given container. For
instance, if I have a public Internet IPv4 /26 allocated to a physical
host by a hosting provider, I'll want to assign only one or two IP
addresses to each container. Currently, I can have an LXD container
just spuriously decide to use any arbitrary IP, and I haven't found a
way to prevent it from doing that if an untrusted user has root access
in the container. They can just run ifconfig and specify the IP
address they want to use.
How can I configure the host environment (LXD or something else on the
host, assuming I'm running a very recent Ubuntu 16.04 Beta nightly) so
Just wanted to clarify that I am *not* using or intending to use a
pre-release of 16.04 in a production environment. I'm currently
satisfied with LXD 0.24 on Ubuntu Server 14.04.4 LTS. I'm not
currently in a situation where I have untrusted root users with access
to containers, but I am planning to open up that type of usage in the
future if LXD turns out to be able to support it. And of course that
would be using the final release of Ubuntu Server 16.04 LTS.
Thanks,
Sean
Post by Sean McNamara
that no packets can be transmitted to/from the guest unless the guest
is using a specific IP or set of IPs? I also want to make sure that no
broadcasting is occurring; i.e., the root user in the container should
not be able to sniff layer 2 and see all the packets going to all the
other containers.
...Or is LXD not suitable for this use case? If it isn't, will it ever be?
As LXD doesn't configure the bridge itself but just lets you use any
bridge that you want, this is currently outside the scope of LXD itself.

That being said, if you use our OpenStack plugin, nova-lxd, then it will
let you integrate LXD containers with whatever SDN your OpenStack
deployment is using (openvswtich by default) in which mode LXD
containers will be bound to the restrictions applied to their virtual
switch port just as you would get when running a virtual machine on
OpenStack.

You could probably do something similar without using full openstack by
installing openvswitch, configuring it the way you like then telling LXD
to use that instead of the default minimal bridge it uses (lxcbr0).
Post by Sean McNamara
Post by Sean McNamara
Thanks,
Sean
Post by Stéphane Graber
- lxc.network.X.ipv4
- lxc.network.X.ipv4.gateway
- lxc.network.X.ipv6
- lxc.network.X.ipv6.gateway
Note that we require you set the interface index (X above) as mixing
those raw entris with the LXD generated config would otherwise randomly
cause an invalid config and container startup failure.
The recommended way to manage IPs with LXD is to do it exactly the same
way you would do it for your VMs or physical machines, so either
configure your DHCP server to give a static lease or configure the
container to use a static IP (you can use lxc file pull/push/edit to do
it on a stopped container).
Post by Sean McNamara
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.
I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.
raw.lxc: "lxc.network.ipv4=1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0
On Ubuntu, you can then set up your bridge as follows in
auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off
This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.
It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.
Thanks,
Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw and
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own IP
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I followed
instructions from https://wiki.debian.org/LXC/SimpleBridge. But turns out
that I am probably running a different version of LXC and that this page is
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container). This
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail to find
within the manual the path where this file should be saved! If you write
documentation, please always provide the path where configuration files are
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not found any
option/instruction on how to apply that profile on my existing image. 'lxc
start server2' does not provide any option to start the container with a
particular profile. BTW, where are profile configuration files stored?
I need clear step by step instructions, with full paths on how to set things
up and I fail to find any on the web. Anybody has a useful link to suggest?
I have a KVM image running (server1) and it works flawlessly with a static
IP on my bridge. And it wasn't hard to find instructions on how to set it
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
Fajar A. Nugraha
2016-03-18 23:21:51 UTC
Permalink
Post by Sean McNamara
As part of that, I was expecting some way to tell LXD to restrict the
IP addresses that can be claimed/used by a given container. For
instance, if I have a public Internet IPv4 /26 allocated to a physical
host by a hosting provider, I'll want to assign only one or two IP
addresses to each container. Currently, I can have an LXD container
just spuriously decide to use any arbitrary IP, and I haven't found a
way to prevent it from doing that if an untrusted user has root access
in the container. They can just run ifconfig and specify the IP
address they want to use.
The same is true for lxd, lxc, xen, kvm, and
whatever-solution-that-uses-plain-bridge.
Post by Sean McNamara
How can I configure the host environment (LXD or something else on the
host, assuming I'm running a very recent Ubuntu 16.04 Beta nightly) so
that no packets can be transmitted to/from the guest unless the guest
is using a specific IP or set of IPs? I also want to make sure that no
broadcasting is occurring; i.e., the root user in the container should
not be able to sniff layer 2 and see all the packets going to all the
other containers.
...Or is LXD not suitable for this use case? If it isn't, will it ever be?
As Stephane said, it's not lxc's job (or xen, or kvm) when using bridge.

That being said, I've been using a workaroud for lxc, and can be adapted to lxd:
- use veth interface for containers, with a persistent name (e.g.
veth-r-0), without a bridge. For lxd's case, this is slightly
complicated as it doesn't allow you to define bridgeless veth
interface. I had to create and assign it to my custom bridge (e.g.
br-dummy), but have a script take it out of the bridge later

- configure the host and the container to use the the veth pair as
point to point

- configure the host to use proxyarp (and optionally, set the host to
send arp update for container's ip, so that other hosts in the network
knows about the change immediately)


The host configuration part looks like this

### container device config (lxc config device show CONTAINER_NAME),
relevant nic config only
eth0:
host_name: veth-r-0
name: eth0
nictype: bridged
parent: br-dummy
type: nic
###

### /etc/sysctl.d/50-proxyarp.conf
net.ipv4.conf.eth0.proxy_arp=1
net.ipv4.ip_forward = 1
###

### addition to /etc/network/interfaces ###
# dummy bridge. Needed later for bridgeless container
auto br-dummy
iface br-dummy inet manual
bridge_ports none

# host veth pair config
allow-hotplug veth-r-0
iface veth-r-0 inet static
address 10.0.0.1/32
scope link
pointopoint 192.168.0.101
up /etc/lxc/script/proxyarp 192.168.0.101 192.168.0.1 eth0
up sleep 1 && brctl delif br-dummy veth-r-0
###

### /etc/lxc/script/proxyarp ###
#!/bin/bash
export IP=$1
export GW=$2
export DEV=$3
ip ad add dev $DEV $IP/32;arping -c 1 -I $DEV -s $IP $GW;ip ad del dev
$DEV $IP/32
###

The container's /etc/network/interfaces looks something like this
###
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 192.168.0.101/32
pointopoint 10.0.0.1
gateway 10.0.0.1
dns-nameservers 8.8.8.8 8.8.4.4
###


With that setup, the container is only connected to the host (thru the
pair of veth interface), and does not share L2 networking with other
containers (so things like snooping other container's traffic won't
work). If the container change its network config, it simply can't
connect anywhere.

It works for me, but might not be suitable for mass-hosting setup (due
to additions needed on the host's /etc/network/interfaces). In that
case it'd probably be more suitable to use something like openvswitch
and perform the configuration there.
--
Fajar
Witold Filipczyk
2017-01-23 17:01:07 UTC
Permalink
Post by Fajar A. Nugraha
Post by Sean McNamara
As part of that, I was expecting some way to tell LXD to restrict the
IP addresses that can be claimed/used by a given container. For
instance, if I have a public Internet IPv4 /26 allocated to a physical
host by a hosting provider, I'll want to assign only one or two IP
addresses to each container. Currently, I can have an LXD container
just spuriously decide to use any arbitrary IP, and I haven't found a
way to prevent it from doing that if an untrusted user has root access
in the container. They can just run ifconfig and specify the IP
address they want to use.
The same is true for lxd, lxc, xen, kvm, and
whatever-solution-that-uses-plain-bridge.
Post by Sean McNamara
How can I configure the host environment (LXD or something else on the
host, assuming I'm running a very recent Ubuntu 16.04 Beta nightly) so
that no packets can be transmitted to/from the guest unless the guest
is using a specific IP or set of IPs? I also want to make sure that no
broadcasting is occurring; i.e., the root user in the container should
not be able to sniff layer 2 and see all the packets going to all the
other containers.
...Or is LXD not suitable for this use case? If it isn't, will it ever be?
As Stephane said, it's not lxc's job (or xen, or kvm) when using bridge.
- use veth interface for containers, with a persistent name (e.g.
veth-r-0), without a bridge. For lxd's case, this is slightly
complicated as it doesn't allow you to define bridgeless veth
interface. I had to create and assign it to my custom bridge (e.g.
br-dummy), but have a script take it out of the bridge later
- configure the host and the container to use the the veth pair as
point to point
- configure the host to use proxyarp (and optionally, set the host to
send arp update for container's ip, so that other hosts in the network
knows about the change immediately)
The host configuration part looks like this
### container device config (lxc config device show CONTAINER_NAME),
relevant nic config only
host_name: veth-r-0
name: eth0
nictype: bridged
parent: br-dummy
type: nic
###
Can it be done with nictype 'p2p' somehow?
Could you provide lxd commands how to achieve this?
Number of examples is too low in documentation, IMO.
Post by Fajar A. Nugraha
### /etc/sysctl.d/50-proxyarp.conf
net.ipv4.conf.eth0.proxy_arp=1
net.ipv4.ip_forward = 1
###
### addition to /etc/network/interfaces ###
# dummy bridge. Needed later for bridgeless container
auto br-dummy
iface br-dummy inet manual
bridge_ports none
# host veth pair config
allow-hotplug veth-r-0
iface veth-r-0 inet static
address 10.0.0.1/32
scope link
pointopoint 192.168.0.101
up /etc/lxc/script/proxyarp 192.168.0.101 192.168.0.1 eth0
up sleep 1 && brctl delif br-dummy veth-r-0
###
### /etc/lxc/script/proxyarp ###
#!/bin/bash
export IP=$1
export GW=$2
export DEV=$3
ip ad add dev $DEV $IP/32;arping -c 1 -I $DEV -s $IP $GW;ip ad del dev
$DEV $IP/32
###
The container's /etc/network/interfaces looks something like this
###
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 192.168.0.101/32
pointopoint 10.0.0.1
gateway 10.0.0.1
dns-nameservers 8.8.8.8 8.8.4.4
###
With that setup, the container is only connected to the host (thru the
pair of veth interface), and does not share L2 networking with other
containers (so things like snooping other container's traffic won't
work). If the container change its network config, it simply can't
connect anywhere.
It works for me, but might not be suitable for mass-hosting setup (due
to additions needed on the host's /etc/network/interfaces). In that
case it'd probably be more suitable to use something like openvswitch
and perform the configuration there.
Hans Deragon
2016-03-18 20:08:43 UTC
Permalink
Greetings,
Post by Stéphane Graber
The recommended way to manage IPs with LXD is to do it
exactly the same way you would do it for your VMs or
physical machines, so either configure your DHCP server to
give a static lease or configure the container to use a
static IP (you can use lxc file pull/push/edit to do it on
a stopped container).
I have this in my container:

***@server2:/etc/network# cat interfaces
auto lo
iface lo inet loopback

auto eth0
iface eth0 inet static
address 192.168.1.5
network 192.168.1.0
netmask 255.255.255.0
broadcast 192.168.1.255

Same configuration as my KVM image (different IP, of course). Within the
container, it seams to work fine:

***@server2:/etc/network# ping 192.168.1.5
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=0.044 ms

Now I need clear instructions on how to tell LDX that my container
needs to connect to the bridge (br0) device on the host. I fail
to find such instructions on the web for the version I am running:

LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1

Best regards,
Hans Deragon
Post by Stéphane Graber
Our stance hasn't changed. LXD doesn't know nor care about layer-3
networking, all it does is setup your layer-2.
Having LXD pre-initialize your network namespace confuses the heck out
of a bunch of distros which expect all network to be unconfigured by the
time they apply their own config (they don't clean things up so
duplicate entries lead to failure).
- lxc.network.X.ipv4
- lxc.network.X.ipv4.gateway
- lxc.network.X.ipv6
- lxc.network.X.ipv6.gateway
Note that we require you set the interface index (X above) as mixing
those raw entris with the LXD generated config would otherwise randomly
cause an invalid config and container startup failure.
The recommended way to manage IPs with LXD is to do it exactly the same
way you would do it for your VMs or physical machines, so either
configure your DHCP server to give a static lease or configure the
container to use a static IP (you can use lxc file pull/push/edit to do
it on a stopped container).
Post by Sean McNamara
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.
I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.
raw.lxc: "lxc.network.ipv4=1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0
On Ubuntu, you can then set up your bridge as follows in
auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off
This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.
It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.
Thanks,
Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw and
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own IP
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I followed
instructions from https://wiki.debian.org/LXC/SimpleBridge. But turns out
that I am probably running a different version of LXC and that this page is
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container). This
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail to find
within the manual the path where this file should be saved! If you write
documentation, please always provide the path where configuration files are
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not found any
option/instruction on how to apply that profile on my existing image. 'lxc
start server2' does not provide any option to start the container with a
particular profile. BTW, where are profile configuration files stored?
I need clear step by step instructions, with full paths on how to set things
up and I fail to find any on the web. Anybody has a useful link to suggest?
I have a KVM image running (server1) and it works flawlessly with a static
IP on my bridge. And it wasn't hard to find instructions on how to set it
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Hans Deragon
2016-03-20 21:16:25 UTC
Permalink
Greetings,

I solved my issue. Sean Mcnamara opened my eyes he wrote "First of all,
there's no such thing as LX[C|D]. You're either using LXC or LXD.".
This brought a paradigm shift where I though I use 'lxc' as the command,
now I search the web for "LXD 2.0 <keywords>". I then found
instructions for the version of LXD I used, filtering many now useless
web pages. Following instructions found, I created the
following profile:

lxc profile create bridged
lxc profile device add bridged eth0 nic nictype=bridged parent=br0

And then, I ran:

lxc config edit <container name>

Where I found "default", I replaced with "bridged". Restarted the
container with:

lxc restart <container name>

and voilà, now I have my container working with a static IP (static IP
configuration per se is the same as for any VM images;
/etc/network/interfaces of container needs to be configured and on the
host, a bridged network usually named br0)

Best regards and thanks to all, particularly Sean, for the help.
Hans Deragon
Post by Hans Deragon
Greetings,
Post by Stéphane Graber
The recommended way to manage IPs with LXD is to do it
exactly the same way you would do it for your VMs or
physical machines, so either configure your DHCP server to
give a static lease or configure the container to use a
static IP (you can use lxc file pull/push/edit to do it on
a stopped container).
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 192.168.1.5
network 192.168.1.0
netmask 255.255.255.0
broadcast 192.168.1.255
Same configuration as my KVM image (different IP, of course). Within the
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=0.044 ms
Now I need clear instructions on how to tell LDX that my container
needs to connect to the bridge (br0) device on the host. I fail
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards,
Hans Deragon
Post by Stéphane Graber
Our stance hasn't changed. LXD doesn't know nor care about layer-3
networking, all it does is setup your layer-2.
Having LXD pre-initialize your network namespace confuses the heck out
of a bunch of distros which expect all network to be unconfigured by the
time they apply their own config (they don't clean things up so
duplicate entries lead to failure).
- lxc.network.X.ipv4
- lxc.network.X.ipv4.gateway
- lxc.network.X.ipv6
- lxc.network.X.ipv6.gateway
Note that we require you set the interface index (X above) as mixing
those raw entris with the LXD generated config would otherwise randomly
cause an invalid config and container startup failure.
The recommended way to manage IPs with LXD is to do it exactly the same
way you would do it for your VMs or physical machines, so either
configure your DHCP server to give a static lease or configure the
container to use a static IP (you can use lxc file pull/push/edit to do
it on a stopped container).
Post by Sean McNamara
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.
I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.
raw.lxc: "lxc.network.ipv4=1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0
On Ubuntu, you can then set up your bridge as follows in
auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off
This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.
It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.
Thanks,
Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a simple
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here btw and
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own IP
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I followed
instructions from https://wiki.debian.org/LXC/SimpleBridge. But turns out
that I am probably running a different version of LXC and that this page is
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container). This
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail to find
within the manual the path where this file should be saved! If you write
documentation, please always provide the path where configuration files are
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not found any
option/instruction on how to apply that profile on my existing image. 'lxc
start server2' does not provide any option to start the container with a
particular profile. BTW, where are profile configuration files stored?
I need clear step by step instructions, with full paths on how to set things
up and I fail to find any on the web. Anybody has a useful link to suggest?
I have a KVM image running (server1) and it works flawlessly with a static
IP on my bridge. And it wasn't hard to find instructions on how to set it
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Mahesh Patade
2016-03-21 04:14:20 UTC
Permalink
Hi,

here is simple and sweet howto on bridge networking.

https://insights.ubuntu.com/2015/11/10/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan/

Google is your friend. ;)
Post by Hans Deragon
Greetings,
I solved my issue. Sean Mcnamara opened my eyes he wrote "First of all,
there's no such thing as LX[C|D]. You're either using LXC or LXD.".
This brought a paradigm shift where I though I use 'lxc' as the command,
now I search the web for "LXD 2.0 <keywords>". I then found
instructions for the version of LXD I used, filtering many now useless
web pages. Following instructions found, I created the
lxc profile create bridged
lxc profile device add bridged eth0 nic nictype=bridged parent=br0
lxc config edit <container name>
Where I found "default", I replaced with "bridged". Restarted the
lxc restart <container name>
and voilà, now I have my container working with a static IP (static IP
configuration per se is the same as for any VM images;
/etc/network/interfaces of container needs to be configured and on the
host, a bridged network usually named br0)
Best regards and thanks to all, particularly Sean, for the help.
Hans Deragon
Post by Hans Deragon
Greetings,
Post by Stéphane Graber
The recommended way to manage IPs with LXD is to do it
exactly the same way you would do it for your VMs or
physical machines, so either configure your DHCP server to
give a static lease or configure the container to use a
static IP (you can use lxc file pull/push/edit to do it on
a stopped container).
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
address 192.168.1.5
network 192.168.1.0
netmask 255.255.255.0
broadcast 192.168.1.255
Same configuration as my KVM image (different IP, of course). Within the
PING 192.168.1.5 (192.168.1.5) 56(84) bytes of data.
64 bytes from 192.168.1.5: icmp_seq=1 ttl=64 time=0.044 ms
Now I need clear instructions on how to tell LDX that my container
needs to connect to the bridge (br0) device on the host. I fail
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards,
Hans Deragon
Post by Stéphane Graber
Our stance hasn't changed. LXD doesn't know nor care about layer-3
networking, all it does is setup your layer-2.
Having LXD pre-initialize your network namespace confuses the heck out
of a bunch of distros which expect all network to be unconfigured by the
time they apply their own config (they don't clean things up so
duplicate entries lead to failure).
Nevertheless, we have recently allowed the following key through
- lxc.network.X.ipv4
- lxc.network.X.ipv4.gateway
- lxc.network.X.ipv6
- lxc.network.X.ipv6.gateway
Note that we require you set the interface index (X above) as mixing
those raw entris with the LXD generated config would otherwise randomly
cause an invalid config and container startup failure.
The recommended way to manage IPs with LXD is to do it exactly the same
way you would do it for your VMs or physical machines, so either
configure your DHCP server to give a static lease or configure the
container to use a static IP (you can use lxc file pull/push/edit to do
it on a stopped container).
Post by Sean McNamara
First of all, there's no such thing as LX[C|D]. You're either using
LXC or LXD. They're different enough in their configuration and
operation that you can't ask an "either-or" question. Pick one
solution and focus on that.
I just wanted to chime in to say that I have this same question. I'm
stuck using a pre-2.0 release of LXD because it allows me to use the
"raw.lxc" config parameter to specify the IP settings for the guest.
This configuration parameter was removed at some point prior to the
2.0 RC, so I ended up editing the source code of LXD to bring it back.
I haven't found any equivalent configuration that works without using
raw.lxc.
raw.lxc: "lxc.network.ipv4=
1.2.3.4/32\nlxc.network.ipv4.gateway=5.6.7.8\nlxc.network.hwaddr=00:11:22:33:44:55\nlxc.network.flags=up
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
\ \nlxc.network.mtu=1500\n"
volatile.eth0.hwaddr: 00:11:22:33:44:55
volatile.eth0.name: eth1
hwaddr: 00:11:22:33:44:55
nictype: bridged
parent: br0
On Ubuntu, you can then set up your bridge as follows in
auto br0
iface br0 inet static
address 1.2.3.4
netmask 255.255.255.0
broadcast 5.6.7.8
gateway 9.10.11.12
bridge_ports eth0
bridge_stp off
This is fine with LXD 0.24 that was built about a month before the 2.0
release candidates started hitting (and with edited source code to
un-block the raw.lxc param) but I'm afraid to upgrade to LXD 2.0
because I don't know the way forward.
It seems like support for certain basic network topologies are still
being worked out with LXD. It should be easy, well-documented and
flexible a la OpenVZ, but it's really not, as far as I have seen. The
best way to make any progress that I've found thus far is to start
learning Google Go and reading the source code.
Thanks,
Sean
Post by Hans Deragon
Greetings,
Ok, this is ridiculous and I apologize for asking help for such a
simple
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
task, but I fail to find the answers by myself. I fail to find proper
documentation to setup bridge networking and static IP. Newbie here
btw and
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
setup details at the end of this email.
I got the container running and with DHCP configured, it has its own
IP
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
which the host can address with.
Obviously, I attempted to setup the static IP many times following
instructions found on many web pages, to no vail. For example, I
followed
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
instructions from https://wiki.debian.org/LXC/SimpleBridge. But
turns out
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
that I am probably running a different version of LXC and that this
page is
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
now obsolete.
I went so far to run 'strace lxc restart server2' to realize that
/var/lib/lxc/server2/config is not read (server2 is the container).
This
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
seams to be confirmed by the post at
http://ubuntuforums.org/showthread.php?t=2275372.
I found 'man lxc.container.conf'. Seams promising. However, I fail
to find
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
within the manual the path where this file should be saved! If you
write
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
documentation, please always provide the path where configuration
files are
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
supposed to be stored.
I created a profile named 'bridged' using commands, but I have not
found any
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
option/instruction on how to apply that profile on my existing
image. 'lxc
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
start server2' does not provide any option to start the container
with a
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
particular profile. BTW, where are profile configuration files
stored?
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
I need clear step by step instructions, with full paths on how to set
things
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
up and I fail to find any on the web. Anybody has a useful link to
suggest?
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
I have a KVM image running (server1) and it works flawlessly with a
static
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
IP on my bridge. And it wasn't hard to find instructions on how to
set it
Post by Hans Deragon
Post by Stéphane Graber
Post by Sean McNamara
Post by Hans Deragon
up. But LXD/LXc is another story.
Host: Ubuntu 14.04 LTS.
Container: Ubuntu 14.04 LTS.
LXD: 2.0.0~rc3-0ubuntu4~ubuntu14.04.1~ppa1
LXC: 2.0.0~rc10-0ubuntu2~ubuntu14.04.1~ppa1
Best regards and thanks in advance,
Hans Deragon
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Andrey Repin
2016-03-21 09:26:46 UTC
Permalink
Greetings, Mahesh Patade!
Post by Mahesh Patade
Hi,
here is simple and sweet howto on bridge networking.
https://insights.ubuntu.com/2015/11/10/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan/
Google is your friend.  ;)
That's no better than LXCBR…
Try macvlan bridge one day.


--
With best regards,
Andrey Repin
Monday, March 21, 2016 12:26:15

Sorry for my terrible english...
Kevin LaTona
2016-03-21 11:27:22 UTC
Permalink
On Mar 21, 2016, at 2:26 AM, Andrey Repin <***@yandex.ru> wrote:

Greetings, Mahesh Patade!
Hi,
here is simple and sweet how to on bridge networking.
https://insights.ubuntu.com/2015/11/10/converting-eth0-to-br0-and-getting-all-your-lxc-or-lxd-onto-your-lan/
Google is your friend. ;)
That's no better than LXCBR…
Try macvlan bridge one day.
So yhy do you thinkmacvlan is any better than the default lxcbr0 style of doing this and using iptables… care to share your thoughts?

https://www.flockport.com/lxc-macvlan-networking/



These days with LXD 2.0.0 pretty much ready to go…... all the infomation anyone can find on Google is getting to be very dated and now will confuse most users….. new or otherwise ….rather than help them to get going or solve a problem.

Meaning much of what anyone can find on Google is about worthless and is a huge time suck trying to pull out any little nuggets from it these days.



The one positive thing in the works is Stephane Graber is in the process of rewriting / updating his blog post series to reflect the state of LXD 2.0 vs LXC .08

https://insights.ubuntu.com/2016/03/16/lxd-2-0-installing-and-configuring-lxd-212/


Until the LXC/D community comes up with a way to create a singular information site that makes it easy for people to both learn from and share their experiences about LXD.

Docker is going to continue to hog the spotlight and working with LXD will forever require more work than the average typically user will dive into inorder to get up to speed with it.


That’s my take on it….. LXD is pretty much amazing… but has a huge info / documentation brick wall in front of it.

-Kevin
Hans Deragon
2016-05-06 12:31:43 UTC
Permalink
Greetings,

I created a few days ago a new container on an updated Ubuntu Trusty
Thar 14.04 LTS host using the following command:

lxc launch images:ubuntu/xenial/amd64 server1

Now, it hangs when I run:

lxc stop server1

This problem was reported at https://github.com/lxc/lxd/issues/1570

What puzzles me, is that stopping a container is such a basic feature
that I am surprise to find that it is not being addressed. Both Trusty
and Xenial are now out and production ready, yet one cannot stop a
Xenial container. Are we the only one that have this problem? Should I
open a new bug report or can someone open this one again since it is not
fixed?

Best regards,
Hans Deragon
Alan Hoffmeister
2016-03-18 13:58:35 UTC
Permalink
Thanks for the reply, I'm trying to allow root access to my containers so
adjusting limits inside the rootfs isn't a good idea for me.


--
Alan Hoffmeister
https://twitter.com/alan_hoff
https://github.com/alanhoff
https://keybase.io/alanhoff
Post by Guido Jäkel
Dear Alan,
adjust the limits per Container in /etc/security/limits.conf , e.g. add something like
* hard nofile 8192
* soft nofile 8000
* hard nproc 1024
* soft nproc 1000
and maybe adjust the values at the host, too.
Greetings
Guido
Post by Alan Hoffmeister
Hello folks,
I'm having some troubles where one container can drain the amount of file
descriptors available in the host system. Does somebody knows how to
limit
Post by Alan Hoffmeister
file descriptors per container?
I'm running lxd v2.0.0.rc3 and lxc v2.0.0.rc10
Cheers
--
Alan Hoffmeister
https://twitter.com/alan_hoff
https://github.com/alanhoff
https://keybase.io/alanhoff
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Continue reading on narkive:
Loading...