[lxc-users] LXC, unionfs and short lived containers

Discussion:

Frans Meulenbroeks

2017-01-28 21:04:17 UTC

Hi,

I'm working on migrating from LXC 1.x to LXC 2.
While doing so I bumped upon the following issue:

My containers are short-lived (say an hour or so).
In LXC 1 we used an overlay filesystem in order to speed up the lxc create.
However I understood LXC 2 does not have this capability.
Any idea how to create containers quickly and efficiently in LXC 2

Complication is that at some times we have a fair amount of containers
alive (say around 50), so creating all containers and reverting to a
snapshot is probably not efficient (apart from the space taken up by the 50
rootfs-es).

Thanks in advance for any suggestions how to tacke this!
Frans

Fajar A. Nugraha

2017-01-28 21:49:22 UTC

Permalink

On Sun, Jan 29, 2017 at 4:04 AM, Frans Meulenbroeks <

Post by Frans Meulenbroeks
Hi,
I'm working on migrating from LXC 1.x to LXC 2.
My containers are short-lived (say an hour or so).
In LXC 1 we used an overlay filesystem in order to speed up the lxc create.
However I understood LXC 2 does not have this capability.

Where did you read that?

Post by Frans Meulenbroeks
Any idea how to create containers quickly and efficiently in LXC 2
Complication is that at some times we have a fair amount of containers
alive (say around 50), so creating all containers and reverting to a
snapshot is probably not efficient

Why is it not efficient?

Post by Frans Meulenbroeks
(apart from the space taken up by the 50 rootfs-es).
Thanks in advance for any suggestions how to tacke this!

--
Fajar

Luis Felipe Marzagao

2017-01-29 00:27:15 UTC

Permalink

I have a Digium Analog PCI Card and would like to access it from lxc
container.

I've searched the internet a lot but it seems information out there is
incomplete and a bit outdated.

It seems that bind mounting /dev/dahdi to
/var/lib/lxd/containers/ct1/rootfs/dev/dahdi does not help, since the
mount point does not appear from within the container.

Actually ct1/rootfs/dev seems completely different from /dev inside the
container.

Could you give any pointers?

Here is my setup:

$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.1 LTS
Release: 16.04
Codename: xenial

$ lxc --version
2.8

$ lxd --version
2.8

$ lspci -v

(...)

02:00.0 Ethernet controller: Digium, Inc. Wildcard TDM410 4-port analog
card (rev 11)
Subsystem: Digium, Inc. Wildcard TDM410 4-port analog card
Flags: bus master, medium devsel, latency 32, IRQ 16
I/O ports at e000 [size=256]
Memory at df020000 (32-bit, non-prefetchable) [size=1K]
Expansion ROM at df000000 [disabled] [size=128K]
Capabilities: <access denied>
Kernel driver in use: wctdm24xxp
Kernel modules: wctdm24xxp

$ dahdi_hardware
pci:0000:02:00.0 wctdm24xxp+ d161:8005 Wildcard TDM410P

$ ls -l /dev/dahdi/
total 0
lrwxrwxrwx 1 root root 12 Jan 28 21:39 1 -> chan/001/001
lrwxrwxrwx 1 root root 12 Jan 28 21:39 2 -> chan/001/002
lrwxrwxrwx 1 root root 12 Jan 28 21:39 3 -> chan/001/003
lrwxrwxrwx 1 root root 12 Jan 28 21:39 4 -> chan/001/004
drwxr-xr-x 3 root root 60 Jan 28 21:39 chan
crw-rw---- 1 root dialout 196, 254 Jan 28 21:32 channel
crw-rw---- 1 root dialout 196, 0 Jan 28 21:32 ctl
drwxr-xr-x 3 root root 60 Jan 28 21:39 devices
crw-rw---- 1 root dialout 196, 255 Jan 28 21:32 pseudo
crw-rw---- 1 root dialout 196, 253 Jan 28 21:32 timer

I should have a /dev/dahdi with the above structure inside the container
in order for it to work.

How do I do that?

Thanks a lot.

Serge E. Hallyn

2017-01-29 16:36:46 UTC

Permalink

Post by Fajar A. Nugraha
On Sun, Jan 29, 2017 at 4:04 AM, Frans Meulenbroeks <

Where did you read that?

Why is it not efficient?

Post by Frans Meulenbroeks
(apart from the space taken up by the 50 rootfs-es).
Thanks in advance for any suggestions how to tacke this!

I'm pretty sure you can still use overlayfs with lxc-2.
My suggestion though, is go with lxd and zfs instead. You can have a

Agreed, either zfs or btrfs would work great. And would avoid the known
issues which overlayfs has (i.e. inotify breakage)

Post by Fajar A. Nugraha
"golden" container, keep it stopped, and simply create your other
containers with "lxc copy". With zfs, the "copy" process will be instaneus,

lxc copy would work, or you can also create a local published image of
your golden container,

lxc publish mygolden --alias=goldenimage

then launch containers fromthat image

for i in `seq 1 10`; do
lxc launch goldenimage c${i}
done

Post by Fajar A. Nugraha
and the "clone" will be its own filesystem (no lower/base directory
restriction like in aufs/overlayfs).
If you need to modify the "golden" container (which will affect all NEW
containers copied from it), simply start it and perform-your-changes like
on a normal container (don't forget to stop it afterwards). Note that this
is different from aufs/overlayfs, where generally you shouldn't touch the
lower/base directory.
--
Fajar
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users

Frans Meulenbroeks

2017-01-29 12:25:55 UTC

Permalink

Hi Fajar, all,

Thanks for your reply.

On Sun, Jan 29, 2017 at 4:04 AM, Frans Meulenbroeks <

Where did you read that?

Here: https://github.com/lxc/lxd/issues/1878 see the response of Stephane.
Of course this reply is almost 10 months old.

Why is it not efficient?

I'm worried about disk storage and creation time, but I noticed your
suggestion below.

Post by Frans Meulenbroeks
(apart from the space taken up by the 50 rootfs-es).
Thanks in advance for any suggestions how to tacke this!

I'm pretty sure you can still use overlayfs with lxc-2.
My suggestion though, is go with lxd and zfs instead. You can have a
"golden" container, keep it stopped, and simply create your other
containers with "lxc copy". With zfs, the "copy" process will be instaneus,
and the "clone" will be its own filesystem (no lower/base directory
restriction like in aufs/overlayfs).

Ah ok, currently we already have a golden container from which we derive
new containers using aufs.
I've just tried the above and it works like a charm!

If you need to modify the "golden" container (which will affect all NEW
containers copied from it), simply start it and perform-your-changes like
on a normal container (don't forget to stop it afterwards). Note that this
is different from aufs/overlayfs, where generally you shouldn't touch the
lower/base directory.

Yeah I am aware of the latter. We always have to take care in lxc-1 when
updating the golden container that no instances were running.

This is great info! Thanks again!
Frans

--
Fajar

Andrey Repin

2017-01-31 12:00:46 UTC

Permalink

Greetings, Frans Meulenbroeks!

Post by Fajar A. Nugraha
On Sun, Jan 29, 2017 at 4:04 AM, Frans Meulenbroeks <

Where did you read that?
Here: https://github.com/lxc/lxd/issues/1878 see the response of Stephane.
Of course this reply is almost 10 months old.

You're linking LXD isue, not LXC.

--
With best regards,
Andrey Repin
Tuesday, January 31, 2017 15:00:25

Sorry for my terrible english...