Discussion:
[Unable to start using lvm backend]
(too old to reply)
Andrea Masi
2015-01-23 16:02:08 UTC
Permalink
Hi,
I'm using lxc 1.0.6 on ubuntu 14.04.
I've no problems creating/running on dir backing store but when I use lvm I
cannot start containers getting these errors:

lxc-start 1422026234.562 ERROR bdev - failed to determine fs type for
'/dev/lxc/lvm-cont'
lxc-start 1422026234.563 ERROR lxc_conf - failed to determine fs type
for '/dev/dm-0'
lxc-start 1422026234.564 ERROR lxc_conf - failed to mount rootfs
lxc-start 1422026234.564 ERROR lxc_conf - failed to setup rootfs for
'lvm-cont'
lxc-start 1422026234.565 ERROR lxc_conf - Error setting up rootfs mount
after spawn
lxc-start 1422026234.565 ERROR lxc_start - failed to setup the container
lxc-start 1422026234.566 ERROR lxc_sync - invalid sequence number 1.
expected 2
lxc-start 1422026234.566 ERROR lxc_start - failed to spawn 'lvm-cont'
lxc-start 1422026234.574 ERROR lxc_start_ui - The container failed to
start.
lxc-start 1422026234.575 ERROR lxc_start_ui - Additional information can
be obtained by setting the --logfile and --logpriority options.

I've tried different template createds with -t download for example:
lxc-create -t download -n lvm-cont -B lvm -- -d ubuntu -r utopic -a amd64

I can manually mount with no problem /dev/lxc/lvm-cont

Any idea on what can it be wrong?

Thanks.
--
www.eraclitux.com
Andrea Masi
2015-01-24 13:41:38 UTC
Permalink
The problem seems related to unprivileged containers that seems cannot work
with -B lvm.
Must we assume that lvm backed store (and maybe others) actually cannot
work with unprivileged containers?
Post by Andrea Masi
Hi,
I'm using lxc 1.0.6 on ubuntu 14.04.
I've no problems creating/running on dir backing store but when I use lvm
lxc-start 1422026234.562 ERROR bdev - failed to determine fs type for
'/dev/lxc/lvm-cont'
lxc-start 1422026234.563 ERROR lxc_conf - failed to determine fs type
for '/dev/dm-0'
lxc-start 1422026234.564 ERROR lxc_conf - failed to mount rootfs
lxc-start 1422026234.564 ERROR lxc_conf - failed to setup rootfs for
'lvm-cont'
lxc-start 1422026234.565 ERROR lxc_conf - Error setting up rootfs mount
after spawn
lxc-start 1422026234.565 ERROR lxc_start - failed to setup the container
lxc-start 1422026234.566 ERROR lxc_sync - invalid sequence number 1.
expected 2
lxc-start 1422026234.566 ERROR lxc_start - failed to spawn 'lvm-cont'
lxc-start 1422026234.574 ERROR lxc_start_ui - The container failed to
start.
lxc-start 1422026234.575 ERROR lxc_start_ui - Additional information
can be obtained by setting the --logfile and --logpriority options.
lxc-create -t download -n lvm-cont -B lvm -- -d ubuntu -r utopic -a amd64
I can manually mount with no problem /dev/lxc/lvm-cont
Any idea on what can it be wrong?
Thanks.
--
www.eraclitux.com
--
www.eraclitux.com
Serge Hallyn
2015-01-25 00:11:32 UTC
Permalink
Yes, unprivileged users cannot manipulate the lvm devices on the host.

You can still use user namespaces though. I have a few containers on my main
server which do that. They each run in a unique uid range, but are started by
root, so that they can use lvm (actually luks-encrypted lvm) backends.

Just add the lxc.id_map lines as per usual to containers which are owned by
root. And make sure to allocate the ranges to root in /etc/sub[ug]id.

-serge
Post by Andrea Masi
The problem seems related to unprivileged containers that seems cannot work
with -B lvm.
Must we assume that lvm backed store (and maybe others) actually cannot
work with unprivileged containers?
Post by Andrea Masi
Hi,
I'm using lxc 1.0.6 on ubuntu 14.04.
I've no problems creating/running on dir backing store but when I use lvm
lxc-start 1422026234.562 ERROR bdev - failed to determine fs type for
'/dev/lxc/lvm-cont'
lxc-start 1422026234.563 ERROR lxc_conf - failed to determine fs type
for '/dev/dm-0'
lxc-start 1422026234.564 ERROR lxc_conf - failed to mount rootfs
lxc-start 1422026234.564 ERROR lxc_conf - failed to setup rootfs for
'lvm-cont'
lxc-start 1422026234.565 ERROR lxc_conf - Error setting up rootfs mount
after spawn
lxc-start 1422026234.565 ERROR lxc_start - failed to setup the container
lxc-start 1422026234.566 ERROR lxc_sync - invalid sequence number 1.
expected 2
lxc-start 1422026234.566 ERROR lxc_start - failed to spawn 'lvm-cont'
lxc-start 1422026234.574 ERROR lxc_start_ui - The container failed to
start.
lxc-start 1422026234.575 ERROR lxc_start_ui - Additional information
can be obtained by setting the --logfile and --logpriority options.
lxc-create -t download -n lvm-cont -B lvm -- -d ubuntu -r utopic -a amd64
I can manually mount with no problem /dev/lxc/lvm-cont
Any idea on what can it be wrong?
Thanks.
--
www.eraclitux.com
--
www.eraclitux.com
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Xavier Gendre
2015-04-12 17:39:46 UTC
Permalink
Hi Serge,

please, can you give more details about your settings of root owned
unprivileged container with LVM backend? Indeed, I encounter the same
problem as Andrea. I have tried to set the container as you explain but
it fails to run...

root # grep lxc.id_map /var/lib/lxc/test/config
lxc.id_map = u 0 558752 65536
lxc.id_map = g 0 558752 65536

root # grep root /etc/sub[ug]id
/etc/subgid:root:558752:65536
/etc/subuid:root:558752:65536

root # lxc-start -n test
lxc-start: failed to determine fs type for '/dev/Pool/test'
lxc-start: failed to determine fs type for '/dev/dm-7'
lxc-start: failed to mount rootfs
lxc-start: failed to setup rootfs for 'test'
lxc-start: Error setting up rootfs mount after spawn
lxc-start: failed to setup the container
lxc-start: invalid sequence number 1. expected 2
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the
--logfile and --logpriority options.

I use the download template with a Debian Wheezy 64 for this container.

Thanks,
Xavier
Post by Serge Hallyn
Yes, unprivileged users cannot manipulate the lvm devices on the host.
You can still use user namespaces though. I have a few containers on my main
server which do that. They each run in a unique uid range, but are started by
root, so that they can use lvm (actually luks-encrypted lvm) backends.
Just add the lxc.id_map lines as per usual to containers which are owned by
root. And make sure to allocate the ranges to root in /etc/sub[ug]id.
-serge
Post by Andrea Masi
The problem seems related to unprivileged containers that seems cannot work
with -B lvm.
Must we assume that lvm backed store (and maybe others) actually cannot
work with unprivileged containers?
Post by Andrea Masi
Hi,
I'm using lxc 1.0.6 on ubuntu 14.04.
I've no problems creating/running on dir backing store but when I use lvm
lxc-start 1422026234.562 ERROR bdev - failed to determine fs type for
'/dev/lxc/lvm-cont'
lxc-start 1422026234.563 ERROR lxc_conf - failed to determine fs type
for '/dev/dm-0'
lxc-start 1422026234.564 ERROR lxc_conf - failed to mount rootfs
lxc-start 1422026234.564 ERROR lxc_conf - failed to setup rootfs for
'lvm-cont'
lxc-start 1422026234.565 ERROR lxc_conf - Error setting up rootfs mount
after spawn
lxc-start 1422026234.565 ERROR lxc_start - failed to setup the container
lxc-start 1422026234.566 ERROR lxc_sync - invalid sequence number 1.
expected 2
lxc-start 1422026234.566 ERROR lxc_start - failed to spawn 'lvm-cont'
lxc-start 1422026234.574 ERROR lxc_start_ui - The container failed to
start.
lxc-start 1422026234.575 ERROR lxc_start_ui - Additional information
can be obtained by setting the --logfile and --logpriority options.
lxc-create -t download -n lvm-cont -B lvm -- -d ubuntu -r utopic -a amd64
I can manually mount with no problem /dev/lxc/lvm-cont
Any idea on what can it be wrong?
Thanks.
--
www.eraclitux.com
--
www.eraclitux.com
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Xavier Gendre
2015-04-13 11:42:46 UTC
Permalink
It may be useful to give more details about what i am trying to do ;-) I
work with a Debian Jessie and LXC 1.0.6 from the Debian repository.

First, i give an ID range to root and i set the container's
configuration with this range:

root # grep root /etc/sub[ug]id
/etc/subgid:root:558752:65536
/etc/subuid:root:558752:65536

root # cat test.conf
lxc.id_map = u 0 558752 65536
lxc.id_map = g 0 558752 65536
lxc.network.type = empty

Then, i create a container with the 'download' template,

root # lxc-create -n test -f test.conf -t download -B lvm --vgname Pool
-- -d debian -r wheezy -a amd64
File descriptor 3 (/var/lib/lxc/test/partial) leaked on lvcreate
invocation. Parent PID 1506: lxc-create
Logical volume "test" created
Using image from local cache
Unpacking the rootfs
[...]

Finally, i try to start this container but it miserably fails,

root # lxc-start -n test --logfile test.log --logpriority DEBUG
lxc-start: failed to determine fs type for '/dev/Pool/test'
lxc-start: failed to determine fs type for '/dev/dm-7'
lxc-start: failed to mount rootfs
lxc-start: failed to setup rootfs for 'test'
lxc-start: Error setting up rootfs mount after spawn
lxc-start: failed to setup the container
lxc-start: invalid sequence number 1. expected 2
lxc-start: failed to spawn 'test'
lxc-start: The container failed to start.
lxc-start: Additional information can be obtained by setting the
--logfile and --logpriority options.

Here is the content of the log file if it helps,

root # cat test.log
lxc-start 1428924388.945 INFO lxc_start_ui - using rcfile
/var/lib/lxc/test/config
lxc-start 1428924388.945 INFO lxc_confile - read uid map:
type u nsid 0 hostid 558752 range 65536
lxc-start 1428924388.945 INFO lxc_confile - read uid map:
type g nsid 0 hostid 558752 range 65536
lxc-start 1428924388.945 WARN lxc_log - lxc_log_init called
with log already initialized
lxc-start 1428924388.945 INFO lxc_lsm - LSM security driver nop
lxc-start 1428924388.947 DEBUG lxc_conf - allocated pty
'/dev/pts/3' (5/6)
lxc-start 1428924388.947 DEBUG lxc_conf - allocated pty
'/dev/pts/4' (7/8)
lxc-start 1428924388.947 DEBUG lxc_conf - allocated pty
'/dev/pts/5' (9/10)
lxc-start 1428924388.947 DEBUG lxc_conf - allocated pty
'/dev/pts/6' (11/12)
lxc-start 1428924388.947 INFO lxc_conf - tty's configured
lxc-start 1428924388.947 DEBUG lxc_start - sigchild handler set
lxc-start 1428924388.947 DEBUG lxc_console - opening /dev/tty
for console peer
lxc-start 1428924388.947 DEBUG lxc_console - using '/dev/tty'
as console
lxc-start 1428924388.947 DEBUG lxc_console - 1587 got SIGWINCH
fd 17
lxc-start 1428924388.947 DEBUG lxc_console - set winsz
dstfd:14 cols:145 rows:58
lxc-start 1428924388.947 INFO lxc_start - 'test' is initialized
lxc-start 1428924388.948 DEBUG lxc_start - Not dropping
cap_sys_boot or watching utmp
lxc-start 1428924388.961 DEBUG bdev - trying to mount
'/dev/Pool/test'->'/usr/lib/x86_64-linux-gnu/lxc/rootfs' with fstype 'ext3'
lxc-start 1428924388.971 INFO bdev - mounted '/dev/Pool/test'
on '/usr/lib/x86_64-linux-gnu/lxc/rootfs', with fstype 'ext3'
lxc-start 1428924388.971 DEBUG lxc_conf - mounted
'/dev/Pool/test' on '/usr/lib/x86_64-linux-gnu/lxc/rootfs'
lxc-start 1428924388.971 INFO lxc_start - Set up container
rootfs as host root
lxc-start 1428924388.971 INFO lxc_start - Cloning a new user
namespace
lxc-start 1428924388.971 INFO lxc_cgroup - cgroup driver
cgroupfs initing for test
lxc-start 1428924388.983 NOTICE lxc_start - switching to
gid/uid 0 in new user namespace
lxc-start 1428924388.984 DEBUG bdev - trying to mount
'/dev/Pool/test'->'/usr/lib/x86_64-linux-gnu/lxc/rootfs' with fstype 'ext3'
lxc-start 1428924388.984 DEBUG bdev - mount failed with error:
Operation not permitted
lxc-start 1428924388.984 DEBUG bdev - trying to mount
'/dev/Pool/test'->'/usr/lib/x86_64-linux-gnu/lxc/rootfs' with fstype 'ext2'
lxc-start 1428924388.984 DEBUG bdev - mount failed with error:
Operation not permitted
lxc-start 1428924388.984 DEBUG bdev - trying to mount
'/dev/Pool/test'->'/usr/lib/x86_64-linux-gnu/lxc/rootfs' with fstype 'ext4'
lxc-start 1428924388.984 DEBUG bdev - mount failed with error:
Operation not permitted
lxc-start 1428924388.984 ERROR bdev - failed to determine fs
type for '/dev/Pool/test'
lxc-start 1428924388.985 DEBUG lxc_conf - trying to mount
'/dev/dm-7'->'/usr/lib/x86_64-linux-gnu/lxc/rootfs' with fstype 'ext3'
lxc-start 1428924388.985 DEBUG lxc_conf - mount failed with
error: Operation not permitted
lxc-start 1428924388.985 DEBUG lxc_conf - trying to mount
'/dev/dm-7'->'/usr/lib/x86_64-linux-gnu/lxc/rootfs' with fstype 'ext2'
lxc-start 1428924388.985 DEBUG lxc_conf - mount failed with
error: Operation not permitted
lxc-start 1428924388.985 DEBUG lxc_conf - trying to mount
'/dev/dm-7'->'/usr/lib/x86_64-linux-gnu/lxc/rootfs' with fstype 'ext4'
lxc-start 1428924388.985 DEBUG lxc_conf - mount failed with
error: Operation not permitted
lxc-start 1428924388.985 ERROR lxc_conf - failed to determine
fs type for '/dev/dm-7'
lxc-start 1428924388.986 ERROR lxc_conf - failed to mount rootfs
lxc-start 1428924388.986 ERROR lxc_conf - failed to setup
rootfs for 'test'
lxc-start 1428924388.987 ERROR lxc_conf - Error setting up
rootfs mount after spawn
lxc-start 1428924388.988 ERROR lxc_start - failed to setup the
container
lxc-start 1428924388.988 ERROR lxc_sync - invalid sequence
number 1. expected 2
lxc-start 1428924388.989 ERROR lxc_start - failed to spawn 'test'
lxc-start 1428924388.989 WARN lxc_conf - Failed to locate
autodev /dev/.lxc and /dev/.lxc/user.
lxc-start 1428924388.990 ERROR lxc_start_ui - The container
failed to start.
lxc-start 1428924388.990 ERROR lxc_start_ui - Additional
information can be obtained by setting the --logfile and --logpriority
options.

The problem seems to be that the subuid is not allowed to mount the
rootfs. Naively, i thought that starting the container as root would
avoid such a complication. It is the case at time 1428924388.971 but it
begins to fail after switching to the new user namespace at time
1428924388.983.

Thanks for help!
Xavier
Post by Serge Hallyn
Yes, unprivileged users cannot manipulate the lvm devices on the host.
You can still use user namespaces though. I have a few containers on my main
server which do that. They each run in a unique uid range, but are started by
root, so that they can use lvm (actually luks-encrypted lvm) backends.
Just add the lxc.id_map lines as per usual to containers which are owned by
root. And make sure to allocate the ranges to root in /etc/sub[ug]id.
-serge
Post by Andrea Masi
The problem seems related to unprivileged containers that seems cannot work
with -B lvm.
Must we assume that lvm backed store (and maybe others) actually cannot
work with unprivileged containers?
Post by Andrea Masi
Hi,
I'm using lxc 1.0.6 on ubuntu 14.04.
I've no problems creating/running on dir backing store but when I use lvm
lxc-start 1422026234.562 ERROR bdev - failed to determine fs type for
'/dev/lxc/lvm-cont'
lxc-start 1422026234.563 ERROR lxc_conf - failed to determine fs type
for '/dev/dm-0'
lxc-start 1422026234.564 ERROR lxc_conf - failed to mount rootfs
lxc-start 1422026234.564 ERROR lxc_conf - failed to setup rootfs for
'lvm-cont'
lxc-start 1422026234.565 ERROR lxc_conf - Error setting up rootfs mount
after spawn
lxc-start 1422026234.565 ERROR lxc_start - failed to setup the container
lxc-start 1422026234.566 ERROR lxc_sync - invalid sequence number 1.
expected 2
lxc-start 1422026234.566 ERROR lxc_start - failed to spawn 'lvm-cont'
lxc-start 1422026234.574 ERROR lxc_start_ui - The container failed to
start.
lxc-start 1422026234.575 ERROR lxc_start_ui - Additional information
can be obtained by setting the --logfile and --logpriority options.
lxc-create -t download -n lvm-cont -B lvm -- -d ubuntu -r utopic -a amd64
I can manually mount with no problem /dev/lxc/lvm-cont
Any idea on what can it be wrong?
Thanks.
--
www.eraclitux.com
--
www.eraclitux.com
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Fajar A. Nugraha
2015-04-13 12:05:04 UTC
Permalink
Post by Xavier Gendre
It may be useful to give more details about what i am trying to do ;-) I
work with a Debian Jessie and LXC 1.0.6 from the Debian repository.
You should realy use at least 1.0.7 from experimental, or better yet, 1.1.1.
Post by Xavier Gendre
Finally, i try to start this container but it miserably fails,
root # lxc-start -n test --logfile test.log --logpriority DEBUG
lxc-start: failed to determine fs type for '/dev/Pool/test'
lxc-start: failed to determine fs type for '/dev/dm-7'
lxc-start: failed to mount rootfs
Works for me, just needed a chmod (which is shown on the helpful error
message that I get). This is lxc 1.1.1+master~20150407-0 from ubuntu
daily ppa.

# cat /etc/subuid
lxc-dnsmasq:100000:10000
user:100000:65537
root:1000000:100000

# cat /etc/subgid
lxc-dnsmasq:100000:10000
user:100000:65537
root:1000000:100000

# cat << END > /tmp/test.conf
Post by Xavier Gendre
lxc.id_map = u 0 1000000 100000
lxc.id_map = g 0 1000000 100000
lxc.network.type = empty
END
# lxc-create -n test -f /tmp/test.conf -t download -B lvm --vgname lxc
-- -d ubuntu -r utopic -a amd64
File descriptor 3 (/var/lib/lxc/test/partial) leaked on lvcreate
invocation. Parent PID 24304: lxc-create
Logical volume "test" created
Using image from local cache
Unpacking the rootfs

---
You just created an Ubuntu container (release=utopic, arch=amd64,
variant=default)

To enable sshd, run: apt-get install openssh-server

For security reason, container images ship without user accounts
and without a root password.

Use lxc-attach or chroot directly into the rootfs to set a root password
or create user accounts.

# lxc-start -n test
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the
container in foreground mode.
lxc-start: lxc_start.c: main: 348 Additional information can be
obtained by setting the --logfile and --logpriority options.

# lxc-start -F -n test
lxc-start: start.c: print_top_failing_dir: 102 Permission denied -
could not access /var/lib/lxc. Please grant it 'x' access, or add an
ACL for the container root.
lxc-start: sync.c: __sync_wait: 51 invalid sequence number 1. expected 2
lxc-start: start.c: __lxc_start: 1157 failed to spawn 'test'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be
obtained by setting the --logfile and --logpriority options.

# chmod o+x /var/lib/lxc

# lxc-start -n test

# lxc-ls -f --running
NAME STATE IPV4 IPV6 GROUPS AUTOSTART
--------------------------------------------
test RUNNING - - - NO

# lxc-attach -n test -- cat /proc/1/uid_map
0 1000000 100000
--
Fajar
Xavier Gendre
2015-04-13 15:41:43 UTC
Permalink
Hello Fajar,
Post by Fajar A. Nugraha
Post by Xavier Gendre
It may be useful to give more details about what i am trying to do ;-) I
work with a Debian Jessie and LXC 1.0.6 from the Debian repository.
You should realy use at least 1.0.7 from experimental, or better yet, 1.1.1.
That was a good advice... with 1.0.7 from experimental, everything works
like a charm. I can create root owned unprivileged containers with lvm
backend and start them without any tweak (no chmod, ...).

Thank you for spending some time on my problem and giving me hope with a
working example ;-)

Xavier
Post by Fajar A. Nugraha
Post by Xavier Gendre
Finally, i try to start this container but it miserably fails,
root # lxc-start -n test --logfile test.log --logpriority DEBUG
lxc-start: failed to determine fs type for '/dev/Pool/test'
lxc-start: failed to determine fs type for '/dev/dm-7'
lxc-start: failed to mount rootfs
Works for me, just needed a chmod (which is shown on the helpful error
message that I get). This is lxc 1.1.1+master~20150407-0 from ubuntu
daily ppa.
# cat /etc/subuid
lxc-dnsmasq:100000:10000
user:100000:65537
root:1000000:100000
# cat /etc/subgid
lxc-dnsmasq:100000:10000
user:100000:65537
root:1000000:100000
# cat << END > /tmp/test.conf
Post by Xavier Gendre
lxc.id_map = u 0 1000000 100000
lxc.id_map = g 0 1000000 100000
lxc.network.type = empty
END
# lxc-create -n test -f /tmp/test.conf -t download -B lvm --vgname lxc
-- -d ubuntu -r utopic -a amd64
File descriptor 3 (/var/lib/lxc/test/partial) leaked on lvcreate
invocation. Parent PID 24304: lxc-create
Logical volume "test" created
Using image from local cache
Unpacking the rootfs
---
You just created an Ubuntu container (release=utopic, arch=amd64,
variant=default)
To enable sshd, run: apt-get install openssh-server
For security reason, container images ship without user accounts
and without a root password.
Use lxc-attach or chroot directly into the rootfs to set a root password
or create user accounts.
# lxc-start -n test
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 346 To get more details, run the
container in foreground mode.
lxc-start: lxc_start.c: main: 348 Additional information can be
obtained by setting the --logfile and --logpriority options.
# lxc-start -F -n test
lxc-start: start.c: print_top_failing_dir: 102 Permission denied -
could not access /var/lib/lxc. Please grant it 'x' access, or add an
ACL for the container root.
lxc-start: sync.c: __sync_wait: 51 invalid sequence number 1. expected 2
lxc-start: start.c: __lxc_start: 1157 failed to spawn 'test'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be
obtained by setting the --logfile and --logpriority options.
# chmod o+x /var/lib/lxc
# lxc-start -n test
# lxc-ls -f --running
NAME STATE IPV4 IPV6 GROUPS AUTOSTART
--------------------------------------------
test RUNNING - - - NO
# lxc-attach -n test -- cat /proc/1/uid_map
0 1000000 100000
Continue reading on narkive:
Loading...