Discussion:
Can not start containers using latest tools from the git
(too old to reply)
John Drescher
2010-01-21 04:29:07 UTC
Permalink
localhost ~ # lxc-create -n guest_1 -f /etc/lxc/guest_1.conf
'guest_1' created
localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-wXIi91/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container

localhost ~ # cat /etc/lxc/guest_1.conf
lxc.utsname = guest_1
lxc.mount = /etc/lxc/guest_1.fstab
lxc.rootfs = /lxc/guest_1
lxc.tty = 4
lxc.pts = 1024
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.name = veth0
lxc.network.mtu = 1500

lxc.cgroup.devices.deny = a # Deny all access to devices

lxc.cgroup.devices.allow = c 1:3 rwm # dev/null
lxc.cgroup.devices.allow = c 1:5 rwm # dev/zero

lxc.cgroup.devices.allow = c 5:1 rwm # dev/console
lxc.cgroup.devices.allow = c 5:0 rwm # dev/tty
lxc.cgroup.devices.allow = c 4:0 rwm # dev/tty0
lxc.cgroup.devices.allow = c 4:1 rwm # dev/tty1
lxc.cgroup.devices.allow = c 4:2 rwm # dev/tty2


lxc.cgroup.devices.allow = c 1:9 rwm # dev/urandon
lxc.cgroup.devices.allow = c 1:8 rwm # dev/random
lxc.cgroup.devices.allow = c 136:* rwm # dev/pts/*
lxc.cgroup.devices.allow = c 5:2 rwm # dev/pts/ptmx

#rtc
lxc.cgroup.devices.allow = c 254:0 rwm


localhost ~ # cat /etc/lxc/guest_1.fstab
none /lxc/guest_1/dev/pts devpts defaults 0 0
#none /lxc/guest_1/proc proc defaults 0 0
#none /lxc/guest_1/sys sysfs defaults 0 0
none /lxc/guest_1/dev/shm tmpfs defaults 0 0
/usr/portage /lxc/guest_1/usr/portage none rw,bind 1 0

Thanks,
John M. Drescher
Michael H. Warfield
2010-01-21 04:51:33 UTC
Permalink
Post by John Drescher
localhost ~ # lxc-create -n guest_1 -f /etc/lxc/guest_1.conf
'guest_1' created
localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-wXIi91/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container
When did you pull that git? I had that problem a week or so ago and
reported it on lxc-devel. Just yesterday, they checked in some stuff
that should fix it but I haven't had a chance to retest. Do a "git
pull" to update and check to see if the latest patches fix that.

Just for something really REALLY off the wall, try unmounting /dev/shm
and then retry the start. If it starts, that's exactly my problem that
I have not been able to reproduce. I'm hoping the developers are now on
this list as well.

Regards,
Mike
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20100120/5177d726/attachment.pgp>
John Drescher
2010-01-21 04:56:37 UTC
Permalink
Post by Michael H. Warfield
Post by John Drescher
localhost ~ # lxc-create -n guest_1 -f /etc/lxc/guest_1.conf
'guest_1' created
localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-wXIi91/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container
When did you pull that git?
Less than 1 hour ago and I double checked that it was current from the
sourceforge git

http://lxc.git.sourceforge.net/git/gitweb.cgi?p=lxc/lxc;a=summary
Post by Michael H. Warfield
?I had that problem a week or so ago and
reported it on lxc-devel. ?Just yesterday, they checked in some stuff
that should fix it but I haven't had a chance to retest. ?Do a "git
pull" to update and check to see if the latest patches fix that.
Just for something really REALLY off the wall, try unmounting /dev/shm
and then retry the start. ?If it starts, that's exactly my problem that
I have not been able to reproduce. ?I'm hoping the developers are now on
this list as well.
I will try to test this tonight if I get a chance. I just closed the
vm I was testing that in..

John
John Drescher
2010-01-21 05:29:26 UTC
Permalink
Post by John Drescher
Post by Michael H. Warfield
?I had that problem a week or so ago and
reported it on lxc-devel. ?Just yesterday, they checked in some stuff
that should fix it but I haven't had a chance to retest. ?Do a "git
pull" to update and check to see if the latest patches fix that.
Just for something really REALLY off the wall, try unmounting /dev/shm
and then retry the start. ?If it starts, that's exactly my problem that
I have not been able to reproduce. ?I'm hoping the developers are now on
this list as well.
I will try to test this tonight if I get a chance. I just closed the
vm I was testing that in..
I did not have /dev/shm mounted in the first place. I tried mounting,
unmounting and in all cases same result:
localhost ~ # mount /dev/shm
localhost ~ # rm /lxc/guest_1/lxc-oldrootfs* -rf
localhost ~ # lxc-start -n guest_1
lxc-start: Device or resource busy - could not unmount old rootfs
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container
localhost ~ # umount /dev/shm
localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-DUZVTO/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container
localhost ~ # rm /lxc/guest_1/lxc-oldrootfs* -rf
localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-adU2gT/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container

John
Daniel Lezcano
2010-01-21 08:26:15 UTC
Permalink
Post by John Drescher
Post by John Drescher
Post by Michael H. Warfield
I had that problem a week or so ago and
reported it on lxc-devel. Just yesterday, they checked in some stuff
that should fix it but I haven't had a chance to retest. Do a "git
pull" to update and check to see if the latest patches fix that.
Just for something really REALLY off the wall, try unmounting /dev/shm
and then retry the start. If it starts, that's exactly my problem that
I have not been able to reproduce. I'm hoping the developers are now on
this list as well.
I will try to test this tonight if I get a chance. I just closed the
vm I was testing that in..
I did not have /dev/shm mounted in the first place. I tried mounting,
localhost ~ # mount /dev/shm
localhost ~ # rm /lxc/guest_1/lxc-oldrootfs* -rf
localhost ~ # lxc-start -n guest_1
lxc-start: Device or resource busy - could not unmount old rootfs
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container
localhost ~ # umount /dev/shm
localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-DUZVTO/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container
localhost ~ # rm /lxc/guest_1/lxc-oldrootfs* -rf
localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-adU2gT/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container
Hi John,

can you give the following information ?
* content of the /proc/mounts file
* content of the /dev/shm directory (and a fuser of each file there is in)
* kernel version
* distro

Thanks
-- Daniel
John Drescher
2010-01-21 14:36:02 UTC
Permalink
Post by Daniel Lezcano
can you give the following information ?
?* content of the /proc/mounts file
localhost ~ # cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/sda2 / ext4 rw,noatime,barrier=1,data=ordered 0 0
proc /proc proc rw,relatime 0 0
rc-svcdir /lib64/rc/init.d tmpfs
rw,nosuid,nodev,noexec,relatime,size=1024k,mode=755 0 0
none /dev/pts devpts rw,relatime,mode=600,ptmxmode=000 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
udev /dev tmpfs rw,nosuid,relatime,size=10240k,mode=755 0 0
none /cgroup cgroup rw,relatime,freezer,devices,memory,cpuacct,cpu,ns,cpuset 0 0
usbfs /proc/bus/usb usbfs rw,nosuid,noexec,relatime,devgid=85,devmode=664 0 0
none /dev/pts devpts rw,relatime,mode=600,ptmxmode=000 0 0
Post by Daniel Lezcano
?* content of the /dev/shm directory (and a fuser of each file there is in)
localhost ~ # ls -al /dev/shm
total 0
drwxr-xr-x 2 root root 40 Jan 21 09:31 .
drwxr-xr-x 11 root root 3100 Jan 21 09:31 ..
Post by Daniel Lezcano
?* kernel version
localhost ~ # uname -a
Linux localhost 2.6.32-gentoo #10 SMP Sat Jan 16 23:50:38 Local time
zone must be set--see zic x86_64 Intel(R) Core(TM)2 Quad CPU Q9550 @
2.83GHz GenuineIntel GNU/Linux
Post by Daniel Lezcano
?* distro
gentoo x86_64

localhost ~ # emerge --info
Portage 2.2_rc61 (default/linux/amd64/10.0, gcc-4.3.4,
glibc-2.10.1-r1, 2.6.32-gentoo x86_64)
=================================================================
System uname: Linux-2.6.32-gentoo-x86_64-Intel-R-_Core-TM-2_Quad_CPU_Q9550_ at _2.83GHz-with-gentoo-2.0.1
Timestamp of tree: Wed, 20 Jan 2010 05:00:02 +0000
app-shells/bash: 4.0_p35
dev-lang/python: 2.5.4-r3, 2.6.4
sys-apps/baselayout: 2.0.1
sys-apps/openrc: 0.6.0
sys-apps/sandbox: 2.2
sys-devel/autoconf: 2.63-r1
sys-devel/automake: 1.9.6-r2, 1.10.2
sys-devel/binutils: 2.18-r3
sys-devel/gcc-config: 1.4.1
sys-devel/libtool: 2.2.6b
virtual/os-headers: 2.6.30-r1
ACCEPT_KEYWORDS="amd64"
ACCEPT_LICENSE="* - at EULA"
CBUILD="x86_64-pc-linux-gnu"
CFLAGS="-march=core2 -O2 -pipe"
CHOST="x86_64-pc-linux-gnu"
CONFIG_PROTECT="/etc"
CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/env.d /etc/gconf
/etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo
/etc/udev/rules.d"
CXXFLAGS="-O2 -pipe"
DISTDIR="/usr/portage/distfiles"
EMERGE_DEFAULT_OPTS="--keep-going --jobs 2"
FEATURES="assume-digests distlocks fixpackages news parallel-fetch
preserve-libs protect-owned sandbox sfperms strict unmerge-logs
unmerge-orphans userfetch"
GENTOO_MIRRORS="http://distfiles.gentoo.org
http://distro.ibiblio.org/pub/linux/distributions/gentoo"
LDFLAGS="-Wl,-O1"
LINGUAS="en en_US"
MAKEOPTS="-j5"
PKGDIR="/usr/portage/packages"
PORTAGE_CONFIGROOT="/"
PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times
--compress --force --whole-file --delete --stats --timeout=180
--exclude=/distfiles --exclude=/local --exclude=/packages"
PORTAGE_TMPDIR="/var/tmp"
PORTDIR="/usr/portage"
PORTDIR_OVERLAY="/usr/local/portage/layman/jmd-gentoo"
SYNC="rsync://rsync.gentoo.org/gentoo-portage"
USE="acl amd64 bash-completion berkdb bzip2 cli cracklib crypt cups
cxx dri examples fortran gdbm git gpm iconv ipv6 lxc mmx modules
mudflap multilib ncurses nls nptl nptlonly openmp pam pcre perl pppd
python readline reflection session spl sse sse2 ssl sysfs tcpd unicode
xorg zlib" ALSA_CARDS="ali5451 als4000 atiixp atiixp-modem bt87x
ca0106 cmipci emu10k1x ens1370 ens1371 es1938 es1968 fm801 hda-intel
intel8x0 intel8x0m maestro3 trident usb-audio via82xx via82xx-modem
ymfpci" ALSA_PCM_PLUGINS="adpcm alaw asym copy dmix dshare dsnoop
empty extplug file hooks iec958 ioplug ladspa lfloat linear meter
mmap_emul mulaw multi null plug rate route share shm softvol"
APACHE2_MODULES="actions alias auth_basic authn_alias authn_anon
authn_dbm authn_default authn_file authz_dbm authz_default
authz_groupfile authz_host authz_owner authz_user autoindex cache dav
dav_fs dav_lock deflate dir disk_cache env expires ext_filter
file_cache filter headers include info log_config logio mem_cache mime
mime_magic negotiation rewrite setenvif speling status unique_id
userdir usertrack vhost_alias" ELIBC="glibc" INPUT_DEVICES="keyboard
mouse evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk
hd44780 lb216 lcdm001 mtxorb ncurses text" LINGUAS="en en_US"
RUBY_TARGETS="ruby18" USERLAND="GNU" VIDEO_CARDS="fbdev glint intel
mach64 mga neomagic nv r128 radeon savage sis tdfx trident vesa via
vmware voodoo"
Unset: CPPFLAGS, CTARGET, FFLAGS, INSTALL_MASK, LANG, LC_ALL,
PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS




John
Daniel Lezcano
2010-01-21 15:21:39 UTC
Permalink
Post by John Drescher
Post by Daniel Lezcano
can you give the following information ?
* content of the /proc/mounts file
localhost ~ # cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/sda2 / ext4 rw,noatime,barrier=1,data=ordered 0 0
proc /proc proc rw,relatime 0 0
rc-svcdir /lib64/rc/init.d tmpfs
rw,nosuid,nodev,noexec,relatime,size=1024k,mode=755 0 0
none /dev/pts devpts rw,relatime,mode=600,ptmxmode=000 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev,noexec,relatime 0 0
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
udev /dev tmpfs rw,nosuid,relatime,size=10240k,mode=755 0 0
none /cgroup cgroup rw,relatime,freezer,devices,memory,cpuacct,cpu,ns,cpuset 0 0
usbfs /proc/bus/usb usbfs rw,nosuid,noexec,relatime,devgid=85,devmode=664 0 0
none /dev/pts devpts rw,relatime,mode=600,ptmxmode=000 0 0
Thank you for the details.

I think the problem is coming from the /dev/shm tmpfs directory is
overwritten by the next udev tmpfs mount.
So the lxc code finds, in /proc/mounts, the /dev/shm mount point and
tries to unmount it, but this one is not visible yet because /dev has to
be unmounted before.

This is something to be fixed before the 0.6.5 release.
But I think the /dev/shm mount point is useless and IMO you can comment
it from your /fstab and check if the container starts.
John Drescher
2010-01-21 15:28:12 UTC
Permalink
Post by Daniel Lezcano
I think the problem is coming from the /dev/shm tmpfs directory is
overwritten by the next udev tmpfs mount.
So the lxc code finds, in /proc/mounts, the /dev/shm mount point and tries
to unmount it, but this one is not visible yet because /dev has to be
unmounted before.
This is something to be fixed before the 0.6.5 release.
But I think the /dev/shm mount point is useless and IMO you can comment it
from your /fstab and check if the container starts.
The container did not start without /dev/shm either.

Actually I did not have /dev/shm mounted or in my /etc/fstab until
Michael Warfield suggested he had a similar problem caused when
/dev/shm was mounted so I tried to see if toggling between mounted and
not mounted caused it to work at least 1 time.

I have not had this issue with 0.6.4 or 0.6.3 in the few hours (6 to
12) I have tested each.

Thanks,
John
Daniel Lezcano
2010-01-21 15:38:57 UTC
Permalink
Post by John Drescher
Post by Daniel Lezcano
I think the problem is coming from the /dev/shm tmpfs directory is
overwritten by the next udev tmpfs mount.
So the lxc code finds, in /proc/mounts, the /dev/shm mount point and tries
to unmount it, but this one is not visible yet because /dev has to be
unmounted before.
This is something to be fixed before the 0.6.5 release.
But I think the /dev/shm mount point is useless and IMO you can comment it
from your /fstab and check if the container starts.
The container did not start without /dev/shm either.
Actually I did not have /dev/shm mounted or in my /etc/fstab until
Michael Warfield suggested he had a similar problem caused when
/dev/shm was mounted so I tried to see if toggling between mounted and
not mounted caused it to work at least 1 time.
I have not had this issue with 0.6.4 or 0.6.3 in the few hours (6 to
12) I have tested each.
Do you mind to test the patch in attachment ?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: fix-unmount-order.patch
Type: text/x-diff
Size: 913 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20100121/f901e806/attachment.patch>
John Drescher
2010-01-21 15:39:53 UTC
Permalink
Post by Daniel Lezcano
Do you mind to test the patch in attachment ?
Will do in 10 hours. I have to get to the day job..

John
Daniel Lezcano
2010-01-21 15:43:02 UTC
Permalink
Post by John Drescher
Post by Daniel Lezcano
Do you mind to test the patch in attachment ?
Will do in 10 hours. I have to get to the day job..
Thanks John.
John Drescher
2010-01-22 04:15:26 UTC
Permalink
Post by Daniel Lezcano
Do you mind to test the patch in attachment ?
Same problem.

localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-fsBvCO/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container


John
Daniel Lezcano
2010-01-22 07:58:39 UTC
Permalink
Post by John Drescher
Post by Daniel Lezcano
Do you mind to test the patch in attachment ?
Same problem.
localhost ~ # lxc-start -n guest_1
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-fsBvCO/dev/shm'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start: failed to setup the container
Gah ! ok, the last request : can you launch the container with the
following command line ?

lxc-start -n guest_1 -l DEBUG -o $(tty)

and give the output.

Is it possible the /dev/shm was mounted at the boot time on the initrd
disk ?
John Drescher
2010-01-22 13:54:16 UTC
Permalink
Gah ! ok, the last request : can you launch the container with the following
command line ?
? lxc-start -n guest_1 -l DEBUG -o $(tty)
and give the output.
localhost ~ # lxc-start -n guest_1 -l DEBUG -o $(tty)
lxc-start 1264150065.375 INFO lxc_conf - tty's configured
lxc-start 1264150065.375 DEBUG lxc_start - sigchild handler set
lxc-start 1264150065.375 INFO lxc_start - 'guest_1' is initialized
lxc-start 1264150065.379 DEBUG lxc_conf - instanciated veth
'vethBhWIhY/vetheRH7qk', index is '10'
lxc-start 1264150065.438 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264150065.438 DEBUG lxc_cgroup - '/cgroup/2876'
renamed to '/cgroup/guest_1'
lxc-start 1264150065.513 DEBUG lxc_conf - move 'br0' to '2876'
lxc-start 1264150065.513 INFO lxc_conf - 'guest_1' hostname
has been setup
lxc-start 1264150065.516 DEBUG lxc_conf - 'veth0' has been setup
lxc-start 1264150065.516 INFO lxc_conf - network has been setup
lxc-start 1264150065.516 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.deny' set to 'a # Deny all access to devices'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:3 rwm # dev/null'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:5 rwm # dev/zero'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:1 rwm # dev/console'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:0 rwm # dev/tty'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:0 rwm # dev/tty0'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:1 rwm # dev/tty1'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:2 rwm # dev/tty2'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:9 rwm # dev/urandon'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:8 rwm # dev/random'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 136:* rwm # dev/pts/*'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:2 rwm # dev/pts/ptmx'
lxc-start 1264150065.517 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 254:0 rwm'
lxc-start 1264150065.517 INFO lxc_conf - cgroup has been setup
lxc-start 1264150065.517 DEBUG lxc_conf - mounted none on
/lxc/guest_1/dev/pts, type devpts
lxc-start 1264150065.517 DEBUG lxc_conf - mounted none on
/lxc/guest_1/dev/shm, type tmpfs
lxc-start 1264150065.517 DEBUG lxc_conf - mounted
/usr/portage on /lxc/guest_1/usr/portage, type none
lxc-start 1264150065.517 INFO lxc_conf - mount points have been setup
lxc-start 1264150065.517 INFO lxc_conf - mount points have been setup
lxc-start 1264150065.518 INFO lxc_conf - console
'/dev/pts/0' mounted to '/lxc/guest_1/dev/console'
lxc-start 1264150065.518 INFO lxc_conf - 4 tty(s) has been setup
lxc-start 1264150065.518 DEBUG lxc_conf - temporary
mountpoint for old rootfs is './lxc-oldrootfs-FJUuhq'
lxc-start 1264150065.518 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-FJUuhq' successful
lxc-start 1264150065.518 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-FJUuhq/proc/bus/usb'
lxc-start 1264150065.518 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-FJUuhq/lib64/rc/init.d'
lxc-start 1264150065.518 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-FJUuhq/dev/pts'
lxc-start 1264150065.519 ERROR lxc_conf - Invalid argument -
failed to umount '/lxc-oldrootfs-FJUuhq/dev/shm'
lxc-start: Invalid argument - failed to umount '/lxc-oldrootfs-FJUuhq/dev/shm'
lxc-start 1264150065.519 ERROR lxc_conf - failed to
pivot_root to '/lxc/guest_1'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start 1264150065.519 ERROR lxc_conf - failed to set
rootfs for 'guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start 1264150065.519 ERROR lxc_start - failed to setup
the container
lxc-start: failed to setup the container
lxc-start 1264150065.519 NOTICE lxc_start - '/sbin/init'
started with pid '2876'
lxc-start 1264150065.519 DEBUG lxc_utils - closing fd '1'
lxc-start 1264150065.519 DEBUG lxc_utils - closing fd '0'
lxc-start 1264150065.519 DEBUG lxc_utils - closed all
inherited file descriptors
lxc-start 1264150065.553 DEBUG lxc_start - child exited
lxc-start 1264150065.553 INFO lxc_error - child <2876> ended
on error (255)
lxc-start 1264150065.553 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264150065.593 DEBUG lxc_cgroup - '/cgroup/guest_1' unlinked
Is it possible the /dev/shm was mounted at the boot time on the initrd disk
?
On the host yes that is possible. I believe it would be easy to remove
the initrd on this machine since it's pretty scaled down being that it
runs inside of virtualbox.
--
John M. Drescher
Daniel Lezcano
2010-01-22 13:59:03 UTC
Permalink
Post by John Drescher
Gah ! ok, the last request : can you launch the container with the following
command line ?
lxc-start -n guest_1 -l DEBUG -o $(tty)
and give the output.
Thanks for sending the output.

[ ... ]
Post by John Drescher
Is it possible the /dev/shm was mounted at the boot time on the initrd disk
?
On the host yes that is possible. I believe it would be easy to remove
the initrd on this machine since it's pretty scaled down being that it
runs inside of virtualbox.
A did a change and commited it in the git tree. I think you should be
able to start the container now. If it's the case I would be interested
to see the log output with the fix.

Thanks
-- Daniel
John Drescher
2010-01-22 14:10:52 UTC
Permalink
Post by Daniel Lezcano
Post by John Drescher
Gah ! ok, the last request : can you launch the container with the following
command line ?
?lxc-start -n guest_1 -l DEBUG -o $(tty)
and give the output.
Thanks for sending the output.
[ ... ]
Post by John Drescher
Is it possible the /dev/shm was mounted at the boot time on the initrd disk
?
On the host yes that is possible. I believe it would be easy to remove
the initrd on this machine since it's pretty scaled down being that it
runs inside of virtualbox.
A did a change and commited it in the git tree. I think you should be able
to start the container now. If it's the case I would be interested to see
the log output with the fix.
Still did not start. BTW, I disabled the initrd on the host however I
did see a line during the host boot
mounting /dev/shm it is possible that openrc (new gentoo init system)
is mounting /dev/shm in the guest
as well. I can probably disable this after I find out what entry is
causing that.

localhost ~ # lxc-start -n guest_1 -l DEBUG -o $(tty)
lxc-start 1264151162.397 INFO lxc_conf - tty's configured
lxc-start 1264151162.397 DEBUG lxc_start - sigchild handler set
lxc-start 1264151162.397 INFO lxc_start - 'guest_1' is initialized
lxc-start 1264151162.400 DEBUG lxc_conf - instanciated veth
'vethSCsXtA/vethuC7t3w', index is '10'
lxc-start 1264151162.447 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264151162.448 DEBUG lxc_cgroup - '/cgroup/8422'
renamed to '/cgroup/guest_1'
lxc-start 1264151162.503 DEBUG lxc_conf - move 'br0' to '8422'
lxc-start 1264151162.503 INFO lxc_conf - 'guest_1' hostname
has been setup
lxc-start 1264151162.506 DEBUG lxc_conf - 'veth0' has been setup
lxc-start 1264151162.506 INFO lxc_conf - network has been setup
lxc-start 1264151162.506 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.deny' set to 'a # Deny all access to devices'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:3 rwm # dev/null'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:5 rwm # dev/zero'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:1 rwm # dev/console'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:0 rwm # dev/tty'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:0 rwm # dev/tty0'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:1 rwm # dev/tty1'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:2 rwm # dev/tty2'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:9 rwm # dev/urandon'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:8 rwm # dev/random'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 136:* rwm # dev/pts/*'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:2 rwm # dev/pts/ptmx'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 254:0 rwm'
lxc-start 1264151162.507 INFO lxc_conf - cgroup has been setup
lxc-start 1264151162.507 DEBUG lxc_conf - mounted none on
/lxc/guest_1/dev/pts, type devpts
lxc-start 1264151162.507 DEBUG lxc_conf - mounted none on
/lxc/guest_1/dev/shm, type tmpfs
lxc-start 1264151162.507 DEBUG lxc_conf - mounted
/usr/portage on /lxc/guest_1/usr/portage, type none
lxc-start 1264151162.507 INFO lxc_conf - mount points have been setup
lxc-start 1264151162.507 INFO lxc_conf - mount points have been setup
lxc-start 1264151162.507 INFO lxc_conf - console
'/dev/pts/0' mounted to '/lxc/guest_1/dev/console'
lxc-start 1264151162.507 INFO lxc_conf - 4 tty(s) has been setup
lxc-start 1264151162.508 DEBUG lxc_conf - temporary
mountpoint for old rootfs is './lxc-oldrootfs-WIzwQn'
lxc-start 1264151162.508 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-WIzwQn' successful
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/proc/bus/usb'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lib64/rc/init.d'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/dev/pts'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/sys'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/dev'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/cgroup'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/pts'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/shm'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/usr/portage'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/console'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/tty1'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/tty2'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/tty3'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/tty4'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/proc'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/dev/shm'
lxc-start 1264151162.508 ERROR lxc_conf - Device or resource
busy - could not unmount old rootfs
lxc-start: Device or resource busy - could not unmount old rootfs
lxc-start 1264151162.508 ERROR lxc_conf - failed to
pivot_root to '/lxc/guest_1'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start 1264151162.508 ERROR lxc_conf - failed to set
rootfs for 'guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start 1264151162.509 ERROR lxc_start - failed to setup
the container
lxc-start: failed to setup the container
lxc-start 1264151162.509 NOTICE lxc_start - '/sbin/init'
started with pid '8422'
lxc-start 1264151162.509 DEBUG lxc_utils - closing fd '1'
lxc-start 1264151162.509 DEBUG lxc_utils - closing fd '0'
lxc-start 1264151162.509 DEBUG lxc_utils - closed all
inherited file descriptors
lxc-start 1264151162.565 DEBUG lxc_start - child exited
lxc-start 1264151162.565 INFO lxc_error - child <8422> ended
on error (255)
lxc-start 1264151162.565 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264151162.645 DEBUG lxc_cgroup - '/cgroup/guest_1' unlinked


Also thanks a lot for your help!
John
John Drescher
2010-01-22 14:41:16 UTC
Permalink
It turns out I was mounting /dev/shm in the guest. After commenting
out that I still can not start my container.

localhost ~ # cat /etc/lxc/guest_1.fstab
none /lxc/guest_1/dev/pts devpts defaults 0 0
#none /lxc/guest_1/proc proc defaults 0 0
#none /lxc/guest_1/sys sysfs defaults 0 0
#none /lxc/guest_1/dev/shm tmpfs defaults 0 0
/usr/portage /lxc/guest_1/usr/portage none rw,bind 1 0


localhost ~ # lxc-start -n guest_1 -l DEBUG -o $(tty)
lxc-start 1264153081.350 INFO lxc_conf - tty's configured
lxc-start 1264153081.350 DEBUG lxc_start - sigchild handler set
lxc-start 1264153081.350 INFO lxc_start - 'guest_1' is initialized
lxc-start 1264153081.353 DEBUG lxc_conf - instanciated veth
'vethJ2NA8g/vethuoeij0', index is '28'
lxc-start 1264153081.413 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153081.414 DEBUG lxc_cgroup - '/cgroup/1466'
renamed to '/cgroup/guest_1'
lxc-start 1264153081.481 DEBUG lxc_conf - move 'br0' to '1466'
lxc-start 1264153081.482 INFO lxc_conf - 'guest_1' hostname
has been setup
lxc-start 1264153081.483 DEBUG lxc_conf - 'veth0' has been setup
lxc-start 1264153081.483 INFO lxc_conf - network has been setup
lxc-start 1264153081.484 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.deny' set to 'a # Deny all access to devices'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:3 rwm # dev/null'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:5 rwm # dev/zero'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:1 rwm # dev/console'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:0 rwm # dev/tty'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:0 rwm # dev/tty0'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:1 rwm # dev/tty1'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:2 rwm # dev/tty2'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:9 rwm # dev/urandon'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:8 rwm # dev/random'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 136:* rwm # dev/pts/*'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:2 rwm # dev/pts/ptmx'
lxc-start 1264153081.485 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 254:0 rwm'
lxc-start 1264153081.485 INFO lxc_conf - cgroup has been setup
lxc-start 1264153081.485 DEBUG lxc_conf - mounted none on
/lxc/guest_1/dev/pts, type devpts
lxc-start 1264153081.485 DEBUG lxc_conf - mounted
/usr/portage on /lxc/guest_1/usr/portage, type none
lxc-start 1264153081.485 INFO lxc_conf - mount points have been setup
lxc-start 1264153081.486 INFO lxc_conf - mount points have been setup
lxc-start 1264153081.486 INFO lxc_conf - console
'/dev/pts/0' mounted to '/lxc/guest_1/dev/console'
lxc-start 1264153081.486 INFO lxc_conf - 4 tty(s) has been setup
lxc-start 1264153081.486 DEBUG lxc_conf - temporary
mountpoint for old rootfs is './lxc-oldrootfs-Jw2y9c'
lxc-start 1264153081.486 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-Jw2y9c' successful
lxc-start 1264153081.486 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/proc/bus/usb'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/lib64/rc/init.d'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/dev/pts'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/sys'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/dev'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/cgroup'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/lxc/guest_1/dev/pts'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/lxc/guest_1/usr/portage'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/lxc/guest_1/dev/console'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/lxc/guest_1/dev/tty1'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/lxc/guest_1/dev/tty2'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/lxc/guest_1/dev/tty3'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/lxc/guest_1/dev/tty4'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/proc'
lxc-start 1264153081.487 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-Jw2y9c/dev/shm'
lxc-start 1264153081.487 ERROR lxc_conf - Device or resource
busy - could not unmount old rootfs
lxc-start: Device or resource busy - could not unmount old rootfs
lxc-start 1264153081.487 ERROR lxc_conf - failed to
pivot_root to '/lxc/guest_1'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start 1264153081.487 ERROR lxc_conf - failed to set
rootfs for 'guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start 1264153081.487 ERROR lxc_start - failed to setup
the container
lxc-start: failed to setup the container
lxc-start 1264153081.487 NOTICE lxc_start - '/sbin/init'
started with pid '1466'
lxc-start 1264153081.487 DEBUG lxc_utils - closing fd '1'
lxc-start 1264153081.487 DEBUG lxc_utils - closing fd '0'
lxc-start 1264153081.487 DEBUG lxc_utils - closed all
inherited file descriptors
lxc-start 1264153081.521 DEBUG lxc_start - child exited
lxc-start 1264153081.521 INFO lxc_error - child <1466> ended
on error (255)
lxc-start 1264153081.522 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153081.561 DEBUG lxc_cgroup - '/cgroup/guest_1' unlinked

I then disabled all mounts in the guest_1.fstab and it still did not work:
localhost ~ # lxc-start -n guest_1 -l DEBUG -o $(tty)
lxc-start 1264153153.126 INFO lxc_conf - tty's configured
lxc-start 1264153153.126 DEBUG lxc_start - sigchild handler set
lxc-start 1264153153.126 INFO lxc_start - 'guest_1' is initialized
lxc-start 1264153153.127 DEBUG lxc_conf - instanciated veth
'vethwvedNM/vethgwP5Gh', index is '34'
lxc-start 1264153153.183 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153153.183 DEBUG lxc_cgroup - '/cgroup/1482'
renamed to '/cgroup/guest_1'
lxc-start 1264153153.261 DEBUG lxc_conf - move 'br0' to '1482'
lxc-start 1264153153.262 INFO lxc_conf - 'guest_1' hostname
has been setup
lxc-start 1264153153.263 DEBUG lxc_conf - 'veth0' has been setup
lxc-start 1264153153.263 INFO lxc_conf - network has been setup
lxc-start 1264153153.264 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.deny' set to 'a # Deny all access to devices'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:3 rwm # dev/null'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:5 rwm # dev/zero'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:1 rwm # dev/console'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:0 rwm # dev/tty'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:0 rwm # dev/tty0'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:1 rwm # dev/tty1'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:2 rwm # dev/tty2'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:9 rwm # dev/urandon'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:8 rwm # dev/random'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 136:* rwm # dev/pts/*'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:2 rwm # dev/pts/ptmx'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 254:0 rwm'
lxc-start 1264153153.265 INFO lxc_conf - cgroup has been setup
lxc-start 1264153153.265 INFO lxc_conf - mount points have been setup
lxc-start 1264153153.265 INFO lxc_conf - mount points have been setup
lxc-start 1264153153.265 INFO lxc_conf - console
'/dev/pts/0' mounted to '/lxc/guest_1/dev/console'
lxc-start 1264153153.265 INFO lxc_conf - 4 tty(s) has been setup
lxc-start 1264153153.265 DEBUG lxc_conf - temporary
mountpoint for old rootfs is './lxc-oldrootfs-WYH6KN'
lxc-start 1264153153.266 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-WYH6KN' successful
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/proc/bus/usb'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lib64/rc/init.d'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/dev/pts'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/sys'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/dev'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/cgroup'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/console'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/tty1'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/tty2'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/tty3'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/tty4'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/proc'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/dev/shm'
lxc-start 1264153153.266 ERROR lxc_conf - Device or resource
busy - could not unmount old rootfs
lxc-start: Device or resource busy - could not unmount old rootfs
lxc-start 1264153153.266 ERROR lxc_conf - failed to
pivot_root to '/lxc/guest_1'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start 1264153153.266 ERROR lxc_conf - failed to set
rootfs for 'guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start 1264153153.266 ERROR lxc_start - failed to setup
the container
lxc-start: failed to setup the container
lxc-start 1264153153.266 NOTICE lxc_start - '/sbin/init'
started with pid '1482'
lxc-start 1264153153.266 DEBUG lxc_utils - closing fd '1'
lxc-start 1264153153.266 DEBUG lxc_utils - closing fd '0'
lxc-start 1264153153.266 DEBUG lxc_utils - closed all
inherited file descriptors
lxc-start 1264153153.321 DEBUG lxc_start - child exited
lxc-start 1264153153.321 INFO lxc_error - child <1482> ended
on error (255)
lxc-start 1264153153.321 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153153.361 DEBUG lxc_cgroup - '/cgroup/guest_1' unlinked

John
Daniel Lezcano
2010-01-22 15:26:20 UTC
Permalink
Post by John Drescher
Post by Daniel Lezcano
Post by John Drescher
Gah ! ok, the last request : can you launch the container with the following
command line ?
lxc-start -n guest_1 -l DEBUG -o $(tty)
and give the output.
Thanks for sending the output.
[ ... ]
Post by John Drescher
Is it possible the /dev/shm was mounted at the boot time on the initrd disk
?
On the host yes that is possible. I believe it would be easy to remove
the initrd on this machine since it's pretty scaled down being that it
runs inside of virtualbox.
A did a change and commited it in the git tree. I think you should be able
to start the container now. If it's the case I would be interested to see
the log output with the fix.
Still did not start. BTW, I disabled the initrd on the host however I
did see a line during the host boot
mounting /dev/shm it is possible that openrc (new gentoo init system)
is mounting /dev/shm in the guest
as well. I can probably disable this after I find out what entry is
causing that.
localhost ~ # lxc-start -n guest_1 -l DEBUG -o $(tty)
lxc-start 1264151162.397 INFO lxc_conf - tty's configured
lxc-start 1264151162.397 DEBUG lxc_start - sigchild handler set
lxc-start 1264151162.397 INFO lxc_start - 'guest_1' is initialized
lxc-start 1264151162.400 DEBUG lxc_conf - instanciated veth
'vethSCsXtA/vethuC7t3w', index is '10'
lxc-start 1264151162.447 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264151162.448 DEBUG lxc_cgroup - '/cgroup/8422'
renamed to '/cgroup/guest_1'
lxc-start 1264151162.503 DEBUG lxc_conf - move 'br0' to '8422'
lxc-start 1264151162.503 INFO lxc_conf - 'guest_1' hostname
has been setup
lxc-start 1264151162.506 DEBUG lxc_conf - 'veth0' has been setup
lxc-start 1264151162.506 INFO lxc_conf - network has been setup
lxc-start 1264151162.506 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.deny' set to 'a # Deny all access to devices'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:3 rwm # dev/null'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:5 rwm # dev/zero'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:1 rwm # dev/console'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:0 rwm # dev/tty'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:0 rwm # dev/tty0'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:1 rwm # dev/tty1'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:2 rwm # dev/tty2'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:9 rwm # dev/urandon'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:8 rwm # dev/random'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 136:* rwm # dev/pts/*'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:2 rwm # dev/pts/ptmx'
lxc-start 1264151162.507 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 254:0 rwm'
lxc-start 1264151162.507 INFO lxc_conf - cgroup has been setup
lxc-start 1264151162.507 DEBUG lxc_conf - mounted none on
/lxc/guest_1/dev/pts, type devpts
lxc-start 1264151162.507 DEBUG lxc_conf - mounted none on
/lxc/guest_1/dev/shm, type tmpfs
The container mounts /dev/shm ...
Post by John Drescher
lxc-start 1264151162.507 DEBUG lxc_conf - mounted
/usr/portage on /lxc/guest_1/usr/portage, type none
lxc-start 1264151162.507 INFO lxc_conf - mount points have been setup
lxc-start 1264151162.507 INFO lxc_conf - mount points have been setup
lxc-start 1264151162.507 INFO lxc_conf - console
'/dev/pts/0' mounted to '/lxc/guest_1/dev/console'
lxc-start 1264151162.507 INFO lxc_conf - 4 tty(s) has been setup
lxc-start 1264151162.508 DEBUG lxc_conf - temporary
mountpoint for old rootfs is './lxc-oldrootfs-WIzwQn'
lxc-start 1264151162.508 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-WIzwQn' successful
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/proc/bus/usb'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lib64/rc/init.d'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/dev/pts'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/sys'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/dev'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/cgroup'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/pts'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/shm'
... and umounts it. The algo seems bogus.
Post by John Drescher
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/usr/portage'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/console'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/tty1'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/tty2'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/tty3'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/lxc/guest_1/dev/tty4'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/proc'
lxc-start 1264151162.508 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WIzwQn/dev/shm'
lxc-start 1264151162.508 ERROR lxc_conf - Device or resource
busy - could not unmount old rootfs
lxc-start: Device or resource busy - could not unmount old rootfs
lxc-start 1264151162.508 ERROR lxc_conf - failed to
pivot_root to '/lxc/guest_1'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start 1264151162.508 ERROR lxc_conf - failed to set
rootfs for 'guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start 1264151162.509 ERROR lxc_start - failed to setup
the container
lxc-start: failed to setup the container
lxc-start 1264151162.509 NOTICE lxc_start - '/sbin/init'
started with pid '8422'
lxc-start 1264151162.509 DEBUG lxc_utils - closing fd '1'
lxc-start 1264151162.509 DEBUG lxc_utils - closing fd '0'
lxc-start 1264151162.509 DEBUG lxc_utils - closed all
inherited file descriptors
lxc-start 1264151162.565 DEBUG lxc_start - child exited
lxc-start 1264151162.565 INFO lxc_error - child <8422> ended
on error (255)
lxc-start 1264151162.565 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264151162.645 DEBUG lxc_cgroup - '/cgroup/guest_1' unlinked
Ok, finally I reproduced it.
Can you remove the mount option in the container configuration file
(devpts and shm) ?
John Drescher
2010-01-22 16:49:03 UTC
Permalink
Post by Daniel Lezcano
Ok, finally I reproduced it.
Can you remove the mount option in the container configuration file (devpts
and shm) ?
You mean in my

/etc/lxc/guest_1.fstab ?

I disabled all mounts in that in the other email. Here was the output of that:


localhost ~ # lxc-start -n guest_1 -l DEBUG -o $(tty)
lxc-start 1264153153.126 INFO lxc_conf - tty's configured
lxc-start 1264153153.126 DEBUG lxc_start - sigchild handler set
lxc-start 1264153153.126 INFO lxc_start - 'guest_1' is initialized
lxc-start 1264153153.127 DEBUG lxc_conf - instanciated veth
'vethwvedNM/vethgwP5Gh', index is '34'
lxc-start 1264153153.183 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153153.183 DEBUG lxc_cgroup - '/cgroup/1482'
renamed to '/cgroup/guest_1'
lxc-start 1264153153.261 DEBUG lxc_conf - move 'br0' to '1482'
lxc-start 1264153153.262 INFO lxc_conf - 'guest_1' hostname
has been setup
lxc-start 1264153153.263 DEBUG lxc_conf - 'veth0' has been setup
lxc-start 1264153153.263 INFO lxc_conf - network has been setup
lxc-start 1264153153.264 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.deny' set to 'a # Deny all access to devices'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:3 rwm # dev/null'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:5 rwm # dev/zero'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:1 rwm # dev/console'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:0 rwm # dev/tty'
lxc-start 1264153153.264 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:0 rwm # dev/tty0'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:1 rwm # dev/tty1'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:2 rwm # dev/tty2'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:9 rwm # dev/urandon'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:8 rwm # dev/random'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 136:* rwm # dev/pts/*'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:2 rwm # dev/pts/ptmx'
lxc-start 1264153153.265 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 254:0 rwm'
lxc-start 1264153153.265 INFO lxc_conf - cgroup has been setup
lxc-start 1264153153.265 INFO lxc_conf - mount points have been setup
lxc-start 1264153153.265 INFO lxc_conf - mount points have been setup
lxc-start 1264153153.265 INFO lxc_conf - console
'/dev/pts/0' mounted to '/lxc/guest_1/dev/console'
lxc-start 1264153153.265 INFO lxc_conf - 4 tty(s) has been setup
lxc-start 1264153153.265 DEBUG lxc_conf - temporary
mountpoint for old rootfs is './lxc-oldrootfs-WYH6KN'
lxc-start 1264153153.266 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-WYH6KN' successful
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/proc/bus/usb'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lib64/rc/init.d'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/dev/pts'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/sys'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/dev'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/cgroup'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/console'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/tty1'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/tty2'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/tty3'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/lxc/guest_1/dev/tty4'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/proc'
lxc-start 1264153153.266 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-WYH6KN/dev/shm'
lxc-start 1264153153.266 ERROR lxc_conf - Device or resource
busy - could not unmount old rootfs
lxc-start: Device or resource busy - could not unmount old rootfs
lxc-start 1264153153.266 ERROR lxc_conf - failed to
pivot_root to '/lxc/guest_1'
lxc-start: failed to pivot_root to '/lxc/guest_1'
lxc-start 1264153153.266 ERROR lxc_conf - failed to set
rootfs for 'guest_1'
lxc-start: failed to set rootfs for 'guest_1'
lxc-start 1264153153.266 ERROR lxc_start - failed to setup
the container
lxc-start: failed to setup the container
lxc-start 1264153153.266 NOTICE lxc_start - '/sbin/init'
started with pid '1482'
lxc-start 1264153153.266 DEBUG lxc_utils - closing fd '1'
lxc-start 1264153153.266 DEBUG lxc_utils - closing fd '0'
lxc-start 1264153153.266 DEBUG lxc_utils - closed all
inherited file descriptors
lxc-start 1264153153.321 DEBUG lxc_start - child exited
lxc-start 1264153153.321 INFO lxc_error - child <1482> ended
on error (255)
lxc-start 1264153153.321 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264153153.361 DEBUG lxc_cgroup - '/cgroup/guest_1' unlinked
--
John M. Drescher
Daniel Lezcano
2010-01-22 17:03:58 UTC
Permalink
Post by John Drescher
Post by Daniel Lezcano
Ok, finally I reproduced it.
Can you remove the mount option in the container configuration file (devpts
and shm) ?
You mean in my
/etc/lxc/guest_1.fstab ?
Oh right, I missed it.
Mmmh, ok. I was able to reproduce it so I will try to fix properly.

Thanks.
-- Daniel
Brian K. White
2010-01-22 20:39:41 UTC
Permalink
This post might be inappropriate. Click to display it.
Brian K. White
2010-01-22 21:11:38 UTC
Permalink
However, now when I go to make a 2nd container, I can't start it.
I can create it, but not execute or start.
Well I'm more boggled now.
I stopped my first container nj12.
lxc-ls shows nothing, screen -ls shows nothing, mount shows nothing
extra, yet trying to start nj13 still fails, and trying to start nj12
still succeeds.

I can't find anything functionally different between nj12 and nj13...
What could I be missing???

nj10:~ # lxc-info -n nj12
'nj12' is STOPPED
nj10:~ # lxc-info -n nj13
'nj13' is STOPPED
nj10:~ # lxc-destroy -n nj12
'nj12' does not exist
nj10:~ # lxc-destroy -n nj13
'nj13' does not exist
nj10:~ # mount
/dev/md0 on / type reiserfs (rw,acl,user_xattr)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
debugfs on /sys/kernel/debug type debugfs (rw)
udev on /dev type tmpfs (rw)
devpts on /dev/pts type devpts (rw,mode=0620,gid=5)
/dev/sdg1 on /boot type ext2 (rw,acl,user_xattr)
fusectl on /sys/fs/fuse/connections type fusectl (rw)
securityfs on /sys/kernel/security type securityfs (rw)
none on /cgroup type cgroup (rw)
nj10:~ # lxc-ls
nj10:~ # ls -lR /etc/lxc
/etc/lxc:
total 0
drwxr-xr-x 2 root root 96 Jan 21 10:45 nj12
drwxr-xr-x 2 root root 96 Jan 22 12:00 nj13

/etc/lxc/nj12:
total 8
-rw-r--r-- 1 root root 275 Jan 21 10:45 config
-rw-r--r-- 1 root root 172 Jan 21 06:55 fstab

/etc/lxc/nj13:
total 8
-rw-r--r-- 1 root root 275 Jan 22 12:00 config
-rw-r--r-- 1 root root 172 Jan 22 12:00 fstab
nj10:~ #
nj10:~ # ls -l /lxc
total 1
drwxr-xr-x 23 root root 672 Jan 22 15:38 nj12
drwxr-xr-x 31 root root 904 Jan 22 15:39 nj13
nj10:~ #
nj10:~ # cat /etc/lxc/nj12/config
lxc.utsname = nj12

lxc.network.type = veth
lxc.network.flags = up

lxc.network.link = br1
lxc.network.hwaddr = 00:16:3e:00:01:4c
lxc.network.ipv4 = 71.187.206.76/24
lxc.network.name = eth0

#lxc.pts = 1024
lxc.tty = 1

lxc.mount = /etc/lxc/nj12/fstab
lxc.rootfs = /lxc/nj12
nj10:~ # cat /etc/lxc/nj13/config
lxc.utsname = nj13

lxc.network.type = veth
lxc.network.flags = up

lxc.network.link = br1
lxc.network.hwaddr = 00:16:3e:00:01:4d
lxc.network.ipv4 = 71.187.206.77/24
lxc.network.name = eth0

#lxc.pts = 1024
lxc.tty = 1

lxc.mount = /etc/lxc/nj13/fstab
lxc.rootfs = /lxc/nj13
nj10:~ # cat /etc/lxc/nj12/fstab
none /lxc/nj12/dev/pts devpts defaults 0 0
none /lxc/nj12/proc proc defaults 0 0
none /lxc/nj12/sys sysfs defaults 0 0
none /lxc/nj12/dev/shm tmpfs defaults 0 0
nj10:~ # cat /etc/lxc/nj13/fstab
none /lxc/nj13/dev/pts devpts defaults 0 0
none /lxc/nj13/proc proc defaults 0 0
none /lxc/nj13/sys sysfs defaults 0 0
none /lxc/nj13/dev/shm tmpfs defaults 0 0
nj10:~ #
nj10:~ # screen lxc-start -f /etc/lxc/nj13/config -l DEBUG -o
nj13log.txt -n nj13
nj10:~ #
nj10:~ # cat nj13log.txt
lxc-start 1264193483.594 INFO lxc_conf - tty's configured
lxc-start 1264193483.594 DEBUG lxc_start - sigchild handler set
lxc-start 1264193483.594 INFO lxc_start - 'nj13' is initialized
lxc-start 1264193483.609 DEBUG lxc_conf - instanciated veth
'vethGA8xND/vethpWLLCp', index is '110'
lxc-start 1264193483.627 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264193483.627 DEBUG lxc_cgroup - '/cgroup/22546'
renamed to '/cgroup/nj13'
lxc-start 1264193483.663 DEBUG lxc_conf - move 'br1' to '22546'
lxc-start 1264193483.663 INFO lxc_conf - 'nj13' hostname has
been setup
lxc-start 1264193483.671 DEBUG lxc_conf - mac address
'00:16:3e:00:01:4d' on 'eth0' has been setup
lxc-start 1264193483.687 DEBUG lxc_conf - 'eth0' has been setup
lxc-start 1264193483.687 INFO lxc_conf - network has been setup
lxc-start 1264193483.687 DEBUG lxc_conf - mounted none on
/lxc/nj13/dev/pts, type devpts
lxc-start 1264193483.687 DEBUG lxc_conf - mounted none on
/lxc/nj13/proc, type proc
lxc-start 1264193483.687 DEBUG lxc_conf - mounted none on
/lxc/nj13/sys, type sysfs
lxc-start 1264193483.687 DEBUG lxc_conf - mounted none on
/lxc/nj13/dev/shm, type tmpfs
lxc-start 1264193483.687 INFO lxc_conf - mount points have
been setup
lxc-start 1264193483.688 INFO lxc_conf - mount points have
been setup
lxc-start 1264193483.688 INFO lxc_conf - console '/dev/pts/2'
mounted to '/lxc/nj13/dev/console'
lxc-start 1264193483.688 WARN lxc_conf - failed to mount
'/dev/pts/3'->'/lxc/nj13/dev/tty1'
lxc-start 1264193483.688 INFO lxc_conf - 1 tty(s) has been setup
lxc-start 1264193483.688 DEBUG lxc_conf - temporary mountpoint
for old rootfs is './lxc-oldrootfs-2JcmIJ'
lxc-start 1264193483.688 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-2JcmIJ' successful
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/dev/pts'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/proc'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/sys/kernel/debug'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/sys/fs/fuse/connections'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/sys/kernel/security'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/boot'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/cgroup'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/lxc/nj13/proc'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/lxc/nj13/dev/pts'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/lxc/nj13/sys'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/lxc/nj13/dev/shm'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/lxc/nj13/dev/console'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/dev'
lxc-start 1264193483.688 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-2JcmIJ/sys'
lxc-start 1264193483.688 ERROR lxc_conf - Device or resource
busy - could not unmount old rootfs
lxc-start 1264193483.688 ERROR lxc_conf - failed to pivot_root
to '/lxc/nj13'
lxc-start 1264193483.688 ERROR lxc_conf - failed to set rootfs
for 'nj13'
lxc-start 1264193483.688 ERROR lxc_start - failed to setup the
container
lxc-start 1264193483.689 NOTICE lxc_start - '/sbin/init'
started with pid '22546'
lxc-start 1264193483.689 DEBUG lxc_utils - closing fd '1'
lxc-start 1264193483.689 DEBUG lxc_utils - closing fd '0'
lxc-start 1264193483.689 DEBUG lxc_utils - closed all
inherited file descriptors
lxc-start 1264193483.706 DEBUG lxc_start - child exited
lxc-start 1264193483.706 INFO lxc_error - child <22546> ended
on error (255)
lxc-start 1264193483.706 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264193483.726 DEBUG lxc_cgroup - '/cgroup/nj13'
unlinked
nj10:~ #
nj10:~ # lxc-create -f /etc/lxc/nj13/config -n nj13
'nj13' created
nj10:~ # lxc-execute -l DEBUG -o nj13bash.txt -n nj13 /bin/bash
lxc-execute: Device or resource busy - could not unmount old rootfs
lxc-execute: failed to pivot_root to '/lxc/nj13'
lxc-execute: failed to set rootfs for 'nj13'
lxc-execute: failed to setup the container
nj10:~ #

nj10:~ # cat nj13bash.txt
lxc-execute 1264193928.289 INFO lxc_conf - tty's configured
lxc-execute 1264193928.289 DEBUG lxc_start - sigchild handler set
lxc-execute 1264193928.289 INFO lxc_start - 'nj13' is initialized
lxc-execute 1264193928.304 DEBUG lxc_conf - instanciated veth
'vethyTNObU/vethKfJ6Ok', index is '122'
lxc-execute 1264193928.326 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-execute 1264193928.327 DEBUG lxc_cgroup - '/cgroup/24454'
renamed to '/cgroup/nj13'
lxc-execute 1264193928.354 DEBUG lxc_conf - move 'br1' to '24454'
lxc-execute 1264193928.354 INFO lxc_conf - 'nj13' hostname has
been setup
lxc-execute 1264193928.362 DEBUG lxc_conf - mac address
'00:16:3e:00:01:4d' on 'eth0' has been setup
lxc-execute 1264193928.378 DEBUG lxc_conf - 'eth0' has been setup
lxc-execute 1264193928.378 INFO lxc_conf - network has been setup
lxc-execute 1264193928.378 DEBUG lxc_conf - mounted none on
/lxc/nj13/dev/pts, type devpts
lxc-execute 1264193928.378 DEBUG lxc_conf - mounted none on
/lxc/nj13/proc, type proc
lxc-execute 1264193928.379 DEBUG lxc_conf - mounted none on
/lxc/nj13/sys, type sysfs
lxc-execute 1264193928.379 DEBUG lxc_conf - mounted none on
/lxc/nj13/dev/shm, type tmpfs
lxc-execute 1264193928.379 INFO lxc_conf - mount points have
been setup
lxc-execute 1264193928.379 INFO lxc_conf - mount points have
been setup
lxc-execute 1264193928.379 INFO lxc_conf - console '/dev/pts/0'
mounted to '/lxc/nj13/dev/console'
lxc-execute 1264193928.379 WARN lxc_conf - failed to mount
'/dev/pts/4'->'/lxc/nj13/dev/tty1'
lxc-execute 1264193928.379 INFO lxc_conf - 1 tty(s) has been setup
lxc-execute 1264193928.379 DEBUG lxc_conf - temporary mountpoint
for old rootfs is './lxc-oldrootfs-356AjD'
lxc-execute 1264193928.379 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-356AjD' successful
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/dev/pts'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/proc'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/sys/kernel/debug'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/sys/fs/fuse/connections'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/sys/kernel/security'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/boot'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/cgroup'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/lxc/nj13/proc'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/lxc/nj13/dev/pts'
lxc-execute 1264193928.379 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/lxc/nj13/sys'
lxc-execute 1264193928.380 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/lxc/nj13/dev/shm'
lxc-execute 1264193928.380 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/lxc/nj13/dev/console'
lxc-execute 1264193928.380 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/dev'
lxc-execute 1264193928.380 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-356AjD/sys'
lxc-execute 1264193928.380 ERROR lxc_conf - Device or resource
busy - could not unmount old rootfs
lxc-execute 1264193928.380 ERROR lxc_conf - failed to pivot_root
to '/lxc/nj13'
lxc-execute 1264193928.380 ERROR lxc_conf - failed to set rootfs
for 'nj13'
lxc-execute 1264193928.380 ERROR lxc_start - failed to setup the
container
lxc-execute 1264193928.380 NOTICE lxc_start -
'/usr/lib/lxc/lxc-init' started with pid '24454'
lxc-execute 1264193928.380 DEBUG lxc_utils - closing fd '1'
lxc-execute 1264193928.380 DEBUG lxc_utils - closing fd '0'
lxc-execute 1264193928.380 DEBUG lxc_utils - closed all
inherited file descriptors
lxc-execute 1264193928.396 DEBUG lxc_start - child exited
lxc-execute 1264193928.396 INFO lxc_error - child <24454> ended
on error (255)
lxc-execute 1264193928.396 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-execute 1264193928.414 DEBUG lxc_cgroup - '/cgroup/nj13'
unlinked
nj10:~ #

...Yet I can still start or create+execute nj12 just fine
This was all still with 0.6.5 btw. Now I'll try downgrading.
--
bkw
Daniel Lezcano
2010-01-23 21:44:05 UTC
Permalink
Post by Brian K. White
However, now when I go to make a 2nd container, I can't start it.
I can create it, but not execute or start.
Well I'm more boggled now.
I stopped my first container nj12.
lxc-ls shows nothing, screen -ls shows nothing, mount shows nothing
extra, yet trying to start nj13 still fails, and trying to start nj12
still succeeds.
I can't find anything functionally different between nj12 and nj13...
What could I be missing???
Yep, there is a problem with the pivot root and the umount of the
different mount point in the old rootfs.
This problem appears with some configuration (I didn't figure out which
one yet), I did a hot fix, which is more a workaround than a real fix,
(I didnt't understand where is coming the real problem).

As soon as I find the culprit, I will release a 0.6.6 version to fix
that as the 0.6.5 is bogus.

In the meantime, if you wish to test, I attached the patch to this email.

Thanks for reporting the problem.

-- Daniel
-------------- next part --------------
A non-text attachment was scrubbed...
Name: force-umount-rootfs.patch
Type: text/x-diff
Size: 2103 bytes
Desc: not available
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20100123/a430285c/attachment.patch>
Brian K. White
2010-01-24 18:27:15 UTC
Permalink
Post by Daniel Lezcano
Post by Brian K. White
However, now when I go to make a 2nd container, I can't start it.
I can create it, but not execute or start.
Well I'm more boggled now.
I stopped my first container nj12.
lxc-ls shows nothing, screen -ls shows nothing, mount shows nothing
extra, yet trying to start nj13 still fails, and trying to start nj12
still succeeds.
I can't find anything functionally different between nj12 and nj13...
What could I be missing???
Yep, there is a problem with the pivot root and the umount of the
different mount point in the old rootfs.
This problem appears with some configuration (I didn't figure out which
one yet), I did a hot fix, which is more a workaround than a real fix,
(I didnt't understand where is coming the real problem).
As soon as I find the culprit, I will release a 0.6.6 version to fix
that as the 0.6.5 is bogus.
In the meantime, if you wish to test, I attached the patch to this email.
Thanks for reporting the problem.
-- Daniel
Thanks. Seems to be working. Both containers start up , run , shutdown
ok. I'm still working on my install/setup procedures so I don't have a
3rd or more containers to test yet.

Downgrading to 0.6.3 worked also. I didn't try 0.6.4, no particular
reason, just 0.6.3 is what comes stock with my OS.

I'm running 0.6.5 for now as long as it works.

Thanks much!
--
bkw
Michael H. Warfield
2010-01-24 19:48:55 UTC
Permalink
Post by Brian K. White
Post by Daniel Lezcano
Post by Brian K. White
However, now when I go to make a 2nd container, I can't start it.
I can create it, but not execute or start.
Well I'm more boggled now.
I stopped my first container nj12.
lxc-ls shows nothing, screen -ls shows nothing, mount shows nothing
extra, yet trying to start nj13 still fails, and trying to start nj12
still succeeds.
I can't find anything functionally different between nj12 and nj13...
What could I be missing???
Yep, there is a problem with the pivot root and the umount of the
different mount point in the old rootfs.
This problem appears with some configuration (I didn't figure out which
one yet), I did a hot fix, which is more a workaround than a real fix,
(I didnt't understand where is coming the real problem).
As soon as I find the culprit, I will release a 0.6.6 version to fix
that as the 0.6.5 is bogus.
In the meantime, if you wish to test, I attached the patch to this email.
Thanks for reporting the problem.
-- Daniel
Thanks. Seems to be working. Both containers start up , run , shutdown
ok. I'm still working on my install/setup procedures so I don't have a
3rd or more containers to test yet.
Downgrading to 0.6.3 worked also. I didn't try 0.6.4, no particular
reason, just 0.6.3 is what comes stock with my OS.
Oh, 0.6.4 will "work", for some value of work, just like 0.6.3 does.
Neither of them contained the problematical pivot root operation at all.
But... They have other problems for which the introduction of the pivot
root was required. They will have bogus mount points in
their /proc/mounts table and there is a security problem where a guest
could break out of the chrooted jail. Both of these are fixed in 0.6.5
but it's that fix that's causing this problem.

It's very interesting that you ran into this on a second container. I
ran into it on a system with a large number of containers (>30) but only
if I had tmpfs mounted on /dev/shm in the host. In the process of
debugging it, I moved all but 4 of those containers to other hosts and
then strangely discovered I could no longer reproduce the problem with
or without /dev/shm mounted. I have one other way in which I can
reproduce the problem and that's where I have the rootfs mounted through
a bind mount from another directory AND the rootfs is also listed in the
containers fstab for the same ordered pair. That was an accidental
mistake I made in migrating a container (I've been testing both methods
as a compatibility / migration mode from OpenVZ to LXC). Either method
alone works (and I'm settling in on using the later instead of the
former, since the former clutters the hosts mount tables horribly) but
together they reproduce this exact same problem. I'm going to retest
with 0.6.5 both with and without the patch on that improper
configuration and see what happens. 0.6.5 actually contained fixes for
that problem under a very good explanation about WHY it happens that
explains both of my failure conditions but then it didn't seem to fix
all the cases.
Post by Brian K. White
I'm running 0.6.5 for now as long as it works.
Thanks much!
--
Michael H. Warfield (AI4NB) | (770) 985-6132 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20100124/12f1e721/attachment.pgp>
John Drescher
2010-01-23 02:57:57 UTC
Permalink
Post by Daniel Lezcano
Hi John,
I cooked a patch, (in attachment). I am hesitating to commit it in the
mainstream because I don't think it's the right way to fix it but it should
fix the problem temporarily for you ... well I hope :)
Success. :)

localhost ssh # lxc-start -n guest_1 -l DEBUG -o $(tty)
lxc-start 1264197385.322 INFO lxc_conf - tty's configured
lxc-start 1264197385.323 DEBUG lxc_start - sigchild handler set
lxc-start 1264197385.323 INFO lxc_start - 'guest_1' is initialized
lxc-start 1264197385.325 DEBUG lxc_conf - instanciated veth
'vethCmD5zS/veth066Rdf', index is '13'
lxc-start 1264197385.386 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264197385.386 DEBUG lxc_cgroup - '/cgroup/21804'
renamed to '/cgroup/guest_1'
lxc-start 1264197385.453 DEBUG lxc_conf - move 'br0' to '21804'
lxc-start 1264197385.453 INFO lxc_conf - 'guest_1' hostname
has been setup
lxc-start 1264197385.455 DEBUG lxc_conf - 'veth0' has been setup
lxc-start 1264197385.455 INFO lxc_conf - network has been setup
lxc-start 1264197385.456 DEBUG lxc_cgroup - using cgroup
mounted at '/cgroup'
lxc-start 1264197385.456 DEBUG lxc_conf - cgroup
'devices.deny' set to 'a # Deny all access to devices'
lxc-start 1264197385.456 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:3 rwm # dev/null'
lxc-start 1264197385.456 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:5 rwm # dev/zero'
lxc-start 1264197385.456 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:1 rwm # dev/console'
lxc-start 1264197385.456 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:0 rwm # dev/tty'
lxc-start 1264197385.456 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:0 rwm # dev/tty0'
lxc-start 1264197385.456 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:1 rwm # dev/tty1'
lxc-start 1264197385.456 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 4:2 rwm # dev/tty2'
lxc-start 1264197385.457 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:9 rwm # dev/urandon'
lxc-start 1264197385.457 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 1:8 rwm # dev/random'
lxc-start 1264197385.457 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 136:* rwm # dev/pts/*'
lxc-start 1264197385.457 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 5:2 rwm # dev/pts/ptmx'
lxc-start 1264197385.457 DEBUG lxc_conf - cgroup
'devices.allow' set to 'c 254:0 rwm'
lxc-start 1264197385.457 INFO lxc_conf - cgroup has been setup
lxc-start 1264197385.457 INFO lxc_conf - mount points have been setup
lxc-start 1264197385.457 INFO lxc_conf - mount points have been setup
lxc-start 1264197385.457 INFO lxc_conf - console
'/dev/pts/2' mounted to '/lxc/guest_1/dev/console'
lxc-start 1264197385.457 INFO lxc_conf - 4 tty(s) has been setup
lxc-start 1264197385.457 DEBUG lxc_conf - temporary
mountpoint for old rootfs is './lxc-oldrootfs-M7Lqrm'
lxc-start 1264197385.457 DEBUG lxc_conf - pivot_root syscall
to '/lxc-oldrootfs-M7Lqrm' successful
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/proc/bus/usb'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/lib64/rc/init.d'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/dev/pts'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/sys'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/dev'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/cgroup'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/lxc/guest_1/dev/console'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/lxc/guest_1/dev/tty1'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/lxc/guest_1/dev/tty2'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/lxc/guest_1/dev/tty3'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/lxc/guest_1/dev/tty4'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm/proc'
lxc-start 1264197385.458 DEBUG lxc_conf - umounted
'/lxc-oldrootfs-M7Lqrm'
lxc-start 1264197385.458 INFO lxc_conf - pivoted to '/tmp/lxc-rjB8QmZ'
lxc-start 1264197385.458 INFO lxc_conf - created new pts instance
lxc-start 1264197385.458 DEBUG lxc_conf - capabilities has been setup
lxc-start 1264197385.458 NOTICE lxc_conf - 'guest_1' is setup.
lxc-start 1264197385.458 NOTICE lxc_start - exec'ing '/sbin/init'
lxc-start 1264197385.459 NOTICE lxc_start - '/sbin/init'
started with pid '21804'
lxc-start 1264197385.459 DEBUG lxc_utils - closing fd '1'
lxc-start 1264197385.459 DEBUG lxc_utils - closing fd '0'
lxc-start 1264197385.459 DEBUG lxc_utils - closed all
inherited file descriptors
INIT: version 2.87 booting
Loading /lib64/rc/console/keymap

OpenRC 0.6.0 is starting up Gentoo Linux (x86_64)

* Mounting /proc ... [ ok ]
* Setting hostname to gentoo-lxc-guest ... [ ok ]
* Configuring kernel parameters ... [ ok ]
* Creating user login records ... [ ok ]
* Cleaning /var/run ... [ ok ]
* Wiping /tmp directory ... [ ok ]
* Setting terminal encoding [UTF-8] ...
/etc/init.d/termencoding: line 31: /dev/tty5: Operation not permitted
/etc/init.d/termencoding: line 31: /dev/tty6: Operation not permitted
/etc/init.d/termencoding: line 31: /dev/tty7: Operation not permitted
/etc/init.d/termencoding: line 31: /dev/tty8: Operation not permitted
/etc/init.d/termencoding: line 31: /dev/tty9: Operation not permitted
/etc/init.d/termencoding: line 31: /dev/tty10: Operation not permitted
/etc/init.d/termencoding: line 31: /dev/tty11: Operation not permitted
/etc/init.d/termencoding: line 31: /dev/tty12: Operation not permitted [ ok ]
* Setting keyboard mode [UTF-8] ...
Couldn't open /dev/tty1
Couldn't open /dev/tty2
Couldn't open /dev/tty3
Couldn't open /dev/tty4
Couldn't open /dev/tty5
Couldn't open /dev/tty6
Couldn't open /dev/tty7
Couldn't open /dev/tty8
Couldn't open /dev/tty9
Couldn't open /dev/tty10
Couldn't open /dev/tty11
Couldn't open /dev/tty12 [ ok ]
* Loading key mappings [us] ... [ ok ]
* Starting network [ ok ]
* Adding static routes ... [ ok ]
* Initializing random number generator ... [ ok ]
INIT: Entering runlevel: 3
* Mounting network filesystems ... [ ok ]
* Starting syslog-ng ...
Jan 22 21:56:27 gentoo-lxc-guest syslog-ng[239]: syslog-ng starting
up; version='3.0.4'
[ ok ]
* Starting sshd ...
Jan 22 21:56:28 gentoo-lxc-guest sshd[253]: Server listening on 0.0.0.0 p [ ok ]
* Starting local ... [ ok ]


This is gentoo-lxc-guest.unknown_domain (Linux x86_64 2.6.32-gentoo) 21:56:34

gentoo-lxc-guest login:
Continue reading on narkive:
Loading...