Discussion:
using cgroups
(too old to reply)
Mike Wright
2016-06-26 18:08:35 UTC
Permalink
Hi all,

cgmanager and cgmanager-utils are installed.

Environment is ubuntu-xenial, lxc-2.0.1, cgm-0.29

Trying to explore cgroups using:

http://linuxcontainers/cgmanager/getting-started.

"sudo cgm create all me" no errors;
"sudo cgm chown all me $(id -u) $(id -g)" errors with:

"call to cgmanager_chown_sync failed: invalid request"

Not making it very far here :)

Any ideas what is lacking?

Thanks,
Mike Wright
Serge E. Hallyn
2016-06-26 20:01:33 UTC
Permalink
Post by Mike Wright
Hi all,
cgmanager and cgmanager-utils are installed.
Environment is ubuntu-xenial, lxc-2.0.1, cgm-0.29
why 0.29? xenial should have 0.39-2ubuntu5. I'm on xenial
using 0.41-2~ubuntu16.04.1~ppa1 from the ubuntu-lxc
ppa.

0 ✓ ***@sl ~ $ sudo cgm create all me
[sudo] password for serge:
0 ✓ ***@sl ~ $ sudo cgm chown all me $(id -u) $(id -g)
0 ✓ ***@sl ~ $

Now, I'm not running systemd so it's possible systemd is
doing something unorthodox again. But really it sounds
like a bug that shouldve been fixed in 0.27-0ubuntu6 -
where cgmanager didn't deal well with comounted controllers.
Post by Mike Wright
http://linuxcontainers/cgmanager/getting-started.
"sudo cgm create all me" no errors;
"call to cgmanager_chown_sync failed: invalid request"
Not making it very far here :)
Any ideas what is lacking?
Thanks,
Mike Wright
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Mike Wright
2016-06-26 21:51:24 UTC
Permalink
Post by Serge E. Hallyn
Post by Mike Wright
Hi all,
cgmanager and cgmanager-utils are installed.
Environment is ubuntu-xenial, lxc-2.0.1, cgm-0.29
why 0.29? xenial should have 0.39-2ubuntu5. I'm on xenial
using 0.41-2~ubuntu16.04.1~ppa1 from the ubuntu-lxc
ppa.
Thanks for the response, Serge.

This is interesting.

sudo apt install -s cgmanager
cgmanager is already the newest version (0.39-2ubuntu5)

cgm --version
0.29

Added ppa:ubuntu-lxc/stable, updated and upgraded.

sudo apt install -s cgmanager
cgmanager is already the newest version (0.41-2~ubuntu16.04.1~ppa1)

cgm --version
0.29
Post by Serge E. Hallyn
Now, I'm not running systemd so it's possible systemd is
doing something unorthodox again. But really it sounds
like a bug that shouldve been fixed in 0.27-0ubuntu6 -
where cgmanager didn't deal well with comounted controllers.
Still failing at cgm chown...

Ideas on how would I go about determining the problem?
Post by Serge E. Hallyn
Post by Mike Wright
http://linuxcontainers/cgmanager/getting-started.
"sudo cgm create all me" no errors;
"call to cgmanager_chown_sync failed: invalid request"
Not making it very far here :)
Any ideas what is lacking?
Thanks,
Mike Wright
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Serge E. Hallyn
2016-06-27 01:41:36 UTC
Permalink
Post by Mike Wright
Post by Serge E. Hallyn
Post by Mike Wright
Hi all,
cgmanager and cgmanager-utils are installed.
Environment is ubuntu-xenial, lxc-2.0.1, cgm-0.29
why 0.29? xenial should have 0.39-2ubuntu5. I'm on xenial
using 0.41-2~ubuntu16.04.1~ppa1 from the ubuntu-lxc
ppa.
Thanks for the response, Serge.
This is interesting.
sudo apt install -s cgmanager
cgmanager is already the newest version (0.39-2ubuntu5)
cgm --version
0.29
Added ppa:ubuntu-lxc/stable, updated and upgraded.
sudo apt install -s cgmanager
cgmanager is already the newest version (0.41-2~ubuntu16.04.1~ppa1)
cgm --version
0.29
Oh, huh. Yeah, that seems to be a cgmanager bug :)
Post by Mike Wright
Post by Serge E. Hallyn
Now, I'm not running systemd so it's possible systemd is
doing something unorthodox again. But really it sounds
like a bug that shouldve been fixed in 0.27-0ubuntu6 -
where cgmanager didn't deal well with comounted controllers.
Still failing at cgm chown...
Ideas on how would I go about determining the problem?
Edit /lib/systemd/system/cgmanager.service and add '--debug' to the
end of the ExecStart line. Do 'systemctl daemon-reload' followed
by 'systemctl restart cgmanager'. Then do the above again, and
do 'journalctl -u cgmanager' and list the results here. Also
show the contents of /proc/self/cgroup and /proc/self/mountinfo.
That should give us what we need.

thanks,
-serge
Mike Wright
2016-06-27 02:52:09 UTC
Permalink
Post by Serge E. Hallyn
Post by Mike Wright
Ideas on how would I go about determining the problem?
Edit /lib/systemd/system/cgmanager.service and add '--debug' to the
end of the ExecStart line. Do 'systemctl daemon-reload' followed
by 'systemctl restart cgmanager'. Then do the above again, and
do 'journalctl -u cgmanager' and list the results here. Also
show the contents of /proc/self/cgroup and /proc/self/mountinfo.
That should give us what we need.
Following clean boot; no cli cgm commands given.

Attached is journalctl -u cgmanager, /proc/self/{cgroup,mountinfo}
rob e
2016-06-28 11:23:17 UTC
Permalink
hi,
I'm experiencing the same problem. I use "lxc.cgroup" to constrain
resource usage and to provide access to devices

in trying to re-use containers established under Trusty, I find that
lxc.cgroup clauses prevent the container starting

furthermore, if I create a new "test" container on Xenial, it will start
and run ok until I start adding lxc.cgroup clauses, at which point it
will no longer start.

LXC is installed. LXD is NOT installed. CGMANAGER is installed. All
packages are current from Xenial LTS

Is there anything I can do to help pin this down ? Will a test conducted
in a KVM based VM be valid / useful ?

Rob
Post by Serge E. Hallyn
Post by Mike Wright
Post by Serge E. Hallyn
Post by Mike Wright
Hi all,
cgmanager and cgmanager-utils are installed.
Environment is ubuntu-xenial, lxc-2.0.1, cgm-0.29
why 0.29? xenial should have 0.39-2ubuntu5. I'm on xenial
using 0.41-2~ubuntu16.04.1~ppa1 from the ubuntu-lxc
ppa.
Thanks for the response, Serge.
This is interesting.
sudo apt install -s cgmanager
cgmanager is already the newest version (0.39-2ubuntu5)
cgm --version
0.29
Added ppa:ubuntu-lxc/stable, updated and upgraded.
sudo apt install -s cgmanager
cgmanager is already the newest version (0.41-2~ubuntu16.04.1~ppa1)
cgm --version
0.29
Oh, huh. Yeah, that seems to be a cgmanager bug :)
Post by Mike Wright
Post by Serge E. Hallyn
Now, I'm not running systemd so it's possible systemd is
doing something unorthodox again. But really it sounds
like a bug that shouldve been fixed in 0.27-0ubuntu6 -
where cgmanager didn't deal well with comounted controllers.
Still failing at cgm chown...
Ideas on how would I go about determining the problem?
Edit /lib/systemd/system/cgmanager.service and add '--debug' to the
end of the ExecStart line. Do 'systemctl daemon-reload' followed
by 'systemctl restart cgmanager'. Then do the above again, and
do 'journalctl -u cgmanager' and list the results here. Also
show the contents of /proc/self/cgroup and /proc/self/mountinfo.
That should give us what we need.
thanks,
-serge
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Benoit GEORGELIN - Association Web4all
2016-06-28 13:32:08 UTC
Permalink
Hi,

For once, simple question :)

Do you have experienced any "disk quota exceeded" with a container using BTRFS as a backing storage ?

LXD using btrfs
Subvolume with quota applied to 10Go
Container can't do any write after 24h uptime and less than 2Go used
If I restart the container, looks like it's ok for the next 24 hours or so.

Other containers don't look to have the same issue , but I did delete this one and create a new one and still the same issue

Merci

Cordialement,
Benoît
Mike Wright
2016-06-28 14:22:22 UTC
Permalink
Post by Benoit GEORGELIN - Association Web4all
Hi,
For once, simple question :)
Hello Benoit,

Please don't hijack threads. (That's where you take an existing thread
and change the subject). It messes up the message flow.

It's the same thing as people having a conversation and somebody walks
up and changes the topic. Many consider it impolite.
Benoit GEORGELIN - Association Web4all
2016-06-28 14:56:48 UTC
Permalink
Hello,
Sorry I did not know the system was based on header informations so I'm use to open an existing email in the list, remove the content , change the subject and send it.
I apologize this was not on purpose. Next time I'll start a new email.

Thanks for your consideration about that.

Cordialement,




De: "Mike Wright" <***@nospam.hostisimo.com>
À: "lxc-users" <lxc-***@lists.linuxcontainers.org>
Envoyé: Mardi 28 Juin 2016 10:22:22
Objet: Re: [lxc-users] btrfs and LXC/LXD - disk quota exceeded
Post by Benoit GEORGELIN - Association Web4all
Hi,
For once, simple question :)
Hello Benoit,

Please don't hijack threads. (That's where you take an existing thread
and change the subject). It messes up the message flow.

It's the same thing as people having a conversation and somebody walks
up and changes the topic. Many consider it impolite.

_______________________________________________
lxc-users mailing list
lxc-***@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Rob Edgerton
2016-06-28 06:01:34 UTC
Permalink
hi,I have the same problem (cgroups not working as expected) on a clean Xenial build (lxc PPA NOT installed, LXD not installed)In my case I have some Ubuntu Trusty containers I really need to use on Xenial, but they won't start because I use cgroups.If I change the existing containers to remove the "lxc.cgroup" clauses from config they start, but not otherwise.Similarly, I created a new Xenial container for testing. It works, until I add "lxc.cgroups" clauses at which point it also fails to ***@virt-host:~$ lxc-start -n trusty_unp_ibvpn -F -l debug -o lxc.log
lxc-start: cgfsng.c: cgfsng_setup_limits: 1662 No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
lxc-start: start.c: lxc_spawn: 1180 failed to setup the cgroup limits for 'trusty_unp_ibvpn'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'trusty_unp_ibvpn'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority  options.

Logfile Contents=============
      lxc-start 20160628155820.562 INFO     lxc_start_ui - lxc_start.c:main:264 - using rcfile /mnt/lxc_images/containers/trusty_unp_ibvpn/config
      lxc-start 20160628155820.562 WARN     lxc_confile - confile.c:config_pivotdir:1879 - lxc.pivotdir is ignored.  It will soon become an error.
      lxc-start 20160628155820.562 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 100000 range 65536
      lxc-start 20160628155820.562 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 100000 range 65536
      lxc-start 20160628155820.564 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .reject_force_umount  # comment this to allow umount -f;  not recommended.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for reject_force_umount action 0
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts

      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for reject_force_umount action 0
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts

      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .[all].
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .kexec_load errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for kexec_load action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for kexec_load action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .open_by_handle_at errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for open_by_handle_at action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for open_by_handle_at action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .init_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for init_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for init_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .finit_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for finit_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for finit_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .delete_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for delete_module action 327681
      lxc-start 20160628155820.565 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for delete_module action 327681
      lxc-start 20160628155820.565 INFO     lxc_seccomp - seccomp.c:parse_config_v2:456 - Merging in the compat seccomp ctx into the main one
      lxc-start 20160628155820.565 DEBUG    lxc_start - start.c:setup_signal_fd:289 - sigchild handler set
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_peer_default:431 - opening /dev/tty for console peer
      lxc-start 20160628155820.565 INFO     lxc_caps - caps.c:lxc_caps_up:101 - Last supported cap was 36
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_peer_default:437 - using '/dev/tty' as console
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_sigwinch_init:145 - 3234 got SIGWINCH fd 9
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_winsz:72 - set winsz dstfd:6 cols:212 rows:73
      lxc-start 20160628155820.611 INFO     lxc_start - start.c:lxc_init:488 - 'trusty_unp_ibvpn' is initialized
      lxc-start 20160628155820.611 DEBUG    lxc_start - start.c:__lxc_start:1326 - Not dropping cap_sys_boot or watching utmp
      lxc-start 20160628155820.611 INFO     lxc_start - start.c:resolve_clone_flags:1013 - Cloning a new user namespace
      lxc-start 20160628155820.611 INFO     lxc_cgroup - cgroup.c:cgroup_init:68 - cgroup driver cgroupfs-ng initing for trusty_unp_ibvpn
      lxc-start 20160628155820.614 DEBUG    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1667 - cgroup 'devices.allow' set to 'c 10:200 rwm'
      lxc-start 20160628155820.614 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1662 - No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
      lxc-start 20160628155820.615 ERROR    lxc_start - start.c:lxc_spawn:1180 - failed to setup the cgroup limits for 'trusty_unp_ibvpn'
      lxc-start 20160628155820.615 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'trusty_unp_ibvpn'
      lxc-start 20160628155820.659 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'trusty_unp_ibvpn', config section 'lxc'
      lxc-start 20160628155821.172 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 20160628155821.172 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
  
Repeating the commands you were discussing with Mike

cgmanager is already the newest version (0.39-2ubuntu5).
@virt-host:~$cgm --version
0.29

@virt-host:~$ls /proc/self/cgroup
/proc/self/cgroup

@virt-host:~$ls /proc/self/mountinfo
/proc/self/mountinfo

@virt-host:~$ sudo nano /lib/systemd/system/cgmanager.service
@virt-host:~$ sudo systemctl daemon-reload
@virt-host:~$
@virt-host:~$ systemctl restart cgmanager
@virt-host:~$
@virt-host:~$ sudo cgm create all me
@virt-host:~$ sudo cgm chown all me $(id -u) $(id -g)
call to cgmanager_chown_sync failed: invalid request
-- Logs begin at Tue 2016-06-28 15:08:37 AEST, end at Tue 2016-06-28 15:44:23 AEST. --
Jun 28 15:08:40 virt-host systemd[1]: Started Cgroup management daemon.
Jun 28 15:40:14 virt-host systemd[1]: Stopping Cgroup management daemon...
Jun 28 15:40:14 virt-host systemd[1]: Stopped Cgroup management daemon.
Jun 28 15:40:14 virt-host systemd[1]: Started Cgroup management daemon.
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount systemd onto /run/cgmanager/fs/none,name=systemd
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount memory onto /run/cgmanager/fs/memory
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount net_cls onto /run/cgmanager/fs/net_cls
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount net_prio onto /run/cgmanager/fs/net_prio
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount cpu onto /run/cgmanager/fs/cpu
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount cpuacct onto /run/cgmanager/fs/cpuacct
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount cpuset onto /run/cgmanager/fs/cpuset
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount blkio onto /run/cgmanager/fs/blkio
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount devices onto /run/cgmanager/fs/devices
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount freezer onto /run/cgmanager/fs/freezer
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount perf_event onto /run/cgmanager/fs/perf_event
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount hugetlb onto /run/cgmanager/fs/hugetlb
Jun 28 15:40:14 virt-host cgmanager[2990]: Arranged to mount pids onto /run/cgmanager/fs/pids
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: all unique controllers: blkio,cpu,cpuset,devices,freezer,hugetlb,memory,net_cls,perf_event,pids,name=systemd
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 0: controller blkio
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src blkio path /run/cgmanager/fs/blkio options blkio
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: yes comounted: blkio
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     unified: no
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 1: controller cpu
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src cpu path /run/cgmanager/fs/cpu options cpu,cpuacct
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: yes comounted: cpuacct                                                                                                     
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     unified: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 2: controller cpuacct                                                                                                                      
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src cpuacct path /run/cgmanager/fs/cpuacct options cpu,cpuacct                                                                         
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)                                                                                                                          
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: yes comounted: cpu                                                                                                         
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     unified: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 3: controller cpuset                                                                                                                       
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src cpuset path /run/cgmanager/fs/cpuset options cpuset                                                                                
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)                                                                                                                          
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: yes comounted: cpuset                                                                                                      
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     unified: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 4: controller devices                                                                                                                      
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src devices path /run/cgmanager/fs/devices options devices                                                                             
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)                                                                                                                          
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: yes comounted: devices                                                                                                     
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     unified: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 5: controller freezer                                                                                                                      
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src freezer path /run/cgmanager/fs/freezer options freezer                                                                             
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)                                                                                                                          
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: yes comounted: freezer                                                                                                     
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     unified: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 6: controller hugetlb                                                                                                                      
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src hugetlb path /run/cgmanager/fs/hugetlb options (none)                                                                              
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)                                                                                                                          
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: no comounted: (none)                                                                                                       
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     unified: no                                                                                                                            
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 7: controller memory                                                                                                                       
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src memory path /run/cgmanager/fs/memory options memory
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: yes comounted: memory
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     unified: no
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info: 8: controller net_cls
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     src net_cls path /run/cgmanager/fs/net_cls options net_cls,net_prio
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     agent: (none)
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     skipped: no
Jun 28 15:40:14 virt-host cgmanager[2990]: print_debug_controller_info:     premounted: yes comounted: net_prio
Jun 28 15:41:00 virt-host cgmanager[2990]: print_debug_controller_info:     src net_prio path /run/cgcgmanager: Invalid path /run/cgmanager/fs/freezer/user/root/1/me
Jun 28 15:41:00 virt-host cgmanager[2990]: cgmanager:do_chown_main: Invalid path /run/cgmanager/fs/freezer/user/root/1/me
@virt-host:~$ ls /proc/self/cgroup
/proc/self/cgroup
@virt-host:~$ ls /proc/self/mountinfo
/proc/self/mountinfo

Note that I did NOT upgrade to the "stable" PPA first, still using the standard Xenial package as originally installed (the build is only a few days old)
Does this help ? Anything else I can do ?
Rob
Post by Mike Wright
Post by Mike Wright
Hi all,
cgmanager and cgmanager-utils are installed.
Environment is ubuntu-xenial, lxc-2.0.1, cgm-0.29
why 0.29?  xenial should have 0.39-2ubuntu5.  I'm on xenial
using 0.41-2~ubuntu16.04.1~ppa1 from the ubuntu-lxc
ppa.
Thanks for the response, Serge.
This is interesting.
sudo apt install -s cgmanager
  cgmanager is already the newest version (0.39-2ubuntu5)
cgm --version
  0.29
Added ppa:ubuntu-lxc/stable, updated and upgraded.
sudo apt install -s cgmanager
  cgmanager is already the newest version (0.41-2~ubuntu16.04.1~ppa1)
cgm --version
  0.29
Oh, huh.  Yeah, that seems to be a cgmanager bug :)
Post by Mike Wright
Now, I'm not running systemd so it's possible systemd is
doing something unorthodox again.  But really it sounds
like a bug that shouldve been fixed in 0.27-0ubuntu6 -
where cgmanager didn't deal well with comounted controllers.
Still failing at cgm chown...
Ideas on how would I go about determining the problem?
Edit /lib/systemd/system/cgmanager.service and add '--debug' to the
end of the ExecStart line.  Do 'systemctl daemon-reload' followed
by 'systemctl restart cgmanager'.  Then do the above again, and
do 'journalctl -u cgmanager' and list the results here.  Also
show the contents of /proc/self/cgroup and /proc/self/mountinfo.
That should give us what we need.

thanks,
-serge
Serge E. Hallyn
2016-06-30 00:36:20 UTC
Permalink
Post by Rob Edgerton
lxc-start: cgfsng.c: cgfsng_setup_limits: 1662 No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
lxc-start: start.c: lxc_spawn: 1180 failed to setup the cgroup limits for 'trusty_unp_ibvpn'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'trusty_unp_ibvpn'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority  options.
Logfile Contents=============
      lxc-start 20160628155820.562 INFO     lxc_start_ui - lxc_start.c:main:264 - using rcfile /mnt/lxc_images/containers/trusty_unp_ibvpn/config
      lxc-start 20160628155820.562 WARN     lxc_confile - confile.c:config_pivotdir:1879 - lxc.pivotdir is ignored.  It will soon become an error.
      lxc-start 20160628155820.562 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 100000 range 65536
      lxc-start 20160628155820.562 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 100000 range 65536
      lxc-start 20160628155820.564 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .reject_force_umount  # comment this to allow umount -f;  not recommended.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for reject_force_umount action 0
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for reject_force_umount action 0
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .[all].
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .kexec_load errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for kexec_load action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for kexec_load action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .open_by_handle_at errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for open_by_handle_at action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for open_by_handle_at action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .init_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for init_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for init_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .finit_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for finit_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for finit_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .delete_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for delete_module action 327681
      lxc-start 20160628155820.565 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for delete_module action 327681
      lxc-start 20160628155820.565 INFO     lxc_seccomp - seccomp.c:parse_config_v2:456 - Merging in the compat seccomp ctx into the main one
      lxc-start 20160628155820.565 DEBUG    lxc_start - start.c:setup_signal_fd:289 - sigchild handler set
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_peer_default:431 - opening /dev/tty for console peer
      lxc-start 20160628155820.565 INFO     lxc_caps - caps.c:lxc_caps_up:101 - Last supported cap was 36
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_peer_default:437 - using '/dev/tty' as console
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_sigwinch_init:145 - 3234 got SIGWINCH fd 9
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_winsz:72 - set winsz dstfd:6 cols:212 rows:73
      lxc-start 20160628155820.611 INFO     lxc_start - start.c:lxc_init:488 - 'trusty_unp_ibvpn' is initialized
      lxc-start 20160628155820.611 DEBUG    lxc_start - start.c:__lxc_start:1326 - Not dropping cap_sys_boot or watching utmp
      lxc-start 20160628155820.611 INFO     lxc_start - start.c:resolve_clone_flags:1013 - Cloning a new user namespace
      lxc-start 20160628155820.611 INFO     lxc_cgroup - cgroup.c:cgroup_init:68 - cgroup driver cgroupfs-ng initing for trusty_unp_ibvpn
      lxc-start 20160628155820.614 DEBUG    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1667 - cgroup 'devices.allow' set to 'c 10:200 rwm'
      lxc-start 20160628155820.614 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1662 - No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
ENOENT - that's unexpected...
Post by Rob Edgerton
      lxc-start 20160628155820.615 ERROR    lxc_start - start.c:lxc_spawn:1180 - failed to setup the cgroup limits for 'trusty_unp_ibvpn'
      lxc-start 20160628155820.615 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'trusty_unp_ibvpn'
      lxc-start 20160628155820.659 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'trusty_unp_ibvpn', config section 'lxc'
      lxc-start 20160628155821.172 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 20160628155821.172 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
  
Repeating the commands you were discussing with Mike
cgmanager is already the newest version (0.39-2ubuntu5).
@virt-host:~$cgm --version
0.29
Can you show 'dpkg -l | grep cgmanager' ?

as well as cat /etc/*release
Post by Rob Edgerton
@virt-host:~$ls /proc/self/cgroup
/proc/self/cgroup
@virt-host:~$ls /proc/self/mountinfo
/proc/self/mountinfo
Hi,
For /proc/self/cgroup and /proc/self/mountinfo, we actually need to see
the contents. Can you show 'cat /proc/self/cgroup' and
'cat /proc/self/mountinfo'?

-serge
Rob
2016-06-30 01:24:25 UTC
Permalink
Post by Serge E. Hallyn
lxc-start 20160628155820.614 ERROR lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1662 - No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
ENOENT - that's unexpected...
0.29
Can you show 'dpkg -l | grep cgmanager' ?
as well as cat /etc/*release
Hi, For /proc/self/cgroup and /proc/self/mountinfo, we actually need
to see the contents. Can you show 'cat /proc/self/cgroup' and 'cat
/proc/self/mountinfo'? -serge
_______________________________________________ lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
here is the follow up info (note that I cut the msg above in order to
reduce size)

$ dpkg -l | grep cgmanager
ii cgmanager 0.39-2ubuntu5 amd64
Central cgroup manager daemon
ii libcgmanager0:amd64 0.39-2ubuntu5
amd64 Central cgroup manager daemon (client library)

$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"
NAME="Ubuntu"
VERSION="16.04 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial


$ cat /proc/self/cgroup
11:blkio:/user.slice
10:hugetlb:/
9:freezer:/user/redger/1
8:pids:/user.slice/user-1000.slice
7:perf_event:/
6:cpu,cpuacct:/user.slice
5:net_cls,net_prio:/
4:devices:/user.slice
3:memory:/user/redger/1
2:cpuset:/
1:name=systemd:/user.slice/user-1000.slice/session-1.scope

$ cat /proc/self/mountinfo
19 25 0:18 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
20 25 0:4 / /proc rw,nosuid,nodev,noexec,relatime shared:12 - proc proc rw
21 25 0:6 / /dev rw,nosuid,relatime shared:2 - devtmpfs udev
rw,size=8026104k,nr_inodes=2006526,mode=755
22 21 0:14 / /dev/pts rw,nosuid,noexec,relatime shared:3 - devpts devpts
rw,gid=5,mode=620,ptmxmode=000
23 25 0:19 / /run rw,nosuid,noexec,relatime shared:5 - tmpfs tmpfs
rw,size=1615856k,mode=755
25 0 8:41 / / rw,relatime shared:1 - ext4 /dev/sdc9
rw,errors=remount-ro,data=ordered
26 19 0:12 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime
shared:8 - securityfs securityfs rw
27 21 0:21 / /dev/shm rw,nosuid,nodev shared:4 - tmpfs tmpfs rw
28 23 0:22 / /run/lock rw,nosuid,nodev,noexec,relatime shared:6 - tmpfs
tmpfs rw,size=5120k
29 19 0:23 / /sys/fs/cgroup rw shared:9 - tmpfs tmpfs rw,mode=755
30 29 0:24 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime
shared:10 - cgroup cgroup
rw,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
31 19 0:25 / /sys/fs/pstore rw,nosuid,nodev,noexec,relatime shared:11 -
pstore pstore rw
32 29 0:26 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime
shared:13 - cgroup cgroup rw,cpuset,clone_children
33 29 0:27 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime
shared:14 - cgroup cgroup rw,memory
34 29 0:28 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime
shared:15 - cgroup cgroup rw,devices
35 29 0:29 / /sys/fs/cgroup/net_cls,net_prio
rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup
rw,net_cls,net_prio
36 29 0:30 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime
shared:17 - cgroup cgroup rw,cpu,cpuacct
37 29 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime
shared:18 - cgroup cgroup
rw,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event
38 29 0:32 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime
shared:19 - cgroup cgroup
rw,pids,release_agent=/run/cgmanager/agents/cgm-release-agent.pids
39 29 0:33 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime
shared:20 - cgroup cgroup rw,freezer
40 29 0:34 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime
shared:21 - cgroup cgroup
rw,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb
41 29 0:35 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime
shared:22 - cgroup cgroup rw,blkio
42 20 0:36 / /proc/sys/fs/binfmt_misc rw,relatime shared:23 - autofs
systemd-1 rw,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct
43 19 0:7 / /sys/kernel/debug rw,relatime shared:24 - debugfs debugfs rw
44 21 0:37 / /dev/hugepages rw,relatime shared:25 - hugetlbfs hugetlbfs rw
45 23 0:38 / /run/rpc_pipefs rw,relatime shared:26 - rpc_pipefs sunrpc rw
46 21 0:17 / /dev/mqueue rw,relatime shared:27 - mqueue mqueue rw
47 20 0:39 / /proc/fs/nfsd rw,relatime shared:28 - nfsd nfsd rw
48 19 0:40 / /sys/fs/fuse/connections rw,relatime shared:29 - fusectl
fusectl rw
49 25 8:34 / /mnt/snd480_boot_01 rw,relatime shared:30 - ext4 /dev/sdc2
rw,data=ordered
50 25 8:35 / /mnt/snd480_root_01 rw,relatime shared:31 - ext4 /dev/sdc3
rw,data=ordered
51 25 8:42 / /home rw,relatime shared:32 - ext4 /dev/sdc10 rw,data=ordered
53 25 8:40 / /boot rw,relatime shared:33 - ext4 /dev/sdc8 rw,data=ordered
52 25 8:36 / /mnt/snd480_home_01 rw,relatime shared:34 - ext4 /dev/sdc4
rw,data=ordered
56 25 8:39 / /mnt/snd480_boot_03_wintemp rw,relatime shared:35 - vfat
/dev/sdc7
rw,gid=46,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro
54 25 8:51 / /mnt/wd2t_home_01 rw,relatime shared:36 - ext4 /dev/sdd3
rw,data=ordered
55 25 8:1 / /mnt/video rw,relatime shared:37 - ext4 /dev/sda1
rw,data=ordered
57 25 8:50 / /mnt/wd2t_root_01 rw,relatime shared:38 - ext4 /dev/sdd2
rw,data=ordered
58 25 8:17 / /mnt/dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1
rw,data=ordered
60 25 8:18 / /mnt/music rw,relatime shared:40 - ext4 /dev/sdb2
rw,data=ordered
59 25 8:49 / /mnt/wd2t_boot_01 rw,relatime shared:41 - ext4 /dev/sdd1
rw,data=ordered
61 25 252:4 / /mnt/lxc_images rw,noatime shared:42 - ext4
/dev/mapper/wd2t--lvm--data-lxc_images rw,data=ordered
63 25 0:41 / /var/lib/lxd rw,noatime shared:43 - btrfs
/dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/
64 61 252:4
/containers/utopic_browse_normal_backup_151115/rootfs/home/ubuntu/Downloads
/mnt/lxc_images/containers/trusty-mythserver/rootfs/mnt/lxc_container_normal_downloads
rw,noatime shared:42 - ext4 /dev/mapper/wd2t--lvm--data-lxc_images
rw,data=ordered
65 61 8:1 / /mnt/lxc_images/containers/browse_danger/rootfs/mnt/video
rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
62 25 252:0 / /mnt/programming_data rw,relatime shared:44 - ext4
/dev/mapper/wd2t--lvm--data-programming_data rw,data=ordered
66 61 8:17 /
/mnt/lxc_images/containers/browse_danger/rootfs/mnt/dvd_rips rw,relatime
shared:39 - ext4 /dev/sdb1 rw,data=ordered
67 61 8:1 /
/mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/video
rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
68 61 8:18 /
/mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/music
rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
69 61 8:18 / /mnt/lxc_images/containers/browse_danger/rootfs/mnt/music
rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
70 61 8:17 /
/mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/dvd_rips
rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
71 25 252:4
/containers/utopic_browse_normal_backup_151115/rootfs/home/ubuntu/Downloads
/mnt/lxc_container_normal_downloads rw,noatime shared:42 - ext4
/dev/mapper/wd2t--lvm--data-lxc_images rw,data=ordered
72 25 0:41 / /mnt/lxd_images rw,noatime shared:43 - btrfs
/dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/
142 23 0:45 / /run/cgmanager/fs rw,relatime shared:113 - tmpfs cgmfs
rw,size=100k,mode=755
146 23 0:47 / /run/lxcfs/controllers rw,relatime shared:115 - tmpfs
tmpfs rw,size=100k,mode=700
148 146 0:35 / /run/lxcfs/controllers/blkio rw,relatime shared:117 -
cgroup blkio rw,blkio
150 146 0:34 / /run/lxcfs/controllers/hugetlb rw,relatime shared:119 -
cgroup hugetlb
rw,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb
152 146 0:33 / /run/lxcfs/controllers/freezer rw,relatime shared:121 -
cgroup freezer rw,freezer
155 146 0:32 / /run/lxcfs/controllers/pids rw,relatime shared:123 -
cgroup pids
rw,pids,release_agent=/run/cgmanager/agents/cgm-release-agent.pids
157 146 0:31 / /run/lxcfs/controllers/perf_event rw,relatime shared:125
- cgroup perf_event
rw,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event
159 146 0:30 / /run/lxcfs/controllers/cpu,cpuacct rw,relatime shared:127
- cgroup cpu,cpuacct rw,cpu,cpuacct
161 146 0:29 / /run/lxcfs/controllers/net_cls,net_prio rw,relatime
shared:129 - cgroup net_cls,net_prio rw,net_cls,net_prio
163 146 0:28 / /run/lxcfs/controllers/devices rw,relatime shared:131 -
cgroup devices rw,devices
165 146 0:27 / /run/lxcfs/controllers/memory rw,relatime shared:133 -
cgroup memory rw,memory
167 146 0:26 / /run/lxcfs/controllers/cpuset rw,relatime shared:135 -
cgroup cpuset rw,cpuset,clone_children
169 146 0:24 / /run/lxcfs/controllers/name=systemd rw,relatime
shared:137 - cgroup name=systemd
rw,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
171 25 0:48 / /var/lib/lxcfs rw,nosuid,nodev,relatime shared:139 -
fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
176 63 0:41 /shmounts /var/lib/lxd/shmounts rw,noatime shared:43 - btrfs
/dev/mapper/wd2t--lvm--data-lxd_images
rw,space_cache,subvolid=5,subvol=/shmounts
177 72 0:41 /shmounts /mnt/lxd_images/shmounts rw,noatime shared:43 -
btrfs /dev/mapper/wd2t--lvm--data-lxd_images
rw,space_cache,subvolid=5,subvol=/shmounts
180 23 0:50 / /run/user/1000 rw,nosuid,nodev,relatime shared:143 - tmpfs
tmpfs rw,size=1615856k,mode=700,uid=1000,gid=1000
292 42 0:55 / /proc/sys/fs/binfmt_misc rw,relatime shared:151 -
binfmt_misc binfmt_misc rw
182 63 8:1 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-video
rw,relatime master:37 - ext4 /dev/sda1 rw,data=ordered
183 72 8:1 / /mnt/lxd_images/devices/xenial-mythserver/disk.mnt-video
rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
188 63 8:18 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-music
rw,relatime master:40 - ext4 /dev/sdb2 rw,data=ordered
189 72 8:18 / /mnt/lxd_images/devices/xenial-mythserver/disk.mnt-music
rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
194 63 8:17 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-dvd_rips
rw,relatime master:39 - ext4 /dev/sdb1 rw,data=ordered
195 72 8:17 /
/mnt/lxd_images/devices/xenial-mythserver/disk.mnt-dvd_rips rw,relatime
shared:39 - ext4 /dev/sdb1 rw,data=ordered
Serge E. Hallyn
2016-06-30 01:35:57 UTC
Permalink
Post by Rob
Post by Serge E. Hallyn
lxc-start 20160628155820.614 ERROR lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1662 - No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
ENOENT - that's unexpected...
0.29
Can you show 'dpkg -l | grep cgmanager' ?
as well as cat /etc/*release
Hi, For /proc/self/cgroup and /proc/self/mountinfo, we actually
need to see the contents. Can you show 'cat /proc/self/cgroup' and
'cat /proc/self/mountinfo'? -serge
_______________________________________________ lxc-users mailing
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
here is the follow up info (note that I cut the msg above in order
to reduce size)
$ dpkg -l | grep cgmanager
ii cgmanager 0.39-2ubuntu5 amd64
Central cgroup manager daemon
ii libcgmanager0:amd64 0.39-2ubuntu5
amd64 Central cgroup manager daemon (client library)
$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"
NAME="Ubuntu"
VERSION="16.04 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial
$ cat /proc/self/cgroup
11:blkio:/user.slice
10:hugetlb:/
9:freezer:/user/redger/1
8:pids:/user.slice/user-1000.slice
7:perf_event:/
6:cpu,cpuacct:/user.slice
5:net_cls,net_prio:/
4:devices:/user.slice
3:memory:/user/redger/1
2:cpuset:/
Oh, ok. I'm sorry, this should have been obvious to me from the start.

You need to edit /etc/pam.d/common-session and change the line that's
something like

session optional pam_cgfs.so -c freezer,memory,name=systemd

to add ",cpuset" at the end, i.e.

session optional pam_cgfs.so -c freezer,memory,name=systemd,cpuset

It has been removed from the default because on systems which do a lot
of cpu hotplugging it can be a problem: with the legacy (non-unified)
cpuset hierarchy, when you unplug a cpu that is part of /user, it gets
removed, but when you re-plug it it does not get re-added.
Rob Edgerton
2016-06-30 02:39:37 UTC
Permalink
Post by Rob
Post by Serge E. Hallyn
      lxc-start 20160628155820.614 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1662 - No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
ENOENT - that's unexpected...
0.29
Can you show 'dpkg -l | grep cgmanager' ?
as well as cat /etc/*release
Hi, For /proc/self/cgroup and /proc/self/mountinfo, we actually
need to see the contents. Can you show 'cat /proc/self/cgroup' and
'cat /proc/self/mountinfo'? -serge
_______________________________________________ lxc-users mailing
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
here is the follow up info (note that I cut the msg above in order
to reduce size)
$ dpkg -l | grep cgmanager
ii  cgmanager 0.39-2ubuntu5                              amd64
Central cgroup manager daemon
ii  libcgmanager0:amd64 0.39-2ubuntu5
amd64        Central cgroup manager daemon (client library)
$ cat /etc/*release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"
NAME="Ubuntu"
VERSION="16.04 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
UBUNTU_CODENAME=xenial
$ cat /proc/self/cgroup
11:blkio:/user.slice
10:hugetlb:/
9:freezer:/user/redger/1
8:pids:/user.slice/user-1000.slice
7:perf_event:/
6:cpu,cpuacct:/user.slice
5:net_cls,net_prio:/
4:devices:/user.slice
3:memory:/user/redger/1
2:cpuset:/
Oh, ok.  I'm sorry, this should have been obvious to me from the start.

You need to edit /etc/pam.d/common-session and change the line that's
something like

session optional    pam_cgfs.so -c freezer,memory,name=systemd

to add ",cpuset" at the end, i.e.

session optional    pam_cgfs.so -c freezer,memory,name=systemd,cpuset

It has been removed from the default because on systems which do a lot
of cpu hotplugging it can be a problem:  with the legacy (non-unified)
cpuset hierarchy, when you unplug a cpu that is part of /user, it gets
removed, but when you re-plug it it does not get re-added.
hi Serge,thanks for the response.
I updated pam.d/common-session# ================= RE Changed ================= #
#session        optional        pam_cgfs.so -c freezer,memory,name=systemd
session optional        pam_cgfs.so -c freezer,memory,name=systemd,cpuset
# ================= RE Changed ================= #
then restarted, with similar result. Further, the config contains auth for using USB devices too# USB devices
lxc.cgroup.devices.allow = c 10:200 rwm# CPU & Memory limits
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.cpu.shares = 256
lxc.cgroup.memory.limit_in_bytes = 4G
lxc.cgroup.blkio.weight = 500

Commenting out the first line still results in start failure, as do the other lines. Even just uncommenting the memory.limit lines leads to failure with$ lxc-start -n trusty_unp_ibvpn -F
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup setup for trusty_unp_ibvpn
lxc-start: start.c: lxc_spawn: 1226 failed to setup the devices cgroup for 'trusty_unp_ibvpn'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'trusty_unp_ibvpn'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.

here's a sample log sequence where ONLY "lxc.cgroup.memory.limit_in_bytes = 4G" was uncommented
     lxc-start 20160630023739.583 INFO     lxc_conf - conf.c:lxc_create_tty:3303 - tty's configured
      lxc-start 20160630023739.583 INFO     lxc_conf - conf.c:setup_tty:995 - 4 tty(s) has been setup
      lxc-start 20160630023739.583 INFO     lxc_conf - conf.c:setup_personality:1393 - set personality to '0x0'
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'mac_admin' (33)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'mac_override' (32)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'sys_time' (25)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'sys_module' (16)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'sys_rawio' (17)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2065 - capabilities have been setup
      lxc-start 20160630023739.583 NOTICE   lxc_conf - conf.c:lxc_setup:3839 - 'trusty_unp_ibvpn' is setup.
      lxc-start 20160630123739.583 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1645 - No devices cgroup setup for trusty_unp_ibvpn
      lxc-start 20160630123739.583 ERROR    lxc_start - start.c:lxc_spawn:1226 - failed to setup the devices cgroup for 'trusty_unp_ibvpn'
      lxc-start 20160630123739.583 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'trusty_unp_ibvpn'
      lxc-start 20160630123739.633 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'trusty_unp_ibvpn', config section 'lxc'
      lxc-start 20160630123740.147 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 20160630123740.147 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
Serge E. Hallyn
2016-06-30 02:48:53 UTC
Permalink
On Thu, Jun 30, 2016 at 02:39:37AM +0000, Rob Edgerton wrote:
...
Post by Rob Edgerton
I updated pam.d/common-session# ================= RE Changed ================= #
#session        optional        pam_cgfs.so -c freezer,memory,name=systemd
session optional        pam_cgfs.so -c freezer,memory,name=systemd,cpuset
# ================= RE Changed ================= #
then restarted, with similar result. Further, the config contains auth for using USB devices too# USB devices
lxc.cgroup.devices.allow = c 10:200 rwm# CPU & Memory limits
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.cpu.shares = 256
lxc.cgroup.memory.limit_in_bytes = 4G
lxc.cgroup.blkio.weight = 500
Ok two things here - first, you'll of course need to add every controller
that you want to use to the pam_cgfs.so line in /etc/pam.d/common-session.

Second, in order to set devices cgroup entries you may need to use cgmanager,
as unprivileged users are not allowed to write those. But then, you
shouldn't need the devices.allow line at all, because your container is
unprivileged and therefore no devices cgroup limits are set.
Post by Rob Edgerton
Commenting out the first line still results in start failure, as do the other lines. Even just uncommenting the memory.limit lines leads to failure with$ lxc-start -n trusty_unp_ibvpn -F
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup setup for trusty_unp_ibvpn
lxc-start: start.c: lxc_spawn: 1226 failed to setup the devices cgroup for 'trusty_unp_ibvpn'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'trusty_unp_ibvpn'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority options.
here's a sample log sequence where ONLY "lxc.cgroup.memory.limit_in_bytes = 4G" was uncommented
     lxc-start 20160630023739.583 INFO     lxc_conf - conf.c:lxc_create_tty:3303 - tty's configured
      lxc-start 20160630023739.583 INFO     lxc_conf - conf.c:setup_tty:995 - 4 tty(s) has been setup
      lxc-start 20160630023739.583 INFO     lxc_conf - conf.c:setup_personality:1393 - set personality to '0x0'
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'mac_admin' (33)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'mac_override' (32)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'sys_time' (25)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'sys_module' (16)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2056 - drop capability 'sys_rawio' (17)
      lxc-start 20160630023739.583 DEBUG    lxc_conf - conf.c:setup_caps:2065 - capabilities have been setup
      lxc-start 20160630023739.583 NOTICE   lxc_conf - conf.c:lxc_setup:3839 - 'trusty_unp_ibvpn' is setup.
      lxc-start 20160630123739.583 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1645 - No devices cgroup setup for trusty_unp_ibvpn
      lxc-start 20160630123739.583 ERROR    lxc_start - start.c:lxc_spawn:1226 - failed to setup the devices cgroup for 'trusty_unp_ibvpn'
      lxc-start 20160630123739.583 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'trusty_unp_ibvpn'
      lxc-start 20160630123739.633 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'trusty_unp_ibvpn', config section 'lxc'
      lxc-start 20160630123740.147 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 20160630123740.147 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
rob e
2016-06-30 22:42:03 UTC
Permalink
Post by Serge E. Hallyn
Oh, ok. I'm sorry, this should have been obvious to me from the start.
You need to edit /etc/pam.d/common-session and change the line that's
something like
session optional pam_cgfs.so -c freezer,memory,name=systemd
to add ",cpuset" at the end, i.e.
session optional pam_cgfs.so -c freezer,memory,name=systemd,cpuset
It has been removed from the default because on systems which do a lot
of cpu hotplugging it can be a problem: with the legacy (non-unified)
cpuset hierarchy, when you unplug a cpu that is part of /user, it gets
removed, but when you re-plug it it does not get re-added.
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
thanks Serge,
I tried that. Same result. Additionally, even when I comment out the CPU
controls, leaving only Memory limits, it still fails.

To confirm, I have 3 uses for cgroups -
1) Resource control on CPU, Memory, Disk, Network etc eg.
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.memory.limit_in_bytes = 4G
2) Access to devices, particularly USB tuners
lxc.cgroup.devices.allow = c 212:* rwm
3) Access to TAP / TUN devices in order to run VPN in a container
lxc.cgroup.devices.allow = c 10:200 rwm

All 3 fail in the same way. Any one of them leads to failure (including
Memory limits)

Here's the current value from /etc/pam.d/common-session
session optional pam_cfgs.so -c freezer,memory,name=systemd,cpuset
the memory clause already existed before edits. Memory limit setting has
failed with default and after the above edit

Error is "No devices group set up for ......"

thanks for your help
Rob

PS Some emails appear to have been "lost", apologies if this is a
logical duplicate
Serge E. Hallyn
2016-07-01 00:58:34 UTC
Permalink
Post by rob e
Post by Serge E. Hallyn
Oh, ok. I'm sorry, this should have been obvious to me from the start.
You need to edit /etc/pam.d/common-session and change the line that's
something like
session optional pam_cgfs.so -c freezer,memory,name=systemd
to add ",cpuset" at the end, i.e.
session optional pam_cgfs.so -c freezer,memory,name=systemd,cpuset
It has been removed from the default because on systems which do a lot
of cpu hotplugging it can be a problem: with the legacy (non-unified)
cpuset hierarchy, when you unplug a cpu that is part of /user, it gets
removed, but when you re-plug it it does not get re-added.
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
thanks Serge,
I tried that. Same result. Additionally, even when I comment out the
CPU controls, leaving only Memory limits, it still fails.
To confirm, I have 3 uses for cgroups -
1) Resource control on CPU, Memory, Disk, Network etc eg.
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.memory.limit_in_bytes = 4G
Let's address them one at a time. For starters,

if you only leave in the
lxc.cgroup.cpuset.cpus = 1-3
does that now work? If not, please post the log output to show exactly
how it fails.
And if you only have
lxc.cgroup.memory.limit_in_bytes = 4G
how does that fail, exactly?

Also, what is /proc/self/cgroup now when you login?
Post by rob e
2) Access to devices, particularly USB tuners
lxc.cgroup.devices.allow = c 212:* rwm
3) Access to TAP / TUN devices in order to run VPN in a container
lxc.cgroup.devices.allow = c 10:200 rwm
All 3 fail in the same way. Any one of them leads to failure
(including Memory limits)
Here's the current value from /etc/pam.d/common-session
session optional pam_cfgs.so -c freezer,memory,name=systemd,cpuset
the memory clause already existed before edits. Memory limit setting
has failed with default and after the above edit
Error is "No devices group set up for ......"
thanks for your help
Rob
PS Some emails appear to have been "lost", apologies if this is a
logical duplicate
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
rob e
2016-07-01 03:33:57 UTC
Permalink
Post by Serge E. Hallyn
Post by rob e
thanks Serge,
I tried that. Same result. Additionally, even when I comment out the
CPU controls, leaving only Memory limits, it still fails.
To confirm, I have 3 uses for cgroups -
1) Resource control on CPU, Memory, Disk, Network etc eg.
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.memory.limit_in_bytes = 4G
Let's address them one at a time. For starters,
if you only leave in the
lxc.cgroup.cpuset.cpus = 1-3
does that now work? If not, please post the log output to show exactly
how it fails.
And if you only have
lxc.cgroup.memory.limit_in_bytes = 4G
how does that fail, exactly?
Also, what is /proc/self/cgroup now when you login?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
thanks for the response, data follows

--------------------------------------
From "my" session
$ cat /proc/self/cgroup
11:blkio:/user.slice
10:hugetlb:/
9:freezer:/user/redger/2
8:pids:/user.slice/user-1000.slice
7:memory:/user/redger/2
6:cpu,cpuacct:/user.slice
5:net_cls,net_prio:/
4:perf_event:/
3:cpuset:/user/redger/2
2:devices:/user.slice
1:name=systemd:/user.slice/user-1000.slice/session-11.scope

--------------------------------------
Lines from PAM (original commented and new line inserted)
#session optional pam_cgfs.so -c freezer,memory,name=systemd
session optional pam_cgfs.so -c freezer,memory,name=systemd,cpuset

--------------------------------------
current config for test system - Note Memory limit (only), no other
cgroup usage
# Template used to create this container:
/usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d ubuntu -r trusty -a amd64
# For additional config options, please look at lxc.container.conf(5)

# Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
lxc.arch = x86_64

# Container specific configuration
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.rootfs = /mnt/lxc_images/containers/xenial_test_01/rootfs
lxc.rootfs.backend = dir
lxc.utsname = xenial_test_01

# Network configuration
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:19:3c:15

## Set resource limits ----- Cause problems in Xenial
lxc.cgroup.memory.limit_in_bytes = 4G

--------------------------------------
And the result of starting (copied and pasted from konsole)
$ lxc-start -n xenial_test_01 -F -o lxc_test.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup setup
for xenial_test_01
lxc-start: start.c: lxc_spawn: 1226 failed to setup the devices cgroup
for 'xenial_test_01'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'xenial_test_01'
lxc-start: lxc_start.c: main: 344
The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained
by setting the --logfile and --logpriority options.

--------------------------------------
Last few lines of the log file
lxc-start 20160701032506.416 DEBUG lxc_conf -
conf.c:setup_caps:2056 - drop capability 'sys_rawio' (17)
lxc-start 20160701032506.416 DEBUG lxc_conf -
conf.c:setup_caps:2065 - capabilities have been setup
lxc-start 20160701032506.416 NOTICE lxc_conf -
conf.c:lxc_setup:3839 - 'xenial_test_01' is setup.
lxc-start 20160701132506.417 ERROR lxc_cgfsng -
cgfsng.c:cgfsng_setup_limits:1645 - No devices cgroup setup for
xenial_test_01
lxc-start 20160701132506.417 ERROR lxc_start -
start.c:lxc_spawn:1226 - failed to setup the devices cgroup for
'xenial_test_01'
lxc-start 20160701132506.417 ERROR lxc_start -
start.c:__lxc_start:1353 - failed to spawn 'xenial_test_01'
lxc-start 20160701132506.449 INFO lxc_conf -
conf.c:run_script_argv:367 - Executing script
'/usr/share/lxcfs/lxc.reboot.hook' for container 'xenial_test_01',
config section 'lxc'
lxc-start 20160701132506.960 ERROR lxc_start_ui -
lxc_start.c:main:344 - The container failed to start.
lxc-start 20160701132506.960 ERROR lxc_start_ui -
lxc_start.c:main:348 - Additional information can be obtained by setting
the --logfile and --logpriority options.

I can attach the logfile if that helps, tho it may delay the email due
to size
Serge E. Hallyn
2016-07-01 15:02:27 UTC
Permalink
Post by Rob
Post by Serge E. Hallyn
Let's address them one at a time. For starters,
if you only leave in the
lxc.cgroup.cpuset.cpus = 1-3
does that now work? If not, please post the log output to show exactly
how it fails.
And if you only have
lxc.cgroup.memory.limit_in_bytes = 4G
how does that fail, exactly?
Also, what is /proc/self/cgroup now when you login?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
thanks for the response, data follows
--------------------------------------
From "my" session
$ cat /proc/self/cgroup
11:blkio:/user.slice
10:hugetlb:/
9:freezer:/user/redger/2
8:pids:/user.slice/user-1000.slice
7:memory:/user/redger/2
6:cpu,cpuacct:/user.slice
5:net_cls,net_prio:/
4:perf_event:/
3:cpuset:/user/redger/2
2:devices:/user.slice
1:name=systemd:/user.slice/user-1000.slice/session-11.scope
--------------------------------------
Lines from PAM (original commented and new line inserted)
#session optional pam_cgfs.so -c freezer,memory,name=systemd
session optional pam_cgfs.so -c freezer,memory,name=systemd,cpuset
--------------------------------------
current config for test system - Note Memory limit (only), no other
cgroup usage
/usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d ubuntu -r trusty -a amd64
# For additional config options, please look at lxc.container.conf(5)
# Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
lxc.arch = x86_64
# Container specific configuration
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.rootfs = /mnt/lxc_images/containers/xenial_test_01/rootfs
lxc.rootfs.backend = dir
lxc.utsname = xenial_test_01
# Network configuration
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:19:3c:15
## Set resource limits ----- Cause problems in Xenial
lxc.cgroup.memory.limit_in_bytes = 4G
--------------------------------------
And the result of starting (copied and pasted from konsole)
$ lxc-start -n xenial_test_01 -F -o lxc_test.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup
setup for xenial_test_01
Wait, why is it still showing this error? You don't
have any lxc.cgroup.deivces in the above config!

Can you please show

/usr/share/lxc/config/ubuntu.common.conf
/usr/share/lxc/config/ubuntu.userns.conf

?
rob e
2016-07-01 21:39:34 UTC
Permalink
Post by Serge E. Hallyn
Post by Rob
Post by Serge E. Hallyn
Let's address them one at a time. For starters,
if you only leave in the
lxc.cgroup.cpuset.cpus = 1-3
does that now work? If not, please post the log output to show exactly
how it fails.
And if you only have
lxc.cgroup.memory.limit_in_bytes = 4G
how does that fail, exactly?
Also, what is /proc/self/cgroup now when you login?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
thanks for the response, data follows
Wait, why is it still showing this error? You don't
have any lxc.cgroup.deivces in the above config!
Can you please show
/usr/share/lxc/config/ubuntu.common.conf
/usr/share/lxc/config/ubuntu.userns.conf
?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
okey dokes, here they are (plus the direct "include" elements)

------------------------------------------------------------------------------------------
$ cat /usr/share/lxc/config/ubuntu.common.conf
# This derives from the global common config
lxc.include = /usr/share/lxc/config/common.conf

# Default mount entries
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none bind,optional 0 0
lxc.mount.entry = /sys/kernel/security sys/kernel/security none
bind,optional 0 0
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none bind,optional 0 0
lxc.mount.entry = mqueue dev/mqueue mqueue
rw,relatime,create=dir,optional 0 0

# When using LXC with apparmor, the container will be confined by default.
# If you wish for it to instead run unconfined, copy the following line
# (uncommented) to the container's configuration file.
#lxc.aa_profile = unconfined

# Uncomment the following line to autodetect squid-deb-proxy
configuration on the
# host and forward it to the guest at start time.
#lxc.hook.pre-start = /usr/share/lxc/hooks/squid-deb-proxy-client

# If you wish to allow mounting block filesystems, then use the following
# line instead, and make sure to grant access to the block device and/or
loop
# devices below in lxc.cgroup.devices.allow.
#lxc.aa_profile = lxc-container-default-with-mounting

# Extra cgroup device access
## rtc
lxc.cgroup.devices.allow = c 254:0 rm
## tun
lxc.cgroup.devices.allow = c 10:200 rwm
## hpet
lxc.cgroup.devices.allow = c 10:228 rwm
## kvm
lxc.cgroup.devices.allow = c 10:232 rwm
## To use loop devices, copy the following line to the container's
## configuration file (uncommented).
#lxc.cgroup.devices.allow = b 7:* rwm

------------------------------------------------------------------------------------------
$ cat /usr/share/lxc/config/ubuntu.userns.conf
# This derives from the global userns config
lxc.include = /usr/share/lxc/config/userns.conf

# Extra fstab entries as mountall can't mount those by itself
lxc.mount.entry = /sys/firmware/efi/efivars sys/firmware/efi/efivars
none bind,optional 0 0
lxc.mount.entry = /proc/sys/fs/binfmt_misc proc/sys/fs/binfmt_misc none
bind,optional 0 0

------------------------------------------------------------------------------------------
$ cat /usr/share/lxc/config/userns.conf
# CAP_SYS_ADMIN in init-user-ns is required for cgroup.devices
lxc.cgroup.devices.deny =
lxc.cgroup.devices.allow =

# We can't move bind-mounts, so don't use /dev/lxc/
lxc.devttydir =

# Extra bind-mounts for userns
lxc.mount.entry = /dev/console dev/console none bind,create=file 0 0
lxc.mount.entry = /dev/full dev/full none bind,create=file 0 0
lxc.mount.entry = /dev/null dev/null none bind,create=file 0 0
lxc.mount.entry = /dev/random dev/random none bind,create=file 0 0
lxc.mount.entry = /dev/tty dev/tty none bind,create=file 0 0
lxc.mount.entry = /dev/urandom dev/urandom none bind,create=file 0 0
lxc.mount.entry = /dev/zero dev/zero none bind,create=file 0 0

------------------------------------------------------------------------------------------
$ cat /usr/share/lxc/config/common.conf
# Default configuration shared by all containers

# Setup the LXC devices in /dev/lxc/
lxc.devttydir = lxc

# Allow for 1024 pseudo terminals
lxc.pts = 1024

# Setup 4 tty devices
lxc.tty = 4

# Drop some harmful capabilities
lxc.cap.drop = mac_admin mac_override sys_time sys_module sys_rawio

# Set the pivot directory
lxc.pivotdir = lxc_putold

# Ensure hostname is changed on clone
lxc.hook.clone = /usr/share/lxc/hooks/clonehostname

# CGroup whitelist
lxc.cgroup.devices.deny = a
## Allow any mknod (but not reading/writing the node)
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
## Allow specific devices
### /dev/null
lxc.cgroup.devices.allow = c 1:3 rwm
### /dev/zero
lxc.cgroup.devices.allow = c 1:5 rwm
### /dev/full
lxc.cgroup.devices.allow = c 1:7 rwm
### /dev/tty
lxc.cgroup.devices.allow = c 5:0 rwm
### /dev/console
lxc.cgroup.devices.allow = c 5:1 rwm
### /dev/ptmx
lxc.cgroup.devices.allow = c 5:2 rwm
### /dev/random
lxc.cgroup.devices.allow = c 1:8 rwm
### /dev/urandom
lxc.cgroup.devices.allow = c 1:9 rwm
### /dev/pts/*
lxc.cgroup.devices.allow = c 136:* rwm
### fuse
lxc.cgroup.devices.allow = c 10:229 rwm

# Setup the default mounts
lxc.mount.auto = cgroup:mixed proc:mixed sys:mixed
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none
bind,optional 0 0

# Blacklist some syscalls which are not safe in privileged
# containers
lxc.seccomp = /usr/share/lxc/config/common.seccomp

# Lastly, include all the configs from /usr/share/lxc/config/common.conf.d/
lxc.include = /usr/share/lxc/config/common.conf.d/

------------------------------------------------------------------------------------------
Serge E. Hallyn
2016-07-01 23:28:16 UTC
Permalink
Post by rob e
Post by Serge E. Hallyn
Post by Rob
Post by Serge E. Hallyn
Let's address them one at a time. For starters,
if you only leave in the
lxc.cgroup.cpuset.cpus = 1-3
does that now work? If not, please post the log output to show exactly
how it fails.
And if you only have
lxc.cgroup.memory.limit_in_bytes = 4G
how does that fail, exactly?
Also, what is /proc/self/cgroup now when you login?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
thanks for the response, data follows
Wait, why is it still showing this error? You don't
have any lxc.cgroup.deivces in the above config!
Can you please show
/usr/share/lxc/config/ubuntu.common.conf
/usr/share/lxc/config/ubuntu.userns.conf
?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
okey dokes, here they are (plus the direct "include" elements)
Thanks. Yeah, this is making no sense. There should be no
lxc.cgroup.devices.*. Can you add

lxc.cgroup.devices.allow =
lxc.cgroup.devices.deny =

to the end of your config and try again?
rob e
2016-07-02 01:32:01 UTC
Permalink
Post by Serge E. Hallyn
Post by rob e
Post by Serge E. Hallyn
Post by Rob
Post by Serge E. Hallyn
Let's address them one at a time. For starters,
if you only leave in the
lxc.cgroup.cpuset.cpus = 1-3
does that now work? If not, please post the log output to show exactly
how it fails.
And if you only have
lxc.cgroup.memory.limit_in_bytes = 4G
how does that fail, exactly?
Also, what is /proc/self/cgroup now when you login?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
thanks for the response, data follows
Wait, why is it still showing this error? You don't
have any lxc.cgroup.deivces in the above config!
Can you please show
/usr/share/lxc/config/ubuntu.common.conf
/usr/share/lxc/config/ubuntu.userns.conf
?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
okey dokes, here they are (plus the direct "include" elements)
Thanks. Yeah, this is making no sense. There should be no
lxc.cgroup.devices.*. Can you add
lxc.cgroup.devices.allow =
lxc.cgroup.devices.deny =
to the end of your config and try again?
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just the
shutdown messages. But it does get to a login prompt

-----------------------------------------------------------------
$ lxc-start -n xenial_test_01 -F -o lxc_test_shall_160702a.log -l debug
Ubuntu 14.04.4 LTS xenial_test_01 console

xenial_test_01 login: * Stopping save kernel messages ...done.
wait-for-state stop/waiting
* Asking all remaining processes to terminate...
...done.
* All processes ended within 1 seconds...
...done.
* Deactivating swap...
swapoff: Not superuser.
...fail!
* Unmounting local filesystems...
umount2: Permission denied
umount: /dev/zero: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/urandom: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/tty: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/random: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/null: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/full: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/console: block devices not permitted on fs
umount2: Permission denied
umount: /dev/mqueue: block devices not permitted on fs
umount2: Permission denied
...fail!
mount: cannot mount block device /dev/mapper/wd2t--lvm--data-lxc_images
read-only
* Will now halt
-----------------------------------------------------------------
Log file attached as lxc_test_shall_160702b.log

-----------------------------------------------------------------
Then I ran it again, this time with cgroup for
memory(lxc.cgroup.memory.limit_in_bytes = 4G), and it died

$ lxc-start -n xenial_test_01 -F -o lxc_test_shall_mem_160702b.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup setup
for xenial_test_01
lxc-start: start.c: lxc_spawn: 1226 failed to setup the devices cgroup
for 'xenial_test_01'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'xenial_test_01'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained
by setting the --logfile and --logpriority options.

-----------------------------------------------------------------
Log file attached as lxc_test_shall_mem_160702b.log

R
Serge E. Hallyn
2016-07-02 02:14:05 UTC
Permalink
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
Serge E. Hallyn
2016-07-02 02:22:23 UTC
Permalink
Post by Serge E. Hallyn
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
https://github.com/lxc/lxc/pull/1070
rob e
2016-07-02 02:35:51 UTC
Permalink
Post by Serge E. Hallyn
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
sorry about this ... didn't work. Tried 2 forms of Pam clause & 2 forms
of config

------------------------------------------------------
PAM line
session optional pam_cgfs.so -c
freezer,memory,name=systemd,cpuset,devices

Config elements
lxc.cgroup.memory.limit_in_bytes = 4G
lxc.cgroup.devices.allow =
lxc.cgroup.devices.deny =

$ lxc-start -n xenial_test_01 -F -o lxc_test_mem_160702c.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup setup
for xenial_test_01
lxc-start: start.c: lxc_spawn: 1226 failed to setup the devices cgroup
for 'xenial_test_01'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'xenial_test_01'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained
by setting the --logfile and --logpriority options.


See attached log file lxc_test_mem_160702c.log
------------------------------------------------------
PAM line (no cpuset this time)
session optional pam_cgfs.so -c freezer,memory,name=systemd,devices

Config elements
lxc.cgroup.memory.limit_in_bytes = 4G
lxc.cgroup.devices.allow =
lxc.cgroup.devices.deny =

$ lxc-start -n xenial_test_01 -F -o lxc_test_mem_160702d.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup setup
for xenial_test_01
lxc-start: start.c: lxc_spawn: 1226 failed to setup the devices cgroup
for 'xenial_test_01'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'xenial_test_01'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained
by setting the --logfile and --logpriority options.

See attached log file lxc_test_mem_160702d.log

------------------------------------------------------
PAM line (no cpuset this time)
session optional pam_cgfs.so -c freezer,memory,name=systemd,devices

Config elements (removed the extra test clauses)
lxc.cgroup.memory.limit_in_bytes = 4G

$ lxc-start -n xenial_test_01 -F -o lxc_test_mem_160702e.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup setup
for xenial_test_01
lxc-start: start.c: lxc_spawn: 1226 failed to setup the devices cgroup
for 'xenial_test_01'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'xenial_test_01'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained
by setting the --logfile and --logpriority options.


See attached log file lxc_test_mem_160702e.log
-----------------------------------------------------

still trying :)

R
Serge E. Hallyn
2016-07-02 02:41:45 UTC
Permalink
Post by rob e
Post by Serge E. Hallyn
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
sorry about this ... didn't work. Tried 2 forms of Pam clause & 2
forms of config
------------------------------------------------------
PAM line
session optional pam_cgfs.so -c
freezer,memory,name=systemd,cpuset,devices
Jus to make sure, did you log back in after this? what does /proc/self/cgroup
look like?
rob e
2016-07-02 03:04:53 UTC
Permalink
Post by Serge E. Hallyn
Post by rob e
Post by Serge E. Hallyn
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
sorry about this ... didn't work. Tried 2 forms of Pam clause & 2
forms of config
------------------------------------------------------
PAM line
session optional pam_cgfs.so -c
freezer,memory,name=systemd,cpuset,devices
Jus to make sure, did you log back in after this? what does /proc/self/cgroup
look like?
errr ... no. When I read the PAM documentation I formed the impression
all is dynamic so no need to log in again. Oops, sorry

Logged Off, then back in and retried and now looks MUCH BETTER :)
............

$ lxc-start -n xenial_test_01 -F -o lxc_test_mem_160702f.log -l debug
-------------- startup lines cleared from console
Ubuntu 14.04.4 LTS xenial_test_01 console

xenial_test_01 login: wait-for-state stop/waiting
* Asking all remaining processes to terminate...
...done.
* All processes ended within 1 seconds...
...done.
* Deactivating swap...
swapoff: Not superuser.
...fail!
* Unmounting local filesystems...
umount2: Permission denied
umount: /dev/zero: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/urandom: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/tty: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/random: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/null: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/full: block devices not permitted on fs
umount2: Permission denied
umount2: Permission denied
umount: /dev/console: block devices not permitted on fs
umount2: Permission denied
umount: /dev/mqueue: block devices not permitted on fs
umount2: Permission denied
...fail!
mount: cannot mount block device /dev/mapper/wd2t--lvm--data-lxc_images
read-only
* Will now halt

I've attached the log just in case
----------------------------
it runs !!!

NAME STATE AUTOSTART GROUPS
IPV4 IPV6
xenial_test_01 RUNNING 0 - 10.0.3.57 -

Yay .. thanks. Now I can constrain them which is important. Now for
passing devices :)

Great work. Thanks for your help

R
rob e
2016-07-02 03:24:44 UTC
Permalink
Post by Serge E. Hallyn
Post by rob e
Post by Serge E. Hallyn
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
sorry about this ... didn't work. Tried 2 forms of Pam clause & 2
forms of config
------------------------------------------------------
PAM line
session optional pam_cgfs.so -c
freezer,memory,name=systemd,cpuset,devices
Jus to make sure, did you log back in after this? what does /proc/self/cgroup
look like?
hmmm ... Now I tried the TAP TUN device (for openvpn & proxy server)
.... FAILED .. on CPUSET

------------------------------------------------------------------------
Config

# Template used to create this container:
/usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d ubuntu -r trusty -a amd64
# For additional config options, please look at lxc.container.conf(5)

# Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
lxc.arch = x86_64

# Container specific configuration
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.rootfs = /mnt/lxc_images/containers/trusty_unp_ibvpn/rootfs
lxc.utsname = trusty_unp_ibvpn

# Network configuration
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = lxcbr0
lxc.network.hwaddr = 00:16:3e:2e:b3:54
#
## Allow Tap / Tun Devices ----- Cause problems in Xenial
lxc.cgroup.devices.allow = c 10:200 rwm

lxc.pts = 1024
lxc.kmsg = 0

## Set resource limits
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.cpu.shares = 256
lxc.cgroup.memory.limit_in_bytes = 4G
lxc.cgroup.blkio.weight = 500


------------------------------------------------------------------------
$ lxc-start -n trusty_unp_ibvpn -F -o lxc_test_taptun__160702a.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1662 No such file or directory
- Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
lxc-start: start.c: lxc_spawn: 1180 failed to setup the cgroup limits
for 'trusty_unp_ibvpn'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'trusty_unp_ibvpn'
lxc-start:
lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained
by setting the --logfile and --logpriority options.

logfile attached
------------------------------------------------------------------------

So then I added back the CPUSET clause to PAM, logged out, back in and
tried again with my test container ...

session optional pam_cgfs.so -c
freezer,memory,name=systemd,cpuset,devices
------------------------------------------------------------------------
Config

# Template used to create this container:
/usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d ubuntu -r trusty -a amd64
# For additional config options, please look at lxc.container.conf(5)

# Uncomment the following line to support nesting containers:
#lxc.include = /usr/share/lxc/config/nesting.conf
# (Be aware this has security implications)


# Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
lxc.arch = x86_64

# Container specific configuration
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.rootfs = /mnt/lxc_images/containers/xenial_test_01/rootfs
lxc.rootfs.backend = dir
lxc.utsname = xenial_test_01

# Network configuration
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:19:3c:15

## Set resource limits ----- Cause problems in Xenial
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.cpu.shares = 256
lxc.cgroup.memory.limit_in_bytes = 4G
lxc.cgroup.blkio.weight = 500

------------------------------------------------------------------------
$ lxc-start -n xenial_test_01 -F -o lxc_test_cpu_160702a.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1662 No such file or directory
- Error setting cpu.shares to 256 for xenial_test_01
lxc-start: start.c: lxc_spawn: 1180 failed to setup the cgroup limits
for 'xenial_test_01'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'xenial_test_01'
lxc-start: lxc_start.c:
main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained
by setting the --logfile and --logpriority options.

Log attached
------------------------------------------------------------------------

So, Memory constraint worked after adding "Devices" ... but CPU didn't.
Not sure about access to devices .....

Sorry about this ...

R
Serge E. Hallyn
2016-07-02 03:40:40 UTC
Permalink
Post by rob e
Post by Serge E. Hallyn
Post by rob e
Post by Serge E. Hallyn
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
sorry about this ... didn't work. Tried 2 forms of Pam clause & 2
forms of config
------------------------------------------------------
PAM line
session optional pam_cgfs.so -c
freezer,memory,name=systemd,cpuset,devices
Jus to make sure, did you log back in after this? what does /proc/self/cgroup
look like?
hmmm ... Now I tried the TAP TUN device (for openvpn & proxy server)
.... FAILED .. on CPUSET
Nope, cpu and cpuset are actually two different controllers. It's failing on
cpu.shares in the cpu controller.

Note, I think you'll be happiest if you just drop the "-c xxxxx" from
/etc/pam.d/common-session. That will tell pam_cgfs to use all controllers.

-serge
rob e
2016-07-02 04:44:45 UTC
Permalink
Post by Serge E. Hallyn
Post by rob e
Post by Serge E. Hallyn
Post by rob e
Post by Serge E. Hallyn
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
sorry about this ... didn't work. Tried 2 forms of Pam clause & 2
forms of config
------------------------------------------------------
PAM line
session optional pam_cgfs.so -c
freezer,memory,name=systemd,cpuset,devices
Jus to make sure, did you log back in after this? what does /proc/self/cgroup
look like?
hmmm ... Now I tried the TAP TUN device (for openvpn & proxy server)
.... FAILED .. on CPUSET
Nope, cpu and cpuset are actually two different controllers. It's failing on
cpu.shares in the cpu controller.
Note, I think you'll be happiest if you just drop the "-c xxxxx" from
/etc/pam.d/common-session. That will tell pam_cgfs to use all controllers.
-serge
That was Better ! CPU and Memory constraints now don't cause failure :)

-----------------------------------------------------------------------------------
Tried VPN ... TAP / TUN FAILED. Container starts, but unable to create
device (where this worked on Trusty)

openvpn will not start ... looks like an AppArmor issue. Is this your
department ?

messages on host syslog.log

Jul 2 14:21:35 virt-host kernel: [111148.961739] IPv6:
ADDRCONF(NETDEV_CHANGE): vethS3C86K: link becomes ready
Jul 2 14:21:35 virt-host kernel: [111148.961777] lxcbr0: port
3(vethS3C86K) entered forwarding state
Jul 2 14:21:35 virt-host kernel: [111148.961785] lxcbr0: port
3(vethS3C86K) entered forwarding state
Jul 2 14:21:35 virt-host kernel: [111149.061396] audit: type=1400
audit(1467433295.584:1118): apparmor="DENIED" operation="mount"
info="failed flags match" error=-13
profile="lxc-container-default-with-mounting" name="/" pid=25762
comm="cgmanager" flags="rw, rprivate"
Jul 2 14:21:35 virt-host kernel: [111149.061437] audit: type=1400
audit(1467433295.584:1119): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/blkio/" pid=25762 comm="cgmanager"
fstype="cgroup" srcname="blkio"
Jul 2 14:21:35 virt-host kernel: [111149.061447] audit: type=1400
audit(1467433295.584:1120): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/cpu/" pid=25762 comm="cgmanager" fstype="cgroup"
srcname="cpu"
Jul 2 14:21:35 virt-host kernel: [111149.061457] audit: type=1400
audit(1467433295.584:1121): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/cpuacct/" pid=25762 comm="cgmanager"
fstype="cgroup" srcname="cpuacct"
Jul 2 14:21:35 virt-host kernel: [111149.061466] audit: type=1400
audit(1467433295.584:1122): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/cpuset/" pid=25762 comm="cgmanager"
fstype="cgroup" srcname="cpuset"
Jul 2 14:21:35 virt-host kernel: [111149.061475] audit: type=1400
audit(1467433295.584:1123): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/devices/" pid=25762 comm="cgmanager"
fstype="cgroup" srcname="devices"
Jul 2 14:21:35 virt-host kernel: [111149.061484] audit: type=1400
audit(1467433295.584:1124): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/freezer/" pid=25762 comm="cgmanager"
fstype="cgroup" srcname="freezer"
Jul 2 14:21:35 virt-host kernel: [111149.061492] audit: type=1400
audit(1467433295.584:1125): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/hugetlb/" pid=25762 comm="cgmanager"
fstype="cgroup" srcname="hugetlb"
Jul 2 14:21:35 virt-host kernel: [111149.061501] audit: type=1400
audit(1467433295.584:1126): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/memory/" pid=25762 comm="cgmanager"
fstype="cgroup" srcname="memory"
Jul 2 14:21:35 virt-host kernel: [111149.061510] audit: type=1400
audit(1467433295.584:1127): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting"
name="/run/cgmanager/fs/net_cls/" pid=25762 comm="cgmanager"
fstype="cgroup" srcname="net_cls"
Jul 2 14:21:35 virt-host libvirtd[32021]: Failed to open file
'/sys/class/net/vethS3C86Kp/operstate': No such file or directory
Jul 2 14:21:35 virt-host libvirtd[32021]: unable to read:
/sys/class/net/vethS3C86Kp/operstate: No such file or directory
Jul 2 14:21:37 virt-host avahi-daemon[1190]: Joining mDNS multicast
group on interface vethS3C86K.IPv6 with address fe80::fc29:c4ff:fe45:3afa.
Jul 2 14:21:37 virt-host avahi-daemon[1190]: New relevant interface
vethS3C86K.IPv6 for mDNS.
Jul 2 14:21:37 virt-host avahi-daemon[1190]: Registering new address
record for fe80::fc29:c4ff:fe45:3afa on vethS3C86K.*.
Jul 2 14:21:50 virt-host kernel: [111164.003628] lxcbr0: port
3(vethS3C86K) entered forwarding state
J
-----------------------------------------------------------------------------------

and will have to wait a while to test USB-DVB passthrough - currently
allocated to kvm machine and in use, would prefer to use lxc / lxd

(didn't work too well with LXD .. passes through ok, Frontend device
works but DMUX device inoperable, though it's present - will write a
separate stream on this one. Possible it's also Apparmor mediated)

R
rob e
2016-07-02 05:16:21 UTC
Permalink
Post by Serge E. Hallyn
Post by rob e
Post by Serge E. Hallyn
Post by rob e
Post by Serge E. Hallyn
Post by Rob
hi Serge,
with JUST those clauses (and no cgroup set clauses) ... it sort of
works. Initial messages are cleared from the console(?) leaving just
the shutdown messages. But it does get to a login prompt
D'oh. Thanks for your patience. I see the bug. I'll post a
PR for a fix. I'm surprised so few people run into this. But
as a workaround just add ",devices" to the end of the pam_cgfs
line in /etc/pam.d/common-session.
sorry about this ... didn't work. Tried 2 forms of Pam clause & 2
forms of config
------------------------------------------------------
PAM line
session optional pam_cgfs.so -c
freezer,memory,name=systemd,cpuset,devices
Jus to make sure, did you log back in after this? what does /proc/self/cgroup
look like?
hmmm ... Now I tried the TAP TUN device (for openvpn & proxy server)
.... FAILED .. on CPUSET
Nope, cpu and cpuset are actually two different controllers. It's failing on
cpu.shares in the cpu controller.
Note, I think you'll be happiest if you just drop the "-c xxxxx" from
/etc/pam.d/common-session. That will tell pam_cgfs to use all controllers.
-serge
ok, tried to pass through USB-DVB devices. This worked in Trusty using
the same config, but not on Xenial. Again, Apparmor is intervening. The
container starts ok, but doesn't map the /dev/dvb devices in (even tho I
had previously bind mounted /dev/dvb into the container, as was working
in Trusty)

sudo mount --bind /dev/dvb
/mnt/lxc_images/containers/trusty-mythserver/rootfs/dev/dvb/
sudo chown -R xxx:xxx
/mnt/lxc_images/containers/trusty-mythserver/rootfs/dev/dvb/

then look for devices in the container - nothing found :(

$ lxc-start -n trusty-mythserver
$ lxc-attach -n trusty-mythserver

***@trusty-mythserver:~#
***@trusty-mythserver:~# ls /dev/dvb
***@trusty-mythserver:~#

---------------------------------------------------------------------------------------
Syslog elements

Jul 2 15:09:17 virt-host libvirtd[32021]: Failed to open file
'/sys/class/net/veth1XDS50p/operstate': No such file or directory
Jul 2 15:09:17 virt-host libvirtd[32021]: unable to read:
/sys/class/net/veth1XDS50p/operstate: No such file or directory
Jul 2 15:09:17 virt-host kernel: [114010.904958] audit_printk_skb: 47
callbacks suppressed
Jul 2 15:09:17 virt-host kernel: [114010.904960] audit: type=1400
audit(1467436157.402:1273): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28339 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs"
Jul 2 15:09:17 virt-host kernel: [114010.904994] audit: type=1400
audit(1467436157.402:1274): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28339 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs" flags="ro"
Jul 2 15:09:17 virt-host kernel: [114011.015576] audit: type=1400
audit(1467436157.514:1275): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28498 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs"
Jul 2 15:09:17 virt-host kernel: [114011.015604] audit: type=1400
audit(1467436157.514:1276): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28498 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs" flags="ro"
Jul 2 15:09:17 virt-host kernel: [114011.053063] audit: type=1400
audit(1467436157.550:1277): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28552 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs"
Jul 2 15:09:17 virt-host kernel: [114011.053100] audit: type=1400
audit(1467436157.550:1278): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28552 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs" flags="ro"
Jul 2 15:09:17 virt-host kernel: [114011.077650] audit: type=1400
audit(1467436157.574:1279): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28584 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs"
Jul 2 15:09:17 virt-host kernel: [114011.077686] audit: type=1400
audit(1467436157.574:1280): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28584 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs" flags="ro"
Jul 2 15:09:17 virt-host kernel: [114011.089934] audit: type=1400
audit(1467436157.590:1281): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28609 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs"
Jul 2 15:09:17 virt-host kernel: [114011.089968] audit: type=1400
audit(1467436157.590:1282): apparmor="DENIED" operation="mount"
info="failed type match" error=-13
profile="lxc-container-default-with-mounting" name="/run/rpc_pipefs/"
pid=28609 comm="mount" fstype="rpc_pipefs" srcname="rpc_pipefs" flags="ro"
Jul 2 15:09:18 virt-host avahi-daemon[1190]: Joining mDNS multicast
group on interface veth1XDS50.IPv6 with address fe80::fca3:d7ff:fe0c:a9d8.
Jul 2 15:09:18 virt-host avahi-daemon[1190]: New relevant interface
veth1XDS50.IPv6 for mDNS.
Jul 2 15:09:18 virt-host avahi-daemon[1190]: Registering new address
record for fe80::fca3:d7ff:fe0c:a9d8 on veth1XDS50.*.
Jul 2 15:09:32 virt-host kernel: [114025.529956] rebr0: port
2(veth1XDS50) entered forwarding state

---------------------------------------------------------------------------------------
Container config (tried with and without the apparmor profile )

# Template used to create this container:
/usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d ubuntu -r trusty -a amd64
# For additional config options, please look at lxc.container.conf(5)

# Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
lxc.arch = x86_64

# Container specific configuration
# ------- Replaced -------#
#lxc.id_map = u 0 100000 65536
#lxc.id_map = g 0 100000 65536

lxc.id_map = u 0 100000 1000
lxc.id_map = g 0 100000 1000
lxc.id_map = u 1000 1000 1
lxc.id_map = g 1000 1000 1
lxc.id_map = u 1001 101001 64535
lxc.id_map = g 1001 101001 64535

lxc.rootfs = /mnt/lxc_images/containers/trusty-mythserver/rootfs
lxc.utsname = trusty-mythserver

## Network configuration
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = rebr0
lxc.network.hwaddr = xx.xx.xx.....

## devices - set profile to allow mounting block devices (constrained by
default)
lxc.aa_profile = lxc-container-default-with-mounting

## Allow access to USB devices (major part # 189), see
##
https://wiki.archlinux.org/index.php/Linux_Containers#Cgroups_device_configuration
## Use "ls -la /dev/bus/usb/003/" or "ls -la /dev/dvb/adapter0" to find
the major / minor numbers to permit
## DVB
lxc.cgroup.devices.allow = c 212:* rwm

## Set resource limits
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.cpu.shares = 256
lxc.cgroup.memory.limit_in_bytes = 4G
lxc.cgroup.blkio.weight = 500

---------------------------------------------------------------------------------------

on the plus side, the container starts ok.

I have not checked CPU and Memory limits yet ie. do they apply as expected

Are these Apparmor conditions your department ?

R
rob e
2016-07-01 03:40:20 UTC
Permalink
Post by Serge E. Hallyn
Post by rob e
thanks Serge,
I tried that. Same result. Additionally, even when I comment out the
CPU controls, leaving only Memory limits, it still fails.
To confirm, I have 3 uses for cgroups -
1) Resource control on CPU, Memory, Disk, Network etc eg.
lxc.cgroup.cpuset.cpus = 1-3
lxc.cgroup.memory.limit_in_bytes = 4G
Let's address them one at a time. For starters,
if you only leave in the
lxc.cgroup.cpuset.cpus = 1-3
does that now work? If not, please post the log output to show exactly
how it fails.
And if you only have
lxc.cgroup.memory.limit_in_bytes = 4G
how does that fail, exactly?
Also, what is /proc/self/cgroup now when you login?
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
hi Serge,
thanks for the response, data follows - CPU limits set this time

--------------------------------------
From "my" session
$ cat /proc/self/cgroup
11:blkio:/user.slice
10:hugetlb:/
9:freezer:/user/redger/2
8:pids:/user.slice/user-1000.slice
7:memory:/user/redger/2
6:cpu,cpuacct:/user.slice
5:net_cls,net_prio:/
4:perf_event:/
3:cpuset:/user/redger/2
2:devices:/user.slice
1:name=systemd:/user.slice/user-1000.slice/session-11.scope

--------------------------------------
Lines from PAM (original commented and new line inserted)
#session optional pam_cgfs.so -c freezer,memory,name=systemd
session optional pam_cgfs.so -c freezer,memory,name=systemd,cpuset

--------------------------------------
current config for test system - Note CPU limit (only), no other cgroup
usage
# Template used to create this container:
/usr/share/lxc/templates/lxc-download
# Parameters passed to the template: -d ubuntu -r trusty -a amd64
# For additional config options, please look at lxc.container.conf(5)

# Distribution configuration
lxc.include = /usr/share/lxc/config/ubuntu.common.conf
lxc.include = /usr/share/lxc/config/ubuntu.userns.conf
lxc.arch = x86_64

# Container specific configuration
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.rootfs = /mnt/lxc_images/containers/xenial_test_01/rootfs
lxc.rootfs.backend = dir
lxc.utsname = xenial_test_01

# Network configuration
lxc.network.type = veth
lxc.network.link = lxcbr0
lxc.network.flags = up
lxc.network.hwaddr = 00:16:3e:19:3c:15

## Set resource limits ----- Cause problems in Xenial
lxc.cgroup.cpuset.cpus = 1-3


--------------------------------------
And the result of starting (copied and pasted from konsole)
$ lxc-start -n xenial_test_01 -F -o lxc_test_cpu_160701a.log -l debug
lxc-start: cgfsng.c: cgfsng_setup_limits: 1645 No devices cgroup setup
for xenial_test_01
lxc-start: start.c: lxc_spawn: 1226 failed to setup the devices cgroup
for 'xenial_test_01'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'xenial_test_01'
lxc-start: lxc_start.c: main: 344
The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained
by setting the --logfile and --logpriority options.

--------------------------------------
Last few lines of the log file
lxc-start 20160701033803.685 DEBUG lxc_conf -
conf.c:setup_caps:2056 - drop capability 'sys_rawio' (17)
lxc-start 20160701033803.685 DEBUG lxc_conf -
conf.c:setup_caps:2065 - capabilities have been setup
lxc-start 20160701033803.685 NOTICE lxc_conf -
conf.c:lxc_setup:3839 - 'xenial_test_01' is setup.
lxc-start 20160701133803.685 ERROR lxc_cgfsng -
cgfsng.c:cgfsng_setup_limits:1645 - No devices cgroup setup for
xenial_test_01
lxc-start 20160701133803.685 ERROR lxc_start -
start.c:lxc_spawn:1226 - failed to setup the devices cgroup for
'xenial_test_01'
lxc-start 20160701133803.685 ERROR lxc_start -
start.c:__lxc_start:1353 - failed to spawn 'xenial_test_01'
lxc-start 20160701133803.721 INFO lxc_conf -
conf.c:run_script_argv:367 - Executing script
'/usr/share/lxcfs/lxc.reboot.hook' for container 'xenial_test_01',
config section 'lxc'
lxc-start 20160701133804.232 ERROR lxc_start_ui -
lxc_start.c:main:344 - The container failed to start.
lxc-start 20160701133804.233 ERROR lxc_start_ui -
lxc_start.c:main:348 - Additional information can be obtained by setting
the --logfile and --logpriority options.

I can attach the logfile if that helps, tho it may delay the email due
to size
Rob Edgerton
2016-06-30 00:43:51 UTC
Permalink
Post by Rob Edgerton
lxc-start: cgfsng.c: cgfsng_setup_limits: 1662 No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
lxc-start: start.c: lxc_spawn: 1180 failed to setup the cgroup limits for 'trusty_unp_ibvpn'
lxc-start: start.c: __lxc_start: 1353 failed to spawn 'trusty_unp_ibvpn'
lxc-start: lxc_start.c: main: 344 The container failed to start.
lxc-start: lxc_start.c: main: 348 Additional information can be obtained by setting the --logfile and --logpriority  options.
Logfile Contents=============
      lxc-start 20160628155820.562 INFO     lxc_start_ui - lxc_start.c:main:264 - using rcfile /mnt/lxc_images/containers/trusty_unp_ibvpn/config
      lxc-start 20160628155820.562 WARN     lxc_confile - confile.c:config_pivotdir:1879 - lxc.pivotdir is ignored.  It will soon become an error.
      lxc-start 20160628155820.562 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type u nsid 0 hostid 100000 range 65536
      lxc-start 20160628155820.562 INFO     lxc_confile - confile.c:config_idmap:1500 - read uid map: type g nsid 0 hostid 100000 range 65536
      lxc-start 20160628155820.564 INFO     lxc_lsm - lsm/lsm.c:lsm_init:48 - LSM security driver AppArmor
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .reject_force_umount  # comment this to allow umount -f;  not recommended.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for reject_force_umount action 0
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for reject_force_umount action 0
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:do_resolve_add_rule:216 - Setting seccomp rule to reject force umounts
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .[all].
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .kexec_load errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for kexec_load action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for kexec_load action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .open_by_handle_at errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for open_by_handle_at action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for open_by_handle_at action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .init_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for init_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for init_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .finit_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for finit_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for finit_module action 327681
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:342 - processing: .delete_module errno 1.
      lxc-start 20160628155820.564 INFO     lxc_seccomp - seccomp.c:parse_config_v2:446 - Adding native rule for delete_module action 327681
      lxc-start 20160628155820.565 INFO     lxc_seccomp - seccomp.c:parse_config_v2:449 - Adding compat rule for delete_module action 327681
      lxc-start 20160628155820.565 INFO     lxc_seccomp - seccomp.c:parse_config_v2:456 - Merging in the compat seccomp ctx into the main one
      lxc-start 20160628155820.565 DEBUG    lxc_start - start.c:setup_signal_fd:289 - sigchild handler set
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_peer_default:431 - opening /dev/tty for console peer
      lxc-start 20160628155820.565 INFO     lxc_caps - caps.c:lxc_caps_up:101 - Last supported cap was 36
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_peer_default:437 - using '/dev/tty' as console
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_sigwinch_init:145 - 3234 got SIGWINCH fd 9
      lxc-start 20160628155820.565 DEBUG    lxc_console - console.c:lxc_console_winsz:72 - set winsz dstfd:6 cols:212 rows:73
      lxc-start 20160628155820.611 INFO     lxc_start - start.c:lxc_init:488 - 'trusty_unp_ibvpn' is initialized
      lxc-start 20160628155820.611 DEBUG    lxc_start - start.c:__lxc_start:1326 - Not dropping cap_sys_boot or watching utmp
      lxc-start 20160628155820.611 INFO     lxc_start - start.c:resolve_clone_flags:1013 - Cloning a new user namespace
      lxc-start 20160628155820.611 INFO     lxc_cgroup - cgroup.c:cgroup_init:68 - cgroup driver cgroupfs-ng initing for trusty_unp_ibvpn
      lxc-start 20160628155820.614 DEBUG    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1667 - cgroup 'devices.allow' set to 'c 10:200 rwm'
      lxc-start 20160628155820.614 ERROR    lxc_cgfsng - cgfsng.c:cgfsng_setup_limits:1662 - No such file or directory - Error setting cpuset.cpus to 1-3 for trusty_unp_ibvpn
ENOENT - that's unexpected...
Post by Rob Edgerton
      lxc-start 20160628155820.615 ERROR    lxc_start - start.c:lxc_spawn:1180 - failed to setup the cgroup limits for 'trusty_unp_ibvpn'
      lxc-start 20160628155820.615 ERROR    lxc_start - start.c:__lxc_start:1353 - failed to spawn 'trusty_unp_ibvpn'
      lxc-start 20160628155820.659 INFO     lxc_conf - conf.c:run_script_argv:367 - Executing script '/usr/share/lxcfs/lxc.reboot.hook' for container 'trusty_unp_ibvpn', config section 'lxc'
      lxc-start 20160628155821.172 ERROR    lxc_start_ui - lxc_start.c:main:344 - The container failed to start.
      lxc-start 20160628155821.172 ERROR    lxc_start_ui - lxc_start.c:main:348 - Additional information can be obtained by setting the --logfile and --logpriority options.
  
Repeating the commands you were discussing with Mike
cgmanager is already the newest version (0.39-2ubuntu5).
@virt-host:~$cgm --version
0.29
Can you show 'dpkg -l | grep cgmanager' ?

as well as cat /etc/*release
Post by Rob Edgerton
@virt-host:~$ls /proc/self/cgroup
/proc/self/cgroup
@virt-host:~$ls /proc/self/mountinfo
/proc/self/mountinfo
Hi,
For /proc/self/cgroup and /proc/self/mountinfo, we actually need to see
the contents.  Can you show 'cat /proc/self/cgroup' and
'cat /proc/self/mountinfo'?

-serge
_______________________________________________
lxc-users mailing list
lxc-***@lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users

hi Serge,thanks for your response
$ dpkg -l | grep cgmanager
ii  cgmanager                                       0.39-2ubuntu5                              amd64        Central cgroup manager daemon
ii  libcgmanager0:amd64                             0.39-2ubuntu5                              amd64        Central cgroup manager daemon (client library)
$ cat /proc/self/cgroup
11:blkio:/user.slice
10:hugetlb:/
9:freezer:/user/redger/1
8:pids:/user.slice/user-1000.slice
7:perf_event:/
6:cpu,cpuacct:/user.slice
5:net_cls,net_prio:/
4:devices:/user.slice
3:memory:/user/redger/1
2:cpuset:/
1:name=systemd:/user.slice/user-1000.slice/session-1.scope

$ cat /proc/self/mountinfo
19 25 0:18 / /sys rw,nosuid,nodev,noexec,relatime shared:7 - sysfs sysfs rw
20 25 0:4 / /proc rw,nosuid,nodev,noexec,relatime shared:12 - proc proc rw
21 25 0:6 / /dev rw,nosuid,relatime shared:2 - devtmpfs udev rw,size=8026104k,nr_inodes=2006526,mode=755
22 21 0:14 / /dev/pts rw,nosuid,noexec,relatime shared:3 - devpts devpts rw,gid=5,mode=620,ptmxmode=000
23 25 0:19 / /run rw,nosuid,noexec,relatime shared:5 - tmpfs tmpfs rw,size=1615856k,mode=755
25 0 8:41 / / rw,relatime shared:1 - ext4 /dev/sdc9 rw,errors=remount-ro,data=ordered
26 19 0:12 / /sys/kernel/security rw,nosuid,nodev,noexec,relatime shared:8 - securityfs securityfs rw
27 21 0:21 / /dev/shm rw,nosuid,nodev shared:4 - tmpfs tmpfs rw
28 23 0:22 / /run/lock rw,nosuid,nodev,noexec,relatime shared:6 - tmpfs tmpfs rw,size=5120k
29 19 0:23 / /sys/fs/cgroup rw shared:9 - tmpfs tmpfs rw,mode=755
30 29 0:24 / /sys/fs/cgroup/systemd rw,nosuid,nodev,noexec,relatime shared:10 - cgroup cgroup rw,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
31 19 0:25 / /sys/fs/pstore rw,nosuid,nodev,noexec,relatime shared:11 - pstore pstore rw
32 29 0:26 / /sys/fs/cgroup/cpuset rw,nosuid,nodev,noexec,relatime shared:13 - cgroup cgroup rw,cpuset,clone_children
33 29 0:27 / /sys/fs/cgroup/memory rw,nosuid,nodev,noexec,relatime shared:14 - cgroup cgroup rw,memory
34 29 0:28 / /sys/fs/cgroup/devices rw,nosuid,nodev,noexec,relatime shared:15 - cgroup cgroup rw,devices
35 29 0:29 / /sys/fs/cgroup/net_cls,net_prio rw,nosuid,nodev,noexec,relatime shared:16 - cgroup cgroup rw,net_cls,net_prio
36 29 0:30 / /sys/fs/cgroup/cpu,cpuacct rw,nosuid,nodev,noexec,relatime shared:17 - cgroup cgroup rw,cpu,cpuacct
37 29 0:31 / /sys/fs/cgroup/perf_event rw,nosuid,nodev,noexec,relatime shared:18 - cgroup cgroup rw,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event
38 29 0:32 / /sys/fs/cgroup/pids rw,nosuid,nodev,noexec,relatime shared:19 - cgroup cgroup rw,pids,release_agent=/run/cgmanager/agents/cgm-release-agent.pids
39 29 0:33 / /sys/fs/cgroup/freezer rw,nosuid,nodev,noexec,relatime shared:20 - cgroup cgroup rw,freezer
40 29 0:34 / /sys/fs/cgroup/hugetlb rw,nosuid,nodev,noexec,relatime shared:21 - cgroup cgroup rw,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb
41 29 0:35 / /sys/fs/cgroup/blkio rw,nosuid,nodev,noexec,relatime shared:22 - cgroup cgroup rw,blkio
42 20 0:36 / /proc/sys/fs/binfmt_misc rw,relatime shared:23 - autofs systemd-1 rw,fd=28,pgrp=1,timeout=0,minproto=5,maxproto=5,direct
43 19 0:7 / /sys/kernel/debug rw,relatime shared:24 - debugfs debugfs rw
44 21 0:37 / /dev/hugepages rw,relatime shared:25 - hugetlbfs hugetlbfs rw
45 23 0:38 / /run/rpc_pipefs rw,relatime shared:26 - rpc_pipefs sunrpc rw
46 21 0:17 / /dev/mqueue rw,relatime shared:27 - mqueue mqueue rw
47 20 0:39 / /proc/fs/nfsd rw,relatime shared:28 - nfsd nfsd rw
48 19 0:40 / /sys/fs/fuse/connections rw,relatime shared:29 - fusectl fusectl rw
49 25 8:34 / /mnt/snd480_boot_01 rw,relatime shared:30 - ext4 /dev/sdc2 rw,data=ordered
50 25 8:35 / /mnt/snd480_root_01 rw,relatime shared:31 - ext4 /dev/sdc3 rw,data=ordered
51 25 8:42 / /home rw,relatime shared:32 - ext4 /dev/sdc10 rw,data=ordered
53 25 8:40 / /boot rw,relatime shared:33 - ext4 /dev/sdc8 rw,data=ordered
52 25 8:36 / /mnt/snd480_home_01 rw,relatime shared:34 - ext4 /dev/sdc4 rw,data=ordered
56 25 8:39 / /mnt/snd480_boot_03_wintemp rw,relatime shared:35 - vfat /dev/sdc7 rw,gid=46,fmask=0007,dmask=0007,allow_utime=0020,codepage=437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro
54 25 8:51 / /mnt/wd2t_home_01 rw,relatime shared:36 - ext4 /dev/sdd3 rw,data=ordered
55 25 8:1 / /mnt/video rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
57 25 8:50 / /mnt/wd2t_root_01 rw,relatime shared:38 - ext4 /dev/sdd2 rw,data=ordered
58 25 8:17 / /mnt/dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
60 25 8:18 / /mnt/music rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
59 25 8:49 / /mnt/wd2t_boot_01 rw,relatime shared:41 - ext4 /dev/sdd1 rw,data=ordered
61 25 252:4 / /mnt/lxc_images rw,noatime shared:42 - ext4 /dev/mapper/wd2t--lvm--data-lxc_images rw,data=ordered
63 25 0:41 / /var/lib/lxd rw,noatime shared:43 - btrfs /dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/
64 61 252:4 /containers/utopic_browse_normal_backup_151115/rootfs/home/ubuntu/Downloads /mnt/lxc_images/containers/trusty-mythserver/rootfs/mnt/lxc_container_normal_downloads rw,noatime shared:42 - ext4 /dev/mapper/wd2t--lvm--data-lxc_images rw,data=ordered
65 61 8:1 / /mnt/lxc_images/containers/browse_danger/rootfs/mnt/video rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
62 25 252:0 / /mnt/programming_data rw,relatime shared:44 - ext4 /dev/mapper/wd2t--lvm--data-programming_data rw,data=ordered
66 61 8:17 / /mnt/lxc_images/containers/browse_danger/rootfs/mnt/dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
67 61 8:1 / /mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/video rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
68 61 8:18 / /mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/music rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
69 61 8:18 / /mnt/lxc_images/containers/browse_danger/rootfs/mnt/music rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
70 61 8:17 / /mnt/lxc_images/containers/utopic_browse_normal_backup_151115/rootfs/mnt/dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
71 25 252:4 /containers/utopic_browse_normal_backup_151115/rootfs/home/ubuntu/Downloads /mnt/lxc_container_normal_downloads rw,noatime shared:42 - ext4 /dev/mapper/wd2t--lvm--data-lxc_images rw,data=ordered
72 25 0:41 / /mnt/lxd_images rw,noatime shared:43 - btrfs /dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/
142 23 0:45 / /run/cgmanager/fs rw,relatime shared:113 - tmpfs cgmfs rw,size=100k,mode=755
146 23 0:47 / /run/lxcfs/controllers rw,relatime shared:115 - tmpfs tmpfs rw,size=100k,mode=700
148 146 0:35 / /run/lxcfs/controllers/blkio rw,relatime shared:117 - cgroup blkio rw,blkio
150 146 0:34 / /run/lxcfs/controllers/hugetlb rw,relatime shared:119 - cgroup hugetlb rw,hugetlb,release_agent=/run/cgmanager/agents/cgm-release-agent.hugetlb
152 146 0:33 / /run/lxcfs/controllers/freezer rw,relatime shared:121 - cgroup freezer rw,freezer
155 146 0:32 / /run/lxcfs/controllers/pids rw,relatime shared:123 - cgroup pids rw,pids,release_agent=/run/cgmanager/agents/cgm-release-agent.pids
157 146 0:31 / /run/lxcfs/controllers/perf_event rw,relatime shared:125 - cgroup perf_event rw,perf_event,release_agent=/run/cgmanager/agents/cgm-release-agent.perf_event
159 146 0:30 / /run/lxcfs/controllers/cpu,cpuacct rw,relatime shared:127 - cgroup cpu,cpuacct rw,cpu,cpuacct
161 146 0:29 / /run/lxcfs/controllers/net_cls,net_prio rw,relatime shared:129 - cgroup net_cls,net_prio rw,net_cls,net_prio
163 146 0:28 / /run/lxcfs/controllers/devices rw,relatime shared:131 - cgroup devices rw,devices
165 146 0:27 / /run/lxcfs/controllers/memory rw,relatime shared:133 - cgroup memory rw,memory
167 146 0:26 / /run/lxcfs/controllers/cpuset rw,relatime shared:135 - cgroup cpuset rw,cpuset,clone_children
169 146 0:24 / /run/lxcfs/controllers/name=systemd rw,relatime shared:137 - cgroup name=systemd rw,xattr,release_agent=/lib/systemd/systemd-cgroups-agent,name=systemd
171 25 0:48 / /var/lib/lxcfs rw,nosuid,nodev,relatime shared:139 - fuse.lxcfs lxcfs rw,user_id=0,group_id=0,allow_other
176 63 0:41 /shmounts /var/lib/lxd/shmounts rw,noatime shared:43 - btrfs /dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/shmounts
177 72 0:41 /shmounts /mnt/lxd_images/shmounts rw,noatime shared:43 - btrfs /dev/mapper/wd2t--lvm--data-lxd_images rw,space_cache,subvolid=5,subvol=/shmounts
180 23 0:50 / /run/user/1000 rw,nosuid,nodev,relatime shared:143 - tmpfs tmpfs rw,size=1615856k,mode=700,uid=1000,gid=1000
292 42 0:55 / /proc/sys/fs/binfmt_misc rw,relatime shared:151 - binfmt_misc binfmt_misc rw
182 63 8:1 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-video rw,relatime master:37 - ext4 /dev/sda1 rw,data=ordered
183 72 8:1 / /mnt/lxd_images/devices/xenial-mythserver/disk.mnt-video rw,relatime shared:37 - ext4 /dev/sda1 rw,data=ordered
188 63 8:18 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-music rw,relatime master:40 - ext4 /dev/sdb2 rw,data=ordered
189 72 8:18 / /mnt/lxd_images/devices/xenial-mythserver/disk.mnt-music rw,relatime shared:40 - ext4 /dev/sdb2 rw,data=ordered
194 63 8:17 / /var/lib/lxd/devices/xenial-mythserver/disk.mnt-dvd_rips rw,relatime master:39 - ext4 /dev/sdb1 rw,data=ordered
195 72 8:17 / /mnt/lxd_images/devices/xenial-mythserver/disk.mnt-dvd_rips rw,relatime shared:39 - ext4 /dev/sdb1 rw,data=ordered
thanks for your help  Rob

Serge E. Hallyn
2016-06-30 00:37:13 UTC
Permalink
Post by Rob Edgerton
hi,I have the same problem (cgroups not working as expected) on a clean Xenial build (lxc PPA NOT installed, LXD not installed)In my case I have some Ubuntu Trusty containers I really need to use on Xenial, but they won't start because I use cgroups.If I change the existing containers to remove the "lxc.cgroup" clauses from config they start,
Please show the exact clauses you are using as well.

-serge
Continue reading on narkive:
Loading...