[lxc-users] setcap capabilities

[Mark Constable]

Outside a container on the host I can...

~ /sbin/setcap cap_net_bind_service=+ep /usr/bin/caddy
~ getcap /usr/bin/caddy
/usr/bin/caddy = cap_net_bind_service+ep

but inside a container I get...

~ /sbin/setcap cap_net_bind_service=+ep /usr/bin/caddy
Failed to set capabilities on file `/usr/bin/caddy' (Invalid argument)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file

What procedure should I follow to allow the above cap_net_bind_service=+ep to be
set inside a 2.0.0~beta1 container?

[Serge Hallyn]

Post by Mark Constable
Outside a container on the host I can...
~ /sbin/setcap cap_net_bind_service=+ep /usr/bin/caddy
~ getcap /usr/bin/caddy
/usr/bin/caddy = cap_net_bind_service+ep
but inside a container I get...
~ /sbin/setcap cap_net_bind_service=+ep /usr/bin/caddy
Failed to set capabilities on file `/usr/bin/caddy' (Invalid argument)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file
What procedure should I follow to allow the above cap_net_bind_service=+ep to be
set inside a 2.0.0~beta1 container?

If the container is in a user namespace, you cannot, until we get
the namespaced file capabilities going. You'll have to do it from
the initial user namespace.

If not in a user namespace, ... well it works for me, but
you may have to edit the files under /usr/share/lxc which get
lxc.include'd to make sure they're not dropping CAP_SETFCAP,
and check your apparmor/selinux policy. I'm not going more
into detail on that until we're sure you're not in a user
namespace :)

[Mark Constable]

Post by Mark Constable
but inside a container I get...
~ /sbin/setcap cap_net_bind_service=+ep /usr/bin/caddy
Failed to set capabilities on file `/usr/bin/caddy' (Invalid argument)

If not in a user namespace, ... well it works for me, but you may
have to edit the files under /usr/share/lxc which get lxc.include'd
to make sure they're not dropping CAP_SETFCAP, and check your
apparmor/selinux policy. I'm not going more into detail on that until
we're sure you're not in a user namespace :)

Woops, sorry, xenial host with a xenial lxd 2.0.0~beta2 unprivileged
container.

***

This could be a separate list post but it may involve a similar solution.

The above is my main concern but this is also related and that is when
I install certain packages (like courier-mta) it requires some suid
capabilities to suid some symlink'd binaries and was failing inside
this same unpriv container. The solution in this case was to set...

# for containers to allow suid exec
echo 0 > /proc/sys/fs/protected_hardlinks

on the host but that is going to be awkward for folks who do not happen
to know this "trick" meaning generally trying to install the courier-mta
package on unpriv containers is going to fail in an ugly way that messes
up package install/upgrades.

Any comment on how to make this easier to deal with?

[Serge Hallyn]

Post by Mark Constable
# for containers to allow suid exec
echo 0 > /proc/sys/fs/protected_hardlinks
on the host but that is going to be awkward for folks who do not happen
to know this "trick" meaning generally trying to install the courier-mta
package on unpriv containers is going to fail in an ugly way that messes
up package install/upgrades.
Any comment on how to make this easier to deal with?

I'm afraid not. It's the exact case which the authors of the
protected_hardlinks mechanism wanted to protect against...

Thanks for the response Serge but this "problem" all but makes unpriv
containers (xenial at least) unusable. Todays example...
Unpacking systemd (229-1ubuntu2) over (228-5ubuntu3) ...
unable to make backup link of './bin/systemctl' before installing new version: Operation not permitted

Are you using overlayfs clones? Or using a readonly mount of
the host's / ? Otherwise this shouldn't be happening. I can hardlink
/bin/systemctl just fine as root in an unprivileged container.

dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
addgroup: The group `systemd-journal' already exists as a system group. Exiting.
Failed to set capabilities on file `/usr/bin/systemd-detect-virt' (Invalid argument)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file
Processing triggers for ureadahead (0.100.0-19) ...
Processing triggers for dbus (1.10.6-1ubuntu1) ...
/var/cache/apt/archives/systemd_229-1ubuntu2_amd64.deb
E: Sub-process /usr/bin/dpkg returned an error code (1)

-serge

[Mark Constable]

Post by Mark Constable
echo 0 > /proc/sys/fs/protected_hardlinks

Thanks for the response Serge but this "problem" all but makes unpriv
containers (xenial at least) unusable. Todays example...
Unpacking systemd (229-1ubuntu2) over (228-5ubuntu3) ...
unable to make backup link of './bin/systemctl' before installing new version: Operation not permitted

Are you using overlayfs clones? Or using a readonly mount of the
host's / ?

No, nothing other than a stock standard launch on a btrfs host.

Otherwise this shouldn't be happening. I can hardlink
/bin/systemctl just fine as root in an unprivileged container.

Thanks for the clarification.

I'm using an old container that I have been upgrading for almost a year
now so goodness knows what state it really is in. I'll launch another
one from scratch and see if that fixes this issue without turning off
/proc/sys/fs/protected_hardlinks.

[Serge Hallyn]

Post by Mark Constable

Post by Mark Constable
echo 0 > /proc/sys/fs/protected_hardlinks

Thanks for the response Serge but this "problem" all but makes unpriv
containers (xenial at least) unusable. Todays example...
Unpacking systemd (229-1ubuntu2) over (228-5ubuntu3) ...
unable to make backup link of './bin/systemctl' before installing new version: Operation not permitted

Are you using overlayfs clones? Or using a readonly mount of the
host's / ?

No, nothing other than a stock standard launch on a btrfs host.

Otherwise this shouldn't be happening. I can hardlink
/bin/systemctl just fine as root in an unprivileged container.

Thanks for the clarification.
I'm using an old container that I have been upgrading for almost a year

What does ls -l /bin/systemctl show?

You can see the conditions under which the sysctl takes effect
in fs/namei.c:safe_hardlink_source()

Post by Mark Constable
now so goodness knows what state it really is in. I'll launch another
one from scratch and see if that fixes this issue without turning off
/proc/sys/fs/protected_hardlinks.

[Serge Hallyn]

Post by Serge Hallyn

Unpacking systemd (229-1ubuntu2) over (228-5ubuntu3) ...
unable to make backup link of './bin/systemctl' before installing new version: Operation not permitted

[...]
What does ls -l /bin/systemctl show?

~ ls -l /bin/systemctl
-rwxr-xr-x 1 root root 659848 Feb 14 22:41 /bin/systemctl
I did an "echo 0 > /proc/sys/fs/protected_hardlinks" on the host and
reran the update which proceeded and installed the rest of the package

... AFAICS this shouldn't be happening. It's not setuid, not a
non-regular file, not setgid, and you should be privileged over
the file. Could you try with a newer kernel, 3.16 or 4.2 or 4.4?

updates but along the way I got this...
Failed to set capabilities on file `/usr/bin/systemd-detect-virt' (Invalid argument)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file

Yes, again that is going to happen, but it should only be a warning,
the package should not fail to install, if it does it is a bug in
the systemd packaging.

~ ls -l /usr/bin/systemd-detect-virt
-rwxr-xr-x 1 root root 35248 Feb 14 22:41 /usr/bin/systemd-detect-virt
~ lsattr /usr/bin/systemd-detect-virt
---------------- /usr/bin/systemd-detect-virt
~ getcap -v /usr/bin/systemd-detect-virt
/usr/bin/systemd-detect-virt
Whereas on the xenial host I get...
~ getcap -v /usr/bin/systemd-detect-virt
/usr/bin/systemd-detect-virt = cap_dac_override,cap_sys_ptrace+ep
So is no one else reporting this problem when upgrading to systemd_229-1ubuntu2?

I get that message,

Installing new version of config file /etc/systemd/resolved.conf ...
addgroup: The group `systemd-journal' already exists as a system group. Exiting.
Failed to set capabilities on file `/usr/bin/systemd-detect-virt' (Invalid argument)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file
Setting up udev (229-1ubuntu2) ...
addgroup: The group `input' already exists as a system group. Exiting.
update-initramfs: deferring update (trigger activated)
Setting up libpam-systemd:amd64 (229-1ubuntu2) ...
Processing triggers for initramfs-tools (0.120ubuntu6) ...
***@upgr:~# echo $?
0

but it's just a warning as you can see ^.

[Mark Constable]

FWIW another package that requires setcap. This is the first one I've seen
that falls back to setuid OOTB.

Setting up mtr-tiny (0.86-1) ...
Failed to set capabilities on file `/usr/bin/mtr' (Invalid argument)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file
Setcap failed on /usr/bin/mtr, falling back to setuid

[Serge Hallyn]

Post by Mark Constable
FWIW another package that requires setcap. This is the first one I've seen
that falls back to setuid OOTB.
Setting up mtr-tiny (0.86-1) ...
Failed to set capabilities on file `/usr/bin/mtr' (Invalid argument)
The value of the capability argument is not permitted for a file. Or the file is not a regular (non-symlink) file
Setcap failed on /usr/bin/mtr, falling back to setuid

Nice. Although, looking at the package, in the setuid-root case
it pre-opens network then setuids to the caller's uid. But in
the file capabilities case it does the same thing - but setuid
in that case won't do anything (because it's not switching gid
or uid to/from root) and so won't drop the cap_net_raw capability.
Ideally it would drop those by hand when it knows how to.

[Mark Constable]

Post by Mark Constable
but inside a container I get...
~ /sbin/setcap cap_net_bind_service=+ep /usr/bin/caddy
Failed to set capabilities on file `/usr/bin/caddy' (Invalid argument)

If not in a user namespace, ... well it works for me, but you may
have to edit the files under /usr/share/lxc which get lxc.include'd
to make sure they're not dropping CAP_SETFCAP, and check your
apparmor/selinux policy. I'm not going more into detail on that until
we're sure you're not in a user namespace :)

xenial host with a xenial lxd 2.0.0~beta2 unprivileged container

lxd 2.0.0~beta3 now. Can you spare a moment for a little more detail please?

[Serge Hallyn]

Post by Mark Constable

Post by Mark Constable
but inside a container I get...
~ /sbin/setcap cap_net_bind_service=+ep /usr/bin/caddy
Failed to set capabilities on file `/usr/bin/caddy' (Invalid argument)

If not in a user namespace, ... well it works for me, but you may
have to edit the files under /usr/share/lxc which get lxc.include'd
to make sure they're not dropping CAP_SETFCAP, and check your
apparmor/selinux policy. I'm not going more into detail on that until
we're sure you're not in a user namespace :)

xenial host with a xenial lxd 2.0.0~beta2 unprivileged container

lxd 2.0.0~beta3 now. Can you spare a moment for a little more detail please?

Sorry apparently I was not clear. If you are in an unprivileged
container, there is nothing you can do to set file capabilities, apart
from writing the kernel patch (and libcap patch) to make namespaaced
capabilities happen.

However any packages in ubuntu should not break due to not being able
to set file capabilities. I want the namespaced capabilties so we can
stop having fallbacks, but right now if that happens then it is valid
to file a bug against the package which is failing to install.