Discussion:
lxc-start fails at apparmor detection
(too old to reply)
Tom Weber
2014-08-05 11:53:58 UTC
Permalink
Hello,

my setup:
debian7
lxc-1.0.4 from debian testing
vanilla kernel.org kernel 3.14.14

i'm new to lxc and apparmor, so this took me a couple of hours to
figure:
lxc-start won't assign an apparmor-profile to a container since it's
test for apparmor will always fail on my setup:
in src/lxc/lsm/apparmor:
the apparmor_enabled() tests for AA_MOUNT_RESTR
(/sys/kernel/security/apparmor/features/mount/mask) first, which will
never exist without that apparmor mount patch in the kernel.

commenting out that test gives me apparmor functionality (except for
that mount feature of course).

Is that intentional or just an ancient relict?
I'd prefer to have apparmor profile support without mount restrictions
over no apparmor profile support at all. apparmor gives me warnings
like:

Warning from /etc/apparmor.d/lxc-containers (/etc/apparmor.d/lxc-containers line 8): profile lxc-container-default mount rules not enforced

when starting up, which is what I expect and something I can deal with
as admin. I think lxc-start should activate the requested profile
anyway.

Oh, and a little log message wether lxc-start detected apparmor or not
and activates it would be _very_ helpfull :)

related question: dropping sys_admin cap for the container should render
all the mount protections from apparmor unnecessary, right?

Regards,
Tom
Serge Hallyn
2014-08-05 16:07:40 UTC
Permalink
Post by Tom Weber
Hello,
debian7
lxc-1.0.4 from debian testing
vanilla kernel.org kernel 3.14.14
i'm new to lxc and apparmor, so this took me a couple of hours to
lxc-start won't assign an apparmor-profile to a container since it's
the apparmor_enabled() tests for AA_MOUNT_RESTR
(/sys/kernel/security/apparmor/features/mount/mask) first, which will
never exist without that apparmor mount patch in the kernel.
commenting out that test gives me apparmor functionality (except for
that mount feature of course).
Is that intentional or just an ancient relict?
I'd prefer to have apparmor profile support without mount restrictions
over no apparmor profile support at all. apparmor gives me warnings
Warning from /etc/apparmor.d/lxc-containers (/etc/apparmor.d/lxc-containers line 8): profile lxc-container-default mount rules not enforced
when starting up, which is what I expect and something I can deal with
as admin. I think lxc-start should activate the requested profile
anyway.
Oh, and a little log message wether lxc-start detected apparmor or not
and activates it would be _very_ helpfull :)
related question: dropping sys_admin cap for the container should render
all the mount protections from apparmor unnecessary, right?
What you say makes sense. What do you think of the following (untested)
patch?
Tom Weber
2014-08-05 22:12:51 UTC
Permalink
Post by Serge Hallyn
What you say makes sense. What do you think of the following (untested)
patch?
From 05864ae7f8b42724fb15ddea8a6d3d3ea9cf8749 Mon Sep 17 00:00:00 2001
From: Serge Hallyn <serge.hallyn at ubuntu.com>
Date: Tue, 5 Aug 2014 11:01:55 -0500
Subject: [PATCH 1/1] apparmor: only warn if mount restrictions lacking
Up to now we've refused to load apparmor profiles if mount
restrictions are missing. With this patch, we'll only warn
but continue loading the profile.
Lack of mount restrictions allows malicious container users
to work around file restrictions by say remounting /proc.
However, as Tom points out containers with no cap_sys_admin
are not vulnerable to this. So it doesn't make sense to not
allow them to use apparmor as well.
Reported-by: Tom Weber <l_lxc-users at mail2news.4t2.com>
Signed-off-by: Serge Hallyn <serge.hallyn at ubuntu.com>
---
src/lxc/lsm/apparmor.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/src/lxc/lsm/apparmor.c b/src/lxc/lsm/apparmor.c
index f4c8d26..e730aba 100644
--- a/src/lxc/lsm/apparmor.c
+++ b/src/lxc/lsm/apparmor.c
@@ -48,8 +48,10 @@ static int apparmor_enabled(void)
int ret;
ret = stat(AA_MOUNT_RESTR, &statbuf);
- if (ret != 0)
- return 0;
+ if (ret != 0) {
+ WARN("WARNING: Apparmor ount restrictions missing from kernel");
+ WARN("WARNING: mount restrictions will not be enforced");
+ }
fin = fopen(AA_ENABLED_FILE, "r");
if (!fin)
return 0;
The patch works in the regard that the container starts and the apparmor
profile is set.
But I can't find the Warning message anywhere (tried lxc-start -n webv1
-d -l DEBUG) - but maybe thats a more general problem. Oh, and there is
a typo: Apparmor ount

My opinion as an admin is that this check isn't needed in lxc itself.
Apparmor spits a warning during aa lxc-profile loading - sane admins
wouldn't ignore this.
If one messes with the aa lxc-profiles and disables the mount
restrictions there, your check wont help (or report) anything - even on
a kernel with mount restriction patch.
All you can do is provide sane aa profiles in the lxc package - the rest
is aa related business, not lxc related.
But thats just my oponion.

Thanks alot for the quick patch!
Tom
Serge Hallyn
2014-08-05 23:34:35 UTC
Permalink
Post by Tom Weber
Post by Serge Hallyn
What you say makes sense. What do you think of the following (untested)
patch?
From 05864ae7f8b42724fb15ddea8a6d3d3ea9cf8749 Mon Sep 17 00:00:00 2001
From: Serge Hallyn <serge.hallyn at ubuntu.com>
Date: Tue, 5 Aug 2014 11:01:55 -0500
Subject: [PATCH 1/1] apparmor: only warn if mount restrictions lacking
Up to now we've refused to load apparmor profiles if mount
restrictions are missing. With this patch, we'll only warn
but continue loading the profile.
Lack of mount restrictions allows malicious container users
to work around file restrictions by say remounting /proc.
However, as Tom points out containers with no cap_sys_admin
are not vulnerable to this. So it doesn't make sense to not
allow them to use apparmor as well.
Reported-by: Tom Weber <l_lxc-users at mail2news.4t2.com>
Signed-off-by: Serge Hallyn <serge.hallyn at ubuntu.com>
---
src/lxc/lsm/apparmor.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/src/lxc/lsm/apparmor.c b/src/lxc/lsm/apparmor.c
index f4c8d26..e730aba 100644
--- a/src/lxc/lsm/apparmor.c
+++ b/src/lxc/lsm/apparmor.c
@@ -48,8 +48,10 @@ static int apparmor_enabled(void)
int ret;
ret = stat(AA_MOUNT_RESTR, &statbuf);
- if (ret != 0)
- return 0;
+ if (ret != 0) {
+ WARN("WARNING: Apparmor ount restrictions missing from kernel");
+ WARN("WARNING: mount restrictions will not be enforced");
+ }
fin = fopen(AA_ENABLED_FILE, "r");
if (!fin)
return 0;
The patch works in the regard that the container starts and the apparmor
profile is set.
But I can't find the Warning message anywhere (tried lxc-start -n webv1
-d -l DEBUG) - but maybe thats a more general problem. Oh, and there is
a typo: Apparmor ount
My opinion as an admin is that this check isn't needed in lxc itself.
Apparmor spits a warning during aa lxc-profile loading - sane admins
wouldn't ignore this.
We're not just talking about "sane admins" though. We're talking about
everyday users using containers. And they're not building their own
misconfigured kernels. It happens, certainly while using the development
release, that you get a kernel for which the apparmor set wasn't ready
yet and mount restrictions weren't ready.

Maybe the patch should be modified to only allow the container to
proceed if cap_sys_admin is being dropped.

Or maybe it's fine as is. I'm feeling undecided.
Post by Tom Weber
If one messes with the aa lxc-profiles and disables the mount
restrictions there, your check wont help (or report) anything - even on
a kernel with mount restriction patch.
All you can do is provide sane aa profiles in the lxc package - the rest
is aa related business, not lxc related.
But thats just my oponion.
Thanks alot for the quick patch!
Tom
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Tom Weber
2014-08-06 09:09:21 UTC
Permalink
Post by Serge Hallyn
Post by Tom Weber
The patch works in the regard that the container starts and the apparmor
profile is set.
But I can't find the Warning message anywhere (tried lxc-start -n webv1
-d -l DEBUG) - but maybe thats a more general problem. Oh, and there is
a typo: Apparmor ount
My opinion as an admin is that this check isn't needed in lxc itself.
Apparmor spits a warning during aa lxc-profile loading - sane admins
wouldn't ignore this.
We're not just talking about "sane admins" though. We're talking about
everyday users using containers. And they're not building their own
misconfigured kernels. It happens, certainly while using the development
release, that you get a kernel for which the apparmor set wasn't ready
yet and mount restrictions weren't ready.
Maybe the patch should be modified to only allow the container to
proceed if cap_sys_admin is being dropped.
So if I _want_ an insecure container with cap_sys_admin (for whatever
reason like testing or development - and yes sometimes I might want
this!) you'd force me to install an apparmor mount supported kernel
where i'd comment out the mount rules in the apparmor profile? Just to
make that thing start?

Just because there's a feature in the kernel (and it's nothing else your
stat does) doesn't mean that the other end of the system that's
responsible for enforcing/using it does really use it. This test
implies security where no security is.

I dont think a readable /proc/kcore inside a container or access to
dmesg is very secure either - as in the default config.
I could mount proc on /proc_insecure and create whatever /dev/ nodes I
like anywhere I want and lxc wouldn't warn me about this at all.
But you wouldn't allow me to start a container if the _kernel_ lacks
aa-mount support and i don't drop cap_sys_admin? Really?

This test belongs in lxc-checkconfig and should print out a big fat
warning - right now it's not even mentioned there.

Regards,
Tom
Serge Hallyn
2014-08-06 14:51:18 UTC
Permalink
Post by Tom Weber
Post by Serge Hallyn
Post by Tom Weber
The patch works in the regard that the container starts and the apparmor
profile is set.
But I can't find the Warning message anywhere (tried lxc-start -n webv1
-d -l DEBUG) - but maybe thats a more general problem. Oh, and there is
a typo: Apparmor ount
My opinion as an admin is that this check isn't needed in lxc itself.
Apparmor spits a warning during aa lxc-profile loading - sane admins
wouldn't ignore this.
We're not just talking about "sane admins" though. We're talking about
everyday users using containers. And they're not building their own
misconfigured kernels. It happens, certainly while using the development
release, that you get a kernel for which the apparmor set wasn't ready
yet and mount restrictions weren't ready.
Maybe the patch should be modified to only allow the container to
proceed if cap_sys_admin is being dropped.
So if I _want_ an insecure container with cap_sys_admin (for whatever
reason like testing or development - and yes sometimes I might want
Well in the end it's open source and you can comment out two lines and
build your own :)

We could also add a set of security flags to specify what the container
considers required. So it might include 'selinux must be enabled', and
'mount restrictions in apparmor must be enabled.'
Post by Tom Weber
this!) you'd force me to install an apparmor mount supported kernel
where i'd comment out the mount rules in the apparmor profile? Just to
make that thing start?
There are many many more people who might adversely affect their
system by not having that. The *defaults* should be safe, and the
burden should be on the one who wants to run insecurely to enable
that. I admit the burden shouldn't necessarily be "rebuild the
package". I'm liking the idea of security flags. I think they
merit some more discussion first, to make sure we get an API that'll
continue to be useful. Hopefully Dwight (cc:d) wlil have some input.
But worst case, we could just make it an explicit

lxc.apparmor = [safe|unsafe|off]

Maybe I'll send a patch for that later today.

It's worth considering also whether there is anything we require from
apparmor for the unprivileged containers. If there is then we cannot
allow an unprivileged container to set lxc.apparmor = unsafe|off. The
only thing I can think of is if there happens to be an 0-day in the
handling of some namespaced procfile that results in privilege
escalation... I'm cc:ing jjohansen for input from him.
Post by Tom Weber
Just because there's a feature in the kernel (and it's nothing else your
stat does) doesn't mean that the other end of the system that's
responsible for enforcing/using it does really use it. This test
implies security where no security is.
Are you arguing that I shouldn't check whether the feature is
enabled bc it might be buggy and not working anyway?
Post by Tom Weber
I dont think a readable /proc/kcore inside a container or access to
dmesg is very secure either - as in the default config.
I could mount proc on /proc_insecure and create whatever /dev/ nodes I
Not from inside the container.
Post by Tom Weber
like anywhere I want and lxc wouldn't warn me about this at all.
Yes, you can configure it however you want. The difference is that
the kernel not having the mount restrictions support means it has
*incomplete* apparmor support. You've requested an apparmor profile,
and the kernel cannot completely implement it.

Really the solution to all this is to get the mount restrictions into
the upstream kernel.
Post by Tom Weber
But you wouldn't allow me to start a container if the _kernel_ lacks
aa-mount support and i don't drop cap_sys_admin? Really?
Yes, because the container was designed a certain way to be safe,
and a piece making up that design is missing.

If you simply want to disable apparmor, you can always set

lxc.aa_profile = unconfined.
Post by Tom Weber
This test belongs in lxc-checkconfig and should print out a big fat
warning - right now it's not even mentioned there.
Agreed. Patches welcome.

-serge
Tom Weber
2014-08-06 17:15:11 UTC
Permalink
Post by Serge Hallyn
Post by Tom Weber
Post by Serge Hallyn
Post by Tom Weber
The patch works in the regard that the container starts and the apparmor
profile is set.
But I can't find the Warning message anywhere (tried lxc-start -n webv1
-d -l DEBUG) - but maybe thats a more general problem. Oh, and there is
a typo: Apparmor ount
My opinion as an admin is that this check isn't needed in lxc itself.
Apparmor spits a warning during aa lxc-profile loading - sane admins
wouldn't ignore this.
We're not just talking about "sane admins" though. We're talking about
everyday users using containers. And they're not building their own
misconfigured kernels. It happens, certainly while using the development
release, that you get a kernel for which the apparmor set wasn't ready
yet and mount restrictions weren't ready.
Maybe the patch should be modified to only allow the container to
proceed if cap_sys_admin is being dropped.
So if I _want_ an insecure container with cap_sys_admin (for whatever
reason like testing or development - and yes sometimes I might want
Well in the end it's open source and you can comment out two lines and
build your own :)
We could also add a set of security flags to specify what the container
considers required. So it might include 'selinux must be enabled', and
'mount restrictions in apparmor must be enabled.'
none of these would be usefull if you don't also check for a correctly
configured apparmor profile.
So you tell me that i can always recompile if i don't want your security
checks, but everyone can render the security you check for completely
useless by editing an apparmor profile - and none of your checks would
even notify.
Post by Serge Hallyn
Post by Tom Weber
this!) you'd force me to install an apparmor mount supported kernel
where i'd comment out the mount rules in the apparmor profile? Just to
make that thing start?
There are many many more people who might adversely affect their
system by not having that. The *defaults* should be safe, and the
burden should be on the one who wants to run insecurely to enable
that. I admit the burden shouldn't necessarily be "rebuild the
package". I'm liking the idea of security flags. I think they
merit some more discussion first, to make sure we get an API that'll
continue to be useful. Hopefully Dwight (cc:d) wlil have some input.
But worst case, we could just make it an explicit
I agree that the default setting should be safe. 3 days ago your default
on a debian or vanilla kernel built system was to _silently_ ignore
apparmor _completely_ - not even the slightest protection through
apparmor was possible _at all_! I wonder how many people run lxc and
think that apparmor is active while it isn't at all due to this.

But it should not be a bourdon to me to do things the way I want, even
if I decide to have an insecure setup. I'd buy/use Macs if i'd want the
developer to limit me to the things he thinks I should be allowed to do.
Post by Serge Hallyn
lxc.apparmor = [safe|unsafe|off]
that sounds nice, but the problem is that you can't say anything about
apparmor security for a container if you don't check the apparmor
profile for correctness. And thats out of the scope for lxc.

so people will turn on lxc.apparmor = safe.
lxc won't start and complain about security and apparmor.
so they'll try with lxc.apparmor = unsafe and it will start.
now people will start messing up apparmor profiles until they figure
that they need a kernel with aa-mount support.
they turn on lxc.apparmor = safe and the aa profiles are still messed
up.
but lxc won't complain about security anymore and they'll end up feeling
safe with a messed up container.
you're giving a false sense of security here.

Or think about someone who creates (maybe by mistake) and uses an all
allow (or partly insecure) apparmor profile.
lxc.apparmor = safe would spit in his face on a debian system
lxc.apparmor = safe wouldn't complain at all on an ubuntu system
that makes no sense for me.

if you implement a 'safe' flag that's supposed to prevent an unsafe
container from starting, you better implement it correctly or not at
all.
It's like a button labeled: make this container apparmor safe. people
push it and expect the container to be apparmor safe while it doesn't
need to be.
Post by Serge Hallyn
Maybe I'll send a patch for that later today.
It's worth considering also whether there is anything we require from
apparmor for the unprivileged containers. If there is then we cannot
allow an unprivileged container to set lxc.apparmor = unsafe|off. The
only thing I can think of is if there happens to be an 0-day in the
handling of some namespaced procfile that results in privilege
escalation... I'm cc:ing jjohansen for input from him.
Post by Tom Weber
Just because there's a feature in the kernel (and it's nothing else your
stat does) doesn't mean that the other end of the system that's
responsible for enforcing/using it does really use it. This test
implies security where no security is.
Are you arguing that I shouldn't check whether the feature is
enabled bc it might be buggy and not working anyway?
basically yes. it could be misconfigured in apparmor as well.
The availability of a feature doesn't tell you if it's used correctly.
Post by Serge Hallyn
Post by Tom Weber
I dont think a readable /proc/kcore inside a container or access to
dmesg is very secure either - as in the default config.
I could mount proc on /proc_insecure and create whatever /dev/ nodes I
Not from inside the container.
dmesg and /proc/kcore? both are readable from inside - in the default
config.

and i was talking about mounting /proc* or /sys* whatever way i want in
lxc.conainer config. You allow me that, but you want to force me on an
apparmor mount patched kernel if i want to use apparmor.
Post by Serge Hallyn
Post by Tom Weber
like anywhere I want and lxc wouldn't warn me about this at all.
Yes, you can configure it however you want. The difference is that
the kernel not having the mount restrictions support means it has
*incomplete* apparmor support. You've requested an apparmor profile,
and the kernel cannot completely implement it.
Yes, i have requested apparmor support from lxc. lxc doesn't know
anything about my apparmor profil I requested - _THATS_ my whole point.
Maybe I dont have mount limiting features in the profile I configured
lxc to use. Thats entirely apparmor's business, not lxc.
Making asumptions in lxc-start what I should/could/would have to
configure in my apparmor profile is wrong.

If apparmor cannot fullfill what I request in the apparmor profile, it's
up to apparmor to complain about this - and apparmor does so.
Post by Serge Hallyn
Really the solution to all this is to get the mount restrictions into
the upstream kernel.
that would help indeed.
there could as well be other ways (maybe kernel patches) far outside the
scope of apparmor to solve that mount problem - and again, your check
for that feature wouldn't do no good.
Post by Serge Hallyn
Post by Tom Weber
But you wouldn't allow me to start a container if the _kernel_ lacks
aa-mount support and i don't drop cap_sys_admin? Really?
Yes, because the container was designed a certain way to be safe,
and a piece making up that design is missing.
Your default setup can be designed to be safe.
But you cant design them to be safe in every possible allowed setup - It
just happened that you forbid me my personal (and most likely secure)
setup by your security checks - whereas, otoh you do nothing to prevent
people from running insecure setups.
Post by Serge Hallyn
If you simply want to disable apparmor, you can always set
lxc.aa_profile = unconfined.
I do want to use lxc containers the way I (as an experienced admin)
want. I don't want lxc to forcefeed me some kind of security.
I do want to be able to run containers, with or without apparmor
protection (that means with or without apparmor mount protection), with
or without cap_sys_admin in any imaginable combination.
Post by Serge Hallyn
This test belongs in lxc-checkconfig and should print out a big fat
Post by Tom Weber
warning - right now it's not even mentioned there.
Agreed. Patches welcome.
I guess so :) If I only had more time and wasn't out of coding for like
20years.
BTW: does lxc-start start a container even if any of the features
lxc-checkconfig checks for are missing? Could such a container be
secure? :)

Regards,
Tom
Serge Hallyn
2014-08-06 17:47:37 UTC
Permalink
Post by Tom Weber
Post by Serge Hallyn
Post by Tom Weber
Post by Serge Hallyn
Post by Tom Weber
The patch works in the regard that the container starts and the apparmor
profile is set.
But I can't find the Warning message anywhere (tried lxc-start -n webv1
-d -l DEBUG) - but maybe thats a more general problem. Oh, and there is
a typo: Apparmor ount
My opinion as an admin is that this check isn't needed in lxc itself.
Apparmor spits a warning during aa lxc-profile loading - sane admins
wouldn't ignore this.
We're not just talking about "sane admins" though. We're talking about
everyday users using containers. And they're not building their own
misconfigured kernels. It happens, certainly while using the development
release, that you get a kernel for which the apparmor set wasn't ready
yet and mount restrictions weren't ready.
Maybe the patch should be modified to only allow the container to
proceed if cap_sys_admin is being dropped.
So if I _want_ an insecure container with cap_sys_admin (for whatever
reason like testing or development - and yes sometimes I might want
Well in the end it's open source and you can comment out two lines and
build your own :)
We could also add a set of security flags to specify what the container
considers required. So it might include 'selinux must be enabled', and
'mount restrictions in apparmor must be enabled.'
none of these would be usefull if you don't also check for a correctly
configured apparmor profile.
So you tell me that i can always recompile if i don't want your security
checks, but everyone can render the security you check for completely
useless by editing an apparmor profile - and none of your checks would
even notify.
Post by Serge Hallyn
Post by Tom Weber
this!) you'd force me to install an apparmor mount supported kernel
where i'd comment out the mount rules in the apparmor profile? Just to
make that thing start?
There are many many more people who might adversely affect their
system by not having that. The *defaults* should be safe, and the
burden should be on the one who wants to run insecurely to enable
that. I admit the burden shouldn't necessarily be "rebuild the
package". I'm liking the idea of security flags. I think they
merit some more discussion first, to make sure we get an API that'll
continue to be useful. Hopefully Dwight (cc:d) wlil have some input.
But worst case, we could just make it an explicit
I agree that the default setting should be safe. 3 days ago your default
on a debian or vanilla kernel built system was to _silently_ ignore
apparmor _completely_ - not even the slightest protection through
Yeah that shouldn't be the case, they should fail to start.
Post by Tom Weber
apparmor was possible _at all_! I wonder how many people run lxc and
think that apparmor is active while it isn't at all due to this.
But it should not be a bourdon to me to do things the way I want, even
But it should! It just should be a minimal burden.
Post by Tom Weber
if I decide to have an insecure setup. I'd buy/use Macs if i'd want the
developer to limit me to the things he thinks I should be allowed to do.
I don't want to limit you. I want to provide you a configurable option
to do what you want. We're just still discussing what that should look
like.

But you come back to my earlier point nicely - on a mac you wouldn't
have to option to recompile when they've failed to provide you the
option. I fully agree you shouldn't have to recompile. I'm just
saying that until we work this out, you have that option.
Post by Tom Weber
Post by Serge Hallyn
lxc.apparmor = [safe|unsafe|off]
that sounds nice, but the problem is that you can't say anything about
apparmor security for a container if you don't check the apparmor
profile for correctness. And thats out of the scope for lxc.
Here I think you are missing the point. I don't want to guarantee that
your container is safe. I want to guarantee that when you use the
defaults, you are getting a certain level of safety. If you don't use
the defaults, you should know what you're doing of course.
Post by Tom Weber
so people will turn on lxc.apparmor = safe.
lxc won't start and complain about security and apparmor.
so they'll try with lxc.apparmor = unsafe and it will start.
now people will start messing up apparmor profiles until they figure
that they need a kernel with aa-mount support.
they turn on lxc.apparmor = safe and the aa profiles are still messed
up.
but lxc won't complain about security anymore and they'll end up feeling
safe with a messed up container.
you're giving a false sense of security here.
Or think about someone who creates (maybe by mistake) and uses an all
allow (or partly insecure) apparmor profile.
lxc.apparmor = safe would spit in his face on a debian system
lxc.apparmor = safe wouldn't complain at all on an ubuntu system
that makes no sense for me.
if you implement a 'safe' flag that's supposed to prevent an unsafe
container from starting, you better implement it correctly or not at
all.
You're arguing with my choice of terms. Offer better terms then :) I
agree safe is probably misleading. We want something connoting
"do not start if not fully available in the kernel".

Perhaps it should be

lxc.apparmor.require_mount = [0|1], with default being 1?
Post by Tom Weber
It's like a button labeled: make this container apparmor safe. people
push it and expect the container to be apparmor safe while it doesn't
need to be.
Post by Serge Hallyn
Maybe I'll send a patch for that later today.
It's worth considering also whether there is anything we require from
apparmor for the unprivileged containers. If there is then we cannot
allow an unprivileged container to set lxc.apparmor = unsafe|off. The
only thing I can think of is if there happens to be an 0-day in the
handling of some namespaced procfile that results in privilege
escalation... I'm cc:ing jjohansen for input from him.
Post by Tom Weber
Just because there's a feature in the kernel (and it's nothing else your
stat does) doesn't mean that the other end of the system that's
responsible for enforcing/using it does really use it. This test
implies security where no security is.
Are you arguing that I shouldn't check whether the feature is
enabled bc it might be buggy and not working anyway?
basically yes. it could be misconfigured in apparmor as well.
The availability of a feature doesn't tell you if it's used correctly.
Likewise there are probably scores of kernel exploits in the lastest
kernel. That doesn't mean that if selinux isn't enabled sestatus should
lie to you and say your policy is enforcing.
Post by Tom Weber
Post by Serge Hallyn
Post by Tom Weber
I dont think a readable /proc/kcore inside a container or access to
dmesg is very secure either - as in the default config.
I could mount proc on /proc_insecure and create whatever /dev/ nodes I
Not from inside the container.
dmesg and /proc/kcore? both are readable from inside - in the default
config.
and i was talking about mounting /proc* or /sys* whatever way i want in
lxc.conainer config. You allow me that, but you want to force me on an
Again I'm talking about the defaults.
Post by Tom Weber
apparmor mount patched kernel if i want to use apparmor.
Post by Serge Hallyn
Post by Tom Weber
like anywhere I want and lxc wouldn't warn me about this at all.
Yes, you can configure it however you want. The difference is that
the kernel not having the mount restrictions support means it has
*incomplete* apparmor support. You've requested an apparmor profile,
and the kernel cannot completely implement it.
Yes, i have requested apparmor support from lxc. lxc doesn't know
anything about my apparmor profil I requested - _THATS_ my whole point.
Maybe I dont have mount limiting features in the profile I configured
lxc to use. Thats entirely apparmor's business, not lxc.
Making asumptions in lxc-start what I should/could/would have to
configure in my apparmor profile is wrong.
If apparmor cannot fullfill what I request in the apparmor profile, it's
up to apparmor to complain about this - and apparmor does so.
Post by Serge Hallyn
Really the solution to all this is to get the mount restrictions into
the upstream kernel.
that would help indeed.
there could as well be other ways (maybe kernel patches) far outside the
scope of apparmor to solve that mount problem - and again, your check
for that feature wouldn't do no good.
Post by Serge Hallyn
Post by Tom Weber
But you wouldn't allow me to start a container if the _kernel_ lacks
aa-mount support and i don't drop cap_sys_admin? Really?
Yes, because the container was designed a certain way to be safe,
and a piece making up that design is missing.
Your default setup can be designed to be safe.
But you cant design them to be safe in every possible allowed setup - It
just happened that you forbid me my personal (and most likely secure)
setup by your security checks - whereas, otoh you do nothing to prevent
people from running insecure setups.
Post by Serge Hallyn
If you simply want to disable apparmor, you can always set
lxc.aa_profile = unconfined.
I do want to use lxc containers the way I (as an experienced admin)
want. I don't want lxc to forcefeed me some kind of security.
I do want to be able to run containers, with or without apparmor
protection (that means with or without apparmor mount protection), with
or without cap_sys_admin in any imaginable combination.
Post by Serge Hallyn
This test belongs in lxc-checkconfig and should print out a big fat
Post by Tom Weber
warning - right now it's not even mentioned there.
Agreed. Patches welcome.
I guess so :) If I only had more time and wasn't out of coding for like
20years.
BTW: does lxc-start start a container even if any of the features
lxc-checkconfig checks for are missing? Could such a container be
secure? :)
Regards,
Tom
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Tom Weber
2014-08-07 13:57:23 UTC
Permalink
Post by Serge Hallyn
Post by Tom Weber
Post by Serge Hallyn
Post by Tom Weber
Post by Serge Hallyn
Post by Tom Weber
The patch works in the regard that the container starts and the apparmor
profile is set.
But I can't find the Warning message anywhere (tried lxc-start -n webv1
-d -l DEBUG) - but maybe thats a more general problem. Oh, and there is
a typo: Apparmor ount
My opinion as an admin is that this check isn't needed in lxc itself.
Apparmor spits a warning during aa lxc-profile loading - sane admins
wouldn't ignore this.
We're not just talking about "sane admins" though. We're talking about
everyday users using containers. And they're not building their own
misconfigured kernels. It happens, certainly while using the development
release, that you get a kernel for which the apparmor set wasn't ready
yet and mount restrictions weren't ready.
Maybe the patch should be modified to only allow the container to
proceed if cap_sys_admin is being dropped.
So if I _want_ an insecure container with cap_sys_admin (for whatever
reason like testing or development - and yes sometimes I might want
Well in the end it's open source and you can comment out two lines and
build your own :)
We could also add a set of security flags to specify what the container
considers required. So it might include 'selinux must be enabled', and
'mount restrictions in apparmor must be enabled.'
none of these would be usefull if you don't also check for a correctly
configured apparmor profile.
So you tell me that i can always recompile if i don't want your security
checks, but everyone can render the security you check for completely
useless by editing an apparmor profile - and none of your checks would
even notify.
Post by Serge Hallyn
Post by Tom Weber
this!) you'd force me to install an apparmor mount supported kernel
where i'd comment out the mount rules in the apparmor profile? Just to
make that thing start?
There are many many more people who might adversely affect their
system by not having that. The *defaults* should be safe, and the
burden should be on the one who wants to run insecurely to enable
that. I admit the burden shouldn't necessarily be "rebuild the
package". I'm liking the idea of security flags. I think they
merit some more discussion first, to make sure we get an API that'll
continue to be useful. Hopefully Dwight (cc:d) wlil have some input.
But worst case, we could just make it an explicit
I agree that the default setting should be safe. 3 days ago your default
on a debian or vanilla kernel built system was to _silently_ ignore
apparmor _completely_ - not even the slightest protection through
Yeah that shouldn't be the case, they should fail to start.
They should do what I told them - use the apparmor profile i specified,
not rely on a silly security check and ignore it completely.
Post by Serge Hallyn
Post by Tom Weber
apparmor was possible _at all_! I wonder how many people run lxc and
think that apparmor is active while it isn't at all due to this.
But it should not be a bourdon to me to do things the way I want, even
But it should! It just should be a minimal burden.
No. Software should help me to do things the right way, not get into my
way if i prefer to do things the author didn't think of.
Post by Serge Hallyn
Post by Tom Weber
if I decide to have an insecure setup. I'd buy/use Macs if i'd want the
developer to limit me to the things he thinks I should be allowed to do.
I don't want to limit you. I want to provide you a configurable option
to do what you want. We're just still discussing what that should look
like.
And I'm trying to explain you why you can't and shouldn't check other
components this way.
Post by Serge Hallyn
But you come back to my earlier point nicely - on a mac you wouldn't
have to option to recompile when they've failed to provide you the
option. I fully agree you shouldn't have to recompile. I'm just
saying that until we work this out, you have that option.
Post by Tom Weber
Post by Serge Hallyn
lxc.apparmor = [safe|unsafe|off]
that sounds nice, but the problem is that you can't say anything about
apparmor security for a container if you don't check the apparmor
profile for correctness. And thats out of the scope for lxc.
Here I think you are missing the point. I don't want to guarantee that
your container is safe. I want to guarantee that when you use the
defaults, you are getting a certain level of safety. If you don't use
the defaults, you should know what you're doing of course.
Your package is called lxc - it's not called apparmor. The defaults are
what the distribution packages together.
lxc itself can't provide secure containers (yet). There is nothing you
can do about this - you can only get lxc's job right.
apparmor itself can help increase security - with or without mount
patching the kernel - but thats out of the scope of lxc to judge on
apparmor.
there might be other ways to work around the mount problem (maybe a
cap_mount patch in the kernel) - and you don't know nothing about these
things.
Yet you imply that either the setup is insecure or that you care that
much about security that everyone thinks the system is secure if they
turn on these switches and the software doesn't complain - both is
wrong.

It's up to the distribution to pull the various components together and
create a sane and safe default solution for the regular ubuntu, debian,
redhat whatever user. All you can do is provide sane default profiles
for the various parts like apparmor or selinux or whatever. But it's
entirely the distribution's job to verify and implement whatever makes
sense.
A check like you do for that mount feature - and actions based on it -
just lead into some kind of dependency hell where one or a distribution
can't easily build the system the way they want.

And 'security' checks that give different results on different
platforms/setups even though the real security is the same are just
wrong.

If the someone chooses to install it's own stuff and packages it's
always up to them to take care about a secure setup - you're lost, you
can't check, you can't enforce or help with checks that say nothing
because you simply don't know anything of the intentions or setup.

Linux systems are build from many packages. Each should get it's own
stuff right. It's the system builder / distributions job to deliver a
secure system with sane defaults - not that of a package.

Does rsync check for misconfigured/insecure/missing ssh
settings/features when relying on ssh as transport for filetransfers?
No.

What's next? A mail reader that tells me that my X11 setup is insecure
for whatever reason and wont start because someone might steal my
password?
Post by Serge Hallyn
Post by Tom Weber
so people will turn on lxc.apparmor = safe.
lxc won't start and complain about security and apparmor.
so they'll try with lxc.apparmor = unsafe and it will start.
now people will start messing up apparmor profiles until they figure
that they need a kernel with aa-mount support.
they turn on lxc.apparmor = safe and the aa profiles are still messed
up.
but lxc won't complain about security anymore and they'll end up feeling
safe with a messed up container.
you're giving a false sense of security here.
Or think about someone who creates (maybe by mistake) and uses an all
allow (or partly insecure) apparmor profile.
lxc.apparmor = safe would spit in his face on a debian system
lxc.apparmor = safe wouldn't complain at all on an ubuntu system
that makes no sense for me.
if you implement a 'safe' flag that's supposed to prevent an unsafe
container from starting, you better implement it correctly or not at
all.
You're arguing with my choice of terms. Offer better terms then :) I
agree safe is probably misleading. We want something connoting
"do not start if not fully available in the kernel".
Perhaps it should be
lxc.apparmor.require_mount = [0|1], with default being 1?
You'll just end up with howtos and forum posts like
Q) my container wont start
A) try putting lxc.apparmor.require_mount = 0 in your container config

and people will do so by default because lxc + apparmor won't work on
more than 50% of the systems otherwise. There goes your sane default.
And it will be argued that turning this check off will make the
containers more secure because it will allow them to make use of
apparmor - maybe not for mount protection, but at least for file access
protection.

I really would prefer proper logging and debugging output from lxc over
all that apparmor fiddling you try to think of.
Like I said, the WARN() messages of your patch don't appear anywhere on
my system.
lxc not giving me any feedback about it's refusal to activate the
apparmor profile I configured did cost me hours to figure the problem.
IMHO these things are way more important to fix than to implement
switches to test and judge other components of the system.
Post by Serge Hallyn
Post by Tom Weber
It's like a button labeled: make this container apparmor safe. people
push it and expect the container to be apparmor safe while it doesn't
need to be.
Post by Serge Hallyn
Maybe I'll send a patch for that later today.
It's worth considering also whether there is anything we require from
apparmor for the unprivileged containers. If there is then we cannot
allow an unprivileged container to set lxc.apparmor = unsafe|off. The
only thing I can think of is if there happens to be an 0-day in the
handling of some namespaced procfile that results in privilege
escalation... I'm cc:ing jjohansen for input from him.
Post by Tom Weber
Just because there's a feature in the kernel (and it's nothing else your
stat does) doesn't mean that the other end of the system that's
responsible for enforcing/using it does really use it. This test
implies security where no security is.
Are you arguing that I shouldn't check whether the feature is
enabled bc it might be buggy and not working anyway?
basically yes. it could be misconfigured in apparmor as well.
The availability of a feature doesn't tell you if it's used correctly.
Likewise there are probably scores of kernel exploits in the lastest
kernel. That doesn't mean that if selinux isn't enabled sestatus should
lie to you and say your policy is enforcing.
huh? don't see what you wanna tell me?
apparmor doesn't lie about anything. It tells me that it can't enforce
the mount rules - but everything else works. and apparmor is the only
thing about to judge on this.
Post by Serge Hallyn
Post by Tom Weber
Post by Serge Hallyn
Post by Tom Weber
I dont think a readable /proc/kcore inside a container or access to
dmesg is very secure either - as in the default config.
I could mount proc on /proc_insecure and create whatever /dev/ nodes I
Not from inside the container.
dmesg and /proc/kcore? both are readable from inside - in the default
config.
and i was talking about mounting /proc* or /sys* whatever way i want in
lxc.conainer config. You allow me that, but you want to force me on an
Again I'm talking about the defaults.
the defaults are the job of the distribution. You can't enforce, judge
or rely on defaults if people start installing or compiling their own
stuff. And people that do so are usually supposed to know what they do.


Regards,
Tom
Dwight Engen
2014-08-07 21:23:28 UTC
Permalink
On Tue, 05 Aug 2014 13:53:58 +0200
Post by Tom Weber
Hello,
debian7
lxc-1.0.4 from debian testing
vanilla kernel.org kernel 3.14.14
i'm new to lxc and apparmor, so this took me a couple of hours to
lxc-start won't assign an apparmor-profile to a container since it's
the apparmor_enabled() tests for AA_MOUNT_RESTR
(/sys/kernel/security/apparmor/features/mount/mask) first, which will
never exist without that apparmor mount patch in the kernel.
commenting out that test gives me apparmor functionality (except for
that mount feature of course).
Is that intentional or just an ancient relict?
I'd prefer to have apparmor profile support without mount restrictions
over no apparmor profile support at all. apparmor gives me warnings
Warning from /etc/apparmor.d/lxc-containers
(/etc/apparmor.d/lxc-containers line 8): profile
lxc-container-default mount rules not enforced
when starting up, which is what I expect and something I can deal with
as admin. I think lxc-start should activate the requested profile
anyway.
Oh, and a little log message wether lxc-start detected apparmor or not
and activates it would be _very_ helpfull :)
lsm_init() INFO()s which lsm backend was detected, and
apparmor_process_label_set() INFO()s which profile its setting so you
should see those in the log if your --logpriority is set accordingly.
Post by Tom Weber
related question: dropping sys_admin cap for the container should
render all the mount protections from apparmor unnecessary, right?
Regards,
Tom
_______________________________________________
lxc-users mailing list
lxc-users at lists.linuxcontainers.org
http://lists.linuxcontainers.org/listinfo/lxc-users
Tom Weber
2014-08-07 22:19:50 UTC
Permalink
Post by Dwight Engen
On Tue, 05 Aug 2014 13:53:58 +0200
Post by Tom Weber
Oh, and a little log message wether lxc-start detected apparmor or not
and activates it would be _very_ helpfull :)
lsm_init() INFO()s which lsm backend was detected, and
apparmor_process_label_set() INFO()s which profile its setting so you
should see those in the log if your --logpriority is set accordingly.
yes, but only if it activates apparmor (which would have been only the
case if that mount patch is in the kernel). It silently ignored my
apparmor settings completely - how should I know what should have been
in the log if I only see these messages when everything works? :)

The problem with Serge's patch, which turns the failed detection of a
mount patched kernel into a WARN(), is that these WARN()s don't appear
anywhere.

Regards,
Tom

Continue reading on narkive:
Loading...