Discussion:
Limits kind of not working
(too old to reply)
Pavol Cupka
2017-09-11 19:23:39 UTC
Permalink
Hello list!

I am running LXD on gentoo, but without systemd (with openrc). I have
trouble setting limits.cpu, limits.memory and disk size (using btrfs). I
haven't tried the network and io limits yet. The values for limits.cpu can
be found in /sys/fs/cgroup/cpuset/lxc/c1/cpuset.cpus for example, but
/proc/cpuinfo inside the container still shows all the processors.

LXC 2.0.8
LXD 2.16
CGMANAGER 0.41

Thank you for any pointers.
Pavol
Serge E. Hallyn
2017-09-11 20:24:34 UTC
Permalink
Post by Pavol Cupka
Hello list!
I am running LXD on gentoo, but without systemd (with openrc). I have
trouble setting limits.cpu, limits.memory and disk size (using btrfs). I
haven't tried the network and io limits yet. The values for limits.cpu can
be found in /sys/fs/cgroup/cpuset/lxc/c1/cpuset.cpus for example, but
/proc/cpuinfo inside the container still shows all the processors.
/proc/cpuinfo isn't virtualized to show only the cpus you have
available. For that you have to use lxcfs, which is a fuse filesystem
that overmounts /proc and filters the results.

github.com/lxc/lxcfs
Post by Pavol Cupka
LXC 2.0.8
LXD 2.16
CGMANAGER 0.41
Thank you for any pointers.
Pavol
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Pavol Cupka
2017-09-12 05:36:54 UTC
Permalink
OK, thank you for your answer. So I can confirm CPU and memory limits
working. IO and network have to be tested on my part yet. What doesn't work
for me is disk "quota" I am using btrfs and I have set 20GB for the
container root, but was still able to allocate more the 90GB writing with
dd to a file inside the container. What am I doing wrong?

Thank you in advance
Pavol
Post by Serge E. Hallyn
Post by Pavol Cupka
Hello list!
I am running LXD on gentoo, but without systemd (with openrc). I have
trouble setting limits.cpu, limits.memory and disk size (using btrfs). I
haven't tried the network and io limits yet. The values for limits.cpu
can
Post by Pavol Cupka
be found in /sys/fs/cgroup/cpuset/lxc/c1/cpuset.cpus for example, but
/proc/cpuinfo inside the container still shows all the processors.
/proc/cpuinfo isn't virtualized to show only the cpus you have
available. For that you have to use lxcfs, which is a fuse filesystem
that overmounts /proc and filters the results.
github.com/lxc/lxcfs
Post by Pavol Cupka
LXC 2.0.8
LXD 2.16
CGMANAGER 0.41
Thank you for any pointers.
Pavol
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Serge E. Hallyn
2017-09-12 06:58:41 UTC
Permalink
Hm, sorry, I don't know. As far as I know this should work. Stéphane,
any ideas?

-serge
Post by Pavol Cupka
OK, thank you for your answer. So I can confirm CPU and memory limits
working. IO and network have to be tested on my part yet. What doesn't work
for me is disk "quota" I am using btrfs and I have set 20GB for the
container root, but was still able to allocate more the 90GB writing with
dd to a file inside the container. What am I doing wrong?
Thank you in advance
Pavol
Post by Serge E. Hallyn
Post by Pavol Cupka
Hello list!
I am running LXD on gentoo, but without systemd (with openrc). I have
trouble setting limits.cpu, limits.memory and disk size (using btrfs). I
haven't tried the network and io limits yet. The values for limits.cpu
can
Post by Pavol Cupka
be found in /sys/fs/cgroup/cpuset/lxc/c1/cpuset.cpus for example, but
/proc/cpuinfo inside the container still shows all the processors.
/proc/cpuinfo isn't virtualized to show only the cpus you have
available. For that you have to use lxcfs, which is a fuse filesystem
that overmounts /proc and filters the results.
github.com/lxc/lxcfs
Post by Pavol Cupka
LXC 2.0.8
LXD 2.16
CGMANAGER 0.41
Thank you for any pointers.
Pavol
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Pavol Cupka
2017-09-12 07:13:33 UTC
Permalink
Maybe I am doing it wrong.

lxc profile show def-disk20g
config: {}
description: Default LXD profile with root disk limit - 20 GB
devices:
eth0:
nictype: bridged
parent: lxdbr0
type: nic
root:
path: /
pool: default
size: 20GB
type: disk
name: def-disk20g
used_by:
- /1.0/containers/c1

lxc storage list
+---------+-------------+--------+------------------------------------+---------+
| NAME | DESCRIPTION | DRIVER | SOURCE
| USED BY |
+---------+-------------+--------+------------------------------------+---------+
| default | | btrfs | /var/lib/lxd/storage-pools/default
| 10 |
+---------+-------------+--------+------------------------------------+---------+


c1 ~ # dd if=/dev/zero of=/test.file
^C180675599+0 records in
180675599+0 records out
92505906688 bytes (93 GB) copied, 508.106 s, 182 MB/s
Post by Serge E. Hallyn
Hm, sorry, I don't know. As far as I know this should work. Stéphane,
any ideas?
-serge
Post by Pavol Cupka
OK, thank you for your answer. So I can confirm CPU and memory limits
working. IO and network have to be tested on my part yet. What doesn't work
for me is disk "quota" I am using btrfs and I have set 20GB for the
container root, but was still able to allocate more the 90GB writing with
dd to a file inside the container. What am I doing wrong?
Thank you in advance
Pavol
Post by Serge E. Hallyn
Post by Pavol Cupka
Hello list!
I am running LXD on gentoo, but without systemd (with openrc). I have
trouble setting limits.cpu, limits.memory and disk size (using btrfs). I
haven't tried the network and io limits yet. The values for limits.cpu
can
Post by Pavol Cupka
be found in /sys/fs/cgroup/cpuset/lxc/c1/cpuset.cpus for example, but
/proc/cpuinfo inside the container still shows all the processors.
/proc/cpuinfo isn't virtualized to show only the cpus you have
available. For that you have to use lxcfs, which is a fuse filesystem
that overmounts /proc and filters the results.
github.com/lxc/lxcfs
Post by Pavol Cupka
LXC 2.0.8
LXD 2.16
CGMANAGER 0.41
Thank you for any pointers.
Pavol
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Marat Khalili
2017-09-12 07:06:25 UTC
Permalink
What doesn't work for me is disk "quota" I am using btrfs and I have
set 20GB for the container root, but was still able to allocate more
the 90GB writing with dd to a file inside the container. What am I
doing wrong?
AFAIU at least LXC 2.0.8 with BTRFS back-end it does not propagate quota
settings from LXC config to the actual filesystem. There's a reason for
this: BTRFS quotas are very beta quality. You can enable them manually
(using btrfs commands) and they will work to some extent, but you are in
for some bad surprises.

--

With Best Regards,
Marat Khalili
Pavol Cupka
2017-09-12 07:14:31 UTC
Permalink
so you would recommend ZFS instead?
Post by Marat Khalili
What doesn't work for me is disk "quota" I am using btrfs and I have set
20GB for the container root, but was still able to allocate more the 90GB
writing with dd to a file inside the container. What am I doing wrong?
AFAIU at least LXC 2.0.8 with BTRFS back-end it does not propagate quota
settings from LXC config to the actual filesystem. There's a reason for
this: BTRFS quotas are very beta quality. You can enable them manually
(using btrfs commands) and they will work to some extent, but you are in for
some bad surprises.
--
With Best Regards,
Marat Khalili
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Pavol Cupka
2017-09-12 07:19:13 UTC
Permalink
well it looks like it gets set correctly also for btrfs

btrfs qgroup show -reF /var/lib/lxd/containers/c1/
qgroupid rfer excl max_rfer max_excl
-------- ---- ---- -------- --------
0/528 1.02GiB 156.54MiB none 20.00GiB
Post by Pavol Cupka
so you would recommend ZFS instead?
Post by Marat Khalili
What doesn't work for me is disk "quota" I am using btrfs and I have set
20GB for the container root, but was still able to allocate more the 90GB
writing with dd to a file inside the container. What am I doing wrong?
AFAIU at least LXC 2.0.8 with BTRFS back-end it does not propagate quota
settings from LXC config to the actual filesystem. There's a reason for
this: BTRFS quotas are very beta quality. You can enable them manually
(using btrfs commands) and they will work to some extent, but you are in for
some bad surprises.
--
With Best Regards,
Marat Khalili
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Marat Khalili
2017-09-12 07:39:37 UTC
Permalink
Post by Pavol Cupka
so you would recommend ZFS instead?
I personally don't use ZFS (it's alien to Linux kernel and I don't want
to deal with support problems arising from this situation). Many people
claim it's magical and revolutionary, so it's up to you to test and decide.

I use LXC on BTRFS with manually-enabled quotas, but I do have problems
with it. If you want it simple I'd recommend to either forget quotas or
use partitions.
Post by Pavol Cupka
well it looks like it gets set correctly also for btrfs
btrfs qgroup show -reF /var/lib/lxd/containers/c1/
qgroupid rfer excl max_rfer max_excl
-------- ---- ---- -------- --------
0/528 1.02GiB 156.54MiB
Not sure why it doesn't work then, try calling `btrfs quota enable` for
the filesystem. Are you sure you can really exceed the quota? You have
it set for amount of exclusive data, so extents shared with other
subvolumes do not count towards it.

--

With Best Regards,
Marat Khalili
Pavol Cupka
2017-09-12 16:36:15 UTC
Permalink
I want to use btrfs even though it is not "production ready", but I
like it for the features zfs is missing (balancing, growing/shrinking
volumes, dedup not eating your ram ...).

I wrote 50GB using dd if=/dev/zero and the quota show is showing this now
btrfs qgroup show -reF /var/lib/lxd/containers/c1/
qgroupid rfer excl max_rfer max_excl
-------- ---- ---- -------- --------
0/528 2.59GiB 1.72GiB none 20.00GiB

so technicaly speaking it is working because excl is smaller then
max_excl but in practice 50GB is not smaller than 20GB.

I will try to ask in btrfs mailing list. thank you for your help
Post by Pavol Cupka
so you would recommend ZFS instead?
I personally don't use ZFS (it's alien to Linux kernel and I don't want to
deal with support problems arising from this situation). Many people claim
it's magical and revolutionary, so it's up to you to test and decide.
I use LXC on BTRFS with manually-enabled quotas, but I do have problems with
it. If you want it simple I'd recommend to either forget quotas or use
partitions.
Post by Pavol Cupka
well it looks like it gets set correctly also for btrfs
btrfs qgroup show -reF /var/lib/lxd/containers/c1/
qgroupid rfer excl max_rfer max_excl
-------- ---- ---- -------- --------
0/528 1.02GiB 156.54MiB
Not sure why it doesn't work then, try calling `btrfs quota enable` for the
filesystem. Are you sure you can really exceed the quota? You have it set
for amount of exclusive data, so extents shared with other subvolumes do not
count towards it.
--
With Best Regards,
Marat Khalili
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Pavol Cupka
2017-09-12 16:57:26 UTC
Permalink
this is strange!

I have changed the quota to 2GB and run dd
dd if=/dev/zero of=/test.file
dd: error writing '/test.file': Disk quota exceeded
0+1 records in
0+0 records out
1982988288 bytes (2.0 GB) copied, 1.7051 s, 1.2 GB/s

if I change it to 20GB dd creates 47GB without problem before I stop it
dd if=/dev/zero of=/test.file
^C0+22 records in
0+22 records out
47244550144 bytes (47 GB) copied, 33.1497 s, 1.4 GB/s

ok it works better with if=/dev/urandom probably because of the
compress=lzo being enabled

thanks for help
Post by Pavol Cupka
I want to use btrfs even though it is not "production ready", but I
like it for the features zfs is missing (balancing, growing/shrinking
volumes, dedup not eating your ram ...).
I wrote 50GB using dd if=/dev/zero and the quota show is showing this now
btrfs qgroup show -reF /var/lib/lxd/containers/c1/
qgroupid rfer excl max_rfer max_excl
-------- ---- ---- -------- --------
0/528 2.59GiB 1.72GiB none 20.00GiB
so technicaly speaking it is working because excl is smaller then
max_excl but in practice 50GB is not smaller than 20GB.
I will try to ask in btrfs mailing list. thank you for your help
Post by Pavol Cupka
so you would recommend ZFS instead?
I personally don't use ZFS (it's alien to Linux kernel and I don't want to
deal with support problems arising from this situation). Many people claim
it's magical and revolutionary, so it's up to you to test and decide.
I use LXC on BTRFS with manually-enabled quotas, but I do have problems with
it. If you want it simple I'd recommend to either forget quotas or use
partitions.
Post by Pavol Cupka
well it looks like it gets set correctly also for btrfs
btrfs qgroup show -reF /var/lib/lxd/containers/c1/
qgroupid rfer excl max_rfer max_excl
-------- ---- ---- -------- --------
0/528 1.02GiB 156.54MiB
Not sure why it doesn't work then, try calling `btrfs quota enable` for the
filesystem. Are you sure you can really exceed the quota? You have it set
for amount of exclusive data, so extents shared with other subvolumes do not
count towards it.
--
With Best Regards,
Marat Khalili
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Marat Khalili
2017-09-12 18:00:59 UTC
Permalink
Post by Pavol Cupka
ok it works better with if=/dev/urandom probably because of the
compress=lzo being enabled
limit [options] <size>|none [<qgroupid>] <path>
[...]
Post by Pavol Cupka
Options
-c
limit amount of data after compression. This is the default, it is currently not possible to turn off this option.
( https://btrfs.wiki.kernel.org/index.php/Manpage/btrfs-qgroup#SUBCOMMAND )

Obviously /dev/zero compresses to nearly zero (not really due to btrfs on-the-fly compression limitations, but still by much). Now I don't even understand why it limits you in 2GB case.
--
With Best Regards,
Marat Khalili
Continue reading on narkive:
Loading...