Discussion:
lxc publish - how to use "pigz" (parallel gzip) for compression?
(too old to reply)
Tomasz Chmielewski
2018-09-21 06:12:42 UTC
Permalink
"lxc publish $container --alias $container" can take quite long even on
a server with multiple CPUs.

This is because "gzip" is used, which can only use one CPU, i.e.:

gzip -c -n
/var/snap/lxd/common/lxd/images/lxd_build_015181701/lxd_build_tar_202915839


Is it possible to specify an alternative compression program which is
able to do parallel compression?

I see it doesn't work for pigz:

# lxc publish $container --alias $container --compression pigz
Error: exec: "pigz": executable file not found in $PATH


This is because it looks up the binary in /snap/core/4917/bin/ and pigz
does not exist there.


xz is able to do parallel compression (i.e. xz -T 0), but sadly, lxd is
not using it, so it's even slower than gzip:


# lxc publish $container --alias $container --compression xz


And this notation is not allowed:

# lxc publish $container --alias $container --compression "xz -T 0"


Are there any possible workarounds to use parallel compression for "lxc
publish"?


Tomasz Chmielewski
https://lxadm.com
Tomasz Chmielewski
2018-09-21 07:22:46 UTC
Permalink
maybe not what you are looking for, but could work as workaround for
the
mv /snap/core/4917/bin/gzip /snap/core/4917/bin/gzip_dist
ln -s /usr/bin/pigz /snap/core/4917/bin/gzip
Nope, it's snap, read-only:

# touch /snap/core/4917/bin/anything
touch: cannot touch '/snap/core/4917/bin/anything': Read-only file
system


Tomasz Chmielewski
https://lxadm.com
Stéphane Graber
2018-09-21 07:28:27 UTC
Permalink
Post by Tomasz Chmielewski
maybe not what you are looking for, but could work as workaround for the
mv /snap/core/4917/bin/gzip /snap/core/4917/bin/gzip_dist
ln -s /usr/bin/pigz /snap/core/4917/bin/gzip
mount -o bind /usr/bin/pigz /snap/core/4917/bin/gzip

That may work. But won't survive core snap update or reboots.
Post by Tomasz Chmielewski
# touch /snap/core/4917/bin/anything
touch: cannot touch '/snap/core/4917/bin/anything': Read-only file system
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
Tomasz Chmielewski
2018-09-21 07:53:35 UTC
Permalink
Post by Stéphane Graber
maybe not what you are looking for, but could work as workaround for the
mv /snap/core/4917/bin/gzip /snap/core/4917/bin/gzip_dist
ln -s /usr/bin/pigz /snap/core/4917/bin/gzip
mount -o bind /usr/bin/pigz /snap/core/4917/bin/gzip
That may work. But won't survive core snap update or reboots.
Are there any plans to support pigz (or "xz -T 0") "out of the box"?


Tomasz Chmielewski
https://lxadm.com

Continue reading on narkive:
Loading...