[lxc-users] lxc publish - how to use "pigz" (parallel gzip) for compression?

[Tomasz Chmielewski]

"lxc publish $container --alias $container" can take quite long even on
a server with multiple CPUs.

This is because "gzip" is used, which can only use one CPU, i.e.:

gzip -c -n
/var/snap/lxd/common/lxd/images/lxd_build_015181701/lxd_build_tar_202915839


Is it possible to specify an alternative compression program which is
able to do parallel compression?

I see it doesn't work for pigz:

# lxc publish $container --alias $container --compression pigz
Error: exec: "pigz": executable file not found in $PATH


This is because it looks up the binary in /snap/core/4917/bin/ and pigz
does not exist there.


xz is able to do parallel compression (i.e. xz -T 0), but sadly, lxd is
not using it, so it's even slower than gzip:


# lxc publish $container --alias $container --compression xz


And this notation is not allowed:

# lxc publish $container --alias $container --compression "xz -T 0"


Are there any possible workarounds to use parallel compression for "lxc
publish"?


Tomasz Chmielewski
https://lxadm.com

[Tomasz Chmielewski]

maybe not what you are looking for, but could work as workaround for
the
mv /snap/core/4917/bin/gzip /snap/core/4917/bin/gzip_dist
ln -s /usr/bin/pigz /snap/core/4917/bin/gzip

Nope, it's snap, read-only:

# touch /snap/core/4917/bin/anything
touch: cannot touch '/snap/core/4917/bin/anything': Read-only file
system


Tomasz Chmielewski
https://lxadm.com

[Stéphane Graber]

Post by Tomasz Chmielewski

maybe not what you are looking for, but could work as workaround for the
mv /snap/core/4917/bin/gzip /snap/core/4917/bin/gzip_dist
ln -s /usr/bin/pigz /snap/core/4917/bin/gzip

mount -o bind /usr/bin/pigz /snap/core/4917/bin/gzip

That may work. But won't survive core snap update or reboots.

Post by Tomasz Chmielewski
# touch /snap/core/4917/bin/anything
touch: cannot touch '/snap/core/4917/bin/anything': Read-only file system

[Tomasz Chmielewski]

Post by Stéphane Graber

maybe not what you are looking for, but could work as workaround for the
mv /snap/core/4917/bin/gzip /snap/core/4917/bin/gzip_dist
ln -s /usr/bin/pigz /snap/core/4917/bin/gzip

mount -o bind /usr/bin/pigz /snap/core/4917/bin/gzip
That may work. But won't survive core snap update or reboots.

Are there any plans to support pigz (or "xz -T 0") "out of the box"?


Tomasz Chmielewski
https://lxadm.com