Discussion:
[lxc-users] How to recover from ERROR state
Kees Bakker
2018-09-11 13:13:40 UTC
Permalink
Hey,

Every now and then we have one or more containers in state ERROR.
Is there a clever method to recover from that, other than
rebooting the LXD server?

Killing the monitor and the forkstart does help. And also a kworker
process (kworker/u16:0) is eating up one of the CPUs with 100% load.
lxc info gives "error: Monitor is hung"

I'm running Ubuntu 16.04 with BTRFS. The kernel is 4.15.0-33-generic
--
Kees Bakker
Christian Brauner
2018-09-11 13:40:17 UTC
Permalink
Post by Kees Bakker
Hey,
Every now and then we have one or more containers in state ERROR.
Is there a clever method to recover from that, other than
rebooting the LXD server?
Killing the monitor and the forkstart does help. And also a kworker
process (kworker/u16:0) is eating up one of the CPUs with 100% load.
lxc info gives "error: Monitor is hung"
If I'm not mistaken this is usually caused by a hanging lxc-monitord
process which older LXC versions still use and which is removed in
newer LXC versions.
Can you check whether you see a lxc-monitord process when such a hang
happens. If so, kill it. Afterwards things should work fine again.
Post by Kees Bakker
I'm running Ubuntu 16.04 with BTRFS. The kernel is 4.15.0-33-generic
Cc stgraber since I don't have in mind what LXC version is used
and if it is one that has already gotten rid of lxc-monitord.

Christian
Kees Bakker
2018-09-11 13:54:30 UTC
Permalink
Post by Christian Brauner
Post by Kees Bakker
Hey,
Every now and then we have one or more containers in state ERROR.
Is there a clever method to recover from that, other than
rebooting the LXD server?
Killing the monitor and the forkstart does help. And also a kworker
process (kworker/u16:0) is eating up one of the CPUs with 100% load.
lxc info gives "error: Monitor is hung"
If I'm not mistaken this is usually caused by a hanging lxc-monitord
process which older LXC versions still use and which is removed in
newer LXC versions.
Can you check whether you see a lxc-monitord process when such a hang
happens. If so, kill it. Afterwards things should work fine again.
Killing lxc-monitord did not help.
I had to kill a "[lxc monitor]" process as well. Then the container
got back to state "STOPPED".

But after trying to start the container again, the state went back
to "ERROR".

Meanwhile the kworker/u16:0 process continued at 100% load.
Post by Christian Brauner
Post by Kees Bakker
I'm running Ubuntu 16.04 with BTRFS. The kernel is 4.15.0-33-generic
Cc stgraber since I don't have in mind what LXC version is used
and if it is one that has already gotten rid of lxc-monitord.
ii  lxc-common     2.0.8-0ubuntu1~16.04.2  amd64        Linux Containers userspace tools (common tools)
ii  lxcfs          2.0.8-0ubuntu1~16.04.2  amd64        FUSE based filesystem for LXC
ii  lxd            2.0.11-0ubuntu1~16.04.4 amd64        Container hypervisor based on LXC - daemon
ii  lxd-client     2.0.11-0ubuntu1~16.04.4 amd64        Container hypervisor based on LXC - client
--
Kees Bakker
Kees Bakker
2018-09-11 14:29:47 UTC
Permalink
Post by Kees Bakker
Post by Christian Brauner
Post by Kees Bakker
Hey,
Every now and then we have one or more containers in state ERROR.
Is there a clever method to recover from that, other than
rebooting the LXD server?
Killing the monitor and the forkstart does help. And also a kworker
process (kworker/u16:0) is eating up one of the CPUs with 100% load.
lxc info gives "error: Monitor is hung"
If I'm not mistaken this is usually caused by a hanging lxc-monitord
process which older LXC versions still use and which is removed in
newer LXC versions.
Can you check whether you see a lxc-monitord process when such a hang
happens. If so, kill it. Afterwards things should work fine again.
Killing lxc-monitord did not help.
I had to kill a "[lxc monitor]" process as well. Then the container
got back to state "STOPPED".
But you still have a lxc-monitord process. That's usually the culprit
for all that mess.
No, that process was killed. Didn't help.
If you can update liblxc and feel confident doing so
you should do it.
Update liblxc to what version?
--
Kees Bakker
Andrey Repin
2018-09-11 19:56:22 UTC
Permalink
Greetings, Kees Bakker!
Post by Kees Bakker
ii  lxc-common     2.0.8-0ubuntu1~16.04.2  amd64        Linux Containers userspace tools (common tools)
ii  lxcfs          2.0.8-0ubuntu1~16.04.2  amd64        FUSE based filesystem for LXC
ii  lxd            2.0.11-0ubuntu1~16.04.4 amd64        Container hypervisor based on LXC - daemon
ii  lxd-client     2.0.11-0ubuntu1~16.04.4 amd64        Container hypervisor based on LXC - client
Upgrade from PPA.
add-apt-repository ppa:ubuntu-lxc/stable

ii lxc-common 2.1.1-0ubuntu1 amd64 Linux Containers userspace tools (common t
ii lxc-templates 2.1.1-0ubuntu1 amd64 Linux Containers userspace tools (template
ii lxc1 2.1.1-0ubuntu1 amd64 Linux Containers userspace tools
ii lxcfs 2.0.8-1ubuntu2 amd64 FUSE based filesystem for LXC


--
With best regards,
Andrey Repin
Tuesday, September 11, 2018 22:52:22

Sorry for my terrible english...
Kees Bakker
2018-09-12 08:14:36 UTC
Permalink
Post by Andrey Repin
Greetings, Kees Bakker!
Post by Kees Bakker
ii  lxc-common     2.0.8-0ubuntu1~16.04.2  amd64        Linux Containers userspace tools (common tools)
ii  lxcfs          2.0.8-0ubuntu1~16.04.2  amd64        FUSE based filesystem for LXC
ii  lxd            2.0.11-0ubuntu1~16.04.4 amd64        Container hypervisor based on LXC - daemon
ii  lxd-client     2.0.11-0ubuntu1~16.04.4 amd64        Container hypervisor based on LXC - client
Upgrade from PPA.
add-apt-repository ppa:ubuntu-lxc/stable
ii lxc-common 2.1.1-0ubuntu1 amd64 Linux Containers userspace tools (common t
ii lxc-templates 2.1.1-0ubuntu1 amd64 Linux Containers userspace tools (template
ii lxc1 2.1.1-0ubuntu1 amd64 Linux Containers userspace tools
ii lxcfs 2.0.8-1ubuntu2 amd64 FUSE based filesystem for LXC
Hmm. That PPA does not have liblxc1 2.1.1, but 3.0.1

# apt list --upgradable
Listing... Done
liblxc1/xenial 3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.8-0ubuntu1~16.04.2]
libseccomp2/xenial 2.3.1-2.1ubuntu3~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.3.1-2.1ubuntu2~16.04.1]
lxc-common/xenial 2.1.1-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.8-0ubuntu1~16.04.2]
lxcfs/xenial 3.0.1-0ubuntu2~18.04.1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.8-0ubuntu1~16.04.2]

# apt policy liblxc1
liblxc1:
  Installed: 2.0.8-0ubuntu1~16.04.2
  Candidate: 3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1
  Version table:
     3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1 500
        500 http://ppa.launchpad.net/ubuntu-lxc/stable/ubuntu xenial/main amd64 Packages
     3.0.1-0ubuntu1~16.04.2 100
        100 http://nl.archive.ubuntu.com/ubuntu xenial-backports/main amd64 Packages
 *** 2.0.8-0ubuntu1~16.04.2 500
        500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     2.0.7-0ubuntu1~16.04.2 500
        500 http://security.ubuntu.com/ubuntu xenial-security/main amd64 Packages
     2.0.0-0ubuntu2 500
        500 http://nl.archive.ubuntu.com/ubuntu xenial/main amd64 Packages

I could upgrade to the 3.0 packages, but that would be more challenging.
Any advice?
--
Kees
Fajar A. Nugraha
2018-09-12 08:51:57 UTC
Permalink
Post by Kees Bakker
Post by Andrey Repin
Greetings, Kees Bakker!
ii lxc-common 2.0.8-0ubuntu1~16.04.2 amd64 Linux
Containers userspace tools (common tools)
Post by Andrey Repin
ii lxcfs 2.0.8-0ubuntu1~16.04.2 amd64 FUSE based
filesystem for LXC
Post by Andrey Repin
ii lxd 2.0.11-0ubuntu1~16.04.4 amd64 Container
hypervisor based on LXC - daemon
Post by Andrey Repin
ii lxd-client 2.0.11-0ubuntu1~16.04.4 amd64 Container
hypervisor based on LXC - client
Post by Andrey Repin
Upgrade from PPA.
add-apt-repository ppa:ubuntu-lxc/stable
ii lxc-common 2.1.1-0ubuntu1 amd64 Linux Containers
userspace tools (common t
Post by Andrey Repin
ii lxc-templates 2.1.1-0ubuntu1 amd64 Linux Containers
userspace tools (template
Post by Andrey Repin
ii lxc1 2.1.1-0ubuntu1 amd64 Linux Containers
userspace tools
Post by Andrey Repin
ii lxcfs 2.0.8-1ubuntu2 amd64 FUSE based
filesystem for LXC
Hmm. That PPA does not have liblxc1 2.1.1, but 3.0.1
# apt list --upgradable
Listing... Done
liblxc1/xenial 3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1 amd64
[upgradable from: 2.0.8-0ubuntu1~16.04.2]
libseccomp2/xenial 2.3.1-2.1ubuntu3~ubuntu16.04.1~ppa1 amd64 [upgradable
from: 2.3.1-2.1ubuntu2~16.04.1]
lxc-common/xenial 2.1.1-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable
from: 2.0.8-0ubuntu1~16.04.2]
lxcfs/xenial 3.0.1-0ubuntu2~18.04.1~ubuntu16.04.1~ppa1 amd64 [upgradable
from: 2.0.8-0ubuntu1~16.04.2]
# apt policy liblxc1
Installed: 2.0.8-0ubuntu1~16.04.2
Candidate: 3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1
3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1 500
500 http://ppa.launchpad.net/ubuntu-lxc/stable/ubuntu xenial/main
amd64 Packages
3.0.1-0ubuntu1~16.04.2 100
100 http://nl.archive.ubuntu.com/ubuntu xenial-backports/main
amd64 Packages
*** 2.0.8-0ubuntu1~16.04.2 500
500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main amd64
Packages
100 /var/lib/dpkg/status
2.0.7-0ubuntu1~16.04.2 500
500 http://security.ubuntu.com/ubuntu xenial-security/main amd64
Packages
2.0.0-0ubuntu2 500
500 http://nl.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
I could upgrade to the 3.0 packages, but that would be more challenging.
2.1 is not 'LTS' version, so it's expected that it won't be available
anymore. Personally I'd upgrade to 3.0. But backup /var/lib/lxd (when ALL
containers are stopped) beforehand.

FWIW, I'm more inclined to think your 'kworker' issue might be related to
btrfs instead of lxd, but it might be harder to debug that.
--
Fajar
Kees Bakker
2018-09-12 09:08:30 UTC
Permalink
Post by Kees Bakker
Post by Andrey Repin
Greetings, Kees Bakker!
ii  lxc-common     2.0.8-0ubuntu1~16.04.2  amd64        Linux Containers userspace tools (common tools)
ii  lxcfs          2.0.8-0ubuntu1~16.04.2  amd64        FUSE based filesystem for LXC
ii  lxd            2.0.11-0ubuntu1~16.04.4 amd64        Container hypervisor based on LXC - daemon
ii  lxd-client     2.0.11-0ubuntu1~16.04.4 amd64        Container hypervisor based on LXC - client
Upgrade from PPA.
add-apt-repository ppa:ubuntu-lxc/stable
ii  lxc-common         2.1.1-0ubuntu1 amd64          Linux Containers userspace tools (common t
ii  lxc-templates      2.1.1-0ubuntu1 amd64          Linux Containers userspace tools (template
ii  lxc1               2.1.1-0ubuntu1 amd64          Linux Containers userspace tools
ii  lxcfs              2.0.8-1ubuntu2 amd64          FUSE based filesystem for LXC
Hmm. That PPA does not have liblxc1 2.1.1, but 3.0.1
# apt list --upgradable
Listing... Done
liblxc1/xenial 3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.8-0ubuntu1~16.04.2]
libseccomp2/xenial 2.3.1-2.1ubuntu3~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.3.1-2.1ubuntu2~16.04.1]
lxc-common/xenial 2.1.1-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.8-0ubuntu1~16.04.2]
lxcfs/xenial 3.0.1-0ubuntu2~18.04.1~ubuntu16.04.1~ppa1 amd64 [upgradable from: 2.0.8-0ubuntu1~16.04.2]
# apt policy liblxc1
  Installed: 2.0.8-0ubuntu1~16.04.2
  Candidate: 3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1
     3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1 500
        500 http://ppa.launchpad.net/ubuntu-lxc/stable/ubuntu <http://ppa.launchpad.net/ubuntu-lxc/stable/ubuntu> xenial/main amd64 Packages
     3.0.1-0ubuntu1~16.04.2 100
        100 http://nl.archive.ubuntu.com/ubuntu <http://nl.archive.ubuntu.com/ubuntu> xenial-backports/main amd64 Packages
 *** 2.0.8-0ubuntu1~16.04.2 500
        500 http://nl.archive.ubuntu.com/ubuntu <http://nl.archive.ubuntu.com/ubuntu> xenial-updates/main amd64 Packages
        100 /var/lib/dpkg/status
     2.0.7-0ubuntu1~16.04.2 500
        500 http://security.ubuntu.com/ubuntu <http://security.ubuntu.com/ubuntu> xenial-security/main amd64 Packages
     2.0.0-0ubuntu2 500
        500 http://nl.archive.ubuntu.com/ubuntu <http://nl.archive.ubuntu.com/ubuntu> xenial/main amd64 Packages
I could upgrade to the 3.0 packages, but that would be more challenging.
2.1 is not 'LTS' version, so it's expected that it won't be available anymore. Personally I'd upgrade to 3.0. But backup /var/lib/lxd (when ALL containers are stopped) beforehand.
Upgrading to 3.0 gives me two options: 1) xenial-backports, 2) the suggested PPA (ppa:ubuntu-lxc/stable).
Which one would you pick?
Post by Kees Bakker
FWIW, I'm more inclined to think your 'kworker' issue might be related to btrfs instead of lxd, but it might be harder to debug that.
OK, I'll keep that in mind.
--
Kees
Fajar A. Nugraha
2018-09-12 09:10:58 UTC
Permalink
Post by Fajar A. Nugraha
Post by Kees Bakker
Post by Andrey Repin
Greetings, Kees Bakker!
ii lxc-common 2.0.8-0ubuntu1~16.04.2 amd64 Linux
Containers userspace tools (common tools)
Post by Andrey Repin
ii lxcfs 2.0.8-0ubuntu1~16.04.2 amd64 FUSE based
filesystem for LXC
Post by Andrey Repin
ii lxd 2.0.11-0ubuntu1~16.04.4 amd64 Container
hypervisor based on LXC - daemon
Post by Andrey Repin
ii lxd-client 2.0.11-0ubuntu1~16.04.4 amd64 Container
hypervisor based on LXC - client
Post by Andrey Repin
Upgrade from PPA.
add-apt-repository ppa:ubuntu-lxc/stable
ii lxc-common 2.1.1-0ubuntu1 amd64 Linux Containers
userspace tools (common t
Post by Andrey Repin
ii lxc-templates 2.1.1-0ubuntu1 amd64 Linux Containers
userspace tools (template
Post by Andrey Repin
ii lxc1 2.1.1-0ubuntu1 amd64 Linux Containers
userspace tools
Post by Andrey Repin
ii lxcfs 2.0.8-1ubuntu2 amd64 FUSE based
filesystem for LXC
Hmm. That PPA does not have liblxc1 2.1.1, but 3.0.1
# apt list --upgradable
Listing... Done
liblxc1/xenial 3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1 amd64
[upgradable from: 2.0.8-0ubuntu1~16.04.2]
libseccomp2/xenial 2.3.1-2.1ubuntu3~ubuntu16.04.1~ppa1 amd64 [upgradable
from: 2.3.1-2.1ubuntu2~16.04.1]
lxc-common/xenial 2.1.1-0ubuntu1~ubuntu16.04.1~ppa1 amd64 [upgradable
from: 2.0.8-0ubuntu1~16.04.2]
lxcfs/xenial 3.0.1-0ubuntu2~18.04.1~ubuntu16.04.1~ppa1 amd64 [upgradable
from: 2.0.8-0ubuntu1~16.04.2]
# apt policy liblxc1
Installed: 2.0.8-0ubuntu1~16.04.2
Candidate: 3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1
3.0.1-0ubuntu1~18.04.2~ubuntu16.04.1~ppa1 500
500 http://ppa.launchpad.net/ubuntu-lxc/stable/ubuntu
xenial/main amd64 Packages
3.0.1-0ubuntu1~16.04.2 100
100 http://nl.archive.ubuntu.com/ubuntu xenial-backports/main
amd64 Packages
*** 2.0.8-0ubuntu1~16.04.2 500
500 http://nl.archive.ubuntu.com/ubuntu xenial-updates/main
amd64 Packages
100 /var/lib/dpkg/status
2.0.7-0ubuntu1~16.04.2 500
500 http://security.ubuntu.com/ubuntu xenial-security/main amd64
Packages
2.0.0-0ubuntu2 500
500 http://nl.archive.ubuntu.com/ubuntu xenial/main amd64 Packages
I could upgrade to the 3.0 packages, but that would be more challenging.
2.1 is not 'LTS' version, so it's expected that it won't be available
anymore. Personally I'd upgrade to 3.0. But backup /var/lib/lxd (when ALL
containers are stopped) beforehand.
Upgrading to 3.0 gives me two options: 1) xenial-backports, 2) the
suggested PPA (ppa:ubuntu-lxc/stable).
Which one would you pick?
I'd go with xenial-backports. IIRC this is more-tested and recommended
compared to ppa (can't find the relevant email that point this off-hand
though, sorry).
--
Fajar
Kees Bakker
2018-09-24 12:03:36 UTC
Permalink
Same question again: what is the best approach to recover
from a container in an ERROR state?

This time it happened with Ubuntu 18.04 and LVM storage.

The steps leading to this were as follows. It's just an FYI, I don't think it
really matters, except for the stop and start.

  lvextend -L 20G local/containers_xyz
  resize2fs /dev/local/containers_xyz
  lxc stop xyz
  e2fsck -f /dev/local/containers_
  lxc start xyz

... the start command hanged.

Some output os ps auxfwww

root      6224  0.0  0.0  22912  4096 pts/1    S    sep06   0:00  |               \_ -bash
root     20900  0.0  0.0 1136140 12092 pts/1   Sl+  12:19   0:00  |                   \_ lxc start xyz
--
root     18157  3.5  4.2 5581444 1398904 ?     Ssl  sep12 611:36 /usr/lib/lxd/lxd --group lxd --logfile=/var/log/lxd/lxd.log
root     20918  0.0  0.0 521720 19780 ?        Sl   12:19   0:00  \_ /usr/lib/lxd/lxd forkstart xyz /var/lib/lxd/containers /var/log/lxd/xyz/lxc.conf
root     20925  0.0  0.0      0     0 ?        Z    12:19   0:00      \_ [lxd] <defunct>
--
root     20926  0.0  0.0 530432  7280 ?        Ss   12:19   0:00 [lxc monitor] /var/lib/lxd/containers xyz
root     20943  0.0  0.0 530432  3484 ?        D    12:19   0:00  \_ [lxc monitor] /var/lib/lxd/containers xyz
Post by Kees Bakker
Hey,
Every now and then we have one or more containers in state ERROR.
Is there a clever method to recover from that, other than
rebooting the LXD server?
Killing the monitor and the forkstart does help. And also a kworker
process (kworker/u16:0) is eating up one of the CPUs with 100% load.
lxc info gives "error: Monitor is hung"
I'm running Ubuntu 16.04 with BTRFS. The kernel is 4.15.0-33-generic
Christian Brauner
2018-09-24 12:11:38 UTC
Permalink
Post by Kees Bakker
Same question again: what is the best approach to recover
from a container in an ERROR state?
Please show me the dmesg output. If it is a kernel bug you're hitting
there's nothing that LXD can do to help you.
Post by Kees Bakker
This time it happened with Ubuntu 18.04 and LVM storage.
The steps leading to this were as follows. It's just an FYI, I don't think it
really matters, except for the stop and start.
lvextend -L 20G local/containers_xyz
resize2fs /dev/local/containers_xyz
lxc stop xyz
e2fsck -f /dev/local/containers_
lxc start xyz
... the start command hanged.
Some output os ps auxfwww
root 6224 0.0 0.0 22912 4096 pts/1 S sep06 0:00
| \_ -bash
root 20900 0.0 0.0 1136140 12092 pts/1 Sl+ 12:19 0:00
| \_ lxc start xyz
--
root 18157 3.5 4.2 5581444 1398904 ? Ssl sep12 611:36
/usr/lib/lxd/lxd --group lxd --logfile=/var/log/lxd/lxd.log
root 20918 0.0 0.0 521720 19780 ? Sl 12:19 0:00 \_
/usr/lib/lxd/lxd forkstart xyz /var/lib/lxd/containers
/var/log/lxd/xyz/lxc.conf
root 20925 0.0 0.0 0 0 ? Z 12:19 0:00 \_
[lxd] <defunct>
--
root 20926 0.0 0.0 530432 7280 ? Ss 12:19 0:00 [lxc
monitor] /var/lib/lxd/containers xyz
root 20943 0.0 0.0 530432 3484 ? D 12:19 0:00 \_ [lxc
monitor] /var/lib/lxd/containers xyz
Post by Kees Bakker
Hey,
Every now and then we have one or more containers in state ERROR.
Is there a clever method to recover from that, other than
rebooting the LXD server?
Killing the monitor and the forkstart does help. And also a kworker
process (kworker/u16:0) is eating up one of the CPUs with 100% load.
lxc info gives "error: Monitor is hung"
I'm running Ubuntu 16.04 with BTRFS. The kernel is 4.15.0-33-generic
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Christian Brauner
2018-09-24 13:30:17 UTC
Permalink
Post by Christian Brauner
Post by Kees Bakker
Same question again: what is the best approach to recover
from a container in an ERROR state?
So another thing I would like to see is the current stack of the hung
monitor process. Could you please paste (or send privately) the output
of:

cat /proc/<pid-of-hung-monitor-process>/stack

Also, in what state is the monitor hung. Again in D state?

Christian
Post by Christian Brauner
Please show me the dmesg output. If it is a kernel bug you're hitting
there's nothing that LXD can do to help you.
Post by Kees Bakker
This time it happened with Ubuntu 18.04 and LVM storage.
The steps leading to this were as follows. It's just an FYI, I don't think it
really matters, except for the stop and start.
lvextend -L 20G local/containers_xyz
resize2fs /dev/local/containers_xyz
lxc stop xyz
e2fsck -f /dev/local/containers_
lxc start xyz
... the start command hanged.
Some output os ps auxfwww
root 6224 0.0 0.0 22912 4096 pts/1 S sep06 0:00
| \_ -bash
root 20900 0.0 0.0 1136140 12092 pts/1 Sl+ 12:19 0:00
| \_ lxc start xyz
--
root 18157 3.5 4.2 5581444 1398904 ? Ssl sep12 611:36
/usr/lib/lxd/lxd --group lxd --logfile=/var/log/lxd/lxd.log
root 20918 0.0 0.0 521720 19780 ? Sl 12:19 0:00 \_
/usr/lib/lxd/lxd forkstart xyz /var/lib/lxd/containers
/var/log/lxd/xyz/lxc.conf
root 20925 0.0 0.0 0 0 ? Z 12:19 0:00 \_
[lxd] <defunct>
--
root 20926 0.0 0.0 530432 7280 ? Ss 12:19 0:00 [lxc
monitor] /var/lib/lxd/containers xyz
root 20943 0.0 0.0 530432 3484 ? D 12:19 0:00 \_ [lxc
monitor] /var/lib/lxd/containers xyz
Post by Kees Bakker
Hey,
Every now and then we have one or more containers in state ERROR.
Is there a clever method to recover from that, other than
rebooting the LXD server?
Killing the monitor and the forkstart does help. And also a kworker
process (kworker/u16:0) is eating up one of the CPUs with 100% load.
lxc info gives "error: Monitor is hung"
I'm running Ubuntu 16.04 with BTRFS. The kernel is 4.15.0-33-generic
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Loading...