Discussion:
"error: LXD still not running after 5 minutes" - failed lxd.migrate - how to recover?
(too old to reply)
Tomasz Chmielewski
2018-08-08 19:06:40 UTC
Permalink
I've tried to migrate from deb to snap on Ubuntu 18.04.

Unfortunately, lxd.migrate failed with "error: LXD still not running
after 5 minutes":

***@b1 ~ # lxd.migrate
=> Connecting to source server
=> Connecting to destination server
=> Running sanity checks

=== Source server
LXD version: 3.0.1
LXD PID: 2656
Resources:
Containers: 6
Images: 4
Networks: 1
Storage pools: 1

=== Destination server
LXD version: 3.3
LXD PID: 12791
Resources:
Containers: 0
Images: 0
Networks: 0
Storage pools: 0

The migration process will shut down all your containers then move your
data to the destination LXD.
Once the data is moved, the destination LXD will start and apply any
needed updates.
And finally your containers will be brought back to their previous
state, completing the migration.

WARNING: /var/lib/lxd is a mountpoint. You will need to update that
mount location after the migration.

Are you ready to proceed (yes/no) [default=no]? yes
=> Shutting down the source LXD
=> Stopping the source LXD units
=> Stopping the destination LXD unit
=> Unmounting source LXD paths
=> Unmounting destination LXD paths
=> Wiping destination LXD clean
=> Backing up the database
=> Moving the /var/lib/lxd mountpoint
=> Updating the storage backends
=> Starting the destination LXD
=> Waiting for LXD to come online

error: LXD still not running after 5 minutes.



***@b1 ~ # lxd.migrate
=> Connecting to source server
error: Unable to connect to the source LXD: Get http://unix.socket/1.0:
dial unix /var/lib/lxd/unix.socket: connect: no such file or directory



***@b1 ~ # lxc list
Error: Get http://unix.socket/1.0: dial unix /var/lib/lxd/unix.socket:
connect: no such file or directory



Not sure how to recover now? The containers seem intact in /var/lib/lxd/


Tomasz Chmielewski
https://lxadm.com
Tomasz Chmielewski
2018-08-08 19:33:04 UTC
Permalink
This post might be inappropriate. Click to display it.
Stéphane Graber
2018-08-08 20:26:49 UTC
Permalink
Post by Tomasz Chmielewski
I've tried to migrate from deb to snap on Ubuntu 18.04.
Unfortunately, lxd.migrate failed with "error: LXD still not running after 5
=> Connecting to source server
=> Connecting to destination server
=> Running sanity checks
=== Source server
LXD version: 3.0.1
LXD PID: 2656
Containers: 6
Images: 4
Networks: 1
Storage pools: 1
=== Destination server
LXD version: 3.3
LXD PID: 12791
Containers: 0
Images: 0
Networks: 0
Storage pools: 0
The migration process will shut down all your containers then move your data
to the destination LXD.
Once the data is moved, the destination LXD will start and apply any needed
updates.
And finally your containers will be brought back to their previous state,
completing the migration.
WARNING: /var/lib/lxd is a mountpoint. You will need to update that mount
location after the migration.
Are you ready to proceed (yes/no) [default=no]? yes
=> Shutting down the source LXD
=> Stopping the source LXD units
=> Stopping the destination LXD unit
=> Unmounting source LXD paths
=> Unmounting destination LXD paths
=> Wiping destination LXD clean
=> Backing up the database
=> Moving the /var/lib/lxd mountpoint
=> Updating the storage backends
=> Starting the destination LXD
=> Waiting for LXD to come online
error: LXD still not running after 5 minutes.
=> Connecting to source server
error: Unable to connect to the source LXD: Get http://unix.socket/1.0: dial
unix /var/lib/lxd/unix.socket: connect: no such file or directory
connect: no such file or directory
Not sure how to recover now? The containers seem intact in /var/lib/lxd/
What do you get if you do "journalctl -u snap.lxd.daemon -n 300" and
anything useful looking in /var/snap/lxd/common/lxd/logs/lxd.log?

It's expected that "systemctl start lxd" won't work anymore since the
data was moved over to the snap which then likely caused your database
to be upgraded, making it unreadable for your older deb version of LXD.

I'd recommend you do:
- systemctl stop lxd lxd.socket
- systemctl mask lxd lxd.socket

To prevent any accidental startup of your old LXD until the snap
migration is done and it can be safely removed.
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
Tomasz Chmielewski
2018-08-08 21:26:10 UTC
Permalink
Post by Stéphane Graber
Post by Tomasz Chmielewski
Not sure how to recover now? The containers seem intact in
/var/lib/lxd/
What do you get if you do "journalctl -u snap.lxd.daemon -n 300" and
-- Logs begin at Thu 2018-07-12 06:07:13 UTC, end at Wed 2018-08-08
21:07:13 UTC. --
Aug 08 18:21:12 b1 systemd[1]: Started Service for snap application
lxd.daemon.
Aug 08 18:21:12 b1 lxd.daemon[12581]: => Preparing the system
Aug 08 18:21:12 b1 lxd.daemon[12581]: ==> Creating missing snap
configuration
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Loading snap configuration
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up mntns symlink
(mnt:[4026532794])
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up kmod wrapper
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Preparing /boot
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Preparing a clean copy of /run
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Preparing a clean copy of /etc
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up ceph configuration
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up LVM configuration
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Rotating logs
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up ZFS (0.7)
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Escaping the systemd cgroups
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Escaping the systemd process
resource limits
Aug 08 18:21:41 b1 systemd[1]: Stopping Service for snap application
lxd.daemon...
Aug 08 18:21:42 b1 lxd.daemon[13595]: => Stop reason is: host shutdown
Aug 08 18:21:42 b1 lxd.daemon[13595]: => Stopping LXD (with container
shutdown)
Aug 08 18:21:42 b1 lxd.daemon[13595]: => Stopping LXCFS
Aug 08 18:21:43 b1 systemd[1]: Stopped Service for snap application
lxd.daemon.
Aug 08 18:21:44 b1 systemd[1]: Started Service for snap application
lxd.daemon.
Aug 08 18:21:44 b1 lxd.daemon[13676]: => Preparing the system
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Loading snap configuration
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up mntns symlink
(mnt:[4026532794])
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up kmod wrapper
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Preparing /boot
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Preparing a clean copy of /run
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Preparing a clean copy of /etc
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up ceph configuration
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up LVM configuration
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Rotating logs
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up ZFS (0.7)
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Escaping the systemd cgroups
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Escaping the systemd process
resource limits
Aug 08 18:21:44 b1 lxd.daemon[13676]: => Starting LXCFS
Aug 08 18:21:44 b1 lxd.daemon[13676]: => Starting LXD
Aug 08 18:21:44 b1 lxd.daemon[13676]: lvl=warn msg="AppArmor support has
been disabled because of lack of kernel support"
t=2018-08-08T18:21:44+0000
Aug 08 18:21:44 b1 lxd.daemon[13676]: lvl=warn msg="CGroup memory swap
accounting is disabled, swap limits will be ignored."
t=2018-08-08T18:21:44+0000
Aug 08 18:21:44 b1 lxd.daemon[13676]: mount namespace: 5
Aug 08 18:21:44 b1 lxd.daemon[13676]: hierarchies:
Aug 08 18:21:44 b1 lxd.daemon[13676]: 0: fd: 6: hugetlb
Aug 08 18:21:44 b1 lxd.daemon[13676]: 1: fd: 7: pids
Aug 08 18:21:44 b1 lxd.daemon[13676]: 2: fd: 8: cpuset
Aug 08 18:21:44 b1 lxd.daemon[13676]: 3: fd: 9: perf_event
Aug 08 18:21:44 b1 lxd.daemon[13676]: 4: fd: 10: freezer
Aug 08 18:21:44 b1 lxd.daemon[13676]: 5: fd: 11: memory
Aug 08 18:21:44 b1 lxd.daemon[13676]: 6: fd: 12: devices
Aug 08 18:21:44 b1 lxd.daemon[13676]: 7: fd: 13: blkio
Aug 08 18:21:44 b1 lxd.daemon[13676]: 8: fd: 14: cpu,cpuacct
Aug 08 18:21:44 b1 lxd.daemon[13676]: 9: fd: 15: net_cls,net_prio
Aug 08 18:21:44 b1 lxd.daemon[13676]: 10: fd: 16: rdma
Aug 08 18:21:44 b1 lxd.daemon[13676]: 11: fd: 17: name=systemd
Aug 08 18:21:44 b1 lxd.daemon[13676]: 12: fd: 18: unified
Aug 08 18:28:07 b1 systemd[1]: Stopping Service for snap application
lxd.daemon...
Aug 08 18:28:07 b1 lxd.daemon[18773]: => Stop reason is: host shutdown
Aug 08 18:28:07 b1 lxd.daemon[18773]: => Stopping LXD (with container
shutdown)
Aug 08 18:37:24 b1 lxd.daemon[18773]: => Stopping LXCFS
Aug 08 18:37:25 b1 systemd[1]: Stopped Service for snap application
lxd.daemon.
-- Reboot --
Aug 08 18:38:55 b1 systemd[1]: Started Service for snap application
lxd.daemon.
Aug 08 18:38:57 b1 lxd.daemon[2345]: => Preparing the system
Aug 08 18:38:57 b1 lxd.daemon[2345]: ==> Loading snap configuration
Aug 08 18:38:57 b1 lxd.daemon[2345]: ==> Setting up mntns symlink
(mnt:[4026532463])
Aug 08 18:38:57 b1 lxd.daemon[2345]: ==> Setting up kmod wrapper
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Preparing /boot
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Preparing a clean copy of /run
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Preparing a clean copy of /etc
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Setting up ceph configuration
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Setting up LVM configuration
Aug 08 18:38:59 b1 lxd.daemon[2345]: ==> Rotating logs
Aug 08 18:38:59 b1 lxd.daemon[2345]: ==> Setting up ZFS (0.7)
Aug 08 18:38:59 b1 lxd.daemon[2345]: ==> Escaping the systemd cgroups
Aug 08 18:38:59 b1 lxd.daemon[2345]: ==> Escaping the systemd process
resource limits
Aug 08 18:38:59 b1 lxd.daemon[2345]: => Starting LXCFS
Aug 08 18:38:59 b1 lxd.daemon[2345]: => Starting LXD
Aug 08 18:38:59 b1 lxd.daemon[2345]: mount namespace: 5
Aug 08 18:38:59 b1 lxd.daemon[2345]: hierarchies:
Aug 08 18:38:59 b1 lxd.daemon[2345]: 0: fd: 6: devices
Aug 08 18:38:59 b1 lxd.daemon[2345]: 1: fd: 7: cpuset
Aug 08 18:38:59 b1 lxd.daemon[2345]: 2: fd: 8: blkio
Aug 08 18:38:59 b1 lxd.daemon[2345]: 3: fd: 9: memory
Aug 08 18:38:59 b1 lxd.daemon[2345]: 4: fd: 10: rdma
Aug 08 18:38:59 b1 lxd.daemon[2345]: 5: fd: 11: cpu,cpuacct
Aug 08 18:38:59 b1 lxd.daemon[2345]: 6: fd: 12: freezer
Aug 08 18:38:59 b1 lxd.daemon[2345]: 7: fd: 13: hugetlb
Aug 08 18:38:59 b1 lxd.daemon[2345]: 8: fd: 14: net_cls,net_prio
Aug 08 18:38:59 b1 lxd.daemon[2345]: 9: fd: 15: perf_event
Aug 08 18:38:59 b1 lxd.daemon[2345]: 10: fd: 16: pids
Aug 08 18:38:59 b1 lxd.daemon[2345]: 11: fd: 17: name=systemd
Aug 08 18:38:59 b1 lxd.daemon[2345]: 12: fd: 18: unified
Aug 08 18:38:59 b1 lxd.daemon[2345]: lvl=warn msg="AppArmor support has
been disabled because of lack of kernel support"
t=2018-08-08T18:38:59+0000
Aug 08 18:38:59 b1 lxd.daemon[2345]: lvl=warn msg="CGroup memory swap
accounting is disabled, swap limits will be ignored."
t=2018-08-08T18:38:59+0000
Post by Stéphane Graber
anything useful looking in /var/snap/lxd/common/lxd/logs/lxd.log?
# cat /var/snap/lxd/common/lxd/logs/lxd.log
lvl=info msg="LXD 3.3 is starting in normal mode"
path=/var/snap/lxd/common/lxd t=2018-08-08T18:38:59+0000
lvl=info msg="Kernel uid/gid map:" t=2018-08-08T18:38:59+0000
lvl=info msg=" - u 0 0 4294967295" t=2018-08-08T18:38:59+0000
lvl=info msg=" - g 0 0 4294967295" t=2018-08-08T18:38:59+0000
lvl=info msg="Configured LXD uid/gid map:" t=2018-08-08T18:38:59+0000
lvl=info msg=" - u 0 1000000 1000000000" t=2018-08-08T18:38:59+0000
lvl=info msg=" - g 0 1000000 1000000000" t=2018-08-08T18:38:59+0000
lvl=warn msg="AppArmor support has been disabled because of lack of
kernel support" t=2018-08-08T18:38:59+0000
lvl=warn msg="CGroup memory swap accounting is disabled, swap limits
will be ignored." t=2018-08-08T18:38:59+0000
lvl=info msg="Initializing local database" t=2018-08-08T18:38:59+0000
lvl=info msg="Initializing database gateway" t=2018-08-08T18:39:07+0000
address= id=1 lvl=info msg="Start database node"
t=2018-08-08T18:39:07+0000
lvl=info msg="Raft: Initial configuration (index=1): [{Suffrage:Voter
ID:1 Address:0}]" t=2018-08-08T18:39:07+0000
lvl=info msg="Raft: Node at 0 [Leader] entering Leader state"
t=2018-08-08T18:39:07+0000
lvl=info msg="LXD isn't socket activated" t=2018-08-08T18:39:07+0000
lvl=info msg="Starting /dev/lxd handler:" t=2018-08-08T18:39:07+0000
lvl=info msg=" - binding devlxd socket"
socket=/var/snap/lxd/common/lxd/devlxd/sock t=2018-08-08T18:39:07+0000
lvl=info msg="REST API daemon:" t=2018-08-08T18:39:07+0000
lvl=info msg=" - binding Unix socket"
socket=/var/snap/lxd/common/lxd/unix.socket t=2018-08-08T18:39:07+0000
lvl=info msg="Initializing global database" t=2018-08-08T18:39:07+0000
lvl=info msg="Initializing storage pools" t=2018-08-08T18:39:08+0000
lvl=info msg="Initializing networks" t=2018-08-08T18:39:08+0000
lvl=info msg="Loading configuration" t=2018-08-08T18:39:08+0000
lvl=info msg="Connected to MAAS controller" t=2018-08-08T18:39:08+0000
lvl=info msg="Pruning expired images" t=2018-08-08T18:39:08+0000
lvl=info msg="Done pruning expired images" t=2018-08-08T18:39:08+0000
lvl=info msg="Updating instance types" t=2018-08-08T18:39:08+0000
lvl=info msg="Expiring log files" t=2018-08-08T18:39:08+0000
lvl=info msg="Updating images" t=2018-08-08T18:39:08+0000
lvl=info msg="Done updating images" t=2018-08-08T18:39:08+0000
lvl=info msg="Done expiring log files" t=2018-08-08T18:39:08+0000
lvl=info msg="Done updating instance types" t=2018-08-08T18:39:10+0000
Post by Stéphane Graber
It's expected that "systemctl start lxd" won't work anymore since the
data was moved over to the snap which then likely caused your database
to be upgraded, making it unreadable for your older deb version of LXD.
- systemctl stop lxd lxd.socket
- systemctl mask lxd lxd.socket
To prevent any accidental startup of your old LXD until the snap
migration is done and it can be safely removed.
Let's try:

# systemctl stop lxd lxd.socket
# systemctl mask lxd lxd.socket
Created symlink /etc/systemd/system/lxd.service → /dev/null.
Created symlink /etc/systemd/system/lxd.socket → /dev/null.

# lxd.migrate
=> Connecting to source server
error: Unable to connect to the source LXD: Get http://unix.socket/1.0:
dial unix /var/lib/lxd/unix.socket: connect: no such file or directory


# /snap/bin/lxc list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+



So I did the following:

- mv /var/snap/lxd/common/lxd /var/snap/lxd/common/lxd.orig

- modified /etc/fstab to mount the previous /var/lib/lxd to
/var/snap/lxd/common/lxd

- rebooted

- removed lxd deb packages

And it now started properly.

Thanks for helping!


Tomasz Chmielewski
https://lxadm.com
Stéphane Graber
2018-08-08 21:48:16 UTC
Permalink
Post by Stéphane Graber
Post by Tomasz Chmielewski
Not sure how to recover now? The containers seem intact in
/var/lib/lxd/
What do you get if you do "journalctl -u snap.lxd.daemon -n 300" and
-- Logs begin at Thu 2018-07-12 06:07:13 UTC, end at Wed 2018-08-08 21:07:13
UTC. --
Aug 08 18:21:12 b1 systemd[1]: Started Service for snap application
lxd.daemon.
Aug 08 18:21:12 b1 lxd.daemon[12581]: => Preparing the system
Aug 08 18:21:12 b1 lxd.daemon[12581]: ==> Creating missing snap
configuration
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Loading snap configuration
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up mntns symlink
(mnt:[4026532794])
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up kmod wrapper
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Preparing /boot
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Preparing a clean copy of /run
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Preparing a clean copy of /etc
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up ceph configuration
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up LVM configuration
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Rotating logs
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Setting up ZFS (0.7)
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Escaping the systemd cgroups
Aug 08 18:21:13 b1 lxd.daemon[12581]: ==> Escaping the systemd process
resource limits
Aug 08 18:21:41 b1 systemd[1]: Stopping Service for snap application
lxd.daemon...
Aug 08 18:21:42 b1 lxd.daemon[13595]: => Stop reason is: host shutdown
Aug 08 18:21:42 b1 lxd.daemon[13595]: => Stopping LXD (with container
shutdown)
Aug 08 18:21:42 b1 lxd.daemon[13595]: => Stopping LXCFS
Aug 08 18:21:43 b1 systemd[1]: Stopped Service for snap application
lxd.daemon.
Aug 08 18:21:44 b1 systemd[1]: Started Service for snap application
lxd.daemon.
Aug 08 18:21:44 b1 lxd.daemon[13676]: => Preparing the system
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Loading snap configuration
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up mntns symlink
(mnt:[4026532794])
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up kmod wrapper
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Preparing /boot
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Preparing a clean copy of /run
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Preparing a clean copy of /etc
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up ceph configuration
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up LVM configuration
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Rotating logs
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Setting up ZFS (0.7)
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Escaping the systemd cgroups
Aug 08 18:21:44 b1 lxd.daemon[13676]: ==> Escaping the systemd process
resource limits
Aug 08 18:21:44 b1 lxd.daemon[13676]: => Starting LXCFS
Aug 08 18:21:44 b1 lxd.daemon[13676]: => Starting LXD
Aug 08 18:21:44 b1 lxd.daemon[13676]: lvl=warn msg="AppArmor support has
been disabled because of lack of kernel support" t=2018-08-08T18:21:44+0000
Aug 08 18:21:44 b1 lxd.daemon[13676]: lvl=warn msg="CGroup memory swap
accounting is disabled, swap limits will be ignored."
t=2018-08-08T18:21:44+0000
Aug 08 18:21:44 b1 lxd.daemon[13676]: mount namespace: 5
Aug 08 18:21:44 b1 lxd.daemon[13676]: 0: fd: 6: hugetlb
Aug 08 18:21:44 b1 lxd.daemon[13676]: 1: fd: 7: pids
Aug 08 18:21:44 b1 lxd.daemon[13676]: 2: fd: 8: cpuset
Aug 08 18:21:44 b1 lxd.daemon[13676]: 3: fd: 9: perf_event
Aug 08 18:21:44 b1 lxd.daemon[13676]: 4: fd: 10: freezer
Aug 08 18:21:44 b1 lxd.daemon[13676]: 5: fd: 11: memory
Aug 08 18:21:44 b1 lxd.daemon[13676]: 6: fd: 12: devices
Aug 08 18:21:44 b1 lxd.daemon[13676]: 7: fd: 13: blkio
Aug 08 18:21:44 b1 lxd.daemon[13676]: 8: fd: 14: cpu,cpuacct
Aug 08 18:21:44 b1 lxd.daemon[13676]: 9: fd: 15: net_cls,net_prio
Aug 08 18:21:44 b1 lxd.daemon[13676]: 10: fd: 16: rdma
Aug 08 18:21:44 b1 lxd.daemon[13676]: 11: fd: 17: name=systemd
Aug 08 18:21:44 b1 lxd.daemon[13676]: 12: fd: 18: unified
Aug 08 18:28:07 b1 systemd[1]: Stopping Service for snap application
lxd.daemon...
Aug 08 18:28:07 b1 lxd.daemon[18773]: => Stop reason is: host shutdown
Aug 08 18:28:07 b1 lxd.daemon[18773]: => Stopping LXD (with container
shutdown)
Aug 08 18:37:24 b1 lxd.daemon[18773]: => Stopping LXCFS
Aug 08 18:37:25 b1 systemd[1]: Stopped Service for snap application
lxd.daemon.
-- Reboot --
Aug 08 18:38:55 b1 systemd[1]: Started Service for snap application
lxd.daemon.
Aug 08 18:38:57 b1 lxd.daemon[2345]: => Preparing the system
Aug 08 18:38:57 b1 lxd.daemon[2345]: ==> Loading snap configuration
Aug 08 18:38:57 b1 lxd.daemon[2345]: ==> Setting up mntns symlink
(mnt:[4026532463])
Aug 08 18:38:57 b1 lxd.daemon[2345]: ==> Setting up kmod wrapper
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Preparing /boot
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Preparing a clean copy of /run
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Preparing a clean copy of /etc
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Setting up ceph configuration
Aug 08 18:38:58 b1 lxd.daemon[2345]: ==> Setting up LVM configuration
Aug 08 18:38:59 b1 lxd.daemon[2345]: ==> Rotating logs
Aug 08 18:38:59 b1 lxd.daemon[2345]: ==> Setting up ZFS (0.7)
Aug 08 18:38:59 b1 lxd.daemon[2345]: ==> Escaping the systemd cgroups
Aug 08 18:38:59 b1 lxd.daemon[2345]: ==> Escaping the systemd process
resource limits
Aug 08 18:38:59 b1 lxd.daemon[2345]: => Starting LXCFS
Aug 08 18:38:59 b1 lxd.daemon[2345]: => Starting LXD
Aug 08 18:38:59 b1 lxd.daemon[2345]: mount namespace: 5
Aug 08 18:38:59 b1 lxd.daemon[2345]: 0: fd: 6: devices
Aug 08 18:38:59 b1 lxd.daemon[2345]: 1: fd: 7: cpuset
Aug 08 18:38:59 b1 lxd.daemon[2345]: 2: fd: 8: blkio
Aug 08 18:38:59 b1 lxd.daemon[2345]: 3: fd: 9: memory
Aug 08 18:38:59 b1 lxd.daemon[2345]: 4: fd: 10: rdma
Aug 08 18:38:59 b1 lxd.daemon[2345]: 5: fd: 11: cpu,cpuacct
Aug 08 18:38:59 b1 lxd.daemon[2345]: 6: fd: 12: freezer
Aug 08 18:38:59 b1 lxd.daemon[2345]: 7: fd: 13: hugetlb
Aug 08 18:38:59 b1 lxd.daemon[2345]: 8: fd: 14: net_cls,net_prio
Aug 08 18:38:59 b1 lxd.daemon[2345]: 9: fd: 15: perf_event
Aug 08 18:38:59 b1 lxd.daemon[2345]: 10: fd: 16: pids
Aug 08 18:38:59 b1 lxd.daemon[2345]: 11: fd: 17: name=systemd
Aug 08 18:38:59 b1 lxd.daemon[2345]: 12: fd: 18: unified
Aug 08 18:38:59 b1 lxd.daemon[2345]: lvl=warn msg="AppArmor support has been
disabled because of lack of kernel support" t=2018-08-08T18:38:59+0000
Aug 08 18:38:59 b1 lxd.daemon[2345]: lvl=warn msg="CGroup memory swap
accounting is disabled, swap limits will be ignored."
t=2018-08-08T18:38:59+0000
Post by Stéphane Graber
anything useful looking in /var/snap/lxd/common/lxd/logs/lxd.log?
# cat /var/snap/lxd/common/lxd/logs/lxd.log
lvl=info msg="LXD 3.3 is starting in normal mode"
path=/var/snap/lxd/common/lxd t=2018-08-08T18:38:59+0000
lvl=info msg="Kernel uid/gid map:" t=2018-08-08T18:38:59+0000
lvl=info msg=" - u 0 0 4294967295" t=2018-08-08T18:38:59+0000
lvl=info msg=" - g 0 0 4294967295" t=2018-08-08T18:38:59+0000
lvl=info msg="Configured LXD uid/gid map:" t=2018-08-08T18:38:59+0000
lvl=info msg=" - u 0 1000000 1000000000" t=2018-08-08T18:38:59+0000
lvl=info msg=" - g 0 1000000 1000000000" t=2018-08-08T18:38:59+0000
lvl=warn msg="AppArmor support has been disabled because of lack of kernel
support" t=2018-08-08T18:38:59+0000
lvl=warn msg="CGroup memory swap accounting is disabled, swap limits will be
ignored." t=2018-08-08T18:38:59+0000
lvl=info msg="Initializing local database" t=2018-08-08T18:38:59+0000
lvl=info msg="Initializing database gateway" t=2018-08-08T18:39:07+0000
address= id=1 lvl=info msg="Start database node" t=2018-08-08T18:39:07+0000
lvl=info msg="Raft: Initial configuration (index=1): [{Suffrage:Voter ID:1
Address:0}]" t=2018-08-08T18:39:07+0000
lvl=info msg="Raft: Node at 0 [Leader] entering Leader state"
t=2018-08-08T18:39:07+0000
lvl=info msg="LXD isn't socket activated" t=2018-08-08T18:39:07+0000
lvl=info msg="Starting /dev/lxd handler:" t=2018-08-08T18:39:07+0000
lvl=info msg=" - binding devlxd socket"
socket=/var/snap/lxd/common/lxd/devlxd/sock t=2018-08-08T18:39:07+0000
lvl=info msg="REST API daemon:" t=2018-08-08T18:39:07+0000
lvl=info msg=" - binding Unix socket"
socket=/var/snap/lxd/common/lxd/unix.socket t=2018-08-08T18:39:07+0000
lvl=info msg="Initializing global database" t=2018-08-08T18:39:07+0000
lvl=info msg="Initializing storage pools" t=2018-08-08T18:39:08+0000
lvl=info msg="Initializing networks" t=2018-08-08T18:39:08+0000
lvl=info msg="Loading configuration" t=2018-08-08T18:39:08+0000
lvl=info msg="Connected to MAAS controller" t=2018-08-08T18:39:08+0000
lvl=info msg="Pruning expired images" t=2018-08-08T18:39:08+0000
lvl=info msg="Done pruning expired images" t=2018-08-08T18:39:08+0000
lvl=info msg="Updating instance types" t=2018-08-08T18:39:08+0000
lvl=info msg="Expiring log files" t=2018-08-08T18:39:08+0000
lvl=info msg="Updating images" t=2018-08-08T18:39:08+0000
lvl=info msg="Done updating images" t=2018-08-08T18:39:08+0000
lvl=info msg="Done expiring log files" t=2018-08-08T18:39:08+0000
lvl=info msg="Done updating instance types" t=2018-08-08T18:39:10+0000
Post by Stéphane Graber
It's expected that "systemctl start lxd" won't work anymore since the
data was moved over to the snap which then likely caused your database
to be upgraded, making it unreadable for your older deb version of LXD.
- systemctl stop lxd lxd.socket
- systemctl mask lxd lxd.socket
To prevent any accidental startup of your old LXD until the snap
migration is done and it can be safely removed.
# systemctl stop lxd lxd.socket
# systemctl mask lxd lxd.socket
Created symlink /etc/systemd/system/lxd.service → /dev/null.
Created symlink /etc/systemd/system/lxd.socket → /dev/null.
# lxd.migrate
=> Connecting to source server
error: Unable to connect to the source LXD: Get http://unix.socket/1.0: dial
unix /var/lib/lxd/unix.socket: connect: no such file or directory
# /snap/bin/lxc list
+------+-------+------+------+------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------+-------+------+------+------+-----------+
- mv /var/snap/lxd/common/lxd /var/snap/lxd/common/lxd.orig
- modified /etc/fstab to mount the previous /var/lib/lxd to
/var/snap/lxd/common/lxd
- rebooted
- removed lxd deb packages
And it now started properly.
Ah yeah, not sure why it didn't work immediately after upgrade, the logs
don't help much there.

The fstab change was indeed necessary and was hinted by lxd.migrate with:

WARNING: /var/lib/lxd is a mountpoint. You will need to update that mount location after the migration.

We preferred that over trying to mess with people's fstab ourselves :)
Thanks for helping!
Tomasz Chmielewski
https://lxadm.com
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
Saint Michael
2018-08-08 23:04:31 UTC
Permalink
Has anybody invented a procedure, a script, etc., to convert a running
machine to a LXC container? I was thinking to create a container of the
same OS, and then use rsync, excluding /proc /tmp/ /sys etc. Any ideas?
Stéphane Graber
2018-08-08 23:16:19 UTC
Permalink
Post by Saint Michael
Has anybody invented a procedure, a script, etc., to convert a running
machine to a LXC container? I was thinking to create a container of the
same OS, and then use rsync, excluding /proc /tmp/ /sys etc. Any ideas?
We have that for LXD, it's lxd-p2c and does pretty much exactly what you
describe but uses the LXD migration API as the rsync target effectively
(so rsync over custom websocket over https).
--
Stéphane Graber
Ubuntu developer
http://www.ubuntu.com
David Favor
2018-08-08 23:17:01 UTC
Permalink
Post by Saint Michael
Has anybody invented a procedure, a script, etc., to convert a running
machine to a LXC container? I was thinking to create a container of the
same OS, and then use rsync, excluding /proc /tmp/ /sys etc. Any ideas?
Use the fabulous lxd-p2c script.
Saint Michael
2018-08-09 02:18:41 UTC
Permalink
The question is how can I use that for plan LXC.
I can install a box with LXD, bring the computer in, but then I want a
plain LXC container.
Is it doable?
Post by David Favor
Post by Saint Michael
Has anybody invented a procedure, a script, etc., to convert a running
machine to a LXC container? I was thinking to create a container of the
same OS, and then use rsync, excluding /proc /tmp/ /sys etc. Any ideas?
Use the fabulous lxd-p2c script.
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Fajar A. Nugraha
2018-08-09 03:21:05 UTC
Permalink
I've converted (manually) some lxc containers to lxd and back in the past.
IIRC the biggest difference was that lxd does not need to output anything
to console, while lxc needs it (e.g. for lxc-attach). Depending on what
container distro and version you use, it might not matter (e.g. it should
"just work" for newer ubuntu), and you can use the same container rootfs
for both.

If you use anything else and simple 'rsync --numeric-ids)' doesn't work,
take a look at the customizations done by lxc-template to find out what
else need to be adjusted, e.g.
https://github.com/lxc/lxc-templates/blob/master/templates/lxc-debian.in#L67
for old debian system that still use sysvinit.
--
Fajar
Post by Saint Michael
The question is how can I use that for plan LXC.
I can install a box with LXD, bring the computer in, but then I want a
plain LXC container.
Is it doable?
Post by David Favor
Post by Saint Michael
Has anybody invented a procedure, a script, etc., to convert a running
machine to a LXC container? I was thinking to create a container of the
same OS, and then use rsync, excluding /proc /tmp/ /sys etc. Any ideas?
Use the fabulous lxd-p2c script.
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Saint Michael
2018-08-09 03:59:09 UTC
Permalink
LXD does not support lxc-attach?
I thought that LXD was a superset of LXC, that added on top of it.
Maybe somebody care to explain how LXC and LXD compare.
Post by Fajar A. Nugraha
I've converted (manually) some lxc containers to lxd and back in the past.
IIRC the biggest difference was that lxd does not need to output anything
to console, while lxc needs it (e.g. for lxc-attach). Depending on what
container distro and version you use, it might not matter (e.g. it should
"just work" for newer ubuntu), and you can use the same container rootfs
for both.
If you use anything else and simple 'rsync --numeric-ids)' doesn't work,
take a look at the customizations done by lxc-template to find out what
else need to be adjusted, e.g.
https://github.com/lxc/lxc-templates/blob/master/templates/lxc-debian.in#L67
for old debian system that still use sysvinit.
--
Fajar
Post by Saint Michael
The question is how can I use that for plan LXC.
I can install a box with LXD, bring the computer in, but then I want a
plain LXC container.
Is it doable?
Post by Saint Michael
Post by Saint Michael
Has anybody invented a procedure, a script, etc., to convert a running
machine to a LXC container? I was thinking to create a container of
the
Post by Saint Michael
same OS, and then use rsync, excluding /proc /tmp/ /sys etc. Any
ideas?
Use the fabulous lxd-p2c script.
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Fajar A. Nugraha
2018-08-09 04:19:24 UTC
Permalink
Sorry, my mistake. I meant "lxc-console". And I just rechecked, apparently
there's the equivalent "lxc console" command as well.

Ignore my comment about "lxc-attach" earlier. You should be able to use lxd
rootfs for lxc as long as:
- you have the correct uid mapping (it's simpler if you just use privileged
container. Otherwise just setup uid map on lxc config file manually)
- you have a suitable lxc config file (the easiest way is probably to
create a new lxc container, then replace the original rootfs with the one
from rsync/lxd-p2c)
--
Fajar
Post by Saint Michael
LXD does not support lxc-attach?
I thought that LXD was a superset of LXC, that added on top of it.
Maybe somebody care to explain how LXC and LXD compare.
Post by Fajar A. Nugraha
I've converted (manually) some lxc containers to lxd and back in the
past. IIRC the biggest difference was that lxd does not need to output
anything to console, while lxc needs it (e.g. for lxc-attach). Depending on
what container distro and version you use, it might not matter (e.g. it
should "just work" for newer ubuntu), and you can use the same container
rootfs for both.
If you use anything else and simple 'rsync --numeric-ids)' doesn't work,
take a look at the customizations done by lxc-template to find out what
else need to be adjusted, e.g. https://github.com/lxc/
lxc-templates/blob/master/templates/lxc-debian.in#L67 for old debian
system that still use sysvinit.
--
Fajar
Post by Saint Michael
The question is how can I use that for plan LXC.
I can install a box with LXD, bring the computer in, but then I want a
plain LXC container.
Is it doable?
Post by Tomasz Chmielewski
Post by Saint Michael
Has anybody invented a procedure, a script, etc., to convert a
running
Post by Saint Michael
machine to a LXC container? I was thinking to create a container of
the
Post by Saint Michael
same OS, and then use rsync, excluding /proc /tmp/ /sys etc. Any
ideas?
Use the fabulous lxd-p2c script.
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Loading...