Discussion:
Kubernetes Storage Provisioning using LXD
(too old to reply)
Butch Landingin
2017-02-15 09:16:44 UTC
Permalink
Hi all,

I've been trying out Canonical Kubernetes via conjure-up and juju charms on
a local LXD cluster. Following the tutorials, I've set up the cluster
running on lxd with a zfs file system.

Everything's been great and I've pretty much exhausted the tutorials
(running microbot, etc).

I'm now at the point where I want to try provisioning some persistent
volumes or even trying
dynamic storage allocation.

By this time, I'm 99 percent sure the answer is no (and I've searched
extensively) ,
but is there anyway to create Kubernetes persistent volumes on a multi node
set up (not using hostpath)
on a local LXD cluster?

If there isn't, does Canonical have this in their roadmap?

Best regards,
Butch
Eric
2017-02-16 20:08:06 UTC
Permalink
That's is what I've also been trying to do

Kubernetes has a list of supported persistent volume types, of which the
only one's that aren't cloud-based that I've tried are NFS, CephFS,
Glusterfs, and HostPath

https://kubernetes.io/docs/user-guide/persistent-volumes/#types-of-persistent-volumes

With LXC + ZFS you can't:

- provide a raw block device (/dev/sad) to heketi, or a loopback device
(/dev/loop0). so glusterfs is out of the picture (
https://github.com/heketi/heketi/issues/665)
- cephfs is like glusterfs, so cephfs is out
- NFS requires kernel modules, so nfs is out
- HostPath doesn't work over multiple nodes

So your only option is to use KVM

I use Proxmox. I knew LXC/LXD wasn't going to be able to fulfill what I
needed to do on a single server, so I looked for a hypervisor that had a
polished UI for creating both LXC and KVM VM

I'm still going to go with glusterfs, which will also need heketi, and will
be running it in a fedora kvm. And the using it as a persistent volume for
kubernetes
Post by Butch Landingin
Hi all,
I've been trying out Canonical Kubernetes via conjure-up and juju charms on
a local LXD cluster. Following the tutorials, I've set up the cluster
running on lxd with a zfs file system.
Everything's been great and I've pretty much exhausted the tutorials
(running microbot, etc).
I'm now at the point where I want to try provisioning some persistent
volumes or even trying
dynamic storage allocation.
By this time, I'm 99 percent sure the answer is no (and I've searched
extensively) ,
but is there anyway to create Kubernetes persistent volumes on a multi
node set up (not using hostpath)
on a local LXD cluster?
If there isn't, does Canonical have this in their roadmap?
Best regards,
Butch
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Adam Stokes
2017-02-16 20:11:50 UTC
Permalink
Cross posting to juju lists. It is my understanding that if you add ceph
units to your juju environment or setup an NFS export that kubernetes can
make use of both of those. Someone from the containers team would know more.
Post by Eric
That's is what I've also been trying to do
Kubernetes has a list of supported persistent volume types, of which the
only one's that aren't cloud-based that I've tried are NFS, CephFS,
Glusterfs, and HostPath
https://kubernetes.io/docs/user-guide/persistent-volumes/#types-of-persistent-volumes
- provide a raw block device (/dev/sad) to heketi, or a loopback device
(/dev/loop0). so glusterfs is out of the picture (
https://github.com/heketi/heketi/issues/665)
- cephfs is like glusterfs, so cephfs is out
- NFS requires kernel modules, so nfs is out
- HostPath doesn't work over multiple nodes
So your only option is to use KVM
I use Proxmox. I knew LXC/LXD wasn't going to be able to fulfill what I
needed to do on a single server, so I looked for a hypervisor that had a
polished UI for creating both LXC and KVM VM
I'm still going to go with glusterfs, which will also need heketi, and
will be running it in a fedora kvm. And the using it as a persistent volume
for kubernetes
Hi all,
I've been trying out Canonical Kubernetes via conjure-up and juju charms on
a local LXD cluster. Following the tutorials, I've set up the cluster
running on lxd with a zfs file system.
Everything's been great and I've pretty much exhausted the tutorials
(running microbot, etc).
I'm now at the point where I want to try provisioning some persistent
volumes or even trying
dynamic storage allocation.
By this time, I'm 99 percent sure the answer is no (and I've searched
extensively) ,
but is there anyway to create Kubernetes persistent volumes on a multi
node set up (not using hostpath)
on a local LXD cluster?
If there isn't, does Canonical have this in their roadmap?
Best regards,
Butch
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Jared Folkins
2017-02-16 20:16:29 UTC
Permalink
FWIW, it is worth keeping your eye on this github issue. Personally I'd
love *full* support for LXD in K8s.

https://github.com/kubernetes/kubernetes/issues/6862
Post by Adam Stokes
Cross posting to juju lists. It is my understanding that if you add ceph
units to your juju environment or setup an NFS export that kubernetes can
make use of both of those. Someone from the containers team would know more.
Post by Eric
That's is what I've also been trying to do
Kubernetes has a list of supported persistent volume types, of which the
only one's that aren't cloud-based that I've tried are NFS, CephFS,
Glusterfs, and HostPath
https://kubernetes.io/docs/user-guide/persistent-volumes/
#types-of-persistent-volumes
- provide a raw block device (/dev/sad) to heketi, or a loopback device
(/dev/loop0). so glusterfs is out of the picture (
https://github.com/heketi/heketi/issues/665)
- cephfs is like glusterfs, so cephfs is out
- NFS requires kernel modules, so nfs is out
- HostPath doesn't work over multiple nodes
So your only option is to use KVM
I use Proxmox. I knew LXC/LXD wasn't going to be able to fulfill what I
needed to do on a single server, so I looked for a hypervisor that had a
polished UI for creating both LXC and KVM VM
I'm still going to go with glusterfs, which will also need heketi, and
will be running it in a fedora kvm. And the using it as a persistent volume
for kubernetes
Hi all,
I've been trying out Canonical Kubernetes via conjure-up and juju charms on
a local LXD cluster. Following the tutorials, I've set up the cluster
running on lxd with a zfs file system.
Everything's been great and I've pretty much exhausted the tutorials
(running microbot, etc).
I'm now at the point where I want to try provisioning some persistent
volumes or even trying
dynamic storage allocation.
By this time, I'm 99 percent sure the answer is no (and I've searched
extensively) ,
but is there anyway to create Kubernetes persistent volumes on a multi
node set up (not using hostpath)
on a local LXD cluster?
If there isn't, does Canonical have this in their roadmap?
Best regards,
Butch
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
_______________________________________________
lxc-users mailing list
http://lists.linuxcontainers.org/listinfo/lxc-users
Charles Butler
2017-02-16 23:22:32 UTC
Permalink
This post might be inappropriate. Click to display it.
Serge E. Hallyn
2017-02-16 20:34:48 UTC
Permalink
Post by Eric
That's is what I've also been trying to do
Kubernetes has a list of supported persistent volume types, of which the
only one's that aren't cloud-based that I've tried are NFS, CephFS,
Glusterfs, and HostPath
https://kubernetes.io/docs/user-guide/persistent-volumes/#types-of-persistent-volumes
- provide a raw block device (/dev/sad) to heketi, or a loopback device
(/dev/loop0). so glusterfs is out of the picture (
https://github.com/heketi/heketi/issues/665)
- cephfs is like glusterfs, so cephfs is out
- NFS requires kernel modules, so nfs is out
- HostPath doesn't work over multiple nodes
Fwiw, while lxc currently does not support ceph or glusterfs, patches to add
such support would be welcome.

-serge
Loading...