[lxc-devel] Integration Kubernetes and LXD/LXC

[Free Ekanayaka]

Oliver Schad <oliver.schad at automatic-server.com> writes:

> On Tue, 02 Oct 2018 11:29:30 +0200
> Free Ekanayaka <free.ekanayaka at canonical.com> wrote:
>
>> Oliver Schad <oliver.schad at automatic-server.com> writes:
>> > If the container layer is unstable, you can't build a stable
>> > service on top of it.  
>> 
>> How does LXE solve the issue of undesired restarts? I imagine that the
>> restarts are triggered by the k8s control plane by connecting to the
>> kubelet which in turns triggers some imperative CRI API which says
>> "Please restart this pod". If that's the case, does LXE somehow ignore
>> the restart request? I'm confused about this part.
>
> It's true, that Kubelet does sometimes stuff which it shouldn't do and
> we filter some things from Kubelet. The imperative nature of CRI is bad
> in fact.
>
> But: it really makes a difference, if sometimes Kubelet is wrong with
> some things or the container engine itself creates problems (in case of
> restart, update, too much logs, ...).
>
> If the container engine dies for whatever reason in case of LXD,
> nothing happens. If Docker dies, all container dies. If Kubelet creates
> trouble, we're able to try to work around that problem with filtering.
> We saw and see a lot of restarts of Docker.
>
> In the area of platform services, it's hard to work one process
> containers. And in the area of platform services it's hard to kill your
> container, just because of updating a file. Both requirements you have
> especially for stateful services. Avoid restart as much as you can and
> if you have to restart something: do it in a planned/controlled way,
> with fine grained options (i.e. notify other cluster members about a
> node restart).

I know that folks to run stateful services on k8s, PostgreSQL is one of
those IIRC. I wouldn't expect MySQL do be fundamentally different.

Although LXE might be an approach that solves your immediate needs, it
feels like a band aid. If you haven't already, I'd recommend approaching
the k8s team/community describing the issues that you're seeing when
using standard CRI implementations such as containerd/docker and
cri-o. If you have already reached them out, I'd be interested get
pointers to their replies, since *in theory* stateful k8s services
shouldn't need any particular CRI implementation (and if that's not the
case in real-world, better report that).

Free