[lxc-devel] lxc_container->start fork keeping server sockets alive?

[John Keiser]

Okey doke, thanks.  I've worked around the problem myself by shelling out
to the lxc_start executable, but I've filed
https://github.com/lxc/lxc/issues/191 so this doesn't get lost.  Would be
nice to not pay the cost of starting an executable, but I doubt that's
going to be my bottleneck for a long time :)


On Thu, Mar 27, 2014 at 3:35 PM, Serge Hallyn <serge.hallyn at ubuntu.com>wrote:

> The container doesn't do it bc it's a daemon, but because we don't
> want the container init to inherit (and be able to use) fds.
>
> The lxc-monitor has quite a few fds which it legitimately needs to have
> open - for instance signalfd, command socket, etc.  So while it should
> be possible and would be nice to do, it won't be particularly easy to
> get right.  I'll happily look at any patch you come up with though.
>
> Quoting John Keiser (jkeiser at getchef.com):
> > Ah!  Should it?  Like, should it do all the daemon things that the
> > container itself does?
> >
> >
> > On Thu, Mar 27, 2014 at 12:30 PM, Serge Hallyn <serge.hallyn at ubuntu.com
> >wrote:
> >
> > > Quoting John Keiser (jkeiser at getchef.com):
> > > > Hi all,
> > > >
> > > > I am trying to use lxc_container->start() from within a process that
> > > > happens to have started a TCP server, and even when my program
> exits, the
> > > > server socket stays alive.  This can be tested by running this
> program
> > > > (which starts a server and then starts a container) twice--the second
> > > time
> > > > resulting in a bind() error:
> > > >
> > > > #include <sys/socket.h>
> > > > #include <netinet/in.h>
> > > > #include <stdio.h>
> > > > #include <errno.h>
> > > > #include <strings.h>
> > > > #include <stdlib.h>
> > > > #include <lxc/lxccontainer.h>
> > > >
> > > > int main(int argc, char**argv)
> > > > {
> > > >   struct lxc_container *c;
> > > >   int listenfd,connfd,n;
> > > >   struct sockaddr_in servaddr;
> > > >   int port;
> > > >
> > > >   // Start server
> > > >   port = atoi(argv[2]);
> > > >   printf("Listening on port %d ...\n", port);
> > > >   listenfd=socket(AF_INET,SOCK_STREAM,0);
> > > >   bzero(&servaddr,sizeof(servaddr));
> > > >   servaddr.sin_family = AF_INET;
> > > >   servaddr.sin_addr.s_addr=htonl(INADDR_ANY);
> > > >   servaddr.sin_port=htons(port);
> > > >   if (bind(listenfd,(struct sockaddr *)&servaddr,sizeof(servaddr)))
> > > >   {
> > > >     perror("Error calling bind()");
> > > >     exit(1);
> > > >   }
> > > >   listen(listenfd,1024);
> > > >
> > > >   // Start
> > > >   printf("Starting container ...\n");
> > > >   c = lxc_container_new(argv[1], NULL);
> > > >   c->want_daemonize(c, true);
> > > >   c->want_close_all_fds(c, true);
> > > >   c->start(c, 0, NULL);
> > > >
> > > >   return 0;
> > > > }
> > > >
> > > >
> > > > *> gcc test.c -llxc -o test*
> > > > *> lxc-create -n blarghle -t ubuntu*
> > > >
> > > > *> ./test blarghle 9050*
> > > > Listening on port 9050 ...
> > > > Starting container ...
> > > > *> ./test blarghle 9050*
> > > > Listening on port 9050 ...
> > > > Error calling bind(): Address already in use
> > > >
> > > > As you can see, even though ./test finished the first time, the
> second
> > > > ./test cannot bind to the port because it is still in use.
> > > >
> > > > This appears to be because the lxc init process has a ghost "./test"
> as
> > > its
> > > > parent:
> > > >
> > > > *> ps -ef | grep 30276*
> > > > root     30276     1  0 18:37 ?        00:00:00 ./test blarghle 9050
> > > > root     30285 30276  0 18:37 ?        00:00:00 /sbin/init
> > > > *> lsof | grep 9050*
> > > > test      30276              root    3u     IPv4             366649
> > > >  0t0        TCP *:9050 (LISTEN)
> > >
> > > Hm.  That's the container monitor.  It doesn't close all fds like
> > > container init does.
> > >
> > > > I am stymied on how to investigate further.  My best guess is it has
> to
> > > do
> > > > with the double forking in
> > > > lxccontainer.h<
> > >
> https://github.com/lxc/lxc/blob/master/src/lxc/lxccontainer.c#L608-L621>,
> > > > though I don't really understand what that is for yet :)  Any
> thoughts?
> > > >
> > > > Thanks!
> > > >
> > > > --John
> > >
> > > > _______________________________________________
> > > > lxc-devel mailing list
> > > > lxc-devel at lists.linuxcontainers.org
> > > > http://lists.linuxcontainers.org/listinfo/lxc-devel
> > >
> > > _______________________________________________
> > > lxc-devel mailing list
> > > lxc-devel at lists.linuxcontainers.org
> > > http://lists.linuxcontainers.org/listinfo/lxc-devel
> > >
>
> > _______________________________________________
> > lxc-devel mailing list
> > lxc-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-devel
>
> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.linuxcontainers.org/pipermail/lxc-devel/attachments/20140327/714543c4/attachment.html>