[lxc-devel] lxc_container->start fork keeping server sockets alive?

[Serge Hallyn]

The container doesn't do it bc it's a daemon, but because we don't
want the container init to inherit (and be able to use) fds.

The lxc-monitor has quite a few fds which it legitimately needs to have
open - for instance signalfd, command socket, etc.  So while it should
be possible and would be nice to do, it won't be particularly easy to
get right.  I'll happily look at any patch you come up with though.

Quoting John Keiser (jkeiser at getchef.com):
> Ah!  Should it?  Like, should it do all the daemon things that the
> container itself does?
> 
> 
> On Thu, Mar 27, 2014 at 12:30 PM, Serge Hallyn <serge.hallyn at ubuntu.com>wrote:
> 
> > Quoting John Keiser (jkeiser at getchef.com):
> > > Hi all,
> > >
> > > I am trying to use lxc_container->start() from within a process that
> > > happens to have started a TCP server, and even when my program exits, the
> > > server socket stays alive.  This can be tested by running this program
> > > (which starts a server and then starts a container) twice--the second
> > time
> > > resulting in a bind() error:
> > >
> > > #include <sys/socket.h>
> > > #include <netinet/in.h>
> > > #include <stdio.h>
> > > #include <errno.h>
> > > #include <strings.h>
> > > #include <stdlib.h>
> > > #include <lxc/lxccontainer.h>
> > >
> > > int main(int argc, char**argv)
> > > {
> > >   struct lxc_container *c;
> > >   int listenfd,connfd,n;
> > >   struct sockaddr_in servaddr;
> > >   int port;
> > >
> > >   // Start server
> > >   port = atoi(argv[2]);
> > >   printf("Listening on port %d ...\n", port);
> > >   listenfd=socket(AF_INET,SOCK_STREAM,0);
> > >   bzero(&servaddr,sizeof(servaddr));
> > >   servaddr.sin_family = AF_INET;
> > >   servaddr.sin_addr.s_addr=htonl(INADDR_ANY);
> > >   servaddr.sin_port=htons(port);
> > >   if (bind(listenfd,(struct sockaddr *)&servaddr,sizeof(servaddr)))
> > >   {
> > >     perror("Error calling bind()");
> > >     exit(1);
> > >   }
> > >   listen(listenfd,1024);
> > >
> > >   // Start
> > >   printf("Starting container ...\n");
> > >   c = lxc_container_new(argv[1], NULL);
> > >   c->want_daemonize(c, true);
> > >   c->want_close_all_fds(c, true);
> > >   c->start(c, 0, NULL);
> > >
> > >   return 0;
> > > }
> > >
> > >
> > > *> gcc test.c -llxc -o test*
> > > *> lxc-create -n blarghle -t ubuntu*
> > >
> > > *> ./test blarghle 9050*
> > > Listening on port 9050 ...
> > > Starting container ...
> > > *> ./test blarghle 9050*
> > > Listening on port 9050 ...
> > > Error calling bind(): Address already in use
> > >
> > > As you can see, even though ./test finished the first time, the second
> > > ./test cannot bind to the port because it is still in use.
> > >
> > > This appears to be because the lxc init process has a ghost "./test" as
> > its
> > > parent:
> > >
> > > *> ps -ef | grep 30276*
> > > root     30276     1  0 18:37 ?        00:00:00 ./test blarghle 9050
> > > root     30285 30276  0 18:37 ?        00:00:00 /sbin/init
> > > *> lsof | grep 9050*
> > > test      30276              root    3u     IPv4             366649
> > >  0t0        TCP *:9050 (LISTEN)
> >
> > Hm.  That's the container monitor.  It doesn't close all fds like
> > container init does.
> >
> > > I am stymied on how to investigate further.  My best guess is it has to
> > do
> > > with the double forking in
> > > lxccontainer.h<
> > https://github.com/lxc/lxc/blob/master/src/lxc/lxccontainer.c#L608-L621>,
> > > though I don't really understand what that is for yet :)  Any thoughts?
> > >
> > > Thanks!
> > >
> > > --John
> >
> > > _______________________________________________
> > > lxc-devel mailing list
> > > lxc-devel at lists.linuxcontainers.org
> > > http://lists.linuxcontainers.org/listinfo/lxc-devel
> >
> > _______________________________________________
> > lxc-devel mailing list
> > lxc-devel at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-devel
> >

> _______________________________________________
> lxc-devel mailing list
> lxc-devel at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-devel