[lxc-users] Question about existing defaults
Michael H. Warfield
mhw at WittsEnd.com
Fri Apr 18 16:07:54 UTC 2014
On Fri, 2014-04-18 at 00:30 -0400, CDR wrote:
> I can give you direct access to the box where I am developing this.
> Please write me to my email directly.
> There is nothing strange about my Fedora20 box. It is fully yum-updated.
Wait a minute. There is something very wrong here...
> ldd /usr/bin/lxc-start
> linux-vdso.so.1 => (0x00007fff0d7d1000)
> liblxc.so.1 => /lib/liblxc.so.1 (0x0000003d98c00000)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
This isn't right.
This is the line from my system...
liblxc.so.1 => /lib64/liblxc.so.1 (0x00007fd4a42cf000)
Why is your version referencing a library in the 32 bit section and not
the 64 bit section???
Do you have a liblxc.so.1 in /lib? What's in /lib64?
[mhw at hydra lxc]$ ls -l /lib/liblxc*
ls: cannot access /lib/liblxc*: No such file or directory
[mhw at hydra lxc]$ ls -l /lib64/liblxc*
lrwxrwxrwx. 1 root root 11 Apr 17 14:53 /lib64/liblxc.so -> liblxc.so.1
lrwxrwxrwx. 1 root root 15 Apr 17 14:52 /lib64/liblxc.so.1 -> liblxc.so.1.0.0
-rwxr-xr-x. 1 root root 434720 Apr 17 14:52 /lib64/liblxc.so.1.0.0
[Note: I'm building off the git master branch head, my version numbers
may vary]
You've got something really out of whack here if you're on a 64 bit
system and with the x86_64 lxc rpm's installed yet there's some 32 bit
lxc library in /lib/ lurking.
If you do have a /lib/liblxc.so.1 run this:
rpm -qf /lib/liblxc.so.1
See where it came from.
Regards,
Mike
> libcap.so.2 => /lib64/libcap.so.2 (0x00000039d1600000)
> libutil.so.1 => /lib64/libutil.so.1 (0x00000039e6600000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x00000039bba00000)
> libc.so.6 => /lib64/libc.so.6 (0x00000039bb200000)
> /lib64/ld-linux-x86-64.so.2 (0x00000039bae00000)
> libattr.so.1 => /lib64/libattr.so.1 (0x00000039c8600000)
> On Thu, Apr 17, 2014 at 10:22 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> > On Thu, 2014-04-17 at 21:25 -0400, CDR wrote:
> >> Dear friends
> >> I installed the RPMs again, and rebooted. Previous, I issued an
> >> "ldconfig", just in case. It still fails.
> >> Maybe one of the developers of lxc may take a look at the trace below
> >> and spot the issue. The advantages of the current version over the one
> >> issued in Fedora are too great to ignore, like "lxc-autostart", etc.
> >
> > This is not occurring on my F20 host and I'm rotating development and
> > release versions almost on a daily basis. This isn't making any sense.
> > That function should exist. It does exist on a clean install. It's
> > defined in caps.c.
> >
> > What do you get from this:
> >
> > ldd /usr/bin/lxc-start
> >
> > You seem to still be hooking up with the wrong library for some reason.
> >
> > Regards,
> > Mike
> >
> >> strace lxc-start -d -n dialer-1
> >> execve("/usr/bin/lxc-start", ["lxc-start", "-d", "-n", "dialer-1"],
> >> [/* 31 vars */]) = 0
> >> brk(0) = 0x604000
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> >> 0) = 0x7fb2a9a35000
> >> access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)
> >> open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
> >> fstat(3, {st_mode=S_IFREG|0644, st_size=103464, ...}) = 0
> >> mmap(NULL, 103464, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7fb2a9a1b000
> >> close(3) = 0
> >> open("/lib/liblxc.so.1", O_RDONLY|O_CLOEXEC) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\313\300\230=\0\0\0"...,
> >> 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=1497976, ...}) = 0
> >> mmap(0x3d98c00000, 2488264, PROT_READ|PROT_EXEC,
> >> MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3d98c00000
> >> mprotect(0x3d98c5e000, 2093056, PROT_NONE) = 0
> >> mmap(0x3d98e5d000, 12288, PROT_READ|PROT_WRITE,
> >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x5d000) = 0x3d98e5d000
> >> close(3) = 0
> >> open("/lib64/libcap.so.2", O_RDONLY|O_CLOEXEC) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0
> >> \26`\3219\0\0\0"..., 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=21424, ...}) = 0
> >> mmap(0x39d1600000, 2114112, PROT_READ|PROT_EXEC,
> >> MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x39d1600000
> >> mprotect(0x39d1604000, 2093056, PROT_NONE) = 0
> >> mmap(0x39d1803000, 8192, PROT_READ|PROT_WRITE,
> >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x39d1803000
> >> close(3) = 0
> >> open("/lib64/libutil.so.1", O_RDONLY|O_CLOEXEC) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20\17`\3469\0\0\0"...,
> >> 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=17536, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> >> 0) = 0x7fb2a9a1a000
> >> mmap(0x39e6600000, 2105616, PROT_READ|PROT_EXEC,
> >> MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x39e6600000
> >> mprotect(0x39e6602000, 2093056, PROT_NONE) = 0
> >> mmap(0x39e6801000, 8192, PROT_READ|PROT_WRITE,
> >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1000) = 0x39e6801000
> >> close(3) = 0
> >> open("/lib64/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\340m\240\2739\0\0\0"...,
> >> 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=150800, ...}) = 0
> >> mmap(0x39bba00000, 2213104, PROT_READ|PROT_EXEC,
> >> MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x39bba00000
> >> mprotect(0x39bba18000, 2093056, PROT_NONE) = 0
> >> mmap(0x39bbc17000, 8192, PROT_READ|PROT_WRITE,
> >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x39bbc17000
> >> mmap(0x39bbc19000, 13552, PROT_READ|PROT_WRITE,
> >> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x39bbc19000
> >> close(3) = 0
> >> open("/lib64/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
> >> read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0p\36\"\2739\0\0\0"...,
> >> 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=2100672, ...}) = 0
> >> mmap(0x39bb200000, 3924576, PROT_READ|PROT_EXEC,
> >> MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x39bb200000
> >> mprotect(0x39bb3b4000, 2097152, PROT_NONE) = 0
> >> mmap(0x39bb5b4000, 24576, PROT_READ|PROT_WRITE,
> >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1b4000) = 0x39bb5b4000
> >> mmap(0x39bb5ba000, 16992, PROT_READ|PROT_WRITE,
> >> MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x39bb5ba000
> >> close(3) = 0
> >> open("/lib64/libattr.so.1", O_RDONLY|O_CLOEXEC) = 3
> >> read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\320\23`\3109\0\0\0"...,
> >> 832) = 832
> >> fstat(3, {st_mode=S_IFREG|0755, st_size=22160, ...}) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> >> 0) = 0x7fb2a9a19000
> >> mmap(0x39c8600000, 2113904, PROT_READ|PROT_EXEC,
> >> MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x39c8600000
> >> mprotect(0x39c8604000, 2093056, PROT_NONE) = 0
> >> mmap(0x39c8803000, 8192, PROT_READ|PROT_WRITE,
> >> MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x3000) = 0x39c8803000
> >> close(3) = 0
> >> mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> >> 0) = 0x7fb2a9a18000
> >> mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1,
> >> 0) = 0x7fb2a9a16000
> >> arch_prctl(ARCH_SET_FS, 0x7fb2a9a167c0) = 0
> >> mprotect(0x39bb5b4000, 16384, PROT_READ) = 0
> >> mprotect(0x39c8803000, 4096, PROT_READ) = 0
> >> mprotect(0x39bbc17000, 4096, PROT_READ) = 0
> >> mprotect(0x39e6801000, 4096, PROT_READ) = 0
> >> mprotect(0x39d1803000, 4096, PROT_READ) = 0
> >> mprotect(0x3d98e5d000, 4096, PROT_READ) = 0
> >> mprotect(0x602000, 4096, PROT_READ) = 0
> >> mprotect(0x39bb01f000, 4096, PROT_READ) = 0
> >> munmap(0x7fb2a9a1b000, 103464) = 0
> >> set_tid_address(0x7fb2a9a16a90) = 3388
> >> set_robust_list(0x7fb2a9a16aa0, 24) = 0
> >> rt_sigaction(SIGRTMIN, {0x39bba068c0, [], SA_RESTORER|SA_SIGINFO,
> >> 0x39bba0f750}, NULL, 8) = 0
> >> rt_sigaction(SIGRT_1, {0x39bba06950, [],
> >> SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x39bba0f750}, NULL, 8) = 0
> >> rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
> >> getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=RLIM64_INFINITY}) = 0
> >> writev(2, [{"lxc-start", 9}, {": ", 2}, {"symbol lookup error", 19},
> >> {": ", 2}, {"lxc-start", 9}, {": ", 2}, {"undefined symbol:
> >> lxc_caps_init", 31}, {"", 0}, {"", 0}, {"\n", 1}], 10lxc-start: symbol
> >> lookup error: lxc-start: undefined symbol: lxc_caps_init
> >> ) = 75
> >> exit_group(127) = ?
> >> +++ exited with 127 +++
> >>
> >> On Thu, Apr 17, 2014 at 2:27 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> >> > On Thu, 2014-04-17 at 07:29 -0400, CDR wrote:
> >> >> I followed the instructions and it works, indeed.
> >> >> I erased first all lxc rpms and the installed the new ones
> >> >
> >> >> rpm -qa |grep lxc
> >> >> lxc-libs-1.0.3-1.fc20.x86_64
> >> >> lxc-debuginfo-1.0.3-1.fc20.x86_64
> >> >> lxc-1.0.3-1.fc20.x86_64
> >> >> libvirt-daemon-driver-lxc-1.1.3.4-4.fc20.x86_64
> >> >
> >> >> However, I hit a problem when starting a container.
> >> >> lxc-start -d -n dialer-1
> >> >> lxc-start: symbol lookup error: lxc-start: undefined symbol: lxc_caps_init
> >> >
> >> >> This means that the code is left to pull this function from package
> >> >> that is not there?
> >> >
> >> > That usually means you've got an old library dangling around or you have
> >> > processes with a deleted library locked in memory. That most often
> >> > happens when you've done a manual "./configure ; make ; make install"
> >> > and pieces are now installed where you don't want them. In that case,
> >> > ldconfig may have the lxc-libs located in a different path location than
> >> > what you've installed. It could also occur if you have running
> >> > containers with an incompatible library.
> >> >
> >> > First thing I would do is inspect /usr/local and look for any lxc
> >> > components and get rid of them.
> >> >
> >> > Second thing I would do is reboot to make sure you have no stale
> >> > processes holding an old library in common memory.
> >> >
> >> > Then try it again.
> >> >
> >> > This is why I strictly go with the yum/rpm route when working with
> >> > managed packages like LXC.
> >> >
> >> >> Any help would be appreciated, or a patch that would not use this
> >> >> function. My containers are unbound, only for internal use, so I want
> >> >> them to use as much resources as needed.
> >> >
> >> > What rpm is showing is that you have a coherent set of packages that
> >> > should not be encountering this problem. The error indicates the
> >> > presence of an unmanaged library in memory on on disk. You might track
> >> > it down using "ldd /usr/bin/lxc-start" and see what that gives you.
> >> >
> >> >> Philip
> >> >
> >> > Regards,
> >> > Mike
> >> >
> >> >> On Wed, Apr 16, 2014 at 11:41 PM, Michael H. Warfield <mhw at wittsend.com> wrote:
> >> >> > On Wed, 2014-04-16 at 22:04 -0400, CDR wrote:
> >> >> >> I am using Fedora 20 and try to compile the very latest sources,
> >> >> >> because they bring new utilities that are very useful.
> >> >> >> You would assume, I think, that by using ./configure
> >> >> >> --with-distro=fedora, the right defaults would be pulled over. It is
> >> >> >> not the case. I need to know how to apply the existing defaults if the
> >> >> >> software is already running in the box, so the libraries and utilities
> >> >> >> do not get written to a different places. Particularly annoying is the
> >> >> >> location of the config files.
> >> >> >
> >> >> >> In summary, don't we agree that by using the distro name, the software
> >> >> >> should either extract or know the right defaults? Unless somebody has
> >> >> >> a secret way to extract that information from the RPM offered by the
> >> >> >> distribution.
> >> >> >
> >> >> > Interesting. I guess this is one I might field, since I'm responsible
> >> >> > for the Fedora template (which does NOT imply the Fedora build). I
> >> >> > don't know who the Fedora package manager for the LXC package is and I
> >> >> > haven't heard from him or her.
> >> >> >
> >> >> > I, for one, would NOT expect "./configure --with-distro=fedora" to pull
> >> >> > the correct results. That's from experience, although I've never used
> >> >> > it myself. It's just something none of us have thought to maintain and
> >> >> > I have never looked into it. I didn't even know the option was there in
> >> >> > configure.
> >> >> >
> >> >> > The Fedora Project itself does not even use the default lxc.spec rpm
> >> >> > spec file from the project but chooses their own .spec file and
> >> >> > packaging. Largely, the defaults are similar...
> >> >> >
> >> >> > I had been using an explicit build based on our .spec file to create tar
> >> >> > files and build rpm files using "rpmbuild -ta" but another contributor,
> >> >> > Dwight (Oracle), pointed out to me that I could just do a "make rpm" in
> >> >> > a source directory and it would to the correct things to create a Fedora
> >> >> > rpm. I've been very successful doing that and now that's all I do. It
> >> >> > does the right things. It uses slightly different packaging from the
> >> >> > "official" Fedora packages but one that's compatible with the Fedora
> >> >> > packaging.
> >> >> >
> >> >> > If you are manually running "./configure" and "make" on our sources on
> >> >> > Fedora, my first advice would be DON'T. You're doing things the hard
> >> >> > way. I don't even do that with any stock rpm packages I'm working on.
> >> >> > Work through the yum/rpm system. Run "make rpm" and install the rpms
> >> >> > using "yum localinstall". That will prevent you from ending up with
> >> >> > conflicting binaries in differing locations.
> >> >> >
> >> >> > In this case, it's actually easier to do the right thing than it is to
> >> >> > do the manual thing the hard way. You're trying too hard.
> >> >> >
> >> >> >> Philip
> >> >> >
> >> >> > Regards,
> >> >> > Mike
> >> >> > --
> >> >> > Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> >> >> > /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> >> >> > NIC whois: MHW9 | An optimist believes we live in the best of all
> >> >> > PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
> >> >> >
> >> >> >
> >> >> > _______________________________________________
> >> >> > lxc-users mailing list
> >> >> > lxc-users at lists.linuxcontainers.org
> >> >> > http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >> _______________________________________________
> >> >> lxc-users mailing list
> >> >> lxc-users at lists.linuxcontainers.org
> >> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >> >
> >> > --
> >> > Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> >> > /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> >> > NIC whois: MHW9 | An optimist believes we live in the best of all
> >> > PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
> >> >
> >> >
> >> > _______________________________________________
> >> > lxc-users mailing list
> >> > lxc-users at lists.linuxcontainers.org
> >> > http://lists.linuxcontainers.org/listinfo/lxc-users
> >> _______________________________________________
> >> lxc-users mailing list
> >> lxc-users at lists.linuxcontainers.org
> >> http://lists.linuxcontainers.org/listinfo/lxc-users
> >
> > --
> > Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
> > /\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
> > NIC whois: MHW9 | An optimist believes we live in the best of all
> > PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
> >
> >
> > _______________________________________________
> > lxc-users mailing list
> > lxc-users at lists.linuxcontainers.org
> > http://lists.linuxcontainers.org/listinfo/lxc-users
> _______________________________________________
> lxc-users mailing list
> lxc-users at lists.linuxcontainers.org
> http://lists.linuxcontainers.org/listinfo/lxc-users
--
Michael H. Warfield (AI4NB) | (770) 978-7061 | mhw at WittsEnd.com
/\/\|=mhw=|\/\/ | (678) 463-0932 | http://www.wittsend.com/mhw/
NIC whois: MHW9 | An optimist believes we live in the best of all
PGP Key: 0x674627FF | possible worlds. A pessimist is sure of it!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 482 bytes
Desc: This is a digitally signed message part
URL: <http://lists.linuxcontainers.org/pipermail/lxc-users/attachments/20140418/952c112f/attachment.sig>
More information about the lxc-users
mailing list