<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
</head>
<body style="word-wrap:break-word; color:rgb(0,0,0); font-family:Calibri,sans-serif">
<div>Hi All,</div>
<div><br>
</div>
<div><br>
<div style="word-wrap:break-word">
<div style="font-size:14px">We're trying to mount an rbd image inside of a linux container that has been created with docker (<a href="https://www.docker.io/" target="_blank">https://www.docker.io/</a>). We seem to have access to the rbd kernel module from
inside the container:</div>
<div style="font-size:14px"><br>
</div>
<div style="font-size:14px">
<div># lsmod | grep ceph</div>
<div>libceph 218854 1 rbd</div>
<div>libcrc32c 12603 3 xfs,libceph,dm_persistent_data</div>
</div>
<div style="font-size:14px"><br>
</div>
<div style="font-size:14px">And we can query the pool for available rbds and create rbds from inside the container:</div>
<div style="font-size:14px"><br>
</div>
<div style="font-size:14px">
<div># rbd -p dockers --id dockers --keyring /etc/ceph/ceph.client.dockers.keyring create lxctest --size 51200</div>
<div># rbd -p dockers --id dockers --keyring /etc/ceph/ceph.client.dockers.keyring ls</div>
<div>lxctest</div>
</div>
<div style="font-size:14px"><br>
</div>
<div style="font-size:14px">But for some reason, we can't seem to map the device to the container:</div>
<div style="font-size:14px"><br>
</div>
<div style="font-size:14px">
<div># rbd -p dockers --id dockers --keyring /etc/ceph/ceph.client.dockers.keyring map lxctest</div>
<div><b>rbd: add failed: (22) Invalid argument</b></div>
</div>
<div style="font-size:14px"><br>
</div>
<div style="font-size:14px">I don't see anything particularly interesting in dmesg or messages on either the container or the host box. Any ideas on how to troubleshoot this?</div>
<div style="font-size:14px"><br>
</div>
<div style="font-size:14px">….</div>
<div style="font-size:14px"><br>
</div>
<div>
<div style="font-size:14px">An strace produces the below results:</div>
<div style="font-size:14px"><br>
</div>
<div><span class="Apple-tab-span" style="font-size:14px; white-space:pre"></span><span style="font-size:16px"># strace rbd -p dockers --id dockers --keyring /etc/ceph/ceph.client.dockers.keyring map test</span></div>
<div style="font-size:14px"><br>
</div>
<blockquote style="font-size:14px; margin:0px 0px 0px 40px; border:none; padding:0px">
…</blockquote>
<div style="font-size:14px">
<blockquote style="margin:0px 0px 0px 40px; border:none; padding:0px">
<div>futex(0xb5637c, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0xb56378, {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1</div>
<div>futex(0xb562f8, FUTEX_WAKE_PRIVATE, 1) = 1</div>
<div>add_key(0x424408, 0x7fff82c4e210, 0x7fff82c4e140, 0x22, 0xfffffffe) = 607085216</div>
<div>stat("/sys/bus/rbd", {st_mode=S_IFDIR|0755, st_size=0, ...}) = 0</div>
<div><b>open("/sys/bus/rbd/add", O_WRONLY) = 3</b></div>
<div><b>write(3, "<a href="http://10.198.41.6:6789/" target="_blank">10.198.41.6:6789</a>,<a href="http://10.198.41.8:678/" target="_blank">10.198.41.8:678</a>"..., 96) = -1 EINVAL (Invalid argument)</b></div>
<div>close(3) = 0</div>
<div>rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, 0x7fbf8a7efa90}, {SIG_DFL, [], 0}, 8) = 0</div>
<div>rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, 0x7fbf8a7efa90}, {SIG_DFL, [], 0}, 8) = 0</div>
<div>rt_sigprocmask(SIG_BLOCK, [CHLD], [PIPE], 8) = 0</div>
<div>clone(child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD, parent_tidptr=0x7fff82c4e040) = 22</div>
<div>wait4(22, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = 22</div>
<div>rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, 0x7fbf8a7efa90}, NULL, 8) = 0</div>
<div>rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, 0x7fbf8a7efa90}, NULL, 8) = 0</div>
<div>rt_sigprocmask(SIG_SETMASK, [PIPE], NULL, 8) = 0</div>
<div>write(2, "rbd: add failed: ", 17rbd: add failed: ) = 17</div>
<div>write(2, "(22) Invalid argument", 21(22) Invalid argument) = 21</div>
<div>write(2, "\n", 1</div>
<div>) = 1</div>
<div>exit_group(1) = ?</div>
<div>+++ exited with 1 +++</div>
<div><br>
</div>
<div><br>
</div>
</blockquote>
</div>
<div style="font-size:14px">
<div style="font-size:medium">
<div style="font-size:14px">The app is run inside the container with setuid = 0 and the container is able to mount all required filesystems … </div>
</div>
<div>Could it be that is not possible to write to /sys/bus/rbd/add inside an lxc container ?</div>
<div><br>
</div>
<div>Thx,</div>
<div>Om</div>
</div>
</div>
<div style="font-size:14px">
<div style="font-family:Calibri; font-size:medium"></div>
</div>
</div>
</div>
<br>
<hr>
<font face="Arial" color="Gray" size="1"><br>
The information in this e-mail is intended only for the person or entity to which it is addressed.<br>
<br>
It may contain confidential and /or privileged material. If someone other than the intended recipient should receive this e-mail, he / she shall not be entitled to read, disseminate, disclose or duplicate it.<br>
<br>
If you receive this e-mail unintentionally, please inform us immediately by "reply" and then delete it from your system. Although this information has been compiled with great care, neither IMC Financial Markets & Asset Management nor any of its related entities
shall accept any responsibility for any errors, omissions or other inaccuracies in this information or for the consequences thereof, nor shall it be bound in any way by the contents of this e-mail or its attachments. In the event of incomplete or incorrect
transmission, please return the e-mail to the sender and permanently delete this message and any attachments.<br>
<br>
Messages and attachments are scanned for all known viruses. Always scan attachments before opening them.<br>
</font>
</body>
</html>