ubuntu-14.04 – Ceph OSD总是在Ubuntu 14.04.1中“关闭”

前端之家收集整理的这篇文章主要介绍了ubuntu-14.04 – Ceph OSD总是在Ubuntu 14.04.1中“关闭”前端之家小编觉得挺不错的,现在分享给大家,也给大家做个参考。
我正在尝试安装和部署ceph集群.由于我没有足够的物理服务器,我使用官方Ubuntu 14.04映像在我的OpenStack上创建了4个VM.我想部署一个带有1个监控节点和3个OSD节点的集群,其中ceph版本为0.80.7-0ubuntu0.14.04.1.我按照 manual deployment document的步骤,成功安装了监控节点.但是,在安装OSD节点后,似乎OSD守护程序正在运行但未正确报告给监视节点.当我请求命令ceph –cluster cephcluster1 osd tree时,osd树总是显示出来.

以下是可能与我的问题相关的命令和相应结果.

root@monitor:/home/ubuntu# ceph --cluster cephcluster1 osd tree
# id    weight  type name       up/down reweight
-1      3       root default
-2      1               host osd1
0       1                       osd.0   down    1
-3      1               host osd2
1       1                       osd.1   down    1
-4      1               host osd3
2       1                       osd.2   down    1

root@monitor:/home/ubuntu# ceph --cluster cephcluster1 -s
    cluster fd78cbf8-8c64-4b12-9cfa-0e75bc6c8d98
     health HEALTH_WARN 192 pgs stuck inactive; 192 pgs stuck unclean; 3/3 in osds are down
     monmap e1: 1 mons at {monitor=172.26.111.4:6789/0},election epoch 1,quorum 0 monitor
     osdmap e21: 3 osds: 0 up,3 in
      pgmap v22: 192 pgs,3 pools,0 bytes data,0 objects
            0 kB used,0 kB / 0 kB avail
                 192 creating

所有节点上的配置文件/etc/ceph/cephcluster1.conf:

[global]
fsid = fd78cbf8-8c64-4b12-9cfa-0e75bc6c8d98
mon initial members = monitor
mon host = 172.26.111.4
public network = 10.5.0.0/16
cluster network = 172.26.111.0/24
auth cluster required = cephx
auth service required = cephx
auth client required = cephx
osd journal size = 1024
filestore xattr use omap = true
osd pool default size = 2
osd pool default min size = 1
osd pool default pg num = 333
osd pool default pgp num = 333
osd crush chooseleaf type = 1

[osd]
osd journal size = 1024

[osd.0]
osd host = osd1

[osd.1]
osd host = osd2

[osd.2]
osd host = osd3

通过start ceph-osd cluster = cephcluster1 id = x启动其中一个osd守护进程时的日志,其中x是OSD ID:

OSD节点#1上的/var/log/ceph/cephcluster1-osd.0.log:

2015-02-11 09:59:56.626899 7f5409d74800  0 ceph version 0.80.7 (6c0127fcb58008793d3c8b62d925bc91963672a3),process ceph-osd,pid 11230
2015-02-11 09:59:56.646218 7f5409d74800  0 genericfilestorebackend(/var/lib/ceph/osd/cephcluster1-0) detect_features: FIEMAP ioctl is supported and appears to work
2015-02-11 09:59:56.646372 7f5409d74800  0 genericfilestorebackend(/var/lib/ceph/osd/cephcluster1-0) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-02-11 09:59:56.658227 7f5409d74800  0 genericfilestorebackend(/var/lib/ceph/osd/cephcluster1-0) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2015-02-11 09:59:56.679515 7f5409d74800  0 filestore(/var/lib/ceph/osd/cephcluster1-0) limited size xattrs
2015-02-11 09:59:56.699721 7f5409d74800  0 filestore(/var/lib/ceph/osd/cephcluster1-0) mount: enabling WRITEAHEAD journal mode: checkpoint is not enabled
2015-02-11 09:59:56.700107 7f5409d74800 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
2015-02-11 09:59:56.700454 7f5409d74800  1 journal _open /var/lib/ceph/osd/cephcluster1-0/journal fd 20: 1073741824 bytes,block size 4096 bytes,directio = 1,aio = 0
2015-02-11 09:59:56.704025 7f5409d74800  1 journal _open /var/lib/ceph/osd/cephcluster1-0/journal fd 20: 1073741824 bytes,aio = 0
2015-02-11 09:59:56.704884 7f5409d74800  1 journal close /var/lib/ceph/osd/cephcluster1-0/journal
2015-02-11 09:59:56.725281 7f5409d74800  0 genericfilestorebackend(/var/lib/ceph/osd/cephcluster1-0) detect_features: FIEMAP ioctl is supported and appears to work
2015-02-11 09:59:56.725397 7f5409d74800  0 genericfilestorebackend(/var/lib/ceph/osd/cephcluster1-0) detect_features: FIEMAP ioctl is disabled via 'filestore fiemap' config option
2015-02-11 09:59:56.736445 7f5409d74800  0 genericfilestorebackend(/var/lib/ceph/osd/cephcluster1-0) detect_features: syncfs(2) syscall fully supported (by glibc and kernel)
2015-02-11 09:59:56.756912 7f5409d74800  0 filestore(/var/lib/ceph/osd/cephcluster1-0) limited size xattrs
2015-02-11 09:59:56.776471 7f5409d74800  0 filestore(/var/lib/ceph/osd/cephcluster1-0) mount: WRITEAHEAD journal mode explicitly enabled in conf
2015-02-11 09:59:56.776748 7f5409d74800 -1 journal FileJournal::_open: disabling aio for non-block journal.  Use journal_force_aio to force use of aio anyway
2015-02-11 09:59:56.776848 7f5409d74800  1 journal _open /var/lib/ceph/osd/cephcluster1-0/journal fd 21: 1073741824 bytes,aio = 0
2015-02-11 09:59:56.777069 7f5409d74800  1 journal _open /var/lib/ceph/osd/cephcluster1-0/journal fd 21: 1073741824 bytes,aio = 0
2015-02-11 09:59:56.783019 7f5409d74800  0 <cls> cls/hello/cls_hello.cc:271: loading cls_hello
2015-02-11 09:59:56.783584 7f5409d74800  0 osd.0 11 crush map has features 1107558400,adjusting msgr requires for clients
2015-02-11 09:59:56.783645 7f5409d74800  0 osd.0 11 crush map has features 1107558400 was 8705,adjusting msgr requires for mons
2015-02-11 09:59:56.783687 7f5409d74800  0 osd.0 11 crush map has features 1107558400,adjusting msgr requires for osds
2015-02-11 09:59:56.783750 7f5409d74800  0 osd.0 11 load_pgs
2015-02-11 09:59:56.783831 7f5409d74800  0 osd.0 11 load_pgs opened 0 pgs
2015-02-11 09:59:56.792167 7f53f9b57700  0 osd.0 11 ignoring osdmap until we have initialized
2015-02-11 09:59:56.792334 7f53f9b57700  0 osd.0 11 ignoring osdmap until we have initialized
2015-02-11 09:59:56.792838 7f5409d74800  0 osd.0 11 done with init,starting boot process

监视节点上的/var/log/ceph/ceph-mon.monitor.log:

2015-02-11 09:59:56.593494 7f24cc41d700  0 mon.monitor@0(leader) e1 handle_command mon_command({"prefix": "osd crush create-or-move","args": ["host=osd1","root=default"],"id": 0,"weight": 0.05} v 0) v1
2015-02-11 09:59:56.593955 7f24cc41d700  0 mon.monitor@0(leader).osd e21 create-or-move crush item name 'osd.0' initial_weight 0.05 at location {host=osd1,root=default}

虽然在使用相同的安装步骤将环境从Ubuntu 14.04更改为CentOS 6.6时,ceph OSD正常显示,但我仍然希望解决这个问题,因为我对Ubuntu比CentOS更熟悉.

任何建议都表示赞赏.非常感谢!

我在非常相同的环境中遇到了同样的问题.我终于将问题跟踪到了一个混乱的OSD UUID.放弃它的是MON日志中的以下行(不是OSD日志!):
... mon.minion-001@0(leader).osd e75 preprocess_boot from osd.0 10.208.66.2:6800/3427 clashes with existing osd: different fsid (ours: 71b33e7f-b464-4ba9-96b3-8c814921fea2 ; theirs: 5401be6f-b4ff-42ef-8531-78ee73772d5b)

我通过首先手动删除OSD,销毁其文件系统并从头开始手动重新创建来解决了这个问题.问题是如何形成的,我随后将要追查.

鉴于我使用木偶来设置OSD并且它弄乱的原因可能与我的环境特有的意义有关,这意味着您遇到的问题很可能是另一个问题,但也许您可以检查一下无论如何都是MON日志.但是,您必须在MON上启用调试,在ceph.conf中声明类似的内容

[mon]
        debug mon = 9

有问题的消息记录在第7级,因此这为您提供了更多细节,而不会使所有内容都非常繁琐.

@LoicDachary:在0级记录此错误/警告消息是否有意义?如果它立刻被记录下来,我肯定会早点发现这个问题.

猜你在找的Ubuntu相关文章