ceph集群至少三个节点，推荐5节点或者更多，实验环境中，我们使用三台虚拟机部署ceph集群，/dev/sda大小为300G，/dev/sdb/、/dev/sdc、/dev/sdd都为100G，共四块硬盘，添加两块网卡，第一张网卡为nat模式，第二张网卡为仅主机模式（集群网）
1、修改主机名
# hostnamectl set-hostname ceph01
# hostnamectl set-hostname ceph02
# hostnamectl set-hostname ceph03
2、修改网卡配置
ceph01
# vi /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
NAME="ens33"
DEVICE="ens33"
ONBOOT="yes"
IPADDR=192.168.58.200
GATEWAY=192.168.58.2
NETMASK=255.255.255.0
DNS1=8.8.8.8
DNS2=114.114.114.114
# vi /etc/sysconfig/network-scripts/ifcfg-ens36
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
NAME="ens36"
DEVICE="ens36"
ONBOOT="yes"
IPADDR=172.16.10.200
NETMASK=255.255.255.0
ceph02
# vi /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
NAME="ens33"
DEVICE="ens33"
ONBOOT="yes"
IPADDR=192.168.58.201
GATEWAY=192.168.58.2
NETMASK=255.255.255.0
DNS1=8.8.8.8
DNS2=114.114.114.114
# vi /etc/sysconfig/network-scripts/ifcfg-ens36
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
NAME="ens36"
DEVICE="ens36"
ONBOOT="yes"
IPADDR=172.16.10.201
NETMASK=255.255.255.0
ceph03
# vi /etc/sysconfig/network-scripts/ifcfg-ens33
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
NAME="ens33"
DEVICE="ens33"
ONBOOT="yes"
IPADDR=192.168.58.202
GATEWAY=192.168.58.2
NETMASK=255.255.255.0
DNS1=8.8.8.8
DNS2=114.114.114.114
# vi /etc/sysconfig/network-scripts/ifcfg-ens36
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
NAME="ens36"
DEVICE="ens36"
ONBOOT="yes"
IPADDR=172.16.10.202
NETMASK=255.255.255.0
重启网络：
# systemctl restart network
3、关闭防火墙，关闭selinux、修改配置（全部主机）
关闭防火墙
# systemctl stop firewalld
# systemctl disable firewalld
关闭selinux
# sed -i s/SELINUX=enforcing/SELINUX=disabled/ /etc/selinux/config
# setenforce 0
设置 /etc/security/limits.conf
# vim /etc/security/limits.conf
# End of file
* hard nofile 655360
* soft nofile 655360
* hard nproc 655360
* soft nproc 655360
* soft core 655360
* hard core 655360
设置 /etc/security/limits.d/20-nproc.conf
# vim /etc/security/limits.d/20-nproc.conf
* soft nproc unlimited
root soft nproc unlimited
4、修改本地域名解析（全部主机）
# vim /etc/hosts
# openstack
192.168.58.100 controller
192.168.58.101 compute01
192.168.58.102 compute02
# cpeh-public
192.168.58.200 ceph01
192.168.58.201 ceph02
192.168.58.202 ceph03
# ceph-cluster
172.16.10.200 ceph01-cluster
172.16.10.201 ceph02-cluster
172.16.10.202 ceph03-cluster
5、NTP客户端配置，在ceph所有节点（ceph01、ceph02、ceph03）上执行
1、统一时区，在所有节点上执行
# ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
2、安装chrony，默认已安装
# yum -y install chrony
然后，修改配置/etc/chrony.conf中服务器为 controller，即
# vim /etc/chrony.conf
#server 0.centos.pool.ntp.org iburst
#server 1.centos.pool.ntp.org iburst
#server 2.centos.pool.ntp.org iburst
#server 3.centos.pool.ntp.org iburst
server controller iburst
设置开机自启动
# systemctl enable chronyd
# systemctl restart chronyd
验证时间同步：
# chronyc sources
6、设置yum源仓库
ceph01、ceph02、ceph03
# curl -o /etc/yum.repos.d/CentOS-Base.repo https://mirrors.aliyun.com/repo/Centos-7.repo
# curl -o /etc/yum.repos.d/epel.repo https://mirrors.aliyun.com/repo/epel-7.repo
全部主机
# vim /etc/yum.repos.d/ceph.repo
[ceph]
name=Ceph packages for $basearch
baseurl=http://mirrors.aliyun.com/ceph/rpm-nautilus/el7/x86_64
enabled=1
gpgcheck=0
[ceph-noarch]
name=Ceph noarch packages
baseurl=http://mirrors.aliyun.com/ceph/rpm-nautilus/el7/noarch
enabled=1
gpgcheck=0
[ceph-source]
name=Ceph source packages
baseurl=http://mirrors.aliyun.com/ceph/rpm-nautilus/el7/SRPMS
enabled=1
gpgcheck=0
7、安装软件
在controller、compute01、compute02安装ceph软件包
# yum -y install ceph ceph-radosgw
在ceph01上这装ceph-deploy
# yum -y install ceph-deploy ceph ceph-mds ceph-radosgw
/*
说明：ceph-deploy是ceph软件定义存储系统的一部分，用来方便地配置和管理Ceph存储集群。
*/
在ceph02、ceph03上这装ceph
# yum -y install ceph ceph-mds ceph-radosgw
全部主机上安装
# yum install vim bash-completion wget lrzsz ntpdate -y
8、配置SSH免密登录（在ceph01节点上）
1、在ceph01节点生成公钥，然后发放到所有的主机/客户机节点上，即包括ceph02、ceph03、controler、compute01、compute02
# ssh-keygen
2、将公钥发放到所有主机上
ssh-copy-id ceph01
ssh-copy-id ceph02
ssh-copy-id ceph03
ssh-copy-id controller
ssh-copy-id compute01
ssh-copy-id compute02
3、验证从ceph01访问所有主机是否可以免密登录
# ssh ceph01
# ssh ceph02
# ssh ceph03
# ssh controller
# ssh compute01
# ssh compute02
