基于centos6.6的IPSAN iscsi配置
iscsi:监听端口:3260/tcp
san的类型:FC SAN ISCSI:IP SAN
iscsi storage:构建 target:linux主机,iscsi Target ,iscsi LUN 案例:实践iscsi IP SAN ,对于认证,基于IP认证,CHAP挑战握手认证协议 程序包:scsi-target-utils 管理工具: 1,tgtadm命令全功能配置工具,其配置结果在OS重启后失效; 2,tgt-admin,通过读取配置文件/etc/tgt/targets.conf进行配置的工具
启动服务后,模拟是iscsi总线 管理多个target,每个target能管理多个lun(32)
initiator: 程序包:iscsi-initiator-utils iscsi工具:链接target iscsid工具:守护进程
iscsi在识别二者时,target和initiator都需要使用基于IQN(iscsi qualified name)机制来进行标识 iqn格式:iqn.YEAR-MONTH.tld,domain:string[.substring] 例如:iqn.2015-05.com.linuxea.i1.IDF-1
配置iscsi server;(node3) 1,准备磁盘设备 2,安装程序包,启动服务 3,创建target 4,在target上创建lun 5,授权访问 1. yum install scsi-target-utils 1. service tgtd start Starting SCSI target daemon: [ OK ] 1. chkconfig tgtd on 1. ss -untl | grep :3260 tcp LISTEN 0 128 :::3260 :::* tcp LISTEN 0 128 *:3260 *:*
配置文件的定义分为两种模式 1,命令行配置 2,修改配置文件
1,命令行的配置 模式化工具: target:管理target logicalunit:管理lun account:管理用户账号
管理操作: 查看:show new:新建 delete:删除 update:修改 bind:绑定,即实现授权,通过将Ip或账号于target绑定进行 unbind:接触授权 查看 1. tgtadm --lld iscsi --mode target --op show
如何添加tgtd
1,# tgtadm --lld iscsi --mode target --op new --tid 1 --targetname iqn.2015-06.com.linuxea:s1.t1 (tgtadm --lld iscsi --mode target --op new新建 --tid为1 --targetname名字是(遵循iqn格式)iqn.2015年6月,域名linuxea.com在s1机架上的第一个target1) 1. tgtadm --lld iscsi --mode target --op show Target 1: iqn.2015-06.com.linuxea:s1.t1 System information: Driver: iscsi State: ready I_T nexus information: LUN information: LUN: 0 Type: controller SCSI ID: IET 00010000 SCSI SN: beaf10 Size: 0 MB, Block size: 1 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: null Backing store path: None Backing store flags: Account information: ACL information: 1. 2,# tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 1 --backing-store /dev/sdb (tgtadm --lld iscsi --mode logicalunit --op new新建一个tid1,lun为1 ,--backing-store 指明磁盘,这里的/dev/sdb是新增的硬盘)
lun已经在列表中 1. tgtadm --lld iscsi --mode target --op show Target 1: iqn.2015-06.com.linuxea:s1.t1 System information: Driver: iscsi State: ready I_T nexus information: LUN information: LUN: 0 Type: controller SCSI ID: IET 00010000 SCSI SN: beaf10 Size: 0 MB, Block size: 1 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: null Backing store path: None Backing store flags: LUN: 1 Type: disk SCSI ID: IET 00010001 SCSI SN: beaf11 Size: 21475 MB, Block size: 512 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: rdwr Backing store path: /dev/sdb Backing store flags: Account information: ACL information:
在添加一块硬盘/dev/sdc,--lun数要唯一,不能重叠 1. tgtadm --lld iscsi --mode logicalunit --op new --tid 1 --lun 2 --backing-store /dev/sdc 1. tgtadm --lld iscsi --mode target --op show LUN: 2 Type: disk SCSI ID: IET 00010002 SCSI SN: beaf12 Size: 21475 MB, Block size: 512 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: rdwr Backing store path:/dev/sdc Backing store flags: Account information: ACL information: 删除lun2 1. tgtadm --lld iscsi --mode logicalunit --op delete --tid 1 --lun 2 授权 对于账号的授权可能比较靠谱,但是实际中基于ip的授权可能较多
bind绑定授权访问,比如:授权给172.16.0.0网络中所有主机访问
# tgtadm --lld iscsi --mode target --op bind --tid 1 --initiator-address 172.16.0.0/16 授权后在来查看,ACL已经添加了172.16.0.0/16 1. tgtadm --lld iscsi --mode target --op show
ACL information:
172.16.0.0/16
PS:这些选项是可以简化的,如下所示:
--lld:-L
配置iscsi initiator: 1,安装程序包,配置initiator名字,并启动服务 (1) 1. yum -y install iscsi-initiator-utils 1. service iscsi start 1. service iscsid start
(2) 配置initiator的iqn格式的名字,保存至:/etc/iscsi/initiatorname.iscsi 使用iscsi-iname生成名字,例如: 1. iscsi-iname -p iqn.2015-06.com.linuxea iqn.2015-06.com.linuxea:6db732c2a231
这样也可以的: 1. echo "InitiatorName=$(iscsi-iname -p iqn.2015-06.com.linuxea)" > /etc/iscsi/initiatorname.iscsi 1. cat /etc/iscsi/initiatorname.iscsi InitiatorName=iqn.2015-06.com.linuxea:38e310d61d4
使用专用工具iscsiadm配置实现target的发现,注册等 discovery:发现模式 iscsiadm -m discpvery -d # -t sendtagets -p ip:port TYPE类型:sendtagets,可简写为st node:节点模式 iscsiadm -m node [ -hV ] [ -d debug_level ] [ -P printlevel ] [ -L all,manual,automatic ] [ -U all,manual,automatic ] [ -S ] [ [ -T targetname -p ip:port -I ifaceN ] [ -l | -u | -R | -s] ] [ [ -o operation ] [ -n name ] [ -v value ] ]
使用iscsiadm -h查看帮助 (3),发现,发现的同时会重启服务 1. iscsiadm -m discovery -d 3 -t st -p 172.16.249.99:3260 iscsiadm: ip 172.16.249.99, port 3260, tgpt -1 iscsiadm: Max file limits 1024 4096
Starting iscsid: [ OK ] iscsiadm: Could not open /var/lib/iscsi/send_targets/172.16.249.99,3260: No such file or directory
iscsiadm: starting sendtargets discovery, address 172.16.249.99:3260, iscsiadm: connecting to 172.16.249.99:3260 iscsiadm: connected local port 45387 to 172.16.249.99:3260 iscsiadm: connected to discovery address 172.16.249.99 iscsiadm: login response status 0000 iscsiadm: discovery process to 172.16.249.99:3260 exiting iscsiadm: disconnecting conn 0x1a7dce8, fd 3 172.16.249.99:3260,1 iqn.2015-06.com.linuxea:s1.t1
一旦发现,会在/var/lib/iscsi/下有/目录/ip/端口/和发现的名字 1. ls /var/lib/iscsi/send_targets/172.16.249.99,3260/ iqn.2015-06.com.linuxea:s1.t1,172.16.249.99,3260,1,default st_config
(4)登陆,登陆后本机会多出两块硬盘,出现successful表明登陆成功,本地会多出两块硬盘
# iscsiadm -m node -d 1 -T iqn.2015-06.com.linuxea:s1.t1 -p 172.16.249.99:3260 -l (iscsiadm -m node指明节点,-d 1级别 ,-T指明发现的tgtd,iqn.2015-06.com.linuxea:s1,-p指明主机ip端口 -l登陆) Logging in to [iface: default, target: iqn.2015-06.com.linuxea:s1.t1, portal: 172.16.249.99,3260] (multiple) Login to [iface: default, target: iqn.2015-06.com.linuxea:s1.t1, portal: 172.16.249.99,3260] successful.
多出的两块硬盘 1. fdisk -l /dev/sd[a-z]
Disk /dev/sda: 128.8 GB, 128849018880 bytes 255 heads, 63 sectors/track, 15665 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0005c1f3
Device Boot Start End Blocks Id System /dev/sda1 * 1 26 204800 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 26 7859 62914560 8e Linux LVM
Disk /dev/sdb: 21.5 GB, 21474836480 bytes 64 heads, 32 sectors/track, 20480 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 Disk /dev/sdc: 21.5 GB, 21474836480 bytes 64 heads, 32 sectors/track, 20480 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000 登出:-u 1. iscsiadm -m node -d 1 -T iqn.2015-06.com.linuxea:s1.t1 -p 172.16.249.99:3260 -u 当你登出后,在下次系统启动时,还会继续自动挂载这两块硬盘,使用delete删除 #iscsiadm -m node -d 1 -T iqn.2015-06.com.linuxea:s1.t1 -p 172.16.249.99:3260 -哦 delete 在查看/var/lib/iscsi/send_targets/172.16.249.99,3260/目录下将会为空 1. ls /var/lib/iscsi/send_targets/172.16.249.99,3260/ st_config 如果你想和它彻底断绝关系,从此相忘于江湖,分手,分手你要把照片还给我 直接把/172.16.249.99,3260/目录删掉
有一天,你知道你错了,想起了172.16.249.99的好,你想重燃旧情,这时候需要再次发现它,如下 1. iscsiadm -m discovery -d 3 -t st -p 172.16.249.99:3260 然后,在登陆 1. iscsiadm -m node -d 1 -T iqn.2015-06.com.linuxea:s1.t1 -p 172.16.249.99:3260 -l (5)格式化,挂载 我们开始分区格式化,一切重新开始吧 #fdisk /dev/sdb n新分区 p主分区 1分区号 从0扇区0磁道开始到+1G w保存退出 #mke2fs -t ext4 /dev/sdb1 #mount /dev/sdb1 /mnt #cd /mnt 1. ls lost+found
为了演示效果,复制一个文件测试 cp /etc/issue /mnt vim /mnt/issue 添加node2
但ls发现出现lost+found后就证明你们真的好了,当然,一切的一切才刚开始 另外,这里创建的分区是透明的,172.16.249.99是可以看见的。
(6)node1 1,安装包 1. yum -y install iscsi-initiator-utils 2,修改iqn文件 1. echo "InitiatorName=$(iscsi-iname -p iqn.2015-06.com.linuxea)" > /etc/iscsi/initiatorname.iscsi 3,启动iscsi 1. for i in iscsi iscsid; do service $i start; done 4,发现 #iscsiadm -m discovery -t st -p 172.16.249.99:3260 5,登陆 #iscsiadm -m node -T iqn.2015-06.com.linuxea:s1.t1 -p 172.16.249.99:3260 -l 查看 #fdisk -l /dev/sd[a-z] 挂载 #mount /dev/sdb1 /mnt #cd /mnt #cat /mnt/issue #vim /mnt/issue 删除node2
注意: 回到noed2查看,issue中的node2是还在的,这是因为缓存在内存中,文件中改变和未改变取决于谁最后一次操作。在双方都没退出之前,这个数据是处于不确定状态。 这里就出现一个问题了,每个主机在使用iscsi时候,只能同时单个使用同一个文件系统,不能同时使用同一个文件系统,应该为不同的主机分配不同的文件系统。
基于用户的账号认证:两端进行交涉,都需要给对方提供认证 查看是否有账号认证信息 server端: #tgtadm -L iscsi -m account -o show #tgtadm -L iscsi -m account -o new --user mark --password mark 1. tgtadm -L iscsi -m account -o show Account list: mark
绑定用户和tgtd #tgtadm -L iscsi -m account -m account -o bind --tid 1 --user mark 解绑 #tgtadm -L iscsi -m account -m account -o unbind --tid 1 --user mark 客户端: 1. vim /etc/iscsi/iscsid.conf node.session.auth.username = username node.session.auth.password = password discovery.sendtargets.auth.username = username discovery.sendtargets.auth.password = password 配置完后重启就可以使用! iscsi在windwos也是可发现的,win7直接支持,winxp需要装插件! 这种配置在系统重启会失效的,要想永久有效,需要修改配置文件!
配置target第二种方式:
先节点上删除 1. iscsiadm -m node -T iqn.2015-06.com.linuxea:s1.t1 -p 172.16.249.99:3260 -u 1. iscsiadm -m node -T iqn.2015-06.com.linuxea:s1.t1 -p 172.16.249.99:3260 -o delete
在服务端删除
tgtadm -L iscsi -m logicalunit -o delete -t 1 -l 2
tgtadm -L iscsi -m logicalunit -o delete -t 1 -l 1
tgtadm -L iscsi -m target -o delete -t 1
1. tgtadm -L iscsi -m target -o show
show应该没有任何信息了
1. cd /etc/tgt/
1. cp targets.conf {,.bak}
1. vim targets.conf (61行添加)
启动 #for i in tgtd; do service $i start; done 查看 1. tgtadm -L iscsi -m target -o show Target 1: iqn.2015-6.com.linuxea:s2.t1 System information: Driver: iscsi State: ready I_T nexus information: LUN information: LUN: 0 Type: controller SCSI ID: IET 00010000 SCSI SN: beaf10 Size: 0 MB, Block size: 1 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: null Backing store path: None Backing store flags: LUN: 1 Type: disk SCSI ID: IET 00010001 SCSI SN: beaf11 Size: 21475 MB, Block size: 512 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: rdwr Backing store path: /dev/sdb Backing store flags: LUN: 2 Type: disk SCSI ID: IET 00010002 SCSI SN: beaf12 Size: 21475 MB, Block size: 512 Online: Yes Removable media: No Prevent removal: No Readonly: No Backing store type: rdwr Backing store path: /dev/sdc Backing store flags: Account information: ACL information: 172.16.0.0/16 [root@node3 tgt]# 重启服务生效。 在node2上发现,没有问题 1. iscsiadm -m discovery -t st -p 172.16.249.99:3260 172.16.249.99:3260,1 iqn.2015-6.com.linuxea:s2.t1
# iscsiadm -m node -T iqn.2015-6.com.linuxea:s2.t1 -p 172.16.249.99:3260 -l Logging in to [iface: default, target: iqn.2015-6.com.linuxea:s2.t1, portal: 172.16.249.99,3260] (multiple) Login to [iface: default, target: iqn.2015-6.com.linuxea:s2.t1, portal: 172.16.249.99,3260] successful. 1. fdisk -l /dev/sd[a-z]
Disk /dev/sda: 128.8 GB, 128849018880 bytes 255 heads, 63 sectors/track, 15665 cylinders Units = cylinders of 16065 * 512 = 8225280 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x0005c1f3
Device Boot Start End Blocks Id System /dev/sda1 * 1 26 204800 83 Linux Partition 1 does not end on cylinder boundary. /dev/sda2 26 7859 62914560 8e Linux LVM
Disk /dev/sdb: 21.5 GB, 21474836480 bytes 64 heads, 32 sectors/track, 20480 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0xce2431b7
Device Boot Start End Blocks Id System /dev/sdb1 1 1025 1049584 83 Linux
Disk /dev/sdc: 21.5 GB, 21474836480 bytes 64 heads, 32 sectors/track, 20480 cylinders Units = cylinders of 2048 * 512 = 1048576 bytes Sector size (logical/physical): 512 bytes / 512 bytes I/O size (minimum/optimal): 512 bytes / 512 bytes Disk identifier: 0x00000000
[root@[node2] ~]# PS:基于图像界面管理的开源软件有以下: Openfiler(centos),FreeNAS(FreeBSD),Nexenta.. Openfiler分两个版本一个商业版,一个社区版,社区版被阉割的较为严重,而且它的更新周期太慢。