path: root/README

=================================================================

                Linux* Open-iSCSI

=================================================================

                                                     Jan 26, 2007

Contents
========

- 1. In This Release
- 2. Introduction
- 3. Installation
- 4. Open-iSCSI daemon
- 5. Open-iSCSI Configuration Utility
- 6. Configuration
- 7. Getting Started
- 8. iSCSI System Info


1. In This Release
==================

This file describes the Linux* Open-iSCSI Initiator. The software was
tested on AMD Opteron (TM) and Intel Xeon (TM). 

The latest development release is available at:
http://www.open-iscsi.org

For questions, comments, contributions send e-mail to:
open-iscsi@googlegroups.com 

    1.1. Features
    
    - highly optimized and very small-footprint data path;
    - persistent configuration database;
    - SendTargets discovery;
    - CHAP;
    - PDU header Digest;
    - multiple sessions;
    
    For the most recent list of features please refer to:
    http://www.open-iscsi.org/cgi-bin/wiki.pl/Roadmap

2. Introduction
===============

Open-iSCSI project is a high-performance, transport independent,
multi-platform implementation of RFC3720 iSCSI.

Open-iSCSI is partitioned into user and kernel parts.

The kernel portion of Open-iSCSI is a from-scratch code
licensed under GPL. The kernel part implements iSCSI data path
(that is, iSCSI Read and iSCSI Write), and consists of three
loadable modules: scsi_transport_iscsi.ko, libiscsi.ko and iscsi_tcp.ko.

User space contains the entire control plane: configuration
manager, iSCSI Discovery, Login and Logout processing,
connection-level error processing, Nop-In and Nop-Out handling,
and (in the future:) Text processing, iSNS, SLP, Radius, etc.

The user space Open-iSCSI consists of a daemon process called
iscsid, and a management utility iscsiadm.


3. Installation
===============

As of today, the Open-iSCSI Initiator requires a host running the
Linux operating system with kernel version 2.6.16, or later. See
http://www.open-iscsi.org/cgi-bin/wiki.pl/Supported_Kernels
for a more information about which kernels Open-iSCSI supports.
You need to enable "Cryptographic API" under "Cryptographic options" in the
kernel config. And you must enable "CRC32c CRC algorithm" even if
you do not use header or data digests. They are the kernel options,
CONFIG_CRYPTO and CONFIG_CRYPTO_CRC32C, respectively.

By default the kernel source found at
/lib/modules/`uname -a`/build
will be used to compile the open-iscsi modules. To specify a different
kernel to build against use:

	make KSRC=<kernel-src>

or cross-compilation:

	make KSRC=<kernel-src> KARCH="ARCH=um"

To compile on SUSE Linux you'll have to use

	make KSRC=/usr/src/linux \
	     KBUILD_OUTPUT=/usr/src/linux-obj/<arch>/<config>

where <config> is the kernel configuration to use (eg. 'smp').

If you choose to install the Debian packages instead of building from source,
please read the file /usr/share/doc/linux-iscsi/README.debian for information
on how to build kernel modules against your specific kernel.

For Red Hat/Fedora and Debian distributions open-iscsi can be installed by
typing "make install". This will copy iscsid and iscsiadm to /usr/sbin, the
init script to /etc/init.d, and the kernel modules: iscsi_tcp.ko, libiscsi.ko
and scsi_transport_iscsi to /lib/modules/`uname -r`/kernel/drivers/scsi/
overwriting existing iscsi modules.

4. Open-iSCSI daemon
====================

The daemon implements control path of iSCSI protocol, plus some management
facilities. For example, the daemon could be configured to automatically 
re-start discovery at startup, based on the contents of persistent 
iSCSI database (see next section).

For help, run:

	./iscsid --help

Usage: iscsid [OPTION]

  -c, --config=[path]     Execute in the config file (/etc/iscsi/iscsid.conf).
  -f, --foreground        run iscsid in the foreground
  -d, --debug debuglevel  print debugging information
  -u, --uid=uid           run as uid, default is current user
  -g, --gid=gid           run as gid, default is current user group
  -h, --help              display this help and exit
  -v, --version           display version and exit



5. Open-iSCSI Configuration Utility
===================================

Open-iSCSI persistent configuration is implemented as a DBM database
available on all Linux installations.

The database contains two tables:

- Discovery table (/etc/iscsi/send_targets);
- Node table (/etc/iscsi/nodes).

The regular place for iSCSI database files: /etc/iscsi/nodes

The iscsiadm utility is a command-line tool to manage (update, delete,
insert, query) the persistent database.

The utility presents set of operations that a user can perform 
on iSCSI nodes, sessions, connections, and discovery records.

Open-iscsi does not use the term node as defined by the iSCSI RFC,
where a node is a single iSCSI initiator or target. Open-iscsi uses the
term node to refer to a portal on a target, so tools like iscsiadm
require that --targetname and --portal argument be used when in node mode.

For session mode, a session id (sid) is used. The sid of a session can be
found by running iscsiadm -m session -i. The session id is not currently
persistent and is partially determined by when the session is setup.

Note that some of the iSCSI Node and iSCSI Discovery operations 
do not require iSCSI daemon (iscsid) loaded.

For help, run:

	./iscsiadm --help

Usage: iscsiadm [OPTION]

  -m, --mode <op>         specify operational mode op = <discovery|node>
  -m discovery --type=[type] --portal=[ip:port] --login
                          perform [type] discovery for target portal with
                          ip-address [ip] and port [port]. Initiate Login for
                          each discovered target if --login is specified
  -m discovery            display all discovery records from internal
                          persistent discovery database
  -m discovery --portal=[ip:port] --login
                          perform discovery based on portal in database
  -m discovery --portal=[ip:port] --op=[op] [--name=[name] --value=[value]]
                          perform specific DB operation [op] for specific
                          discovery portal. It could be one of:
                          [new], [delete], [update] or [show]. In case of
                          [update], you have to provide [name] and [value]
                          you wish to update
  -m node                 display all discovered nodes from internal
                          persistent discovery database
  -m node --targetname=[name] --portal=[ip:port] --interface=[HWaddress] \
					[--login|--logout|--rescan|--stats]
  -m node --targetname=[name] --portal=[ip:port] --interface=[HWaddress] \
				--op=[op] [--name=[name] --value=[value]]
  -m node --targetname=[name] --portal=[ip:port] --interface=[HWaddress] \
				--print=[level]
                          perform specific DB operation [op] for specific
                          interface on host that will connect to portal on
			  target. targetname, portal and interface are optional.

			  HWaddress must be he formed as: XX:XX:XX:XX:XX:XX.

			  op could be one of:
                          [new], [delete], [update] or [show]. In case of
                          [update], you have to provide [name] and [value]
                          you wish to update

			  Print level can be 0 to 1.

			  Rescan will perform a SCSI layer scan of the session
			  to find new LUNs.

			  Stats prints the iSCSI stats for the session.
  -m node --logoutall=[all,manual,automatic]
			  Logout "all" the running sessions or just the ones
			  with a node or conn startup value manual or automatic.
			  Nodes marked as ONBOOT are skipped.
  -m node --loginall=[all,manual,automatic]
			  Login "all" the running sessions or just the ones
			  with a node or conn startup value manual or automatic.
			  Nodes marked as ONBOOT are skipped.
  -m session              display all active sessions and connections
  -m session --sid=[sid] [ --print=level | --rescan | --logout ]
				--op=[op] [--name=[name] --value=[value]]
                          perform operation for specific session with
                          session id sid. If no sid is given the operation
			  will be performed on all running sessions if possible.
			  --logout and --op work like they do in node mode,
			  but in session mode targetname and portal info is
			  is not passed in.

			  Print level can be 0 to 2.

			  If no sid and no operation is given print out the
			  running sessions.
  -d, --debug debuglevel  print debugging information
  -V, --version           display version and exit
  -h, --help              display this help and exit


    Usage examples using the one-letter options (see iscsiadm man page
    for long options):

    Discovery mode:

    - SendTargets iSCSI Discovery:

	    ./iscsiadm -m discovery -t sendtargets -p 192.168.1.1:3260


    Node mode. In node mode you can specify which records you want to log
    into by specifying the targetname, ip address, port or interface
    (if specifying the interface it must already be setup in the node db).
    iscsiadm will search the node db, for records which match the values
    you pass in, so if you pass in the targetname and interface, iscsiadm
    will search for records with those values and operate on only them.
    Passing in none of them will result in all node records being operated on.

    - iSCSI Login to all portals on every node/starget through each interface
	set in the db:

	    ./iscsiadm -m node -l

    - iSCSI login to all portals on a node/target through each interface set
	in the db:

	    ./iscsiadm -m node -T iqn.2005-03.com.max -l

    - iSCSI login to a specific portal through each interface set in the db:

	    ./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 -l

	To specify a IPv6 address the following can be used:

	    ./iscsiadm -m node -T -p 2001:c90::211:9ff:feb8:a9e9 -l

	The above command would use the default port, 3260. To specify a
	port use the following:

	    ./iscsiadm -m node -T -p [2001:c90::211:9ff:feb8:a9e9]:3260 -l

    - iSCSI Login to a specific portal through the NIC with MAC/HW
	address 00:0F:1F:92:6B:BF set in the db:

	    ./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 \
				-I 00:0F:1F:92:6B:BF -l

    - iSCSI Logout to all portals on every node/starget through each interface
	set in the db:

	    ./iscsiadm -m node -u

	Warning: this does not check startup values like the logout/login all
	option. Do not use this if you are running iscsi on your root disk.	

    - iSCSI logout to all portals on a node/target through each interface set
	in the db:

	    ./iscsiadm -m node -T iqn.2005-03.com.max -u

    - iSCSI logout to a specific portal through each interface set in the db:

	    ./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 -u

    - iSCSI Logout to a specific portal through the NIC with MAC/HW
	address 00:0F:1F:92:6B:BF set in the db:

	    ./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 \
				-I 00:0F:1F:92:6B:BF -u

    - Changing iSCSI parameter:

	    ./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260 \
	       -o update -n node.cnx[0].iscsi.MaxRecvDataSegmentLength -v 65536

	You can also change paramaters for multiple records at once, by
	specifying different combinations of the target, portal and interface
	like above.

    - Adding custom iSCSI portal:

	    ./iscsiadm -m node -o new -T iqn.2005-03.com.max \
			-p 192.168.0.1:3260 -I 00:0F:1F:92:6B:BF

	The -I/--interface is optiniol. If not passed in, "default" is used.
	For iscsi_tcp, this would allow the network layer to decide what is
	best.

    - Removing iSCSI portal:

	    ./iscsiadm -m node -o delete -T iqn.2005-03.com.max -p 192.168.0.4:3260

	You can also delete multiple records at once, by specifying different
	combinations of the target, portal and interface like above.

    - Display iSCSI portal onfiguration:

	    ./iscsiadm -m node -T iqn.2005-03.com.max -p 192.168.0.4:3260

	or

	    ./iscsiadm -m node -o show -T iqn.2005-03.com.max -p 192.168.0.4:3260
	You can also display multiple records at once, by specifying different
	combinations of the target, portal and interface like above.

	Note: running "iscsiadm -m node" will only display the records. It
	will not display the configuration info. You must run,
	"iscsiadm -m node -o show".

    - Show all node records:

            ./iscsiadm -m node

	This will print the nodes using the old flat format where the
	interface and driver are not displayed. To display that info
	use the -P argument with the arguent "1":

	    ./iscsiadm -m node -P 1

    - Show all records in discovery database:

            ./iscsiadm -m discovery

    - Display discovery record setting:

            ./iscsiadm -m discovery -p 192.168.0.4:3260

    - Display session statistics:

            ./iscsiadm -m session -r 1 --stats

    - Display running sessions:

	    ./iscsiadm -m session -P 1

6. Configuration
================

The default configuration file is /etc/iscsi/iscsid.conf. This file contains
only configuration that could be overwritten by iSCSI Discovery,
or manualy updated via iscsiadm utility. Its OK if this file does not
exist in which case compiled-in default configuration will take place
for newer discovered Target nodes.

See the man page and the example file for the current syntax.
The manpages for iscsid, iscsiadm are in the doc subdirectory and can be
installed in the appropriate man page directories and need to be manually
copied into e.g. /usr/local/share/man8.

7. Getting Started
==================
There are three steps needed to set up a system to use iSCSI storage:
7.1. iSCSI startup using the init script or manual startup.
7.2. Discover targets.
7.3. Automate target logins for future system reboots.

The init scripts will start the iSCSI daemon and log into any
connections or nodes that are set up for automatic login. If your distro
does not have a init script, then you will have to start the daemon
and log into the targets manually.

7.1.1 iSCSI startup using the init script
-----------------------------------------------

Red Hat or Fedora:
-----------------
To start open-iscsi in Red Hat/Fedora you can do:

	service open-iscsi start

To get open-iscsi to automatically start at run time you may have to
run:
	chkconfig --level <levels> open-iscsi on
Where <levels> are the run levels.

And, to automatically mount a file system during startup
you must have the partition entry in /etc/fstab marked with the "_netdev"
option. For example this would mount a iscsi disk sdb:

	/dev/sdb /mnt/iscsi ext3 _netdev 0 0

SUSE or Debian:
---------------
Otherwise, if there is a initd script for your distro in etc/initd that
gets installed with "make install"

	/etc/init.d/open-iscsi start

will usually get you started.

7.1.2 Manual Startup:
---------------------

7.1.2.1 Starting up the iSCSI daemon (iscsid) and loading modules:
-----------------------------------------------------------------
If there is no initd script, you must start the tools by hand. First load the
iscsi modules with:

	modprobe -q iscsi_tcp

after that start iSCSI daemon process:

	./iscsid

or alternatively, start it with debug enabled and with output
redirected to the current console:

	./iscsid -d8 -f &

7.1.2.2 Logging into Targets:
---------------------------
Use the configuration utility, iscsiadm, to add/remove/update Discovery
records, iSCSI Node records or monitor active iSCSI sessions (see above or the
iscsiadm man files and see section 7.2 below for how to discover targets).

	./iscsiadm  -m node

will print out the nodes that have been discovered as:

	10.15.85.19:3260,3 iqn.1992-08.com.netapp:sn.33615311 
	10.15.84.19:3260,2 iqn.1992-08.com.netapp:sn.33615311

The format is:

ip:port,target_portal_group_tag targetname

If you are using the iface.name argument or want to see the driver
info use the following:

	./iscsiadm -m node -P 1

target: iqn.2001-04.com.home:max2
        portal: 20.15.0.5:3260
                driver: tcp hwaddress: 00:0F:1F:92:6B:BF
                driver: tcp hwaddress: 00:C0:DD:08:63:E7

The format is:

target: targetname
	portal ip_address:port
		driver: driver hwaddress: hwaddress

where targetname is the name of the target and ip_address:port is the address
and port of the portal. target_portal_group_tag, is the portal group tag of
the portal, and is not used in iscsiadm commands. Driver is the iscsi
transport/interconnect driver that will be used to access the portal
and HWaddress is the address of the NIC/HBA which the session will go through.
For tcp/iscsi_tcp "default" means that we will create one session to the
portal and that the network layer will decide how to best route it through
the NICs on the system.

To login, take the ip, port and targetname from above and run:

	./iscsiadm -m node -T targetname -p ip:port -l

In this example we would run

	./iscsiadm -m node -T iqn.1992-08.com.netapp:sn.33615311 -p 10.15.84.19:3260 -l

	Note: drop the portal group tag from the "iscsiadm -m node" output.

7.2. Discover Targets
---------------------
Once the iSCSI service is running, you can perform discovery using
SendTarget with:

iscsiadm -m discovery -t sendtargets -p ip:port

where "ip" is the address of the portal and port is the port.

Or you can you perform discovery using iSNS by setting the address
of the iSNS server in iscsid.conf with the "isns.address" value and
running:

iscsiadm -m discovery -t isns

Both commands will print out the list of all discovered targets and their
portals:

# iscsiadm -m discovery -t st -p 10.15.85.19:3260
tcp:default 10.15.85.19:3260,3 iqn.1992-08.com.netapp:sn.33615311
tcp:default 10.15.84.19:3260,2 iqn.1992-08.com.netapp:sn.33615311

Note: this prints out every node in the db including the ones just discovered.
This is a bug and will change in future releases.

The format for the output is:

driver:HWaddress ip:port,target_portal_group_tag targetname

In this example, for the first target the ip address is 10.15.85.19.
The port is 3260. The target portal group is 3. The target name
is iqn.1992-08.com.netapp:sn.33615311. The driver that will be used
is tcp/iscsi_tcp, and we will allow the network layer to decide
how to best route IO through the NICs on the system.

While discovery targets are kept in the discovery db, they are
usefull only for re-discovery. The discovered targets (a.k.a. nodes)
are stored as records in the node db.

The discovered targets are not logged into yet. Rather than logging
into the discovered nodes (making LUs from those nodes available as
storage), it is better to automate the login to the nodes we need.

If you wish to log into a target manually now, see section
"7.1.2.2 Logging in targets" above.

7.3. Automate Target Logins for Future System Statups
-----------------------------------------------------
Note: this may only work for distros with init scripts.

To automate login to a node, use the following with the record ID
(record ID is the targetname and portal) of the node discovered in the
discovery above:
	iscsiadm -m node -T targetname -p ip:port --op update -n node.conn[0].startup -v automatic

Or to set the "node.conn[0].statup" attribute to "startup" as default for
all sessions add the following to the /etc/iscsi/iscsid.conf:

	node.conn[0].startup = automatic

Setting this in iscsid.conf, will not affect existing nodes. It will only
affect nodes that are discovered after setting the value.

To login to all the automated nodes, simply restart the iscsi service:
e.g /etc/init.d/open-iscsi restart. On your next startup the nodes will
be logged into autmotically.


8. iSCSI System Info
====================

To get information about the running sessions: including the session and
device state, session ids (sid) for session mode, and some of the
negioated parameters, run:

	iscsiadm -m session -P 2

If you are looking for something shorter like just the sid to node mapping
run:

	iscsiadm -m session -P 0

This will print the list of running sessions with the format:

driver [sid] ip:port,target_portal_group_tag targetname

# iscsiadm -m session
tcp [2] 10.15.84.19:3260,2 iqn.1992-08.com.netapp:sn.33615311
tcp [3] 10.15.85.19:3260,3 iqn.1992-08.com.netapp:sn.33615311

To print the hw address info use the -P option with "1":

       iscsiadm -m session -P 1

This will print the sessions with the following format:

target: targetname
	portal: ip_address:port
		driver: driver hwaddress: hwaddress sid: sid

[root@madmax usr]# ./iscsiadm -m session -P 1
target: iqn.2001-04.com.home:max2
        portal: 20.15.0.5:3260
                driver: tcp hwaddress: 00:C0:DD:08:63:E7 sid: 20
		driver: tcp hwaddress: 00:C0:DD:08:63:E7 sid: 21

For example this first node is using the iscsi_tcp kernel module, has
session id (sid) 2, is connected to a portal with address and port
10.15.84.19:3260 in portal group 2 on the target,
iqn.1992-08.com.netapp:sn.33615311.