Scsi_debug adapter driver for Linux

Introduction

A small but hopefully useful set of SCSI commands is supported along with some crude error checking. The number of simulated devices and the shared RAM size for storage can be given as module parameters or boot time parameters if the scsi_debug driver is built into the kernel. The number of simulated devices (and hosts) can be varied at run time via sysfs. Various error conditions can be optionally generated to test the reaction of upper levels of the kernel and applications to abnormal situations. 

This page describes the this driver as found in the Linux kernel version 3.17.0 and earlier versions of this driver worked with the Linux kernel 2.6 series.  For information about the scsi_debug driver found in the lk 2.4 production series see this page.

Parameters

The delay parameter is the number of jiffies by which the driver will delay responses. The default is 1 jiffy unless the ndelay parameter is given, see its description. Setting this parameter to 0 will cause the response to be sent back to the mid level before the request function is completed. The "jiffy" is a kernel space jiffy (typically the largest HZ figure yields a 1 millisecond on i386) rather than a user space jiffy (USER_HZ is typically 10 milliseconds on i386). HZ and USER_HZ are configurable in the kernel build. Both delayed and immediate responses are permitted however delayed responses are more realistic. For delayed responses, a kernel timer is used. [Real adapters would generate an interrupt when the response was ready (i.e. the command had completed).] For a fast ram disk set the delay parameter to 0. These SCSI commands ignore the delay parameter and respond immediately: INQUIRY, REPORT LUNS, REQUEST SENSE, SYNCHRONIZE CACHE plus various other non "media access" commands. TEST UNIT READY is considered a media access command.

The delay parameter may be set to -1 or -2 which uses a kernel tasklet to generate a more or less immediate response (but in a different kernel thread). The -1 variant schedules a high priority tasklet while -2 schedules a normal priority tasklet. Trying to write a new value to delay while there are queued command responses may result in an EBUSY error.

The dev_size_mb parameter allows the user to specify the size of the simulated storage. The unit is Mebibytes (each 2**20 bytes and a bit larger than a Megabyte) and the default value is 8. The maximum value depends on the capabilities of the vmalloc() call on the target architecture. If the module fails to load with a "cannot allocate memory" message then a "vmalloc=nn{KMG}" boot time argument may be needed. [See the kernel source file: Documentation/kernel-parameters.txt for more information on this.] The RAM reserved for storage is initialized to zeros which leads the sd (scsi disk) driver and the block layer to believe there is no partition table present. Partitions can be simulated with num_parts (see below). All simulated dummy devices share the same RAM. If a value of 0 or less is given then dev_size_mb is forced to 1 so 1 MB of RAM is used. Given 512 byte logical blocks, the largest ramdisk that can be allocated is 2 TB but it is unlikely a system would be able to allocate that much ram (a situation that would be bypassed if fake_rw=1). Very large amounts of "virtual" storage can be simulated with the virtual_gb parameter (see below).

The every_nth parameter takes a decimal number as an argument. When this number is greater than zero, then incoming commands are counted and when <n> is reached then the associated command generates some sort of error. Currently the available errors are timeout (when "opts & 4" is true) and RECOVERED_ERROR (when "opts & 8" is true) . Once the command count reaches <n> then it is reset to zero. For example setting every_nth to 3 and opts to 4 will cause every third command to be ignored (and hence a timeout). If every_nth is not given it is defaulted to 0 and timeouts and recovered errors will not be generated.

If every_nth is negative then an internal command counter counts down to that value and when it is reached, continually generates the error condition (specified in opts) on each newly received command. The driver flags this continual error state by setting every_nth to -1 . The user can stop error conditions being generated on receipt of every subsequent command by writing 0 to every_nth (or opts ).

The fake_rw parameter instructs the scsi_debug driver to ignore all READ and WRITE commands and return a GOOD status. This means the data "read" when fake_rw is set is whatever was previously in the scatter gather list. The default value is 0 (i.e. process READ and WRITE commands). This parameter is for testing and when set can confuse the kernel or utilities that look for partitions and other information on a "disk".

The guard parameter when set to zero (the default) use T10 defined CRC in the protection information. When set to one the IP (internet protocol) checksum (as used by iSCSI ?) is used.

The host_lock parameter indicates whether each command (excluding its response delay and associated callback into the mid-layer) is surrounded by a per host host_lock (which is a kernel "spin lock"). In a SCSI multi-queue system the presence of this host lock will have the effect of serializing all commands form a host; and that is detrimental to system performance. Prior to version 1.84 this parameter was not available and the host_lock surround all commands. In version 1.84 and later the default is 0 which means the host_lock is not applied. Set host_lock=1 for the old behaviour.

The lbpu parameter, if set, causes the logical block partitioning VPD page to set the field of the same name. The default is to set the LBPU field to 0. When set this field indicates the UNMAP command is supported.

The lpbws and lbpws10 parameters cause the corresponding bits in the logical block partitioning VPD page to be set. The imply the the UNMAP field within the WRITE SAME(16) and WRITE_SAME(10) respectively are supported.

The lbprz parameter, if set, causes the logical block partitioning VPD page to set the field of the same name. When this field is set reading unmapped logical blocks will yield block(s) of data full of xeros to be returned.

The lowest_aligned parameter sets the field called LOWEST ALIGNED LOGICAL BLOCK ADDRESS in the READ CAPACITY (16) command response.
The default is zero which implies the logical block size and the physical block size are the same.

The max_luns parameter allows an upper limit to be placed on the logical unit number (lun) that the scsi_debug driver will respond to. A value of 2 means that this driver will respond to logical unit numbers 0 and 1. If max_luns is modified by a sysfs write then the scsi_debug driver modifies the scsi_host::max_lun member of all hosts that it owns. When max_luns is modified by a sysfs write then it will take effect the next time a host is added (see add_host) or when a scan is done on any existing host. The mid level scanning code will scan for up to but not including max_scsi_luns which is a SCSI mid level boot and module load time parameter.

The max_queue parameter indicates the maximum number of queued responses the driver can handle. This defaults to an internal define in the scsi_debug driver called SCSI_DEBUG_CANQUEUE which is currently 576. If both the delay and ndelay parameters are 0, no commands have queued responses. If there is an attempt to exceed this value then either SCSI_MLQUEUE_HOST_BUSY is returned to the mid-layer (the default) or a status of TASK_SET_FULL (if the 0x200 opts mask is set). Sysfs can be used at any time to change the value of max_queue, even when the are queued command responses.

The ndelay parameter is the response delay whose units are nanoseconds. This mechanism depends on high resolution timers in the kernel which may not be supported on small or old system (it is a kernel build config option). Its default value is 0 which means the delay parameter is operative. If ndelay is a positive value then a response delay for that many nanoseconds is active (and to indicate the delay parameter is overridden, it is set to -9999). Depending on the hardware, setting ndelay to less than a few microseconds probably causes no further reduction in the observed response delays. Trying to write a new value to ndelay while there are queued command responses may result in an EBUSY error.

The opts parameter takes a  number as an argument which is the bitwise "or" of several flags. The flags that mention "nth" are only active when every_nth != 0 . So-called "read-write" commands include some others such as VERIFY. The flags supported are:

•     1  -  "noisy" flag: all calls to entry points of driver are logged. Commands to be executed are shown in hex. Additional information such as check conditions, command aborts and resets are logged
  
•     2  -  "medium error" flag: simulates a SCSI MEDIUM ERROR when sector 0x1234 (that is 4660 in decimal) is read
•     4  -  ignore "nth" command causing a timeout.
•     8  -  cause "nth" read or write command to yield a RECOVERED_ERROR.
•     0x10 -  cause "nth" read-write command to yield an ABORTED_COMMAND (ack/nak timeout) which is a SAS transport error.
  
•     0x20 -  cause "nth" read-write command to yield an ABORTED_COMMAND (logical block guard check failed), nominally a DIF (Protection Information) error
•     0x40 -  cause "nth" read-write command to yield an ABORTED_COMMAND (logical block guard check failed), nominally a DIX error
•     0x80 -  ignore "nth" media access command causing a timeout
•     0x100 - cause "nth" read command to yield half the data it was requested to read
•     0x200 - log generation of TASK SET FULL and host busy plus changes to queue depth and type
•     0x400 - if max_queue is exceeded yield a TEST SET FULL (default: host busy)
•     0x800 - cause "nth" read-write command whose queue_depth is at it maximum value to yield a status of TASK SET FULL
•     0x1000 - set WCE field in the caching page to 0 (default WCE=1)
•     0x2000 - log only abort commands and the various levels of reset
•     0x4000 - used together with the noisy flag (1) to suppress the logging of cdbs; additional information (if any) is still logged.
  

Supported SCSI commands

Below is a list of supported commands. Some do nothing (e.g. SYNCHRONIZE CACHE). Those that have interesting functionality have notes in brackets. If the feature was introduced in a recent version (i.e. since 1.76) then that is noted. 

• ALLOW MEDIUM REMOVAL
• GET LBA STATUS
  
• INQUIRY [vital product data pages: 0, 0x80, 0x83] [1.77: VPD pages: 0x85, 0x86, 0x87, 0x88, 0x89, 0xb0]
  
• LOG SENSE [1.78: temperature(0xd) and informational exceptions(0x2f)] [1.80: support log subpages]
  
• MODE SELECT (6), MODE SELECT (10) [1.84: changeable pages: 0x8 (caching), 0xa (control) and 0x1c (informational exceptions)]
  
• MODE SENSE (6), MODE_SENSE (10) [sense pages: 1 (rw error recovery), 2 (disconnect), 3 (format), 8 (caching), 0xa (control), 0x1c (informational exceptions), 0x3f (read all)] [1.77: subpage support plus SAS pages: 0x19,0 0x19,1 and 0x19,2]
  
• READ (6), READ (10), READ(12), READ(16), READ(32)
  
• READ CAPACITY (10), READ CAPACITY (16) [1.79: added 16 byte command]
  
• RELEASE (6), RELEASE (10)
• REPORT LUNS [1.77: shows REPORT LUNS wlun]
• REPORT TARGET PORT GROUPS
  
• REQUEST SENSE [1.79: shows MRIE=6 failure prediction, power states]
  
• RESERVE (6), RESERVE (10)
• REZERO UNIT (which is REWIND for tapes)
  
• SEND DIAGNOSTIC
• START STOP [1.78: maintains start and stop states, when stopped fails media access commands]
  
• SYNCHRONIZE CACHE
  
• TEST UNIT READY [1.78: in stopped state gives appropriate error]
  
• VERIFY (10)
  
• WRITE (6), WRITE (10), WRITE (12), WRITE (16), WRITE(32)
• WRITE SAME(10), WRITE SAME(16)
• UNMAP
• XDWRITEREAD [which is a bidirectional command]
  

Modern SCSI devices use vital product page 0x83 for identification. This driver yields both "T10 vendor identification" and "NAA" descriptors. The former yields an ASCII string like "Linux   scsi_debug      4000" where the "4000" is the ((host_no + 1) * 2000) + (target_id * 1000) + lun). In this case "4000" corresponds to host_no==1,  target_id==0 and lun==0. The "NAA-5" descriptor is an 8 byte binary value that looks like this hex sequence: "51 23 45 60 00 00 0f a0" where the IEEE company id is 0x123456 (fake) and the vendor specific identifier in the least significant bytes is 4000 (which is fa0 in hex). [The "4000" is derived the same way for both descriptors.]

Read and write commands executed by the scsi_debug driver are atomic (i.e. a write to one scsi_debug device will not interrupt (split) a read from another scsi_debug device. So a  read command will either yield the contents of ram before a co-incident write, or after the co-incident write has finished

Logical and physical block size

Thin provisioning

• unmap_max_desc specifies the maximum number of ranges that can be unmapped using a single UNMAP command. If this is set to 0, only WRITE SAME is supported and UNMAP will cause a check condition.
• unmap_granularity specifies the granularity at which to track mapped blocks (specified in number of logical blocks). 2048 (1 MB) is a realistic value for disk arrays although some may have a finer granularity.
• unmap_alignment specifies the first LBA which is naturally aligned on an unmap_granularity boundary.
• unmap_max_blocks specifies the maximum number of blocks that can be unmapped using a single UNMAP command. Default is 0xffffffff.
  

Examples

Basic

# modprobe scsi_debug
# lsscsi -s
[0:0:0:0]    disk    SEAGATE  ST33000650SS     0005  /dev/sda   3.00TB
[0:0:1:0]    enclosu Intel    RES2SV240        0d00  -               -
[4:0:0:0]    disk    ATA      ST3160812AS      D     /dev/sdb    160GB
[7:0:0:0]    disk    Linux    scsi_debug       0184  /dev/sdc   8.38MB

In this case there is a 3 TB SAS disk, an ATA disk and a small scsi_debug pseudo disk. The other device (at [0:0:1:0]) is a SCSI Enclosure Service (SES) device.  The /dev/sdc pseudo disk is full of zeros and has no partitions. To get a partition the num_parts parameter could have been used on the modprobe line or it could be done from the command line with the fdisk /dev/sdc command. Assuming one ext3 partition is allocated to the whole pseudo disk (8 MB in this case) then the mkfs.ext3 /dev/sdc1 command can be used to make an ext3 file system. Now /dev/sdc1 can be mounted and treated like a normal file system. Naturally when the power is turned off anything stored in /dev/sdc1 will be forgotten.

Rather than mounting the pseudo disk, the sg3_utils package could be used to carry out various tests on it.

Information about the scsi_debug driver version, its current parameters and some other data can be found in the "proc" file system. The trailing number in the path is the scsi_debug host number which is the first element in the 4 item tuple shown in the lsscsi above :

# cat /proc/scsi/scsi_debug/7
scsi_debug adapter driver, version 1.84 [20140706]
num_tgts=1, shared (ram) size=8 MB, opts=0x0, every_nth=0
delay=1, ndelay=0, max_luns=1, q_completions=1859648
sector_size=512 bytes, cylinders=64, heads=8, sectors=32
command aborts=0; RESETs: device=0, target=0, bus=0, host=0
dix_reads=0 dix_writes=0 dif_errors=0 usec_in_jiffy=1000

Here is an important sysfs directory for the scsi_debug driver:

# cd /sys/bus/pseudo/drivers/scsi_debug/
# ls
adapter0  dev_size_mb  fake_rw    max_queue  num_tgts    sector_size
add_host  dif          guard      ndelay     opts        uevent
ato       dix          host_lock  no_lun_0   ptype       unbind
bind      dsense       map        no_uld     removable   virtual_gb
delay     every_nth    max_luns   num_parts  scsi_level  vpd_use_hostno

Those files are most of the scsi_debug parameters, those that are writable can be modified and the scsi_debug actions will change accordingly thereafter. Certain parameters cannot be changed while the driver is busy (e.g. it has queued command responses), in which case EBUSY is returned if the user attempts to change one. Reading one can be done with the cat command and changing one can be done with the echo command:

# cat every_nth
0
# echo 2000 > every_nth

Another important sysfs directory for (any) disks is /sys/block/<disk_node_name> and its queue sub-directory. So in this case of this scsi_debug pseudo disk that directory would be /sys/block/sdc/queue . Also there is the scsi_device sysfs directory that has the form /sys/class/scsi_device/<h:c:t:l>/device where the <h:c:t:l> tuple is found at the left hand side of each device listed by lsscsi. This sysfs directory contains many important SCSI device parameters some of which can be modified.

Adding and removing hosts and devices

• increase the value of num_tgts or max_luns then use a line like 'echo "0 - -" > scan' (shown above) to a host already owned by the scsi_debug driver.
• add more hosts with a line like 'echo 3 > add_host'. Each new host will create (num_tgts * max_luns) new scsi devices. Of course num_tgts or max_luns can be modified prior to calling 'echo 3 > add_host'.

Mode pages

• caching (WCE field is changeable) [added in version 1.84]
  
• control (D_SENSE field is acted upon)
• informational exceptions control (MRIE and TEST fields are acted upon by REQUEST SENSE)

REPORT LUNS Well Known LU

SAS personality

• device identification page [0x83] (yields naa-5 addresses for the lu, the accessing target port and the target device, plus some other designators)
  
• SCSI ports [0x88] (shows the naa-5 addresses of both ports)
• ATA information [0x89] (simulates a SATA disk in a SAS domain, defined in SAT)

• protocol specific port page (SAS): short format page [0x19,0x0]
• protocol specific port page (SAS): phy control and discover subpage [0x19,0x1]
• protocol specific port page (SAS): shared mode subpage [0x19,0x2] (sas2 version)

Downloads

Conclusion

 Back  to main page

Douglas Gilbert <dgilbert at interlog dot com>
with additions from
Martin K. Petersen <martin dot petersen at oracle dot com>

Last updated: 10th July 2014