DataMover For Linux Print E-mail

System Requirements

  • Gimp Tool Kit (GTK) version 2.0 or greater
  • SCSI Generic Interface (type 'modprobe -v sg' to start if it's not already running)

Installation Procedure

  1. Untar the package (tar -xvf datademo.tar)
  2. Run install script ('./datamoverinstall.sh')
  3. Create launcher to run 'datamover' binary or type 'datamover' in any terminal window.

Getting Started

DataMover consists of two main components - the Main Dialog (MD), and the I/O Characteristic Dialog (IOD).  The MD is used to scan for and select target(s) for test.

Image

The IOD is used to select the characteristics of the I/O that will be performed on the selected targets. 

Image

License registration can be checked by opening the About dialog from the MD.

Image

The following steps outline the procedures required to import and register a license which is required to activate the advanced features of DataMover.   If using the HASP key, please make sure it's plugged into a USB port and it's LED is on before starting the application.

Step 1:  Import the license by selecting Import from the main menu, a new dialog will appear that allows use to select the license file.

Image

Image

Step 2: Register the license by selecting Register from the main menu, a dialog will be displayed indicating whether the registration was successful or not.

Image

Step 3: Verify license registration was successful by opening the About dialog.

Image  

Main Dialog Operation

  • Clear - Clears MD listing and stops any currently running I/O.
  • Exit - Stops I/O and exits program.
  • License Import - Imports license file.
  • License Register - Registers license file.
  • License UnRegister - Unregisters license file.
  • Help About - Displays About dialog which shows application version and registration status.
  • Scan - Performs device rescan and stops any currently running I/O. 
  • Start - Open the IOD if atleast one target has been selected.
  • Stop - Stops I/O on the currently selected targets.
  • Devices - Number of devices found.
  • Performance - If IOD characteristc performance selected, displays total throughput in MB/s.
  • IOPS - If IOD characteristic performance selected, displays total IOPS.
  • Status - IDLE, RUN or STOP.
  • Extended - Displays summary of error message, see /var/log/messages for detailed error information.
  • Elapsed Time - Total elapsed time since I/O began.
  • Device Column Listings
    • Adapter - Adapter designator that devices are connected to.
    • Device - Vendor and product strings returned in Inquiry response.
    • B-T-L - Bus, target, LUN designation.
    • Type - Device Type returned in Inquiry response.
    • Revision - Revision string returned in Inquiry response.
    • Status - General test status of device.
    • MB/s - Megabytes per second.
    • IOPS - Input/Output per second.
    • Error - Short error description if one occurs.
    • Operation - Synchronous (SYNC) only.
    • Chunk - Transfer size.
    • Buffer - Buffer size.
    • Performance - Enabled/Disabled.
    • Error Checking - Enabled/Disabled.
    • I/O Type - RD, WR or RDWR.
    • Threads - Number of threads per device to spawn.
    • Pattern - Data pattern selected.
    • Special - Special test being performed.

I/O Characteristic Dialog Operation

  • Advanced Options Performance Test - Enables automated performance testing.
  • Advanced Patterns (CRPAT, CJTPAT, CSPAT) -  CSPAT, CRPAT and CJTPAT are modified versions of the test patterns defined by FC-MJSQ Specification on www.t11.org.  They differ in two key areas:  1.)  The pattern is repeated until it fills the buffer size selected and 2.)  Valid fibre channel headers are used vs. using these 24 bytes for data pattern.  As a third party vendor application, it's impossible for DataMover to modify fibre channel frame formatting and if it were done, the frame would not be recognizable to another fibre channel device or fabric.
  • Advanced Patterns Import Data File - Imports custom data file.
  • Advanced Patterns Reset - Resets Pattern to 0xAAh.
  • Status - IDLE, RUN or STOP.
  • Operation - Displays description of an advanced test procedure currently running, PERFTEST, for example.
  • Location - Selected targets.
  • Read - Selects READ direction.
  • Write - Selects WRITE direction.
  • Both - Selects READ and WRITE directions.
  • Threads - Number of threads to run.
  • Pattern - BYTE pattern selectable by the user.  Only alphanumeric values are accepted, 0 - 9 and A - F.  Default value is 0xAAh.
  • Transfer Size - Range 512B to 4MB.  Transfer size must be less than or equal to buffer size.
  • Timeout - Adjustable I/O timeout setting, defaults to 30 seconds.
  • Buffer Size - Range 512B to 4MB.  Buffer size must be greater than or equal to transfer size.
  • Error Checking - If checked, enables data integrity verification.
  • Performance - If checked, enables performance data acquisition and display.

Memory Considerations

DataMover's memory requirements are dependent on the settings chosen by the user; the maximum memory required by DataMover when the maximum thread count = 25, I/O direction = BOTH, performance checkbox is checked, and buffer size = 4MB is 200MB per target.  Based on this calculation, it's easy to see that the user must have a good understanding of the memory configuration in the system under test.  Avoid using virtual memory that's in the system's page file, this will have a negative impact on performance.

Importing Custom Data Patterns

Custom data pattern files can be created using any hex-binary editor.  The Moojit prefers HHD's Free Hex Editor.  Each file must begin with a 16 byte header that consists of a NULL terminated variable length ASCII string up to 16 bytes (this includes the NULL value).  Data files created by The Moojit can be accessed here, please use these as examples when creating your own.

Performance Testing

The objective of the performance test suite is to record 15 measurements for all transfer sizes ranging from 512B to 4MB for half duplex READ, half duplex WRITE and full duplex READ/WRITE operations.  Synchronous I/O is used with a thread count of 1 and 5 for each transfer size and Asynchronous I/O is used with an depth setting of 8 and 16.  The combination that produces the best performance number will be recorded.  After all samples have been taken, the mean and standard deviation from the mean will be calculated for throughput (MB/s) and IOPS (IO/s) for all transfer sizes.  This information will be displayed in the Performance Results dialog that is displayed when the test completes.  The user is given the option to save these results to a comma delimited file which can be later imported into a spreadsheet program such as Excel to generate graphs and data tables.  When the Performance Dialog closes, the original settings selected prior to the Performance test will be restored.  The following pictures demonstrate how to enable the Performance test, what the Performance Results dialog looks like, how to save to a comma delimited file, what the raw imported data looks like in Excel, and what the file report can look like in less than 10 minutes.

Enable the performance test by selecting Performance Test from the IOD menu and selecting OK.

Image

When the test completes, the following dialog will be displayed allowing the user to save the results to a comma delimited file which can be easily imported into excel.

Image

Error Reporting

In the licensed version, DataMover logs all error messages to /var/log/messages.  These messages can also be viewed remotely using a syslog viewer.  To enable your server to forward syslog messages to a specific IP address, follow these steps:

  • Edit /etc/syslog.conf and add the following line:  ' *.*    @192.168.0.2 ' (address of the machine viewing syslog messages).
  • Restart syslog service by typing 'service syslog restart'

Target Testing

Standalone mode - a single client performs I/O on a target with maximum LBA count of N.  Odd numbered threads start at LBA 0 and increment to LBA N, then repeat.  Even numbered threads start at LBA N and decrement to LBA 0, then repeat.  This process is repeated until the test is stopped.  The Standalone test is a one-to-one relationship, only a single DM client may perform I/O to a given target.

Bus Analyzer Signatures

Bus signatures are automatically enabled if Error Checking is turned on.  A specific pattern will be written to the target allowing a bus analyzer such as a SCSI or Fibre Channel analyzer to trigger at the point of failure.  The signature is not written for all errors reported by DataMover - only Data Integrity, READ API and WRITE API errors will trigger the signature pattern.  The patterns for API and Data Integrity failures are as follows:

 ASCII HEX
 "MOOJIT GOT CRC ERROR ON LAST I/O         "4D4F4F4A 49542047 4F542043 52432045 52524F52 204F4E20 4C415354 20492F4F 20202020 20202020
 "MOOJIT GOT API ERROR ON LAST I/O         "4D4F4F4A 49542047 4F542041 50492045 52524F52 204F4E20 4C415354 20492F4F 20202020 20202020

 In most cases, the first DWORD (4D4F4F4A) should be sufficient to trigger the analyzer.

The bus analyzer trigger should be set to look for this pattern at offset 0x00h in an FCP_DATA frame, payload offset 0x00h in an iSCSI WRITE command, or offset 0x00h in an TCP PAYLOAD.