Summary

• Got three Seagate SATA 750GB on Trevi -- NCQ, 5-year warranty (use XFS) and run them on RAID
  
• See Netcell comments here and here -- superfast RAID
  
• The new libata driver, which uses SCSI, is adding support for the Promise PDC20269 card!! This should significantly speed up those drives (as if they needed it)
  
• On 14 June 2002 I reset the system bus speed for all the hard drives in lilo -- except spello's, which found udma6 on their own. Cf. Installation Master.
• Firmware upgrade (Windows only) to the Deathstar drive, currently in Cyberspace (do it on some Win machine?)
• For external harddrives, see firewire and usb
• file:///usr/src/linux/Documentation/ide.txt
• Online help for the 3ware 3DM2 manager

Diagnostics and maintenance

• iostat -E  (show IO disk errors)
  
• e2fsck /dev/hda2
• e2defrag /dev/hdb2 (with a curses gui)
  
• "The new golden rule on ATA is the 40C rule.  If a drive exceeds 40°C, its lifetime is drastically reduced.  Since 7,200rpm is found everywhere -- if it isn't cooled, 40°C is a real problem" (source)
  
• Serial ATA for Linux (Garzik)
• Linux Mafia on SATA
  
• Large hard drives in Linux -- Andries Brower's guide
  
• smartmontools (in Debian, supercedes smartsuite)
• examples
  
• etc/smartd.conf
• /etc/default/smartmontools
• just restart smartmontools
• with SATA drives
• SMART supported in 2.6.15 (ATA passthru)
  
• smart-notifier
• smartctl -A /dev/sda              List all smart attribute options
  
• smartctl -t offline /dev/sda    Run an immediate self-test
  
• smartctl -l selftest /dev/sda   Show self-test log
• smartctl -l selftest /dev/twe0 -d 3ware,5
• smartctl -l selftest /dev/twe0 -d 3ware,6
• smartctl -H /dev/sda              Overall health test results
  
• smartctl -a /dev/hda               See all results
• smartctl -a /dev/twe0 -d 3ware,5
  
• sdparm for SCSI devices, including SATA
  
• Use these commands to tell the video drives to take a break:
  • hdparm -S 242 /dev/hde <== set the timeout to an hour -- add keepsettings:
  • hdparm -k1 /dev/hdh    <== note that you must give the drive name, not the partition name!
  • hdparm -y /dev/hdh  <== put the drive to sleep now
  • hdparm -c1 /dev/hdh  <== enable 32-bit I/O (cf. guide and more guide)
  • hdparm -C /dev/hdh  <== display current state
  • hdparm /dev/hdh <== display basic settings
  • hdparm -i /dev/hdh  <== display full information on the drive
  • hdparm -d0 /dev/hde <== turn off DMA (only for troubled drives; see below)
    
  • hdparm -X08 /dev/hde <== set PIO mode to 0 (only for troubled drives; see below)
    

To make these changes stick, add something like this to /etc/init.d/bootmisc.sh (in SuSE to /etc/init.d/boot.local)

#
# Set hdparm parameters (Frode, 14 June 2003)
#
hdparm -c 1 -S 242 -k 1 /dev/hda
hdparm -c 1 -S 242 -k 1 /dev/hde
hdparm -c 1 -S 242 -k 1 /dev/hdf

• Good information: http://linux.oreilly.com/news/lnxkernel_1100.html
• smartctl -X /dev/hda (install s.m.a.r.t suite from UCSC S.M.A.R.T. Suite for Linux or from Linux-IDE)
• use smartmontools instead (see above)
  
• Recovering Linux files and filesystems (Oct 03)
  
• atapci (user space monitoring program) -- installed on Spello.
• Bad sector check -- you can force the disk to replace the bad sectors by doing a disk-level write:
• dd if=/dev/zero of=/dev/sda1 bs=4k (erases the disk of course)
  
• /proc queries:
  • sudo cat /proc/ide/piix
  • sudo cat /proc/ide/drivers
  • sudo cat /proc/ide/ide1/hdc/identify
  • sudo cat /proc/ide/ide1/hdc/settings
  • sudo cat /proc/ide/ide1/hdc/capacity
  • sudo cat /proc/ide/ide1/hdc/media
  • sudo cat /proc/ide/ide1/hdc/mode

Munin

Install munin and munin-node and edit /etc/munin/plugin-conf.d/munin-node, adding:

[hddtemp_smartctl]
user root
env.drives hda hdb twe0_0 twe0_1 twe0_2 twe0_3
env.type_twe0_0 3ware,0
env.type_twe0_1 3ware,1
env.type_twe0_2 3ware,2
env.type_twe0_3 3ware,3

[smart_*]
user root
group disk

[smart_twe0-0]
env.smartargs -H -i -c -l error -l selftest -l selective -a -d 3ware,0

[smart_twe0-1]
env.smartargs -a -d 3ware,1

[smart_twe0-2]
env.smartargs -a -d 3ware,2

[smart_twe0-3]
env.smartargs -a -d 3ware,3

Create symlinks in /etc/munin/plugins

smart_twe0-0 -> /usr/share/munin/plugins/smart_
smart_twe0-1 -> /usr/share/munin/plugins/smart_
smart_twe0-2 -> /usr/share/munin/plugins/smart_
smart_twe0-3 -> /usr/share/munin/plugins/smart_

Run "/etc/init.d/munin-node restart" and check /var/log/munin/*log to verify the plugins are working -- you'll get
errors if they're not. SATA drives:

[hddtemp_smartctl]
user root
env.drives hda sda
env.type_sda ata

SMART monitoring of SATA drives

Add this to /etc/smartd.conf on Chianti:

First (primary) ATA/IDE hard disk.  Monitor all attributes, enable
# automatic online data collection, automatic Attribute autosave, and
# start a short self-test every day between 2-3am, and a long self test
# Saturdays between 3-4am.
/dev/hda -a -o on -S on -s (S/../.././02|L/../../6/03)

# Secondary SATA drives -- ditto
# Since the disk is addressed via a SCSI device, but the
# underlying disk type is actually ATA, you need -d ata
/dev/sda -d ata -a -o on -S on -s (S/../.././02|L/../../6/03)
/dev/sdb -d ata -a -o on -S on -s (S/../.././02|L/../../6/03)

Add this to /etc/smartd.conf on Clitunno:

# Monitor 6 ATA disks connected to a 3ware 6/7/8000 controller which uses
# the 3w-xxxx driver. Start long self-tests Sundays between 1-2, 2-3, 3-4,
# and 4-5 am. Each card is a character device, /dev/twe0, /dev/twe1, ...
# Note that the drives may skip a number, as in my setup (8 Oct 2006)
/dev/twe0 -d 3ware,0 -a -s L/../../7/01
/dev/twe0 -d 3ware,1 -a -s L/../../7/02
/dev/twe0 -d 3ware,3 -a -s L/../../7/04
/dev/twe0 -d 3ware,5 -a -s L/../../7/05
/dev/twe0 -d 3ware,6 -a -s L/../../7/06
/dev/twe0 -d 3ware,7 -a -s L/../../7/07

Or use this sort of line:

/dev/twe0 -d 3ware,0 -H -C 0 -U 0 -m fsteen@ucla.edu -o on -S on -s L/../../7/00

Cf. installation history October 2006. It may be worth having a hot spare -- cf. 3Ware 9000 instructions.

3Ware RAID

• 3ware 3DM2 manager
• Online help for the 3DM2 manager
• 3Ware 9000 instructions and examples
  
• To switch out a drive and rebuild an array
• unmount the array (umount /ssb)
  
• open the 3DM2 manager
• select Management | Maintenance
• check the affected unit and select Remove drive
• remove the drive and clean the enclosure
• insert the new drive
• select Rescan Controller
• it's possible that at this point you should instruct the system to create a new unit with the currently good drives and the new drive -- cf. the notes from 3ware support at the bottom ofGentoo's RAID rebuild instructions
  
• select Rebuild Unit
• You can also use the commands in the 3ware-cli-binary
• see Gentoo's RAID rebuild instructions
  

Superblock records

Don't delete the lost+found directory -- it's used by fsck.

Cyberspace:/mnt/vc

I ran mke2fs -j /dev/hdb1 to put ext3 on cyberspace's vc drive and I got Superblock backups stored on blocks:

32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872

Keep these numbers -- if you need to recover a corrupt file system, this is where you indicate the superblock is.

Note that these are likely identical on all the big drives, even though the sizes differ.

Encrypted directories

You can use the device mapper module to create on-the-fly encrypted directories, using these (added to config-2.6.9-ac6-9 on clitunno):

 Device Drivers -> Multi-Device Support (RAID and LVM) -> Device Mapper Support
 Device Drivers -> Multi-Device Support (RAID and LVM) -> Crypt Target Support
 Device Drivers -> Block Devices -> Loopback Device Support
 Cryptographic Options -> AES (you could use or add other types)

For instructions, see http://www.linuxjournal.com/article/8599; cf. Security.

To do (7 Sep 02)

• IBM drive: A REPORT ON the excellent Storage Review Web site reports IBM as warning that its 120GXP hard drives shouldn't be powered for more than eight hours a day. If this is so, you should give the new drive to Tom. "A three-year warranty backs the drive." http://www.storagereview.com/ -- in fact, IBM recommends just 333 power-on hours per month in the drive's spec sheet (pdf) and the service person said 8 hours a day. Let Tom borrow it, or just return it; it really won't be useful for you.
  
• Consider doing software IDE RAID
• Possibly get a BIOS update for the Promise controller (done in Windows?)
• Consider this report on a stable system -- no kernel options, updated BIOS :

Installation history

October 2006 update

SMART monitoring now appears to be working for SATA drives too -- this is on Chianti:

# smartctl -d ata -a /dev/sdb
smartctl version 5.36 [x86_64-unknown-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF INFORMATION SECTION ===
Device Model:     ST3300622AS
Serial Number:    4NF22YGR
Firmware Version: 3.AAH
User Capacity:    300,069,052,416 bytes
Device is:        Not in smartctl database [for details use: -P showall]
ATA Version is:   7
ATA Standard is:  Exact ATA specification draft version not indicated
Local Time is:    Fri Oct  6 00:41:40 2006 PDT
SMART support is: Available - device has SMART capability.
SMART support is: Enabled

=== START OF READ SMART DATA SECTION ===
SMART overall-health self-assessment test result: PASSED

General SMART Values:
Offline data collection status:  (0x82) Offline data collection activity
                                        was completed without error.
                                        Auto Offline Data Collection: Enabled.
Self-test execution status:      (   0) The previous self-test routine completed
                                        without error or no self-test has ever
                                        been run.
Total time to complete Offline
data collection:                 ( 430) seconds.
Offline data collection
capabilities:                    (0x5b) SMART execute Offline immediate.
                                        Auto Offline data collection on/off support.
                                        Suspend Offline collection upon new
                                        command.
                                        Offline surface scan supported.
                                        Self-test supported.
                                        No Conveyance Self-test supported.
                                        Selective Self-test supported.
SMART capabilities:            (0x0003) Saves SMART data before entering
                                        power-saving mode.
                                        Supports SMART auto save timer.
Error logging capability:        (0x01) Error logging supported.
                                        General Purpose Logging supported.
Short self-test routine
recommended polling time:        (   1) minutes.
Extended self-test routine
recommended polling time:        ( 105) minutes.

SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x000f   108   097   006    Pre-fail  Always       -       178204415
  3 Spin_Up_Time            0x0003   087   085   000    Pre-fail  Always       -       0
  4 Start_Stop_Count        0x0032   100   100   020    Old_age   Always       -       16
  5 Reallocated_Sector_Ct   0x0033   100   100   036    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x000f   078   060   030    Pre-fail  Always       -       61883049
  9 Power_On_Hours          0x0032   099   099   000    Old_age   Always       -       1276
 10 Spin_Retry_Count        0x0013   100   100   097    Pre-fail  Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   020    Old_age   Always       -       20
187 Unknown_Attribute       0x0032   100   100   000    Old_age   Always       -       0
189 Unknown_Attribute       0x003a   100   100   000    Old_age   Always       -       0
190 Unknown_Attribute       0x0022   078   061   045    Old_age   Always       -       370540566
194 Temperature_Celsius     0x0022   022   040   000    Old_age   Always       -       22 (Lifetime Min/Max 0/22)
195 Hardware_ECC_Recovered  0x001a   058   054   000    Old_age   Always       -       106684178
197 Current_Pending_Sector  0x0012   100   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0010   100   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x003e   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x0000   100   253   000    Old_age   Offline      -       0
202 TA_Increase_Count       0x0032   100   253   000    Old_age   Always       -       0

SMART Error Log Version: 1
No Errors Logged

SMART Self-test log structure revision number 1

SMART Selective self-test log data structure revision number 1
 SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
    1        0        0  Not_testing
    2        0        0  Not_testing
    3        0        0  Not_testing
    4        0        0  Not_testing
    5        0        0  Not_testing
Selective self-test flags (0x0):
  After scanning selected spans, do NOT read-scan remainder of disk.
If Selective self-test is pending on power-up, resume after 0 minute delay.

I edited /etc/smartd.conf as follows:

# First (primary) ATA/IDE hard disk.  Monitor all attributes, enable
# automatic online data collection, automatic Attribute autosave, and
# start a short self-test every day between 2-3am, and a long self test
# Saturdays between 3-4am.
/dev/hda -a -o on -S on -s (S/../.././02|L/../../6/03)

# Secondary SATA drives -- ditto
# Since the disk is addressed via a SCSI device, but the
# underlying disk type is actually ATA, you need -d ata
/dev/sda -d ata -a -o on -S on -s (S/../.././02|L/../../6/03)
/dev/sdb -d ata -a -o on -S on -s (S/../.././02|L/../../6/03)

Restart the daemon:

/etc/init.d/smartmontools restart

dmesg says,

ata4.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2 frozen
ata4.00: tag 0 cmd 0xb0 Emask 0x2 stat 0x50 err 0x0 (HSM violation)
ata4: soft resetting port
ata4: SATA link up 1.5 Gbps (SStatus 113 SControl 300)
ata4.00: configured for UDMA/133
ata4: EH complete
SCSI device sdb: 586072368 512-byte hdwr sectors (300069 MB)
sdb: Write Protect is off
sdb: Mode Sense: 00 3a 00 00
SCSI device sdb: drive cache: write back

At least this is working. However, this discussion suggests a patch is needed for smarmontools 5.36 for the nv_sata chipset, so disable for now.

You can tweak this later, maybe get the system to send e-mail warnings. So for the first time this is working for SATA.

On Clitunno, I get this after some tweaking:

smartd version 5.36 [x86_64-unknown-linux-gnu] Copyright (C) 2002-6 Bruce Allen
Home page is http://smartmontools.sourceforge.net/
Opened configuration file /etc/smartd.conf
Configuration file /etc/smartd.conf parsed.
Device: /dev/hda, opened
Device: /dev/hda, found in smartd database.
Device: /dev/hda, enabled SMART Attribute Autosave.
Device: /dev/hda, enabled SMART Automatic Offline Testing.
Device: /dev/hda, is SMART capable. Adding to "monitor" list.
Device: /dev/hdb, opened
Device: /dev/hdb, not found in smartd database.
Device: /dev/hdb, enabled SMART Attribute Autosave.
Device: /dev/hdb, enabled SMART Automatic Offline Testing.
Device: /dev/hdb, is SMART capable. Adding to "monitor" list.
Device: /dev/twe0 [3ware_disk_00], opened
Device: /dev/twe0 [3ware_disk_00], found in smartd database.
Device: /dev/twe0 [3ware_disk_00], is SMART capable. Adding to "monitor" list.
Device: /dev/twe0 [3ware_disk_01], opened
Device: /dev/twe0 [3ware_disk_01], found in smartd database.
Device: /dev/twe0 [3ware_disk_01], is SMART capable. Adding to "monitor" list.
Device: /dev/twe0 [3ware_disk_03], opened
Device: /dev/twe0 [3ware_disk_03], found in smartd database.
Device: /dev/twe0 [3ware_disk_03], is SMART capable. Adding to "monitor" list.
Device: /dev/twe0 [3ware_disk_05], opened
Device: /dev/twe0 [3ware_disk_05], found in smartd database.
Device: /dev/twe0 [3ware_disk_05], is SMART capable. Adding to "monitor" list.
Device: /dev/twe0 [3ware_disk_06], opened
Device: /dev/twe0 [3ware_disk_06], found in smartd database.
Device: /dev/twe0 [3ware_disk_06], is SMART capable. Adding to "monitor" list.
Device: /dev/twe0 [3ware_disk_07], opened
Device: /dev/twe0 [3ware_disk_07], found in smartd database.
Device: /dev/twe0 [3ware_disk_07], is SMART capable. Adding to "monitor" list.
Monitoring 8 ATA and 0 SCSI devices
Device: /dev/hdb, 1 Currently unreadable (pending) sectors
Device: /dev/twe0 [3ware_disk_01], 1 Currently unreadable (pending) sectors
Device: /dev/twe0 [3ware_disk_05], 4 Currently unreadable (pending) sectors
Device: /dev/twe0 [3ware_disk_05], 2 Offline uncorrectable sectors
smartd has fork()ed into background mode. New PID=4340.
file /var/run/smartd.pid written containing PID 4340

June 2005 update

Installed smartmontools 5.32-3 on chianti, no changes to config files. It finds what it needs on /dev/hda, but not on the sata drives -- though this may be in the works in a libata driver update, possibly already for 2.6.12. I get

smartd[12689]: Device: /dev/sdc, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdc' to turn on SMART features
Jun 24 21:54:02 localhost smartd[12689]: Unable to register SCSI device /dev/sdc at line 23 of file /etc/smartd.conf

April 2005 update

On the new machine Chianti, there is sata_sil and sata_nv. The Seagate drives have a problem on sata_sil and Timing buffered disk reads is 20MB/s. On the sata_nv drives it's 60MB/s.  The Seagate drives are blacklisted and hobbled under the sata_sil driver; this may be fixable with a firmware update for the drive and/or the Silicon Image BIOS.

As for the drive firmware, dmesg | grep Rev: shows I have revision 3.01 for drive model ST3200822AS.

March 2004 update

The old /vs drive, since divided into /vsa, /vsb, /vsc, and /vsd, is misbehaving after getting lots of capuchin data. I see dma error messages of this type

gubbio kernel: hde: read_intr: error=0x40 { UncorrectableError }, LBAsect=1312083, high=0, low=1312083, sector=1312008

and wonder if I can turn dma off, so as to make less demands on the drive. Here's what SuSE says:

A good first step in reducing IDE problems, is to disable DMA. This can be done with the command hdparm -d0 /dev/<device>. For example, if your CDROM is set to master on the secondary controller, use the command

    hdparm -d0 /dev/hdc

If this is not sufficient, you may want to reduce the transfer rates. hdparm -XNN /dev/device will set the transfer mode. Here is what values are valid for NN:

    08 - PIO mode 0
    09 - PIO mode 1
    10 - PIO mode 2
    11 - PIO mode 3
    12 - PIO mode 4
    13 - PIO mode 5

Sometimes it is best to get a bit paranoid and set the device to the absolute basics, of course replacing /dev/hdc with your IDE device in question.

    hdparm -d0 /dev/hdc
    hdparm -X08 /dev/hdc

This of course will slow the device down somewhat, but if it stops the symptoms, you've located the problem.

root@gubbio:~# hdparm -d0 /dev/hde

/dev/hde:
 setting using_dma to 0 (off)
 using_dma    =  0 (off)

Mar  1 17:02:11 gubbio kernel: blk: queue c0326f68, I/O limit 4095Mb (mask 0xffffffff)
Mar  1 17:02:11 gubbio kernel: hde: DMA disabled

root@gubbio:~# hdparm -X08 /dev/hde

/dev/hde:
 setting xfermode to 8 (PIO flow control mode0)
root@gubbio:~#

gubbio kernel: end_request: I/O error, dev 21:01 (hde), sector 1312008
gubbio kernel: hde: irq timeout: status=0xd0 { Busy }
gubbio kernel:
gubbio kernel: PDC202XX: Primary channel reset.
gubbio kernel: ide2: reset: success
gubbio kernel: hde: irq timeout: status=0xd0 { Busy }
gubbio kernel:
gubbio kernel: PDC202XX: Primary channel reset.
gubbio kernel: ide2: reset: success

gubbio kernel: hde: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }
gubbio kernel: hde: read_intr: error=0x40 { UncorrectableError }, LBAsect=9440682, high=0, low=9440682, sector=9440616

rsync -v * /vc/video/Perry/1997/Tape-01/xvid-2

read errors mapping "/mnt/vsc/Perry/1997-C/Tape-08/xvid-2/1997-T08-016.xvid": (5) Input/output error

hde: read_intr: error=0x40 { UncorrectableError }, LBAsect=401612131, high=23, low=15736163, sector=85195888
end_request: I/O error, dev 21:03 (hde), sector 85195888
hde: read_intr: status=0x59 { DriveReady SeekComplete DataRequest Error }

hde: status error: status=0x58 { DriveReady SeekComplete DataRequest }
hde: drive not ready for command

ac-tex damaged at 43 8
Error at MB: 411
marker does not match f_code
concealing errors
header damaged

February 2004 update

The large drives are quite vulnerable to failures from overheating. Give them breaks, and use this command to spin them down (standby mode):

        hdparm -y /dev/hde

Verify standby with

        hdparm -C /dev/hde

The drives will spin down on their own after an hour of no use, because of the command in /etc/init.d/bootmisc.sh:

        hdparm -c 1 -S 242 -k 1 /dev/hde

Use hdparm -y to give the drive a rest between copying sessions, and limit each copy session to around 30GB or less.

In the October 2003, one 250GB drive was destroyed after copying more than 200GB straight to another drive; it had to be returned (they have three year warranties). No data was lost, as the drive was fully copied by the time it broke down.

In February 2004, a 320GB had lots of errors after copying 300GB to the new storage server; this was done in sections. The data was successfully copied, the file system had corruptions but the data may not have been damaged, and the drive was not destroyed. I reformatted the drive into four partitions and it's working fine as gubbio:/vsa /vsb/vsc and /vsd. Use it as a storage drive and try not to move stuff much.

October 2003 update

In early October I installed two new 320GB drives on gubbio, vf and vs. I moved about 230GB straight from spello:/vh to gubbio:/vs, and at the end of it /vh crashed. The warranty on the two 250GB drives on spello, /ve and /vh, runs for 3 years until the end of 2005.

I went to Maxtor's web site and got an RMA. The new drive is A80317DE.
To avoid massive losses, you need to backup all of the compressed files to a different drive.

November 2002 update

On 18 November 2002, I installed the two new 250GB Maxtor drives on spello, attached to a new ATA controller card that came in the same box. The dmesg reads,

Uniform Multi-Platform E-IDE driver Revision: 6.31
ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
PIIX4: IDE controller on PCI bus 00 dev 39
PIIX4: chipset revision 1
PIIX4: not 100% native mode: will probe irqs later
ide0: BM-DMA at 0xffa0-0xffa7, BIOS settings: hda:DMA, hdb:pio
ide1: BM-DMA at 0xffa8-0xffaf, BIOS settings: hdc:DMA, hdd:pio
PDC20269: IDE controller on PCI bus 02 dev 50
PCI: Found IRQ 11 for device 02:0a.0
PCI: Sharing IRQ 11 with 00:07.2
PDC20269: chipset revision 2
PDC20269: not 100% native mode: will probe irqs later
PDC20269: ROM enabled at 0xfb000000
PDC20269: (U)DMA Burst Bit ENABLED Primary PCI Mode Secondary MASTER Mode.
ide2: BM-DMA at 0xdcc0-0xdcc7, BIOS settings: hde:pio, hdf:pio
ide3: BM-DMA at 0xdcc8-0xdccf, BIOS settings: hdg:pio, hdh:pio
hda: Maxtor 91024D4, ATA DISK drive
hdc: TOSHIBA DVD-ROM SD-M1212, ATAPI CD/DVD-ROM drive
hdd: IOMEGA ZIP 250 ATAPI, ATAPI FLOPPY drive
hde: Maxtor 4A250J0, ATA DISK drive
hdh: Maxtor 4A250J0, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
ide2 at 0xdcf8-0xdcff,0xdcf2 on irq 11
ide3 at 0xdce0-0xdce7,0xdcda on irq 11
hda: 19999728 sectors (10240 MB) w/512KiB Cache, CHS=1244/255/63, UDMA(33)
hde: 490234752 sectors (251000 MB) w/2048KiB Cache, CHS=30515/255/63, UDMA(133)
hdh: 490234752 sectors (251000 MB) w/2048KiB Cache, CHS=30515/255/63, UDMA(133)

Incidentally, I'm writing this straight into Dreamweaver under Win98 in VMware -- there's no appreciable delay! So note that:

• the controller card is a Promise PDC20269!! (which means I have a spare of those for cyberspace)
• the UDMA 133 was detected on startup, with no information added in Lilo!!

This was an extremely unproblematic 500GB harddrive install. I left the jumpers on cable-select, which Linux seems to handle fine. I created a single Linux 83 partition on both, using cfdisk /dev/hde and /dev/hdh, and then put down ext3 on both, with this result:

# mke2fs -j /dev/hdh1
mke2fs 1.27 (8-Mar-2002)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
30654464 inodes, 61277926 blocks
3063896 blocks (5.00%) reserved for the super user
First data block=0
1871 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 21 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

Now, the fact that I can cut and past directly from konsole into Dreamweaver -- first time ever! -- is a huge timesaver. This was clearly the way to go; eventually, it may be that wine will do the trick, but there's no reason to wait for it. Wow. I now have a terabyte of storage. I added this to /etc/fstab, modeled on gubbio:

/dev/hde1 /mnt/ve ext3 user,rw 0 0
/dev/hdh1 /mnt/vh ext3 user,rw 0 0

I made the mount points and put in symlinks; everything worked the first time! I then added the following hdparm settings:

hdparm -c1 /dev/hde <== enable 32-bit I/O
hdparm -S242 /dev/hde <== standby mode after an hour
hdparm -k1 /dev/hde  <== keep settings through a warm reboot

I didn't see that enabling 32-bit I/O made any difference, but it might for copying files, and it might in later kernels. 2.5 is supposed to speed up 1/O. See guide and more guide. I also set hdparm -u1 /dev/hde --this allows the system to attend to other tasks while waiting for disk access. I didn't set this on /dev/hdh. I'll now run a trial copy from gubbio -- the previous trial went up to 10 MB/sec and wavered down to 8.2 or so.

October 2002 update

The Promise Controller card, Ultra 133TX2 with the PDC200269 chip, came with Bios version 2.20.0050.10.

On 11 September 2002 I discovered the upgrade page:
http://support.promise.com/Support/All.asp?product=ultra133TX2

The latest BIOS version is 2.20.0.12 -- apparently a bit more recent, from 2001-11-28.

The ftp directory is here: ftp://ftp.promise.com/Controllers/IDE/Ultra133TX2/

The file is an exe file for Windows. There's also a new driver. I download both and boot into Windows -- the first time in months -- and (after not loading memory managers) successfully flash the card from DOS.

September 2002

On 20 September, I got smartsuite 2.1 from http://www.linux-ide.org/smart/smartsuite-2.1.tar.gz and untarred it in
/usr/src/packages/SOURCES. I did make (no configuration) and make install; it failed here:

cp ./smartd.initd /etc/rc.d/init.d/smartd
cp: cannot create regular file `/etc/rc.d/init.d/smartd': No such file or
directory
make: *** [install] Error 1

SuSE 7.3 skips the rc.d directory and uses /etc/init.d -- I put the file there and this seemed fine. The instructions are in man smartctl. I ran /usr/local/sbin/smartctl -i /dev/hde on the IBM Deskstart and got

Device: IC35L120AVVA07-0 Supports ATA Version 5
Drive supports S.M.A.R.T. and is disabled
Use option -e to enable

I issued /usr/local/sbin/smartctl -e /dev/hde and enabled the feature. I then issued /usr/local/sbin/smartctl -c /dev/hde to check for warranty failures.

gubbio:/etc/init.d # /usr/local/sbin/smartctl -c /dev/hde
Device: IC35L120AVVA07-0 Supports ATA Version 5
Drive supports S.M.A.R.T. and is enabled
Check S.M.A.R.T. Passed.

I then ran the short self test (unmounted only):

/usr/local/sbin/smartctl -s /dev/hde

Smart Offline failed: Input/output error
Smartctl: Smart Short Self Test Failed

I ran S.M.A.R.T. Extended Self Test in Captive Mode.

gubbio:/etc/init.d # /usr/local/sbin/smartctl -x /dev/hde
Smart Offline failed: Input/output error
S.M.A.R.T. Extendend Self Test Failed

I then put the ext3 file system on the drive:

gubbio:/etc/init.d # mke2fs -j /dev/hde
mke2fs 1.27 (8-Mar-2002)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
15089664 inodes, 30156840 blocks
1507842 blocks (5.00%) reserved for the super user
First data block=0
921 block groups
32768 blocks per group, 32768 fragments per group
16384 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632,
2654208,
4096000, 7962624, 11239424, 20480000, 23887872

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 25 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to override.

And reran the test:

gubbio:/etc/init.d # /usr/local/sbin/smartctl -x /dev/hde
Smart Offline failed: Input/output error
S.M.A.R.T. Extendend Self Test Failed

The www.mainpc.com people who sold me the drive have a policy of mandatory 15% restocking on harddrives and only repair or replacement, so I guess I'm still stuck with it -- at least I want to see if I can upgrade the firmware or test it with IBM's own tools.

I found this: http://techupdate.zdnet.co.uk/story/0,,t481-s2119615-p3,00.html

Despite recent bad (and sensationalistic) press due to problems with its 75GXP model, I still have a very favorable opinion of IBM's Deskstar IDE drives. I find them to be reliable, and they are certainly fast for 7,200-rpm ATA 100 drives. IBM also offers a free download of the IBM Feature Tool utility, which you can use to change the drive's Automatic Acoustic Management settings to the lowest acoustic emanation setting (Quiet Seek Mode),making them quiet enough to use in low-noise PCs.

I got the Download Drive Fitness Test at http://www.storage.ibm.com/hdd/support/download.htm with instructions How to create bootable DFT diskette using Linux at http://www.storage.ibm.com/hdd/support/dft-linux-readme.htm

The Drive Fitness Test image file (dft-vxxximg.bin) includes IBM DOS2000 and the IBM Drive Fitness Test. Running the Linux Disk Dump (dd) utility creates an IBM DOS 2000 bootable diskette that contains the Drive Fitness Test Utility. A formatted 1.44 MB diskette is required to create the diskette.

To create a bootable DFT diskette under Linux:

Download the bootable version of DFT (dft-vxxximg.bin) into a known directory on your PC.
Insert a formatted 1.44 MB diskette into your disk drive.
Make sure that the floppy disk is unmounted.
Change directory to where you saved the DFT image.

Run the Disk Dump (dd) utility with the following parameters:

dd if=dft-vxxximg.bin of=/dev/fd0 bs=8k

Note: /dev/fd0 is the first diskette drive. Use /dev/fd1 if you would like to use the second diskette drive to create the DFT diskette.

The bootable DFT diskette will now be created, Note this could take a couple of minutes.

See the users guide for information on how to run the Drive Fitness Test.

I put the user guide on /vc and got the BIN file. I successfully created the floppy with

dd if=DFTV310_IMG.BIN of=/dev/fd0 bs=8k

gubbio:~ # dd if=DFTV310_IMG.BIN of=/dev/fd0 bs=8k
180+0 records in
180+0 records out

In the meantime, Opera got stuck and I had to kill it. Ready for reboot.

Putting a drive to sleep

Summary

After installing IBM's DeskStar, popularly known as DeathStar, I looked for ways to spin it down and perhaps put it to sleep. I found out that this is indeed possible with the command hdparm -Y /dev/hdd. It's not good for the drive to be woken up frequently from this sleep mode, but it serves precisely my purposes. This drive should be used for stuff that is accessed rarely -- a kind of archival drive.

Configuration history

man hdparm:

-C Check the current IDE power mode status, which will always be one of unknown (drive does not support this command), active/idle (normal operation), standby (low power mode, drive has spun down), or sleeping (lowest power mode, drive is completely shut down). The -S, -y, -Y, and -Z flags can be used to manipulate the IDE power modes.

cyberspace:~ # hdparm -C /dev/hdd

/dev/hdd:
drive state is: active/idle

So it hasn't spun down on its own!

man hdparm again:

-S Set the standby (spindown) timeout for the drive. This value is used by the drive to determine how long to wait (with no disk activity) before turning off the spindle motor to save power. Under such circumstances, the drive may take as long as 30 seconds to respond to a subsequent disk access, though most drives are much quicker. The encoding of the timeout value is somewhat peculiar. A value of zero means "off". Values from 1 to 240 specify multiples of 5 seconds, for timeouts from 5 seconds to 20 minutes. Values from 241 to 251 specify from 1 to 11 units of 30 minutes, for timeouts from 30 minutes to 5.5 hours. A value of 252 signifies a timeout of 21 minutes, 253 sets a vendor-defined timeout, and 255 is interpreted as 21 minutes plus 15 seconds.

Just to test this, I try

hdparm -S 1 /dev/hdd

I get:

/dev/hdd:
setting standby to 1 (5 seconds)

From hdparm -C /dev/hdd I now get:

/dev/hdd:
drive state is: standby

Of course, an hour is a more reasonable permanent value -- that would be hdparm -S 242 /dev/hdd. This could also be applied to /vm and /vc!

Now, that puts it into standby -- but what about putting it to sleep, as it were?

man hdparm again:

-y Force an IDE drive to immediately enter the low power consumption standby mode, usually causing it to spin down. The current power mode status can be checked using the -C flag.

-Y Force an IDE drive to immediately enter the lowest power consumption sleep mode, causing it to shut down completely. A hard or soft reset is required before the drive can be accessed again (the Linux IDE driver will automatically handle issuing a reset if/when needed). The current power mode status can be checked using the -C flag.

NOTE: hdparm -Y is not working! Known bug.

cyberspace:~ # hdparm /dev/hdd

/dev/hdd:
multcount = 16 (on)
I/O support = 0 (default 16-bit)
unmaskirq = 0 (off)
using_dma = 1 (on)
keepsettings = 0 (off)
nowerr = 0 (off)
readonly = 0 (off)
readahead = 8 (on)
geometry = 255296/15/63, sectors = 241254720, start = 0

The keepsettings have to do with the soft reset.

A drive in standby mode will automatically awake whenever a disk operation is requested. Conversely, a drive in sleeping mode will awake only by a hard or soft reset of the IDE controller. However, the EIDE driver of the Linux kernel will automatically issue a reset of the controller when necessary. Be warned, however, that your drive may take as long as 30 seconds to wake up both from the standby mode and from the sleeping mode. Moreover, it'd be better to avoid too many spin downs and spin ups of your disk, so you may want to set the standby timer to a reasonably long interval of time.

So this is the story. You can put the drive to sleep, but it's a strain to weke it up again -- so this should only be done for /vd. Maybe I should put some more stuff on the drive first? I only have Spencer and Yulia on /vm, and Carrie and Jirina on /vc. I guess I'll leave it like this and put /vd to sleep.

I added "noauto" to the vd line in gubbio:/etc/fstab and unmounted it from gubbio. I added "noauto" also to cyberspace's fstab, but it wouldn't unmount -- claims to be busy. Here's the whole story:

cyberspace:/ # fuser -m /mnt/vd
cyberspace:/ # umount /mnt/vd
umount: /mnt/vd: device is busy
cyberspace:/ # fuser -k /mnt/vd
/mnt/vd:
No process references; use -v for the complete list
No automatic removal. Please use umount /mnt/vd
cyberspace:/ # umount /mnt/vd
umount: /mnt/vd: device is busy

So it claims to be busy even though the drive is on standby. I issue
hdparm -Y /dev/hdd and it does seem to have some effect -- but when I ask
hdparm -C /dev/hdd it takes a while and then still says standby. I may
have to reboot to get this right, and I won't do that until I'm there. So
this resolves the issue for the moment; I've spun it down and figured out
how to put it to sleep.

Some additional information:

A Mandrake help screen says, "Note that the hard drive 'sleep' function puts a great deal of mechanical strain on the hard drive when 'waking it up'. Furthermore, the Linux drive access cache prevents a drive with mounted partitions on it from going into 'sleep' mode. The 'noflushd' package from your Mandrake Linux CD enables you to work around that. You should make sure to its man page before using it, though. Once you've found a suitable setting, execute the 'hdparm' command again, adding the 'k' option. This will preserve your settings beyond the current session."

http://www.mandrakeuser.org/docs/hardware/hide2.html

So this is what I likely need: noflushd. It's not on cyberspace. Here's info (http://www.loh-fi.de/howto/acer_en.htm):

noflushd

The noflushd watches the activity of drives and spins them down after a certain time without access. The timeout can be defined by the user. At the same time it watches the kernel update daemon (kupdated), which periodically flushes all buffers to disk. So noflushd prevents kupdated to write the buffers to disk, if the disk is spun down and there are no other disk accesses.

Using the noflushd is quite easy, because it is already included in SuSE 7.3. Configuration is done best with SuSEconfig, which provides three keys:

START_NOFLUSHD = "yes"
NOFLUSHD_DISKS = "/dev/hda"
NOFLUSHD_TIMEOUT = "2"

The first key causes . With the keys as set above, noflushd is started while booting and monitors the activity on / dev/hda and spins it down after 2 minutes with no accesses.

Installing noflushd makes no sense if there are no additional steps taken to reduce hard disk activity. One thing to do is to set the noatime attribute for mount. Usually Linux logs the most recent access time for all files, which leads to many short-time disk accesses. On the other hand logging that is not essential, neither for Linux nor for the user. Setting the attribute noatime for a partition in /etc/fstab prevents Linux from logging the access time of files:

/dev/hda6 / reiserfs noatime 0 0

For /etc/cryptotab it is just the same:

/dev/loop0 /dev/hda8 /home reiserfs twofish noatime

That the buffers are written regularly even if there are no other disk accesses, I recommend to set up a cron job. This is done by creating the file /etc/cron.d/sync which contains the following line:

*/6 7-23 * * * nobody /bin/sync

After that the buffers are written between 7:00 and 23:00 every 6 minutes. The buffers should also be flushed when changing into standby mode. Nothing must be done for it, SuSE provides the necessary lines in /usr/sbin/ apmd_proxy.

Not my situation of course -- this is for a laptop. All this is assuming that the drive is the primary drive -- which in my case it's not. I can safely put all the video drives to sleep.

Make sure you set the following:

cyberspace: hdparm -S 241 /dev/hdd -- half-our timeout for /vd
cyberspace: hdparm -S 242 /dev/hdb -- one hour timeout for /vc
gubbio : hdparm -S 242 /dev/hde -- one hour timeout for /vm

Try this for a while and see how you like it. So I did this on gubbio and it worked fine -- you can also put the drive in
standby by hdparm -y /dev/hde.

December 2001

I first patched a downloaded kernel 2.4.16, using the patch from http://www.linuxdiskcert.org/ to enable detection and use of the 160GB drive, and called it 2.4.16p. Details:

• Configuration help: http://www.uwsg.iu.edu/hypermail/linux/kernel/0201.0/0286.html
  
• IBM patch and ATA-knowledgeable SuSE developer
  
• Andre Henrick's 8 Jan 02 update (the patch maker)
  
• Patch location (23 Feb 02 for 2.4.16, 17, and 18, as well as 2.5.3)
• Patch location (11 Apr 02 for 2.4.18 and 2.5.5 and 2.5.7)
  

This is how I applied the patch successfully:

• Uncompress the 2.4.16 kernel to /usr/src/common
  
• It automatically untars into the subdirectory linux
  
• Rename the directory linux to linux-2.4.16
  
• Copy the compressed patch into linux-2.4.16
  
• Enter the directory linux-2.4.16
  
• type bzip2 -dc *patch.bz2 | patch -p0

In other words, place the patch in the directory named after the kernel version (not linux, which is the default on untarring, but linux-2.4.16). Then from that common directory, type

bzip2 -dc *patch.bz2 | patch -p0

This worked. Linux is absurd when you don't know it. Note that I discovered how to do it by using two important files:

1. The patch itself
   -- it clearly assumed the directory was linux-2.4.16
   
2. The untarred file README in the linux kernel
   -- it gave the patch syntax

This is a good rule of thumb in Linux: read the accompanying files, don't assume that there are generally valid standards. The typical situation is some minor but absolutely crucial variation from the standard.

For the new harddrive, the important thing is to discover the support for the PDC20269 IDE controller chipset on the new PCI bus. I used the grep and then the search function / in vi to discover that the parameter is called CONFIG_BLK_DEV_PDC202XX. I think I got it on the second try.

Alan Cox writes on 9 March 2002 (see also press release -- sells for $500 says Pricewatch):

For anyone who hasn't noticed yet 3ware have put out firmware updates to their IDE raid cards to support LBA48 (thats the new IDE standard for very big disks) and you can now run hardware raid with 8 160Gb disks and attach over a terabyte of IDE disk to a single PCI slot.

Configuration

System bus speed

After booting, I noted that the bus speed of the new drive is set to 33MHZ instead of 133:

Revision: 6.31 ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx

On 14 June 2002, I came across some news on that topic. The 2.4.16m5 build (and all my previous 2.4.16 builds on gubbio) defines the ide bus speed to 33 instead of 133 (or 66, or 100). See kernel boot log. What went wrong?

It's possible you need to get a more recent BIOS from Promise -- someone wrote, "Promise BIOS version is 2.20.0.12," but what is mine? Here's an answer: "grab the latest hdparm, and the latest kernel (ensuring it supports udma mode 6, and compile both." Here's an attempt to fix the speed with hdparm.

But this may not be the problem. Here's another answer: "This is a bug in the driver [2.4.19 p7]. You need to boot with 'ide0=ata66' (0 being the channel number) to be able to switch to udma3+ modes." Source.

To get the channel numbers, see the boot log:

ide0: BM-DMA at 0xf000-0xf007, BIOS settings: hda:DMA, hdb:DMA
ide1: BM-DMA at 0xf008-0xf00f, BIOS settings: hdc:DMA, hdd:DMA
ide2: BM-DMA at 0xbc00-0xbc07, BIOS settings: hde:pio, hdf:pio
ide3: BM-DMA at 0xbc08-0xbc0f, BIOS settings: hdg:pio, hdh:pio

Or you could do this:

l /proc/ide/       --see which hard drives are on which controllers

Note that each channel has two slots for drives. hdb is the CDROM, hdd the CDRW -- they get taken over by SCSI. Of course /dev/hdf doesn't exist, and ide3 is not used -- you basically have room for three more devices on your IDE cards!

Since (at this point I believe that) all my drives should work at a speed faster than 33, I add these parameters to lilo:

ide0=ata66 ide1=ata66 ide2=ata66

Or possibly you could do,

ide0=ide1=ide2=ata66

When I just do the first one -- I try this incrementally, to be on the safe side -- I get the following:

ide: Assuming 33MHz system bus speed for PIO modes; override with idebus=xx
ide0: BM-DMA at 0xf000-0xf007, BIOS settings: hda:DMA, hdb:DMA
ide1: BM-DMA at 0xf008-0xf00f, BIOS settings: hdc:DMA, hdd:DMA
PDC20269: ATA-66/100 forced bit set (WARNING)!!
ide2: BM-DMA at 0xbc00-0xbc07, BIOS settings: hde:pio, hdf:pio
ide3: BM-DMA at 0xbc08-0xbc0f, BIOS settings: hdg:pio, hdh:pio
hda: Maxtor 91360D8, ATA DISK drive
hdb: CD-ROM 48X/AKU, ATAPI CD/DVD-ROM drive
hdc: Maxtor 51536U3, ATA DISK drive
hdd: CD-RW CRX100E, ATAPI CD/DVD-ROM drive
hde: Maxtor 4G160J8, ATA DISK drive
ide0 at 0x1f0-0x1f7,0x3f6 on irq 14
ide1 at 0x170-0x177,0x376 on irq 15
ide2 at 0xac00-0xac07,0xb002 on irq 7
hda: 26563824 sectors (13601 MB) w/256KiB Cache, CHS=1653/255/63, UDMA(33)
hdc: 30015216 sectors (15368 MB) w/2048KiB Cache, CHS=29777/16/63, UDMA(33)
hde: 320173056 sectors (163929 MB) w/2048KiB Cache, CHS=317632/255/63, UDMA(133)

As you can see, the drive is now defined as running UDMA(133), as it should, and that in spite of it starting out running in pio mode, and being set to ata66. Not terribly logical!

DMA vs pio

Looking at the bootlog, the gigadrive at first doesn't appear to run in DMA mode:

ide0: BM-DMA at 0xf000-0xf007, BIOS settings: hda:DMA, hdb:DMA
ide1: BM-DMA at 0xf008-0xf00f, BIOS settings: hdc:DMA, hdd:DMA
ide2: BM-DMA at 0xbc00-0xbc07, BIOS settings: hde:pio, hdf:pio
ide3: BM-DMA at 0xbc08-0xbc0f, BIOS settings: hdg:pio, hdh:pio

Note, however, that later it still defines hde as running UDMA(133) as above. Here is some information:

• The Linux Ultra-DMA Mini-Howto -- http://www.tldp.org/HOWTO/mini/Ultra-DMA.html

"Most non-SCSI hard drives and drive interfaces that you can buy today or are likely to be using are EIDE, although many of the larger drives now available are UDMA. Bus Master DMA is a technology for increasing the speed of hard disk data transfers which requires support from the motherboard and the BIOS, and at least some support from the drive." DMA improves the transfer rate.

LVM

Note that cyberspace's /dev/hdb has been configured as a logical drive, using Logical Volume Manager; see details. This was subsequently removed.

Speed tests

Spello's drives

hdparm -tT /dev/hda

/dev/hda:
Timing buffer-cache reads: 128 MB in 0.93 seconds =137.63 MB/sec
Timing buffered disk reads: 64 MB in 7.97 seconds = 8.03 MB/sec

hdparm -tT /dev/hde

/dev/hde:
Timing buffer-cache reads: 128 MB in 0.93 seconds =137.63 MB/sec
Timing buffered disk reads: 64 MB in 2.64 seconds = 24.24 MB/sec

hdparm -tT /dev/hdh

/dev/hdh:
Timing buffer-cache reads: 128 MB in 0.93 seconds =137.63 MB/sec
Timing buffered disk reads: 64 MB in 2.88 seconds = 22.22 MB/sec

Note that the buffer-cache reads are fast -- even a tad better than cyberspace, possibly because of the 256 MB RAM. The buffered disk reads are disappointing -- gubbio gets 35 MB/sec on its UDMA(133) drive.

I must have done something -- but I don't know that I did! A few minutes later I got the speed I had been expecting (since this is what I get on gubbio's 133 MHz bus drive):

hdparm -tT /dev/hdh

/dev/hdh:
Timing buffer-cache reads: 128 MB in 0.92 seconds =139.13 MB/sec
Timing buffered disk reads: 64 MB in 1.89 seconds = 38.10 MB/sec

/dev/hde:
Timing buffer-cache reads: 128 MB in 0.93 seconds =137.63 MB/sec
Timing buffered disk reads: 64 MB in 1.66 seconds = 38.55 MB/sec

Now that is spectacular. However, it's not that I've done something -- results are simply very uneven. Here are the results of tests in quick succession on /dev/hdh:

Timing buffered disk reads: 64 MB in 2.31 seconds = 27.71 MB/sec
Timing buffered disk reads: 64 MB in 2.68 seconds = 23.88 MB/sec
Timing buffered disk reads: 64 MB in 2.44 seconds = 26.23 MB/sec
Timing buffered disk reads: 64 MB in 1.67 seconds = 38.32 MB/sec

From 23 to 38 MB/sec -- it may be that the drive is just warming up and cooling down. The subsequent five reads held steady at 38.10 MB/sec. My best reading for /dev/hde was 139.13 MB/sec for buffer-cache and 38.55 MB/sec for buffered disk. It looks like this is what you get once the disk is spinning at full speed.

Gubbio's drives

An easy way to measure the sustained transfer rate is to use hdparm, like this:

gubbio:/mnt # hdparm -Tt /dev/hdc8

/dev/hdc8:
Timing buffer-cache reads: 128 MB in 1.60 seconds = 80.00 MB/sec
Timing buffered disk reads: 64 MB in 2.60 seconds = 24.62 MB/sec

Or again for the gigadrive (this is after defining ide2=ata66 in lilo and getting UDMA(133) bus speed at bootup):

gubbio:/mnt # hdparm -Tt /dev/hde1

/dev/hde1:
Timing buffer-cache reads: 128 MB in 1.61 seconds = 79.50 MB/sec
Timing buffered disk reads: 64 MB in 1.83 seconds = 34.97 MB/sec

These results are actually not bad -- no difference on the timing buffer cache, but the gigadrive is significantly faster on the buffered disk read (35 vs 25MB/sec). Not orders of magnitude, but clearly faster. So maybe the 133MHz is working?

Here's the final drive -- this is a FAT32 drive running Windows:

gubbio:/windows # hdparm -Tt /dev/hda5

/dev/hda5:
Timing buffer-cache reads: 128 MB in 1.63 seconds = 78.53 MB/sec
Timing buffered disk reads: 64 MB in 5.33 seconds = 12.01 MB/sec

Now, that buffered disk read is truly slow -- so you're talking about 35 vs 25 vs 12. Let's see what you get on an NFS drive -- no, I can't get it to test it.

Now, let's try to speed up the other drives, or maybe first slow down gigadrive. I set ide1=ata66 instead of ide2 -- that should put the faster bus speed on the main hdc drive. Here are the results. In the bootlog, I get this:

ide0: BM-DMA at 0xf000-0xf007, BIOS settings: hda:DMA, hdb:DMA
        PIIX4: ATA-66/100 forced bit set (WARNING)!!
ide1: BM-DMA at 0xf008-0xf00f, BIOS settings: hdc:DMA, hdd:DMA
ide2: BM-DMA at 0xbc00-0xbc07, BIOS settings: hde:pio, hdf:pio
ide3: BM-DMA at 0xbc08-0xbc0f, BIOS settings: hdg:pio, hdh:pio

But the actual configuration shows that /dev/hdc did not get reset to a higher bus speed:

hda: 26563824 sectors (13601 MB) w/256KiB Cache, CHS=1653/255/63, UDMA(33)
hdc: 30015216 sectors (15368 MB) w/2048KiB Cache, CHS=29777/16/63, UDMA(33)
hde: 320173056 sectors (163929 MB) w/2048KiB Cache, CHS=317632/255/63, UDMA(33)

Now, that means the gigadrive should be slower:

gubbio:/mnt # hdparm -Tt /dev/hde1

/dev/hde1:
Timing buffer-cache reads: 128 MB in 1.54 seconds = 83.12 MB/sec
Timing buffered disk reads: 64 MB in 2.17 seconds = 29.49 MB/sec

Well, the buffer cache is actually a bit faster (83 vs 79.5MB/sec). However, the buffered disk is significantly slower -- 29.5 vs 35MB/sec. So this is more or less as predicted.

While /dev/hdc should be unchanged:

gubbio:/mnt # hdparm -Tt /dev/hdc8

/dev/hdc8:
Timing buffer-cache reads: 128 MB in 1.54 seconds = 83.12 MB/sec
Timing buffered disk reads: 64 MB in 2.60 seconds = 24.62 MB/sec

Similar numbers here -- for some reason, the buffer cache is now faster on all drives. The buffered disk is exactly what it was before.

The FAT32 drive should be unchanged:

gubbio:/mnt # hdparm -Tt /dev/hda5

/dev/hda5:
Timing buffer-cache reads: 128 MB in 1.61 seconds = 79.50 MB/sec
Timing buffered disk reads: 64 MB in 5.36 seconds = 11.94 MB/sec

The FAT32 drive is a touch slower than before, but insignificantly so. Now try setting ide0=ata66 to see if the FAT32 drive can handle some extra speed. I also reset the high speed for the gigadrive (ide2-ata66). Of course what this makes me wonder is whether I should switch the other two drives over to the IDE133 card!

After a reboot, I see that neither /dev/hda nor /dev/hdc will accept the higher bus speed. Indeed, I look up the /dev/hda drive, which is a Maxtor 91303D, and it can only handle ATA33 (see specs). The fast drive I bought for Stef is a Maxtor 51536U3 -- this actually does support ATA66 -- this is the DiamondMax 40 series, featuring an average seek time of sub-9.0 ms, a fast 2 MB 100 MHz SDRAM cache buffer, and UltraDMA 66 interface (see specs).

Now, this may well mean that the motherboard currently only supports ATA33 natively, and that the fast drive is slower than it need be -- this could be an important restriction in working with digital video! So can you move the hdc drive to a new location and still hope to boot to Linux? Or should you make a boot disk first and see where it ends up?

Make a bloody boot disk. Oh well, I tried very carefully to follow instructions, and it failed -- it just wouldn't boot. In the meantime, I realized that in order to make the fast 15GB drive boot off the 133MHz PCI bus card, I need another 80-wide IDE cable. So add that to the shopping list.

Speed tests on cyberspace

cyberspace:~ # hdparm -Tt /dev/hda (the main drive, with the OS and applications)

/dev/hda:

Timing buffer-cache reads: 128 MB in 0.98 seconds =130.61 MB/sec
Timing buffered disk reads: 64 MB in 2.54 seconds = 25.20 MB/sec

cyberspace:~ # hdparm -Tt /dev/hdb (this is the vc drive)

/dev/hdb:

Timing buffer-cache reads: 128 MB in 0.96 seconds =133.33 MB/sec
Timing buffered disk reads: 64 MB in 2.37 seconds = 27.00 MB/sec

hdparm -tT /dev/hdd (this is the IBM DiskStar drive, vd)

/dev/hdd:
Timing buffer-cache reads: 128 MB in 0.98 seconds =130.61 MB/sec
Timing buffered disk reads: 64 MB in 2.18 seconds = 29.36 MB/sec

For some reason, buffer-cache reads are twice as fast on cyberspace! Note that this is machine-specific -- and the Dells are simply faster than the Abit BH6 board on gubbio. The amount of memory doesn't seem to matter. Gubbio's gigadrive is a lot faster on buffered disk reads, however -- around 35 MB/sec. This is due to the Promise ATA133 card -- on Spello, you get a gain from 29 MB/sec to around 38 MB/sec, while Gubbio's 133 holds back at 35 MB/sec. Copying from Gubbio to Spello should be fast! I tested it, and copying is between 8.4 and 9.4 MB/sec, so drive speed wasn't a factor anyway. This is the ethernet holding us back.

Drive characteristics

More drive characteristics with hdparm -I.(note that Spello has a newer version of hdparm):

Gubbio A drive -- hdparm -I /dev/hda5

/dev/hda5:

Model=aMtxro9 31068D , FwRev=AP2SB351, SerialNo=8V70N6AQ
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=29
BuffType=DualPortCache, BuffSize=256kB, MaxMultSect=16, MultSect=16
CurCHS=26353/16/63, CurSects=-66060037, LBA=yes, LBAsects=26563824
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 *udma2
AdvancedPM=no
Drive Supports : ATA/ATAPI-4 T13 1153D revision 17 : ATA-1 ATA-2 ATA-3 ATA-4
Kernel Drive Geometry LogicalCHS=1653/255/63 PhysicalCHS=26353/16/63

Gubbio C drive -- hdparm -I /dev/hdc8

/dev/hdc8:

Model=aMtxro5 51633U , FwRev=AD26700V, SerialNo=3K90Y1C4
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=57
BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=30015216
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 *udma2 udma3 udma4
AdvancedPM=yes: disabled (255)
Drive Supports : ATA/ATAPI-4 T13 1153D revision 17 : ATA-1 ATA-2 ATA-3 ATA-4 ATA-5
Kernel Drive Geometry LogicalCHS=1868/255/63 PhysicalCHS=29777/16/63

Gubbio E drive -- hdparm -I /dev/hde1

/dev/hde1:

Model=aMtxro4 1G068J , FwRev=AG8K910K, SerialNo=8G301EE0
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=57
BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=268435455
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 udma2 udma3 udma4 udma5 *udma6
AdvancedPM=yes: disabled (255)
Drive Supports : ATA/ATAPI-6 T13 1410D revision 0 : ATA-1 ATA-2 ATA-3 ATA-4 ATA-5 ATA-6
Kernel Drive Geometry LogicalCHS=317632/255/63 PhysicalCHS=317632/255/63

Note that Gubbio's drive A and C are now both running at their most advanced udma -- udma2 for the A drive and udma6 for the E drive. It's the C drive that should be running at udma4 instead of udma2. This could be the result of two factors (the drive will automatically pick the highest available speed -- at least it will when prompted by idex=ATA66):

• The cable is the wrong kind for UDMA66 (which it is -- just try to change that first)
• The motherboard controller only handles UDMA33

It turns out that the BH6 board in fact does not handle UDMA66 natively (you need a card, which you have), and you also need the 80-pin cable for the card. Transfer rates on the C drive should have some room for improvement.

Cyberspace A drive: cyberspace:~ # hdparm -I /dev/hda1

/dev/hda1:

Model=UQNAUT MIFERABLL PML012. , FwRev=3A.57000, SerialNo=880251644183
Config={ HardSect NotMFM HdSw>15uSec Fixed DTR>10Mbs }
RawCHS=16383/16/63, TrkSize=32256, SectSize=21298, ECCbytes=4
BuffType=DualPortCache, BuffSize=1900kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=19925880
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 *udma2 udma3 udma4
AdvancedPM=no
Drive Supports : ATA/ATAPI-5 T13 1321D revision 1 : ATA-1 ATA-2 ATA-3 ATA-4 ATA-5
Kernel Drive Geometry LogicalCHS=1240/255/63 PhysicalCHS=19767/16/63

Cyberspace B drive: cyberspace:~ # hdparm -I /dev/hdb1

/dev/hdb1:

Model=aMtxro4 1G026J , FwRev=AG8K910K, SerialNo=6GB000EB
Config={ Fixed }
RawCHS=16383/16/63, TrkSize=0, SectSize=0, ECCbytes=57
BuffType=DualPortCache, BuffSize=2048kB, MaxMultSect=16, MultSect=16
CurCHS=16383/16/63, CurSects=-66060037, LBA=yes, LBAsects=240121728
IORDY=on/off, tPIO={min:120,w/IORDY:120}, tDMA={min:120,rec:120}
PIO modes: pio0 pio1 pio2 pio3 pio4
DMA modes: mdma0 mdma1 mdma2 udma0 udma1 *udma2 udma3 udma4 udma5 udma6
AdvancedPM=yes: disabled (255)
Drive Supports : ATA/ATAPI-6 T13 1410D revision 0 : ATA-1 ATA-2 ATA-3 ATA-4 ATA-5 ATA-6
Kernel Drive Geometry LogicalCHS=238216/16/63 PhysicalCHS=238216/16/63

Note that both of these drives support higher DMAs -- the Quantum Fireball supports UDMA4, or 66MHz, and the Maxtor supports UDMA6, or 133MHz. This means for instance that you could put it on the 133MHz PCI board along with the gigadrive -- it also means that it may conceivably benefit from an 80-cable -- you might want to get three of them.

Drive characteristics

Drive performance (MB/sec)

So this is a big surprise: why does cyberspace have twice the buffer cache speed? Is this an OS module effect -- something I'm not including in the module -- or a motherboard effect? At any rate, it doesn't seem to have any visible results -- video actually plays better on gubbio, at least in aviplay.

More queries:

Note that you query operating system version by uname -r:

gubbio:/proc/ide # cat /proc/ide/pdc202xx

PDC20269 TX2 Chipset.

gubbio:/proc/ide # cat /proc/ide/ide2/hde/settings
name value min max mode
---- ----- --- --- ----
acoustic 0 0 254 rw
address 1 0 2 rw
bios_cyl 317632 0 65535 rw
bios_head 255 0 255 rw
bios_sect 63 0 63 rw
breada_readahead 8 0 255 rw
bswap 0 0 1 r
current_speed 70 0 69 rw
failures 0 0 65535 rw
file_readahead 124 0 16384 rw
ide_scsi 0 0 1 rw
init_speed 70 0 69 rw
io_32bit 0 0 3 rw
keepsettings 0 0 1 rw
lun 0 0 7 rw
max_failures 1 0 65535 rw
max_kb_per_request 128 1 255 rw
multcount 16 0 16 rw
nice1 1 0 1 rw
nowerr 0 0 1 rw
number 0 0 3 rw
pio_mode write-only 0 255 w
slow 0 0 1 rw
unmaskirq 0 0 1 rw
using_dma 1 0 1 rw
wcache 0 0 1 rw