Re: RAID-5 softwerowy i wolny zapis <10MB/s

Autor: Bartosz <lot_skasujto_at_algrim.pl>
Data: Mon 20 Sep 2010 - 01:14:57 MET DST
Message-ID: <i765hh$ii5$1@news.onet.pl>
Content-Type: text/plain; format=flowed; charset="iso-8859-2"; reply-type=original

> To faktycznie jest już zabytek. Spróbuj rozpiąć RAID i przetestować
> transfer z pojedynczego dysku. Jeśli też będzie źle, to problem leży w
> obsłudze dysków w systemie, a nie w RAID.
> --
0 wirusów, 0 spywares, zresztą to nie wina systemu tak jak wpsomniałem
identyczny efekt był na czyściutkim 2003serwer i czysciutkim XP.
Tyle, że w 2003serwer był fajny myk, że keszował sobie ok 1GB danych do RAM
i zapisywał w tle ;)
Nie czuć było spowolnienia póki nie dokonywało się operacji na wiekszych niż
1GB danych.
Płyta CD 700MB szła w 9 sekund ;) ale 1,5GB już w 2~3minuty ;)

Bez żadnego rozpinania na Linuxie wykonano pomiary.
[root@localhost ~]# hdparm -tT /dev/sda

/dev/sda: //wester digital
 Timing cached reads: 5316 MB in 2.00 seconds = 2658.37 MB/sec
  Timing buffered disk reads: 288 MB in 3.02 seconds = 95.49 MB/sec
  [root@localhost ~]# hdparm -tT /dev/sdb

  /dev/sdb: // Seagate RAID-5 disk nr1
   Timing cached reads: 5556 MB in 2.00 seconds = 2778.72 MB/sec
    Timing buffered disk reads: 326 MB in 3.00 seconds = 108.60 MB/sec
    [root@localhost ~]# hdparm -tT /dev/sdc

    /dev/sdc: // Seagate RAID-5 disk nr2
     Timing cached reads: 5544 MB in 2.00 seconds = 2772.67 MB/sec
      Timing buffered disk reads: 332 MB in 3.01 seconds = 110.47 MB/sec
      [root@localhost ~]# hdparm -tT /dev/sdd

      /dev/sdd: // Seagate RAID-5 disk nr3
       Timing cached reads: 5352 MB in 2.00 seconds = 2676.76 MB/sec
        Timing buffered disk reads: 332 MB in 3.00 seconds = 110.53 MB/sec
        [root@localhost ~]#

/dev/sde: // Seagate 1TB
Timing cached reads: 5504 MB in 2.00 seconds = 2752.76 MB/sec
      (odczyt cache jak widać chyba magistrala SATA II ogranicza )
Timing buffered disk reads: 376 MB in 3.00 seconds = 125.17 MB/sec
[root@localhost ~]#

smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce
Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION === /dev/sdc
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate 0x000f 118 099 006 Pre-fail
s - 178492115
  3 Spin_Up_Time 0x0003 095 092 000 Pre-fail
s - 0
  4 Start_Stop_Count 0x0032 099 099 020 Old_age
ys - 1056
  5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
s - 0
  7 Seek_Error_Rate 0x000f 073 060 030 Pre-fail
s - 23832805
  9 Power_On_Hours 0x0032 097 097 000 Old_age
ys - 2782
 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail
s - 2
 12 Power_Cycle_Count 0x0032 100 100 020 Old_age
ys - 664
184 Unknown_Attribute 0x0032 100 100 099 Old_age
ys - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age
ys - 0
188 Unknown_Attribute 0x0032 100 099 000 Old_age
ys - 3
189 High_Fly_Writes 0x003a 100 100 000 Old_age
ys - 0
190 Airflow_Temperature_Cel 0x0022 069 053 045 Old_age
ys - 31 (Lifetime Min/Max
29/31)
194 Temperature_Celsius 0x0022 031 047 000 Old_age
ys - 31 (0 15 0 0)
195 Hardware_ECC_Recovered 0x001a 052 026 000 Old_age
ys - 178492115
197 Current_Pending_Sector 0x0012 100 100 000 Old_age
ys - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age
line - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
ys - 0

SMART Error Log Version: 1
No Errors Logged

smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce
Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION === /dev/sdd
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate 0x000f 115 099 006 Pre-fail
s - 98325876
  3 Spin_Up_Time 0x0003 094 089 000 Pre-fail
s - 0
  4 Start_Stop_Count 0x0032 099 099 020 Old_age
ys - 1629
  5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
s - 0
  7 Seek_Error_Rate 0x000f 071 060 030 Pre-fail
s - 25861374430
  9 Power_On_Hours 0x0032 095 095 000 Old_age
ys - 4865
 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail
s - 7
 12 Power_Cycle_Count 0x0032 099 099 020 Old_age
ys - 1190
184 Unknown_Attribute 0x0032 100 100 099 Old_age
ys - 0
187 Reported_Uncorrect 0x0032 100 100 000 Old_age
ys - 0
188 Unknown_Attribute 0x0032 100 099 000 Old_age
ys - 65539
189 High_Fly_Writes 0x003a 100 100 000 Old_age
ys - 0
190 Airflow_Temperature_Cel 0x0022 068 049 045 Old_age
ys - 32 (Lifetime Min/Max
28/32)
194 Temperature_Celsius 0x0022 032 051 000 Old_age
ys - 32 (0 15 0 0)
195 Hardware_ECC_Recovered 0x001a 046 033 000 Old_age
ys - 98325876
197 Current_Pending_Sector 0x0012 100 100 000 Old_age
ys - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age
line - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
ys - 0

SMART Error Log Version: 1
No Errors Logged

smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce
Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION === /dev/sdb
SMART Attributes Data Structure revision number: 10
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED
WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate 0x000f 106 099 006 Pre-fail
s - 12159428
  3 Spin_Up_Time 0x0003 095 091 000 Pre-fail
s - 0
  4 Start_Stop_Count 0x0032 099 099 020 Old_age
ys - 1628
  5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail
s - 0
  7 Seek_Error_Rate 0x000f 079 060 030 Pre-fail
s - 84832422
  9 Power_On_Hours 0x0032 095 095 000 Old_age
ys - 4870
 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail
s - 8
 12 Power_Cycle_Count 0x0032 099 037 020 Old_age
ys - 1190
184 Unknown_Attribute 0x0032 100 100 099 Old_age
ys - 0
187 Reported_Uncorrect 0x0032 095 095 000 Old_age
ys - 5
188 Unknown_Attribute 0x0032 100 099 000 Old_age
ys - 3
189 High_Fly_Writes 0x003a 100 100 000 Old_age
ys - 0
190 Airflow_Temperature_Cel 0x0022 069 049 045 Old_age
ys - 31 (Lifetime Min/Max
29/31)
194 Temperature_Celsius 0x0022 031 051 000 Old_age
ys - 31 (0 15 0 0)
195 Hardware_ECC_Recovered 0x001a 049 034 000 Old_age
ys - 12159428
197 Current_Pending_Sector 0x0012 100 100 000 Old_age
ys - 0
198 Offline_Uncorrectable 0x0010 100 100 000 Old_age
line - 0
199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age
ys - 0

SMART Error Log Version: 1
ATA Error Count: 5
        CR = Command Register [HEX]
        FR = Features Register [HEX]
        SC = Sector Count Register [HEX]
        SN = Sector Number Register [HEX]
        CL = Cylinder Low Register [HEX]
        CH = Cylinder High Register [HEX]
        DH = Device/Head Register [HEX]
        DC = Device Command Register [HEX]
        ER = Error register [HEX]
        ST = Status register [HEX]
Powered_Up_Time is measured from power on, and printed as
DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes,
SS=sec, and sss=millisec. It "wraps" after 49.710 days.

Error 5 occurred at disk power-on lifetime: 583 hours (24 days + 7 hours)
  When the command that caused the error occurred, the device was active or
idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
  -- -- -- -- -- -- -- -- ---------------- --------------------
  60 00 3c ff ff ff 4f 00 09:07:28.069 READ FPDMA QUEUED
  60 00 03 d7 9c 20 40 00 09:07:28.064 READ FPDMA QUEUED
  60 00 01 00 0c 6f 40 00 09:07:28.044 READ FPDMA QUEUED
  00 00 00 00 00 00 00 ff 09:07:24.044 NOP [Abort queued commands]
  60 00 3c ff ff ff 4f 00 09:07:11.044 READ FPDMA QUEUED

Error 4 occurred at disk power-on lifetime: 583 hours (24 days + 7 hours)
  When the command that caused the error occurred, the device was active or
idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
  -- -- -- -- -- -- -- -- ---------------- --------------------
  60 00 3c ff ff ff 4f 00 09:07:11.044 READ FPDMA QUEUED
  61 00 08 87 29 30 40 00 09:07:11.044 WRITE FPDMA QUEUED
  00 00 00 00 00 00 00 ff 09:07:07.044 NOP [Abort queued commands]
  60 00 3c ff ff ff 4f 00 09:06:54.064 READ FPDMA QUEUED
  60 00 08 87 29 30 40 00 09:06:54.044 READ FPDMA QUEUED

Error 3 occurred at disk power-on lifetime: 583 hours (24 days + 7 hours)
  When the command that caused the error occurred, the device was active or
idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
  -- -- -- -- -- -- -- -- ---------------- --------------------
  60 00 3c ff ff ff 4f 00 09:06:54.064 READ FPDMA QUEUED
  60 00 08 87 29 30 40 00 09:06:54.044 READ FPDMA QUEUED
  61 00 02 d7 9c 20 40 00 09:06:54.044 WRITE FPDMA QUEUED
  00 00 00 00 00 00 00 ff 09:06:50.044 NOP [Abort queued commands]
  60 00 3c ff ff ff 4f 00 09:06:37.061 READ FPDMA QUEUED

Error 2 occurred at disk power-on lifetime: 583 hours (24 days + 7 hours)
  When the command that caused the error occurred, the device was active or
idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
  -- -- -- -- -- -- -- -- ---------------- --------------------
  60 00 3c ff ff ff 4f 00 09:06:37.061 READ FPDMA QUEUED
  60 00 02 d7 9c 20 40 00 09:06:37.057 READ FPDMA QUEUED
  60 00 07 79 0c 6f 40 00 09:06:37.044 READ FPDMA QUEUED
  00 00 00 00 00 00 00 ff 09:06:33.044 NOP [Abort queued commands]
  60 00 3c ff ff ff 4f 00 09:06:20.142 READ FPDMA QUEUED

Error 1 occurred at disk power-on lifetime: 583 hours (24 days + 7 hours)
  When the command that caused the error occurred, the device was active or
idle.

  After command completion occurred, registers were:
  ER ST SC SN CL CH DH
  -- -- -- -- -- -- --
  40 51 00 ff ff ff 0f

  Commands leading to the command that caused the error were:
  CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name
  -- -- -- -- -- -- -- -- ---------------- --------------------
  60 00 3c ff ff ff 4f 00 09:06:20.142 READ FPDMA QUEUED
  60 00 04 ff ff ff 4f 00 09:06:20.133 READ FPDMA QUEUED
  60 00 40 ff ff ff 4f 00 09:06:20.133 READ FPDMA QUEUED
  60 00 3c ff ff ff 4f 00 09:06:20.132 READ FPDMA QUEUED
  60 00 04 ff ff ff 4f 00 09:06:20.129 READ FPDMA QUEUED

Coś sporo wygenerowal kompunikatów ten sdb. wszystkie dyski w macieży sa
identyczne.
Grafika 8600GTS ze wsparciem GPU (32 rdzenie) + CPU 4 Rdzenie.
Nvidia chwali się, że GPU ich kart wspiera obsługe RAID-5.
Przy takim sprzecie spokojnie powinien wycisnąć pełną przepustowość PCI
czyli min ~250MB/s i zostawić w tyle większość sprzetowych kontrolerów
klasując się gdzieś w czołówce najszybszych RAID-5 SATA II na rynku.
Cos mi sie wydaje ze sie pokusze w tym tygodniu o zgranie danych z macieży i
odpalenie tego na programowym RAID-5 pod linuxem aby pokazać że to nie wina
sprzętu tylko softu (windowsa).
Zaznaczam co do ODCZYTU nie mam zastrzeżeń ~60MB/s.

Ma ktoś jeszcze jakieś pomysły skąd ten bardzo wolny zapis?
Spróbuję równolegle uderzyć do producenta NVIDIA z tym problemem zobaczymy
co odpowiedzą.
Received on Mon Sep 20 01:15:03 2010

To archiwum zostało wygenerowane przez hypermail 2.1.8 : Mon 20 Sep 2010 - 01:42:01 MET DST