Poor SSD's File Writing Performance on Windows

SSD (Solid State Drive) is undoubtly fast and will be the favor of performance choices over the current mechanical hard drive.

However, when this SSD is run with Windows OS (e.g. XP or Vista), the performance will drop significantly.

Active applications can also stop responding for about 1-2 minutes (followed by irresponsive mouse), when browsers like IE/firefox are being opened up and navigated to certain heavy websites.

This is because the Windows' file read/write mechanism is optimized for hard drive's mechanical movement (as compared to Linux, which is block-by-block)

When writing a file, the entire block on SSD has to be erased before it can be written. The drive cannot be written over the top of pre-written data.

This adds latency as every bit written to the drive has to be preceded by a bit erased and the random writes fill up the write buffer on the drive, causing the SATA controller to pause when the data are being flushed.

Visit these links for more detail about SSD's block-rewrite penalty:
- http://techreport.com/articles.x/17136/2
- http://techreport.com/articles.x/17136/4

So, the bigger the SSD capacity you have, the bigger the block you will have (and the slower the XP or Vista you will get :-))

Any workaround?

Yes, just install Windows SteadyState (which currently only supports for XP/Vista 32-bit only), or use Linux with NILFS.

---
Update on 09/13/2012 :

It has been very busy for me in these past three years to spare the time on writing the blog. Perhaps I will not have luxury to update it so frequently anymore.
 
Too bad that the Windows SteadyState is already discontinued with some limitations in which it can only work in the 32-bit Windows. 

It was also found out that the partition misalignment was actually the major cause for these performance penalties. 

So, partition misalignment combined by old XP or bad vista, will just cause our SSD to become useless :(

For some background on it, SSD and new generation of HD require the partitions to be aligned with the 4K sector size. 

When the partitions are misaligned, there will be overhead in the reading/writing of random block files to cause performance penalties as well as shortening the write endurance (called as "write amplification"):
 
- http://dfarq.homeip.net/2011/07/why-you-should-align-your-ssd/
- http://www.msc-ge.com/en/news/pressroom/essays/7480-www/6318-www.html#dsy6318-www_3.2

Some more links about partition alignment on SSD:

- http://dl.paragon-software.com/free/Paragon%20Alignment%20Tool%20-%20White%20Paper.pdf 
- http://lifehacker.com/5837769/make-sure-your-partitions-are-correctly-aligned-for-optimal-solid-state-drive-performance
- http://siduction.org/index.php?module=news&func=display&sid=78&lang=en 

Happy upgrading the HD to SSD :)

Clone and Resize Multiple Windows/Linux Partitions for the new Harddisk

One of my old laptop harddisk (250GB) was nearly full, and its partitions were structured like this:
- MBR: grub
- /dev/sda1: Windows XP (fat32, windows boot)
- /dev/sda2: data (ntfs)
- /dev/sda3: Suse 11.0 (ext3)
- /dev/sda4: Linux swap

Getting new harddisk with 500GB, I would like to copy the whole old HD data into the new one with the same structure like this:
- MBR: grub
- /dev/sdb1: Windows XP (fat32, windows boot, with bigger size)
- /dev/sdb2: data (ntfs, with bigger size)
- /dev/sdb3: Suse 11.0 (ext3, with bigger size)
- /dev/sdb4: Linux swap

I am too lazy to wait for the cloning if I have to use dd to read raw image of the entire 250GB of /dev/sda into /dev/sdb. After dd-ing, I will still have to rearrange and resize the partitions to fit them into 500GB new size with partitioner tool like gparted (resizing the partitions in the gparted will need another long time I have to wait as I have 90% data of 250GB size to be re-arranged for /dev/sdb1,2,3,4).

So, in order to speed up the imaging, I prefer to do some manual work like these:
1) Attach the new HD to the laptop as USB storage.
2) Clone the Windows boot partition with dd from /dev/sda1 to /dev/sdb1 (new sdb1 has to be created with same size using Linux fdisk)
3) Resize /dev/sdb1 only (with gparted)
4) Create new empty partition of /dev/sdb2,3,4 manually (using Linux fdisk)
5) Copy the rest of the files manually from old partitions to the newly created partitions (/dev/sdb2,3,4)
   - On /dev/sdb2, use manual dragging from Windows explorer (can also be done from Linux)
   - On /dev/sdb3, use rsync from localbind mount to the target
6) Remove the old HD and attach the new one as SATA.
7) Revive back the grub in the new HD.

Okay done. After doing step #1-5, I could remove my old HD and attached the new one as SATA. With one single drive, my new HD should be seen as "/dev/sda", and the last step to do would be to revive back the grub on the newly cloned HD.

Afterwards, everything was fine (my Suse Linux booted nicely) except the main Windows XP got stuck at blue login welcome screen!

What happened?

Oh, after digging around the web, this seems to be caused by the so called "Windows feature" on memorizing the partition signatures :-(( . It must be due to my itchy hand on step #1, attaching the USB HD from Windows.

So bad. When the Windows assigned wrong drive letter during the bootup, it also created junk files on my new D: drive with folders like "My Documents" and "Program Files".

What could I do next? Do I need to boot my old Windows and clear the partition signature and re-clone again?

I am a lazy guy to wait for another imaging for the whole 250GB, and luckilly I could still boot my linux from the new /dev/sda3 partition.

After booting into Linux, life became easier and I could do the fix with some file cleanup:
- Clear the partition signature: dd if=/dev/zero of=[my new harddisk] bs=446 count=1
   (be extra careful on running this command as it can wipe out your entire disk if it points to the wrong target!)
   (Take note 446 is the safe bootstrap code area to be zero-ed, while 447-512 contain master partition table info and the 0xaa55 signature
   (If you are not familiar with dd, you can also use "fdisk /mbr" from Win98 as per described by the previous link)
- Install back the grub (grub-install /dev/sda)
- Delete all junk folders/files on my new D: drive (/dev/sda2)
- Also, I don't like the folder/files like "System Volume Informations", pagefile.sys, hiberfil.sys (from my previous old HD at /dev/sda1,2), so I deleted them too :-))

What a pain when we have to deal with Windows.
Linux makes our life simpler ;-)

Make FAT16/32 Thumbdrive Bootable with syslinux

Is it so special?

Mostly people know default Linux file systems are ext2/ext3.
If you'd like to boot it, you have to use one of those file systems.

As a result, if you format your thumbdrive with those Linux file system, the thumbdrive will not be accessible by Windows (unless you install special driver on it).

This may become uncomfortable if you have to bring your thumbdrive to Windows PCs and enquire your colleages/friends to follow your "techy" style when you need to share your files with ext2/3 file system.

FAT16/32 is commonly supported by Windows and Linux.
You may want to make your thumbdrive to be multifunctions: can store/share files easily for Windows, and can boot Linux as well.

So, the answer is syslinux.
You can maintain to keep FAT16/32 on your thumbdrive, and at the same time, make it bootable.

Here are the steps:

1. Download the latest syslinux
2. Extract it to let say C:\utils\syslinux-3.73
3. Mark the thumbdrive bootable by executing syslinux.exe located at win32 folder
(for example: if your thumbdrive is mapped at F: drive, execute "syslinux.exe -ma f:"
4. Create a folder in your thumbdrive to place linux binaries (e.g. F:\puppy412)
5. Copy/extract one of linux distribution (e.g. Puppy Linux) that has USB live booting support into it
(get 7-zip if you need to extract ISO file for it)

6. Create/edit syslinux.cfg (at main directory of the thumbdrive, e.g. F:\syslinux.cfg) and point the boot file to your linux folder

Example syslinux.cfg:
(for Puppy Linux, you just need to copy/rename isolinux.cfg to syslinux.cfg)

default puppy
display /puppy412/boot.msg
prompt 1
label puppy
   kernel /puppy412/vmlinuz
   append initrd=/puppy412/initrd.gz pmedia=usbflash noapic acpi=off
label linux
   kernel /syslinux/chain.c32
   append hd1
timeout 50

7. Boot your thumbdrive and see it booting with your favorite Linux























Wanna have more fun with syslinux?
Well, just dig more on its web.

SSD vs RAID0 vs 10k rpm drive

I recently bought 32GB SSD drive (S$195) for my laptop (HP Pavilion 9612CA).

With 2 SATA slots, so I tweaked it as follows:
- Slot 1: 32GB SSD
- Slot 2: 250GB 5.4k rpm

It rocks !
It beats both RAID0 and 10k rpm drive performance.

Kernel Recompilation

Well, there are a lot of complicated steps we need to follow, and this is what I usually do:

0. yast -i ncurses-devel     #ensure ncurses-devel is installed  
1. make mrproper              #clean config
2. cp /boot/config-x.x.x ./.config (or: cp /boot/config-`uname -r` ./.config)
3. make menuconfig (or: make oldconfig)

For normal steps:

4. make dep; make clean         #clean compilation
5. make bzImage
6. make modules
7. cp src/arch/i386/boot/bZimage /boot/vmlinuz-x.y.z
8. make modules_install
9. mkinitrd

For SUSE steps:

4. make rpm
5. Secure the backup:
      cp /usr/lib/rpm/find-provides.ksyms /usr/lib/rpm/find-provides.ksyms.orig
      cp /usr/lib/rpm/find-requires.ksyms /usr/lib/rpm/find-requires.ksyms.orig
      cp /usr/lib/rpm/find-supplements.ksyms /usr/lib/rpm/find-supplements.ksyms.orig
6. Change "kernel-*)" to "kernel *)"
      #kernel-*)           is_kernel_package=1 ;;
      kernel-*)           is_kernel_package=1 ;;
7. -
8. rpm -ivh kernel-x.x.x.rpm
9. make initrd

Reviving the grub in the rsync's Backup Disk

Incremental backup with rsync is very effective for daily backup.
However, it cannot be used to boot the system after failure.

This is because the disk is just simply a storage without capability to boot the Linux.
In order to make it bootable, the boot loader has to be installed back.

Assuming the grub should be installed into MBR, the steps would be:

1. Boot the system with rescue CD

2. Invoke the following command:

# mount /dev/sda1 /mnt/      # Assuming the main disk resides on single partition
# mount –t proc none /mnt/proc
# mount –o bind /dev /mnt/dev
# chroot /mnt/ /bin/bash
# grub
At grub:
grub> root (hd0,1)
grub> setup (hd0)
grub> quit

3. Check (and adjust if necessary) the config files like menu.lst, device.map, fstab, and mtab.

4. If needed, run: "grub-install --recheck --root-directory=/mnt /dev/sda"

Still cannot boot?
Well, be happy with dd and just troubleshoot the MBR of your disk:
# dd if=/dev/sda count=1 bs=512 | xxd

Incremental Backup with rsync

There are few reasons I don't want to use dd for full backup:
- My HD is having 250GB data and I don't want to do daily backup for it
- My data do not change so frequently
- I don't want to burden my server with 250GB of daily thrashing just for backup only

To do incremental backup, we can make use of rsync:
rsync -azvr /mnt/localbind root@mystorages:/mybackup/

To simplify the life, just use single partition for each physical disks, and bind them in the fstab.

#!/bin/bash

TIMESTAMP=`date +%Y%m%d_%H%M%S`

mount -o bind,ro / /mnt/localbind
mount -o remount,rw /backup/mainbackup

nice -n 19 rsync -azvrb --suffix=.rsync_$TIMESTAMP /mnt/localbind/ /backup/mainbackup/
echo "Backup performed on $TIMESTAMP" >>/backup/backup.log
sync

umount /mnt/localbind
mount -o remount,ro /backup/mainbackup

Simple Backup using dd

Backup the whole disk becomes easier with dd command:

1. Fill empty zero till full: "dd if=/dev/zero of=/tmp/delete.me bs=8M; rm delete.me"
2. Invoke the dd command combined with gzip: "dd if=/dev/sda bs=4096 | gzip > /mnt/nfs/diskboot.img.gz"

3. To restore the data from image into the disk, execute: "gzip -dc /mnt/nfs/diskboot.img.gz | dd of=/dev/xxx" (replace /dev/xxx with your target disk)

Warning: Be extra careful when invoking "dd of=/dev/xxx" command since it can wipe out all of your data on your disk if you are pointing to the wrong device.

My big LCD

Huraah !
I bought new 22" LCD that supported resolution up to 1680x1050