Two months ago, I purchased a Western Digital My Book Studio Edition II.
This little gem is a 2TB external storage system that sports a quadruple interface (including eSATA, which was one of the the reasons I purchased it), and a RAID0 or RAID1 configuration (which is the other reason). It was my intention to employ the eSATA interface, which is the fastest external interface available, to allow me to seemlessly move between my desktop and notebook environments. I envisioned accomplishing this by moving all my “volatile” file data, including files in use by running applications, onto the SEII in a RAID1 configuration, having both my desktop and notebook pointing at these files via their local discs using Windows Vista’s new symbolic link support. In this fashion, I could simply disconnect the SEII from one machine, connect it to the other, and have all my most recent file changes instantly available.
Well, this ambitious undertaking came to fruition. Running in RAID1, the SEII mirrors my files from one 1TB drive onto the second for safety. I substituted the application data files and folders (Skype, Firefox, Thunderbird, etc.) on the local HDD with Vista symlinks which redirect to the files on the SEII. My applications don’t even know that their data files are not where they are being found. For example, here’s a portion of my application data folder as it looks today:
... 8/25/2008 21:22 <DIR> Forte 4/26/2009 19:14 <SYMLINKD> G7PS [W:\Users\Administrator\AppData\G7PS] 12/12/2008 10:44 <DIR> GnuPG 2/12/2009 12:34 <DIR> gtk-2.0 2/22/2009 15:16 <DIR> HP 3/20/2008 19:52 <DIR> Media Center Programs 10/18/2008 16:31 <DIR> mIRC 4/11/2009 10:47 <SYMLINKD> Mozilla [W:\Users\Administrator\AppData\Mozilla] 2/08/2008 20:04 <DIR> Nero 4/11/2009 10:47 <SYMLINKD> Skype [W:\Users\Administrator\AppData\Skype] ...
Both machines contain the same symlinks pointing at the same locations, so, as long as the external drive is connected and powered on, my environment follows me.
However, a problem surfaced: The SEII was narcoleptic! If left too long without any kind of activity, it falls asleep, and if it is asleep for too long, its assigned drive drops off of Windows’ radar. Now, I’m all for being “green”, but when I have open files on the drive and it decides to take a permanent nap (as far as Windows is concerned), not only do my applications complain loudly, there’s a real likelihood of damaging the file system on the SEII. This latter disaster has already happened to me once, and I had to run through the resulting “found.000” folder to try and figure out what files went to the right location. Of course, I went to the Western Digital site, looking for a solution to this sleeping sickness. Unfortunately, the impression I got from Western Digital is that it is a programmed behavior, and they are not at all interested in changing it.
Can nothing ever go smoothly?
As I’m fond of telling others, you have to be smarter than the hardware. Obviously, the firmware in the device is programmed to sleep after a predefined amount of time passes without disc activity. As long as I’m getting IM’s, or mail is arriving, or I’m browsing the Interwebs, the drive is busy changing files. However, if some passage of time occurs where nothing happens on the drive (perhaps it’s the weekend and I’m playing an MMO), I can say ‘goodnight’ to my SEII, and the integrity of its file system. The solution is to keep the device “busy”, even when those things using it are not. Luckily, I’m a software engineer, and know how to outsmart the hardware.
After trying a few approaches, I lit upon one that so far appears to work. The device has not fallen asleep on me for nearly a week now, even after letting it sit idle for a few hours while I went to the movies. The solution is simple enough: keep it “busy”, without impacting its performance. The solution I devised takes the form of a Python script:
#!/usr/local/bin/python import os import time import random import binascii file_list =  def path_visitor((files, max),thisdir,nameshere): if len(files) >= max: # remove all sub-folders from the list to stop any further search for name in nameshere: full_path = os.path.join(thisdir,name) if os.path.isdir(full_path): nameshere.remove(name) return for name in nameshere: full_path = os.path.join(thisdir,name) if not os.path.isdir(full_path): value = random.randint(0, 200000) if value >= 199000: files.append(full_path) def calculate_crc32(file): crc = 0 f = open(file, 'rb') data = 'bob' while len(data) != 0: data = f.read(1024*512) crc = binascii.crc32(data, crc) f.close() open('w:\\crc32.txt', 'w').write('0x%08x\n' % (crc & 0xffffffff)) def construct_file_list(): global file_list file_list =  os.path.walk('w:\\',path_visitor,(file_list, 1440)) random.seed() while True: if (len(file_list) == 0) or (not os.path.exists(file_list[-1])): construct_file_list() time.sleep(60) calculate_crc32(file_list[-1]) file_list.pop()
The script initially gathers a list of random files on the device up to a maximum value (I use 1440, because there are that many minutes in 24 hours), and then, at intervals of 1 minute, processes each file, reading it to calculate a CRC32 value, and then writing this value to a small file on the drive’s root partition. When it runs out of files to process, or when it encounters a file that is now missing from the device, it refreshes its random file cache, and starts again. You can see that I have hard-coded some things (like the drive path, the maximum number of files to cache, etc.), but this is typical for a special-purpose tool like this. Also, the scanning code assumes something about the population of the device (I currently have over 400,000 files on it), so the numbers I use to randomly select files are tuned to give me the 1440 files I want across as wide a sampling of the files on the devices as possible. You’ll likely need to tweak these numbers for your specific situation.
You can run this script manually from a command prompt, and then minimize it to the task bar. However, I opted to install the script as a Windows service (using srvany). Once installed, I set the service to run manually, and then created a batch file in my Startup folder that simply calls ‘net start KeepWDS2Alive’ to start the new service whenever I log into my Windows account. This way, the service (and, thus, the script) is only running when I actually have applications dependent upon the alertness of the SEII.
It is a shame that Western Digital decided that they know what is best for the person who wishes to use their hardware. If it wasn’t simply arrogance on their part, they could at least explain the hardware requirements that made them decide to hamper this otherwise fine device. However, software is always the brains of the beast, and if you know how, you can always be smarter than the hardware.