Life with the My Book Studio Edition II

Two months ago, I purchased a Western Digital My Book Studio Edition II.

Western Digital My Book Studio Edition II

Western Digital My Book Studio Edition II

This little gem is a 2TB external storage system that sports a quadruple interface (including eSATA, which was one of the the reasons I purchased it), and a RAID0 or RAID1 configuration (which is the other reason).  It was my intention to employ the eSATA interface, which is the fastest external interface available, to allow me to seemlessly move between my desktop and notebook environments.  I envisioned accomplishing this by moving all my “volatile” file data, including files in use by running applications, onto the SEII in a RAID1 configuration, having both my desktop and notebook pointing at these files via their local discs using Windows Vista’s new symbolic link support.  In this fashion, I could simply disconnect the SEII from one machine, connect it to the other, and have all my most recent file changes instantly available.

Well, this ambitious undertaking came to fruition.  Running in RAID1, the SEII mirrors my files from one 1TB drive onto the second for safety.  I substituted the application data files and folders (Skype, Firefox, Thunderbird, etc.) on the local HDD with Vista symlinks which redirect to the files on the SEII.  My applications don’t even know that their data files are not where they are being found.  For example, here’s a portion of my application data folder as it looks today:

...
 8/25/2008  21:22         <DIR>    Forte
 4/26/2009  19:14    <SYMLINKD>    G7PS [W:\Users\Administrator\AppData\G7PS]
12/12/2008  10:44         <DIR>    GnuPG
 2/12/2009  12:34         <DIR>    gtk-2.0
 2/22/2009  15:16         <DIR>    HP
 3/20/2008  19:52         <DIR>    Media Center Programs
10/18/2008  16:31         <DIR>    mIRC
 4/11/2009  10:47    <SYMLINKD>    Mozilla [W:\Users\Administrator\AppData\Mozilla]
 2/08/2008  20:04         <DIR>    Nero
 4/11/2009  10:47    <SYMLINKD>    Skype [W:\Users\Administrator\AppData\Skype]
...

Both machines contain the same symlinks pointing at the same locations, so, as long as the external drive is connected and powered on, my environment follows me.

However, a problem surfaced:  The SEII was narcoleptic!  If left too long without any kind of activity, it falls asleep, and if it is asleep for too long, its assigned drive drops off of Windows’ radar.  Now, I’m all for being “green”, but when I have open files on the drive and it decides to take a permanent nap (as far as Windows is concerned), not only do my applications complain loudly, there’s a real likelihood of damaging the file system on the SEII.  This latter disaster has already happened to me once, and I had to run through the resulting “found.000” folder to try and figure out what files went to the right location.  Of course, I went to the Western Digital site, looking for a solution to this sleeping sickness.  Unfortunately, the impression I got from Western Digital is that it is a programmed behavior, and they are not at all interested in changing it.

Can nothing ever go smoothly?

As I’m fond of telling others, you have to be smarter than the hardware.  Obviously, the firmware in the device is programmed to sleep after a predefined amount of time passes without disc activity.  As long as I’m getting IM’s, or mail is arriving, or I’m browsing the Interwebs, the drive is busy changing files.  However, if some passage of time occurs where nothing happens on the drive (perhaps it’s the weekend and I’m playing an MMO), I can say ‘goodnight’ to my SEII, and the integrity of its file system.  The solution is to keep the device “busy”, even when those things using it are not.  Luckily, I’m a software engineer, and know how to outsmart the hardware.

After trying a few approaches, I lit upon one that so far appears to work.  The device has not fallen asleep on me for nearly a week now, even after letting it sit idle for a few hours while I went to the movies.  The solution is simple enough:  keep it “busy”, without impacting its performance.  The solution I devised takes the form of a Python script:

#!/usr/local/bin/python

import os
import time
import random
import binascii

file_list = []

def path_visitor((files, max),thisdir,nameshere):
    if len(files) >= max:
        # remove all sub-folders from the list to stop any further search
        for name in nameshere:
            full_path = os.path.join(thisdir,name)
            if os.path.isdir(full_path):
                nameshere.remove(name)
        return

    for name in nameshere:
        full_path = os.path.join(thisdir,name)
        if not os.path.isdir(full_path):
            value = random.randint(0, 200000)
            if value >= 199000:
                files.append(full_path)

def calculate_crc32(file):
    crc = 0
    f = open(file, 'rb')
    data = 'bob'

    while len(data) != 0:
        data = f.read(1024*512)
        crc = binascii.crc32(data, crc)

    f.close()

    open('w:\\crc32.txt', 'w').write('0x%08x\n' % (crc & 0xffffffff))

def construct_file_list():
    global file_list

    file_list = []
    os.path.walk('w:\\',path_visitor,(file_list, 1440))

random.seed()

while True:
    if (len(file_list) == 0) or (not os.path.exists(file_list[-1])):
        construct_file_list()

    time.sleep(60)

    calculate_crc32(file_list[-1])
    file_list.pop()

The script initially gathers a list of random files on the device up to a maximum value (I use 1440, because there are that many minutes in 24 hours), and then, at intervals of 1 minute, processes each file, reading it to calculate a CRC32 value, and then writing this value to a small file on the drive’s root partition.  When it runs out of files to process, or when it encounters a file that is now missing from the device, it refreshes its random file cache, and starts again. You can see that I have hard-coded some things (like the drive path, the maximum number of files to cache, etc.), but this is typical for a special-purpose tool like this. Also, the scanning code assumes something about the population of the device (I currently have over 400,000 files on it), so the numbers I use to randomly select files are tuned to give me the 1440 files I want across as wide a sampling of the files on the devices as possible. You’ll likely need to tweak these numbers for your specific situation.

You can run this script manually from a command prompt, and then minimize it to the task bar.  However, I opted to install the script as a Windows service (using srvany).  Once installed, I set the service to run manually, and then created a batch file in my Startup folder that simply calls ‘net start KeepWDS2Alive’ to start the new service whenever I log into my Windows account.  This way, the service (and, thus, the script) is only running when I actually have applications dependent upon the alertness of the SEII.

It is a shame that Western Digital decided that they know what is best for the person who wishes to use their hardware.  If it wasn’t simply arrogance on their part, they could at least explain the hardware requirements that made them decide to hamper this otherwise fine device.  However, software is always the brains of the beast, and if you know how, you can always be smarter than the hardware.

Advertisements

2 thoughts on “Life with the My Book Studio Edition II

  1. Really smart stuff Bob 🙂

    Since I am not a software engineer, I was thinking of easier approaches like a program (maybe SETI’s tool) or something else to keep the Ext on the run.

    Wouldn’t it be better, if it were a native win App ? which a user can add to startup ? perl would require stuff like activeperl to run I guess.

  2. bobhood says:

    Yeah, software engineers tend to construct things that fit into their daily environments. That’s the reason I went with this approach. In addition, as a Python script, the functionality of the script can pretty much be altered ‘on-the-fly’, requiring only a restart of the service.

    I can probably convert the functionality of the Python script into a stand-alone binary application that could simply be dropped into the Startup folder. If I find the time to create that, I’ll post it here as an alternative to the script/srvany approach.

Comments are closed.