Organizing Data


Now that I have most of the software I use on a recurring basis installed on the MacBook Pro, it is time to start addressing my data. Pictures and music were the low-hanging fruit, and have already been transferred. (Although I am considering deleting all the pictures and re-transferring them via the Migration Assistant. Doing it manually certainly worked, but the folder hierarchy I had before was lost. I really don’t want to recreate it and re-sort thousands of images.)

The rest of my data breaks down into about eight types: E-books, Manuals, Word documents, Excel spreadsheets, Powerpoint presentations, diagrams, PDF, and RFT files. Figuring out where to put all of this on the new machine isn’t as easy as simply copying it across. The current organization scheme grew organically and without much thought at times. It is less an organization and more a hodge-podge.

A Word on the Home Directory Hierarchy

When a new user account is created on a Macintosh computer there are some directories included in the user’s home directory by default. They are:

The Dropbox folder itself, not the software that enables the client, is located in my home directory. This is the default location and I’m used to it and think it works very well.

bin is the folder where my program scripts, either bash, Ruby, or Python, live. bin is short for “binary.”

Projects is where I keep any source code that I am developing, or the files associated with a software project, e.g., WordPress site themes or web site files.

With a precedent set of adding home directories for specific uses, I need to decide whether my remaining files go into one of the established directories or into a specially created directory just for them.

E-books

I have been an avid e-book reader for several years. I got hooked on the format with a Palm and consequently most of my e-books are in the eReader format. I also have a number of PDF or Word copies of books. Heretofore my eBook folder has been located under the Documents folder in my home directory. However, eBooks aren’t Documents, they are specialized in format and purpose. I think it’s time to move eBooks out of Documents and into it’s own directory, Ebooks.

Manuals

Over the years I’ve collected a number of technical books and references, some as PDFs, others as self-contained web sites. While these could be considered eBooks none of them require specialized or proprietary software to be viewed. I’ve always kept them in a directory in my home directory. I see no reason to change that now.

Word, Excel, Powerpoint

Historically I have tended to keep a directory under my Documents directory for each of these file types. I think this grew out of wanting to set a default save location for each in their respective programs. Over time these three directories have become the most jumbled and chaotic.  Each is littered with files and sub-directories. The sub-directories were an attempt to impose order. “Resumes,” for example is a sub-folder under Word that houses many examples of my resume. It also holds some Rich Text Format (RTF) copies of my resume, and some PDF copies as well. I put the non-Word format files in with the Word-format ones since that is where my resumes were. Adding insult to injury not all of my resumes are in the Resume directory, there are a few running around loose in the Documents folder.

The Excel directory is equally messy for the same reasons. Powerpoint isn’t too bad simply because I don’y have too many slide shows. However it does hold a couple of OpenOffice Presentation files.

Clearly it is time for a major rethinking regarding how to organize files produced by my office productivity suite. Since I am switching from Microsoft Office to OpenOffice this seems like a good time to abandon the Word - Excel - Powerpoint triumvirate. The better approach I think would be to promote the sub-directories from inside Word, Excel, and Powerpoint, so that they live directly under Documents. Instead of saving things based on their file type, save them by topic. All resumes, regardless of file type, go in the resume directory.

The hard part will be moving everything around and developing a good sense for topics.

PDF and RTF

These files have their own directory on my old machine just like Word, Excel, and Powerpoint. And just like those three, it is a somewhat chaotic jumble of files. Once again, grouping files by type wasn’t a good idea. Henceforth PDFs and RTFs will be stored in the topical directory where they belong.

Diagrams

The final broad category of files are diagrams. Some of these are Visio documents I’ve saved from previous employment engagements and the rest are Omni Graffle files. And like all their sibling file types, these have tended to be segregated by file extension. So, just like the rest these files will be stored in the topic for which they were created.

Summary

With the addition of the new top-level directories, and the shifting of file-type directories to topics, the new hierarchy looks like this:


Restoring iTunes Backups


In Backing Up iTunes Purchases I talked about how I made DVD backups of a portion of my iTunes library. Now that I have a significantly larger hard drive (320 GB versus 100 GB) it’s time to put the restore feature to the test.

After an initial scare when I couldn’t find the backup where I thought it should be, everything seems to be working swimmingly. My recollection was that I had backup my iTunes purchases to one of my external backup drives, however I wasn’t able to find the material on either of those disks. For a few minutes I wondered if the backup had been there only to have been overlain with a subsequent full-drive backup. Then it occurred to me to re-read the posting I made, maybe I had left my self a clue.

Turns out the backup wants to be to CD or DVD. Armed with that knowledge it was relatively short work to find the five DVDs I made in May 2008.

Restoring from the DVDs is simple. Insert the disk into the computer and iTunes asks if you want to restore. There’s an option to overwrite existing material, but otherwise you just click restore and sit back while the process happens. It could not be simpler.


Initial Software


The following is a list of the software I installed on the new MacBook Pro in the first 24 hours.

There are some tools (VirtualBox being a good example) that insist on installing themselves in the root application folder, and I am content to allow that to happen. It means my /Applications folder won’t be pristine, with only Apple provided applications, but it will be much cleaner than the hodge-podge of nearly 300 applications my old machine had collected.

In addition to these installs I also imported 8,652 pictures via FireWire from the PowerBook. I also imported 7,081 MP3 files to the new machine. I setup Mail to read my IMAP Gmail account, and I imported the latest Address Book backup, via Dropbox, to enable email and chat.

The new machine is amazingly fast and responsive - importing pictures from the digital camera was astonishing. The whole process was over in less time than it takes the PowerBook to recognize that a camera is attached and to prepare the import. It also synchronized my 30,000 or so email messages relatively quickly, having more than an 802.11b wireless card makes a huge difference.

There are still a couple of outlier programs to install, and some development tools to download, build, and install, but by and large the machine is ready for 95% of my daily use. Now I need to focus on the 50 GB of documents, spreadsheets and other data files on the PowerBook to determine what to migrate, and how to better organize it on the MacBook Pro.


Not Since February 2003


Not since February 2003 (maybe late January of that year) have I gone to work without carrying my PowerBook G4. It has been the ultimate filofax, holding pictures, music, words, code, and all the other electronic flotsam and jetsam near and dear to me.

Tomorrow will be the first day I’ve ventured off to work without it.

Of course tomorrow will also be the first (full) day that I carry the new MacBook Pro to work.

Not a bad trade all things considered. Still, there is a feeling of nostalgia, of loss tonight. Macs were thin on the ground in central Illinois 7 years ago. My PowerBook was unique and drew attention as a result. Even in 2004 when I took it to work in Kansas City it was different enough to attract some attention. Working for the University I am surrounded by MacBooks, MacBook Airs, MacBook Pros, and Mac Pros.

Seven years ago having a PowerBook and taking it with me nearly everywhere I went made me stand out in the crowd a little bit. It was an introverts way of being different. While I may not stand out in the crowd of Mac fans at the University, I am pleased to be one of the Mac-toting crowd there.


Welcome to the BlackPerl


My new MacBook Pro was delivered this morning, less then two-and-a-half days after it shipped from Shanghai China. It is stunning and sleek, simply beautiful. I am very pleased with my choice.

As some of you might have guessed from the title of this posting, I’ve named the new machine BlackPerl. I thoroughly enjoyed the movie Pirates of the Caribbean: The Curse of the Black Pearl, and particularly liked the bits about the pirate’s code being more guidelines than rules. Anytime now when I hear the work guidelines used, mentally I say to myself, “They’re more like guidelines, welcome to the Black Pearl.” Typing the word pearl is difficult as I tend to spell it perl, which is the name of a powerful computer programming language. Sibylle pointed out that I was tending to spell Black Pearl, Black Perl, and the name stuck. I like the irony and the pun.

Migrating data, installing applications, and generally configuring the new laptop is now the order of the day. Probably the next several days.

Unboxing pictures will be up later this evening on Flickr, and over the next few days I’ll have postings about setting the machine, and some performance comparisons with my venerable PowerBook. (By the way, my Powerbook has been on continuously for 40 days and 22 hours as I write this; not bad for a 7-year old computer that travels everyday.)


On Naming Computers


In Nerd Angst I mentioned briefly that I need to think up a suitable name for the new laptop due to arrive sometime Friday. Being a nerd through-and-through this is actually a more significant undertaking that you might otherwise suspect.

Until I got my PowerBook I had never named a computer. Upon seeing it and using it for the first time I immediately thought it should be called Eeyore. The Winne-the-Pooh books are still among my favorites more than four decades after my parents first read them to me. I always identified with Eeyore, and while a sleek new Apple laptop is hardly a donkey with a tacked tail, the name fit.

With the beginnings of a theme at work here my subsequent computers and external hard drives have been named from the Winne-the-Pooh universe. I’ve got a USB key named Piglet, and two older machines called Kanga and Roo. The IBM ThinkPad I bought 4 years ago is called Tigger. For a time the iMac was called 100AcreWoods, but names that start with numbers don’t always play well with others, so it was renamed Rivendell.

The problem with the Pooh universe is the rather limited number of character names. The Tolkien universe on the other hand is littered with names. Unfortunately many of them are hard to spell and harder still to pronounce. I actually use Tolkien names for my work supplied desktop and laptop, Palantir and Orthanc respectively. (Yes, it would make more sense to name the desktop tower after  the tower and the portable laptop after the far-seeing stone, but I wasn’t thinking clearly when I choose the names. Renaming isn’t easy as there are static IP addressing issues involved.)

I read once about using the Periodic Table of the Elements as computer names, and leveraging the atomic number as the final octet of the IP address. But that would mean moving away from DHCP assigned IP addressing on our home network, and renaming computers here to take full advantage of the system.

One former employer used rivers (e.g., Orinoco and Danube) for development servers and planets for production servers (e.g., Mars and Jupiter). My current employer uses Jedi character names from Wookiepedia for servers. Although they are moving away from that creative scheme and toward one like dev-ome-as1. Which is functional but not really what I’m looking for.

The Internet Engineering Task Force (IETF) uses Request for Comments (a form of memo) to capture ideas and standards. There is a RFC on computer names, that offers many guidelines but no real answers.

(Every time I mention guidelines I am reminded of the first Pirate movie, and the phrase, “Welcome to the Black Pearl.” Hmmm….)

I’ve got until Friday to solve this mental exercise. Well, actually longer than that as the name doesn’t have to be set immediately. Decisions, decisions, decisions.


Timing Is Everything


If I were to coin a phrase about buy consumer electronics it would be, “don’t watch the price after completing your purchase.” Usually, over time, the price of consumer electronics goes down, or, if the item is already at its lowest sustainable price, the features or extras go up for the same price.

Personal computers, both desktop and laptop, have razor thin margins. Manufacturer suggested list prices barely cover the cost of components and assembly. Prices may drop a little once in a while, but more often than not, speedier processors or more memory or larger hard drives are made available for the same price point. All of which means there’s no good time to buy a new laptop if you are going to be upset by a newer, sleeker, faster one being released after your purchase. Whether it is six months later or six days later, it is going to happen.

Buying a new computer also has a bit of the popular “washed the car yesterday so it is raining today” cause and effect pattern. On Saturday I purchased a new computer, so naturally the Apple Store is currently “down” today while it is being updated. Rumors indicate new plastic MacBooks, new iMac minis, new iMacs, and maybe a surprise. I know that the MacBook Pro I just bought will sooner (rather than later) be upgraded. I’m okay with that.

I just hope that it’s not today.


Nerd Angst


The new MacBook Pro has been ordered. It’s status is still “not yet shipped” more than 48 hours after the order was completed, apparently ordering on a Saturday has disadvantages. However, I can make good use of the waiting time by preparing for how to transferring nearly 90 GB of data and applications from my PowerBook to the new MacBook Pro.

Options

Apple provides a migration assistant that will let you transfer user accounts, files, folders, applications, and your Library folder. Moving all my stuff in one fell swoop would be the easiest in terms of decision making for me, however it would move the cruft as well as the cream to the new hard drive. For example, there’s stuff on the PowerBook from my last full-time Windows machine, circa 2001. Plus I tend to download and experiment with all kinds of applications, many of which fall into disuse quickly. There’s no real need to move all of that clutter to a new machine.

Option two would be to start using the MacBook Pro and only install applications as I use them, or data only when I need it. Doing it incrementally would minimize the amount of cruft that was blindly copied to the new machine, but it would stretch out the transition period considerably.

The third option would be to identify those programs I use on a daily basis, and those files/folders I want immediately, and transfer those in one marathon session to get things up and running. Then over time as I discover infrequently used applications or forgotten but still necessary data, transfer them.

A Plan Emerges

Web Browsing and Email

After creating a user account on the new machine (need to think up a good name for the machine), use Dropbox to transfer my Safari bookmarks and address book entries. Configure Apple Mail to read my Gmail accounts via IMAP, and my domain accounts. With 35,000 messages in GMail this will synchronize for a long time.

Music and Pictures

Many months ago, when I was running out of room on the current 100 GB hard drive I backed up all the television shows I had purchased from iTunes. I am very tempted to copy my iTunes library across as is, import the back up and call it good. I am also tempted to re-rip my music collection, paying more attention to bit rates, and complete tags. I have a second copy of my music on my work computer, which is where I do most of my listening, so a delay here wouldn’t be the end of the world. One thing I do need to figure out is how to synchronize my 30 GB iPod with the new laptop.

iPhoto is a great tool for my pictures, but the 867 MHz processor in my old PowerBook is overwhelmed by my relatively small 4000 image library. I am looking forward to transferring to a new, faster machine, and to a newer version of iPhoto.

With pictures in place, and music either copied directly across or being slowly re-ripped it’ll be time for third-party applications.

Chat and Twitter

I am a huge fan of Adium, so this software will be installed right away. I’ve been running the nightly build for several months now with no problems at all. My current favorite message theme is “Twitterrific” with the Geeky list layout. And I use the “iPhone Dark Mini” message theme; I’ll have to download and install both of these extras along with Adium.

I’ve been a staunch Twitterrific user since it was released. My only complaint is a lack of multiple computer synchronization support. TweetDeck and Tweetie provide options for this (built-in and via Dropbox respectively) so I may switch for a while and see how I like the competition.

Quicksilver, Google QuickSearchBox, and Spotlight

Quicksilver has been an integral part of my Mac experience for as long as it has been available. I’ve tried Spotlight and Google’s QuickSearchBox as replacements on a couple of occasions, mostly since development on Quicksilver has ceased. I haven’t decided if I’ll use the new machine as an excuse to leave Quicksilver once and for all or not. Several years of ingrained user experience expectation has a lot of inertia to overcome, new machine or not.

Editors and Development

TextMate is my all-time favorite editor so it will be quickly installed on the new machine. Along with the PlasticCodeWrap theme. Since I’m making more use of Mercurial for source version control I’ll be adding the hg bundle as well.

Some of my code is still housed in a local Subversion repository, so that’ll be transferred next.

I make good use of MAMP for local testing of web sites, so a fresh install of that come next. Once MAMP is up and running I’ll need to download and install WordPress and establish its database in MySQL.

My /usr/local directory is filled with all sorts of software goodness. Junit, Maven, Subversion, Ant, JBoss, Scala, et cetera. These are all strong candidates for the wait and transfer as needed category.

Office Productivity

When I purchased the PowerBook, and its sibling 17-inch iMac G4, I bought a license to Microsoft Office for OS X. Coming from a Windows environment, and being a heavy user of Word and Excel, I wanted to have those tools available on the (then) new platform. I still use Word and Excel these days, but I also use Google Docs and OpenOffice for word processing, spreadsheets and presentations. Rather than continue with the now seven-year-old copy of Office, or springing for a new copy, I am planning on installing OpenOffice and existing in the open source world for my office productivity tools.

Virtualization

Believe it or not my aging PowerBook was capable of virtualization. I’ve had a copy of VirtualPC running a Windows 2000 guest OS for several years. It’s slow and cumbersome, but it does work. On my work-supplied Mac Pro and MacBook Pro I run both VMWare Fusion and VirtualBox. For my personal use at home I am planning on using VirtualBox.

Summary

This initial batch of transfers and installations will enable my email, chat, and twitter communication channels, my web browsing favorites, programming editor, and interface enhancements. I’ll also have my pictures and either a plan for re-ripping music, or the music itself. I’ll be able to install a guest OS or two through virtualization, and I’ll have word processing and spreadsheet support. Should be enough to get me through a day or two.


PowerBook G4 vs. MacBook Pro


For the past seven years my primary day-to-day computer has been a Titanium PowerBook G4. In a word this laptop has been outstanding. Except for a faulty CD drive early in its life, I have had no mechanical difficulty with this machine. I replaced the original hard drive with a new 100 GB model about 4 years ago; the original 40 GB drive has continued to perform faithful service in a USB/FireWire enclosure. I bumped the memory up from its original 256 MB to the maximum 1 GB. I’m still using the original battery, and the AirCard I added weeks after getting the machine. In short it has been a phenomenal machine, well worth every penny of it’s $2500 cost.

This morning I purchased a new 15" MacBook Pro, a uni-body model with considerably more oomph in the CPU cycles, RAM, and battery life areas. Oh, and the ability to run the newest operating system, Snow Leopard. There will no doubt be several entries about the new machine once it arrives, but I wanted to do a straight up technical specification comparison now, just for grins.

Technical Specifications

Solid-state trackpad provides precise cursor control; supports tap, double-tap, and drag capabilities

Multi-Touch trackpad for precise cursor control; supports two-finger scrolling, pinch, rotate, swipe, three-finger swipe, four-finger swipe, tap, double-tap, and drag capabilities


Synchronize Apple AddressBook Using Dropbox


Proceed with this how-to with caution and a good backup…

In order to synchronize your Apple AddressBook using Dropbox you need to have Dropbox installed on all the computers you wish to sync, and you need to have some comfort running commands in Terminal. This synchronization is relatively easy to accomplish, but missing a step or mistyping a command could leave you without ready access to your address book. Make backups and follow the steps carefully and you should be okay.

First Step

Get a copy of Dropbox if you haven’t already. It’s an amazingly useful tool that gives you 2 GB of storage on the “cloud” for free. With clients for Linux, Windows, and Mac operating systems, I use it every day. The installation and setup is well explained on the Dropbox site so I won’t cover that here.

Second Step

With Dropbox installed on each machine you wish to sync you need to prepare and backup your AddressBook. To prepare my Address Book I combined all my addresses into a single address book. This combined address book is the one I pushed to Dropbox. But before that I made backups of all my address books. Under File | Export you’ll find the Address Book Archive… option.

Third Step

Once your backups are completed, you are ready to move the combined Address Book to Dropbox. To do this you will use the Terminal application located in the Applications | Utilities folder. Make sure that you have quit Address Book before doing these commands. Also, these commands assume your Dropbox is located in your home folder. If you placed it elsewhere, you’ll have to adjust the Dropbox path accordingly.

$ mv ~/Library/Application\ Support/AddressBook ~/Dropbox/
This will move your AddressBook directory to Dropbox. Next create a symbolic link on your machine to point to the new location for the AddressBook directory. Like this:
$ ln -s ~/Dropbox/AddressBook ~/Library/Application\ Support/AddressBook
Now when you start AddressBook it will use the symbolic link in ~/Library/Application \Support to find the actual AddressBook in ~/Dropbox. Start AddressBook to make sure you’ve typed everything correctly to this point.

Forth Step

With AddressBook working on the first machine it is time to setup the other machine or machines. Since you combined the AddressBook already, all you need to do is move the AddressBook directory from ~/Library/Application \Support and add a symbolic link to the Dropbox location. Again use the Terminal to issue these commands:

$ cd
$ mkdir backups
$ mv ~/Library/Application\ Support/AddressBook/ ~/backups
$ ln -s ~/Dropbox/AddressBook/ ~/Library/Application\ Support/AddressBook
The cd moves you to your home directory. mkdir backups creates a backups directory. mv ~/Library/Application \Support/AddressBook/ ~/backups moves the existing AddressBook directory to the backups directory. This is perhaps safer than outright deleting the directory. The ln command creates the symbolic link necessary for the Address Book application to find the data now located in Dropbox.

Repeat the Forth step for each machine you want to share your Address Book with.

Caveat

Address Book wasn’t written with multiple copies accessing a single, shared set of data in mind. I haven’t experimented with having the Address Book application open on more than one machine at the same time. I would expect this to cause errors and possible data corruption. I rarely have Address Book open at all, usually only to update an address or add or remove people. If your usage paradigm is different than mine and you have Address Book open all the time, you may need to be prepared to restore from the backups you made.