Categories
Personal Computing

Making Good Passwords for the Rest of Us

Every week brings stories of a new software exploit or corporate data breach in which user names and passwords are stolen. Security “experts” are advising us to make ever longer and more complex softwoods and to use unique passwords for each site that we visit. Why is this good advice and how do we go about making good but usable passwords? And how do we remember them all? I’ll attempt to answer these questions in this article in a way that is accessible to my retired buddies and family.

Why a User Name

There are lots of reasons to have a user name and password at sites that you visit regularly. Some benefit you while others benefit the site operator in a way that allows them to continue to provide the site’s service to you.

  • The site can provide personalized service
  • Some services are provided only to authenticated users having a standing business relationship with the organization (like your bank)
  • The site accumulates information that allows it to provide better services to you.

Some of these things can be done with or without an E-mail address using your home’s IP address as a substitute. Without an account, the site has no way of knowing which of several users at an address is actually visiting: you, your spouse, your 13 year old son? Without a sign-in, the site has to make somewhat general decisions about what to show you.

Why a Unique Password?

A password is a secret shared between you and the web site. When the user name and password pair are unique to the site, successful presentation of the user name and the associated password verifies you to the site and the site to you. You’ve shared that particular secret with only one web site. You each know that the intended party is at the other end of the line.

If login fails, you may have miss-typed the URL. Double check the URL before doing missing password procedures. It is not uncommon for unscrupulous operators to attempt to collect user name and password pairs by impersonating a site on a common misspelling of the URL.

Why a “Complex” Password?

The primary reason to use a complex password is to avoid well-known passwords or passwords composed from information known about you that an impersonator can obtain. This basically prevents an unscrupulous unskilled individual from committing theft of service or tampering with your relationship with the various sites that you visit. It is not intended to protect you from an organized and systematic attack.

Intelligence agencies and criminal syndicates make sophisticated attacks to break into web sites. One thing they try is to use statistically common passwords like “password12”. And surprisingly, they can make all the mess-ups like “pa$$word12”, “pas$word12”, etc. Most sites attempt to protect accounts against password guessing using several techniques.

  • Limiting the number of failed log in attempts in a time period
  • Increasing the delay between log in opportunities
  • Locking the account and requiring use of lost password procedures which involve different shared secrets.

How Passwords are Stored

Reputable web sites do not store your password. Rather they store the results of operating on your user name and password and possibly some well-known (to the site, anyway) other stuff to compute a hashed value. The addition of other stuff is called salting the hash or just salt for short. A hash function is a function that maps a string of data into another pseudo-random string. It is easy to compute the hash but prohibitively expensive or impossible to retrieve the original string from the hash because the hash function makes a many to 1 mapping. The hash is useful because the inputs that map to a given output are wildly different. No recognizable variation of the input string will give the same result as the correct string .

As a result, Google can not tell you your G-mail password. Google only has the hash. It is likely that the hash input and hash algorithm are designed to give different results when a common password is used with multiple Google accounts. Compromise of one account does not imply compromise of other accounts. Others may not be so clever or careful as Google.

What is taken in a break-in?

In many break-ins, the attackers gain the password database which is basically a list of user names with their hashed passwords. Many sites, especially entertainment sites, use a well known authentication process. Each such site produces the same hashed value from your E-mail address and password.

It is common for attackers to sell lists of user name password hash pairs. Today it is possible to break the hash to recover the original password. In other cases, the password list is stored in the clear and lists of user names with passwords are also available. Availability of these lists allow others to compromise your account to steal from you or to impersonate you.

What do I do?

To limit the consequences of password compromise and to authenticate my bank and broker’s web sites, I do the following.

  1. I use unique passwords for each site
  2. I use a password manager to store all my passwords
  3. I use a password manager that syncs password data among computer, phone, and tablet
  4. I chose the password manager carefully.

Making Unique Passwords

It is hard to make up good passwords. Choosing words myself generally results in using words associated with me, my interests, or my experiences. They’re not really random. Same with numbers. They’re usually the last 4 of an ancient phone number.

Instead, I use dice ware to make good but easy to type passwords whose parts are chosen randomly. Dice ware is a word list used with a dice cup and 5 dice. Do use real dice as computer random number generators are “pseudo-random”. That is, started with a seed, the random number generator will always make the same sequence of numbers. Which is to say, that they’re not random, they only appear to be. You can’t guess the next one given this one but you can reset the seed and recreate the sequence.

For web sites, I use three rolls to pick two strings (usually words) and a number. Each roll has 6 to the 5th power outcomes that are independent so there are 6 to the 15th power outcomes. Two rolls select a word from the word list. The advantage of the dice ware technique is that two words and a number stick in short term memory long enough to allow them to be typed easily yet the search space is big and fairly random. And you will come to remember those you use frequently. And there is nothing about the passwords that suggests you are using dice ware to make them. The dice ware word lists are available for a number of languages in addition to English that use the Latin alphabet.

Most sites will hold a password made this way. The troublesome sites are those that have a high complexity requirement but short string length. It is difficult to produce easily remembered 8 character passwords that have 2 digits, 2 punctuation, and one or more caps. You’re down to 4 letter words or going random. Also troublesome are sites that don’t tell you the maximum password length. These sites are truncating your password so the numbers, caps and special characters can be lost if they’re near the end.

Remembering 200 Passwords

I can’t do it. I don’t think computer security expert and ace cryptographer Bruce Schneirer can do it. So I use the OS X/IOS built in password manager and commercial product 1Password. The built in pass word manager works in the web browser and stores passwords in the OS X key chain. It can also store your SSH keys (for geeks) as of Mavericks and is synchronized via iCloud as of IOS 7.

I keep everything in 1Password also because I can use 1Password to keep track of security questions and responses and other information about the site and my relationship with the site that Key Chain will not store. As of IOS 8, applications will be able to ask 1Password for data. Agile Bits explains this interface and the actions they’ve taken to prevent misuse in their blog.

1Password never gives anything up without you authenticating using your 1Password master pass phrase. I have a good one that I can remember that I made using Dice Ware. It is guess proof.

The folks at 1Password understand cryptography and know how to build secure cryptographic applications. All 1Password data is protected using AES256 encryption and care is taken that the plain text and cypher text are not left lying around in memory.

Categories
Personal Computing Retired Live Web hacking

Second Life, Web Hacking Edition

To keep busy, I’ve been doing web sites for two non-profits, my church and the local Road Scholar Lifetime Learning Institute Network affiliate sponsored by Old Dominion University.  Both web sites were in need of updates for the brave new world of iPhone and iPad. Neither site was responsive and both had become disorganized as the sponsor’s activities grew in scale and complexity.

Categories
Personal Computing

New Life for an Old (early 2009) Mac Mini

My beloved Oswald (named after Nick’s grand sire) was getting as slow as his deceased name sake. The internal disk was failing, boot and shutdown times were long, and the machine was getting unstable. Time for a new iMac? Being a retired moocher, the thought of parting with $2500 while totally out of pocket was a bit unsettling. What could I do with an overhaul?

The Symptoms

The machine’s symptoms were

  • Dying in its sleep. I’d find the forbidden icon up on a gray background
  • Slow to log in
  • Slow to log out
  • Programs like Aperture ran slowly
  • Machine was not CPU bound
  • Machine was not swapping
  • Disk I/O looked reasonable. Most things read, modify in memory, then write.

Initial Investigation

  1. Review syslog using Console.app. Nothing scary. No panics called, no device errors for disks mentioned.
  2. Reinstall Mavericks. This helped for a while
  3. Check /Library/LaunchAgents and /Library/LaunchDaemons. They were full of crap from 12 years of Mac OS X updates and retired software. Clean these out.
  4. Do a general clean up using Clean My Mac 2. Remove broken startup items and broken preferences. There were some.
  5. Run About This Mac and check the kernel extensions. I found some from PPC days and the OS was actually trying to load one.
  6. Check and remove all KEXT’s older than Intel only OS X, say 2009. Remove all that were PPC only.

At this point the machine was somewhat improved. At least log in and log out were moving nicely. But the machine died in its sleep a week later.

On to Hardware

Now that the system was cleaned up, was the hardware old, ailing, or failing? Time for a visit to the Genius Bar.

I took the machine and power supply to the local Genius Bar at the MacArthur Mall Apple Store. After a few minutes to review the symptoms and my corrective actions, the Genius rounded up a monitor and keyboard and began a quick inspection. Once complete, he recommended running diagnostics. The disk phase quickly found a failing Hitachi Death Star disk. Apple could only put a disk like the original back in. Apple business rules did not allow Apple to make an alteration equivalent to repair. So I elected to reinstall Mavericks at the Genius Bar and restore the disk from Time Machine upon my return home.

On the way out, I launched a few things on the Mac Pro. Blinding fast. What’s in that sucker? About this Mac found a 256 GB SSD. Ah Ha! What can I do?

Alternative Courses of Action

While Time Machine was chugging, about 8 hours for 1/4 TB to restore, I did some research.

  • How hard was it to replace a disk? Not very.
  • How hard was it to reinstall and restore? Been there, done that, got the tee shirt!
  • Could I increase the memory? Yes, from 4 GB to 8 GB if the last firmware update had been installed. It was.
  • Could I put an SSD in? Yes.
  • Whose SSD?

SSD Selection

After some reading, I concluded that Samsung and Crucial were the go-to SSD suppliers. Both made their own flash and Samsung made its own controllers. Crucial was using recent Marvel controllers that were well regarded.

Could I get the memory and SSD from the same source? Maybe. Who?

  • Amazon did not have a good memory advisor AI so I ruled them out.
  • Samsung did not have a good memory advisor so I ruled them out.
  • Tiger Direct and NewEgg? They did not have Mac savvy memory  advisors so I ruled them out.
  • Crucial has supplied memory upgrades in the past and had a good Mac memory advisor. Did they also have a good SSD? The consensus of Ars, Toms’s Hardware, and AnandTech was that Crucial’s M550 was in the hunt.

So, I ordered 8 GB of expansion memory, and a 512 GB M550 laptop form factor eSATA 3 SSD. The SSD included a 9 MM spacer that would be needed in the Mac Mini. I also ordered Crucial’s Apple tools which included a spudger and small screw drivers.

Installation

Crucial was a bit back ordered so it was 10 days waiting for parts to come. Oswald took another header so I put an OS image on my media Drobo Gen2 to limp along while waiting for parts.

Parts arrived in Tuesday’s evening UPS run so I elected wisely to do the installation Wednesday morning.

 

  1. Are you satisfied with your backup? No. Run Time Machine and be sure things are up to date. They weren’t so I kicked that off around noon on Tuesday. Note which TM volume of three had the fresh backup.
  2. TM1 was mounted read only. Why? Run  Disk Utility to repair the disk. Nothing was wrong but it was 12 hours to find that out. Better safe than sorry.
  3. Does a recovery partition boot and run? Yes, from thumb drive made using the recovery disk tool from the App store, and also the recovery partition on the external media disk.
  4. Clean up and draw file an edge on a putty knife as described at iFixit.
  5. Do a normal shutdown before breakfast on Wednesday.
  6. After breakfast do the replacement following OWC’s 2009 Mac Mini disk replacement video.

OWC advises that the replacement is easy but not so easy. As to be expected, I found out why.

  • Getting the old disk out and the new one in looks easy when you watch an experienced tech do it. In practice, there are some sticky bits
  • Getting the drive tabs into the riser socket is tricky because there are no guides for the drive body. But it can be done with patience.
  • Getting the drive carrier tab into the mother board connector is a bit tricky. It took me 3 tries.
  • Seating the ribbon cable on the disk connector is a bit tricky. It needs a good push.
  • Replacing the memory was trivial. No skinned knuckles like desk top memory transplanting produces.

Once all was back together (well, almost all, one screw went missing), I fired the machine up. No happy chord. I let the machine boot. No internal disk. Three checks to find all the stuff mentioned above. Then the lost chord was back.

System Installation

Mac OS X installation goes like this.

  1. Start from the thumb drive (Alt/Opt down while booting until the drop down box shows).
  2. Start disk utility and partition the SSD. One 64GB Win81 partition and the balance to OS X HFS+ Journaled. ESPlanner brought the camel into the tent. Frown!
  3. Connect the Time Machine Drobo and restore the system disk from Time Machine. This took 8 hours for 1/4 TB of data.
  4. When Time Machine completes, the machine restarts.
  5. Complete the setup wizard.

Other than being agonizingly slow, the whole process was without drama. Only a bit of futzing around to get connectors seated.

The Results

For $500 and a day of BS&T, I have a new machine that is quick to boot, quick to log in, and pleasant to use. Even Aperture launch and Aperture import, both painful, are reasonably quick. This without making a working Aperture library on the SSD. Aperture is quick enough that there is no need to make a working library in addition to the archival library on the Drobo. Even image correction, which was slow before, is reasonably quick. Here’s why.

  • 4GB of memory was too little although nothing appeared to be swapped. My normal workload shows about 4.5 GB of App memory so stuff that was paging is no longer paging.
  • There is about 3GB of buffer cache. Enough said.
  • The SSD eliminates seek latency and rotational latency. Apps load much more quickly because they page in without mechanical waits.

Why the slow logins?

Just what were those LaunchAgents and LaunchDaemons? Would  you believe

  • A Google daemon to enhance the user experience?
  • An Adobe daemon to find the latest screwed up version of Flash?
  • An Oracle daemon to find the latest Java vulnerabilities?

Any or all of these were ill behaved. They’re worm food now. And the machine is happy. And Google’s helper is not missed.

Categories
Personal Computing

New Windows, SOS

Retired life is a zero sum game for now. For the first five years, I’ve chosen to live completely out of pocket by delaying the start of Social Security payments until age 70. To keep my retirement finances on track, I use ESPlanner to estimate my annual discretionary spending, See Finance to track expenses, and TurboTax to do my income taxes.

ESPlanner is a Windows program and I keep a Windows Free Household. Well, something had to give and I let the Windows camel into the tent but keep it in a corral using Parallels Desktop. This article describes my initial experiences with Parallels Desktop and Windows 8. Windows 8 is not your father’s Windows but it is awfully familiar in all the bad old ways.

Parallels Desktop

Parallels Desktop is a low administration version of the Parallels virtualization product designed for use by mere end users like me. Although I’ve been in the industry for over 30 years, I consider myself an end user when it comes to virtual machines and products so Parallels Desktop is my kind of product.

Terminology

Guest      An operating system running in a Parallels Desktop virtual machine

Host        The operating system running directly on the hardware, in this case, OS X.

Hypervisor            The part of Parallels Desktop that mediates between the guest operating system and the underlying hardware.

Virtual Machine     A simulated computer provided by Parallels Desktop using the Intel virtual machine facilities.

Installation

Parallels Desktop installs from a down loaded disk image. The retail box gets you a license key that you enter once installation is complete. The installation process is the one Mac OS X users know well. Just run the installer, let it verify that the host environment satisfies the preconditions, and then do the install. The product installs as a normal application bundle into /Applications. No surprises here.

Gest Installation

Parallel’s guest installation is straight forward. Parallels Desktop walks you through creating a virtual disk, starting the virtual machine, and loading and starting the Windows 8.1 installer. From there, it is the Windows installation experience  you know and love. Walk through the installation wizard answering its questions, let her rip, let the VM restart, and let Windows get itself caught up to date. As you remember, Windows will do an update download and a mandatory restart. That mandatory restart can be put off a couple of times but sooner or later, Windows 8 will insist on restarting. Might as well save yourself pain and get it over with.

Windows 8.1

When you are shopping for Windows, do pay extra for the standard Home Edition new system version. This version is somewhat more permissive in that it will allow you to make installations on a small number of virgin disks on the local subnet without requiring an earlier product to be present.

The System Builder edition lets you make one installation. Subsequent installations require contacting Microsoft to have them deauthorize the earlier versions. System Builder considers an installation to be subsequent if any part of the hardware has changed including the disk. Based on my reading, if you mess up a virtual machine, it’s likely you will need to call Microsoft and ask mother, may I to create a new instance of Windows. 

So, if you are an infrequent Microsoft customer, go to Best Buy or some such and confirm that you’re buying the correct version. And for their help, give them some love. You’ll probably not find the home/family version at Amazon or NewEgg. (I couldn’t but then again, Microsoft product jargon is mind numbing).

Remember the classic Steve Jobs jab, “Home Edition, 29.95. Business Edition, 29.95, Galactic Edition, 29.95”

Windows first impressions

Windows 8 is better than its predecessors in many ways.

  • The stack is execute disabled — this closes many buffer overflow attacks.
  • The heap is execute disabled — this closes many buffer overflow attacks.
  • Things work correctly for users that are not the administrator.

But Windows 8 hides many things. The UI has been reorganized around touch screens and touch gestures. Some touch gestures have mouse equivalents but they are not thought out in the same way as in OS X. And Windows is unclear about which gestures are mouse and touch screen and which are touch screen only. And it fails to explain the mousing technique for most mouse gestures. You just have to futz around until something useful happens.

Summoning the charms

One particularly frustrating thing in Parallels Desktop is summoning the Charrs. The charms are UI dingbats that let you search, see the list of programs like Launch Pad, and do some other common actions. Moving the mouse to the upper right corner is supposed to make them appear. Unfortunately, OS X gets first dibs on mouse events and the charms don’t appear.

The administrator user

The installer gives administrator rights to the first user created. As with earlier versions of Windows, the user holds these rights continually but unlike XP, the various system administration operations will ask for confirmation. On first launch of a downloaded image, Windows will ask if you really want to run this random thing from childporn.xxx. So it is a bit harder for things to be installed behind your back. But I don’t trust Redmond to get it right.

Just a user user

So the first thing I did after all the initial updating and restarting was over was to create a second user dave with regular user rights for every day use. This gives me another layer of insulation from acts of malware. Before performing administrative actions, Windows will tell me that I’ve initiated an administrative action and will ask for the administrator password. Not as elegant as sudo but an improvement over XP. So you give the admin password and you will be asked for additional confirmations for each admin action. So it is harder to be had than in the past.

Should something sneak by, running as dave prevents a process from touching the system files. Important system files are writable only by the administrative user so a process holding user id dave can’t alter them or install executables in Program Files, etc. Just a bit safer.

Active X

I guess Active X is still around and but less able to commit mayhem. Any Active X widget will be running as user dave with dave’s object access rights. Any Active X thing asking for administrator rights will be outed and I can kill it with extreme prejudice.

As a rule, I do everything I can in OS X where Apple and BSD sandboxing are in effect. The BSD Jails are pretty effective at keeping things out of mischief and I have OS X set up only to run signed executables built by developers who have purchased signing keys from Apple. This stops a lot of malware but $100 is chump change for a pro black hat. But, get caught and Apple kills your keys.

I don’t know if Microsoft is doing the same with signing of images, but the new versions are much more robust than the prior Microsoft art. The attack surface is still pretty large so Parallels Desktop provides another layer of containment. But Parallels can be exploited. Again, keep the attack surface small. I’m pretty much keeping this Windows instance stock.

Parallels Tools

Parallels Tools allow the guest to create native windows and to see a chroot subset of the file system. Once a guest process is launched, you can pretty much ignore the guest and interact with the user process in a regular Aqua window. And keep the data in the shared file system branch where the files are visible to Time Machine for backup.

Unfortunately, there is no Parallels Tool to summon the charms.

Categories
Personal Computing

Mavericks, Drobo, and More

For the past several months, I’d had a full volume on my Gen 2 Drobo. What’s a Drobo? It is a Firewire/USB external disk drive designed for use by small offices and professionals. Drobo is popular with photographers, musicians, video editors and such as an external storage device because it has some interesting properties.

  • It is disk failure tolerant
  • It is easy to expand
  • Failure recovery is easy
  • You don’t have to be a professional storage admin to manage it

I’ve had this device since the Fall of 2009 when I purchased my current Intel Mac Mini. For the past several months, the system had been giving me fits with slow performance and unfinished disk backups. Being newly retired, I had the time to look into these problems in depth.

Some History

I’ve been a Mac OS X (say ten) user since 2002 when I purchased my dualie G4 mirror door machine. I’ve been migrating stuff forward through 8 OS X updates and a hardware swap so things had gotten crufty with abandoned executables, senseless start up items, etc, passe widgets, etc. Time for a house cleaning.

Second, in 2012, living in the east coast hurricane alley, I decided off site disk backup was good and began using Carbonite for this. In 2013, I became curious about Pogoplug and began using it too. Thus, I had 2 off-site backup daemons churning the file system.

In June 2013, the Time Machine volume became full in a way that caused it to mount Read Only. The drive reported write protected status to Time Machine rather than end of medium status confusing Time Machine’s end of medium logic so it just sat there and told syslog it was in trouble. No really clear user messages

Summer of 2013 also saw the Snowden Disclosures about NSA agressively trying to spy on all Internet traffic, FISA, Patriot 2, etc called to our attention. I decided to discontinue off-site backup to minimize exposure to acts of my government.

Winter Cleaning

I went through the start up items using Clean My Mac and disabled those not being used. Once comfortable that these changes are safe, I’ll use Clean My Mac to delete the old items. Clean My Mac knows what can be bushwacked and what must be kept around. For example, it won’t propose deleting or disabling the OS X UI server (a really bad thing).

I cleaned up old stuff from my TIvO days, iSTAT menus days, etc. Anything not essential to normal operation that I was able to identify as a 3 rd party add-in became history. Carbonite gone, PogoPlug gone, Quicksilver gone, ISTAT menus gone, SMART monitor gone. Amazingly, with the the third party crud gone and the external disks having free space again, life was good.

How Time Machine works

Time Machine is a really cool Apple OS X system service that has been around for several years. Time Machine is designed to back up a UNIX file system to external USB/Firewire disk storage. It doesn’t do tape because appropriate tape drives are $5000 devices that require SCSI interfaces not found on personal computers. The way TIme Machine works is to maintain a current virtual image of the file system on the external drive by making  a baseline backup plus hourly changes. As the program runs, it consolidates the hourlys into daily snapshots, weekly snapshots, and monthly snapshots.

The way it does this magic is to make a copy of each unique version of a file to disk hourly. Files that don’t change are written just once. Files you are working on are written hourly. Say you are writing a letter and do the following.

  1. Save the template
  2. Edit the template to make something useful
  3. Save the draft
  4. Print the draft to preview for proof reading
  5. Edit the draft
  6. Save the draft
  7. Print for mailing

Depending on how this activity spreads out over time, Time Machine will capture one or more versions of the letter, the final draft which is persistent and perhaps one of the two working copies.

How does Time Machine keep things straight? Behind the scenes, Time Machine keeps a version history for each file listing the current version and each available previous version. This history is organized for easy query by directory and date. When you enter Time Machine, it opens the directory and shows you your home directory for the current day in a Finder style window. You can poke around the current time in Finder. To move back in time, you use a slider at the right side to pick the available date and time.

Time Machine uses hard links to construct these virtual views in the Time Machine universe. This allows normal UNIX file operations to show what is available and to restore a directory or file becomes a simple UNIX copy.

More about the full volume

Drobo originally held the following disks

  1. 1TB Western Digital Caviar Green
  2. 1TB Western Digital Caviar Green
  3. 2TB Western Digital Caviar Green

Drobo proposed creating a 2 TB volume but reports 16TB total size, the maximum for this model. Drobo takes some off the top to store metadata so 2 TB is actually 1.8TB or so. Thus, to have a true 2 TB HFS+ volume requires more than 2 TB of installed disk. With the drives I had, the best Drobo could do was 1.8 TB and if I added a disk, it would give me a true 2 TB volume. The bad bit was that it went write protected rather than staying read write and returning end of medium status.

Drobo is file system aware unlike RAID which is disk block aware. Drobo tires to be smart about how it splits up file blocks across the available disks and how it creates and organizes forward error correction data for the file data. When there are two identical disks as the volume started out, Drobo mirrors. When I added the 2 TB disk, it reorganized the files and forward error correction to be spread across all 3 disks in such a way that the volume could be supported by any 2 of the disks.

Because this volume was made from 1 TB, 1 TB, and 2 TB, the Drobo tax still prevented creation of a 2 TB OS volume. About 1.8 TB were available because Drobo needed 200 GB (about 10 percent) for its internal record keeping.

Adding the third disk

In December, I ordered a 3 TB Western Digital Caviar Green disk from MacSales.com. When it came, I allowed it to warm up to room temperature and installed it. Drobo recognized the disk, filled out the 2 TB HFS+ volume and proposed creating a second smaller volume from the available extra. I took up Drobo on this offer. After a couple of days, it had done the necessary data reorganization and life was good.

The disk failure

One of the 1 TB disks failed early in the new year. Drobo became write protected again and Time Machine became unhappy. After a couple of days, Drobo had reorganized the data and forward error correction and again became read write. Magic. I didn’t have to do a thing other than leave the beast alone, a hard feat for a former Navy Nuke trained to do something when things go wrong! The something I did

  • Stop Time Machine
  • Unmount the volume
  • Tell Drobo to go to standby
  • Let Drobo do its thing unmolested

Replacing the disk

When I came back to replace the failed disk with a new Western Digital Caviar Red 3 TB disk, I recabled the drive on Fire Wire. The drive woke up on its own in healthy status. I inserted the drive in the slot of the failed drive. After a minute or so, Drobo discovered the drive and proposed creating an additional volume. I opted out of this. So here’s what I did.

  • Cabled the drive to the Fire Wire bus.
  • Added the Audio Interface to the bus as last device
  • Removed the bad drive from Slot 2
  • Installed the new 3 TB drive in Slot 2
  • Waited some
  • Started Time Machine

Time Machine is still sorting things out but has begun doing backups again. It will be writing data for a good bit of today. Once it finishes, I’ll unmount both volumes on that Drobo, restart the Drobo, and add volumes covering the new space. I should be able to create a second volume that is a full 2 TB.

What to do with the new space

Time Machine is aware of disks and slices. Time Machine is smart enough not to back up the backup volume. OS X is also aware of disks and slices and knows which volumes are on which disks and slices. Although there are new volumes out there, they should not be used for data because they are on the same disk as the Time Machine volume. That means they can’t be backed up. Death of the Time Machine disk is also death of these volumes.

Time Machine will let you add volumes to the backup pool. I believe these may be on the current Time Machine disk. So once I have Time Machine happy and the available 5.4 TB formatted, I’ll add the new storage to Time Machine’s volume pool.

Choosing Disks

Best Buy and the other local retailers have consumer grade disks on the shelf. Both WD and Seagate package disks with cables, screws, etc for retail sale as internal disks. The catches

  • Unknown handling by customers. Was one dropped? ++ungood
  • Designed for light use

I’ve always had poor luck with disks bought at retail. They seem to last a year or two before going tango uniform (toes up). So I prefer to buy from Mac Sales or Amazon who package disks properly (like Sun packaged its repair part disks). I’ve had good service with these, 4 years for the WD 1 TB that failed.

WD makes its OEM disks in several grades identified by color. The 5400 RPM disks are Caviar Red and Caviar Green branded. The Red are rated for small storage array use. The Green are rated for PC use. The difference is that the Red are designed for a bit more activity as the storage array may not spin the disks down as often as a PC will. Several things are at work here.

  • Multiple hosts can read and write to the array
  • The array can do read and heal passes over the data
  • The array is less aggressive about saving power by spinning disks down
Categories
Personal Computing

Pogoplug Cloud and Disk Backup

Introduction

This article is about disk backup as much as it is about Pogoplug and Pogoplug Cloud. Disk backup is the computing system operations practices that protect a computer system’s file system from hardware, software, and user failures like dos> format c:. So disk backup is a combination of things

  • A saved image of the file system state
  • The media that stores that image
  • The device and programs that read and write that media
  • The user practices needed to create the backup save sets

Backups come in two varieties, local backups and off-site backups. Local backups are complete copies of the file system that permit complete restoration of the file system to the last saved state. These protect primarily against disk failures or other hardware failures that cause the disk to become corrupt. This can include a dropped, lost, or stolen laptop computer. Off-site backups permit recovery of essential records when one’s home is damaged by fire, flooding, windstorms, or theft of a machine or the local backups were destroyed, stolen, or unusable.

In days of yore, off-site backups were local backups that were carried to a safe off-site storage location such as your safety deposit box. Today, they are transmitted over the Internet to a cold storage provider such as Google Drive or Amazon Glacier. Some people use Google Drive or Amazon Glacier directly while most mere mortals use a service such as Carbonite, Pogoplug Cloud, etc to create and restore off-site backups. This article describes Pogoplug Cloud and the things used with it.

Some Definitions

I’ll try to use the terms defined below in this article. These may not exactly match Pogoplug’s usage of them but I can match them up to things on my Mac.

Cloud Engines: Makers of Pogoplug and providers of Pogoplug Cloud services

Local backup: A locally kept copy of a group of files made for the purpose of restoring lost or damaged files.

Off-site backup: A remotely kept copy of a group of files made for the purpose of restoring important files when recovering from loss or damage to the local site.

Pogoplug: A local server running the pogoplug service. This can be a PC running pogoplug pc or a purpose built Pogoplug device.

Pogoplug Cloud: A remote server running the pogoplug service.

Pogoplug Backup: The local pogoplug service client that conducts backups and restores backups.

Pogoplug Companion: The Android/IOS application that allows a phone or tablet to use files served by the user’s pogoplug service.

PogoplugPC: A Windows/Mac OS X application providing the pogoplug service from user space.

Time Machine: The Mac OS X system backup and system recovery application.

Web interface: http://my.pogoplug.com

Introducing Pogo plug

Pogoplug is the trademark of Cloud Engines, an international software company headquartered in Israel with offices in Silicon Valley. Cloud Engines got its start making a gadget called Pogoplug, a computer built into a wall wart (hence plug) that provided a small amount of Internet accessible shared storage using a user provided USB disk drive. Over the years, the product has changed form and new services have evolved around the original USB media server. This page tries to coherently describe the benefits of the Cloud Engines product and the associated services, something Cloud Engines has trouble doing, probably for want of a good technical writer.

Pogoplug branded products and services support a number of use cases so I’ll try to present the products in terms of these uses cases.

  • Local backup of one or more computers
  • Off-site backup of one or more computers
  • Local and Internet file sharing (a personal cloud service).
  • Remote file access

A bit about me

I’m a scientific software professional who has worked for 40 years on nuclear power plant application software, nuclear power plant simulator software, and wargaming software. My career spans the period from the introduction of disk drives to embedded systems to pocket sized “super computers”. Back in my simulator days, I’d stayed ’til one AM doing an integration build. I was tired and cranky so I blew off the backup. The head crash occurred as I was driving to work the next morning. This was back in the days of CDC storage module drives and dinner plated sized multi-platter removable media disk packs. A head crash was a big deal. The repairs were $10,000, a new pack was $1000 or so not to mention the lost data and the day that it would take to do the repair and a second day putting humpty dumpty back together.

My professional background is in several extinct minicomputer operating systems, SunOS 4, and Linux. My hobby background is a mix of early Windows ’95, OS-2, and Mac OS X, mostly the latter. I’m an amateur system administrator these days looking after RedHat workstation and plain old Mac OS X.

I’m not at all familiar with Windows backup and recovery procedures and my recollections of them are over a decade old. I’ve done one OS X system recovery using Time Machine and it is a joy. I’ve done several SunOS 4 system recoveries that were a pain courtesy of all of the media handling of 8 mm video tapes.

Windows Local Backup

To this day, Microsoft continues to leave disk backup to third parties. Microsoft does include a backup product but nobody uses it. Most 3rd party backup products are designed for use by corporate high priests in corporate settings where tape drives or tape library robots are available. Tape drives, tape handling, and automated tape libraries are too complex for most home users so something different is needed at home. Pogoplug attempts to fill the Windows user data backup gap. Pogoplug Windows local user data backup requires the following kit.

  • A USB disk large enough to hold the files to be protected
  • A Pogoplug server, either a Pogoplug kit machine or a local host running PogoplugPC software.
  • Pogoplug Backup installed on each machine to be protected

Pogoplug Backup is a free companion program available at http://my.pogoplug.com/downloads. This link provides both Mac OS X and MS Windows versions.

A Pogoplug server turns a USB disk drive into a local backup server. A companion software product, Pogoplug Backup backs up selected parts of its host file system to the Pogoplug connected disk. Typically, each user’s home directory is backed up plus any public directories such as those used by photo libraries, music libraries, and video libraries. Pogoplug Backup allows selection of the directories to be backed up. The directories and files to be backed up must be readable by the logged in user running Pogoplug Backup. Once the user has nominated files and directories for backup, Pogoplug Backup transfers the files to Pogoplug which maintains an image of the most recent version of the file. The machine must remain running and logged in until the transfer completes.

This process is sufficiently fast that it should be possible to save the entire file system. In my initial exploration of Pogoplug Backup, I did not attempt to set up a full disk save because I’m using Pogoplug Backup with Pogoplug Cloud for off-site backup.

Pogoplug Backup runs as a user program rather than as a service. This means that the user account must be logged in and active while the program is running. I’m not familiar with Windows but I suspect some care is needed to tell the machine to skip hibernation while the backup completes.

I’ve skipped over little issues like restoring applications and the Windows registry. These make Windows recovery a royal pain, about as bad as buying a new machine. Everything has to be reinstalled from media to recreate the registry which is difficult to back up while the machine is running. I’m not a Windows guy so I don’t know the details or of any tools that would make this easier.

Pogoplug Backup Limitations

Pogoplug backup runs as a logged in user process so it has the following limitations.

  • Files must be readable by the user
  • Pogoplug cannot back up complex data structures like a MySql database or an Aperture photo library. The UI will not let you choose things that Pogoplug backup does not recognize.
  • Pogoplug backup will not let you select directory Applications or Program Files content for backup.

Pogoplug’s limitations make it suitable for backup of user data. It is not designed to support system recovery. In this way, Pogoplug and Carbonite are similar. They will save user directories that reside on the system disk. Pogoplug appears to give more user control of what is to be saved.

Mac OS X Local Backup

Pogo Backup and Pogoplug can support Mac OS X backup and the procedures and prerequisites are the same as for the Windows use case. But, most OS X users continue to use OS X Time Machine for disk backup. Time Machine maintains the current state of the file system on an external drive, either directly connected or network storage provided by an Airport Time Capsule, Airport Extreme, or a network storage array such as a Droboshare. Time Machine differs from Pogoplug Backup by maintaining back versions of a file in the archive and permitting recovery of any back version that remains available.

Time Machine also differs in that it was designed to save the entire file system including directly connected external disks. This is easily configured and you can tell Time Machine to skip directories whose contents are transient.

I use a USB Drobostore with Time Machine that currently has 4TB of raw disk storage configured as a 2 TB virtual disk. I have a second 1 TB Drobostore that holds music and photos and the system’s internal 320 mB disk. Time Machine maintains the current state of the complete file system except the part dedicated to Time Machine itself. Time Machine allows user configuration to specify the storage device to be used and those parts of the file system to be included or excluded from protection. The 2 TB storage array is adequate to backup both if I have Time Machine skip the EyeTV spool directory and the iTunes spool directory. There’s no need to back up transient TV shows and movies. Eventually, I’ll have to put a 4th disk in the Time Machine Drobostore.

Time Machine uses hard links to maintain a current virtual image of the protected file system tree while retaining back versions of files. The hard links point to the current versions with the back versions on disk. The Time Machine user interface allows you to retrieve the current version of a file or any earlier version still in the archive. When Time Machine needs space, it starts shedding oldest versions of files.

I’ve needed Time Machine once when a system disk failed. The recovery procedure was dead simple, install OS X from external media, open Time Machine, and restore the volume. The next morning, Oswald was ready for use. I’ve used Time Machine a second time when my Aperture photo library fell victim to a disk malfunction. A Drobostore, then connected by FireWire became befuddled and my Aperture Library went missing. Time Machine came to the rescue. I had to rummage back a week or two but I found one that was usable.

Pogoplug Off Site Backup

The offsite backup process is identical for Windows and OS X. The material tha follows applies to both.

I’ve tried two solutions for offsite backup, Carbonite and now Pogoplug Cloud. Pogoplug Cloud is a $60/year service that uses Amazon Glacier to store the portion of the file system that you wish to protect. To use Pogoplug Cloud, you need the following

  • A robust Internet connection such as Cox Preferred
  • A Pogoplug Cloud subscription
  • Pogoplug Backup installed on the machine to be protected

Once these conditions are satisfied, you use Pogoplug Backup to nominate directories and files for backup. Pogoplug Backup transfers these to Pogoplug Cloud in much the same way that it transfers files to a local Pogoplug server but the process is limited by Internet speed rather than local Ethernet speed and disk drive write performance. My initial Carbonite backup took a good week and I had to pause it while watching Netflix or iTunes content. I expect that Pogoplug Cloud will be the same.

Pogoplug Local and Off-site Backup

If you have both a Pogoplug Server and a Pogoplug Cloud service subscription, your Pogoplug server will transfer backed up files to both its local disk and the Pogoplug Cloud subscription. Once properly setup, the off-site backup process is an extension of the local process and no additional user actions are needed.

Local and Off Site File Access

This is how Carbonite and Pogoplug differ. Carbonite permits backup and recovery only. Pogoplug permits on the go file access by web service at http://my.pogoplug.com. From here, you can retrieve any media saved in a home Pogoplug server or in the Pogoplug Cloud.

The Recovery Process

The recovery process is similar using OS X Time Machine or Pogoplug. For this purpose, we’ll assume a failed disk drive replacement. The work flow is

  • Repair the hardware by replacing the bad system disk
  • Reinstall the operating system and applications
  • Install Pogoplug Backup
  • Restore the user data system

With OS X Time Machine, recovery is relatively easy. Newer Macs include an Internet boot loader that will start OS X from the Internet for the purpose of recovering the system If you have an old machine like mine, initial startup requires an OS X installation disk for older versions of OS X or an installation thumb drive for Lion, Mountain Lion, and Mavericks.

  • Start the machine from a recovery disk or thumb drive image
  • Format the new disk using Disk Utility
  • Install OS X from the boot media
  • Connect the external Time Machine volume and allow it to mount
  • Restore the complete file system from Time Machine

With Pogoplug Backup and Pogoplug, the work flow is

  • Start the machine from a recovery disk or thumb drive
  • Format the new disk using Disk Utility
  • Install OS X from the boot media
  • Reinstall applications
  • Install Pogoplug Backup from the Pogoplug.com website.
  • Sign in using to your Pogoplug your Pogoplug credentials, usually your primary E-mail address and password
  • Restore the user directories and files saved

I can’t really comment on Windows recovery because I’ve not needed to do it. In OS X land, things are not so bad. There is no registry. Applications are saved as application packages (basically a directory) in /Applications which may be backed up. A Time Machine restoration brings everything back. I’ve been there, done that, and have a working system to prove it. Time Machine rocks.

I’ve not needed to do a network recovery and hopefully I never will. But I live in hurricane country and a Cat 2 will damage my home and a Cat 3 will blow it down and probably wash it away. So I’m careful. I rely on my Mac for all of my tax and financial record keeping. And my photos and home movies reside there. So I back up off site and it is money well spent because little of this can be replaced even if I knew what it all was. The choice of a Mac Mini was deliberate. I can throw the mini and Drobostores in a bugout bag should we need to evacuate.

Categories
Personal Computing

Mountain Lion Arrives

After yesterday evening’s fireworks wound down, I installed Apple OS X Mountain Lion on Oswald Cobblepot, my middle-aged Mac Mini that does photo, movie, and music chores. Mountain Lion installs in two steps, purchase of the installer from the Mac App Store and running of the installer. The installation process takes a couple of hours but requires minimal attention once started. At least, that’s the case when upgrading from Lion to Mountain Lion.

I was a little bit nervous. Although the skies were quiet when I kicked off the update, another wave of storms came through the area about mid-way through. The thunder gods were kind and left the power alone. Once installation is complete, the machine restarts using the new OS image and updated programs. The changes from Lion to Mountain Lion are subtle. Apple has revised the OS X applications to look a lot like their iPad counterparts in IOS 5. They’ve added messages, reminders, notes, and a notification system similar to that in IOS. A lot of the release is about integration with iCloud. Mountain Lion syncs notes and reminders in addition to calendar items, contacts, and mail.

At the moment, there appear to be no downsides. The applications that I use weekly work without fuss. These include iBank and Investoscope, both purchased outside the App Store. Gatekeeper is a new feature of Mountain Lion that is baked into the process launch services of the operating system. The process manager checks each application being started to verify that it was signed by the Apple App Store or a registered developer. Gatekeeper will let you run unsigned applications by presenting a dialog reporting that the image is unsigned and requesting authorization to run it.

Apple did not tinker with Air Play other than to make it possible to redisplay the Mac OS X desktop on an Air Play display server. iPad, iPhone, iPod Touch, Apple TV, and other Mountain Lion equipped Macs can be Air Play display servers. This is a nice touch for business. Buy an HD HDMI interface projector, connect it to an Apple TV, and presenters can show visuals without all the silliness that goes on at meetings. And you can play Hulu+ content on your telly, even things not permitted to play in iPad/iPhone.

Another thing Apple left alone is the annoying habit of the window manager of going into beach ball mode and refusing to let you work with another application when you make a slow to finish request. In 10 years of life with OS X, I’ve yet to see a pattern to when the window manager does this. Modal dialogs are evil, particularly those that kidnap the mouse until dismissed. Apple is slowly eliminating this sort of thing but there is still some of it left. OS X’s great strength is that it is multi-user and multi-tasking from the kernel up. No need to act like Windows 3.1.

A third party audio player went into a hard run playing a high resolution FLAC file while I was writing this article. That’s about it so far for troubles. And this may be the player’s fault, not the OS update.