Sunday, January 18, 2009

The Terabyte Challenge

Computer stores have started to advertise 1.5 terabyte drives. The standard hard disk drive is now the 1 terabyte drive, which you can buy for about $100. What this means is that we have finally reached the point at which the average computer user can use a computer for years without any particular concern about running out of disk space.

My first brush with a terabyte came years ago in one of my most highly visible computing projects. I was involved, along with nearly a hundred other people, in creating a data warehouse for one of the largest banks in the world. This project would, for the first time, give the bank’s marketing analysts ready access to the complete body of customer data. The gee-whiz aspect of this project, from the computer industry’s point of view at the time, was the size of the data. After the historical data was all loaded in, a process that was expected to take almost a year, the size of the database would be one terabyte. There was no computer that could comfortably process a terabyte of data, so we split up the work among a room full of computers that cost, we were proud to say at the time, less than half a million dollars.

So when a terabyte is the most ordinary hard disk drive, a 3.5 inch drive that weighs about a kilogram and fits in just any desktop computer, it says that times have changed.

What does it take to fill up a one-terabyte drive with your computer files? Here are some comparisons, for perspective:

  • A terabyte is a million megabytes. An average novel fits in a megabyte. If you read one novel a week for a year, those novels could take up 50 megabytes on your hard drive. That’s just 1/20,000 of your one terabyte drive.
  • A current high-resolution photo, lightly compressed, takes up about 2 megabytes. If you took 40 photos a month, after 10 years you would fill up just 1 percent of your one terabyte drive with photos.
  • A typical major computer application takes up 100 megabytes. If you learned one new application every week, in ten years you could fill up 5 percent of a one terabyte drive.
  • A typical music album takes up 50 to 150 megabytes depending on the compression scheme. If you had 1,000 albums in your music collection, it might still fill up only a tenth of a one terabyte drive.

To fill up a one terabyte drive, you have to do something unusual, such as:

  • Having a software failure . . . and letting it run for hours.
  • Keeping business data . . . for a Fortune 500 company.
  • Collecting a huge music collection . . . that you store uncompressed.
  • Making a feature film . . . that’s more than 2 hours long.
  • Watching TV shows on your computer . . . and never deleting any of them.

I’m sure people will come up with other ways, but you get the idea. You can basically put your whole life on your hard drive. Of course, that brings up another question: what happens when people start to do that?