Harddrives 7.2k vs. 10k vs. 15k speeds

A DB server would probably benefit from the higher speeds. Hundreds of thousands of records per minute is quite a lot. Faster disks would help, of course intelligent application design and indexing would also be critical.

Camelot is quite familiar with these kinds of loads... With Resin and MySQL, perhaps he can speak to more specifics. It's just just as simple as a disk question.

Again it depends on what these CRUD operations really are.

If you're hitting the disk spindles, I found when working with Oracle for their Low Cost Storage Initiative white paper that a 7200 RPM ATA or SATA hard drive can do about 80 IOs per second (IOPS) whereas a high-end fiber drive (SAS drives should be similar) can do about 150 IOPS. So that's 4800 or 9000 IOs per minute, respectively. You can't go faster than that to a single drive, and it assumes a fairly small I/O size.

Given those are both above the numbers you're quoting, you're going to need to be hitting cache (which could be in your application server or database server) or throwing a number of spindles at the problem.

Spindle rotational speeds are the disk drive analog of the classic megahertz or gigahertz ratings of processors; certainly useful information, but very far from a reliable comparison. And this is information that can be easily misconstrued.

Do look at the individual latency and transfer rates for the devices. The disk latency limits random access (which is a pattern seen with many of the typical application mixes), and the spindle transfer rates are the upper bound on and limit (big) reads and (big) writes.

I've worked with some disks with silly-fast rotation that provided rotten access times; that had very large latencies. Some solid-state devices can run afoul of this; cases where the access times are gonzo-fast (no half-rotation wait for the data to arrive under the head, and no track-to-track actuator; no drive head at all) and read transfer speeds can look good, but the write speeds are glacial.

For my customers, I tend to recommend a combination of disks. Some big and slow, and some small and fast. The former for nearline and archival purposes, and the latter when the CRUD is really flying. The former are for infrequent and low-bandwidth, and the latter are for the hot files.

As for what works and what is needed, you're probably going to end up prototyping it. Popular sites usually scale up into the range, and end up with some idea of what the specific traffic patterns are. As you scale up, you'll end up with a much better idea of your specific load, and the various corner cases -- such as what happens when the web site gets busy right when an archival pass wanders through. (And there are seemingly always corner cases.)

Individual disk spindles are (all things considered) fairly cheap up through and including many of the direct-attached storage (DAS) options, and the prices don't really jump until you get to the SAN configurations. Some vendors can re-target DAS storage shelves up onto SAN controllers, too.

I'm only mentioning this because of the change above from 100's of thousands to 10-80K database updates.

Are you really talking database updates here, where changed data needs to be saved to disk, or is it more about reading data, web pages served, etc?

I'm not a database expert, but for the kind of load you're talking about, with 10-80,000 database updates per minute, designing and managing your storage is so far beyond the question of which speed drive you should buy that it makes me wonder if (with all respect) you might be in over your head.

Depending on the complexity of the surrounding website that needs to be served, any transactions generated by database changes, etc., you could be talking about an Amazon.com level of data center to handle that kind of load. It's almost certainly beyond the capacity of a single Xserve, and most likely beyond the capacity of even a couple servers with a gob of RAM, dedicated SAN storage, and a well designed database. Probably somewhere between there and a small data center to make it work, and perhaps much more. All of which is waaaaay beyond which speed hard drive to consider in complexity.