Hard Drive Reliability

A few months ago we asked and answered one of computing’s oldest questions: How long do hard drives actually last? That story missed one vital piece of information, though —who makes the most reliable hard drives? Well, we can now answer that question too.

Just like last time, this information comes from Backblaze, an all-you-can-eat online backup company. Backblaze currently has around 28,000 hard drives powered up and constantly spinning, storing a total of around 80,000 terabytes (80 petabytes) of user data. As you can imagine, it is very much in Backblaze’s interests to ensure that it buys reliable hard drives. Every time a drive fails, it takes considerable time and effort to pull the drive, slot in a new one, and rebuild the RAID array.

Which hard drive manufacturer is the most reliable?

Backblaze breaks down its data in two ways — by manufacturer, and by specific drive. The data is fairly complex, but we’ll try to break it down into morsels of easy-to-digest, actionable information. (Read: How a hard drive works.)

As of the end of December 2013, Backblaze had 12,765  Seagate drives, 12,956 Hitachi drives, and 2,838 Western Digital drives. These drives are not all the same age — some are almost four years old, while many were installed in the past year. The odd numbers are because Backblaze basically buys whatever drive offers the most competitive dollar-per-gigabyte ratio, with reliability being a secondary factor. For most of the last four years, Seagate and Hitachi have offered the best price-per-gig, with Western Digital Red drives only now becoming a viable option for Backblaze.

Hard drive annual failure rate, broken down by maker (Hitachi, Seagate, Western Digital) and size

As you can see from the graph above, Hitachi drives are by far the most reliable. Even though most of Backblaze’s Hitachi drives are now older than two years, they only have an annual failure rate of around 1%. The “annual failure rate” is the chance of a drive dying within a 12-month period. After three years of being powered up 24/7, 96.9% of Hitachi drives are still running.

Western Digital is slightly worse, but still impressive: After three years of operation, 94.8% of Western Digital drives are still running. Backblaze lists the annual failure rate of the WD drives at around 3% (I don’t think the numbers quite add up, but I could be wrong).

Seagate drives are not very reliable at all. As you can see in the second graph below, Seagate drives are fine for the first year, but failures quickly start building up after 18 months. By the end of the third year, just 73.5% of Backblaze’s Seagate drives are still running. This equates to an annual failure rate of 8-9%.

Hard drive failure rate, plotted by month

In Backblaze’s words: “If the price were right, we would be buying nothing but Hitachi drives. They have been rock solid, and have had a remarkably low failure rate.”

Which single hard drive is the most reliable? (And which is the least?)

In general, then, if you want a reliable hard drive you should go for a Hitachi or Western Digital. If you’re looking for a specific drive model that has good longevity, the numbers break down interestingly.

The two best drives, with 0.9% annual failure rate over more than two years, are the Hitachi GST Deskstar 5K3000, and Hitachi Deskstar 7K3000. Get one of these drives and you’re almost guaranteed (97-98%) to make it through three years without a dead drive. If you want a 4TB drive, the Hitachi Deskstar 5K4000 is your best bet — it has a slightly higher failure rate, but still below WD and Seagate’s offerings.

As far as poor reliability goes, Seagate has some nasty offenders. The 1.5TB Seagate Barracuda 7200 (an old drive now) has a very high chance of failing after three or four years. Even the newer 3TB Seagate Barracuda has a pretty high failure rate, at 9.8% per year.

Backblaze also notes that some drives (the Western Digital Green 3TB and Seagate Barracuda LP 2TB) start producing errors as soon as they’re slotted into a storage pod. They think this is due to the large amounts of vibration caused by thousands of other hard drives. (They also think that their aggressive spin-down setting, which is ostensibly to save power, causes a lot of wear to the drive.)

Samsung and Toshiba

Unfortunately, Backblaze doesn’t have a statistically significant number of Samsung or Toshiba drives installed. Even so, because Samsung’s hard drive division was acquired by Seagate in 2011, it’s hard to say if an older, pre-acquisition Samsung drive would be more or less reliable than a post-acquisition drive. Toshiba/Fujitsu still have a reasonable wedge (~10%) of the market share pie, but unfortunately we’ll have to wait for another study to see how they compare to Seagate, Western Digital, and Hitachi.

On the topic of acquisitions, you may also remember that Western Digital acquired Hitachi GST almost two years ago. If we compare Hitachi drives from before and after the acquisition, the annual failure rate seems to stay the same (around 1%). It would seem that Western Digital and Hitachi have the reliable hard drive business sewn up — and this is before we’ve had a chance to see what WD/HGST’s helium-filled hard drive can do!

Security at Risk

A U.S. cybersecurity firm says it has gathered evidence that the Russian government spied on hundreds of American, European and Asian companies, the first time Moscow has been linked to cyber attacks for alleged economic – rather than political – gains.

According to the firm, CrowdStrike, the victims of the previously unreported cyber espionage campaign include energy and technology firms, some of which have lost valuable intellectual property.

CrowdStrike declined to go into detail about those losses or to name any victims, citing confidentiality agreements related to its investigation.

Officials with the Russian Interior Ministry could not be reached for comment early on Wednesday in Moscow.

“These attacks appear to have been motivated by the Russian government’s interest in helping its industry maintain competitiveness in key areas of national importance,” Dmitri Alperovitch, chief technology officer of CrowdStrike, told Reuters on Tuesday evening.

Cybersecurity researchers have in the past said that China’s government was behind cyber espionage campaigns against various corporations dating back as far as 2005, but China has vehemently denied those allegations. Alperovitch said this is the first time the Russian government has been linked to cyber intrusions on companies.

Governments have been using computer networks to spy on each other for more than 30 years in the type of surveillance programs conducted by virtually every nation, according to CrowdStrike. It is only in the past decade that some nations have started using cyber espionage as a platform for gaining data to help promote their national economic interests, according to Alperovitch.

CrowdStrike has been following the activities of the Russian group of hackers, which it dubbed “Energetic Bear,” for two years. The firm believes the Russian government is behind the campaign because of technical indicators, as well as analysis of the targets chosen and the data stolen, according to Alperovitch.

“We are very confident about this,” he said. Victims include European energy companies, defence contractors, technology companies and government agencies, according to the CrowdStrike report.

Manufacturing and construction firms in the United States, Europe and Middle East as well as U.S. healthcare providers were also cited as targets in the report that was posted on the web early on Wednesday morning.

CrowdStrike described the activities of the Energetic Bear hackers in its annual cyber threat report, released on Wednesday. It also documented attacks by hacking groups in China and Iran and described the activities of the activist Syrian Electronic Army.

Alperovitch, who is of Russian ethnic origin and now lives in the Washington, D.C., area, is an expert on cyber espionage who rose to prominence while working for McAfee Inc. While there he managed a team of researchers who produced a landmark January 2010 report that described how Chinese hackers had launched an unprecedented series of attacks known as “Operation Aurora” on Google Inc and dozens of other companies.

In 2012, he co-founded CrowdStrike, which collects intelligence about the activities of hacking groups around the world and sells software to thwart such attacks.

He told Reuters that the data his firm has obtained about Energetic Bear suggests that authorities in Moscow have decided to start using cyber espionage to promote Russia’s national economic interests.

“They are copying the Chinese play book,” he said. “Cyber espionage is very lucrative for economic benefit to a nation.”

Heat Waves under the scope

Scientists have fingerprinted a distinctive atmospheric wave pattern high above the Northern Hemisphere that can foreshadow the emergence of summertime heat waves in the United States more than two weeks in advance.

The new research, led by scientists at the National Center for Atmospheric Research (NCAR), could potentially enable forecasts of the likelihood of U.S. heat waves 15-20 days out, giving society more time to prepare for these often-deadly events.

The research team discerned the pattern by analyzing a 12,000-year simulation of the atmosphere over the Northern Hemisphere. During those times when a distinctive "wavenumber-5" pattern emerged, a major summertime heat wave became more likely to subsequently build over the United States.

"It may be useful to monitor the atmosphere, looking for this pattern, if we find that it precedes heat waves in a predictable way," says NCAR scientist Haiyan Teng, the lead author. "This gives us a potential source to predict heat waves beyond the typical range of weather forecasts."

The wavenumber-5 pattern refers to a sequence of alternating high- and low-pressure systems (five of each) that form a ring circling the northern midlatitudes, several miles above the surface. This pattern can lend itself to slow-moving weather features, raising the odds for stagnant conditions often associated with prolonged heat spells.

The study is being published next week in Nature Geoscience. It was funded by the U.S. Department of Energy, NASA, and the National Science Foundation (NSF), which is NCAR's sponsor. NASA scientists helped guide the project and are involved in broader research in this area.

Predicting a lethal event

Heat waves are among the most deadly weather phenomena on Earth. A 2006 heat wave across much of the United States and Canada was blamed for more than 600 deaths in California alone, and a prolonged heat wave in Europe in 2003 may have killed more than 50,000 people.

To see if heat waves can be triggered by certain large-scale atmospheric circulation patterns, the scientists looked at data from relatively modern records dating back to 1948. They focused on summertime events in the United States in which daily temperatures reached the top 2.5 percent of weather readings for that date across roughly 10 percent or more of the contiguous United States. However, since such extremes are rare by definition, the researchers could identify only 17 events that met such criteria -- not enough to tease out a reliable signal amid the noise of other atmospheric behavior.

The group then turned to an idealized simulation of the atmosphere spanning 12,000 years. The simulation had been created a couple of years before with a version of the NCAR-based Community Earth System Model, which is funded by NSF and the Department of Energy.

By analyzing more than 5,900 U.S. heat waves simulated in the computer model, they determined that the heat waves tended to be preceded by a wavenumber-5 pattern. This pattern is not caused by particular oceanic conditions or heating of Earth's surface, but instead arises from naturally varying conditions of the atmosphere. It was associated with an atmospheric phenomenon known as a Rossby wave train that encircles the Northern Hemisphere along the jet stream.

During the 20 days leading up to a heat wave in the model results, the five ridges and five troughs that make up a wavenumber-5 pattern tended to propagate very slowly westward around the globe, moving against the flow of the jet stream itself. Eventually, a high-pressure ridge moved from the North Atlantic into the United States, shutting down rainfall and setting the stage for a heat wave to emerge.

When wavenumber-5 patterns in the model were more amplified, U.S. heat waves became more likely to form 15 days later. In some cases, the probability of a heat wave was more than quadruple what would be expected by chance.

In follow-up work, the research team turned again to actual U.S. heat waves since 1948. They recognized that some historical heat wave events are indeed characterized by a large-scale circulation pattern that indicated a wavenumber-5 event.

Extending forecasts beyond 10 days

The research finding suggests that scientists are making progress on a key meteorological goal: forecasting the likelihood of extreme events more than 10 days in advance. At present, there is very limited skill in such long-term forecasts.

Previous research on extending weather forecasts has focused on conditions in the tropics. For example, scientists have found that El Niño and La Niña, the periodic warming and cooling of surface waters in the central and eastern tropical Pacific Ocean, are correlated with a higher probability of wet or dry conditions in different regions around the globe. In contrast, the wavenumber-5 pattern does not rely on conditions in the tropics. However, the study does not exclude the possibility that tropical rainfall could act to stimulate or strengthen the pattern.

Now that the new study has connected a planetary wave pattern to a particular type of extreme weather event, Teng and her colleagues will continue searching for other circulation patterns that may presage extreme weather events.

"There may be sources of predictability that we are not yet aware of," she says. "This brings us hope that the likelihood of extreme weather events that are damaging to society can be predicted further in advance."

The University Corporation for Atmospheric Research manages the National Center for Atmospheric Research under sponsorship by the National Science Foundation. Any opinions, findings and conclusions, or recommendations expressed in this release are those of the author(s) and do not necessarily reflect the views of the National Science Foundation
.

VR is on its way

If mere texting, talking, e-mailing and snapping pictures on mobile devices aren’t enough to satisfy your data cravings, now there’s the prospect of accessing and displaying 3-D virtual reality simulations and animations on them. New information architecture from researchers in Offenburg, Germany puts 3-D visualizations in the palm of your hand to make this possible.

By devising a novel information and communication architecture with optics technology, researchers created a new approach based on outsourcing to servers all the heavy number crunching required by computer animations and virtual reality simulations. After churning through it, the servers then provide the information either as stream (avi, motion JPEG) or as vector-based data (VRML, X3D) displayable as 3-D on mobile devices. Dan Curticapean and his colleagues Andreas Christ and Markus Feisst of Offenburg’s University of Applied Science devised the approach.

"Since the processing power of mobile phones, smart phones and personal digital assistants is increasing—along with expansion in transmission bandwidth—it occurred to us that it is possible to harness this power to create 3-D virtual reality," says Curticapean. "So we designed a system to optimize and send the virtual reality data to the mobile phone or other mobile device."

Their approach works like this: Virtual reality data sent by the server to a mobile phone can be visualized on the phone’s screen, or on external display devices, such as a stereoscopic two-video projector system or a head-mounted stereoscopic display. The displays are connected to the mobile phone by wireless Bluetooth so the user’s mobility is preserved. In order to generate stereoscopic views on the mobile display screens, a variety of means can be used, such as a built-in 3-D screen or using lenticular lenses or anaglyph images viewed with special glasses having lenses of two different colors to create the illusion of depth.

The upshot of this new approach is improved realistic 3-D presentation, enhanced user ability to visualize and interact with 3-D objects and easier presentation of complex 3-D objects. "Perhaps most important," says Curticapean, "is the prospect of using mobile devices such as cell phones as a user interface to communicate more data with more people as an important component of mobile-Learning (m-Learning), given the ubiquity of mobile devices, particularly in developing countries."

The scientists are presenting their research at the 92nd Annual Meeting of the Optical Society (OSA), being held from Oct. 19-23 in Rochester, N.Y
.