Problem solve Get help with specific problems with your technologies, process and projects.

Storage management software flies high for Air Force lab

Starfire Optical Range moves away from DAS to explore the options of a SAN.

The Starfire Optical Range (SOR) project's goal is to remove distortion from the images viewed in telescopes. Put simply, SOR's researchers are taking the twinkle out of the stars.

To get an accurate picture of space over a period of time, thousands of sample images must be collected and stored, according to Terry Duncan, engineering director for the Air Force Research Laboratory Directed Energy Directorate's SOR project. The trouble was, any image stored on the Lab's direct-attached storage (DAS) system was almost as hard to find as a life on another planet.

SOR researchers are developing the world's fastest and most accurate adaptive optics. In adaptive optics, the blurring and visual aberrations created when optical waves pass through the Earth's atmosphere are corrected by changing the shape of a telescope's mirror. The distortions differ with the angle of view and atmospheric conditions, which change constantly. To collect a true image of space a "picture" must be taken over a period of time and then composed using accurate statistical data.

The SOR is "the most important advance in astronomy since the invention of the telescope," said Duncan.

Unfortunately, the facility's storage system was not advanced. In SOR's first-generation storage system, data was stored on removable hard drives and magnetic media. Those drives and cartridges were stacked, and researchers had to search through the files on each one for data.

"Our storage system was very haphazard," said Duncan. "Consequently, we were not able to collect as much data as we needed, because we had no organized way to store it."

Studying the options, the lab's IT team decided to replace the old system with network-attached storage (NAS) appliances. With NAS, data could be copied over the network to filers. The team discovered that NAS succeeded in improving data organization, but failed elsewhere. Even at 20 MB/sec, the NAS "just wasn't efficient given the quantities of data we were trying to move," said Duncan. "It didn't help us get the quantity of data we needed to collect."

Duncan spent 18 months researching storage solutions and running into walls. While not as complex as SOR technology, the project's requirements for storage were pretty daunting. Duncan needed a unified solution that would easily integrate the hardware and software elements of SOR's storage tasks -- file sharing, data migration and end-to-end data management. Of course, this being an Air Force project, data security had to be tight -- very tight.

Drilling down, Duncan figured his system had to handle more than 500 GB of collected data daily and provide throughput of hundreds of MB/sec with volumes of tens of terabytes (TB). SOR's storage system must allow up to 50 Dell and Compaq workstations to have access to collected data and give multiple users access to the same files at the same time. Add to that the fact that researchers needed dynamic access to the lab's entire 250 TB data set.

Data protection requirements were just as strenuous. Data must be replicated as soon as it is written to tape. "We want to never have more than an hour of single data set exposure," said Duncan.

Duncan decided that a storage area network (SAN) had the flexibility and strength SOR needed. With his IT team, he designed a SAN system that seemed perfect on paper. The design included a SAN fabric of four Slingshot 4218 switches from Santa Clara, CA-based Gadzoox and four 10 TB RAID 5 disk arrays.

Although the switches and arrays worked well, SOR faced another year of problems. Initially, Duncan chose storage network management software from Ft. Lauderdale, FL-based DataCore Software Corp. and OTG, since acquired by Mountain View, CA-based Legato. SOR was at the bleeding edge of testing for both of these software packages, and bugs caused a lot of bleeding.

Cutting cords with OTG, Duncan brought in ADIC's StorNext Management Suite, keeping the tried-and-tweaked DataCore software in some applications. ADIC hails from Redmond, Wash.

"StorNext Management Suite gives us fast shared access for highly parallel processing, and its integrated data management software automatically moves data between our Scalar 10K library and our disk," said Duncan. "Any time researchers need data to analyze, the system writes it from the library to the disk, and it's available almost immediately."

Putting ADIC's 10K Scalar tape library to work brought everything into sharp focus, Duncan said. Highly scalable, the library accommodates AIT and LTO tapes in the same chassis, holding up to 16 drives and 2,100 pieces of media. High availability is accomplished with storage networking technology that provides direct FC connections, redundant path capability and integrated management utilities.

"It was the only system we found that integrated the hardware and software elements of our storage task-file sharing, FC data access speeds, data migration, backup, archiving and end-to-end data management into a single unified solution," said Duncan.

The ADIC system has allowed SOR to gather as much data as is necessary for its research tasks. "It enables us to do things we always said we couldn't do," said Duncan. "It enables us to do the impossible."

For more information on Starfire Optical Range, visit its Web site.

Additional information on ADIC can be found here.


>> SAN vs. NAS: What's the difference?

>> The principle benefits of moving to a SAN

>> Best Web Links: SAN

Dig Deeper on Win Development Resources

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.