|
ResearchSUBARU TelescopeReturn to Research Success Story List
Contact:
Tools Used:
Background: The SUBARU Telescope captures starlight data for analysis by researchers around the world. The first data were gathered in 1999 and full operation is expected in 2000. The captured light is saved as image data, video data and numerical data. Once the data is collected, it is sent to the supercomputer at the main facility located in the University of Hawaii at Hilo. The data is examined by researchers at that location and in Japan via an international leased line.
Problem: The SUBARU group needed to develop a flexible and scalable analytical system that satisfied the complex needs of astronomical observations. The analytical system had to account for contradicting elements such as flexibility and integrity; changeability and persistence; and independence and collaboration. At the data processing center in Hawaii, several analytical applications were working with the Fujitsu VPP700 supercomputer to manage a large amount of data from the SUBARU telescope. The AP3000 connected with the archival system (a magnetic data library with a capacity of about 150 terabytes with 1.2 terabyte RAID discs and many workstations.) The old analytical applications lacked interoperability and could not manage the stratified data necessary for the astronomical studies. The lack of data management made it hard for researchers to get the data that they needed in a timely fashion. The poor quality of the old system also made it difficult for the researchers to get the image data at the quality level they expected.
Solution: The SUBARU group introduced a CORBA- based solution, using Fujitsu’s INTERSTAGE ORB, to solve these problems. The solution is capable of sharing various analytical data, tracking processes, using data archives, and building analytical engines with fewer resources than the old system. CORBA was selected because it gives the ability to easily change the system’s configuration to limit the effects of system damage and to balance the load. CORBA, which supports a multi-vendor environment and interoperability, also makes it easy to integrate legacy systems. New observatory instruments were added to the capabilities of the SUBARU telescope, so it was necessary to efficiently develop data processing software for each instrument. It also had to be easy to improve the analytical system, keeping up with the improvement of the observatory instruments. "CORBA was the only way to make full use of the existing resources to build an integrated research environment capable of providing usability to the researchers," Dr. Mizumoto recalls. The researchers also needed a high-level abstraction of the business model, represented by real world objects that would make it easy to communicate with remote personnel. By creating specialized terminology (the "restaurant model", PROCube and the "recipe model"), this task was completed more easily. In the restaurant model, the "restaurant" represents a whole astronomical data analytical system and the "recipe"; does the analysis. It applies these abstract reference models to the system architecture. These models created a mutual understanding between the managers, the architects, and the programmers, which improved when developers analyzed, designed, and reviewed the models. The feeling that they had a clear grasp of the entire system, along with easy extraction of the objects, made an efficient development environment. The development group also devised PROCube, an object that represents all the information of an analytical procedure. PROCube is equal to a "recipe" and responds to the "order voucher" in the restaurant model. With PROCube, the raw data that comes from the telescope is processed in a series of transactions. The basic structure is a 3-dimensional flow chart. Multiple PROCubes work on the CORBA-based platform and the computers for transactions are chosen automatically. Through a 3-year, multi-phase process, the researchers were able to develop and refine the prototype of the whole distributed system, which they called the DASH (Distributed Analysis System Hierarchy). Targets were reviewed and rearranged every 6 months. After completing the 3-year plan, the developers were able to realize the following functions: enable data transactions on one single system; integrate applications that had been implemented with different languages; create a seamless and easy-to-use user interface; reuse the existing software; wrap the legacy applications with CORBA; and establish a collaboration of Java, CORBA, Web and C++ with a customized GUI. The GUI clients implemented with Java send requests to the INTERSTAGE server through IIOP transmission using Java IDL, a standard package in Java 2. All the servers are working on INTERSTAGE from Fujitsu Ltd. The project was successfully executed through the hard work of an exceptionally large distributed system and an exceptionally small number of staff. The development staff of the ORB vendor, Fujitsu, also contributed to the project’s success by developing unique metaphorical models and fine-tuning their original middleware product to handle unprecedented amounts of huge and complex data. As technologies such as Java, CORBA and XML grow; the development group will apply these applications to improve the component oriented system.
For complete version of this story, including technical details and diagrams,
please download this Adobe PDF file. |