Object Oriented Data Technology
for Interferometry Systems

Objective

Object Oriented Data Technology, or OODT, develops and promotes science data management and archiving systems that span scientific disciplines and enable interoperability among astrophysics, planetary, and space science data systems. Using OODT's framework of distributed objects and databases, the data collected by scientists and engineers in these disparate disciplines can be jointly searched, stored, retrieved, and analyzed, from anywhere in the world.

Our goal is to pioneer this framework, architect it with continual input from the communities that will use it, and implement it efficiently. We have expanded our focus beyond interferometry, our initial risk-reduction effort, and plan on producing the next-generation archiving and analysis framework for use by JPL, NASA, and beyond. This framework allows common data management services to be instantiated and specialized for each data system implementation. In addition, science tools can access distributed data based on a standard generic interface. This improves the scientific results by allowing scientists to correlate data across multiple systems while reducing the overall implementation cost to scientific missions and projects by promoting software reuse through component design.

Approach

OODT has concentrated on three key areas of data management: data archiving, data search and retrieval systems, and data analysis. Working with the scientists of the Palomar Testbed Interferometer (PTI), we prototyped an integrated data system to archive and analyze key interferometry data using a distributed archiving service from the OODT framework. Unlike many prototypes, this effort resulted in an operational and maintained system with an easy-to-use web interface.

Drawing on and expanding upon the prototype, we set out to create a next-generation, multidisciplanary framework for data management that includes a set of distributed services for archiving and analysis using the following technologies:

  • The Common Object Request Broker Architecture (CORBA). CORBA provides a way for software objects to communicate with each other, regardless of the computer language in which they're written or the computer platform on which they run. Using this system, we can distribute the framework world-wide, making even the remotest database or service appear as if it were local. A resource location service provides information about the location of other objects in the framework.
  • The Extensible Markup Language (XML). XML, a cutting-edge web technology, gives us a standard for searching for and retrieving data, and for describing data formats. Our profile service accepts sophisticated queries for science data, determines the correct databases to use, fetches the data, and assembles the result into self-documenting XML. The use of XML allows for data to be converted and represented from proprietary formats into a neutral format described by a set of generic XML definitions.
  • Relational database technology is used to catalog science, engineering and operations data. These systems provide an environment for storing data reliably and provide efficient management and retrieval.
  • Component-based software objects hide the underlying database details, including their physical locations.
  • Java technology is used in the framework to promote portability across heterogeneous platforms.

Combining these technologies with standard distributed system principles we set out to create an architecture that promotes location independence, resource optimization, maintainability, and legacy system support. Based on these requirements, OODT is creating a plug-in architecture that enables scientists to access data in sophisticated ways (including web interfaces) without having to understand the location of data or how to access it. Smart software components are deployed over the Internet that allow clients that support OODT's interfaces to access the partner science data systems through the Resource Location Service. The following figure depicts the architecture:

architecture_sm.jpg
OODT is using XML technology to manage metadata that describes data system contents in a repository that is called a profile. Profiles are made available over the Internet using CORBA technology to share the information. Tools can use APIs to submit parallel queries to distributed profiles. Profiles that can support the query will return results of data systems that contain the product, or in some cases it will return a direct hypertext link to the product itself. Products which are inside proprietary databases can be accessed through a capability that OODT is developing called a product server. Product servers wrap legacy data systems and provide a mapping from the OODT query system into the proprietary data system which may be a database, a flat file system, or some other data store. OODT queries can be made using an XML query definition and is supported by both profile and product servers. The Resource Location Service provides the integrated profile and product server environment that allows for queries and products to be searched and retrieved from a distributed data system environment. The following figure depicts the Resource Location Service that interconnects a graph of profile and product servers to support an OODT query:
Image40_sm.jpg
Accomplishments

Our prototype for PTI is one of the first web-based integrated analysis tools for interferometery. It's also the first integrated, real-time data archiving system, developed for PTI.

OODT is now tightly integrated and supporting real time archiving of data from the Palomar Testbed Interferometer. Every nightly run at the PTI observatory stores the observations into our operational archiving system. Scientists can analyze that data using a familiar web browser interface with point-and-click controls. The following screen shot shows a portion of the system's analysis tools in action:

pti_sm.jpg
We are also building the resource location service that allows for the search and retrieval part of the framework. We have designed and built an XML profile database for the Planetary Data System (PDS) and the Palomar Testbed Interferometer Archive system. In addition, we have designed a CORBA profile and query service that allows for distributed data system information to be located. OODT has built a web-based client that can now query for information across multiple data systems. The profile service now enables new databases to be plugged into the framework without modifying the layout of the data or the data itself.

In addition, OODT has secured partnerships to help research and investigate transferring the archive service to other JPL projects including the TMOD Technology program, and the SeaWinds task. These activities will help support and validate the archive architecture for use in their research activities.

Significance

The systems we have and will develop will enable scientists on missions and other research efforts to avoid having to custom-design yet another data management, archiving, and analysis system. The framework developed promotes interoperability and reduces cost and risk using state-of-the-art and state-of-the-practice distributed and object design technologies and principles. It also increase scientific analysis capabilities by allowing interdisciplinary research through data sharing, data interoperability and data access. Finally, it provides an opportunity to continue research on other fronts to improve science data analysis using technologies such as data mining.

This framework is also generic. It solves data management issues that exist in many disciplines. Through technology transfer, we can provide object oriented data technology to fields outside astrophysics, planetary, and space science. For example, medicine, pathology, and biology can make use of our efforts and promote sharing of patient histories and prognoses between hospitals and medical research institutions, resulting in improved healthcare at lower costs.

Status/Plans

Currently, we have migrated the PTI archiving and analysis system to a high-performance Linux server. Low-cost PC hardware and the Linux operating system bring our framework a high level of performance at a fraction of the cost of typical Unix server hardware. Future developments of our new framework will also be hosted on this platform.

We have also developed an initial prototype of a new archive service that is completely CORBA 2.0 compliant. We are currently testing and evaluating our new archive server and will work a migration plan to move the existing PTI archive service over to the CORBA version. In addition, we are working to extend our archiving capabilities to support new partners from TMOD and SeaWinds.

We have also developed the CORBA profile service and a generic XML DTD specification to support PDS, PTI and other future data systems. In addition, we have designed an XML database and deployed an initial profile server to support queries against data from PDS. Future plans include replacing the PDS Distributed Inventory System (DIS) with the profile service.

We are currently working to create product servers to access information in proprietary databases for PTI and PDS. As previously discussed these systems will convert data products from the internal data system representation to a neutral format described by XML. The product servers will be integrated with the profile servers to create the Resource Location Service. This service will be integrated with a web search interface that will allow scientists to locate and retrieve products from the systems.

Finally, we are working with JPL's Navigation and Ancillary Information Facility (NAIF) in an effort to make their data available as a component in the OODT framework. NAIF's SPICE provides modules useful in computing position, velocity and instrument observation geometry parameters. The integration of SPICE will be useful in other components of the OODT framework including archiving, search and retrieval, and science analysis.

Our current schedule of deliverables is available.

Point of Contact,

Daniel J. Crichton

Jet Propulsion Laboratory

dan.crichton@jpl.nasa.gov

(818) 354-9155

L2banner.jpg