Distributed Information System--DATAGRAF Software System In Reply Refer To: November 2, 1984 WGS-Mail Stop 412 QUALITY OF WATER BRANCH TECHNICAL MEMORANDUM NO. 85.03 Subject: Distributed Information System--DATAGRAF Software System Through the efforts of the Quality of Water Branch, the Water Resources Division has purchased the DATAGRAF software system from M/A-Com Information Systems, Rockville, Maryland. DATAGRAF is a flexible, user-friendly interactive system for data analysis and graphic display. The system was designed especially for nonprogrammers or persons with neither the time nor the inclination to learn several different systems or write their own application software. The DATAGRAF user learns one system but has access to a full range of application programs which can come from any number of sources. DATAGRAF has been installed on both the Amdahl V7 mainframe computer and on the Headquarters Operations Prime (QVARSB); however, the two versions are slightly different. The primary difference between the Amdahl and Prime versions of DATAGRAF is that the Amdahl version includes an extensive interface to the Statistical Analysis System (SAS). About a dozen of the most commonly used procedures are available along with a U.S. Geological Survey (USGS) designed EXTRACT procedure for converting SAS data sets to DATAGRAF data sets and vice versa, and for filtering SAS data sets with Boolean logic expressions. Prime DATAGRAF utilizes the IMSL subroutine library for its statistical procedures and the relational data base management system, RIM, for the data management features (sorting, merging, aggregation, filtering, etc). If and when SAS becomes available on the Prime, the DATAGRAF interface to SAS will be downloaded to the Prime. The Prime version of DATAGRAF is more powerful than the Amdahl version because of the data management capabilities available from the RIM data base management system and the fact that the Quality of Water Branch, NAWDEX, and the National Water Summary program have concentrated their financial resources on the Prime version. In fact, the Amdahl version still has a few bugs related to the difficulty of working with the IBM TSO system. The remainder of this memo deals primarily with Prime DATAGRAF. Both the Amdahl and Prime versions of DATAGRAF have extensive mapping capabilities for States, counties, hydrologic boundaries, and Environmental Protection Agency (EPA) river reach traces. The program includes the capability for overlaying boundaries, point plotting for site locations, and quantitative representation of points and areas. The old and flawed hydrologic boundary files on the Prime have been replaced with newly digitized files. The initial list of application procedures currently available for Prime DATAGRAF includes data management, data display, basic statistics, business graphics, regression analysis and plotting, probability plotting, piper diagrams, box and whisker plots, and the comprehensive mapping capability. Details of these procedures are given in the appendix to this memorandum. Additional enhancements are under development and more are planned for fiscal year 1985. Background Several years ago M/A-Com Information Systems (then called Sigma Data) designed and built a system similar to DATAGRAF called UPGRADE for the President's Council on Environmental Quality (CEQ). The purpose of UPGRADE was to give the nonprogramming government manager an easy-to-use interactive computer capability to analyze and graphically display data. The USGS was one of several government agencies to participate in the UPGRADE project. The National Stream Quality Accounting Network (NASQAN) program, the NAWDEX program, and the Office of Water Data Coordination (OWDC) contributed funds for the development of additional graphic capabilities such as quantitative point plotting on maps. A few District Offices used the system on a test basis and the response was very enthusiastic. Unfortunately, the UPGRADE project was cancelled because of changing priorities at CEQ; however, the interest in the capabilities provided by UPGRADE continued to be of interest to several WRD programs. With the demise of the UPGRADE program, M/A-COM became interested in possible commercial applications of the UPGRADE concept; however, UPGRADE could not be marketed commercially because it was in the public domain. UPGRADE also had another problem that had to be resolved before M/A-COM could market this type of system. UPGRADE was not a well designed program from a coding point of view and had been patched together for several years. The result was a very difficult system to maintain or enhance with new procedures. M/A-COM then designed DATAGRAF utilizing more modern structured coding techniques and produced a modularized system that was easy to maintain and was capable of almost infinite expansion. The USGS has been actively involved in the design of specifications for many of the procedures currently operational in the two USGS versions of DATAGRAF, especially the mapping program. What is DATAGRAF? DATAGRAF is an easy to use interactive system for passing a formatted list of user specifications to an interfaced computer program for execution. When a procedure is completed, control is returned to the user in the DATAGRAF user interface where additional procedures can be specified. An important point to remember is that DATAGRAF has no inherent capability of its own other than the interfacing functions of accepting and verifying the user's specifications for a given procedure and then passing these specifications via a specifications table to the application program in question. Even the data management functions are carried out by a specially designed application program. 1 During a DATAGRAF session, specifications tables are filled in indirectly in prompt mode by the user answering questions prompted by the system, or directly in table mode by the use of assignment statements, e.g., color = blue. Once a table has been completed (and edited, if necessary) the information is passed to the procedure for execution along with the data required by the procedure. The data to be used in a given procedure are also specified (selected, subsetted, sorted, merged, filtered, etc.) according to instructions entered into a separate specifications table. Once a specifications table has been filled in, it can be saved for future (saving a table created in prompt mode is slightly more involved than in table mode but is still quite simple). The user then has the option of starting with an incomplete default table or using an already filled in table. In any case, a table can be used, edited, and resaved under different names as many times as needed. The number of application procedures available through DATAGRAF is potentially unlimited, depending only upon the number of procedures an organization wants to interface to it. Procedures may be added by M/A-COM through the contracting process, or USGS could add its own procedures utilizing an interactive procedure (NEWPROC) developed for USGS by M/A-COM. In any case, the new procedure is coded like any other computer program except that one or more common statements must be constructed to pass procedure arguments (if any) between the DATAGRAF user interface and the program. Once the program is written, it is interfaced to DATAGRAF using the NEWPROC procedure. NEWPROC allows the installer to interactively input (1) the user-help files, (2) the screen prompts used during prompt mode, and (3) other information required by DATAGRAF to format the specifications tables. A DATAGRAF procedure can be any computer program written in any language supported by the host computer. The program may be a home-grown routine or it may make extensive use of the major subroutine libraries such as DISSPLA, SAS, IMSL, etc. The procedure may also be a data base management system program for retrieving data from INFO, Prime DBMS, etc. Such a program, using instructions passed to it from a specifications table, would retrieve the required data out of the data base and put it into a PRIMOS flat file. This Primos file would then in turn be accesed by the DATAGRAF DATAIN procedure which would put the data into a DATAGRAF dataset. The user need not know anything about the data base system because someone would have already written a general purpose retrieval program and interfaced it to DATAGRAF. If DATAGRAF interfaces individual programs that ordinarily could run in a stand-alone mode, what function does DATAGRAF really serve? The DATAGRAF concept is desirable because it allows the user at any level of computer expertise to learn just one system which in turn provides all the necessary data management capabilities and a wide variety of procedures taken from any number of software resources. Comprehensive statistical and graphical procedures can be written with sufficient options to meet most of the routine needs of project personnel. Even the more specialized and less frequently used applications would still be maintained in an operational mode on a continuous basis. This continued availablility of application procedures within a familiar user environment is the basis of the design for the NWIS interactive controller and as the reason why DATAGRAF is being evaluated for possible use in the interactive controller. Regardless of the final decision regarding DATAGRAF and the interactive controller, most of the procedures in DATAGRAF should be usable in the NWIS or any other DIS application. USE OF DATAGRAF In order to use DATAGRAF, the potential user must first become a registered user on the Operations Prime (QUARSB) by calling Gail Kalen at FTS 928-7955. Before using DATAGRAF for the first time, a work area directory should be created because DATAGRAF will create six system files the first time it is executed in a given directory. These files are used to store data catalogs and files, procedure specifications tables, and other system-related information. Do not delete these files unless you are prepared to start again from scratch. To execute DATAGRAF, simply give the command-- DATAGRAF. Only one copy of DATAGRAF has been obtained for the Prime at this time and is installed on the Operation's Prime (QVARSB) in Headquarters. Plans have been made to move DATAGRAF to the National Node Prime (QVARSA) in the next few months when the Geologic Division moves its work to its own Prime. The Quality of Water Branch is encouraging all interested WRD personnel to use DATAGRAF and submit comments for improvements and any new procedures they would like to have added to the system. If the response to DATAGRAF is sufficiently positive, arrangements will be made to purchase additional copies for other DIS Primes. Draft documentation and additional information are available from Jim Schornick in the Quality of Water Branch, FTS 928-6834. David A. Rickert Attachment WRD Distribution: A, B, FO Key Words: Computer, statistics, graphics, data analysis, data management This memorandum does not supersede any previous memorandums.