Water Resources Division ITAC Home Page

Nevada ITAC "Charting Software" Discussion

From:	Thomas C Wood
To:	GS-W ITAC
Cc:	Thomas C Wood/WRD/USGS/DOI@USGS, Alan M Lumb/WRD/USGS/DOI@USGS
Date:	01/11/2002 11:40 AM
Subject:	Information for ITAC Charting Software Discussion

ITAC,

Please see the attached/embedded information related to the charting software discussion at our meeting next week:

1. Powerpoint presentation or (pdf format) for the 1/15-17/02 meeting
2. Word document on NWIS Graphics for background (pdf format)
3. Summary of selected field personnel comments on charting software in WRD, included below in this email message, for background information.

Thanks, Tom

Comments from Lucky Sultz, Hydrologic Technican, MT District:

"I can't speak to the publications issues however, from a technicians viewpoint, I am a very big fan of Tkg2 for producing graphics for records processing. I am only aware of a couple of points where Tkg2 does not produce proper publication formats. Tkg2 is basically a rewrite of G2 in Perl that has vastly outstripped the functionality and usability of G2. I was an avid user of G2, but since switching to Tkg2, I could never go back to the limitations of G2. I would think that if anyone who had previously used G2 were to understand how to use Tkg2, they would feel the same way. Tkg2 is very flexible and stable. It can produce linear, log, probability, time-series, bar plots, boxplots, and several other types of plots. It is very flexible as to the file types for input and produces output in PostScript, MIF, PDF, and PNG formats.

Tkg2 was written to make full use of batch processing and scripting and is a very easy way to create custom graphics. The Montana District is running shell scripts to create all our SW records plots this year, bypassing the ADAPS graphics and I have heard nothing but praise for the ease with which Tkg2 can create these plots. We have used Tkg2 for a couple of years and have been using the latest pre-release version for a week or so. The Tkg2 developers have been VERY receptive to my requests for changes and/or additions and there is very little that Tkg2 cannot do at this point to meet our needs.

I don't think there will ever be a single, commercial package that will meet all the needs of NWIS and pubs folks, but I think Tkg2 comes pretty close. Just as with G2, the Tkg2 graphics may need to be imported into a publication package for final tweaking, but it can do the vast majority of the work needed. I have used S-Plus a little (very little), but in my opinion it does not hold a candle to Tkg2 for ease of use and the graphics it produces are not nearly as flexible as those from Tkg2. The best thing about Tkg2 is that we own not only the program, but the developers as well, and our needs come first in developing new functionality.

The main problem with Tkg2 right now is that there is very little documentation and very few folks who understand what it can do. I get questions almost daily about how to get Tkg2 to do a particular function that is available, but not documented. There has been talk, and there is a real need for some sort of workshop on advanced Tkg2 use and scripting/batch processing. Perhaps something like that would be a good opportunity for some publication folks to get a better understanding of Tkg2 and to exchange ideas with the developers.

Shameless plug: If you are unfamiliar with Tkg2, see the Montana Technicians Page at; http://montana.usgs.gov/office/techfiles/tkg2/intro.htm . It is somewhat out of date at this point, being based on an early version, but still a good introduction to Tkg2."

Comments from Burl Goree, Hydrologic Technician, LA District:

"I waited for Lucky Sultz to reply first because I knew he would be a proponent of tkg2 and I didn't want to have to write some of the same praise, for I also use tkg2. It does a nice job of providing our office with the graphics we need. Having said that I would like to make the some other points.

Although tkg2 does a good job there are many functions/options that as a technician I will never use. I still believe people need to recognize the differences between the groups that are being targeted (Technicians, Hydrologists, Illustrators). The reason you can't find a graphics package that will make everyone happy is because no one realizes that the people you are trying to group together are not compatible in their needs/requirements. I (a technician) am interested in displaying and plotting values in order to qualify, estimate, and edit data. The only graphs that I publish are very crude hydrographs in our annual data report. The other 99% are used in house. As a technician, I would like to see the qualifying and editing tools merged with our plotting programs.

I couldn't care less whether the ticks were inside, outside or upside down. It doesn't matter to me whether the colors on the screen accurately reflect the RGB values when printed or not; and to this day I haven't used step plots for plotting daily mean values. If you talk with Illustrators I'm sure you will find that they couldn't care less about whether the data contains bad values from ice effect, silted can, or corrupt DCP transmissions. I believe Hydrologist to be somewhere in-between looking for a graphics package that has at least basic statistical functions.

When you try to combine the graphics requirements of Technicians, Hydrologists, and Illustrators you end up not pleasing anyone. I see the need for the report writers to have at least a common file format with the publishers but I don't think they need the same graphing program and we (non-report writing) technicians just want good tools to make sure that the data that drives everything is accurate.

Since we end up supporting two or three different packages anyway, has WRD looked into addressing the needs of Technicians, Hydrologists, and Illustrators separately?"

Additional Comments from Lucky Sultz:

"Burl makes a very valid point here that I would agree with. Although in Montana we do much better than "very crude hydrographs" for our annual report :> He points out there there really is a need for at least two graphics program. Project folks and others have a need for a statistical package such as S-Plus or SAS that can compute and display data relationships. As data folks we have a program (ADAPS) that does most, if not all, of our data computations. What we need is just a way to display the finished data in a meaningful way. While we would like to have our display software better integrated with our database program, it is not necessary for the charting software to have the ability to create data relationships and we don't need the added complexity of a program like S-Plus."

Comments from Keith Kirk, WR Reports Specialist, specifically addressing quotes from Greg Granato's email to Bob Hirsch:

"Here is my two cents on the G2 issue:/P>

"G2 is currently the only program that will produce graphs with the proper format (for example, proper fonts, proper layouts, with the ticks on the inside on all four sides, etc.).""

This is incorrect,

1. S-Plus also has this capability and it may be configured so that the "proper format" is the default. The CAPP web site gives details of how this is accomplished, see URL:  http://water.usgs.gov/usgs/publishing/Production/graphmodifications.pdf
2. Tkg2, a Perl script based program developed in TX district, has much of the utility of G2 and also meets these format requirements. Tkg2 also produces PDF file output. PDF files are now the preferred graphic format for Illustrator 9 or higher according to Adobe.

"G2 will also produce graphs in postscript formats compatible with publications graphics formats."

- G2 does create postscript formatted output, so do most other graphics packages including S-Plus and Tkg2. In a Windows, UNIX or Mac environment this is irrelevant since virtually any document (text or graphics) can be printed through a postscript driver to produce a postscript file that can be edited in Illustrator, which is the current (January 2002) "standard" graphics package for WRD pubs units.

Consider S-Plus has a robust C++ interface this allows the development of special S-Plus "projects" that could perform the same functions as G2 while maintaining user friendliness and batch processing. This may be a better path to pursue than expending effort on revitalizing an old software package such as G2. My personal experience from managing a national computer system is that it is much more cost effective in the long run to allow the "off-the-shelf" software vendor to develop and maintain the needed capability than to have the fed do the development. In general the fed has a rather poor record of keeping software developed in house up to date. Our developers tend to retire and we are yet again left holding the bag.

However, consider in the short-term better documentation for Tkg2, it has much if not all of the utility of G2 and it runs in a Windows environment using an UNIX emulator."

Comments from Joyce Williamson, Hydrologist, SD District:

"I think that most issues concerning comparisons of G2 to Splus from a hydrologists perspective have been made by Keith. I will add a couple of items. For the hydrologist I work with, and myself also, G2 was what we had used for a long time and generating plots was pretty easy. The problem with Splus is that we have to learn a new package. I get similar feedback from the techs here regarding TKG2. However, a few quick "how-to's" and people are up and running. They may not be as comfortable as with G2, but they will get there. Our DG died a couple of months ago so we are currently in that learning curve. Here are the strengths and weaknesses of Splus as I see them.

• Publication requirements can be set as the default for graphs in Splus.
• Scripts can be written and run to facilitate mass production of graphs as well as reproduction of the same graph.
• Postscript is an option for graph output including directly from scripts.
• Splus doesn't crash when I am in the middle of my plots.
• So far, Splus can not generate a boxplot on censored data. This is being worked on.
• Splus has some limitations in regards to setting mins/maxs on axes and with date/time axes. There are ways to work around these but it is not straightforward.
• Time series data takes some additional manipulation to get in the proper format.

Greg is correct that when G2 is gone, you lose all those old graphs. But that was the case when we changed from whatever it was on the prime to G2 too."

Address questions or comments to Charles Merk, ITAC Secretary cfmerk on (703) 648-5680