CPAC GOAL 1:
Develop Policy on how WRD will select and support our corporate software and hardware.
A Method for Ranking WRD Corporate Software Support Activities
The purpose
of this document is to describe a method for ranking software packages deployed
nationally by the WRD to determine the relative importance of software-support
activities. A pair of indices are described for ranking the importance of
centrally provided national: (1) installation support, and (2) user support.
The goal of the index computations is to provide a more thorough and objective
basis for determining which software packages receive various levels of
support.
The two
indices do not provide guidance on several other pertinent issues related to
software deployment and support. The decisions related to these other issues
might be influenced by the results of the index computations, however the
indices are not designed to rank packages on the basis of these issues. The
additional issues pertain to decisions on whether to: (1) nationally procure
(or deploy) any particular package, (2) support a package on more than one
hardware/software environment, or (3) develop software to enhance, interface,
or integrate the package.
The index
score for each package is computed by summing the scores for several factors.
The factor scores are determined by characterizing a package for pairs of
support issues and finding the appropriate factor score in a factor matrix.
After index scores have been determined for all nationally deployed packages,
the packages are sorted and ranked by the index scores to determine the
relative importance of providing software support among the packages.
Installation-Support Index
Installation support is designed to help system administrators to successfully install a software package for a particular hardware and software environment. Different levels of support can be provided: (1) collation of the software installation files with other software packages to minimize the number software-installation activities, (2) preparation of guidance documentation regarding installation of the software package, (3) preparation of installation scripts to copy (or link) files to appropriate directories, (4) configuration for particular hardware environments, and (5) preparation (or integration) of software user documentation into the on-line "help" system.
The relative
need for installation support is determined by the installation-support index,
which is computed from several factors. The factors considered are described
below.
o EXTERNAL SUPPORT QUALITY: The quality and availability of external sources for installation support. The support might be provided by the software vendor or another contractual entity. External support is characterized as "Unavailable", "Poor", or "Good".
o VERSION CONTROL: The importance of version control and patches among the
various installations. If data will be exchanged among the users of the
software from different installations, it may be important that all users
employ the same version of the software (e.g.: SAS). Other times, the output
files of the software are version independent (e.g.: emacs). Version control is
characterized as: "Important", or "Unimportant".
o INSTALLATION COMPLEXITY: The number of options and actions required to
successfully install the software package. With more decisions and actions
required to install a package, it is more likely that benefits will be obtained
from additional guidance and prepackaging. Software-installation complexity is
characterized as: "Simple", "Average", or
"Complex". Support received
directly from the vendor should be considered.
o NUMBER OF INSTALLERS: The number of persons performing the package installation. Greater efficiency and standardization may be obtained by prepackaging software. With more persons puzzling through the installation, it is more likely that benefits will be accrued from some prepackaging. The number of installers is somewhat arbitrarily characterized as: "Less than 25", or "25 or more".
These four
factors are paired in factor matrices, for the purpose of determining factor
scores, as shown below. Characterizations for each of the four factors are
determined, then a pair of factor scores are determined from the DELIVERY and
SCOPE factor matrices. The two factor scores are added to compute the
installation-support index.
Installation Support Index
|
DELIVERY FACTOR
MATRIX |
|
||||
|
EXTERNAL
SUPPORT QUALITY |
|||||
|
Unavailable |
Poor |
Good |
|||
|
VERSION CONTROL |
Important |
3 |
2 |
1 |
|
|
Not Important |
2 |
1 |
0 |
||
Figure 1.
Delivery-factor matrix for the installation-support index.
|
SCOPE FACTOR
MATRIX |
|
||||
|
INSTALLATION COMPLEXITY |
|
||||
|
Simple |
Average |
Complex |
|
||
|
NUMBER OF
INSTALLERS |
< 25 |
0 |
1 |
2 |
|
|
>= 25 |
1 |
2 |
3 |
|
|
Figure 2.
Scope-factor matrix for the installation-support index
User Support Index
A process
analogous to the installation-support index is used to prepare a user-support
index. However, the levels of potential user support, and the factors
determining the need for user support are different. User support can be provided
by: (1) on-line help files, (2) tutorials, (3) lists of Frequently Asked
Questions (FAQ) and answers, (4) establishment of a newsgroup designated for
discussion of the package, (5) maintenance of e-mail mailing lists of package
experts (who agree to answer questions), or (6) centrally funded package
experts who participate in newsgroups, answer e-mail queries, and provide
telephone consultation.
The relative
need for user support is determined from user-support index, computed from
several factors. The factors considered are described below.
o EXTERNAL SUPPORT QUALITY: The timeliness and usefulness of user support provided
by an external entity. Similar to one of the factors for installation support,
this factor measures the quality of external support available to package
users, and is characterized as "Unavailable", "Poor", or
"Good".
o COST RATIO: The relative cost of obtaining the external support versus
providing software support using internal expertise. The cost of user support
may be significant, and savings may (or may not) be obtained by providing
internal expertise. All costs associated with internal support (including
salary, benefits, expert training, and overhead) are computed for the expected
package usage timeframe, and divided by the costs expected for obtaining
external support for the same timeframe. A ratio of internal versus external
user-support costs is thus determined. The ratio is characterized as "Half
or less", "Between half and double", or "Twice as much or
more". However, given the intrinsically hydrologic mission of the
organization and the pressure to use scarce manpower to support directly the
mission, the external cost is deprecated in determination of the factor score
(see figure 3).
o USE COMPLEXITY: The number of options, features, navigation paths, and
interfaces of a software package. The more decisions a user must make regarding
software use, the more likely it is that the user will require assistance.
Software-use complexity is characterized as: "Simple", "Average",
or "Complex".
o USER ABILITY: The computing proficiency of the expected user community.
The more experienced and capable computer users will need less assistance than
other less proficient users. User ability is characterized as: "Below
Average", "Average", or "Above Average".
o NUMBER OF USERS: The size of the supported user community. As the number of
supported package users increases, the variability of user abilities and
occurrences of unusual software problems increases. The number of package users
is rather arbitrarily characterized as: "Less than 100",
"Between 100 and 500", and "500 or more".
o SCOPE OF FAILURE: The organizational scope of the consequences for the
complete failure of the software to function at an installation. Software is
redundantly installed in the organization's distributed computing environment.
Each of these installations may be configured and maintained in different ways,
with some of the installations incorrect in some catastrophic manner. This
factor attempts to capture the consequences for the organization of the
complete failure of the software package at one installation on the network.
Scope of failure is characterized as: "Project", "Office",
or "Division".
"Project"
consequences occur if only a few projects located at the installation are
adversely affected by software failure (such as a ground-water model).
"Office" consequences occur if most or all of one office is adversely
affected (most of the currently supported software is in this category, e.g.:
spreadsheet, GIS, graphics, electronic publishing). "Division"
consequences occur if the package failure causes significant consequences
beyond the office where the software is installed. Examples of software with
potential divisional consequences are: NWIS, AIS, computer security, LRGS, and
e-mail.
These six
factors are paired in factor matrices, for the purpose of determining factor
scores, as shown below. Characterizations for each of the six factors are
determined, then factor scores are determined from the MARKET, FRUSTRATION, and
IMPACT factor matrices. The three factor scores are added to compute the
user-support index.
User Support Index
|
MARKET FACTOR MATRIX |
|
||||
|
EXTERNAL
SUPPORT QUALITY |
|
||||
|
Unavailable |
Poor |
Good |
|
||
|
COST RATIO |
<= 0.5 |
3 |
3 |
2 |
|
|
0.5 < x < 2.0 |
3 |
2 |
1 |
|
|
|
>= 2.0 |
3 |
1 |
0 |
|
|
Figure 3. Market-factor matrix for the user-support index.
|
FRUSTRATION FACTOR
MATRIX |
|
||||
|
USE
COMPLEXITY |
|
||||
|
Simple |
Average |
Complex |
|
||
|
USER ABILITY |
Below Average |
1 |
2 |
3 |
|
|
Average |
0 |
1 |
2 |
|
|
|
Above Average |
0 |
0 |
1 |
|
|
Figure 4. Frustration-factor matrix for the user-support index.
|
IMPACT FACTOR
MATRIX |
|
||||
|
NUMBER OF
USERS |
|
||||
|
< 100 |
100 – 500 |
500 |
|
||
|
SCOPE OF FAILURE |
Project |
0 |
1 |
2 |
|
|
Office |
1 |
2 |
3 |
|
|
|
Division |
2 |
3 |
3 |
|
|
|
Outside |
2 |
3 |
3 |
|
|
Figure
5. Impact-factor matrix for the SA
user-support index.
Product Support Index
A process to prepare a product support index is
basically a yes or no answer to the type of resource needing support.
The internal need to provide is based on whether
there is any outside support available and if so, if used.
The type of resource for which internal support is or
is not needed is in the form of technical support, updates and releases of
software, training, and if there is a need for investigation of new tools.
This Scope of Support Factor Matrix is used to
compute the product support index.
Product Support Index
|
SCOPE OF
SUPPORT FACTOR
MATRIX |
|
||||
|
INTERNAL
NEED TO PROVIDE |
|
||||
|
YES |
NO |
|
|
||
|
RESOURCE |
Technical |
1 |
0 |
|
|
|
Updates & Release |
1 |
0 |
|
|
|
|
Training |
1 |
0 |
|
||
|
Investigationof
New Tools |
1 |
0 |
|
||
Figure 6.
Scope of support factor matrix for product support index.
Limitations to
this Method
The index
approach to software-support prioritization is not a panacea whereby anyone can
objectively and reproducibly determine the relative needs of the Division. The
approach has several limitations.
First,
continuous factor variables have been simplified as categorical variables. This
truncation results in loss of precision in the computed index score. Therefore,
judgment must be used when ranking similar or equal index scores. However,
software packages with grossly different index scores should probably receive
different levels of national support.
Second,
different weighting factors within the factor matrices could be assigned. Those
provided herein incorporate the author's judgments and biases. However,
computation of an index score does require the consistent evaluation of
multiple factors when determining how much support is needed for different
software packages.
Third, the
characterization of the various factors is, at times, a subjective process. The
biases resulting from factor characterization can be mitigated by having
several individuals evaluate the factors and compute index scores. The factor
scores or the index scores can be averaged from multiple responses to provide a
more balanced perspective.
A Method for Ranking WRD Corporate
Hardware Support Activities
The purpose
of this document is to describe a method for ranking hardware systems
(platforms) deployed nationally by the WRD to determine the relative importance
of hardware-support activities. A pair of indices are described for ranking the
importance of centrally provided national: (1) installation support, and (2)
user support. The goal of the index computations is to provide a more thorough
and objective basis for determining which hardware systems receive various
levels of support.
The two
indices do not provide guidance on several other pertinent issues related to
hardware deployment and support. The decisions related to these other issues
might be influenced by the results of the index computations, however the
indices are not designed to rank hardware on the basis of these issues. The
additional issues pertain to decisions on whether to: (1) nationally procure
(or deploy) any particular system, (2) support a system, or (3) develop
software to enhance, interface, or integrate the systems.
The index score for each system is computed by summing the scores for several factors. The factor scores are determined by characterizing a package for pairs of support issues and finding the appropriate factor score in a factor matrix. After index scores have been determined for all systems used in WRD, the systems are sorted and ranked by the index scores to determine the relative importance of providing hardware support among the systems.
Installation-Support Index
Installation
support is designed to help system administrators to successfully install an
operating system for a particular hardware environment. Different levels of
support can be provided: (1) collation of the hardware installation files and
service patches to minimize the number of installation activities, (2)
preparation of guidance documentation regarding installation of the operating
software package, (3) preparation of installation scripts to copy (or link)
files to appropriate directories, (4) configuration for particular hardware
environments, and (5) preparation (or integration) of hardware system user
documentation into the on-line "help" system.
The relative
need for installation support is determined by the installation-support index,
which is computed from several factors. The factors considered are described
below.
o EXTERNAL SUPPORT QUALITY: The
quality and availability of external sources for installation support. The
support might be provided by the hardware vendor or another contractual entity.
External support is characterized as "Unavailable", "Poor",
or "Good".
o VERSION CONTROL: The
importance of version control and patches among the various installations. If
data will be exchanged among the users of the operating systems from different
installations, it may be important that all users employ the same version of
operating system. Version control is characterized as: "Important",
or "Unimportant".
o INSTALLATION COMPLEXITY: The
number of options and actions required to successfully install the operating
system software package. With more decisions and actions required to install a
package, it is more likely that benefits will be obtained from additional
guidance and prepackaging operating systems. Software-installation complexity
is characterized as: "Simple", "Average", or
"Complex". Support received directly from the vendor should be
considered.
o NUMBER OF INSTALLERS: The
number of persons performing the package installation. Greater efficiency and
standardization may be obtained by prepackaging operating system software. With
more persons puzzling through the installation, it is more likely that benefits
will be accrued from some prepackaging. The number of installers is somewhat
arbitrarily characterized as: "Less than 25", or "25 or
more".
These four
factors are paired in factor matrices, for the purpose of determining factor
scores, as shown below. Characterizations for each of the four factors are
determined, then a pair of factor scores are determined from the DELIVERY and
SCOPE factor matrices. The two factor scores are added to compute the
installation-support index.
Installation-Support Index
|
DELIVERY FACTOR
MATRIX |
|
||||
|
EXTERNAL SUPPORT
QUALITY |
|
||||
|
Unavailable |
Poor |
Good |
|
||
|
VERSION CONTROL |
Important |
3 |
2 |
1 |
|
|
Not Important |
2 |
1 |
0 |
|
|
Figure 7. Delivery-factor matrix for the installation-support index.
|
SCOPE FACTOR
MATRIX |
|
||||
|
INSTALLATION COMPLEXITY |
|
||||
|
Simple |
Average |
Complex |
|
||
|
NUMBER OF
INSTALLERS |
< 25 |
0 |
1 |
2 |
|
|
>= 25 |
1 |
2 |
3 |
|
|
Figure 8. Scope-factor matrix for the installation-support index.
System Administrator/User Support Index
A process analogous to the installation-support index is used to prepare a user-support index. However, the levels of potential System Administrator (SA) user support, and the factors determining the need for user support are different. User support can be provided by: (1) on-line help files, (2) tutorials, (3) lists of Frequently Asked Questions (FAQ) and answers, (4) establishment of a newsgroup designated for discussion of the package, (5) maintenance of e-mail mailing lists of package experts (who agree to answer questions), or (6) centrally funded package experts who participate in newsgroups, answer e-mail queries, and provide telephone consultation.
The relative need for SA user support is determined from user-support index, computed from several factors. The factors considered are described below.
o EXTERNAL SUPPORT QUALITY: The
timeliness and usefulness of SA user support provided by an external entity.
Similar to one of the factors for installation support, this factor measures
the quality of external support available to package users, and is
characterized as "Unavailable", "Poor", or
"Good".
o COST RATIO: The
relative cost of obtaining the external support versus providing hardware
support using internal expertise. The cost of user support may be significant,
and savings may (or may not) be obtained by providing internal expertise. All
costs associated with internal support (including salary, benefits, expert
training, and overhead) are computed for the expected package usage timeframe,
and divided by the costs expected for obtaining external support for the same
timeframe. A ratio of internal versus external user-support costs is thus
determined. The ratio is characterized as "Half or less",
"Between half and double", or "Twice as much or more".
However, given the intrinsically hydrologic mission of the organization and the
pressure to use scarce manpower to support directly the mission, the external
cost is deprecated in determination of the factor score (see figure 9).
o USE COMPLEXITY: The number of options, features, navigation paths,
and interfaces of a software package. The more decisions a user must make
regarding software use, the more likely it is that the user will require
assistance. Software-use complexity is characterized as: "Simple",
"Average", or "Complex".
o USER ABILITY: The
computing proficiency of the expected user community. The more experienced and
capable computer users will need less assistance than other less proficient
users. User ability is characterized as: "Below Average",
"Average", or "Above Average".
o NUMBER OF USERS: The
size of the supported user community. As the number of supported package users
increases, the variability of user abilities and occurrences of unusual
software problems increases. The number of package users is rather arbitrarily
characterized as: "Less than 100", "Between 100 and 500",
and "500 or more".
o SCOPE OF FAILURE: The
organizational scope of the consequences for the complete failure of the
software to function at an installation. Software is redundantly installed in
the organization's distributed computing environment. Each of these
installations may be configured and maintained in different ways, with some of
the installations incorrect in some catastrophic manner. This factor attempts
to capture the consequences for the organization of the complete failure of the
software package at one installation on the network. Scope of failure is
characterized as: "Project", "Office", or
"Division".
"Project" consequences occur if only a few projects located at the installation are adversely affected by software failure (such as a ground-water model). "Office" consequences occur if most or all of one office is adversely affected (most of the currently supported software is in this category, e.g.: spreadsheet, GIS, graphics, electronic publishing). "Division" consequences occur if the package failure causes significant consequences beyond the office where the software is installed. Examples of software with potential divisional consequences are: NWIS, AIS, computer security, LRGS, and e-mail.
These six
factors are paired in factor matrices, for the purpose of determining factor
scores, as shown below. Characterizations for each of the six factors are
determined, then factor scores are determined from the MARKET, FRUSTRATION, and
IMPACT factor matrices. The three factor scores are added to compute the
user-support index.
System Administration/User
Support Index
|
MARKET FACTOR MATRIX |
|
||||
|
EXTERNAL
SUPPORT QUALITY |
|
||||
|
Unavailable |
Poor |
Good |
|
||
|
COST RATIO |
<= 0.5 |
3 |
3 |
2 |
|
|
0.5 < x < 2.0 |
3 |
2 |
1 |
|
|
|
>= 2.0 |
3 |
1 |
0 |
|
|
Figure 9. Market-factor matrix for the user-support index.
|
FRUSTRATION FACTOR
MATRIX |
|
||||
|
USE
COMPLEXITY |
|
||||
|
Simple |
Average |
Complex |
|
||
|
USER ABILITY |
Below Average |
1 |
2 |
3 |
|
|
Average |
0 |
1 |
2 |
|
|
|
Above Average |
0 |
0 |
1 |
|
|
Figure 10. Frustration-factor matrix for the user-support index.
|
IMPACT FACTOR
MATRIX |
|
||||
|
NUMBER OF
USERS |
|
||||
|
< 100 |
100 – 500 |
500 |
|
||
|
SCOPE OF FAILURE |
Project |
0 |
1 |
2 |
|
|
Office |
1 |
2 |
2 |
|
|
|
Division |
3 |
3 |
3 |
|
|
Figure 11. Impact-factor matrix for the SA user-support index.
Product Support Index
A process to prepare a product support index is
basically a yes or no answer to the type of resource needing support.
·
Vendor
Support Quality
Determining the level of vendor support quality for the need.
·
Need
Determine the need for cost to buy, maintenance costs, peripherals available,
performance, and reliability.
The internal need to provide is based on whether
there is any outside support available and if so, if used.
·
Resource
The type of resource for which internal support is or is not needed is in the
form of technical support, updates and releases of software, training, and if
there is a need for investigation of new tools.
The Hardware Systems Factor Matrix and the Scope of
Support Factor Matrix are used to compute the Product Support Index.
Product Support Index
|
HARDWARE
SYSTEMS FACTOR MATRIX |
|
||||
|
VENDOR
SUPPORT QUALITY |
|
||||
|
LOW |
MEDIUM |
HIGH |
|
||
|
NEED |
Cost
to Buy |
3 |
2 |
1 |
|
|
Maint. Costs |
3 |
2 |
1 |
|
|
|
Peripherals Available |
3 |
2 |
1 |
||
|
Performance |
3 |
2 |
1 |
||
|
Reliability |
3 |
2 |
1 |
||
Figure
12. Hardware systems factor matrix for
product support index.
|
SCOPE OF
SUPPORT FACTOR
MATRIX |
|
||||
|
INTERNAL
NEED TO PROVIDE |
|
||||
|
YES |
NO |
|
|
||
|
RESOURCE |
Technical |
1 |
0 |
|
|
|
Updates & Release |
1 |
0 |
|
|
|
|
Training |
1 |
0 |
|
||
|
Investigation
of New Tools |
1 |
0 |
|
||
Figure
13. Scope of support factor matrix for
product support index.
Limitations To This
Method
The index approach to hardware
systems prioritization is not a panacea whereby anyone can objectively and
reproducibly determine the relative needs of the Division. The approach has
several limitations.
First, continuous factor variables
have been simplified as categorical variables. This truncation results in loss
of precision in the computed index score. Therefore, judgment must be used when
ranking similar or equal index scores. However, hardware systems with grossly
different index scores should probably receive different levels of national
support.
Second, different weighting
factors within the factor matrices could be assigned. Those provided herein
incorporate the authors' judgments and biases. However, computation of an index
score does require the consistent evaluation of multiple factors when
determining how much support is needed for different hardware systems.
Third, the characterization of
the various factors is, at times, a subjective process. The biases resulting
from factor characterization can be mitigated by having several individuals
evaluate the factors and compute index scores. The factor scores or the index
scores can be averaged from multiple responses to provide a more balanced
perspective.
FUTCOM3 Software Table
(attached)