Unidata User Support

Tom Yoksas
August 25, 2006

Unidata Inquiry Tracking System

The UPC's transition to the use of a new inquiry tracking system was announced to the community in Unidata Transitions to a New Inquiry Tracking System. We interpret the fact that we have received no end-user comments about the new system to mean that the transition was smooth, problem free, and painless for the community. This was one of our original transition goals.

Since January 26, over 2900 community member "transactions" (new inquiries and follow-ups) have been processed through the Unidata inquiry tracking system (SupportSuite from Kayako Infotech Ltd). Comments by Unidata staff indicate that transitioning to use of the new system occurred without difficulty. It typically takes only a small number of interactions with the system before it feels natural to the UPC staff providing end-user support. OPeNDAP experts outside of the UPC were given access to the tracking system for DODS/OPeNDAP inquiries. It is our intention to allow non-UPC, domain-specific, expert users access to the tracking system if they are willing to assist in the direct support of community members.

The next phase in our transition will be the integration of web-based access to the new system by end-users. This effort will require integration with our single sign on management system (CAMS) and conversion of the user interface to the look-and-feel of our website. We expected to unveil the web-based access by Users Workshop that was held in July, but other, higher priority activities forced a delay in our schedule. The current timeline is to unveil the web-based access to the tracking system sometime after the completion of the Fall 2006 series of training workshops.

A Metric for International Use of Unidata-supported Software Packages

Following the example set in the NetCDF Status report, an effort was made to quantify the international use of several software packages supported by the UPC: netCDF, LDM, THREDDS, McIDAS, IDV, UDUNITS, and GEMPAK. We used the number of web pages per million in each domain of interest that include the name of the package (without regard to capitalization) on all web pages indexed by Google™. Call this metric nGh/Mwp (number of Google hits per million web pages). We then combined the numbers for each package by continent. The results, ordered by netCDF rank, are shown in the following table:

 

Unidata Software Usage Ranking (nGh/Mwp) by Continent on August 17, 2006

Grouping	netCDF	LDM	THREDDS	McIDAS	IDV	UDUNITS	GEMPAK
Australia	46.90	0.09	0.07	0.08	0.03	1.14	0.00
North America	25.73	5.46	2.41	2.51	2.04	1.09	1.82
Europe	20.79	0.17	0.13	0.16	0.03	1.04	0.00
Asia	9.92	0.50	0.03	0.51	0.15	0.78	0.01
South America	4.63	0.28	0.03	0.27	0.04	0.63	0.07
Africa	1.21	0.02	0.00	0.00	0.00	0.21	0.00
Antarctica	0.00	0.00	0.00	0.00	0.00	0.00	0.00

Comments:

• since the Google search algorithm appears to vary periodically, the web page reference numbers were collected within a 4 hour period
• since attempts to query the Google search engine programatically (Rew) were rebuffed by Google, all queries were done "by hand". The procedure is laborious and time consuming.
• the search key used for GEMPAK, IDV, LDM, and McIDAS were limited to pages that also contained the word unidata (e.g., 'GEMPAK +unidata'). This restriction was necessary since all keys except McIDAS are not uniquely identified with the software package of the same name. The search for McIDAS was restricted in an attempt to differentiate between the SSEC and Unidata distributions of the package.
• there are no web pages that use the country code for Antarctica (.aq). The null Antarctic numbers are included so that all seven continents are represented in the table.
• the metrics presented are meant to be provocative; their interpretations are subject to debate. We believe, however, that the numbers clearly show that netCDF is the most widely used Unidata package both domestically and internationally.

User Support by Domain - Historical Perspective

As part of the overall effort to provide metrics for the Unidata Program evaluation effort, the User Support email archives backend of our inquiry tracking system was mined to determine the effort expended for the various communities we support. The following figure, which represents part of this investigation, depicts the volume of support expended for calendar years 2000 - 2006, inclusive.

Unidata Support by Domain for the period January 1, 2000 - August 27, 2006

Comments:

• the number of responses represent emails recorded in our inquiry tracking system sent by User Support personnel to end users. Since it does not include numbers from individuals providing support outside of the inquiry tracking system, it is a underestimate of the level of effort expended.
• the numbers for calendar year 2006 (depicted as 2006p) are a projection that assumes that the inquiry rate is evenly distributed throughout the year.
• the X axis labels represent:

  US	responses to email addresses ending in: .bitnet, .biz, .com, .dnet, .edu, .gov, .info, .mil, .net, .org, .us, and .uucp
  EDU	responses to email addresses ending in .edu
  GOV	responses to email addresses ending in .gov
  INT	responses from domains other than those in US
  COM	responses to email addresses ending in .com
  MIL	responses to email addresses ending in .mil
  ORG	responses to email addresses ending in .org
  NET	responses to email addresses ending in .net

• overall, the level of support effort is reasonably consistent from year-to-year with the exception of calendar year 2000 in which there was a greater than average level of support for the US research and education community (.edu suffix)
• there is a subtle, and yet apparent trend for increased support to groups outside of the Unidata core community. Support of US government (GOV), international (INT), commercial entities (COM), and organizations (ORG) is steadily increasing.
• not depicted in the graph above is the level of support effort expended for the totality of the Unidata program tracking system archive. The effort represented for calendar years 1994 - 2006 are:

  Total	71914
  US	66843
  EDU	53131
  GOV	6972
  INT	5071
  COM	4513
  MIL	971
  ORG	560
  NET	515

• the number of foreign countries supported by year are:

  1994-2006	87
  2000	32
  2002	42
  2002	40
  2003	36
  2004	43
  2005	44
  2006	40 (projected)

Unidata User Support Inquiry Metrics

Above are histograms that portray the number of Unidata email responses for categories of support for a one year period ending August 23, 2006. The histograms are arranged by yearly activity averages with the highest on the left and lowest on the right. Each quarter year within the period is depicted from oldest to newest from left to right. The number of responses has been normalized to weekly averages so that the support load over the various periods can be easily compared.

Total support averaged 148 responses/week over the entire year; 140 for the first quarter; 132 for the second quarter; 157 for the third quarter; and 164 for the current quarter.

Individual support activities included in the categories depicted above are listed in the following table.

Category	Packages, Groups, and Lists
data	casestudies, casestudies-list, conduit, c2-linda, craft, craft-nws, craft-ty, datastream, difax, level2, level2-linda, level2-ty, noaaport, noaaport-ty, noaaportldm
dods	dods, dods-core, dods-list, dods-tech, dods-team, linda-dods, opendap
gempak	gempak, gembud-list
ldm	ldm, ldm-users-list
lead	lead
idd	cluster, idd, idd-antarctic, idd-brasil, idd-caribe, idd-inject, idd-status, scoop, suominet, tigge, venezuela
idv	idv, idvlist, idvsteering, metapps, visad-list, visad-renderer
mcidas	mcdevelop, mcidas, mcidas-list
miscellaneous	esupport, fxlinux, license, misc, network, notrack, platforms, wxp, wxp-list
netcdf	data-models, libcf, ncml, netcdf, netcdf-miss, netcdfgroup-list, netcdf-hdf-list, netcdf-java, netcdf-perl
nws-changes	nws-changes
outreach	agu-linda, agu-ty, ams-linda, announce, barbados-ty, cbmet-ty, chile-ty, egrants, egu-linda, external, iai-ty, meteoforum-ty, noaa-linda, nws-linda, unidata, workshop
thredds	java-dev, java-dev-list, thredds
utilities	decoders, ldm-mcidas, udunits

Comments:

• The LEAD numbers are an underestimate of its activity since very few LEAD email transactions go through User Support.
• Support numbers for individual packages, groups, and lists can be seen in full support statistics.

Some tentative conclusions:

• The total support provided by the UPC remains high. Overall support activities are for the most part seasonally invariant. The numbers presented are virtually identical to those reported in the Spring 2006 User Committee Support status report.
• Support required for the legacy visualization packages GEMPAK and McIDAS continues to be substantial with GEMPAK leading the way as the most widely distributed analysis application. The IDV support load has surpassed that for McIDAS for the first time.
• Support required for LDM, IDD, and data continues at a high level and shows some variability throughout the year.
• Support for netCDF remains substantial, but is considered modest given the number of users of the package worldwide.
• The overall higher numbers for outreach reflect inclusion of more categories of support/outreach provided by Linda and the high level of activity in organizing a special session for the Spring 2006 AGU Joint Assembly that was held in Baltimore, MD on 23-26 May.
• Taken as a whole, the support required for visualization packages (GEMPAK, IDV, and McIDAS) is comparable to the support related to data reception (LDM, IDD, noaaport).
• Support for any package increases after a new version of the package is made available.

Notes:

These numbers and conclusions should not be taken too literally, for several reasons:

• For some packages, multiple responses in the same thread may be bundled into a single archived email. Other packages have each response in a thread counted separately.
• After a new release of software, there may be a flurry of the same or similar questions, which can be answered in separate emails or in a single mailing list posting.
• The graph primarily represents support of end users and site administrators, not developers. Support for non-Unidata developers in projects such as OPeNDAP/DODS, THREDDS, IDV, GEMPAK, and McIDAS requires significant resources, but is difficult to assess.
• Not all support records were indexable for this report. Given this, the above numbers are an underestimate of the actual support being provided by the UPC.

by Tom Yoksas