Re: [idvusers] [THREDDS #GCY-293173]: THREDDS 4.6 and Mixed Interval Variables

Dear THREDDS Support-

Not all IDV users are on an academic schedule. I use this bundle continually through the year (daily during times of precip) and it is served up by my RAMADDA server to all my users. In this case, I do have to remake the display (unless I edit the bundle) which means I have to redo the formula and set any colortables, ranges, etc that I had in the original bundle (if I can remember what they were). It's not a trivial issue as you make it seem.

Don

On 5/18/15 1:10 PM, Unidata THREDDS Support wrote:
Hi All,

Don - thank you for your initial report of the issue, and apologies for the 
delay. Given that
the academic summer semesters typically start at the beginning of June, there 
should
be enough time to address these changes.

I'm pretty sure the bundle will load what it can, and the new field can be 
added quite easily
(which I've done with your RR 20km bundle Don - would you like me to send a 
copy?). I do
not believe there is any need to remake a bundle from scratch in this case.

In terms of this change specifically, if anyone has issues with their bundles 
not being able
to be salvaged due to the mixed interval variable name changes, please send a 
copy of your
bundle to support@xxxxxxxxxxxxxxxx and we will get it fixed.

If you're feeling adventurous, you  can use the list of name changes provided 
below to
replace the old names in the .xidv  bundle file yourself, since xidv bundles 
are just text files,
which is exactly what we will be doing here. Generally it's a bad idea to hand 
edit bundle
files as they tend to be very brittle and not at all forgiving when it comes to 
hand editing,
but variable name changes are pretty straight forward. You would basically need 
to change
occurrences of the string "_Mixed_Intervals" with "_1_Hour" or "_3_Hour", 
depending on
the correct name.

Again, no need that I can see to remake a bundle.

Cheers,

THREDDS Support


Thank you for finally addressing this issue that was reported several weeks 
ago.  I wish
  there could have been more notification than on the day of the server switch 
so I could
  have time to fix things.

One point of clarification.  Loading a bundle in IDV 5.1u2 will not prompt you 
for a new variable
  name. It just gives an error that the variable can't be found and hence no 
display is created.
  This means that anyone using bundles that have variables with new names will 
have to
  remake their bundles.

Sigh.....

Don

Greetings IDV users,

As thredds.ucar.edu is switched to the 4.6 version of the THREDDS Data Server 
(TDS),
you may notice the same names of variables have changed. Specifically, you will 
likely
encounter the case where Mixed Interval variable have been renamed to indicate 
that
they are no longer mixed, but rather fixed interval variables. As an example, 
if we consider
the Rapid Refresh Model, the following old variable name:

Convective_precipitation_surface_Mixed_intervals_Accumulation

has been changed to:

Convective_precipitation_surface_1_Hour_Accumulation

Why did this happen? The THREDDS group has looked over all cases of Mixed 
Interval
Variables in the data we serve out and we noticed that, in some cases, mixed 
intervals were
not needed, and we we able to reduce these to single interval variables.

Again, considering the Convective Precip variable from the Rapid Refresh, the 
old variable:

Convective_precipitation_surface_Mixed_intervals_Accumulation

actually contained two types of intervals over which the accumulation occurred 
(thus
the "mixed interval" in the name). One interval was the 1 hour interval:

accumulation over forecast hours 0-1, 1-2, 2-3, 3-4., etc.

The second interval contained a total run accumulation:

accumulation over forecast hours 0-1, 0-2, 0-3, 0-4, etc.

When served as a mixed interval, the data would look something like:

accumulation over forecast hours 0-1, 0-1, 1-2, 0-2,  2-3, 0-3...

The end result is that, when displayed in the IDV, the precip field would 
appear to
"jump" around in magnitude when animated. However, given the simplicity of how
these intervals were actually "mixed", we were able to, on the server side, 
remove
the total accumulation over the run portion of the grids while only serving the 
1 hour
accumulation grids. Since the total run accumulation can be computed from the
1 hour interval accumulations, this seems like the reasonable thing to do.

Depending on the model, the mixed interval grid may be rename as a
1_hour_Accumulation, or a 3_hour_Accumulation (other interval lengths
will be possible in the future, I'm sure). That means replacing all of the Mixed
Interval Variable names with a 1 hour Accumulation variable name is not a
safe option.

After much discussion with the IDV group over the past year+, we've agreed that
in these cases (where data fundamentally change meaning), it's best to allow
users of the IDV to see that these changes have occurred. We are not big fans of
silently changing the meaning of the variables. Further, we aren't sure how 
aware
users are of the complexity of these mixed interval data, so this will bring 
these issues
to the forefront. For users of IDV  bundles, this means you will need to 
re-save your
bundle after being prompted to select a new variable.

We hope you find these reduced mixed interval variables are more useful, and
please know that we are working on a solution for the more complex types of
mixed intervals (for example, every mixed interval in the GFS output). Because
these more complex mixed interval variables still show up as Mixed Interval
variables (and are very nuanced), be very cautious when using them as they are
likely NOT what you think or hope they might be.

In the coming weeks we will be doing a series of blog posts explaining in detail
how these mixed interval grids come about, and why there are cases where
the reduction of a mixed interval variable is not as simple as the case above.

Here is a list of the datasets, and associated variables, where these changes 
occur:

Rapid Refresh collections (for old name, replace 1_Hour with Mixed_intervals):

Convective_precipitation_surface_1_Hour_Accumulation
Large-scale_precipitation_non-convective_surface_1_Hour_Accumulation

NAM 12km NOAAPORT (for old name, replace 3_Hour with Mixed_intervals):

Convective_precipitation_surface_3_Hour_Accumulation
Percent_of_Frozen_Precipitation_surface_3_Hour_Accumulation
Probability_of_0p01_inch_of_precipitation_POP_surface_3_Hour_Accumulation
Probability_of_Freezing_Precipitation_surface_3_Hour_Accumulation
Thunderstorm_probability_surface_3_Hour_Accumulation
Total_precipitation_surface_3_Hour_Accumulation

NAM 12km CONDUIT  (for old name, replace 3_Hour with Mixed_intervals):

Baseflow-Groundwater_Runoff_surface_3_Hour_Accumulation
Convective_precipitation_surface_3_Hour_Accumulation
Deep_Convective_Heating_Rate_entire_atmosphere_single_layer_3_Hour_Average
Downward_Short-Wave_Radiation_Flux_surface_3_Hour_Average
Large_Scale_Condensate_Heating_Rate_entire_atmosphere_single_layer_3_Hour_Average
Snow_melt_surface_3_Hour_Accumulation
Storm_Surface_Runoff_surface_3_Hour_Accumulation
Total_precipitation_surface_3_Hour_Accumulation
Water_equivalent_of_accumulated_snow_depth_surface_3_Hour_Accumulation

NAM 20km NOAAPORT (for old name, replace 3_Hour with Mixed_intervals):

Convective_precipitation_surface_3_Hour_Accumulation
Percent_of_frozen_precipitation_surface_3_Hour_Accumulation
Probability_of_freezing_precipitation_surface_3_Hour_Accumulation
Probability_of_precipitation_surface_3_Hour_Accumulation
Snow_depth_surface_3_Hour_Accumulation
Thunderstorm_probability_surface_3_Hour_Accumulation
Total_precipitation_surface_3_Hour_Accumulation

NAM 40km CONDUIT  (for old name, replace 3_Hour with Mixed_intervals):

Baseflow-Groundwater_Runoff_surface_3_Hour_Accumulation
Convective_precipitation_surface_3_Hour_Accumulation
Downward_Long-Wave_Radp_Flux_surface_3_Hour_Average
Downward_Short-Wave_Radiation_Flux_surface_3_Hour_Average
Evaporation_surface_3_Hour_Accumulation
Ground_Heat_Flux_surface_3_Hour_Average
Large-scale_precipitation_non-convective_surface_3_Hour_Accumulation
Latent_heat_net_flux_surface_3_Hour_Average
Potential_Evaporation_surface_3_Hour_Accumulation
Sensible_heat_net_flux_surface_3_Hour_Average
Snow_Phase_Change_Heat_Flux_surface_3_Hour_Average
Snow_melt_surface_3_Hour_Accumulation
Storm_Surface_Runoff_surface_3_Hour_Accumulation
Total_cloud_cover_entire_atmosphere_single_layer_3_Hour_Average
Total_precipitation_surface_3_Hour_Accumulation
Upward_Long-Wave_Radp_Flux_atmosphere_top_3_Hour_Average
Upward_Long-Wave_Radp_Flux_surface_3_Hour_Average
Upward_Short-Wave_Radiation_Flux_atmosphere_top_3_Hour_Average
Upward_Short-Wave_Radiation_Flux_surface_3_Hour_Average
Water_equivalent_of_accumulated_snow_depth_surface_3_Hour_Accumulation

Thank you,

The THREDDS Development Team


Ticket Details
===================
Ticket ID: GCY-293173
Department: Support THREDDS
Priority: Low
Status: Open

_______________________________________________
idvusers mailing list
idvusers@xxxxxxxxxxxxxxxx
For list information, to unsubscribe, visit: 
http://www.unidata.ucar.edu/mailing_lists/


--
Don Murray
NOAA/ESRL/PSD and CU-CIRES
303-497-3596
http://www.esrl.noaa.gov/psd/people/don.murray/



  • 2015 messages navigation, sorted by:
    1. Thread
    2. Subject
    3. Author
    4. Date
    5. ↑ Table Of Contents
  • Search the idvusers archives: