NOTE: The bufrtables mailing list is no longer active. The list archives are made available for historical reasons.
Hi Jeff: Thanks for clarifying this, along with why best practice is to keep previous versions around. Interesting example in para 2; we have seen messages that use elements from subsequent versions. I take this to mean that producers aren't always that careful with what version they say they're using. If the message is otherwise well-formed, I hate to reject it. However, immediate feedback to the producer could fix the problem. In terms of collecting versioned tables: Unidata and the British Met Office (Gil Ross) have independently extracted the info from the current (version 13) WMO master tables B and D. We are comparing and reconciling any differences, so Im feeling pretty good that we will end up with an accurate representation of whats in those documents. We should finish that work when Gil is back from vacation.However, there is the problem of earlier versions. Unidata has extracted and compared the versioned mel-bufr tables. But cross-referencing with the ECMWF tables (from BUFRDC package) fails to confirm the differences. So at the moment I am having doubts about the mel-bufr tables. In short, we are still looking for a reliable way to generate earlier versions of the tables.
But the good news is that, in principle, we should be able to decode all previous versions with the current master tables. I hadn't fully realized that new table C operators are similar to a format change, so the edition number has to increase. Ill have to digest that a bit more. BTW, if anyone knows who at the ECMWF might be interested in joining our discussion, let me know (or just invite them). Regards, John Jeff Ator wrote:
Hello Everyone,I'm now back from a restful vacation and can add my proverbial $0.02 to the discussion.First of all, Stan is correct that scale, reference and bit width values should never change between table versions. Once a descriptor has been included in an operational version it is considered static. This means that you theoretically only ever need the latest version of Table B in order to be able to decode everything, and any discrepancies for a particular descriptor between successive table versions are indeed typos.Having said that, I do believe it is useful to maintain previous table versions, because they do allow you to go back and determine when certain descriptors became available for use. For example, if you received a message that said it was using version 8 of the tables, but it contained a particular descriptor that only became available beginning with version 10, having all of the previous versions available would allow you to easily diagnose such an error.Please note that I'm only sharing with you what is the current WMO practice, and I'm not making any personal statement of opinion one way or the other as to whether this is a good thing. While the current practice, whereby each successive Table B is always a superset of all the previous versions, does simplify the task of decoding, it also has caused some of the descriptor classes to fill up much more quickly than would otherwise be necessary. A good example is Class 12, where we initially had a descriptor for every conceivable type of temperature value, but with scale 1 which only allowed accuracy to one digit beyond the decimal. After these were in use for a while, it was discovered that we had a problem, because most instruments report temperatures in Celsius whereas BUFR uses the SI unit of Kelvin, but the conversion factor between Celsius and Kelvin is 273.15, and different architectures didn't always handle the rounding the same way. This meant you could have one center take an observation in Celsius, then convert it to Kelvin for encoding into BUFR, then send that BUFR message to a different center, and the second center could, when converting back to Celsius, obtain a value that was off by as much as two-tenths of a degree from the original observed value. With hindsight, we realized that the best solution was to just store all temperatures using scale 2, but the practice was (and still is!) to never modify the characteristics of an existing descriptor once it has been in the table. So we ended up creating a second copy of each existing temperature descriptor, each with it's own new reference number and a scale factor of 2, when an alternative solution could have been to simply increase the scale of each existing temperature descriptor effective with the next version of the tables, and then rely on everyone's encoder/decoder software to differentiate properly between the old and new versions of each descriptor based on the table version number encoded within the message.I realize that was a somewhat long-winded historical example, but an important point is that WMO could possibly change their practice at some point in the future (especially as some of the Table B classes are now close to completely full!), and this is another reason why it would be a good idea for us to hold on to all previous versions of these tables, even though right now it's not technically necessary. The easiest way to do this would be to include the version number somewhere in the filename within our collective archive.As for BUFR edition numbers (about which I've also sensed some confusion in the ongoing email thread), these are a different animal entirely. Whereas table version number changes do not require any corresponding change to BUFR encoder/decoder software, the edition number is incremented only when changes are introduced which require corresponding changes to actual encoding and decoding software (for example, if new Table C operators are introduced, or the format of BUFR Section 1 is modified). In this case, the software needs to be able to simultaneously handle multiple editions (i.e. formats) of BUFR messages, which of course adds to programming complexity, and so such changes are made much less frequently and with much more advance notice.I hope this helps clarify any confusion! Best regards, -Jeff
bufrtables archives: