NOTE: The netcdf-hdf mailing list is no longer active. The list archives are made available for historical reasons.
Hi all,
I've spent some time updating my previous draft of changes to the group API
as a result of the changes necessary to provide "creation order" support for
users. I've attached the original version and the updated 2nd draft below.
Please send me feedback, as I'll be starting coding on this soon.
Quincey
NOTE #1: These changes are mainly ramifications from the addition of an
[optional] index that allows applications to access objects in a group
according to the order they were created.
NOTE #2: We still need to decide on scheme for re-naming existing functions
when their parameters change.
Existing API routine changes:
============================
Old Name: Purpose: Change:
========= ======== ======
H5Gcreate Create new group Will need to drop 'size_hint'
parameter and acquire "group
creation" and "group access"
property lists
H5Gopen Open an existing Add "group access" property
group list
H5Gclose Close a group ID <none>
H5Gset_comment Set a "comment" on a <none> (deprecated)
group
H5Gget_comment Get the "comment" on a <none> (deprecated)
group
H5Glink/H5Glink2 Creates a link to an <none>
object in another group
by name in the other
group
H5Gmode/H5Gmove2 Renames a link to an <none>
object in current group
by name
H5Giterate Iterate through objects Change to allow choice of which
in a group, with user index to use (and direction?).
callback routine
H5Gget_objinfo Retrieve information Update H5G_stat_t with any new
about an object in a information. Possibly add
group H5Gget_objinfo_by_idx routine?
Absorb H5Gget_<foo>_by_idx
routines?
H5Gget_num_objs Retrieve number of <none>
objects in a group
H5Gget_objname_by_idx Retrieve name of n'th Needs a "type of index"
object in a group, parameter, in order to choose
according to link name index to use to compute the
order "n'th object" of.
H5Gget_objtype_by_idx Retrieve type of n'th Needs a "type of index"
object in a group, parameter, in order to choose
according to link name index to use to compute the
order "n'th object" of.
Added API routines:
==================
Name: Purpose:
========= =======
H5Glink_order Create a link to an object in another group by creation
order in the other group.
H5Gunlink_order Remove the n'th object (by creation order) in a group.
H5Gchange_time Change the creation time of an object in a group,
potentially changing its position in creation order.
H5Gget_linkval_by_idx Retrieve the link information about a soft link, using
the order with an index.
H5Gopen_by_idx Open the n'th object in a group, accordding to
the order with an index.
New property lists:
==================
Name: Purpose:
===== =======
"Group Creation" Control various parameters of a group, like it's B-tree
properties and whether it has an index allowing it
track the order that objects are created. Possibly
other parameters as well.
"Group Access" Control various parameters of accessing a group, like
possibly the default index to use for various
operations, etc.
NOTE #1: These changes are mainly ramifications from the addition of an
[optional] index that allows applications to access objects in a group
according to the time the object (or link) was created.
NOTE #1a: We will track creation order as the offset into an index on the
creation time of an object/link. The creation time of an object/link will
be encoded as three parts: the number of seconds since the UNIX epoch when
the object was created (as a signed 64-bit number), the number of
nanoseconds since the second (as an unsigned 32-bit number) and a "counter"
field (as an unsigned 32-bit number).
The counter field is needed due to the lack of actual nanosecond
granularity of most system clocks which could allow objects to appear be
created at the same second/nanosecond. If the same second/nanosecond
would be used to create a new link, the counter value will be incremented
until a unique second/nanosecond/counter triplet is arrived at.
NOTE #1b: We could create create a little API for these unique timestamps, that
used the algorithm above. This might be nice for users who want to
timestamp various events that occur. However, we will have to do some
additional work to make this work in parallel. Additionally, this may be a
problem for us, since users will want to store these in a file and would
assume that we should provide a datatype for them, etc.
NOTE #2: If a multiple indices are active in a group when objects are created
in that group (with H5Dcreate(), etc.), they will automatically be updated
to include the newly created object.
NOTE #3: We still need to decide on scheme for re-naming existing functions
when their parameters change.
=====================
Existing API routines:
=====================
--------------------------------------------------------------------------------
Current Name:
H5Gclose
Prototype:
herr_t H5Gclose(hid_t group_id);
Purpose:
Close a group ID from H5Gcreate() or H5Gopen()
API Change:
<none>
Behavior Change:
<none>
--------------------------------------------------------------------------------
Current Name:
H5Gcreate
Prototype:
hid_t H5Gcreate(hid_t loc_id, const char *name, size_t size_hint);
Purpose:
Create new group
API Change:
Will need to drop 'size_hint' parameter and acquire "group creation" and
"group access" property list ID parameters.
Behavior Change:
Property list settings will determine behavior changes.
--------------------------------------------------------------------------------
Current Name:
H5Gget_comment
Prototype:
int H5Gget_comment(hid_t loc_id, const char *name, size_t bufsize,
char *buf);
Purpose:
Get the "comment" on a group.
API Change:
<none>
Behavior Change:
<none> (deprecated)
--------------------------------------------------------------------------------
Current Name:
H5Gget_linkval
Prototype:
herr_t H5Gget_linkval(hid_t loc_id, const char *name, size_t size,
char *buf/*out*/);
Purpose:
Retrieves the "link value" for a soft link in a group
API Change:
<none>
Behavior Change:
Will need to retrieve the "link value" of new "external" link type
Possibly add H5Gget_linkval_by_idx routine?
--------------------------------------------------------------------------------
Current Name:
H5Gget_num_objs
Prototype:
herr_t H5Gget_num_objs(hid_t loc_id, hsize_t *num_objs);
Purpose:
Retrieve number of objects in a group
API Change:
<none>
Behavior Change:
<none>
--------------------------------------------------------------------------------
Current Name:
H5Gget_objinfo
Prototype:
herr_t H5Gget_objinfo(hid_t loc_id, const char *name,
hbool_t follow_link, H5G_stat_t *statbuf/*out*/);
Purpose:
Retrieve information about a link in a group
API Change:
Update H5G_stat_t with any new information. (Including creation time, etc)
Behavior Change:
Possibly add H5Gget_objinfo_by_idx routine? (Absorb H5Gget_<foo>_by_idx
routines, if so?)
--------------------------------------------------------------------------------
Current Name:
H5Gget_objname_by_idx
Prototype:
ssize_t H5Gget_objname_by_idx(hid_t loc_id, hsize_t idx, char* name,
size_t size);
Purpose:
Retrieve name of n'th object in a group.
API Change:
<none>
Behavior Change:
Choice of which index to use (and direction of use) will be determined by
group access property list used in group create or open call.
--------------------------------------------------------------------------------
Current Name:
H5Gget_objtype_by_idx
Prototype:
H5G_obj_t H5Gget_objtype_by_idx(hid_t loc_id, hsize_t idx);
Purpose:
Retrieve type of n'th object in a group.
API Change:
<none>
Behavior Change:
Choice of which index to use (and direction of use) will be determined by
group access property list used in group create or open call.
--------------------------------------------------------------------------------
Current Name:
H5Giterate
Prototype:
herr_t H5Giterate(hid_t loc_id, const char *name, int *idx,
H5G_iterate_t op, void *op_data);
Purpose:
Iterate through objects in a group, with user callback routine.
API Change:
Change "int *idx" to "hsize_t *idx".
Change to iterating over the objects in the "loc_id" instead of the
objects in the object pointed to by loc_id/name (and drop the "name"
parameter).
Behavior Change:
Choice of which index to use (and direction of use) will be determined by
group access property list used in group create or open call.
Possibly allow recursive iteration through groups within current group?
(Which would require an API change)
Possibly allow use of regex for determining which objects to iterate over?
(Maybe with a new API routine?)
--------------------------------------------------------------------------------
Current Name:
H5Glink/H5Glink2
Prototype:
herr_t H5Glink(hid_t src_loc, const char *cur_name, H5G_link_t type,
const char *new_name);
herr_t H5Glink2(hid_t src_loc, const char *src_name, H5G_link_t type,
hid_t dst_loc, const char *dst_name);
Purpose:
Creates a link to an object in another group, using names (as opposed to
another type of index).
API Change:
H5G_link_t will need to be expanded to include new "external" link type.
We could add a property list to this routine to allow future link
information (access control information, etc.) to be set for the new link.
Behavior Change:
This routine will need to create the new "external" link type, which allows
links to point to objects in other HDF5 files.
The new link will automatically be assigned a creation time and inserted
into the creation time index (if a creation time index is active for the
group) when the new link is created.
--------------------------------------------------------------------------------
Current Name:
H5Gmove/H5Gmove2
Prototype:
herr_t H5Gmove(hid_t src_loc, const char *src, const char *dst);
herr_t H5Gmove2(hid_t src_loc, const char *src, hid_t dst_loc,
const char *dst);
Purpose:
Renames a link to an object in current group, using names (as opposed to
another type of index).
API Change:
<none>
We could add a property list to this routine to control how other link
information (creation time and other link information/behavior we haven't
thought of yet) will be set/modified for this link as it is moved.
Behavior Change:
When a link is moved (ie. renamed) within a group, it's creation time is
_not_ changed (and thus it's offset in the creation time index for that
group is the same). If the link is moved to another group, the link's
creation time "counter" field may be modifed if existing object(s) in the
destination group have duplicate creation second/nanosecond values. The
counter field of the new link will be modified in that case to be a value
one greater than the highest counter for any object with that
second/nanosecond. The link will be inserted into the destination group's
creation time index (if a creation time index is active for that group).
Generally speaking, moving a link within a group or to another group should
not modify any other aspect of the link information aside from the name of
the link.
--------------------------------------------------------------------------------
Current Name:
H5Gopen
Prototype:
hid_t H5Gopen(hid_t loc_id, const char *name);
Purpose:
Open an existing group
API Change:
Add "group access" property list parameter
Behavior Change:
Property list settings will determine behavior changes.
--------------------------------------------------------------------------------
Current Name:
H5Gset_comment
Prototype:
herr_t H5Gset_comment(hid_t loc_id, const char *name, const char *comment);
Purpose:
Set a "comment" on a group.
API Change:
<none>
Behavior Change:
<none> (deprecated)
--------------------------------------------------------------------------------
Current Name:
H5Gunlink
Prototype:
herr_t H5Gunlink(hid_t loc_id, const char *name);
Purpose:
Removes a link from a group to an object
API Change:
<none>
Behavior Change:
<none>
--------------------------------------------------------------------------------
==================
Added API routines:
==================
--------------------------------------------------------------------------------
Current Name:
H5Gset_create_time
Prototype:
herr_t H5Gset_create_time(hid_t loc_id, const char *name,
int64_t new_seconds, uint32_t new_nanoseconds);
Purpose:
Change the creation time of a link in a group (which may cause its location
in a creation time index to change).
If a creation time second/nanosecond is chosen which is already used by
another link in the group, the counter field is used to internally
disambiguate the two links.
This routine is analogous to H5Gmove2, only operating on the creation time
index instead of the name index.
--------------------------------------------------------------------------------
Current Name:
H5Glink_by_idx
Prototype:
herr_t H5Glink_by_idx(hid_t src_loc, const char *src_name, H5G_link_t type,
hid_t dst_loc, hsize_t dst_idx);
Purpose:
Create a link to an object in another group by index order.
Is this routine useful?
--------------------------------------------------------------------------------
Current Name:
H5Gunlink_by_idx
Prototype:
herr_t H5Gunlink(hid_t loc_id, hsize_t idx);
Purpose:
Remove the n'th object (by current index order) in a group.
--------------------------------------------------------------------------------
Current Name:
H5Gopen_by_idx
Prototype:
herr_t H5Gopen_by_idx(hid_t loc_id, hsize_t idx);
Purpose:
Open the n'th object in a group, according to the order with an index.
Or, should this be H5Dopen_by_idx, etc? (Which is how our APIs currently
work)
--------------------------------------------------------------------------------
Current Name:
H5Gget_num_indices
Prototype:
herr_t H5Gget_num_indices(hid_t loc_id, unsigned *num_idx);
Purpose:
Determine the number of indices available on links of a group
--------------------------------------------------------------------------------
Current Name:
H5Gget_index_type
Prototype:
herr_t H5Gget_index_type(hid_t loc_id, unsigned idx, H5G_index_t *type);
Purpose:
Determine the type of the n'th index on links of a group
--------------------------------------------------------------------------------
New property lists:
==================
Name: Purpose:
===== =======
"Group Creation" Control various parameters of a group, like it's B-tree
properties and whether it has an index allowing it
track the order that objects are created. Possibly
other parameters as well.
"Group Access" Control various parameters of accessing a group, like
the default index to use for various operations, etc.
Interesting ideas:
=================
- Make routines that operate on key of index more generic?
- Accept name if index is on names, accept creation time if index is on
creation time, etc. We would do this by accepting a "void *" instead
of a "char *" and interpreting the value pointed to as a string or a
time, based on the index chosen for that group ID.
- Affected routines:
- H5Gget_linkval
- H5Gget_objinfo
- H5Giterate (?)
- H5Glink/H5Glink2 (?)
- H5Gmove/H5Gmove2 (?)
- H5Gopen
- H5Gunlink
- Allow which group is the "root" group to be changed
netcdf-hdf archives: