looks good!
On 12/6/2012 12:38 AM, Nils Hoffmann wrote:
Hi John,
thank you for your answer. To clarify, I was not trying to realize
long-term persistence of Array objects. I just
needed short term persistence in order to use ehcache to cache arrays
and spill over to disk in case of memory exhaustion.
Ehcache does not seem to have a straightforward way of plugging in a
customized serialization mechanism apart from
using the Java default one, which is why I had to stick to that. It
would probably also be possible to use other serialization
libraries like Kryo or, as you mentioned "protocol buffers" within the
writeExternal/readExternal methods.
I ended up implementing a wrapper object for Array that handles
the serialization and deserialization, maybe not in the most efficient
way, but it currently meets my requirements and looks quite
simple:
public class SerializableArray implements Externalizable {
...
@Override
public void writeExternal(ObjectOutput oo) throws IOException {
if(array!=null) {
oo.writeObject(DataType.getType(array.getElementType()));
oo.writeObject(array.getShape());
oo.writeObject(array.getStorage());
}
}
@Override
public void readExternal(ObjectInput oi) throws IOException,
ClassNotFoundException {
array = Array.factory((DataType)oi.readObject(),
(int[])oi.readObject(), oi.readObject());
}
}
And it passes all tests. So I am currently good to go with that solution.
Thanks again for the pointers!
Nils
Am 13.11.2012 22:35, schrieb John Caron:
Hi Nils:
Default Java serialization has serious problems. See chapter 11 of
"Effective Java" for details. Early versions of Netcdf-Java had lots
of serialization, but I have removed it mostly.
If you want to do serialization, I would create specialized objects
that take the original object as an argument, and transfer the contents.
Recently Ive been using Google's "protocol buffer" library to do the
encoding, and I like that approach. Fast, and the serialized objects
are readable from other languages besides java.
May seem like more work, but in the long run, IMO default java
serialization is not a good persistence solution except in certain
limited cases.
John
On 11/13/2012 2:13 AM, Nils Hoffmann wrote:
Hi netcdf-java developers,
I recently had the requirement of serializing/deserializing a
ucar.ma2.Array instance for local client-side caching
and was wondering why the current implementation (4.2.32) does not
implement java.io.Serializable or java.io.Externalizable?
Are there any particular reasons for this, or am I overlooking
something?
In order to check whether any severe problems would wait down the road,
I created a delegate for ucar.ma2.Array and wrote an associated unit
test, showing that it is possible to successfully serialize and
deserialize arbitrary arrays. I would however prefer to be able to
extend ucar.ma2.Array directly, which is currently not possible
unless I create the new class within the ucar.ma2 namespace. This is due
to three methods in
ucar.ma2.Array that are using default visibility (abstract Array
createView(Index index), abstract void copyFrom1DJavaArray(IndexIterator
iter, Object javaArray),
and abstract void copyTo1DJavaArray(IndexIterator iter, Object
javaArray)). Their visibility effectively disallows extension
of ucar.ma2.Array from outside the ucar.ma2 package.
Would it be possible to set the visibility of these methods to public in
the netcdf-java api without other negative side effects or would anyone
propose a different approach?
Maybe someone else has experience with Array serialization and would be
interested in sharing or providing me some further pointers?
I would greatly appreciate any help on this.
Sincerely,
Nils
_______________________________________________
netcdf-java mailing list
netcdf-java@xxxxxxxxxxxxxxxx
For list information or to unsubscribe, visit:
http://www.unidata.ucar.edu/mailing_lists/
