Hi John,
thank you for your answer. To clarify, I was not trying to realize
long-term persistence of Array objects. I just
needed short term persistence in order to use ehcache to cache arrays
and spill over to disk in case of memory exhaustion.
Ehcache does not seem to have a straightforward way of plugging in a
customized serialization mechanism apart from
using the Java default one, which is why I had to stick to that. It
would probably also be possible to use other serialization
libraries like Kryo or, as you mentioned "protocol buffers" within the
writeExternal/readExternal methods.
I ended up implementing a wrapper object for Array that handles
the serialization and deserialization, maybe not in the most efficient
way, but it currently meets my requirements and looks quite
simple:
public class SerializableArray implements Externalizable {
...
@Override
public void writeExternal(ObjectOutput oo) throws IOException {
if(array!=null) {
oo.writeObject(DataType.getType(array.getElementType()));
oo.writeObject(array.getShape());
oo.writeObject(array.getStorage());
}
}
@Override
public void readExternal(ObjectInput oi) throws IOException,
ClassNotFoundException {
array = Array.factory((DataType)oi.readObject(),
(int[])oi.readObject(), oi.readObject());
}
}
And it passes all tests. So I am currently good to go with that solution.
Thanks again for the pointers!
Nils
Am 13.11.2012 22:35, schrieb John Caron:
> Hi Nils:
>
> Default Java serialization has serious problems. See chapter 11 of
> "Effective Java" for details. Early versions of Netcdf-Java had lots
> of serialization, but I have removed it mostly.
>
> If you want to do serialization, I would create specialized objects
> that take the original object as an argument, and transfer the contents.
>
> Recently Ive been using Google's "protocol buffer" library to do the
> encoding, and I like that approach. Fast, and the serialized objects
> are readable from other languages besides java.
>
> May seem like more work, but in the long run, IMO default java
> serialization is not a good persistence solution except in certain
> limited cases.
>
> John
>
> On 11/13/2012 2:13 AM, Nils Hoffmann wrote:
>> Hi netcdf-java developers,
>> I recently had the requirement of serializing/deserializing a
>> ucar.ma2.Array instance for local client-side caching
>> and was wondering why the current implementation (4.2.32) does not
>> implement java.io.Serializable or java.io.Externalizable?
>> Are there any particular reasons for this, or am I overlooking
>> something?
>>
>> In order to check whether any severe problems would wait down the road,
>> I created a delegate for ucar.ma2.Array and wrote an associated unit
>> test, showing that it is possible to successfully serialize and
>> deserialize arbitrary arrays. I would however prefer to be able to
>> extend ucar.ma2.Array directly, which is currently not possible
>> unless I create the new class within the ucar.ma2 namespace. This is due
>> to three methods in
>> ucar.ma2.Array that are using default visibility (abstract Array
>> createView(Index index), abstract void copyFrom1DJavaArray(IndexIterator
>> iter, Object javaArray),
>> and abstract void copyTo1DJavaArray(IndexIterator iter, Object
>> javaArray)). Their visibility effectively disallows extension
>> of ucar.ma2.Array from outside the ucar.ma2 package.
>>
>> Would it be possible to set the visibility of these methods to public in
>> the netcdf-java api without other negative side effects or would anyone
>> propose a different approach?
>>
>> Maybe someone else has experience with Array serialization and would be
>> interested in sharing or providing me some further pointers?
>>
>> I would greatly appreciate any help on this.
>>
>> Sincerely,
>>
>> Nils
>>
>
> _______________________________________________
> netcdf-java mailing list
> netcdf-java@xxxxxxxxxxxxxxxx
> For list information or to unsubscribe, visit:
> http://www.unidata.ucar.edu/mailing_lists/
--
Nils Hoffmann phone: +49-521-106-4342
Bielefeld University room: U10-144
Faculty of Technology, Genome Informatics
P.O. Box 10 01 31
33501 Bielefeld, Germany
http://www.cebitec.uni-bielefeld.de/~hoffmann
