More Pickling

Following on from my experiments with Scala Pickling, I thought I would try unpickle() on the JSON file. This is where I discovered that things hadn’t gone well at all. First of all, I had to increase the maximum heap – 8GB was not enough. I used 14GB, but perhaps 9 would have been enough. It’s still a lot. At this point I started to get exceptions from the unpickle() call. It turned out that a lot of my data structure had turned into references like this:

{
 "$type": "java.lang.Object"
 }

Unsurprisingly Scala can’t unpickle this into anything useful ^[1]I suppose it could create the Java object instance, but it wouldn’t be usable anywhere with a type declaration.

Now I am going through my code converting the Java collections to Scala collections – mostly mutable.Map and mutable.ArrayBuffer to start with. I’ve spent quite a few hours on this now. The code is no doubt better for it, but pickling is still losing most of my object graph. One class retains one String field but loses another. This is puzzling to say the least. If this was some random library from the internet I would give up and move on, but this is the right way to do Scala serialization from EPFL. Although it’s fairly new, I’m pretty sure it should work a lot better than this.