ScappShots

Chapter 4 of The Definitive Guide to Terracotta says:

When you traverse a reference to a clustered object that isn’t currently instantiated on the local physical heap, Terracotta will automatically request that object’s data from the Terracotta server, instantiate it on the local heap, and wire up the reference you are traversing, and your application will be none the wiser.

Likewise, Terracotta can artificially add null references to clustered objects as memory pressure increases, which lets arbitrarily large clustered object graphs fit within a constrained physical heap space.

I ran a couple of experiments to help me understand how these capabilities translate into application behaviour. You can download the source code for the simple Java programs I used in these experiments here.

The first question I wanted to answer was whether or not Terracotta’s ability to page heap objects in and out would allow a Terracotta client to create an object graph that was too big for its physical heap. The book states that a program can traverse an object graph that is too large for its heap, but it does not claim that the virtualization effect applies to object creation. Nevertheless I thought the possibility was worth investigating.

I designed the experiment to test this as follows:

1. Determine how many instances of a class of a fixed size (just over 3 kB) can be created and added to a HashMap by a non-Terracotta Java application running with a heap space of a known size before the application runs out of heap space.

2. Determine how many instances of the same fixed size class can be created by the same Java application running as a Terracotta client application with the HashMap holding the object instances configured as a Terracotta root, and with a heap space of the same known size.

I foresaw two possible outcomes:

1. The numbers of objects created by the two programs would be about the same, indicating that Terracotta’s ability to relieve memory pressure does not apply to pressure caused by the creation of new objects.

2. The Terracotta client would be able to create many more instances of the test object than its plain Java counterpart did, indicating that Terracotta’s object paging feature detects and handles memory pressure caused by the creation of new objects.

The actual results matched the second of the scenarios I had envisioned. When I ran the test with a heap size of 100 megabytes, the plain Java client created 2,988 instances of the test object before running out of memory.

dan@scapps1:~/workspace/bigHeap$ java -Xms100m -Xmx100m -classpath ./bin bigHeap.BigHeapMain 2988

Done so far: 1000 total memory: 100532224 free memory: 68213168

Done so far: 2000 total memory: 100532224 free memory: 36038896

dan@scapps1:~/workspace/bigHeap$

Running as a Terracotta client, the program successfully instantiated more than 11,000 instances running with a server provisioned with 512 megabytes of heap:

dan@scapps1:~/workspace/bigHeap$ dso-java.sh -Xms100m -Xmx100m -classpath ./bin bigHeap.BigHeapMain 11010

Starting BootJarTool…

2009-06-01 00:08:30,423 INFO – Terracotta 3.0.0, as of 20090409-180411 (Revision 12431 by cruise@su10mo5 from 3.0)

2009-06-01 00:08:30,748 INFO – Configuration loaded from the file at ‘/home/dan/workspace/bigHeap/tc-config.xml’.

Starting Terracotta client…

2009-06-01 00:08:32,353 INFO – Terracotta 3.0.0, as of 20090409-180411 (Revision 12431 by cruise@su10mo5 from 3.0)

2009-06-01 00:08:32,673 INFO – Configuration loaded from the file at ‘/home/dan/workspace/bigHeap/tc-config.xml’.

2009-06-01 00:08:32,804 INFO – Log file: ‘/home/dan/workspace/bigHeap/terracotta/client-logs/terracotta-client.log’.

2009-06-01 00:08:34,786 INFO – Connection successfully established to server at 192.168.1.20:9510

Done so far: 1000 total memory: 93257728 free memory: 39313736

Done so far: 2000 total memory: 93257728 free memory: 34530984

Done so far: 3000 total memory: 93257728 free memory: 33617040

Done so far: 4000 total memory: 93323264 free memory: 36051464

Done so far: 5000 total memory: 93257728 free memory: 24922912

Done so far: 6000 total memory: 93061120 free memory: 10904496

Done so far: 7000 total memory: 93257728 free memory: 39410008

Done so far: 8000 total memory: 93257728 free memory: 32656648

Done so far: 9000 total memory: 93585408 free memory: 28937584

Done so far: 10000 total memory: 93257728 free memory: 23353536

Done so far: 11000 total memory: 93257728 free memory: 29487976

dan@scapps1:~/workspace/bigHeap$

My second experiment was designed to confirm the book’s claim that a client program can operate on an object graph that is too big for the client’s heap. To test this I used a program that iterates over all of the objects in the HashMap created by the program I used for the first experiment. With a 19 megabyte heap the program successfully reads all of the objects in the HashMap, the actual size of which is on the order of 33 megabytes:

dan@scapps1:~/workspace/bigHeap$ dso-java.sh -Xms19m -Xmx19m -classpath ./bin bigHeap.CountEm

Starting BootJarTool…

2009-06-01 00:22:37,684 INFO – Terracotta 3.0.0, as of 20090409-180411 (Revision 12431 by cruise@su10mo5 from 3.0)

2009-06-01 00:22:38,010 INFO – Configuration loaded from the file at ‘/home/dan/workspace/bigHeap/tc-config.xml’.

Starting Terracotta client…

2009-06-01 00:22:39,880 INFO – Terracotta 3.0.0, as of 20090409-180411 (Revision 12431 by cruise@su10mo5 from 3.0)

2009-06-01 00:22:40,204 INFO – Configuration loaded from the file at ‘/home/dan/workspace/bigHeap/tc-config.xml’.

2009-06-01 00:22:40,334 INFO – Log file: ‘/home/dan/workspace/bigHeap/terracotta/client-logs/terracotta-client.log’.

2009-06-01 00:22:42,349 INFO – Connection successfully established to server at 192.168.1.20:9510

Found: 11010

dan@scapps1:~/workspace/bigHeap$

These simple experiments demonstrate that Terracotta’s virtual heap can be used to allow client programs to manage datasets that are too large for their physical heaps. The virtual heap is just one of Terracotta’s distinctive and powerful features. I’ll examine others in subsequent posts.