13. Managing Memory
Executing your application code will naturally require access to previously committed objects in the repository. These objects are faulted into the Gem session’s memory to be examined or updated. When new objects are created, they must reside in memory while they are being modified.
GemStone automatically garbage-collects temporary objects that are no longer referenced, and clears out the space used by persistent, committed objects, when memory is needed. However, the memory available for any session is finite. If you need to create large temporaries or modify many objects within a transaction, you may need to tune your application to increase the available memory, or to use memory more efficiently.
This chapter discusses the following topics:
13.1 Memory Organization
Each Gem session has a temporary object memory that is private to the Gem process and its corresponding session. This local object memory is divided into the following regions:
- new — Young temporary objects; includes two subspaces named eden and survivor
- old — Older temporary objects
- pom — Unmodified faulted-in objects, divided into ten subspaces
- perm — Faulted-in or created Classes and Metaclasses
- code — Instances of GsNMethod being executed, or recently executed
- mE — oopMap entries that map objId to in-memory objects for committed objects
Temporary objects are created in the new area of local object memory. When the new area fills up, a scavenge occurs which throws away unreferenced objects in new. After an object has survived a number of scavenges, it is copied to the old area. After the old area has grown by some amount or is almost full, a mark/sweep takes place, finding all live objects and then compacting the new, old, perm, and code areas as needed to remove dead objects.
Committed objects referenced by the session are copied from the shared page cache into the pom, perm, or code areas at the point they are first referenced by interpreter execution or a GCI call. (This is called a "copy-on-read" design.)
If a committed object in the pom area has been modified, it is copied to the old area if a scavenge occurs before the change is committed. Objects that are sent to the GCI client, such as GBS, are also moved to the old area, whether or not they are modified.
When the pom area becomes full, the contents of its oldest subspace (that is, the oldest 10%) are discarded, and that subspace is reused to continue faulting-in committed objects. Before the oldest subspace is recycled, any objects in the subspace that have been modified, or that are currently referenced from the interpreter stack, are copied to the old area.
At transaction commit, any committed objects that have been modified, and any new objects transitively reachable from those modified objects, are copied to new data pages in the shared cache. A transaction conflict check is then performed. If the commit succeeds, the in-memory state of all new objects copied to the shared cache is changed to "committed". The newly committed objects are now eligible to be removed from temporary memory by a mark/sweep or scavenge if they are no longer directly referenced from temporary objects.
13.2 Configuring Temporary Memory Usage
You may encounter an OutOfMemory error if you create too large a graph of live temporary objects at any time, or if you try to modify too many committed objects in a single transaction. OutOfMemory is a fatal error that terminates the session.
Very large numbers of Classes can also fill up temporary object memory. Any class that is referenced by message send or iteration is loaded into the perm area, and its method dictionaries, classHistory, and so on are loaded into the old area.
Persistent objects that are in the export set for a GCI client, such as GemBuilder for Smalltalk, are also moved to the old area. This includes objects that are replicated but not modified.
If you find that your application is running out of temporary memory, you can use several GemStone environment variables to help you identify which parts of your application are triggering garbage collection. Once you’ve done that, you can set GemStone configuration options to provide the needed memory.
Configuration Options
The values for these options are set when the gem or topaz -l process is initialized. You cannot change these values without restarting the VM. For more about these options, see the descriptions that begin here.
GEM_TEMPOBJ_CACHE_SIZE
The maximum size (in KB) of temporary object memory. (This limit also applies to linked Topaz sessions and linked GemBuilder applications.) When you only change this setting, and the other GEM_TEMPOBJ... configuration options use default values, then all of the various spaces remain in proportion to each other.
GEM_TEMPOBJ_MESPACE_SIZE
The maximum size (in KB) of the mE (Map Entries) space within temporary object memory. Unless you are trying to minimize the memory footprint on HP-UX or AIX, you should always leave GEM_TEMPOBJ_MESPACE_SIZE at its default value so that the system can calculate the appropriate value. Otherwise, you are at risk of premature OutOfMemory errors.
GEM_TEMPOBJ_OOPMAP_SIZE
The size of the hash table (that is, the number of 8-byte entries) in the objId-to-object map within temporary object memory. This option should normally be left at its default setting.
GEM_TEMPOBJ_POMGEN_SIZE
The maximum size (in KB) of the POM generation area in temporary object memory. The POM generation area holds unmodified copies of committed objects that have been faulted into a Gem, and is divided into ten subspaces. This option should normally be left at its default setting so that the system can calculate the value.
Methods for Computing Temporary Object Space
To find out how much space is left in the old area of temporary memory, the following methods in class System (category Performance Monitoring) are provided:
System _tempObjSpaceUsed
Returns the approximate number of bytes of temporary object memory being used to store objects.
System _tempObjSpaceMax
Returns the approximate maximum number of bytes of temporary object memory that are usable for storing objects.
System _tempObjSpacePercentUsed
Returns the approximate percentage of temporary object memory that is in use to store temporary objects. This is equivalent to the expression:
(System _tempObjSpaceUsed * 100) // System _tempObjSpaceMax.
Note that it is possible for the result to be slightly greater than 100%. Such a result indicates that temporary memory is almost completely full.
Sample Configurations
This section presents several sample configurations:
These examples assume that you have already set the GS_DEBUG_VMGC... environment variables (Configuration Options) to produce the resulting printouts. The examples shown are for Solaris and may vary on other platforms.
Smaller Configuration
A value of 10000 for GEM_TEMPOBJ_CACHE_SIZE (that is, 10 MB) produces a limit of about 7 MB of temporary plus modified-committed objects (old), space for a working set of about 8 MB of unmodified committed objects (pom), and a maximum memory footprint on the order of 25 MB.
The following example shows the printout for this configuration:
vmGc spaceSizes: eden init 1864K max 1864K, survivor init 320K max 320K,
vmGc old max 7496K, code max 2000K, perm max 1000K, pom 10 * 840K=8400K,
vmGc remSet 216K, meSpace max 9592K oopMapSize 65536
(The internal structures remSet, meSpace, and oopMapSize are not of interest here.)
Larger old, Smaller pom
The following settings configure the application for a 5 MB working set of unmodified committed objects (smaller than the default), and a maximum of 25 MB of temporary plus modified objects (larger than the default).
GEM_TEMPOBJ_CACHE_SIZE = 25000;
GEM_TEMPOBJ_POMGEN_SIZE = 5000;
The following example shows the printout for this configuration:
vmGc spaceSizes: eden init 2000K max 4688K, survivor init 400K max 784K
vmGc old max 18744K, code max 5000K, perm max 2504K, pom 10 * 504K=5040K
vmGc remSet 360K, meSpace max 16824K oopMapSize 65536
Smaller old Area, Larger pom
The following settings configure an application with a large working set of committed objects and small temporary object space.
GEM_TEMPOBJ_CACHE_SIZE = 7000;
GEM_TEMPOBJ_POMGEN_SIZE = 100000;
The following example shows the printout for this configuration:
vmGc spaceSizes: eden init 1304K max 1304K , survivor init 224K max 224K
vmGc old max 5248K, code max 1400K, perm max 704K, pom 10 * 10000K=100000K
vmGc remSet 1008K, meSpace max 48880K oopMapSize 524288
Debugging out-of-memory errors
When any of the following environment variables are set to a positive non-zero value, they have the effect described here for each Gem or linkable Topaz (topaz -l) process that you subsequently start. For all of these environment variables, the printout goes to the "output push" file of a linkable Topaz (topaz -l) session, for use in testing your application. If that file is not defined, the printouts go to standard output of the session’s Gem or topaz -l process.
To set any of these environment variables, you must first uncomment them in the $GEMSTONE/sys/gemnetdebug file. Be aware that the contents of gemnetdebug are subject to change at any time. For the most current information about these and other variables, examine the gemnetdebug file.
GS_DEBUG_COMPILE_TRACE
Trace method compiles. The following are valid values:
0 - no tracing
1 - one line (class,selector) of each method compiled
2 - in addition to above, bytecode disassembly
3 - in addition to above, native code assembly listing
GS_DEBUG_VMGC_MKSW_MEMORY_USED_SOFT_BREAK
At the end of each mark/sweep, if the percent of memory used is greater than the threshold specified by this variable, a SoftBreak (error 6003) is generated, and the threshold is raised by 5 percent. We suggest a setting of 75%.
GS_DEBUG_VMGC_MKSW_PRINT_STACK
The mark/sweep count at which to begin printing the Smalltalk stack at each mark/sweep.
GS_DEBUG_VMGC_MKSW_PRINT_C_STACK
The mark/sweep count at which to begin printing the C stack at each mark/sweep. This variable is very expensive, consuming two seconds plus the cost of fork() for each printout.
GS_DEBUG_VMGC_PRINT_MKSW
The mark/sweep count at which to begin printing mark/sweeps.
GS_DEBUG_VMGC_PRINT_MKSW_MEMORY
The mark/sweep count at which to begin printing detailed memory usage (20 lines) for each mark/sweep.
GS_DEBUG_VMGC_PRINT_MKSW_MEMORY_USED
Specifies when Smalltalk stack printing starts as the application approaches OutOfMemory conditions. At the end of each mark/sweep, if the percent of memory used is greater than the threshold specified by this variable, the mark/sweep is printed, the Smalltalk stack is printed, and the threshold is raised by 5 percent. In a situation producing an OutOfMemory error, you should get several Smalltalk stacks printed in the Gem log file before the session dies.
GS_DEBUG_VMGC_PRINT_SCAV
The scavenge count at which to begin printing scavenges. Once this takes effect, all mark/sweeps will also be printed. Be aware that printing scavenges can produce large quantities of output.
GS_DEBUG_VM_PRINT_TRANS
Print transaction boundaries (begin/commit/abort) in the log file.
GS_DEBUG_VMGC_SCAV_PRINT_STACK
The scavenge count at which to begin printing the Smalltalk stack at each scavenge. Be aware that this print activity can produce large quantities of output.
GS_DEBUG_VMGC_SCAV_PRINT_C_STACK
The scavenge count at which to begin printing the C stack at each scavenge. This variable is very expensive, consuming 2 seconds plus the cost of fork() for each printout.
GS_DEBUG_VMGC_VERBOSE_OUTOFMEM
Automatically call the primitive for System class>>_vmPrintInstanceCounts:0 when an OutOfMemory error occurs, and also print the Smalltalk stack. (For details about this method, see the comments in the image.) This applies to each Gem or linkable Topaz (topaz -l) process that you subsequently start.
GS_DEBUG_VMGC_VERIFY_MKSW
The mark/sweep count at which to begin verifying object memory before and after each mark/sweep.
GS_DEBUG_VMGC_VERIFY_SCAV
The scavenge count at which to begin verifying object memory before and after each scavenge. Once this takes effect, GS_DEBUG_VMGC_VERIFY_MKSW will also be in effect. Be aware that this activity uses significant amounts of CPU time.
Recording Out of Memory Information to CSV file
The Out of memory and almost out of memory information that is written to a Gem log can also be directed to write to a disk log file in (CSV) comma separated format, which can be loaded into spreadsheet applications for analysis.
To configure your system to write such a CSV file, set the configuration option GEM_TEMPOBJ_OOMSTATS_CSV to TRUE.
If a Gem or linked topaz process encounters an almost out of memory or out of memory condition, a file will be created or appended to with the name gemnetobjectpid.csv or topazpid.csv.
The config parameter GEM_TEMPOBJ_OOMSTATS_CSV and the environment variable GS_DEBUG_VMGC_VERBOSE_OUTOFMEM operate independently.
Signal on low memory condition
When a session runs low on temporary object memory, there are actions it can take to avoid running out of memory altogether. By enabling handling for the signal AlmostOutOfMemory, an application can take appropriate action before memory is entirely full. This signal is asynchronous, so may be received at any time memory use is greater than the threshold the end of an in-memory markSweep. However, if the session is executing a user action, or is in index maintenance, the error is deferred and generated when execution returns.
When performing index operations, such as creating indexes for large collections, the IndexManager can be configured to use this facility to automatically commit when memory is low. Committing objects allows them to be removed from memory, since they can be re-loaded as needed from the persistent object. See the Programming Guide for details on IndexManager autoCommit, and for more information on handling AlmostOutOfMemory.