2. Changes in Administration

One-time passwords allow login without further authentication

A new feature has been added, to allow a session to create a one-time, time-limited password for a specific UserProfile, which will allow a login as that user without further authentication. This allows, for example, writing code that uses GsExternalSession to login other sessions to divide tasks over multiple sessions, without needing to hard-code the GemStone password in code or query for it on the command line. The current session’s authentication is "carried over" for a one-time authentication without re-entering a password. One time passwords can be used to login as the same user as the current session or another user configured on the session’s allowlist of UserProfiles.

Topaz and GsTsExternalSession/GsExternalSession include additional methods to support one-time passwords. With GemBuilder for C, you must specify the new flag GCI_ONETIME_PASSWORD with the login parameters using GciLoginEx() or GciLoginEx_(). GemBuilder for Smalltalk products do not yet support one-time password login.

System myUserProfile addOnetimePasswordUserId: 'SubordinateUser'.

tempPW := GsCurrentSession currentSession

createOnetimePasswordForUserId: 'SubordinateUser'

validForSeconds: 30. 

sess := (GsTsExternalSession newDefault)

username: 'SubordinateUser';

onetimePassword: tempPW;

yourself.

sess login.

Tracking one-time passwords

New privilege

The privilege #CreateOnetimePassword has been added. This privilege is granted to DataCurator and SystemUser, but existing and newly created users will not have this privilege.

Added methods

GsCurrentSession >> createOnetimePasswordForUserId: aUserId validForSeconds: seconds

GsCurrentSession >> createOnetimePasswordForUserProfile: aUserProfile validForSeconds: seconds 

GsCurrentSession >> createOnetimePasswordValidForSeconds: seconds

GemStoneParameters >> onetimePassword: pword

GemStoneParameters >> onetimePasswordFlag

GemStoneParameters >> onetimePasswordLoginFlags 

GemStoneParameters >> setLoginWithOnetimePassword  

GsExternalSession >> onetimePassword: aString

GsTsExternalSession >> onetimePassword: aString

UserProfile >> addOnetimePasswordUserId: aUserId 

UserProfile >> addOnetimePasswordUserProfile: aUserProfile

UserProfile >> addOnetimePasswordUserProfiles: anArray

UserProfile >> onetimePasswordUserProfiles

UserProfile >> removeAllOnetimePasswordUserProfiles

UserProfile >> removeOnetimePasswordUserId: aUserId

UserProfile >> removeOnetimePasswordUserProfile: aUserProfile

UserProfile >> removeOnetimePasswordUserProfiles: anArray

CreateOnetimePassword privilege

This new privilege allows creating a one-time password that allow another session to login; see One-time Password for details.

MigrateObjects privilege

This privilege was present in earlier versions, but not implemented. This privilege is required for the new optimized instance migration; see Multithreaded Instance Migration.

Full control over compression levels

GemStone programmatic fullBackup and secureFullBackup allow you to compress backups using gzip (zlib) or lz4. Generally speaking, lz4 backups are faster but do not compress as much.

Both gzip and lz4 support different compression levels, with lower compression level values providing faster performance with less compression. gzip supports compression levels 1 to 9, while lz4 supports 0 to 12. For GemStone backups, the upper ranges provide little addition compression and take much longer.

Repository scan operations now allowed in solo sessions

A number of repository scan operations that were previously disallowed in solo sessions can now be used.

This includes Repository methods for allInstances*, listInstances*, allReferences*, listReferences*, and related methods; and GsObjectInventory scans.

In restore mode, compiler now generates arguments and temporaries as Strings

When a repository is in restore mode, it cannot create new Symbols, since Symbols are canonicalized and require the SymbolGem to commit, which would create a state difference between the restoring repository and the repository it is restoring from. This created problems executing code that includes arguments or temporary variables with names that did not already exist in the repository, since variable names are created as symbols for efficiency.

Now, when the repository is in restore mode, temporary variable names in compiled code are created as Strings, which will not require the SymbolGem commit. (#49984)

Upgrade via hot standby

A process that allows upgrade of a hot standby configuration without having to recreate the slave was introduced in recent 3.6.x releases.

A number of methods have been added to make this process somewhat more streamlined; these added methods are describe in the following sections.

The documentation for this process is now provided as a supplemental document, at this location: https://docs.gemtalksystems.com/current/upgradeViaHotStandby.

Note that v3.7 includes a symbol, #OptimizedSelectors, that impacts upgrade via hot standby. If using the upgrade via hot standby process to upgrade from v3.6.x to 3.7.x, you must first resolve this symbol in the 3.6.6 environment. See the documentation for details.

suspendCommitsForFailover no longer supported

Repository >> suspendCommitsForFailover should no longer be used; instead, use the alternate method Repository >> failOverToSlave.

suspendCommitsForFailover allowed you to keep the master system running as a master, while you created a forked duplicate system on the slave node. After suspendCommitsForFailover to setup the former slave as a master, you have two masters that are not compatible, since they have separate, incompatible tranlog sequences. With suspendCommitsForFailover, to setup the former master as a slave of the new master, you had to restore a new backup of the former slave, new master, into the former master.

If you do want to continue running the master as an independent fork along with the system running on the former-slave, execute commitRestore on the former master after executing failOverToSlave.

Clarification on hot Standby with failover and multiple slaves

As in previous releases, it is possible for multiple slave stones to be restoring from a single master stone. While only one logsender (the primary logsender) can attach to the master stone, the primary logsender may service up to 5 slave logreceivers. Additional logsenders can run on the master node; these read from the tranlogs, but are not attached to the Stone, so the master Stone is not aware of them, and they are not notified if, for example, another tranlog directory is added.

The first Slave system’s logreceiver to connect to the Master’s primary logsender is the one for which failOverStatus is reported. There is no other difference between multiple slaves, other than this reporting information. If the system is restarted with the slaves connected in a different order, the failOverStatus information may not describe the same slave as before the restart. When you have multiple slaves, it is recommended to use the new gslist options (described here) rather than failOverStatus.

A failOverToSlave operation prepares all slaves to become the new master; whichever slave performs the commitRestore, and the application API connects to, will be the new master. The other slaves can seamlessly connect to this new master.

Repository >> failOverStatus improved report

The failOverStatus method results now include more details about the status of the master Stone executing it, and the slave Stone being restored into (the first to attach to the logsender, if there is more than one slave). This includes the most recent tranlog record as well as the most recent checkpoint, and other critical information on the master repository. This allows you to verify the exact restore status of the slave and compare to the status of the current master.

For example:

This repository last tranlog commit file 6 record 206

  last checkpoint file 6 record 15

Other repository restored to commit file 6 record 77

  last restored checkpoint file 6 record 15

This repository commits allowed

Note that this may be misleading if you have multiple slaves. If you have more than one slave, you may check on the intended failover slave itself, or use gslist -j -v logsender, as described here.

continuousRestoreFromArchiveLogs* now has configurable timeout

These methods available in previous releases waited for up to 20 minutes for the logreceiver to start writing log records, before giving up and returning an error. It is possible that very slow hosts under certain circumstances could take this long to start writing logs.

The timeout has been decreased to a more reasonable 60 seconds. Additional methods have been added that include an explicit timeout:

Repository >> continuousRestoreFromArchiveLogs:timeout:

Repository >> continuousRestoreFromArchiveLogs: withDelay:timeout: 

These methods are similar to the existing methods, but will timeout and error after the given number of seconds.

Terminating logsender and logreceiver programmatically

You may now stop the primary logsender and the logreceiver using Smalltalk expressions. The following methods have been added.

Note that these methods are in System class and require SystemControl privilege, as with any method that terminates server processes.

System class >> killLogReceiver
Causes stone to send SIGTERM to the log receiver process. Returns true if the SIGTERM signal was sent, false if no signal was sent because no log receiver process was running or a SIGTERM signal had already been sent.

System class >> killLogSender
Causes stone to send SIGTERM to the log sender process. Returns true if the SIGTERM signal was sent, false if no signal was sent because no log sender process was running or a SIGTERM signal had already been sent.

commitRestoreForFailoverAfterWaitingUpTo: now stops logreceiver

This method now will stop the logreceiver, if it is running, as well as stopping continuous restore. (#50512)

Full details in JSON from gslist

logsenders and logreceivers write keys into the locks directory, which allows them to be queried by gslist. This provides a way to see the status of all logsenders or logreceivers.

gslist has a new features which allows you to query for information on a logsender and all the logreceivers attached to it; using gslist -j -v (both options are required).

For example, the following is the output on the Stone’s node, for a system running on host benton, with a primary slave on host fossa and a secondary slave on host santiam, on the stone’s node. The "statusInMasterCache:true" indicates that this is the primary slave, whose details are reported by failOverStatus.

os$ gslist -v -j logsender_57222

{"GemStoneServers":[

        {

        "Name":"logsender_57222",

        "Host":"benton",

        "HostId":"69621bb0476b1937",

        "Ip":"10.94.441.15",

        "Status":"OK",

        "Type":"Logsender",

        "Version":"3.7.0",

        "Creator":"lalmarod",

        "Started":"2023-04-12T18:13:29.000-08:00",

        "Pid": 15613,

        "Port": 57222,

        "Options":{

                "-d":null,

                "-P":"57222",

                "-A":"benton.gemtalksystems.com",

                "-l":"/benton/admin/logs/logsender57222.log",

                "-s":"bentonstone"

        },

        "LogName":"/benton/admin/logs/logsender57222.log",

        "Sysconf":null,

        "Execonf":null,

        "GEMSTONE":"/benton/admin/GS6437",

        "Exe":"/benton/admin/GS6437/sys/gem",

    "Logreceivers":[   {

        "StatusInMasterCache":"true",

        "PeerIp":"10.94.441.127",

        "PeerHost":"fossa.gemtalksystems.com",

        "ProcessId": 3637081,

        "ReplayedCommit":{"File": 4,"Record": 5235},

        "ReplayedCheckpoint":{"File": 4,"Record": 5235}

  },

        {

        "StatusInMasterCache":"false",

        "PeerIp":"10.94.441.104",

        "PeerHost":"santiam.gemtalksystems.com",

        "ProcessId": 791089,

        "ReplayedCommit":{"File": 4,"Record": 5235},

        "ReplayedCheckpoint":{"File": 4,"Record": 5235}

  },

]

markForCollection performance improvement for repositories with long object chains

When a repository has very long object chains, performance of markForCollection can become slow. The code was improved substantially in v3.6.5 (bug 50025). v3.7 includes additional improvements; thread sleep time has adjustments to moderately improve performance under these circumstances.

ReclaimGem configuration parameters renamed

To accommodate shorter waits on modern hardware, several configuration parameters have been renamed and are specified in smaller units.

While application settings in earlier released are converted to the new units, it is strongly recommended to review these settings and adjust or reset them to default values. Old settings may limit system commit performance.

• sleepTimeBetweenReclaimMs is replaced by sleepTimeBetweenReclaimUs, with units of microseconds. The value for sleepTimeBetweenReclaimMs in the previous release has the units converted and applied to sleepTimeBetweenReclaimUs.
• sleepTimeWithCrBacklogMs is replaced by sleepTimeWithCrBacklogUs, with units of microseconds. The value for sleepTimeWithCrBacklogMs in the previous release has the units converted and applied to sleepTimeWithCrBacklogUs.
• maxTransactionDuration is replaced by maxTransactionDurationUs, with units of microseconds. maxTransactionDuration was previously in units of seconds and could be as high as 300 (5 minutes). The new parameter maxTransactionDurationUs has a maximum of 20000000 (20 seconds) and a default value of 100000 (0.1 second). A value defined in a previous release is converted and applied, but values over 20 will be limited to 20000000.

Number of threads configured for AdminGem

The configuration file value for STN_NUM_GC_RECLAIM_SESSIONS is now used to ensure that the AdminGem uses a number of threads appropriate for the repository configuration to perform write set union sweep and epoch.

In 3.7, when an epoch or write set union sweep is run, the AdminGem uses the Stone’s computed value for STN_NUM_GC_RECLAIM_SESSIONS. The larger of this value or the AdminGem’s setting for #epochGcMaxThreads or #sweepWsUnionMaxThreads, is used to determine the number of AdminGem threads used to perform the operation. The number of threads used is printed in the AdminGem’s log entry for the operation.

Renamed method to configure number of reclaim threads

A new method has been added:

System class >> changeNumberOfReclaimThreads: 

Which has identical behavior as the existing method System class >> changeNumberOfReclaimGemSessions:. The older method continues to be usable and is not deprecated. The new method is preferred; the name reflects the current reclaim implementation.

Reclaim GcGem Logging

The ReclaimGem configuration option #verboseLogging now has multiple levels. Higher levels may produce a very large amount of output for debugging errors and evaluating performance, and are not recommended for general use.

For compatibility with earlier versions of GemStone, in which only booleans were expected, true and false are accepted: false = 0 and true = 1.

These are the definitions for the levels:

0 - only summary information every 5 minutes in slow, 15 minutes in a fast

1 - log sigAbort, gcGemAlert, gcHwPage, etc

2 - numDead from stone, numDead processed per thread

3 - summary: commitCount, stayInTrans, crPage, numLive, numDead, numReclPages

4 - abort, start phases

5 - conflict set

6 - num live and dead found per page

7 - summary of info added to ot slot

8 - deltas in rootPage

9 - push live or dead

The default ReclaimGem logging (with level 0) is now on one line, making the log easier to read. Note at this level, summaries of reclaim operations are printed every 15 minutes.

Determining sessions that have not voted

The following method has been added:

System class >> notVotedSessionsReport
Returns a String describing sessions that have not voted since the last epochGc or markForCollection.

Added -E argument to simplify configuration file specification

startstone, startnetldi, topaz, and gemnetobject now include the -E argument, which specifies a configuration file (not a directory).

When the -E argument is specified, use of the GEMSTONE_EXE_CONF and GEMSTONE_SYS_CONF environment variables is disabled, and the process does not search for default-named files (such as system.conf, hostname.conf, stonename.conf, and gem.conf) in any of the usual default locations.

When using the -E argument, the internal environment variable GEMSTONE_DATA_DIR is set to the directory in which the -E argument file is located. This can be used along with the $GEMSTONE/bin/gemstone_data.conf configuration file to setup a GemStone installation. This location is also used for GemStone system log files.

Configuration file with GEMSTONE_DATA_DIR

An additional template configuration file, gemstone_data.conf, is now provided in the distribution. This specifies a default location for the extents and transaction logs relative to a configuration file path, rather than using $GEMSTONE/data. This is intended to facilitate RPM installations, in which the GemStone distribution is installed in a shared, read-only location and thus using $GEMSTONE/data for application files is not appropriate.

$GEMSTONE/bin/gemstone_data.conf specifies the location for extents and transaction logs using the internal environment variable GEMSTONE_DATA_DIR. This variable is not set directly by the user. Rather, when using the -E argument to startstone or other utilities, GEMSTONE_DATA_DIR is defined as the directory in which this configuration file is located.

Cache Warming

SuperDoit scripts using GemStone base extent

SuperDoit solo scripts by default use a Rowanized extent ($GEMSTONE/bin/extent0.rowan.dbf), although it was possible to configure a script to use the base gemstone extent ($GEMSTONE/bin/extent0.dbf). In most cases, script behavior was identical between the base image and Rowan.

Now, the solo script may invoke a new support script, $GEMSTONE/bin/superdoit_baseimage_solo, that uses the base image, to more easily configuration a script for situations in which a Rowan image is not suitable.

See $GEMSTONE/examples/superDoit for examples:

• versionReport.gs_solo has been added, using the base image.
• template.gs_solo has been added, as a template for scripts that use the base GemStone image.
• template.solo has been removed; replaced by template.rw_solo, which uses the Rowan image.

Changes in abort behavior in solo session

A solo session (a GemStone login that does not connect to a Stone process) has no transactional behavior nor ability to make persistent changes. System class >> commitTransaction has always been disallowed in solo sessions.

When converting existing scripts to solo, calls to System class >> abortTransaction or System class >> beginTransaction that had been present within the scripts previously, could result in losing changes to persistent objects that cannot be committed in a solo session (such as adding classes to UserGlobals).

To make code conversion more reliable, abortTransaction now signals an error if invoked in a solo session where changes to persistent objects would be lost. This allows you to find and remove the abort.

A new method, System class >> soloAbort, can be used within a solo session to perform a logical abort.

System class >> beginTransaction is also disallowed, if in a solo session and changes to persistent objects would be lost.

Methods added to determine recursive size of an object tree

The following methods have been added to support repository analysis.

Object >> recursiveSize
Returns an Array of the form { size . numberOfObjects } where size is approximate physical size on disk of an object and all of the objects referenced by it and numberOfObjects is the count of those objects. Excludes referenced objects in the symbolList, referenced Symbols and referenced special objects.

Object >> recursiveSizeInMemory
Returns an Array of the form { size . numberOfObjects } where size is approximate physical size on disk of an object and all of the objects referenced by it and numberOfObjects is the count of those objects. Excludes committed objects in memory unless those objects have uncommitted changes. Excludes referenced objects in the symbolList, referenced Symbols and referenced special objects.

Object >> recursiveSizeInMemoryReport
Return the result of recursiveSizeInMemory as a readable String.

Object >> recursiveSizeReport
Return the result of recursiveSize as a readable String.

Fast variants of findAllReferences

The following methods have been added:

Object >> fastAllReferences
Same as Object >> findAllReferences, but uses more threads.

Object >> fastPrivateReferences
Same as Object >> findAllReferences, but uses more threads, and returns references from internal nodes of large objects and Nscs.

Debug tracing for SSL, SSO, SSH now allowed in fast builds

Previously, using a number of debugging environment variables to print tracing information to log files was disallowed on fast (normal) builds, for security reasons. Now, sensitive information is redacted, so use of these environment variable in regular fast builds is allowed.

The affected environment variables are:

GS_DEBUG_SSL_LOG_DIR
when set to a directory, enables logging of SSL calls to a file named GsSSLDebug_pid.log

GS_DEBUG_SSO_LOG_DIR
when set to a directory, enables logging of Kerberos/single sign-on calls to a file named GsSSODebug_pid.log

GS_DEBUG_LIBSSH_DIR
when set to a directory, enables logging of SSH calls to a file named GsSSHDebug_pid.log

Tracing LDAP

The previous debugging trace information was supported by GS_DEBUG_LDAP, which wrote information to stdout. This variable remains, but a new environment variable has been added to support writing trace information to a file.

GS_DEBUG_LDAP_DIR
when set to a directory, enables logging of LDAP calls to a file named GsLDAPDebug_pid.log

If you enable GS_DEBUG_LDAP_DIR, GS_DEBUG_LDAP has no effect.

Note that GS_DEBUG_LDAP_DIR redacts sensitive information, but GS_DEBUG_LDAP does not.

Added Errors

The following exceptions have been added:

RT_ERR_STEP_St 6023
Single-step breakpoint signalled to Smalltalk.

RT_ERR_CODE_BREAKPOINT_St 6024
Method breakpoint signalled to Smalltalk.

RT_ERR_STACK_BREAKPOINT_St 6025
Stack breakpoint signalled to Smalltalk.

Added Error and Exception Classes

The SshSocketError class, associated with the new feature Support for ssh and sftp using OpenSSL, was added to support the error ERR_SshSocketError 2759. ERR_SshSocketError was present, but reserved, in previous releases.

The error GcFinalizeNotification, which is private to GemStone internals, has been added; it is used in finalization of Ephemerons.

Removed Errors

The following errors have been removed:

RT_ERR_TERMINATE_PROCESS 6018

RT_ERR_TRACKED_OBJS_NEEDS_INIT 2381

Cannot return now not resumable

The CannotReturn error returned true for isResumable, although this error could not actually be resumed. (#49677)

OS-origin error messages changed

Previously, GemStone provided it’s own strings to describe errors that orginate from the OS, for example, "ENOENT, The file or directory specified cannot be found".

Now, the wording provided by the OS (on Unix) is used, which may be different; for example, "No such file or directory".

2503 changed from FloatingPointError to NumericError

This error number is now associated with the class NumericError.

AbstractException >> signalToGci

The following method has been added:

AbstractException >> signalToGci
Receiver is signaled and it will not be trappable by any exception handler. An Error will be return to the GCI.

Print GCI trace records on protocol error

To aid in analysis of GCI protocol errors, recent GCI call history is now automatically printed to the gem log when a GCI protocol error occurs. GCI call trace records are maintained in a wraparound buffer, and the most recent 100 records will be printed to the log.

MFC Warning now includes possibleDeadSymbols

The warning signaled by markForCollection now includes the possibleDeadSymbols count, as well as live and dead object counts. This change was present in 3.6.4, but not documented.

Many utilities faster on Linux

The startstone, stopstone, stopnetldi, and waitstone utilities have been tuned and optimized for modern hardware, and require much less overhead time than in previous releases. In practice, the amount of startstone performance improvement will depend on the specific startstone tasks for an application.

Utilities startstone, topaz, NetLDI and gemnet* support -E argument

These utilities now support new rules for resolving configuration files, as described in Section 2.6  .

Some corner cases of behavior in the non-E handling have been cleaned up.

The documentation in the System Administration Guide has been extensively revised and expanded.

Custom gemnetobject locations

It is allowed to make a copy of any of the gemnetobject scripts and edit to add your desired behavior.

Previously, it was documented that the custom gemnetobject would be found in the home directory of the current user. However, the code was incorrectly looking in the startup directory of the NetLDI. This behavior has been removed.

To use a custom gemnetobject script, you may either:

• use the full path to the custom script in the NRS login parameters.
• edit $GEMSTONE/sys/services.dat to translate the custom script name to its location path and file.

Note that relative paths are not allowed.

Changes in largememorypages

secure_backup_extract added

The secure_backup_extract utility provides a way to verify the signature of a secure backup, that can be used on secure backup files from any earlier version (the secure backup feature was added in v3.4), without requiring a GemStone environment. This utility is available on little-endian hosts only (Linux and Mac).

Usage:

secure_backup_extract (sig | cert) secureBackupFile outputFile

where:

sig - extract the digital signature and write to outputFile

cert - extract the public signing key and write to

		outputFile

secureBackupFile - a valid GemStone secure backup

outputFile - location to write script results; this file

		must not already exist

secure_backup_extract extracts the public signing cert or the digital signature from a secure backup file, and writes it to a new file. While $GEMSTONE is not needed, you must have openssl and perl available.

This script produces the same result as copydbf, using copydbf -X to extract the public signing cert, or copydbf -Y to extract the digital signature. To verify the digital signature, compare the binary files using the unix utility cmp.

For example, if you have a secure backup file backup.sdbf that was signed using GemStone's example private cert backup_sign_2_clientkey.pem, either of the following will extract the public signing cert:

secure_backup_extract cert backup.sdbf signingCert.secbkext

copydbf -X signingCert.copydbf backup.sdbf 

This can be compared with each other, or against the signing cert used to make the backup; for example:

cmp -l signingCert.secbkext $GEMSTONE/examples/openssl/certs/backup_sign_2_clientcert.pem

To extract the digital signature itself:

secure_backup_extract sig backup.sdbf digitalSig.secbkext

copydbf -Y digitalSig.copydbf backup.sdbf 

which can be compared for validation:

cmp -l digitalSig.secbkext digitalSig.copydbf

Topaz changes

Removed configuration parameters

The following obsolete and nonfunctional parameters have been removed:

DBF_SCRATCH_DIR

GEM_PRIVATE_PAGE_CACHE_KB

STN_PRIVATE_PAGE_CACHE_KB

GEM_RPCGCI_TIMEOUT

The following legacy parameters have also been removed; applications should use the parameter STN_CACHE_WARMER_ARGS for automatic cache warming on startup.

STN_CACHE_WARMER

STN_CACHE_WARMER_SESSIONS

GEM_NATIVE_CODE_ENABLED

Previously, a setting of 0 disabled native code, 1 enabled native code, and 2 enabled native code with inlining some math primitives.

Now, there are only two settings: 0 to disable native code, and 2 to enable native code. A setting of 1 is reserved for future use, but is legal to use and interpreted as a 2 to enable native code.

SHR_PAGE_CACHE_LARGE_MEMORY_PAGE_SIZE_MB

The default setting for this parameter was previously 2 to specify 2MB large pages, although a value of 0 was intended to specify the linux default value. Now, with no setting or a setting of 0, the system determine a default by looking at:

• /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
• /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages
• the value of Hugepagesize in /proc/meminfo

If there are 1GB pages configured, the value defaults to 1GB; otherwise, if there are 2MB pages configured, the value defaults to 2MB; otherwise, it will use the Linux default size in /proc/meminfo.

SHR_PAGE_CACHE_SIZE_KB default increased

The default for SHR_PAGE_CACHE_SIZE_KB, when a specific value is not defined in a configuration file used by the Stone, has increased from 75MB to 200MB.

STN_GEM_TIMEOUT

If the value of this parameter is 0, the login timeout used was previously 5 minutes, and is now 1 minute.

STN_NUM_GC_RECLAIM_SESSIONS now applies to Admin Gem

The number of sessions used by the Admin GcGem now may be determined using the setting for reclaim sessions, which is calculated based on the setting for STN_MAX_GC_RECLAIM_SESSIONS. See Number of threads configured for AdminGem.

Performance improvement for Shared Page Cache Monitor

Statistics collection in the Shared Page Cache Monitor has been adjusted so expensive per-frame statistics are collected less frequently, not more than once per second, to reduce overhead for large systems.

Improved lz4 compression and new VSD required

Compressing statmonitor data with lz4 now has improved compression (between 3% and 25%) by using a C dictionary; this dictionary is in the distribution as $GEMSTONE/bin/lz4.statmon.dict.bin, and is used by VSD to read lz4-compressed statmonitor data files in VSD v5.6 and later.

Statmonitor files generated by 3.7 and compressed using lz4 can only be read using VSD v5.6 or later. VSD 5.6 can read lz4-compressed statmonitor files generated by earlier versions of GemStone, and older versions of VSD can read statmonitor files generated by GemStone v3.7 that are not compressed or that are compressed using gz.

To uncompress an lz4-compressed statmonitor data file on the command line, you must use an additional -D $GEMSTONE/bin/lz4.statmon.dict.bin argument; for example:

os$ lz4 -D $GEMSTONE/bin/lz4.statmon.dict.bin stats.out.lz4

Statmonitor -z -Z arguments now accept compression level

The statmonitor -z (gzip) and -Z (lz4) compression arguments now accept an optional argument to specify the compression level. Higher levels provide greater compression, at the expense of performance.

-z accepts 1...9; the default, if no level is included, is 6.

-Z accepts 1...12; the default, if no level is included, is 3.

Statistics Types reorganized

Several Process Types have been split up. For statistics that applied to the previous general process type, these are now associated only with the relevant process types. For example., the PageMgr* stats that were previously of type Stn, since the PageManager is a thread in the Stone, are now specifically of type PageMgrThr.

Changes in memory statistics on Linux

The existing statistics to capture memory use on Linux did not reliably capture important information, and in many cases were misleading.

Now, on Linux, statmonitor collects memory use related statistics from /proc/<pid>/smaps instead of /proc/<pid>/maps.

There is some performance impact, since it is more expensive to read smaps than maps, particularly on large systems with many processes and short sampling intervals. A new thread in statmonitor is dedicated to reading smaps; this thread is designed to consume a upper maximum of 25% of 1 core, although it will normally take less; much less on smaller systems.

os$ setcap cap_sys_ptrace=pe $GEMSTONE/bin/statmonitor

os$ cd $GEMSTONE/sys

os$ chown root $GEMSTONE/bin/statmonitor 

os$chmod u+s $GEMSTONE/bin/statmonitor

os$ cp $GEMSTONE/sys/gem $GEMSTONE/sys/gemTrace

os$ cp $GEMSTONE/bin/topaz $GEMSTONE/bin/topazTrace

os$ setcap cap_sys_ptrace=pe $GEMSTONE/bin/topazTrace

os$ setcap cap_sys_ptrace=pe $GEMSTONE/sys/gemTrace

os$ cp $GEMSTONE/sys/gemnetobject $GEMSTONE/sys/gemnetobject_trace

gemname="gemTrace" 

topaz> set gemnetid gemnetobject_trace

Other changes in existing statistics

Removed Statistics

The following statistics have been removed (in addition to Linux per-process stats listed above, and SessionWithGcLock):

AbortInProgress

ClientAbortInProgress

CodeGenFullCount

ObjectMemoryGrowCount

PreemptedBitmapPages

PreemptedCommitRecordPages

PreemptedDataPages

PreemptedObjectTablePages

PreemptedOtherPages

SpinLockNewSymSleepCount

TimeSleepingMs

TrackedSetSize

TteCrPageFreeCount

Added Statistics