1. Release Notes for 3.7.4

Next chapter

Overview

GemStone/S 64 Bitâ„¢ 3.7.4 is a new version of the GemStone/S 64 Bit object server. Version 3.7.4 includes feature enhancements and bug fixes, including significant additional features for login.

These Release Notes include changes between the previous version of GemStone/S 64 Bit, v3.7.2, and v3.7.4. v3.7.3 was an alpha-only release; all changes in this version are included in these Release Notes.

If you are upgrading from a version prior to 3.7.2, review the release notes for each intermediate release to see the full set of changes.

The Installation Guide has not been updated for this release. For installation, upgrade and conversion instructions, use the Installation Guide for version 3.7.2.

Supported Platforms

Platforms for Version 3.7.4

GemStone/S 64 Bit version 3.7.4 is supported on the following platforms:

  • Red Hat-compatible Linux 7.9, 8.10, and 9.5, and Ubuntu 22.04 and 24.04, on x86_64.
    GemStone is tested on a mixture of Red Hat, CentOS, and Rocky; these are all considered fully certified platforms. Any reference to Red Hat applies to any Red Hat-compatible distribution.
  • Ubuntu 22.04 on ARM. Linux on ARM is for development only, not for production.
  • macOS 15.3.2 (Sequoia) and 11.7.10 (Big Sur), on x86 and Apple silicon (ARM). macOS distributions are for development only, not for production.

Note that GemStone/S 64 Bit v3.7 and later on Linux are built using machine instructions that are not available on older x86_64 CPUs (more than about 12 years old); v3.7.4 will not run on these CPUs, regardless of the Linux OS version.

For more information and detailed requirements for each supported platforms, please refer to the GemStone/S 64 Bit v3.7.2 Installation Guide for that platform.

GemBuilder for Smalltalk (GBS) Versions

The following versions of GBS are supported with GemStone/S 64 Bit version 3.7.4:

GBS/VW version 8.8.1

VisualWorks
9.4.1

64-bit

VisualWorks
9.3.1

64-bit

VisualWorks
8.3.2

64-bit

VisualWorks
8.3.2

32-bit

  • Windows 10, Windows 11
  • RedHat ES 7.9, 8.10, and 9.5; Ubuntu 22.04 and 24.04
  • Windows 10, Windows 11
  • RedHat ES 7.9, 8.10, and 9.5; Ubuntu 22.04 and 24.04
  • Windows 10, Windows 11
  • RedHat ES 7.9, 8.10, and 9.5; Ubuntu 22.04 and 24.04
  • Windows 11
GBS/VA version 5.4.7

VAST Platform
11.0.1

VAST Platform
10.0.2

VA Smalltalk
8.6.3

  • Windows Server 2016, Windows 10, and
    Windows 11
  • Windows Server 2016, Windows 10, and
    Windows 11
  • Windows Server 2016, Windows 10, and
    Windows 11

For more details on GBS and client Smalltalk platforms and requirements, see the GemBuilder for Smalltalk Installation Guide for that version of GBS.

VSD Version

The GemStone/S 64 Bit v3.7.4 distribution includes VSD version 5.6.1. This is the same version of VSD that was included with the previous release, v3.7.2.

VSD 5.6.1 is included with the GemStone distribution, and can also be downloaded as a separate product from https://gemtalksystems.com/vsd/

Updated library versions

The version of openSSL has been updated to 3.0.16

Added Shared Libraries

The distribution now includes additional shared libraries,

$GEMSTONE/lib/libcurl-3.7.4-64.so
$GEMSTONE/lib/libcurlutl-3.7.4-64.so

1. Additional Login Parameter Support

GemStone supports a number of types of authentication, in addition to the standard GemStone-native password: UNIX, LDAP, Single-sign on via Kerberos. This release adds support for login via JSON Web Tokens (JWT). These all utilize the same login parameters (userId and password).

In addition, you may also now provide passwords other than as strings. Passwords can be the value of an environment variable or in a text file on disk. The environment variable or filename is resolved using environment and filesystem access of the GCI client process, or the gem process that is creating the instances of GsTsExternalSession.

To use these alternate password sources, set the appropriate flag in the login parameters, and put the environment variable name or a filename in the password field. During login, the environment variable is accessed or the file with the given name is read and the result is used to login.

Ordinary logins using password strings do not require any changes from previous releases.

There are additional topaz commands, GsTsExternalSession parameters messages, or GCI login flags that allow you to specify the new password source options.

The environment variable $GS_DEBUG_LOGIN can be set in the client environment to print debugging information on the login to stdout of the Gem process. This is primarily intended to print the multiple validations that can be done by a JWT login; it also prints the authentication scheme that is used by the UserProfile.

Support for JWT Authentication

GemStone now supports authentication using JSON Web Tokens (JWTs), as another alternative authentication scheme for GemStone login. See Chapter 2 for details.

Note that this feature should be considered preview in this version of GemStone.While JWT logins have been carefully tested, the API may have unknown gaps and is subject to change.

The JsonWebToken class has been added to support testing for JWT login; this class also supports general uses of JWTs.

GsTsExternalSession added methods

The following methods have been added; these are convenience methods that invoke the method on the GsTsExternalSession’s instance of GemStoneParameters.

GsTsExternalSession >> jwtPassword: aString
Consider aString to be a JSON Web Token (JWT) and set it as the password.

GsTsExternalSession >> passwordEnvVar: aString
Consider aString to be the name of an environment variable which contains the actual password. aString is evaluated in this current session's gem or topaz -l process.

GsTsExternalSession >> passwordFileName: aString
Consider aString to be the name of a file on the gem host which contains the actual password, which may be a JWT, GemStone, or other authentication password. The filename will use the current directory of this session's gem or topaz -l process to evaluate aString.

GsTsExternalSession >> setLoginDebug
Enables writing debug info to the gem's log file.

GemStoneParameters added and changed methods

the GemStoneParameters have been extensively updated to support alternate password source locations as well as JWT authentication.

New methods accessing the new GCI login flags have been added, and some existing methods (onetimePasswordFlag, onetimePasswordLoginFlags, and passwordlessLoginFlags), have been renamed for consistency, and new *flag methods have been added. The *flag methods are not expected to be accessed directly.

The following instance methods have been added.

GemStoneParameters >> passwordEnvVar: envVarName
Configure the parameters to interpret the password instance variable as an environment variable, and use the value of envVarName in the password field. Applies to JWT and GemStone authentication schemes.

GemStoneParameters >> passwordFileName: fileName
Configure the parameters to interpret the password instance variable as a filename, and use the contents of fileName in the password field. Applies to JWT and GemStone authentication schemes.

GemStoneParameters >> setPasswordIsEnvVar
Configure the parameters to interpret the password instance variable as an environment variable and apply the value as the password field. Applies to JWT and GemStone authentication schemes.

GemStoneParameters >> setPasswordIsFileName
Configure the parameters to interpret the password instance variable as a filename, and apply the contents of that file as the password field. Applies to JWT and GemStone authentication schemes.

GemStoneParameters >> setLoginDebug
Enable writing login debug details to the gem's log file

GemStoneParameters >> clearLoginDebug
Disables writing debug info to the gem's log file.

The following methods support login via JWTs:

GemStoneParameters >> setLoginWithJwt
Configure the parameters to login with a JSON Web Token.

GemStoneParameters >> jwtPassword: aString
Configure the parameters to authenticate using JWT, and set the password instance variable to aString, which must be an encoded JWT.

Topaz added set subcommands

Topaz includes additional set subcommands to configure login.

SET ENVPASSWORD onOrOff

When ON, the password field is expected to contain the name of an environment variable that references the password. This password may be a native GemStone, UNIX, or LDAP password or encoded JWT. May be abbreviated as ENVP or ENVPASS.

SET FILEPASSWORD onOrOff

When ON, the password field is expected to contain the name of disk file that contains the password. This password may be a native GemStone, UNIX, or LDAP password or encoded JWT. May be abbreviated as FILEP or FILEPASS.

SET JWTJSONFILENAME filename key

Enables login using a JWT that is in a specifically structured JSON file. For more information on JWTs, see Chapter 2.

SET JWTPASSWORD onOrOff

When on, enables login using a JWT as the password. For more information on JWTs, see Chapter 2.

2. Other significant new features

Configurable remote commit

A configuration parameter has been added to control the location of the commit for Gems that are running on a different node than the Stone. The default for non-X509 Gems is false, so the critical section of a commit runs in the remote Gem, as in past releases. When GEM_REMOTE_COMMIT is set to true, the commit will run in Gem’s page server (or HostAgent) on the Stone’s node.

For Gems in X509-secured GemStone environments, the default is true, which is the behavior in previous releases. This can be configured so the commit runs in the remote X509 Gem by setting GEM_REMOTE_COMMIT to false.

GEM_REMOTE_COMMIT
If TRUE, a Gem on a remote cache will execute the critical region of commit in the session's thread in the gempgsvr or hostagent process on Stone host. Can only be enabled if gem and stone host have same byte order, and has no effect if the Gem is not remote from Stone

  • For normal (non-X509) sessions, default is false.
  • For X509 sessions, default is true if gem is remote.

Runtime equivalent: #GemRemoteCommit

Remote cache page server changes for keeping cache warm

Remote cache can be warmed on startup, but there was previously no mechanism to keep these warm as processes on the Stone’s cache modify objects. This meant that a remote Gem server with cache warming was not effective over time in avoiding performance impacts of slow networks configurations.

Now, a page pushing mechanism has been added. When a remote Gem logs in and triggers the startup of a remote cache on that node, its configuration file may specify for the remote cache’s SPC monitor to start page pusher and receiver threads, to keep the remote cache warm.

The default is 0, which provides the same behavior as in previous releases.

SHR_PAGE_RECEIVER_THREADS
In a gem config file for a gem login that triggers startup of a remote cache, the number of page pusher and receiver thread pairs that will be started to keep the remote cache warm.

Default: 0 max: 20, recommended: 2 to 4.

KeepAlive now configured per socket

Most firewalls terminate idle sockets after a period of time, which terminates a logged in Gem session. The system wide TCP_KEEPALIVE defaults to 2 hours, while most firewalls have a shorter idle socket timeout.

To allow this to be controlled within a GemStone environment rather than system-wide at the OS level, on Linux GemStone now sets SO_KEEPALIVE on individual interprocess sockets. The timeout default to 30 minutes, but can be configured by a GemStone configuration parameter, depending on the firewall timeout.

The following configuration parameter applies to all GemStone processes that open sockets.

SOCKET_KEEPALIVE

Timeout value in seconds to be used for setsockopt(fd, SOCKOPT_KEEPALIVE, timeout), for gem's socket to the GCI client, socket between gem and stone, socket between gem and a pgsvr or hostagent, socket between stone and a remote cache pgsvr. SOCKET_KEEPALIVE may be used in stone, gem and x509 netldi config files.

Default: 1800 min: 15 max 2147483647

SIGHUP handling for log rotation

To support log rotation, on Linux the following GemStone processes now handle SIGHUP:

stoned, netldid, pgsvrmain, shrpcmonitor, gem, reclaimgem, admingem, symbolgem, hostagent, pgsvrmain

If the log file/s written to by this process no longer exist, a new file will be opened with the same name, using freopen().

The stone process's login log (enabled by STN_LOGIN_LOG_ENABLED) is reopened with a new timestamp in the file name.

3. Collection Changes

Collection printing changes

Collection printing changes

In previous releases, printing a collection using printString printed a limited string of around 700 characters, stopping after printing the collection element that brought the size over the limit. This produced highly variable result string sizes, depending on the size of the elements in the collection.

Now, printing stops in the middle of a collection element’s printString, not at an element boundary, and the resulting string is reliably limited to the given size (700, or as specified by the maxSize: argument to printingOn:maxSize:, see bug below) plus 4 to allow for elipses and collection close character. The result string size will usually be smaller than in previous releases.

For example, given an array of integers,

previously: <first part of printString>, 1348, 1349, 1350, ...)

3.7.4: <first part of printString>, 1348, 1349, 13...)

Collection printing did not conform to PrintStream maxSize limit

Printing a dictionary or other collection using printString uses an instance of PrintStream and returns a string limited to about 700 characters, terminating in ...) to indicate truncation. This is designed to be controlled by creating an instance of PrintStream using printingOn:maxSize: and using that for printing; however, printing did not respect the maxSize: argument and still limited printing to about 700 characters. (#51281)

GsBitmap changes

Arguments to addAll: and removeAll:

Formerly, GsBitmap >> addAll: and removeAll: only accepted an Array argument. Now, these methods accept IdentitySet collections as arguments.

addAll: return value

addAll: also was documented to return the number of elements added to the GsBitmap, but it incorrectly returned the argument. This has been fixed. (#51339)

Added CharacterCollection and ByteArray HMAC methods

Additional HMAC methods have been added to ByteArray and CharacterCollection which support returning the HMAC as a ByteArray. Previous methods only allowed returning the HMAC as a LargeInteger or hexadecimal String. The new methods in ByteArray and CharacterCollection are:

asMd5HmacByteArrayWithKey:
asSha1HmacByteArrayWithKey:
asSha256HmacByteArrayWithKey:
asSha3_224HmacByteArrayWithKey:
asSha3_256HmacByteArrayWithKey:
asSha3_384HmacByteArrayWithKey:
asSha3_512HmacByteArrayWithKey:
asSha512HmacByteArrayWithKey:

Base64Url encoding

For JWT support, the following methods have been added:

ByteArray >> asBase64UrlString
Return a String which represents the receiver represented in base64Url format.

CharacterCollection >> asBase64UrlString
Return a String which represents the receiver represented in base64Url format.

4. Backup and Restore Changes

Improved performance for commitRestoreForFailoverAfterWaitingUpTo:

The method Repository >> commitRestoreForFailoverAfterWaitingUpTo: was added to make the failover process simpler, by combining the wait, stopLogReceiver and commitRestore operations. The performance was previously similar to the sum of the separate operations, and could be somewhat worse.

The performance is now much faster than the individual operations executed in series.

Added Repository methods

Repository >> restorePosition
Return the position of the tranlog being restored as a ScaledDecimal with scale 10.

Repository >> restoreStatusDateAndTimeRestoredTo
Returns a DateAndTime which represents time of the last checkpoint that the repository was restored to or nil if restore not active, or the restored-to time is not available. The result will be nil if restore is active but has not yet replayed any checkpoints from tranlogs.

Repository >> restoreStatusPosixTimeRestoredTo
Returns an Integer, or nil if restore is not active.

restoreStatusInfo additional field

The method Repository >> restoreStatus now includes an additional field:

17: a SmallInteger, posix time in seconds of time of last checkpoint restored to.

The existing field 11, which contains the same information in string format, is now legacy.

Added System methods

System class >> lastCommitTranlogPosition
Returns a ScaledDecimal representation of the Tranlog position of the commit record of this sessions' last successful commit. Zero is returned if this session has not committed since login, or if the stone process is configured to write tranlogs to /dev/null.

Methods that now accept DateAndTime in addition to DateTime

The following methods now accept a DateAndTime as well as a DateTime as an argument:

Repository >> restoreFromArchiveLogs:toPointInTime:
Repository >> restoreFromArchiveLogs:toPointInTime:withPrefix:
Repository >> restoreToPointInTime: 

See also restoreToPointInTime: failed with OS/Repository timezone mismatch; changes in _checkPointInTime: affect this method.

5. Other Image Changes

See Chapter 2 for additional image changes related to JSON Web Tokens (JWT) and JWT login support.

Class categories

Client products such as Jade use Class categories as a way to navigate the class hierarchy, rather than SymbolDictionaries. Historically these were not maintained for most server classes, and class categories are not visible using GBS (other than when inspecting a class).

All kernel classes now have a preliminary category assigned. The category layout and assignments are not final; further modifications for consistency and usability should be expected.

Added GsTsExternalSessions methods

GsTsExternalSession class >> newDefault: aGciTsLibrary forGemHost: aHostName
Creates a new external session that is set to the user, and stone of the current gem with the password 'swordfish', and the argument library version. The gem host is set to aHostName. You may update any of these parameters before login for more complex environments.

GsTsExternalSession class >> newDefaultForGemHost: aHostName
Creates a new external session that is set to the user, and stone of the current gem with the password 'swordfish', and the same library version as the current image. The gem host is set to aHostName. You may update any of these parameters before login for more complex environments.

GsTsExternalSession >> useOnetimePassword
Obtains a onetime password for the UserProfile of the current session and sets it for the receiver. The UserProfile of the current session must have the #CreateOnetimePassword privilege else an exception is raised. The UserProfile of the current session must also not be SystemUser, otherwise an exception will be raised at login time. The onetime password is valid for 300 seconds.

GsNetworkResourceString class >> defaultGemNRSFromCurrentForHost: aHostName
Return a new gem NRS with the gem's host being set to aHostName. Use default gem service, and default netldi, or as set by environment variable GEMSTONE_NRS_ALL.If GEMSTONE_NRS_ALL contains a #dir component, include that in result. Ignore any other parameters in the NRS of the current login.

ChildError now includes stdout

The instance variable has been added, and accessor methods:

ChildError >> stdout 
ChildError >> stdout: aString 

Additional field in descriptionOfSession:

The method System class descriptionOfSession: includes an additional field:

29. A SmallInteger, 0 or processId of the gempgsvr servicing this remote non-X509 session.

Kerberos Principal reporting methods

The following methods have been added:

KerberosPrincipal >> configurationReport 
KerberosPrincipal >> printConfigurationOn: aStream 
KerberosPrincipal >> printGroupsOn: aStream 

FileSystem changes

Handling of Undefined environment variables

When an environment variable was undefined, this previously caused low level methods to return nil, which resulted in a message not understood.

Now, this will signal an error, Error occurred (error 2710), Environment variable undefined in envName.

Incorrect result for SystemResolver >> gemLogDirectory

Linked sessions do not have gem logs; this method was incorrect in linked sessions. It incorrectly returned the root directory, and could result in MNUs under some conditions. This method now returns nil for linked sessions. (#51363)

Removed origin methods

FileLocator and SystemResolver classes previously provided methods to access the paths for the GemStone extent and tranlogs. Since FileSystem is implement for the Gem's file system (not the server’s), this was not correct and not reliable. (#51359)

The following methods have been removed:

SystemResolver >> extent1
SystemResolver >> extent1Directory
SystemResolver >> tranlog
SystemResolver >> _extent:
FileLocator class >> extent1
FileLocator class >> extent1Directory
FileLocator class >> extent:
FileLocator class >> tranlog

Changes in SystemResolver >> supportedOrigins

SystemResolver >> supportedOrigins returns different results in v3.7.4.

  • extent1, extent1Directory, and tranlog are no longer included.
  • gemLogDirectory is now included in supportedOrigins, only for RPC sessions.

Other added methods

The following methods have been added:

Boolean >> asSymbol
Returns #true if self==true, #false otherwise.

GsFile class >> serverChangeDirectory: aString
Changes the directory of the gem or topaz -l process on the server to aString. Returns true on success or false if the directory change was not successful. Raises an error is aString is not a kind of String or Utf8.

GsTestClass class >> resultDir
Return the directory into which results for tests will be written.

Locale >> name
Return the name of the Locale.

SmallInteger >> asSymbol
Returns a Symbol that indicates the numeric value of the receiver. Positive values do not include a leading +.

System class >> epochGcEnabled
Return true if epoch GC is enabled.

Removed Methods

The following methods have been removed:

FileLocator class >> extent1
FileLocator class >> extent1Directory
FileLocator class >> extent:
FileLocator class >> tranlog
GemStoneParameters >> onetimePasswordFlag
GemStoneParameters >> onetimePasswordLoginFlags
GemStoneParameters >> passwordlessLoginFlags
Repository >> _scaledDecimalFromFileId:blockId:
SystemResolver >> extent1
SystemResolver >> extent1Directory
SystemResolver >> tranlog
SystemResolver >> _extent:
TimeZone >> shouldWriteInstVar:

6. GcGem Improvements

AdminGem only runs when needed

Previously, the AdminGem was always running, although it was only needed to manage voting after an MFC, or when epoch is running. Now, the AdminGem is started up when needed, and shuts down when no longer needed.

ReclaimGem further performance and scaling improvements

In v3.7.2 and earlier 3.7.x versions, the ReclaimGem could reclaim pages at a rate that caused unreasonable extent growth. Many of these issues were addressed in v3.7.2; this release contains additional fixes to avoid issues of growth that were not fully resolved in v3.7.2.

ReclaimGem now limits frame demand when free frames are low

On a heavily loaded system, the cache may run low in free frames in the shared page cache. Since the system must search for free frames to clear, there is usually a serious performance impact. The ReclaimGem now monitors the FreeFrameCount stat produced by the shrpcmonitor and pauses when the free frame list is low. (#51272)

Reclaim of dead checkpoint frequency

The ReclaimGem reclaims dead objects after a mark/sweep, and shadow objects continually while the application is running.

Reclaim of dead objects now handles some limits differently than during reclaim of shadow objects. During dead reclaim, the free space threshold is set higher, checkpoints are triggered more often, and reclaim pauses to allow checkpoints to complete. (#51340)

STN_CR_BACKLOG_THRESHOLD was not working correctly

This configuration parameter sets a limit on the commit record backlog, above which commit records are disposed more aggressively. Since reclaimed pages are not available until after the commit record is disposed, this reduced reclaim efficiency. (#51285)

7. Utilities Changes

largememorypages -C now optional

The largememorypages script now uses a default of 1900 for the number of shared counters, and the -C argument is optional.

The computations done by this script also did not produce correct results for very large (terabyte-range) shared cache size arguments. (#51331)

startnetldi -l argument now accepts NRS %P and %M

When specifying the name of the NetLDI’s log file using the -l argument, you may now include the %N and %P NRS identifiers in the argument filename. %P is expanded to the PID of the NetLDI, and %M is expanded to the host name that this NetLDI is running on.

Improvements to copydbf -i/-I output

Identifying extents in restore mode

When an extent is in restore mode, the status was previously reported as "Extent was not cleanly shutdown; recovery is needed", since the extent requires additional actions to be usable. Now, it more specifically reports:

Extent in restore from tranlogs, restored to tranlog position N.NNNN
Oldest tranlog needed to resume restore is fileId N ( tranlogN.dbf ).

Checkpoint numeric position as well as timestamp

Previously, checkpoints were reported by start time; now, the numeric position of the checkpoint in the tranlog, extent or backup is included, as well as the timestamp. There are also minor changes in the timestamp printing format.

searchlogs output timestamps now include timezone information

The searchlogs script prints the date and time of operations (localized to the OS timezone); the output previously did not include timezone information. For example:

Old format: 2025-03-25-15:13:55.617

New format: 2025-03-25T15:13:55.617-04:00

8. GCI changes

Added login flags

Login flags have been to support password or JWT tokens for login in a file or environment variable.

Added login flags for GciLoginEx() and GsTsLogin()

The following login flags have been added. These flags apply to both classic GCI logins using GciLoginEx() and related methods, and thread-safe GCI logins using GsTsLogin() and related methods.

GCI_LOGIN_PW_FILE
Setting this bit indicates the password buffer contains a file name of a file which contains the actual password. The file name is evaluated against the file system or environment of the GCI client process. The type of password within the file is dependent on which other login flags are set. By default, the file is expected to contain a standard GemStone password string. If the GCI_JWT_PASSWORD flags is also set, then the file is expected to contain a JWT.

GCI_LOGIN_PW_ENVVAR
Setting this bit indicates the password buffer contains the name of an environment variable that references the actual password. The env var is evaluated against the file system or environment of the GCI client process. The type of password referenced by the env var is dependent on which other login flags are set. By default, the env var is expected to reference a standard GemStone password string. If the GCI_JWT_PASSWORD flags is also set, then the env var is expected to reference a JWT.

GCI_LOGIN_DEBUG
Setting this bit causes the gem process to write login debugging information to the GCI logger to assist with login debugging. No debugging information is written to the GCI client or its log file. Setting this bit also causes the gem to not automatically delete its log file.

The following only applies for JWT logins:

GCI_JWT_PASSWORD
Setting this bit indicates the provided password is a JWT in base64url format. The JWT may be provided in a buffer, in a file, or referenced by an environment variable depending if any of the other new login bit flags have been set.

GCI login flags are now validated to ensure only legal combinations of flags can be used. Invalid combinations will result in an error.

Added GCI thread-safe functions

These functions are identical to the similar existing GCI TS calls (GciTsLogin() and GciTsNbLogin()) except that the new calls accept netldiName as an additional argument.

(GciSession) GciTsLogin_(
const char *StoneNameNrs,
const char *HostUserId,
const char *HostPassword,
BoolType hostPwIsEncrypted,
const char *GemServiceNrs,
const char *gemstoneUsername,
const char *gemstonePassword,
const char *netldiName,
unsigned int loginFlags /* per GCI_LOGIN* in gci.ht */ ,
int haltOnErrNum,
BoolType *executedSessionInit, /*output*/
GciErrSType *err);
 
(GciSession) GciTsNbLogin_(
const char *StoneNameNrs,
const char *HostUserId,
const char *HostPassword,
BoolType hostPwIsEncrypted,
const char *GemServiceNrs,
const char *gemstoneUsername,
const char *gemstonePassword,
const char *netldiName,
unsigned int loginFlags /* per GCI_LOGIN* in gci.ht */ ,
int haltOnErrNum,
int *loginPollSocket /* output */);

9. Cache Statistics Changes

Cache Statistics Changes and Fixes

Stone cache statistic TimeInCheckpoint not working

This statistic was not incremented for most checkpoints. (#51342)

Added cache statistics

The following cache statistics have been added:

CommitsAfterReplay (Gem)
The number of commits that succeeded after executing RC replay.

CompletedCheckpointCount (Stn)
The number of checkpoints that have been completed since the stone was last started. See CheckpointCount

DataPagesCommitted (Gem)
Total number of data pages made persistent (committed) by the session. This statistic is updated during commit processing.

TimeInLastCheckpointMs (Stn)
Time in milliseconds from start to end of most recent checkpoint.

Removed cache statistics

The following cache statistics have been removed:

LastCommandFromClient

PageDisposesDeferred

10. Bugs Fixed

The following bugs were present in v3.7.2 and are fixed in v3.7.4:

Gem runs hot in detached execution

When a session logs in using the recently added GCI login flag GCI_PERFORM_DETACH, the Gem may run hot and consume a large percent of CPU due to excessive socket polls. (#51287)

Multithreaded operations on remote Gem with encrypted extents corrupts cache

When a Gem running on a host other than the Stone's host, and the Stone is using encrypted extents, executing a multithreaded scan operation (such as listInstances, allReferences, or markForCollection) in the remote Gem may cause cache corruption. (#51417)

ReclaimGem was not responsive while in sleepTimeBetweenReclaimUs

To control the impact of reclaim, the ReclaimGem can be configured to sleep between reclaim operations, using the parameter #sleepTimeBetweenReclaimUs. This has a maximum setting equivalent to 5 minutes. There was a bug such that the ReclaimGem did not respond to sigAborts while sleeping. If #sleepTimeBetweenReclaimUs is set to a value greater than the sigAbort timeout, on a busy system, the ReclaimGem could be killed repeatedly by lostOT processing. (#51328)

GEM_TEMPOBJ_POMGEN_SIZE always about 83%

When GEM_TEMPOBJ_POMGEN_SIZE is set to the default (0), the value should be calculated as a percentage of the GEM_TEMPOBJ_CACHE_SIZE, with the percentage decreasing as the cache size increases. However, this decreasing percentage was not applied, and the value of GEM_TEMPOBJ_POMGEN_SIZE was always about 83%. (#51375)

ReclaimGem fails to release commit token on fatal error in thread

If a ReclaimGem thread gets a fatal error (this has been seen only in a case of cache corruption), the ReclaimGem does not correctly release the commit token, preventing further commits until the ReclaimGem is manually killed or the Stone is restarted. (#51422).

Gem may be unresponsive to SIGTERM when executing in a primitive

When a Gem was executing in a primitive, it could become unresponsive to SIGTERM, and require kill -9. Now, 5 seconds after a second SIGTERM the process will exit. (#51341)

installgs reported that a valid keyfile was invalid

The installgs script was affected by changes in v3.7.2 for the new configuregs script, and the validation check when installing a keyfile reported that the keyfile was invalid. The keyfile is installed and provided it is valid, will work as usual. (#51355)

Timezone and DateTime related issues

restoreToPointInTime: failed with OS/Repository timezone mismatch

The method Repository >> restoreToPointInTime: does argument validation checks to ensure the argument is with the range of the repository birth time to the last time restored to in the repository. The check for the restore time did not calculate the time correctly when the repository was not configured to be in the same timezone as the OS. The validation was the only thing that failed, the actual restore operations did not have an issue. (#51350).

This method also now accepts an instance of DateAndTime, described here.

DateTime/TimeZone passivate-activate issues

When a DateTime is passivated, the associated instance of TimeZone, which is the current Timezone when the DateTime was created, is also passivated. The Timezone’s passivation was incomplete; recreatable data was not explicitly included. While the reactivated DateTime worked correctly, some messages sent to its TimeZone failed. (#51389)

Time instances with non-Integer milliseconds could break DateTime creation

In versions earlier than 3.7, it was possible create instance of Time with a non-Integer milliseconds field, such as a SmallDouble. After upgrade to 3.7.2, using these instances to create an instance of DateTime errored. (#51332)

System class >> processKindForSlot: could return zero

There was a very small window during the period where a session is logging in, for which System class >> processKindForSlot: for the new session's slot (invoked by another session) may return 0. The method System class >> cacheStatisticsDescriptionAt: expects a non-zero result, and reported a primitive error. (#51324)

DBF_EXTENT_SIZES was not cleared correctly on Stone restart

When DBF_EXTENT_SIZES is set and the repository is pregrown to that size, removing that configuration from the configuration file and restarting the stone reported that it was setting to unlimited, but in fact it restricted growth to the previously set limit. (#51343)

GsFileIn did not handle server file access errors in RPC environments

The class GsFileIn provides the ability to file topaz-format code into GemStone programmatically, from files on the client or server file systems. It uses GsFile to open and read files. If an error occurred when opening a server file, the incorrect (client) method was used to fetch the error; this method worked in linked sessions, but returned nil in RPC sessions. In a thread-safe environment, since client GsFile operations are disallowed, this error returned was a less understandable GciTransportError. (#51345).

pthread_join or other issues due to thread not shutdown

For a remote session that is connected to a mid-level cache on a third node, there is a thread for that session’s slot in the Stone’s cache that services reads from the mid-level cache. This thread was not being shut down at session logout. This could result in pthread_join issues or other issues. (#51367)

Upgrade could fail if Published SymbolDictionary is missing

If the Published SymbolDictionary was removed, upgrade could have failed.

Remote cache warming did not report progress count correctly

The log files for cache warming on remote gems incorrectly reported progress count as 0 until complete. (#51306)

Sending _selectiveAbort to a temporary object SEGVed

Object >> _selectiveAbort is not intended to be used on temporary objects; this method SEGVed on the attempt. #51274)

Split-tranlogs logsender slow to see commits

In a split tranlogs configuration, a logsender writes tranlogs in multiple files based on a time interval, which is separate from a logsender’s role in a hot standby configuration. The split tranlogs logsender could be slow to detect Stone commits, and commits could be missing in the split tranlogs after Stone shutdown. (#51388)

numElements and numCollisions may have become incorrect in a StringKeyValueDictionary

It was possible for the numElements and numCollections instance variables to become incorrect in an instance of StringKeyValueDictionary under certain conditions of removing entries. (#51278)

In-memory GC may have failed during load of a large method

If garbage collection of temporary object memory runs while a large method was being loaded, there is a small window in which it may have encountered an uninitialized value and fail. (#51270)

Improved error message for error #2110, OBJ_ERR_BAD_OFFSET

This error was signalled with incorrectly ordered arguments, and the resulting message was unclear. (#51409)

X509-Secured GemStone Issues

X509-Secure GemStone remote commits could error

In X509-Secured GemStone, Gems on nodes remote from the Stone execute the commit in the hostagent by default. This could result in connection errors during a Gem commit, or thread shutdown of page pusher/receiver threads for remote cache warming. (#51353)

GsTsX509ExternalSession spurious "call already in progress" errors

The initialization of a GsTsX509ExternalSession is incomplete, which resulted in invalid "call already in progress" errors. (#51382)

Killing a remote cache pageserver could cause stone issues

When a remote pageserver that is part of an X509-secured GemStone configuration is killed, it could cause Stone issues; the Stone may not be aware the remote cache is gone, and sessions may be stuck in after logout. (#51385)

Port range for mid-level cache warmer gems

When cache warming an X509-secured mid-level cache, the Gem on the mid-level cache node that is handling the warming listens for connections to the cache warmer pusher threads on the Stone’s node. This now uses the port range specified by NETLDI_PORT_RANGE.

Mid-level cache warming threads could have exited prematurely

It was possible for the threads warming a mid-level cache to exit before transferring all of the data pages found in the Stone cache. (#51364)

TotalSessionCount not decremented on when cache or hostagent killed

When an X509 remote cache or hostagent is killed, the Stone’s TotalSessionCount is not decremented for the sessions that were logged in using that hostagent. This can lead to Too many sessions logged in errors. (#51387)

Risk of hostagent SEGV when killed

When the hostegents gets a kill -TERM immediately after starting, there is a risk it may SEGV rather than terminating normally. (#51395, #51402).

X509 Netldi may have shutdown when midcache hostagent startup fails

When an X509 mid-level cache hostagent fails during startup, the X509 NetLDI may also shut down. (#51400)

 

Next chapter