1. Release Notes for 3.7.2

Platforms for Version 3.7.2

GemStone/S 64 Bit version 3.7.2 is supported on the following platforms:

• Red Hat-compatible Linux 7.9, 8.9, and 9.4, and Ubuntu 20.04, 22.04, and 24.04 on x86_64. GemStone is tested on a mixture of Red Hat, CentOS, and Rocky; these are all considered fully certified platforms. Any reference to Red Hat applies to any Red Hat-compatible distribution.
• Ubuntu 20.04 on ARM. Linux on ARM is for development only, not for production.
• macOS 14.5 (Sonoma) and 11.7.10 (Big Sur), on x86 and Apple silicon (ARM). macOS distributions are for development only, not for production.

Note that GemStone/S 64 Bit v3.7 and later on Linux are built using machine instructions that are not available on older x86_64 CPUs (more than about 10 years old); v3.7.2 will not run on these CPUs, regardless of the Linux OS version.

For more information and detailed requirements for each supported platforms, please refer to the GemStone/S 64 Bit v3.7 Installation Guide for that platform.

GemBuilder for Smalltalk (GBS) Versions

The following versions of GBS are supported with GemStone/S 64 Bit version 3.7.2:

VSD Version

The GemStone/S 64 Bit v3.7.2 distribution includes VSD version 5.6.1. This is the same version of VSD that was included with the previous release, v3.7.1.

VSD 5.6.1 is included with the GemStone distribution, and can also be downloaded as a separate product from https://gemtalksystems.com/vsd/

Rowan

Rowan 2 and Rowan 3 are provided with v3.7.2:

• Rowan 2 is version 2.8, on masterV2.8
• Rowan 3 is version 3.2, on masterV3.2

GemBuilder for Java

You must use the concurrently released GemBuilder for Java (GBJ) v3.2 with GemStone/S 64 Bit v3.7.2; GBJ versions 3.1.3 and earlier are not supported. GBJ v3.2 is provided for Linux only, and does not support clients on other platforms; please contact GemTalk Technical Support if you require other platforms.

There are a number of infrastructure changes in GBJ v3.2; see the Release Notes for GemBuilder for Java v3.2 for more information.

GBJ installation now requires that the Stone’s TimeZone match the OS TimeZone.

The v3.7.2 distribution includes the shared libraries that support GBJ 3.2 on Linux:

libgbjlnk320-3.7.2-64.so

libgbjts320-3.7.2-64.so

GemConnect

Both GemConnect v2.4 and v2.5 are supported with GemStone/S 64 Bit 3.7.2, on Linux only.

On some recent platforms, there are OS library changes needed:

• In the Ubuntu 24.04 release, the library libaio.so.1 was replaced with a 64-bit version, libaio.so.1t64.0.2. Users on Ubuntu 24 and later should setup a symbolic link to accommodate the name change.
• Installations of RedHat 8.x and 9.x no longer automatically include the library libnsl.so.1. Users on RedHat 8 and later should manually install this library.

The GemStone/S 64 Bit v3.7.2 distribution includes the shared library that supports GemConnect v2.4. To use GemConnect v2.5, you may use make to link with the 3.7.2 release, or contact GemTalk Technical Support.

New script to support OS configuration

the configuregs script has been added to making basic configuration of simple systems easier; for example, setup of the /opt/gemstone/locks and logs directories. This avoids the need to use installgs, or to manually configure these files.

This script writes answers to an answers file, which can be edited and reused.

New script to setup a simple repository using data-conf

A new script, createNewGemStoneRepository has been added, to make setting up a simple Data-conf configured repository simple. A data-conf repository is created with extents and other files in a repository other than $GEMSTONE/data, and uses the -E argument and a copy of the gemstone_data.conf configuration file.

Installation Guide

Chapter 1 of the Installation Guide for v3.7.2 has been extensively updated.

• OS configuration information has been moved to the end of the chapter, since this is primarily optimizations that do not need to be considering when performing the installation.
• The material regarding creating a new repository has been moved to a separate chapter, Chapter 2.
• Use of the new configuregs script has been added, in addition to the installgs description, which has been simplified.
• Use of the new createNewGemStoneRepository script has been added. The classic way of setting up a basic new GemStone repository in $GEMSTONE/data is also now documented.

Upgrade

GemStone 3.7.2 documentation now includes support for upgrade directly from 3.3.x. This was tested and supported for earlier 3.7.x versions, but not documented.

Improvements/fixes to RPM installation process

The RPM installation process has been reviewed and improved.

Previously, RPM installation was documented to use rpm -i. RPM installation more correctly should be done using dnf (or yum on older OS versions). The previous RPM distribution’s package name for GemStone, however, was GemStone64.x86_64. Since this was the same for all GemStone versions, installing a new version using dnf/yum would override any earlier RPM versions. (#51189)

With v3.7.2, the RPM package is named GemStone64-3.7.2.x86_64, and dnf/yum is safe.

On earlier versions of GemStone, you should continue to use rpm -i.

RPM installations could have failed in v3.7 and later with spurious dependency errors.

In addition, on minimal systems in which Perl was not installed, RPM installations could have failed due to Perl dependencies. Now, the RPM installation using dnf/yum will install a number of required Perl packages if they are not already present. On earlier GemStone versions, with rpm -i, you should include the flag --nodeps. (#50965).

On a system in which /opt/gemstone/locks and /opt/gemstone/log directories were not already defined, the permissions of the newly created directories were not correct. (#50966)

TimeZone installation now supported from the image

The GemStone distribution extent has the TimeZone of America/Los_Angeles, so most new installations will need to update the TimeZone. Previously, this was done by editing and executing the $GEMSTONE/upgrade/installtimezone script.

Now, this can be done from the image, using the new methods. These methods must be executed as SystemUser.

TimeZone class >> installNamedZone: zonenameString
Install the named timezone, from the GemStone distribution Olson zoneinfo database in $GEMSTONE/pub/timezone. The new TimeZone is made the default/current. The existing default TimeZone is sent become: with the new TimeZone, so existing references to the old TimeZone automatically now refer to the new TimeZone.

TimeZone class >> installOsTimeZone
Install a TimeZone based on the OS time zone settings for the host on which the Gem executing this command is running. This uses /etc/localtime which is correct for Linux and Mac. The new TimeZone is made the default/current. The existing default TimeZone is sent become: with the new TimeZone, so existing references to the old TimeZone automatically now refer to the new TimeZone.

Change in behavior in 3.7.2 that may affect existing application code

In earlier versions, there was a file descriptor leak in GsHostProcess. This bug has been fixed, which results in an apparent behavior change in GsSocket >> readWillNotBlock; see File descriptor leak in GsHostProcess.

The "leaked" file descriptor was the Gem process's writing end of the stdout and stderr pipes, and the read end of the stdin pipe. After the child was forked, the Gem previously failed to close its stdout/sterr write file descriptor, and this socket is inherited by the child. When the child process exited, it closed its own write file descriptor, but the Gem's was still open (although there was no way for data to appear on this socket).

As a result, in previous releases GsSocket >> read:* operations from the Gem to the child process, after the child process had exited, hung waiting for data to appear on the still-open socket.

With the fix for #47630, GsSocket >> read:* operations from the Gem to the child process after the child process had exited now correctly see the EOF and return 0.

However, GsSocket >> readWillNotBlock also now (correctly) returns true in the case when the child has exited. Since the read:* operation on this socket will return an EOF without blocking, this is intended behavior, although it was not visible in earlier releases since the socket was not actually closed.

If existing code does not also check for read:* returning 0, and only checks the result of readWillNotBlock, then it may fail to recognize when the child has exited. Applications that are upgrading should check their application code to ensure the correct loop checks are done when communicating with the child process in GsHostProcess.

In general, readWillNotBlock is not a reliable way to detect child status, although it is useful to avoid a read:* call waiting indefinitely for data to appear. For instance, there can be a false answer if the child process has not started writing or if it pauses in its writing.

Provided that your code can handle the GsProcess blocking until the child process has sent some data or closed its end of the pipe, there is no need to invoke readWillNotBlock.

Updated library versions

The version of openssl has been updated to 3.0.15.

The version of libssh has been updated to 0.10.90.

32-bit Linux no longer supported

With v3.7.2, GemStone/S 64 Bit does not support 32-bit Linux. The directories /bin32 and /lib32 are no longer included in the distribution.

Likewise, GBS/VW with 32-bit VisualWorks is no longer supported on Linux.

The Windows client does include 32-bit libraries and GBS is supported with 32-bit client Smalltalk on Windows.

Removing GemStone system locks on reboot

If GemStone server processes are terminated uncleanly, such as a SEGV or an OS shutdown, they may leave their lock files behind in /opt/gemstone/locks/. These can normally be cleaned up using gslist -c. However, if the OS is rebooted and some process running as a different UNIX user gets the process ID contained in one of the stale .LCK files, then GemStone code cannot safely delete the stale .LCK file.

It is recommended to clear out all .LCK files in /opt/gemstone/locks/ on reboot. An example has been added to demonstrate this:

$GEMSTONE/examples/admin/systemd/gemstone-clean-locks.service

The example services for the Stone and NetLDI, gemstone.service and netldi.service, have been updated to use gemstone-clean-locks.service.

Gem log header includes additional NRS information

The Gem log header includes a section specific to the Gem’s login, in addition to the header information by the NetLDI that spawned it. This now includes an additional line providing the NRS from the Gem’s environment, since this may be different from that used by the NetLDI.

Multithreaded scan operations now have a thread limit of 128

Issues were found in which the C code supporting multithreaded scans was not sufficiently protected by mutexes when moving to a new phase of the operation, which could result in unexpected errors or missing results (#50904, #50929). This code has been adjusted to include the appropriate mutexes, and there is now a hard limit of 128 on the number of threads used in a multithreaded scan.

Multithreaded-related query methods no longer require SystemControl

Methods that fetch the current settings for mtMaxThreads, mtThreadsLimit, and mtPercentCpuActiveLimit previously required both SessionAccess and SystemControl privileges; now, only SessionAccess is required. SystemControl is still required to modify these settings.

Improved reclaim behavior with low free space

Previously, the ReclaimGem provided the parameter #reclaimMinFreeSpaceMb, which was designed to suspend reclaim under low free space conditions, to avoid excessive extent file growth before any of the newly reclaimed pages became available. This parameter was not checked at the correct point and was thus ineffective, with the risk of runaway extent growth during reclaim. (#51260, #51258, also here)

However, pausing reclaim has risk of reclaim becoming stuck and the repository remaining in a state of low free space, since reclaim is the way pages are made available again.

To avoid the risk of either stuck reclaim or runaway growth, the Reclaim Gem’s parameter #reclaimMinFreeSpaceMb has been removed. Reclaim now is suspended when free space is below the Stone’s STN_FREE_SPACE_THRESHOLD. Under low space conditions, reclaim will proceed slowly until pages become available.

The ReclaimGem’s SessionStat29, #FreeSpaceLowCount, will report the number of times reclaim was paused due to low free space.

SequenceableCollection + removed

In 32-Bit GemStone/S before v5.0, SequenceableCollection>>+ was used to combine collections and strings; this was made obsolete in version 5.0, but has remained in the image to support customers who have progressively upgraded from very old versions.

This method creates issues with Pharo compatibility, and has been removed in this version.

This should not affect customers who have used the deprecation mechanism to remove dependence on deprecated methods. SequenceableCollection >>+ has been deprecated using GemStone’s deprecation mechanism since that mechanism was introduced in v3.0 (2011).

fillFrom:resizeTo:with: no longer usable

The primitive invoked by Array >> fillFrom:resizeTo:with: and KeyValueDictionary >> fillFrom:resizeTo:with: was subject to out of memory errors (see Memory issues from fillFrom:resizeTo:with:).

The use of this method in KeyValueDictionary table resize was unnecessary; resetting the collection is just as efficient. (#50990). This method has been removed from the image.

Array >> fillFrom:resizeTo:with: remains in the image to allow detection using the deprecation mechanism, but is obsolete and signals an error.

The method Array >> fillFrom:to:with: has been added, as a replacement for filling an Array. Note that code that previously used fillFrom:resizeTo:with: will need modification, since the Array sizing must be done separately.

Array >> fillFrom: startIdx to: endIdx with: anObject
Store anObject into instVars startIdx..endIdx of the receiver. The receiver will be grown if necessary. Attempts to grow the receiver beyond 2034 total instance variables (named and unnamed) signals an error.

Double and QuadByteStrings disallow 16rD800..16rDFFF

Characters with codepoints in the range 16rD800..16rDFFF are not valid for Unicode. Characters with these codepoints are disallowed in UnicodeStrings.

Now, they are also disallowed from DoubleByteString and QuadByteString instances, to avoid the problem conversion of DoubleByteString and QuadByteString instances to Unicode16, Unicode32, or a UTF class. Strings containing codePoints in this range are inherently unusable for any string-related purpose.

Improvements to reduced-conflict collections

A number of bugs have been fixed and feature requests implemented, related to RC collections. In addition to the changes listed here, see specific bugs under Conflict handling in Reduced-conflict collections.

Non-identity-dictionary >> keys now returns appropriate class of result

Sending #keys to instances of following classes previously returned instances of IdentitySet, although the dictionary is not identity-based. For these classes, sending #keys now returns an instance of Set:

KeyValueDictionary

IntegerKeyValueDictionary

KeySoftValueDictionary

StringKeyValueDictionary

RcKeyValueDictionary

Optimizations and bug fixes in TreeDictionary/TreeSet

TreeDictionary/TreeSet are kinds of Dictionary/Set optimized to avoid performance variability in adding elements. These classes were added in v3.7.1. There have been a number of optimizations in the supporting code in v3.7.2. In addition, the following bugs have been fixed:

Support for OpenSSH keys and conversion between key types

To support OpenSSH keys, and allow conversion between RSA and PEM format keys, the classes GsSshPublicKey and GsSshPrivateKey have been added.

GsTlsCredential >> isDsa

GsTlsCredential >> isEllipticCurve 

GsTlsCredential >> isRsa

GsTlsCredential >> isOpenSshKey

GsTlsCredential >> isOpenSslKey

GsTlsCredential >> class isOpenSshClass

GsTlsCredential >> class isOpenSslClass

Validating an SNI name against the target host name

GsSecureSocket now supports validating that a client connection’s SNI (Server Name Indication) name matches a host name in the peer certificate.

By default, this additional validation is not done; the same validation is done as in previous releases.

To enable validation, add the expected host name to the GsSecureSocket before making the secure connection, using GsSecureSocket >> addExpectedHost: or GsSecureSocket >> setExpectedHost:. Multiple hosts can be added to a Socket’s expected hosts.

During the connection TLS handshake, if the certificate’s names don’t match the expected host name, the connection is not completed. This validation is done automatically during the connection, if the expected hosts is set.

To disable validation, set the expected hosts to nil using
GsSecureSocket >> setExpectedHosts: nil.

The matching can be made more strict by setting additional flags using
GsSecureSocket >> setExpectedHostFlags:.

#X509_CHECK_FLAG_ALWAYS_CHECK_SUBJECT

#X509_CHECK_FLAG_NO_WILDCARDS

#X509_CHECK_FLAG_NO_PARTIAL_WILDCARDS

#X509_CHECK_FLAG_MULTI_LABEL_WILDCARDS

#X509_CHECK_FLAG_SINGLE_LABEL_SUBDOMAINS

#X509_CHECK_FLAG_NEVER_CHECK_SUBJECT

Other added methods for managing certificates

The following methods on GsSecureSocket and GsX509Certificate allow you to query for more details about the peer certificate.

GsSecureSocket >> matchedPeerName
Returns a String representing the DNS hostname or subject CommonName from the peer certificate that matched one of the hosts set by the #setExpectedHost: or #addExpectedHost: methods. Returns nil if no host matching was performed. Raises an exception if the receiver is not a client socket or if the secure connection has not been established.

GsSecureSocket >> peerCertificate
Answer an instance of GsX509Certificate representing the peer's certificate. Raises an exception if the receiver has not completed the TLS handshake with its peer. Returns nil if no certificate was sent by the peer. This will always happen if anonymous TLS is used and can happen when certificate verification is disabled.

GsSecureSocket >> peerCertificateChain
Answer an instance of GsX509CertificateChain containing the peer's certificate chain. Raises an exception if the receiver has not completed the TLS handshake with its peer. Returns nil if no certificate was sent by the peer. This always happens if anonymous TLS is used and may happen when certificate verification is disabled.

GsX509Certificate >> isSelfSigned
Answer a Boolean indicating if the receiver is a self-signed certificate.

GsX509Certificate >> issuerName
Returns a string representing the issuer common name of the receiver.

GsX509Certificate >> isValidNow
Answer a Boolean indicating if the receiver is valid at this point in time, that is the current time falls within the window between the receiver's 'not before' and 'not after' times.

GsX509Certificate >> notAfterTime
Returns a SmallDateAndTime representing the 'not after' time of the receiver in GMT.

GsX509Certificate >> notAfterTimeGmtSeconds
Returns a SmallInteger representing 'not after' time of the receiver expressed as the number of seconds since 00:00:00UTC January 1, 1970.

GsX509Certificate >> notBeforeTime
Returns a SmallDateAndTime representing the 'not before' time of the receiver in GMT. GsX509Certificate >> notBeforeTimeGmtSeconds. Returns a SmallInteger representing 'not before' time of the receiver expressed as the number of seconds since 00:00:00UTC January 1, 1970.

GsX509Certificate >> notBeforeTimeGmtSeconds
Returns a SmallInteger representing 'not before' time of the receiver expressed as the number of seconds since 00:00:00UTC January 1, 1970.

GsX509Certificate >> subjectAlternateNames
Returns an Array of Strings representing the contents of the subject alternate extension contained in the receiver, or an empty array if the receiver does not contain the extension.

GsX509Certificate >> subjectName
Returns a string representing the subject common name of the receiver.

GsSftpSockets may not be nonblocking

There is a known bug in libssh, #58: Opening sftp fails in non-blocking mode, that prevents non-blocking GsSftpSockets from being used. (#50942)

The following methods are not supported for GsSftpSocket:

makeNonBlocking

nbSshConnect

nbSshConnectTimeout:

GsSecureSocket examples

New debugging environment variable

GsSecureSocket secureAccept may fail with SSL error

During secureAccept, GsSecureSocket calls readWillNotBlockWithin:, which calls _peek, which calls SSL_peek(). This call would fail with an error if the socket was not yet connected, depending on the timing. (#50937)

GsHostProcess with Arguments

Using GsHostProcess to execute C code with arguments that include whitespace and quotes has Smalltalk-specific requirements for escaping/formatting. Hand applying formatting is error-prone; and including unmatched single quotes was not possible.

Now, arguments can be supplied as an array of strings to an additional argument. The following methods have been added:

GsHostProcess class >> execute: commandLineString args: anArray
A variant of the existing GsHostProcess class >> execute: method with the added args: argument.

GsHostProcess class >> execute: commandLineString input: stdinString args: anArray
A variant of the existing GsHostProcess class >> execute:input: method with the added args: argument.

GsHostProcess class >> fork: commandLineString args: anArray
A variant of the existing GsHostProcess class >> fork: method with the added args: argument.

In these methods, the commandLineString is parsed for space separated arguments. anArray may be nil or an Array of instances of String, Unicode7 or Utf8.

Elements of anArray are appended to the arguments contained in commandLineString, to form the total argv array of the child; any quoting or whitespace within an element of anArray is passed directly to the child in the corresponding element of child's argv.

When using GsHostProcess, if any argument has complicated quoting or whitespace, it is recommended that commandLineString should contain only the full path to the executable, and arguments should be passed as elements of anArray.

GsHostProcess execute: '/bin/cat ''a b.txt'''

GsHostProcess execute: '/bin/cat' args: {'a b.txt'}

Support for warming AllSymbols

When doing limited cache warming on repository startup, you may now also manually warm AllSymbols, using the new methods:

System class >> warmAllSymbols:

System class >> warmAllSymbols: numSessions parameters: aGemStoneParameters 

These functions start an external session to perform the warming. Note that warmAllSymbols: takes its login parameters from the current session. This is not sufficient to login if the current user’s password is not swordfish or the NetLDI is performing authentication.

In most cases you should use warmAllSymbols:parameters:, which allows you to pass in an instance of GemStoneParameters, either based on the current session or using other parameters.

For example:

| params | 

params := GemStoneParameters newDefault.

params 

password: 'currentGemStoneUserPassword'; 

hostPassword: 'myHostPassword';

yourself.

System warmAllSymbols: 5 parameters: params.

Changes in commit conflict reports

AllSymbols audit

The method CanonSymbolDict >> _audit has been added, providing a way to audit AllSymbols.

System class

External Session changes

Other Added methods

GsTestCase logging changes

GsTestCase logging now reports stacks for assertion failures more similarly to the way errors are reported. Instead of just FAILURE:, the leading line now will report:

ERROR 2753  a ResumableTestFailure occurred (error 2753)

GsTestCase >> assert:identical: has improved printing when SmallInteger arguments do not match.

JsonParser parsing of exponential notation

Previously, JsonParser parsed expressions using scientific notation, such as '1E-1', into instance of Fraction. Now, instances of SmallDouble or Float are returned.

Changes in Number>>kind for some classes

A fraction of 0/0 previously reported kind as infinity, now this is a NaN.

FixedPoint previously always returned kind as normal; now it will return NaN, infinity, and zero if appropriate.

Logging to GCI server rather than client

There were remaining maintenance methods that logged output to the client rather than the server. To allow these to execute using the thread-safe GCI, such as via GsTsExternalSession, these have been changed to log output to the server. Logging to client or server is the same in linked sessions, but in a topaz RPC sessions, output now goes to the Gem log. The following methods are affected:

GsNMethod class>>convertArrayBuildersIn:list:

GsNMethod class>>convertArrayBuildersInTopazScript:

GsNMethod class>>_convertArrayBuildersIn:list:refDir:

Repository>>_shrinkExtents

System class>>startCheckpointSync

System class>>_commitPrintingDiagnostics

Upgrade3Init class >> _allUsers_addInversePrivilege

Newly deprecated methods

Removed methods

DecimalFloat >> _asStringE

GciLibrary >> GciHostInstallFaultHandler_:_:

GemStoneParameters >> _getArg:key:

GemStoneParameters >> _secureArg:key:

GemStoneParameters >> _securePasswords

GemStoneParameters >> _zeroArgPrim:

GsTestCase >> repeatIfInterruptedByGC:

GsTestCase >> shouldRepeatIfInterruptedByGC

GsTlsCredential class >> newFromPemString:

HtInternalNode >> makeRoomAtIndex:

KeyValueDictionary >> fillFrom:resizeTo:with:

LogEntry >> argArray:

LogEntry >> receiver:

LogEntry >> selector:

Object >> _primitiveSelectiveAbort

PrivateObject >> _primitiveSelectiveAbort

RcCounter >> _getInvalidSessionIds

RcCounter >> _getSessionElementFor:

RcIdentityBag >> _components:

RcIdentityBag >> _indexForSessionId:

RcIdentityBag >> _privateRemove:fromRemovalBag:logging:updateRcValueCache:

RcIdentityBag >> _privateRemove:logging:updateRcValueCache:

RcIdentityBag >> _processRemovalBagFor:logging:

RcIdentityBag >> _thisSessionAdditionIndex

RcKeyValueDictionary >> _rebuildTable:logging:

RcKeyValueDictionary >> _removeKey:logging:

RcQueueSessionComponent class >> speciesOfElements

RedoLog >> addLogEntry:forConflictObject:

RedoLog >> addLogEntry:forLargeConflictObject:

RedoLog >> clear

System class >> _cancelWaitForDebugExitClient

System class >> _removeFromRcQueue

TimeZone class >> fromSolaris

TimeZone class >> _initialize1

TimeZone class >> _initialize2

startlogsender

startlogsender has the following additional arguments:

-W <path>  path to write-once filesystem for writing tranlogX.dbf.sN split tranlogs. Requires -F.

-F <writeTimeoutSecs> Write tranlogs as split tranlogs to the path specified by -W. <writeTimeoutSecs> is the allowed time for holding a tranlogX.dbf.sN open before closing that file and opening tranlogX.dbf.sM where M equals N + 1. Only used for split tranlogs. Requires -D.

When -F and -W are used, -A is disallowed, as are the SSL options -C -J -K -Q -S -V.

The logsender can be stopped and restarted, and the writing of records to split tranlogs will seamlessly resume where the previous logsender stopped.

startlogreceiver

startlogreceiver has the following additional argument:

-F <writeTimeoutSecs> Write tranlogs as split tranlogs to the directory specified by -T. <writeTimeoutSecs> is the allowed time for holding a tranlogX.dbf.sN open before closing that file and opening tranlogX.dbf.sM where M equals N + 1. Only used for split tranlogs. Requires -D.

There are no argument restrictions when using -F.

Tranlog chunks

The tranlog chunks are named tranlogTLNumber.dbf.schunkID, where the first chunkID is 0001. The chunk ID is not limited to 4 digits; 0-padding is for convenience for testing using cat, etc. With a 30 second write timeout, 4 digits provides enough chunk ids for a single tranlog to be copied as split tranlogs for more than 80 hours before requiring 5 digits. The new chunk file is not actually created until there are transactions ready to be written, so an idle system will not continuously create new chunk files.

Once the timeout has expired and the logsender is ready to start a new chunk file, the previous tranlog is fsynced to disk and made user read-only, with permission 400.

If a write, fsync, or chmod on a chunk file returns an error, the logsender will exit.

Image support

Added System methods to control split tranlogs logsender process:

System class >> killLogSenderSplitLogs
Causes stone to send SIGTERM to the log sender process that was started with the arguments -F -W.

System class >> logsenderSessionIdSplitLogs
Return the session ID of the logsender process started with -W -F, or zero if no logsender is connected to the stone process. Note a logsender can exist without a connection to the stone.

Example

For example, to start writing split tranlogs, with chunks written every 30 seconds to a directory named /gshost/splitTranlogs/, use an expression such as the following:

startlogsender -F 30 -W /gshost/splitTranlogs/ -P 54321 -s myStoneName

If this runs for 90 seconds in an active system, it would produce the following split tranlog chunk files, in addition to writing tranlog1.dbf in the normal tranlog location:

/gshost/splitTranlogs/tranlog1.dbf.s0001

/gshost/splitTranlogs/tranlog1.dbf.s0002

/gshost/splitTranlogs/tranlog1.dbf.s0003

Copydbf

Chunk files are only opened read-only.

Chunk files can be used directly for copydbf or restore, by specifying the first tranlog chunk file (tranlog1.dbf.s0001). This will look for all tranlogs chunk files tranlog1.dbf.s0001...tranlog1.dbf.sN until a file N+1 is not found.

For example,

copydbf -i tranlog1.dbf.s0001

will report the status of all tranlog1.dbf.s* files.

You can use copydbf to create a merged file, which is the same as the normal tranlog1.dbf written by the Stone. The copydbf argument of the first chunk file (tranlog1.dbf.s0001) will automatically pick up the remaining chunk files.

For example:

copydbf tranlog1.dbf.s0001 tranlog1.copy.dbf

Creates a merged file containing all the tranlog1.dbf.s* files (with monotonically increasing numbers). If the stone’s ordinary tranlog is named tranlog1.dbf, then:

diff tranlog1.copy.dbf tranlog1.dbf 

returns a status of zero.

File-based concatenation

Chunk files can also be concatenated using cat, which produces a file that is identical to the ordinary tranlog that was written by the Stone.

For example,

cat tranlog1.dbf.s* > tranlog1.mergedChunks.dbf

This requires that all chunks be part of the monotonically increasing set; missing files in the sequence are not detected.

Also note that if more than 4 digits have been used when writing files (that is, a file tranlog1.dbf.s10000 was created), any scripts using cat would need to generate the proper sequence of input files, to ensure the files are in the correct order.

Restore from Tranlogs

When tranlogs are opened for restore, it will look for s tranlog named tranlog1.dbf.s0001 as well as the other supported filename patterns (tranlog1.dbf, tranlog1.dbf.lz4, and tranlog1.dbf.gz). If tranlog1.dbf.s0001 is found, it is opened, read-only, as a split tranlog and all the chunk files tranlog1.dbf.s0001... tranlog1.dbf.sN until a file N+1 is not found, are restored.

Once all the chunk files that composed the tranlog are restored, it will continue restoring the next tranlog (tranlog2), which may be in any of the four supported filename patterns.

Other access

Low-level pgsvr tranlog access can read split tranlogs without combining.

However, printlogs and searchlogs do not support access of split tranlogs. To use these utilities, merge the split tranlogs using copydbf or file-based concatenation.

Added configuration parameter

The following has been added, to avoid out of memory conditions in which the code_gen region size is the limiting factor.

GemCommitConflictDetails now accepts 0...3

The gem configuration parameter GemCommitConflictDetails previously accepted true or false as an argument.

Now, it is a numeric value between 0 and 3 (inclusive). 1 and true are equivalent and 0 and false are equivalent.

Values of 2 and 3 provide additional tracing of RC replay.

Cache warming for X509-secured GemStone

The argument string supplied to the X509-Secured GemStone-only parameter NETLDI_WARMER_ARGS now accepts an additional argument to support cache warming:

-w integer -- interval in minutes for shrpcmonitor to write a working set file

stopstone changes

Previously, if the Stone took a long time to shutdown, stopstone could return before the stone was completely shut down and the shared memory segment released, which meant that a subsequent startstone failed.

In v3.7.2, the shutdown sequence has been redesigned to avoid this situation. stopstone will block until the shared memory segment is released and startstone will be able to succeed. The Stone itself will now shut down before the Shared Page Cache Monitor. The stone and SPCMon log file messages reflect this new sequence.

The stopstone -t secs argument allows you to specify a timeout. By default (if -t is not specified), this is -1, and stopstone will wait forever for the Stone to stop and the shared memory segment to be available. Previously, stopstone errored with an explicit -t -1 argument.

System >> shutDown also blocks until shutdown is complete.

startnetldi further checking for consistent port and numeric name

In recent releases, startnetldi has tightened checking to ensure that a setting in $GEMSTONE_NRS_ALL matches the specified netldi server name.

Now, startnetldi also checks the -P and -X arguments, and any mappings in the services database.

• If the services database maps a alphabetic-named netldi to a port, this port number must now match any -P argument, startnetldi numeric netldi name, or a numeric-named #netldi specification in $GEMSTONE_NRS_ALL or an argument passed with -X.
• When the startnetldi -X argument is used, a netldi name, or numeric name signifying port use must agree with a numeric netldi name provided as a startnetldi argument. Since the -X argument overrides a $GEMSTONE_NRS_ALL, this does not need to match.
• A port number supplied to -P must match a numeric netldi name provided as a startnetldi argument, or a numeric-named #netldi specification in $GEMSTONE_NRS_ALL or an argument passed with -X.

Utility failures have additional [ERROR] result

When utilities such as startstone and startnetldi have argument errors or other startup issues, they report an error message. A number of cases in which the error message was not preceded by [ERROR] or Error: now correctly include this prefix.

stoned command requires -l argument

The stoned command is normally invoked by startstone, however it is legal to invoke it directly. In 3.7.x, the -l argument to stoned is required.

In earlier v3.7 versions, the stoned utility SEGVed in absence of -l argument; now, it errors and the stone does not start. (#51144)

startstone, startnetldi only print full usage if -h is specified

If an error occurs with a GemStone utility, usually the full usage is printed after the error message. For some utilities this is long, obscuring the actual error. Now, startstone and startnetldi will print only the usage lines, not the option details, if an error occurs. To see full details, use the -h argument.

Topaz Changes

set gemnetid !@hostx#netldi:12345!gemnetobject

set stone myStoneName user DataCurator

set gemnetid !#netldi:12345!gemnetobject

topaz> set stone myStoneName user DataCurator netldi 54321

set stone myStoneName user DataCurator

set gemnetid !@hostx#netldi:12345!gemnetobject

topaz> set stone myStoneName user DataCurator netldi 54321

topaz> set gemnet !@hostx!gemnetobject

topaz 1> list method addToLog:

addToLog: aString

resultToReport isNil ifTrue: [resultToReport := String new]. 

resultToReport add: aString. 

---ORIGIN: Trait method #'LoggingTrait'---