1. GemStone/S 64 Bit 3.5.3 Release Notes 
Overview
GemStone/S 64 Bit™ 3.5.3 is a new version of the GemStone/S 64 Bit object server. This release includes a number of new features and bug fixes. We recommend everyone using or planning to use GemStone/S 64 Bit upgrade to this new version.
These release notes describe changes between the previous version of GemStone/S 64 Bit, version 3.5.2, and version 3.5.3. If you are upgrading from a version prior to 3.5.2, review the release notes for each intermediate release to see the full set of changes.
For details about installing GemStone/S 64 Bit 3.5.3 or upgrading from earlier versions of GemStone/S 64 Bit, see the GemStone/S 64 Bit Installation Guide for v3.5.3 for your platform.
Supported Platforms
Platforms for Version 3.5.3
GemStone/S 64 Bit version 3.5.3 is supported on the following platforms:
-
Red Hat Enterprise Linux Server 6.9, 7.6, and 8.1; and
Ubuntu 16.04, 18.04, and 20.04 - Solaris 10 and 11.4 on x86
- AIX 6.1, 7.1, and 7.2
-
OS X 10.14.6 (Mojave) with Darwin 18.7.0 kernel, and OSX 10.15.1 (Catalina) with Darwin 19.6.0 kernel, on x86
(Mac is supported for development only)
For more information and detailed requirements for each supported platform, please refer to the GemStone/S 64 Bit Installation Guide for v3.5.3 for that platforms.
GemBuilder for Smalltalk (GBS) Versions
GemStone/S 64 Bit version 3.5.3 requires GBS version 8.4 or later for VisualWorks Smalltalk, or version 5.4.5 or later for VA Smalltalk.
The following versions of GBS are supported with GemStone/S 64 Bit version 3.5.3:
GBS version 8.4
VisualWorks 7.10.1 is no longer considered fully certified, since it is out of support with Cincom.
GBS version 5.4.5
Windows 7 reached end of life in January of 2020. While GemTalk continues to test on and support GemStone clients on Windows 7, it will no longer be considered fully certified.
For more details on supported GBS and client Smalltalk platforms and requirements, see the GemBuilder for Smalltalk Installation Guide for that version of GBS.
VSD Version
The GemStone/S 64 Bit v3.5.3 distribution includes VSD v5.5. This is the same version that was included with GemStone/S Bit v3.5.2; the updated build number reflects added statistics definitions.
VSD versions are not tied to GemStone server versions: both older and newer versions of VSD can be used to read statmonitor files generated by both older and newer versions of GemStone/S and GemStone/S 64 Bit.
Changes in this release
Updated Linux platform support
Support has been added for Red Hat 8.1 and Ubuntu 20.04. Note that previous versions of GemStone cannot run on Ubuntu 20.04 due to OS changes in mmap().
GemStone is no longer certifying on SUSE, although it is expected to work. Please contact GemTalk Technical Support if you need certification on this platform.
GemStone on Mac is now distributed as an Apple-notarized .dmg
On more recent versions of OSX 10.15 (Catalina), the GateKeeper security restrictions have been changed for downloaded executables, resulting in applications being quarantined by default. With earlier distributions of GemStone, downloading the GemStone distribution via a web browser resulted in LSFileQuarantineEnabled being set, and the application being quarantined. (#48841)
To allow downloaded distributions to run correctly, GemStone for the Mac has modified the distribution to conform to Apple’s security protocols. The GemStone distribution is now provided as a .dmg, rather than a .zip file, and the build process includes changes such that the distribution is signed and notarized by Apple.
Improvements for using Large Pages
Adjusted values for alignment
When large pages are being used, GemStone will automatically increase the number of frames in the SPC to minimize memory waste due to large page alignment requirements.
The number of large pages used, and the physical size of the cache, is never larger than the values that would have been used under the previous behavior of leaving the extra memory wasted.
largememorypages script on Linux
The output produced by the script largememorypages has been improved.
- the minimum settings for shmall and shmmax are now included in the output
- The instructions have been updated to use grub, and provide distribution-specific instructions
- The instructions have been simplified to omit transient configuration of large pages.
References to the script startshrpcmonitor have been removed from the instructions; this script is not present in 3.5 and later.
The undocumented obsolete script getlargepages is no longer included in the distribution.
GsTlsPrivateKey methods renamed for consistent capitalization
The following methods capitalized the final P in PassPhrase, which was inconsistent with other usages in the image. These methods have been renamed:
newFromPemFile:withPassPhrase:
newFromPemFile:withPassPhraseFile:
newFromPemString:withPassPhrase:
newFromPemString:withPassPhraseFile:
newFromPemFile:withPassphrase:
newFromPemFile:withPassphraseFile:
newFromPemString:withPassphrase:
newFromPemString:withPassphraseFile:
External Sessions using Thread-safe API
In addition to the standard GCI libraries, GemStone also includes a thread-safe GCI (defined in gcits.hf), with a simplified set of functions providing comparable functionality.
The existing GsExternalSession uses the standard GCI library to run GemStone code in a different session. GsExternalSessions may log into different Stones than the Gem executing the code, but both sessions must be the same GemStone version.
This release adds the class GsTsExternalSession, which provides a way to run external sessions using the thread-safe GCI library. This may provide faster performance for some applications.
GsTsExternalSession can also be used to login sessions running a different version of GemStone. However, use of this features requires support in both versions; currently, only v3.5.3 and recent alpha versions of 3.6 support thread-safe external sessions.
GsTsExternalSession uses the added class GciTsLibrary to access the thread-safe GCI.
Example using GsTsExternalSession to log into an alpha 3.6 Stone:
| param sess result |
param := GemStoneParameters new
gemStoneName: 'gemstone36';
username: 'DataCurator';
password: 'swordfish' copy;
gemService: '!#netldi:ldi_36!gemnetobject';
yourself.
sess := GsTsExternalSession
parameters: param
library: (GciTsLibrary
newForVersion: '3.6.0'
product: '/lark/users/gsadmin/GemStone64_360').
sess login.
result := sess executeString:
'System stoneVersionReport at: ''gsRelease'''.
sess logout.
result
'3.6.0'
Added GsObjectInventory methods
GsObjectInventory can now create profiles based on GsBitmaps, both from instances of GsBitmap and from GsBitmap files.
GsObjectInventory class >> profileObjectsInBm: aGsBitmap
Generate a profile of the objects in the given GsBitmap.
GsObjectInventory class >> profileObjectsInFile: aGsBitmapFile
Generate a profile of the objects in the bitmap file.
Repository scan disallowed while vote is not idle
After a garbage collection operations such as a markForCollection, Gems must vote on the dead objects. During this voting phase, there is a small risk of issues with symbol finalization. To avoid risk, it is now disallowed to run repository scan operations such as objectAudit and backup, while voting is not idle.
See also Symbol GarbageCollection issue.
Performance reimplementation of ReadStream >> upTo:
The method ReadStream >> upTo: has been reimplemented to be faster.
skip: added to BtreePlusGsReversedIndexReadStream, BtreePlusGsIndexReadStream
When iterating over the results of an index query using BtreePlus structures, you may now use skip: to advance over elements. Note this does not apply to indexes with other structure types.
Behavior >> isSpecial should not be used; replaced by Behavior >> areInstancesSpecial
The method Object >> isSpecial can be sent to any object to determine if it is a special (the OOP encodes the value, such as a Character or SmallInteger). Historically, this method was overridden by Behavior >> isSpecial, which returned if instances of the class would be Specials, rather than the status of receiver itself.
This inconsistency is being removed over coming releases.
The method Behavior >> areInstancesSpecial has been added, and should be used to test if instances of a class are Specials.
In some future release, the method Behavior >> isSpecial will be removed, so the behavior will be inherited from Object >> isSpecial. The means that sending #isSpecial to a kind of Behavior will return false, since no instance of a kind of Behavior is itself special.
Other added methods
The following methods have been added:
CFunction >> name
CFunction >> coerceNameToSymbol
GciInterface >> lastErrorString
GciInterface >> login: aGemStoneParameters flags: flagsInt
GciError >> error: aGciErrSType in: anExternalSession details: aString
ErrorDescription >> asString
GemStoneParameters >> clearQuietLogin
GemStoneParameters >> setSoloLogin
GsExternalSession >> waitForReadReady
Deprecation Change
The method DecimalFloat class >> on:do:, which has had no code functionality in GemStone/S 64 Bit, has been marked as deprecated.
Removed Private Methods
The following private methods have been removed:
GsExternalSession >> _errorIfCallInProgress
GsExternalSession >> _getBytes:
GsExternalSession >> _postLogin
GsExternalSession >> _setSessionId
GsExternalSession >> _signalIfError
Repository >> _buildAllRefsResult:
Repository >> _buildAllRefsResult:withLimit:
startstone -A
The startstone utility supports the -A argument, which performs the pageaudit function.
Configuration Parameter Changes
Added configuration parameter
The configuration parameter STN_RC_LOOKAHEAD_LIMIT has been added to allow control over performance for applications with heavy use of reduced-conflict collections. See Commit performance when using RC collections.
STN_RC_LOOKAHEAD_LIMIT
Maximum number of entries in the queue of RC elements in the stone that will be analyzed after a session that was on the queue commits.
Runtime equivalent: #StnRcLookaheadLimit
Default: 5
Minimum: 1
Maximum: 100
Changes in STN_STATMONITOR_ARGS with -X
The statmonitor -X argument runs a statmonitor that collects host statistics only, and does not attach to a Stone. When this was started during Stone startup by including it in the argument to STN_STATMONITOR_ARGS or a similar configuration parameter, the statmonitor was launched, but did not naturally stop when the Stone was stopped. Since using statmonitor with the -X argument is independent from the Stone, -X has been disallowed in STN_STATMONITOR_ARGS, GEM_STATMONITOR_ARGS, and GEM_STATMONITOR_MID_CACHE_ARGS.
Statmonitor recording some information on NetLDIs
statmonitor now records some process information on NetLDI processes, which is available for viewing in VSD. Statistics recorded include the added HeapKBytes, which records C Heap memory used by the process, and can detect leaks such as described under Memory leaks in NetLDI.
To disable collecting NetLDI statistics, the statmonitor -T option has been added.
New Cache Statistics
The following cache statistics have been added:
HeapKBytes (All)
The size of the process's heap in kilobytes.
RcReadSetSize (Gem)
Number of objects in the session's RcReadSet.
RcReadSetSizeLastCommit (Gem)
Number of objects in the session's RcReadSet at the beginning of the last commit.
RcTransQueueUnionSize (Stone)
Last computed value of size of rcReadSetUnion in stone's RcTransQueue code paths.
TotalFailedCommits (Stone)
The total number of failed commits performed by all sessions since the stone was last started.
Added GCI Thread-Safe functions
The following functions have been added to the GCI thread-safe API.
GciTsNbExecute
Non-blocking version of GciTsExecute. This function returns TRUE if execution started successfully. Use GciTsNbResult to get the result.
(BoolType) GciTsNbExecute(
GciSession sess,
const char* sourceStr,
OopType sourceOop,
OopType contextObject,
OopType symbolList,
int flags,
ushort environmentId /* normally zero*/,
GciErrSType *eerrrr
);
GciTsNbLogout
Non-blocking version of GciTsLogout, which logs out the session. Do not wait for a result from the gem process. If the return value is FALSE, an error is returned in *err.
(BoolType) GciTsNbLogout(
GciSession sess,
GciErrSType *err
);
GciTsNbPerform
Non-blocking version of GciTsPerform. This function returns TRUE if execution started successfully. Use GciTsNbResult to get the result
(BoolType) GciTsNbPerform(
GciSession sess,
OopType receiver,
OopType aSymbol,
const char* selectorStr,
const OopType *args,
int numArgs,
int flags,
ushort environmentId /* normally zero*/,
GciErrSType *err
);
Bug Fixes
Memory leaks in NetLDI
There were several memory leaks that occurred on requests to the NetLDI.
When not running in guest mode, there was a 16 bytes per-request leak in v3.4.x and v3.5.x. In all NetLDI modes in v3.5.x, there was a 2k byte per-request leak. (#48875)
To ensure leaks such as this are caught, a cache statistic HeapKBytes has been added, with the size of the process's heap in kilobytes; and statmonitor by default records statistics on NetLDI processes. Continually increasing HeapKBytes in a NetLDI demonstrates the problem.
Unnecessary LDAP calls from NetLDI in guest mode
When the NetLDI is in guest mode, it does not need to authenticate the host user that is connecting. However, it was making calls to perform the validation, which created unnecessary calls to LDAP. Now, this validation call is not performed by the NetLDI. (#48427)
Commit performance when using RC collections
Reduced-conflict collections perform processing on the Stone to try to determine if the changes in a commit will be a conflict. While this processing can improve overall system commit performance, when the number of objects affected by RC operations becomes very large it may use excessive CPU and so reduce performance. (#48778)
The Gem cache statistics RcReadSetSize and RcReadSetSizeLastCommit have been added to monitor this behavior.
Limiting analysis
The configuration parameter STN_RC_LOOKAHEAD_LIMIT has been added to allow applications to throttle back the look-ahead analysis. In earlier releases, a limit of 10 was always used. Applications that have heavy use of RC collections and excessive Stone CPU use during commit may benefit from reducing the value further.
Temporary objects unnecessarily added to RcReadSet
When using Reduced-conflict Collections, objects are added to the RcReadSet, to ensure they are preserved for retry of conflicts. Temporary objects, which are not involved in commit conflict resolution, were being unnecessarily added to the RcReadSet and passed to the Stone. This seriously impacted commit performance when there was significant activity on larger RC Collections. (#48819)
Symbol GarbageCollection issue
Symbol GarbageCollection was significantly updated for scalability in v3.5.2. Further changes have been made in this release. This code included a codepath within the internal step of unhiding symbols, which could encountered an object does not exist and error. (#48535, #48861).
It is now disallowed to run most repository scans while voting is not idle.
Timeouts during RPC GCI logins
When logging in an RPC client, where the Gem and client are on separate hosts over a slower connection, the timeouts in some phases of the login process were not correct, resulting in occasional unnecessary login failures. (#48793)
Indexes with optionalPathTerms error on removing elements without the term
GemStone indexing allows you to specify the optionalPathTerms option, which allows you to add objects to an indexed collection that do not have an instance variable matching the path term that the index is on. Attempting to remove an object that did not have that instance variable from a collection with optional path term index, however, encountered an error. (#48965)
FFI CHeader parsing issues
Upgrade Issues
Incorrect version test during upgrade broke session methods
During upgrade, session methods, which are used by applications with Seaside code loaded, are handled according to the original version of the repository. The check for the previous version was not correct, leaving session methods disabled. (#48866)
Upgrade did not remove #defaultValues in GcUser's UserGlobals.
The key in which GcGem parameter definitions are located was changed in v3.5. Previously, all definitions were in #defaultValues, now, they are #defaultReclaimConfigValues and #defaultAdminConfigValues. Upgrade did not remove the obsolete key. (#48757)
listReferences: results did not respect order of argument
The method Repository >> listReferences: takes an array of classes and returns an array of arrays of instances. In v3.5 and later, the order of subarrays within the results array did not match the order in the argument. (#48789)
GsObjectInventory could report 0 bytes for some collection classes
For some kinds of NSCs, such as Set and RcIdentityBag, the bytes reported could have incorrectly been zero. This is related to an NSC with the actual bytes held in a sub-object. (#48824)
Backup made during reclaim may have object audit failures after restore
When making a programmatic backup while the dead object reclaim is running, following a markForCollection, there have been cases where the resulting backup, when restored, has object audit errors (with object does not exist errors). There were fixes in v3.5.2 to address this issue; these were incomplete, and further changes are included in this release. (#48583)
Solo login when /tmp/gemRO_<pid>_extent1.dbf exists
A Solo (no stone) login expects to be able to create a file in the /tmp directory; this is used only for a few operations such as GsBitmap that require pages, and login is allowed even if this file cannot be created. Previously, if this file could not be created, operations that required pages would SEGV; now, these operations will fail with a repository full error. (#48801)
Slight risk of Gem fatal error on kill -USR1
Sending kill -USR1 pid to a GemStone process causes the C and Smalltalk stack traces to be printed to the log file for that process. If this is sent to a Gem (including a linked topaz) that is performing in-memory GC within a call to a Stone or a PageLocate, this can result in a Gem fatal error composing the stack. (#48774)
Checking stack for self may incorrectly report corrupt object error
Operations such as become: and migrate are disallowed when the receiver is on the stack. The check for this, in Object >> _onStackAsSelfBelow:, could report a corrupt object error. (#48878)
Cache Statistics Fixes
The Gem cache statistics TimeProcessingCommit and TimeWaitingForCommit were updated incorrectly or not updated (always zero). (#48808)
The ShrPcMonitor cache statistic LocalDirtyPageCount was not updated and always zero. (#48807)
The values for the BmCHeapPages and MemMappedSize configuration parameters could incorrectly be negative. (#48810)