1. Release Notes for 3.6.1 
Overview
GemStone/S 64 Bit™ 3.6.1 is a new version of the GemStone/S 64 Bit object server. Version 3.6.1 fixes a number of serious bugs, as well as including a feature enhancements and bug fixes. We recommend everyone using or planning to use GemStone/S 64 Bit upgrade to this new version.
These Release Notes include changes between the previous version of GemStone/S 64 Bit, v3.6, and v3.6.1. If you are upgrading from a version prior to 3.6, review the release notes for each intermediate release to see the full set of changes.
The Installation Guide for Mac, but not for other platforms, has been updated for v3.6.1, to reflect added support for Big Sur and the Apple M1 architecture. For details on installing or upgrading on other platforms, see the GemStone/S 64 Bit Installation Guide for v3.6 for that platform.
Supported Platforms
Platforms for Version 3.6.1
GemStone/S 64 Bit version 3.6.1 is supported on the following platforms:
-
Red Hat Enterprise Linux Server and CentOS Linux 7.9 and 8.3; and
Ubuntu 16.04, 18.04, and 20.04
GemStone performs testing on a mixture of Red Hat and CentOS servers; both are fully certified platforms. Any reference to Red Hat applies to both distributions. - Solaris 10 on x86
- AIX 7.1 and 7.2
-
OSX 11.1 (Big Sur) with Darwin 20.2.0 kernel on x86 and Apple M1, and
OSX 10.15.6 (Catalina) with Darwin 19.6.0 kernel, on x86
(Mac is supported for development only)
For more information and detailed requirements for each supported platforms, please refer to the GemStone/S 64 Bit Installation Guide for that platform.
GemBuilder for Smalltalk (GBS) Versions
GemStone/S 64 Bit version 3.6.1 requires GBS version 8.5 or later for VisualWorks Smalltalk, or version 5.4.6 or later for VA Smalltalk.
The following versions of GBS are supported with GemStone/S 64 Bit version 3.6.1:
VSD Version
The GemStone/S 64 Bit v3.6.1 distribution includes VSD version 5.5.1. This is the same version of VSD that was included with the previous version of GemStone/S 64 Bit, v3.6.
Note that in GemStone/S 64 Bit v3.6 and later, statmonitor writes additional information to the statmonitor file. As a result, statmonitor files from v3.6 and later cannot be read by versions of VSD earlier than v5.5. VSD 5.5.1 can read statmonitor files generated in older versions of GemStone/S 64, 32-bit GemStone, and GBS, as well as those generated by GemStone/S 64 Bit v3.6.
VSD 5.5.1 is included with the GemStone distribution, and can also be downloaded as a separate product from https://gemtalksystems.com/vsd/
Changes in this release
Updated library versions
The version of OpenSSL has been updated from 1.1.1h to 1.1.1k
The version of LZ4 has been updated from 1.9.2 to 1.9.3
The version of OpenLDAP has been updated from 2.4.55 to 2.4.58
The version of MIT Kerberos has been updated from 1.18.2 to 1.19.1
Updated Mac Support
Additional platform support has been added for the Mac; the Installation Guide for the Mac has been updated for this release.
Distribution with native support for Apple M1
GemStone now provides a distribution for Darwin running on the Apple M1 chip, arm64. This distribution has the name GemStone64Bit3.6.0-arm64.Darwin. While the ordinary Mac distribution GemStone64Bit3.6.0-i386.Darwin can be run on the Mac on Apple M1, the emulated performance may be poor.
Note that different keyfiles are required for Mac on x86 vs. Mac on Apple M1. Contact keyfiles@gemtalksystems.com if you need an updated keyfile.
Support for Mac "Big Sur"
Configuring shared memory on "Big Sur" requires special handling. The Installation Guide for Mac for v3.6.1 includes details, or see the (informational) bugnote for 49319.
Ubuntu on arm64
A development-only distribution is now available for Ubuntu Linux v18.04 and later on the arm64 chip. This distribution has the name GemStone64Bit3.6.0-arm64.Linux.zip.
Installation is the same on arm64 as on x86; refer to the Installation Guide for v3.6 for installation details. Note that 64-bit arm is required for the GemStone server.
Distribution changes
PACKING file no longer included in the distribution
The file PACKING is no longer included in the distribution. The list of files can be seen in the zip file. To verify the contents, the website download provides the SHA256, SHA1, and MD5 checksums.
Improved handling of crashed slot recovery in Shared Page Cache Monitor
When Gems and Page Servers die, the SPCMonitor must recover their slots in the SPC before these slots can be reused for other processes. This was previously done with a single thread which recovered one slot at a time, which was insufficient for large SPCs when many Gems/Page Servers died in a short period of time.
Crashed slot recovery has been redesigned, so the SPC Mon can recovery multiple slots when it scans the cache for stuck locks. It will start 1 or more threads to check frame locks, and 1 or more threads to check hash table locks. The number of threads is determined by the cache size and number of CPU cores; 1 is the minimum, the maximum is 75% of available cores
Added cache statistics for SPC Monitor
The following cache statistics have been added:
CleanSlotsWaitingForReuse (SPCMonitor)
Number of slots in the shared cache which were cleanly detached and are currently waiting to be reused.
CleanSlotsWaitingForReuseMs (SPCMonitor)
Total real time in milliseconds taken for all cleanly detached slots in the shared page cache to be reused.
CleanSlotsWaitingForReuseTotal (SPCMonitor)
Total number of slots in the shared cache which were cleanly detached waited to be reused since the shared cache was created.
NumFrameLockLockTestThreads (SPCMonitor)
The number of threads in the shared page cache monitor allocated to test frame locks during crashed slot recovery.
NumHashTableLockTestThreads (SPCMonitor)
The number of threads in the shared page cache monitor allocated to test hash locks during crashed slot recovery.
SlotRecovThreadState (SPCMonitor)
The state of the slot recovery thread in the shared page cache monitor process.
TimeWaitingForLockThds (SPCMonitor)
The total real time in milliseconds spent by the shared page cache monitor's main recovery thread waiting for the hash and frame lock test threads to complete their tasks.
GsHostProcess now signals newly added ChildError
Previously, if the child of a GsHostProcess returned a non-zero exit status, it signalled an Error. Now, it signals a ChildError, a newly added subclass of Error.
GsBitmap handling of page order oop files
When performing operations on a large number of objects, it is more efficient to operate on the objects sequentially in page order, rather than in OOP order. A GsBitmap is inherently in OOP order, so to use page order GsBitmap files, a method has been added that will return all or some of the objects from a page-order bitmap file as an Array.
The bitmap files created by Repository >> listInstancesInPageOrder:toFile: are interchangeable with bitmap files written by GsBitmap >> writeToFileInPageOrder:.
However, the page-order bitmap file format is not compatible with ordinary GsBitmap file format. You cannot use GsBitmap >> readFromFile: with a page-order bitmap file, or vice-versa.
The following added methods support reading objects from page order files:
GsBitmap class >> numberOfObjectsInPageOrderFile: fileName
Answer the number of objects in a page order oop file. The specified file must be in page order. The session must be in a transaction when this method is invoked.
GsBitmap class >> readObjectsFromPageOrderFile: fileName startingAt: startIndex upTo: endIndex
Reads, validates and returns an array containing the valid oops in page order from a page-ordered bitmap file. startIndex is the index of the first object (1-based), endIndex is the index of the last object to return from the file.
Object identifiers (OOPS) contained in the file may no be longer valid, due to garbage collection since the file was written. Objects which are no longer valid have nil stored in their place in the returned array.
Auditing GsBitmaps
When GsBitmaps are read from file, if objects have been garbage collected, some of the objects may no longer exist. The following methods detect this condition.
Note that if the oop has been subsequently reused for a new object, that cannot be detected by this audit code.
The following methods have been added:
GsBitmap class >> auditFile: fileName
Reads the GsBitmap-format file and counts the number of objects in the file which are no longer valid. Returns an Array containing two numbers: the total number of oops in the file, and the number of invalid oops in the file.
Objects are considered invalid if they no longer exist (i.e., are not present in the shared object table) or are present in the dead objects set maintained by stone.
The session must be in a transaction when this method is invoked.
GsBitmap class >> auditPageOrderOopFile: fileName
Reads the page order oop file and counts the number of objects in the file which are no longer valid. Returns an Array containing two numbers: the total number of oops in the file, and the number of invalid oops in the file.
Objects are considered invalid if they no longer exist (i.e., are not present in the shared object table) or are present in the dead objects set maintained by stone.
The session must be in a transaction when this method is invoked.
Object >> initialize default implementation
The method Object >> initialize has been added, with an empty operation. This allows subclasses to send "super initialize" from their initialize methods without considering the location within the hierarchy.
General implementation of removeAllSuchThat:
The method removeAllSuchThat: aBlock was implemented previously for SequenceableCollection, Interval, and SymbolList.
A more general but inefficient implementation has been added to Collection, so this function is available for all kinds of Collection.
Excluding Globals from ClassOrganizer queries
When performing senders and implementors, it can be helpful to not include matches that are in GemStone kernel code, that is, on classes in Globals.
To use this feature in topaz: use the added method ClassOrganizer >> _newExcludingGlobals, and define the topaz variable CurrentClassOrganizer. This allows later commands senders, implementors, literals, strings to exclude Globals.
topaz 1> send ClassOrganizer _newExcludingGlobals
topaz 1> define CurrentClassOrganizer **
After this, organizer-related commands in topaz (such as senders, implementors, literals, and strings) will exclude methods in classes in Globals.
Other added method
The following method has been added:
PositionableStreamPortable >> beforeEnd
Returns true if the receiver can access more objects, false otherwise.
Other Cache Statistics changes
The following cache statistics have been added, in addition to the statistics documented here.
CommitRecordOfView (Gem)
CommitRecordOfView is the commit record that is being referenced by the session's view.
PageMgrRemoveFrameId (Stone)
The frameId that the Page Manager thread is attempting to recycle.
PageMgrRemovePageId (Stone)
The pageId that the Page Manager thread is attempting to remove from the stone cache.
Bugs Fixed
Incorrect incremental tranlogging of changes to kinds of IdentityBag from within nested transactions
With incremental tranlogging, only specific changes to collections are logged to the transaction logs, for efficiency. These incremental tranlog entries were incorrect for kinds of IdentityBag, from within nested transactions. Changes to an IdentityBag or a subclass of IdentityBag that are made within a nested transaction, may be written incorrectly to the transaction logs. If the database is later restored from backup and the transaction logs replayed, or if the transaction logs are applied to a warm or hot standby, the collection will be incorrect in the replayed repository; the collection could contain nils instead of valid objects. (#49382)
Added config parameter STN_TRAN_INCREMENTAL_LOGGING
To manually control incremental tranlogging, the configuration parameter STN_TRAN_INCREMENTAL_LOGGING has been added. The default is true, provides behavior from previous releases. When false, operations that are usually incrementally logged are instead logged in full. This will result in larger tranlog volume.
STN_TRAN_INCREMENTAL_LOGGING
FALSE shuts off the incremental logging of additions or deletions to nodes of large objects, with side effect of more bytes written to tranlogs.
Default TRUE.
Symbol Garbage Collection may still collect referenced Symbol
A Symbol whose only reference is from an object in the possible dead set, may be garbage collected even if the referencing object is voted to stay alive and not promoted to dead. (#49339)
logsender may read incomplete tranlog records
In a hotstandby system, when the logsender transmit records to the logreceiver, there is the possibility that while the write operation completed, the disk fsync was not complete, and so the record read from disk by the logsender and sent to the logreceiver may be incomplete. (#49449)
A related problem still exists in this version, in the case where the logsender is not connected to the master Stone (using the -s flag), and therefore has to rely on disk file writes to detect when a new record is available. A logical record may cross a file system block boundary, and this is not detected by the logsender’s disk file checks. It is strongly encouraged to use the -s flag to avoid this risk; the Stone tracks the logical record boundaries and provides this information to the logsender.
Logins could have failed due to stuck page manager thread
The Stone's page manager thread could get stuck in page removal under some load conditions, which prevented further logins. (#49381, #49407)
Upgrade issues
Upgrade required DataCurator password to be manually reset
The upgradeImage step requires that you change the password for SystemUser to the default. The SystemUser then changes the password for DataCurator to perform some steps of the login. The order of operations was incorrect in some recent versions, such that DataCurator’s password had to be manually reset as well. (#49365)
Upgrade required DataCurator not have NoGsFileOnServer permission
In v3.6, GsFile is used for some steps of upgrade. upgradeImage failed if DataCurator had the NoGsFileOnServer reverse permission.(#49365)
DataCurator permission issues with performOnServer whitelist
The added feature of a whitelist for performOnServer: required adding an instance variable to UserSecurityData. However, UserSecurityData instances were not explicitly migrated during upgrade, and only SystemUser has sufficient permissions to view or auto-migrate UserSecurityData instances.
If DataCurator attempted to add a path to the whitelist for a UserProfile in an upgraded repository, this resulted in a permission error. (#48291)
Behavior >> instSize insufficient to iterate named instance variables
When iterating the named instance variables, previously the method instSize was used, but this did not handle some cases of instance variables in leaf classes. (#49223)
The method Object >> namedSize has been added, is used instead of instSize.
Object >> namedSize
Returns a SmallInteger, the number of named instance variables in the receiver.
Error from ProcessScheduler
The ProcessScheduler may encounter an error "ProcessScheduler _runNextProcess failed, new 20 old 20" under some timing circumstances (an EINTR occurs, and the method _findReadyProcess returns nil). This may have been possible in earlier versions, but became more likely in 3.6. (#46280).
GEM_ABORT_MAX_CRS impacts GBS traversals
The configuration parameter GEM_ABORT_MAX_CRS limits the number of commit records examined on abort. If commit record backlog is larger than this limit, the subsequent traversal result seen by GBS does not include all of the exported objects; updates to replicated but not modified objects in that session, that were changed by other sessions, may not be visible. (#49434)
In addition, the parameter documentation incorrectly stated that a value of 0 meant scan all records; in fact, 0 meant scan no records. (#49448)
Issues related to automatic startup of statmonitor
Stone did not auto-start statmonitor early enough when in recovery
When STN_STATMONITOR_ARGS is set, the Stone starts statmonitor on startup; which is possible as soon as Stone connects to the shared page cache. However, when the Stone was in recovery (after an unclean shutdown), it did not start statmonitor until recovery was complete, which meant no statistics information was available on the recovery itself (#49442)
GsSocket issues
Code paths in which tranlog full may result in Stone shutdown
When tranlogs are full, operations such as checkpoints should pause and wait for tranlog space to become available. There were some code paths, such as epoch GC and fullBackup, in which they to start a checkpoint, in which the Stone shut down instead of waiting for tranlog space to be available. It is always recommended to avoid exhausting disk space. (#49208)
GcGem logging issues
Reclaim log entries omitted
Entries providing information on reclaim activity, after the first entry, were not printed to the ReclaimGem’s log file. (#49388)
Warnings on range test incorrect
Some out of range warnings in the ReclaimGem log displayed incorrect information. (#49396).
ProfMonitor did not automatically delete its file
While ProfMonitor is performing monitored, e.g., using monitorBlock:, it records data in a disk file. These files were not being automatically deleted. (#49406)
Issues with Multi-Threaded scan operations
Additional fix for "Repository scan may fault in dead objects"
When a repository scan operation is running while sessions are voting on dead objects, there were a small number of code paths that allowed objects to be faulted into memory that were being promoted to dead. This would potentially allow references to dead objects to be committed. The known cases of this bug were fixed in v3.6; an additional case was found and fixed in this release. (#49051)
DateAndTime now performance issue in 3.6
With the changes to DateAndTime in v3.6, the performance of DateAndTime now became much slower, which impacted operations, such as logging using GsEventLog, that make frequent calls. Performance has been restored to the pre-3.6 level. (#49425)
Missing server library may cause GCI client process termination
GCI clients such as VisualWorks load the main GemStone library, libgcirpc, which in turn loads libssl and libkrb5. If either of these libraries is not available, the load fails. This terminated the GCI client in recent GemStone server versions. (#49245)
On Windows clients, the "RUNNING ON" platform displayed is incorrect
The log files on Windows 10 clients printed a RUNNING ON value of NT 6.2, which is incorrect for Windows 10. (#49294)
Topaz issues
Non-interactive topazwaitfordebug in script difficult to terminate
When the topazwaitfordebug command is invoked from a non-interactive script, it does not respond to control-C (which is correct behavior). However, attempting to kill the process using UNIX kill -TERM did not work; kill required locating and killing a second process. (#49198)
GsTsExternalSession issues
ArgumentError during GsTsExternalSession >> send:to:withArguments:
If the initial use of send:to:withArguments: for a particular instance of GsTsExternalSession has fewer arguments than a later use of the same method, the later use may get an error such as "offset not within bounds of memory owned by aCByteArray". (#49284)
Memory leak in X509-secured NetLDI and HostAgent
The specialized NetLDI and HostAgent processes that support X509-Secure GemStone logins had a memory leak in the login/logout process. (#49411)
System _writtenObjects from within nested transaction is incomplete
When System _writtenObjects is executed within a nested transaction, it does not return the full closure of the transaction. This usage should be replaced with System enumerateDirtyList: -1. (#49211)
Some cases of GsPackagePolicy may break filein
In some configurations with GsPackagePolicy enabled, in v3.6 the ordinary filein of GemStone code (e.g. using the topaz input command) may be broken.
With GsPackagePolicy enabled, the use of the transient symbol dictionary was inadvertently enabled due to change in v3.6 handling of the transient and persistent symbolLists. This meant the filed-in changes were put in the transient symbol dictionary, which would therefore disappear on logout.
In addition, when filing in class fileouts, the fileout format includes topaz directives to remove all methods on that class. The effects of removeAllMethods are not applied to the transient symbol list, so after filein and logout, if the class previously existed, it was left with no methods. (#49328)
Gem login failures may not report details
When a login fails with error 4138/REP_ERR_SYSTEM_REP_NOT_ATTACHED, it did not report the details of why the login failed. The reported case was an extent file permissions issue. (#49273)