1. Release Notes for 3.6.7 
Overview
GemStone/S 64 Bit™ 3.6.7 is a new version of the GemStone/S 64 Bit object server. Version 3.6.7 provides new features and fixes for a number of significant bugs. We recommend everyone using or planning to use GemStone/S 64 Bit upgrade to this new version.
These Release Notes include changes between the previous version of GemStone/S 64 Bit, v3.6.6, and v3.6.7. If you are upgrading from a version prior to 3.6.6, review the release notes for each intermediate release to see the full set of changes.
For details about installing GemStone/S 64 Bit 3.6.7 or upgrading from earlier versions of GemStone/S 64 Bit, see the GemStone/S 64 Bit Installation Guide for v3.6.2 for your platform.
Supported Platforms
Platforms for Version 3.6.7
GemStone/S 64 Bit version 3.6.7 is supported on the following platforms:
-
Red Hat Enterprise Linux Server, CentOS Linux, Rocky Linux, and AlmaLinux 7.9, 8.8, and 9.2; Ubuntu 20.04 and 22.04; all on x86.
Ubuntu 20.04 on ARM (Ubuntu on ARM is supported for development only)
GemStone performs testing on a mixture of Red Hat, CentOS, Rocky Linux and AlmaLinux servers; these are all fully certified platforms. Any reference to Red Hat applies to all these distributions. - AIX 7.1 and 7.2
-
macOS 13.1 (Ventura) with Darwin 22.2.0 kernel, and macOS 12.6 (Monterey) with Darwin 21.6.0 kernel on x86; and macOS 11.6 (Big Sur) with Darwin 20.6.0 kernel on Apple silicon.
(Mac is supported for development only)
Distributions for Solaris/x86 and Solaris/SPARC are no longer available.
For more information and detailed requirements for each supported platforms, please refer to the GemStone/S 64 Bit Installation Guide for v3.6.2 for that platform.
GemBuilder for Smalltalk (GBS) Versions
The following versions of GBS are supported with GemStone/S 64 Bit version 3.6.7:
GBS/VW version 8.7
GBS/VW version 8.6
VSD Version
The GemStone/S 64 Bit v3.6.7 distribution includes VSD version 5.6.1. The previous version of GemStone/S 64 Bit, v3.6.6, included VSD v5.6. VSD version 5.6.1 includes minor bug fixes. For details on the changes, see the Release Notes for VSD v5.6.1.
VSD v5.6.1 is included with the GemStone distribution, and can also be downloaded as a separate product. For details or to download, go to https://gemtalksystems.com/vsd/.
Changes in this version
Using setcap for large pages and OOM protection on Linux
In previous releases, the Installation Guide has specified operating system configuration steps to enable use of large memory pages and for protecting processes from the OOM killer, using setcap to give the executables specific Linux capabilities. This is not optimal for security, and has a side effect of disallowing stack traces, since gdb did not have privileges to attach to the process, and prevented statmonitor from reading certain memory statistics. (#50580)
Alternate ways to configure large memory pages and OOM protection have been determined, and the way GemStone uses capabilities has been modified for v3.6.7. The previously existing techniques work, but for security and usability reasons are no longer recommended.
Large Memory Pages
Previously, it was recommended to give the Shared Page Cache Monitor the cap_ipc_lock capability.
Now, you can avoid giving this capability, by instead selecting and configuring a huge pages group. This is the procedure:
Select or create a group that includes the Linux user that will be starting up the shared page cache, and all users who will be accessing the shared page cache.
Then create the file /etc/sysctl.d/64-gemstone-local.conf, to set this group’s numeric id to the huge pages group. Add a line to the file:
vm.hugetlb_shm_group = numericGidOfGroup
You can determine the gid of a group from the name using:
os$ getent group groupName
This will take effect on reboot; you can apply immediately by executing:
os$ sysctl --system
To verify the current value, cat proc/sys/vm/hugetlb_shm_group, which will show the group id. To enable large pages transiently without rebooting, you can set the gid of the group directly in /proc/sys/vm/hugetlb_shm_group. This will be reset on reboot if not configured as by /etc/sysctl.d/64-gemstone-local.conf.
For full details, see the updated instructions in the Installation Guide for v3.6.2.
OOM Killer protection
Previously, a number of executables were given the cap_sys_resource capability, to allow them to modify their oom_score_adjust; critical processes reduced their scores, and less critical processes, such as gems, increased their score.
Now, only the Stone itself, and the pgsvrmain for remote configurations, need the capability. The only setcap executions needed are:
os$ setcap cap_sys_resource=pe $GEMSTONE/sys/stoned
os$ setcap cap_sys_resource=pe $GEMSTONE/sys/pgsvrmain
On startup, these processes use this capability to set their oom_score_adjust, then release the capability. Other critical processes are spawned by the Stone and inherit the Stone’s oom_score_adjust, or for remote caches, are spawned by the pgsvrmain and likewise inherit oom_score_adjust. The spawned processes that inherit an oom_score_adjust do not report the protection in their log files. The scores for Gems can be safely left at the default.
The GemStone code changes to release the capability, avoid the previous issues with gdb and statmonitor, including for processes that have explicit setcap.
Raw partitions on Character devices no longer supported
The RAW driver for direct I/O has been deprecated for many years (superseded by the O_DIRECT flag for a block device). With Linux kernel 5.14, the RAW driver was dropped; this or later versions are used by Ubuntu 22.04 and RedHat 9.x.
With v3.6.7, raw partitions on character devices are no longer supported for GemStone dbf files, on older as well as recent kernels. v3.6.7 supports raw partitions on block devices.
GsTsExternalSession forkAndDetachBlock:
Normally, processes started by GsTsExternalSession are limited to execute while the GsTsExternalSession is logged in. This limits its value for starting background processes and running scripts.
The following methods have been added to support this use case:
GsTsExternalSession >> forkAndDetachBlock:
GsTsExternalSession >> forkAndDetachBlock:withArguments:
GsTsExternalSession >> forkAndDetachString:
These methods perform a non-blocking execute the given string or block. If the operation returns, it must be followed by GsTsExternalSession >> nbResult. Execution will continue running after nbLogout of this session, but until logout, it will respond to softBreak or hardBreak from the external session.
Note that the argument should include error handling and printing, so that there is enough information in the Gem log to allow problems to be debugged.
Terminating logsender and logreceiver programmatically
You may now stop the primary logsender and the logreceiver using Smalltalk expressions. The following methods have been added.
Note that these methods are in System class and require SystemControl privilege, as with any method that terminates server processes.
System class >> killLogReceiver
Causes stone to send SIGTERM to the log receiver process. Returns true if the SIGTERM signal was sent, false if no signal was sent because no log receiver process was running or a SIGTERM signal had already been sent.
System class >> killLogSender
Causes stone to send SIGTERM to the log sender process. Returns true if the SIGTERM signal was sent, false if no signal was sent because no log sender process was running or a SIGTERM signal had already been sent.
ClassOrganizer >> newExcludingGlobals
This method allows you to perform ClassOrganizer queries that do not return results exclusive to Globals. This can be used, for example, to find application methods that reference GsHostProcess or JsonParser classes (which may need to be recompiled; see the Installation Guide upgrade instructions), without including kernel methods that do not need recompiling.
ClassOrganizer >> newExcludingGlobals replaces the private method _newExcludingGlobals, which was available in some recent releases. Note that _newExcludingGlobals did not reliably exclude methods on classes in Globals.
GsNMethod >> recompileFromSource
This method has been added for convenient recompilation of a method from source code, such as the recompile needed for GsHostProcess and JsonParser. Existing recompile methods were designed for recompiling after bytecode changes; since they relied on the existing literal references, they did not update references to the obsolete class.
Note that if the symbolList of the user that is executing the recompile resolves literal names to different objects than the original compile, this may result in unexpected changes in behavior. Depending on the complexity of your application’s use of users’ symbolLists, you may wish to examine the source code before recompiling.
Converting Non-specials in special range
A special object is one in which the OOP encodes the value, and thus requires no repository space; this is a canonical form of that value. A number of magnitude classes have subclasses with the prefix ’Small’, which provide a special/canonical form of a subset of that class’s potential values. While new values are created in canonical form, it is possible to have non-canonical instances that have an exactly equivalent canonical value. For example, an upgraded application may have an instance of Fraction with the value 1/3, rather than a SmallFraction with the value 1/3.
To convert a non-canonical to the canonical equivalent, the method asCanonicalForm has been added. This either returns self, or the canonical object exactly equivalent to the receiver. Specific implementations apply to Float, Date, DateAndTime, Time, Fraction, LargeInteger, and ScaledDecimal.
Note that this cannot perform a conversion in place; you must modify the object referring to the non-canonical value to refer to the canonical value.
codePointAt:put: now performs auto-conversion if needed
Previously, sending codePointAt:put: to a String or DoubleByteString, with a codePoint that is out of range for that class, resulted in an OutOfRange error.
Now, the receiver will be transparently converted to the class that can contain the given codePoint; that is, converted from a String or DoubleByteString to a DoubleByteString or QuadByteString).
Empty Unicode7 string literals now canonicalized
In traditional String comparison mode, the string literal '' creates an instance of String, which is canonicalized (to the kernel OOP 233473). In Unicode comparison mode, in which '' creates an instance of Unicode7, these were previously not canonicalized. Now, these are canonicalized (to kernel OOP 165377).
GsTsExternalSession no longer requires cpp/gcc/g++
Previously, GsTsExternalSessions computed the FFI calls for the GciTsLibrary on demand, using cpp to parse gcits.hf. This required cpp, and the gcc/g++ compilation environment; including for production use of GsTsExternalSessions.
Now, the FFI calls are precomputed for the current version, and provided in a new file that is loaded during upgrade, $GEMSTONE/upgrade/GciTsLibrary.dat.
Note that this applies to GsTsExternalSessions that login with the same GemStone version as the current Gem. For logins to Stones running a different version of GemStone/S 64 Bit, gcits.hf still must still be parsed, and cpp/gcc/g++ continues to be required.
Better error message if reverse DNS fails for Single Sign-On (SSO)/Kerberos logins
In order to use SSO logins using Kerberos, it was and is required that you have reverse DNS lookup setup and operating correctly; see the informational bugnote for #50507. This case will now provide a more useful error message.
IndexManager added method
The method IndexManager >> allIndexesReport has been added to provide a report on all indexes in the image.
Added Repository methods
The following methods have been added:
Repository >> countInstances: anArray numThreads: maxThreads objMaxSize: aSize
Scans the entire Repository and generates an array of instance counts of the the classes contained in anArray, counting only objects with size <= aSize. Does not include in-memory objects.
listInstances: anArrayOfClasses toDirectory: aString numThreads: maxThreads objMaxSize: aSize
List instances of specified classes, listing only objects with size <= aSize. Writes files named className-instances.bm files in the directory specified by aString.
listReferencesFromBmFiles: anArray numThreads: numThreads toDirectory: aDirectoryString
Reads bitmap files of instances whose paths are listed in anArray. Writes files, one per thread, in the directory specified by aDirectoryString. Files are named references-N.txt, where N is thread number, containing one line per object containing a reference, with the format
<objId> <1-based iv offset>:<objId of value> <ivOffset>:<valueId>
Excludes references to OOP_EMPTY_INVARIANT_ARRAY, OOP_EMPTY_INVARIANT_STRING, and OOP_EMPTY_SYMBOL, which have oops 233217, 233473, and 233729. Also excludes references from instances of VariableContext, GsNMethod, and GsProcess.
Returns a report of classes and number of instances of each class containing references to objects specified by anArray.
Bugs Fixed
Very slow restore of compressed tranlogs
Restoring compressed tranlogs is unusably slow; the restore is doing excessive seeks due to a prematurely cleared buffer. This affects both gzip and lz4 compressed tranlogs. (#50692)
Performance regression in markForCollection
In v3.6, code changes were introduced to avoid thread-safety issues in garbage collection leaf caches. This change resulted in significantly slower performance (for some configurations) in v3.6 and later 3.6.x version, vs. 3.5.x and earlier. The fix has been rewritten and performance is now significantly faster than pre-3.6 releases as well as faster than 3.6.6 and earlier 3.6.x versions. (#50770).
Stone may have shutdown on checkpoint when tranlog disks full
When the tranlog disks are full, and the stone has paused checkpoints waiting for tranlog disk space to become available, if the checkpoint timeout interval expires and a checkpoint initiated, the stone shut down. (#50744)
NetLDI issues
NetLDI shutdown timing issue resulted in unable to restart
When a NetLDI services a request to fork a gem, it forks a child process and then does an exec. During the period after the fork and before the exec, a thread in the child process makes system calls to collect information for printing the log file banner. If the NetLDI is stopped using either kill -TERM or stopnetldi while a thread is making a system call, this can result in a deadlock in the child.
The fix removes the system call in which the deadlock has been observed. The forked child also now immediately closes its inherited file descriptor for the NetLDI's listening socket, rather than waiting for the close-on-exec of that socket to take effect.
Since informational system calls are no longer made during the forked period, the Gem log headers now show a reduced section for "NetLDI Child Task", which includes only the Gem command line. The system information is already duplicated in the header information printed by the Gem. (#50697)
Collection issues
Finding references failed to report objects in large IdentityBags
The results returned by the operations Repository >> allReferences:, Object >> findAllReferences, and Object >> findReferences, and methods that invoked these methods, did not include references for objects that were located in a large IdentityBag (more than 2K elements). (#50163)
Array >> replaceFrom:to:with:startingAt: may not copy elements correctly
Using replaceFrom:to:with:startingAt: where the source for the replace is the same as the receiver and the starting position in the source for the replace is less than the target starting index, the replace must copy in reverse. This case did not correctly perform all the copy operations for large collections, and produced incorrect results. (#50721)
GsSocket >> writeWillNotBlockWithin: does not handle nil returned by writeWillNotBlock
It is possible for GsSocket >> writeWillNotBlock to return a nil, which resulted in an error in GsSocket >> writeWillNotBlockWithin:. (#50775)
Issues relating to upgrades
Upgrading did not preserve default legacy indexes
In v3.4, the BtreePlus indexes were introduced, and are used by default in new repositories. Upgraded repositories defaulted to legacy indexes. This behavior on upgrade was lost due to code reorganization in v3.6; upgrading from pre-3.4 to 3.6 or later would result in the default for GsIndexOptions to include btreePlusIndex, rather than legacyIndex. (#50318)
Session methods must be disabled for upgrade
Session methods are a feature used by GsDevkit and GLASS, but should not be enabled in the ordinary base GemStone without knowledgable assistance. If GsPackagePolicy current is inadvertently enabled, it causes a number of problems. This is not a supported configuration for upgrade; now, upgrade will error if GsPackagePolicy current enabled returns true. (#50453)
copydbf returns wrong version for secondary extents after upgrade
After upgrade, the dbf version in the extent is updated with the new version; this is the version reported by copydbf -i. This update was incorrectly only applied to the primary extent. For repositories with multiple extents, copydbf -i on any of the secondary extents reported the old version. (#50749)
GLASS/Seaside auto-migrate may error after GC operation
In the GLASS/Seaside environment, Class versioning (including that done by upgradeSeasideImage, as well as code development), automatically migrates instances to the new Class version, which invokes a repository scan operation to detect the instances. This scan operation may conflict with voting or promote operations that occurs as part of an Epoch or after a markForCollection. (#50078)
Issues with stacks and memory stats when using setcap to enable LargeMemoryPages or OOM killer protection
To configure Linux to allow huge pages, to protect critical processes from the Linux OOM killer, and to lock the shared page cache in memory, the Installation Guide directed using setcap to provide specific capabilities to GemStone executables. Setting these capabilities has the effect of preventing gdb from attaching to generate C stacks, and statmonitor from collecting certain host process statistics. (#50580).
See Using setcap for large pages and OOM protection on Linux and the updated Installation Guide for more information.
Numeric and Time related issues
Float = and ~= always returned false for SmallScaledDecimal arguments
The primitives supporting the Float methods = and ~= return false, rather than failing the primitive, for SmallScaledDecimal arguments. This bypasses the handling that converts the SmallScaledDecimal to a comparable value. (#50597)
Strings with leading '-0.' lose sign when converted using asNumber or Number fromString/Stream:
Converting a string containing a decimal number that is less than zero and greater than -1, such as '-0.1', using an unspecific conversion asNumber or Number fromString: or fromStream:, resulted in a positive value, the absolute value of the correct result. (#50751)
SmallDateAndTime passivate activate failed for years 2035-2072
Activating a passivated SmallDateAndTime in the approximate year range 2035 to 2072 errored; the activation expected a SmallScaledDecimal value in the internal storage, but the stored value was not in the SmallScaledDecimal range. (#50172)
asJson problems with Date, Time, DateTime, and DateAndTime
The results produced by sending asJson were empty for instances of Date, Time, and DateAndTime. While DateTime produced a valid string, it was not quoted correctly. (#50391)
Duration new created an uninitialized instance
Duration new now returns Duration zero rather than an uninitialized instance. (#50710)
File handle leak in GsTsExternalSession
In-memory GC of a logged-in GsTsExternalSession caused a file handle leak (#50624)
Spurious OutOfMemory when in-memory pom_gen scavenge fails
When pom_gen scavenge fails, in-memory GC may signal an OutOfMemory error when not actually out of memory. (#50711)
commitRestoreForFailoverAfterWaitingUpTo: now stops logreceiver
This method now will stop the logreceiver, if it is running, as well as stopping continuous restore. (#50512)
stoned exited with code 3 on clean shutdown
On a clean stone shutdown using stopstone, stopstone exited with code 0. However, the stoned process itself exited with code 3. Now, stoned will exit with code 0 on clean stopstone. (#50222)
SEGV when a terminated GsProcess is sent terminate
From GBS, it was possible to interrupt a GsProcess that was in the process of being terminated, and send another #terminate message to it. This SEGVed if the timing was such that the GsProcess had already been terminated. (#50702)
gemnetobject -T, -N argument was ignored if no separator
When setting the temporary object cache size or native code using arguments to the gemnetobject script (e.g. set gemnetid 'gemnetobject -T value'), the -T -N argument was ignored if there was no space between the -T or -N and the argument. (#50338)
CodeModificaton required to list Pragmas with session methods enabled
When session methods are enabled (this is a feature that is used in Seaside and GsDevKit), finding or enumerating Pragmas required the user to have #CodeModification privilege; which should not be needed for a view-only operation. (#50620)