1. Release Notes for 3.7 
Overview
GemStone/S 64 Bit™ 3.7 is a new version of the GemStone/S 64 Bit object server. Version 3.7 includes a number of important new features, including compiler optimizations, changes to process scheduling and debugging, new FileSystem support, multithreaded instance migration, customer special classes, and other new and enhanced features and bug fixes.
These Release Notes include changes between the previous version of GemStone/S 64 Bit, v3.6.6, and v3.7. If you are upgrading from a version prior to 3.6.6, review the release notes for each intermediate release to see the full set of changes.
For details about installing GemStone/S 64 Bit 3.7 or upgrading from earlier versions of GemStone/S 64 Bit, see the GemStone/S 64 Bit Installation Guide for v3.7 for your platform.
New keyfiles required
The keyfiles for v3.6.x and earlier cannot be used with v3.7; new keyfiles are required for this release. To obtain a new keyfile for GemStone/S v3.7, write to keyfiles@gemtalksystems.com. In your request, include your license information, platform and any updates to contact information.
Please contact GemTalk Technical Support if you have issues or questions.
Supported Platforms
Platforms for Version 3.7
GemStone/S 64 Bit version 3.7 is supported on the following platforms:
-
Red Hat-compatible Linux 7.9, 8.7, and 9.2, and Ubuntu 20.04 and 22.04, on x86;
Ubuntu 20.04 on ARM (Ubuntu on ARM is supported for development only)
GemStone is tested on a mixture of Red Hat, CentOS, Rocky and Alma; these are all considered fully certified platforms. Any reference to Red Hat applies to any Red Hat-compatible distribution. - AIX 7.1 and 7.2
- macOS 12.6.8 (Monterey) and 11.7.9 (Big Sur) on x86; macOS 13.4.1 (Ventura) and 11.7.8 (Big Sur) on Apple silicon (ARM)
Note that GemStone/S 64 Bit v3.7 was built using compilation features that are not available on older CPUs (more than about 10 years old); v3.7 will not run on these CPUs, regardless of the Linux OS version. See here for more details.
Distributions for Solaris/x86 and Solaris/SPARC are no longer available.
For more information and detailed requirements for each supported platforms, please refer to the GemStone/S 64 Bit v3.7 Installation Guide for that platform.
X509-Secured GemStone feature is tested and supported on Linux platforms only.
GemBuilder for Smalltalk (GBS) Versions
The following versions of GBS are supported with GemStone/S 64 Bit version 3.7:
GBS/VW version 8.7
VSD Version
The GemStone/S 64 Bit v3.7 distribution includes VSD version 5.6.1. The previous version of GemStone/S 64 Bit, v3.6.6, included VSD v5.6.
VSD version 5.6.1 includes improved handling for appended files. For details on the changes, see the Release Notes for VSD v5.6.1.
With GemStone/S 64 Bit v3.7, statmonitor uses an additional optimization file to improve lz4 compression; this file is used by VSD when reading an lz4-compressed statmonitor data file. Further details on this change can be found here.
As a result, lz4-compressed statmonitor files from v3.7 cannot be read by versions of VSD earlier than v5.6. VSD 5.6.1 can read statmonitor files generated in older versions of GemStone/S 64, 32-bit GemStone, and GBS, as well as those generated by GemStone/S 64 Bit v3.7.
VSD 5.6.1 is included with the GemStone distribution, and can also be downloaded as a separate product from https://gemtalksystems.com/vsd/
GemBuilder for Java (GBJ) Version
The most recent version of GBJ, v3.1.3, can be used with GemStone/S 64 Bit v3.7.
GBJ 3.1.3 has not been tested on AIX with v3.7, although it is expected to work, and the distribution on AIX does not include the GBJ shared library. Contact GemTalk Technical Support if you require GBJ on AIX.
GemConnect Version
The most recent version of GemConnect, v2.4, can be used with GemStone/S 64 Bit v3.7.
Sparkle
Sparkle, the Pharo IDE for GemStone, provides GemStone development tools in the Pharo client Smalltalk environment. GemStone/S 64 Bit v3.7 is required with Sparkle.
Sparkle is under active development, and there are ongoing changes in GemStone server code to support Sparkle.
See the Installation guide for Sparkle in the Documentation subdirectory of the project, under https://github.com/GemTalk/Sparkle/.
Upgrade
Upgrade is supported from all 3.4.x, 3.5.x, and 3.6.x versions. Upgrade from 3.3.x is not disallowed, but is not tested or supported. To upgrade from earlier versions, upgrade to a supported upgrade origin version, and then upgrade to 3.7.
Note that JsonParser has changed definition in v3.7, and GsHostProcess changed definition in 3.6.5. If you have subclasses of these classes, or methods that reference either class directly or by name, you may need to recompile methods and/or edit source code, to ensure these methods work as intended. See the Installation Guide for v3.7 for details.
Note that several obsolete or deprecated configuration parameters have been removed. These will now error when read, rather than being ignored. You may need to update legacy configuration files if you have not done so previously. See Removed configuration parameters.
For large systems with many symbols, in which upgrade performance is critical, we are now recommending to run System clusterAllSymbols in the old environment, prior to upgrade.
isNil/notNil
GemStone/S 64 Bit v3.7 includes new configurable optimized selectors, described here; these optimized selectors include isNil, notNil, and yourself. If your application includes method overrides to implement isNil or notNil, you will need to either modify your application, or disable the affected optimized selectors. Instructions for this are in the Programming Guide and are summarized here.
Seaside and GsDevKit
The code for webSocket, which is optional but used in Seaside/GLASS, includes classes that reimplement isNil. As a result of the isNil/notNil issue, the isNil selector is not optimized in the distribution seaside extent, nor when bootstrapping GLASS into extent0.dbf. See https://github.com/GsDevKit/GsDevKit_home/issues/331 for more information.
The upgradeSeasideImage has been updated to disable optimization of isNil by default. The -E argument has been added to upgradeSeasideImage, which bypasses the step of disabling optimization (and thus results in isNil remaining optimized). This argument can be used if your application does not override or load code with a custom isNil implementation.
Documentation Changes
Documentation has been revised for this release, with modifications to incorporate new and changed features, as well as corrections and improvements.
In addition to the maintenance changes, and addition of new information and features, the following improvements have been made:
- The Topaz User’s Guide has an additional chapter describing scripting, including existing shell script information and new superDoit scripts.
- The System Administration Guide section on Hot Standbys has been updated, and the Appendix describing Configuration File handling has been rewritten for completeness and clarity.
- The Programming Guide has a new chapter for File handling, including GsFile as well as File System; and the FFI chapter has been extensively revised.
- A new Installation Guide specific for Linux RPM installations has been added, the Installation Guide for Linux RPM.
A number of recently added features are documented in Supplemental Documentation; see awskeymanagement, monitorwithprometheus, upgradeViaHotStandby, and the new AzureKeyVault.
The GemStone/S 64 Bit X509-Secured GemStone System Administration Guide has not been updated for this release.
1.1 Library and Distribution changes
Updated Library Versions
The version of OpenSSL has been updated to 3.0.10. This is a major version update from 3.6.x.
The version of OpenLDAP has been updated to 2.6.4.
The version of libssh has been updated to 0.10.5.
The version of Kerberos has been updated to 1.20.1.
The version of lz4 has been updated to 1.9.4.
The version of prometheus has been updated to 1.0.1.
The version of aws-sdk-cpp has been updated to 1.9.362.
RPM distributions available
RPM (Red Hat Package Manager) distributions are now available.
RPM installations create a shared read-only distribution, in which it is not appropriate to use the $GEMSTONE/data directory for configuration files, extents, transaction logs, and other repository-specific files. Likewise, using the default settings in $GEMSTONE/data/system.conf is not appropriate.
When using an RPM installation, you must explicitly specify a configuration file location, and the configuration file must specify an application-specific directory in which the extents and transaction logs will be located.
Two features have been added to improve support basic GemStone RPM installations:
- The new -E configFile option has been added to many of the utilities; this is similar to -e, but with a simplified search path. In addition, an environment variable $GEMSTONE_DATADIR is defined as the directory in which the -E configuration file is located.
- The distribution now includes an additional template configuration file, $GEMSTONE/bin/gemstone_data.conf. This template defines the default extent and transaction log locations using $GEMSTONE_DATADIR, rather than $GEMSTONE.
RPM Installation process
After installing GemStone via RPM, a simple GemStone environment can be setup by:
- Create a directory for all GemStone files, gemstoneApplicationDir.
- Copy $GEMSTONE/bin/extent0.dbf and $GEMSTONE/bin/gemstone_data.conf to this directory, and give these files write permission.
- Execute startstone, including -E gemstoneApplicationDir/gemstone_data.conf.
See the new Installation Guide for Linux RPM for more detail.
clientLibs distributions
Along with the GemStone distribution, GemStone now includes a clientlibs directory tree. This provides a way to flexibly handling multiple versions of client library files. The structure of this directory tree is:
/clientlibs
versionNum
64bit
specific library files
32bit
specific library files
When a new version of the server is released, the newVersionNum tree from that release can be added to the clientlibs directory without perturbing other uses.
$GEMSTONE/projects moved
Earlier 3.6.x releases included the projects directory under $GEMSTONE/upgrade/. This directory includes rowan-format source code for projects such as superDoit.
This directory has been moved up and is now a base directory, $GEMSTONE/projects/.
Further changes in permissions within the distribution
The GemStone distribution included subdirectories and files with incorrect permissions; primarily non-executable files with execute permission; and some cases of subdirectories with owner write. These permissions have been corrected.
$GEMSTONE/data and $GEMSTONE/ualib continue to have owner and group write permission.
1.2 OS Configuration and Installation Changes
Using setcap for large pages and OOM protection on Linux
In previous releases, the Installation Guide has specified operating system configuration steps to enable use of large memory pages and for protecting processes from the OOM killer, using setcap to give the executables specific Linux capabilities. This is not optimal for security, and has a side effect of disallowing stack traces, since gdb did not have privileges to attach to the process, and prevented statmonitor from reading certain memory statistics. (#50580)
Alternate ways to configure large memory pages and OOM protection have been determined, and the way GemStone uses capabilities has been modified for v3.7. The previously existing techniques work, but for security and usability reasons are no longer recommended.
Large Memory Pages
Previously, it was recommended to give the Shared Page Cache Monitor the cap_ipc_lock capability.
Now, you can avoid giving this capability, by instead selecting and configuring a huge pages group. This is the procedure:
Select or create a group that includes the Linux user that will be starting up the shared page cache, and all users who will be accessing the shared page cache.
Then create the file /etc/sysctl.d/64-gemstone-local.conf, to set this group’s numeric id to the huge pages group. Add a line to the file:
vm.hugetlb_shm_group = numericGidOfGroup
You can determine the gid of a group from the name using:
os$ getent group groupName
This will take effect on reboot; you can apply immediately by executing:
os$ sysctl --system
To verify the current value, cat proc/sys/vm/hugetlb_shm_group, which will show the group id. To enable large pages transiently without rebooting, you can set the gid of the group directly in /proc/sys/vm/hugetlb_shm_group. This will be reset on reboot if not configured as by /etc/sysctl.d/64-gemstone-local.conf.
For full details, see the updated instructions in the Installation Guide for v3.7.
OOM Killer protection
Previously, a number of executables were given the cap_sys_resource capability, to allow them to modify their oom_score_adjust; critical processes reduced their scores, and less critical processes, such as gems, increased their score.
Now, only the Stone itself, and the pgsvrmain for remote configurations, need the capability. The only setcap executions needed are:
os$ setcap cap_sys_resource=pe $GEMSTONE/sys/stoned
os$ setcap cap_sys_resource=pe $GEMSTONE/sys/pgsvrmain
On startup, these processes use this capability to set their oom_score_adjust, then release the capability. Other critical processes are spawned by the Stone and inherit the Stone’s oom_score_adjust, or for remote caches, are spawned by the pgsvrmain and likewise inherit oom_score_adjust. The spawned processes that inherit an oom_score_adjust do not report the protection in their log files. The scores for Gems can be safely left at the default.
The GemStone code changes to release the capability, avoid the previous issues with gdb and statmonitor, including for processes that have explicit setcap.
Example scripts for setting up GemStone in systemd on Linux
Under $GEMSTONE/examples/admin/, the following directory and files have been added:
/systemd
netldi.service
gemstone.env
netldi.env
gemstone.service
These scripts include the details needed to set up the Stone and NetLDI to run as services managed by systemd.
Note that, whether using systemd or not to run GemStone, /etc/systemd/logind.conf should include:
RemoveIPC=no
to avoid Stone shutdown when the ssh or gui session that started the Stone logs out, and the Stone’s shared semaphore array is deleted.
Raw partitions on Character devices no longer supported
The RAW driver for direct I/O has been deprecated for many years (superseded by the O_DIRECT flag for a block device). With Linux kernel 5.14, the RAW driver was dropped; this or later versions are used by Ubuntu 22.04 and RedHat 9.x.
With v3.7, raw partitions on character devices are no longer supported for GemStone dbf files, on older as well as recent kernels. v3.7 supports raw partitions on block devices.
Reverse DNS lookups required for Single Sign-On (SSO)/Kerberos logins
In order to use SSO logins using Kerberos, it was, and is, required that you have reverse DNS lookup setup and operating correctly; see the informational bugnote for #50507.
The error reporting for this case has been improved, and SSO tracing enabled; see Debug tracing for SSL, SSO, SSH now allowed in fast builds.
1.3 Process and Debugger Optimizations
New native code generation and compiler optimizations
The native code generator has been extensively improved for performance, and the compiler has been further optimized.
Some operations, such as integer arithmetic, have 2x-5x performance improvements, overall a 10% improvement can be expected.
Note that operations that are not CPU-bound, i.e. that are I/O bound waiting on disk reads or writes, will not observe performance improvements.
Native code not reimplemented on AIX, Mac/Linux on ARM
In this release, AIX, Linux on ARM, and Darwin on Apple silicon (ARM) run in interpreted mode only.
Older CPU machines cannot run v3.7
GemStone has been compiled using optimizations that require CPU features that are not available on older CPUs (more than about 10 years old).
- Intel CPUs with architecture Sandy Bridge or newer (excluding Atom CPUs)
- AMD CPUs with architecture Bulldozer version 1 or newer
v3.7 cannot run on older CPUs. Attempting to run any executable will result in an error such as:
[Error]: Program cannot start because the CPU is missing the following feature(s):
[Error]: avx pclmul
[Error]: For more information, please contact GemTalk technical support.
Conditionally Optimized Selectors
The selectors isNil, notNil, and yourself are now optimized by default. The optimization is configurable, and can be removed.
The new method GsNMethod class >> configurableOptimizedSelectors returns the optimized selectors.
To remove the optimization, as SystemUser you must reset GsNMethods’s class variable OptimizedSelectors to an Array containing the selectors that will be optimized (minus any you do not want optimized). Symbols other than the three specified will be ignored. You may use an expression such as GsNMethod _classVars at: #OptimizedSelectors put: anArray.
To update the bytecodes in compiled methods, run upgradeImage or otherwise recompile all GemStone kernel methods, and recompile all application methods.
Process Scheduling and Debugger Support
GemStone/S 64 Bit v3.7 includes additional infrastructure and changes in process scheduling and management, breakpoints, and stepping from v3.6.x. These differences includes both additional methods, and behavior changes in specific methods.
Scheduler now preemptive
The scheduling of a higher priority process previously required an explicit yield or wait in GemStone; now, GemStone does preemptive scheduling, similar to the behavior in Pharo and other Smalltalk dialects.
For example ExecBlock>>forkAt: no longer requires a subsequent yield, to allow a high priority GsProcess to start running. GsProcess >> resume and GsProcess >> priority: will automatically yield, if the receiver is or becomes higher priority than the current process.
Code with coordinating multiple processes may see differences in behavior in this version of GemStone.
Breakpoint Handling and Debugger Support
The GsProcess, Behavior and GsNMethod API that supports breakpoints and debugging has been redesigned, to improve and facilitate debugger implementation. Additional exception codes have been added to distinguish signaling to the GCI vs. signaling to Smalltalk. These changes are likely to affect behavior in corner cases, but should be generally transparent to end users. If you have implemented a debugger or made extensive customizations, contact GemTalk Engineering for more information.
Interrupting sockets
When a socket is waiting on activity (such as read or accept) in one GsProcess, while another GsProcess is running, previously the socket that is waiting could not get scheduled when the wait was complete and it was ready for further execution.
Now, a socket can be configured as interrupting. The following methods have been added:
GsSocket >> interrupting
GsSocket >> interrupting: aBoolean
After executing aSocket interrupting: true, then a GsProcess waiting for that socket to be ready for read or write in an instance method of GsSocket or a subclass of GsSocket will be scheduled to run when that socket is ready.
Process and debugging bugs fixed
A number of bugs have been fixed in the underlying code. These issues may or may not have been in previous releases.
- Step over in recursive method stopped in a frame of the same selector, which may be incorrect (#49539)
- Process terminate may be ignored if unwind block continuously runs. (#49464)
- GsProcess >> suspend did not suspend a process waiting for a delay or on a semaphore (#49526)
- GsProcess >> resume resumed a process that was waiting on a delay but not suspended (#49802)
- Terminated GsProcess does not allow ensure block in progress to complete, risking stuck semaphore (#48271)
- After AlmostOutOfStack, attempting to execute further encountered error clearing stack (#47373)
- GsProcess terminate on a process waiting on a socket without a timeout was not terminated correctly. (#47196)
Handling asynchronous events in a designated process
In a multi-process application, you may now configure a specific process to receive and handle these signals. The following methods have been added. For an example of how these can be used, see GsSignalingSocket class >> _readNotificationExample.
GsSignalingSocket class >> newForAsyncExceptions: anArray
Returns a kind of GsSignalingSocket which has been registered to receive data representing asynchronous Notifications. The socket returned represents one end of an underlying UNIX domain socket pair, the other end of that pair is written to by the C thread that is the source of the Notification.
anArray must contain one or more of the classes InterSessionSignal, ObjectsCommittedNotification, TransactionBacklog, and GcFinalizeNotification, to specify the type of Notification to receive. Use GsSignalingSocket >> readNotification to receive the Notifications.
Do not use InterSessionSignal class >> enableSignalling, ObjectsCommittedNotification class >> enableSignalling, or TransactionBacklog class >> enableSignalling in conjunction with this method.
Only one socket can be registered in a session to receive asynchronous Notifications. GsSignalingSocket class >> disableAsyncExceptions must be used to cancel a previous registration before the next execution of newForAsyncExceptions:.
GsSignalingSocket >> readNotification
Read a notification from the receiver, which must be the result of executing GsSignalingSocket class >> newForAsyncExceptions:.
GsSignalingSocket class >> disableAsyncExceptions
Cancel a registration for receiving asynchronous notifications.
Added and renamed Process and related methods
GsProcess "Step" methods renamed
The methods GsProcess >> step*FromLevel:* have been renamed to more accurately describe their function; the new names are setStep*BreaksAtLevel:*. The earlier methods are still present and usable, but deprecated.
Signaling the GCI
The following methods have been added:
AbstractException >> signalToGci
Receiver is signaled and it will not be trappable by any exception handler. An Error will be returned to the GCI.
AbstractException class >> signalToGci
An instance of the concrete subclass is created and signaled; not be trappable by any exception handler. An Error will be returned to the GCI.
Other added methods
GsProcess >> abandonUnwindAndTrimStackToLevel:
GsProcess >> breakpointLevel
GsProcess >> breakpointLevel:
GsProcess >> terminateTries:eachTimeoutMs:
GsProcess >> trimStackToLevel:
GsProcess >> unsafeTerminate
GsProcess >> unsafeTrimStackToLevel:
GsProcess class >> current
GsMethod >> setBreakAtStepPoint:breakpointLevel:
ProcessorScheduler class >> highestPriority
ProcessorScheduler class >> lowestPriority
Removed Methods
The following related methods have been removed:
Breakpoint class >> trappable:
ExecBlock >> valueNowOrOnUnwindDo:
Other removed methods are listed under Deprecated and Removed Classes and Methods.