GemStone/S 64 Bit 3.3.1 is a new version of the GemStone/S 64 Bit object server. This release adds a number of new features, and fixes a number of bugs in v3.3; we recommend everyone using or planning to use GemStone/S 64 Bit upgrade to this new version.
These release notes describe changes between the previous version of GemStone/S 64 Bit, version 3.3, and version 3.3.1. If you are upgrading from a version prior to 3.3, review the release notes for each intermediate release to see the full set of changes. In particular, if you are upgrading from version 2.4.x, note that there were substantial changes in v3.0 that impact your application.
For details about installing GemStone/S 64 Bit 3.3.1 or upgrading from earlier versions of GemStone/S 64 Bit, see the GemStone/S 64 Bit Installation Guide for v3.3.1 for your platform.
Upgrade is supported from GemStone/S 64 Bit v2.4.x, 3.1.x, 3.2.x. and 3.3. For details on the conversion and upgrade, see the GemStone/S 64 Bit Installation Guide for v3.3.1.
GemStone/S 64 Bit v3.3 included an updated version of the ICU libraries that support unicode string collation and Unicode Comparison Mode. If you are upgrading from a version 3.2.x, you may continue to use the older ICU library version; an additional step is required when upgrading to set your application to use the legacy ICU version.
The underlying issue and the changes in v3.3.1 are described below, under ICU version change caused problems in upgrade to 3.3 and Handling of ICU shared libraries. The updated Installation Guides for v3.3.1 provides the specific steps that are required as part of upgrade.
GemStone/S 64 Bit version 3.3.1 is supported on the following platforms:
For more information and detailed requirements for each supported platforms, please refer to the GemStone/S 64 Bit v3.3.1 Installation Guide for that platform.
GemStone/S 64 Bit version 3.3.1 requires GBS version 8.1 or later for VisualWorks Smalltalk, or version 5.4.3 or later for VA Smalltalk. Older versions of GBS cannot login to v3.3.1.
The following versions of GBS are supported with GemStone/S 64 Bit version 3.3.1:
The GemStone/S 64 Bit v3.3.1 distribution includes VSD version 5.2. The previous version of GemStone/S 64 Bit, v3.3, included VSD v5.1.3.
VSD v5.2 includes many changes and bug fixes, especially in support for working with multiple files. For a description of the changes, see the Release Notes for VSD v5.2.
This listing includes enhancements between v3.3 and v3.3.1, and includes changes also introduced in versions 3.2.13, 3.2.14, or 3.2.15.
The version of OpenSSL used by GemStone/S 64 Bit v3.3.1 has been updated to 1.0.2h.
For some time, GemStone distributions on certain platforms have included a limited starter keyfile, intended primarily for use with Seaside/GLASS/GsDevKit applications and therefore located under the $GEMSTONE/seaside directory. These keyfiles are suitable for evaluation.
In v3.3.1, on Linux and Darwin, the GemStone distribution includes a starter keyfile under $GEMSTONE/sys, with the name community.starter.key. If there is no keyfile at the location $GEMSTONE/sys/gemstone.key, then the stone will look for a keyfile $GEMSTONE/sys/community.starter.key and use that for startup.
If the configuration parameter KEYFILE refers to an invalid file, or a file other than $GEMSTONE/sys/gemstone.key that does not exist, then the stone will not use the starter keyfile and will fail to start.
Note that the starter keyfile does not have permissions for GBS, GemConnect, or GBJ; if you set up a new installation and do not set your license keyfile, the stone will now start rather than failing with an error. Operations such as attempting to login from GBS, for example, will fail.
The keyfile that is distributed under the seaside directory continues to be included in the distribution in the old location on Linux and Macintosh, in addition to the new community keyfile.
New versions of ICU are released periodically to support changes in the official Unicode standard. For example, the ruble symbol (Character codePoint: 8381) is a new addition in Unicode 7.0, and was added to ICU in v53. The ICU library version included with GemStone/S 64 Bit v3.3 was ICU v54.1, while version 3.2.x included version 51.2. This created problems with SortedCollections and Indexes that involved ICU-based comparisons; see the description here for details.
To ensure correct sort ordering and query results, the upgrade process and internal configurations have been changed so that repositories will continue to use the same ICU library version over upgrade. To avoid inadvertent changes, it is now disallowed to login with an ICU library version other than the one expected by the repository.
The ICU version varies between the supported originating repositories for upgrade to 3.3.1, and 2.x and 3.0.x did not have the ICU feature. The upgrade scripts and Installation Guide instructions have been updated for each upgrade path.
The result of following these instructions for each version is as follows:
GsDevKit_home systems that are upgraded using the upgradeStone script do not have to take any further action; this script performs all necessary steps on all supported originating versions.
Both 51.2 and 54.1 ICU libraries are included in the distribution, and the library interfaces are designed so that Gems in version 3.2.14 or later 3.2.x versions, or version 3.3.1, may load either version.
A class variable and access method has been added to IcuCollator to request the version of the ICU libraries that are currently loaded:
IcuCollator class >> libraryVersion
A new entry has been added to Globals to store the version of the ICU libraries that the repository should be using:
Globals at: #IcuLibraryVersion
This is a string that is set to '54.1' or '51.2' on upgrade, depending on what version you are upgrading from.
The new configuration parameter STN_GEM_LIBICU_VERSION has been added, and is used by the Gem during login to determine which version of libicu*.so should be loaded. This configuration parameter is not usable on AIX due to the way libraries are loaded, and you must use the following environment variable.
The default setting for this is 54.1. An additional manual step is required on upgrade from 3.2.x to set this configuration parameter to 51.2 in the Stone’s configuration file.
The new GS_LIB_ICU_VERSION environment variable can override the setting for the configuration parameter. This may be set to either ’51.2’ or ’54.1’.
During login, if the loaded ICU library does not match the repository’s configuration, if
IcuCollator libraryVersion ~= (Globals at: #IcuLibraryVersion)), then an error is signalled and further commits are disallowed by this session. This avoids any risk of committing incorrectly sorted results.
The version test on login can be bypassed for upgrades, if the user is SystemUser and the environment variable GS_LIB_ICU_BYPASS is defined.
After subsequent upgrades, the repository will automatically continue to use the same ICU version; you must manually upgrade to use a later ICU version.
A string specifying the version of the libicu* libraries that a gem or topaz -l should load during login. This can be overridden by the GS_LIB_ICU_VERSION environment variable in the environment of a gem (i.e. environment of the startnetldi) or the environment of a topaz -l process.
The method SortedCollection class >> resortInstancesFromFilesForGem: gemNum of: totalGems has been added. This method reads the files in $upgradeLogDir/scbm that contain OOPs of instances of SortedCollection, and performs a resort.
Code has been added to IndexManager to support rebuild of indexes that rely on ICU library collation.
This code is designed for use in future improved index management during upgrade and ICU version upgrade.
When application code recompile is required, as with the 3.2.x to 3.3.x upgrade, sortBlocks must also be recompiled; see bug here. postconv is now a required step following the 3.2.x to 3.3.x upgrade, and support for this has been improved in v3.3.1.
The following methods have been added:
needsRecompile
Returns true if the receiver's block needs recompilation, false otherwise.
recompile
Attempts to recompile the receiver's method, assuming it is a simple block without references to self or method variables.
These methods test if block recompile is needed, and perform the simple block recompiles. They are invoked by postconv after a GemStone server upgrade that requires code recompile.
Complex blocks that have references to self or method variables cannot be automatically compiled, since the referents are in other methods. Handling such blocks requires individual attention; you may have to manually construct code that recreates the block.
The sortWithBlock:* methods now provide variants with an added collator: argument. This allows you to pass in the IcuCollator as a third argument in a sort block. This allows IcuSortedCollection sortBlocks to be simple blocks, which can be automatically recompiled.
The class SortBlockUnicodeNode has been added to support this functionality. This is a subclass of SortBlockNode, used by BlockSorter to implement the sort used by sortWithBlock:*.
ByteArray >> replaceFrom:to:with:startingAt:
String >> replaceFrom:to:with:startingAt:
now permit the with: argument to be a CByteArray. Note that the startingAt: argument is a 1-based offset into the otherwise 0-based instance of CByteArray.
The read limit for the method GsSocket >> read:into:startingAt: was previously 10000000 (10MB), now it is 500000000 (500MB)
When a GCI protocol error occurs, GemStone will now print the recent GCI calls and associated packets to allow diagnosis.
The way CCallout handles the C global errno has been changed in this release.
Now, the errno resulting from a C-level call via CCallout is cached in VM memory. Each time that CCallout >> callWith: is invoked, errno is set to the cached value before the function, and the cached value is set to errno after the function executes. This preserves the value against other GemStone internal calls.
CCallout has added class methods to access the cached value:
CCallout class >> errno
Returns the SmallInteger value of the C global variable errno that was saved by the most recent invocation of CCallout >> callWith:.
CCallout class >> errno: aSmallInteger
If aSmallInteger == -1, Returns the SmallInteger value of the cached errno value, ffiErrno. If aSmallInteger >= 0, stores aSmallInteger into the session's cache.
GemStone automatically sets the limits on memory use using setrlimit().
Now, this can be disabled using the environment variable GEMSTONE_NO_SETRLIMIT, for customers who want to configure memory limits manually.
Previously, up to 100 files could be specified; now, 1000 are allowed.
The following option has been added to startnetldi:
-N Use numeric IP addresses for printing peer info in logs, do not do reverse DNS lookups to get hostnames of clients.
The output of the printlogs utility is used for debugging and for analysis of repository transaction. When printing page information, the output can include data incidental to the actual issue being examined, in the form of strings that may include sensitive information.
Now, the added nostrings argument to printlogs suppresses printing the contents of all kinds of strings and similar byte collections, including ByteArrays.
The following statistic has been added:
AlmostOutOfMemoryCount (Gem)
AlmostOutOfMemoryCount is the number of times that either an AlmostOutOfMemory notification was signalled to the application, or smalltalk stack was printed to the gem log because temporary object memory was approaching full.
The following functions have been added:
GciPerformFetchBytes is similar to GciPerform, with the addition of maxResultSize, the maximum number of bytes to be returned in *result . The function returns the number of bytes returned in *result, or -1 if an error is available to be fetched with GciErr.
(int64) GciPerformFetchBytes(
OopType receiver,
const char *selector,
const OopType *args,
int numArgs,
ByteType *result,
ssize_t maxResultSize
);
The following bugs in 3.3 are fixed in v3.3.1. Note that this list includes bugs that are also fixed in 3.2.13, 3.2.14, and 3.2.15.
In an internal C function, a variable was declared as uint32 while a uint64 is needed. This may have caused failures or infinite loops in operations such as markForCollection, object audit, and reclaim. (#46252)
gslist looks at GemStone lock (..LCK) files to determine what GemStone server processes (Stones, NetLDIs, etc.) are running. The lock files include information about the server process, including the PID of the process.
In earlier versions, gslist only verified that the PID was running, not that the process using that PID was the same GemStone server process. If a process was killed, leaving the lock file behind, and that PID was reused, gslist assumed the PID was valid. The server name/lock file were stuck, and the lock file had to be manually deleted.
Now, on Linux only, lock files involving reused PIDs are handled correctly, by checking in /proc/PID/.
On platforms other than Linux, you must still manually delete the lock file.
Each host with a GemStone server node finds or creates a file /opt/gemstone/locks/gemstone.hostid to identify the host. If this file exists, but is not readable by a process such as linked topaz, the process will hang. (#46219)
If a reverse IP lookup returned an EAI_AGAIN ("The name could not be resolved at this time. Try again later.", the NetLDI retried immediately until a valid response was returned. On systems that persistently return this error, it resulted in the NetLDI hanging. (#46156)
An option has been added to startnetldi to skip the reverse lookup and print the numeric IP address; see startnetldi added option to skip reverse DNS lookups.
The environment variable STN_LOGIN_LOG_DIR, added in version 3.2.2, did not permit environment variables such as $GEMSTONE. (#46028)
A login from a client on Mac to a server on another machine failed if there was no /opt/gemstone/locks directory on the client. (#46181)
The ICU library version was updated for v3.3. This created issues with indexes and SortedCollections that used the ICU libraries for collation. (#46056).
The way ICU versioning is handle is changed substantially in this release, see Handling of ICU shared libraries.
After upgrade to v3.3, all code requires recompile. This includes code in SortedCollection sortBlocks and other persistent blocks. The upgrade instructions did not include this requirement. (#46055)
The Installation Guides have been updated for v3.3.1, and the code that supports postConv now recompiles simple sortBlocks from 3.2.x.
When upgrading from v2.x to 3.x, the setting for epochGcPageBufferSize was not being converted from a previous setting to a power of two. (#46009)
After additions of GLASS/Seaside/GsDevKit classes GsFailedMethodCompilationDuringCopyMethodToNewClass, and NumberParserGsDevKitIssue75Test, the upgradeSeasideImage step failed. (#46059, #46217)
If a problem was detected while a user action was being loaded, the user action table was left in an invalid state. This may have caused the Gem to crash when the system attempted to execute a user action . (#46131).
If a application class’s class variable referenced an instance of the class, but there are no other references to the class or any instance of the class, and the size of the instance is 0, the class may have been immune from garbage collection. (#46116)
There was an optimization in the multithreaded MFC that performed a recursion; it was possible for excessive recursion to occur which resulted in running out of Cstack space. This optimization was determined to provide very little value and has been removed. (#46194)
When the stone starts up and has to recover due to an unclean shutdown, and there were pages to reclaim, then the Reclaim Gem started up in transaction. This could cause a commit record backlog until recovery completed, or Reclaim Gems’ maxTransactionDuration was reached. (#46005)
If the Admin Gem is not running following an operation that holds the GC lock, a request for a markForCollection cannot get the gcLock and fails to start with an error. The error did not indicate the cause of the problem; now, the error message includes a warning that the Admin Gem is not running. (#46047)
If an incomplete transaction log was replayed, and then later the same transaction log was replayed after it had completed (and therefore contained more records), it was possible for some records in the later part of the transaction log to not be replayed. This resulted in the restored system being corrupted. (#46148)
The code for GsQuery reject: included a bug and returned the wrong results. (#46184)
Rather than returning the results of the ifNone: block, on a negative result it returned the query. (#46184)
There was a code path in which a remote shared page cache could have deadlocked on a stuck spin lock. (#46175)
If a remote cache fails to complete startup within the time period of STN_REMOTE_CACHE_STARTUP_TIMEOUT, the stone starts shutting down and cleaning up after the startup attempt. There were timing cases in which the remote cache could complete the startup and shut itself down, leaving the Stone status incorrect. (#46020)
When a GsFile was open for update (that is, for both reading and writing), in certain cases if a read operation is followed by a write, the text to be written was appended to the end, not at the desired location. (#46120)
If a read performed on a GsSecureSocket did not read all available characters, a subsequent call to GsSecureSocket>>readWillNotBlockWithin: incorrectly returned false. (#46174)
Operations on hidden sets, such as System>>hiddenSetEnumerate:limit:, against very large bitmaps could encounter a logic error that filled the ME space part of temporary object memory. In-memory garbage collection was not triggered when ME Space was full but other spaces did not need GC. This resulted in an error "VM temporary object memory is full , ME-space overflow" errors (4067), which could not be trapped using the AlmostOutOfMemory exception. (#46054, #46058)
When a KeyValueDictionary become: occurs, any CollisionBuckets within the KeyValueDictionary should have their parent reset to the new containing dictionary. This step was not being done. Operations on collision buckets do not normally invoke the parent, so no consequences of this bug are known. (#46194)
If the classInstanceVariable: argument to a class creation method is invalid, an error is returned. Due to the way class instance variables are added after class creation, the class was still created, but was not made invariant. (#46166)
Several private subclass creation methods have been modified to include a classInstanceVariable: argument, and the private methods without this argument have been removed.