1. GemStone/S 64 Bit 3.4.4 Release Notes

Overview

GemStone/S 64 Bitâ„¢ 3.4.4 is a maintenance release of the GemStone/S 64 Bit object server. This release includes several feature enhancements and fixes a number of bugs.

We recommend everyone using GemStone/S 64 Bit upgrade to the latest version, currently v3.5. The v3.4.4 release provides bug fixes and enhancements to customers who are not yet able to upgrade to v3.5.

These release notes describe changes between the previous version of GemStone/S 64 Bit, version 3.4.3, and version 3.4.4. If you are upgrading from a version prior to 3.4.3, review the release notes for each intermediate release to see the full set of changes.

The Installation Guide has not been updated for this release. For installation, upgrade and conversion instructions, use the Installation Guide for version 3.4.

Supported Platforms

Platforms for Version 3.4.4

GemStone/S 64 Bit version 3.4.4 is supported on the following platforms:

  • Red Hat Enterprise Linux Server 6.9 and 7.4, and
    Ubuntu 16.04 and 18.04, and SUSE Linux Enterprise 12, on x86.
    Testing has been done on some earlier Red Hat Enterprise Linux Server 6.x and 7.x versions with the latest security patches.
  • Solaris 10 and 11.3 on x86
  • AIX 6.1, 7.1, and 7.2
  • OS X 10.13.2 (High Sierra), with Darwin 17.3.0 kernel, and
    OS X 10.14.3 (Mojave) with Darwin 18.2.0 kernel, on x86
    (Mac is supported for development only)

For more information and detailed requirements for each supported platforms, please refer to the GemStone/S 64 Bit v3.4 Installation Guide for that platform.

GemBuilder for Smalltalk (GBS) Versions

GemStone/S 64 Bit version 3.4.4 requires GBS version 8.3 or later for VisualWorks Smalltalk, or version 5.4.4 or later for VA Smalltalk.

The following versions of GBS are supported with GemStone/S 64 Bit version 3.4.4:

GBS version 8.4

VisualWorks
8.3.2

32-bit and 64-bit

VisualWorks
7.10.1

32-bit

VisualWorks
7.10.1

64-bit

  • Windows 10 and Windows 7
  • RedHat ES 7.4 and 6.9; Ubuntu 18.04 and 16.04
  • Windows 10 and Windows 7
  • RedHat ES 7.4 and 6.9; Ubuntu 16.04
  • Windows 10
  • RedHat ES 7.4 and 6.9
GBS version 8.3

VisualWorks
8.2.1

32-bit and 64-bit

VisualWorks
7.10.1

32-bit

VisualWorks
7.10.1

64-bit

  • Windows 10 and Windows 7
  • RedHat ES 7.4 and 6.9; Ubuntu 16.04
  • Windows 10 and Windows 7
  • RedHat ES 7.4 and 6.9; Ubuntu 16.04
  • Windows 10
  • RedHat ES 7.4 and 6.9
GBS version 5.4.5

VA Smalltalk
8.6.3

  • Windows 10
  • Windows 8.1, Professional or above
  • Windows 7, Professional or above
GBS version 5.4.4

VA Smalltalk
8.6.3

  • Windows 10
  • Windows 8.1, Professional or above
  • Windows 7, Professional or above

For more details on supported GBS and client Smalltalk platforms and requirements, see the GemBuilder for Smalltalk Installation Guide for that version of GBS.

VSD Version

The GemStone/S 64 Bit v3.4.4 distribution includes VSD version 5.4. This is unchanged from the previous version of GemStone/S, v3.4.3.

VSD versions are not tied to GemStone server versions: both older and newer versions of VSD can be used to read statmonitor files generated by both older and newer versions of GemStone/S and GemStone/S 64 Bit.

Changes in this release

Distribution changes

Updated library versions

  • The version of lz4 has been updated to v1.9.1.
  • The version of OpenSSL has been updated to 1.1.1c.
  • The version of OpenLDAP has been updated to 2.4.47.

Updated ZoneInfo (TimeZone database)

The zoneinfo TimeZone database has been updated from version 2017b to 2019a. As part of this update, the files within the timezone directory tree have been reorganized to adhere more closely to standard directory naming conventions.

Raspberry Pi

An initial version of support for GemStone clients running on Raspberry Pi is now provided. The Raspberry Pi runs Linux distribution "Raspian", which is a Debian Linux variant.

Raspberry Pi is a client-only distribution with the name GemStone64BitClient3.4.4-arm.Linux. No specific Installation Guide has been created; the process is similar to a client-only installation on Linux.

Clients Smalltalks have been verified:

  • GBS/VA v5.4.5 with VA Smalltalk v9.2 early access (ECAP). Raspberry Pi is provided as a preview feature in this unreleased distribution of VA Smalltalk.
  • GBS/VW 8.4 with VisualWorks Smalltalk v8.3.2 Raspberry Pi is provided as unsupported platform in this release.

These configurations are provided for experimentation and are not certified nor fully supported. They should be tested carefully for suitability and should not be used for critical systems.

gslist no longer distributed on Windows

The gslist utility was previously included in the distribution on Windows, but was of minor value since no GemStone server processes can run on Windows. It is no longer included in the Windows distribution.

Enhancements and bug fixes in Backup and Restore

Compressed backups using lz4 have improved compression

Lz4 backups could be significant larger than the gzip version. The default lz4 file compression level has been changed from 0 to 3, reducing the resulting compressed backup size differential.

The restore of a fullbackup now reduces the OOP high water mark

When a programmatic backup of a GemStone database is made, it now records the actual highest value of an OOP in the repository. This is used on restore of the backup, and may mean the size and number of pages required by the object table is reduced. (#48094).

Reduced space used by object table leaf pages after restore

During restore of backup, leaf pages may be created in the object table that do not end up being needed. The allocation of these pages has changed so the pages are only allocated if needed. (#48093)

pageaudit additional statistics

pageaudit, in addition to reporting if there are any errors found in the page allocation with a repository, also reports statistics about the repository pages. This information is written to the pageaudit log file, tmpstnname-pageAudit.log.

The additional lines of information include:

OT Internal Pages                0.05 Mbytes          3 pages
OT Leaf Pages                    0.81 Mbytes         52 pages
Empty OT leaf pages              0.00 Mbytes          0 pages
Empty data pages                 0.00 Mbytes          0 pages
Data pages with 25%+ free        5.94 Mbytes        380 pages
Data pages with 50%+ free        4.05 Mbytes        259 pages
Free space in data pages         4.25 Mbytes     272.25 pages

This additional information allows you to verify the page distribution following a restore.

During restore, possible C heap growth and CR backlog

When restoring transaction logs, such as in a warm standby, the tranlog reader thread was reading ahead too far, resulting in excessive C heap growth. In addition, the commit record was not always released in a timely manner during tranlog restore, which lead to a commit record backlog. (#48214)

The statmonitor stat BmCHeapPages, that measures C heap usage, wrapped and was unreliable (#48216)

backup start time was not reported in stone log

The stone log normally contains the time a full programmatic backup was started and the time it completed. The start message was not being included. (#48100)

Converting from ByteArray containing UTF-8 to decoded string

The following methods have been added:

ByteArray >> decodeFromUTF8ToString
The contents are bytes in UTF-8 encoding; decode and return as a String, DoubleByteString or QuadByteString.

ByteArray >> decodeFromUTF8ToUnicode
The contents are bytes in UTF-8 encoding; decode and return as Unicode7, Unicode16 or Unicode32.

As in v3.4.3, Utf8 class >> withAll: no longer accepts a ByteArray argument.

Floating point exceptions on Ubuntu 18.04

In Linux’s libc v2.24 and later, the inexact exception has been removed from the arithmetic functions floor, ceiling, round and truncate. This impacts GemStone executing on the newly supported Ubuntu 18.04 distribution, which includes libc 2.27. Ubuntu 16.04, Red Hat and SUSE include older versions of libc.

If floating point exceptions are enabled, GemStone operations such as 9.75 ceiling do not signal any exception on Ubuntu 18.04, although they continue to signal the inexact exception on other platforms.

Note that in GemStone/S 64 Bit v3.5 and later, ceiling and similar operations do not signal inexact on any platform.

Statmonitor new feature and bug fixes

Statmonitor automatic delete of older data files

You can now configure statmonitor with automatic new file creation to automatically delete older files. Using the -K numFiles flag to statmonitor, you can specify the number of older files to save (not including the file currently being updated). You may only use -K with either -r or -R.

Statmonitor automatic start failed with -X

Starting up a statmonitor -X using configuration parameters such as STN_STATMONITOR_ARGS did not work, as these methods automatically include the stone or cache name as an argument. The -X option provides host-only monitoring and no stone or cache name is permitted as an argument. The -X option is now handled, but note that there are limitations in the log file names permitted within the -F argument in this mode. (#47885)

Automatic start of statmonitor did not occur during pageaudit

Setting a value for STN_STATMONITOR_ARGS means that statmonitor should start automatically when the stone runs; this was not occurring during pageaudit. (#47674)

gemnetobject supports -U argument

The -U argument has been specifically added to allow passing in arbitrary strings; this ensures that future changes to the gemnetobject arguments will continue to support -U.

In addition to the existing gemnetobject arguments (-T tocSizeKB, -N nativeCodeArg, -e exeConfPath, -C configParamsString, and -U ignoredArg), you may include additional arguments, provided no leading dash is included, at the end or within the known arguments. Unrecognized arguments and tokens are passed along to the gem and ignored. Note that minimal error checking is done; you should verify that your arguments were entered correctly by examining the gem log file.

Temporary files on Linux now created in /tmp, not /usr/tmp

Recent linux distributions do not define /usr/tmp/, as the standard location for temporary files is now /tmp/. GemStone now creates temporary files in /tmp/.

Changes and bug fixes relating to use of GCI

Linker symbols now weak on Linux

Support for GCI application on Ubuntu 18.04 required some changes in the way GemStone shared libraries are built. The shared libraries now contain weak symbols, and the linker flags for linking a GCI application have changed.

Otherwise, this change should have no impact on customer applications.

Stack traces on Windows for fatal errors

On Windows, access violation errors (e.g. SEGV) from RPC GCI will now provide a C stack for subsequent debugging. All GCI calls will handle access violations this way. The stack is written to the GCI trace file, if that is active, or to a file with the name GCI*.tmp in the windows temporary directory,

c:\users\userid\AppData\Local\Temp\

Note that the .pdb files from the GemStone Windows Client distribution must be present in order to capture a stack trace.

GciErrSType class >> new incorrect malloc size

The implementation was configured to malloc 2163 bytes, which did not allow for the round-up to 32-bit word boundaries, and should have been 2168. (#47304)

SIGTERM handler not chained to installed handler in linked GCI

In the libgcilnk library, but not in the libgcirpc library, a SIGTERM handler is not chained to a previously installed handler. (#47862)

Reported bytes sent/received incorrect

Initial value incorrect

When setting up logging using GciDbgEstablishToFile(), the bytes sent and bytes received were not initialized. This results in, for example, GBS call tracing to report a large number of bytes sent and received by the first server call. (#47996)

Printing incorrect on 32-bit Windows

With 32-bit clients on Windows, integer printing may incorrectly print 32-bit values as if they were 64 bit. (#47997)

Changes in Cache Statistics

Removed statistics

The following statistic has been removed:

ServiceMs

Added statistics

The following statistics have been added:

ReadWaitMs (HardDisk)
The average number of milliseconds to service a read I/O, including both time spent in the I/O queue and time performing the read.

WriteWaitMs (HardDisk)
The average number of milliseconds to service a write I/O, including both time spent in the I/O queue and time performing the write.

Bug Fixes

The following bugs that were present in v3.4.3 are fixed in this version.

libgcirpc shared library load performance

Due to changes in the way openSSL libraries were located, there were cases, such as when loading libraries over NFS on heavily loaded system, in which the time to load the libraries was much slower. (#48171)

Remote cache related issues

Failure to recover after loss of pageserver for a remote cache

When the pageserver on the Stone's note that services a remote cache dies, the remote cache should shutdown and a subsequent login on that remote node should trigger the restart of the remote cache. However, login attempts during the period could prevent the remote cache from shutting down to complete the recovery. (#48166)

Extent names not updated on remote cache

When a new extent was created at runtime, the cached list of extent names was not updated in a remote cache unless a new gem logged into the remote cache. (#48183)

Symbol Garbage Collection not working reliably

The process that implemented the garbage collection of unreferenced Symbols could commit interior nodes of the SymbolDictionary structure such that many Symbols were not actually garbage collected. (#47891)

User Action SEGV or Access Violation

If the OOP of the name of a user action is larger than 549755813633, and this OOP has bit 0x8000000000 set, errors in the code produce a negative hash value, which causes a SEGV on Unix and an Access violation error on Windows. (#48126)

startstone problem with checkpoint-related records in different tranlogs

It is possible for the precheckpoint and checkpoint records to be written to two different tranlogs. In this case, startstone may fail in spite of a clean shutdown. (#48169)

Issues related to configuration parameters

STN_COMMITS_ASYNC risky and use has been disabled

Testing encountered a case in which with this configuration parameter set to true, a record was missing from the transaction log during restore after the Stone ran out of tranlog disk space. (#48098) To avoid risk, this parameter has been disabled in v3.4.4.

SHR_PAGE_CACHE_PERMISSIONS printed without leading 0

The value for SHR_PAGE_CACHE_PERMISSIONS was printed in log files without the leading ’0’, which caused it to be interpreted as decimal. Now, the logs show the usable value that includes the leading 0. (#48157)

Errors in STN_GEM_INITIAL_TRANSACTION_MODE caused log bloat

The error reported when there was an parsing error in the value for the configuration parameter STN_GEM_INITIAL_TRANSACTION_MODE was not handled correctly and caused startup to fail and excess printing in the stone log.

STN_FREE_SPACE_THRESHOLD value greater than 4GB was truncated

If the setting for STN_FREE_SPACE_THRESHOLD is specified to be greater than 232-1 (4294967295), the value that is actually set is the value truncated to the low-order 32 bits of the number, resulting in a value no greater than 65K (#48211).

GsExternalSession Issues

GsExternalSession failed with gemNRS: string argument

GsExternalSession login parameters can accept an instance of GsNetworkResourceString, or a String containing NRS as an argument. The use of strings for the gemNRS: parameter was broken. (#47868)

Logout problems in certain configurations

The tests to determine if Gem was correctly logged out did not take into account the possible existence of a stone with the same name running on the Gem host but not used to login the External session; nor a Gem running on a remote node. While the logout proceeded correctly, the return would not complete and the logout would appear to hang. (#47663)

System >> hostCpuUsage results were not always reliable

On the first use, or if two calls are performed too closely together, the method System >> hostCpuUsage returned invalid results, either arrays of zeros or zeros with the second value of 100. (#48220).

The code supporting this method also did not round predictably; the percentages of CPU did not always add up to 100. (#48232)

topaz errors on input of a file with BOM

Files may include an initial byte order mark (BOM) character, which is metadata indicating file encoding. Previously topaz would error when this character was encountered; now, provided topaz as set fileformat utf8, a UTF-8 BOM character is ignored at the beginning of a file. It is an error to input files with a UTF-16 or UTF-32 BOM character. (#47665)

Hot Standby issues

Logreceiver race condition could cause tranlog read failure

There is a race condition in logreceiver logic that can cause the slave stone to attempt to read from a newly transferred tranlog before any data has been permanently flushed to disk. This will generate errors claiming that the file is not a GemStone transaction log file. (#47844)

Logsender added padding to tranlog records when compressing

During compression, the logsender added padding to the tranlog records. This was unnecessary; and the cumulative padding could cause buffer overrun. (#47916)

Issues related to upgrade

Invalid version number in extents, confusing later upgrade errors

The GemStone extents store internal version numbers, to verify compatibility between versions. While the numbers for the checks were correct, the version number stored for reporting errors was ’820.0’, which made the error messages unclear. This has been corrected, so upgrading from 3.4.4 or later 3.4.x versions to future versions will be able to report useful error messages. (#49752)

After upgrade, GcGems not running

During the Stone restart during upgrade, the Reclaim and Admin GcGem/s were started, but could not login due to a version mismatch, and no further startup attempts were made after upgrade. This left the upgraded system without the GcGems running. (#47884)

GsSysLog writeErrorMessage fails with ArgumentError

Execution of the method GsSysLog >> writeErrorMessage: failed with an Invalid priority ArgumentError. (#47878)

Incorrect error from before: and after: on out of range

When argument to SequentialCollection >> before: was the first element, or the argument to after: was the last element, the error incorrectly stated that the argument was not found in the collection. (#48072)

instVarAt:put: did not work for a class instance variables

Using instVarAt:put: to assign a class instance variables value reported an error. (#47430)

Unable to write to stdout on Windows GCI client

While it was possible to get a result from GsFile stdout (and stderr, etc.), from a Windows client application (such as GBS), it was not possible to perform write or flush operations to write data to the application's console. (#47998)

copydbf -Z did not work

The copydbf code supporting the -Z option (compress with zlib) used a compression argument that was out of range. (#48188)

Host Memory Banner incorrect on the Mac

The amount of physical memory reported for the host of a GemStone process was not reported correctly on the Mac; the number provided was always 2048 MB. (#47905)

enableUnixAuthenticationWithAlias: failed in Unicode Mode

In a repository that has enabled Unicode Comparison Mode, the method UserProfile >> enableUnixAuthenticationWithAlias: fails with a primitive failure. (#47251)

Issues only applicable to slow (debugging) gems

GemStone provides "slow" libraries, which have less optimization and additional assertion checks, for debugging. In v3.4.x, using gemnetobject_slow could fail in loading the slow shared library, in which case it fell back to using the regular fast library (#48219).

Also, when running with a slow gem, certain code compilations could result in a SEGV (#48224)