2. Configuring Gem Session Processes
This chapter tells how to configure the GemStone/S 64 Bit session processes for your application. For additional information about running session processes on a node remote from the Stone repository monitor, refer also to Chapter 3.
2.1 Overview
As shown in Figure 2.1, a GemStone session involves the following components in a client-server relationship:
- The user application
- A session manager process (Gem), which acts as a server for a particular application
- The Stone repository monitor
- The shared page cache and shared page cache monitor
- The Stone’s AIO page server and other server processes
- The repository extent files
Figure 2.1 GemStone Session Elements
As seen by the application, the Gem session process provides the bulk of the repository capabilities. From the viewpoint of the application, the Gem is the object server:
- It logs in to the repository through the Stone repository monitor, and it obtains object locks, free object identifiers, and free pages from the repository monitor.
- It presents the application with a consistent view of the repository during a transaction and tracks which objects the application accesses.
- It executes Smalltalk methods within the repository.
- It reads the repository as the application accesses objects, and (with the help of the AIO page server) it updates the repository when the application successfully commits a transaction.
Linked and RPC Applications
The Gem session process can be run as a separate process (as in Figure 2.1) or integrated with the application into a single process, in which case the application is called a linked application.
When the Gem runs as a separate process, it responds to Remote Procedure Calls (RPCs) from the application; this is called an RPC application. Applications that use a separate Gem process start that process automatically (from the user’s viewpoint) while logging in to the repository.
Either type of application can be used on a single node or across a network.
An application can have only one linked login, since only one Gem can be integrated with the process. It may have multiple RPC logins, or one linked and multiple RPC logins.
GemStone provides both linked and RPC versions of topaz for repository administration, and shared client libraries that allow linked and RPC. The linked version allows both linked and RPC logins, but the RPC version allows only RPC logins.
C programmers should always use an RPC version during development and debugging to protect Gem data structures from possible corruption.
The Session Configuration File
At start-up time, each Gem session process looks for a configuration file, which by default is the same system-wide configuration file sought by the repository monitor when it starts. However, there are three important differences:
- The session configuration file is optional. If one is not found, the session process uses system defaults.
- All session processes read those configuration options that begin with “GEM_” and the few that are used by both Stones and Gems (such as DUMP_OPTIONS and LOG_WARNINGS). Other settings that the Gem needs are obtained from the repository monitor by network protocol and are the same for all sessions logged in to that Stone.
- The first session process on a node remote from the Stone and extents uses the shared page cache configuration options (SHR_), which determine the configuration of the cache on that node.
It can be useful for certain sessions to use a different configuration. Appendix A, “GemStone Configuration Options” tells how to specify an alternate configuration file and how to use supplementary files to adjust the system-wide configuration for a specific session process. There are always ways to include individual parameter settings by passing these as arguments to the Gem or to Topaz. This appendix also includes descriptions of each configuration option.
2.2 How To Configure Gem Session Processes
To configuring a Gem session process, you perform the following steps:
1. Gather application specifics about the number of sessions that will be logged in to the repository simultaneously from this node.
2. Plan the operating system resources that will be needed: memory and swap (paging) space.
3. Set the Gem configuration options.
If this node is remote from the repository monitor, enable a local GemStone shared page cache. Gem session processes running on a server node always use the Stone’s shared page cache.
4. Set GemStone file permissions to allow session processes access while providing adequate security.
Gathering Application Information
System resources needed for session processes primarily depend on the number of sessions that will be logged in to a particular repository from this node. Remember that in some applications each user can have more than one session logged in.
Planning Operating System Resources
GemStone session processes need adequate memory and swap space to run efficiently. In addition, kernel parameters can limit the number of sessions that can connect to the shared page cache.
Estimating Session Memory Needs
Two factors determine the memory needs for session processes:
- The size of the shared page cache on a node remote from the Stone and extents will depend on the configuration of the Gem that starts the cache. (There is only one cache on each node for a particular repository; session processes running on the server node attach to the Stone repository monitor’s cache.)
- The amount of memory required by a Gem session is dependent on how it is configured, as determined by the system requirements. To avoid out-of-memory conditions, Gems must be configured with an adequate temporary object cache. The default of 50MB is suitable for light use; it is likely that particular operations in an application will require more, possibly much more.
With the 50 MB default Gem’s temporary object cache, the first Gem session process or linked application on a node ordinarily requires about 80 MB of memory, of which 5 MB is for code that can be shared by other session processes. Each additional session process requires about 65 MB. If you tune the cache size for Gems, add any increase to the amount given here.
In addition to the memory needs for session processes, you must also allocate memory for object server processes. For details, see Planning Operating System Resources.
For Gem session processes running on machines that are remote from the object server, there are additional memory needs on the server. For information about this, see Estimating Server Memory Needs.
Estimating Client Swap Space Needs
Swap (paging) space on machines remote from the Stones should follow the same general guidelines given for servers under Estimating Server Swap Space Needs. In determining the additional swap space needed for GemStone session processes, use the memory requirements derived in the preceding section (Estimating Session Memory Needs), including space for the number of sessions you expect. The resulting figures will approximate the client’s needs, and are in addition to the swap requirement for the object server and non-GemStone processes.
Estimating Session File Descriptor Needs
When a Gem session process starts, it attempts to raise the file descriptor limit from the default (soft) limit to the hard limit set by the operating system. GemBuilder applications and page servers do the same. Gem session processes use file descriptors this way:
7 for stdin, stdout, stderr, and internal communication
1 for each connection between the Gem and an RPC application
1 for each local extent within a file system
2 for each local extent that is a raw partition
1 for each extent on a remote node
GemBuilder applications that start a large number of RPC Gems need a correspondingly large number of file descriptors.
You can override the default behavior of raising the file descriptor limit to the hard limit by setting the GEMSTONE_MAX_FD environment variable to a positive integer. A lower limit may be desirable in some cases to reduce the amount of virtual memory used by the process. A value of 0 disables attempts to raise the default limit.
The value of GEMSTONE_MAX_FD in the environment of a NetLDI (Network Long Distance Information) server is passed to its child processes.
Reviewing Kernel Tunable Parameters
The kernel parameter of primary relevance to GemStone session processes is the maximum number of semaphores per semaphore id (typically semmsl or a similar name, although it is not tunable under all operating systems). This parameter limits the number of sessions than can connect to the shared page cache, because each session uses two semaphores.
How you determine the existing limits depends on your operating system. If the information is not readily available, proceed with setup. A later step shows how to verify that the limits are adequate for the GemStone configuration you set up.
To Set Ownership and Permissions for Session Processes
The primary consideration in setting file ownership permissions for client access is to make sure the Gem session process can read and write both the extents and the shared page cache. You must also take into account the following factors:
- Is the Gem session process linked or RPC?
- Does the Gem session process runs on the server or on a node remote from the Stone.?
- Does the server uses setuid bit and protection mode 600 for the extents (as recommended here), or does it use the alternative of group write permission?
Extents
The extents may be protected as read-write only by their owner (protection 600) if you use the setuid (S) bit for repository executables as recommended here. Otherwise, the extents must be writable by a group to which the GemStone users belong (protection 660).
Shared Page Cache
The shared memory and semaphore resources used by GemStone are created and owned by the user account under which the Stone repository monitor is running and have the same group membership. Access for the shared page cache is set by the SHR_PAGE_CACHE_PERMISSIONS configuration option; by default, it is read-write for the owner and read for the group (the equivalent of file protection 640). You can inspect the cache ownership and permissions by using the ipcs command.
For a session to log in using a shared page cache, the Operating System user account of the linked application or Gem session process must either be the same as that of the Stone (such as the gsadmin account) or be one that belongs to the same group as the Stone. The same requirement applies to page server processes, which are discussed in Chapter 3, “Connecting Distributed Systems”.
If the setuid bit is set on repository executables as recommended in Table 1.2, the Stone process and shared page cache will belong to the owner you specify for those files (such as gsadmin).
To Set Access for Linked Applications
For linked applications on the server, we recommend you try using the setuid bit on the application’s executable file. Have the file owned by gsadmin as it is defined here. This works well for topaz -l. The installgs script offers to set the file ownership and permissions for you. To do it manually, do this while logged in as root:
# cd $GEMSTONE/bin
# chmod u+s topaz
# chown gsadmin topaz
You may prefer not to use the setuid bit with linked applications that do not distinguish between real and effective user IDs. GemStone’s Topaz executable performs repository reads and writes as the effective user (the account that owns the executable’s file), but performs other reads and writes as the real user (the one who invoked it). Linked applications that do not make this distinction, such as a third-party Smalltalk used with GemBuilder, are likely to perform all I/O as the effective user, or gsadmin. If this result is unsatisfactory, remove the S bit on that executable and add group write permission to the extents.
To Set Access for All Other Applications
All RPC applications require a GemStone NetLDI service to start a separate Gem session process and/or a page server. For these sessions, we recommend that the Gem session process and page server always be owned by (run as) the gsadmin account. That arrangement ensures that the Gem will be able to read and write both the extents and the shared page cache. The ownership and protection of the application executables themselves is not a factor.
To Set Access to Other Files
GemStone creates log files and other special files for session processes in several locations, which are described below. In a multi-user environment, the protection of these resources must be such that the appropriate file can be created or updated in response to actions by several users.
$HOME
GemStone ordinarily creates log files for spawned processes (such as RPC Gem session processes and page servers) in the home directory of the user or the NetLDI captive account. In situations where the home directory cannot be writable, the environment variable GEMSTONE_NRS_ALL can be used to specify an alternative location; see To Set a Default NRS.
/opt/gemstone
All users should have read/write/execute access to the directories /opt/gemstone/log and /opt/gemstone/locks on each host (or an equivalent location, as discussed here).
By default, NetLDI (Network Long Distance Information) and hot standby processes create log files in the log directory.
Repository-wide lock files are created in the locks directory, and other processes use this location to look up access information for them.
To Set Configuration for Remote Shared Page Caches
The configuration file used by Gem sessions will also be used, on remote nodes, to configure the associated remote shared page cache monitor.
The Stone always creates a shared page cache monitor and cache on its own node, based on parameters in the Stone’s configuration file. This cache is always used by Gems that run on this node.
On other nodes, when the first Gem session process logs in, if there is not already a shared page cache running on that remote node, a remote cache is started. Parameters specified for the first Gem that logs in determine the size of the cache and the number of processes that can attach to it (SHR_PAGE_CACHE_SIZE_KB and SHR_PAGE_CACHE_NUM_PROCS, respectively). These configuration parameter settings usually come from the Gem’s configuration file, but may also be specified by Topaz arguments or arguments with the Gem’s NRS.
All subsequent sessions that log in from that remote node will use the same cache.
2.3 Set the Gem Configuration Options
There are a number of configuration options for Gem session processes.
Configure Temporary Object Space
Gems use temporary object memory for a number of purposes, described in detail in Chapter 11. The upper limit on this memory is configured by the GEM_TEMPOBJ_CACHE_SIZE configuration option. It is important to provide sufficient temporary object space. If temporary object memory is exhausted, the Gem can encounter an out-of-memory condition and terminate.
The default of 50000 (50 MB) should be adequate for normal user sessions. For sessions that place a high demand on the temporary object cache, modifying large numbers of objects or working with large collections, a large value will be needed.
The configured value must be large enough to accommodate the memory needs of any gem using that configuration. The amount of temporary object memory required may be different for different application gems, depending on the specific tasks of the gem; you may wish to set up special configuration files for application sessions that have a particularly high demand on memory.
The full amount of GEM_TEMPOBJ_CACHE_SIZE is not allocated on gem login, but it is reserved and will impact memory usage per user.
The configured size of the Stone’s shared page cache plus temporary object memory size should not be greater than the amount of memory available on the machine on which the Gem will be running, allowing for overhead for the operating system and other GemStone requirements. The Gem temporary object cache should also be smaller than the size of the Stone’s shared page cache, otherwise it may be impossible for operations such as commit to execute.
You will probably need to experiment somewhat before you determine the optimum size of the temporary object space.
Memory management is discussed in greater detail in Chapter 11, “Managing Memory”.
Configure SSL for remote Gem sessions
During the RPC login process, a Secure Socket Layer (SSL) socket is used to establish the login. After that, for gem processes on the same node as their client application, communication reverts to a normal socket connection. For Gem processes that are running on a different node than their client, communication continues to use SSL.
This results in a slightly slower connection, due to the overhead of encrypting and decrypting data. To configure the gem to use normal socket communication for remote connections, set the GEM_RPC_USE_SSL configuration option to false.
2.4 How To Access the Configuration at Run Time
GemStone provides several methods in class System that let you examine, and in certain cases modify, the session configuration parameters at run time.
To Access Current Settings at Run Time
System class methods let you examine the configuration of your current Gem session process. There are three access methods for session processes:
gemConfigurationReport
Returns a SymbolDictionary whose keys are the names of configuration file parameters, and whose values are the current settings of those parameters in the current session’s Gem process.
gemConfigurationAt: aName
Returns the value of the specified configuration parameter from the current session, or returns nil if that parameter is not applicable to a session process.
configurationAt: aName
Returns the value of the specified configuration parameter, giving preference to the current session process if the parameter applies to a Gem.
To Change Settings at Run Time
The class method System class >>configurationAt: aName put: aValue lets you change the value of the internal run-time parameters in Table 2.1, provided you have the appropriate privileges.
Parameters read from the configuration file at process startup are in all uppercase, and are read-only. Parameters that can be changed at runtime have similar, but not necessarily identical, names that are in mixed case without underscores.
CAUTION
Do not change configuration parameters unless there is a clear reason for doing so. Incorrect settings can have serious adverse effects on GemStone performance. Appendix A provides additional guidance about run-time changes to specific parameters.
The following example changes the value of the configuration option #GemFreeFrameLimit:
topaz 1> printit
System configurationAt: #GemFreeFrameLimit put: 4000
%
4000
For more information about the parameters that can be changed at run time, see Appendix A, “GemStone Configuration Options”.
2.5 Tuning Gem Performance
Private Page Cache
The setting for the GEM_PRIVATE_PAGE_CACHE_KB configuration option specifies the size (in KB) of the Gem’s private page cache. The default value of this option is 1000; in most cases, this value is acceptable, and you do not need to do any further tuning.
When monitoring the Gem, if the value of the statistic LocalCacheOverflowCount is non-zero, you may want to increase the setting for GEM_PRIVATE_PAGE_CACHE_KB as needed.
Native Code
By default, generation of native code for Smalltalk methods is enabled. This is configured using GEM_NATIVE_CODE_ENABLED configuration option. When native code is disabled, execution is interpreted; behavior will be identical but somewhat slower.
Native code generation can be disabled using the runtime parameter #GemNativeCodeEnabled, but it cannot then be re-enabled for that session’s lifetime.
If any breakpoints are set in any methods, native code is disabled. Native code is re-enabled when all breakpoints have been removed.
To determine if native code is in use by the currently executing session, execute:
GsProcess usingNativeCode
Under some configurations on x86, in particular on the Macintosh, the 32-bit offset limit may be exceeded in some cases with a very large temporary object cache. If this occurs, native code is disabled.
Note that the Foreign Function Interface (FFI) feature has additional limitations when native code is disabled. FFI is discussed in the Programming Guide for GemStone/S 64 Bit.
2.6 How To Install a Custom Gem
The GemBuilder for C manual explains how to create a custom Gem session executable containing your own C functions to be called from Smalltalk. One way to make this custom Gem available to all users is to perform the following steps as system administrator:
Step 1. Copy the shell script gemnetobject from $GEMSTONE/sys to your working directory. This shell script is used to start Gem session processes under the UNIX shell. You will modify this script to start your custom Gem executable instead of the standard one.
Step 2. In your copy of gemnetobject, find the section labeled User-definable symbols. In that section, replace gem in the line
gemname="gem"
with the name of the new Gem executable. For example:
gemname="MyGem"
Step 3. Rename your modified copy of the shell script gemnetobject so that it has a distinct filename. For example:
% mv gemnetobject MyGemnetObject
Step 4. Copy the new shell script to $GEMSTONE/sys. Make sure that all GemStone users have read and execute (r-x) permission for the script. For example:
-r-xr-xr-x 1 root 912 Feb 2 20:22 MyGemnetObject
If necessary, change the permissions:
% chmod 555 MyGemnetObject
Step 5. Add an entry for the new shell script to the services database, $GEMSTONE/sys/services.dat. A NetLDI checks that file to translate the name of a service to a command it can execute. For example:
MyGemnetObject $GEMSTONE/sys/MyGemnetObject
Step 6. Copy the new Gem executable to the GemStone system directory. For example:
% cp MyGem $GEMSTONE/sys
Step 7. Make sure that all GemStone users have read and execute (r-x) permission for the new Gem executable.