GemStone Utility Commands
GemStone provides the following utility commands in the $GEMSTONE/bin directory.
- copydbf – copy an extent, transaction log or backup.
- gemnetobject – script that starts a Gem during RPC login.
- gslist – list running GemStone server processes.
- largememorypages – calculate large memory page requirements.
- netldidebug – turn debugging on or off for a NetLDI process.
- pageaudit – offline audit of repository pages.
- printlogs – print contents of a transaction log.
- pstack – get C‑level and Smalltalk stack traces.
- removedbf – delete an extent, transaction log or backup.
- searchlogs – search for OOPs within a transaction log.
- secure_backup_extract – extract signing information from a backup.
- startcachewarmer – start cache warmers.
- startlogreceiver – start logreceiver for hot standby system.
- startlogsender – start logsender for hot standby system.
- startnetldi – start a NetLDI.
- startstone – start a Stone.
- statmonitor – start statmonitor to record statistics.
- statprom – start statprom to serve statistics to Prometheus.
- stoplogreceiver – stop logreceiver for hot standby system.
- stoplogsender – stop logsender for hot standby system.
- stopnetldi – shut down a NetLDI.
- stopstone – shut down a Stone.
- topaz – command line scripting tool for GemStone.
- updatesecuredbf – update key in an encrypted dbf file.
- verify_backup_with_openssl – verify signature of signed backup
- vsd – graphic tool to analyze GemStone statistics files.
- waitstone – verify status of a Stone.
copydbf
copydbf sourceFile destFileOrDir [ ‑z [ comprLev ]| ‑Z [ comprLev ]| ‑u ] [ ‑E ] [ ‑f filePrefix ] [ ‑s Mbytes ] [ ‑l | ‑m | ‑P ] [ ‑F ]
Copy an extent, transaction log or backup file, converting per the arguments.
copydbf sourceFile ( ‑a | stdout ) [ ‑z [ comprLev ]| ‑Z [ comprLev ]| ‑u ] [ ‑l | ‑m | ‑P ]
Copy an extent, transaction log or backup file to stdout.
Report information on an extent, transaction log or backup file to stdout.
Report information on the copydbf utility to stdout.
copydbf ‑V sourceFile [ ‑K keyRingDirs ]
Verify a digitally signed backup.
copydbf sourceFile destFile ‑e encryptionCert ‑s keyLength ‑K dir [ ‑K dir+ ]
Encrypt an extent or transaction log.
copydbf sourceFile destFile ‑D privKey ‑K dir [ ‑K dir+ ] [ ‑c ] [ ‑j passPhrase | ‑J pfFile ] [ ‑O ]
Decrypt an encrypted extent, transaction log or backup file; or verify it can be decryted.
copydbf sourceFile ‑W encrCertFN | ‑X signCertFN | ‑Y sigFN
Extract the encryption certificate from an encrypted extent, transaction log or backup file.
copydbf reads GemStone extent files, transaction logs, and backups, and either reports the details, or writes a copy of the file with or without performing state changes such as compression/ decompression, encryption/unencryption, and byte order changes. Finally, you can use copydbf to extract the public key from an encrypted file.
For a copy that does not involve raw partitions and does not make state changes, GemStone repository files can be copied using the ordinary cp command, as well as using copydbf.
To make copies of extent files or transaction logs, the user executing copydbf must have read permission to the file. If you attempt to copy extent files that are in use, and if checkpoints are not suspended, the resulting repository may be corrupt and unusable. See How To Make an Extent Snapshot Backup for more information.
copydbf reports information about the file that is being copied.
For example, the following makes a copy of a clean GemStone extent into the data directory:
os$ copydbf $GEMSTONE/bin/extent0.dbf $GEMSTONE/data
Source file: /gshost/GemStone3.7/bin/extent0.dbf
File type: extent fileId: 0 in a repository with 1 files
File size: 50331648 bytes (48 MB), 3072 records
ByteOrder: Intel (LSB first) compatibilityLevel: 844
Last checkpoint written at: 2023‑08‑14‑15:08:02.292.
Extent was shutdown cleanly; no recovery needed.
GemStone Version: 3.7.0, Fri Aug 11 15:50:43 2023 0b86af16998557
Destination file: /gshost/GemStone3.7/data/extent0.dbf
Compression of copydbf output
copydbf preserves the compression state of the original. The output of copydbf can be compressed during the copy using gzip (‑C or ‑z) or LZ4 (‑Z) compression. Restore from backup, transaction log restore, and copydbf can read all compression types.
Compresssed backups, transaction logs and extents can be uncompressed using ‑u.
copydbf will append the correct extension (.gz., .lz4, .dbf, or .sdbf) to the destination filename, if the copydbf target filename does not already end in that extension.
Encrypted files cannot be compressed or uncompressed using copydbf.
Byte Order
GemStone executables write .dbf files in the native byte ordering for the platform they are executing on. When moving between operating systems with different byte orderings, such as between AIX/RS6000 and Linux/x8664, it is strongly recommended to use copydbf to update the byte order. Files with non‑native byte order can be used, but are inefficient. See the ‑l and ‑m arguments.
Using copydbf to access information on a file
To obtain the same source file information (but not the size) without making a copy, use the second form of the command: copydbf ‑i sourceFile. In this usage, you do not specify a destination.
The following copydbf ‑i example displays information for an extent, and indicates the oldest transaction log that would be needed to recover from a system crash:
os$ copydbf ‑i $GEMSTONE/bin/extent0.dbf
Source file: /users/GemStone/bin/extent0.dbf
File type: extent fileId: 0 in a repository with 1 files
File size: 50331648 bytes (48 MB), 3072 records
ByteOrder: Intel (LSB first) compatibilityLevel: 844
Last checkpoint written at: 2023‑08‑14‑15:08:02.292.
Extent was shutdown cleanly; no recovery needed.
GemStone Version: 3.7.0, Fri Aug 11 15:50:43 2023 0b86af16998557
Encryption: NONE
The next example displays information for a backup, and indicates the oldest transaction log that would be needed to restore subsequent transactions.
os$ copydbf ‑i backup.dat
Source file: /gshost/GemStone3.7/backup.dbf
File type: backup fileId: 0 in a backup set with 1 files
File size: 49152000 bytes (46 MB), 375 records
ByteOrder: Intel (LSB first) compatibilityLevel: 860
The file was created at: 08/14/2023 12:22:17 PDT
Full backup started from checkpoint at: 08/14/202316:03:46.
Oldest tranlog needed for restore is fileId 3 ( tranlog3.dbf ).
Backup was created by GemStone Version: 3.7.0, Fri Aug 11 15:50:43 2023 0b86af16998557.
Backup Attributes:
Compression: NONE
Encryption: NONE
Signature Hash: NONE
Encryption Keys: 0
For a listing of all checkpoints recorded in a transaction log, use copydbf ‑I sourceFile. This information is helpful in restoring a GemStone backup to a particular point in time. For example:
os$ copydbf ‑I tranlog5.sdbf
Source file: /gshost/GemStone3.7/tranlogs/tranlog5.sdbf
File type: secure tranlog fileId: 5
File size: 14336 bytes (14 KB), 28 records
ByteOrder: Intel (LSB first) compatibilityLevel: 950
The file was created at: 08/14/202313:17:53 PDT
The previous file last recordId is 8
Encryption: AES_256_XTS
Scanning tranlog to find last checkpoint...
Checkpoint 1 started at: 08/14/2023 13:17:52 PDT.
oldest transaction references fileId ‑1 ( this file ).
Checkpoint 2 started at: 08/14/2023 13:17:52 PDT.
oldest transaction references fileId ‑1 ( this file ).
Checkpoint 3 started at: 08/14/2023 13:17:52 PDT.
oldest transaction references fileId ‑1 ( this file ).
Checkpoint 4 started at: 08/14/20230 13:18:57 PDT.
oldest transaction references fileId ‑1 ( this file ).
Checkpoint 5 started at: 08/14/2023 13:19:06 PDT.
oldest transaction references fileId ‑1 ( this file ).
Checkpoint 6 started at: 08/14/2023 13:24:07 PDT.
oldest transaction references fileId ‑1 ( this file ).
Checkpoint 7 started at: 08/14/2023 15:48:09 PDT.
oldest transaction references fileId ‑1 ( this file ).
Checkpoint 8 started at: 08/14/2023 16:44:47 PDT.
oldest transaction references fileId ‑1 ( this file ).
Checkpoint 9 started at: 08/14/2023 16:45:35 PDT.
oldest transaction references fileId ‑1 ( this file ).
copydbf with raw partitions
To moved extent files or transaction logs onto or off of raw partitions, you should use copydbf. copydbf of backup files to raw partitions is not supported.
The following example copies a fresh repository extent to an existing raw disk partition. If the raw partition already contains a repository file or backup, use removedbf first to mark it as being empty.
os$ copydbf $GEMSTONE/bin/extent0.dbf /dev/rsd3h
Copying a transaction log from a raw partition to a file system directory generates a file name having the same form as if that transaction log had originated in a file system. If the internal fileId is 43, this example would name the destination file /dsk1/tranlogs/tranlog43.dbf:
os$ copydbf /dev/rsd3h /dsk1/tranlogs/
copydbf with encrypted backups, extents, and transaction logs
copydbf can copy encrypted files without special arguments; the contents are copied without decryption. The ‑e, ‑s, ‑k, ‑D, ‑j, ‑J, and ‑O flags apply to encrypting and decrypting dbf files. Note that you cannot encrypt a backup file using copydbf, although you can decrypt an encrypted backup.
copydbf for encrypted dbf files does not allow encryption, or changes to byte order.
To change the encryption key for a .dbf file, without a separate decrypt and re‑encryption, you an use updatesecuredbf, which changes the encryption key in place.
You can extract the public key of an encrypted .dbf using the ‑W argument. With a secure backup, which is also signed, you can also verify the signature and extract the signature and signing key using copydbf with the ‑V, ‑X or ‑Y arguments.
See Chapter 12, “Encrypted Extents and Transaction Logs” for details on using copydbf to manage encrypted extents and transaction logs.
gemnetobject
The gemnetobject script is not invoked on the command line. It, and related scripts such as gemnetdebug, are used to set the environment and invoke the Gem process during an RPC login.
Some of the arguments to gemnetobject are passed in by the NetLDI process, but other arguments can be included with the gemnetobject specification during login. This allows specification of the Gem environment for a specific login without requiring the definition of a configuration file.
gemnetobject TCP socketFD [ ‑T tocSizeKB ] [ ‑N nativeCodeArg ] [ ‑e exeConfPath | ‑E exeConfigFile ] [ ‑C configParamsString ] [ ‑U ignoredArg ] otherApplicationArgs
|
File descriptor of client socket inherited from fork. Provided automatically by NetLDI. |
|
|
Passes in a configuration file; the settings in the specified configuration file override environment settings. Only one of ‑e, ‑E can be used. |
|
|
The GemStone executable configuration file.Only one of ‑e, ‑E can be used. Suppresses use of configuration file environment variables and default configuration file locations; recommended for use in RPM installations. See Executable Configuration File for RPC Gem. |
|
|
The argument is a string containing configuration value settings. Multiple configuration parameter settings are accepted, and must be separated by ‘;’ (with no additional spaces). |
|
|
Arguments are 0, 1, or 2. A setting overrides other settings for GEM_NATIVE_CODE_ENABLED. |
|
|
a setting overrides other settings for GEM_TEMPOBJ_CACHE_SIZE. |
|
|
Other application‑specific arguments may be included, which are passed to the Gem without interpretation. |
The user‑supplied arguments may be specified in any order. If any of the arguments ‑C, ‑N, and ‑T are included twice, only the last specification is used.
Any other arguments or tokens can be included at the end or within the string passed to gemnetobject.
All arguments, including ignoredArg and otherApplicationArgs, are passed to the Gem and available to the application via System class >> commandLineArguments.
The Gem configuration parameters to be used by the Gem are determined in the following precedence order (first one found is used).
- values in ‑T or ‑N argument
- values in ‑C argument
- values in the executable config file.
- values in the system config file.
The system configuration file is the first one found when searching:
- path from $GEMSTONE_SYS_CONF environment variable (if ‑E is not used)
- $GEMSTONE/data/system.conf (if ‑E is not used)
The executable configuration file is the first one found when searching:
- path from ‑e or ‑E argument
- path from $GEMSTONE_EXE_CONF environment variable (if ‑E is not used)
- a file named 'gem.conf' in current directory of the gem process (the current directory is set from a #dir directive, or from a NetLDI ‑D argument; Controlling log file directory locations) (if ‑E is not used)
The higher precedence setting is used and lower precedence settings are ignored. If there are multiple settings for a specific parameter at the same level, that is, in the same executable or system configuration file, or within the ‑C argument, the last one that is found applies and earlier settings for that parameter within the file or string are ignored.
The following variants of gemnetobject are included in the distribution. Any of these can be used in place of gemnetobject.
gemnetdebug—sets debugging environment variables that are particularly useful for debugging memory issues
gemnetobject_noop—starts a non‑optimized Gem executable
gemnetobject_slow—starts a slow Gem executable with assertion checks
gemnetobject_keeplog—starts an ordinary Gem executable, configured to not delete the gem log on clean shutdown.
Examples
gemnetobject is used in topaz, GemBuilder for Smalltalk, and GemBuilder for C applications in the login parameters for gemnetid. The following examples are in topaz.
Note that single quotes may be needed when the gemnetid arguments include spaces.
topaz> set gemnetid 'gemnetobject ‑T 500MB ‑e devStone.conf'
This example passes in a temporary object cache size and a configuration file named devStone.conf with other configuration parameter settings.
topaz> set gemnetid '!#netldi:54321!gemnetobject ‑C GEM_RPCGCI_TIMEOUT=5;GEM_ABORT_MAX_CRS=2; ‑N 1 devStone'
This example specifies NetLDI port (54321) to fork the Gem. It includes two configuration parameter settings and modifies the Native Code setting. The token devStone is ignored, but visible in ps output and can be used by code that fetches the command line arguments.
Custom gemnetobject scripts
You may create your own copies of gemnetobject and customize these as needed. See Creating a custom gemnetobject.
Error checking
Note that minimal argument error checking is done and unrecognized tokens or invalid argument values are ignored during login. Check the gem log to ensure that intended values are used.
For example, an extra space as in the following, results in the second configuration value silently being ignored:
'gemnetobject ‑C GEM_RPCGCI_TIMEOUT=5; GEM_ABORT_MAX_CRS=2;'
gslist
gslist [ ‑c ] [ ‑l | ‑p | ‑x ] [ ‑H ] [ ‑m host ]+ [ [ ‑n ] name ]+ [ ‑N netldiName ] [ ‑q ] [ ‑s key | ‑S key ] [ ‑t secs ] [ ‑u user ] [ ‑v ] [ ‑U certFile ‑R keyFile ‑J caCertFile ]+
The gslist command prints information about GemStone servers. The default listing prints the following server attributes in columns:
When you include the ‑l or ‑x option, the following additional columns are printed:
The ‑x option prints the preceding attributes on separate lines, and adds lines for the following as appropriate:
|
The GemStone system configuration file. See Stone configuration files. |
|
|
The GemStone executable configuration file. See Executable Configuration File for Stone. |
|
If many servers are reported as frozen to gslist ‑v, try increasing ‑t secs.
By default, status is returned for all servers on the current host. To specify a particular server, use the ‑n switch, or just include the server name on the command line (since the ‑n is optional). To specify multiple server names, include the ‑n name option for each server.
Remote queries
The ‑m option allows you to list servers on a remote host. To specify more than one remote host, include the ‑m host option multiple times, one for each host. Names on remote hosts are prefixed by host:, where host is the name of the remote machine.
To list servers on a remote host, gslist must be able to contact a NetLDI running on the remote machine.
- The remote NetLDI must be a GemStone version that is compatible with the version of gslist. This limitation applies to the remote NetLDI, but not to the Stone and other server processes on the remote host, for which gslist fetches information.
- The remote NetLDI must be named with the default name, or the ‑N argument should be included to specify the name. The default name here is the default for the environment in which gslist is running. This is either “gs64ldi”, or a name specified by $GEMSTONE_NRS_ALL.
Date and Time format
The date and time that the process started is normally printed in a format specific to gslist and fitting into the table display. To get a parse‑able but potentially less readable format, you may use the environment variable GS_GSLIST_TIME_FORMAT to specify a UNIX‑style date format string.
Json output
gslist ‑j provides the gslist ‑x contents, but in json format. This can be easily processed as needed for system administration. For example:
JsonParser new parse: (System performOnServer: 'gslist ‑j')
This creates a structure of Dictionaries and Arrays in GemStone Smalltalk containing all information about all GemStone services. Note that all keys and values in the output are capitalized. This includes gslist status values (such as ’Exists’), and command line arguments.
logsender details in Json
gslist ‑j ‑v is a special case of gslist ‑j, which includes printing of logreceiver details for a logsender process. See gslist ‑v ‑j information for a logsender for more information.
largememorypages
largememorypages [ ‑r ] [ ‑e exeConfig ] [ ‑z systemConfig ] [ ( ‑M sharedCacheKB | ‑F sharedCacheFrames ) ‑P maxProcesses ‑C maxSharedCounters ‑p largeMemoryPageSize ]
largememorypages computes the number of large or huge memory pages that will be required for a given shared page cache size and other system requirements.
Large or huge memory pages are available for use with GemStone on Linux and AIX.
- On Linux, GemStone supports 2MB and 1024 MB (1 GB) pages.
- On AIX, 16GB large memory pages are supported.
Large pages are not supported on other platforms.
Using the largememorypages utility is described in detail in the Installation Guide for Linux and for AIX.
For example, for a 20GB shared page cache with 1 GB pages:
os$ largememorypages ‑p 1GB ‑M 20GB ‑C 1900 ‑P 200
Cache kind is: Primary SPC (stone shared page cache)
Cache config is 1373248 pages = 21457 MB, total is 22528 MB, overhead 4% of configured size
HashTableSize 2097152 TotalPces 3483008 freeListPces 1385856
Large page size requested is: 1024 MB.
Large page overhead: 0.09 MB
[Info]: Increasing number of pages from 1310720 to 1373248 to reduce large memory page overhead.
For 1373248 pages, 200 processes and 1900 shared counters, 0 pusherThreads
minimum sizing for cache shmmax 23622320128, shmall 5767168.
Number of 1024 MB large pages required: 22
Commands to enable large memory pages on Linux. Run as root:
<details to configure linux>
os$ largememorypages ‑p 2MB ‑M 20GB ‑C 1900 ‑P 200
Cache kind is: Primary SPC (stone shared page cache)
Cache config is 1310720 pages = 20480 MB, total is 21534 MB, overhead 5% of configured size
HashTableSize 2097152 TotalPces 3420480 freeListPces 1323328
Large page size requested is: 2 MB.
Large page overhead: 0.91 MB
For 1310720 pages, 200 processes and 1900 shared counters, 0 pusherThreads
minimum sizing for cache shmmax 22580035584, shmall 5512704.
Number of 2 MB large pages required: 10767
Commands to enable large memory pages on Linux. Run as root:
<details to configure linux>
netldidebug
netldidebug [ ‑h ] netldiNameOrPort actionToTake
|
The action to take, which must one of the keywords enabledebug, extradebug, or disabledebug. |
The netldidebug command allows you to modify the debug state of a running NetLDI (that is, the state resulting from omitting or including ‑d with startnetldi), without the need to shut down and restart the NetLDI.
pageaudit
pageaudit [ ‑e exeConfig | ‑E exeConfigFile ] [ ‑z systemConfig ] [ ‑f ] [ ‑d ] [ ‑l logfile ] [ gemStoneName ]
pageaudit [ ‑e exeConfig | ‑E exeConfigFile ] [ ‑z systemConfig ] [ ‑f ] [ ‑d ] [ ‑l logFile ] [ ‑D privateKey ‑K dir [ ‑K dir+ ] [ ‑j passPhrase | ‑J passphraseFile ] [ gemStoneName ]
|
Name of the GemStone repository monitor; the default is gs64stone‑audit. Network resource syntax is not permitted. |
|
|
Disable audit of data pages. Only audit Object Table pages, bitmaps, and other non‑data pages. |
|
|
For encrypted extents, specifies the private key file corresponding to the public key used to encrypt the extents. The file must be in PEM format. Requires ‑K, and if the private key has a passphrase, also ‑j or ‑J. |
|
|
The GemStone executable configuration file, or a directory containing a specifically named file. See Executable Configuration File for Stone. |
|
|
The GemStone executable configuration file.Only one of ‑e , ‑E can be used. When ‑E is specified, the ‑z argument must be a file, not a directory. Suppresses use of configuration file environment variables and default configuration file locations; defines an internal environment variable $GEMSTONE_DATA_DIR as the directory in which exeConfigFile resides. See Executable Configuration File for Stone. |
|
|
For encrypted extents, use the given passphrase to read the private key specified by ‑D. |
|
|
For encrypted extents, use the passphrase contained in file passphraseFile to read the private key specified by ‑D. passphraseFile must be the full path and filename. |
|
|
For encrypted extents, specifies one or more colon‑separated directories which contain keys and certificates. May be included more than once. |
|
|
Write output to the file with the given name. The file is created if it does not exist. If there is an existing file with this name, the pageaudit output is appended to the end of this file. |
|
|
Run with the specified number of threads. The default is numExtents + numCpus. |
|
|
The GemStone system configuration file, or a directory containing a specifically named file. If used with ‑E, must be a file. See System Configuration File for Stone. |
Audit the pages in a GemStone repository, which must not be in use. pageaudit opens the repository specified by the relevant configuration files. The arguments ‑e exeConfig, and ‑z systemConfig determine which configuration files pageaudit reads.
pageaudit by default runs aggressively, using a large number of threads (based on the number of CPUs and extents for this system), in order to complete as quickly as possible; the ‑n option can be used to specify fewer threads, and reduce the impact on your system.
By default, all pages in the repository are verified; this includes data pages as well as Object Table, bitmap, and other pages containing internal information. Audit of data pages can be disabled using the ‑d option.
An error is returned if another Stone is running as gemStoneName or has opened the same repository.
When you include the ‑f switch, pageaudit prints all errors possible. Without ‑f, the default is to stop after the first error is found.
pageaudit writes a number of log files, including a stone log. This stone log includes the detailed audit results.
This utility can take a long time to run, so it is best to run it as a background job.
For additional information about pageaudit and a description of its output, see Page Audit.
printlogs
printlogs [ ‑h ] [ arguments, see below ] [ filters, see below ] all | tlog1...tlogN
Arguments
Search filter criteria:
printlogs prints out the contents of transaction logs in human‑readable form. This may produce a very large amount of output; without filters, the output will be at least as large as the original and may be multiple times larger depending on the options used.
Tranlogs to be printed are by default named tranlogNNN.dbf. This can be overridden by setting the environment variable $GS_TRANLOG_PREFIX to the tranlog prefix.
When using multiple filters, records that match any of the filters are printed. Each filter key can be used only once.
A maximum of 256 transaction logs can be analyzed at once.
Examples
To print out the entire contents of all tranlogs in the current working directory:
printlogs all
To print out all entries in a selected number of tranlogs:
printlogs tranlog5.dbf tranlog6.dbf tranlog7.dbf
To print out all tranlog entries for the user DataCurator in any tranlog:
printlogs user DataCurator all
To print out detailed information for all entries in tranlog5.dbf for the user DataCurator:
printlogs full user DataCurator tranlog5.dbf
pstack
pstack [ ‑b | ‑c | ‑C | ‑h ] processPid
The pstack command attaches a debugger to the process with the given pid, outputs the C and Smalltalk stacks of that process to stderr, and detaches. The Smalltalk stack is printed to the Gem log or topaz linked session console.
Execution of the running process briefly pauses while the debugger is attaching, but the process will subsequently continue running normally.
pstack is similar to functions provided with some OS platforms.
By default, the stack summary is printed with one line per frame for each thread, followed by the complete stack details. The ‑b and ‑C options allow you to specify brief format stacks.
By default, both C stack and (if there is a Smalltalk context) Smalltalk stacks are printed. The ‑c and ‑C options allow you to omit the Smalltalk stack.
removedbf
|
The GemStone repository filename or the raw partition for the repository to be removed. |
|
This command removes (erases) a GemStone repository file. It is provided primarily for erasing an extent or transaction log from a raw partition, but it also works on files in local or NFS‑mounted file systems. It does not work for disks on remote file systems.
If you specify a file in the file system, this command is equivalent to the rm command. If you specify a raw disk partition, GemStone metadata in the partition is overwritten so that GemStone will no longer think there is a repository file on the partition.
For example, to remove an extent on the raw partition /dev/rsd3h:
removedbf /dev/rsd3h
This command does not disconnect an extent from the logical repository. To alter the configuration by disconnecting an existing extent, see To Remove an Extent.
searchlogs
searchlogs [ ‑h ] [arguments, see below] [filters, see below] oop1...oop2
Arguments
Seach Filters:
Searchlogs searches all tranlogs in the current directory for selected oops, filtering results by user, host, and other criteria. Filters are optional.
Tranlogs to be searched are by default named tranlogNNN.dbf. This can be overridden by setting the environment variable $GS_TRANLOG_PREFIX to the tranlog prefix.
When using multiple filters, records that match any of the filters are printed. Each filter key can be used only once.
A maximum of 256 transaction logs can be analyzed at once.
secure_backup_extract
secure_backup_extract ( cert | sig ) secureBackupFile outputFile
|
destination in which to write the script results. This must not specify an existing file. |
The secure_backup_extract utility provides a way to verify the signature of a secure backup, that can be used on secure backup files from any earlier version (the secure backup feature was added in v3.4), without requiring a GemStone environment.
secure_backup_extract extracts the public signing cert or the digital signature from a secure backup file, and writes it to a new file. While $GEMSTONE is not needed, you must have openssl and perl available.
This script produces the same result as copydbf, using copydbf ‑X to extract the public signing cert, or copydbf ‑Y to extract the digital signature. To verify the digital signature, compare the binary files use the unix utility cmp.
Example
For example, if you have a secure backup file backup.sdbf that was signed using GemStone's example private cert backup_sign_2_clientkey.pem, either of the following will extract the public signing cert:
secure_backup_extract cert backup.sdbf signingCert.secbkext
copydbf ‑X signingCert.copydbf backup.sdbf
This can be compared with each other, or against the signing cert used to make the backup; for example:
cmp ‑l signingCert.secbkext $GEMSTONE/examples/openssl/certs/backup_sign_2_clientcert.pem
To extract the digital signature itself:
secure_backup_extract sig backup.sdbf digitalSig.secbkext
copydbf ‑Y digitalSig.copydbf backup.sdbf
which can be compared for validation:
cmp ‑l digitalSig.secbkext digitalSig.copydbf
startcachewarmer
startcachewarmer [ ‑d | ‑D ] [ ‑h ] [ ‑l limit ] [ ‑L path ] [ ‑n numSessions ] [ ‑s stoneName ] [ ‑w writeInterval ] [ ‑W ] [ ‑C midCacheSizeKb ] [ ‑e gemConfigPath ] [ ‑H stoneHost ] [ ‑M midCacheHost ] [ ‑N midCacheMaxProcs ]
The startcachewarmer command warms up the shared page cache on startup, by preloading object table pages and optionally data pages into the cache. This allows the overhead of initial page loading to occur in a controlled way on system startup, rather than more gradually as the repository is in use.
The object table and dependency map pages are always loaded; data pages are loaded based on the ‑d flag or the presence of the working set file.
- If the working set file /opt/gemstone/locks/stoneName~hostid.workingSet.lz4 exists, the valid data pages in this file are loaded.
- If the working set file does not exist, then behavior depends on the use of the ‑d and ‑D options:
- If the ‑d option is specified, all data pages in page order are loaded.
- With the ‑D option, data pages are loaded into the OS file buffer but not into the shared page cache.
The ‑d and ‑D flags are mutually exclusive, if both are specified, then the later one is used.
Cache warming writes messages to stone log when it starts and with status when it completes. When the ‑W option is specified, it will also write the results to the console. If an error occurs during cache warming, details are preserved in a file named stonename_cachewarmer.log in the directory from which this utility was executed.
If a remote shared page cache is to be warmed (i.e., the ‑H option is used), then the remote cache will be created if it does not already exist. The configuration used to start the remote cache are controlled by the ‑e argument, a gem.conf in the current directory, or the GEMSTONE_EXE_CONF environment variable.
If a mid‑level cache host name or IP address is specified (via ‑M), the mid‑level cache will be created if it does not already exist. The ‑C and ‑N options will be used to specify the size and number of processes that can attach the mid‑cache respectively. If the mid‑cache already exists, the ‑C and ‑N options are ignored.
For greater efficiency, you can set the configuration file parameters STN_CACHE_WARMER_ARGS, GEM_CACHE_WARMER_ARGS, and GEM_CACHE_WARMER_MID_CACHE_ARGS to invoke cache warming automatically on startup.
startlogreceiver
startlogreceiver ‑P listeningPort ‑A listeningAddress ‑T tranlogDir+ [ ‑l logFile ] [ ‑s stoneName ] [ ‑p alternatePort ] [ ‑d | ‑D ] [ ‑t timeoutSeconds ] [ ‑f flushIntervalSecs ] [ ‑C certFileName ] [ ‑J certAuthFileName ] [ ‑K keyFileName ] [ ‑Q passphrasestring ] [ ‑S ] [ ‑V ]
This utility starts up a logreceiver process. As part of a hot standby setup, the logreceiver process runs on the slave system to receive transaction log records, as they are generated, from a logsender process that is running on a master system.
The received transaction log records are written to files in a specified directory on the slave system. The slave system can run continuousRestoreFromArchiveLogs: to restore these transaction log records.
The logreceiver writes logging output to a file with the path and name or in the directory specified by the ‑l argument. If this file is a directory, the file name is logreceiver_listeningPort.log. If no ‑l argument is used, it writes to /opt/gemstone/log/logreceiver_listeningPort.log. If a file with the specified name exists, it is appended to. It is not deleted on process exit.
The logreceiver process will continue running until explicitly stopped using the stoplogreceiver command. If connection to the stone or the logsender process is lost, it will attempt to reconnect.
gslist will report running logreceiver processes.
To start the logsender using SSL to establish a secure connection between logsender and log receiver, both startlogsender and startlogreceiver can be provided with SSL credentials. When SSL arguments are provided, startlogreceiver will start the logreceiver in secure mode, requiring an SSL connection with the logsender. This is described under Connecting using SSL Mode
startlogsender
startlogsender ‑P listening port ‑A listeningAddress+ ( ‑T tranlogDir+ | ‑s stoneName ) [ ‑l logFile ] [ ‑d ] [ ‑g ] [ ‑t numberOfThreads ] [ ‑C certFileName ] [ ‑J certAuthFileName ] [ ‑K keyFileName ] [ ‑Q passphrasestring ] [ ‑S ] [ ‑V ]
This utility starts up a logsender process. As part of a hot standby setup, the logsender process runs on the master system to transmit transaction log records, as they are generated, to a logreceiver process that is running on a slave system. See Hot Standby for complete information on setting up a hotstandby system.
The logsender writes logging output to a file with the path and name or in the directory specified by the ‑l argument. If this file is a directory, the file name is logsender_listeningPort.log. If no ‑l argument is used, it writes to /opt/gemstone/log/logsender_listeningPort.log. If a file with the specified name exists, it is appended to. It is not deleted on process exit.
The logsender process will continue running until explicitly stopped using the stoplogsender command. If connection to the stone is lost, it will attempt to reconnect.
gslist will report running logsender processes.
To start the logsender using SSL to establish a secure connection between logsender and log receiver, both startlogsender and startlogreceiver can be provided with SSL credentials. When SSL arguments are provided, startlogsender will start the logsender in secure mode, requiring an SSL connection with the logreceiver. This is described under Connecting using SSL Mode.
startnetldi
startnetldi [ netLdiName ] [ ‑g | [ ‑s ] [ ‑k keytab ] ] [ ‑a name ] [ ‑A addresses ]+ [ ‑b maxBacklog ] [ ‑d ] [ ‑H ] [ ‑X nrs ] [ ‑D directoryPath ] [ ‑l logFile ] [ ‑n ] [ ‑P portNumber ] [ ‑r ] [ ‑t numThreads ] [ ‑E exeConfigFile ]
startnetldi [ x509NetLdiName ] [ ‑b ] [ ‑d ] [ ‑E configFileName ] [ ‑l logFile ] [ ‑n ] [ ‑X nrs ] [ ‑H ] [ ‑P portNumber ] [‑r ] ‑D directoryPath ‑S ‑J certAuthFileName ‑R keyFileName ‑U certFileName ‑L crlFileName
|
The name or port number of the GemStone network server. |
|
|
Captive account; all child processes created by the NetLDI will belong to the account named name. By default, child processes belong to the client’s account. |
|
|
Address to listen on. Up to 10 arguments are accepted. If no ‑A arguments are provided, listening is on the default wildcard address ::. If this default wildcard address is included, then other ‑A arguments are ignored. If the ‑A entry arguments do not include :: or ::1, then ::1 is also listened on. |
|
|
Debug mode; inserts more extensive messages in the log file. You can modify the debug mode during runtime using netldidebug. |
|
|
Specifies a directory that will be used to compose log file paths for process log files. It is the default log path if the NRS does not include a #dir directive. This directory must exist and be writable, and the $GEMSTONE_NRS_ALL in the startnetldi’s environment must not include a #dir directive. If the NRS includes the %D pattern in the #dir or #log, when spawning a child process, the %D in the NRS is replaced with directoryPath before forking. Path separator is appended if necessary. |
|
|
A GemStone executable configuration file, which will be provided to RPC Gems spawned by this NetLDI, if the NRS does not contain a ‑E argument. Suppresses use of environment variables and default configuration file locations. Recommended for use in RPM installations. See Executable Configuration File for RPC Gem. |
|
|
Guest mode; no accesses are authenticated. This option is not allowed if the NetLDI’s effective user id is the root account. Not compatible with ‑k. |
|
|
Do reverse DNS lookup on IP addresses of hostnames, and print both name and IP address in the log. |
|
|
Enable Kerberos and set the keytab file name; logins with empty host password will authenticate using Kerberos. Not compatible with ‑g. |
|
|
The logged output of the NetLDI; the default is |
|
|
Do not allow any ad hoc processes to be created (ad hoc processes are ones not listed in $GEMSTONE/sys/services.dat). |
|
|
Use numeric IP addresses for printing peer info in logs. This is the default. |
|
|
If there is a running NetLDI (version 3.3 or above) with this name, and that NetLDI has different version than specified in the current environment, stop the running NetLDI and restart it. |
|
|
Start the NetLDI with numThreads threads. For an X509‑Secured NetLDI, or if the NetLDI will be running as root, only uses one thread. Default 10, min 1, max 500. |
|
|
Additional arguments for use only when starting a X509‑Secured NetLDI: |
|
|
For use when starting a X509‑Secured NetLDI on a remote node (not the Stone’s node). Provides specifications for starting the remote cache. The actual cache startup is initiated by starthostagent on the Stone's node. See the GemStone/S 64 Bit X509‑Secured GemStone System Administration Guide. |
|
|
Specifies a certificate authority certificate (CA) in PEM format to use. Requires ‑S. |
|
|
Specifies a certificate revocation list (CRL) file in PEM format. |
|
|
Specifies host private key in PEM format to use. Requires ‑S. |
|
|
Start a X509‑Secured NetLDI. Certificates must also be provided using ‑U, ‑R, and ‑J, and ‑e is required for remote caches. The NetLDI will do mutual authentication for all connections. |
|
|
Specifies a host X509 certificate in PEM format to use. Requires ‑S. |
|
|
Specifies an NRS containing one or more of #log:, #dir:, and #netldi:, which take precedence over an GEMSTONE_NRS_ALL setting in the environment of the startnetldi. |
|
This command starts a GemStone network server with the specified netLdiName and timeout (given in seconds). The server spawns GemStone processes in response to login requests from remote applications and requests for remote repository access from Stone repository monitor processes. If your machine is slow or heavily loaded, and RPC logins time out before completing, specify a larger timeout value.
Legal characters in a NetLDI name are A‑Z, a‑z, 0‑9, and the characters period, underscore, and dash (. _ ‑). NetLDI names with other characters are disallowed. NetLDI names may not start with period, dash, or underscore. Also, NetLDI names must not match an existing system service, such as ldap, syslog, etc.
The NetLDI listens for requests, including RPC login requests, on a specified or configured port and optionally on a specific address. During the RPC login process, it uses a separate set of ports to establish communication between the Gem and its client. If you are running over a firewall, you can specify the port using the ‑P option, and configure your firewall to permit access via this port.
The locations and names of log files for process started by the NetLDI are defined by NRS directives in the login parameters or by settings in the GEMSTONE_NRS_ALL.
Using the ‑D logfile argument to startnetldi sets a default log file path, which is used instead of the user’s home directory for gem log files. For more on managing log file paths, see Controlling log file directory locations.
NetLDI can be configured to authenticate starting processes, or all accesses. It may configure to authenticate the UNIX userId (including by using Kerberos) or to run in guest mode, which does not authenticate. These options work in conjunction with permissions for the extent files to provide security while also ensuring the appropriate access to GemStone for authorized users. How to configure the NetLDI is described in Chapter 4, “NetLDI and Interprocess Access”.
To use startnetldi within an X509‑Secured GemStone configuration, see the GemStone/S 64 Bit X509‑Secured GemStone System Administration Guide.
For assistance with startup failures, refer to To Troubleshoot NetLDI Startup Failures.
netldid
$GEMSTONE/sys/netldid is the executable that is invoked by startnetldi. While startnetldi blocks until startup is complete, then returns control to the command line, executing netldid directly does not return. This can allow you to run the NetLDI as a background process.
The options described for startnetldi are also available for netldid, with the exception of ‑r.
startstone
startstone [ gemStoneName ] [ ‑l logFile ] [ ‑e exeConfig | ‑E exeConfigFile ] [ ‑z systemConfig ] [ ‑R ] [ ‑N ] [ ‑H ]
startstone [ gemStoneName ] [ ‑l logFile ] [ ‑e exeConfig ] [ ‑z systemConfig ] [ ‑R ] [ ‑N ] [ ‑H ] ‑D privateKey ‑K dir [ ‑K dir+ ] [ ‑j passPhrase | ‑J passphraseFile ]
|
Name of the GemStone repository monitor, default is gs64stone. Network resource syntax is not permitted. |
|
|
For encrypted extents, specifies the private key file corresponding to the public key used to encrypt the extents. The file must be in PEM format. Requires ‑K, and if the private key has a passphrase, also ‑j or ‑J. |
|
|
The GemStone executable configuration file, or a directory containing a specifically named file. See Executable Configuration File for Stone. |
|
|
The GemStone executable configuration file.Only one of ‑e , ‑E can be used. When ‑E is specified, the ‑z argument must be a file, not a directory. Suppresses use of configuration file environment variables and default configuration file locations; defines an internal environment variable $GEMSTONE_DATA_DIR as the directory in which exeConfigFile resides, which is used by the configuration file template $GEMSTONE/bin/gemstone_data.conf. Recommended over ‑e (see discusson here) particularly for use in RPM installations. See Executable Configuration File for Stone. |
|
|
Do reverse DNS lookup on IP addresses of hostnames, and print both name and IP address in the log. |
|
|
For encrypted extents, use the given passphrase to read the private key specified by ‑D. |
|
|
For encrypted extents, use the passphrase contained in file passphraseFile to read the private key specified by ‑D. passphraseFile must be the full path and filename. |
|
|
For encrypted extents, specifies one or more colon‑separated directories which contain keys and certificates. May be included more than once. |
|
|
The location (or filename) for the logged output of the stone; the default is (1) the setting of the $GEMSTONE_LOG environment variable, if defined; (2) $GEMSTONE/data/gemStoneName.log |
|
|
Do not replay transaction logs as part of startup. Unless used with the ‑R option, and transaction logs are replayed manually, work may be lost. |
|
|
Start up from the most recent checkpoint in the extents, and go into restore mode so that transaction logs can be manually restored. Used when restoring from backup and for hot standbys. With this option, the stone does not create or write to a transaction log on startup. |
|
|
The GemStone system configuration file, or a directory containing a specifically named file. If used with ‑E, must be a file. See System Configuration File for Stone. |
This command starts up the GemStone repository specified by the configuration files, which may be specified using startstone arguments, environment variables, or using file default locations and names. For details on how startstone determines which the configuration file/s to use, see How GemStone Processes Find Configuration Files.)
Legal characters in a Stone name are A‑Z, a‑z, 0‑9, and the characters period, underscore, and dash (. _ ‑). Stone names with other characters are disallowed. Stone names should not start with period, dash, or underscore, and must contain at least one non‑digit. Also, Stone names must not match an existing system service, such as ldap, syslog, etc.
If the Stone did not shutdown cleanly, and transaction logs are available in the transaction log directories, startstone (without the ‑R or ‑N arguments), will automatically replay them starting from the last checkpoint. This recovery applies to systems in both full and partial transaction logging modes. Provided startstone completes without error, recovery was successful and no further action is needed. See To Troubleshoot Stone Startup Failures and Recovering from an Unexpected Shutdown for more information.
The ‑N option is intended for use when the repository needs recovery (was not cleanly shutdown), but no transaction logs can be found, or the transaction logs have become corrupted. The ‑N starts up the Stone without restoring any transaction logs. If your repository is in partial logging mode, any transactions committed after the last checkpoint in the extents are lost. If you are in full logging mode, ‑N is also used to setup hot standbys and for some manual restore operations. A new transaction log will be initialized as part of the startup.
The ‑R option is used when restoring from backup or setting up a hot standby. This starts up the repository at the point of the most recent checkpoint in the extents, and puts the repository into restore more. No recovery is done. If there are any transaction logs present in the transaction log directories, startup will fail. If you are manually restoring transaction logs, you must either include the ‑N option along with ‑R, or move the tranlogs to another location that is not on the list of tranlog directories. With ‑R, a new transaction log is not created. Use of the ‑R flag is described under How to Restore from Backup and Warm and Hot Standbys.
If you are using encrypted extent files, you must also specify the private key corresponding to the public key used to encrypt the extents, with ‑D, and the directory or directories containing public and private keys, with ‑K. If the private key has a pass phrase, you must also provide this with the ‑j or ‑J arguments. If the repository was uncleanly shutdown, the associated encrypted transaction logs must be encrypted with the same key as the repository extents, for the automatic recovery to succeed. However, if you are restoring tranlogs from backup, you may specify alternate private keys. See Chapter 12 for more details on using encrypted extents and tranlogs.
stoned
$GEMSTONE/sys/stoned is the executable that is invoked by startstone. While startstone blocks until startup is complete, then returns control to the command line, executing stoned directly does not return. This can allow you to run the Stone as a background process.
The options described for startstone are also available for stoned.
Return values
The startstone utility returns one of the following codes to the command line:
statmonitor
statmonitor stoneName [ options, see below ]
statmonitor records statistics for a repository and/or the operating system to a disk file.
statmonitor runs in the background, sampling specified data at a specified interval and recording the data to a text file. The data in this file can be viewed by the graphical application VSD, version 5.5 or later; v5.6 or later if using ‑Z to compress via lz4. See vsd and the VSD Users Guide for more information.
For details on the statistics that statmonitor records, see the VSD help (Main Window menu Main > Show Statistics Info).
Statistics are collected from the shared page cache and directly from the OS. Only data for GemStone processes attached to the shared page cache on the host on which statmonitor is running, are collected. To monitor Gems on systems with remote Gem servers, you must run statmonitor both on the Stone’s machine and on the Gem server machine.
The stonename argument is used to specify the cache to monitor, both for caches that are local to the Stone and caches that are remote from the Stone with that name. Configurations in which a single machine is hosting remote caches for multiple stones that are running with the same Stone name are ambiguous and will not work correctly; this configuration is strongly discouraged.
In addition to running statmonitor from the command line, you can also specify that statmonitor be automatically started at stone startup, and the startup for remote gem caches and remote mid‑level caches, to ensure statmonitor data is collected. See the configuration parameters STN_STATMONITOR_ARGS, GEM_STATMONITOR_ARGS, and GEM_STATMONITOR_MID_CACHE_ARGS.
Statmonitor Compression
Statmonitor can write compressed files, which is particularly critical for production systems performing continuous monitoring. Both gzip and lz4 compression are supported.
Note to uncompress an lz4‑compressed statmonitor data file on the command line, you must use an additional ‑D argument to provide the C dictionary used for optimization, which is $GEMSTONE/bin/lz4.statmon.dict.bin.
os$ lz4 ‑D $GEMSTONE/bin/lz4.statmon.dict.bin stats.out.lz4
Statmonitor Filenames and locations
Statmonitor by default writes to a file named statmonNNNNN.out, statmonNNNNN.out.gz, or statmonNNNNN.out.lz4., in the current directory from where statmonitor was started. NNNNN is the pid of the statmonitor process.
You can control both the directory and the filename, including pattern‑based file naming.
To use the default statmonitor data file name, but write the statmonitor files to a different directory, use the ‑d option. This cannot be used with the ‑f or ‑F filename options. To specify both directory and filename, include the directory as part of the file name specification.
To simply provide a filename and optionally directory, use the ‑f option. For example,
statmonitor ‑f /gsadmin/stats/ManagerStats gs64stone
If this file already exists, statmonitor will append an ‑N to the filename, where N starts at 1, or the next available number, incrementing as needed.
statmonitor will append the correct .out, .out.gz, or .out..lz4 extension to the specified filename, if it does not already exist, to ensure the file is correctly detected by VSD. It is recommended that you omit the extension, since using the incorrect ending will result in extra, possibly duplicate extensions.
The ‑F option allows you to specify a pattern used to generate the statmonitor filename. The following patterns are available:
GemStone specific filename pattern options:
%%C ‑ name of the shared page cache
%%i ‑ sample interval in seconds
%%I ‑ sample interval in milliseconds
Some useful values from stftime()
Note that this list does not include all possible values and may vary by operating system.
%b The abbreviated month name according to the current locale.
%B The full month name according to the current locale.
%c The preferred date and time representation for the current locale.
%d The day of the month as a decimal number (range 01 to 31).
%H The hour as a decimal number using a 24‑hour clock (range 00 to 23).
%m The month as a decimal number (range 01 to 12).
%M The minute as a decimal number (range 00 to 59).
%R The time without seconds in 24‑hour notation (%H:%M).
%S The second as a decimal number (range 00 to 60).
%T The time with seconds in 24‑hour notation (%H:%M:%S).
%y The year as a decimal number without a century (range 00 to 99).
Automatic restart
It is recommended to have statmonitor running at all times, since if an error occurs, the information leading up to the problem may be important.
There are a number of feature that make this process easy to automate.
Using the ‑r or ‑R option, statmonitor will automatically restart monitoring when an existing statmonitor stops itself (this does not apply if you manually stop statmonitor).
Statmonitor will stop itself if:
- When using the ‑h option, the given number of hours has passed
- When using the ‑t option, the given number of samples are recorded
- If a list of times was passed in using the ‑k option, one of these times was reached
With the ‑R option, on restart, a new filename is calculated based on the pattern. With ‑r, on restart, the same filename is used (appending the ‑N argument as needed)
Automatic deletion of older statmonitor files
When running statmonitor regularly, many data files are generated, and while older statmonitor files can be useful to examine changes in performance over time, you will need to establish systems for deleting excess files.
When using the ‑r or ‑R options, that provide for automatic restart and thus generate a series of files by the same statmonitor instance, you may use the ‑K aNumber option. This specified how many old files you wish to keep. The number specified does not include the statmonitor file that is currently being written.
Note that this will delete the older files that are part of a single series. When statmonitor is restarted (and statmonitor must be restarted if the Stone is restarted), it starts a new and distinct series. The remaining files in the older series are not automatically deleted. Depending on the ‑K argument, the frequency of starting new files, and the frequency of statmonitor restarts, many files may still require manual deletion.
Limiting what is recorded
Statmonitor by default records a moderate amount of data that is most commonly needed. It can be configured to record fewer kinds of data, or more kinds of data.
The following options limit the GemStone processes that are collected:
‑S — record stone and shrpcmon only
‑P — record stone, shrpcmon, and aio pgsrvs only
‑J — record stone, shrpcmon, and page manager only
‑G — record stone, shrpcmon, admin and reclaim gems only
‑T — do not record statistics on NetLDI processes.
‑ppid — do not record gem statistics other than for the specified PID.
‑UuserId — do not record gem statistics other than for processes owned by the given UNIX userID
‑Y — Disable collection of all system stats, including per‑process data.
The following collect additional host information
‑A — Collect all available system statistics.
‑a — Collect all available system statistics except per‑core CPU statistics.
‑C — Collect system statistics for each individual CPU.
‑D — Collect system statistics for all disks and partitions
‑N — Collect system statistics for all network interfaces.
‑Qpid — Collect system statistics for processIds, which do not need to be GemStone processes.
To not collect any GemStone statistics, but only statistics on the host machine itself, use:
This can be combined with the other host information statistics to manage the amount of host information collected.
Example
To run statmonitor on an application named plantis, with:
- 30 second sample rate
- files zipped using gzip
- collect all system statistics except CPU stats, as well as GemStone statistics
- automatic restart at 2am every day
- keep three old files but delete older files as new ones are generated
- each file has a filename that includes the date and time. The stone name is hard coded into the filename, but statmonitor will automatically add the correct zip extension.
The following expression can be used:
statmonitor ‑i30 ‑z ‑a ‑R ‑K3 ‑k'02:00' ‑F /gshost/GemStone3.7/statmon/plantis30sec‑%y%m%d_%H%M plantis
This creates files with a filename such as plantis30sec.200520_0500.out.gz.
Up to four files will be in the directory /gshost/GemStone3.7/statmon/ (this includes three older files and the current file).
Example 2
The previous example requires manual restart every time the stone is started. The following example shows setting up start statmonitor automatically every time the stone is started, using the STN_STATMONITOR_ARGS configuration parameter.
This example also sets up to collection two series of statistics; statistics with a small sample rate that are discarded after a week, and statistics with a longer sample rate that are kept indefinitely.
- 1 second sample rate
- collects system statistics as well as GemStone statistics)
- files zipped using LZ4
- new files are started every day at midnight; old files are deleted after one week
- each file has a filename that includes the stone name (the %%S pattern), and the date and time.
- 5 minute sample rate
- files zipped using LZ4
- new files are started weekly (after 168 hours); old files are kept forever.
- each file has a filename that includes the stone name (the %%S pattern), and the date.
The following is added to the Stone’s configuration file:
STN_STATMONITOR_ARGS =
"‑i1 ‑a ‑Z ‑R ‑k '00:00' ‑K7 ‑F /gshost/GemStone3.7/statmon/%%S‑dailyStats_%F_%H‑%M",
"‑i360 ‑Z ‑R ‑h168 ‑F /gshost/GemStone3.7/statmon/%%S‑weeklyStats_%F";
When the stone is started, this creates files with a filenames such as:
plantis‑dailyStats_2023‑08‑12_16‑23.out.lz4
plantis‑weeklyStats_2023‑08‑12.out.lz4
statprom
statprom ‑f configFile [ ‑c ] [ ‑d ] [ ‑r ]
This utility sets up a process that listens for connection from Prometheus, and provides cache statistics from the shared cache to Prometheus. Specifications of the process types and statistics to be monitored are provided in a JSON format file.
For more information, see the supplemental documentation: https://docs.gemtalksystems.com/current/monitorwithprometheus
Configuration file JSON format
A sample JSON file, that could be used as an argument to statprom, is included in $GEMSTONE/examples/Prometheus/stats.json. This provides an example of the main features of statprom configuration.
“http” Section
-
listen_addresses
This argument povides informatoin on the port that statprom will listen on for connections from Prometheus. It is an Array of strings representing port numbers and interfaces/addresses. Both IPv4 and IPv6 addresses are supported. The default port number for statprom is 9995. The address format is specific to the civetweb webserver, see: https://civetweb.github.io/civetweb/UserManual.html
“gemstone” Section
Only one Stone or one remote cache can be monitored by a statprom process.
-
cache_name
a String which represents the name of the remote shared page cache to monitor or NULL. Only relevant when monitoring a remote shared page cache; this must be NULL when monitoring the Stone’s cache. The cache name may be obtained from gslist. -
stone_name
a String which represents the name of the Stone to monitor or NULL. Only relevant when monitoring a primary shared page cache; should be NULL when monitoring a remote cache. -
sample_interval
an Integer representing the sample interval, in seconds.
“metrics” Section
Three types of metrics are supported: monitor (shared page cache monitor), stone, and gem. For each type, there should be an array of specific metrics to be monitored for that type. The JSON metrics objects have the following members:
-
vsd_name (required)
a String which represents the vsd name of the statistic to monitor. Must exactly match the name of the statistic. It is an error if the statistic name does not exist for the given metric type. Note that only statistics that originate in the cache can be monitored; host statistics cannot be monitored. -
metric_name (required)
a String which represents the name of the statistic in Prometheus. All values of metric_name in the configuration file must be unique. This is a limitation of Prometheus which does not allow duplicate names. -
metric_type (required)
a String representing the Prometheus type for the stat. Accepted values are: - Counter – values that only increase and never decrease
- Gauge – values that may either increase or decrease
- Histogram – values that will be plotted as a histogram.
-
metric_help (required)
a String which describes the function of the statistic. -
metric_units (required)
a String which describes the measurement units of the statistic.
Since there may be multiple Gems in a cache, and Prometheus does not support multiple instances, special configuration is required to distinguish specific Gems.
In addition to the above object members, Gem entries require either cache_name_regex, or both gem_kind_min and gem_kind_max, or all three.
-
cache_name_regex
a String which represents a regex expression used to match against the cache names of all gems in the cache. -
gem_kind_min
an Integer indicating the minimum value of the GemKind statistic used to match against gems in the cache. Requires gem_kind_max be specified. -
gem_kind_max
an Integer indicating the maximum value of the GemKind statistic used to match against gems in the cache. Must be greater than or equal to gem_kind_min. Requires gem_kind_min be specified.
stoplogreceiver
stoplogreceiver ‑P listeningPort
This utility stops a logreceiver process that was started by a previous startlogreceiver command. This may only be executed on the same node as the logreceiver is running.
stoplogsender
stoplogsender ‑P listeningPort
|
The port on which this logsender is listening. It must be the same port as specified in the ‑P argument of the startlogsender. |
|
This utility stops a logsender process that was started by a previous startlogsender command. This may only be executed on the same node as the logsender is running.
stopnetldi
|
The name of the GemStone network server; the default is gs64ldi. Network resource syntax is not permitted. |
|
Gracefully stop a GemStone network server.
If name is not specified, the name of the NetLDI to stop is taken from GEMSTONE_NRS_ALL environment variable, if defined there. If GEMSTONE_NRS_ALL does not define a NetLDI name, then a NetLDI with the default name gs64ldi is stopped.
If a portNumber was used to start the netldi process, then the portNumber can be used to stop the netldi process.
Return values
The stopnetldi utility returns one of the following codes to the command line:
stopstone
stopstone [ gemStoneName [ gemStoneUserName [ gemStonePassword ] ] ] [ ‑i ] [ ‑t ] [ ‑h ]
Gracefully shut down a GemStone repository monitor. In the process, a checkpoint is performed in which all committed transactions are written to the extents.
If you specify the ‑i (immediate) option, the repository monitor is shut down even if there are GemStone users logged in. The ‑t option specifies how long to wait for all session to detach from the cache before returning; if omitted, stopstone will wait forever.
If you do not supply the gemStoneName, gemStoneUserName, and gemStonePassword on the command line, this command will prompt you for them.
Because this command uses a Gem session process to connect to gemStoneName, it fails if the Gem is unable to connect for any reason, such as inability to attach a shared page cache. For assistance, refer to To Troubleshoot Session Login Failures.
Return Values
stopstone returns one of the following codes:
- 0 (success) Successful stop
- 1 (informational) No running stone with gemStoneName
- 3 or above (error) an error occurred and gemStoneName was not stopped.
- 10 syntax error
- 11 bad user or password
- 12 gemStoneUserName does not have privilege to stop the stone.
- 13 stone not stopped; other users are logged in. Use ‑i to override.
topaz
topaz [ ‑r ] [ ‑q ] [ ‑i | ‑I topazini ] [ ‑S scriptFile ] [ ‑n hostName:netldiName ] [ ‑u useName ] [ ‑X caCertPaths ] [ ‑‑ other args ]
topaz ‑l | ‑L [ ‑q ] [ ‑i | ‑I topazini ] [ ‑e exeConfig | ‑E exeConfigFile ] [ ‑z systemConfig ] [ ‑T tocSizeKB ] [ ‑C configParams ] [ ‑S scriptFile ] [ ‑u useName ] [ ‑‑ other args ]
|
Provides configuration parameters, in configuration file syntax, that override settings in the configuration files. Only applies to linked sessions (RPC sessions may use the ‑C syntax in the Gem’s NRS). |
|
|
The GemStone executable configuration file or directory; only valid for linked sessions. Only one of ‑e, ‑E can be used. See Executable Configuration File for Linked Gem. |
|
|
The GemStone executable configuration file; only valid for linked sessions. Only one of ‑e, ‑E can be used. When ‑E is specified, the ‑z argument must be a file, not a directory. Suppresses use of configuration file environment variables and default configuration file locations; recommended for use in RPM installations. See Executable Configuration File for Linked Gem. |
|
|
Specify a complete path and file to a topazini initialization files, and use this rather then any .topazini in the default location. |
|
|
Invoke the linked version of Topaz, and do not apply any command set gemnetid that may appear in the .topazini file or a file passed in using ‑I. |
|
|
For a login using X509‑Secured GemStone, to specify the NetLDI to spawn the Gem, part of the X509‑secured GemStone login parameters. The host name or IP, and the netldi name or listening port, for an X509‑secured NetLDI. |
|
|
Start Topaz in quiet mode, suppressing printout of the banner and other information. |
|
|
Specifies a script file that will be processed with INPUT, suppressing output except if an error occurs. |
|
|
The GEM_TEMPOBJ_CACHE_SIZE that will be used. Overrides any settings provided in configuration files passed as arguments with the ‑e or ‑z options. Only applies to linked sessions. |
|
|
Sets the cache name, as recorded by statmonitor for viewing in VSD. This is also useful for identifying processes in OS utilities such as top or ps. |
|
|
On Windows client only; forces terminal behavior regardless of I/O device. |
|
|
For a login using X509‑Secured GemStone, to set certificates. Requires additional infrastructure to be running, including X509‑secured NetLDIs and caches. The argument specifies the CA cert, user cert, and user private key with 3 paths in this order: 'stoneCA‑dev.cert.pem;DataCurator.chain.pem;DataCurator.privkey.pem' |
|
|
The GemStone system configuration file or directory; only valid for linked sessions. Must be a file if used with the ‑E option. See Executable Configuration File for Linked Gem. |
|
|
Arbitrary text arguments otherArgs may be included after the “‑‑” end of arguments marker, which must follow any of the above topaz arguments are included. |
This command invokes various forms of topaz. Topaz is described in details in a separate manual, the Topaz Programming Environment.
By default, the remote procedure call (RPC) version of topaz is opened, the equivalent of topaz ‑r. To invoke the linked version of topaz, which is recommended or required for some maintenance operations, use the ‑l or ‑L option. Settings within topaz can allow linked topaz to perform RPC logins. The advantage of the ‑L option is that a .topazini initialization file command cannot inadvertently configure the login to be RPC rather than linked.
Several topaz arguments only apply in linked topaz. Linked topaz uses a configuration file to configure its Gem, this can be specified using command line arguments, environment variables, or rely on default configuration file names and locations, or on default Gem configuration parameters. See Linked Gem configuration files for more information.
Topaz is available for Windows, in addition to regular server platforms, as part of the Windows Client installation. Options are more limited, since linked logins are not possible.
updatesecuredbf
updatesecuredbf [ ‑F ] [‑d] [ ‑O ] extentFile ‑e newEncrCert ‑D oldPrivEncrKey ‑K keyRingDir [ ‑K keyRingDir+ ] [ ‑j encrPP | ‑J encrPPFile ]
updatesecuredbf tranlogFile [ ‑F ] [‑d] ‑e newEncrCert ‑D oldPrivEncrKey ‑K keyRingDir [ ‑K keyRingDir+ ][ ‑j encrPP | ‑J encrPPFile ]
updatesecuredbf backupFile [ ‑F ] [‑d] ‑e newEncrCert ‑D oldPrivEncrKey ‑K keyRingDir [ ‑K keyRingDir+ ] ‑S sigPrivKey [ ‑j encrPP | ‑J encrPPFile ] [ ‑t sigPP | ‑T sigPPFile ]
updatesecuredbf accepts an encrypted extent, tranlog file, or full backup file, and replaces the private key with a new private key. the modification is done in place. It is strongly recommended to make a backup copy of the file prior to executing updatesecurebackup.
In order to decrypt the dbf file, required arguments include the private key corresponding to the public key that was used to encrypt the database file previously, and if this key has a passphrase, this must also be provided.
In addition, the new public key that will be used to re‑encrypt the file is required. Since encrypted backups are always signed, additional arguments when using updatesecuredbf for a backup are required to provide the signing key and passphrase.
When using updatesecuredbf with encrypted backups, which may have up to 8 encryption keys, any single one of the keys can be replaced using updatesecuredbf. The other keys are not affected. Note that encrypted backups, which are always signed, also require the signing key and passphrase.
For example, for a backup that was encrypted using the backup_encrypt_1_clientcert.pem with passphrase backup_encrypt_1_client_passwd.txt, the following will replace that with backup_encrypt_3_clientcert.pem.
os$ updatesecuredbf backupEn1.sdbf
‑e backup_encrypt_3_clientcert.pem
‑D backup_encrypt_1_clientkey.pem
‑J $GEMSTONE/allcerts/backup_encrypt_1_client_passwd.txt
‑S backup_sign_2_clientkey.pem
‑T $GEMSTONE/allcerts/backup_sign_2_client_passwd.txt
‑K $GEMSTONE/allcerts
For more information on encrypted extents, transaction logs and backup files, and using copydbf and updatesecuredbf to manage them, see Chapter 12, “Encrypted Extents and Transaction Logs”.
verify_backup_with_openssl
verify_backup_with_openssl [ ‑h ] backupFile [ publicKeyPathAndFile ]
|
The full back to the public certificate file corresponding to the private key that was used to sign backupFile. |
Use openSSL to verify the backup file backupFile, with the public certificate publicKeyPathAndFile. Note that for verfication, you cannot use just a public key, the certificate containing the key is required.
The public certificate can be omitted, in which case it performs verification using the public key extracted from the private key, which is not useful for verification, and only demonstrates that the backup is signed.
vsd
vsd [ ‑b color ] [ ‑u | ‑a | ‑A ] statmonDataFile+ ] [ directory ]
|
Append mode. Files following the ‑a are appended to the first one; the data in these files is merged so statistics are viewed as if they were a single file. |
|
|
Like ‑a, and also enable continuous appending with auto‑append mode. |
|
|
Set the master background color. Color may be a hex RGB value (e.g.: #d3d3ff) or a valid color name (e.g.: white). The list of recognized color names may be found at: http://www.tcl.tk/man/tcl8.6/TkCmd/colors.htm |
|
|
The name of a statistics data file to load into the VSD as it is started. |
|
|
directory is the name of a directory; VSD will set this as the working directory for the executable, and after startup, opens the Open Statmonitor File dialog on this directory. |
This command starts VSD, the graphical tool to view and analyze statmonitor data. The default is to open an empty instance of VSD, into which you can then load one or multiple statmonitor data files. After loading data files, you may select one or multiple GemStone processes, and view charts on one or multiple statistics for these processes.
VSD is available for Windows, in addition to regular server platforms, as part of the Windows Client installation. VSD can also be downloaded independent of GemStone/S 64 Bit, see https://gemtalksystems.com/products/vsd/.
For a complete description of VSD, see the VSD User’s Guide.
waitstone
waitstone [ ‑h ] [ gemStoneName | netLdiNameOrPort ] [ timeout ] [ waitForWarming ]
This command reports whether the GemStone repository gemStoneName is ready to accept logins or whether netLdiName is ready to accept requests. If neither gemStoneName nor netLdiName are specified, waitstone will report on the default stone name, gs64stone.
Waitstone checks every 0.5 seconds to see if the Stone or NetLDI is ready. When the service is ready, waitstone issues a message to stdout. If the service is not ready by the time the specified number of seconds have elapsed, waitstone reports an error.
With a ‑1 timeout, waitstone will make one connection attempt to the Stone or NetLDI. Note that if the stone is inaccessible, the timeout may be longer, since the configured TCP/IP timeout may apply. This can result in a 20 second timeout regardless of the specified timeout value.
Use of timeout requires that the Stone or NetLDI name is also specified.
If a positive value for the waitForWarming argument is specified, waitstone will block until cache warming completes; this is useful when cache warming is automatically run on startup. The specific value of the waitForWarming is not important. Use of the waitForWarming positional argument requires specifying each of the preceding arguments.
You may specify the gemStoneName and netLdiName arguments as a GemStone network resource string. (See Appendix C, “Network Resource Strings (NRS)”)
This command returns 0 exit status if the operation is successful; otherwise, it returns a non‑zero value.