10. Managing Transaction Logs
Transaction logs hold records of changes to the repository, which allow you to recover any changes made between backups. This chapter describes how to manage these transaction logs.
10.1 Overview
A transaction log contains the information necessary to redo transactions to the repository that have been committed by GemStone sessions since the last checkpoint or orderly shutdown. This log is used to recover from crashes such as those caused by a power failure, an operating system failure, or the killing of GemStone monitor processes.
If you need to restore the repository from a backup, transaction logs written in full-logging mode can be used to recreate all transactions committed since the most recent backup was written.
The transaction log is implemented as a sequence of files having names of the form tranlog1.dbf ... tranlogNNN.dbf. The numeric fileId starts at 1 when the Stone starts with a copy of the initial repository extent ($GEMSTONE/bin/extent0.dbf). If the Stone starts on an existing repository without any logs present, the fileId will be one greater than when the repository was last shut down cleanly.
The filename prefix, by default “tranlog”, can be controlled by setting the STN_TRAN_LOG_PREFIX configuration option.
The transaction logs are written to a list of directories or raw partitions specified by the STN_TRAN_LOG_DIRECTORIES configuration option, which is treated as a circular list. Each log is limited to the size set for that directory or raw partition by the STN_TRAN_LOG_SIZES configuration parameter. When one log is full, logging switches to the next directory or raw partition. (What happens when logs have been created in all directories is discussed in Table 10.1) Collectively, the transaction log files logically form an extremely large sequential file with a maximum size of 4 x 106 GB.
As users commit changes to the repository, GemStone writes the changes to a transaction log. Periodically, the stone repository monitor performs a checkpoint, at which point all committed changes are guaranteed to be written to the extents. During the periods between checkpoints, the transaction logs hold the record of the changes, so they are available if an unexpected shutdown occurs.
Use ordinary UNIX utilities to backup the transaction logs in the file system. To backup a transaction log that is on a raw disk partition, use copydbf to copy it to a file system. You’ll also need to use removedbf to clear the partition for reuse.
Tranlogs with Encrypted Extents
Transaction logs written by a Stone that is using encrypted extents are written in encrypted form, using the same keypair as the extents. When the Stone is started, if recovery is needed, it will be able to read the transaction logs for recovery, since they were encrypted using the same key as was used to start the Stone. However, extra care must be taken when changing the encryption key for the Stone.
If the encryption key of the extents have changed, on startup after a clean shutdown, the Stone will start a new transaction log that is automatically encrypted with the new key. In this case, you must keep the older keypairs, or modify the older tranlogs to use the new encryption key, if you want to be able to restore these older tranlogs into an older backup.
Logging Modes
GemStone provides two modes of transaction logging, selected by setting the STN_TRAN_FULL_LOGGING configuration option:
- To provide real-time incremental backup of the repository, set STN_TRAN_FULL_LOGGING to True. All transactions are logged regardless of their size. This mode is strongly recommended for deployed GemStone systems.
- To allow a simple operation without critical data to run unattended for an extended period, set STN_TRAN_FULL_LOGGING to False. This mode, known as partial logging, provides automatic recovery from system crashes that do not corrupt the repository, such as fatal errors or loss of power. However, if you do experience extent corruption and need to restore from backup, you cannot recover any changes made after the backup was taken.
Table 10.1 compares full and partial transaction logging.
In the event of a system crash, GemStone can recover by automatically replaying transactions recorded in the transaction log, from the latest checkpoint before the crash to the end of the log at the moment of the crash (Figure 10.2). This allows you to restart from where you were at the time of the crash.
No special administrative action is required to perform this recovery. Every time the system does a normal startup, it checks to see if there are transactions that need to be recovered. Thus, if the extents and transaction logs are good, the system will automatically recover from a crash.
Restoring Transactions to a Restored Backup
An important use of transaction logs is to restore transactions that were committed between the last full backup and a system failure. If you experience system failure and your current extents are no longer usable, you can still recover all data, provided that the repository already is in full transaction logging mode and that the backup was made in that mode.
For details on restoring transactions to a backup, see How to Restore Transaction Logs.
How the Logs Are Used
You can only use transaction logs to restore your system from backup if you are in full logging mode.
After creating a backup, you need to retain all the transaction logs that are created during and after the backup. To determine the oldest transaction log that will be required, use the copydbf utility to query the backup file. For example:
For a programmatic full backup
os$ copydbf -i backup.dat
File type: backup fileId: 0 in a backup set with 1 files
File size: 25952256 bytes (24 MB), 198 records
ByteOrder: Intel (LSB first) compatibilityLevel: 860
The file was created at: 08/07/23 18:16:35 PDT
Full backup started from checkpoint at: 08/07/23 18:16:35 PDT
Oldest tranlog needed for restore is fileId 4 ( tranlog4.dbf )
Backup was created by GemStone Version: 3.7.0, Tue Aug 08 14:14:38 2023 15f18029e261daf4
Backup Attributes:
Compression: NONE
Encryption: NONE
Signature Hash: NONE
Encryption Keys: 0
For an extent copy backup
os$ copydbf -i extent0.dbf
Source file: /gshost/GemStone3.7/data/extent0.dbf
File type: extent fileId: 0 in a repository with 1 files
File size: 83886080 bytes (80 MB), 5120 records
ByteOrder: Intel (LSB first) compatibilityLevel: 844
Last checkpoint written at: 2023-08-07-12:01:59.591.
Oldest tranlog needed for recovery/restore is fileId 4 ( tranlog4.dbf ).
Extent was shutdown cleanly; no recovery needed.
GemStone Version: 3.7.0, Tue Aug 08 14:14:38 2023 15f18029e261daf4
Encryption: NONE
For this example, tranlog4.dbf and later must be available. If any of these transaction logs are deleted or lost, you will not be able to recover completely. These transaction logs may be archived elsewhere, as long as they can be readily be made available if you do need to restore from backup. If the tranlogs are encrypted as when using encrypted extents, and the Stone’s encryption key changes, you must also keep the old encryption keys for the tranlogs, or update the encryption key in the old tranlogs.
If you do need to restore your system, first restore from backup using the using the appropriate procedure, then replay transactions committed since the backup by reading the transaction logs in the order in which they were created.
For detailed procedures on restoring from backup and replaying transaction logs, see the instructions in Chapter 11, “Making and Restoring Backups”
NOTE
Restoring a repository from backup resets it - both the GemStone kernel and your application classes and data - to the state it was in at the time the backup was created. Anything that was done after that can be recovered only by replaying transaction logs in order, or by restoring a more recent backup.
10.2 How To Manage Full Logging
When the system is operating with the STN_TRAN_FULL_LOGGING configuration option set to True, you (as system administrator) should monitor the available log space. If the log space defined by STN_TRAN_LOG_DIRECTORIES becomes full, users will be unable to commit transactions to the repository until space is made available.
For transaction logs in file system directories, “full” means that there is no free space in the file systems containing those directories.
For transaction logs in raw partitions, “full” means that all partitions listed already contain a GemStone transaction log or other repository file. After archiving an existing log, you must invoke removedbf before that partition can be reused.
There are two recovery situations to consider in managing transaction logs under full logging:
- Recovery from a system crash requires logs for all transactions committed since the last checkpoint. Because of the way GemStone logs changes involving large objects, parts of these transactions may be in earlier logs. The method Repository>>oldestLogFileIdForRecovery returns the fileId of the oldest log that would be needed if a crash were to occur at that point. All logs needed for crash recovery should be kept online.
- Recovery from damaged extents, such as a media failure, requires all transaction logs since the last backup, and earlier logs may be needed if lengthy transactions were in progress at the time the backup started. Log files not needed for crash recovery may be archived offline, although restoring them will take longer.
To Archive Logs
Ordinary UNIX tools, such as tar and cp, can be used to move log files to other locations or to archival media. We recommend that you archive and free one complete log directory at a time, in the order listed in the STN_TRAN_LOG_DIRECTORIES configuration option.
Two special commands are provided for working with raw disk partitions:
- The copydbf command copies a repository file (extent, transaction log, or full backup) to or from a raw disk partition.
- The removedbf command writes over a raw partition so that GemStone will no longer think it contains a repository file.
To determine the current size of a transaction log that is in a raw partition, use the method Repository>>currentTranlogSizeMB. This method returns the log size (in MB) as an Integer.
To determine the oldest transaction log that would be needed to recover from the most recent checkpoint, use the method Repository>>oldestLogFileIdForRecovery. This method returns the internal fileId, which is part of the filename for transaction logs in the file system. If a session was in a lengthy transaction at the time of a system crash, the oldest log needed during recovery may be one that was written before the last checkpoint occurred.
You can obtain similar information by applying copydbf -i to an extent, as shown in the examples here.
For an example script showing how to archive transaction logs out of raw partitions, see $GEMSTONE/examples/archivelog.sh. You will need to edit the script to conform to your own partition names and archive location, and then test it.
To Add a Log at Run Time
Privileges required: FileControl.
You can add a directory or a raw partition for transaction logs to the existing list without shutting down the Stone repository monitor. When you do this, the repository monitor also records the change in its configuration file so that the addition becomes permanent. Send the following message:
SystemRepository addTransactionLog: deviceOrDirectory size: aSize
topaz 1> printit
SystemRepository addTransactionLog: '/users/tlogs2' size: 100
%
The argument aSize sets the maximum log size (in MB) for deviceOrDirectory. It will be added to the list in STN_TRAN_LOG_SIZES.
You can use the method System class>>stoneConfigurationAt: to examine the contents of STN_TRAN_LOG_DIRECTORIES at run time. For information, see How To Access the Server Configuration at Run Time. The Repository methods in Table 10.2 return other information that is helpful in managing transaction logs.
To Force a New Transaction Log
Privileges required: FileControl.
You can force closure of the current log and opening of a new log at almost any time by using the method Repository>>startNewLog. The method performs the following sequential actions:
2. Waits until the checkpoint completes.
4. Returns a SmallInteger, which is the fileId of the new log.
In the following example, the new transaction log file would be tranlog9.dbf.
topaz 1> printit
SystemRepository startNewLog
%
9
If a checkpoint is already in progress when you execute startNewLog, the method will fail and return –1 instead. If you’re using this method in an application, therefore, you need to accommodate the possibility of such a failure with code such as:
| id |
id := SystemRepository startNewLog.
[ id < 0 ] whileTrue: [
System sleep: 1.
id := SystemRepository startNewLog ].
To Initiate a Checkpoint
Privileges required: SystemControl.
System class provides two methods that you can use to start checkpoints manually. These methods do not commit, abort, or otherwise modify the current transaction.
Starts a new checkpoint and returns when the new checkpoint has completed. If a checkpoint is already in progress, this method waits until the current checkpoint completes, then starts a new checkpoint. Returns true if a new checkpoint was successfully completed, returns false if a new checkpoint could not be started because transaction logs are full or checkpoints are suspended.
Starts a new checkpoint if a checkpoint is not already in progress and returns immediately, without waiting for the checkpoint to finish. Returns true if a checkpoint is in progress or was started; returns false if a checkpoint could not be started because transaction logs are full or checkpoints are suspended.
To Change to Partial Logging
Once the full transaction logging has been started on a repository, the STN_TRAN_FULL_LOGGING state of True persists regardless of later changes to the configuration file.
To terminate full logging, the procedure is:
Step 1. Make a Smalltalk full backup (not an extent snapshot), following the instructions under How To Make a Smalltalk Full Backup.
Step 2. Edit the configuration file to set the STN_TRAN_FULL_LOGGING option to False.
Step 3. Stop the Stone repository monitor, and restore the backup following the instructions under Restoring from a Full Backup. Be sure to restore into a copy of the empty distribution extent ($GEMSTONE/bin/extent0.dbf).
10.3 How To Manage Partial Logging
Partial logging provides ease of administration along with some protection against loss of data from system crashes. The Stone repository monitor treats the log directories as a circular list. If the file in the current directory reaches the limit set by STN_TRAN_LOG_SIZES, the Stone switches to the next directory in the list that does not contain a transaction log. In the process of creating log n, the Stone attempts to find and delete log (n – size_of_STN_TRAN_LOG_DIRECTORIES); for example, if the new log will be tranlog7.dbf and there are three elements in STN_TRAN_LOG_DIRECTORIES, the Stone searches all three in attempting to delete tranlog4.dbf.
If there is only one directory specified in by the STN_TRAN_LOG_DIRECTORIES, then the stone deletes the log file in this directory before starting a new log. This means that badly-timed crashes may not be recoverable. You should ensure that there are two directories specified and that there is sufficient disk space for at least two log files, so that one can be preserved when the next is opened.
To Change to Full Logging
To change a repository from partial to full logging, simply change the STN_TRAN_FULL_LOGGING setting to True and restart the Stone repository monitor.
Be sure to make a new GemStone full backup in full-logging mode so that you will be able to restore from the transaction logs if necessary. Transaction logs cannot be restored to backups that were made in partial-logging mode.
10.4 How To Recover from Tranlog-Full Conditions
Transaction Log Space Full
If the space for transaction logs becomes full, the Stone stops processing commits or other requests that initiate a write to the transaction log. Sessions performing these operations are blocked until the condition is resolved and may appear to the user to be hung. The Stone writes a message like the following in its log:
The tranlog directories are full and the stone process is waiting
for an operator to make more space available by either cleaning up
the existing files (copying them to archive media and deleting
them) or by adding a new tranlog directory.
Also, the Exception RepositoryError is signaled in any sessions that have enabled receipt of this error by sending System enableSignalTranlogsFull, and setting up a handler for this error.
Once enabled, you can disable receipt of this error by sending System disableSignalTranlogsFull, and determine the current status by System signalTranlogsFullStatus.
If the transaction log space is full, you have the following options:
- You can free space by taking some existing log files offline. Archive them using operating system utilities and then remove them. GemStone can reuse that slot in the circular list of log directories. (To archive and remove a log file from a raw partition, use copydbf and then removedbf.)
- You can increase the available log space by adding a raw partition or a directory on another disk drive to this list specified for the configuration option STN_TRAN_LOG_DIRECTORIES. Add its maximum file size to STN_TRAN_LOG_SIZES. For information on how to make these changes while GemStone is running, see To Add a Log at Run Time.
When transaction log space becomes available, waiting sessions can complete operations that were blocked.