10. Managing Transaction Logs

Previous chapter

Next chapter

Transaction logs hold records of changes to the repository, which allow you to recover any changes made between backups. This chapter describes how to manage these transaction logs.

Overview
explains transaction logs, how to configure them, and how to use them when restoring from backup.

How To Manage Full Logging
describes transaction log management when transaction logs are configured for full logging.

How To Manage Partial Logging
describes transaction log management when transaction logs are configured for partial logging.

How To Recover from Tranlog-Full Conditions
describes how the system responds when there is no disk space for transaction logs, and what to do in these circumstances.

10.1  Overview

A transaction log contains the information necessary to redo transactions to the repository that have been committed by GemStone sessions since the last checkpoint or orderly shutdown. This log is used to recover from crashes such as those caused by a power failure, an operating system failure, or the killing of GemStone monitor processes.

If you need to restore the repository from a backup, transaction logs written in full-logging mode can be used to recreate all transactions committed since the most recent backup was written.

The transaction log is implemented as a sequence of files having names of the form tranlog1.dbf ... tranlogNNN.dbf. The numeric fileId starts at 1 when the Stone starts with a copy of the initial repository extent ($GEMSTONE/bin/extent0.dbf). If the Stone starts on an existing repository without any logs present, the fileId will be one greater than when the repository was last shut down cleanly.

The filename prefix, by default “tranlog”, can be controlled by setting the STN_TRAN_LOG_PREFIX configuration option.

The transaction logs are written to a list of directories or raw partitions specified by the STN_TRAN_LOG_DIRECTORIES configuration option, which is treated as a circular list. Each log is limited to the size set for that directory or raw partition by the STN_TRAN_LOG_SIZES configuration parameter. When one log is full, logging switches to the next directory or raw partition. (What happens when logs have been created in all directories is discussed in Table 10.1) Collectively, the transaction log files logically form an extremely large sequential file with a maximum size of 4 x 106 GB.

As users commit changes to the repository, GemStone writes the changes to a transaction log. Periodically, the stone repository monitor performs a checkpoint, at which point all committed changes are guaranteed to be written to the extents. During the periods between checkpoints, the transaction logs hold the record of the changes, so they are available if an unexpected shutdown occurs.

Figure 10.1   Normal Operation

 

Use ordinary UNIX utilities to backup the transaction logs in the file system. To backup a transaction log that is on a raw disk partition, use copydbf to copy it to a file system. You’ll also need to use removedbf to clear the partition for reuse.

Tranlogs with Encrypted Extents

Transaction logs written by a Stone that is using encrypted extents are written in encrypted form, using the same keypair as the extents. When the Stone is started, if recovery is needed, it will be able to read the transaction logs for recovery, since they were encrypted using the same key as was used to start the Stone. However, extra care must be taken when changing the encryption key for the Stone.

If the encryption key of the extents have changed, on startup after a clean shutdown, the Stone will start a new transaction log that is automatically encrypted with the new key. In this case, you must keep the older keypairs, or modify the older tranlogs to use the new encryption key, if you want to be able to restore these older tranlogs into an older backup.

Logging Modes

GemStone provides two modes of transaction logging, selected by setting the STN_TRAN_FULL_LOGGING configuration option:

  • To provide real-time incremental backup of the repository, set STN_TRAN_FULL_LOGGING to True. All transactions are logged regardless of their size. This mode is strongly recommended for deployed GemStone systems.
  • To allow a simple operation without critical data to run unattended for an extended period, set STN_TRAN_FULL_LOGGING to False. This mode, known as partial logging, provides automatic recovery from system crashes that do not corrupt the repository, such as fatal errors or loss of power. However, if you do experience extent corruption and need to restore from backup, you cannot recover any changes made after the backup was taken.

Table 10.1 compares full and partial transaction logging.

Table 10.1 Comparison of Full and Partial Transaction Logging

Characteristic

 

Full Logging

STN_TRAN_FULL_LOGGING
=TRUE

Partial Logging

STN_TRAN_FULL_LOGGING
=FALSE

Type of
transaction logged

All transactions

Only those transactions smaller than STN_TRAN_LOG_LIMIT; successful commits of larger transactions cause an immediate checkpoint

Recovery from system crash
(extents are OK)

Yes, automatic recovery during restart using checkpoint and log

Yes, automatic recovery during restart using checkpoint and log

Replay of transactions since last backup (as after media failure)

Yes—can carry forward GemStone backup by recreating subsequently committed transactions

No—cannot replay transactions since the backup

Action when current log is full

Logging moves to the next directory or to the head of the list. If a file system directory, the Stone opens a new log file, and retains existing transaction logs. If a raw partition, a new log can be opened only if the previous one has been removed.

There is no limit on the number of file system logs in the directory, but the number of raw partition logs is limited to the number of partitions listed in STN_TRAN_LOG_DIRECTORIES.

Logging moves to the next directory or to the head of the list. The Stone removes the existing transaction log before opening a new one.

The maximum number of logs on line at one time depends on the number of directories or raw partitions in the list.

Action when log space becomes full

The Stone pauses execution if it cannot find space in any of the specified directories or raw partitions.

The Stone deletes log files from the circular list of directories and keeps running.

Administrative task

Monitor log space; archive log files and delete them as necessary

None

In the event of a system crash, GemStone can recover by automatically replaying transactions recorded in the transaction log, from the latest checkpoint before the crash to the end of the log at the moment of the crash (Figure 10.2). This allows you to restart from where you were at the time of the crash.

No special administrative action is required to perform this recovery. Every time the system does a normal startup, it checks to see if there are transactions that need to be recovered. Thus, if the extents and transaction logs are good, the system will automatically recover from a crash.

Figure 10.2   System Crash

Restoring Transactions to a Restored Backup

An important use of transaction logs is to restore transactions that were committed between the last full backup and a system failure. If you experience system failure and your current extents are no longer usable, you can still recover all data, provided that the repository already is in full transaction logging mode and that the backup was made in that mode.

For details on restoring transactions to a backup, see How to Restore Transaction Logs.

How the Logs Are Used

You can only use transaction logs to restore your system from backup if you are in full logging mode.

After creating a backup, you need to retain all the transaction logs that are created during and after the backup. To determine the oldest transaction log that will be required, use the copydbf utility to query the backup file. For example:

Example 10.1 Determining the oldest tranlog needed for restore

For a programmatic full backup
os$ copydbf -i backup.dat
File type: backup  fileId: 0 in a backup set with 1 files
   File size: 25952256 bytes (24 MB), 198 records
   ByteOrder: Intel (LSB first)  compatibilityLevel: 860 
   The file was created at: 08/07/23 18:16:35 PDT
   Full backup started from checkpoint at: 08/07/23 18:16:35 PDT
   Oldest tranlog needed for restore is fileId 4 ( tranlog4.dbf )
   Backup was created by GemStone Version: 3.7.0, Tue Aug 08 14:14:38 2023 15f18029e261daf4
   Backup Attributes:
         Compression: NONE
          Encryption: NONE
      Signature Hash: NONE
     Encryption Keys: 0
For an extent copy backup
os$ copydbf -i extent0.dbf
Source file: /gshost/GemStone3.7/data/extent0.dbf
   File type: extent  fileId: 0 in a repository with 1 files
   File size: 83886080 bytes (80 MB), 5120 records
   ByteOrder: Intel (LSB first)  compatibilityLevel: 844 
   Last checkpoint written at: 2023-08-07-12:01:59.591.
   Oldest tranlog needed for recovery/restore is fileId 4 ( tranlog4.dbf ).
   Extent was shutdown cleanly; no recovery needed.
   GemStone Version: 3.7.0, Tue Aug 08 14:14:38 2023 15f18029e261daf4
   Encryption: NONE
 
 

For this example, tranlog4.dbf and later must be available. If any of these transaction logs are deleted or lost, you will not be able to recover completely. These transaction logs may be archived elsewhere, as long as they can be readily be made available if you do need to restore from backup. If the tranlogs are encrypted as when using encrypted extents, and the Stone’s encryption key changes, you must also keep the old encryption keys for the tranlogs, or update the encryption key in the old tranlogs.

If you do need to restore your system, first restore from backup using the using the appropriate procedure, then replay transactions committed since the backup by reading the transaction logs in the order in which they were created.

For detailed procedures on restoring from backup and replaying transaction logs, see the instructions in Chapter 11, “Making and Restoring Backups”

NOTE
Restoring a repository from backup resets it - both the GemStone kernel and your application classes and data - to the state it was in at the time the backup was created. Anything that was done after that can be recovered only by replaying transaction logs in order, or by restoring a more recent backup.

10.2  How To Manage Full Logging

When the system is operating with the STN_TRAN_FULL_LOGGING configuration option set to True, you (as system administrator) should monitor the available log space. If the log space defined by STN_TRAN_LOG_DIRECTORIES becomes full, users will be unable to commit transactions to the repository until space is made available.

For transaction logs in file system directories, “full” means that there is no free space in the file systems containing those directories.

For transaction logs in raw partitions, “full” means that all partitions listed already contain a GemStone transaction log or other repository file. After archiving an existing log, you must invoke removedbf before that partition can be reused.

There are two recovery situations to consider in managing transaction logs under full logging:

To Archive Logs

Ordinary UNIX tools, such as tar and cp, can be used to move log files to other locations or to archival media. We recommend that you archive and free one complete log directory at a time, in the order listed in the STN_TRAN_LOG_DIRECTORIES configuration option.

Two special commands are provided for working with raw disk partitions:

  • The copydbf command copies a repository file (extent, transaction log, or full backup) to or from a raw disk partition.
  • The removedbf command writes over a raw partition so that GemStone will no longer think it contains a repository file.

To determine the current size of a transaction log that is in a raw partition, use the method Repository>>currentTranlogSizeMB. This method returns the log size (in MB) as an Integer.

To determine the oldest transaction log that would be needed to recover from the most recent checkpoint, use the method Repository>>oldestLogFileIdForRecovery. This method returns the internal fileId, which is part of the filename for transaction logs in the file system. If a session was in a lengthy transaction at the time of a system crash, the oldest log needed during recovery may be one that was written before the last checkpoint occurred.

You can obtain similar information by applying copydbf -i to an extent, as shown in the examples here.

For an example script showing how to archive transaction logs out of raw partitions, see $GEMSTONE/examples/archivelog.sh. You will need to edit the script to conform to your own partition names and archive location, and then test it.

To Add a Log at Run Time

Privileges required: FileControl.

You can add a directory or a raw partition for transaction logs to the existing list without shutting down the Stone repository monitor. When you do this, the repository monitor also records the change in its configuration file so that the addition becomes permanent. Send the following message:

SystemRepository addTransactionLog: deviceOrDirectory size: aSize

For example:

topaz 1> printit
SystemRepository addTransactionLog: '/users/tlogs2' size: 100
%

The argument aSize sets the maximum log size (in MB) for deviceOrDirectory. It will be added to the list in STN_TRAN_LOG_SIZES.

You can use the method System class>>stoneConfigurationAt: to examine the contents of STN_TRAN_LOG_DIRECTORIES at run time. For information, see How To Access the Server Configuration at Run Time. The Repository methods in Table 10.2 return other information that is helpful in managing transaction logs.

Table 10.2 Repository Methods for Information About Transaction Logs

Method

Description

currentLogDirectoryId

Returns a positive SmallInteger, which is the one-based offset of the current log file into the list of log directory names.

currentLogFile

Returns a String containing the name of the transaction log file to which records currently are being appended.

currentTranlogSizeMB

Returns an Integer that is the size (in MB) of the currently active transaction log.

allTranlogDirectories

Returns an Array of Strings corresponding to the STN_TRAN_LOG_DIRECTORIES configuration.

allTranlogSizes

Returns an Array of SmallIntegers corresponding to the STN_TRAN_LOG_SIZES configuration.

logOriginTime

Returns the log origin time of the receiver, the time when a new sequence of log files was started. For details, see the method comment in the image.

oldestLogFileIdForRecovery

Returns a positive SmallInteger, which is the internal fileId of the oldest transaction log needed to recover from the most recent checkpoint, if the Stone were to crash as of now.

restoreStatusOldestFileId

Returns a SmallInteger, which is the internal fileId of the oldest transaction log needed for the next restore from log operation.

To Force a New Transaction Log

Privileges required: FileControl.

You can force closure of the current log and opening of a new log at almost any time by using the method Repository>>startNewLog. The method performs the following sequential actions:

1. Starts a checkpoint.

2. Waits until the checkpoint completes.

3. Starts the new log.

4. Returns a SmallInteger, which is the fileId of the new log.

In the following example, the new transaction log file would be tranlog9.dbf.

topaz 1> printit
SystemRepository startNewLog
%
9

If a checkpoint is already in progress when you execute startNewLog, the method will fail and return –1 instead. If you’re using this method in an application, therefore, you need to accommodate the possibility of such a failure with code such as:

| id  |
id := SystemRepository startNewLog.
[ id < 0 ] whileTrue: [
  System sleep: 1.
  id := SystemRepository startNewLog ].

To Initiate a Checkpoint

Privileges required: SystemControl.

System class provides two methods that you can use to start checkpoints manually. These methods do not commit, abort, or otherwise modify the current transaction.

startCheckpointSync

Starts a new checkpoint and returns when the new checkpoint has completed. If a checkpoint is already in progress, this method waits until the current checkpoint completes, then starts a new checkpoint. Returns true if a new checkpoint was successfully completed, returns false if a new checkpoint could not be started because transaction logs are full or checkpoints are suspended.

startCheckpointAsync

Starts a new checkpoint if a checkpoint is not already in progress and returns immediately, without waiting for the checkpoint to finish. Returns true if a checkpoint is in progress or was started; returns false if a checkpoint could not be started because transaction logs are full or checkpoints are suspended.

To Change to Partial Logging

Once the full transaction logging has been started on a repository, the STN_TRAN_FULL_LOGGING state of True persists regardless of later changes to the configuration file.

To terminate full logging, the procedure is:

Step 1. Make a Smalltalk full backup (not an extent snapshot), following the instructions under How To Make a Smalltalk Full Backup.

Step 2. Edit the configuration file to set the STN_TRAN_FULL_LOGGING option to False.

Step 3. Stop the Stone repository monitor, and restore the backup following the instructions under Restoring from a Full Backup. Be sure to restore into a copy of the empty distribution extent ($GEMSTONE/bin/extent0.dbf).

Step 4. Restart GemStone.

10.3  How To Manage Partial Logging

Partial logging provides ease of administration along with some protection against loss of data from system crashes. The Stone repository monitor treats the log directories as a circular list. If the file in the current directory reaches the limit set by STN_TRAN_LOG_SIZES, the Stone switches to the next directory in the list that does not contain a transaction log. In the process of creating log n, the Stone attempts to find and delete log (n – size_of_STN_TRAN_LOG_DIRECTORIES); for example, if the new log will be tranlog7.dbf and there are three elements in STN_TRAN_LOG_DIRECTORIES, the Stone searches all three in attempting to delete tranlog4.dbf.

If there is only one directory specified in by the STN_TRAN_LOG_DIRECTORIES, then the stone deletes the log file in this directory before starting a new log. This means that badly-timed crashes may not be recoverable. You should ensure that there are two directories specified and that there is sufficient disk space for at least two log files, so that one can be preserved when the next is opened.

To Change to Full Logging

To change a repository from partial to full logging, simply change the STN_TRAN_FULL_LOGGING setting to True and restart the Stone repository monitor.

Be sure to make a new GemStone full backup in full-logging mode so that you will be able to restore from the transaction logs if necessary. Transaction logs cannot be restored to backups that were made in partial-logging mode.

10.4  How To Recover from Tranlog-Full Conditions

Transaction Log Space Full

If the space for transaction logs becomes full, the Stone stops processing commits or other requests that initiate a write to the transaction log. Sessions performing these operations are blocked until the condition is resolved and may appear to the user to be hung. The Stone writes a message like the following in its log:

The tranlog directories are full and the stone process is waiting
for an operator to make more space available by either cleaning up
the existing files (copying them to archive media and deleting
them) or by adding a new tranlog directory.

Also, the Exception RepositoryError is signaled in any sessions that have enabled receipt of this error by sending System enableSignalTranlogsFull, and setting up a handler for this error.

Once enabled, you can disable receipt of this error by sending System disableSignalTranlogsFull, and determine the current status by System signalTranlogsFullStatus.

If the transaction log space is full, you have the following options:

  • You can free space by taking some existing log files offline. Archive them using operating system utilities and then remove them. GemStone can reuse that slot in the circular list of log directories. (To archive and remove a log file from a raw partition, use copydbf and then removedbf.)
  • You can increase the available log space by adding a raw partition or a directory on another disk drive to this list specified for the configuration option STN_TRAN_LOG_DIRECTORIES. Add its maximum file size to STN_TRAN_LOG_SIZES. For information on how to make these changes while GemStone is running, see To Add a Log at Run Time.

When transaction log space becomes available, waiting sessions can complete operations that were blocked.

Previous chapter

Next chapter