9. Transactions and Concurrency Control

Note on Terminology

GemStone’s terminlogy differs from commonly used DBMS terminlogy. A GemStone "view" is a consistent representation of all objects in the repository at a given point in time; essentially a snapshot. This document uses the term "snapshot view" for clarity.

GemStone uses the term "commit record" for the internal structure that tracks what composes a view or snapshot. This term may be used equivalently to view; for example, the oldest commit record is the oldest snapshot view of the repository.

Snapshot Views and Transactions

As shown in Figure 9.1, every user session maintains its own consistent snapshot view of the repository state. Objects that the repository contained at the beginning of your session are preserved in your snapshot view, even if you are not using them—and even if other users’ actions have rendered them obsolete. The storage that those objects are using cannot be reclaimed until you commit or abort your transaction. Depending upon the characteristics of your particular installation (such as the number of users and the commit frequency), this burden can be trivial or significant.

When you log in to GemStone, you get a snapshot view of the latest repository state. After login, you may start a transaction automatically or manually, or remain outside of transaction. The repository snapshot view you get on login is updated when you begin a transaction or abort. When you commit a transaction, your changes are merged with other changes to the shared data in the repository, and your snapshot view is updated. When you obtain a new snapshot view of the repository, by commit, abort, or continuing, any new or modified objects that have been committed by other users become visible to you.

The transaction mode controls if a transaction is automatically started, or if you must manually begin a transaction. For details, see Committing Transactions.)

Figure 9.1 View States

---

Transaction State and Transaction Modes

A GemStone session is always either in a transaction or not in a transaction. When in transaction, changes can be committed to the repository. When not in transaction, you can make changes to objects in the repository, but these changes cannot be committed.

A session that is in transaction may be in one of a number of transaction levels, depending on if nested transactions are involved.

When not in transaction, the session may merely be not in transaction, or it may be in the specialized transactionless mode. In transactionless mode, the session is not in transaction, but its snapshot view may be updated automatically at any time. Transactionless mode is primarily for idle sessions that do not need consistent repository data, since objects may change at any time; the topics that this chapter discusses, for the most part, do not apply to transactionless mode sessions.

The transaction modes provide different behavior with respect to starting new transactions. When in automatic transaction mode, the session is always in transaction. When in manual transaction mode, you may be in transaction or not in transaction, depending on specific messages your session sends.

The following are the GemStone transaction modes:

System transactionMode: #manualBegin

System beginTransaction

System transactionMode: #transactionless

System transactionMode

System transactionLevel

System inTransaction

Reading and Writing in Transactions

GemStone considers the operations that take place in a transaction (or outside of a transaction) as reading or writing objects. Any operation that sends a message to an object, or accesses any instance variable of an object, is said to read that object. An operation that stores something in one of an object’s instance variables is said to write the object. While you can read without writing, writing an object always implies reading it. GemStone must read the internal state of an object in order to store a new value in the object.

Operations that fetch information about an object also read the object. In particular, fetching an object’s size, class, or security policy reads the object. An object also gets read in the process of being stored into another object.

The following expression sends a message to obtain the name of an employee and so reads the object:

theName := anEmployee name. "reads anEmployee" 

The following example reads aName in the same operation that anEmployee is written:

anEmployee name: aName. "writes anEmployee, reads aName"

Some less common operations cause objects to be read or written. For example, assigning an object to a new object security policy, using the message assignToObjectSecurityPolicy:, writes the object and reads both the old and the new GsObjectSecurityPolicy. Modifying an object that participates in an index may write support objects built and maintained as part of the indexing mechanism.

For the purposes of detecting conflict among concurrent users, GemStone keeps separate sets of the objects you have written during a transaction and the objects you have only read. These sets are called the write set and the read set; the read set is always a superset of the write set.

Reading and Writing Outside of Transactions

Outside of a transaction, reading an object is accomplished precisely the same way. You can write objects in the same way as well, but you cannot commit these changes to make them a permanent part of the repository.

When Should You Commit a Transaction?

Most applications create or modify objects in logically separate steps, combining trivial operations in sequences that ultimately do significant things. To protect other users from reading or using intermediate results, you want to commit after your program has produced some stable and usable results. Changes become visible to other users only after you’ve committed.

Your chance of being in conflict with other users increases with the time between commits.

Nested In-memory Transactions

Within a transaction, GemStone allows you to group units of work into logical transactions, which can be committed or aborted within the given session. These logical transactions can be nested with up to 16 levels of nesting (including the outer level actual transaction). When the full set of changes are ready to be committed, committing the outer transaction will make the changes persistent and detect any conflicts.

While the same protocol is used to commit the actual (outer) transaction and the nested transactions, the semantics are different. A commit of a nested transaction does not detect conflicts with changes by other users, does not update current session state, and does not make the changes persistent if the session exits unexpectedly or recoverable on system shutdown. Abort of a nested transaction returns the session to the state it was in at the beginning of the nested transaction, without updating the session’s snapshot view with any changes by other users.

When transactions are discussed, unless specified otherwise, it only refers to an outer level actual transaction, not to a nested transaction.

To begin a nested transaction, use

System beginNestedTransaction

You should be already in transaction when executing this method.

Executing commit, commitTransaction, abort, or abortTransaction when in a nested transaction preserve or discard in-memory changes and return to the parent level of transaction. The same protocol is used at the outer level, actual transaction to perform the commit or abort.

continueTransaction cannot be used when in a nested transaction.

You can commit or abort all levels of nested transactions at once, including performing the outer level actual commit or abort, using the messages:

System commitAll

System abortAll

Concurrency Management

As the application designer, you determine your approach to concurrency control.

• Using the optimistic approach to concurrency control, you simply read and write objects as if you were the only user. The object server detects conflicts with other sessions only at the time you try to commit your transaction. Your chance of being in conflict with other users increases with the time between commits and the size of your write set.

Although easy to implement in an application, this approach entails the risk that you might lose the work you’ve done if conflicts are detected and you are unable to commit.

• Using the pessimistic approach to concurrency control, you detect and prevent conflicts by explicitly requesting locks that signal your intentions to read or write objects. By locking an object, other users are unable to use the object in a way that conflicts with your purposes. If you are unable to acquire a lock, then someone else has already locked the object and you cannot use the object. You can then abort the transaction immediately instead of doing work that can’t be committed.
• Using reduced-conflict (RC) classes in places where write-write conflicts are likely. RC classes use internal structures and additional logic to allow a commit to succeed in spite of a write-write conflict, when the changes do not actually conflict with each other.

The GemStone reduced-conflict classes include: RcCounter, RcIdentityBag, RcIdentitySet, RcArray, RcPipe, RcQueue, and RcKeyValueDictionary. See Classes That Reduce the Chance of Conflict.

Committing Transactions

Committing a transaction has two effects:

• It makes your new and changed objects visible to other users as a permanent part of the repository.
• It makes visible to you any new or modified objects that have been committed by other users in an the most recent state of the repository.

When you tell GemStone to commit your transaction, the object server performs these actions:

1. Checks whether other concurrent sessions have committed transactions that modify an object that you modified during your transaction.

2. Checks to see whether other concurrent sessions have added, removed, or changed indexes on an object that you have modified during your transaction.

3. Checks for locks set by other sessions that indicate the intention to modify objects that you have read.

If none of these conditions is found, GemStone commits the transaction.

The methods System class >> commit or System class >> commitTransaction commit the current transaction.

System commitTransaction returns true if GemStone commits your transaction and false if the commit fails and it cannot commit.

System commit performs the same commit, but returns true if GemStone commits your transaction and signals an error if it fails to commit.

Handling Commit Failure in a Transaction

If GemStone refuses to commit your transaction, the transaction read or wrote an object that another user modified and committed to the repository (or involved in indexing operations) since your transaction began. Because you can’t undo a read or a write operation, simply repeating the attempt to commit will not succeed.

You must abort the transaction in order to get a new snapshot view of the repository and, along with it, an empty read set and an empty write set. A subsequent attempt to run your code and commit can succeed. If the competition for shared data is heavy, subsequent transactions can also fail to commit. In this situation, locking objects that are frequently modified by other transactions gives you a better chance of committing.

Transaction Conflicts

To find why your transaction failed to commit, you can send the message:

System transactionConflicts

This method returns a symbol dictionary that contains an Association whose key is #commitResult and whose value is one of the following symbols:

The remaining Associations in the dictionary, if any, are used to report the conflicts found. Each Association’s key indicates the kind of conflict detected; its associated value is an Array of OOPs for the objects that are conflicting.

Table 9.1 lists the possible keys for the conflict.

If there are no conflicts for the transaction, the returned symbol dictionary has no additional Associations.

Conflict sets are cleared at the beginning of a commit or abort and thus can be examined until the next commit, continue, or abort.

NOTE
If you save a reference to the conflict set, be sure to clear this references to avoid making the conflict set persistent.

To determine whether the current transaction has write-write conflicts, you can send the following message before attempting to commit the transaction:

System currentTransactionHasWWConflicts

Similarly, to determine whether the current transaction has write-dependency conflicts, you can send this message:

System currentTransactionHasWDConflicts

If the above message returns true, you can send the appropriate message to obtain a list of write-write (or write-dependency) conflicts in the current transaction:

System currentTransactionWWConflicts (write-write)

or:

System currentTransactionWDConflicts (write-dependency)

More details about transaction conflicts

The information provided by transactionConflicts lets you know the objects that were committed by another session, but does not help in tracking down which session or user committed the changes that were the cause of the conflict. You can enable tracking in your session that lets you collect these details, but this must be enabled before your commit performs a commit that fails. Determining the session whose commit caused the conflict has performance overhead, so it is not recommended that you run this way by default, but it can be useful when you have ongoing conflicts that are difficult to track down.

To enable tracking, set the runtime-only configuration parameter GemCommitConflictDetails to true. For example,

System gemConfigurationAt: #GemCommitConflictDetails put: true

Then, once the commit fails, execute the method System class >> detailedConflictReportString, which returns a string containing information about the conflicting other commit. For example,

System detailedConflictReportString 

%

Commit failed , failure

Attempt to commit at: 2023-05-14 15:39:55.959

1 Write-Write Conflicts

  ( 12200193(a SymbolDictionary))

  1 commits by other sessions

  session 7 at 2023-05-14 15:39:45.607 userId DataCurator

    ( 12200193(a SymbolDictionary))

Indexes and Concurrency Control

It is also possible that you can encounter conflict on the internal indexing structures used by GemStone. For example, if two transactions modify the salaries of different employees that participate in the same indexed set, it is possible that both transactions will modify the same internal indexing structure and therefore conflict, despite the fact that neither transaction has explicitly accessed an object written by the other transaction. It is true even if the collection itself is an Rc collection and does not encounter transaction conflicts.

To check this possibility, examine the dictionary returned by evaluating System transactionConflicts (described here). If that dictionary includes any Associations whose key is #'Write-Dependency', you have experienced a conflict on some portion of an indexing structure. In that case, you can abort the transaction and try the modification again.

If you encounter conflicts in the internal indexing structures, you can create a reduced-conflict index. See Reduced-Conflict.

Aborting Transactions

If GemStone refuses to commit your modifications, your snapshot view remains intact with all of the new and modified objects it contains. However, your repository now also includes other users’ modifications to objects that are visible to you, but that you have not modified.

You must take some action to save the modifications in your session or in a file outside GemStone.

Then you need to abort the transaction. This discards all of the modifications from the aborted transaction, and gives you a new snapshot view containing all shared, committed objects. Depending on the activities of other users, you can repeat your operations using the new values and commit the new transaction without encountering conflicts.

The messages abort or abortTransaction discard the modified objects in your snapshot view. If you are in automatic transaction mode, these messages also begin a new transaction.

Aborting a transaction discards any changes you have made to shared objects during the transaction. However, work you have done within your own object space is not affected by an abortTransaction. GemStone gives you a new snapshot view of the repository that does not include any changes you made to permanent objects during the aborted transaction—because the transaction was aborted, your changes did not affect objects in the repository. The new snapshot view, however, does include changes committed by other users since your last transaction started.

Objects that you have created in the GemBuilder for Smalltalk object space, outside the repository, remain until you remove them or end your session.

System enableSignaledAbortError

System disableSignaledAbortError

System signaledAbortErrorStatus

System enableSignaledFinishTransactionError

System disableSignaledFinishTransactionError

System signaledFinishTransactionErrorStatus

Handlers for abort or continueTransaction notifications

Not only do you need to enable the receipt of the notification to abort or continueTransaction, you must also set up a signal handler to take the appropriate action. Sending enableSignaledAbortError and enableSignaledFinishTransactionError control whether you receive the TransactionBacklog notification when you are not in transaction or when you are in transaction, respectively. The handler for the TransactionBacklog notification needs to take both possible situations into account.

Lock Types

GemStone provides two kinds of locks you may use on any objects: read and write. A session may hold only one kind of lock on an object at a time.

GemStone also provides another type of lock, applicationWriteLock, which is limited to a single unique lock object; it provides similar locking, but is used to provide a mutex. While the applicationWriteLock behaves similarly to read and write locks, it is used differently is discussed separately starting here.

Acquiring Locks

The kernel class System is the receiver of all lock requests. The following statements request one lock of each kind:

System readLock: SharedDictionary. 

System writeLock: myEmployees. 

System readLock: anObject 

ifDenied: [block1]

ifChanged: [block2].

Object subclass: #Dummy

	instVarNames: #()

	classVars: #()

	classInstVars: #()

	poolDictionaries: #()

	inDictionary: UserGlobals

	options: #()

%

method: Dummy

getReadLockOn: anObject tries: numTries

	"This method tries to lock anObject. If the lock is denied, 

	it tries again, making up to numTries attempts."

| n |

n := 1.

[n <= numTries] whileTrue: [

	System readLock: anObject

		ifDenied: [System sleep: 1.]

		ifChanged: [System abortTransaction.].

	n := n + 1	].

^(System myLockKind: anObject) = #read

%

UserGlobals at: #testObject put: Object new.

System commitTransaction.

%

Dummy new getReadLockOn: testObject tries: 3

%

System readLock: anObject

	ifDenied: [block1]

	ifChanged: [block2].

System readLock: anObject

	ifDenied: []

	ifChanged: [System abortTransaction]

| dirtyBag |

dirtyBag := IdentityBag new.

myEmployees do: [:anEmp |

   System readLock: anEmp

      ifDenied: []

      ifChanged: [ dirtyBag add: anEmp ] ].

dirtyBag isEmpty

   ifTrue: [ ^true ]

   ifFalse: [ System abortTransaction ].

UserGlobals at: #myArray put: Array new;

   at: #myBag put: IdentityBag new.

System readLockAll: myArray.

System writeLockAll: myBag.

System writeLockAll: aCollection ifIncomplete: aBlock

classmethod: Dummy

handleDenialOn: deniedObjs

^ deniedObjs

%

classmethod: Dummy

getWriteLocksOn: aCollection

System writeLockAll: aCollection

   ifIncomplete: [:denied :dirty :unused |

      denied isEmpty ifFalse: [self handleDenialOn: denied].

      dirty isEmpty ifFalse: [System abortTransaction] ]

%

System readLockAll: myEmployees

%

Dummy getWriteLocksOn: myEmployees

%

Locking and Indexed Collections

When indexes are present, locking can fail to prevent conflict. The reasons are similar to those discussed in the section Indexes and Concurrency Control. Briefly, GemStone maintains indexing structures in your snapshot view and does not lock these structures when an indexed collection or one of its elements is locked. Therefore, despite having locked all of the visible objects that you touched, you can be unable to commit.

Specifically, this means that:

• if an object is either an element of an indexed collection, or participates in an index (meaning it is a component of an element bearing an index);
• and another session can access the object, an indexed collection of which the object is a member, or one of its predecessors along the same indexed path;
• then locking the object does not guarantee that you can commit after reading or writing the object.

Therefore, don’t rely on locking an object if the object participates in an index.

Removing or Releasing Locks

Once you lock an object, its default behavior is to remain locked until you either log out or explicitly remove the lock; unless you specify otherwise, locks persist through aborts and commits. In general, remove a lock on an object when you have used the object, committed the resulting values to the repository, and no longer foresee an immediate need to maintain control of the object.

Class System provides the following messages for removing locks:

System removeLock: anObject

Removes any lock you might hold on a single object. If anObject is not locked, GemStone does nothing. If another session holds a lock on anObject, this message has no effect on the other session’s lock.

System removeLockAll: aCollection

Removes any locks you might hold on the elements of a collection.

If you intend to continue your session, but the next transaction is to work on a different set of objects, you might wish to remove all the locks held by your session. Class System provides two mechanisms for doing so.

System commitTransaction; removeLocksForSession

Attempts to commit the present transaction and removes all locks it holds, even if the commit does not succeed.

System commitAndReleaseLocks

Attempts to commit your transaction and release all the locks you hold in a single operation. If your transaction fails to commit, all locks are held instead of released.

System addToCommitReleaseLocksSet: aLockedObject

System addToCommitOrAbortReleaseLocksSet: aLockedObject

System addAllToCommitReleaseLocksSet: aLockedCollection

System addAllToCommitOrAbortReleaseLocksSet: aLockedCollection

System removeFromCommitReleaseLocksSet: aLockedObject

System removeFromCommitOrAbortReleaseLocksSet: aLockedObject

System removeAllFromCommitReleaseLocksSet: aLockedCollection

System removeAllFromCommitOrAbortReleaseLocksSet: aLockedCollection

System clearCommitReleaseLocksSet

System clearCommitOrAbortReleaseLocksSet

Inquiring About Locks

GemStone provides messages for inquiring about locks held by your session and other sessions. Most of these messages are intended for diagnostic use, but some may be useful to ordinary applications.

The message sessionLocks gives you a complete list of all the locks held by your session. This message returns a three-element array. The first element is an array of read-locked objects; the second is an array of write-locked objects, and the third is "deferred unlocks", objects that have been unlocked, but the request is waiting for another session to release the commit token.

For example, the following code uses this information to remove all write locks held by the current session:

System removeLockAll: 

(System sessionLocks at: 2)

Other useful messages systemLocksQuick, systemLocks, systemLocksReport, and systemLocksDetailedReport, which report locks on all objects held by all sessions currently logged in to the repository. Note that these methods do not report on locks that other sessions are holding on their temporary objects—that is, objects that they have never committed to the repository. These objects are not visible to your session, so they are unlikely to be a cause of commit conflict.

Another lock inquiry message, lockOwners: anObject, is useful if you’ve been unable to acquire a lock because of conflict with another session. This message returns an array of SmallIntegers representing the sessions that hold locks on anObject. The method in Example 9.7 uses lockOwners: to build an array of the userIDs of all users whose sessions hold locks on a particular object.

classmethod: Dummy

getNamesOfLockOwnersFor: anObject

| userIDArray sessionArray |

sessionArray := System lockOwners: anObject. 

userIDArray := Array new. 

sessionArray do:

	[:aSessNum | userIDArray add:

		 (System userProfileForSession: aSessNum) userId].

^userIDArray

%

Dummy getNamesOfLockOwnersFor: 

	(myEmployees detect: {:e | e.name = ’Conan’ })

%

System commitReleaseLocksSetIncludes: anObject

System  commitOrAbortReleaseLocksSetIncludes: anObject

Application Write Locks

Unlike read and write locks, application write locks can only be placed on a single object per lock queue (there are ten lock queues available). The object can be any persistent non-special object; the first time an application lock write is invoked on a lock queue, the object that is locked is registered for that lock queue, and all subsequent uses of that lock queue can only lock this particular object until the next Stone restart.

This allows it to be used as a mutex, or simplifies serializing modifications to a single critical object, such as a collection.

The call to acquire an application write lock also does not return until the lock is acquired, or the lock wait times out. This frees you from having to repeatedly request a lock if it is not immediately available. The timeout is controlled by the configuration parameter STN_OBJ_LOCK_TIMEOUT.

To set an application write lock on an object, send the message:

System waitForApplicationWriteLock: lockObject 

queue: lockIdx

autoRelease: aBoolean

lockIdx must be a SmallInteger between 1 and 10, depending on which lock queue is being used. If aBoolean is true, the lock is released automatically on commit or abort, otherwise you must manually remove the lock when you are done.

This method errors if you attempt to lock a temporary object or AllSymbols, otherwise returns an integer code, one of the following:

1 - lock granted
2071 - undefined lock (lockIdx out of range)
2074 - dirty; the lock object written by other session since start of this transaction
2075 - lock denied (lockObject is an invalid object)
2418 - lock not granted, deadlock
2419 - lock not granted, wait for lock timed out

RcCounter

The class RcCounter can be used instead of a simple number in order to keep track of the amount of something. It allows multiple users to increment or decrement the amount at the same time without experiencing conflicts.

The class RcCounter is not a kind of number. It encapsulates a number—the counter—but it also incorporates other intelligence; you cannot use an RcCounter to replace a number anywhere in your application. It only increments and decrements a counter.

For example, imagine an application to keep track of the number of items in a warehouse bin. Workers increment the counter when they add items to the bin, and decrement the counter when they remove items to be shipped. This warehouse is busy; if each concurrent increment or decrement operation produced a conflict, work slows unacceptably.

Furthermore, the conflicts are mostly unnecessary. Most of the workers can tolerate a certain amount of inaccuracy in the value of the bin count at any time. They do not need to know the exact number of items in the bin at every moment; they may not even worry if the bin count goes slightly negative from time to time. They may simply trust that the data is not completely up-to-date, and that their fellow workers have added to the bin in the time since the data was last refreshed. For such an application, an RcCounter is helpful.

Instances of RcCounter understand messages such as increment, decrement, and value. For additional protocol, see the image.

For example, assuming that binCount refers to an instance of RcCounter, the following operations can take place concurrently from different sessions without causing a conflict:

!session 1

binCount incrementBy: 36.

System commitTransaction.

%

!session 2

binCount incrementBy: 24.

System commitTransaction.

%

!session 3

binCount decrementBy: 48

   ifLessThan: 0

   thenExecute: [^'Not enough widgets to ship today.'].

System commitTransaction.

%

Reduced-Conflict Collection Classes

GemStone provides a variety of reduced-conflict collection classes:

• RcArray, providing Array-like semantics
• RcKeyValueDictionary, producing dictionary-like semantics
• RcIdentityBag, RcLowMaintenanceIdentityBag, and RcIdentitySet, providing IdentityBag- and IdentitySet-like semantics.
• GsPipe, RcPipe, and RcQueue, providing queue semantics

In addition to varying collection semantics, individual classes have specific types of conflicts they are designed to avoid, and the amount of internal infrastructure or the cost of resolving a conflict varies. Selection of an RC class should consider the demands of the application and also the costs of the automatic conflict resolution.

RcArray, GsPipe, RcPipe, RcKeyValueDictionary, RcIdentitySet and RcLowMaintenanceIdentityBag provide reduced-conflict by automatic replay; when performing specific supported operations, if conflict occurs, the changes can be replayed, slowing down the commit by the second session but allowing the commit to occur.

In cases where there are likely to be many concurrent updates, there is a risk of developing a backlog of sessions replaying the operations; application in which a high degree of concurrent operations are expected may benefit by using an RcQueue or RcIdentityBag. RcIdentityBag and RcQueue provide add and remove sets for each session. This avoids the risk of conflict between sessions at the cost of additional time required to access elements, and some use patterns may require periodic manual cleanup.