8. Transactions and Concurrency Control
GemStone users can share code and data objects by maintaining common dictionaries that refer to those objects. However, if operations that modify shared objects are interleaved in any arbitrary order, inconsistencies can result.
This chapter describes how GemStone manages concurrent sessions to prevent inconsistencies resulting from multiple concurrent updates.
8.1 GemStone’s Conflict Management
GemStone prevents conflict between users by encapsulating each session’s operations (computations, stores, and fetches) in units called transactions. The operations that make up a transaction act on what appears to you to be a private view of GemStone objects. When you tell GemStone to commit the current transaction, GemStone tries to merge the modified objects in your view with the shared object store.
Views and Transactions
As shown in Figure 8.1, every user session maintains its own consistent view of the repository state. Objects that the repository contained at the beginning of your session are preserved in your view, even if you are not using them—and even if other users’ actions have rendered them obsolete. The storage that those objects are using cannot be reclaimed until you commit or abort your transaction. Depending upon the characteristics of your particular installation (such as the number of users and the commit frequency), this burden can be trivial or significant.
When you log in to GemStone, you get a view of repository state. After login, you may start a transaction automatically or manually, or remain outside of transaction. The repository view you get on login is updated when you begin a transaction or abort. When you commit a transaction, your changes are merged with other changes to the shared data in the repository, and your view is updated. When you obtain a new view of the repository, by commit, abort, or continuing, any new or modified objects that have been committed by other users become visible to you.
The transaction mode controls if a transaction is automatically started, or if you must manually begin a transaction. For details, see Committing Transactions.)
Transaction State and Transaction Modes
A GemStone session is always either in a transaction or not in a transaction. When in transaction, changes can be committed to the repository. When not in transaction, you can make changes in your view but these changes cannot be committed.
A session that is in transaction may be in one of a number of transaction levels, depending on if nested transactions are involved.
When not in transaction, the session may merely be not in transaction, or it may be in the specialized transactionless mode. In transactionless mode, the session is not in transaction, but its view may be updated automatically at any time. Transactionless mode is primarily for idle sessions that do not need a reliable view of repository data; the topics that this chapter discusses for the most part do not apply to transactionless mode sessions.
The transaction modes provide different behavior with respect to starting new transactions. When in automatic transaction mode, the session is always in transaction. When in manual transaction mode, you may be in transaction or not in transaction, depending on specific messages your session sends.
The following are the GemStone transaction modes:
Automatic transaction mode
In this mode, GemStone begins a transaction when you log in, and starts a new one after each commit or abort message. In this default mode, you are in a transaction the entire time you are logged into a GemStone session. Use caution with this mode in busy production systems, since your session will not receive the signals that your view is causing a strain on system resources.
This is the default transaction mode on login.
To change to automatic transaction mode, send the message:
System transactionMode: #autoBegin
This aborts the current transaction and starts a new transaction.
Manual transaction mode
In this mode, you can be logged in and be outside of a transaction. You explicitly control whether your session starts a transaction, makes changes, and commits. Although a transaction is started for you when you log in, you can set the transaction mode to manual, which aborts the current transaction and leaves you outside a transaction. You can subsequently start a transaction when you are ready to start making changes that you wan to commit. Manual transaction mode provides a method of minimizing the transactions, while still managing the repository for concurrent access.
In manual transaction mode, you can view the repository, browse objects, and make computations based upon object values. You cannot, however, make your changes permanent, nor can you add any new objects you may have created while outside a transaction. You can start a transaction at any time during a session; you can carry temporary results that you may have computed while outside a transaction into your new transaction, where they can be committed, subject to the usual constraints of conflict-checking.
To change to manual transaction mode, send the message:
System transactionMode: #manualBegin
This aborts the current transaction and leaves the session not in transaction.
To begin a transaction, execute
System beginTransaction
This message gives you a fresh view of the repository and starts a transaction. When you commit or abort this new transaction, you will again be outside of a transaction until you either explicitly begin a new one or change transaction modes.
Transactionless mode
In transactionless mode, you remain outside a transaction. This mode is intended primarily for idle sessions. If all you need to do is browse objects in the repository, transactionless mode can be a more efficient use of system resources. However, you are at risk of obtaining inconsistent views.
To change to transactionless transaction mode, send the message:
System transactionMode: #transactionless
Determining transaction mode and transaction state
To determine the transaction mode you are in, send the message:
System transactionMode
To determine the transaction level you are at, send the message:
System transactionLevel
To determine if you are in transaction, send the message
System inTransaction
A transaction level of 1 or more means your session is in transaction, with values greater than 1 indicating the number of levels of transaction. A transaction level of 0 is not in transaction, while -1 indicates transactionless.
You can determine whether you are currently in a transaction by sending the message:
System inTransaction
This message returns true if you are in a transaction and false if you are not.
Reading and Writing in Transactions
GemStone considers the operations that take place in a transaction (or view) as reading or writing objects. Any operation that sends a message to an object, or accesses any instance variable of an object, is said to read that object. An operation that stores something in one of an object’s instance variables is said to write the object. While you can read without writing, writing an object always implies reading it. GemStone must read the internal state of an object in order to store a new value in the object.
Operations that fetch information about an object also read the object. In particular, fetching an object’s size, class, or security policy reads the object. An object also gets read in the process of being stored into another object.
The following expression sends a message to obtain the name of an employee and so reads the object:
theName := anEmployee name. "reads anEmployee"
The following example reads aName in the same operation that anEmployee is written:
anEmployee name: aName. "writes anEmployee, reads aName"
Some less common operations cause objects to be read or written. For example, assigning an object to a new object security policy, using the message assignToObjectSecurityPolicy:, writes the object and reads both the old and the new GsObjectSecurityPolicy. Modifying an object that participates in an index may write support objects built and maintained as part of the indexing mechanism.
For the purposes of detecting conflict among concurrent users, GemStone keeps separate sets of the objects you have written during a transaction and the objects you have only read. These sets are called the write set and the read set; the read set is always a superset of the write set.
Reading and Writing Outside of Transactions
Outside of a transaction, reading an object is accomplished precisely the same way. You can write objects in the same way as well, but you cannot commit these changes to make them a permanent part of the repository.
When Should You Commit a Transaction?
Most applications create or modify objects in logically separate steps, combining trivial operations in sequences that ultimately do significant things. To protect other users from reading or using intermediate results, you want to commit after your program has produced some stable and usable results. Changes become visible to other users only after you’ve committed.
Your chance of being in conflict with other users increases with the time between commits.
Nested In-memory Transactions
Within a transaction, GemStone allows you to group units of work into logical transactions, which can be committed or aborted within the given session. These logical transactions can be nested with up to 16 levels of nesting (including the outer level actual transaction). When the full set of changes are ready to be committed, committing the outer transaction will make the changes persistent and detect any conflicts.
While the same protocol is used to commit the actual (outer) transaction and the nested transactions, the semantics are different. A commit of a nested transaction does not detect conflicts with changes by other users, does not update current session state, and does not make the changes persistent if the session exits unexpectedly or recoverable on system shutdown. Abort of a nested transaction returns the session to the state it was in at the beginning of the nested transaction, without updating the session’s view with any changes by other users.
When transactions are discussed, unless specified otherwise, it only refers to an outer level actual transaction, not to a nested transaction.
To begin a nested transaction, use
System beginNestedTransaction
You should be already in transaction when executing this method.
Executing commit, commitTransaction, abort, or abortTransaction when in a nested transaction preserve or discard in-memory changes and return to the parent level of transaction. The same protocol is used at the outer level, actual transaction to perform the commit or abort.
continueTransaction cannot be used when in a nested transaction.
You can commit or abort all levels of nested transactions at once, including performing the outer level actual commit or abort, using the messages:
System commitAll
System abortAll
8.2 How GemStone Detects and Manages Conflict
GemStone detects conflict by comparing your write set with those of all other transactions that committed since your transaction began. The following conditions signal a possible concurrency conflict:
- An object in your write set is also in the write set of another transaction—a write-write conflict. Write-write conflicts can involve only a single object.
- An object in your write set is also in another session’s dependency list—a write-dependency conflict. An object belongs to a session’s dependency list if the session has added, removed, or changed a dependency (index) for that object. For details about how GemStone creates and manages indexes on collections, see Chapter 7, “Indexes and Querying”.
If a write-write or write-dependency conflict is detected, then your transaction cannot commit; you must abort, and try again. The following section describes some approaches to handling this kind of situation.
Concurrency Management
As the application designer, you determine your approach to concurrency control.
- Using the optimistic approach to concurrency control, you simply read and write objects as if you were the only user. The object server detects conflicts with other sessions only at the time you try to commit your transaction. Your chance of being in conflict with other users increases with the time between commits and the size of your write set.
Although easy to implement in an application, this approach entails the risk that you might lose the work you’ve done if conflicts are detected and you are unable to commit.
- Using the pessimistic approach to concurrency control, you detect and prevent conflicts by explicitly requesting locks that signal your intentions to read or write objects. By locking an object, other users are unable to use the object in a way that conflicts with your purposes. If you are unable to acquire a lock, then someone else has already locked the object and you cannot use the object. You can then abort the transaction immediately instead of doing work that can’t be committed.
- Using reduced-conflict (RC) classes in places where write-write conflicts are likely. RC classes use internal structures and additional logic to allow a commit to succeed in spite of a write-write conflict, when the changes do not actually conflict with each other.
The GemStone reduced-conflict classes include: RcCounter, RcIdentityBag, RcIdentitySet, RcArray, RcPipe, RcQueue, and RcKeyValueDictionary. See Classes That Reduce the Chance of Conflict.
Committing Transactions
Committing a transaction has two effects:
- It makes your new and changed objects visible to other users as a permanent part of the repository.
- It makes visible to you any new or modified objects that have been committed by other users in an up-to-date view of the repository.
When you tell GemStone to commit your transaction, the object server performs these actions:
1. Checks whether other concurrent sessions have committed transactions that modify an object that you modified during your transaction.
2. Checks to see whether other concurrent sessions have added, removed, or changed indexes on an object that you have modified during your transaction.
3. Checks for locks set by other sessions that indicate the intention to modify objects that you have read.
If none of these conditions is found, GemStone commits the transaction. The messages commit or commitTransaction commit the current transaction:
Handling Commit Failure in a Transaction
If GemStone refuses to commit your transaction, the transaction read or wrote an object that another user modified and committed to the repository (or involved in indexing operations) since your transaction began. Because you can’t undo a read or a write operation, simply repeating the attempt to commit will not succeed.
You must abort the transaction in order to get a new view of the repository and, along with it, an empty read set and an empty write set. A subsequent attempt to run your code and commit the view can succeed. If the competition for shared data is heavy, subsequent transactions can also fail to commit. In this situation, locking objects that are frequently modified by other transactions gives you a better chance of committing.
Transaction Conflicts
To find why your transaction failed to commit, you can send the message:
System transactionConflicts
This method returns a symbol dictionary that contains an Association whose key is #commitResult and whose value is one of the following symbols:
|
There were no modified objects to commit, so the commit did not do writes. |
|
|
Commit failed, and the previous commit attempt failed with an rcFailure. |
|
The remaining Associations in the dictionary, if any, are used to report the conflicts found. Each Association’s key indicates the kind of conflict detected; its associated value is an Array of OOPs for the objects that are conflicting.
Table 8.1 lists the possible keys for the conflict.
If there are no conflicts for the transaction, the returned symbol dictionary has no additional Associations.
Conflict sets are cleared at the beginning of a commit or abort and thus can be examined until the next commit, continue, or abort.
NOTE
If you save a reference to the conflict set, be sure to clear this references to avoid making the conflict set persistent.
To determine whether the current transaction has write-write conflicts, you can send the following message before attempting to commit the transaction:
System currentTransactionHasWWConflicts
Similarly, to determine whether the current transaction has write-dependency conflicts, you can send this message:
System currentTransactionHasWDConflicts
If the above message returns true, you can send the appropriate message to obtain a list of write-write (or write-dependency) conflicts in the current transaction:
System currentTransactionWWConflicts (write-write)
System currentTransactionWDConflicts (write-dependency)
More details about transaction conflicts
The information provided by transactionConflicts lets you know the objects that were committed by another session, but does not help in tracking down which session or user committed the changes that were the cause of the conflict. You can enable tracking in your session that lets you collect these details, but this must be enabled before your commit performs a commit that fails. Determining the session whose commit caused the conflict has performance overhead, so it is not recommended that you run this way by default, but it can be useful when you have ongoing conflicts that are difficult to track down.
To enable tracking, set the runtime-only configuration parameter GemCommitConflictDetails to true. For example,
System gemConfigurationAt: #GemCommitConflictDetails put: true
Then, once the commit fails, execute the method System class >> detailedConflictReportString, which returns a string containing information about the conflicting other commit. For example,
System detailedConflictReportString
%
Commit failed , failure
Attempt to commit at: 2019-05-14 15:39:55.959
1 Write-Write Conflicts
( 12200193(a SymbolDictionary))
1 commits by other sessions
session 7 at 2019-05-14 15:39:45.607 userId DataCurator
( 12200193(a SymbolDictionary))
Indexes and Concurrency Control
It is also possible that you can encounter conflict on the internal indexing structures used by GemStone. For example, if two transactions modify the salaries of different employees that participate in the same indexed set, it is possible that both transactions will modify the same internal indexing structure and therefore conflict, despite the fact that neither transaction has explicitly accessed an object written by the other transaction. It is true even if the collection itself is an Rc collection and does not encounter transaction conflicts.
To check this possibility, examine the dictionary returned by evaluating System transactionConflicts (described here). If that dictionary includes any Associations whose key is #'Write-Dependency', you have experienced a conflict on some portion of an indexing structure. In that case, you can abort the transaction and try the modification again.
If you encounter conflicts in the internal indexing structures, you can create a reduced-conflict index. See Reduced-Conflict.
Aborting Transactions
If GemStone refuses to commit your modifications, your view remains intact with all of the new and modified objects it contains. However, your view now also includes other users’ modifications to objects that are visible to you, but that you have not modified. You must take some action to save the modifications in your session or in a file outside GemStone.
Then you need to abort the transaction. This discards all of the modifications from the aborted transaction, and gives you a new view containing the shared, committed objects. Depending on the activities of other users, you can repeat your operations using the new values and commit the new transaction without encountering conflicts.
The messages abort or abortTransaction discard the modified objects in your view. If you are in automatic transaction mode, these messages also begin a new transaction.
Example 8.2
SharedDictionary at: #testData put: 'a string'.
"modifies private view"
System abortTransaction.
"discard the modified copy of SharedDictionary
and all other modified objects, get a new view,
and start a new transaction"
Aborting a transaction discards any changes you have made to shared objects during the transaction. However, work you have done within your own object space is not affected by an abortTransaction. GemStone gives you a new view of the repository that does not include any changes you made to permanent objects during the aborted transaction—because the transaction was aborted, your changes did not affect objects in the repository. The new view, however, does include changes committed by other users since your last transaction started. Objects that you have created in the GemBuilder for Smalltalk object space, outside the repository, remain until you remove them or end your session.
Updating the View Without Committing or Aborting
The message System continueTransaction gives you a new, up-to-date view of other users’ committed work without discarding the objects you have modified in your current session.
The message continueTransaction returns true if a commit on your transaction would succeed, or false if a commit would fail. After continueTransaction returns false, you may view System transactionConflicts to see what objects have conflicts.
Unlike commitTransaction and abortTransaction, continueTransaction does not end your transaction. It has no effect on object locks, and it does not discard any changes you have made or commit any changes. Objects that you have modified or created do not become visible to other users.
Work you have done locally within your own interface is not affected by a continueTransaction. Objects that you have created in your own application remain. Similarly, any execution that you have begun continues, unless the execution explicitly depends upon a successful commit operation.
Note that if you were unable to commit your transaction due to conflicts, you cannot use continueTransaction until you abort the transaction.
Being Signaled To Abort
As mentioned earlier, being in a transaction incurs certain costs. When you are in a transaction, GemStone waits until you commit or abort before it attempts to reclaim obsolete objects in your view. While you are in a transaction, your session will not be signalled to abort, nor is it subject to losing it’s view of the repository or being terminated as a result of sigAbort mechanisms. A session in transaction may cause your repository to grow until it runs out of disk space.
When you are outside of a transaction, GemStone warns you when your view is outdated and this is imposing a burden on the system, by sending your session the TransactionBacklog notification. You are allowed a certain amount of time to abort your current view, as specified in the STN_GEM_ABORT_TIMEOUT parameter in your configuration file. When you abort your current view (by sending the message System abortTransaction), GemStone can reclaim storage and you get a fresh view of the repository.
If you do not respond within the specified time period, the object server sends your session the exception RepositoryViewLost and then terminates the Gem.
Work that you have done locally (such as references to objects within your application) is retained, and you still cannot commit work to the repository when running outside of a transaction. However, you must read again those objects that you had previously read from the repository, and recompute the results of any computations performed on them, because the object server no longer guarantees that the application values are valid.
Your GemStone session controls whether it is signalled to abort by receiving the TransactionBacklog notification when it is out of transaction. To enable receiving it, send the message:
System enableSignaledAbortError
To disable receiving it, send the message:
System disableSignaledAbortError
To determine whether receiving this notification is currently enabled or disabled, send the message:
System signaledAbortErrorStatus
This method returns true if the notification is enabled, and false if it is disabled. By default, GemStone sessions disable receiving this notification. The GemBuilder interfaces may change this default. If you wish to be notified, then you must explicitly enable the signaled abort error, and re-enable it after each time the signal is received.
Being Signaled to continueTransaction
As described earlier, when you are in a transaction, GemStone does not signal the session to abort, nor are you subject to losing your view of the repository. This entails a risk that your repository may grow until it runs out of disk space.
To avoid this problem, you can enable your GemStone session to receive the TransactionBacklog notification when you are in transaction. This prompts your session that it is now holding the oldest view of the repository, and potentially causing your repository to grow. When your session receives this signal, it may execute a continueTransaction, or abort or commit its changes.
Your GemStone session controls whether it receives the TransactionBacklog notification when in transaction. To enable receiving it, send the message:
System enableSignaledFinishTransactionError
To disable receiving it, send the message:
System disableSignaledFinishTransactionError
To determine whether receiving this error message is currently enabled or disabled, send the message:
System signaledFinishTransactionErrorStatus
This method returns true if the notification is enabled, and false if it is disabled. By default, GemStone sessions disable receiving this notification. If you wish to be notified, then you must explicitly enable it after each time the signal is received.
Handlers for abort or continueTransaction notifications
Not only do you need to enable the receipt of the notification to abort or continueTransaction, you must also set up a signal handler to take the appropriate action. Sending enableSignaledAbortError and enableSignaledFinishTransactionError control whether you receive the TransactionBacklog notification when you are not in transaction or when you are in transaction, respectively. The handler for the TransactionBacklog notification needs to take both possible situations into account.
8.3 Controlling Concurrent Access with Locks
If many users are competing for shared data in your application, or you can’t tolerate even an occasional inability to commit, then you can implement pessimistic concurrency control by using locks.
Locking an object is a way of telling GemStone (and, indirectly, other users) your intention to read or write the object. Holding locks prevents transactions whose activities would conflict with your own from committing changes to the repository. Unless you specify otherwise, GemStone locks persist across aborts as well as commits. If you lock on an object and then abort, your session still holds the lock after the abort. Aborting the current transaction (and starting another, if you are in manual transaction mode) gives you an up-to-date value for the locked object without removing the lock.
Remember, locking improves one user’s chances of committing only at the expense of other users. Use locks sparingly to prevent an overall degradation of system performance.
Lock Types
GemStone provides two kinds of locks you may use on any objects: read and write. A session may hold only one kind of lock on an object at a time.
GemStone also provides another type of lock, applicationWriteLock, which is limited to a single unique lock object; it provides similar locking, but is used to provide a mutex. While the applicationWriteLock behaves similarly to read and write locks, it is used differently is discussed separately starting here.
Read Locks
Holding a read lock on an object means that you can use the object’s value, and then commit without fear that some other transaction has committed a new value for that object during your transaction. Another way of saying this is that holding a read lock on an object guarantees that other sessions cannot:
To understand the utility of read locks, imagine that you need to compute the average age of a large number of employees. While you are reading the employees and computing the average, another user changes an employee’s age and commits (right at the end of a birthday party). You have now performed the computation using out-of-date information. You can prevent this frustration by read-locking the employees at the outset of your transaction; this prevents changes to those objects.
Multiple sessions can hold read locks on the same object. A maximum of 1 million read locks can be held concurrently. Because locking incurs a cost at commit time, you should keep the aggregate number of locked objects as small as possible.
NOTE
If you have a read lock on an object and you try to write that object, your attempt to commit that transaction will fail.
Write Locks
Holding a write lock on an object guarantees that you can write the object and commit. That is, it ensures that you won’t find that someone else has prevented you from committing by writing the object and committing it before you, while your transaction was in progress. Another way of looking at this is that holding a write lock on an object guarantees that other sessions cannot:
Write locks are useful, for example, if you want to change the addresses of a number of employees. If you write-lock the employees at the outset of your transaction, you prevent other sessions from modifying one of the employees and committing before you can finish your work. This guarantees your ability to commit the changes.
Write locks differ from read locks in that only one session can hold a write lock on an object. In fact, if a session holds a write lock on an object, then no other session can hold any kind of lock on the object. This prevents another session from receiving the assurance implied by a read lock; that the value of the object it sees in its view will not be out of date when it attempts to commit a transaction.
Acquiring Locks
The kernel class System is the receiver of all lock requests. The following statements request one lock of each kind:
When locks are granted, these messages return System.
Commits and aborts do not necessarily release locks, although locks can be set up so that they will do so. Unless you specify otherwise, once you acquire a lock, it remains in place until you log out or remove it explicitly. (Subsequent sections explain how to remove locks.)
When a lock is requested, GemStone grants it unless one of the following conditions is true:
- You do not have suitable authorization. Read locks require read authorization; write locks require write authorization.
- The object is an instance of SmallInteger, Boolean, Character, SmallDouble, or nil. Trying to lock these special objects is meaningless.
- The object is already locked in an incompatible way by another session (remember, only read locks can be shared).
Variants of the readLock: and writeLock: messages allow you to lock collections of objects en masse. For details, see Locking Collections of Objects Efficiently.
Lock Denial
If you request a lock on an object and another session already holds a conflicting lock on it, then GemStone denies your request; GemStone does not automatically wait for locks to become available.
If you use one of the simpler lock request messages (such as readLock:), lock denial generates an error. If you want to take some automatic action in response to the denial, use a more complex lock request message, such as this:
System readLock: anObject
ifDenied: [block1]
ifChanged: [block2].
A lock denial causes GemStone to execute the block argument to ifDenied:. The method in Example 8.4 uses this technique to request a lock repeatedly until the lock becomes available.
Example 8.4
Object subclass: #Dummy
instVarNames: #()
classVars: #()
classInstVars: #()
poolDictionaries: #()
inDictionary: UserGlobals
options: #()
%
method: Dummy
getReadLockOn: anObject tries: numTries
"This method tries to lock anObject. If the lock is denied,
it tries again, making up to numTries attempts."
| n |
n := 1.
[n <= numTries] whileTrue: [
System readLock: anObject
ifDenied: [System sleep: 1.]
ifChanged: [System abortTransaction.].
n := n + 1 ].
^(System myLockKind: anObject) = #read
%
UserGlobals at: #testObject put: Object new.
System commitTransaction.
%
Dummy new getReadLockOn: testObject tries: 3
%
Deadlocks
You may never succeed in acquiring a lock, no matter how long you wait. Furthermore, because GemStone does not automatically wait for locks, it does not attempt deadlock detection. It is your responsibility to limit the attempts to acquire locks in some way. For example, you can write a portion of your application in such a way that there is an absolute time limit on attempts to acquire a lock. Or you can let users know when locks are being awaited and allow them to interrupt the process if needed.
Dirty Locks
If another user has written an object and committed the change since your transaction began, then the value of the object in your view is out of date. Although you may be able to acquire a lock on the object, it is a dirty lock because you cannot use the object and commit, despite holding the lock.
This condition is trapped by the argument to the ifChanged: keyword following read lock request message:
System readLock: anObject
ifDenied: [block1]
ifChanged: [block2].
Like its simpler counterpart, this message returns System if it acquires a lock on anObject without complications. It generates an error if the user has no authorization for acquiring the lock, or selects one of the blocks passed as arguments and executes that block, returning the block’s value.
For example, if a conflicting lock is held on anObject, this message executes the block given as an argument to the keyword ifDenied:. Similarly, if anObject has been changed by another session, it executes the argument to ifChanged:. The following sections provide some suggestions about the code such blocks might contain. For example:
To minimize your chances of getting dirty locks, lock the objects you need as early in your transaction as possible. If you encounter a dirty lock in the process, you can keep track of the fact and continue locking. After you finish locking, you can abort your transaction to get current values for all of the objects whose locks are dirty. See Example 8.6.
Locking Collections of Objects Efficiently
In addition to the locking request messages for single objects, GemStone provides messages to request locks on an entire collection of objects. If the objects you need to lock are already in collections, or if they can be gathered into collections without too much work, it is more efficient to use the collection-locking methods than to lock the objects individually.
The following statements request locks on each of the elements of two different collections:
Example 8.7
UserGlobals at: #myArray put: Array new;
at: #myBag put: IdentityBag new.
System readLockAll: myArray.
System writeLockAll: myBag.
The messages in Example 8.7 are similar to the simple, single-object locking-request messages (such as readLock:) that you’ve already seen. If a clean lock is acquired on each element of the argument, these messages return System. If you lack the proper authorization for any object in the argument, GemStone generates an error and grants no locks.
The difference between these methods and their single-object counterparts is in the handling of other errors. The system does not immediately halt to report an error if an object in the collection is changed, or if a lock must be denied because another session has already locked the object. Instead, the system continues to request locks on the remaining elements, acquiring as many locks as possible. When the method finishes processing the entire collection, it generates an error. In the meantime, however, all locks that you acquired remain in place.
You might want to handle these errors from within your GemStone Smalltalk program instead of letting execution halt. For this purpose, class System provides collection-locking methods that pass information about unsuccessful lock requests to blocks that you supply as arguments. For example:
System writeLockAll: aCollection ifIncomplete: aBlock
The argument aBlock that you supply to this method must take three arguments. If locks are not granted on all elements of aCollection (for any reason except authorization failure), the method passes three arrays to aBlock and then executes the block.
- The first array contains all elements of aCollection for which locks were denied.
- The second array contains all elements for which dirty locks were granted.
- The third array is empty, and is there for compatibility with previous versions of GemStone.
You can then take appropriate actions within the block. See Example 8.8.
Example 8.8
classmethod: Dummy
handleDenialOn: deniedObjs
^ deniedObjs
%
classmethod: Dummy
getWriteLocksOn: aCollection
System writeLockAll: aCollection
ifIncomplete: [:denied :dirty :unused |
denied isEmpty ifFalse: [self handleDenialOn: denied].
dirty isEmpty ifFalse: [System abortTransaction] ]
%
System readLockAll: myEmployees
%
Dummy getWriteLocksOn: myEmployees
%
Upgrading Locks
On occasion, you might want to upgrade a read lock to a write lock. For example, you might initially intend to read an object, only to discover later that you must also write the object.
However, if you have a read lock on an object, you cannot successfully write that object. If you attempt to do so, your attempt to commit that transaction will fail.
GemStone currently provides no built-in support for upgrading locks. However, to ensure your ability to commit, you can remove the read lock you currently hold on an object and then immediately request a write lock.
It is important to request the upgraded lock immediately, because between the time that the lock is removed, and the time that the upgraded lock is requested, another session has the opportunity to lock the object, or to write it and commit.
Locking and Indexed Collections
When indexes are present, locking can fail to prevent conflict. The reasons are similar to those discussed in the section Indexes and Concurrency Control. Briefly, GemStone maintains indexing structures in your view and does not lock these structures when an indexed collection or one of its elements is locked. Therefore, despite having locked all of the visible objects that you touched, you can be unable to commit.
Specifically, this means that:
- if an object is either an element of an indexed collection, or participates in an index (meaning it is a component of an element bearing an index);
- and another session can access the object, an indexed collection of which the object is a member, or one of its predecessors along the same indexed path;
- then locking the object does not guarantee that you can commit after reading or writing the object.
Therefore, don’t rely on locking an object if the object participates in an index.
Removing or Releasing Locks
Once you lock an object, its default behavior is to remain locked until you either log out or explicitly remove the lock; unless you specify otherwise, locks persist through aborts and commits. In general, remove a lock on an object when you have used the object, committed the resulting values to the repository, and no longer foresee an immediate need to maintain control of the object.
Class System provides the following messages for removing locks:
System removeLock: anObject
Removes any lock you might hold on a single object. If anObject is not locked, GemStone does nothing. If another session holds a lock on anObject, this message has no effect on the other session’s lock.
System removeLockAll: aCollection
Removes any locks you might hold on the elements of a collection.
If you intend to continue your session, but the next transaction is to work on a different set of objects, you might wish to remove all the locks held by your session. Class System provides two mechanisms for doing so.
System commitTransaction; removeLocksForSession
Attempts to commit the present transaction and removes all locks it holds, even if the commit does not succeed.
System commitAndReleaseLocks
Attempts to commit your transaction and release all the locks you hold in a single operation. If your transaction fails to commit, all locks are held instead of released.
Releasing Locks Upon Aborting or Committing
After you have locked an object, you can add it to either of two special sets. One set contains objects whose locks you wish to release as soon as you commit your current transaction. The other set contains objects whose locks you wish to release as soon as you either commit or abort your current transaction. Executing continueTransaction does not release the locks in either set.
The following statement adds a locked object to the set of objects whose locks are to be released upon the next commit:
System addToCommitReleaseLocksSet: aLockedObject
The following statement adds a locked object to the set of objects whose locks are to be released upon the next commit or abort:
System addToCommitOrAbortReleaseLocksSet: aLockedObject
The following statement adds the locked elements of a collection to the set of objects whose locks are to be released upon the next commit:
System addAllToCommitReleaseLocksSet: aLockedCollection
The following statement adds the locked elements of a collection to the set of objects whose locks are to be released upon the next commit or abort:
System addAllToCommitOrAbortReleaseLocksSet: aLockedCollection
NOTE
If you add an object to one of these sets and then request an updated lock on it, the object is removed from the set.
You can remove objects from these sets without removing the lock on the object. The following statement removes a locked object from the set of objects whose locks are to be released upon the next commit:
System removeFromCommitReleaseLocksSet: aLockedObject
The following statement removes a locked object from the set of objects whose locks are to be released upon the next commit or abort:
System removeFromCommitOrAbortReleaseLocksSet: aLockedObject
The following statement removes the locked elements of a collection from the set of objects whose locks are to be released upon the next commit:
System removeAllFromCommitReleaseLocksSet: aLockedCollection
The following statement removes the locked elements of a collection from the set of objects whose locks are to be released upon the next commit or abort:
System removeAllFromCommitOrAbortReleaseLocksSet: aLockedCollection
You can also remove all objects from either of these sets with one message. The following statement removes all objects from the set of objects whose locks are to be released upon the next commit:
System clearCommitReleaseLocksSet
The following statement removes all objects from the set of objects whose locks are to be released upon the next commit or abort:
System clearCommitOrAbortReleaseLocksSet
The statement System commitAndReleaseLocks also clears both sets if the transaction was successfully committed.
Inquiring About Locks
GemStone provides messages for inquiring about locks held by your session and other sessions. Most of these messages are intended for diagnostic use, but some may be useful to ordinary applications.
The message sessionLocks gives you a complete list of all the locks held by your session. This message returns a three-element array. The first element is an array of read-locked objects; the second is an array of write-locked objects, and the third is "deferred unlocks", objects that have been unlocked, but the request is waiting for another session to release the commit token.
For example, the following code uses this information to remove all write locks held by the current session:
System removeLockAll:
(System sessionLocks at: 2)
Other useful messages systemLocksQuick, systemLocks, systemLocksReport, and systemLocksDetailedReport, which report locks on all objects held by all sessions currently logged in to the repository. Note that these methods do not report on locks that other sessions are holding on their temporary objects—that is, objects that they have never committed to the repository. These objects are not visible to your session, so they are unlikely to be a cause of commit conflict.
Another lock inquiry message, lockOwners: anObject, is useful if you’ve been unable to acquire a lock because of conflict with another session. This message returns an array of SmallIntegers representing the sessions that hold locks on anObject. The method in Example 8.9 uses lockOwners: to build an array of the userIDs of all users whose sessions hold locks on a particular object.
Example 8.9
classmethod: Dummy
getNamesOfLockOwnersFor: anObject
| userIDArray sessionArray |
sessionArray := System lockOwners: anObject.
userIDArray := Array new.
sessionArray do:
[:aSessNum | userIDArray add:
(System userProfileForSession: aSessNum) userId].
^userIDArray
%
Dummy getNamesOfLockOwnersFor:
(myEmployees detect: {:e | e.name = ’Conan’ })
%
You can test to see whether an object is included in either of the sets of locked objects whose locks are to be released upon the next abort or commit operation. The following statement returns true if anObject is included in the set of objects whose locks are to be released upon the next commit:
System commitReleaseLocksSetIncludes: anObject
The following statement returns true if anObject is included in the set of objects whose locks are to be released upon the next commit or abort:
System commitOrAbortReleaseLocksSetIncludes: anObject
For information about the other lock inquiry messages, see the methods on System class in the image.
Application Write Locks
Unlike read and write locks, application write locks can only be placed on a single object per lock queue (there are ten lock queues available). The object can be any persistent non-special object; the first time an application lock write is invoked on a lock queue, the object that is locked is registered for that lock queue, and all subsequent uses of that lock queue can only lock this particular object until the next Stone restart.
This allows it to be used as a mutex, or simplifies serializing modifications to a single critical object, such as a collection.
The call to acquire an application write lock also does not return until the lock is acquired, or the lock wait times out. This frees you from having to repeatedly request a lock if it is not immediately available. The timeout is controlled by the configuration parameter STN_OBJ_LOCK_TIMEOUT.
To set an application write lock on an object, send the message:
System waitForApplicationWriteLock: lockObject
queue: lockIdx
autoRelease: aBoolean
lockIdx must be a SmallInteger between 1 and 10, depending on which lock queue is being used. If aBoolean is true, the lock is released automatically on commit or abort, otherwise you must manually remove the lock when you are done.
This method errors if you attempt to lock a temporary object or AllSymbols, otherwise returns an integer code, one of the following:
1 - lock granted
2071 - undefined lock (lockIdx out of range)
2074 - dirty; the lock object written by other session since start of this transaction
2075 - lock denied (lockObject is an invalid object)
2418 - lock not granted, deadlock
2419 - lock not granted, wait for lock timed out
8.4 Classes That Reduce the Chance of Conflict
Often, concurrent access to an object is structural, but not semantic. GemStone detects a conflict when two users access the same object, even when respective changes to the objects do not collide.
For example, when two users both try to add something to a bag they share, GemStone perceives a write-write conflict on the second add operation, although there is really no reason why the two users cannot both add their objects. As human beings, we can see that allowing both operations to succeed leaves the bag in a consistent state, even though both operations modify the bag. A situation such as this causes commit conflicts that could potentially be avoided.
GemStone provides a number of reduced-conflict classes that you can use instead of their regular counterparts in applications that might otherwise experience too many unnecessary conflicts. Using these classes allows a greater number of transactions to commit successfully, but “reduced conflict” does not mean “no conflict.” For example, while two users should be able to add different objects to a shared collection, the code can’t be expected to resolve the problem of two users attempting to remove the same object.
When a conflict does occur - for example, two users attempting to remove the same object - this is a normal conflict. The second user will see a commit failure with a transaction conflict. When the commit fails, the user loses all changes made to the Rc object during the current transaction, and the persistent state remains in the state left by the earlier user who made the conflicting changes.
Reduced-conflict classes are not always appropriate; some of them require more storage, and may require maintenance under some usage conditions, or may cause commits to take longer to complete under some usage conditions.
RcCounter
The class RcCounter can be used instead of a simple number in order to keep track of the amount of something. It allows multiple users to increment or decrement the amount at the same time without experiencing conflicts.
The class RcCounter is not a kind of number. It encapsulates a number—the counter—but it also incorporates other intelligence; you cannot use an RcCounter to replace a number anywhere in your application. It only increments and decrements a counter.
For example, imagine an application to keep track of the number of items in a warehouse bin. Workers increment the counter when they add items to the bin, and decrement the counter when they remove items to be shipped. This warehouse is busy; if each concurrent increment or decrement operation produced a conflict, work slows unacceptably.
Furthermore, the conflicts are mostly unnecessary. Most of the workers can tolerate a certain amount of inaccuracy in their views of the bin count at any time. They do not need to know the exact number of items in the bin at every moment; they may not even worry if the bin count goes slightly negative from time to time. They may simply trust that their views are not completely up-to-date, and that their fellow workers have added to the bin in the time since their views were last refreshed. For such an application, an RcCounter is helpful.
Instances of RcCounter understand messages such as increment, decrement, and value. For additional protocol, see the image.
For example, assuming that binCount refers to an instance of RcCounter, the following operations can take place concurrently from different sessions without causing a conflict:
Example 8.10
!session 1
binCount incrementBy: 36.
System commitTransaction.
%
!session 2
binCount incrementBy: 24.
System commitTransaction.
%
!session 3
binCount decrementBy: 48
ifLessThan: 0
thenExecute: [^'Not enough widgets to ship today.'].
System commitTransaction.
%
This can result in some variable behavior, depending on the timing of the operations.
For example, if the starting binCount is 0, and these operations happen concurrently, then session 3 will not perform the decrement, and the final binCount will be 60. However, if session 1 and 2 have committed their increment operations, and session 3 updated its view prior to executing the code, then session 3 will perform the decrement and the final binCount will be 12.
Reduced-Conflict Collection Classes
GemStone provides a variety of reduced-conflict collection classes:
- RcArray, providing Array-like semantics
- RcKeyValueDictionary, producing dictionary-like semantics
- RcIdentityBag, RcLowMaintenanceIdentityBag, and RcIdentitySet, providing IdentityBag- and IdentitySet-like semantics.
- GsPipe, RcPipe, and RcQueue, providing queue semantics
In addition to varying collection semantics, individual classes have specific types of conflicts they are designed to avoid, and the amount of internal infrastructure or the cost of resolving a conflict varies. Selection of an RC class should consider the demands of the application and also the costs of the automatic conflict resolution.
RcArray, GsPipe, RcPipe, RcKeyValueDictionary, RcIdentitySet and RcLowMaintenanceIdentityBag provide reduced-conflict by automatic replay; when performing specific supported operations, if conflict occurs, the changes can be replayed, slowing down the commit by the second session but allowing the commit to occur.
In cases where there are likely to be many concurrent updates, there is a risk of developing a backlog of sessions replaying the operations; application in which a high degree of concurrent operations are expected may benefit by using an RcQueue or RcIdentityBag. RcIdentityBag and RcQueue provide add and remove sets for each session. This avoids the risk of conflict between sessions at the cost of additional time required to access elements, and some use patterns may require periodic manual cleanup.
RcArray
The class RcArray provides much of the same functionality as Array. However, no conflict occurs on instances of RcArray with:
If a conflict with other update operations on the RcArray occur, the add is replayed so that the commit can succeed. Only methods that add elements at the end of the RcArray support concurrent updates. During conflict resolution, commit order determines the order of the elements in the RcArray.
Because implementation relies on the replay of the adds when there are conflicts, high levels of concurrency have a risk of creating a backlog, when a convoy of sessions are all trying to commit their additions to the RcArray. For applications with expected high rates of concurrency, consider using an RcQueue to accumulate the additions, and have a single gem process remove elements from the RcQueue, and put them in an RcArray.
RcIdentityBag
The class RcIdentityBag provides much of the same functionality as IdentityBag. No conflict occurs on instances of RcIdentityBag with:
- Multiple producers: any number of users are adding objects .
- Limited multiple consumers: one user removes an object from the bag, or multiple users remove objects but only one tried to remove the last or only occurrence of an object.
When multiple sessions remove different occurrences of the same object, it may take a little longer to commit the second transaction.
RcIdentityBag uses per-session add and remove subcollections to avoid conflict. Each session adds to its individual subcollection, and removals of these items are tracked in a parallel bag.
If you create an index on an RcIdentityBag, you should also create a reduced-conflict index, otherwise the underlying index structure may have a conflict. However, even an indexed instance of RcIdentityBag reduces the possibility of a transaction conflict, compared to an instance of IdentityBag, indexed or not.
RcLowMaintenanceIdentityBag and RcIdentitySet
The class RcLowMaintenanceIdentityBag and RcIdentitySet provide much of the same functionality as IdentityBag and IdentitySet. No conflict occurs on instances of RcLowMaintenanceIdentityBag or RcIdentitySet with:
- Multiple producers: any number of users are adding objects.
- Single consumer: one user removes objects.
RcLowMaintenanceIdentityBag and RcIdentitySet avoid conflict by performing a selective abort and replay of adds. If more than one user removes objects, they are likely to experience a commit failure with a transaction conflict. Instances of RcLowMaintenanceIdentityBag and RcIdentitySet may have indexes on their contents. It is recommended to create a reduced-conflict index.
RcKeyValueDictionary
The class RcKeyValueDictionary provides the same functionality as KeyValueDictionary. No conflict occurs on instances of RcKeyValueDictionary with:
- Limited multiple producers: any number of users add keys and values to the dictionary, as long as the keys do not already exist in the dictionary.
- Limited multiple consumers: any number of users remove keys from the dictionary, as long as only one user removes the same key at a time.
RcKeyValueDictionary avoids conflict by performing a selective abort and replay of the modifications to the dictionary. A session that would otherwise have a commit failure due to a transaction conflict may take slightly longer to complete the commit.
GsPipe
The class GsPipe implements a first-in-first-out queue with a single producer and a single consumer. No conflict occurs on instances of GsPipe with:
- Single producer: only one user at a time adds objects to the pipe.
- Single consumer: only one user at a time removes an object from the pipe.
GsPipe avoids commit conflict between adds and removes by the nature of its implementation, since modifying the head or tail of a linked list doesn’t cause conflict.
RcPipe
The class RcPipe implements a first-in-first-out queue with multiple producers and a single consumer. No conflict occurs on instances of RcPipe with:
- Multiple producers: any number of users are adding objects to the RcPipe.
- Single consumer: only one user at a time removes an object from the RcPipe.
RcPipe avoids conflict by performing a selective abort and replay of adds to the pipe. If more than one user removes objects from the pipe, they are likely to experience a commit failure with a transaction conflict. When the commit fails, the user loses all changes made to the pipe during the current transaction, and the pipe remains in the state left by the earlier user who made the conflicting changes.
RcQueue
The class RcQueue approximates the functionality of a first-in-first-out queue. RcQueues are multiple-producer, single-consumer. No conflict occurs with:
- Multiple producers: any number of users are adding objects to the queue.
- Single consumer: only one user at a time removes an object from the queue.
RcQueue uses per-session add and remove subcollections to avoid conflict. Each session’s modifications are only to its own add and remove subcollections; the RcQueue calculates the next element based on the contents of the individual session subcollections.
RcQueue approximates a first-in-first-out queue, but it cannot implement such functionality exactly because of the nature of repository views during transactions.
An object added to an RcQueue is ordered in the queue according to the time it is added to the RcQueue, but it only becomes visible to other sessions when the session commits. If objects in the RcQueue are "consumed" as soon as they appear, then it is possible for more recently created elements to be consumed before older ones that were not yet committed.
For example, suppose one user adds object A at 10:20, but waits to commit until 10:50. Meanwhile, another user adds object B at 10:35 and commits immediately. A third user viewing the queue at 10:30 will see neither object A nor B. At 10:35, object B will become visible to the third user. At 10:50, object A will also become visible to the third user, and will furthermore appear earlier in the queue, because it was created first.