13. Operating System Access
GemStone applications interact with their operating system environment in many ways. The ability to directly execute operating system commands provides a flexible and powerful way to accomplish many tasks. Sockets provide another powerful way to communicate between processes.
This chapter also includes information on serialized GemStone data using PassiveObject and JSON.
13.1 Executing Operating System Commands
GemStone can execute arbitrary operating system command line expressions, using System performOnServer:, or an instance of the GsHostProcess class.
Privileges and limiting OS access
Performing server operations gives your application broad powers, and you may wish to limit the specific server operations that are permitted.
GemStone user privileges, which are described in the System Administration Guide, may restrict who can execute OS commands, and which OS commands can be executed.
GemStone UserProfiles that have the (reverse) privilege NoPerformOnServer cannot execute any OS commands using performOnServer: or using GsHostProcess.
Users with NoPerformOnServer may be allow to execute specific named commands that are added to the allowlist for their UserProfile. An administrator with OtherPassword privilege can add the full path of a command to the allowlist for a specific user, or remove it, using:
UserProfile >> addPerformOnServerCommand: aString
UserProfile >> removePerformOnServerCommand: aString
See the System Administration Guide for more information, or review methods in the image.
performOnServer: for simple commands
System also understands the message performOnServer: aString, which causes the UNIX shell commands given in aString to execute in a subprocess of the current GemStone process. The output of the commands is returned as an instance of String. For example:
System performOnServer: 'date'
%
Fri Aug 04 13:10:21 PDT 2023
The commands in the argument to performOnServer: can have exactly the same form as a shell script; for example, new lines or semicolons can separate commands; you may embed single quotes or double quotes within the argument starting, to allow arguments with spaces; and the character “\” can be used as an escape character.
System performOnServer: 'cat ''a b.txt'''
contents of file
System performOnServer: 'export NEWENVVAR=44
echo $NEWENVVAR'
%
44
The string returned is whatever an equivalent shell command writes to stdout. If the operating system command reports an error, you must interpret the string to determine the correct action. For example:
System performOnServer: 'cp'
%
[40738561 size:63 String] cp: missing file operand
Try 'cp --help' for more information.
The String returned from the performOnServer: command is always composed of 8-bit Characters. If the operating system command produces UTF-8 encoded results, then the Smalltalk String will be UTF-8 encoded. You will need to send decodeFromUTF8ToString or decodeFromUTF8ToUnicode to decode the results. performOnServer: may also return results as 8-bit non-encoded, extended ASCII, if that is what was returned by the operating system commands that were executed.
Using other shells
By default, performOnServer: executes using /bin/sh. To use other shells, specify the full path to the shell executable using System class >> performOnServer: aString withShell: aShellOrNil.
Not all shells will work as an argument to performOnServer:withShell:. bash (/bin/bash), Korn Shell (/bin/ksh), Z Shell (/bin/zsh) and are known to work. C Shell (/bin/csh) and its variants (/bin/tcsh) are known to not work.
GsHostProcess for more complex interactions
System >> performOnServer: can execute arbitrary OS code on the server, but only operates synchronously; Smalltalk will block until the command has completed.
To interact with an operating system command, perform code asynchronously, and to allow Smalltalk to read from stdout or write to stdin or to redirect stdout or stderr to files, you can use the class GsHostProcess.
To use this, use one of the class methods execute:, execute:input:, or fork:, passing the command line you wish to execute. execute: will execute the command and return the results; fork: will return immediately with an instance of GsHostProcess with sockets on stdin, stdout, and stderr. You can use socket protocol to read from or write to these sockets.
For example, to fetch the date from the operating system:
GsHostProcess execute: '/bin/date'.
%
Fri Aug 04 13:10:21 PDT 2023
Note that pathname resolution is not provided. You must fully qualify executable paths. Build-in shell commands cannot be invoked with GsHostProcess.
As with performOnServer:, GsHostProcess accepts a string, which can include spaces to separate arguments and single or double quotes.
GsHostProcess execute: '/bin/cat helloworld.txt'
GsHostProcess execute: '/bin/cp /gshost/f1.txt /gshost/f2.txt'
However, GsHostProcess does not simply execute a shell command, so you cannot pass multiple commands using newlines, since newlines within the command are treated as argument separators; nor can you use other shell syntax.
When an operating system command executed by GsHostProcess fails, it will signal a ChildError with the error status and message from stderr. For example,
GsHostProcess execute: '/bin/cp'
%
ERROR 2710 , a ChildError occurred (error 2710), status 1, /bin/cp: missing file operand
Try '/bin/cp --help' for more information.
Using execute:input: to pass data to stdin
The method GsHostProcess >> execute:input: allows you to pass in data to the stdin of the command being executed.
GsHostProcess execute: '/bin/grep wo' input: 'hello
world'
%
world
Using fork: for interactive commands
Creating a GsHostProcess using the class method fork: allows you to interact with the GsHostProcess instance. For example:
| hostprocess |
hostprocess := GsHostProcess fork: '/bin/date'.
hostprocess stdout read: 1024
%
Fri Aug 04 13:10:21 PDT 2023
GsHostProcess >> readOutErr will read from both stdout and stderr.
When using fork:, the GsHostProcess can report its status using childStatus; you may kill the child process using killChild or killChild: timeoutSeconds.
Connecting stdin, stdout, and stderr
When using GsHostProcess >> fork or fork:, rather than execute:*, it is recommended to specify one or more of:
GsHostProcess>>stderrPath:
GsHostProcess>>stdinPath:
GsHostProcess>>stdoutPath:
to redirect standard files to the filesystem. Otherwise there is a risk the child may hang when trying to read or write to a pipe connected to the parent process, if the parent is not executing the data reads and writes equivalent to those implemented in GsHostProcess>>_executeWithInput:.
To redirect both stdout and stderr to the same file, use stdoutPath:, followed by GsHostProcess >> redirectStderrToStdout.
13.2 Setting environment variables
You can ask about and set environment variables in the server or client environments using the following methods:
System class >> clientEnvironmentVariable: varName
System class >> clientEnvironmentVariable: varName put: valueStr
System class >> gemEnvironmentVariable: varName
System class >> gemEnvironmentVariable: varName put: valueStr
These methods cannot be used if the NoGsFileOnServer privilege is set for the UserProfile.
run
System gemEnvironmentVariable: 'GS_MY_INDEX_KEY' put: '47'.
System clientEnvironmentVariable: 'GS_MY_KEY' put: '84'.
System performOnServer: 'env | grep GS_MY'
%
GS_MY_KEY=84
GS_MY_INDEX_KEY=47
When methods that accept a filename, such as GsFile methods, include an environment variable, environment variables set in this way are used in the expansion.
13.3 Creating and Using Sockets
Sockets open a connection between two processes, allowing a two-way exchange of data. The class GsSocket provides a mechanism for manipulating operating system sockets from within GemStone Smalltalk.
Methods in the class GsSocket do not use the terms client and server in the same way as the methods in class GsFile. Instead, these terms refer to the roles that two processes play with respect to the socket: the server process creates the socket, binds it to a port number, and listens for the client, while the client connects to an already created socket. Both client and server are processes created (or spawned) by a Gem process.
In addition to standard sockets created by GsSocket, you can create secure SSL sockets using the class GsSecureSocket. GsSecureSocket is a subclass of GsSocket that adds protocol to specify certificates and require authentication.
Both GsSocket and GsSecureSocket contain a number of examples as class methods, in the method category Examples.
GsSocket class methods such as clientExample and serverExample. These methods provide examples of how to create a socket connection between two sessions. The example methods work together; they require two separate sessions running from two independently executing interfaces, one running the server example and one running the client example. You can execute these methods from Topaz or from GemBuilder for Smalltalk, but note that serverExample, which should be started first, will take control of the interface until the clientExample completes the socket connection.
GsSecureSocket has a more extensive collection of examples that include single-method examples, as well as pairs of examples. See the image for the available examples.
GsSocket and GsSignallingSocket
GsSocket and GsSignallingSocket represent a basic socket. Methods on GsSocket generally return nil on error, while methods on GsSignallingSocket signal an Error.
Establishing the connection
To setup a socket connection, you create instances of GsSocket in both the client and server processes.
1. On the server side, create an instance of GsSocket, and call makeServerAtPort: This creates a listening socket on the given port.
To have the operating system select a port, use a wildcard bind using makeServer:, or pass nil as the port argument. You will then need to determine the port that the client should connect at using the port method.
2. On the client side, create an instance of a GsSocket and call one of the following:
- connectTo: for a connection to a process on the same host
- connectTo:on: if the server is on a different machine
- connectTo:on:timeoutMs: to specify a timeout for the connection
Provided there was a listening server socket setup as in step 1, this will initiate the connection to the server.
3. The server then does an accept, or acceptTimeoutMs: (to specify a timeout) . This returns a new instance of GsSocket for the client connection.
Note that the server side has two sockets; a listening socket and the established socket with the client.
Communication on the socket
Each process can write and read to the socket using protocol such as write: and read:. See the image methods in the categories Reading and Writing for specific methods.
Writes and reads are of byte objects such as String or ByteArray. Read operations are for a specified number of bytes, and return the actual number of bytes read if fewer bytes were available (if fewer bytes were written to the socket by the peer). A return value of nil means an error occurred, and for read operations, a return value of 0 means the socket EOF was reached.
Closing the socket
When completed, the client should close its socket and the server close the listening and established sockets. This is done by simply sending close to the sockets.
GsSecureSocket
GsSecureSocket creates a secure socket Secure Sockets Layer (SSL), providing access to the open-source OpenSSL library. This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit. (http://www.openssl.org/).
The protocol is Transport Layer Security (TLS), the successor to SSL; GemStone continues to use the term "SSL" in most cases, since SSL remains commonly used to refer to both,.
To create a secure socket, you create instances of the GsSecureSocket class and first establish the connection as a regular socket. Then, further protocol authenticates the connection to make the socket secure.
GsSecureSocket class protocol includes setup of certificates and types of authentication, which can be set for general socket operations, so each new socket does not have to be separately configured.
Certificates, keys, and passphrases
GsSecureSocket instances must be configured with the CA certificates, private key files, and passphrases, to allow them to complete the secure handshake. This can be done using class methods that apply to all new instances, or you can send instance methods to configure individual settings.
The required certificates, CA certificates, private key files and passphrases may be provided by your organization. The GemStone distribution includes example certificates, keys, and passphrases to verify your code, under the directories:
$GEMSTONE/examples/openssl/certs/
directory containing example public keys
$GEMSTONE/examples/openssl/private/
directory containing example private keys and passphrases
The distribution includes script that will allow you to generate certificates:
$GEMSTONE/examples/openssl/create_ca.sh
$GEMSTONE/examples/openssl/create_new_certs.sh
And the openssl executable, matching the version that GemStone uses:
$GEMSTONE/bin/openssl
Using this openssl executable, rather than any version that may be present on your system, is recommended. For details on the openssl interface, see http://www.openssl.org/docs/apps/openssl.html.
Enable or disable verifying CA Certificate
Certificate Authority (CA) certificates should be setup prior to creating instance of GsSecureSocket. An example CA certificate file is provided here:
$GEMSTONE/examples/openssl/certs/cacert.pem
By default, client sockets verify the certificates presented by the server, and are configured with the CA certificate. By default, servers do not verify certificates. This is the usual case, and if this is your preferred behavior you do not need to explicitly enable or disable verification.
Class methods that configure verification must be executed prior to the creation of the client or server GsSecureSocket instance.
Client Sockets
By default, client sockets verify connections from the server. You will need to specify the client certificate file, or the directory containing CA certificates, using one of the following methods.
GsSecureSocket class >> useCACertificateFileForClients: certfile
GsSecureSocket class >> useCACertificateDirectoryForClients: aDir
A CA certificate file must be in PEM format. It may contain more than one certificate. The certificate has effect only for the life of the session; the state is not committed to the repository.
To disable validation on the client, use the methods:
GsSecureSocket disableCertificateVerificationOnClient
aGsSecureClientSocket disableCertificateVerification
Parallel methods exist to re-enable validation after validation is disabled.
Server Sockets
By default, server sockets do not verify the certificate from the client, so you do not need to specify the CA certificate file or directory.
Setting the CA certificate file or directory also enables verification. To enable, execute one of the following:
GsSecureSocket class >> useCACertificateFileForServers: certfile
GsSecureSocket class >> useCACertificateDirectoryForServers: aDir
Parallel methods exist to disable validation after validation is enabled.
The server accepts additional verification options. Using the methods
GsSecureSocket setCertificateVerificationOptionsForServer:
aGsSecureServerSocket setCertificateVerificationOptions:
The following options may be set:
#SSL_VERIFY_FAIL_IF_NO_PEER_CERT
if the client did not return a certificate, the TLS/SSL handshake is immediately terminated with a 'handshake failure' alert.
#SSL_VERIFY_CLIENT_ONCE
only request a client certificate on the initial TLS/SSL handshake. Do not ask for a client certificate again in case of a renegotiation.
Set certificate, private key, and passphrase
The certificate, private key, and private key passphrase can be setup by class methods to apply to all instances, or by sending messages to the instance of GsSecureSocket.
You can pass the certificate and private key either by using pathnames to the files, or by the string. If a string is used, it must exactly match the contents of the corresponding certificate file (including white space, line feeds, etc.), or the strings will not be accepted.
Both certificate and private key must be in PEM format, and the private key must match the certificate. The same file may be specified for the certificate file and the private key file.
The certificate may contain a certificate chain or a single certificate.
If the private key requires a passphrase, it must be specified as a string. If the private key does not require a passphrase, the argument is expected to be nil.
Example certificates, public and private keys, and passphrases are under the $GEMSTONE/examples/openssl/ directory. A number of examples are provided, for example:
$GEMSTONE/examples/openssl/certs/server_1_servercert.pem
$GEMSTONE/examples/openssl/private/server_1_serverkey.pem
$GEMSTONE/examples/openssl/private/server_1_server_passwd.txt
Class setup for server sockets
To specify the server certificates, private key file, and passphrase (if required), that will be used for all secure server sockets that are created after these methods are invoked, use the method:
GsSecureSocket class >>
useServerCertificateFile: certFilePathAndName
withPrivateKeyFile: privateKeyFilePathAndName
privateKeyPassphrase: passphraseStringOrNil
A variant is available that allows you to pass in the strings containing the certificate and key as string; see the image methods. If a string is used, it must exactly match the contents of the corresponding certificate file (including white space, line feeds, etc.), or the strings will not be accepted.
Class setup for client sockets
By default, certificates are not verified on the client, so the certificate and private key do not need to be setup for clients.
If you need to enable client validation, the follow methods specify the client certificates, private key file, and passphrase, that will be used to validate server connections for secure client sockets that are created after these methods are invoked:
GsSecureSocket class >>
useClientCertificateFile: certFilePathAndName
withPrivateKeyFile: privateKeyFilePathAndName
privateKeyPassphraseFile: passphraseStringOrNil
A variant that accepts the content strings instead of filenames is available.
Instance setup for client or server sockets
You can specify the certificate, private key, and passphrase for a single specific instance of GsSecureSocket (either a server socket or a client socket), using instance method:
GsSecureSocket >>
useCertificateFile: certFilePathAndName
withPrivateKeyFile: privateKeyFilePathAndName
privateKeyPassphraseFile: passphraseStringOrNil
Again, a variant that accepts the content strings instead of filenames is available.
Setup the Cipher list
The list of ciphers that are acceptable to use can be configured, either on the class side for servers and clients, or for specific instances of GsSecureSocket client or server sockets.
The cipher list is specified as a formatted string. See http://www.openssl.org/docs/apps/ciphers.html for details on the format of this string (as well as other information on ciphers).
For example, to use all ciphers except NULL ciphers and anonymous Diffie-Hellman (DH), and sort by strength, use the following string:
'ALL:!ADH:@STRENGTH'
To configure the cipher list for all instances of GsSecureSocket, use the following methods. These methods return true if the specification finds one or more usable ciphers, false if no usable ciphers match the specification.
GsSecureSocket class >>
setClientCipherListFromString: aString
GsSecureSocket class >>
setServerCipherListFromString: aString
To configure the cipher list for a specific instance of a server socket or client socket, the ciphers must be set before secureConnect:/secureAccept are executed. This method returns true if the specification finds one or more usable ciphers, false if no usable ciphers match the specification, and nil if the operation has no affect because the receiver is already connected.
GsSecureSocket class >>
setCipherListFromString: aString
Once an instance of GsSecureSocket is successfully connected, you can fetch the cipher in use using:
GsSecureSocket >> fetchCipherDescription
Establishing the connection
Rather than creating instances using GsSocket class >> new, with GsSecureSocket sockets are instantiated using newClient and newServer.
To establish the socket connection, as with regular GsSocket,
1. The server creates the socket using newServer, and calls makeServerAtPort: on an unused port, to create the server listener socket on that port.
2. The client creates the socket using newClient, and calls connectTo:, specify the same port as in Step 1.
3. The server socket calls accept, or acceptTimeoutMs:, which creates the connected socket on the given port.
This establishes the standard socket, but the connection is not secure. Another client-server interaction is required to make this a secure socket.
At this point, you can setup specific certificates and ciphers that will apply to these sockets only, as described in the preceding sections. This is needed if you have not previously set up certificates and ciphers that apply to all GsSecureSocket connections.
Then continue with the process that makes the socket secure:
4. The client socket calls secureConnect.
5. The server socket calls secureAccept, or secureAcceptTimeoutMs:.
If these methods return true, then the connection is secure. To determine if you have a secure connection, use the method:
GsSecureSocket >> hasSecureConnection
Closing the socket
You can either close the socket connection entirely, or close the secure connection and remain connected for normal (not secure) communication.
To close the socket entirely, use
GsSecureSocket >> close
Which performs both the secure close and the regular close.
Note that the secure close requires a handshake. If the socket is blocking, and the peer does not respond, then the close will hang. To close the socket, we recommend first making it non blocking:
mySecureSocket makeNonBlocking.
mySecureSocket close.
To close only the secure socket and leave the connection available for non-secure communication, you can use the method
GsSecureSocket >> secureClose
Which must be executed by both sockets on the connection. You can then call close later, to close the connection entirely.
Error handling
GsSocket
Many GsSocket operations return nil if an error occurs.
On a nil return, you can query the system for the details on the error that occurred using the following methods. These methods are implemented both for the class and instance of GsSocket. For errors in GsSocket class methods, use the class side error methods, and for errors in GsSocket instance methods, use the instance methods
lastErrorString
Returns a String containing information about the last error or nil if no error has occurred. Clears the error information.
lastErrorCode
Returns an integer representing the last OS error or nil if no error has occurred. Does not clear the error information
lastErrorSymbol
Returns a Symbol representing the last OS error or nil if no error has occurred. Does not clear the error information.
GsSecureSocket
Most GsSecureSocket operations signal a SocketError or a SecureSocketError if an error occurs. Ordinary socket operations will signal a SocketError; errors in the security configuration will signal a SecureSocketError.
You can query for the last error from an instance operation using the instance methods:
GsSecureSocket >> lastErrorString
GsSecureSocket >> fetchLastIoErrorString
These methods fetch and clear the error string from a call to SSL functions for connect, accept, read, or write.
On the class side, the following methods return error strings for any SSL function call errors:
GsSecureSocket class >> fetchErrorStringArray
Returns an Array of error strings generated by the OpenSSL package. The errors returned are cleared from the SSL error queue. The array is ordered from oldest to newest error.
GsSecureSocket class >> lastErrorString
Returns a string describing the elements in the SSL error queue, based on the contents of fetchErrorStringArray. The errors are cleared from the error queue.
GsSecureSocket class >> fetchLastCertificateVerificationErrorForClient
GsSecureSocket class >> fetchLastCertificateVerificationErrorForServer
These methods fetch and clear a string representing the last certificate verification error logged by, respectively, a client SSL socket or a server SSL socket.
To clear the error queue, use the method
GsSecureSocket class >> clearErrorQueue
GsSecureSocket pre-shared keys
Pre-shared keys are symmetric keys shared in advance among the communicating parties. Pre-shared keys are set up using the following method:
GsSecureSocket >> setPreSharedKey: aByteArray
Sets the Pre-Shared Key (psk) for the connection to be the bytes contained in aByteArray. Raises and exception if the receiver is already connected or listening for a connection. aByteArray must have a size of at least 8 bytes, not more than 64 bytes and be a multiple of 8 (16, 24, 32 etc). A key size of at least 16 bytes is recommended.
TLS version handling
Support for pre-shared keys requires callback protocols that are dependent on the TLS version. The following methods allow you to specify the version, and limit the maximum and minimum TLS version:
GsSecureSocket >> tlsActualVersion
GsSecureSocket >> tlsMaxVersion
GsSecureSocket >> tlsMaxVersion: verStringOrSymbol
GsSecureSocket >> tlsMinVersion
GsSecureSocket >> tlsMinVersion:
GsSecureSocket class >> tlsClientMaxVersion
GsSecureSocket class >> tlsClientMaxVersion: verStringOrSymbol
GsSecureSocket class >> tlsClientMinVersion
GsSecureSocket class >> tlsClientMinVersion: verStringOrSymbol
GsSecureSocket class >> tlsServerMaxVersion
GsSecureSocket class >> tlsServerMaxVersion: verStringOrSymbol
SGsSecureSocket class >> tlsServerMinVersion
GsSecureSocket class >> tlsServerMinVersion: verStringOrSymbol
Example in image
preSharedKeyTlsClientExample: logToGciClient usingPort: portNum on: host
Pre-shared key transport layer security (PSK-TLS) requires the client and server to both know a pre-shared secret key before the connection is initiated. The key must be at least 8 bytes in size (16 hex digits) and be stored in a ByteArray.
GsSecureSocket class >> preSharedKeyTlsServerExample: logToGciClient usingPort: portNum on: host
Pre-shared key transport layer security (PSK-TLS) requires the client and server to both know a pre-shared secret key before the connection is initiated. The key must be at least 8 bytes in size (16 hex digits) and be stored in a ByteArray.
13.4 ssh and sftp using OpenSSL
The libssh libraries include support for ssh and sftp. GemStone supports commonly used functions with the following classes:
Examples of using these classes are provided in class side methods.
SSH with GsSshSocket
Creating the SSH connection
To create a SSH connection, create a client GsSshSocket using methods inherited from GsSocket, and connect using sshConnect.
For example, the following code connects to the host hostnameOrIP:
sshSock := GsSshSocket newClient.
sshSock connectToHost: hostnameOrIP timeoutMs: 2000.
sshSock userId: userName; password: userPassword.
sshSock disableHostAuthentication.
sshSock sshConnect.
As with all sockets, you should close the socket when you no longer need the connection.
sshSock close
SFTP with GsSftpSocket
Creating Sftp connection
Creating an SFPT connection is just like creating an SSH connection. To create a SFTP connection, create a client GsSftpSocket using methods inherited from GsSocket, and connect using sshConnect.
For example, the following code connects to the host hostnameOrIP:
sshFtpSock := GsSftpSocket newClient.
sshFtpSock connectToHost: hostnameOrIP timeoutMs: 2000.
sshFtpSock userId: userName; password: userPassword.
sshFtpSock disableHostAuthentication.
sshFtpSock sshConnect.
As with all sockets, you should close the socket when you no longer need the connection.
sshFtpSock close
Sftp operations
Once the GsSftpSocket instance is created and connected, there are a number of methods supported, including:
GsSftpSocket >> remoteFileExists: aFileOrDirectory
GsSftpSocket >> currentRemoteDirectory
GsSftpSocket >> createRemoteDirectory: dirname
GsSftpSocket >> createRemoteDirectory: dirname mode: modeInt
GsSftpSocket >> contentsOfRemoteDirectory: dirname
GsSftpSocket >> contentsOfRemoteDirectory: dirname matchPattern: aString
GsSftpSocket >> renameRemoteFile: oldName to: newName
GsSftpSocket >> removeRemoteFile: filename
GsSftpSocket >> removeRemoteDirectory: dirname
GsSftpRemoteFile for operations on remote files
The class GsSftpRemoteFile represents an instance of a Remote File that can be used for reading and writing. You must have an established GsSftpSocket connection, which is an argument to the instance creation methods.
GsSftpRemoteFile class >> openRemoteFile: fileName mode: openMode permissions: permInt errorIfExists: aBoolean withSftpSocket: aGsSftpSocket
GsSftpRemoteFile class >> openRemoteFileAppend: fileName withSftpSocket: aGsSftpSocket
GsSftpRemoteFile class >> openRemoteFileReadOnly: fileName withSftpSocket: aGsSftpSocket
GsSftpRemoteFile class >> createNewRemoteFile: fileName withSftpSocket: aGsSftpSocket
GsSftpRemoteFile class >> createOrOverwriteRemoteFile: fileName withSftpSocket: aGsSftpSocket
Once an instance of GsSftpRemoteFile has been created, use the instance protocol to read, write, and fetch information about the remote file. See the image for specific methods.
For example, to download a remote file using SFTP:
sshFtpSock := GsSftpSocket newClient.
sshFtpSock connectToHost: hostnameOrIP timeoutMs: 2000.
sshFtpSock userId: userName; password: userPassword.
sshFtpSock disableHostAuthentication.
sshFtpSock sshConnect.
remoteSftpFile := GsSftpRemoteFile
openRemoteFileReadOnly: remoteFileName
withSftpSocket: sftpSock.
localFile := GsFile openWriteOnServer: localFileName.
bytes := remoteSftpFile readAllInto: localFile.
localFile close.
sftpFile close.
13.5 Serializing Data
There are several options to serialize GemStone data. PassiveObject preserves the classes and object relationships, but may not work over versions or if activated in a different repository. JSON is highly portable for simple data types.
PassiveObject
To serialize your data to a file or for transmitting over a socket, you can passivate the objects themselves. Objects representing your data are stored into a serialized, text-based form by the GemStone class PassiveObject. PassiveObject starts with a root object and traces through its instance variables, and their instance variables, recursively until it reaches special objects (instances of SmallInteger, Character, Boolean, SmallDouble, or UndefinedObject), or classes that can be reduced to special objects (strings and numbers that are not integers), creating a representation of the object that preserves all of the values required to re-create it. The resulting network of object descriptions can be written to a file, stream, or string.
Each file can hold only one network—you cannot append additional networks to an existing passive object file, stream, or string.
A few objects and aspects of objects are not preserved:
- Instances of UserProfile cannot be preserved in this way, for obvious security reasons.
- SystemRepository cannot be preserved.
- In-memory FileSystems are not preserved.
- Blocks that refer to globals or other variables outside the scope of the block cannot be reactivated correctly.
- Blocks associated with objects (such as the sort block in SortedCollections) have the block’s source code preserved, and recompiled on activate, which may affect behavior depending on the activation environment’s symbolList.
- Any indexes you have created on the object are lost.
- Identities (OOPs) are not preserved.
Dynamic instance variables are preserved. This can be customized for specific classes, by overriding the method shouldWriteDynamicInstVar: instVarName, with the specific dynamic instance variable that you do not want included in the passivated form.
The relationship between two objects is conserved only so long as they are described in the same network. Similarly, if two separate objects A and B both refer to the same third object C, then making A and B passive in two separate operations will result in duplicating the object C, which will be represented in both A’s and B’s network. Because the resulting network of objects can be quite large anyway, you want to avoid such unnecessary duplication. For this reason, it is usually a good idea to create one collection to hold all the objects you wish to preserve before invoking one of the PassiveObject methods.
In addition, since object identity is not preserved, behavior that depends on identity may not work as expected. For example, for objects that implement = using ==, the re-activated object will not be = to the original.
The class PassiveObject implements the method passivate: anObject toStream: aGsFileOrStream to write objects out to a stream or a file. To write the object AllEmployees out to the file allEmployees.obj in the current directory, execute an expression of the form shown in Example 13.1.
Example 13.1
| empFile |
empFile := GsFile openWriteOnServer: 'allEmployees.obj'.
PassiveObject passivate: AllEmployees toStream: empFile.
empFile close.
The class PassiveObject implements the method newOnStream: aGsFileOrStream to read objects from a stream or file into a repository. The method activate then restores the object to its previous form.
The following example reads the file allEmployees.obj into a GemStone repository:
Example 13.2
| empFile passivatedEmployees |
empFile := GsFile openReadOnServer: 'allEmployees.obj'.
passivatedEmployees := PassiveObject newOnStream: empFile.
AllEmployees := passivatedEmployees activate.
empFile close.
JSON
For transmitting data between sockets, GemStone supports serializing objects in JSON format. JSON allows you to serialize the data only in a language-independant human-readable form.
For example, using the Employee examples from Employee class and AllEmployees:
AllEmployees asJson
%
'[{"firstName":"Lee","lastName":"Smith","job":"librarian",
"age":40},{"firstName":"Kay","lastName":"Adams","job":
"clerk","age":24},{"firstName":"Al","lastName":"Jones","job"
:"busdriver","age":40}]'
Unlike passivation, the default JSON serialized form does not preserve class names, object identity, dynamic variables, or other objects attributes. A pointer object is represerented using {} dictionary sytax, with key-value pairs of instance variable name-value at that instance variable; indexable objects are represented using [] arrays syntax. Other attributes, such as sortBlocks and dynamic instance variables are omitted.
By implementing printJsonOn:, you can customize the output of your own classes to provide additional information, as long as you write corresponding code to understand the resulting JSON string.
The JsonParser allows you to parse a JSON string, either a string you have created based on a GemStone object, or external JSON code. The Json parser returns a Dictionary or Array containing the equivalent of the JSON string.
JsonParser parse: AllEmployees asJson
%
anArray( aDictionary( 'firstName'->'Lee', 'age'->40, 'job'->
'librarian', 'lastName'->'Smith'), aDictionary( 'firstName'
->'Kay', 'age'->24, 'job'->'clerk', 'lastName'->'Adams'),
aDictionary( 'firstName'->'Al', 'age'->40, 'job'
->'busdriver','lastName'->'Jones'))
To recreate the original object, you can write custom come to set the instance variable values.
For example, implement the following method
Employee class >> fromJson: aJsonString
| json n |
n := super new.
json := JsonParser parse: aJsonString.
json keysAndValuesDo: [:key :val |
n perform: key, ':' with: val
].
^n