Configuring LIXA components

Configuring LIXA components
Prev	Chapter 3. Configuration	Next

The picture below shows the LIXA components that must be configured:

Figure 3.6. The LIXA components and the necessary configuration files

The client component (lixac) is configured using the etc/lixac_conf.xml file; the server component (lixad) is configured using the etc/lixad_conf.xml file ^[17].

If a node hosts both the client and the server components, both the files must be configured. If a node hosts only one component, only one configuration file is necessary ^[18].

Configuring the server

The default configuration file is etc/lixad_conf.xml and is located at the root installation path (i.e. /opt/lixa/etc/lixad_conf.xml).

The file is composed of three sections:

Figure 3.7. The structure of lixad_conf.xml

the server section contains the global parameters of the server
the listeners section specifies how many TCP/IP addresses and ports must be listened to accept incoming client connections
the managers section specifies how many server threads must be activated to serve the LIXA clients

Below there is a sample configuration file:

<?xml version="1.0" encoding="UTF-8" ?>
  <server 
    pid_file="/opt/lixa/var/run.pid"
    min_elapsed_sync_time="0" max_elapsed_sync_time="0"
    log_size="524288" max_buffer_log_size="131072"
    log_o_direct="1" log_o_dsync="0" log_o_rsync="0" log_o_sync="0">
  <listeners>
    <listener domain="AF_INET" address="127.0.0.1" port="3456"/>
    <listener domain="AF_INET" address="0.0.0.0" port="2345"/>
  </listeners>
  <managers>
    <manager status_file="/opt/lixa/var/lixad_status1"/>
    <manager status_file="/opt/lixa/var/lixad_status2"/>
    <manager status_file="/opt/lixa/var/lixad_status3"/>
  </managers>
</server>

The tags and the properties of the XML file are described below:

pid_file: it is the file used by the server to store the daemon PID; the server creates the file at start-up and destroys it at shutdown
min_elapsed_sync_time: minimum time expressed in millisecond between a request for file synchronization and the start of the synchronization operation; use 0 to obtain the lowest RPO (Recovery Point Objective), use higher values to obtain higher performances of the server
max_elapsed_sync_time: maximum time expressed in millisecond between a request for file synchronization and the start of the synchronization operation; use 0 to obtain the lowest RPO (Recovery Point Objective), use higher values to obtain higher performances of the server max_elapsed_sync_time can not be less than min_elapsed_sync_time;
log_size: maximum desired size for state log files used by the journal based state engine, it does not apply to traditional state engine; it must be considered a soft limit: if necessary, the state engine can expand it. The lower the storage performance, the higher the space necessary for the log files
max_buffer_log_size: maximum desired size in RAM for log buffering, it does not apply to traditional state engine; it's a hard limit. A large buffer can alleviate the issue deriving by a slow storage
log_o_direct: boolean value (1 or 0) to activate/deactivate the O_DIRECT flag for log files, the flag is described in open(2) man page, it does not apply to the traditional state engine
log_o_dsync: boolean value (1 or 0) to activate/deactivate the O_DSYNC flag for log files, the flag is described in open(2) man page, it does not apply to the traditional state engine
log_o_rsync: boolean value (1 or 0) to activate/deactivate the O_RSYNC flag for log files, the flag is described in open(2) man page, it does not apply to the traditional state engine and it's currently of no relevance for Linux
log_o_sync: boolean value (1 or 0) to activate/deactivate the O_SYNC flag for log files, the flag is described in open(2) man page, it does not apply to the traditional state engine
domain: the type of socket must be used to listen for clients. The only allowed type is AF_INET; this may change in the future
address: the address that must be used to listen for clients; the special value "0.0.0.0" means any address
port: the port that must be used to listen for clients; it must be a free port (use command netstat to find out one)
manager: any configured manager is a server worker and runs as a dedicated thread. Refer to Chapter 11, Tuning for a discussion about configuration and performance results
status_file: the physical path that must be used to create the status files (a couple) for a manager; this generally is a persistent and reliable storage device like a RAID partition. The string specified by the tag status_file is used as a prefix: every manager (thread) creates two files with the same prefix and different suffixes.

Synchronization parameters

Parameters min_elapsed_sync_time, max_elapsed_sync_time, log_size, max_buffer_log_size, log_o_direct, log_o_dsync, log_o_rsync, and log_o_sync plays a fundamental role in reliability and performance of the LIXA state server. The provided default values are valid for general purpose scenarios, for advanced use cases, refer to: the section called “Tuning the journal state engine” and the section called “Tuning the traditional state engine”.

Configuring the state engine

Before version 1.9.0, the state engine was based on memory mapped files flushed through msync(2). Even if the solution proved to be robust, it requires a very fast, in term of IOPS, storage subsystem to perform well.

Version 1.9.0 introduces a new alternative state engine based on a much more sophisticated technology that uses log files and asynchronous IO. The new state engine has been designed to provide sub-millisecond transaction time even in presence of common storage subsystems.

Important

The new journal based state engine passes all the test cases, but it's a fresh new piece of software and it can contain unknown bugs. Activate it in a production environment only after appropriate testing has been performed.

To switch from traditional to journal state engine use the LIXA_STATE_ENGINE environment variable before starting lixad state server. To use the journal state engine:

$ export LIXA_STATE_ENGINE=JOURNAL

to use the traditional state engine (default):

$ export LIXA_STATE_ENGINE=TRADITIONAL

Configuring the client

The default configuration file is etc/lixac_conf.xml and is located at the root installation path (i.e. /opt/lixa/etc/lixac_conf.xml).

The file is composed of three sections:

Figure 3.8. The structure of lixac_conf.xml

the state servers section contains a list of the available LIXA state servers and how they can be contacted
the resource managers section contains a list of the available Resource Managers and how they can be managed by LIXA using XA standard API and protocol
the profiles section contains a set of possibly different profiles: every Application Program can use only one profile; if you had different Application Programs with different requirements, you should create different profiles. Every profile contains a set of servers and a set of Resource Managers that must be used to support the Application Program.

Below there is a sample configuration file:

<?xml version="1.0" encoding="UTF-8" ?>
<client
  connection_timeout="3000">
  <sttsrvs>
    <sttsrv name="local_1" domain="AF_INET" address="10.23.45.67" port="2345" />
    <sttsrv name="local_2" domain="AF_INET" address="10.23.46.91" port="3456" />
  </sttsrvs>
  <rsrmgrs>
    <rsrmgr name="OracleXE_dynreg" switch_file="/opt/lixa/lib/switch_oracle_dynreg.so" xa_open_info="Oracle_XA+Acc=P/hr/hr+SesTm=30+LogDir=/tmp+threads=true+DbgFl=7+Loose_Coupling=true" xa_close_info="" />
    <rsrmgr name="OracleXE_stareg" switch_file="/opt/lixa/lib/switch_oracle_stareg.so" xa_open_info="Oracle_XA+Acc=P/hr/hr+SesTm=30+LogDir=/tmp+threads=true+DbgFl=7+Loose_Coupling=true" xa_close_info="" />
    <rsrmgr name="IBMDB2_dynreg" switch_file="/opt/lixa/lib/switch_ibmdb2_dynreg.so" xa_open_info="axlib=/opt/lixa/lib/liblixac.so,db=sample,tpm=lixa" xa_close_info="" />
    <rsrmgr name="IBMDB2_stareg" switch_file="/opt/lixa/lib/switch_ibmdb2_stareg.so" xa_open_info="axlib=/opt/lixa/lib/liblixac.so,db=sample,tpm=lixa" xa_close_info="" />
  </rsrmgrs>
  <profiles>
    <profile name="GT71">
      <sttsrvs>
        <sttsrv>local_1</sttsrv>
      </sttsrvs>
      <rsrmgrs>
        <rsrmgr>OracleXE_stareg</rsrmgr>
        <rsrmgr>IBMDB2_stareg</rsrmgr>
      </rsrmgrs>
    </profile>
    <profile name="VZ67">
      <sttsrvs>
        <sttsrv>local_1</sttsrv>
      </sttsrvs>
      <rsrmgrs>
        <rsrmgr>OracleXE_dynreg</rsrmgr>
        <rsrmgr>IBMDB2_dynreg</rsrmgr>
      </rsrmgrs>
    </profile>
    <profile name="AG71">
      <sttsrvs>
        <sttsrv>local_2</sttsrv>
      </sttsrvs>
      <rsrmgrs>
        <rsrmgr>IBMDB2_dynreg</rsrmgr>
      </rsrmgrs>
    </profile>
  </profiles>
</client>

The tags and the properties of the XML file are described below:

connection_timeout: this property is used to define the client connection timeout, see the section called “LIXA_CLIENT_CONNECTION_TIMEOUT” for the corresponding environment variable and further information
sttsrv: this section is used to describe a state server (a LIXA server instance) that must be reached by any client described below
sttsrv/name: a name associated to the state server; it is a logical name that is referenced by the profiles defined below
domain: it must be the same domain specified by the listener that must be reached; the listener is configured in lixad_conf.xml (see above) and may be local or remote
address: it must be the same same address specified by the listener that must be reached; the listener is configured in lixad_conf.xml (see above) and may be local or remote
port: it must be the same port specified by the listener that must be reached; the listener is configured in lixad_conf.xml (see above) and may be local or remote
rsrmgr: this section is used to describe a resource manager that will be used by the Application Programs configured below (see profiles)
rsrmgr/name: a name associated to the Resource Manager; it is a logical name that is referenced by the profile defined below
switch_file: name of the file that contains the XA switch structure; the file is produced by the installation procedure
xa_open_info: it is the string of parameters that must be passed to the Resource Manager by the xa_open() function call; the content of the string depends on the Resource Manager, please refer to the documentation distributed with the resource manager you are using. The string can not exceed 255 characters
xa_close_info: it is the string of parameters that must be passed to the Resource Manager by the xa_close() function call; the content of the string depends on the Resource Manager, please refer to the documentation distributed with the resource manager you are using. The string can not exceed 255 characters
profile: it contains the description of the LIXA every transactional profile must be used by the Application Programs needs to be listed here
profile/name: the name associated to the transaction profile; this name is used in different places and it is suggested to avoid special characters, blanks and possibly mixed case (these hints may help you in troubleshooting)
profile/sttsrv: the state server that will be used to store the transactional information associated to this profile; more than one state servers can be specified but only the first one is used with the current release software
profile/rsrmgr: every Resource Manager that must be used by the Application Programs associated to this transactional profile needs to be listed here. There is no a limit: you can specify 1, 2, 3, ... resource managers. Avoid useless resource manager because xa_open() and xa_close() will be performed against all the listed resource managers. If you can choose between a "dynamic" and a "static" version of the same resource manager, the "dynamic" one is more efficient

Client configuration explanation

The client configuration file contains three sections:

sttsrvs: is the list of the LIXA daemons you are running inside your network and you will use to manage the persistent state of the clients that are using the configuration file; many times a single LIXA state server is sufficient, but sometimes you need more (development, test and production environment might use different LIXA servers)
rsrmgrs: the list of the Resource Managers necessary execute the Application Programs; there is no limit to the number of resource managers you can specify from a LIXA point of view, but you should avoid to list useless Resource Managers to obtain the best performance
profiles: the list of the available transactional profiles for your Application Programs. This concept allows you a great configuration flexibility: the same configuration file can be used for completely different Application Programs and completely different environment. As an example, imagine you have 3 distinct applications and every application uses a different mix of resource managers; then you manage 3 different environments (development, test and production): with 9 profiles you can model your transactional needs completely (APP1DEV, APP2DEV, APP3DEV, APP1TEST, APP2TEST, APP3TEST, APP1PROD, APP2PROD, APP3PROD).

Important

Every profile lists some Resource Managers: LIXA guarantees that all the XA functions are called in the same exact order ^[19] of the Resource Managers as enlisted in the profile.

In the above example, profile "VZ67" enlists Resource Managers "OracleXE_dynreg" in first position and "IBMDB2_dynreg" in second position; every XA function, like for example xa_prepare and xa_commit are called by LIXA in the same exact order:

	    ...
	    xa_prepare("OracleXE_dynreg")
	    xa_prepare("IBMDB2_dynreg")
	    ...
	    xa_commit("OracleXE_dynreg")
	    xa_commit("IBMDB2_dynreg")
	    ...

The picture below models an environment that's using the sample etc/lixac_conf.xml showed above:

Figure 3.9. A "real" environment

The scenario is composed of three distinct applications:

GT71: it's an application that uses both an Oracle and an IBM DB2 DBMS; most of the transactions need to perform distributed transaction processing and the profile specifies the static registration versions of the switch files.
AG71: it's an application that uses only an IBM DB2 DBMS; only few transactions need to perform distributed transaction processing and the profile specifies the dynamic registration versions of the switch files. This application does not need an XA Transaction Manager because it uses only one Resource Manager, but the development team decided to use LIXA because it will use a second Resource Manager in the next few months.
VZ67: it's an application that uses both an Oracle and an IBM DB2 DBMS; only few transactions need to perform distributed transaction processing and the profile specifies the dynamic registration versions of the switch files.

The scenario uses two distinct LIXA state servers: this is an unusual situation but it works.

The LIXA_PROFILE environment variable must be used to specify the transactional profile associated to the Application Program. If you do not specify a valid transactional profile, the first profile of the list will be applied (in the above example it's GT71).

Note

There is not an alternate way to specify the transactional profile: tx_open does not allow parameters.

With reference to scenario showed above, Application Program GT71 must export LIXA_PROFILE before execution; if you are using bash shell you must specifiy something like this

export LIXA_PROFILE=GT71

before application execution. The same applies to Application Program AG71 and VZ67:

export LIXA_PROFILE=AG71
[...]
export LIXA_PROFILE=VZ67

Client configuration caution

If you change the content of lixac_conf.xml you will change the signature associated to the configuration file from LIXA Transaction Manager. The signature is very important because LIXA Transaction Manager will not activate automatic recovery of “prepared/recovery pending” transactions if configuration file signatures do not match. In the case of signature mismatch during automatic recovery, the LIXA system reacts with some log messages as below:

Feb  8 21:38:51 ubuntu lixad[2517]: LXD011W a client is performing recovery but config file changed in the meantime for job 'a100c8728292168b21ba7239bffc137d/127.0.0.1      ' and transaction '1279875137.46d1df80428d4d6c85e6919b16b4a744.a100c8728292168b21ba7239bffc137d'
Feb  8 21:38:51 ubuntu lixat[2532]: LXC001E transaction '1279875137.46d1df80428d4d6c85e6919b16b4a744.a100c8728292168b21ba7239bffc137d' can NOT be recovered because current config digest 'a100c8728292168b21ba7239bffc137d' does NOT match server stored config digest '9e4c11057107c73366c9fc421eaa85ca'

LIXA state server issues a warning (LXD011W) while LIXA Transaction Manager (client) issues an error (LXC001E): this behavior is necessary to protect the whole system against undesired recovery operation that could damage your data inside the Resource Managers.

This protection has a cost explained in the section called “Recoverying a transaction associated to a different job”.

Important

Before editing (and saving) lixac_conf.xml, be sure there are no “prepared/recovery pending” transactions (you can use --dump as explained in the section called “Dump execution” and in the section called “Recoverying a “recovery failed” transaction” to verify the content of the state server).

Create a different config file and use LIXA_CONFIG_FILE environment variable as explained in the section called “Environment variables” until you are not sure about the changes you are performing.

^[17] Both the files reside in /opt/lixa/etc/ directory and must be managed by lixa user, the LIXA administrative user

^[18] The standard installation procedure installs both the files with default content; it's your responsability to customize the content of the files.

^[19]XA functions can be skipped for dynamically registered Resource Managers if they are not part of the transaction.