Ubik Batch Server FAQ

From UbikWiki

(Difference between revisions)
(→After a while, Oracle throws this exception: too many open cursors for pool jdbc/console)
(→After a while, Oracle throws this exception: too many open cursors for pool jdbc/console)

Line 125:

Line 125:

=== After a while, Oracle throws this exception: too many open cursors for pool jdbc/console ===
=== After a while, Oracle throws this exception: too many open cursors for pool jdbc/console ===
-
The Oracle JDBC driver doesn't much like to have its prepared statements cached with Hibernate (see [ http://www.hibernate.org/120.html#A10 Hibernate Platform Specific Issues]).
+
The Oracle JDBC driver doesn't much like to have its prepared statements cached with Hibernate (see [[ http://www.hibernate.org/120.html#A10 Hibernate Platform Specific Issues]]).
Disable PreparedStatement caching for the connection pool.
Disable PreparedStatement caching for the connection pool.

Revision as of 13:04, 29 December 2007

Contents

Administration

How do I setup logging parameters

Log4j is used as logging framework for the Batch Server, see:

How do I setup Network Interface

What is network interface:

  • The ability to have 2 or more IP addresses for the same machine

To affect an IP to a batch server, add in <UBIK_SERVER_INSTALL_DIR>/config/config.properties: 

NETWORK_INTERFACE_HOST_NAME=<IP ADDRESS>

What are the available parameters for Database Pool declarations

Ubik Batch Server uses Apache commons-dbcp implementation, see:

What validation Query to Use

Validation Query is mandatory to ensure that a connection returned to client code is valid. It has the following requirements:

  • It must be the most efficient query
  • It must return ONE and only ONE row

To declare it, use the Ubik Web Controler or open <INSTALL_DIR>/config/datasource.xml and add or modify for your pool the parameter validationQuery: 

<parameter>
      <name>validationQuery</name>
      <value>...</value>
</parameter>
Database
Query
DB2 SELECT COUNT(*) FROM SYSIBM.SYSTABLES OR SELECT CURRENT TIMESTAMP FROM SYSIBM.SYSDUMMY1
Informix SELECT COUNT(*) FROM SYSTABLES
Microsoft SQL Server SELECT 1
MySQL SELECT 1
Oracle SELECT 1 FROM DUAL
PointBase SELECT COUNT(*) FROM SYSTABLES
PostgreSQL SELECT 1
Progress SELECT COUNT(*) FROM SYSTABLES
Sybase SELECT 1

How do I set a JDBC connection property

Use the Ubik Web Controler or open <INSTALL_DIR>/config/datasource.xml and add or modify for your pool the parameter connectionProperties(Format of the string must be [propertyName=property;]*: 

<parameter>
      <name>connectionProperties</name>
      <value>currentSchema=TEST</value>
</parameter>

How do I change the retention period for Batches History in Database

History of batches execution is saved in the Database, a system batch runs every day to cleanup lines older than CLEANUP_DELAY days. To setup this parameter, open the file <INSTALL_DIR>/config/config.properties and change: 

# Cleanup delay in days for tables BATCH_HEAD and BATCH_TRT
CLEANUP_DELAY=30

How do I setup my mail server parameters

Open the file <INSTALL_DIR>/config/mail.xml and change: 

<?xml version="1.0" encoding="UTF-8"?>
<!-- #Release @VERSION@ at @DATE@ -->
<mail>
	<mailHost>smtp.ubik-ingenierie.com</mailHost>
	<mailer>UbikBatch</mailer>
	<mailPort>25</mailPort>
</mail>

How do I declare my custom plugin

Open the file <INSTALL_DIR>/config/plugin-manager.xml and add: 

<plug-in className="com.ubikingenierie.console.monitoring.plugin.HealthMonitorPlugin"  
        id="com.ubikingenierie.console.monitoring.plugin.HealthMonitorPlugin">
          <set-property property="propertyOfClass" value="valueToSet"></set-property>
...
          <set-property property="otherValue" value="3000000"></set-property>
</plug-in>

How do I declare my custom extension

Open the file <INSTALL_DIR>/config/extension-manager.xml and add: 

<extension className="com.ubikingenierie.console.si3si.extension.SI3SIExtension" >
</extension>

I get the following message on deployment of a big BAR 

java.lang.OutOfMemoryError: PermGen space

Ensure you use the last JDK 5 available version Open <INSTALL_DIR>/bin/setEnv.bat or <INSTALL_DIR>/bin/setenv.cfg and set correctly the PermGen space using JAVA_OPTS: 

-XX:MaxPermSize=128M

My database type is not correctly detected

Open <INSTALL_DIR>/config/config.properties and set HIBERNATE_CONFIGURED_DIALECT: 

# Force a dialect for Hibernate to override auto detection
# For DB2 : com.ubikingenierie.console.persistance.hibernate.DB2ExtendedDialect(*) 
HIBERNATE_CONFIGURED_DIALECT=org.hibernate.dialect.MySQL5InnoDBDialect

(*) : see DB2Dialect.getCurrentTimestampSQLFunctionName() uses Oracle syntax

After a while, Oracle throws this exception: too many open cursors for pool jdbc/console

The Oracle JDBC driver doesn't much like to have its prepared statements cached with Hibernate (see http://www.hibernate.org/120.html#A10 Hibernate Platform Specific Issues).

Disable PreparedStatement caching for the connection pool. Use Web Controler JDBC Administration page or open <INSTALL_DIR>config/datasource.xml, goto jdbc/console declaration and modifiy parameter poolPreparedStatements: 

<parameter>
      <name>poolPreparedStatements</name>
      <value>false</value>
</parameter>

I have the following messages in the logs 

11:49:27,464 WARN LoadContexts? fail-safe cleanup (collections)... org.hibernate.engine.loading.CollectionLoadContext@cd7045<rs=org.hsqldb.jdbc.jdbcResultSet@192e6b6>

Nothing's wrong, it's a small issue that will be resolved in future versions of Hibernate, since it's indirectly a logging issue. You can hide those messages by adding to <INSTALL_DIR>/config/log.properties: 

log4j.category.org.hibernate.engine.loading.LoadContexts=ERROR

Deployment

My BAR fails to deploy on the Batch server

  • If you have the Ubik Web Controler, go to Deployment > Click on the BAR Name, deployment errors will appear
  • If you don't have it, go to folder <INSTALL_DIR>/batches-deploy-failure and open file <BAR_NAME-version>.bar.failure, it

will contain the main cause of error

  • If you don't find any error, go to log folder

How do I customize mapping with Cold Deployment feature

  • Create in the same folder dedicated to Cold Deployment (identified by property COLD_DEPLOYMENT_DIR

(defaults to <INSTALL_DIR>/batches-cold-deploy)) that contains your BAR a file called <BAR_ID>-binding.xml that will contain the mapping: 

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE bar-resources 
          PUBLIC "-//Ubik-Ingenierie //DTD BAR Binding Resource Configuration 1.0//EN" 
          "http://www.ubik-ingenierie.com/consolebatch/config/bar-binding.dtd">
<bar-resources>
     <bar-resource nameinbar="Logical Name" boundname="Physical Name" />
...
     <bar-resource nameinbar="Logical Name 1" boundname="Physical Name 2" />
</bar-resources>

Configuration

How to make each Batch group log into a different file ?

This is done by:

  1. Using an Ubik Special Log4j Filter
  2. Using XML configuration file (Properties files cannot use Filters)
  3. Configure an Appender by group and add a Filter "com.ubikingenierie.console.log4j.MDCFilter" with a <param> element:
    • <param name="groupName" value="DEMO1"/>
  4. Add an <appender-ref> under each category

Example: 

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration debug='true' xmlns:log4j='http://jakarta.apache.org/log4j/'>
 
	<appender name="FILE" class="org.apache.log4j.RollingFileAppender"> 
		<param name="File" value="${LOG_ROOT_PATH}/demo-batch/all.log" /> 
		<param name="Append" value="true" /> 
		<param name="MaxFileSize" value="10MB" />
		<param name="MaxBackupIndex" value="10" />		
		<layout class="org.apache.log4j.PatternLayout"> 
			<param name="ConversionPattern" value="%d - %5p - %c: %m%n"/> 
		</layout> 
		<!-- Get all other events -->
		<filter class="com.ubikingenierie.console.log4j.MDCFilter">
		</filter>
	</appender>
 
	<!-- FILE -->
	<appender name="GROUP_DEMO1_APPENDER" class="org.apache.log4j.RollingFileAppender"> 
		<param name="File" value="${LOG_ROOT_PATH}/demo-batch/group-demo1.log" /> 
		<param name="Append" value="true" /> 
		<param name="MaxFileSize" value="10MB" />
		<param name="MaxBackupIndex" value="10" />		
		<layout class="org.apache.log4j.PatternLayout"> 
			<param name="ConversionPattern" value="%d - %5p - %c: %m%n"/> 
		</layout> 
		<!-- Special filter with Group name -->
		<filter class="com.ubikingenierie.console.log4j.MDCFilter">
			<param name="groupName" value="DEMO1"/>			
		</filter>
	</appender>
 
	<appender name="GROUP_DEMO2_APPENDER" class="org.apache.log4j.RollingFileAppender"> 
		<param name="File" value="${LOG_ROOT_PATH}/demo-batch/group-demo2.log" /> 
		<param name="Append" value="true" /> 
		<param name="MaxFileSize" value="10MB" />
		<param name="MaxBackupIndex" value="10" />		
		<layout class="org.apache.log4j.PatternLayout"> 
			<param name="ConversionPattern" value="%d - %5p - %c: %m%n"/> 
		</layout> 
		<!-- Special filter with Group name -->
		<filter class="com.ubikingenierie.console.log4j.MDCFilter">
			<param name="groupName" value="DEMO2"/>			
		</filter>
	</appender>
 
	<root>
		<level value="INFO"/>
		<!-- Add appender references -->
		<appender-ref ref="GROUP_DEMO2_APPENDER"/> 
		<appender-ref ref="GROUP_DEMO1_APPENDER"/> 
		<appender-ref ref="FILE"/> 
	</root> 
</log4j:configuration>

How to make each Batch group log into a different file ?

This is done by:

  1. Using an Ubik Special Log4j Filter
  2. Using XML configuration file (Properties files cannot use Filters)
  3. Configure an Appender by group/batch and add a Filter "com.ubikingenierie.console.log4j.MDCFilter" with 2 <param>(s) elements:
    • <param name="groupName" value="DEMO2"/>
    • <param name="batchName" value="ADDRESS_GENERATION_BATCH"/>
  4. Add an <appender-ref> under each category

Example: 

<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE log4j:configuration SYSTEM "log4j.dtd">
<log4j:configuration debug='true' xmlns:log4j='http://jakarta.apache.org/log4j/'>
 
	<appender name="FILE" class="org.apache.log4j.RollingFileAppender"> 
		<param name="File" value="${LOG_ROOT_PATH}/demo-batch/all.log" /> 
		<param name="Append" value="true" /> 
		<param name="MaxFileSize" value="10MB" />
		<param name="MaxBackupIndex" value="10" />		
		<layout class="org.apache.log4j.PatternLayout"> 
			<param name="ConversionPattern" value="%d - %5p - %c: %m%n"/> 
		</layout> 
		<!-- Get all other events -->
		<filter class="com.ubikingenierie.console.log4j.MDCFilter">
		</filter>
	</appender>
 
	<appender name="BATCH_DEMO2_ADDRESS_GENERATION_APPENDER" class="org.apache.log4j.RollingFileAppender"> 
		<param name="File" value="${LOG_ROOT_PATH}/demo-batch/address-generation.log" /> 
		<param name="Append" value="true" /> 
		<param name="MaxFileSize" value="10MB" />
		<param name="MaxBackupIndex" value="10" />		
		<layout class="org.apache.log4j.PatternLayout"> 
			<param name="ConversionPattern" value="%d - %5p - %c: %m%n"/> 
		</layout> 
		<!-- Special filter with Group name -->
		<filter class="com.ubikingenierie.console.log4j.MDCFilter">
			<param name="groupName" value="DEMO2"/>
			<param name="batchName" value="ADDRESS_GENERATION_BATCH"/>			
		</filter>
	</appender>
 
	<root>
		<level value="INFO"/>
		<!-- Add appender references -->
		<appender-ref ref="BATCH_DEMO2_ADDRESS_GENERATION_APPENDER"/> 
		<appender-ref ref="FILE"/> 
	</root> 
</log4j:configuration>

Development

How to make my batch send a Mail alert when an exception occurs

This is a built-in feature of the Ubik Batch Server, just ensure that your code do not catch silently exceptions

How to modify SQL Query parameters depending on business conditions

Sometimes Sql parameters change depending on some conditions that cannot be computed before execution, to enable this, Ubik Batch server offers a way to do it:

  1. Use SqlParameter in DaoSqlBatchStart or SqlBatchStart
  2. In Traitement#initialize() or BaseBatch#executeBeforeData, modify the parameters using: 
getBatchExecutionContext().setSqlBatchStartParameters(new SqlParameter[]{...});

How to pass data safely between initialize/exec/finalizeBatch

You can use BatchExecutionContext object: 

getBatchExecutionContext().setAttribute(...);
getBatchExecutionContext().getAttribute(...);

How to veto a batch

You can veto a batch (stop the execution) in the initialize method invoking method vetoExecution. For example if some conditions for execution are not met. 

vetoExecution("NOT A BUG! The treatment has been stopped by batch, functional flag is disabled");

If you want to stop a batch once you are in the exec method, you will have to manage by an applicative way.

How to check if your batch encountered a problem

You can check in the finalizeBatch method if there was a problem in your treatment. 

if (!getBatchExecutionContext().getExceptions().isEmpty())
{
    // do something
}

Authors

  • Philippe Mouawad

History

  • Page created by Pmouawad, 18 November 2007: New page: === Authors === * Philippe Mouawad === History ===
  • Last modified by Pmouawad, 10 January 2014: /* What is the difference between SUPER ADMIN and ADMIN in Mail notifications ? */
Retrieved from "http://wiki.ubik-ingenierie.com/ubikwiki/index.php?title=Ubik_Batch_Server_FAQ"
Personal tools