StreamBase Server Configuration XML

This topic provides reference information about the StreamBase Server configuration file, sbd.sbconf. To view the default (or skeleton) configuration, enter the following command at a terminal window: 

sbd -s

To write the default configuration to a file for customization, enter a command like the following example: 

sbd -s > /home/myaccount/myapps/mysbd.sbconf

streambase-configuration is the root element that contains all XML code describing a StreamBase Server configuration. It has one or more of the following child elements:

global

Identifies the locations of resources, such as custom function plug-ins or modules, that should be loaded when the StreamBase Server runs your application.

Normally, it is not necessary to identify such resources when they are stored and managed within StreamBase Studio. However, you do need to edit the global section to specify the locations of these resources when you deploy applications outside of StreamBase Studio.

In addition, even while working in StreamBase Studio, you may want to store some resources outside of StreamBase Studio. By identifying them here, you can enable applications in your project to locate those resources.

global has one or more of the following child elements:

Example 1. global

The following UNIX example assumes that you have given a value to the STREAMBASE_HOME environment variable: 

<global>
    <plugin directory="/usr/share/sbplugins"/> 
    <module-search directory="/usr/share/sbapps"/>
    <operator-resource-search directory="${STREAMBASE_HOME}/resources"/>
</global>

plugin

Identifies a custom simple or aggregate function plug-in to be loaded when the application runs. You can have as many plug-in elements as you need, each identifying a different library.

Note that for C++ custom functions, each function name must also be declared in a custom-functions.

Attributes

Attribute Required Description
directory Either this attribute or filename A directory containing one or more custom function plug-ins (simple and/or aggregate) to be loaded. The StreamBase Server scans each specified directory and loads all the files it finds there.
filename Either this attribute or directory A list of one or more paths to individual plug-in files for custom functions (simple and/or aggregate) that you want to load when your application runs. Unlike the plugin directory option, this option causes only the specified files to be loaded.

You can omit the file extension: StreamBase will load the platform-appropriate .so file on UNIX, or .dll file on Windows.


Note

When running StreamBase in daemon mode (with the -b flag), make sure you use fully qualified paths.

Example 2. plugin

See global.


module-search

Identifies a directory containing application modules (sbapp or ssql files) to be used by applications at run time. These external application modules then can be referenced using a module reference in an EventFlow application, or an APPLY statement in a StreamSQL application.

When sbd runs the application, it searches for the referenced module in several places (including this configuration file) using a pre-defined sequence. Please refer to Managing StreamBase Resource Files for the exact sequence.

There are two situations for defining module-search elements:

  • When you deploy an application, its configuration file must identify any modules that are referenced. For more information, see Deploying Applications in the Administration Guide.

  • When you are designing applications in StreamBase Studio, and the referenced modules are outside of your project's workspace instead of within StreamBase Studio.

Attributes

Attribute Required Description
directory Yes Directory where application modules (.sbapp or .ssql files) are stored at run time.

The directory must be local (or shared) to the runtime system, and StreamBase recommends using only full pathnames. In a clustered environment, the directory must exist on the StreamBase Cluster Manager node, sbclusterd.


Example 3. Example

See global.


operator-resource-search

Specifies a directory containing Java operator and adapter resources. These external resources can then be located by StreamBase applications at run time.

When sbd runs the application, it searches for the resources in several places (including this configuration file) using a pre-defined sequence. Please refer to Managing StreamBase Resource Files for the exact sequence.

There are two situations for defining operator-resource-search elements:

  • When you deploy an application, its configuration file must identify any Java operator and adapter resources used by the application. For more information, see Deploying Applications in the Admin Guide.

  • When you are designing applications in StreamBase Studio, and the resources are outside of your project's workspace instead of within Streambase Studio.

Attributes

Attribute Required Description
directory Yes Directory where Java operator or adapter files are stored at run time.

The directory must be local (or shared) to the runtime system, and StreamBase recommends using only full pathnames. In a clustered environment, the directory must exist on the StreamBase Cluster Manager node, sbclusterd.


Example 4. operator-resource-search

See global.


server

Specifies aspects of the StreamBase Server that relate to network I/O. Optionally designates a data directory for disk-based Query Tables (so that data can persist between StreamBase Server sessions). server has the following child elements :

  • One or more of: param with any of these name-value attribute pairs:

name Attribute value Attribute
tcp-port TCP port number to be used for network I/O. Default: 10000.
connection-backlog Number of backlogged connections. Servers with many clients may want to increase the maximum number of backlogged connections to the server. For further details, lookup the manual page for the system call listen. The default is 10.
client-heartbeat-interval The number of milliseconds between heartbeat packets sent to clients. Clients can be configured to require a heartbeat packet from the server at a minimum interval. This is used primarily for network segmentation detection. Setting client-heartbeat-interval to 0 will disable heartbeats from the server. Clients connected to such a server will not have heartbeat protection, regardless of their locally configured minimum heartbeat interval. The default value is 10000.
datadir Directory to be used for persistent data from any Query Tables in your StreamBase application. Note that the datadir value could come from other sources. The precedence order is:

  1. If present on the sbd command line --datadir value.

  2. The STREAMBASE_DATA environment variable's value.

  3. The value in this server section of the sbd configuration file.

  4. Finally, if not specified, or the special value "+TEMP+" is used, a temporary directory is created on server startup and deleted on shutdown. The default temporary locations are shown below:

    • On UNIX:

      /tmp/sbdata-six-random- characters/querytable/ table-name

    • On Windows:

      C:\Documents and Settings\username\Local Settings \Temp\sbdata-six-random- characters\querytable\ table-name

disk-querytable-cache Amount of main memory that will be allocated to any disk-based query table operations. The value units are in MB of main memory and must be a power of 2. When unspecified, the default value is 16 MB. If your application uses disk-based query tables, you may be able to improve performance by setting a higher value. However, use caution with setting this parameter, as too high a value may consume more memory than needed and could negatively impact other resources that require memory during the execution of the StreamBase application or other applications. As with any resource setting, you should establish baseline performance metrics and then test the effect of increasing or decreasing values under normal and peak loads.
disk-querytable-pagesize The page size used internally for disk-based Query Table operations. You may be able to improve performance by setting a higher value. This page size has nothing to do with the StreamBase page size, nor the operating system (OS) page size. The value units are in KB and must be a power of 2. Valid values are 0, 1, 2, 4, 8, 16, 32 and 64. The default is 0 and means use the OS-dependent default value.
disk-querytable-transaction_mode Mode of transaction support for disk-based Query Tables.

Enumeration: { 0 | 1 | 2 }

The default mode is 1, where transactions are enabled, but not flushed to persistent store in any predictable time. Mode 2 will transactionally commit every single table update to persistent store. Mode 0 disables transactional support. Please see the next parameter for a related setting.

disk-querytable-logsize If you are using disk-based Query Tables in mode 1 or 2, your Query Table performance may improve if you increase the size of transaction log files. The value units are in MB. The default is 10MB.
disk-querytable-flush-interval-ms If you are using disk-based Query Tables in transaction mode 1 or 0, writes to tables are not committed to disk in a predictable time frame. Setting the disk-querytable-flush-interval-ms parameter will force disk commits at the interval specified in milliseconds. Also a flush interval is needed in mode 1 and 2 in order to write a checkpoint and remove unneeded log files. The default interval for mode 0 is no flushing (0) and 30000 (30 seconds) for modes 1 and 2.
idle-enqueue-client-timeout-ms, idle-dequeue-client-timeout-ms Settings for disconnecting idle clients.

An idle enqueue client is a client who has enqueued at least one tuple and has been idle for idle-enqueue-client-timeout-ms. An idle dequeue client is a client who has subscribed to at least one stream (at any point) and has been idle for idle-dequeue-client-timeout-ms.

Clients that have enqueued and subscribed are subject to both settings. The server checks clients every idle-client-check-interval-ms. The actual point that a client is disconnected will be approximate modulo idle-client-check-interval-ms.

Values are in milliseconds. Values greater than zero turn these features on. The default is off.

idle-client-check-interval-ms How often the server should check for idle clients. The value is in milliseconds. The default is 60000 (60 seconds).
max-persistent-connections Maximum number of persistent connections. Each persistent connection uses up server resources. To protect the server from errant client connections a user can specify a maximum number of persistent connections. Any attempted client connections over the limit will be disconnected.

Default value = -1 (no limit).

operator-state-change-timeout-ms Each Java Operator or Embedded Adapter changes state along with the server process as a whole. The server waits for each Operator or Adapter to change state before it completes its state change. The value of operator-state-change-timeout-ms is the amount of time (in milliseconds) the server will wait before timing out the Operator or Adapter. If an Operator or Adapter is timed out on a state change, the server shuts down the Operator or Adapter, and proceeds with its own server state change. If unspecified in your sbd.sbconf, the default operator-state-change-timeout-ms value is 10000 milliseconds (ten seconds).

Example 5. server 

<server>
    <param name="tcp-port" value="101000"/>
    <param name="connection-backlog" value="12"/>
    <param name="datadir" value="/tmp/112233/querytable/last10stocks"/>    
    <param name="disk-querytable-cache" value="256"/>
    <param name="disk-querytable-transaction_mode" value="1"/>
    <param name="disk-querytable-flush-interval-ms" value="40000"/>  
</server>

page-pool

Defines memory parameters used by the StreamBase Server. page-pool has the following child elements:

  • One or more of: param with any of these name-value attribute pairs:

name Attribute value Attribute
page-size The size in bytes per page. The StreamBase Server (sbd) defaults to the proper minimum value for the machine's architecture. If you change the page-size value, it must be a multiple of the system page size: 4096 bytes for x86, or 8192 bytes for SPARC.

The StreamBase page size determines the maximum size of a tuple, so you will need to increase this value if you have large tuples. A related parameter is: param name="schema-max-size" .../ in the runtime

In environments where High Availability (HA) is enabled, the page-size value is part of a calculation to determine the total memory allocation for the StreamBase HA application:

page-size * pages-per-chunk * max-chunks

In non-HA environments, the initial memory allocation is determined by the JVM heap size for the hosted StreamBase application. If needed for StreamBase applications with large memory demands, you can set a higher initial allocation of heap by specifying -Xms and -Xmx JVM arguments, as described later in this configuration topic.

max-pooled-pages Sets the maximum number of pages that will be retained in the application-wide dequeuing client buffer pool. When sbd starts, this pool is empty. As client activity begins, it is dynamically sized to accommodate the sum of max-client-pages from all clients. If client demand subsequently recedes, a certain number of pages is retained in the pool. The default is 1024 pages.
max-client-pages Sets the maximum number of pages that a dequeue client connection can allocate. The sbd process will disconnect clients that try to allocate more. This parameter is designed to protect sbd from a slow or hung dequeue client. Therefore to understand the resulting number of bytes allowed, multiply the max-client-pages value by the page-size value. A max-client-pages value of 2048 pages results in about 8 megabytes. If you want to enable a dequeue client connection that would allocate unlimited memory use, change the max-client-pages value to "0".
pages-per-chunk Sets the number of pages to allocate at a time in the page pool. Defaults to 1024, or 4MB at a time (with 4K page size).
max-chunks Sets the maximum number of chunks to allocate. Defaults to 128 chunks, or 512MB total (4MB * 128).

Example 6. page-pool

In the following configuration example, the administrator decided to increase the page-size and the Java Heap for the StreamBase Server that will host the application: 

<page-pool>
    <param name="page-size" value="8192"/>    
</page-pool>

For related details, see the java-vm section below.


runtime

Use this element to specify one or more containers (one per application) that will be hosted by the StreamBase Server process, and other aspects of the server's runtime behavior. This includes selecting the now() function's implementation. You can have one or more application elements in the runtime section. The runtime section has the following child elements:

  • One or more of: application

  • One or more of: param with any of these name-value attribute pairs:

name Attribute value Attribute
schema-max-size The maximum size of a stream's schema; that is, the total number of bytes of all fields you expect in the tuple. The default value is 1048576 bytes, which is 1 MB. While there is no theoretical upper limit, in practice there is a maximum based on the available memory on the machine(s) that will host the processing of your application. Be cautious about creating very large schemas, because applications that move smaller amounts of data perform much better than applications that move tuples containing unnecessary fields.
cpp-string-field-size A default number of characters for strings that are returned between the runtime and C++ custom functions. Default: 32768.

  • If the C++ custom function developer used the setReturntype method to set the string size, the default size is set at runtime by this parameter.

  • If the custom function used the setReturnString method to hard-code a string size, that value is used at run time, and this attribute is ignored.

now-implementation StreamBase provides a now() function to return the current time. With this parameter, you can select one of two now() implementations. Default: system.

  • The system value means that the now() function will use Java's System.currentTimeMillis(). This is the default.

  • The thread value means that the now() function will use a background thread that checks the time approximately every millisecond. This option will result in decreased accuracy, but may be more efficient than "system" if you call now() more frequently than 1000 times per second.

application-file The application-file attribute of the runtime element was deprecated as of StreamBase 3.7. Use the application element instead.

Example 7. runtime 

<runtime>
  <application file="myapp.ssql" container="A"/>
  <application file="app.ssql" container="B">
    <container-connection dest="B.InputStream1" source="A.OutputStream1"/>
    <container-connection dest="B.InputStream2" source="A.InputStream2"/>
  </application>
  <param name="cpp-string-field-size" value="65536"/>
  <param name="now-implementation" value="system"/>  
</runtime>

For related details, see Addressing Entities in Containers in Using Containers to Organize and Connect Multiple Applications, and the java-vm section below.


application

Use this element to specify the .sbapp or .ssql application to load into a named container.

Attributes

Attribute Required Description
file Yes

An .sbapp or .ssql file.

container Yes The name you assign to the container.

application has the following child elements:

Example 8. application 

<application file="myapp.ssql" container="A"/>
<application file="app.ssql" container="B">
  <container-connection dest="B.InputStream1" source="A.OutputStream1"/>
  <container-connection dest="B.InputStream2" source="A.InputStream2"/>
</application>

container-connection

Use this element to identify a container for this server. In the case of multiple containers, the dest and source attributes can define connections between the containers. For notation and guidelines, please refer to the section on addressing entities in Using Containers to Organize and Connect Multiple Applications.

Attributes

Attribute Required Description
dest Yes

The entity (such as an Input Stream) in one named container that will consume the output of the connected source container.

source Yes The entity in one named container (such as an Output Stream) that will send the data to the destination container.

Example 9. container-connection

See the application example.


custom-functions

Element for all the declarations for a custom function. Has the following child elements:

Example 10. custom-functions

The custom-functions section is used for two reasons: to register the type signatures of custom C++ functions used in the server, contained in libraries imported using the <plugin> tag in the <global> section above; and to assign short name aliases to C++ or Java functions so they can be called in the same way as built-in functions.

In the following example, a custom simple function named func1 takes a string and an int argument and returns a string of size 16: 

<custom-functions>
    <custom-function name="func1" type="simple" >
      <args>
        <arg type="int" />    
        <arg type="string" /> 
      </args>
      <return type="string" size="16" />
    </custom-function>
</custom-functions>

Add the alias attribute to tell StreamBase to allow you to use a short function name for a given function. For example, the following would allow you to call the same function in StreamBase expressions in the form f(myString, myInt): 

<custom-functions>
    <custom-function name="func1" type="simple" alias="f">
      <args>
        <arg type="int" />    
        <arg type="string" /> 
      </args>
      <return type="string" size="16" />
    </custom-function>
</custom-functions>

Finally, you can specify the attributes language to be java and class to be the fully qualified class name where a named public static function is found. This allows you to specify short names or alternate names for arbitrary Java functions. You can also specify the attribute args to be auto and leave out the args and return-type tags to ask StreamBase to autodetect them. For example: 

<custom-function language="java" class="java.lang.Math" 
    name="log10" alias="log_base_ten" args="auto"/>

custom-function

Each C++ simple or aggregate plug-in function must be declared in a custom-function element. Each function must also be imported into a directory specified in the global plug-in element. C++ and Java functions can be assigned aliases. Java functions from public Java classes can be aliased for use as if that function were embedded in your StreamBase application.

Attributes

Attribute Required Description
name Yes

Must match the name used in the application expression that calls the function.

type Yes Enumeration: { simple | aggregate }
alias No A substitute name for this function, to be called as alias-name() instead of using calljava().
language No Set to "java".
class No The fully qualified class name where a named public static function is found.
args No Set to "auto" to have the StreamBase Server autodetect the arguments of the public function.

Child Elements

args

Container element for all the argument declarations for a function. Has the following child elements:

  • One or more of: arg

arg

Defines an argument used in a function.

Attributes

Name Required Description
type Yes The data type for each argument that the function expects to be passed from the StreamBase application.

return

Defines the data returned by a function.

Attributes

Name Required Description
type Yes The data type to be returned from the function to the StreamBase application.
size A string requires a size attribute; other data types have implicit sizes The size to be returned, in bytes.

sbmonitor

Use this element to define characteristics of the statistics that will be generated for use by the sbmonitor. sbmonitor has the following child elements:

  • One or more of: param with any of these name-value attribute pairs:

name Attribute value Attribute
enabled Either true or false.

Set to true to enable statistics collection, or false to disable it. If it is set to false, sbmonitor will not be usable with the daemon. The default is true.

period-ms Specifies the period, in milliseconds, over which statistics are aggregated. The default is 1000 ms (every second).

Example 11. sbmonitor 

<sbmonitor>
    <param name="enabled" value="true"/>
    <param name="period-ms" value="2000"/>
</sbmonitor>

security

The security section controls security and authentication in the sbproxy and sbd. If there is a security section, the sbd will only listen for connections on localhost, so connections from remote computers cannot be made. If there is a security section in the sbconf file, the sbproxy server will allow only SSL connections from clients.

Note

This section cannot be configured together with the authentication section, which provides a different authentication method.

security has one or more of the following child elements:

  • One of: ssl-authentication

  • One param element using this name-value attribu te pair:

    name Attribute value Attribute
    perform-authentication
    true

    Restrictions to user actions will be added in this section.

    false

    All users will be able to perform any action. The client connections will still be SSL, but no further authentication will be performed.

  • Zero or one of: transform-principal

  • Zero or one of: client-ssl-connections

  • Zero or more of: ldap

  • Zero or more of: role

  • Zero or one of: user-data

Example 12. security 

<security>
  <ssl-authentication>
    <param name="keystore" value="../test/proxy/security/signed.keystore"/>
    <param name="keystore-password" value="secret"/>
  </ssl-authentication>

  <param name="perform-authentication" value="false"/>

  <transform-principal>
    <param name="searchRegexp" value="cn=([^,]*).*"/>
    <param name="replaceRegexp" value="$1"/>
  </transform-principal>

  <client-ssl-connections>
    <param name="require-client-authentication" value="true"/>
    <param name="cipher-suite" value="SSL_RSA_WITH_3DES_EDE_CBC_SHA"/>
  </client-ssl-connections>

  <ldap>
    <!-- a TLS auth server -->
    <server host="parallels" port="838">
      <param name="principal-root" value="ou=Users,dc=example,dc=com"/>
      <param name="principal-search" value="cn={0}"/>
      <param name="keystore" value="ldapuser.keystore"/>
      <param name="keystore-password" value="secret"/>
    </server>
  </ldap>

  <ldap>
    <!-- a basic auth server -->
    <server host="parallels" port="839">
      <param name="principal-root" value="ou=Users,dc=example,dc=com"/>
      <param name="principal-search" value="cn={0}"/>
      <param name="root-dn" value="cn=Manager,dc=example,dc=com"/>
      <param name="password" value="secret"/>
    </server>

  <role name="StreamBaseSuperuser">
    <param name="action" value="Administrate"/>
  </role>

  <role name="InnocentBystander">
    <param name="action" value="Status"/>
    <param name="action" value="Enqueue default.InputStream1"/>
    <param name="action" value="Dequeue default.OutputStream1"/>
  </role>

  <user-data>
    <user>
      <param name="cn" value="cn=Alice Pleasance Liddell,ou=Users,dc=example,dc=com"/>
      <param name="role" value="InnocentBystander"/>
      <param name="role" value="StreamBaseSuperuser"/>
      <param name="password" value="secret"/>
    </user>
    <user>
      <param name="cn" value="EMAILADDRESS=bobn@streambase.com, CN=Bob Newhart, 
        OU=Users, O=StreamBase Systems, L=Lexington, ST=Massachusetts, C=US"/>
      <param name="role" value="InnocentBystander"/>
    </user>
  </user-data>

</security>

ssl-authentication

Specifies the keystore that holds the signed X.509 server certificate that will be used for SSL connections between clients and the sbproxy server.

ssl-authentication has the following child elements:

  • Two param elements, each using one of these name-value attribute pairs:

    name Attribute value Attribute
    keystore URI of the keystore that identifies a certificate authority (CA). Alternatively, the URI can point to a trust store, which defines multiple CAs that your organization recognizes.
    keystore-password The password to access the keystore.

transform-principal

Defines a search/replace regular expression that will change the name of the principal. The transform in the preceding security example extracts the CN section from the entire principal's name as it appears in a certificate.

transform-principal has the following child elements:

  • Two param elements, each using one of these name-value attribute pairs:

    name Attribute value Attribute
    searchRegexp Matches on a pattern in a field in an LDAP database.
    replaceRegexp Specifies the replacement pattern.

client-ssl-connections

Defines the type of SSL connection and the external tools used for encryption.

client-ssl-connections has the following child elements:

  • One of: param with this name-value attribute pair:

    name Attribute value Attribute
    require-client-authentication
    false

    SSL connections will be accepted from clients that do not have a signed user certificate. This is sometimes called 1-way SSL, and is the default value. The principal and password attributes (user= and password= attributes) must be specified in the URI, unless the perform-authentication parameter is false (see below)

    true

    Only client connections that have a certificate signed by a known certificate authority will be allowed. This is sometimes called 2-way SSL. The principal will be taken from the certificate.

  • Zero or more of: param with this name-value attribute pair:

    name Attribute value Attribute
    cipher-suite Comma-separated list of cipher suites that can be used for SSL encryption. If this element is not listed, sbproxy uses all the cipher suites that are installed with the JVM. When this element is present, sbproxy is limited to the suites you list. Typical suites for the JDK 1.5 JVM include: SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA, SSL_RSA_WITH_NULL_MD5, SSL_RSA_WITH_NULL_SHA, SSL_DH_anon_WITH_RC4_128_MD5, TLS_DH_anon_WITH_AES_128_CBC_SHA, SSL_DH_anon_WITH_3DES_EDE_CBC_SHA, SSL_DH_anon_WITH_DES_CBC_SHA, SSL_DH_anon_EXPORT_WITH_RC4_40_MD5, SSL_DH_anon_EXPORT_WITH_DES40_CBC_SHA, TLS_KRB5_WITH_RC4_128_SHA, TLS_KRB5_WITH_RC4_128_MD5, TLS_KRB5_WITH_3DES_EDE_CBC_SHA, TLS_KRB5_WITH_3DES_EDE_CBC_MD5, TLS_KRB5_WITH_DES_CBC_SHA, TLS_KRB5_WITH_DES_CBC_MD5, TLS_KRB5_EXPORT_WITH_RC4_40_SHA, TLS_KRB5_EXPORT_WITH_RC4_40_MD5, TLS_KRB5_EXPORT_WITH_DES_CBC_40_SHA, TLS_KRB5_EXPORT_WITH_DES_CBC_40_MD5

ldap

Specifies information for connecting to and searching LDAP servers to manage user authentication. Create a separate ldap element for each LDAP server. If more than one LDAP server is defined, the requests will be sent to each ldap server in turn, in a round-robin manner. You can configure one or more LDAP servers.

ldap has the following child elements:

server

Identifies an LDAP server that you want to connect to, and specifies the type of authentication to be used.

Attributes

Attribute Required Description
host Yes The address presented to clients.
port Yes The port number used by the ldap server.

server has the following child elements:

  • Two of: param elements, each with one of these name-attribute pairs:

    name Attribute value Attribute
    principal-root The Distinguished Name pattern that describes the directory information tree on the LDAP server, from the required entry to the directory root. Principal names will be applied to the pattern during authentication.
    principal-search The field name and index within the LDAP to use when looking up principals.

  • One of the following groups of elements:

    To authenticate in the sbconf file

    Two of: param, each with one of these name-attribute pairs

    name Attribute value Attribute
    root-dn The Distinguished Name pattern that identifies a valid user to the LDAP server.
    password The password that must be entered to authenticate a user.
    To authenticate in LDAP (TLS)

    Two of: param, each with one of these name-attribute pairs

    name Attribute value Attribute
    keystore URI of the keystore that identifies users for the LDAP server.
    keystore-password The password to access the keystore.

role

Use the role element to define a role and assign its actions. You can define any number of roles and use any role names, each with a different combination of actions. Later (in the user-data element) you can assign users to roles.

When a client attempts to perform an action, such as dequeue OutputStream1, the user is authenticated, then the authentication source (an LDAP server or a user-data in the sbconf file) is asked for all of the roles the user has. If the user has as role which has the dequeue default.OutputStream1 action listed, then the dequeue operation is allowed. .

Attributes

Attribute Required Description
name Yes A name for a group of actions. You can assign any name.

role has the following child elements:

  • One or more param elements with this name-attribute pair:

name Attribute value Attribute
action

By assigning actions, you determine the commands that a user is authorized to use (and, by omission, not use). Specify one of the following:

Administrate

Matches Administer actions in the LDAP server, and enables the following commands:

shutdown, shutdownsecondary, clusterstart, clustershutdown, suspend, resume, restart, getOperatorProperty, setOperatorProperty, addContainer, removeContainer, checkpoint, loadcheckpoint, listConnections, typecheck
Status

Matches Status actions in the LDAP server, and enables the following commands:

status
isCluster
describe
list
Enqueue container.stream

Matches Status actions in the LDAP server. Enables the user to enqueue the specified stream.

Dequeue container.stream

Matches Status actions in the LDAP server. Enables the user to dequeue the specified stream.

user-data

Defines the Principal-to-role mapping for each user. This section should not be defined if an ldap section is defined.

user-data has the following child elements:

  • One or more of: user

user

A user corresponds to a Principal in an LDAP database. Specifies authentication data and assigns user roles.

user has the following child elements:

  • One of: param with the following name-attribute pair:

    name Attribute value Attribute
    cn The full Distinguished Name of the Principal.

  • One of: param with the following name-attribute pair:

    name Attribute value Attribute
    password The user's password.

  • Zero or more of: param with the following name-attribute pair:

    name Attribute value Attribute
    role One of the roles defined in a role element.

authentication

Use this element to define the authentication parameters that will be used to protect the StreamBase application from unauthorized access. Note that this element must not be present if the node is part of a cluster. For clusters, authentication parameters are specified in the cluster manager's configuration (sbclusterd.sbconf) rather than individual server configurations.

Note

The authentication element cannot be configured together with the security element. You can only use one of these security methods.

authentication has the following child elements:

  • One or more of: param with any of these name-value attribute pairs:

name Attribute value Attribute
enabled Enumerated: { true | false }. Set to true to require authentication information from clients, or false to allow access from anyone. The default is false, but it should be set to true in production systems.
type Enumerated: sb-password-file. This is the only value allowed. It specifies the type of authentication to use to verify user names and password. Required if authentication is enabled.
filepath The path to the sbpasswd file containing user names and passwords. Required if authentication is enabled. The default is passwd in the etc subdirectory of your StreamBase installation.

Example 13. authentication 

<authentication>
    <param name="enabled" value="true"/>
    <param name="type" value="sb-password-file"/>
    <param name="filepath" value="/opt/streambase/etc/passwd"/>
</authentication>

java-vm

Use this element to set initial min and max values for the JVM memory allocation pool. It's important that you examine, test, and modify these values (as needed) for all StreamBase applications. See the table below for caveats.

In addition, the java-vm element is required when:

  • When your application uses Java operators.

  • To load any jars and shared libraries needed by database drivers declared in the data-source element. Use the jar and library child elements for this purpose.

For Java operators you must specify, at a minimum, the jar file that contains the Java operator classes. For details about this feature, see Using the StreamBase Java Operator Wizard in the API Guide.

Note

If you use this section to override the JDK version, remember that you must be using at least the Sun JDK version 1.4.2_07 or later for the 1.4.2_xx versions, or 1.5.0_06 or later for the 1.5.0_xx versions. For the latest information, see the Supported Configurations topic in the Installation Guide. java-vm has the following child elements:

  • Optionally one or more of: jar

  • Optionally one or more of: dir

  • Zero or one of: library

  • Zero or one of: param for each of these name-value attribute pairs:

name Attribute value Attribute
java-home The path to the installed location of the full JDK on your machine. For example: 
<param name="java-home" 
    value="/usr/java/j2sdk1.5.0_06" />

Or: 

<param name="java-home" 
    value="C:\Program Files\Java\jdk1.5.0_06" />

Note

Refer to the Supported Configurations topic in the Installation Guide for details about the required minimum JDK versions.

jvm-args Arguments to send to the JVM. You can set the initial min and max values for the memory allocation pool, Java properties, or other JVM options.

Do not set the classpath here. Also do not specify -server or -client, because the server JVM is always loaded.

Important

  • Although this setting is not required, it is is strongly recommended that you check the heap size arguments

  • As you probably read elsewhere in the documentation, you must have the full Sun JDK installed to run a StreamBase application. This requirement exists on development, staging, and production environments. At runtime, your application is compiled into one or more Java classes. To ensure that your application performs well, it's critical that you use this java-vm section to set the Java Virtual Machine's -Xms (min) and -Xmx (max) heap-size arguments

The default maximum for the JVM is 256 MB. For example: 

<param name="jvm-args" 
    value="-Xms64m -Xmx256m"/>

At runtime, this default may be insufficient for your StreamBase application. We recommend that you experiment with the JVM arguments for memory use and, as needed, increase the values. For example: 

<java-vm>
    <param name="java-home" 
    value="C:\Program Files\Java\jdk1.5.0_06" />
    <param name="jvm-args" 
    value="-Xms512m -Xmx1024m" />
</java-vm>

Additional Cautions

  • If you need to view information about the Java Just-In-Time (JIT) compiler (HotSpot), try adding this JVM argument in the configuration file: 

    -XX:+PrintCompilation

  • If the JVM is using an excessive amount of memory, or spending too much time performing garbage collection, try adding this argument to get more specific information: 

    -verbose:gc

    With this argument, the JVM will periodically generate output telling you how large the Java heap size is and how much time is being spent in garbage collection. If you notice that a large amount of time is being spent in garbage collection (especially "Full GC" phases), please contact your StreamBase representative.

  • If the generated code for StreamBase operator execution is not optimal, try adding -Xprof to the set of JVM arguments. Run your application again, and then send the generated information to your StreamBase representative.

You may find more information about JVM arguments by going to the Sun JDK Tools and Utilities Web page, and navigating to the Java application launcher topic for your platform.

How sbd searches for JVM

In using the java-vm element, it helps to understand how the StreamBase Server instance (sbd) that hosts and processes your application attempts to find a JDK. If any of the following ordered tests succeeds, no further tests are performed:

  1. Uses the java-home attribute in the sbd.sbconf configuration file, if it is defined (it is undefined at install time).

  2. Uses the internal JDK if present and if the STREAMBASE_USE_INTERNAL_JDK environment variable has not been set to FALSE (it is TRUE by default).

  3. Uses the JAVA_HOME environment variable, if defined on your system.

  4. Searches through your PATH environment variable to find a suitable JDK.

  5. Attempts to find a JDK on your system.

  6. If all of the preceding steps fail, sbd will not start.

Example 14. java-vm 

    <java-vm>
      <param name="java-home" value="/usr/java/j2sdk1.5.0_06" />
      <param name="jvm-args" value="-Xms512m -Xmx1024m"/>
      <jar file="olive.jar"/>
      <jar file="othercalcs.jar"/>
      <dir path="/opt/apache/classes""/>
      <dir path="/usr/share/maths/classes""/>
      <library file="native-lib"/>
      <library file="other-lib"/>
    </java-vm>

jar

Specifies a the jar to be loaded at run time, used by a Java Operator or by a JDBC datasource server. Required within a parent java-vm element when Java operators or JDBC Tables are used in the StreamBase application. You can specify multiple jar files, as shown in the java-vm example.

Attributes

Attribute Required Description
file Yes

Path to the jar file.


dir

Use the dir element to add a directory to the classpath. You can use multiple dir elements, as shown in the java-vm example.

Attributes

Attribute Required Description
path Yes

Directory specification to add to the classpath.


library

Specifies a path that will be prepended to the JVM java.library.path when loading native libraries used by your Java code, or libraries used by drivers for JDBC datasources. To load multiple libraries use multiple library elements. See java-vm for an example.

Attributes

Attribute Required Description
file Yes

Path to the library.


data-sources

Container element for data-source declarations, which identify all external database drivers accessed by StreamBase applications using the server. You can define as many datasources as you need, each pointing to a single server. Chronicle and JDBC databases are supported. data-sources has the following child elements:

Example 15. data-sources 

<data-sources>
  <data-source name="myDB" type="jdbc">
    <uri value="jdbc:derby://localhost:1527/myDB;user=me;password=mine"/>
    <driver value="org.apache.derby.jdbc.ClientDriver"/>
    <param name="jdbc-fetch-size" value="10"/>
    <param name="jdbc-timeout" value="15000"/>
    <param name="jdbc-max-column-size" value="2048"/>
    <param name="user" value="me"/>
    <param name="password" value="mine"/>
  </data-source>
</data-sources>

<data-sources>
  <data-source name="myVertica"  type="chronicle">
    <uri value="jdbc:vertica://MyHost:5433/chron_vertica"/>
    <driver value="com.vertica.Driver"/>
    <param name="user" value="MyUser"/>
    <param name="jdbc-share-connection" value="true"/>
    <param name="jdbc-quote-strings" value="true"/>
  </data-source>
</data-sources>

<data-sources>
  <data-source name="msql-dev1"  type="jdbc">
    <uri value="jdbc:mysql://dbserver:3306/qa?user=username&password=secret"/>
    <param name="jdbc-reconnect-regexp" value="Communications link failure"/>
    <param name="jdbc-reconnect-regexp" value="Connection reset"/>
    <param name="jdbc-reconnect-attempts" value="5"/>
    <param name="jdbc-reconnect-sleep" value="250"/>
  </data-source>
</data-sources>

data-source

This element identifies a server to be used with a datasource, and accessed in an EventFlow or StreamSQL application. Sets parameters for JDBC and Chronicle operations, and loads the drivers required by the server.

Note

For a JDBC or Chronicle data source, you must also edit the java-vm element, adding jar and library child elements for the jars and shared libraries needed by the database driver.

Attributes

Attribute Required Description
name Yes A unique identifier for the server.
type Yes value: { jdbc | chronicle }

data-source has the following child elements:

  • One of: uri

  • One of: driver

  • Zero or more of: param with jdbc-retry-sqlstate attribute.

  • Zero or one of: param with any of the other name-value attribute pairs.

Each param within the data-source element specifies one of the following name-value attribute pairs:

name Attribute value Attribute
jdbc-fetch-size This param can be used to control the size of the buffer used by the JDBC driver when fetching rows. Consider limiting fetch size when your StreamBase application experiences memory problems related to large result sets that are returned from queries to the JDBC datasource. Consider increasing fetch size deadlocks occur because you are performing multiple queries against the same JDBC data source and query result sets are larger than the current database fetch size.

Possible values:

  • 0: Uses the JDBC driver's default value (default).

  • -2147483648 (For MySQL databases only): Normally, MySQL retrieves complete result sets and stores in them in memory. Setting the Integer.MIN_VALUE in this parameter serves as a signal to the MySQL driver to stream result sets row-by-row instead. Caution For other databases, a negative value causes an SQLException.

  • positive integer (for non-MySQL databases): A positive integer serves as a "hint" to non-MySQL JDBC drivers for the size to allocate for result set buffers. The actual size allocated depends on how a given driver interprets it, which may vary among drivers. A positive integer is ignored by MySQL drivers.

jdbc-timeout The timeout (in milliseconds) of background JDBC operations in StreamBase Studio. StreamBase Studio communicates with the datasource server in order to typecheck an application that has a JDBC datasource connection. If the datasource server does not respond during this interval, the typecheck fails. The default value of 15 seconds is adequate for normal local area network connections; consider increasing this value if you experience typecheck failures due to a slow network connection. For example, this setting increases the interval to 25 seconds: 
<param name="jdbc-timeout" value="25000"/>
jdbc-max-column-size The maximum column size expected in a JDBC table. Columns that are larger than this value (2048, by default) will cause a typecheck exception. If you override the default, remember that the resulting output tuple's schema must entirely fit into the pool-size. Therefore, also consider increasing the page pool-size, described in page-pool.
jdbc-retry-sqlstate Retry any select, insert, or update jdbc operation that returns the specified sqlstate for this database driver. The operation is retried until it succeeds.

A useful example is when you know that the database may return an exception due to transaction deadlocks in the database. When a deadlock occurs the database returns a specific exception sqlstate indicating deadlock/timeout, and the expected behavior is to roll back the operation or transaction and retry.

You can use multiple params to test for multiple sqlstates.

jdbc-batch-size Enables a Query operator to perform an insert or update operation in batch mode, and sets the number of statements that will be executed at one time. For example, if jdbc-batch-size is set to 10, then when the first nine tuples arrive at the query operator, they are held by the JDBC connection instead of being executed immediately. When the tenth tuple arrives and is added to the batch of statements, the entire batch is executed at once.

If an update is interrupted normally (for example, by stopping the sbd server), the Query operator updates the JDBC table using all the accumulated statements, even if there are fewer than the specified number. However, if the application is stopped abnormally, the updates may not occur.

The value must be an integer greater than 1 (a value of 1 is valid, but not meaningful for a batch operation).

Note that your application may require more memory when you perform query updates in batch mode.

jdbc-reconnect-regexp

If a connection is lost between the StreamBase Server and the database server, the next attempted JDBC operation returns an error. There are no standard error codes (or SQLSTATE) for communications failures, so to detect these errors, we must parse the text of the database-specific error messages. Use the jdbc-reconnect-regexp parameter to specify a regular expression to be compared to any received error messages. If one or more specified string is matched in the text of any error message, the StreamBase server assumes a communications failure has occurred, and attempts to open a new connection to the database server.

You can specify more than one jdbc-reconnect-regexp parameter to specify several error message strings to be considered communications failures. Each specified string is compared in order to the received error message text.

jdbc-reconnect-attempts

The jdbc-reconnect-attempts parameter controls the number of reconnection attempts a StreamBase server will attempt. If the connection to the database server is lost, the configuration example above attempts to establish a connection to the databse server five times. If five connection attempts fail, an error message is printed and the StreamBase server exits with error status. If a connection is reestablished to the databse server, and then the connection is lost some time thereafter, the reconnection count is reset so that five attempts to reconnect will be made again.

A jdbc-reconnect-attempts value of -1 specifies that the server is to continue reconnection attempts without limits. A value of zero disables reconnection attempts.

jdbc-reconnect-sleep

If the jdbc-reconnect-sleep parameter is specified, reconnection attempts are delayed by the specified number of milliseconds. If no jdbc-reconnect-sleep parameter is used, there will be no delay between reconnection attempts.

Just like the initial connection to the database, reconnection attempts use the jdbc-timeout parameter to limit how long the StreamBase server waits to establish a connection. The default value for jdbc-timeout is 15000 milliseconds (15 seconds).

user Username to log in to the datasource server.
password Password to log in to the datasource server.

uri

Use the uri element to specify the location of the server used to access the datasource. Each type of supported database, as shown in the data-sources example.

Attributes

Attribute Required Description
value Yes A URI for the server in this format: 
jdbc:database-prefix://address

Each kind of database requires a specific database-prefix, which must be one of these values:

  • derby (for a JDBC database)

  • db2 (for a DB2 database)

  • sybase (for a Sybase database)

  • vertica (for a Vertica database)

Note that if the URI value contains ampersands, replace them with &amp;; otherwise you will get an error about an "Unterminated entity reference".


Example 16. data-source

The following excerpts show examples of the data-source and uri for several supported databases. See Supported Configurations for the currently supported databases.

JDBC 
<data-source name="myDB" type="jdbc">
  <uri value="jdbc:derby://localhost:1527/myDB;user=me;password=mine"/>
DB2 
<data-source name="DB2" type="chronicle">
  <uri value="jdbc:db2://db2dev:50000/qa_db"/>
Sybase 
<data-source name="SybaseIQ" type="chronicle">
  <uri value="jdbc:sybase:Tds:localhost:2642/chron_db"/>
Vertica 
<data-source name="vertica" type="chronicle">
  <uri value="jdbc:vertica://MyHost:5433/chron_vertica"/>

driver

Identifies the database driver, as shown in the data-sources example.

Attributes

Attribute Required Description
driver Yes The class of the database driver.

operator-parameters

This element is a container for operator-parameter declarations that you want to use as expression parameters in operators. Declared parameters apply to any application that is run under the configured sbd (for example in StreamBase Studio, any application in the sbd.sbconf file's project).

operator-parameters has the following child elements:

operator-parameter

Use this section to declare each operator-parameter and assign its value. Specify the following name-value attribute pair:

Attributes

Attribute Required Description
name yes

Parameter label that can be used in an expression.

value yes Value to which the parameter is resolved.

Example 17. operator-parameters 

<operator-parameters>
    <operator-parameter name="MyStringParam" value="somestring"/>
    <operator-parameter name="MyIntParam" value="2"/>
</operator-parameters>

Applying this example, the following expressions could be used:

SELECT * FROM SourceTagger_1
"${MyStringParam}" AS source
...
WHERE myint > ${MyIntParam}


Note

If an operator-parameter name conflicts with a declared module parameter name, the module parameter value is used.

error-handler

The optional error-handler element is not listed in the skeleton STREAMBASE_HOME/etc/sbd.sbconf file. However, if desired, you can add it to override the default sbd server action, which is to shut down (in most cases) when errors occur. There are some exceptions to this rule, as noted below.

In other words, the purpose of this optional section is to allow you to ignore or continue running the sbd even if errors occur.

error-handler has the following child elements:

Example 18. error-handler 

<error-handler>
<error type='eval-error' action='ignore'/>
</error-handler>

error

Defines a single error type for the error handler. Use multiple error elements to define multiple error types, as shown in the error-handler example.

Attributes

Attribute Required Description
type Yes

  • eval-error

    This exception is caused by a variety of reasons, including errors that occur when evaluating expressions or when flushing disk-based query table logs, and incorrect timestamp formats. Continuing after such errors is not recommended.

  • eval-plugin-error

    Java and C++ plug-ins can get eval-plugin-error exceptions for a variety of reasons, including TupleExceptions, errors, and when setting parameters. Continuing after such errors is not recommended.

  • ordering-error

    The Gather operator throws this error when an order by field value is null. The Aggregate operator throws it when an order by field value is null or the current value is less then a previous value.

action Yes

  • shutdown: Stop the server

  • continue: Continue and display error

  • ignore: Continue and display no error

Note

The types default to a shutdown action, except for ordering-error, which defaults to continue, as shown in the error-handler example.