StreamBase New and Noteworthy

The following method has been added to the Java API:

public void sendErrorOutput(String message)

Use this method to generate error tuples from adapter and operator methods that are not invoked directly or indirectly through the StreamBase operator API. For example, an input adapter that processes messages through a listener interface can use this new method to generate an error tuple.

An adapter or operator typically generates an error tuple by throwing an exception from its processTuple method. However, input adapters that process messages through a listener interface cannot use this technique, because the thrown exception would be caught by the library delivering the message to the adapter rather than by StreamBase Server.

If the configuration file for a JMS Embedded Output Adapter specifies just one server and that server's specification contains just one destination, the schema of tuples processed by the adapter does not require server and destination fields (__JMSServerName and __JMSDestinationName).

The EMS/JMS adapter installation kit previously supported one embedded input adapter and a pair of input and output external adapters. This release adds a fourth adapter to the kit, the EMS/JMS embedded output adapter. The message converter API for using all four EMS/JMS adapters changed, and some limitations in the adapters were fixed. The EMS/JMS adapters now support the TIBCO EMS RELIABLE_DELIVERY mode. See EMS/JMS Input and Output Adapters for details.

The documentation for the SMTP Reader adapter was updated to clarify that the prerequisite libpcap (Linux) or WinPcap (Windows) libraries must be installed and working before using the adapter.

The Feed Simulation Editor in the Test/Debug view was updated to better manage mutually exclusive options during selection and save.

The current Reuters RMDS Subscribing Input Adapter and the deprecated Reuters Subscribing Input Adapter were both updated to emit update tuples only when one or more fields of interest change.

The new Reuters RMDS Subscribing Adapter allows a StreamBase application to receive market data from both Reuters RMDS 5 and RMDS 6 servers. This adapter supersedes the Reuters OMM Subscribing adapter (supporting RMDS 6 only) that was introduced in release 5.1.2. The new RMDS adapter is intended to replace the Reuters Subscribing adapter (supporting RMDS 5 only), which is being phased out.

The Reuters RMDS Subscribing Adapter is described in Reuters RMDS Subscribing Input Adapter.

The new Binary File Reader Input Adapter reads structured binary files created with the new Binary File Writer Output Adapter. The adapters are intended to be used as an adapter pair to write a stream of tuples to a file, then read them back from that file at a different point in your application. You might use the Binary File Writer and Reader as part of a High Availability scenario to preserve the contents of a tuple stream to disk. As part of a failover event, the secondary server in a StreamBase Server cluster pair could restore the contents of a stream with a fast read of the saved binary file.

The Binary File Reader Input Adapter is described in Binary File Reader Input Adapter. The Binary File Writer Output Adapter is described in Binary File Writer Output Adapter.

StreamBase releases 5.1.0 introduced a collection of new features to support clustering of StreamBase Servers and other High Availability (HA) features. The new HA support provides a component approach that replaces the monolithic HA support of previous releases.

As of release 5.1.5, the documentation has caught up with the newly added features. Start with the overview page, Clustering and High Availability, which provides an index to the documentation of the various StreamBase features that support HA and clustering.

In previous 5.1.x releases, the HA sample included Linux scripts to run the sample. The sample now includes Windows scripts as well. Instructions for the sample are in High Availability Sample.

StreamBase releases through 5.0 included support for an earlier version of StreamBase Server clustering that used the sbclusterd and sbnoded servers. In release 5.1.5, all documentation of these features is removed. The sbclusterd and sbnoded commands themselves are still shipped for compatibility with previous releases. However, these commands are deprecated and will be removed from a future StreamBase release. Sites using these features should begin to migrate to the new clustering and high availability features discussed above.

Fields in Studio views that accept input of integers were changed in this release. Previous releases used spinner controls for integer input fields, like the following example from StreamBase 5.0:

Integer input fields in Studio are now simple text boxes, like the corresponding example from StreamBase 5.1.5:

This change allows you to enter negative integers where appropriate, and to enter parameters in the form ${param}.

The expression language was extended with the getLeadershipStatus() function. See StreamBase Expression Language and Functions for details.

The Reuters OMM Subscribing Input Adapter was added with an optional installation kit. This adapter allows StreamBase applications to receive OMM data from Reuters Data Feed Direct and RMDS 6 Point-to-Point servers. This adapter complements the existing Reuters Subscribing Input Adapter.

The Reuters Subscribing Input Adapter now automatically adds metadata fields to the market data output schema, if you do not provide them when using the Edit Schema tab.

To improve disk Query Table write performance, log flushes and checkpoints can be separated. The current disk-querytable-flush-interval-ms configuration parameter remains, but the default has been reduced to ten seconds. A new configuration parameter, disk-querytable-checkpoint-count, specifies how many flush intervals to wait before doing a checkpoint. The default is 3, so by default, a checkpoint is still done every 30 seconds. These new defaults are not expected to change the average write rate but should reduce the variance in write rate over time, and have the additional benefit of reducing the default loss window to ten seconds. Some applications and configurations might see measurable improvements.

In both HTML and Eclipse Help documentation, the pages of the Tuning Guide were moved to the Authoring Guide to improve their searchability in Eclipse Help. See Tuning Applications.

The StreamBase Reuters Subscribing Input Adapter was enhanced with the following features:

This release introduces a new embedded input adapter for data feeds from the Comstock service. This adapter is not part of the base StreamBase kit, and installs separately. Contact your StreamBase Systems representative to obtain the StreamBase Comstock Input Adapter installation kit.

In the StreamBase client API:

The StreamBase expression language was enhanced to include the to_milliseconds() function.

You can implement more robust authentication for client applications using new StreamBase support for X.509 certificates, SSL, and LDAP servers. A new server, sbproxy, was added to support these features. For more information, see Security Options.

The new proxy server, in addition to supporting the new security features, also supports backward compatibility for StreamBase clients. The proxy server provides a way to run existing StreamBase 3.x clients with a 5.x StreamBase Server. For more information, see Using the Proxy Server to Control Client Access and sbproxy.

Query tables can now be accessed across application modules, using the mechanism of module references. See Sharing Query Tables for more information.

You can now detect events across one or more input streams by using StreamBase pattern-matching queries. A new Pattern operator is available for EventFlows. For StreamSQL applications, the FROM PATTERN clause has been added in the SELECT statement. For more information, see Detecting Patterns in StreamBase Applications.

Output streams now accept a predicate that can restrict output tuples in two ways:

For more information, see Defining Output Streams.

Data types are documented in StreamBase Data Types.

The following functions were added to the list of built-in functions you can use in building expressions for StreamBase operators: blob(), ceil(), cosh(), expm1(), floor(), ln(), log1p(), long(), random(), regexmatch(), round(), securerandom(), sign(), sinh(), systemproperty(), tanh(), to_degrees(), and to_radians().

Functions are documented in StreamBase Expression Language and Functions.

You can now use the Query operator to write to a JDBC data source in batch mode. For example, you might might want to commit batches of insert or update statements to the database, instead of every time a tuple arrives at the Query operator. To use this option, edit your sbd.sbconf file, specifying the number of statements using the jdbc-batch-size parameter in the data-source section. For more information, see the StreamBase Server Configuration XML reference.

An optional, high-capacity event-store, StreamBase Chronicle, can be used to provide persistent data storage in association with streaming applications. Supported data stores in this release are DB2, Sybase, and Vertica databases. For EventFlow applications, a new data construct, the Chronicle Connection, is provided.For StreamSQL applications, the APPLY CHRONICLE statement has been added. For more information, see Using Chronicle Data Sources.

The StreamBase .NET API was enhanced to no longer require a Java runtime as an intermediary. StreamBase .NET clients now require the .NET Framework 2.0 or later, and must be developed with Microsoft Visual Studio 2005 or later. See Creating .NET Clients.

The jsbmonitor command was replaced with a preview relea

se of the sbmanager command. See Using StreamBase Manager.

In previous releases, the HTML version of the StreamBase documentation required JavaScript to be enabled in your browser to support its page to page navigation. The 5.0 version of the documentation lifts this restriction. The 5.0 documentation was reorganized and given a simpler, more elegant page design that does not force a standard page width.

In previous releases, StreamBase Studio was implemented as a custom application built on the foundation of the Eclipse Rich Client Platform. Starting with release 5.0, StreamBase Studio is implemented as a set of standard Eclipse plug-ins installed into a base of Eclipse 3.2.2. The plug-in architecture means StreamBase Studio is now more tightly integrated with Eclipse, and takes advantage of Eclipse's built-in Java development tools and open architecture.

Studio in 5.0 uses a new format for its Eclipse workspace that is incompatible with the workspace from previous releases. When Studio 5.0 first starts and prompts for the location of its workspace, use the provided default location, or specify a new, empty directory. Do NOT specify the location of any previous StreamBase Studio workspace. Studio 5.0 issues a warning dialog if it detects an older format workspace.

Despite the new format workspace, your work in previous releases of StreamBase Studio is not lost. It remains in the workspace directory for the previous release, and Studio 5.0 provides a migration wizard to copy your projects from a StreamBase 3.x workspace to the new, empty workspace. See Migrating StreamBase Studio 3.x Projects for details.i

Studio project directories in previous releases required a strict directory structure, with certain file types required to be in certain directories. In Studio 5.0, project directories are standard Eclipse projects, and are free-form. You can organize your project directories as you prefer.

In StreamBase 3.x, you had to download and install StreamBase's Java Toolkit for Eclipse into an independent Eclipse installation in order to use the toolkit's wizard to generate Java stub code for custom extensions to StreamBase. In Studio 5.0, Java support is fully integrated with the underlying Eclipse release. You can create, edit, compile, and run Java files right in your StreamBase project directory. The StreamBase Java code generation wizard is integrated into Studio 5.0, and is no longer a separate download kit.

Now that StreamBase Studio is based on Eclipse 3.2.2, you can use the Eclipse update mechanism to install Eclipse plug-ins from other vendors into StreamBase Studio. This allows you to install the plug-in for a version control system, which would effectively add version control support to StreamBase Studio. For example, you could install the Subclipse plug-in to add Subversion version control to your StreamBase projects.

Certain views in the Test/Debug perspective were removed or replaced with more complete functionality offered by the Eclipse environment or StreamBase tools.

For existing customers and qualified evaluators, StreamBase Systems provides access to a private URL that you can use to install the StreamBase Studio plug-ins into an existing installation of Eclipse. Contact StreamBase Systems Customer Support for further information. Use the instructions to take advantage of this resource.

In previous releases, on initial startup, StreamBase Studio prompted you to choose a local or remote connection to an instance of the StreamBase Server. In Studio 5.0, this choice is not forced at startup. When you start an application to run or debug it, at that time you choose to run against a local or remote StreamBase Server. You can alternate between local and remote servers at will.

StreamBase Studio now provides keyboard equivalents for most mouse actions. For example, with an EventFlow Editor open and selected, you can now add a new Filter operator to the canvas by pressing the O key (letter Oh, not zero), followed by the F key. To see the complete list of EventFlow keyboard shortcuts, start with an EventFlow Editor open and its tab selected, then press Ctrl-Shift-L. Press Esc to close the shortcut list. See Keyboard Shortcuts for further information.

The palette was reorganized for Studio 5.0:

You can now group collections of components on your EventFlow canvas into a visually defined group. Groups are a visual tool that can help you edit more efficiently, but they do not affect the functionality of your application. For more information, see Groups.

You can now add Notes to your EventFlow canvas to add documentation for each component. Notes can attach to one component, to several selected components, to a group, or to the canvas itself. For more information, see Notes.

In the Properties View for any operator, there is a Help subwindow on the right side. In the Chooser sub-tab of the Functions tab, the search function has been enhanced. In previous releases, you entered a string to match the beginning of a function name. Now you can use a regular expression to better focus your search for a function. For example, the search string *int* now matches five function names: intercept(), compound_interest(), interval(), stripinterval(), and int(). The results field now shows only trees with matching functions. Use the X button on the right of the search field to clear the search field.

StreamBase Studio extends the Eclipse Compare function, providing support for comparing the XML structures of sbapp EventFlow application files. See StreamBase Compare Editor for details.

Application icons for ssql and sbapp files displayed in StreamBase Studio now include problem markers if typecheck errors are found in the files.

StreamBase 5.0 supports the installation of one StreamBase 3.x installation plus any number of StreamBase 5.x installations on the same Windows machine. See Coexistence of Multiple StreamBase Installations for important details and limitations.

As part of the support for multiple StreamBase installations, the StreamBase installer no longer automatically sets the STREAMBASE_HOME and PATH environment variables. Instead, use Start → (All) Programs → StreamBase n.m → StreamBase Command Prompt to open a Windows command prompt with pre-set environment. You can also use the new --setenv or --setsysenv options of the sb-config command to set the environment permanently in the registry. Use caution when using these options if you have multiple StreamBase installations. See StreamBase Command Prompt for important details.

As part of the support for multiple StreamBase installations, the default locations for all StreamBase-related directories and registry keys are now decorated with the current version number, to avoid collision with previous releases. Directories affected include:

The RSS Reader Input Adapter is an embedded adapter that reads RSS data from any number of feeds off a socket. Using the schema of RSS, the adapter interprets this data and displays the relevant information from the specified feeds.

New adapters added to recent releases of StreamBase 3.7 are available in StreamBase 5.0. This includes the Opentick, Adobe Flex, and Wombat MAMA adapters.

The default settings in sbd.sbconf for the jvm-args parameter (used by sbd to set maximum and minimum memory in the JVM) have been increased from value="-Xms64m -Xmx256m" to value="-Xms128M, -Xmx512M".

The -[XX]MaxPermGenSize Sun JVM parameter is now used by the sbd server.

Support was added in the JDBC operator to control behavior when a JDBC connection between the StreamBase Server and the database server is lost. Three new data-source configuration parameters implement this control; in their absence, the default behavior for a connection loss is for the StreamBase Server to display an error message and exit with error status. The new jdbc-reconnect-* parameters are documented in the StreamBase Server Configuration XML reference.

You can now specify short names for custom Java functions or public static functions, so that you can call them like any other function instead of using calljava(). See the custom-functions section of the sbd configuration file documentation, StreamBase Server Configuration XML. Also, the StreamBase Custom Java Functions wizard has an option to create aliases for your new functions.

StreamBase 5.0 includes a number of changes to the Java API for building extensions to StreamBase. See the migration guide at API Changes in StreamBase 5.0.

For StreamBase 5.0, the C++ API underwent an overhaul to make it easier to use and to ensure safety while doing so. The changes are described in the migration guide at C++ API Changes in StreamBase 5.0.

The StreamBase .NET API was significantly upgraded, possibly requiring changes to your existing code for .NET client applications. See Migrating .NET Clients for instructions.