StreamBase Release Notes

This release adds an additional way to control the size of the internal queue used between a Java StreamBaseClient's main thread and the dequeuing thread. Previously, you could control the number of batches of tuples in the queue using the Java property streambase.max-queue-size, which has a default size of 500 batches. However, if a client gets behind in dequeuing, the server starts sending larger batches, which requires more memory usage by the client. To remedy this situation, there is a new Java property in the StreamBaseClient API: streambase.client.max-tuples-in-queue. The default value is 10000 tuples. This property limits the absolute number of tuples in the queue, which promotes better client behavior. This same Java property also affects the StreamBase-to-StreamBase input adapter.

There is also a new Java property for the server, streambase,sbd.max-tuples-in-dequeue-packet. This property limits the number of packets the server includes in any one dequeue packet. The default value is 10000, which matches the default of the new client property described above. Administrators can set both server and client with matching values to make sure client dequeues are not overwhelmed.

The rare circumstances that triggered the condition were: an update operator was modifying rows in a query table with a primary key of type string, which was downstream from, but in the same parallel region as, a read operator connected to the same table, which returned three or more rows.

<param name="jvm-args" value="-Xms64m -Xmx256m -Dstreambase.codegen.intermediate-stream-dequeue = true"/>

In the previous release, setting the switch shown did not work properly, and you could only see intermediate streams in debug mode. The problem was fixed.

=RTD("StreamBase.RTD", , "STREAM:myContainer.All_Transactions", ...)

Note that is usually better *NOT* to specify container names in Excel cells. That is,

=RTD("StreamBase.RTD", , "STREAM:All_Transactions" , ...)

is generally more maintainable than:

=RTD("StreamBase.RTD",,"STREAM:myContainer.All_Transactions", ...)

In previous releases, copying the operators in the loop would cause the schema of the looping stream to be shared by the operators. As a result, a change to either looping stream would take effect in both. In this release, such a copy now creates a separate schema for the looping stream. The schemas are initially identical, but you can edit each independently of the other.

Instead, the adapter monitors the failed StreamBase server, and when it becomes available, reconnects in the background. The current state of each StreamBase server is available through the new META:SERVER_STATUS tag. The new meta.xls sample spreadsheet shows how this works.

<param name="uri" value="sb://foo:10000"/>

and a spreadsheet cell contains:

=RTD("StreamBase.RTD",,"URIREF:", ...)

then this cell will reference the StreamBase server running at URI sb://foo:10000.

Note that if you want to use an expression that intentionally sets the result to null, you must use one of the data-type-specific null literals, as shown in the following list:

For more information, see Using Nulls in StreamBase Applications, a topic in the Authoring Guide.

streambase.codegen.javac-Xmx

Use this flag the same way as the java -Xmx command-line option:

Specify the maximum size, in bytes, of the memory allocation pool. This value must a multiple of 1024. Append the letter k or K to indicate kilobytes, or m or M to indicate megabytes. Examples:

-Xmx83886080
-Xmx81920k
-Xmx80m

If the input field in question were of string type, the StreamBase server could experience a null pointer exception. If the input field were another data type, then the output value could be 0 instead of the expected null data type.

package com.streambase.sb.expr.test; 
import com.streambase.sb.operator.AggregateWindow;
/** 
  * Class that implements a custom aggregate function that 
    concatenates strings together. 
*/ 
public class StringConcatAggregate extends AggregateWindow {
    StringBuffer buf = new StringBuffer();
    public static int getResultLength() {
        return 100;
    } 
    public void init() {
        buf.setLength(0); 
    } 
    public void accumulate(String s) {
        buf.append(s);
    } 
    public String calculate() {
        return buf.toString(); 
   }
} 

{"strptime('2006-Feb-10 11:01:03', '%Y-%b-%d %H:%M:%S')", SOME_TIMESTAMP}
{"strptime('2006-fEB-10 11:01:03', '%Y-%B-%d %H:%M:%S')", SOME_TIMESTAMP}

Note that month letter codes are limited to three characters: long month names like February are not supported.

bin/sbd: error while loading shared libraries: libsbclient.so.4:
       cannot open shared object file: No such file or directory

If that socket is already being used by some other program on the same machine, the failover will succeed, but the HA functionality will be disabled. Consequently if the former primary server restarts, it cannot function as the new secondary server. This means that subsequent failures will not be able to failover.

On the new primary server, you will see a message similar to the following:

error Can't start primary server on port XXXX: <reason> Secondary taking over but HA disabled

For example, if the fields available in the Output Settings tab have changed on either the input stream or at the associated Data Construct, the Output Settings tab may (in explicit output field mode) auto-select new output fields.

MOZILLA_FIVE_HOME=/usr/lib/mozilla-1.7.3
export MOZILLA_FIVE_HOME 

This applies both to using fields from other schemas, or to using the Field objects that were used to create the Schema. Fields that are used to create a schema are not fully initialized, and so you must get the actual Field from the new Schema object in order to access the fields of a Tuple.

Note, however, that if you defined regions in pre-3.0 releases of StreamBase, you should examine each component in your application after opening it in StreamBase Studio. In the Properties view for each component, make sure that the concurrency option ("Run this component in a separate thread") is appropriately set - either checked or unchecked. You should only check the concurrency option for a module reference or an individual operator if you know that it can run without dependencies on other components in the application.

If the existing table contains critical data, you can save the data before upgrading StreamBase. For example, create a simple application in StreamBase 2.5 that writes the entire contents of the table to an output stream. Then, use sbc to run the application and write the output to a file. After the upgrade, create a simple application in StreamBase 3.x that reads the data into a persistent Query Table.

For example, the following statement passes typechecking even if the named table or fields do not exist in the JDBC datasource:

APPLY JDBC DB2 "INSERT INTO mytable (id, name) VALUES({id},
{name})" FROM InStream INTO OutStream;

If you experience the problem described, consider limiting the number of rows that MySQL can read at one time. You can do this by enabling the new jdbc-fetch-size parameter in the data-sources section of the StreamBase server configuration file, and setting it to the Java Integer.MIN_VALUE literal, -2147483648. Note that, although other JDBC drivers interpret a positive jdbc-fetch-size value as a hint, MySQL drivers ignore all values except this magic value. See Using JDBC Data Sources for details.

For a full discussion and options, see the section titled, Avoiding Deadlocks With JDBC Queries, in the

For example, one workaround is to mark one or more of the Query operators as parallel to break the deadlock: in an EventFlow application set the concurrency option, as described in Using the Query Operator to Use or Modify Shared Data; in a StreamSQL application add the PARALLEL keyword to the APPLY JDBC statement, as described in APPLY Statement.

One incorrect side effect is that if one predicate in filtered subscribe fails, then predicates that come after it in the list are not executed and tuples are not delivered. Similarly, if other operators are downstream and only one is set for data parallelism, the other operators may not see tuples. Which operators see tuples is effectively non-deterministic.