Sample Code and Applications

About the AutoUnion Operator

Did you ever want to just Union two or more streams together indiscriminately without having to worry about picky niceties such as whether the input schemas were exactly the same or have key fields that match? Did you ever want to Union your Metronome tuples into a stream with a 58-field schema without having to explicitly map all the other 57 fields to null? Is all you really want to do is just have anything that comes in go out and not have to think too hard about the schema?

AutoUnion is like Union in that it takes an input tuple from any of its input ports and passes it through to the output port. The input tuple's values also get automatically mapped into the operator's output schema.

AutoUnion's mapping works in two major modes, controlled by the  MergeInputSchemas property. Briefly and informally, by default, all the output tuples have the same schema as input port #1. Or, if you set the  MergeInputSchemas property to true, then the output schema contains all the fields from all the input schemas.

When  MergeInputSchemas is false (the default), the output schema is always exactly the same as the schema of the first input port. (This is great for Doing The Right Thing with Metronome tuples — have your main data flowing in port 1, and the Metronome tuples flowing into port 2). Tuples from input port 1 are literally passed through. Input tuples from other ports have their schemas mapped to conform to the output schema as follows: if a field with the same name exists in both the input and output schemas, the input field value is copied into the corresponding output field with the same name, doing as much automatic type conversion as it can (null is used where a type conversion attempt fails). If an input field doesn't exist in the output schema, that field disappears from the output. If there's no corresponding input field for an output field, the value of the output field is null.

When  MergeInputSchemas is true, then the output schema consists of all the fields in all the input schemas unioned together into one fat schema. For example, if there are three input ports with schemas {a,b}, {b,c}, and {d,e}, the output schema will be {a,b,c,d,e}. The type of each field in the output schema is the type of the field from the lowest numbered input port whose schema contains that field. Using the previous example, if the b field on input port 1 was an int, but the b field from input port 2 was a string(10), then the output port b field would have type int (and b values coming in on input port 2 would be converted from string to int).

There are two other properties:

  UseTypesFromHighestPort: default false
This property is relevant only when MergeInputSchemas is true. If  UseTypesFromHighestPort is true, then the type of the field in the output schema matches that of its type on the highest numbered input port.

  WarnOnTypeConversionFailure: default true
If true, write a warning to the standard output (System.out) when a field value type conversion attempt fails. If false, don't write a warning. In either case, the value of the output field is null.

Comments/Warnings

AutoUnion is designed to cover some commonly occurring cases, but is not designed to be infinitely flexible. It also will happily drop values if the schemas or types don't match. AutoUnion is implemented as a Custom Java Operator. It's not incredibly fast compared to the built-in operators that generate their own schema-specific Java code. Therefore, depending on application requirements, AutoUnion may not be fast enough. One way to use AutoUnion is to rely on it during early development to save development time while schemas are being designed/discovered, but then replace it with Maps and Union once the design of the application stabilizes.

Downloading and Installing the AutoUnion Operator

To use AutoUnion, you must have Release 3.1 or 3.5 of the Developer or Enterprise Edition of StreamBase Studio installed. Download and Install StreamBase Developer Edition .

The AutoUnion download is a 6 KB JAR file that you load into StreamBase Studio, then access from the Palette View.

1. Download the JAR file. Download AutoUnion .
2. Open StreamBase Studio.
3. Choose File > Import > User-Defined Operators (JAR).
4. Select AutoUnion.jar

To create an instance of an AutoUnion operator in a StreamBase application:

1. Drag a Java operator from the Palette View onto the application diagram canvas.
2. Upon dropping, select com.streambase.services.operators.AutoUnion as the Java operator class, then press OK.

« More Sample Code and Applications .