Container Connections

The following types of connections between containers are available.

StreamBase provides several methods of specifying a connection between containers. Some methods are interactive (for testing and management of containers on running servers), while some methods specify that the connection is made immediately when the enclosing server starts (or when the enclosing container on that server starts). However, all methods are equally effective. Use the method that works best for your application and its stage of development or deployment, but only specify one method per connection for a given run of StreamBase Server. For example, do not specify a connection between containers A and B in both the Run Configuration dialog and in the Advanced tab of a stream.

Each container connection must be specified only once for each connected pair of containers. Do not specify a container connection in the output stream of the sending container and again in the input stream of the receiving container. You can specify the connection in either the sending or receiving container to the same effect. Do not specify the container connection in both places.

The methods of specifying container connections are:

Studio: Connections in Stream Advanced Tabs

In StreamBase Studio, in the Advanced tab of the Properties view for either an input stream or output stream, use the Container connection field to specify a StreamBase expression that resolves to the qualified name of a stream in another container. A qualified stream name follows this syntax:

container-name.stream-name

When using the Advanced tab for an output stream, you are already in the source stream, so you only need to specify the qualified name of the destination stream. Similarly, in the Advanced tab for an input stream, you are in the destination stream, and only need to specify the qualified name of the source stream.

You must express the container-qualified stream name as a StreamBase expression. To express a fixed, known container and stream name, enclose the qualified name in single or double quotes to render the name as an expression. You can also write an expression with variables, field names, and expression language functions that resolves to a qualified container and stream name based on other criteria upstream in your application.

Specify each container connection only once, in either the sending or the receiving stream. Do not specify the container connection in both places.

The following example shows an input stream specifying a connection to the control stream in the system container. This example is taken from the primary HA sample, High Availability Sample.

When you use the Advanced tab to specify a container connection, the connection is stored as part of the EventFlow XML definition of the module, and thus travels with the module. As long as the container names are the same, the same connection will be made whether the module is run in Studio or at the command prompt with the sbd command.

There are restrictions on the order in which you start connected containers, both at the command line and in Studio. See Container Start Order Issues for details.

Studio: Connections in Launch Configuration Dialogs

You can specify both containers and container connections in the Containers tab of Run Configuration dialog (and the Debug and Trace Configuration dialogs).

In the following illustration, running the main application would start two containers: default and sideplay. You must select a line in the top table to add or edit connections for the selected container. The blue highlighting shows that the sideplay container is selected, and it has one connection.

See The Containers Tab for further details on setting up container connections as part of a launch configuration, including information on controlling container start order.

When you specify container connections with the Run Configuration dialog, the connection is stored with the launch configuration for use in Studio only, and not as part of the EventFlow XML. To specify the same containers and connections for running the same application at the command prompt, you must re-specify the container and connections in the server configuration file or with another method.

<runtime>
  <application file="primary.sbapp" container="A" enqueue="ENABLED" dequeue="ENABLED"/>
  <application file="secondary.sbapp" container="B" enqueue="ENABLED" dequeue="DISABLED">
    <container-connection dest="B.InputStream1" source="A.OutputStream1"/>
    <container-connection dest="B.InputStream2" source="A.InputStream2"/>
  </application>
</runtime>

sbadmin addContainer container-name application.[sbapp|ssql|sbar] connection1 connection2 ...

sbadmin addContainer A preprocess.sbapp
sbadmin addContainer B mainprocess.sbapp B.InputMain=A.OutputPre
sbadmin addContainer C postprocess.sbapp C.InputPost=B.OutputMain

dest-container.instream=source-container.outstream

To specify a container connection, you must specify both source and destination stream names, and you must qualify the stream names with their container names. The following syntax is explicitly required when using the sbadmin command, and is expressed in other ways with the other container connection methods:

dest-container.instream=source-container.outstream

If the incoming or outgoing stream is in a module called by a top-level application in a container, then you must specify the module name as well:

dest-container.moduleA.instream=source-container.outstream
dest-container.instream=source-container.moduleB.outstream
dest-container.moduleA.instream=source-container.moduleB.outstream

If the source container is running on a remote StreamBase Server (or if the server is running on a different port on the same host), specify a full StreamBase URI enclosed in parentheses:

dest-container.instream=(sb://remotesbhost:9955/app3.outstream)
dest-container.instream=(sb://localhost:10001/default.outstream6)

You can use a StreamBase URI for the receiving container, as well. For example:

(sb://remotehost:8855/app2.instream)=source-container.outstream
(sb://localhost:10001/default.instream4)=source-containter.outstream

You can use the same syntax for remote hosts in both the dest and source attributes in the server configuration file:

<runtime>
  <application file="primary.sbapp" container="A" />
  <application file="secondary.sbapp" container="B">
    <container-connection dest="B.InputStream1" source="(sb://remotesbhost:9955/app3.outstream)"/>
    <container-connection dest="B.InputStream2" source="A.InputStream2"/>
  </application>
</runtime>

The remote host syntax also works in the Advanced tab of input and output streams, but not when using the Run Configuration dialog in Studio or StreamBase Manager.

You can also specify a JMS server or a CSV file as either source or destination stream. See Connecting Container Streams Through JMS or Connecting Container Streams To a CSV File for the syntax of those connection types.

With a valid container connection specified, when the module containing the connection starts, the hosting server locates the specified container and stream, and makes the connection. As the application runs, tuples exiting one container's output stream are consumed by the specified input stream in the other container.

The schema of both sides of a container connection must match, as described in Schema Matching for Container Connections.

If the hosting server cannot make the container connection for any reason, it stops and writes an error message to the console. The likely reason for a failure to make a container connection is a failure to locate the specified container or the specified input stream in the application running in that container. In this case, check the spelling of your container connection specification.

When a container is removed, the input or output to any dependent container disappears. The dependent container continues to function, but its input or output is removed.

Stream-to-stream container connections are not limited to one container on each side of the connection. That is, you can configure:

When input and output streams are shared, the streams are still owned by the original container. This means that any input or output to the shared stream ultimately needs to go through the original container. When tuples are passed between asynchronous container connections, they are passed using the same queuing mechanism used between parallel modules. This means that the tuple is queued for the other containers. When the tuple is queued, the original container must wait for the tuple to be pushed onto the queue, but will not be blocked by the processing of the tuple (which is done in another thread). Because of the queuing architecture, there is no guarantee of the order of tuples when they are shared between multiple containers.

In the following example, containers A and B share the stream A.InputStream1. When a tuple is enqueued into A.InputStream1 the tuple is first passed to all containers that share that stream (in this case, container B) and then it is processed to completion by itself.

sbadmin addContainer A app.sbapp
sbadmin addContainer B app.sbapp B.InputStream1=A.InputStream1

Nothing prevents you from enqueuing directly to an input stream in container B while that stream also receives input from an upstream connection from container A. Because upstream tuples are queued, any input directly into container B is randomly interleaved with input from container A.

The use of container connections imposes some restrictions on the startup order of containers. In general, the rule is: the container that holds the receiving end of a container connection must start first.

The schemas of the outgoing and incoming streams in a container connection must match. Schema matching is enforced in two different ways for different cases:

Use sbadmin modifyContainer to dynamically add or remove a container connection from an existing container while StreamBase Server is running:

sbadmin modifyContainer container-name [addConnection | removeConnection] 
    connection-expression1 connection-expression2 ...

By default, connections between containers are set up to run asynchronously, with a queue to manage the flow of tuples through the connection.

StreamBase also supports synchronous container connections, which are low latency, direct connections between containers, and do not have queue management. An application might use a synchronous container connection to improve response time of the communication between HA monitoring applications in clustered and high-availability StreamBase design patterns.

Specify a synchronous container connection with the sbadmin command by using := (colon-equals) instead of = (equals) in your container connection assignment statement. For example:

sbadmin addContainer A app1.sbapp
sbadmin addContainer B app2.sbapp
sbadmin modifyContainer addConnection B.incoming:=A.outgoing

You can also specify a synchronous container connection in the server configuration file by adding the synchronicity attribute to the container-connection element that specifies your connection:

...
    <container-connection dest="B.InputStream1" source="A.OutputStream1"
       synchronicity="synchronous" />
...

To determine whether a synchronous container connection will improve your application, you must test your application with and without the synchronous setting, using a test input stream that emulates the actual conditions your application will face in production. Use the Rules of StreamBase Execution Order in the Execution Order and Concurrency page as general guidelines to help decide whether to use synchronous container connections.

StreamBase detects and prevents any attempt to make a synchronous container connection to:

When using a StreamBase URI as part of a stream-to-stream container connection string to specify that one side of the container connection is on a remote StreamBase Server, you can optionally specify URI parameters as part of the remote URI.

The URI parameters you can specify for connections to remote servers depend on which side of the connection is remote. There are two cases:

These parameters have the same meanings for container connections as the similarly named property of the StreamBase to StreamBase adapters described in StreamBase to StreamBase Output Adapter and StreamBase to StreamBase Input Adapter.

Append one or more container connection parameters to a StreamBase URI, each preceded by a semicolon. For example, to specify non-default buffer sizes for a destination connection, use a container connection string like the following:

("sb://remotehost:9900/boxC.instr2;enqueue-buffer-size=300;max-enqueue-buffer-size=2000")=boxD.outstr6

See StreamBase to StreamBase Output Adapter for a discussion of these property settings.