1. Denny Hatzenbihler InfoSphere Streams - Runtime
Stream connection health, blockage and congestion InfoSphere Streams Version 3.0
Denny Hatzenbihler
InfoSphere Streams - Runtime

2. Important Disclaimer
THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONAL PURPOSES ONLY.
WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THE INFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED.
IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.
NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:
CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS OR THEIR SUPPLIERS AND/OR LICENSORS); OR
ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENT GOVERNING THE USE OF IBM SOFTWARE.
The information on the new product is intended to outline our general product direction and it should not be relied on in making a purchasing decision. The information on the new product is for informational purposes only and may not be incorporated into any contract. The information on the new product is not a commitment, promise, or legal obligation to deliver any material, code or functionality. The development, release, and timing of any features or functionality described for our products remains at our sole discretion.
THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.
IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

3. Agenda Stream connection health - additional capabilities
Ability to detect Stream blockage and measure congestion

4. Stream Connection Health
Streams Version 2.0
User can determine the number of broken connections
Streams Version 3.0
User can identify individual broken connections
Has a particular connection ever been connected?
Timely information about connection state
User can identify connections that are broken because their destination PE has died
PE 2
This feature enables the user to identify which connection is broken. The list of functionality updates consists of:
Adding ability to report the health state of individual connections. Streams v2.0 only provides an aggregate summary of connection health inferred from the connection metrics. This feature will provide state information of individual PE-to-PE data connections.
Adding ability to determine connection state changes in relation with the health of the connected PEs. Streams v2.0 lacks functionality to determine the health of connections under certain conditions: for example, when a PE crashes, the system continues to report all its inbound connections as being in a healthy state, although there is no longer any process consuming tuples.
In regards to a Streams instance’s job workload, version 2.0 of Streams Console can display Jobs, PEs, Operators, and Application Streams (Imports/Exports). For application health, Streams Console v2.0 shows the health of Jobs and PEs.
Streams Console v3.0 can display a list of current connections. The connection list supports filtering by connection state, as well as the “normal” list of console filters such as: Job ID, Job name, PE ID, and user. The console would enable the user to drill down from the PE to see connections for a PE which may be the cause of a PE being unhealthy.
Note: this feature does not address any direct Operator to Operator connection conditions.
PE 1
PE 3

5. Stream Connection Health – Streams Console

6. Stream Connection Health – streamtool capturestate
streamtool capturestate –i <instance> --select jobs
<job id="1" name="application::PersonDataApp" submitTime=" " user="warrena" state="running" healthSummary="partiallyUnhealthy">
<pe id="10" host="d0701b01.pok.hpc-ng.ibm.com" processId="0" state="stopped" reasonCode="failure" requiredConnections="disconnected" optionalConnections="disconnected" healthSummary="unhealthy">
<operator name="File">
<outputPort index="0" name="File" streamName="File"/>
</operator>
<outputPort index="0">
<connection inputPeId="11" inputPortIndex="0" required="true" state="closed"/>
</outputPort>
</pe>

7. Stream Blockage and Congestion
When a PE is slow in processing
The sender PE blocks on transmit
This can propagate up to the sources (backpressure)
Blockage: Inability to make forward progress for a longer duration
Caused e.g. by an operator stuck in infinite loop or doing long term I/O
Congestion: Short term blockages
Tuples are being sent faster than the destination operator can process them
Congestion Metric
Per PE output port congestion
Range from 0 (no congestion) to 100 (blockage)
This functionality enables the user to detect connection congestion. 
Because the definition of congestion is subjective, Streams v3.0 provides a metric value that indicates the level of congestion for each PE output port connection.  This metric provides a value from zero to one hundred which represents the percentage ratio of time spent waiting to send data divided by total time to send the data. This allows users to determine if a flow is congested or not. The port congestion metrics will be reported to the customer interfaces (Console, streamtool, IDE) along with other PE metrics to allow appropriate display of PE connections that may be, or are about to be, congested.

8. Streams Congestion – Streams Console

9. Stream Congestion – streamtool capturestate
streamtool capturestate –i <instance> --select jobs=all
The following <connection> element is under <job>, <pe>, <outputport>
<connection inputPeId="1" inputPortIndex="0" required="true" state="closed">
<metric name="congestionFactor" lastChangeObserved=" " userDefined="false">
<metricValue xsi:type="streams:longType" value="0"/>
</metric>
<metric name="nTuplesFilteredOut" lastChangeObserved=" " userDefined="false">
</connection>

10. Demo

11. Questions ?