1. A Study Case: JPEG2000 Compressed Images over a Link 16 Network
IST-083 Symposium Progressive Still Image Transmission over a TDL Network
A Study Case: JPEG2000 Compressed Images over a Link 16 Network
Author: Cdr, Manuel Martinez, PHD (SPN)
Co author: Prof, Antonio Artes, PHD (UC3M)
Co author: Maj, Roberto Sabatini, PHD (IT AF)

2. Scope Operational Requirements Image Technical Requirements
JPEG2000 Standard
Link 16 Case Study
Some Examples

3. Imagery Operational Requirements
Target images required before authorizing weapon release.
Strike Aircraft aircrew using an image depicting a time critical target (TCT) for identification.
Damage Assessment.
UAV imagery supporting urban/roads battlefield.

4. Why in TDLs?
To develop a “composite tactical picture” available to all Joint Units by using “multimedia” information content provided by TDLs in support of network centric warfare.

5. Why in TDLs?
Extend TDLs concept from “textual data” to “multimedia elements” including still image & stream-video for a variety of image types
Visual
IR/FLIR
SAR
Each image shows different characteristics based on the sensor performance and image type.
Don’t ask for new “spectrum allocation” in dense urban areas
Use the current TDL’s (Link 16) allocated spectrum.
Use current AJP capabilities provided by TDLs (Link 16) to avoid jammers /civilian interferences.

6. “Still Image” Compression Requirements
Compression Efficiency.
Excellent performance at low or very low data rate (disadvantage links).
Random access to the “code-stream”.
Embedded code stream that allows to extract the image in different ways.
This shall allow to reconstruct an image based on a progressive scheme.
Scalable.
Resolution.
Quality: definition of QoS and quality layers.
Components: color, B&W,…
Position.
Lossless and Lossy Compression.
Open architecture and standardized.
Definition of Region of Interest (ROI), Zoom, Tiling,..
Robustness in channel error conditions.

7. Current Efforts. DLWG DLCP NUMBER: ML277-054-P33-US-M32-R6
J16.0 Image Transfer Message definition.
NPG 11 dedicated for Imagery
Minimum 32 Time Slots required per frame.
Recommended P4 Packing Limit (Link 16).
Supplement messages from surveillance and control.
Secondary role.
Imagery and image amplification information defined in the standard.
Voice coordination required.

8. Current Efforts. DLWG (2)
Fulfill NATO Secondary Image Format (NSIF).
STANAG 4545, Edition 1 Amendment 1 dated 14 April 2002 – NATO Secondary Imagery Format (NSIF) Version 1.0.
Compression Algorithms: JPEG & EagleEye.
Image packet data of 45 bytes per J 16.0 message.
Geo-position Information is included as a non displayable image segment.
File Transfer protocols: one way & two ways.

9. Proposed Still Image Compression Standard:JPEG2000 (1).
Defines a “canvas coordinate system” based on tiles
Since different components can have different sizes.
Each tile is DWTd Provides “spatial” access to the image (can be operationally required).
Discrete Wavelet Transform (DWT).
Wavelets provides a robust tool for signal and image processing.
More accurate time-frequency representation Can handle signals with discontinuities.
Transform Domain: each sub-band of every resolution level is partitioned into blocks/precints/packets.

10. Proposed Still Image Compression Standard:JPEG2000 (2)
Embedded block coding algorithm (EBCOT): one code-stream can be decompressed in many ways.
Each sub-band image divided into blocks that are coded independently.
Introduces quality for “layers” concept.
Progressive compression.
By quality / resolution / components and position
State of the art compression efficiency.

11. JPEG2000 Canvas Coordinated System

12. JPEG2000 Block Diagram ICT: Inter-component Transformation
DWT: Discrete Wavelet Transform: Daubechies 9/7 (NR) & Daubechies 5/7 (Reversible)
DZQ: Dead Zone Quantification

13. JPEG2000: Wavelet Transform
Sub-band Images from low & high pass filtering and sub-sampling
Same samples as in original image
Transform is reversible
Sub-bands are quantized
(Image from David Taubman)

14. JPEG2000: Wavelet Transform (2)
Multi-resolution representation of the image
Image is LL/HL/LH/HH filtered in various levels
LL: Approximation of image (low frequency)
LH: Horizontal details.
HL: Vertical details.
HH: Diagonal Details (high frequency)
Note: Image by Konstantinos Kamaras Thesis, March 2002.

15. JPEG2000: EBCOT (Embedded Block Coded with Optimal Truncation)
EBCOT defines quality layers: each block stream is optimally truncated.
From low quality imagery (coarse quantization) to the best image (finest quantization)

16. JPEG 2000 Standard. Different Parts
Part #1: Core coding & basic technology (JP2)
Part #2: Extensions (JPX)
Part#3: Motion JPEG2000 (MJ2)
Part #5: Reference SW (Java & C)
Part#8: Encryption and Watermarking (JPSEC)
Part #9: Interactive Protocols and API (JPIP)
Part#11: robust communication for wireless (JPWL)
Part #12: ISO Base Media File Format (Common with MPEG-4)

17. JPEG2000 Code stream (JP2)
JPEG2000 coder produces an embedded code-stream that can be decoded at any length to produce an image.
From 0 bit rate to the maximum (lossless).
JPEG produces an independent code-stream per “target bit-rate”.
Implies multiple coding passes  less efficient.
Embedded code stream support random access to image properties:
Region of Interest (ROI).
Zoom-in / out.

18. JPEG2000 Code stream (JP2)
Tile
Stream

19. Geo Referenced Images (Geography Markup Language)
JPEG2000 doesn’t specify mechanism for geo-referencing the image, describing the sensor characteristics,…
JPEG2000 provides room for “boxes” containing arbitrary XML data.
GML (by Open Geo-Spatial Consortium) provides geo referencing information as XML encoded metadata.
GML & JPEG2000 are interoperable and compatible.

20. JPEG2000/JPEG Comparative Analysis
Progressive lossy to lossless compression
JPEG: not integrated capability.
Random Access to the code-stream:
Implementation of Region of Interest capability.
Better data rate control.
JPEG2000: better compression efficiency.
(Image1 & 2 are referenced in paper)
Bpp
0.125
0.5
2.00
Img1 JPEG
24.42
31.17
35.15
Img1 JPEG2000
28.12
32.95
37.35
Img2 JPEG
22.60
28.92
35.99
Img2JPEG2000
24.85
31.13
38.80

21. JPEG2000/JPEG Comparative Analysis in Noisy Channels
Better performance in noisy channels. NOT EDC Technique Implemented WORS CASE SCENARIO  better if R-S is modeled.
See example: BER: 10^ (-4).
Error Symbols in the code-stream:
JPEG2000: 28
JPEG: 18
Method:
JPEG2000 code stream is built (*.J2C)
A Binary vector representation of the code stream is developed.
A binary vector with the same BER channel is generated: x=(rand(tamano,1)<=ber)
An exclusive OR is made between both vectors for generating a LINK-16 channel corrupted JPEG2000 code stream.
Image reconstruction with “Kakadu”
JPEG
JPEG2000

22. JPEG2000/JPEG Comparative Analysis (2)

23. Requirements Flow Down: From Design to Operation
Note: Link 16 Service channel: 8 TS assigned if required (Free Text)

24. JPEG2000 Compressed Image Over Link 16
Method proposed
Transmit the most sensitive part of JPEG2000 code-stream with maximum Link 16 Anti-Jamming Protection: STD Packing Limit.
Headers/markers/first layers/tiles/
Rest of the image with a less robust Packing Limit. (less AJP)
JPEG2000 Coding option: progressive by quality.
Quality Layers.
Link 16.
“free text” unformatted messages/ JXX.YY standard format message.
NPGs: from (NPG 11 –assigned- & spare NPGs: 15 to 18)
Max TSBs: 64 per Link 16 Terminal (48 in TSR)
Time slots available for imagery
Two study cases: NMT 10% and NMT 5%.

25. JPEG2000 Quality Layer to Time Slot Allocation Algorithm
Match Link 16 available resources with JPEG2000 LRCP “quality layers”

26. Matching Numbers: An Example
Step 1 &2. Define the bit rate iaw network design: 32+x+y+z (time slots available for imagery)
Step 3. Bit rate layer#1: 32 TS *XXX bits/TS_PL=YYY bits.
Step 4. Size of the image=640 x 437 pixel.
Step 5. Bit rate layer #1: r_1=YYY/640*437= bpp (P2DP).
Step 6. Layer #n: r (n)= r(1) +  (r), with (r ) iaw time slots allocation by network design
Step 7. Image JPEG2000 coded LRCP iaw layer(n) bpp.
Nota: Each additional quality layer can be allocated to a different NPG or TSAB of the corresponding NPG.

27. Communication Architecture & Protocol
Sensor Node (SN)
Captures the image.
Image Preprocessing & Compression: filtering/PSNR/Quality Criteria.
Build Geo referencing code stream (If required).
Imagery Management Node (IMN)
Receive the image and request retransmission or changes in quality based on R-S Symbol threshold. (Errors in channel)
Request for ROI.
Request for more time-slots if in TSR.
Reconstruct the original image based on code stream received and operational/tactical requirements.

28. Protocol Note: All joint units receive the same code-stream but only
IMN can interact with Sensor Node unless otherwise specified.

29. Quality Management (QM). (In terms of PSNR)

30. Concept. Example 1 Picture JPEG2000 coded
Progressive LRCP Mode: 4 layers
NGP allocation (if available): Packing Limit and Access Mode
Layers 1 & 2: Dedicated Algorithm #1
Layers 3 & 4: TSR Algorithm #2
Link 16 Transmission

31. Example 2: P/L increased by layer
TS available: 128
Img#1: 3 layers (96 TS/27.26 dB)
Img#2: 1 layer (32 TS/23.21 dB)

32. JPEG2000 & Link 16 Combined Error Detection and Correction Tecniques

33. Error Resilient Mode (Symplified).
JPEG2000 Coded Image: Always include SOP-EPH markers in the code-stream.
Much better syncronization mechanism.
Error Management Protocol Implemented.
Monitor Link 16 Parameter MER (Message Error Rate)
Provides Reed-Solomon Coder Errors and Erasures.
“If MER >Threshold (iaw AJ enviroment):
 Retransmit same packet with a more robust P/L”.
Increase AJP.
Decrease BPP.
Commitment: Binary Rate AJ Protection
Imagen codificada siempre con SOP-EPH.
Implementar Protocolo de gestión de errores Ver siguiente transparencia.
Utrilizar Parámetros MER (Message Error Rate)
Proporciona medida de errores y erasures del codificador R-S.
Caso fallo Retransmitir en P/L de protección +.
Válido para cualquier requisito de distancia.
See Ref: JPEG2000 Image compression and error resilience for transmission over wireless channels (Konstantionos Kamaras, Mach NPS, Monterrey, ca)

34. Error Recovery. An Example (1)
1 packet header corrupted +1 TIME SLOT
JPEG200
Code-stream
#Bytes
P/LResilient Mode
# Time Slots
P/L No resilient Mode
#Time Slots
MAIN HEADER 96
STD 4
TILEPART #1 12 1
PACKET #1 HEADER 11 P2
PACKET#1 13
……………….. ….
……………
………….
……………… …
TOTAL Time Slots 25 24

35. Error Recovery. An Example (2) True JPEG2000 Coded Image
ALL packet corrupted +12 TIME SLOTS
Trama Codificada JPEG2000
#Bytes
Packing Limit Resilient
# Time Slots
Re-TX
Delta #Time Slots
MAIN HEADER 96
STD 4 =
TILEPART #1 12 1
PACKET #1 HEADER 11
PACKET#1 13 P2
(0.462)= 1 (+0)
PACKET #2 HEADER 17
PACKET#2 38
(1.3511)= 2 (+1)
PACKET #3 HEADER
PACKET#3 95 2
(3.3777)= 4 (+2)
PACKET #4 HEADER
PACKET#4
207 P4
P2DP
(3.68)= 4 (+2)
PACKET #5 HEADER
PACKET#5
392
(6.96)=7 (+3)
PACKET #6 HEADER
PACKET#6
440
(7.82)=8 (+4)
TOTAL Time Slots 25
37 (+12 time slots)

36. Some Simulations and Results
Great amount of simulations has been conducted:
One NPG  STD P/L (by default)
Two NPGs  STD/P2
Three NPGs  STD/P2/P4
Four NPGs STD/P2/P4/P4NEDC
DLWG: Only one NPG allocated for imagery (NPG11)
But several NPGs available for future use.
Packing limit can also be changed before each Link 16 transaction via AP: Only one NPG will be required  More “host overhead”.

37. 119 TS/3 Layers/ dB.

38. 140 TS/ 9 Layers/ dBm

39. FLIR Original Image

40. 32 TS (STD) / 1 Layer/ Cr=706.71

41. 128 TS (P4) / 3 Layer/ Cr=78.5

42. Change in Resolution Original Resolution 1 Resolution 2 Resolution 3
Algorithm step
Pixels
bytes
BPP
Time Slots.
NPGs/PL
Original Image
640x480
921600 1 #1
Resolution: 4
40x30
3600
0.7138 4
NPG V1 (STD-TSR) #2
Resolution 3
80x60
14400
0.7277
+16 =20 #3
Resolution 2
160x120
57600
0.7851
+34 (TSR) =54
NPG V2 (P2DP-TSR) #4
Resolution 1
320x240
230400
0.8830
+37 (TSR) =91
NPG V3 (P4-TSR)
Original
Resolution 1
Resolution 2
Resolution 3

43. JPEG In Noisy Channels BER: 10^-4
Original
JPEG

44. JPEG 2000 In Noisy Ch. BER: 10^-4
ORIGINAL
JPEG
JPEG2000

45. Original SAR Image: 8 bpp

46. JPEG2000 SAR Images 1 Layer / 30 TS (P4)/0.05088 bpp/18.84 dB
5 Layers/ 80 TS (P4)/ bpp/ 19.84

47. JPEG2000 Filtered SAR Images
BPP
PSNR
(Filt)
(Orig)
T/S P4
(Filtd)
Original
8 bpp -
Comp
Lay #1
0.0505
22.97
18.84 30
Lay#5
0.1395
25.48
19.84 81 80
Lay#10
0.5143 --
22.30
---
SAR Image median filtered. 5 layers
PSNR regarding to filtered image

48. SAR Image. Region Of Interest
LAYERS
BPP
PSNR db
Link-16 Time Slots
Layer 1 64
Layer 2
64+64=128
Layer 3
10.93
128+32=160 > 154
Layer #1
Layer #3

49. SAR Image. Tiling Lay bitrate TS PSNR Tiles 1 0.0139 32 10.39 T0 2
0.0695 96
11.44
T0 a T3 3
0.1252
128
12.60
T0 a T5 4
0.18
144
20.07
T0 a T8
Layer #1 layer #2
Layer #3 layer #4

50. Summary (1)
Recent conflicts have shown that real time imagery availability is one of the most demanding capabilities required by operational/tactical commanders.
Integrating “imagery content” into the “common tactical picture” provides a better target identification, weapon release criteria, damage assessment and surveillance data in support of network centric warfare. Also avoids “request for frequency allocation.”
Current efforts are focused mainly to support JPEG/EagleEye compressed images packed into J16.0 messages and released in the NPG 11.

51. Summary (2)
JPEG2000 is a more robust open source standard that produces a totally embedded code-stream.
Method proposed to send the JPEG2000 image in a antijamming adaptative way
Most important part is more protected.
Rest of the image in a less protected packing limit.
Algorithm Proposed to adapt JPEG2000 quality layers to Link 16 available time slots.
Results shown a very good compression ratio with less than 10% and good performance with 5% of the available Link 16 time slots.
JPEG2000 also offers a great variety of new capabilities well suited for tactical exploitation such as
ROI
Watermarks
Geo-referencing
More robust error and detection schemes.

52. Way Ahead Develop a technological demonstrator Plan on going.
Propose an ad-hoc J message J XX.YY or modify J16.0 specification for including JPEG2000 codification syntax.
Implement a GML compatible geo-referencing capabilities as a part of XML boxes available at JPX file format.
Implement security and encryption capabilities iaw JPSEC file format.
MPEG2000 for stream-video under assessment.
And more…
Some of these new proposals require an “enhance throughput” Link 16 capability.

53. QUESTIONS ?