Presentation on theme: "A Study Case: JPEG2000 Compressed Images over a Link 16 Network"— Presentation transcript:
1A Study Case: JPEG2000 Compressed Images over a Link 16 Network IST-083 Symposium Progressive Still Image Transmission over a TDL NetworkA Study Case: JPEG2000 Compressed Images over a Link 16 NetworkAuthor: Cdr, Manuel Martinez, PHD (SPN)Co author: Prof, Antonio Artes, PHD (UC3M)Co author: Maj, Roberto Sabatini, PHD (IT AF)
3Imagery Operational Requirements Target images required before authorizing weapon release.Strike Aircraft aircrew using an image depicting a time critical target (TCT) for identification.Damage Assessment.UAV imagery supporting urban/roads battlefield.
4Why in TDLs?To develop a “composite tactical picture” available to all Joint Units by using “multimedia” information content provided by TDLs in support of network centric warfare.
5Why in TDLs?Extend TDLs concept from “textual data” to “multimedia elements” including still image & stream-video for a variety of image typesVisualIR/FLIRSAREach image shows different characteristics based on the sensor performance and image type.Don’t ask for new “spectrum allocation” in dense urban areasUse the current TDL’s (Link 16) allocated spectrum.Use current AJP capabilities provided by TDLs (Link 16) to avoid jammers /civilian interferences.
6“Still Image” Compression Requirements Compression Efficiency.Excellent performance at low or very low data rate (disadvantage links).Random access to the “code-stream”.Embedded code stream that allows to extract the image in different ways.This shall allow to reconstruct an image based on a progressive scheme.Scalable.Resolution.Quality: definition of QoS and quality layers.Components: color, B&W,…Position.Lossless and Lossy Compression.Open architecture and standardized.Definition of Region of Interest (ROI), Zoom, Tiling,..Robustness in channel error conditions.
7Current Efforts. DLWG DLCP NUMBER: ML277-054-P33-US-M32-R6 J16.0 Image Transfer Message definition.NPG 11 dedicated for ImageryMinimum 32 Time Slots required per frame.Recommended P4 Packing Limit (Link 16).Supplement messages from surveillance and control.Secondary role.Imagery and image amplification information defined in the standard.Voice coordination required.
8Current Efforts. DLWG (2) Fulfill NATO Secondary Image Format (NSIF).STANAG 4545, Edition 1 Amendment 1 dated 14 April 2002 – NATO Secondary Imagery Format (NSIF) Version 1.0.Compression Algorithms: JPEG & EagleEye.Image packet data of 45 bytes per J 16.0 message.Geo-position Information is included as a non displayable image segment.File Transfer protocols: one way & two ways.
9Proposed Still Image Compression Standard:JPEG2000 (1). Defines a “canvas coordinate system” based on tilesSince different components can have different sizes.Each tile is DWTd Provides “spatial” access to the image (can be operationally required).Discrete Wavelet Transform (DWT).Wavelets provides a robust tool for signal and image processing.More accurate time-frequency representation Can handle signals with discontinuities.Transform Domain: each sub-band of every resolution level is partitioned into blocks/precints/packets.
10Proposed Still Image Compression Standard:JPEG2000 (2) Embedded block coding algorithm (EBCOT): one code-stream can be decompressed in many ways.Each sub-band image divided into blocks that are coded independently.Introduces quality for “layers” concept.Progressive compression.By quality / resolution / components and positionState of the art compression efficiency.
12JPEG2000 Block Diagram ICT: Inter-component Transformation DWT: Discrete Wavelet Transform: Daubechies 9/7 (NR) & Daubechies 5/7 (Reversible)DZQ: Dead Zone Quantification
13JPEG2000: Wavelet Transform Sub-band Images from low & high pass filtering and sub-samplingSame samples as in original imageTransform is reversibleSub-bands are quantized(Image from David Taubman)
14JPEG2000: Wavelet Transform (2) Multi-resolution representation of the imageImage is LL/HL/LH/HH filtered in various levelsLL: Approximation of image (low frequency)LH: Horizontal details.HL: Vertical details.HH: Diagonal Details (high frequency)Note: Image by Konstantinos Kamaras Thesis, March 2002.
15JPEG2000: EBCOT (Embedded Block Coded with Optimal Truncation) EBCOT defines quality layers: each block stream is optimally truncated.From low quality imagery (coarse quantization) to the best image (finest quantization)
16JPEG 2000 Standard. Different Parts Part #1: Core coding & basic technology (JP2)Part #2: Extensions (JPX)Part#3: Motion JPEG2000 (MJ2)Part #5: Reference SW (Java & C)Part#8: Encryption and Watermarking (JPSEC)Part #9: Interactive Protocols and API (JPIP)Part#11: robust communication for wireless (JPWL)Part #12: ISO Base Media File Format (Common with MPEG-4)
17JPEG2000 Code stream (JP2)JPEG2000 coder produces an embedded code-stream that can be decoded at any length to produce an image.From 0 bit rate to the maximum (lossless).JPEG produces an independent code-stream per “target bit-rate”.Implies multiple coding passes less efficient.Embedded code stream support random access to image properties:Region of Interest (ROI).Zoom-in / out.
19Geo Referenced Images (Geography Markup Language) JPEG2000 doesn’t specify mechanism for geo-referencing the image, describing the sensor characteristics,…JPEG2000 provides room for “boxes” containing arbitrary XML data.GML (by Open Geo-Spatial Consortium) provides geo referencing information as XML encoded metadata.GML & JPEG2000 are interoperable and compatible.
20JPEG2000/JPEG Comparative Analysis Progressive lossy to lossless compressionJPEG: not integrated capability.Random Access to the code-stream:Implementation of Region of Interest capability.Better data rate control.JPEG2000: better compression efficiency.(Image1 & 2 are referenced in paper)Bpp0.1250.52.00Img1 JPEG24.4231.1735.15Img1 JPEG200028.1232.9537.35Img2 JPEG22.6028.9235.99Img2JPEG200024.8531.1338.80
21JPEG2000/JPEG Comparative Analysis in Noisy Channels Better performance in noisy channels. NOT EDC Technique Implemented WORS CASE SCENARIO better if R-S is modeled.See example: BER: 10^ (-4).Error Symbols in the code-stream:JPEG2000: 28JPEG: 18Method:JPEG2000 code stream is built (*.J2C)A Binary vector representation of the code stream is developed.A binary vector with the same BER channel is generated: x=(rand(tamano,1)<=ber)An exclusive OR is made between both vectors for generating a LINK-16 channel corrupted JPEG2000 code stream.Image reconstruction with “Kakadu”JPEGJPEG2000
23Requirements Flow Down: From Design to Operation Note: Link 16 Service channel: 8 TS assigned if required (Free Text)
24JPEG2000 Compressed Image Over Link 16 Method proposedTransmit the most sensitive part of JPEG2000 code-stream with maximum Link 16 Anti-Jamming Protection: STD Packing Limit.Headers/markers/first layers/tiles/Rest of the image with a less robust Packing Limit. (less AJP)JPEG2000 Coding option: progressive by quality.Quality Layers.Link 16.“free text” unformatted messages/ JXX.YY standard format message.NPGs: from (NPG 11 –assigned- & spare NPGs: 15 to 18)Max TSBs: 64 per Link 16 Terminal (48 in TSR)Time slots available for imageryTwo study cases: NMT 10% and NMT 5%.
25JPEG2000 Quality Layer to Time Slot Allocation Algorithm Match Link 16 available resources with JPEG2000 LRCP “quality layers”
26Matching Numbers: An Example Step 1 &2. Define the bit rate iaw network design: 32+x+y+z (time slots available for imagery)Step 3. Bit rate layer#1: 32 TS *XXX bits/TS_PL=YYY bits.Step 4. Size of the image=640 x 437 pixel.Step 5. Bit rate layer #1: r_1=YYY/640*437= bpp (P2DP).Step 6. Layer #n: r (n)= r(1) + (r), with (r ) iaw time slots allocation by network designStep 7. Image JPEG2000 coded LRCP iaw layer(n) bpp.Nota: Each additional quality layer can be allocated to a different NPG or TSAB of the corresponding NPG.
27Communication Architecture & Protocol Sensor Node (SN)Captures the image.Image Preprocessing & Compression: filtering/PSNR/Quality Criteria.Build Geo referencing code stream (If required).Imagery Management Node (IMN)Receive the image and request retransmission or changes in quality based on R-S Symbol threshold. (Errors in channel)Request for ROI.Request for more time-slots if in TSR.Reconstruct the original image based on code stream received and operational/tactical requirements.
28Protocol Note: All joint units receive the same code-stream but only IMN can interact with Sensor Node unless otherwise specified.
32JPEG2000 & Link 16 Combined Error Detection and Correction Tecniques
33Error Resilient Mode (Symplified). JPEG2000 Coded Image: Always include SOP-EPH markers in the code-stream.Much better syncronization mechanism.Error Management Protocol Implemented.Monitor Link 16 Parameter MER (Message Error Rate)Provides Reed-Solomon Coder Errors and Erasures.“If MER >Threshold (iaw AJ enviroment): Retransmit same packet with a more robust P/L”.Increase AJP.Decrease BPP.Commitment: Binary Rate AJ ProtectionImagen codificada siempre con SOP-EPH.Implementar Protocolo de gestión de errores Ver siguiente transparencia.Utrilizar Parámetros MER (Message Error Rate)Proporciona medida de errores y erasures del codificador R-S.Caso fallo Retransmitir en P/L de protección +.Válido para cualquier requisito de distancia.See Ref: JPEG2000 Image compression and error resilience for transmission over wireless channels (Konstantionos Kamaras, Mach NPS, Monterrey, ca)
34Error Recovery. An Example (1) 1 packet header corrupted +1 TIME SLOTJPEG200Code-stream#BytesP/LResilient Mode# Time SlotsP/L No resilient Mode#Time SlotsMAIN HEADER96STD4TILEPART #1121PACKET #1 HEADER11P2PACKET#113………………..….……………………….…………………TOTAL Time Slots2524
35Error Recovery. An Example (2) True JPEG2000 Coded Image ALL packet corrupted +12 TIME SLOTSTrama Codificada JPEG2000#BytesPacking Limit Resilient# Time SlotsRe-TXDelta #Time SlotsMAIN HEADER96STD4=TILEPART #1121PACKET #1 HEADER11PACKET#113P2(0.462)= 1 (+0)PACKET #2 HEADER17PACKET#238(1.3511)= 2 (+1)PACKET #3 HEADERPACKET#3952(3.3777)= 4 (+2)PACKET #4 HEADERPACKET#4207P4P2DP(3.68)= 4 (+2)PACKET #5 HEADERPACKET#5392(6.96)=7 (+3)PACKET #6 HEADERPACKET#6440(7.82)=8 (+4)TOTAL Time Slots2537 (+12 time slots)
36Some Simulations and Results Great amount of simulations has been conducted:One NPG STD P/L (by default)Two NPGs STD/P2Three NPGs STD/P2/P4Four NPGs STD/P2/P4/P4NEDCDLWG: Only one NPG allocated for imagery (NPG11)But several NPGs available for future use.Packing limit can also be changed before each Link 16 transaction via AP: Only one NPG will be required More “host overhead”.
46JPEG2000 SAR Images 1 Layer / 30 TS (P4)/0.05088 bpp/18.84 dB 5 Layers/ 80 TS (P4)/ bpp/ 19.84
47JPEG2000 Filtered SAR Images BPPPSNR(Filt)(Orig)T/S P4(Filtd)Original8 bpp-CompLay #10.050522.9718.8430Lay#50.139525.4819.848180Lay#100.5143--22.30---SAR Image median filtered. 5 layersPSNR regarding to filtered image
48SAR Image. Region Of Interest LAYERSBPPPSNR dbLink-16 Time SlotsLayer 164Layer 264+64=128Layer 310.93128+32=160 > 154Layer #1Layer #3
49SAR Image. Tiling Lay bitrate TS PSNR Tiles 1 0.0139 32 10.39 T0 2 0.06959611.44T0 a T330.125212812.60T0 a T540.1814420.07T0 a T8Layer #1 layer #2Layer #3 layer #4
50Summary (1)Recent conflicts have shown that real time imagery availability is one of the most demanding capabilities required by operational/tactical commanders.Integrating “imagery content” into the “common tactical picture” provides a better target identification, weapon release criteria, damage assessment and surveillance data in support of network centric warfare. Also avoids “request for frequency allocation.”Current efforts are focused mainly to support JPEG/EagleEye compressed images packed into J16.0 messages and released in the NPG 11.
51Summary (2)JPEG2000 is a more robust open source standard that produces a totally embedded code-stream.Method proposed to send the JPEG2000 image in a antijamming adaptative wayMost important part is more protected.Rest of the image in a less protected packing limit.Algorithm Proposed to adapt JPEG2000 quality layers to Link 16 available time slots.Results shown a very good compression ratio with less than 10% and good performance with 5% of the available Link 16 time slots.JPEG2000 also offers a great variety of new capabilities well suited for tactical exploitation such asROIWatermarksGeo-referencingMore robust error and detection schemes.
52Way Ahead Develop a technological demonstrator Plan on going. Propose an ad-hoc J message J XX.YY or modify J16.0 specification for including JPEG2000 codification syntax.Implement a GML compatible geo-referencing capabilities as a part of XML boxes available at JPX file format.Implement security and encryption capabilities iaw JPSEC file format.MPEG2000 for stream-video under assessment.And more…Some of these new proposals require an “enhance throughput” Link 16 capability.