Presentation on theme: "IST-083 Symposium Progressive Still Image Transmission over a TDL Network A Study Case: JPEG2000 Compressed Images over a Link 16 Network Author: Cdr,"— Presentation transcript:
IST-083 Symposium Progressive Still Image Transmission over a TDL Network A Study Case: JPEG2000 Compressed Images over a Link 16 Network Author: Cdr, Manuel Martinez, PHD (SPN) Co author: Prof, Antonio Artes, PHD (UC3M) Co author: Maj, Roberto Sabatini, PHD (IT AF)
Scope Operational Requirements Image Technical Requirements JPEG2000 Standard Link 16 Case Study Some Examples
Imagery Operational Requirements Target images required before authorizing weapon release. Strike Aircraft aircrew using an image depicting a time critical target (TCT) for identification. Damage Assessment. UAV imagery supporting urban/roads battlefield.
To develop a composite tactical picture available to all Joint Units by using multimedia information content provided by TDLs in support of network centric warfare. Why in TDLs?
Extend TDLs concept from textual data to multimedia elements including still image & stream-video for a variety of image types –Visual –IR/FLIR –SAR Each image shows different characteristics based on the sensor performance and image type. Dont ask for new spectrum allocation in dense urban areas –Use the current TDLs (Link 16) allocated spectrum. –Use current AJP capabilities provided by TDLs (Link 16) to avoid jammers /civilian interferences.
Still Image Compression Requirements Compression Efficiency. –Excellent performance at low or very low data rate (disadvantage links). Random access to the code-stream. –Embedded code stream that allows to extract the image in different ways. –This shall allow to reconstruct an image based on a progressive scheme. Scalable. –Resolution. –Quality: definition of QoS and quality layers. –Components: color, B&W,… –Position. Lossless and Lossy Compression. Open architecture and standardized. Definition of Region of Interest (ROI), Zoom, Tiling,.. Robustness in channel error conditions.
Current Efforts. DLWG DLCP NUMBER: ML P33-US-M32-R6 J16.0 Image Transfer Message definition. NPG 11 dedicated for Imagery Minimum 32 Time Slots required per frame. Recommended P4 Packing Limit (Link 16). Supplement messages from surveillance and control. –Secondary role. Imagery and image amplification information defined in the standard. Voice coordination required.
Current Efforts. DLWG (2) Fulfill NATO Secondary Image Format (NSIF). –STANAG 4545, Edition 1 Amendment 1 dated 14 April 2002 – NATO Secondary Imagery Format (NSIF) Version 1.0. Compression Algorithms: JPEG & EagleEye. Image packet data of 45 bytes per J 16.0 message. Geo-position Information is included as a non displayable image segment. File Transfer protocols: one way & two ways.
Proposed Still Image Compression Standard:JPEG2000 (1). Defines a canvas coordinate system based on tiles –Since different components can have different sizes. Each tile is DWTd Provides spatial access to the image (can be operationally required). –Discrete Wavelet Transform (DWT). Wavelets provides a robust tool for signal and image processing. More accurate time-frequency representation Can handle signals with discontinuities. Transform Domain: each sub-band of every resolution level is partitioned into blocks/precints/packets.
Proposed Still Image Compression Standard:JPEG2000 (2) Embedded block coding algorithm (EBCOT): one code-stream can be decompressed in many ways. – Each sub-band image divided into blocks that are coded independently. – Introduces quality for layers concept. –Progressive compression. By quality / resolution / components and position State of the art compression efficiency.
JPEG2000: Wavelet Transform Sub-band Images from low & high pass filtering and sub-sampling Same samples as in original image Transform is reversible Sub-bands are quantized (Image from David Taubman)
JPEG2000: Wavelet Transform (2) Multi-resolution representation of the image Image is LL/HL/LH/HH filtered in various levels LL: Approximation of image (low frequency) LH: Horizontal details. HL: Vertical details. HH: Diagonal Details (high frequency) Note: Image by Konstantinos Kamaras Thesis, March 2002.
JPEG2000: EBCOT (Embedded Block Coded with Optimal Truncation) EBCOT defines quality layers: each block stream is optimally truncated. –From low quality imagery (coarse quantization) to the best image (finest quantization)
JPEG 2000 Standard. Different Parts Part #1: Core coding & basic technology (JP2) Part #2: Extensions (JPX) Part#3: Motion JPEG2000 (MJ2) Part #5: Reference SW (Java & C) Part#8: Encryption and Watermarking (JPSEC) Part #9: Interactive Protocols and API (JPIP) Part#11: robust communication for wireless (JPWL) Part #12: ISO Base Media File Format (Common with MPEG-4)
JPEG2000 Code stream (JP2) JPEG2000 coder produces an embedded code- stream that can be decoded at any length to produce an image. –From 0 bit rate to the maximum (lossless). JPEG produces an independent code-stream per target bit-rate. –Implies multiple coding passes less efficient. Embedded code stream support random access to image properties: – Region of Interest (ROI). –Zoom-in / out.
JPEG2000 Code stream (JP2) Tile Stream
Geo Referenced Images (Geography Markup Language) JPEG2000 doesnt specify mechanism for geo- referencing the image, describing the sensor characteristics,… JPEG2000 provides room for boxes containing arbitrary XML data. GML (by Open Geo-Spatial Consortium) provides geo referencing information as XML encoded metadata. GML & JPEG2000 are interoperable and compatible.
JPEG2000/JPEG Comparative Analysis Progressive lossy to lossless compression –JPEG: not integrated capability. Random Access to the code-stream: –Implementation of Region of Interest capability. –Better data rate control. JPEG2000: better compression efficiency. (Image1 & 2 are referenced in paper) Bpp Img1 JPEG Img1 JPEG Img2 JPEG Img2JPEG
JPEG2000/JPEG Comparative Analysis in Noisy Channels Better performance in noisy channels. NOT EDC Technique Implemented WORS CASE SCENARIO better if R-S is modeled. –See example: BER: 10^ (-4). –Error Symbols in the code-stream: JPEG2000: 28 JPEG: 18 Method: –JPEG2000 code stream is built (*.J2C) –A Binary vector representation of the code stream is developed. –A binary vector with the same BER channel is generated: x=(rand(tamano,1)<=ber) –An exclusive OR is made between both vectors for generating a LINK-16 channel corrupted JPEG2000 code stream. –Image reconstruction with Kakadu JPEG JPEG2000
JPEG2000/JPEG Comparative Analysis (2)
Requirements Flow Down: From Design to Operation Note: Link 16 Service channel: 8 TS assigned if required (Free Text) Requirements flow down
JPEG2000 Compressed Image Over Link 16 Method proposed –Transmit the most sensitive part of JPEG2000 code-stream with maximum Link 16 Anti-Jamming Protection: STD Packing Limit. Headers/markers/first layers/tiles/ –Rest of the image with a less robust Packing Limit. (less AJP) JPEG2000 Coding option: progressive by quality. –Quality Layers. Link 16. –free text unformatted messages/ JXX.YY standard format message. –NPGs: from 1..4 (NPG 11 –assigned- & spare NPGs: 15 to 18) –Max TSBs: 64 per Link 16 Terminal (48 in TSR) –Time slots available for imagery Two study cases: NMT 10% and NMT 5%.
JPEG2000 Quality Layer to Time Slot Allocation Algorithm Match Link 16 available resources with JPEG2000 LRCP quality layers
Matching Numbers: An Example Step 1 &2. Define the bit rate iaw network design: 32+x+y+z (time slots available for imagery) Step 3. Bit rate layer#1: 32 TS *XXX bits/TS_PL=YYY bits. Step 4. Size of the image=640 x 437 pixel. Step 5. Bit rate layer #1: r_1=YYY/640*437= bpp (P2DP). Step 6. Layer #n: r (n)= r(1) + (r), with (r ) iaw time slots allocation by network design Step 7. Image JPEG2000 coded LRCP iaw layer(n) bpp. –Nota: Each additional quality layer can be allocated to a different NPG or TSAB of the corresponding NPG.
Communication Architecture & Protocol Sensor Node (SN) –Captures the image. –Image Preprocessing & Compression: filtering/PSNR/Quality Criteria. –Build Geo referencing code stream (If required). Imagery Management Node (IMN) –Receive the image and request retransmission or changes in quality based on R-S Symbol threshold. (Errors in channel) –Request for ROI. –Request for more time-slots if in TSR. –Reconstruct the original image based on code stream received and operational/tactical requirements.
Protocol Note: All joint units receive the same code-stream but only IMN can interact with Sensor Node unless otherwise specified.
Quality Management (QM). (In terms of PSNR)
Concept. Example 1 Picture JPEG2000 coded Progressive LRCP Mode: 4 layers NGP allocation (if available): Packing Limit and Access Mode Layers 1 & 2: Dedicated Algorithm #1 Layers 3 & 4: TSR Algorithm #2 Link 16 Transmission
JPEG2000 & Link 16 Combined Error Detection and Correction Tecniques
Error Resilient Mode (Symplified). JPEG2000 Coded Image: Always include SOP-EPH markers in the code-stream. Much better syncronization mechanism. Error Management Protocol Implemented. Monitor Link 16 Parameter MER (Message Error Rate) –Provides Reed-Solomon Coder Errors and Erasures. If MER >Threshold (iaw AJ enviroment): Retransmit same packet with a more robust P/L. –Increase AJP. –Decrease BPP. Commitment: Binary Rate AJ Protection
Error Recovery. An Example (1) JPEG200 Code-stream #BytesP/LResilient Mode# Time SlotsP/L No resilient Mode#Time Slots MAIN HEADER96STD4 4 TILEPART #112STD1 1 PACKET #1 HEADER11STD1P21 PACKET#113P21 1 ………………..….……………………….………………… TOTAL Time Slots packet header corrupted +1 TIME SLOT
Error Recovery. An Example (2) True JPEG2000 Coded Image Trama Codificada JPEG2000 #BytesPacking Limit Resilient# Time SlotsRe-TXDelta #Time Slots MAIN HEADER96STD4 = TILEPART #112STD1 = PACKET #1 HEADER11STD1 = PACKET#113P21STD(0.462)= 1 (+0) PACKET #2 HEADER17STD1 = PACKET#238P21STD(1.3511)= 2 (+1) PACKET #3 HEADER17STD1 = PACKET#395P22STD(3.3777)= 4 (+2) PACKET #4 HEADER17STD1 = PACKET#4207P42P2DP(3.68)= 4 (+2) PACKET #5 HEADER17STD1 = PACKET#5392P44P2DP(6.96)=7 (+3) PACKET #6 HEADER17STD1 = PACKET#6440P44P2DP(7.82)=8 (+4) TOTAL Time Slots2537 (+12 time slots) ALL packet corrupted +12 TIME SLOTS
Some Simulations and Results Great amount of simulations has been conducted: –One NPG STD P/L (by default) –Two NPGs STD/P2 –Three NPGs STD/P2/P4 –Four NPGs STD/P2/P4/P4NEDC DLWG: Only one NPG allocated for imagery (NPG11) –But several NPGs available for future use. Packing limit can also be changed before each Link 16 transaction via AP: Only one NPG will be required More host overhead.
JPEG In Noisy Channels BER: 10^-4 Original JPEG
JPEG 2000 In Noisy Ch. BER: 10^-4 ORIGINAL JPEGJPEG2000
Original SAR Image: 8 bpp
JPEG2000 SAR Images 1 Layer / 30 TS (P4)/ bpp/18.84 dB 5 Layers/ 80 TS (P4)/ bpp/ 19.84
JPEG2000 Filtered SAR Images ImageBPPPSNR (Filt) PSNR (Orig) T/S P4 (Filtd) T/S P4 (Orig) Origina l 8 bpp-- Comp Lay # Comp Lay# Comp Lay# SAR Image median filtered. 5 layers PSNR regarding to filtered image
SAR Image. Region Of Interest LAYERSBPPPSNR dbLink-16 Time Slots Layer Layer =128 Layer =160 > 154 Layer #1Layer #3
SAR Image. Tiling Lay bitrate TS PSNRTiles T T0 a T T0 a T T0 a T8 Layer #1layer #2 Layer #3layer #4
Summary (1) Recent conflicts have shown that real time imagery availability is one of the most demanding capabilities required by operational/tactical commanders. Integrating imagery content into the common tactical picture provides a better target identification, weapon release criteria, damage assessment and surveillance data in support of network centric warfare. Also avoids request for frequency allocation. Current efforts are focused mainly to support JPEG/EagleEye compressed images packed into J16.0 messages and released in the NPG 11.
Summary (2) JPEG2000 is a more robust open source standard that produces a totally embedded code-stream. Method proposed to send the JPEG2000 image in a antijamming adaptative way –Most important part is more protected. –Rest of the image in a less protected packing limit. Algorithm Proposed to adapt JPEG2000 quality layers to Link 16 available time slots. Results shown a very good compression ratio with less than 10% and good performance with 5% of the available Link 16 time slots. JPEG2000 also offers a great variety of new capabilities well suited for tactical exploitation such as –ROI –Watermarks –Geo-referencing –More robust error and detection schemes.
Way Ahead Develop a technological demonstrator –Plan on going. Propose an ad-hoc J message J XX.YY or modify J16.0 specification for including JPEG2000 codification syntax. Implement a GML compatible geo-referencing capabilities as a part of XML boxes available at JPX file format. Implement security and encryption capabilities iaw JPSEC file format. MPEG2000 for stream-video under assessment. And more… Some of these new proposals require an enhance throughput Link 16 capability.