1. Combinatorial Testing on ID3v2 Tags of MP3 Files
Zhiqiang Zhang1, Xiaojian Liu2, Jian Zhang1
1 Institute of Software, Chinese Academy of Sciences
2 Institute of Automation, Shandong Academy of Sciences

2. Introduction
MP3 (MPEG-1 or MPEG-2 audio layer III) is an audio file format developed by the Moving Pictures Experts Group (MPEG)
One of the most popular audio formats
Supported by almost all audio players
Usually, an MP3 file comes with an ID3 tag, which stores audio information such as:
Title, artist, album, …
ID3 has two unrelated versions:
ID3v1 & ID3v2

3. Overview ID3v2 tag format Test goals Modeling Experiments & Results
Conclusion

4. Combinatorial Testing
Considerably high fault coverage
Model the system under test (SUT) as a parameterized black box
Usually use a covering array as the test set
Small # of test cases

5. MP3 frames & ID3v2 tags
An MP3 file is built up from a sequence of MP3 frames (MPEG frames)
A short segment of audio data
The first 11 or 12 bits are always set, can be used for synchronization
Sometimes MP3 frames depend on each other, and cannot be freely cut
ID3v2 tag:
A metadata container for audio file information
Located at the beginning of an MP3 file
ID3v2 tag
MP3 frames

6. ID3v2 tag format
In our work, we consider a subset of the ID3v2 tag format
An ID3v2 header
Several ID3v2 frames
ID3v2 header format:
ID3v2/file identifier “ID3”
ID3v2 version $03 00
ID3v2 flags %abc00000
ID3v2 size * %0xxxxxxx
ID3v2 header
ID3v2 frame 1
ID3v2 frame 2
ID3v2 frame 3 …
Tag size

7. ID3v2 Frames
An ID3v2 frame consists of a frame header and the frame content
Frame header
Frame ID $xx xx xx xx (four characters)
Size $xx xx xx xx
Flags $xx xx
Text information frames
Text encoding $xx
Information <text string>
Frame header
Frame content
Frame size

8. Text Encodings Encodings commonly used in China: $00: ASCII or GBK
$01: Unicode (UTF-16)
Little-endian (LE)
Big-endian (BE)
$02: Unicode BE
$03: UTF-8

9. Test Goals
Usually, tag processing and audio playing are performed by separate modules of MP3 players
Audio playing is often done by a decoder (out of the scope of our work)
ID3v2 tag processing
Goal I: text information recognition and display
Text encodings of some frames may influence the processing of other frames
Goal II: robustness against bad header & frame sizes
Offset related
May cause buffer overflow or vulnerable read operations

10. Modeling (Test Goal I) Six types of text information frames The model:
TIT2 (title), TPE1 (artist), TALB(album), TRCK (track number), TCON (genre) and TYER(year)
The model:
TIT2
TPE1
TALB
TRCK
TCON
TYER
NONE
ASCII
GBK
ULE
UBE
UBE2
UTF-8

11. Modeling (Test Goal II)
The sizes indicated by the ID3v2 header & frame size bytes might be faulty
May cause critical failures
For building each test case
Set the header and frame size bytes
Set the actual header and frame sizes
Build an ID3v2 frame
Build an ID3v2 tag by filling the content with frames
Attach MP3 frames

12. Modeling (Test Goal II)
Building an ID3v2 frame
Building an ID3v2 tag
Frame size bytes
Frame header
$00 AAAAAA…
Tag size bytes
ID3v2 header
ID3v2 frame …
Actual frame size
Actual tag size

13. Modeling (Test Goal II)
Building a complete MP3 file
Attach a sequence of MP3 frames after the ID3v2 tag
In some cases, the actual tag size is not an integral multiple of the total frame size
To fill the remaining space: 4 filling modes

14. Filling Modes None Safe Cut Overflow Tag size ID3v2 frame MP3 frames
$ …
Tag size
ID3v2 frame
MP3 frames
ID3v2 frame
MP3 frames
Tag size
None
Safe
Cut
Overflow

15. Modeling (Test Goal II)TS
TSD FS
FSD FM
ATCHMP3
0KB 0B
NONE
YES
4KB
1KB
256B
64B
SAFE NO
64KB
-1KB
-64B
CUT
OVFL
TS: tag size indicated by the ID3v2 header size bytes
TSD: difference of the actual tag size compared with TS
FS: frame size indicated by the frame header size bytes
FSD: difference of the actual frame size compared with FS
FM: filling mode
ATCHMP3: whether MP3 frames are attached or not

16. Modeling (Test Goal II)
Normal constraints
IF TS=0KB THEN TSD<>-1KB
IF FS=0KB THEN FSD<>-64B
When TS=0KB and TSD=0KB, the actual tag size is 0KB. So no frames will be filled. Thus FS, FSD and FM are invalid parameters
Introduce a special parameter value ‘#’ for invalid parameters
IF TS=0KB AND TSD=0KB THEN FS=# AND FSD=# AND FM=#
IF TS<>0KB OR TSD<>0KB THEN FS<># AND FSD<># AND FM<>#

17. Experiments Test subjects: Test generation:
I: an on-vehicle leisure and entertainment system
II: a portable MP3 player
Test generation:
Use Microsoft’s PICT
Test Goal I: 57 test cases (strength=2)
Test Goal II: 59 test cases (strength=3)

18. Experimental Results (Test Goal I, Subject I)
Condition
Results
ALL
ASCII text is correctly displayed.
UBE2 text is incorrectly displayed.*
TIT2=NONE
Title is displayed as file name.
TPE1=NONE
Artist is displayed as folder name.
TPE1{ASCII,UBE2} AND
TIT2{GBK,ULE,UBE,UTF-8}
Title is not displayed.*
TPE1{ASCII,UBE2} AND
TIT2{GBK,ULE,UBE,UTF-8}
Title is displayed as file name.*
Artist is displayed as folder name.*
2 passed, 55 failed
Actually, only ASCII text can be displayed

19. Experimental Results (Test Goal I, Subject II)
Condition
Results
TIT2{UBE2,UTF-8}
File name is displayed as the title.*
TIT2<>NONE AND
TPE1{UBE2,UTF-8}
OTHERWISE
All information is correctly displayed.
33 passed, 24 failed
Problem when processing two encodings
Failures caused by 1-2 parameters

20. Experimental Results (Test Goal II, Subject I)
Condition
Results
TS=0KB AND
TSD=1KB AND
FS<>0KB AND
ATCHMP3=NO
The subject encounters a failure, and all audio
files copied into the subject cannot be played.*
ATCHMP3=YES
The file can be successfully played.
The file is skipped.
52 passed, 7 failed
Failures caused by 4 parameters

21. Experimental Results (Test Goal II, Subject II)
Condition
Results
ATCHMP3=YES
The file can be successfully played.
ATCHMP3=NO
The player shows that the file is damaged
and proceeds to the next file.
All test cases passed
We occasionally found a bug
When ATCHMP3=NO, the subject
Displays that the file is damaged for a few seconds
Proceeds to the next file.
Just between this two moments, it is not possible to play back to the previous file
Not Discovered by CT!!

22. Conclusion Benefit from CT: Small test suite
Test Goal I: 57 out of 7323=2744 exhaustive test cases
Test Goal II: 59 out of 3442=648 exhaustive test cases (including test cases not satisfying the constraints)
Interaction faults discovered
Failures are caused by 1-4 parameters

23. Additional Results Experiments are conducted on two PC audio players
PC audio player I perfectly passes all the tests
PC audio player II:
Test Goal I: similar failures as subject I
Test Goal II:
When all test cases are added to the player playlist, the player’s UI begin to respond slowly
The player got stuck when playing 4 test cases

24. Thank you！