1. Least Significant Bit Steganography Detection with Machine Learning Techniques Shen Ge 1, Yang Gao 1, Ruili Wang 2 1 State Key Laboratory for Novel Software Technology, Nanjing University 2 Institute of Information Sciences and Technology Massey University (Turitea) We are so sorry that we are not able to come due to the visa problem.

2. Outline IntroductionIntroduction MotivationMotivation Conventional MethodsConventional Methods Our Point of ViewOur Point of View Our FrameworkOur Framework Experiment ResultsExperiment Results Conclusions and Future WorkConclusions and Future Work

3. Introduction (1) SteganographySteganography –the goal to ensure secret messages transferred secretlyto ensure secret messages transferred secretly to make the transferred secret messages undetectable.to make the transferred secret messages undetectable. –It is the art of invisible communication, and provide a plausible deniability to secret communication. SteganalysisSteganalysis –the goal to detect the existence of steganographyto detect the existence of steganography to estimate its message lengthto estimate its message length or to extract the hidden informationor to extract the hidden information –The steganalysis algorithms achieve their goals by exploiting the differences between the media files before and after embedding.

4. Introduction (2) An illustrating exampleAn illustrating example We will attack at 5:00 AM tomorrow Embedding (Steganography) Extracting Hidden Message Detecting (Steganalysis) There is embedded message!

5. Motivation (1) The using of steganography can cause security problemsThe using of steganography can cause security problems –Suppose an employee send out an image embedded with commercial secrets, current network firewalls cannot block such communications –So steganalysis is need to analyze the suitable cover media and point out possible embedded ones for further process

6. Motivation (2) In this paper we focus on LSB hidden information detection with machine learning techniquesIn this paper we focus on LSB hidden information detection with machine learning techniques –LSB is the most popular steganography methods –The detection can be used in real applications We need new methods to detect the existence of LSB embedded messageWe need new methods to detect the existence of LSB embedded message –Universal steganalysis seems to be too generic thus low accuracy, conventional LSB steganalysis seems to be too specific –Steganalysis is proposed to estimate the length of hidden message, so if we focus on the detection problem, we have to do more

7. Conventional Methods (1) Our Classification for Steganography MethodsOur Classification for Steganography Methods –Operates in bitmaps: LSB –Embeds in transformed domain images (such as JPEG images): OutGuess, F5 Our Classification for Steganalysis MethodsOur Classification for Steganalysis Methods – Instance based use training sets, and involve a classifier construction processuse training sets, and involve a classifier construction process IQM based, High-Order DWT, Calibrated FeatureIQM based, High-Order DWT, Calibrated Feature – Non-instance based exploit the statistics of the image by an implicated parametric model, and classification is done by heuristics.exploit the statistics of the image by an implicated parametric model, and classification is done by heuristics. Most conventional methodsMost conventional methods C2, RS, SPA, JPEG compatibilityC2, RS, SPA, JPEG compatibility

8. Conventional Methods (2) We focus on three main conventional steganalysis methods of LSBWe focus on three main conventional steganalysis methods of LSB The  2 Method []The  2 Method [Pfitzmann and Westfeld] –Idea: To find the statistical evidence which is left by the embedding process –The LSB embedding process can be viewed as a flip operation, if the pixel ’ s LSB is the same with the bit we want to embed, then this pixel is untouched, else it will be flipped, pixel 2j will be flipped to 2j+1,and 2j+1 will be flipped to 2j

9. Conventional Methods (3) The  2 MethodThe  2 Method –We combine pixel 2j and 2j+1 to a pair which we call PoV (Pair of Value) –From experiment, we can see that before embedding, the frequencies of the two pixels in a specific pair seems to distribute randomly, and after embedding, they will be nearly the same because the pixel ’ s LSB are replaced by the message 0 and 1 bits which is usually uniformly distributed

10. Conventional Methods (4) The  2 MethodThe  2 Method –So we can develop equations of the probability of images being embedded (also the message length) –We define h 2i and h 2i+1 as the frequencies of the two pixels in a certain pair, in order to tell if there is significant difference between the distributions of the two value, we can use  2 test, we need to calculate  2 statistics (h 2i *=(h 2i +h 2i+1 )/2) –And finally we can calculate p as the message length (k is the total number of all possible i) –In fact, the accuracy of this methods decrease sharply when the number of pixels increase, so we often split the images into groups to analyse

11. Conventional Methods (3) The RS Method [J. Fridrich et al]The RS Method [J. Fridrich et al] –Idea: using some experiment justified hypothesis, to estimate the message length –Since this method is very complicated, we don ’ t discuss the theory behind it, we just describe the procedure needed –Terms Group:Group: Discriminate function:Discriminate function: Flip function:Flip function: –F1:–F1:–F1:–F1: –F -1 : –F0 :–F0 :–F0 :–F0 : MaskMask Flip: every group is flipped by F and some mask MFlip: every group is flipped by F and some mask M

12. Conventional Methods (4) The RS MethodThe RS Method –The Group Types RegularRegular SingularSingular UnusableUnusable –We denote the relative number of R and S groups using mask M  {0,1} as R M, S M, using -M  {-1,0} as R -M,S -M –The hypothesis For typical unembedded cover image R M  R -M S M  S -MFor typical unembedded cover image R M  R -M S M  S -M Straight line: R -M (p/2)-R -M (1-p/2) and S -M (p/2)-S -M (1-p/2)Straight line: R -M (p/2)-R -M (1-p/2) and S -M (p/2)-S -M (1-p/2) Parabolas: R M (p/2), R M (1/2), R M (1-p/2) and S M (p/2), S M (1/2), S M (1- p/2)Parabolas: R M (p/2), R M (1/2), R M (1-p/2) and S M (p/2), S M (1/2), S M (1- p/2) In RS diagram (see next page), the intersection point of R M and R -M has the same x coordinate with wich of S M and S -MIn RS diagram (see next page), the intersection point of R M and R -M has the same x coordinate with wich of S M and S -M R M (1/2)=S M (1/2)R M (1/2)=S M (1/2)

13. Conventional Methods (5) The RS MethodThe RS Method –The RS diagram 50% x stands for 100% embedding50% x stands for 100% embedding The value for p/2 is get through the image statistics, the value for 1-p/2 is get though the flipped imageThe value for p/2 is get through the image statistics, the value for 1-p/2 is get though the flipped image Using the hypothesis stated before, we can calculate the final message length pUsing the hypothesis stated before, we can calculate the final message length p –The final formula for calculating p x is one root of equationx is one root of equation p=x p /(x p -1/2)p=x p /(x p -1/2)

14. Conventional Methods (5) The SPA Method [Dumitrescu et al.]The SPA Method [Dumitrescu et al.] –Idea: classifier value pairs into groups, study the transition between them, and use this to estimate the message length –Terms We assume the image is represented by successive samples: s 1,s 2, …,s N (N is the total number of the sample)We assume the image is represented by successive samples: s 1,s 2, …,s N (N is the total number of the sample) A sample pair is a tuple (s i,s j ) 1  i,j  NA sample pair is a tuple (s i,s j ) 1  i,j  N We group the tuples into four types (1  m  2 b -1) (b is the pixel total bits)We group the tuples into four types (1  m  2 b -1) (b is the pixel total bits) –X 2m+1 : (2k-2m-1,2k) or (2k,2k-2m-1) –Y 2m+1 : (2k-2m,2k+1) or (2k+1,2k-2m) –X 2m : (2k-2m,2k) or (2k+1,2k-2m+1) –Y 2m : (2k-2m+1,2k+1) or (2k,2k-2m) –C m : The Union of X 2m-1,X 2m,Y 2m,X 2m+1, closed under embedding –D m : The Union of X 2m and Y 2m Considered the flip of each pixel in a pair, we get the finite state machine which can help calculate the message length pConsidered the flip of each pixel in a pair, we get the finite state machine which can help calculate the message length p

15. Conventional Methods(6) The SPA MethodThe SPA Method –The finite state transition machine –The final equation for calculating p:

16. Our Point of View The detection problem itself can be treated as a standard classification problem. Here our input is the message, the output is the label we assign to indicate whether this image has been embedded or not.The detection problem itself can be treated as a standard classification problem. Here our input is the message, the output is the label we assign to indicate whether this image has been embedded or not. Conventional methods do not focus on detection, if we want to directly use these methods, we have to use thresholdConventional methods do not focus on detection, if we want to directly use these methods, we have to use threshold We introduce machine learning technique to wrap on conventional algorithms, so as to get better performance and generalization abilityWe introduce machine learning technique to wrap on conventional algorithms, so as to get better performance and generalization ability We extract features from the image based on conventional methodsWe extract features from the image based on conventional methods

17. Our Framework The frameworkThe framework We get a series of methods using different classifiersWe get a series of methods using different classifiers We use features based on conventional LSB steganalysis methods because we want to ensure the performance on LSB detectionWe use features based on conventional LSB steganalysis methods because we want to ensure the performance on LSB detection –Sequential case, the features are  2 coefficients –Non-sequential case, the features are RS derived values

18. Experiment Results (1) DatasetDataset –24-bit color images from public domains are used. We embedded different length of messages into the images. We extract features using different methods for sequential and non-sequential case. Every image will result an instance represented by a set of features in the dataset. ClassifiersClassifiers –We build our experiment platform on WEKA –We choose Naive Bayes, Bayes Net, the J48 decision tree, kNN, SVM (SMO algorithm with RBF kernel) and BP (Back Propagation) neural network (here we use the default parameter indicated by WEKA) to build classifiers for a benchmark Other issuesOther issues –We use the entropy of H value of HSV color as the measure of image internal complexity, and divide the image set to five complexity levels and use four different embed rates as 10%, 20%, 50% and 100%. We make data gathering by internal complexity and embed rate to simulate the real situation. –This grouping operation is intend to investigate how will the data properties influence the final result

19. Experiment Results (2) We find that there are two cases in LSB embeddingWe find that there are two cases in LSB embedding Sequential caseSequential case –the bits are successively embedded, so we can find “ clusters ” of bits embedded, resulting in abrupt changes in the bits statistics, and this makes the detection easier. –We test wrapped methods on pov3 algorithm (a variation of  2 ), features are based on  2,so the threshold based-  2 with threshold of 95% and 99% are used for comparison with the 10-fold cross validation results of ML methods –We split the RGB LSB plane to 100 segments, every  2 coefficient is the feature, we have three palnes, so we have 300 features for each image Non-sequential caseNon-sequential case –Non-sequential case: the embedded bits are scattered randomly in the data –We test wrapped methods on the feature we derived from RS, and use threshold RS for comparison –The features is derived from RS, we have 2 features for each image

20. Experiment Results (3) Sequential caseSequential case –The results show that the precision decreases when the image complexity increases and increases when the embed rate increases. –We can see that except for Na ï ve Bayes and Bayes Net, other traditional methods like kNN, J48 and SVM can get a same or better accuracy than simple  2 method.

21. Experiment Results (4) Sequential case:Sequential case: –Because most of the pictures are in high complexity level 2-4, so ML- based methods are generally performs better than simple  2. We can make conclusion that applying machine learning to  2 can effectively improve the accuracy, and the classifier wrapped conventional steganalysis maybe a good solution to detect sequential LSB steganography.

22. Experiment Results (5) Non-sequential case:Non-sequential case: –From table, we can see that J48 performs best in mixed embed rate case, and can get nearly 95% accuracy at all embed levels. We use only two features, this result is comparable to  2 case in sequential embedding and is better than threshold based RS can do. –The best precision of threshold based SPA methods is 93.20%, and the precision of our J48-based method is 94.44%. Precision (RMS)Embed 0.1Embed 0.2Embed 0.5Embed 1.0Embed All Mixed Naijve Bayes50.90%(0.59)53.80%(0.57)54.90%(0.45)94.56%(0.31)80.48%(0.37) Bayes Net89.21%(0.27)95.85%(0.17)99.35%(0.07)99.45%(0.07)95.44%(0.18) kNN92.16%(0.28)97.55%(0.16)99.45%(0.07)99.70%(0.05)96.38%(0.19) J4894.11%(0.27)98.05%(0.14)99.40%(0.08)99.65%(0.06)97.56%(0.14) SMO59.39%(0.64)75.32%(0.50)93.21%(0.26)96.70%(0.18)80.90%(0.44) BP53.10%(0.50)54.10%(0.50)52.70%(0.50)56.80%(0.50)80.00%(0.40) Threshold RS95.30%98.75%99.70%87.11%97.38%

23. Conclusions and Future Work In this paper, we have developed a general framework of applying machine learning to steganalysis for LSB hidden information detection. We justify its superiority by experiments in both sequential LSB and non-sequential LSB case.In this paper, we have developed a general framework of applying machine learning to steganalysis for LSB hidden information detection. We justify its superiority by experiments in both sequential LSB and non-sequential LSB case. Possible future worksPossible future works –the seeking of more theoretical explanation for the effectiveness of our framework. –to use feature selection and nonlinear mapping to construct more effective features. –to extend our framework to non-LSB steganalysis (especially for JPEG steganalysis). –more effective learning techniques like cost-sensitive learning and class- imbalance learning will be incorporated in our framework for more effective classifiers. –Anyway, applying machine learning to steganalysis needs further discussion and more research.

24. Main References A. Westfeld and A. Pfitzmann. Attacks on steganographic systems. In Information Hiding Third International Workshop IH'99 Proceedings, Lecture Notes in Computer Science vol. 1768, pages 61-76, 1999.A. Westfeld and A. Pfitzmann. Attacks on steganographic systems. In Information Hiding Third International Workshop IH'99 Proceedings, Lecture Notes in Computer Science vol. 1768, pages 61-76, 1999. J. J. Fridrich and M. Goljan. Practical steganalysis of digital images { state of the art. In Security and Watermarking of Multimedia Contents, SPIE vol. 4675, pages 1-13, 2002.J. J. Fridrich and M. Goljan. Practical steganalysis of digital images { state of the art. In Security and Watermarking of Multimedia Contents, SPIE vol. 4675, pages 1-13, 2002. S. Dumitrescu, X. Wu, and Z. Wang. Detection of lsb steganography via sample pair analysis. In Information Hiding 5th International Workshop IH 2002 Revised Papers, Lecture Notes in Computer Science vol. 2578, pages 355-372, 2003.S. Dumitrescu, X. Wu, and Z. Wang. Detection of lsb steganography via sample pair analysis. In Information Hiding 5th International Workshop IH 2002 Revised Papers, Lecture Notes in Computer Science vol. 2578, pages 355-372, 2003. A. D. Ker. Improved detection of lsb steganography in grayscale images. In Information Hiding 6 th International Workshop, IH 2004 Revised Selected Papers, Lecture Notes in Computer Science vol. 3200, pages 97-115, 2004A. D. Ker. Improved detection of lsb steganography in grayscale images. In Information Hiding 6 th International Workshop, IH 2004 Revised Selected Papers, Lecture Notes in Computer Science vol. 3200, pages 97-115, 2004

25. Thank you!