People often transmit digital images over the internet and JPEG is one of the most common used formats. Steganography is the art and science of hiding communication; the information hiding process thus uses an image as a cover medium to embed a hidden message. Steganalysis is the inverse process of trying to identify the existence of hidden message in a cover image. In this paper, we present an enhancement to the steganalysis algorithm that successfully attacks F5 steganographic algorithm. The key idea is related to the selection of an "optimal" value of β (the probability that a non-zero AC coefficient will be modified) for the image under consideration. Rather than averaging the values of β for 64 shifting steps worked on an image, an optimal β is determined that corresponds to the shift having minimal distance E from the double compression removal step. Numerical experiments were carried out to validate the proposed enhanced algorithm and compare it against the original one. Both algorithms were tested and compared using two sets of test images. The first set uses reference test data of 20 grayscale images [1], and the second uses 432 images created by manipulating 12 images for various image parameters: two sizes (300×400 and 150×2000), six JPEG old quality factors (50, 60, 70, 80, 90, 100), and 3 message lengths (0, 1kB, 2 kB). The results suggest that the original algorithm may be used as a classifier, since it shows a good detection performance of both clean and stego test images; whereas, the proposed enhanced algorithm may be used as an estimator for the true message length for those images that have been classified by original algorithm as stego images |