I have a binary data with a JPEG file, and I need to compress this. I was try with some tools from the forge but in some images the size increases not decreases.

The challenge has to do with the way that compression is handled by JPEG which makes use of a lossy compression technique.

Whenever a JPEG is compressed there are artefacts that are added to the image. The artefacts may not be visible but the image is different. When a JPEG is compressed again there are additional artefacts that get added to the image. In some cases, this can result in the colour channels being re-sampled which can result in an increase in file size after compression or the artefacts start degrading the ability to compress any further and the size starts increasing on each of the saves.

PNG, on the other hand, makes use of a lossless compression technique which has advantages in the ability to use transparency, colour palettes and retains the pixel clarity of the original PNG image. Each compression does not result in a compression of the previous image but a compression of the original image representation.

I am not a graphics expert but this has been my experience.