Iterative-Pooling

"Multi-patch Aggregation Models for Resampling Detection", IEEE ICASSP 2020

Multi-patch Aggregation Models for Resampling Detection

The project is the official implementation of our IEEE ICASSP paper, “Multi-patch Aggregation Models for Resampling Detection”
— Mohit Lamba, Kaushik Mitra

The arXiv version of the paper can found at arXiv.org.

Images captured nowadays are of varying dimensions with smartphones and DSLR’s allowing users to choose from a list of available image resolutions. It is therefore imperative for forensic algorithms such as resampling detection to scale well for images of varying dimensions. However, in our experiments, we observed that many state-of-the-art forensic algorithms are sensitive to image size and their performance quickly degenerates when operated on images of diverse dimensions despite re-training them using multiple image sizes. To handle this issue, we propose a novel pooling strategy called Iterative Pooling. This pooling strategy can dynamically adjust input tensors of different size and shapes without much loss of information as in ROI Max-pooling. This pooling strategy can be used with any of the existing deep models and for demonstration purposes, we show its utility on Resnet-18 for the case of resampling detection a fundamental operation for any image sought of image manipulation. Compared to existing strategies and Max-pooling it gives up to 7-8% improvement on public datasets.

The project has been summarised in the following video.

Watch the project Video

Sample Output

Image Resolution after Resampling Patch Size
< 1024x1024 128x128
~ 1024x1024 but < 2000x2000 256x256
>2000x2000 512x512

Original Image size: 512x512
Patch size: 128x128
Resampling Factor : Correctly classified images / Total test images
0.6:968/1001 0.8:976/1001 1:961/1001 1.2:935/1001 1.4:936/1001

Original Image size: 1024x1024
Patch size: 128x128
Resampling Factor : Correctly classified images / Total test images
0.6:979/1001

Original Image size: 1024x1024
Patch size: 256x256
Resampling Factor : Correctly classified images / Total test images
0.8:989/1001 1:995/1001 1.2:997/1001 1.4:952/1001

Original Image size: greater than 1024x1024
Patch size: 256x256
Resampling Factor : Correctly classified images / Total test images
0.6:991/1001 0.8:959/1001

Original Image size: greater than 1024x1024
Patch size: 512x512
Resampling Factor : Correctly classified images / Total test images
1:994/1001 1.2:963/1001 1.4:999/1001

How to use the code?

It should run well for Pytorch>=1.00 and torchvision>=0.2.

The Dataset

Our code does not uses the validation dataset during the training phase and so the validation dataset can also be used for evaluation. We additionally provide a MATLAB file which illustrates how this dataste was created for image resolution of 1024x1024 and also describes how to add rotation to create JPEG+Roation+Resampling+JPEG manipulation dataste which is much harder case to deal with.

Cite us

If you find any information provided here useful please cite us,

  @inproceedings{lamba2020multi,
  title={multi-patch aggregation models for resampling detection},
  author={Lamba, Mohit and Mitra, Kaushik},
  booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={2967--2971},
  year={2020},
  organization={IEEE}
}
Mohit Lamba and Kaushik Mitra. "Multi-Patch Aggregation Models for Resampling Detection." IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020.
}