Correct Photos of Documents – Two- and Three-dimensional Skew

As I mentioned in my previous post, Correct Photos of Documents – Ambient Lighting, most images of documents suffer from skew. There are two types of skew: two-dimensional and three-dimensional.

Text is typically parallel to the top and bottom of the paper. Simply put, two-dimensional skew is the angle of the text when compared to the top or bottom edge of the image. This type of skewing can occur in photos of documents as well as document images produced by a scanner. While it is relatively easy to correct with a simple rotate, determining the angle of rotation can be tricky. Fortunately, LEADTOOLS includes a DeskewCommand class that can determine the angle of skew.

private static void AutoDeskew(RasterImage image)
{
    ProcessImage(image,
        new DeskewCommand(RasterColor.White, DeskewCommandFlags.FillExposedArea));
}

Three-dimensional skew is also known as perspective skew and is more complicated than a simple rotation angle distortion. When the document is skewed in three dimensions, a rectangular document is warped into convex quadrilateral. To correct a perspective skew, the corners of the document have to be found and then the image has to be warped back into a rectangular shape. Again, LEADTOOLS includes the PerspectiveDeskewCommand class to automatically find the corners and correct the distortion without user input. As a bonus, the command removes parts of the image that it did not consider to be a part of the document.

private static void PerspectiveDeskew(RasterImage image)
{
    ProcessImage(image, new PerspectiveDeskewCommand());
}

The aforementioned PerspectiveDekewCommand class does not require any user input, which is perfect for my workflow. (I want to right-click any document image in Windows Explorer and convert it to PDF). The PerspectiveDekewCommand usually does a perfect job, but there are some cases where it can incorrectly remove part of the document. An example of an image that can confuse the command is one of a letter that is tri-folded from being in an envelope. Ideally, the paper should be completely flat before taking the picture, but that is not always possible. The fold causes three-dimension distortion in two different directions, so the PerspectiveDeskewCommand corrects the larger part of the image with the distortion.

In the “Before” example above, I highlighted what the PerspectiveDeskewCommand found as the document corners. As shown, the part of the document above the fold is incorrectly cropped out of the “After” image. Depending on the application, LEADTOOLS has other image processing commands, such as the KeyStoneCommand and ManualPerspectiveDeskewCommand, that can be used if the PerspectiveDeskewCommand does not work perfectly. Both require user input to find the corners of the document. Since my workflow does not allow for user input, I cannot use them. However, there is another way.

Stay tuned for my next post where I show how I solved this problem in a way that does not require any user input. 🧙