Rotate image and crop out black borders

Question:

My application: I am trying to rotate an image (using OpenCV and Python)

Rotating Images

At the moment I have developed the below code which rotates an input image, padding it with black borders, giving me A. What I want is B – the largest possible area crop window within the rotated image. I call this the axis-aligned boundED box.

This is essentially the same as Rotate and crop, however I cannot get the answer on that question to work. Additionally, that answer is apparently only valid for square images. My images are rectangular.

Code to give A:

import cv2
import numpy as np


def getTranslationMatrix2d(dx, dy):
    """
    Returns a numpy affine transformation matrix for a 2D translation of
    (dx, dy)
    """
    return np.matrix([[1, 0, dx], [0, 1, dy], [0, 0, 1]])


def rotateImage(image, angle):
    """
    Rotates the given image about it's centre
    """

    image_size = (image.shape[1], image.shape[0])
    image_center = tuple(np.array(image_size) / 2)

    rot_mat = np.vstack([cv2.getRotationMatrix2D(image_center, angle, 1.0), [0, 0, 1]])
    trans_mat = np.identity(3)

    w2 = image_size[0] * 0.5
    h2 = image_size[1] * 0.5

    rot_mat_notranslate = np.matrix(rot_mat[0:2, 0:2])

    tl = (np.array([-w2, h2]) * rot_mat_notranslate).A[0]
    tr = (np.array([w2, h2]) * rot_mat_notranslate).A[0]
    bl = (np.array([-w2, -h2]) * rot_mat_notranslate).A[0]
    br = (np.array([w2, -h2]) * rot_mat_notranslate).A[0]

    x_coords = [pt[0] for pt in [tl, tr, bl, br]]
    x_pos = [x for x in x_coords if x > 0]
    x_neg = [x for x in x_coords if x < 0]

    y_coords = [pt[1] for pt in [tl, tr, bl, br]]
    y_pos = [y for y in y_coords if y > 0]
    y_neg = [y for y in y_coords if y < 0]

    right_bound = max(x_pos)
    left_bound = min(x_neg)
    top_bound = max(y_pos)
    bot_bound = min(y_neg)

    new_w = int(abs(right_bound - left_bound))
    new_h = int(abs(top_bound - bot_bound))
    new_image_size = (new_w, new_h)

    new_midx = new_w * 0.5
    new_midy = new_h * 0.5

    dx = int(new_midx - w2)
    dy = int(new_midy - h2)

    trans_mat = getTranslationMatrix2d(dx, dy)
    affine_mat = (np.matrix(trans_mat) * np.matrix(rot_mat))[0:2, :]
    result = cv2.warpAffine(image, affine_mat, new_image_size, flags=cv2.INTER_LINEAR)

    return result
Asked By: aaronsnoswell

||

Answers:

So, after investigating many claimed solutions, I have finally found a method that works; The answer by Andri and Magnus Hoff on Calculate largest rectangle in a rotated rectangle.

The below Python code contains the method of interest – largest_rotated_rect – and a short demo.

import math
import cv2
import numpy as np


def rotate_image(image, angle):
    """
    Rotates an OpenCV 2 / NumPy image about it's centre by the given angle
    (in degrees). The returned image will be large enough to hold the entire
    new image, with a black background
    """

    # Get the image size
    # No that's not an error - NumPy stores image matricies backwards
    image_size = (image.shape[1], image.shape[0])
    image_center = tuple(np.array(image_size) / 2)

    # Convert the OpenCV 3x2 rotation matrix to 3x3
    rot_mat = np.vstack(
        [cv2.getRotationMatrix2D(image_center, angle, 1.0), [0, 0, 1]]
    )

    rot_mat_notranslate = np.matrix(rot_mat[0:2, 0:2])

    # Shorthand for below calcs
    image_w2 = image_size[0] * 0.5
    image_h2 = image_size[1] * 0.5

    # Obtain the rotated coordinates of the image corners
    rotated_coords = [
        (np.array([-image_w2,  image_h2]) * rot_mat_notranslate).A[0],
        (np.array([ image_w2,  image_h2]) * rot_mat_notranslate).A[0],
        (np.array([-image_w2, -image_h2]) * rot_mat_notranslate).A[0],
        (np.array([ image_w2, -image_h2]) * rot_mat_notranslate).A[0]
    ]

    # Find the size of the new image
    x_coords = [pt[0] for pt in rotated_coords]
    x_pos = [x for x in x_coords if x > 0]
    x_neg = [x for x in x_coords if x < 0]

    y_coords = [pt[1] for pt in rotated_coords]
    y_pos = [y for y in y_coords if y > 0]
    y_neg = [y for y in y_coords if y < 0]

    right_bound = max(x_pos)
    left_bound = min(x_neg)
    top_bound = max(y_pos)
    bot_bound = min(y_neg)

    new_w = int(abs(right_bound - left_bound))
    new_h = int(abs(top_bound - bot_bound))

    # We require a translation matrix to keep the image centred
    trans_mat = np.matrix([
        [1, 0, int(new_w * 0.5 - image_w2)],
        [0, 1, int(new_h * 0.5 - image_h2)],
        [0, 0, 1]
    ])

    # Compute the tranform for the combined rotation and translation
    affine_mat = (np.matrix(trans_mat) * np.matrix(rot_mat))[0:2, :]

    # Apply the transform
    result = cv2.warpAffine(
        image,
        affine_mat,
        (new_w, new_h),
        flags=cv2.INTER_LINEAR
    )

    return result


def largest_rotated_rect(w, h, angle):
    """
    Given a rectangle of size wxh that has been rotated by 'angle' (in
    radians), computes the width and height of the largest possible
    axis-aligned rectangle within the rotated rectangle.

    Original JS code by 'Andri' and Magnus Hoff from Stack Overflow

    Converted to Python by Aaron Snoswell
    """

    quadrant = int(math.floor(angle / (math.pi / 2))) & 3
    sign_alpha = angle if ((quadrant & 1) == 0) else math.pi - angle
    alpha = (sign_alpha % math.pi + math.pi) % math.pi

    bb_w = w * math.cos(alpha) + h * math.sin(alpha)
    bb_h = w * math.sin(alpha) + h * math.cos(alpha)

    gamma = math.atan2(bb_w, bb_w) if (w < h) else math.atan2(bb_w, bb_w)

    delta = math.pi - alpha - gamma

    length = h if (w < h) else w

    d = length * math.cos(alpha)
    a = d * math.sin(alpha) / math.sin(delta)

    y = a * math.cos(gamma)
    x = y * math.tan(gamma)

    return (
        bb_w - 2 * x,
        bb_h - 2 * y
    )


def crop_around_center(image, width, height):
    """
    Given a NumPy / OpenCV 2 image, crops it to the given width and height,
    around it's centre point
    """

    image_size = (image.shape[1], image.shape[0])
    image_center = (int(image_size[0] * 0.5), int(image_size[1] * 0.5))

    if(width > image_size[0]):
        width = image_size[0]

    if(height > image_size[1]):
        height = image_size[1]

    x1 = int(image_center[0] - width * 0.5)
    x2 = int(image_center[0] + width * 0.5)
    y1 = int(image_center[1] - height * 0.5)
    y2 = int(image_center[1] + height * 0.5)

    return image[y1:y2, x1:x2]


def demo():
    """
    Demos the largest_rotated_rect function
    """

    image = cv2.imread("lenna_rectangle.png")
    image_height, image_width = image.shape[0:2]

    cv2.imshow("Original Image", image)

    print "Press [enter] to begin the demo"
    print "Press [q] or Escape to quit"

    key = cv2.waitKey(0)
    if key == ord("q") or key == 27:
        exit()

    for i in np.arange(0, 360, 0.5):
        image_orig = np.copy(image)
        image_rotated = rotate_image(image, i)
        image_rotated_cropped = crop_around_center(
            image_rotated,
            *largest_rotated_rect(
                image_width,
                image_height,
                math.radians(i)
            )
        )

        key = cv2.waitKey(2)
        if(key == ord("q") or key == 27):
            exit()

        cv2.imshow("Original Image", image_orig)
        cv2.imshow("Rotated Image", image_rotated)
        cv2.imshow("Cropped Image", image_rotated_cropped)

    print "Done"


if __name__ == "__main__":
    demo()

Image Rotation Demo

Simply place this image (cropped to demonstrate that it works with non-square images) in the same directory as the above file, then run it.

Answered By: aaronsnoswell

The math behind this solution/implementation is equivalent to this solution of an analagous question, but the formulas are simplified and avoid singularities. This is python code with the same interface as largest_rotated_rect from the other solution, but giving a bigger area in almost all cases (always the proven optimum):

def rotatedRectWithMaxArea(w, h, angle):
  """
  Given a rectangle of size wxh that has been rotated by 'angle' (in
  radians), computes the width and height of the largest possible
  axis-aligned rectangle (maximal area) within the rotated rectangle.
  """
  if w <= 0 or h <= 0:
    return 0,0

  width_is_longer = w >= h
  side_long, side_short = (w,h) if width_is_longer else (h,w)

  # since the solutions for angle, -angle and 180-angle are all the same,
  # if suffices to look at the first quadrant and the absolute values of sin,cos:
  sin_a, cos_a = abs(math.sin(angle)), abs(math.cos(angle))
  if side_short <= 2.*sin_a*cos_a*side_long or abs(sin_a-cos_a) < 1e-10:
    # half constrained case: two crop corners touch the longer side,
    #   the other two corners are on the mid-line parallel to the longer line
    x = 0.5*side_short
    wr,hr = (x/sin_a,x/cos_a) if width_is_longer else (x/cos_a,x/sin_a)
  else:
    # fully constrained case: crop touches all 4 sides
    cos_2a = cos_a*cos_a - sin_a*sin_a
    wr,hr = (w*cos_a - h*sin_a)/cos_2a, (h*cos_a - w*sin_a)/cos_2a

  return wr,hr

Here is a comparison of the function with the other solution:

>>> wl,hl = largest_rotated_rect(1500,500,math.radians(20))
>>> print (wl,hl),', area=',wl*hl
(828.2888697391496, 230.61639227890998) , area= 191016.990904
>>> wm,hm = rotatedRectWithMaxArea(1500,500,math.radians(20))
>>> print (wm,hm),', area=',wm*hm
(730.9511000407718, 266.044443118978) , area= 194465.478358

With angle angle in [0,pi/2[ the bounding box of the rotated image (width w, height h) has these dimensions:

  • width w_bb = w*cos_a + h*sin_a
  • height h_bb = w*sin_a + h*cos_a

If w_r, h_r are the computed optimal width and height of the cropped image, then the insets from the bounding box are:

  • in horizontal direction: (w_bb-w_r)/2
  • in vertical direction: (h_bb-h_r)/2

Proof:

Looking for the axis aligned rectangle between two parallel lines that has maximal area is an optimization problem with one parameter, e.g. x as in this figure:
animated parameter

Let s denote the distance between the two parallel lines (it will turn out to be the shorter side of the rotated rectangle). Then the sides a, b of the sought-after rectangle have a constant ratio with x, s-x, resp., namely x = a sin α and (s-x) = b cos α:

enter image description here

So maximizing the area a*b means maximizing x*(s-x). Because of "theorem of height" for right-angled triangles we know x*(s-x) = p*q = h*h. Hence the maximal area is reached at x = s-x = s/2, i.e. the two corners E, G between the parallel lines are on the mid-line:

enter image description here

This solution is only valid if this maximal rectangle fits into the rotated rectangle. Therefore the diagonal EG must not be longer than the other side l of the rotated rectangle. Since

EG = AF + DH = s/2*(cot α + tan α) = s/(2sin αcos α) = s/sin 2*α

we have the condition s ≤ lsin 2α, where s and l are the shorter and longer side of the rotated rectangle.

In case of s > lsin 2α the parameter x must be smaller (than s/2) and s.t. all corners of the sought-after rectangle are each on a side of the rotated rectangle. This leads to the equation

x*cot α + (s-x)*tan α = l

giving x = sin α*(lcos α – ssin α)/cos 2*α. From a = x/sin α and b = (s-x)/cos α we get the above used formulas.

Answered By: coproc

Congratulations for the great work! I wanted to use your code in OpenCV with the C++ library, so I did the conversion that follows. Maybe this approach could be helpful to other people.

#include <iostream>
#include <opencv.hpp>

#define PI 3.14159265359

using namespace std;

double degree_to_radian(double angle)
{
    return angle * PI / 180;
}

cv::Mat rotate_image (cv::Mat image, double angle)
{
    // Rotates an OpenCV 2 image about its centre by the given angle
    // (in radians). The returned image will be large enough to hold the entire
    // new image, with a black background

    cv::Size image_size = cv::Size(image.rows, image.cols);
    cv::Point image_center = cv::Point(image_size.height/2, image_size.width/2);

    // Convert the OpenCV 3x2 matrix to 3x3
    cv::Mat rot_mat = cv::getRotationMatrix2D(image_center, angle, 1.0);
    double row[3] = {0.0, 0.0, 1.0};
    cv::Mat new_row = cv::Mat(1, 3, rot_mat.type(), row);
    rot_mat.push_back(new_row);


    double slice_mat[2][2] = {
        {rot_mat.col(0).at<double>(0), rot_mat.col(1).at<double>(0)},
        {rot_mat.col(0).at<double>(1), rot_mat.col(1).at<double>(1)}
    };

    cv::Mat rot_mat_nontranslate = cv::Mat(2, 2, rot_mat.type(), slice_mat);

    double image_w2 = image_size.width * 0.5;
    double image_h2 = image_size.height * 0.5;

    // Obtain the rotated coordinates of the image corners
    std::vector<cv::Mat> rotated_coords;

    double image_dim_d_1[2] = { -image_h2, image_w2 };
    cv::Mat image_dim = cv::Mat(1, 2, rot_mat.type(), image_dim_d_1);
    rotated_coords.push_back(cv::Mat(image_dim * rot_mat_nontranslate));


    double image_dim_d_2[2] = { image_h2, image_w2 };
    image_dim = cv::Mat(1, 2, rot_mat.type(), image_dim_d_2);
    rotated_coords.push_back(cv::Mat(image_dim * rot_mat_nontranslate));


    double image_dim_d_3[2] = { -image_h2, -image_w2 };
    image_dim = cv::Mat(1, 2, rot_mat.type(), image_dim_d_3);
    rotated_coords.push_back(cv::Mat(image_dim * rot_mat_nontranslate));


    double image_dim_d_4[2] = { image_h2, -image_w2 };
    image_dim = cv::Mat(1, 2, rot_mat.type(), image_dim_d_4);
    rotated_coords.push_back(cv::Mat(image_dim * rot_mat_nontranslate));


    // Find the size of the new image
    vector<double> x_coords, x_pos, x_neg;
    for (int i = 0; i < rotated_coords.size(); i++)
    {
        double pt = rotated_coords[i].col(0).at<double>(0);
        x_coords.push_back(pt);
        if (pt > 0)
            x_pos.push_back(pt);
        else
            x_neg.push_back(pt);
    }

    vector<double> y_coords, y_pos, y_neg;
    for (int i = 0; i < rotated_coords.size(); i++)
    {
        double pt = rotated_coords[i].col(1).at<double>(0);
        y_coords.push_back(pt);
        if (pt > 0)
            y_pos.push_back(pt);
        else
            y_neg.push_back(pt);
    }


    double right_bound = *max_element(x_pos.begin(), x_pos.end());
    double left_bound = *min_element(x_neg.begin(), x_neg.end());
    double top_bound = *max_element(y_pos.begin(), y_pos.end());
    double bottom_bound = *min_element(y_neg.begin(), y_neg.end());

    int new_w = int(abs(right_bound - left_bound));
    int new_h = int(abs(top_bound - bottom_bound));

    // We require a translation matrix to keep the image centred
    double trans_mat[3][3] = {
        {1, 0, int(new_w * 0.5 - image_w2)},
        {0, 1, int(new_h * 0.5 - image_h2)},
        {0, 0, 1},
    };


    // Compute the transform for the combined rotation and translation
    cv::Mat aux_affine_mat = (cv::Mat(3, 3, rot_mat.type(), trans_mat) * rot_mat);
    cv::Mat affine_mat = cv::Mat(2, 3, rot_mat.type(), NULL);
    affine_mat.push_back(aux_affine_mat.row(0));
    affine_mat.push_back(aux_affine_mat.row(1));

    // Apply the transform
    cv::Mat output;
    cv::warpAffine(image, output, affine_mat, cv::Size(new_h, new_w), cv::INTER_LINEAR);

    return output;
}

cv::Size largest_rotated_rect(int h, int w, double angle)
{
    // Given a rectangle of size wxh that has been rotated by 'angle' (in
    // radians), computes the width and height of the largest possible
    // axis-aligned rectangle within the rotated rectangle.

    // Original JS code by 'Andri' and Magnus Hoff from Stack Overflow

    // Converted to Python by Aaron Snoswell (https://stackoverflow.com/questions/16702966/rotate-image-and-crop-out-black-borders)
    // Converted to C++ by Eliezer Bernart

    int quadrant = int(floor(angle/(PI/2))) & 3;
    double sign_alpha = ((quadrant & 1) == 0) ? angle : PI - angle;
    double alpha = fmod((fmod(sign_alpha, PI) + PI), PI);

    double bb_w = w * cos(alpha) + h * sin(alpha);
    double bb_h = w * sin(alpha) + h * cos(alpha);

    double gamma = w < h ? atan2(bb_w, bb_w) : atan2(bb_h, bb_h);

    double delta = PI - alpha - gamma;

    int length = w < h ? h : w;

    double d = length * cos(alpha);
    double a = d * sin(alpha) / sin(delta);
    double y = a * cos(gamma);
    double x = y * tan(gamma);

    return cv::Size(bb_w - 2 * x, bb_h - 2 * y);
}

// for those interested in the actual optimum - contributed by coproc
#include <algorithm>
cv::Size really_largest_rotated_rect(int h, int w, double angle)
{
  // Given a rectangle of size wxh that has been rotated by 'angle' (in
  // radians), computes the width and height of the largest possible
  // axis-aligned rectangle within the rotated rectangle.
  if (w <= 0 || h <= 0)
    return cv::Size(0,0);

  bool width_is_longer = w >= h;
  int side_long = w, side_short = h;
  if (!width_is_longer)
    std::swap(side_long, side_short);

  // since the solutions for angle, -angle and pi-angle are all the same,
  // it suffices to look at the first quadrant and the absolute values of sin,cos:
  double sin_a = fabs(sin(angle)), cos_a = fabs(cos(angle));
  double wr,hr;
  if (side_short <= 2.*sin_a*cos_a*side_long)
  {
    // half constrained case: two crop corners touch the longer side,
    // the other two corners are on the mid-line parallel to the longer line
    double x = 0.5*side_short;
    wr = x/sin_a;
    hr = x/cos_a;
    if (!width_is_longer)
      std::swap(wr,hr);
  }
  else
  { 
    // fully constrained case: crop touches all 4 sides
    double cos_2a = cos_a*cos_a - sin_a*sin_a;
    wr = (w*cos_a - h*sin_a)/cos_2a;
    hr = (h*cos_a - w*sin_a)/cos_2a;
  }

  return cv::Size(wr,hr);
}

cv::Mat crop_around_center(cv::Mat image, int height, int width)
{
    // Given a OpenCV 2 image, crops it to the given width and height,
    // around it's centre point

    cv::Size image_size = cv::Size(image.rows, image.cols);
    cv::Point image_center = cv::Point(int(image_size.height * 0.5), int(image_size.width * 0.5));

    if (width > image_size.width)
        width = image_size.width;

    if (height > image_size.height)
        height = image_size.height;

    int x1 = int(image_center.x - width  * 0.5);
    int x2 = int(image_center.x + width  * 0.5);
    int y1 = int(image_center.y - height * 0.5);
    int y2 = int(image_center.y + height * 0.5);


    return image(cv::Rect(cv::Point(y1, x1), cv::Point(y2,x2)));
}

void demo(cv::Mat image)
{
    // Demos the largest_rotated_rect function
    int image_height = image.rows;
    int image_width = image.cols;

    for (float i = 0.0; i < 360.0; i+=0.5)
    {
        cv::Mat image_orig = image.clone();
        cv::Mat image_rotated = rotate_image(image, i);

        cv::Size largest_rect = largest_rotated_rect(image_height, image_width, degree_to_radian(i));
        // for those who trust math (added by coproc):
        cv::Size largest_rect2 = really_largest_rotated_rect(image_height, image_width, degree_to_radian(i));
        cout << "area1 = " << largest_rect.height * largest_rect.width << endl;
        cout << "area2 = " << largest_rect2.height * largest_rect2.width << endl;

        cv::Mat image_rotated_cropped = crop_around_center(
                    image_rotated,
                    largest_rect.height,
                    largest_rect.width
                    );

        cv::imshow("Original Image", image_orig);
        cv::imshow("Rotated Image", image_rotated);
        cv::imshow("Cropped image", image_rotated_cropped);

        if (char(cv::waitKey(15)) == 'q')
            break;
    }

}

int main (int argc, char* argv[])
{
    cv::Mat image = cv::imread(argv[1]);

    if (image.empty())
    {
        cout << "> The input image was not found." << endl;
        exit(EXIT_FAILURE);
    }

    cout << "Press [s] to begin or restart the demo" << endl;
    cout << "Press [q] to quit" << endl;

    while (true)
    {
        cv::imshow("Original Image", image);
        char opt = char(cv::waitKey(0));
        switch (opt) {
        case 's':
            demo(image);
            break;
        case 'q':
            return EXIT_SUCCESS;
        default:
            break;
        }
    }

    return EXIT_SUCCESS;
}
Answered By: Eliezer Bernart

Correction to the most favored solution above given by Coprox on May 27 2013: when cosa = cosb infinity results in the last two lines. Solve by adding “or cosa equal cosb” in the preceding if selector.

Addition: if you do not know the original non-rotated nx and ny but only have the rotated frame (or image) then find the box just containing this (I do this by removing blank = monochrome borders) and first run the program reversely on its size to find nx and ny. If the image was rotated into a too small frame so that it was cut along the sides (into octagonal shape) I first find the x and y extensions to the full containment frame.
However, this also does not work for angles around 45 degrees where the result gets square instead of maintaining the non-rotated aspect ratio. For me this routine only works properly up to 30 degrees.

Still a great routine! It solved my nagging problem in astronomical image alignment.

Answered By: Rob Rutten

Inspired by Coprox’s amazing work I wrote a function that forms together with Coprox’s code a complete solution (so it can be used by copying & pasting with no-brainer). The rotate_max_area function below simply returns a rotated image without black boundary.

def rotate_bound(image, angle):
    # CREDIT: https://www.pyimagesearch.com/2017/01/02/rotate-images-correctly-with-opencv-and-python/
    (h, w) = image.shape[:2]
    (cX, cY) = (w // 2, h // 2)
    M = cv2.getRotationMatrix2D((cX, cY), -angle, 1.0)
    cos = np.abs(M[0, 0])
    sin = np.abs(M[0, 1])
    nW = int((h * sin) + (w * cos))
    nH = int((h * cos) + (w * sin))
    M[0, 2] += (nW / 2) - cX
    M[1, 2] += (nH / 2) - cY
    return cv2.warpAffine(image, M, (nW, nH))


def rotate_max_area(image, angle):
    """ image: cv2 image matrix object
        angle: in degree
    """
    wr, hr = rotatedRectWithMaxArea(image.shape[1], image.shape[0],
                                    math.radians(angle))
    rotated = rotate_bound(image, angle)
    h, w, _ = rotated.shape
    y1 = h//2 - int(hr/2)
    y2 = y1 + int(hr)
    x1 = w//2 - int(wr/2)
    x2 = x1 + int(wr)
    return rotated[y1:y2, x1:x2]
Answered By: Naofumi

Swift solution

Thanks to coproc for his great solution. Here is the code in swift

// Given a rectangle of size.width x size.height that has been rotated by 'angle' (in
// radians), computes the width and height of the largest possible
// axis-aligned rectangle (maximal area) within the rotated rectangle.
func rotatedRectWithMaxArea(size: CGSize, angle: CGFloat) -> CGSize {
    let w = size.width
    let h = size.height

    if(w <= 0 || h <= 0) {
        return CGSize.zero
    }

    let widthIsLonger = w >= h
    let (sideLong, sideShort) = widthIsLonger ? (w, h) : (w, h)

    // since the solutions for angle, -angle and 180-angle are all the same,
    // if suffices to look at the first quadrant and the absolute values of sin,cos:
    let (sinA, cosA) = (sin(angle), cos(angle))
    if(sideShort <= 2*sinA*cosA*sideLong || abs(sinA-cosA) < 1e-10) {
        // half constrained case: two crop corners touch the longer side,
        // the other two corners are on the mid-line parallel to the longer line
        let x = 0.5*sideShort
        let (wr, hr) = widthIsLonger ? (x/sinA, x/cosA) : (x/cosA, x/sinA)
        return CGSize(width: wr, height: hr)
    } else {
        // fully constrained case: crop touches all 4 sides
        let cos2A = cosA*cosA - sinA*sinA
        let (wr, hr) = ((w*cosA - h*sinA)/cos2A, (h*cosA - w*sinA)/cos2A)
        return CGSize(width: wr, height: hr)
    }
}
Answered By: Josh Bernfeld

Rotation and cropping in TensorFlow

I personally needed this function in TensorFlow and thanks for Aaron Snoswell, I could implement this function.

def _rotate_and_crop(image, output_height, output_width, rotation_degree, do_crop):
    """Rotate the given image with the given rotation degree and crop for the black edges if necessary
    Args:
        image: A `Tensor` representing an image of arbitrary size.
        output_height: The height of the image after preprocessing.
        output_width: The width of the image after preprocessing.
        rotation_degree: The degree of rotation on the image.
        do_crop: Do cropping if it is True.
    Returns:
        A rotated image.
    """

    # Rotate the given image with the given rotation degree
    if rotation_degree != 0:
        image = tf.contrib.image.rotate(image, math.radians(rotation_degree), interpolation='BILINEAR')

        # Center crop to ommit black noise on the edges
        if do_crop == True:
            lrr_width, lrr_height = _largest_rotated_rect(output_height, output_width, math.radians(rotation_degree))
            resized_image = tf.image.central_crop(image, float(lrr_height)/output_height)    
            image = tf.image.resize_images(resized_image, [output_height, output_width], method=tf.image.ResizeMethod.BILINEAR, align_corners=False)

    return image

def _largest_rotated_rect(w, h, angle):
    """
    Given a rectangle of size wxh that has been rotated by 'angle' (in
    radians), computes the width and height of the largest possible
    axis-aligned rectangle within the rotated rectangle.
    Original JS code by 'Andri' and Magnus Hoff from Stack Overflow
    Converted to Python by Aaron Snoswell
    Source: http://stackoverflow.com/questions/16702966/rotate-image-and-crop-out-black-borders
    """

    quadrant = int(math.floor(angle / (math.pi / 2))) & 3
    sign_alpha = angle if ((quadrant & 1) == 0) else math.pi - angle
    alpha = (sign_alpha % math.pi + math.pi) % math.pi

    bb_w = w * math.cos(alpha) + h * math.sin(alpha)
    bb_h = w * math.sin(alpha) + h * math.cos(alpha)

    gamma = math.atan2(bb_w, bb_w) if (w < h) else math.atan2(bb_w, bb_w)

    delta = math.pi - alpha - gamma

    length = h if (w < h) else w

    d = length * math.cos(alpha)
    a = d * math.sin(alpha) / math.sin(delta)

    y = a * math.cos(gamma)
    x = y * math.tan(gamma)

    return (
        bb_w - 2 * x,
        bb_h - 2 * y
    )

If you need further implementation of example and visualization in TensorFlow, you can use this repository.
I hope this could be helpful to other people.

Answered By: ByungSoo Ko

A small update for brevity that makes use of the excellent imutils library.

def rotated_rect(w, h, angle):
    """
    Given a rectangle of size wxh that has been rotated by 'angle' (in
    radians), computes the width and height of the largest possible
    axis-aligned rectangle within the rotated rectangle.

    Original JS code by 'Andri' and Magnus Hoff from Stack Overflow

    Converted to Python by Aaron Snoswell
    """
    angle = math.radians(angle)
    quadrant = int(math.floor(angle / (math.pi / 2))) & 3
    sign_alpha = angle if ((quadrant & 1) == 0) else math.pi - angle
    alpha = (sign_alpha % math.pi + math.pi) % math.pi

    bb_w = w * math.cos(alpha) + h * math.sin(alpha)
    bb_h = w * math.sin(alpha) + h * math.cos(alpha)

    gamma = math.atan2(bb_w, bb_w) if (w < h) else math.atan2(bb_w, bb_w)

    delta = math.pi - alpha - gamma

    length = h if (w < h) else w

    d = length * math.cos(alpha)
    a = d * math.sin(alpha) / math.sin(delta)

    y = a * math.cos(gamma)
    x = y * math.tan(gamma)

    return (bb_w - 2 * x, bb_h - 2 * y)

def crop(img, w, h):
    x, y = int(img.shape[1] * .5), int(img.shape[0] * .5)

    return img[
        int(np.ceil(y - h * .5)) : int(np.floor(y + h * .5)),
        int(np.ceil(x - w * .5)) : int(np.floor(x + h * .5))
    ]

def rotate(img, angle):
    # rotate, crop and return original size
    (h, w) = img.shape[:2]
    img = imutils.rotate_bound(img, angle)
    img = crop(img, *rotated_rect(w, h, angle))
    img = cv2.resize(img,(w,h),interpolation=cv2.INTER_AREA)
    return img
Answered By: Artemi Krymski

Perhaps an even simplier solution would be:

def crop_image(image, angle):
    h, w = image.shape
    tan_a = abs(np.tan(angle * np.pi / 180))
    b = int(tan_a / (1 - tan_a ** 2) * (h - w * tan_a))
    d = int(tan_a / (1 - tan_a ** 2) * (w - h * tan_a))
    return image[d:h - d, b:w - b]

Instead of calculating the height and width of the rotated rectangle like many have done, it is sufficient to find the height of the black triangles that form when rotating an image.

Answered By: Federico Corazza

Rotate images in correct order

 import cv2
 import pytesseract
 import urllib
 import numpy as np
 import re
 import imutils #added
 import PIL

 image = cv2.imread('my_pdf_madan_m/page_1.jpg')

 gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
 gray = cv2.bitwise_not(gray)

 rot_data = pytesseract.image_to_osd(image);
 print("[OSD] "+rot_data)
 rot = re.search('(?<=Rotate: )d+', 
 rot_data).group(0)

 angle = float(rot)

 # rotate the image to deskew it
 rotated = imutils.rotate_bound(image, angle) #added


 #  TODO: Rotated image can be saved here
 print(pytesseract.image_to_osd(rotated));

 # Run tesseract OCR on image
 text = pytesseract.image_to_string(rotated, 
 lang='eng', config="--psm 6")
 print(text)
Answered By: madan maram

Recently implemented a solution for Pytorch. It might come in handy. Could potentially be used with the ‘Random Rotation Transform’ as well. Just need to read the particular angle used by the transform and then just use it with PyTorch transforms. Function simply takes in a batch of images and does the random rotation with cropping.

import torchvision.transforms as transforms
import math

def _largest_rotated_rect(w, h, angle):
"""
Given a rectangle of size wxh that has been rotated by 'angle' (in
radians), computes the width and height of the largest possible
axis-aligned rectangle within the rotated rectangle.
Original JS code by 'Andri' and Magnus Hoff from Stack Overflow
Converted to Python by Aaron Snoswell
Source: http://stackoverflow.com/questions/16702966/rotate-image-and-crop-out-black-borders
"""

quadrant = int(math.floor(angle / (math.pi / 2))) & 3
sign_alpha = angle if ((quadrant & 1) == 0) else math.pi - angle
alpha = (sign_alpha % math.pi + math.pi) % math.pi

bb_w = w * math.cos(alpha) + h * math.sin(alpha)
bb_h = w * math.sin(alpha) + h * math.cos(alpha)

gamma = math.atan2(bb_w, bb_w) if (w < h) else math.atan2(bb_w, bb_w)

delta = math.pi - alpha - gamma

length = h if (w < h) else w

d = length * math.cos(alpha)
a = d * math.sin(alpha) / math.sin(delta)

y = a * math.cos(gamma)
x = y * math.tan(gamma)

return (
    bb_w - 2 * x,
    bb_h - 2 * y
)


def _rotate_and_crop(image, output_height=32, output_width=32):
"""Rotate the given image with the given rotation degree and crop for the black edges if necessary. For my case, image sizes are 32x32.
Args:
    image: A Batch of Tensors- normally from a dataloader.
    output_height: The height of the image after preprocessing.
    output_width: The width of the image after preprocessing.
Returns:
    A rotated image.
"""

# Rotate the given image with the given rotation degree
rotation_transform = transforms.RandomRotation((0, 360))
angle_rot = rotation_transform.angle_rot #you will have to read it from the pytorch library

lrr_width, lrr_height = _largest_rotated_rect(output_height, output_width, math.radians(angle_rot))
croped_image = transforms.CenterCrop((lrr_height, lrr_width))
resize_transform = transforms.Resize(size=(output_height, output_width))

transform = transforms.Compose([rotation_transform, croped_image, resize_transform, transforms.RandomHorizontalFlip(),
                                               transforms.ToTensor(),
                                               ])

image = transform(image)

return image
Answered By: jmm

By doing the calculations by hand and looking at the original post, I found a minor typo on the gamma calculations. It should actually be:

gamma = math.atan2(bb_w, bb_h) if (w < h) else math.atan2(bb_h, bb_w)

Answered By: user18242760
Categories: questions Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.