Introduction

Yes, OpenCV can help you detect rectangles from corner information, but corners alone are rarely enough for a robust detector. In practice, the most reliable pipeline uses edges, contours, polygon approximation, and geometric validation, with corner detection acting as a supporting signal rather than the only test.

Start With Preprocessing and Contours

A common rectangle detector begins by simplifying the image, extracting edges, and finding contours.

1import cv2
2
3img = cv2.imread("input.jpg")
4gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
5blur = cv2.GaussianBlur(gray, (5, 5), 0)
6edges = cv2.Canny(blur, 50, 150)
7
8contours, _ = cv2.findContours(edges, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)

This works better than relying on raw corners because contours preserve larger shape information rather than isolated point evidence.

Approximate Contours to Quadrilaterals

Once you have contours, reduce them to polygons and keep convex shapes with four vertices.

1candidates = []
2for cnt in contours:
3    perimeter = cv2.arcLength(cnt, True)
4    approx = cv2.approxPolyDP(cnt, 0.02 * perimeter, True)
5
6    if len(approx) == 4 and cv2.isContourConvex(approx):
7        area = cv2.contourArea(approx)
8        if area > 1000:
9            candidates.append(approx)

At this stage you have quadrilateral candidates, not guaranteed rectangles. A trapezoid or an arbitrary convex four-sided shape can still pass this filter.

Validate Rectangle Geometry

To decide whether a quadrilateral is rectangle-like, inspect its angles. A rectangle should have four near-right angles.

1import numpy as np
2
3
4def angle_cos(p0, p1, p2):
5    d1 = (p0 - p1).astype(np.float32)
6    d2 = (p2 - p1).astype(np.float32)
7    denom = np.linalg.norm(d1) * np.linalg.norm(d2) + 1e-8
8    return abs(np.dot(d1, d2) / denom)
9
10
11validated = []
12for quad in candidates:
13    pts = quad.reshape(4, 2)
14    cosines = []
15    for i in range(4):
16        p0 = pts[i]
17        p1 = pts[(i + 1) % 4]
18        p2 = pts[(i + 2) % 4]
19        cosines.append(angle_cos(p0, p1, p2))
20
21    if max(cosines) < 0.3:
22        validated.append(quad)

Smaller cosine values mean the angles are closer to ninety degrees. This removes many non-rectangular quadrilaterals.

Where Corner Detection Still Helps

Corner detectors such as Shi-Tomasi can still be useful, but usually as a support tool rather than the main detector.

1corners = cv2.goodFeaturesToTrack(gray, maxCorners=200, qualityLevel=0.01, minDistance=10)
2
3if corners is not None:
4    for corner in corners:
5        x, y = corner.ravel()
6        cv2.circle(img, (int(x), int(y)), 2, (0, 0, 255), -1)

This is useful for debugging, for visualizing likely candidate points, or for later perspective correction steps.

Rotated and Skewed Rectangles

If the rectangle is rotated or seen in perspective, minAreaRect can still provide a rotated bounding box estimate.

1for cnt in contours:
2    rect = cv2.minAreaRect(cnt)
3    box = cv2.boxPoints(rect)
4    box = np.int32(box)
5    cv2.polylines(img, [box], True, (255, 0, 0), 1)

This is especially helpful in document scanning, tabletop scenes, or camera feeds where the target is not axis-aligned.

The Image Quality Problem

Detection quality often depends more on the input image than on the final geometric tests. Better contrast, less background clutter, and cleaner edges can improve rectangle detection dramatically before you change a single threshold.

That is why preprocessing is not just a preliminary step. It is often half of the detector.

Common Pitfalls

A common mistake is treating any four-corner polygon as a rectangle without angle validation. Four corners only prove that the contour is a quadrilateral.

Another issue is using fixed thresholds for every image regardless of lighting, blur, and scale. Real scenes usually need some tuning.

Developers also often rely on corner detectors alone when contour-based shape information would be much stronger.

Summary

• Rectangle detection from corners is possible, but corners alone are rarely robust enough.
• A practical OpenCV pipeline uses preprocessing, contours, polygon approximation, and angle checks.
• Four vertices give you a quadrilateral candidate, not proof of a rectangle.
• Corner detectors are useful supporting tools for visualization and refinement.
• Rotated cases often benefit from minAreaRect or other geometry-aware post-processing.