Sorting polygon's points
Data Structures & Algorithms practice on Codemia
Step through 300 algorithm problems with animated visualisers that show the data structure changing as the code runs.
Introduction
Sorting a polygon's points means placing its vertices into a consistent clockwise or counterclockwise order so edges can be drawn correctly. The right algorithm depends on what you actually have, because sorting arbitrary 2D points is not the same as reconstructing an unknown polygon.
The first question: what kind of input is this
There are two very different problems that people often mix together:
- you already have the vertices of one polygon, just not in order
- you have an arbitrary point set and want to infer a polygon boundary
The first problem can be straightforward for convex polygons. The second problem may be ambiguous or much harder, especially for concave shapes.
Convex polygons: sort by angle around the centroid
If the points are known to be the vertices of a convex polygon, a common solution is:
- compute the centroid
- compute each point's angle relative to the centroid with
atan2 - sort by that angle
This is simple and works well when the polygon is convex because the centroid lies inside the shape and the radial ordering matches the boundary traversal.
Why angle sorting is not universal
For concave polygons, sorting by angle around the centroid can produce the wrong boundary or even self-intersecting edges. That happens because angular order only captures radial position from one reference point, not actual adjacency along the polygon boundary.
So this pattern is safe when you know the polygon is convex, but it is not a general "sort any polygon points" algorithm.
If the points define a concave polygon
If you truly need to recover the order of a concave polygon from an unordered set of boundary vertices, the problem is more geometric than sorting-based. You may need one of these approaches:
- domain knowledge about neighboring vertices
- an existing edge list
- a triangulation or planar graph reconstruction step
- computing a convex hull first, then placing interior boundary points carefully
Without extra structure, multiple valid simple polygons may pass through the same point set. That means there may not be one uniquely correct ordering to recover.
Choosing clockwise versus counterclockwise
Once you have an ordered polygon, it is often useful to normalize the orientation.
A standard method is to compute the signed area. If the signed area is positive, the points are in one orientation; if negative, they are in the other.
That is useful when downstream code expects a consistent orientation.
When a convex hull is the real goal
Sometimes the points are not really the vertices of one polygon at all. If the goal is to wrap the outer boundary of a point cloud, you may actually want a convex hull rather than point sorting.
In that case, algorithms such as Graham scan or Andrew's monotonic chain are a better fit than angle sorting alone.
Common Pitfalls
The biggest mistake is assuming that sorting by angle around the centroid solves polygon ordering in every case. It does not.
Another issue is ignoring whether the input is convex, concave, or even a valid polygon boundary at all. Geometry algorithms depend heavily on those assumptions.
It is also easy to forget that unordered points may describe several possible simple polygons. If the original edge structure is unknown, the problem may be underdetermined.
Finally, once the points are ordered, make sure the orientation matches what your renderer or geometry code expects. Clockwise and counterclockwise conventions matter in many systems.
Summary
- Sorting polygon points is easy only under the right assumptions.
- For convex polygons, centroid plus
atan2sorting is a practical solution. - For concave polygons, angle sorting can fail or create self-intersections.
- If you only have an arbitrary point set, reconstructing the boundary may require more than sorting.
- Normalize orientation after ordering if later code expects clockwise or counterclockwise vertices.
Related reading
- Sorting result array
- Sorting strings containing numbers in a user friendly way
- Sorting strings so that hamming distance is low between adjacent strings
- Sorting with stochastic comparisions
- Spark Word2vec vector mathematics
- Spatial data structure for finding all points greater than or less than a value in each cartesian dimension
- Sorting zipped locked containers in C using boost or the STL
- Source code for Xiaolin Wu's line algorithm in C?

DSA Fundamentals
Master algorithmic patterns and data structures through hands-on LeetCode-style problems - from arrays and hashing to dynamic programming and advanced graphs.
View the courseTrack what you have practised
A free account saves your progress, solutions and study plan across every problem on Codemia.
Data Structures & Algorithms practice on Codemia
Step through 300 algorithm problems with animated visualisers that show the data structure changing as the code runs.