@tareque_md The count should be one, right, but the check tests not only the count, but also what those 4 points are exactly. It may happen that there are 4 points remaining, but they are wrong points, not the corners of the bounding convex hull.
It fails on this test: [[0,0,1,1],[0,0,2,1],[1,0,2,1],[0,2,2,3]]. Graphically, it looks something like this:
Here, 22 corresponds to three rectangles, one 2x1 and two 1x1 on top of it, thus creating two layers of overlap. This overlap creates even number of corners, and therefore, the bottom rectangles are eliminated. The top is just a single 2x1 rectangle, so it remains in the hash set, leaving there exactly 4 corners. Only they are wrong corners.
The trick is that we don't need to store everything in the middle. Storing the step is enough as everything in the middle will be left + step * k. From the left, right and the step, we can calculate them in the next iteration.