linelog: fix infinite loop vulnerability

Checking len(lines) is not a great way of detecting infinite loops, as
demonstrated in the added test. Therefore check instruction count instead.

The original C implementation does not have this problem. There are a few
other places where the C implementation enforces more strictly, like
a1 <= a2, b1 <= b2, rev > 0. But they are optional.

Test Plan:
Add a test. The old code forces the test to time out.

Differential Revision: https://phab.mercurial-scm.org/D4151