rebase: fix issue 5494 also with --collapse

Authored by martinvonz.