The loop was iterating over all the datafiles and maintaining a set to check
whether filelogs have been processed, manifests have been processed or not.
The code was hard to understand and it assumed that alldatafiles are ordered
in a certain way.
This refactors the for loop in separate parts for each manifests, changelog and
filelogs. This will also help in future work where we will like more better
handling on whether we want to upgrade filelogs or not.