This is an archive of the discontinued Mercurial Phabricator instance.

Differential D1622

[PoC] transaction: Use intbitset for implementing changes['phase']
AbandonedPublic

Authored by joerg.sonnenberger on Dec 8 2017, 7:06 PM.

Details

Reviewers
yuja
Group Reviewers
hg-reviewers
Summary

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

quark added a subscriber: quark.Dec 8 2017, 7:31 PM

Although this is definitely better for the clone case. intbitset does not seem to implement segmented bitsets so the memory usage could in theory be worse for other cases.

mercurial/localrepo.py
1283

It's still worthwhile to have set() as a fallback. So the code runs in pure python or pypy environment

yuja requested changes to this revision.Dec 9 2017, 8:18 AM
yuja added a subscriber: yuja.

Although this is definitely better for the clone case. intbitset does not seem to implement segmented bitsets so the memory usage could in theory be worse for other cases.

Perhaps. And if we need to iterate a sparse bitset, the performance would
be way worse.

This revision now requires changes to proceed.Dec 9 2017, 8:18 AM
joerg.sonnenberger added a comment.Dec 9 2017, 10:15 AM

Like I said, this is primarily a proof of concept. I'm still playing with different approaches and I don't think intbitset is a good fit for other reasons, but it worked the best from the various pre-existing implementations.

yuja retitled this revision from transaction: Use intbitset for implementing changes['phase'] to [PoC] transaction: Use intbitset for implementing changes['phase'].Dec 10 2017, 6:04 AM
yuja added a comment.Dec 10 2017, 6:08 AM
In D1622#27981, @joerg.sonnenberger wrote:

Like I said, this is primarily a proof of concept. I'm still playing with different approaches

Ah, okay. I've added [PoC] to the subject so this won't be queued by mistake.

FWIW, we generally don't move codes to C unless there is significant improvement.

joerg.sonnenberger abandoned this revision.Feb 18 2020, 10:02 AM

Obsoleted by range based representation, which often works even better and doesn't depend on C.

Revision Contents
Changeset List

PathPackages
Mmercurial/localrepo.py (4 lines)

Diff 4279

mercurial/localrepo.py

import errno import errno
import hashlib import hashlib
import inspect import inspect
import os import os
import random import random
import time import time
import weakref import weakref
from intbitset import intbitset
from .i18n import _ from .i18n import _
from .node import ( from .node import (
hex, hex,
nullid, nullid,
short, short,
) )
from . import ( from . import (
bookmarks, bookmarks,
"undo", "undo",
aftertrans(renames), aftertrans(renames),
self.store.createmode, self.store.createmode,
validator=validate, validator=validate,
releasefn=releasefn, releasefn=releasefn,
checkambigfiles=_cachedfiles) checkambigfiles=_cachedfiles)
tr.changes['revs'] = xrange(0, 0) tr.changes['revs'] = xrange(0, 0)
tr.changes['obsmarkers'] = set() tr.changes['obsmarkers'] = set()
tr.changes['phases'] = [set() for i in range(7)] tr.changes['phases'] = [intbitset() for i in range(6)]
quarkUnsubmitted
Not Done

It's still worthwhile to have set() as a fallback. So the code runs in pure python or pypy environment

quark: It's still worthwhile to have `set()` as a fallback. So the code runs in pure python or pypy…
tr.changes['bookmarks'] = {} tr.changes['bookmarks'] = {}
tr.hookargs['txnid'] = txnid tr.hookargs['txnid'] = txnid
# note: writing the fncache only during finalize mean that the file is # note: writing the fncache only during finalize mean that the file is
# outdated when running hooks. As fncache is used for streaming clone, # outdated when running hooks. As fncache is used for streaming clone,
# this is not expected to break anything that happen during the hooks. # this is not expected to break anything that happen during the hooks.
tr.addfinalize('flush-fncache', self.store.write) tr.addfinalize('flush-fncache', self.store.write)
def txnclosehook(tr2): def txnclosehook(tr2):