Page MenuHomePhabricator

marmoute (Pierre-Yves David)
User

Projects

User does not belong to any projects.

User Details

User Since
Jun 29 2017, 2:56 PM (133 w, 4 d)

Recent Activity

Fri, Jan 17

marmoute updated the diff for D7894: nodemap: introduce an option to use mmap to read the nodemap mapping.
Fri, Jan 17, 1:12 PM
marmoute updated the diff for D7888: nodemap: track the maximum revision tracked in the nodemap.
Fri, Jan 17, 1:11 PM
marmoute updated the diff for D7889: nodemap: track the total and unused amount of data in the rawdata file.
Fri, Jan 17, 1:10 PM
marmoute updated the diff for D7887: nodemap: add a flag to dump the details of the docket.
Fri, Jan 17, 1:10 PM
marmoute updated the diff for D7885: nodemap: keep track of the docket for loaded data.
Fri, Jan 17, 1:10 PM
marmoute updated the diff for D7845: nodemap: add basic checking of the on disk nodemap content.
Fri, Jan 17, 1:09 PM
marmoute updated the diff for D7840: nodemap: add a (python) index class for persistent nodemap testing.
Fri, Jan 17, 1:09 PM
marmoute updated the diff for D7847: nodemap: provide the on disk data to indexes who support it.
Fri, Jan 17, 1:08 PM
marmoute updated the diff for D7835: nodemap: write nodemap data on disk.
Fri, Jan 17, 1:07 PM
marmoute updated the diff for D7834: nodemap: have some python code serializing a nodemap.
Fri, Jan 17, 1:07 PM
marmoute updated the diff for D7836: nodemap: add a function to read the data from disk.
Fri, Jan 17, 1:07 PM
marmoute updated the diff for D7837: nodemap: only use persistent nodemap for non-inlined revlog.
Fri, Jan 17, 1:06 PM
marmoute added a comment to D7834: nodemap: have some python code serializing a nodemap.

As stated yesterday, I'll have limited screen exposure time available for the next couple of day. I am going to ignore any minor nits during that time. (Am I happy to look at them afterward and follow up).

Fri, Jan 17, 1:05 PM
marmoute added a comment to D7894: nodemap: introduce an option to use mmap to read the nodemap mapping.

How much does this patch help performance?
I would also like to see performance numbers (even just rough ones) for the Rust version. Sorry about a possibly stupid question, but why will this on-disk nodemap be faster than building it from the index? Is it that the file is smaller and thus faster to read? Or is it more the building of the tree than the reading that's slow? You mentioned you use some private repo for testing this. How large is the 00changelog.n file in that repo and how large is 00changelog.i?

This save hundreds of milli second at initialization of large repositories. The repositories we are looking at are about 2 millions revisions. (but this will help smaller repository too). Mozilla try is a public repository in that range. It 00changelog.i is 103MB

And 00changelog.n?

Fri, Jan 17, 11:30 AM
marmoute added a comment to D7834: nodemap: have some python code serializing a nodemap.

Can you clarify which change you are actually requesting (I see mostly nits).

Fri, Jan 17, 10:39 AM
marmoute closed D7492: localrepo: also fastpath access to working copy parents when possible.
Fri, Jan 17, 10:16 AM
marmoute closed D7502: changectx: mark parent of changesets as non filtered.
Fri, Jan 17, 10:15 AM
marmoute closed D7501: changectx: use unfiltered changelog to walk ancestors in annotate.
Fri, Jan 17, 10:15 AM
marmoute committed rHG98349eddceef: changectx: mark parent of changesets as non filtered.
changectx: mark parent of changesets as non filtered
Fri, Jan 17, 10:15 AM
marmoute committed rHGc472970339d2: changectx: use unfiltered changelog to walk ancestors in annotate.
changectx: use unfiltered changelog to walk ancestors in annotate
Fri, Jan 17, 10:15 AM
marmoute closed D7495: localrepo: recognize trivial request for '.'.
Fri, Jan 17, 10:15 AM
marmoute closed D7498: localrepo: also fast past the parents of working copies parents.
Fri, Jan 17, 10:15 AM
marmoute committed rHG5a012404503b: localrepo: also fast past the parents of working copies parents.
localrepo: also fast past the parents of working copies parents
Fri, Jan 17, 10:14 AM
marmoute closed D7494: localrepo: fastpath access to ".".
Fri, Jan 17, 10:14 AM
marmoute committed rHGbfaf4c673bec: localrepo: recognize trivial request for '.'.
localrepo: recognize trivial request for '.'
Fri, Jan 17, 10:13 AM
marmoute committed rHGd86dede17392: localrepo: fastpath access to ".".
localrepo: fastpath access to "."
Fri, Jan 17, 10:13 AM
marmoute committed rHG85c4cd73996b: localrepo: also fastpath access to working copy parents when possible.
localrepo: also fastpath access to working copy parents when possible
Fri, Jan 17, 10:13 AM
marmoute added a comment to D7492: localrepo: also fastpath access to working copy parents when possible.

Thank you, much appreciated.

Fri, Jan 17, 10:00 AM
marmoute updated subscribers of D7894: nodemap: introduce an option to use mmap to read the nodemap mapping.

How much does this patch help performance?
I would also like to see performance numbers (even just rough ones) for the Rust version. Sorry about a possibly stupid question, but why will this on-disk nodemap be faster than building it from the index? Is it that the file is smaller and thus faster to read? Or is it more the building of the tree than the reading that's slow? You mentioned you use some private repo for testing this. How large is the 00changelog.n file in that repo and how large is 00changelog.i?

Fri, Jan 17, 5:25 AM
marmoute added inline comments to D7886: nodemap: introduce append-only incremental update of the persisten data.
Fri, Jan 17, 5:19 AM
marmoute added inline comments to D7834: nodemap: have some python code serializing a nodemap.
Fri, Jan 17, 5:18 AM

Thu, Jan 16

marmoute added inline comments to D7834: nodemap: have some python code serializing a nodemap.
Thu, Jan 16, 4:52 PM
marmoute closed D7833: transaction: allow finalizer to add finalizer.
Thu, Jan 16, 2:20 PM
marmoute committed rHG2f1d6180737f: transaction: allow finalizer to add finalizer.
transaction: allow finalizer to add finalizer
Thu, Jan 16, 2:20 PM
marmoute added a comment to D7492: localrepo: also fastpath access to working copy parents when possible.

Here is raw perf comparison for status, diff and export. We don't have a benchmark for annottate yet. (not too long to write, but I did not had the time to do that yet)

Thu, Jan 16, 5:24 AM

Wed, Jan 15

marmoute retitled D7890: nodemap: double check the source docket when doing incremental update from nodemapdata: double check the source docket when doing incremental update to nodemap: double check the source docket when doing incremental update.
Wed, Jan 15, 1:09 PM
marmoute created D7894: nodemap: introduce an option to use mmap to read the nodemap mapping.
Wed, Jan 15, 9:59 AM
marmoute created D7888: nodemap: track the maximum revision tracked in the nodemap.
Wed, Jan 15, 9:58 AM
marmoute created D7893: nodemap: update the index with the newly written data (when appropriate).
Wed, Jan 15, 9:58 AM
marmoute created D7892: nodemap: never read more than the expected data amount.
Wed, Jan 15, 9:58 AM
marmoute created D7889: nodemap: track the total and unused amount of data in the rawdata file.
Wed, Jan 15, 9:58 AM
marmoute created D7891: nodemap: write new data from the expected current data length.
Wed, Jan 15, 9:58 AM
marmoute created D7885: nodemap: keep track of the docket for loaded data.
Wed, Jan 15, 9:58 AM
marmoute created D7890: nodemap: double check the source docket when doing incremental update.
Wed, Jan 15, 9:58 AM
marmoute created D7887: nodemap: add a flag to dump the details of the docket.
Wed, Jan 15, 9:57 AM
marmoute updated the diff for D7845: nodemap: add basic checking of the on disk nodemap content.
Wed, Jan 15, 9:56 AM
marmoute created D7886: nodemap: introduce append-only incremental update of the persisten data.
Wed, Jan 15, 9:56 AM
marmoute created D7884: nodemap: introduce an explicit class/object for the docket.
Wed, Jan 15, 9:55 AM
marmoute created D7883: nodemap: keep track of the ondisk id of nodemap blocks.
Wed, Jan 15, 9:55 AM
marmoute updated the diff for D7846: nodemap: all check that revision and nodes match in the nodemap.
Wed, Jan 15, 9:55 AM
marmoute updated the diff for D7844: nodemap: code to parse a serialized nodemap.
Wed, Jan 15, 9:54 AM
marmoute updated the diff for D7843: nodemap: move the iteratio inside the Block object.
Wed, Jan 15, 9:54 AM
marmoute updated the diff for D7842: nodemap: use an explicit "Block" object in the reference implementation.
Wed, Jan 15, 9:54 AM
marmoute updated the diff for D7841: nodemap: add a optional `nodemap_add_full` method on indexes.
Wed, Jan 15, 9:54 AM
marmoute updated the diff for D7839: nodemap: deleted older raw data file when creating a new ones.
Wed, Jan 15, 9:54 AM

Mon, Jan 13

marmoute updated the diff for D7840: nodemap: add a (python) index class for persistent nodemap testing.
Mon, Jan 13, 1:06 PM
marmoute updated the diff for D7835: nodemap: write nodemap data on disk.
Mon, Jan 13, 1:06 PM
marmoute retitled D7847: nodemap: provide the on disk data to indexes who support it from nodemap: provide the on disk data to the index who support it to nodemap: provide the on disk data to indexes who support it.
Mon, Jan 13, 10:36 AM
marmoute created D7840: nodemap: add a (python) index class for persistent nodemap testing.
Mon, Jan 13, 10:10 AM
marmoute created D7845: nodemap: add basic checking of the on disk nodemap content.
Mon, Jan 13, 10:10 AM
marmoute created D7847: nodemap: provide the on disk data to indexes who support it.
Mon, Jan 13, 10:09 AM
marmoute updated the diff for D7835: nodemap: write nodemap data on disk.
Mon, Jan 13, 10:09 AM
marmoute created D7846: nodemap: all check that revision and nodes match in the nodemap.
Mon, Jan 13, 10:08 AM
marmoute created D7844: nodemap: code to parse a serialized nodemap.
Mon, Jan 13, 10:08 AM
marmoute created D7843: nodemap: move the iteratio inside the Block object.
Mon, Jan 13, 10:08 AM
marmoute updated the diff for D7834: nodemap: have some python code serializing a nodemap.
Mon, Jan 13, 10:08 AM
marmoute created D7842: nodemap: use an explicit "Block" object in the reference implementation.
Mon, Jan 13, 10:08 AM
marmoute created D7841: nodemap: add a optional `nodemap_add_full` method on indexes.
Mon, Jan 13, 10:08 AM
marmoute updated the diff for D7836: nodemap: add a function to read the data from disk.
Mon, Jan 13, 10:08 AM
marmoute updated the summary of D7839: nodemap: deleted older raw data file when creating a new ones.
Mon, Jan 13, 10:08 AM
marmoute updated the summary of D7838: nodemap: use an intermediate "docket" file to carry small metadata.
Mon, Jan 13, 10:07 AM
marmoute updated the diff for D7837: nodemap: only use persistent nodemap for non-inlined revlog.
Mon, Jan 13, 10:07 AM
marmoute updated the diff for D7830: revlog: reorder a conditionnal about revlogio.
Mon, Jan 13, 10:06 AM
marmoute updated the diff for D7832: transaction: add a `hasfinalize` method.
Mon, Jan 13, 10:06 AM
marmoute updated the diff for D7831: changelog: fix the diverted opener to accept more kwargs.
Mon, Jan 13, 10:05 AM
marmoute closed D7830: revlog: reorder a conditionnal about revlogio.
Mon, Jan 13, 9:20 AM
marmoute closed D7832: transaction: add a `hasfinalize` method.
Mon, Jan 13, 9:19 AM
marmoute committed rHG8e09551206f5: transaction: add a `hasfinalize` method.
transaction: add a `hasfinalize` method
Mon, Jan 13, 9:19 AM
marmoute committed rHG7f67f53492f7: changelog: fix the diverted opener to accept more kwargs.
changelog: fix the diverted opener to accept more kwargs
Mon, Jan 13, 9:19 AM
marmoute closed D7831: changelog: fix the diverted opener to accept more kwargs.
Mon, Jan 13, 9:19 AM
marmoute committed rHG612225e994ff: revlog: reorder a conditionnal about revlogio.
revlog: reorder a conditionnal about revlogio
Mon, Jan 13, 9:19 AM

Sat, Jan 11

marmoute created D7835: nodemap: write nodemap data on disk.
Sat, Jan 11, 12:05 PM
marmoute created D7834: nodemap: have some python code serializing a nodemap.
Sat, Jan 11, 12:05 PM
marmoute created D7836: nodemap: add a function to read the data from disk.
Sat, Jan 11, 12:04 PM
marmoute created D7839: nodemap: deleted older raw data file when creating a new ones.
Sat, Jan 11, 12:04 PM
marmoute created D7838: nodemap: use an intermediate "docket" file to carry small metadata.
Sat, Jan 11, 12:04 PM
marmoute created D7837: nodemap: only use persistent nodemap for non-inlined revlog.
Sat, Jan 11, 12:04 PM
marmoute created D7830: revlog: reorder a conditionnal about revlogio.
Sat, Jan 11, 12:02 PM
marmoute created D7833: transaction: allow finalizer to add finalizer.
Sat, Jan 11, 12:02 PM
marmoute created D7832: transaction: add a `hasfinalize` method.
Sat, Jan 11, 12:02 PM
marmoute created D7831: changelog: fix the diverted opener to accept more kwargs.
Sat, Jan 11, 12:02 PM

Thu, Jan 9

marmoute added a comment to D7809: revlogutils: move the NodeMap class in a dedicated nodemap module.

utils is older than revlogutils, revlogutils focus on revlog related changes and his worth keeping at this.

Thu, Jan 9, 1:03 PM
marmoute retitled D7807: hgrc: introduce HGRCSKIPREPO to skip reading the repository's hgrc from hgrc: introduce HGRCSKIPREPO to skip reading the hgrc's repository to hgrc: introduce HGRCSKIPREPO to skip reading the repository's hgrc.
Thu, Jan 9, 3:46 AM

Wed, Jan 8

marmoute closed D7808: mmap: add a size argument to mmapread.
Wed, Jan 8, 3:04 PM
marmoute closed D7809: revlogutils: move the NodeMap class in a dedicated nodemap module.
Wed, Jan 8, 3:04 PM
marmoute committed rHG8ed8dfbeabb9: mmap: add a size argument to mmapread.
mmap: add a size argument to mmapread
Wed, Jan 8, 3:04 PM
marmoute committed rHGab595920de0e: revlogutils: move the NodeMap class in a dedicated nodemap module.
revlogutils: move the NodeMap class in a dedicated nodemap module
Wed, Jan 8, 3:04 PM
marmoute closed D7657: rust-index: handle `MixedIndex` in `pyindex_to_graph`.
Wed, Jan 8, 1:11 PM
marmoute committed rHGc627f1b2f3c3: rust-index: handle `MixedIndex` in `pyindex_to_graph`.
rust-index: handle `MixedIndex` in `pyindex_to_graph`
Wed, Jan 8, 1:10 PM

Tue, Jan 7

marmoute added inline comments to D7808: mmap: add a size argument to mmapread.
Tue, Jan 7, 1:20 PM