( )⚙ D3991 absorb: import extension from Facebook's hg-experimental

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3991

absorb: import extension from Facebook's hg-experimental
ClosedPublic

Authored by durin42 on Aug 1 2018, 11:17 AM.

Details

Reviewers
indygreg
Group Reviewers
hg-reviewers
Commits
rHG5111d11b8719: absorb: import extension from Facebook's hg-experimental
Summary

absorb is a wicked-fast command to use blame information to
automatically amend edits to the correct draft revision. Originally
written by Jun Wu, this import is hgext3rd/absorb/__init__.py with:

  • the testedwith value changed
  • the linelog import updated
  • some missing configitems registered
  • some imports reordered per check-code.py
  • some missing future imports added per check-code.py

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

durin42 created this revision.Aug 1 2018, 11:17 AM
indygreg added a subscriber: indygreg.Aug 1 2018, 4:58 PM

I only reviewed about half of this. I'm inclined to take it as-is or close to as-is. But I would like to have a handle of the follow-ups before we forget about them.

hgext/absorb.py
67–69

I'd like to see these moved to [commands] and to use hyphens in the names for readability. Could be deferred to a follow-up to minimize import churn. Worth adding an inline TODO/FUTURE to track though.

68

Do we even need this? Can't [alias] do this?

107–127

I'm pretty sure we have a variant of this code somewhere in core that could be reused.

129

Nit: mutable default argument value

174

Nit: the p1 is not necessarily public/immutable from the perspective of the store (because of stack limits). But it should be immutable from the perspective of the absorb operation.

176

The comment is overstating reality.

237

I was initially worried about this because it returns a reference. But it looks like committablectx.__init__ makes a copy of the dict to avoid surprises. So this is fine.

311–312

The bad API rears its ugly head again :/

322

TIL that Python allows 0 + True and 0 + False. Sadness.

378

Using a set comprehension will avoid having to construct an intermediate list.

381–385

We could do without the cast to a list and do a set.pop() to return its single element. I guess it is wonky either way. But seeing the list(set(...)) followed by a [0] access unsettled me. It's all good because of the len(involvedrevs) == 1 check though.

391

pycompat.xrange

418

pycompat.xrange.

430

pycompat.xrange

592–603

I think a lot of these want to be private.

622–623

self.ui.debug().

938–939

Maybe this should be controlled with --verbose?

979

I'd like to think we could delete this code and leave it up to these extensions to do their own monkeypatching.

Actually, this whole advanced wrapping of amend feels very hacky. We should definitely get this logic moved to the actual definition of the amend command since it is part of this repo.

1003

Nit: use a tuple

1020–1022

The status object is a named tuple of sorts. I'd prefer to see us access the named attributes instead of the integer fields. (And I wish we could deprecate accessing by index because it is hard to read.) Suitable as a follow-up.

1025

I don't believe the string ex. appears in the UI today. We should tweak this string.

Also, this logic is action at a distance and is making inferences. Perhaps we should actually test for the underlying condition and display a more accurate error message?

1030–1035

This wants to be templatized. Follow-up territory.

indygreg accepted this revision.Aug 6 2018, 11:53 AM

I'm fine taking this as-is.

This revision is now accepted and ready to land.Aug 6 2018, 11:53 AM
Closed by commit rHG5111d11b8719: absorb: import extension from Facebook's hg-experimental (authored by durin42). · Explain WhyAug 6 2018, 11:54 AM
This revision was automatically updated to reflect the committed changes.
durin42 added a comment.Aug 7 2018, 10:24 PM

absorb.py:67
+
+configitem('absorb', 'addnoise', default=True)
+configitem('absorb', 'amendflag', default=None)

https://www.mercurial-scm.org/wiki/UIGuideline#adding_new_options

Please feel encouraged to send cleanups to absorb - the initial import was kept as pristine as possible to make history clearer.

Revision Contents
Changeset List

Diff 9682

hgext/absorb.py

  • This file was added.
# absorb.py
#
# Copyright 2016 Facebook, Inc.
#
# This software may be used and distributed according to the terms of the
# GNU General Public License version 2 or any later version.
"""apply working directory changes to changesets (EXPERIMENTAL)
The absorb extension provides a command to use annotate information to
amend modified chunks into the corresponding non-public changesets.
::
[absorb]
# only check 50 recent non-public changesets at most
maxstacksize = 50
# whether to add noise to new commits to avoid obsolescence cycle
addnoise = 1
# make `amend --correlated` a shortcut to the main command
amendflag = correlated
[color]
absorb.node = blue bold
absorb.path = bold
"""
from __future__ import absolute_import
import collections
from mercurial.i18n import _
from mercurial import (
cmdutil,
commands,
context,
crecord,
error,
extensions,
linelog,
mdiff,
node,
obsolete,
patch,
phases,
registrar,
repair,
scmutil,
util,
)
from mercurial.utils import (
stringutil,
)
# Note for extension authors: ONLY specify testedwith = 'ships-with-hg-core' for
# extensions which SHIP WITH MERCURIAL. Non-mainline extensions should
# be specifying the version(s) of Mercurial they are tested with, or
# leave the attribute unspecified.
testedwith = 'ships-with-hg-core'
cmdtable = {}
command = registrar.command(cmdtable)
configtable = {}
configitem = registrar.configitem(configtable)
configitem('absorb', 'addnoise', default=True)
configitem('absorb', 'amendflag', default=None)
indygregUnsubmitted
Not Done

Do we even need this? Can't [alias] do this?

indygreg: Do we even need this? Can't `[alias]` do this?
markandUnsubmitted
Not Done

https://www.mercurial-scm.org/wiki/UIGuideline#adding_new_options

markand: https://www.mercurial-scm.org/wiki/UIGuideline#adding_new_options
configitem('absorb', 'maxstacksize', default=50)
indygregUnsubmitted
Not Done

I'd like to see these moved to [commands] and to use hyphens in the names for readability. Could be deferred to a follow-up to minimize import churn. Worth adding an inline TODO/FUTURE to track though.

indygreg: I'd like to see these moved to `[commands]` and to use hyphens in the names for readability.
colortable = {
'absorb.node': 'blue bold',
'absorb.path': 'bold',
}
defaultdict = collections.defaultdict
class nullui(object):
"""blank ui object doing nothing"""
debugflag = False
verbose = False
quiet = True
def __getitem__(name):
def nullfunc(*args, **kwds):
return
return nullfunc
class emptyfilecontext(object):
"""minimal filecontext representing an empty file"""
def data(self):
return ''
def node(self):
return node.nullid
def uniq(lst):
"""list -> list. remove duplicated items without changing the order"""
seen = set()
result = []
for x in lst:
if x not in seen:
seen.add(x)
result.append(x)
return result
def getdraftstack(headctx, limit=None):
"""(ctx, int?) -> [ctx]. get a linear stack of non-public changesets.
changesets are sorted in topo order, oldest first.
return at most limit items, if limit is a positive number.
merges are considered as non-draft as well. i.e. every commit
returned has and only has 1 parent.
"""
ctx = headctx
result = []
while ctx.phase() != phases.public:
if limit and len(result) >= limit:
break
parents = ctx.parents()
if len(parents) != 1:
break
result.append(ctx)
ctx = parents[0]
result.reverse()
return result
indygregUnsubmitted
Not Done

I'm pretty sure we have a variant of this code somewhere in core that could be reused.

indygreg: I'm pretty sure we have a variant of this code somewhere in core that could be reused.
def getfilestack(stack, path, seenfctxs=set()):
indygregUnsubmitted
Not Done

Nit: mutable default argument value

indygreg: Nit: mutable default argument value
"""([ctx], str, set) -> [fctx], {ctx: fctx}
stack is a list of contexts, from old to new. usually they are what
"getdraftstack" returns.
follows renames, but not copies.
seenfctxs is a set of filecontexts that will be considered "immutable".
they are usually what this function returned in earlier calls, useful
to avoid issues that a file was "moved" to multiple places and was then
modified differently, like: "a" was copied to "b", "a" was also copied to
"c" and then "a" was deleted, then both "b" and "c" were "moved" from "a"
and we enforce only one of them to be able to affect "a"'s content.
return an empty list and an empty dict, if the specified path does not
exist in stack[-1] (the top of the stack).
otherwise, return a list of de-duplicated filecontexts, and the map to
convert ctx in the stack to fctx, for possible mutable fctxs. the first item
of the list would be outside the stack and should be considered immutable.
the remaining items are within the stack.
for example, given the following changelog and corresponding filelog
revisions:
changelog: 3----4----5----6----7
filelog: x 0----1----1----2 (x: no such file yet)
- if stack = [5, 6, 7], returns ([0, 1, 2], {5: 1, 6: 1, 7: 2})
- if stack = [3, 4, 5], returns ([e, 0, 1], {4: 0, 5: 1}), where "e" is a
dummy empty filecontext.
- if stack = [2], returns ([], {})
- if stack = [7], returns ([1, 2], {7: 2})
- if stack = [6, 7], returns ([1, 2], {6: 1, 7: 2}), although {6: 1} can be
removed, since 1 is immutable.
"""
assert stack
if path not in stack[-1]:
return [], {}
fctxs = []
fctxmap = {}
pctx = stack[0].p1() # the public (immutable) ctx we stop at
indygregUnsubmitted
Not Done

Nit: the p1 is not necessarily public/immutable from the perspective of the store (because of stack limits). But it should be immutable from the perspective of the absorb operation.

indygreg: Nit: the p1 is not necessarily public/immutable from the perspective of the store (because of…
for ctx in reversed(stack):
if path not in ctx: # the file is added in the next commit
indygregUnsubmitted
Not Done

The comment is overstating reality.

indygreg: The comment is overstating reality.
pctx = ctx
break
fctx = ctx[path]
fctxs.append(fctx)
if fctx in seenfctxs: # treat fctx as the immutable one
pctx = None # do not add another immutable fctx
break
fctxmap[ctx] = fctx # only for mutable fctxs
renamed = fctx.renamed()
if renamed:
path = renamed[0] # follow rename
if path in ctx: # but do not follow copy
pctx = ctx.p1()
break
if pctx is not None: # need an extra immutable fctx
if path in pctx:
fctxs.append(pctx[path])
else:
fctxs.append(emptyfilecontext())
fctxs.reverse()
# note: we rely on a property of hg: filerev is not reused for linear
# history. i.e. it's impossible to have:
# changelog: 4----5----6 (linear, no merges)
# filelog: 1----2----1
# ^ reuse filerev (impossible)
# because parents are part of the hash. if that's not true, we need to
# remove uniq and find a different way to identify fctxs.
return uniq(fctxs), fctxmap
class overlaystore(patch.filestore):
"""read-only, hybrid store based on a dict and ctx.
memworkingcopy: {path: content}, overrides file contents.
"""
def __init__(self, basectx, memworkingcopy):
self.basectx = basectx
self.memworkingcopy = memworkingcopy
def getfile(self, path):
"""comply with mercurial.patch.filestore.getfile"""
if path not in self.basectx:
return None, None, None
fctx = self.basectx[path]
if path in self.memworkingcopy:
content = self.memworkingcopy[path]
else:
content = fctx.data()
mode = (fctx.islink(), fctx.isexec())
renamed = fctx.renamed() # False or (path, node)
return content, mode, (renamed and renamed[0])
def overlaycontext(memworkingcopy, ctx, parents=None, extra=None):
"""({path: content}, ctx, (p1node, p2node)?, {}?) -> memctx
memworkingcopy overrides file contents.
"""
# parents must contain 2 items: (node1, node2)
if parents is None:
parents = ctx.repo().changelog.parents(ctx.node())
if extra is None:
extra = ctx.extra()
indygregUnsubmitted
Not Done

I was initially worried about this because it returns a reference. But it looks like committablectx.__init__ makes a copy of the dict to avoid surprises. So this is fine.

indygreg: I was initially worried about this because it returns a reference. But it looks like…
date = ctx.date()
desc = ctx.description()
user = ctx.user()
files = set(ctx.files()).union(memworkingcopy.iterkeys())
store = overlaystore(ctx, memworkingcopy)
return context.memctx(
repo=ctx.repo(), parents=parents, text=desc,
files=files, filectxfn=store, user=user, date=date,
branch=None, extra=extra)
class filefixupstate(object):
"""state needed to apply fixups to a single file
internally, it keeps file contents of several revisions and a linelog.
the linelog uses odd revision numbers for original contents (fctxs passed
to __init__), and even revision numbers for fixups, like:
linelog rev 1: self.fctxs[0] (from an immutable "public" changeset)
linelog rev 2: fixups made to self.fctxs[0]
linelog rev 3: self.fctxs[1] (a child of fctxs[0])
linelog rev 4: fixups made to self.fctxs[1]
...
a typical use is like:
1. call diffwith, to calculate self.fixups
2. (optionally), present self.fixups to the user, or change it
3. call apply, to apply changes
4. read results from "finalcontents", or call getfinalcontent
"""
def __init__(self, fctxs, ui=None, opts=None):
"""([fctx], ui or None) -> None
fctxs should be linear, and sorted by topo order - oldest first.
fctxs[0] will be considered as "immutable" and will not be changed.
"""
self.fctxs = fctxs
self.ui = ui or nullui()
self.opts = opts or {}
# following fields are built from fctxs. they exist for perf reason
self.contents = [f.data() for f in fctxs]
self.contentlines = map(mdiff.splitnewlines, self.contents)
self.linelog = self._buildlinelog()
if self.ui.debugflag:
assert self._checkoutlinelog() == self.contents
# following fields will be filled later
self.chunkstats = [0, 0] # [adopted, total : int]
self.targetlines = [] # [str]
self.fixups = [] # [(linelog rev, a1, a2, b1, b2)]
self.finalcontents = [] # [str]
def diffwith(self, targetfctx, showchanges=False):
"""calculate fixups needed by examining the differences between
self.fctxs[-1] and targetfctx, chunk by chunk.
targetfctx is the target state we move towards. we may or may not be
able to get there because not all modified chunks can be amended into
a non-public fctx unambiguously.
call this only once, before apply().
update self.fixups, self.chunkstats, and self.targetlines.
"""
a = self.contents[-1]
alines = self.contentlines[-1]
b = targetfctx.data()
blines = mdiff.splitnewlines(b)
self.targetlines = blines
self.linelog.annotate(self.linelog.maxrev)
annotated = self.linelog.annotateresult # [(linelog rev, linenum)]
indygregUnsubmitted
Not Done

The bad API rears its ugly head again :/

indygreg: The bad API rears its ugly head again :/
assert len(annotated) == len(alines)
# add a dummy end line to make insertion at the end easier
if annotated:
dummyendline = (annotated[-1][0], annotated[-1][1] + 1)
annotated.append(dummyendline)
# analyse diff blocks
for chunk in self._alldiffchunks(a, b, alines, blines):
newfixups = self._analysediffchunk(chunk, annotated)
self.chunkstats[0] += bool(newfixups) # 1 or 0
indygregUnsubmitted
Not Done

TIL that Python allows 0 + True and 0 + False. Sadness.

indygreg: TIL that Python allows `0 + True` and `0 + False`. Sadness.
self.chunkstats[1] += 1
self.fixups += newfixups
if showchanges:
self._showchanges(alines, blines, chunk, newfixups)
def apply(self):
"""apply self.fixups. update self.linelog, self.finalcontents.
call this only once, before getfinalcontent(), after diffwith().
"""
# the following is unnecessary, as it's done by "diffwith":
# self.linelog.annotate(self.linelog.maxrev)
for rev, a1, a2, b1, b2 in reversed(self.fixups):
blines = self.targetlines[b1:b2]
if self.ui.debugflag:
idx = (max(rev - 1, 0)) // 2
self.ui.write(_('%s: chunk %d:%d -> %d lines\n')
% (node.short(self.fctxs[idx].node()),
a1, a2, len(blines)))
self.linelog.replacelines(rev, a1, a2, b1, b2)
if self.opts.get('edit_lines', False):
self.finalcontents = self._checkoutlinelogwithedits()
else:
self.finalcontents = self._checkoutlinelog()
def getfinalcontent(self, fctx):
"""(fctx) -> str. get modified file content for a given filecontext"""
idx = self.fctxs.index(fctx)
return self.finalcontents[idx]
def _analysediffchunk(self, chunk, annotated):
"""analyse a different chunk and return new fixups found
return [] if no lines from the chunk can be safely applied.
the chunk (or lines) cannot be safely applied, if, for example:
- the modified (deleted) lines belong to a public changeset
(self.fctxs[0])
- the chunk is a pure insertion and the adjacent lines (at most 2
lines) belong to different non-public changesets, or do not belong
to any non-public changesets.
- the chunk is modifying lines from different changesets.
in this case, if the number of lines deleted equals to the number
of lines added, assume it's a simple 1:1 map (could be wrong).
otherwise, give up.
- the chunk is modifying lines from a single non-public changeset,
but other revisions touch the area as well. i.e. the lines are
not continuous as seen from the linelog.
"""
a1, a2, b1, b2 = chunk
# find involved indexes from annotate result
involved = annotated[a1:a2]
if not involved and annotated: # a1 == a2 and a is not empty
# pure insertion, check nearby lines. ignore lines belong
# to the public (first) changeset (i.e. annotated[i][0] == 1)
nearbylinenums = set([a2, max(0, a1 - 1)])
indygregUnsubmitted
Not Done

Using a set comprehension will avoid having to construct an intermediate list.

indygreg: Using a set comprehension will avoid having to construct an intermediate list.
involved = [annotated[i]
for i in nearbylinenums if annotated[i][0] != 1]
involvedrevs = list(set(r for r, l in involved))
newfixups = []
if len(involvedrevs) == 1 and self._iscontinuous(a1, a2 - 1, True):
# chunk belongs to a single revision
rev = involvedrevs[0]
indygregUnsubmitted
Not Done

We could do without the cast to a list and do a set.pop() to return its single element. I guess it is wonky either way. But seeing the list(set(...)) followed by a [0] access unsettled me. It's all good because of the len(involvedrevs) == 1 check though.

indygreg: We could do without the cast to a list and do a `set.pop()` to return its single element. I…
if rev > 1:
fixuprev = rev + 1
newfixups.append((fixuprev, a1, a2, b1, b2))
elif a2 - a1 == b2 - b1 or b1 == b2:
# 1:1 line mapping, or chunk was deleted
for i in xrange(a1, a2):
indygregUnsubmitted
Not Done

pycompat.xrange

indygreg: pycompat.xrange
rev, linenum = annotated[i]
if rev > 1:
if b1 == b2: # deletion, simply remove that single line
nb1 = nb2 = 0
else: # 1:1 line mapping, change the corresponding rev
nb1 = b1 + i - a1
nb2 = nb1 + 1
fixuprev = rev + 1
newfixups.append((fixuprev, i, i + 1, nb1, nb2))
return self._optimizefixups(newfixups)
@staticmethod
def _alldiffchunks(a, b, alines, blines):
"""like mdiff.allblocks, but only care about differences"""
blocks = mdiff.allblocks(a, b, lines1=alines, lines2=blines)
for chunk, btype in blocks:
if btype != '!':
continue
yield chunk
def _buildlinelog(self):
"""calculate the initial linelog based on self.content{,line}s.
this is similar to running a partial "annotate".
"""
llog = linelog.linelog()
a, alines = '', []
for i in xrange(len(self.contents)):
indygregUnsubmitted
Not Done

pycompat.xrange.

indygreg: `pycompat.xrange`.
b, blines = self.contents[i], self.contentlines[i]
llrev = i * 2 + 1
chunks = self._alldiffchunks(a, b, alines, blines)
for a1, a2, b1, b2 in reversed(list(chunks)):
llog.replacelines(llrev, a1, a2, b1, b2)
a, alines = b, blines
return llog
def _checkoutlinelog(self):
"""() -> [str]. check out file contents from linelog"""
contents = []
for i in xrange(len(self.contents)):
indygregUnsubmitted
Not Done

pycompat.xrange

indygreg: pycompat.xrange
rev = (i + 1) * 2
self.linelog.annotate(rev)
content = ''.join(map(self._getline, self.linelog.annotateresult))
contents.append(content)
return contents
def _checkoutlinelogwithedits(self):
"""() -> [str]. prompt all lines for edit"""
alllines = self.linelog.getalllines()
# header
editortext = (_('HG: editing %s\nHG: "y" means the line to the right '
'exists in the changeset to the top\nHG:\n')
% self.fctxs[-1].path())
# [(idx, fctx)]. hide the dummy emptyfilecontext
visiblefctxs = [(i, f)
for i, f in enumerate(self.fctxs)
if not isinstance(f, emptyfilecontext)]
for i, (j, f) in enumerate(visiblefctxs):
editortext += (_('HG: %s/%s %s %s\n') %
('|' * i, '-' * (len(visiblefctxs) - i + 1),
node.short(f.node()),
f.description().split('\n',1)[0]))
editortext += _('HG: %s\n') % ('|' * len(visiblefctxs))
# figure out the lifetime of a line, this is relatively inefficient,
# but probably fine
lineset = defaultdict(lambda: set()) # {(llrev, linenum): {llrev}}
for i, f in visiblefctxs:
self.linelog.annotate((i + 1) * 2)
for l in self.linelog.annotateresult:
lineset[l].add(i)
# append lines
for l in alllines:
editortext += (' %s : %s' %
(''.join([('y' if i in lineset[l] else ' ')
for i, _f in visiblefctxs]),
self._getline(l)))
# run editor
editedtext = self.ui.edit(editortext, '', action='absorb')
if not editedtext:
raise error.Abort(_('empty editor text'))
# parse edited result
contents = ['' for i in self.fctxs]
leftpadpos = 4
colonpos = leftpadpos + len(visiblefctxs) + 1
for l in mdiff.splitnewlines(editedtext):
if l.startswith('HG:'):
continue
if l[colonpos - 1:colonpos + 2] != ' : ':
raise error.Abort(_('malformed line: %s') % l)
linecontent = l[colonpos + 2:]
for i, ch in enumerate(l[leftpadpos:colonpos - 1]):
if ch == 'y':
contents[visiblefctxs[i][0]] += linecontent
# chunkstats is hard to calculate if anything changes, therefore
# set them to just a simple value (1, 1).
if editedtext != editortext:
self.chunkstats = [1, 1]
return contents
def _getline(self, lineinfo):
"""((rev, linenum)) -> str. convert rev+line number to line content"""
rev, linenum = lineinfo
if rev & 1: # odd: original line taken from fctxs
return self.contentlines[rev // 2][linenum]
else: # even: fixup line from targetfctx
return self.targetlines[linenum]
def _iscontinuous(self, a1, a2, closedinterval=False):
"""(a1, a2 : int) -> bool
check if these lines are continuous. i.e. no other insertions or
deletions (from other revisions) among these lines.
closedinterval decides whether a2 should be included or not. i.e. is
it [a1, a2), or [a1, a2] ?
"""
if a1 >= a2:
return True
llog = self.linelog
offset1 = llog.getoffset(a1)
offset2 = llog.getoffset(a2) + int(closedinterval)
linesinbetween = llog.getalllines(offset1, offset2)
return len(linesinbetween) == a2 - a1 + int(closedinterval)
def _optimizefixups(self, fixups):
"""[(rev, a1, a2, b1, b2)] -> [(rev, a1, a2, b1, b2)].
merge adjacent fixups to make them less fragmented.
"""
result = []
pcurrentchunk = [[-1, -1, -1, -1, -1]]
def pushchunk():
if pcurrentchunk[0][0] != -1:
result.append(tuple(pcurrentchunk[0]))
for i, chunk in enumerate(fixups):
rev, a1, a2, b1, b2 = chunk
lastrev = pcurrentchunk[0][0]
lasta2 = pcurrentchunk[0][2]
lastb2 = pcurrentchunk[0][4]
if (a1 == lasta2 and b1 == lastb2 and rev == lastrev and
self._iscontinuous(max(a1 - 1, 0), a1)):
# merge into currentchunk
pcurrentchunk[0][2] = a2
pcurrentchunk[0][4] = b2
else:
pushchunk()
pcurrentchunk[0] = list(chunk)
pushchunk()
return result
def _showchanges(self, alines, blines, chunk, fixups):
ui = self.ui
def label(line, label):
if line.endswith('\n'):
line = line[:-1]
return ui.label(line, label)
# this is not optimized for perf but _showchanges only gets executed
# with an extra command-line flag.
a1, a2, b1, b2 = chunk
aidxs, bidxs = [0] * (a2 - a1), [0] * (b2 - b1)
for idx, fa1, fa2, fb1, fb2 in fixups:
for i in xrange(fa1, fa2):
aidxs[i - a1] = (max(idx, 1) - 1) // 2
for i in xrange(fb1, fb2):
bidxs[i - b1] = (max(idx, 1) - 1) // 2
buf = [] # [(idx, content)]
buf.append((0, label('@@ -%d,%d +%d,%d @@'
% (a1, a2 - a1, b1, b2 - b1), 'diff.hunk')))
buf += [(aidxs[i - a1], label('-' + alines[i], 'diff.deleted'))
for i in xrange(a1, a2)]
buf += [(bidxs[i - b1], label('+' + blines[i], 'diff.inserted'))
for i in xrange(b1, b2)]
for idx, line in buf:
shortnode = idx and node.short(self.fctxs[idx].node()) or ''
ui.write(ui.label(shortnode[0:7].ljust(8), 'absorb.node') +
line + '\n')
class fixupstate(object):
"""state needed to run absorb
internally, it keeps paths and filefixupstates.
a typical use is like filefixupstates:
1. call diffwith, to calculate fixups
2. (optionally), present fixups to the user, or edit fixups
3. call apply, to apply changes to memory
4. call commit, to commit changes to hg database
"""
def __init__(self, stack, ui=None, opts=None):
"""([ctx], ui or None) -> None
stack: should be linear, and sorted by topo order - oldest first.
all commits in stack are considered mutable.
"""
assert stack
self.ui = ui or nullui()
self.opts = opts or {}
self.stack = stack
self.repo = stack[-1].repo().unfiltered()
# following fields will be filled later
self.paths = [] # [str]
self.status = None # ctx.status output
self.fctxmap = {} # {path: {ctx: fctx}}
self.fixupmap = {} # {path: filefixupstate}
self.replacemap = {} # {oldnode: newnode or None}
self.finalnode = None # head after all fixups
indygregUnsubmitted
Not Done

I think a lot of these want to be private.

indygreg: I think a lot of these want to be private.
def diffwith(self, targetctx, match=None, showchanges=False):
"""diff and prepare fixups. update self.fixupmap, self.paths"""
# only care about modified files
self.status = self.stack[-1].status(targetctx, match)
self.paths = []
# but if --edit-lines is used, the user may want to edit files
# even if they are not modified
editopt = self.opts.get('edit_lines')
if not self.status.modified and editopt and match:
interestingpaths = match.files()
else:
interestingpaths = self.status.modified
# prepare the filefixupstate
seenfctxs = set()
# sorting is necessary to eliminate ambiguity for the "double move"
# case: "hg cp A B; hg cp A C; hg rm A", then only "B" can affect "A".
for path in sorted(interestingpaths):
if self.ui.debugflag:
self.ui.write(_('calculating fixups for %s\n') % path)
indygregUnsubmitted
Not Done

self.ui.debug().

indygreg: `self.ui.debug()`.
targetfctx = targetctx[path]
fctxs, ctx2fctx = getfilestack(self.stack, path, seenfctxs)
# ignore symbolic links or binary, or unchanged files
if any(f.islink() or stringutil.binary(f.data())
for f in [targetfctx] + fctxs
if not isinstance(f, emptyfilecontext)):
continue
if targetfctx.data() == fctxs[-1].data() and not editopt:
continue
seenfctxs.update(fctxs[1:])
self.fctxmap[path] = ctx2fctx
fstate = filefixupstate(fctxs, ui=self.ui, opts=self.opts)
if showchanges:
colorpath = self.ui.label(path, 'absorb.path')
header = 'showing changes for ' + colorpath
self.ui.write(header + '\n')
fstate.diffwith(targetfctx, showchanges=showchanges)
self.fixupmap[path] = fstate
self.paths.append(path)
def apply(self):
"""apply fixups to individual filefixupstates"""
for path, state in self.fixupmap.iteritems():
if self.ui.debugflag:
self.ui.write(_('applying fixups to %s\n') % path)
state.apply()
@property
def chunkstats(self):
"""-> {path: chunkstats}. collect chunkstats from filefixupstates"""
return dict((path, state.chunkstats)
for path, state in self.fixupmap.iteritems())
def commit(self):
"""commit changes. update self.finalnode, self.replacemap"""
with self.repo.wlock(), self.repo.lock():
with self.repo.transaction('absorb') as tr:
self._commitstack()
self._movebookmarks(tr)
if self.repo['.'].node() in self.replacemap:
self._moveworkingdirectoryparent()
if self._useobsolete:
self._obsoleteoldcommits()
if not self._useobsolete: # strip must be outside transactions
self._stripoldcommits()
return self.finalnode
def printchunkstats(self):
"""print things like '1 of 2 chunk(s) applied'"""
ui = self.ui
chunkstats = self.chunkstats
if ui.verbose:
# chunkstats for each file
for path, stat in chunkstats.iteritems():
if stat[0]:
ui.write(_('%s: %d of %d chunk(s) applied\n')
% (path, stat[0], stat[1]))
elif not ui.quiet:
# a summary for all files
stats = chunkstats.values()
applied, total = (sum(s[i] for s in stats) for i in (0, 1))
ui.write(_('%d of %d chunk(s) applied\n') % (applied, total))
def _commitstack(self):
"""make new commits. update self.finalnode, self.replacemap.
it is splitted from "commit" to avoid too much indentation.
"""
# last node (20-char) committed by us
lastcommitted = None
# p1 which overrides the parent of the next commit, "None" means use
# the original parent unchanged
nextp1 = None
for ctx in self.stack:
memworkingcopy = self._getnewfilecontents(ctx)
if not memworkingcopy and not lastcommitted:
# nothing changed, nothing commited
nextp1 = ctx
continue
msg = ''
if self._willbecomenoop(memworkingcopy, ctx, nextp1):
# changeset is no longer necessary
self.replacemap[ctx.node()] = None
msg = _('became empty and was dropped')
else:
# changeset needs re-commit
nodestr = self._commitsingle(memworkingcopy, ctx, p1=nextp1)
lastcommitted = self.repo[nodestr]
nextp1 = lastcommitted
self.replacemap[ctx.node()] = lastcommitted.node()
if memworkingcopy:
msg = _('%d file(s) changed, became %s') % (
len(memworkingcopy), self._ctx2str(lastcommitted))
else:
msg = _('became %s') % self._ctx2str(lastcommitted)
if self.ui.verbose and msg:
self.ui.write(_('%s: %s\n') % (self._ctx2str(ctx), msg))
self.finalnode = lastcommitted and lastcommitted.node()
def _ctx2str(self, ctx):
if self.ui.debugflag:
return ctx.hex()
else:
return node.short(ctx.node())
def _getnewfilecontents(self, ctx):
"""(ctx) -> {path: str}
fetch file contents from filefixupstates.
return the working copy overrides - files different from ctx.
"""
result = {}
for path in self.paths:
ctx2fctx = self.fctxmap[path] # {ctx: fctx}
if ctx not in ctx2fctx:
continue
fctx = ctx2fctx[ctx]
content = fctx.data()
newcontent = self.fixupmap[path].getfinalcontent(fctx)
if content != newcontent:
result[fctx.path()] = newcontent
return result
def _movebookmarks(self, tr):
repo = self.repo
needupdate = [(name, self.replacemap[hsh])
for name, hsh in repo._bookmarks.iteritems()
if hsh in self.replacemap]
changes = []
for name, hsh in needupdate:
if hsh:
changes.append((name, hsh))
if self.ui.verbose:
self.ui.write(_('moving bookmark %s to %s\n')
% (name, node.hex(hsh)))
else:
changes.append((name, None))
if self.ui.verbose:
self.ui.write(_('deleting bookmark %s\n') % name)
repo._bookmarks.applychanges(repo, tr, changes)
def _moveworkingdirectoryparent(self):
if not self.finalnode:
# Find the latest not-{obsoleted,stripped} parent.
revs = self.repo.revs('max(::. - %ln)', self.replacemap.keys())
ctx = self.repo[revs.first()]
self.finalnode = ctx.node()
else:
ctx = self.repo[self.finalnode]
dirstate = self.repo.dirstate
# dirstate.rebuild invalidates fsmonitorstate, causing "hg status" to
# be slow. in absorb's case, no need to invalidate fsmonitorstate.
noop = lambda: 0
restore = noop
if util.safehasattr(dirstate, '_fsmonitorstate'):
bak = dirstate._fsmonitorstate.invalidate
def restore():
dirstate._fsmonitorstate.invalidate = bak
dirstate._fsmonitorstate.invalidate = noop
try:
with dirstate.parentchange():
dirstate.rebuild(ctx.node(), ctx.manifest(), self.paths)
finally:
restore()
@staticmethod
def _willbecomenoop(memworkingcopy, ctx, pctx=None):
"""({path: content}, ctx, ctx) -> bool. test if a commit will be noop
if it will become an empty commit (does not change anything, after the
memworkingcopy overrides), return True. otherwise return False.
"""
if not pctx:
parents = ctx.parents()
if len(parents) != 1:
return False
pctx = parents[0]
# ctx changes more files (not a subset of memworkingcopy)
if not set(ctx.files()).issubset(set(memworkingcopy.iterkeys())):
return False
for path, content in memworkingcopy.iteritems():
if path not in pctx or path not in ctx:
return False
fctx = ctx[path]
pfctx = pctx[path]
if pfctx.flags() != fctx.flags():
return False
if pfctx.data() != content:
return False
return True
def _commitsingle(self, memworkingcopy, ctx, p1=None):
"""(ctx, {path: content}, node) -> node. make a single commit
the commit is a clone from ctx, with a (optionally) different p1, and
different file contents replaced by memworkingcopy.
"""
parents = p1 and (p1, node.nullid)
extra = ctx.extra()
if self._useobsolete and self.ui.configbool('absorb', 'addnoise'):
extra['absorb_source'] = ctx.hex()
mctx = overlaycontext(memworkingcopy, ctx, parents, extra=extra)
# preserve phase
with mctx.repo().ui.configoverride({
('phases', 'new-commit'): ctx.phase()}):
return mctx.commit()
@util.propertycache
def _useobsolete(self):
"""() -> bool"""
return obsolete.isenabled(self.repo, obsolete.createmarkersopt)
def _obsoleteoldcommits(self):
relations = [(self.repo[k], v and (self.repo[v],) or ())
for k, v in self.replacemap.iteritems()]
if relations:
obsolete.createmarkers(self.repo, relations)
def _stripoldcommits(self):
nodelist = self.replacemap.keys()
# make sure we don't strip innocent children
revs = self.repo.revs('%ln - (::(heads(%ln::)-%ln))', nodelist,
nodelist, nodelist)
tonode = self.repo.changelog.node
nodelist = [tonode(r) for r in revs]
if nodelist:
repair.strip(self.repo.ui, self.repo, nodelist)
def _parsechunk(hunk):
"""(crecord.uihunk or patch.recordhunk) -> (path, (a1, a2, [bline]))"""
if type(hunk) not in (crecord.uihunk, patch.recordhunk):
return None, None
path = hunk.header.filename()
a1 = hunk.fromline + len(hunk.before) - 1
# remove before and after context
hunk.before = hunk.after = []
buf = util.stringio()
hunk.write(buf)
patchlines = mdiff.splitnewlines(buf.getvalue())
# hunk.prettystr() will update hunk.removed
a2 = a1 + hunk.removed
blines = [l[1:] for l in patchlines[1:] if l[0] != '-']
return path, (a1, a2, blines)
def overlaydiffcontext(ctx, chunks):
"""(ctx, [crecord.uihunk]) -> memctx
return a memctx with some [1] patches (chunks) applied to ctx.
[1]: modifications are handled. renames, mode changes, etc. are ignored.
"""
# sadly the applying-patch logic is hardly reusable, and messy:
# 1. the core logic "_applydiff" is too heavy - it writes .rej files, it
# needs a file stream of a patch and will re-parse it, while we have
# structured hunk objects at hand.
# 2. a lot of different implementations about "chunk" (patch.hunk,
# patch.recordhunk, crecord.uihunk)
# as we only care about applying changes to modified files, no mode
# change, no binary diff, and no renames, it's probably okay to
# re-invent the logic using much simpler code here.
memworkingcopy = {} # {path: content}
patchmap = defaultdict(lambda: []) # {path: [(a1, a2, [bline])]}
for path, info in map(_parsechunk, chunks):
if not path or not info:
continue
patchmap[path].append(info)
for path, patches in patchmap.iteritems():
if path not in ctx or not patches:
continue
patches.sort(reverse=True)
lines = mdiff.splitnewlines(ctx[path].data())
for a1, a2, blines in patches:
lines[a1:a2] = blines
memworkingcopy[path] = ''.join(lines)
return overlaycontext(memworkingcopy, ctx)
def absorb(ui, repo, stack=None, targetctx=None, pats=None, opts=None):
"""pick fixup chunks from targetctx, apply them to stack.
if targetctx is None, the working copy context will be used.
if stack is None, the current draft stack will be used.
return fixupstate.
"""
if stack is None:
limit = ui.configint('absorb', 'maxstacksize')
stack = getdraftstack(repo['.'], limit)
if limit and len(stack) >= limit:
ui.warn(_('absorb: only the recent %d changesets will '
'be analysed\n')
% limit)
if not stack:
raise error.Abort(_('no changeset to change'))
if targetctx is None: # default to working copy
targetctx = repo[None]
if pats is None:
pats = ()
if opts is None:
opts = {}
state = fixupstate(stack, ui=ui, opts=opts)
matcher = scmutil.match(targetctx, pats, opts)
if opts.get('interactive'):
diff = patch.diff(repo, stack[-1].node(), targetctx.node(), matcher)
origchunks = patch.parsepatch(diff)
chunks = cmdutil.recordfilter(ui, origchunks)[0]
targetctx = overlaydiffcontext(stack[-1], chunks)
state.diffwith(targetctx, matcher, showchanges=opts.get('print_changes'))
if not opts.get('dry_run'):
state.apply()
if state.commit():
state.printchunkstats()
elif not ui.quiet:
ui.write(_('nothing applied\n'))
return state
@command('^absorb|sf',
[('p', 'print-changes', None,
_('print which changesets are modified by which changes')),
indygregUnsubmitted
Not Done

Maybe this should be controlled with --verbose?

indygreg: Maybe this should be controlled with `--verbose`?
('i', 'interactive', None,
_('interactively select which chunks to apply (EXPERIMENTAL)')),
('e', 'edit-lines', None,
_('edit what lines belong to which changesets before commit '
'(EXPERIMENTAL)')),
] + commands.dryrunopts + commands.walkopts,
_('hg absorb [OPTION] [FILE]...'))
def absorbcmd(ui, repo, *pats, **opts):
"""incorporate corrections into the stack of draft changesets
absorb analyzes each change in your working directory and attempts to
amend the changed lines into the changesets in your stack that first
introduced those lines.
If absorb cannot find an unambiguous changeset to amend for a change,
that change will be left in the working directory, untouched. They can be
observed by :hg:`status` or :hg:`diff` afterwards. In other words,
absorb does not write to the working directory.
Changesets outside the revset `::. and not public() and not merge()` will
not be changed.
Changesets that become empty after applying the changes will be deleted.
If in doubt, run :hg:`absorb -pn` to preview what changesets will
be amended by what changed lines, without actually changing anything.
Returns 0 on success, 1 if all chunks were ignored and nothing amended.
"""
state = absorb(ui, repo, pats=pats, opts=opts)
if sum(s[0] for s in state.chunkstats.values()) == 0:
return 1
def _wrapamend(flag):
"""add flag to amend, which will be a shortcut to the absorb command"""
if not flag:
return
amendcmd = extensions.bind(_amendcmd, flag)
# the amend command can exist in evolve, or fbamend
for extname in ['evolve', 'fbamend', None]:
indygregUnsubmitted
Not Done

I'd like to think we could delete this code and leave it up to these extensions to do their own monkeypatching.

Actually, this whole advanced wrapping of amend feels very hacky. We should definitely get this logic moved to the actual definition of the amend command since it is part of this repo.

indygreg: I'd like to think we could delete this code and leave it up to these extensions to do their own…
try:
if extname is None:
cmdtable = commands.table
else:
ext = extensions.find(extname)
cmdtable = ext.cmdtable
except (KeyError, AttributeError):
continue
try:
entry = extensions.wrapcommand(cmdtable, 'amend', amendcmd)
options = entry[1]
msg = _('incorporate corrections into stack. '
'see \'hg help absorb\' for details')
options.append(('', flag, None, msg))
return
except error.UnknownCommand:
pass
def _amendcmd(flag, orig, ui, repo, *pats, **opts):
if not opts.get(flag):
return orig(ui, repo, *pats, **opts)
# use absorb
for k, v in opts.iteritems(): # check unsupported flags
if v and k not in ['interactive', flag]:
indygregUnsubmitted
Not Done

Nit: use a tuple

indygreg: Nit: use a tuple
raise error.Abort(_('--%s does not support --%s')
% (flag, k.replace('_', '-')))
state = absorb(ui, repo, pats=pats, opts=opts)
# different from the original absorb, tell users what chunks were
# ignored and were left. it's because users usually expect "amend" to
# take all of their changes and will feel strange otherwise.
# the original "absorb" command faces more-advanced users knowing
# what's going on and is less verbose.
adoptedsum = 0
messages = []
for path, (adopted, total) in state.chunkstats.iteritems():
adoptedsum += adopted
if adopted == total:
continue
reason = _('%d modified chunks were ignored') % (total - adopted)
messages.append(('M', 'modified', path, reason))
for idx, word, symbol in [(0, 'modified', 'M'), (1, 'added', 'A'),
(2, 'removed', 'R'), (3, 'deleted', '!')]:
paths = set(state.status[idx]) - set(state.paths)
indygregUnsubmitted
Not Done

The status object is a named tuple of sorts. I'd prefer to see us access the named attributes instead of the integer fields. (And I wish we could deprecate accessing by index because it is hard to read.) Suitable as a follow-up.

indygreg: The status object is a named tuple of sorts. I'd prefer to see us access the named attributes…
for path in sorted(paths):
if word == 'modified':
reason = _('unsupported file type (ex. binary or link)')
indygregUnsubmitted
Not Done

I don't believe the string ex. appears in the UI today. We should tweak this string.

Also, this logic is action at a distance and is making inferences. Perhaps we should actually test for the underlying condition and display a more accurate error message?

indygreg: I don't believe the string `ex.` appears in the UI today. We should tweak this string. Also…
else:
reason = _('%s files were ignored') % word
messages.append((symbol, word, path, reason))
if messages:
ui.write(_('\n# changes not applied and left in '
'working directory:\n'))
for symbol, word, path, reason in messages:
ui.write(_('# %s %s : %s\n') % (
ui.label(symbol, 'status.' + word),
ui.label(path, 'status.' + word), reason))
indygregUnsubmitted
Not Done

This wants to be templatized. Follow-up territory.

indygreg: This wants to be templatized. Follow-up territory.
if adoptedsum == 0:
return 1
def extsetup(ui):
_wrapamend(ui.config('absorb', 'amendflag'))

tests/test-absorb-edit-lines.t

  • This file was added.
$ cat >> $HGRCPATH << EOF
> [extensions]
> absorb=
> EOF
$ hg init repo1
$ cd repo1
Make some commits:
$ for i in 1 2 3; do
> echo $i >> a
> hg commit -A a -m "commit $i" -q
> done
absorb --edit-lines will run the editor if filename is provided:
$ hg absorb --edit-lines
nothing applied
[1]
$ HGEDITOR=cat hg absorb --edit-lines a
HG: editing a
HG: "y" means the line to the right exists in the changeset to the top
HG:
HG: /---- 4ec16f85269a commit 1
HG: |/--- 5c5f95224a50 commit 2
HG: ||/-- 43f0a75bede7 commit 3
HG: |||
yyy : 1
yy : 2
y : 3
nothing applied
[1]
Edit the file using --edit-lines:
$ cat > editortext << EOF
> y : a
> yy : b
> y : c
> yy : d
> y y : e
> y : f
> yyy : g
> EOF
$ HGEDITOR='cat editortext >' hg absorb -q --edit-lines a
$ hg cat -r 0 a
d
e
f
g
$ hg cat -r 1 a
b
c
d
g
$ hg cat -r 2 a
a
b
e
g

tests/test-absorb-filefixupstate.py

  • This file was added.
from __future__ import absolute_import, print_function
import itertools
from hgext import absorb
class simplefctx(object):
def __init__(self, content):
self.content = content
def data(self):
return self.content
def insertreturns(x):
# insert "\n"s after each single char
if isinstance(x, str):
return ''.join(ch + '\n' for ch in x)
else:
return map(insertreturns, x)
def removereturns(x):
# the revert of "insertreturns"
if isinstance(x, str):
return x.replace('\n', '')
else:
return map(removereturns, x)
def assertlistequal(lhs, rhs, decorator=lambda x: x):
if lhs != rhs:
raise RuntimeError('mismatch:\n actual: %r\n expected: %r'
% tuple(map(decorator, [lhs, rhs])))
def testfilefixup(oldcontents, workingcopy, expectedcontents, fixups=None):
"""([str], str, [str], [(rev, a1, a2, b1, b2)]?) -> None
workingcopy is a string, of which every character denotes a single line.
oldcontents, expectedcontents are lists of strings, every character of
every string denots a single line.
if fixups is not None, it's the expected fixups list and will be checked.
"""
expectedcontents = insertreturns(expectedcontents)
oldcontents = insertreturns(oldcontents)
workingcopy = insertreturns(workingcopy)
state = absorb.filefixupstate(map(simplefctx, oldcontents))
state.diffwith(simplefctx(workingcopy))
if fixups is not None:
assertlistequal(state.fixups, fixups)
state.apply()
assertlistequal(state.finalcontents, expectedcontents, removereturns)
def buildcontents(linesrevs):
# linesrevs: [(linecontent : str, revs : [int])]
revs = set(itertools.chain(*[revs for line, revs in linesrevs]))
return [''] + [
''.join([l for l, rs in linesrevs if r in rs])
for r in sorted(revs)
]
# input case 0: one single commit
case0 = ['', '11']
# replace a single chunk
testfilefixup(case0, '', ['', ''])
testfilefixup(case0, '2', ['', '2'])
testfilefixup(case0, '22', ['', '22'])
testfilefixup(case0, '222', ['', '222'])
# input case 1: 3 lines, each commit adds one line
case1 = buildcontents([
('1', [1, 2, 3]),
('2', [ 2, 3]),
('3', [ 3]),
])
# 1:1 line mapping
testfilefixup(case1, '123', case1)
testfilefixup(case1, '12c', ['', '1', '12', '12c'])
testfilefixup(case1, '1b3', ['', '1', '1b', '1b3'])
testfilefixup(case1, '1bc', ['', '1', '1b', '1bc'])
testfilefixup(case1, 'a23', ['', 'a', 'a2', 'a23'])
testfilefixup(case1, 'a2c', ['', 'a', 'a2', 'a2c'])
testfilefixup(case1, 'ab3', ['', 'a', 'ab', 'ab3'])
testfilefixup(case1, 'abc', ['', 'a', 'ab', 'abc'])
# non 1:1 edits
testfilefixup(case1, 'abcd', case1)
testfilefixup(case1, 'ab', case1)
# deletion
testfilefixup(case1, '', ['', '', '', ''])
testfilefixup(case1, '1', ['', '1', '1', '1'])
testfilefixup(case1, '2', ['', '', '2', '2'])
testfilefixup(case1, '3', ['', '', '', '3'])
testfilefixup(case1, '13', ['', '1', '1', '13'])
# replaces
testfilefixup(case1, '1bb3', ['', '1', '1bb', '1bb3'])
# (confusing) replaces
testfilefixup(case1, '1bbb', case1)
testfilefixup(case1, 'bbbb', case1)
testfilefixup(case1, 'bbb3', case1)
testfilefixup(case1, '1b', case1)
testfilefixup(case1, 'bb', case1)
testfilefixup(case1, 'b3', case1)
# insertions at the beginning and the end
testfilefixup(case1, '123c', ['', '1', '12', '123c'])
testfilefixup(case1, 'a123', ['', 'a1', 'a12', 'a123'])
# (confusing) insertions
testfilefixup(case1, '1a23', case1)
testfilefixup(case1, '12b3', case1)
# input case 2: delete in the middle
case2 = buildcontents([
('11', [1, 2]),
('22', [1 ]),
('33', [1, 2]),
])
# deletion (optimize code should make it 2 chunks)
testfilefixup(case2, '', ['', '22', ''],
fixups=[(4, 0, 2, 0, 0), (4, 2, 4, 0, 0)])
# 1:1 line mapping
testfilefixup(case2, 'aaaa', ['', 'aa22aa', 'aaaa'])
# non 1:1 edits
# note: unlike case0, the chunk is not "continuous" and no edit allowed
testfilefixup(case2, 'aaa', case2)
# input case 3: rev 3 reverts rev 2
case3 = buildcontents([
('1', [1, 2, 3]),
('2', [ 2 ]),
('3', [1, 2, 3]),
])
# 1:1 line mapping
testfilefixup(case3, '13', case3)
testfilefixup(case3, '1b', ['', '1b', '12b', '1b'])
testfilefixup(case3, 'a3', ['', 'a3', 'a23', 'a3'])
testfilefixup(case3, 'ab', ['', 'ab', 'a2b', 'ab'])
# non 1:1 edits
testfilefixup(case3, 'a', case3)
testfilefixup(case3, 'abc', case3)
# deletion
testfilefixup(case3, '', ['', '', '2', ''])
# insertion
testfilefixup(case3, 'a13c', ['', 'a13c', 'a123c', 'a13c'])
# input case 4: a slightly complex case
case4 = buildcontents([
('1', [1, 2, 3]),
('2', [ 2, 3]),
('3', [1, 2, ]),
('4', [1, 3]),
('5', [ 3]),
('6', [ 2, 3]),
('7', [ 2 ]),
('8', [ 2, 3]),
('9', [ 3]),
])
testfilefixup(case4, '1245689', case4)
testfilefixup(case4, '1a2456bbb', case4)
testfilefixup(case4, '1abc5689', case4)
testfilefixup(case4, '1ab5689', ['', '134', '1a3678', '1ab5689'])
testfilefixup(case4, 'aa2bcd8ee', ['', 'aa34', 'aa23d78', 'aa2bcd8ee'])
testfilefixup(case4, 'aa2bcdd8ee',['', 'aa34', 'aa23678', 'aa24568ee'])
testfilefixup(case4, 'aaaaaa', case4)
testfilefixup(case4, 'aa258b', ['', 'aa34', 'aa2378', 'aa258b'])
testfilefixup(case4, '25bb', ['', '34', '23678', '25689'])
testfilefixup(case4, '27', ['', '34', '23678', '245689'])
testfilefixup(case4, '28', ['', '34', '2378', '28'])
testfilefixup(case4, '', ['', '34', '37', ''])
# input case 5: replace a small chunk which is near a deleted line
case5 = buildcontents([
('12', [1, 2]),
('3', [1]),
('4', [1, 2]),
])
testfilefixup(case5, '1cd4', ['', '1cd34', '1cd4'])
# input case 6: base "changeset" is immutable
case6 = ['1357', '0125678']
testfilefixup(case6, '0125678', case6)
testfilefixup(case6, '0a25678', case6)
testfilefixup(case6, '0a256b8', case6)
testfilefixup(case6, 'abcdefg', ['1357', 'a1c5e7g'])
testfilefixup(case6, 'abcdef', case6)
testfilefixup(case6, '', ['1357', '157'])
testfilefixup(case6, '0123456789', ['1357', '0123456789'])
# input case 7: change an empty file
case7 = ['']
testfilefixup(case7, '1', case7)

tests/test-absorb-phase.t

  • This file was added.
$ cat >> $HGRCPATH << EOF
> [extensions]
> absorb=
> drawdag=$RUNTESTDIR/drawdag.py
> EOF
$ hg init
$ hg debugdrawdag <<'EOS'
> C
> |
> B
> |
> A
> EOS
$ hg phase -r A --public -q
$ hg phase -r C --secret --force -q
$ hg update C -q
$ printf B1 > B
$ hg absorb -q
$ hg log -G -T '{desc} {phase}'
@ C secret
|
o B draft
|
o A public

tests/test-absorb-rename.t

  • This file was added.
$ cat >> $HGRCPATH << EOF
> [diff]
> git=1
> [extensions]
> absorb=
> EOF
$ sedi() { # workaround check-code
> pattern="$1"
> shift
> for i in "$@"; do
> sed "$pattern" "$i" > "$i".tmp
> mv "$i".tmp "$i"
> done
> }
rename a to b, then b to a
$ hg init repo1
$ cd repo1
$ echo 1 > a
$ hg ci -A a -m 1
$ hg mv a b
$ echo 2 >> b
$ hg ci -m 2
$ hg mv b a
$ echo 3 >> a
$ hg ci -m 3
$ hg annotate -ncf a
0 eff892de26ec a: 1
1 bf56e1f4f857 b: 2
2 0b888b00216c a: 3
$ sedi 's/$/a/' a
$ hg absorb -pq
showing changes for a
@@ -0,3 +0,3 @@
eff892d -1
bf56e1f -2
0b888b0 -3
eff892d +1a
bf56e1f +2a
0b888b0 +3a
$ hg status
$ hg annotate -ncf a
0 5d1c5620e6f2 a: 1a
1 9a14ffe67ae9 b: 2a
2 9191d121a268 a: 3a
when the first changeset is public
$ hg phase --public -r 0
$ sedi 's/a/A/' a
$ hg absorb -pq
showing changes for a
@@ -0,3 +0,3 @@
-1a
9a14ffe -2a
9191d12 -3a
+1A
9a14ffe +2A
9191d12 +3A
$ hg diff
diff --git a/a b/a
--- a/a
+++ b/a
@@ -1,3 +1,3 @@
-1a
+1A
2A
3A
copy a to b
$ cd ..
$ hg init repo2
$ cd repo2
$ echo 1 > a
$ hg ci -A a -m 1
$ hg cp a b
$ echo 2 >> b
$ hg ci -m 2
$ hg log -T '{rev}:{node|short} {desc}\n'
1:17b72129ab68 2
0:eff892de26ec 1
$ sedi 's/$/a/' a
$ sedi 's/$/b/' b
$ hg absorb -pq
showing changes for a
@@ -0,1 +0,1 @@
eff892d -1
eff892d +1a
showing changes for b
@@ -0,2 +0,2 @@
-1
17b7212 -2
+1b
17b7212 +2b
$ hg diff
diff --git a/b b/b
--- a/b
+++ b/b
@@ -1,2 +1,2 @@
-1
+1b
2b
copy b to a
$ cd ..
$ hg init repo3
$ cd repo3
$ echo 1 > b
$ hg ci -A b -m 1
$ hg cp b a
$ echo 2 >> a
$ hg ci -m 2
$ hg log -T '{rev}:{node|short} {desc}\n'
1:e62c256d8b24 2
0:55105f940d5c 1
$ sedi 's/$/a/' a
$ sedi 's/$/a/' b
$ hg absorb -pq
showing changes for a
@@ -0,2 +0,2 @@
-1
e62c256 -2
+1a
e62c256 +2a
showing changes for b
@@ -0,1 +0,1 @@
55105f9 -1
55105f9 +1a
$ hg diff
diff --git a/a b/a
--- a/a
+++ b/a
@@ -1,2 +1,2 @@
-1
+1a
2a
"move" b to both a and c, follow a - sorted alphabetically
$ cd ..
$ hg init repo4
$ cd repo4
$ echo 1 > b
$ hg ci -A b -m 1
$ hg cp b a
$ hg cp b c
$ hg rm b
$ echo 2 >> a
$ echo 3 >> c
$ hg commit -m cp
$ hg log -T '{rev}:{node|short} {desc}\n'
1:366daad8e679 cp
0:55105f940d5c 1
$ sedi 's/$/a/' a
$ sedi 's/$/c/' c
$ hg absorb -pq
showing changes for a
@@ -0,2 +0,2 @@
55105f9 -1
366daad -2
55105f9 +1a
366daad +2a
showing changes for c
@@ -0,2 +0,2 @@
-1
366daad -3
+1c
366daad +3c
$ hg log -G -p -T '{rev}:{node|short} {desc}\n'
@ 1:70606019f91b cp
| diff --git a/b b/a
| rename from b
| rename to a
| --- a/b
| +++ b/a
| @@ -1,1 +1,2 @@
| 1a
| +2a
| diff --git a/b b/c
| copy from b
| copy to c
| --- a/b
| +++ b/c
| @@ -1,1 +1,2 @@
| -1a
| +1
| +3c
|
o 0:bfb67c3539c1 1
diff --git a/b b/b
new file mode 100644
--- /dev/null
+++ b/b
@@ -0,0 +1,1 @@
+1a
run absorb again would apply the change to c
$ hg absorb -pq
showing changes for c
@@ -0,1 +0,1 @@
7060601 -1
7060601 +1c
$ hg log -G -p -T '{rev}:{node|short} {desc}\n'
@ 1:8bd536cce368 cp
| diff --git a/b b/a
| rename from b
| rename to a
| --- a/b
| +++ b/a
| @@ -1,1 +1,2 @@
| 1a
| +2a
| diff --git a/b b/c
| copy from b
| copy to c
| --- a/b
| +++ b/c
| @@ -1,1 +1,2 @@
| -1a
| +1c
| +3c
|
o 0:bfb67c3539c1 1
diff --git a/b b/b
new file mode 100644
--- /dev/null
+++ b/b
@@ -0,0 +1,1 @@
+1a
"move" b to a, c and d, follow d if a gets renamed to e, and c is deleted
$ cd ..
$ hg init repo5
$ cd repo5
$ echo 1 > b
$ hg ci -A b -m 1
$ hg cp b a
$ hg cp b c
$ hg cp b d
$ hg rm b
$ echo 2 >> a
$ echo 3 >> c
$ echo 4 >> d
$ hg commit -m cp
$ hg mv a e
$ hg rm c
$ hg commit -m mv
$ hg log -T '{rev}:{node|short} {desc}\n'
2:49911557c471 mv
1:7bc3d43ede83 cp
0:55105f940d5c 1
$ sedi 's/$/e/' e
$ sedi 's/$/d/' d
$ hg absorb -pq
showing changes for d
@@ -0,2 +0,2 @@
55105f9 -1
7bc3d43 -4
55105f9 +1d
7bc3d43 +4d
showing changes for e
@@ -0,2 +0,2 @@
-1
7bc3d43 -2
+1e
7bc3d43 +2e
$ hg diff
diff --git a/e b/e
--- a/e
+++ b/e
@@ -1,2 +1,2 @@
-1
+1e
2e
$ hg log -G -p -T '{rev}:{node|short} {desc}\n'
@ 2:34be9b0c786e mv
| diff --git a/c b/c
| deleted file mode 100644
| --- a/c
| +++ /dev/null
| @@ -1,2 +0,0 @@
| -1
| -3
| diff --git a/a b/e
| rename from a
| rename to e
|
o 1:13e56db5948d cp
| diff --git a/b b/a
| rename from b
| rename to a
| --- a/b
| +++ b/a
| @@ -1,1 +1,2 @@
| -1d
| +1
| +2e
| diff --git a/b b/c
| copy from b
| copy to c
| --- a/b
| +++ b/c
| @@ -1,1 +1,2 @@
| -1d
| +1
| +3
| diff --git a/b b/d
| copy from b
| copy to d
| --- a/b
| +++ b/d
| @@ -1,1 +1,2 @@
| 1d
| +4d
|
o 0:0037613a5dc6 1
diff --git a/b b/b
new file mode 100644
--- /dev/null
+++ b/b
@@ -0,0 +1,1 @@
+1d

tests/test-absorb-strip.t

  • This file was added.
Do not strip innocent children. See https://bitbucket.org/facebook/hg-experimental/issues/6/hg-absorb-merges-diverged-commits
$ cat >> $HGRCPATH << EOF
> [extensions]
> absorb=
> drawdag=$RUNTESTDIR/drawdag.py
> EOF
$ hg init
$ hg debugdrawdag << EOF
> E
> |
> D F
> |/
> C
> |
> B
> |
> A
> EOF
$ hg up E -q
$ echo 1 >> B
$ echo 2 >> D
$ hg absorb
saved backup bundle to * (glob)
2 of 2 chunk(s) applied
$ hg log -G -T '{desc}'
@ E
|
o D
|
o C
|
o B
|
| o F
| |
| o C
| |
| o B
|/
o A

tests/test-absorb.t

  • This file was added.
$ cat >> $HGRCPATH << EOF
> [extensions]
> absorb=
> EOF
$ sedi() { # workaround check-code
> pattern="$1"
> shift
> for i in "$@"; do
> sed "$pattern" "$i" > "$i".tmp
> mv "$i".tmp "$i"
> done
> }
$ hg init repo1
$ cd repo1
Do not crash with empty repo:
$ hg absorb
abort: no changeset to change
[255]
Make some commits:
$ for i in 1 2 3 4 5; do
> echo $i >> a
> hg commit -A a -m "commit $i" -q
> done
$ hg annotate a
0: 1
1: 2
2: 3
3: 4
4: 5
Change a few lines:
$ cat > a <<EOF
> 1a
> 2b
> 3
> 4d
> 5e
> EOF
Preview absorb changes:
$ hg absorb --print-changes --dry-run
showing changes for a
@@ -0,2 +0,2 @@
4ec16f8 -1
5c5f952 -2
4ec16f8 +1a
5c5f952 +2b
@@ -3,2 +3,2 @@
ad8b8b7 -4
4f55fa6 -5
ad8b8b7 +4d
4f55fa6 +5e
Run absorb:
$ hg absorb
saved backup bundle to * (glob)
2 of 2 chunk(s) applied
$ hg annotate a
0: 1a
1: 2b
2: 3
3: 4d
4: 5e
Delete a few lines and related commits will be removed if they will be empty:
$ cat > a <<EOF
> 2b
> 4d
> EOF
$ hg absorb
saved backup bundle to * (glob)
3 of 3 chunk(s) applied
$ hg annotate a
1: 2b
2: 4d
$ hg log -T '{rev} {desc}\n' -Gp
@ 2 commit 4
| diff -r 1cae118c7ed8 -r 58a62bade1c6 a
| --- a/a Thu Jan 01 00:00:00 1970 +0000
| +++ b/a Thu Jan 01 00:00:00 1970 +0000
| @@ -1,1 +1,2 @@
| 2b
| +4d
|
o 1 commit 2
| diff -r 84add69aeac0 -r 1cae118c7ed8 a
| --- a/a Thu Jan 01 00:00:00 1970 +0000
| +++ b/a Thu Jan 01 00:00:00 1970 +0000
| @@ -0,0 +1,1 @@
| +2b
|
o 0 commit 1
Non 1:1 map changes will be ignored:
$ echo 1 > a
$ hg absorb
nothing applied
[1]
Insertaions:
$ cat > a << EOF
> insert before 2b
> 2b
> 4d
> insert aftert 4d
> EOF
$ hg absorb -q
$ hg status
$ hg annotate a
1: insert before 2b
1: 2b
2: 4d
2: insert aftert 4d
Bookmarks are moved:
$ hg bookmark -r 1 b1
$ hg bookmark -r 2 b2
$ hg bookmark ba
$ hg bookmarks
b1 1:b35060a57a50
b2 2:946e4bc87915
* ba 2:946e4bc87915
$ sedi 's/insert/INSERT/' a
$ hg absorb -q
$ hg status
$ hg bookmarks
b1 1:a4183e9b3d31
b2 2:c9b20c925790
* ba 2:c9b20c925790
Non-mofified files are ignored:
$ touch b
$ hg commit -A b -m b
$ touch c
$ hg add c
$ hg rm b
$ hg absorb
nothing applied
[1]
$ sedi 's/INSERT/Insert/' a
$ hg absorb
saved backup bundle to * (glob)
2 of 2 chunk(s) applied
$ hg status
A c
R b
Public commits will not be changed:
$ hg phase -p 1
$ sedi 's/Insert/insert/' a
$ hg absorb -pn
showing changes for a
@@ -0,1 +0,1 @@
-Insert before 2b
+insert before 2b
@@ -3,1 +3,1 @@
85b4e0e -Insert aftert 4d
85b4e0e +insert aftert 4d
$ hg absorb
saved backup bundle to * (glob)
1 of 2 chunk(s) applied
$ hg diff -U 0
diff -r 1c8eadede62a a
--- a/a Thu Jan 01 00:00:00 1970 +0000
+++ b/a * (glob)
@@ -1,1 +1,1 @@
-Insert before 2b
+insert before 2b
$ hg annotate a
1: Insert before 2b
1: 2b
2: 4d
2: insert aftert 4d
Make working copy clean:
$ hg revert -q -C a b
$ hg forget c
$ rm c
$ hg status
Merge commit will not be changed:
$ echo 1 > m1
$ hg commit -A m1 -m m1
$ hg bookmark -q -i m1
$ hg update -q '.^'
$ echo 2 > m2
$ hg commit -q -A m2 -m m2
$ hg merge -q m1
$ hg commit -m merge
$ hg bookmark -d m1
$ hg log -G -T '{rev} {desc} {phase}\n'
@ 6 merge draft
|\
| o 5 m2 draft
| |
o | 4 m1 draft
|/
o 3 b draft
|
o 2 commit 4 draft
|
o 1 commit 2 public
|
o 0 commit 1 public
$ echo 2 >> m1
$ echo 2 >> m2
$ hg absorb
abort: no changeset to change
[255]
$ hg revert -q -C m1 m2
Use a new repo:
$ cd ..
$ hg init repo2
$ cd repo2
Make some commits to multiple files:
$ for f in a b; do
> for i in 1 2; do
> echo $f line $i >> $f
> hg commit -A $f -m "commit $f $i" -q
> done
> done
Use pattern to select files to be fixed up:
$ sedi 's/line/Line/' a b
$ hg status
M a
M b
$ hg absorb a
saved backup bundle to * (glob)
1 of 1 chunk(s) applied
$ hg status
M b
$ hg absorb --exclude b
nothing applied
[1]
$ hg absorb b
saved backup bundle to * (glob)
1 of 1 chunk(s) applied
$ hg status
$ cat a b
a Line 1
a Line 2
b Line 1
b Line 2
Test config option absorb.maxstacksize:
$ sedi 's/Line/line/' a b
$ hg log -T '{rev}:{node} {desc}\n'
3:712d16a8f445834e36145408eabc1d29df05ec09 commit b 2
2:74cfa6294160149d60adbf7582b99ce37a4597ec commit b 1
1:28f10dcf96158f84985358a2e5d5b3505ca69c22 commit a 2
0:f9a81da8dc53380ed91902e5b82c1b36255a4bd0 commit a 1
$ hg --config absorb.maxstacksize=1 absorb -pn
absorb: only the recent 1 changesets will be analysed
showing changes for a
@@ -0,2 +0,2 @@
-a Line 1
-a Line 2
+a line 1
+a line 2
showing changes for b
@@ -0,2 +0,2 @@
-b Line 1
712d16a -b Line 2
+b line 1
712d16a +b line 2
Test obsolete markers creation:
$ cat >> $HGRCPATH << EOF
> [experimental]
> evolution=createmarkers
> [absorb]
> addnoise=1
> EOF
$ hg --config absorb.maxstacksize=3 sf
absorb: only the recent 3 changesets will be analysed
2 of 2 chunk(s) applied
$ hg log -T '{rev}:{node|short} {desc} {get(extras, "absorb_source")}\n'
6:3dfde4199b46 commit b 2 712d16a8f445834e36145408eabc1d29df05ec09
5:99cfab7da5ff commit b 1 74cfa6294160149d60adbf7582b99ce37a4597ec
4:fec2b3bd9e08 commit a 2 28f10dcf96158f84985358a2e5d5b3505ca69c22
0:f9a81da8dc53 commit a 1
$ hg absorb
1 of 1 chunk(s) applied
$ hg log -T '{rev}:{node|short} {desc} {get(extras, "absorb_source")}\n'
10:e1c8c1e030a4 commit b 2 3dfde4199b4610ea6e3c6fa9f5bdad8939d69524
9:816c30955758 commit b 1 99cfab7da5ffdaf3b9fc6643b14333e194d87f46
8:5867d584106b commit a 2 fec2b3bd9e0834b7cb6a564348a0058171aed811
7:8c76602baf10 commit a 1 f9a81da8dc53380ed91902e5b82c1b36255a4bd0
Test config option absorb.amendflags and running as a sub command of amend:
$ cat >> $TESTTMP/dummyamend.py << EOF
> from mercurial import commands, registrar
> cmdtable = {}
> command = registrar.command(cmdtable)
> @command('amend', [], '')
> def amend(ui, repo, *pats, **opts):
> return 3
> EOF
$ cat >> $HGRCPATH << EOF
> [extensions]
> fbamend=$TESTTMP/dummyamend.py
> [absorb]
> amendflag = correlated
> EOF
$ hg amend -h
hg amend
(no help text available)
options:
--correlated incorporate corrections into stack. see 'hg help absorb' for
details
(some details hidden, use --verbose to show complete help)
$ $PYTHON -c 'print("".join(map(chr, range(0,3))))' > c
$ hg commit -A c -m 'c is a binary file'
$ echo c >> c
$ sedi $'2i\\\nINS\n' b
$ echo END >> b
$ hg rm a
$ hg amend --correlated
1 of 2 chunk(s) applied
# changes not applied and left in working directory:
# M b : 1 modified chunks were ignored
# M c : unsupported file type (ex. binary or link)
# R a : removed files were ignored
Executable files:
$ cat >> $HGRCPATH << EOF
> [diff]
> git=True
> EOF
$ cd ..
$ hg init repo3
$ cd repo3
$ echo > foo.py
$ chmod +x foo.py
$ hg add foo.py
$ hg commit -mfoo
$ echo bla > foo.py
$ hg absorb --dry-run --print-changes
showing changes for foo.py
@@ -0,1 +0,1 @@
99b4ae7 -
99b4ae7 +bla
$ hg absorb
1 of 1 chunk(s) applied
$ hg diff -c .
diff --git a/foo.py b/foo.py
new file mode 100755
--- /dev/null
+++ b/foo.py
@@ -0,0 +1,1 @@
+bla
$ hg diff
Remove lines may delete changesets:
$ cd ..
$ hg init repo4
$ cd repo4
$ cat > a <<EOF
> 1
> 2
> EOF
$ hg commit -m a12 -A a
$ cat > b <<EOF
> 1
> 2
> EOF
$ hg commit -m b12 -A b
$ echo 3 >> b
$ hg commit -m b3
$ echo 4 >> b
$ hg commit -m b4
$ echo 1 > b
$ echo 3 >> a
$ hg absorb -pn
showing changes for a
@@ -2,0 +2,1 @@
bfafb49 +3
showing changes for b
@@ -1,3 +1,0 @@
1154859 -2
30970db -3
a393a58 -4
$ hg absorb -v | grep became
bfafb49242db: 1 file(s) changed, became 1a2de97fc652
115485984805: 2 file(s) changed, became 0c930dfab74c
30970dbf7b40: became empty and was dropped
a393a58b9a85: became empty and was dropped
$ hg log -T '{rev} {desc}\n' -Gp
@ 5 b12
| diff --git a/b b/b
| new file mode 100644
| --- /dev/null
| +++ b/b
| @@ -0,0 +1,1 @@
| +1
|
o 4 a12
diff --git a/a b/a
new file mode 100644
--- /dev/null
+++ b/a
@@ -0,0 +1,3 @@
+1
+2
+3
Use revert to make the current change and its parent disappear.
This should move us to the non-obsolete ancestor.
$ cd ..
$ hg init repo5
$ cd repo5
$ cat > a <<EOF
> 1
> 2
> EOF
$ hg commit -m a12 -A a
$ hg id
bfafb49242db tip
$ echo 3 >> a
$ hg commit -m a123 a
$ echo 4 >> a
$ hg commit -m a1234 a
$ hg id
82dbe7fd19f0 tip
$ hg revert -r 0 a
$ hg absorb -pn
showing changes for a
@@ -2,2 +2,0 @@
f1c23dd -3
82dbe7f -4
$ hg absorb --verbose
f1c23dd5d08d: became empty and was dropped
82dbe7fd19f0: became empty and was dropped
a: 1 of 1 chunk(s) applied
$ hg id
bfafb49242db tip