This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3866

histedit: use cbor to write histedit-state file
Changes PlannedPublic

Authored by pulkit on Jun 30 2018, 11:10 AM.

Details

Reviewers
durin42
Group Reviewers
hg-reviewers
Summary

This patch starts using state.cmdstate.write() which uses cbor as serialization
format to write histedit-state files.

Compatibility for reading old state files using new code is maintained.

Some changes in test which depended on the format of histedit state file.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

pulkit created this revision.Jun 30 2018, 11:10 AM
yuja added a subscriber: yuja.Jul 1 2018, 8:35 AM

Queued the first 4 patches, thanks.

$ HGEDITOR="sh $TESTTMP/editplan.sh" hg histedit --edit-plan
$ cat .hg/histedit-state
  • v1
  • 055a42cdd88768532f9cf79daa407fc8d138de9b
  • 3c6a8ed2ebe862cc949d2caa30775dd6f16fb799
  • False
  • 3
  • drop
  • e860deea161a2f77de56603b340ebbb4536308ae
  • drop
  • 652413bf663ef2a641cab26574e46d5f5a64a55a
  • drop
  • 3c6a8ed2ebe862cc949d2caa30775dd6f16fb799
  • 0
  • strip-backup/177f92b77385-0ebe6a8f-histedit.hg

+ 2

I know anything other than "v1" will make old hg crash, but I think the
version syntax should stay "v%d" (i.e. "v2" in this case.)

def _write(self, fp):
  • fp.write('v1\n')
  • fp.write('%s\n' % node.hex(self.parentctxnode))
  • fp.write('%s\n' % node.hex(self.topmost))
  • fp.write('%s\n' % ('True' if self.keep else 'False'))
  • fp.write('%d\n' % len(self.actions))

+ data = {'parentctxnode': self.parentctxnode,
+ 'topmost': self.topmost,
+ 'keep': self.keep,
+ 'backupfile': self.backupfile,
+ 'replacements': self.replacements}
+ rules = []

for action in self.actions:
  • fp.write('%s\n' % action.tostate())
  • fp.write('%d\n' % len(self.replacements))
  • for replacement in self.replacements:
  • fp.write('%s%s\n' % (node.hex(replacement[0]), ''.join(node.hex(r)
  • for r in replacement[1])))
  • backupfile = self.backupfile
  • if not backupfile:
  • backupfile = ''
  • fp.write('%s\n' % backupfile)

+ rules.append(action.tostate().split('\n'))
+ rules = "\n".join(["%s %s" % (verb, rest) for [verb, rest] in rules])

Isn't it better to serialize [(verb, rest)] as is since CBOR can store that?

pulkit added a comment.Jul 1 2018, 5:11 PM
In D3866#60421, @yuja wrote:

Queued the first 4 patches, thanks.

$ HGEDITOR="sh $TESTTMP/editplan.sh" hg histedit --edit-plan
$ cat .hg/histedit-state
  • v1
  • 055a42cdd88768532f9cf79daa407fc8d138de9b
  • 3c6a8ed2ebe862cc949d2caa30775dd6f16fb799
  • False
  • 3
  • drop
  • e860deea161a2f77de56603b340ebbb4536308ae
  • drop
  • 652413bf663ef2a641cab26574e46d5f5a64a55a
  • drop
  • 3c6a8ed2ebe862cc949d2caa30775dd6f16fb799
  • 0
  • strip-backup/177f92b77385-0ebe6a8f-histedit.hg

+ 2

I know anything other than "v1" will make old hg crash, but I think the
version syntax should stay "v%d" (i.e. "v2" in this case.)

Yeah, I think we need to pass version number in the state.cmdstate.read(). Supporting rebasestate will also need similar tweak.

def _write(self, fp):
  • fp.write('v1\n')
  • fp.write('%s\n' % node.hex(self.parentctxnode))
  • fp.write('%s\n' % node.hex(self.topmost))
  • fp.write('%s\n' % ('True' if self.keep else 'False'))
  • fp.write('%d\n' % len(self.actions))

+ data = {'parentctxnode': self.parentctxnode,
+ 'topmost': self.topmost,
+ 'keep': self.keep,
+ 'backupfile': self.backupfile,
+ 'replacements': self.replacements}
+ rules = []

for action in self.actions:
  • fp.write('%s\n' % action.tostate())
  • fp.write('%d\n' % len(self.replacements))
  • for replacement in self.replacements:
  • fp.write('%s%s\n' % (node.hex(replacement[0]), ''.join(node.hex(r)
  • for r in replacement[1])))
  • backupfile = self.backupfile
  • if not backupfile:
  • backupfile = ''
  • fp.write('%s\n' % backupfile)

+ rules.append(action.tostate().split('\n'))
+ rules = "\n".join(["%s %s" % (verb, rest) for [verb, rest] in rules])

Isn't it better to serialize [(verb, rest)] as is since CBOR can store that?

Yeah, it's better. I will do that.

pulkit planned changes to this revision.Jul 9 2018, 2:31 PM

I am not confident to land some state format changes at this point in the cycle. Let's do it early in next cycle.

Revision Contents
Changeset List

Diff 9378

hgext/histedit.py

actions = parserules(data['rules'], self) actions = parserules(data['rules'], self)
self.actions = actions self.actions = actions
self.keep = data['keep'] self.keep = data['keep']
self.topmost = data['topmost'] self.topmost = data['topmost']
self.replacements = data['replacements'] self.replacements = data['replacements']
self.backupfile = data['backupfile'] self.backupfile = data['backupfile']
def _read(self): def _read(self):
try:
return self.stateobj.read()
except error.CorruptedState:
pass
fp = self.repo.vfs.read('histedit-state') fp = self.repo.vfs.read('histedit-state')
if fp.startswith('v1\n'): if fp.startswith('v1\n'):
data = self._load() data = self._load()
parentctxnode, rules, keep, topmost, replacements, backupfile = data parentctxnode, rules, keep, topmost, replacements, backupfile = data
else: else:
data = pickle.loads(fp) data = pickle.loads(fp)
parentctxnode, rules, keep, topmost, replacements = data parentctxnode, rules, keep, topmost, replacements = data
backupfile = None backupfile = None
rules = "\n".join(["%s %s" % (verb, rest) for [verb, rest] in rules]) rules = "\n".join(["%s %s" % (verb, rest) for [verb, rest] in rules])
return {'parentctxnode': parentctxnode, "rules": rules, "keep": keep, return {'parentctxnode': parentctxnode, "rules": rules, "keep": keep,
"topmost": topmost, "replacements": replacements, "topmost": topmost, "replacements": replacements,
"backupfile": backupfile} "backupfile": backupfile}
def write(self, tr=None): def write(self, tr=None):
if tr: if tr:
tr.addfilegenerator('histedit-state', ('histedit-state',), tr.addfilegenerator('histedit-state', ('histedit-state',),
self._write, location='plain') self._write, location='plain')
else: else:
with self.repo.vfs("histedit-state", "w") as f: with self.repo.vfs("histedit-state", "w") as f:
self._write(f) self._write(f)
def _write(self, fp): def _write(self, fp):
fp.write('v1\n') data = {'parentctxnode': self.parentctxnode,
fp.write('%s\n' % node.hex(self.parentctxnode)) 'topmost': self.topmost,
fp.write('%s\n' % node.hex(self.topmost)) 'keep': self.keep,
fp.write('%s\n' % ('True' if self.keep else 'False')) 'backupfile': self.backupfile,
fp.write('%d\n' % len(self.actions)) 'replacements': self.replacements}
rules = []
for action in self.actions: for action in self.actions:
fp.write('%s\n' % action.tostate()) rules.append(action.tostate().split('\n'))
fp.write('%d\n' % len(self.replacements)) rules = "\n".join(["%s %s" % (verb, rest) for [verb, rest] in rules])
for replacement in self.replacements: data['rules'] = rules
fp.write('%s%s\n' % (node.hex(replacement[0]), ''.join(node.hex(r) self.stateobj.save(2, data)
for r in replacement[1])))
backupfile = self.backupfile
if not backupfile:
backupfile = ''
fp.write('%s\n' % backupfile)
def _load(self): def _load(self):
fp = self.repo.vfs('histedit-state', 'r') fp = self.repo.vfs('histedit-state', 'r')
lines = [l[:-1] for l in fp.readlines()] lines = [l[:-1] for l in fp.readlines()]
index = 0 index = 0
lines[index] # version number lines[index] # version number
index += 1 index += 1

tests/test-histedit-arguments.t

---------------------------------------------- ----------------------------------------------
$ hg histedit . -q --commands - << EOF $ hg histedit . -q --commands - << EOF
> edit 8fda0c726bf2 6 x > edit 8fda0c726bf2 6 x
> EOF > EOF
Editing (8fda0c726bf2), you may commit or record as needed now. Editing (8fda0c726bf2), you may commit or record as needed now.
(hg histedit --continue to resume) (hg histedit --continue to resume)
[1] [1]
Corrupt histedit state file Corrupt histedit state file
$ sed 's/8fda0c726bf2/123456789012/' .hg/histedit-state > ../corrupt-histedit $ sed 's/8fda0c726bf2/123456789012/g; s/\x8f\xda/\x23\x56/g' .hg/histedit-state > ../corrupt-histedit
$ mv ../corrupt-histedit .hg/histedit-state $ mv ../corrupt-histedit .hg/histedit-state
$ hg histedit --abort $ hg histedit --abort
warning: encountered an exception during histedit --abort; the repository may not have been completely cleaned up warning: encountered an exception during histedit --abort; the repository may not have been completely cleaned up
abort: $TESTTMP/foo/.hg/strip-backup/*-histedit.hg: $ENOENT$ (glob) (windows !) abort: $TESTTMP/foo/.hg/strip-backup/*-histedit.hg: $ENOENT$ (glob) (windows !)
abort: $ENOENT$: $TESTTMP/foo/.hg/strip-backup/*-histedit.hg (glob) (no-windows !) abort: $ENOENT$: $TESTTMP/foo/.hg/strip-backup/*-histedit.hg (glob) (no-windows !)
[255] [255]
Histedit state has been exited Histedit state has been exited
$ hg summary -q $ hg summary -q

tests/test-histedit-edit.t

> cat > \$1 <<EOF2 > cat > \$1 <<EOF2
> drop e860deea161a e > drop e860deea161a e
> drop 652413bf663e f > drop 652413bf663e f
> drop 3c6a8ed2ebe8 g > drop 3c6a8ed2ebe8 g
> EOF2 > EOF2
> EOF > EOF
$ HGEDITOR="sh $TESTTMP/editplan.sh" hg histedit --edit-plan $ HGEDITOR="sh $TESTTMP/editplan.sh" hg histedit --edit-plan
$ cat .hg/histedit-state $ cat .hg/histedit-state
v1 2
055a42cdd88768532f9cf79daa407fc8d138de9b \xa6Dkeep\xf4ErulesX\x89drop e860deea161a2f77de56603b340ebbb4536308ae (esc)
3c6a8ed2ebe862cc949d2caa30775dd6f16fb799 drop 652413bf663ef2a641cab26574e46d5f5a64a55a
False drop 3c6a8ed2ebe862cc949d2caa30775dd6f16fb799GtopmostT<j\x8e\xd2\xeb\xe8b\xcc\x94\x9d,\xaa0w]\xd6\xf1o\xb7\x99JbackupfileX.strip-backup/177f92b77385-0ebe6a8f-histedit.hgLreplacements\x80MparentctxnodeT\x05ZB\xcd\xd8\x87hS/\x9c\xf7\x9d\xaa@\x7f\xc8\xd18\xde\x9b (no-eol) (esc)
3
drop
e860deea161a2f77de56603b340ebbb4536308ae
drop
652413bf663ef2a641cab26574e46d5f5a64a55a
drop
3c6a8ed2ebe862cc949d2caa30775dd6f16fb799
0
strip-backup/177f92b77385-0ebe6a8f-histedit.hg
edit the plan via --commands edit the plan via --commands
$ hg histedit --edit-plan --commands - 2>&1 << EOF $ hg histedit --edit-plan --commands - 2>&1 << EOF
> edit e860deea161a e > edit e860deea161a e
> pick 652413bf663e f > pick 652413bf663e f
> drop 3c6a8ed2ebe8 g > drop 3c6a8ed2ebe8 g
> EOF > EOF
$ cat .hg/histedit-state $ cat .hg/histedit-state
v1 2
055a42cdd88768532f9cf79daa407fc8d138de9b \xa6Dkeep\xf4ErulesX\x89edit e860deea161a2f77de56603b340ebbb4536308ae (esc)
3c6a8ed2ebe862cc949d2caa30775dd6f16fb799 pick 652413bf663ef2a641cab26574e46d5f5a64a55a
False drop 3c6a8ed2ebe862cc949d2caa30775dd6f16fb799GtopmostT<j\x8e\xd2\xeb\xe8b\xcc\x94\x9d,\xaa0w]\xd6\xf1o\xb7\x99JbackupfileX.strip-backup/177f92b77385-0ebe6a8f-histedit.hgLreplacements\x80MparentctxnodeT\x05ZB\xcd\xd8\x87hS/\x9c\xf7\x9d\xaa@\x7f\xc8\xd18\xde\x9b (no-eol) (esc)
3
edit
e860deea161a2f77de56603b340ebbb4536308ae
pick
652413bf663ef2a641cab26574e46d5f5a64a55a
drop
3c6a8ed2ebe862cc949d2caa30775dd6f16fb799
0
strip-backup/177f92b77385-0ebe6a8f-histedit.hg
Go at a random point and try to continue Go at a random point and try to continue
$ hg id -n $ hg id -n
3+ 3+
$ hg up 0 $ hg up 0
abort: histedit in progress abort: histedit in progress
(use 'hg histedit --continue' or 'hg histedit --abort') (use 'hg histedit --continue' or 'hg histedit --abort')