( )⚙ D10947 test: use a python script in `test-transaction-rollback-on-sigpipe.t`

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D10947

test: use a python script in `test-transaction-rollback-on-sigpipe.t`
ClosedPublic

Authored by marmoute on Jul 2 2021, 9:42 PM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHGed81f2be5527: test: use a python script in `test-transaction-rollback-on-sigpipe.t`
Summary

This still does not work on Windows, but at least this is a python script now.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

marmoute created this revision.Jul 2 2021, 9:42 PM
Alphare accepted this revision.Jul 5 2021, 5:46 AM
Alphare added a subscriber: Alphare.

I'm not in love with this, but I don't have a much better solution for (eventual) Windows compatibility and debugability.

This revision is now accepted and ready to land.Jul 5 2021, 5:46 AM
Alphare added a comment.Jul 5 2021, 9:11 AM

Btw, does this still require #require bash?

marmoute added a comment.Jul 5 2021, 9:17 AM
In D10947#167837, @Alphare wrote:

Btw, does this still require #require bash?

I don't think it does.

baymax updated this revision to Diff 28817.Jul 5 2021, 5:04 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

baymax updated this revision to Diff 29002.Jul 8 2021, 10:22 AM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

Revision Contents
Changeset List

Diff 28817

tests/test-transaction-rollback-on-sigpipe.t

#require bash
Test that, when an hg push is interrupted and the remote side receives SIGPIPE, Test that, when an hg push is interrupted and the remote side receives SIGPIPE,
the remote hg is able to successfully roll back the transaction. the remote hg is able to successfully roll back the transaction.
$ hg init -q remote $ hg init -q remote
$ hg clone -e "\"$PYTHON\" \"$TESTDIR/dummyssh\"" -q ssh://user@dummy/`pwd`/remote local $ hg clone -e "\"$PYTHON\" \"$RUNTESTDIR/dummyssh\"" -q ssh://user@dummy/`pwd`/remote local
$ SIGPIPE_REMOTE_DEBUG_FILE="$TESTTMP/DEBUGFILE"
$ pidfile=`pwd`/pidfile $ SYNCFILE1="$TESTTMP/SYNCFILE1"
$ >$pidfile $ SYNCFILE2="$TESTTMP/SYNCFILE2"
$ export SIGPIPE_REMOTE_DEBUG_FILE
$ script() { $ export SYNCFILE1
> cat >"$1" $ export SYNCFILE2
> chmod +x "$1" $ PYTHONUNBUFFERED=1
> } $ export PYTHONUNBUFFERED
On the remote end, run hg, piping stdout and stderr through processes that we On the remote end, run hg, piping stdout and stderr through processes that we
know the PIDs of. We will later kill these to simulate an ssh client know the PIDs of. We will later kill these to simulate an ssh client
disconnecting. disconnecting.
$ killable_pipe=`pwd`/killable_pipe.sh $ remotecmd="$RUNTESTDIR/testlib/sigpipe-remote.py"
$ script $killable_pipe <<EOF
> #!/usr/bin/env bash
> echo \$\$ >> $pidfile
> exec cat
> EOF
$ remotecmd=`pwd`/remotecmd.sh
$ script $remotecmd <<EOF
> #!/usr/bin/env bash
> hg "\$@" 1> >($killable_pipe) 2> >($killable_pipe >&2)
> EOF
In the pretxnchangegroup hook, kill the PIDs recorded above to simulate ssh In the pretxnchangegroup hook, kill the PIDs recorded above to simulate ssh
disconnecting. Then exit nonzero, to force a transaction rollback. disconnecting. Then exit nonzero, to force a transaction rollback.
$ hook_script=`pwd`/pretxnchangegroup.sh
$ script $hook_script <<EOF
> #!/usr/bin/env bash
> for pid in \$(cat $pidfile) ; do
> kill \$pid
> while kill -0 \$pid 2>/dev/null ; do
> sleep 0.1
> done
> done
> exit 1
> EOF
$ cat >remote/.hg/hgrc <<EOF $ cat >remote/.hg/hgrc <<EOF
> [hooks] > [hooks]
> pretxnchangegroup.00-break-things=$hook_script > pretxnchangegroup.00-break-things="$RUNTESTDIR/testlib/wait-on-file" 10 "$SYNCFILE2" "$SYNCFILE1"
> pretxnchangegroup.01-output-things=echo "some remote output to be forward to the closed pipe" > pretxnchangegroup.01-output-things=echo "some remote output to be forward to the closed pipe"
> EOF > EOF
$ hg --cwd ./remote tip -T '{node|short}\n' $ hg --cwd ./remote tip -T '{node|short}\n'
000000000000 000000000000
$ cd local $ cd local
$ echo foo > foo ; hg commit -qAm "commit" $ echo foo > foo ; hg commit -qAm "commit"
$ hg push -q -e "\"$PYTHON\" \"$TESTDIR/dummyssh\"" --remotecmd $remotecmd 2>&1 | grep -v $killable_pipe $ hg push -e "\"$PYTHON\" \"$TESTDIR/dummyssh\"" --remotecmd "$remotecmd"
pushing to ssh://user@dummy/$TESTTMP/remote
searching for changes
remote: adding changesets
remote: adding manifests
remote: adding file changes
abort: stream ended unexpectedly (got 0 bytes, expected 4) abort: stream ended unexpectedly (got 0 bytes, expected 4)
[255]
$ cat $SIGPIPE_REMOTE_DEBUG_FILE
SIGPIPE-HELPER: Starting
SIGPIPE-HELPER: Mercurial started
SIGPIPE-HELPER: Redirection in place
SIGPIPE-HELPER: SYNCFILE1 detected
SIGPIPE-HELPER: pipes closed
SIGPIPE-HELPER: creating SYNCFILE2
SIGPIPE-HELPER: Shutting down
SIGPIPE-HELPER: Server process terminated
SIGPIPE-HELPER: Shut down
The remote should be left in a good state The remote should be left in a good state
$ hg --cwd ../remote tip -T '{node|short}\n' $ hg --cwd ../remote tip -T '{node|short}\n'
000000000000 000000000000
$ hg --cwd ../remote recover $ hg --cwd ../remote recover
no interrupted transaction available no interrupted transaction available
[1] [1]

tests/testlib/sigpipe-remote.py

  • This file was added.
PropertyOld ValueNew Value
File Modenull100755
#!/usr/bin/env python3
from __future__ import print_function
import os
import subprocess
import sys
import threading
import time
# we cannot use mercurial.testing as long as python2 is not dropped as the test will only install the mercurial module for python2 in python2 run
def _timeout_factor():
"""return the current modification to timeout"""
default = int(os.environ.get('HGTEST_TIMEOUT_DEFAULT', 360))
current = int(os.environ.get('HGTEST_TIMEOUT', default))
if current == 0:
return 1
return current / float(default)
def wait_file(path, timeout=10):
timeout *= _timeout_factor()
start = time.time()
while not os.path.exists(path):
if (time.time() - start) > timeout:
raise RuntimeError(b"timed out waiting for file: %s" % path)
time.sleep(0.01)
def write_file(path, content=b''):
with open(path, 'wb') as f:
f.write(content)
# end of mercurial.testing content
if sys.version_info[0] < 3:
print('SIGPIPE-HELPER: script should run with Python 3', file=sys.stderr)
sys.exit(255)
def sysbytes(s):
return s.encode('utf-8')
def sysstr(s):
return s.decode('latin-1')
piped_stdout = os.pipe2(os.O_NONBLOCK | os.O_CLOEXEC)
piped_stderr = os.pipe2(os.O_NONBLOCK | os.O_CLOEXEC)
stdout_writer = os.fdopen(piped_stdout[1], "rb")
stdout_reader = os.fdopen(piped_stdout[0], "rb")
stderr_writer = os.fdopen(piped_stderr[1], "rb")
stderr_reader = os.fdopen(piped_stderr[0], "rb")
DEBUG_FILE = os.environ.get('SIGPIPE_REMOTE_DEBUG_FILE')
if DEBUG_FILE is None:
debug_stream = sys.stderr.buffer
else:
debug_stream = open(DEBUG_FILE, 'bw', buffering=0)
SYNCFILE1 = os.environ.get('SYNCFILE1')
SYNCFILE2 = os.environ.get('SYNCFILE2')
if SYNCFILE1 is None:
print('SIGPIPE-HELPER: missing variable $SYNCFILE1', file=sys.stderr)
sys.exit(255)
if SYNCFILE2 is None:
print('SIGPIPE-HELPER: missing variable $SYNCFILE2', file=sys.stderr)
sys.exit(255)
debug_stream.write(b'SIGPIPE-HELPER: Starting\n')
TESTLIB_DIR = os.path.dirname(sys.argv[0])
WAIT_SCRIPT = os.path.join(TESTLIB_DIR, 'wait-on-file')
hooks_cmd = '%s 10 %s %s'
hooks_cmd %= (
WAIT_SCRIPT,
SYNCFILE2,
SYNCFILE1,
)
cmd = ['hg']
cmd += sys.argv[1:]
sub = subprocess.Popen(
cmd,
bufsize=0,
close_fds=True,
stdin=sys.stdin,
stdout=stdout_writer,
stderr=stderr_writer,
)
debug_stream.write(b'SIGPIPE-HELPER: Mercurial started\n')
shut_down = threading.Event()
close_lock = threading.Lock()
def _read(stream):
try:
return stream.read()
except ValueError:
# read on closed file
return None
def forward_stdout():
while not shut_down.is_set():
c = _read(stdout_reader)
while c is not None:
sys.stdout.buffer.write(c)
c = _read(stdout_reader)
time.sleep(0.001)
with close_lock:
if not stdout_reader.closed:
stdout_reader.close()
debug_stream.write(b'SIGPIPE-HELPER: stdout closed\n')
def forward_stderr():
while not shut_down.is_set():
c = _read(stderr_reader)
if c is not None:
sys.stderr.buffer.write(c)
c = _read(stderr_reader)
time.sleep(0.001)
with close_lock:
if not stderr_reader.closed:
stderr_reader.close()
debug_stream.write(b'SIGPIPE-HELPER: stderr closed\n')
stdout_thread = threading.Thread(target=forward_stdout, daemon=True)
stderr_thread = threading.Thread(target=forward_stderr, daemon=True)
try:
stdout_thread.start()
stderr_thread.start()
debug_stream.write(b'SIGPIPE-HELPER: Redirection in place\n')
try:
wait_file(sysbytes(SYNCFILE1))
except RuntimeError as exc:
msg = sysbytes(str(exc))
debug_stream.write(b'SIGPIPE-HELPER: wait failed: %s\n' % msg)
else:
debug_stream.write(b'SIGPIPE-HELPER: SYNCFILE1 detected\n')
with close_lock:
if not stdout_reader.closed:
stdout_reader.close()
if not stderr_reader.closed:
stderr_reader.close()
sys.stdin.close()
debug_stream.write(b'SIGPIPE-HELPER: pipes closed\n')
debug_stream.write(b'SIGPIPE-HELPER: creating SYNCFILE2\n')
write_file(sysbytes(SYNCFILE2))
finally:
debug_stream.write(b'SIGPIPE-HELPER: Shutting down\n')
shut_down.set()
if not sys.stdin.closed:
sys.stdin.close()
try:
sub.wait(timeout=30)
except subprocess.TimeoutExpired:
msg = b'SIGPIPE-HELPER: Server process failed to terminate\n'
debug_stream.write(msg)
else:
debug_stream.write(b'SIGPIPE-HELPER: Server process terminated\n')
debug_stream.write(b'SIGPIPE-HELPER: Shut down\n')