This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11025

ci-windows: introduce manual windows CI
ClosedPublic

Authored by Alphare on Jul 8 2021, 11:53 AM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHGbc01b8ad6134: ci-windows: introduce manual windows CI
Summary

Note: most of the foundational work of this commit was done by Matt Harbison,
but enough has changed that I don't feel comfortable sending this patch as his.

In our quest to remove Python 2 from Mercurial, we need to get Windows CI going
to check that Python 3 support is up to an acceptable standard.
This is the first step: adding a manual step to run a full Windows CI, with a
certain definition of "full" since some things are not installed yet, like SVN.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Alphare created this revision.Jul 8 2021, 11:53 AM
baymax updated this revision to Diff 29031.Jul 8 2021, 7:59 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

baymax updated this revision to Diff 29091.Jul 9 2021, 10:26 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

baymax updated this revision to Diff 29112.Jul 10 2021, 12:09 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

mharbison72 added a subscriber: mharbison72.Jul 10 2021, 7:18 PM

I assume the various things that I was installing in before_script are now baked into to the docker image/runner host? (That's good, it was my intention. But the TEMP variable hack made me wonder if maybe it wasn't yet.)

I'm also a bit surprised you haven't hit any issues by leaving out the make local bits.

contrib/heptapod-ci.yml
152–153

Do we also need to mkdir -p this?

172

What generates this file?

marmoute added a subscriber: marmoute.Jul 11 2021, 7:15 AM
In D11025#169418, @mharbison72 wrote:

I assume the various things that I was installing in before_script are now baked into to the docker image/runner host? (That's good, it was my intention. But the TEMP variable hack made me wonder if maybe it wasn't yet.)

Yes they are.

I'm also a bit surprised you haven't hit any issues by leaving out the make local bits.

I don't know what error you were getting without --local, but the latest run are doing without it. I made a bunch of fixes to run-tests.py that might have helped (and there will be more)

mharbison72 added a comment.Jul 11 2021, 2:02 PM
In D11025#169424, @marmoute wrote:
In D11025#169418, @mharbison72 wrote:

I assume the various things that I was installing in before_script are now baked into to the docker image/runner host? (That's good, it was my intention. But the TEMP variable hack made me wonder if maybe it wasn't yet.)

Yes they are.

I'm also a bit surprised you haven't hit any issues by leaving out the make local bits.

I don't know what error you were getting without --local, but the latest run are doing without it. I made a bunch of fixes to run-tests.py that might have helped (and there will be more)

IDR at this point, but it was scattered throughout the tests. I thought maybe it was not building the C extensions and falling back to pure, and having some differing output.

On a related note, it might be good if we have some way to lock in using cext on Windows (assuming we don't want the overhead of also running pure tests there too). I'm not sure why anyone would use pure there since there are binary installers, and we had a nasty bug in pure relating to corrupting the manifest with a bunch of file removes in 4.7-4.9 or so. I guess there was a bug in the installer that caused it to not load cext, and that's how it was triggered on Windows in the field. But that shouldn't block this.

Alphare added a commit: rHGbc01b8ad6134: ci-windows: introduce manual windows CI.Jul 11 2021, 2:07 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHGbc01b8ad6134: ci-windows: introduce manual windows CI (authored by Alphare). · Explain Why
This revision was automatically updated to reflect the committed changes.
marmoute added a comment.Jul 11 2021, 4:55 PM
In D11025#169487, @mharbison72 wrote:

On a related note, it might be good if we have some way to lock in using cext on Windows (assuming we don't want the overhead of also running pure tests there too).

We I guess we could lock the policy to "c" instead of "allow" on windows. Would you send a patch ?

Revision Contents
Changeset List

PathPackages
Mcontrib/heptapod-ci.yml (32 lines)

Diff 29179

contrib/heptapod-ci.yml

- hg -R /tmp/mercurial-ci/ update `hg log --rev '.' --template '{node}'` - hg -R /tmp/mercurial-ci/ update `hg log --rev '.' --template '{node}'`
- cd /tmp/mercurial-ci/ - cd /tmp/mercurial-ci/
- make local PYTHON=$PYTHON - make local PYTHON=$PYTHON
- $PYTHON -m pip install --user -U pytype==2021.04.15 - $PYTHON -m pip install --user -U pytype==2021.04.15
variables: variables:
RUNTEST_ARGS: " --allow-slow-tests tests/test-check-pytype.t" RUNTEST_ARGS: " --allow-slow-tests tests/test-check-pytype.t"
PYTHON: python3 PYTHON: python3
TEST_HGMODULEPOLICY: "c" TEST_HGMODULEPOLICY: "c"
# `sh.exe --login` sets a couple of extra environment variables that are defined
# in the MinGW shell, but switches CWD to /home/$username. The previous value
# is stored in OLDPWD. Of the added variables, MSYSTEM is crucial to running
# run-tests.py- it is needed to make run-tests.py generate a `python3` script
# that satisfies the various shebang lines and delegates to `py -3`.
.window_runtests_template: &windows_runtests
stage: tests
before_script:
# Temporary until this is adjusted in the environment
- $Env:TEMP="C:/Temp"
- $Env:TMP="C:/Temp"
mharbison72Unsubmitted
Not Done

Do we also need to mkdir -p this?

mharbison72: Do we also need to `mkdir -p` this?
# TODO: find/install cvs, bzr, perforce, gpg, sqlite3
script:
- echo "Entering script section"
- echo "python used, $Env:PYTHON"
- Invoke-Expression "$Env:PYTHON -V"
- Invoke-Expression "$Env:PYTHON -m black --version"
- echo "$Env:RUNTEST_ARGS"
- C:/MinGW/msys/1.0/bin/sh.exe --login -c 'cd "$OLDPWD" && HGTESTS_ALLOW_NETIO="$TEST_HGTESTS_ALLOW_NETIO" HGMODULEPOLICY="$TEST_HGMODULEPOLICY" $PYTHON tests/run-tests.py --color=always $RUNTEST_ARGS'
windows-py3:
<<: *windows_runtests
when: manual
tags:
- windows
timeout: 2h
variables:
RUNTEST_ARGS: "--blacklist /tmp/check-tests.txt"
mharbison72Unsubmitted
Not Done

What generates this file?

mharbison72: What generates this file?
PYTHON: py -3