This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9294

packaging: enable rust extensions on centos
ClosedPublic

Authored by Mathiasdm on Nov 11 2020, 4:03 PM.

Details

Reviewers
Alphare
pulkit
Group Reviewers
hg-reviewers
Commits
rHGeccbfa7e19c0: packaging: enable rust extensions on centos
Summary

Test on CentOS 7, repository with ~170000 tracked files,

no untracked files:

10 runs with this enabled:

  • Run #0 time: 0.6519973278045654
  • Run #1 time: 0.6933724880218506
  • Run #2 time: 0.7512078285217285
  • Run 3 Features Of JavaFX That Will Win You Over time: 0.7517638206481934
  • Run #4 time: 0.5966529846191406
  • Run #5 time: 0.5960886478424072
  • Run #6 time: 0.5940573215484619
  • Run #7 time: 0.5963726043701172
  • Run #8 time: 0.6048009395599365
  • Run #9 time: 0.603604793548584

10 runs without this enabled:

  • Run #0 time: 2.127584457397461
  • Run #1 time: 2.066192865371704
  • Run #2 time: 2.0831892490386963
  • Run 3 Features Of JavaFX That Will Win You Over time: 2.077716588973999
  • Run #4 time: 2.07608962059021
  • Run #5 time: 2.072899341583252
  • Run #6 time: 2.094369888305664
  • Run #7 time: 2.067504644393921
  • Run #8 time: 2.069610834121704
  • Run #9 time: 2.0567898750305176

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

Mathiasdm created this revision.Nov 11 2020, 4:03 PM
indygreg added a subscriber: indygreg.Nov 11 2020, 11:44 PM

Are the Rust extensions considered stable? I'm honestly unsure. I'd love to ship these. But I'd like confirmation from the people maintaining them before we sign off.

valentin.gatienbaron added a subscriber: valentin.gatienbaron.Nov 12 2020, 10:53 PM

Raphael should probably chime in, but as a data point, for us, all installations use rust. And they will have to, when we enable persistent-nodemap soon.
The likelihood of bug and performance bugs is probably higher than without rust, for conditions not covered by tests. That being said, I remember only two bugs from the past 6 months, and they were fairly obscure (type error on linux on filesystems that don't respect exec bits, occasional spurious failure in hg clean when it races on the filesystem with something).

Alphare accepted this revision.Nov 13 2020, 5:22 AM
Alphare added a subscriber: Alphare.

I'm glad to see this patch!

As Valentin said, the extensions are of course less battle-tested than the Python+C code, but they have had a fair amount of user testing by now. The Rust code is gated (in Python code) to only be triggered in supported contexts, if that worried you.

I currently know of one (annoying, but mostly harmless) issue in concurrent contexts I intend to look into today, but that shouldn't deter you from using this code.

Now that Gentoo users have the Rust extensions advertized to them and others like you are starting to push for them, we'll probably need to add support for Rust config in the near future to give a more fined-grained access to them, but that's another discussion.

One thing to be careful of though is the version of Rust has to be at least 1.41.0. It may still work for older versions (like 1.34.2), but they will break in the future for any versions lower than our current MSRV.

Mathiasdm added a comment.Nov 16 2020, 4:03 AM

Rust on CentOS 7 is 1.47.0, if I'm not mistaken.
Could the MSRV be checked automatically in the build of the rust extensions? (separate from this changeset, but might be useful, @Alphare )

Alphare added a comment.Nov 16 2020, 6:44 AM
In D9294#140142, @Mathiasdm wrote:

Rust on CentOS 7 is 1.47.0, if I'm not mistaken.

Cool

Could the MSRV be checked automatically in the build of the rust extensions? (separate from this changeset, but might be useful, @Alphare )

I was simply mentioning this as a caution, but I think that we should let old compilers compile if they can, and that the MSRV should only be a "backwards-compat guarantee", not a hard limit.

pulkit accepted this revision.Nov 21 2020, 2:41 AM
pulkit added a subscriber: pulkit.

From the discussion above, it seems to me that rust extensions are well tested (not battle-tested though). They are used at Jane Street and Gentoo users also have them advertised. Hence I am +1 on pushing this patch and will push in 3-4 days if I don't hear any concerns.

This revision is now accepted and ready to land.Nov 21 2020, 2:41 AM
pulkit added a comment.Nov 27 2020, 5:37 AM

Queuing this, many thanks!

Revision Contents
Changeset List

Diff 23458

contrib/packaging/docker/centos7

make \ make \
python3-devel \ python3-devel \
python36-docutils \ python36-docutils \
rpm-build \ rpm-build \
tar tar
# For creating repo meta data # For creating repo meta data
RUN yum install -y createrepo RUN yum install -y createrepo
# For rust extensions
RUN yum install -y cargo

contrib/packaging/docker/centos8

FROM centos:centos8 FROM centos:centos8
RUN groupadd -g %GID% build && \ RUN groupadd -g %GID% build && \
useradd -u %UID% -g %GID% -s /bin/bash -d /build -m build useradd -u %UID% -g %GID% -s /bin/bash -d /build -m build
RUN yum install -y \ RUN yum install -y \
gcc \ gcc \
gettext \ gettext \
make \ make \
python3-devel \ python3-devel \
python3-docutils \ python3-docutils \
rpm-build rpm-build
# For creating repo meta data # For creating repo meta data
RUN yum install -y createrepo RUN yum install -y createrepo
# For rust extensions
RUN yum install -y cargo

contrib/packaging/mercurial.spec

# these .a are not necessary and they are readonly and strip fails - kill them! # these .a are not necessary and they are readonly and strip fails - kill them!
rm -f %{buildroot}%{hgpyprefix}/lib/{,python2.*/config}/libpython2.*.a rm -f %{buildroot}%{hgpyprefix}/lib/{,python2.*/config}/libpython2.*.a
cd - cd -
cd %{docutilsname} cd %{docutilsname}
LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py install --root="$RPM_BUILD_ROOT" LD_LIBRARY_PATH=$PYPATH $PYPATH/python setup.py install --root="$RPM_BUILD_ROOT"
cd - cd -
PATH=$PYPATH:$PATH LD_LIBRARY_PATH=$PYPATH make install PYTHON=%{pythonexe} DESTDIR=$RPM_BUILD_ROOT PREFIX=%{hgpyprefix} MANDIR=%{_mandir} PATH=$PYPATH:$PATH LD_LIBRARY_PATH=$PYPATH make install PYTHON=%{pythonexe} DESTDIR=$RPM_BUILD_ROOT PREFIX=%{hgpyprefix} MANDIR=%{_mandir} PURE="--rust"
mkdir -p $RPM_BUILD_ROOT%{_bindir} mkdir -p $RPM_BUILD_ROOT%{_bindir}
( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/hg . ) ( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/hg . )
( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/python2.? %{pythonhg} ) ( cd $RPM_BUILD_ROOT%{_bindir}/ && ln -s ../..%{hgpyprefix}/bin/python2.? %{pythonhg} )
%else %else
make install PYTHON=%{pythonexe} DESTDIR=$RPM_BUILD_ROOT PREFIX=%{_prefix} MANDIR=%{_mandir} make install PYTHON=%{pythonexe} DESTDIR=$RPM_BUILD_ROOT PREFIX=%{_prefix} MANDIR=%{_mandir} PURE="--rust"
%endif %endif
install -m 755 contrib/chg/chg $RPM_BUILD_ROOT%{_bindir}/ install -m 755 contrib/chg/chg $RPM_BUILD_ROOT%{_bindir}/
install -m 755 contrib/hgk $RPM_BUILD_ROOT%{_bindir}/ install -m 755 contrib/hgk $RPM_BUILD_ROOT%{_bindir}/
install -m 755 contrib/hg-ssh $RPM_BUILD_ROOT%{_bindir}/ install -m 755 contrib/hg-ssh $RPM_BUILD_ROOT%{_bindir}/
bash_completion_dir=$RPM_BUILD_ROOT%{_sysconfdir}/bash_completion.d bash_completion_dir=$RPM_BUILD_ROOT%{_sysconfdir}/bash_completion.d