This is an archive of the discontinued Mercurial Phabricator instance.

Differential D851

[RFC] setup: increase MAX_PATH limit on Windows 10 Anniversary Update
AbandonedPublic

Authored by indygreg on Sep 30 2017, 5:46 AM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Summary

As documented at
https://msdn.microsoft.com/en-gb/library/windows/desktop/aa365247(v=vs.85).aspx
system settings or per-application settings can now opt in to increasing
the length of MAX_PATH, removing the 260 character restriction. This
impacts the following API calls:

CreateDirectoryW, CreateDirectoryExW, GetCurrentDirectoryW,
RemoveDirectoryW, SetCurrentDirectoryW, CopyFileW, CopyFile2,
CopyFileExW, CreateFileW, CreateFile2, CreateHardLinkW,
CreateSymbolicLinkW, DeleteFileW, FindFirstFileW, FindFirstFileExW,
FindNextFileW, GetFileAttributesW, GetFileAttributesExW,
SetFileAttributesW, GetFullPathNameW, GetLongPathNameW, MoveFileW,
MoveFileExW, MoveFileWithProgressW, ReplaceFileW, SearchPathW,
FindFirstFileNameW, FindNextFileNameW, FindFirstStreamW,
FindNextStreamW, GetCompressedFileSizeW, GetFinalPathNameByHandleW.

Adding an application manifest to hg.exe will activate this setting
and possibly eliminate some 260 character path length limitations
in Mercurial without having to convert all paths to absolute and use
the "\\?\" prefix, which was previously the only supported mechanism
for using longer paths.

I HAVE NOT YET TESTED THIS PATCH.

Diff Detail

Repository
rHG Mercurial
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

indygreg created this revision.Sep 30 2017, 5:46 AM
abuehl added a subscriber: abuehl.Sep 30 2017, 7:18 AM
abuehl added a comment.Sep 30 2017, 7:27 AM

What about file I/O done via Python lib (e.g. python27.dll)?

indygreg added a comment.Sep 30 2017, 7:34 AM
In D851#14300, @abuehl wrote:

What about file I/O done via Python lib (e.g. python27.dll)?

I /think/ this works at the process - not DLL - level. But I need to test on a Windows machine to be sure. One of the reasons this is marked RFC.

abuehl added a comment.Sep 30 2017, 7:53 AM
In D851#14301, @indygreg wrote:
In D851#14300, @abuehl wrote:

What about file I/O done via Python lib (e.g. python27.dll)?

I /think/ this works at the process - not DLL - level. But I need to test on a Windows machine to be sure. One of the reasons this is marked RFC.

Maybe, but what does the Python library itself do with long paths?
What do you expect from calling e.g. os.unlink with a long path? Can the Python lib handle it?

abuehl added a comment.Sep 30 2017, 8:52 AM

In addition, Mercurial is not using the ...W Windows API functions.

I think, for example

https://phab.mercurial-scm.org/diffusion/HG/browse/default/mercurial/cext/osutil.c;da8bdeb1be28b976909a963c89e974264686e2bb$1207

would have to be changed to call CreateFileW in order for this to have an effect.

indygreg abandoned this revision.Dec 24 2017, 12:54 PM

Support for this was added in ed5acd3fd7e1.

Revision Contents
Changeset List

PathPackages
Msetup.py (29 lines)

Diff 2176

setup.py

if err or returncode != 0: if err or returncode != 0:
raise DistutilsExecError(err) raise DistutilsExecError(err)
with open(self._indexfilename, 'w') as f: with open(self._indexfilename, 'w') as f:
f.write('# this file is autogenerated by setup.py\n') f.write('# this file is autogenerated by setup.py\n')
f.write('docs = ') f.write('docs = ')
f.write(out) f.write(out)
# ws2:longPathAware increases MAX_PATH on Windows 10 Anniversary Edition and
# newer to larger than its default of 260.
HGEXE_MANIFEST = '''
<application xmlns="urn:schemas-microsoft-com:asm.v3">
<windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
<ws2:longPathAware>true</ws2:longPathAware>
</windowsSettings>
</application>
'''.strip()
class buildhgexe(build_ext): class buildhgexe(build_ext):
description = 'compile hg.exe from mercurial/exewrapper.c' description = 'compile hg.exe from mercurial/exewrapper.c'
def build_extensions(self): def build_extensions(self):
if os.name != 'nt': if os.name != 'nt':
return return
if isinstance(self.compiler, HackedMingw32CCompiler): if isinstance(self.compiler, HackedMingw32CCompiler):
self.compiler.compiler_so = self.compiler.compiler # no -mdll self.compiler.compiler_so = self.compiler.compiler # no -mdll
pythonlib = 'python%d%d' % (hv >> 24, (hv >> 16) & 0xff) pythonlib = 'python%d%d' % (hv >> 24, (hv >> 16) & 0xff)
log.info('using %s as Python library name' % pythonlib) log.info('using %s as Python library name' % pythonlib)
with open('mercurial/hgpythonlib.h', 'wb') as f: with open('mercurial/hgpythonlib.h', 'wb') as f:
f.write('/* this file is autogenerated by setup.py */\n') f.write('/* this file is autogenerated by setup.py */\n')
f.write('#define HGPYTHONLIB "%s"\n' % pythonlib) f.write('#define HGPYTHONLIB "%s"\n' % pythonlib)
objects = self.compiler.compile(['mercurial/exewrapper.c'], objects = self.compiler.compile(['mercurial/exewrapper.c'],
output_dir=self.build_temp) output_dir=self.build_temp)
try:
fd, manifestpath = tempfile.mkstemp('.manifest')
fd.write(HGEXE_MANIFEST)
fd.close()
dir = os.path.dirname(self.get_ext_fullpath('dummy')) dir = os.path.dirname(self.get_ext_fullpath('dummy'))
target = os.path.join(dir, 'hg') target = os.path.join(dir, 'hg')
self.compiler.link_executable(objects, target, self.compiler.link_executable(objects, target,
libraries=[], libraries=[], output_dir=self.build_temp,
output_dir=self.build_temp) extra_postargs=['/MANIFESTFILE:%s' % manifestpath])
finally:
os.unlink(manifestpath)
@property @property
def hgexepath(self): def hgexepath(self):
dir = os.path.dirname(self.get_ext_fullpath('dummy')) dir = os.path.dirname(self.get_ext_fullpath('dummy'))
return os.path.join(self.build_temp, dir, 'hg.exe') return os.path.join(self.build_temp, dir, 'hg.exe')
class hginstall(install): class hginstall(install):