| hg-reviewers |
.hg/dirstate now only contains some metadata to point to a separate data file
named .hg/dirstate.{}.d with a random hexadecimal identifier. For now every
update creates a new data file and removes the old one, but later we’ll
(usually) append to an existing file.
Separating into two files allows doing the "write to a temporary file then
atomically rename into destination" dance with only a small docket file,
without always rewriting a lot of data.
| Automatic diff as part of commit; lint not applicable. |
| Automatic diff as part of commit; unit tests not applicable. |
This seems good overall, but I have some questions.
| mercurial/debugcommands.py | ||
|---|---|---|
| 995 | How does this work once we move to append-only (I'm not seeing it in the next changesets)? | |
| 1004 | nit: hash is a builtin function in Python | |
| mercurial/dirstateutils/docket.py | ||
| 43 | Why pass around nodeconstants? Shouldn't they be directly imported? | |
| rust/rhg/src/commands/status.rs | ||
| 176 | The opening of the dirstate maybe should be consolidated somehow so we don't duplicate the docket indirection in rhg and hg-cpython, but that's not that big of a deal. | |
| mercurial/debugcommands.py | ||
|---|---|---|
| 995 | Scratch that, it's in the 3rd one. | |
| mercurial/debugcommands.py | ||
|---|---|---|
| 1004 | Yes. Personally I don’t really mind variable shadowing, though once upon a time I worked on a code base where a linter complained about it. I’ll rename to hash_bytes. | |
| rust/rhg/src/commands/status.rs | ||
| 176 | Yes, rhg is in sore need of refactoring to have reusable APIs for various things such as opening the dirstate, but that will be for another patch. The hg-cpython case is slightly different since (so far) we do all the I/O and docket handling on the Python side of FFI. | |
| mercurial/debugcommands.py | ||
|---|---|---|
| 1005 | You probably want ui.write here | |
| mercurial/debugcommands.py | ||
|---|---|---|
| 997 | do you mean or nothing for distate-v1 ? | |
| mercurial/dirstatemap.py | ||
| 637 | We should restrict that amount of data-read to docket.data_size, should we not ? | |
| mercurial/dirstateutils/docket.py | ||
| 31 | We should not use .d extension here. There are too many part of Mercurial that assum .d is attached to a revlog and that will eventually lead to confusion. | |
| mercurial/dirstatemap.py | ||
|---|---|---|
| 637 | never mind, this is done in the next changesets. | |
| mercurial/dirstateutils/docket.py | ||
|---|---|---|
| 31 | why not .docket simply? | |
| mercurial/dirstateutils/docket.py | ||
|---|---|---|
| 31 | This is the data, my bad. | |
| Path | Packages | |||
|---|---|---|---|---|
| M | contrib/dirstatenonnormalcheck.py (4 lines) | |||
| M | mercurial/debugcommands.py (19 lines) | |||
| M | mercurial/dirstate.py (13 lines) | |||
| M | mercurial/dirstatemap.py (132 lines) | |||
| A | M | mercurial/dirstateutils/__init__.py | ||
| A | M | mercurial/dirstateutils/docket.py (62 lines) | ||
| M | mercurial/upgrade_utils/engine.py (1 line) | |||
| M | rust/hg-core/src/dirstate_tree/dirstate_map.rs (12 lines) | |||
| M | rust/hg-core/src/dirstate_tree/dispatch.rs (12 lines) | |||
| M | rust/hg-core/src/dirstate_tree/on_disk.rs (87 lines) | |||
| M | rust/hg-core/src/operations/list_tracked_files.rs (9 lines) | |||
| M | rust/hg-core/src/repo.rs (5 lines) | |||
| M | rust/hg-cpython/src/dirstate/dirstate_map.rs (48 lines) | |||
| M | rust/hg-cpython/src/dirstate/dispatch.rs (8 lines) | |||
| M | rust/hg-cpython/src/dirstate/owning.rs (26 lines) | |||
| M | rust/rhg/src/commands/status.rs (31 lines) | |||
| M | rust/rhg/src/error.rs (7 lines) | |||
| M | setup.py (1 line) | |||
| M | tests/fakedirstatewritetime.py (4 lines) | |||
| M | tests/test-completion.t (2 lines) | |||
| M | tests/test-help.t (2 lines) | |||
| M | tests/test-hgignore.t (7 lines) |
| Status | Author | Revision | |
|---|---|---|---|
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin | ||
| Closed | SimonSapin |
| ui.develwarn(b"%s call to %s\n" % (label, b_orig), config=b'dirstate') | ui.develwarn(b"%s call to %s\n" % (label, b_orig), config=b'dirstate') | ||||
| ui.develwarn(b"inconsistency in nonnormalset\n", config=b'dirstate') | ui.develwarn(b"inconsistency in nonnormalset\n", config=b'dirstate') | ||||
| b_nonnormal = pycompat.sysbytes(repr(_nonnormalset)) | b_nonnormal = pycompat.sysbytes(repr(_nonnormalset)) | ||||
| ui.develwarn(b"[nonnormalset] %s\n" % b_nonnormal, config=b'dirstate') | ui.develwarn(b"[nonnormalset] %s\n" % b_nonnormal, config=b'dirstate') | ||||
| b_nonnormalcomputed = pycompat.sysbytes(repr(nonnormalcomputedmap)) | b_nonnormalcomputed = pycompat.sysbytes(repr(nonnormalcomputedmap)) | ||||
| ui.develwarn(b"[map] %s\n" % b_nonnormalcomputed, config=b'dirstate') | ui.develwarn(b"[map] %s\n" % b_nonnormalcomputed, config=b'dirstate') | ||||
| def _checkdirstate(orig, self, arg): | def _checkdirstate(orig, self, *args, **kwargs): | ||||
| """Check nonnormal set consistency before and after the call to orig""" | """Check nonnormal set consistency before and after the call to orig""" | ||||
| checkconsistency( | checkconsistency( | ||||
| self._ui, orig, self._map, self._map.nonnormalset, b"before" | self._ui, orig, self._map, self._map.nonnormalset, b"before" | ||||
| ) | ) | ||||
| r = orig(self, arg) | r = orig(self, *args, **kwargs) | ||||
| checkconsistency( | checkconsistency( | ||||
| self._ui, orig, self._map, self._map.nonnormalset, b"after" | self._ui, orig, self._map, self._map.nonnormalset, b"after" | ||||
| ) | ) | ||||
| return r | return r | ||||
| def extsetup(ui): | def extsetup(ui): | ||||
| """Wrap functions modifying dirstate to check nonnormalset consistency""" | """Wrap functions modifying dirstate to check nonnormalset consistency""" | ||||
| # emulate that all 'dirstate.normal' results are written out | # emulate that all 'dirstate.normal' results are written out | ||||
| self._lastnormaltime = 0 | self._lastnormaltime = 0 | ||||
| self._updatedfiles.clear() | self._updatedfiles.clear() | ||||
| # delay writing in-memory changes out | # delay writing in-memory changes out | ||||
| tr.addfilegenerator( | tr.addfilegenerator( | ||||
| b'dirstate', | b'dirstate', | ||||
| (self._filename,), | (self._filename,), | ||||
| self._writedirstate, | lambda f: self._writedirstate(tr, f), | ||||
| location=b'plain', | location=b'plain', | ||||
| ) | ) | ||||
| return | return | ||||
| st = self._opener(filename, b"w", atomictemp=True, checkambig=True) | st = self._opener(filename, b"w", atomictemp=True, checkambig=True) | ||||
| self._writedirstate(st) | self._writedirstate(tr, st) | ||||
| def addparentchangecallback(self, category, callback): | def addparentchangecallback(self, category, callback): | ||||
| """add a callback to be called when the wd parents are changed | """add a callback to be called when the wd parents are changed | ||||
| Callback will be called with the following arguments: | Callback will be called with the following arguments: | ||||
| dirstate, (oldp1, oldp2), (newp1, newp2) | dirstate, (oldp1, oldp2), (newp1, newp2) | ||||
| Category is a unique identifier to allow overwriting an old callback | Category is a unique identifier to allow overwriting an old callback | ||||
| with a newer callback. | with a newer callback. | ||||
| """ | """ | ||||
| self._plchangecallbacks[category] = callback | self._plchangecallbacks[category] = callback | ||||
| def _writedirstate(self, st): | def _writedirstate(self, tr, st): | ||||
| # notify callbacks about parents change | # notify callbacks about parents change | ||||
| if self._origpl is not None and self._origpl != self._pl: | if self._origpl is not None and self._origpl != self._pl: | ||||
| for c, callback in sorted( | for c, callback in sorted( | ||||
| pycompat.iteritems(self._plchangecallbacks) | pycompat.iteritems(self._plchangecallbacks) | ||||
| ): | ): | ||||
| callback(self, self._origpl, self._pl) | callback(self, self._origpl, self._pl) | ||||
| self._origpl = None | self._origpl = None | ||||
| # use the modification time of the newly created temporary file as the | # use the modification time of the newly created temporary file as the | ||||
| # multiple of n seconds | # multiple of n seconds | ||||
| clock = time.time() | clock = time.time() | ||||
| start = int(clock) - (int(clock) % delaywrite) | start = int(clock) - (int(clock) % delaywrite) | ||||
| end = start + delaywrite | end = start + delaywrite | ||||
| time.sleep(end - clock) | time.sleep(end - clock) | ||||
| now = end # trust our estimate that the end is near now | now = end # trust our estimate that the end is near now | ||||
| break | break | ||||
| self._map.write(st, now) | self._map.write(tr, st, now) | ||||
| self._lastnormaltime = 0 | self._lastnormaltime = 0 | ||||
| self._dirty = False | self._dirty = False | ||||
| def _dirignore(self, f): | def _dirignore(self, f): | ||||
| if self._ignore(f): | if self._ignore(f): | ||||
| return True | return True | ||||
| for p in pathutil.finddirs(f): | for p in pathutil.finddirs(f): | ||||
| if self._ignore(p): | if self._ignore(p): | ||||
| filename = self._actualfilename(tr) | filename = self._actualfilename(tr) | ||||
| assert backupname != filename | assert backupname != filename | ||||
| # use '_writedirstate' instead of 'write' to write changes certainly, | # use '_writedirstate' instead of 'write' to write changes certainly, | ||||
| # because the latter omits writing out if transaction is running. | # because the latter omits writing out if transaction is running. | ||||
| # output file will be used to create backup of dirstate at this point. | # output file will be used to create backup of dirstate at this point. | ||||
| if self._dirty or not self._opener.exists(filename): | if self._dirty or not self._opener.exists(filename): | ||||
| self._writedirstate( | self._writedirstate( | ||||
| self._opener(filename, b"w", atomictemp=True, checkambig=True) | tr, | ||||
| self._opener(filename, b"w", atomictemp=True, checkambig=True), | |||||
| ) | ) | ||||
| if tr: | if tr: | ||||
| # ensure that subsequent tr.writepending returns True for | # ensure that subsequent tr.writepending returns True for | ||||
| # changes written out above, even if dirstate is never | # changes written out above, even if dirstate is never | ||||
| # changed after this | # changed after this | ||||
| tr.addfilegenerator( | tr.addfilegenerator( | ||||
| b'dirstate', | b'dirstate', | ||||
| (self._filename,), | (self._filename,), | ||||
| self._writedirstate, | lambda f: self._writedirstate(tr, f), | ||||
| location=b'plain', | location=b'plain', | ||||
| ) | ) | ||||
| # ensure that pending file written above is unlinked at | # ensure that pending file written above is unlinked at | ||||
| # failure, even if tr.writepending isn't invoked until the | # failure, even if tr.writepending isn't invoked until the | ||||
| # end of this transaction | # end of this transaction | ||||
| tr.registertmp(filename, location=b'plain') | tr.registertmp(filename, location=b'plain') | ||||
| ) | ) | ||||
| assert srcrepo.dirstate._use_dirstate_v2 == (old == b'v2') | assert srcrepo.dirstate._use_dirstate_v2 == (old == b'v2') | ||||
| srcrepo.dirstate._map._use_dirstate_tree = True | srcrepo.dirstate._map._use_dirstate_tree = True | ||||
| srcrepo.dirstate._map.preload() | srcrepo.dirstate._map.preload() | ||||
| srcrepo.dirstate._use_dirstate_v2 = new == b'v2' | srcrepo.dirstate._use_dirstate_v2 = new == b'v2' | ||||
| srcrepo.dirstate._map._use_dirstate_v2 = srcrepo.dirstate._use_dirstate_v2 | srcrepo.dirstate._map._use_dirstate_v2 = srcrepo.dirstate._use_dirstate_v2 | ||||
| srcrepo.dirstate._dirty = True | srcrepo.dirstate._dirty = True | ||||
| srcrepo.vfs.unlink(b'dirstate') | |||||
| srcrepo.dirstate.write(None) | srcrepo.dirstate.write(None) | ||||
| scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements) | scmutil.writereporequirements(srcrepo, upgrade_op.new_requirements) | ||||
| root: ChildNodes::default(), | root: ChildNodes::default(), | ||||
| nodes_with_entry_count: 0, | nodes_with_entry_count: 0, | ||||
| nodes_with_copy_source_count: 0, | nodes_with_copy_source_count: 0, | ||||
| ignore_patterns_hash: [0; on_disk::IGNORE_PATTERNS_HASH_LEN], | ignore_patterns_hash: [0; on_disk::IGNORE_PATTERNS_HASH_LEN], | ||||
| } | } | ||||
| } | } | ||||
| #[timed] | #[timed] | ||||
| pub fn new_v2( | pub fn new_v2(on_disk: &'on_disk [u8]) -> Result<Self, DirstateError> { | ||||
| on_disk: &'on_disk [u8], | |||||
| ) -> Result<(Self, Option<DirstateParents>), DirstateError> { | |||||
| Ok(on_disk::read(on_disk)?) | Ok(on_disk::read(on_disk)?) | ||||
| } | } | ||||
| #[timed] | #[timed] | ||||
| pub fn new_v1( | pub fn new_v1( | ||||
| on_disk: &'on_disk [u8], | on_disk: &'on_disk [u8], | ||||
| ) -> Result<(Self, Option<DirstateParents>), DirstateError> { | ) -> Result<(Self, Option<DirstateParents>), DirstateError> { | ||||
| let mut map = Self::empty(on_disk); | let mut map = Self::empty(on_disk); | ||||
| &mut packed, | &mut packed, | ||||
| ); | ); | ||||
| } | } | ||||
| } | } | ||||
| Ok(packed) | Ok(packed) | ||||
| } | } | ||||
| #[timed] | #[timed] | ||||
| fn pack_v2( | fn pack_v2(&mut self, now: Timestamp) -> Result<Vec<u8>, DirstateError> { | ||||
| &mut self, | |||||
| parents: DirstateParents, | |||||
| now: Timestamp, | |||||
| ) -> Result<Vec<u8>, DirstateError> { | |||||
| // TODO:!how do we want to handle this in 2038? | // TODO:!how do we want to handle this in 2038? | ||||
| let now: i32 = now.0.try_into().expect("time overflow"); | let now: i32 = now.0.try_into().expect("time overflow"); | ||||
| let mut paths = Vec::new(); | let mut paths = Vec::new(); | ||||
| for node in self.iter_nodes() { | for node in self.iter_nodes() { | ||||
| let node = node?; | let node = node?; | ||||
| if let Some(entry) = node.entry()? { | if let Some(entry) = node.entry()? { | ||||
| if entry.mtime_is_ambiguous(now) { | if entry.mtime_is_ambiguous(now) { | ||||
| paths.push( | paths.push( | ||||
| node.full_path_borrowed(self.on_disk)? | node.full_path_borrowed(self.on_disk)? | ||||
| .detach_from_tree(), | .detach_from_tree(), | ||||
| ) | ) | ||||
| } | } | ||||
| } | } | ||||
| } | } | ||||
| // Borrow of `self` ends here since we collect cloned paths | // Borrow of `self` ends here since we collect cloned paths | ||||
| self.clear_known_ambiguous_mtimes(&paths)?; | self.clear_known_ambiguous_mtimes(&paths)?; | ||||
| on_disk::write(self, parents) | on_disk::write(self) | ||||
| } | } | ||||
| fn status<'a>( | fn status<'a>( | ||||
| &'a mut self, | &'a mut self, | ||||
| matcher: &'a (dyn Matcher + Sync), | matcher: &'a (dyn Matcher + Sync), | ||||
| root_dir: PathBuf, | root_dir: PathBuf, | ||||
| ignore_files: Vec<PathBuf>, | ignore_files: Vec<PathBuf>, | ||||
| options: StatusOptions, | options: StatusOptions, | ||||
| ) -> Result<Vec<u8>, DirstateError>; | ) -> Result<Vec<u8>, DirstateError>; | ||||
| /// Clear mtimes that are ambigous with `now` (similar to | /// Clear mtimes that are ambigous with `now` (similar to | ||||
| /// `clear_ambiguous_times` but for all files in the dirstate map), and | /// `clear_ambiguous_times` but for all files in the dirstate map), and | ||||
| /// serialize bytes to write the `.hg/dirstate` file to disk in dirstate-v2 | /// serialize bytes to write the `.hg/dirstate` file to disk in dirstate-v2 | ||||
| /// format. | /// format. | ||||
| /// | /// | ||||
| /// Note: this is only supported by the tree dirstate map. | /// Note: this is only supported by the tree dirstate map. | ||||
| fn pack_v2( | fn pack_v2(&mut self, now: Timestamp) -> Result<Vec<u8>, DirstateError>; | ||||
| &mut self, | |||||
| parents: DirstateParents, | |||||
| now: Timestamp, | |||||
| ) -> Result<Vec<u8>, DirstateError>; | |||||
| /// Run the status algorithm. | /// Run the status algorithm. | ||||
| /// | /// | ||||
| /// This is not sematically a method of the dirstate map, but a different | /// This is not sematically a method of the dirstate map, but a different | ||||
| /// algorithm is used for the flat v.s. tree dirstate map so having it in | /// algorithm is used for the flat v.s. tree dirstate map so having it in | ||||
| /// this trait enables the same dynamic dispatch as with other methods. | /// this trait enables the same dynamic dispatch as with other methods. | ||||
| fn status<'a>( | fn status<'a>( | ||||
| &'a mut self, | &'a mut self, | ||||
| fn pack_v1( | fn pack_v1( | ||||
| &mut self, | &mut self, | ||||
| parents: DirstateParents, | parents: DirstateParents, | ||||
| now: Timestamp, | now: Timestamp, | ||||
| ) -> Result<Vec<u8>, DirstateError> { | ) -> Result<Vec<u8>, DirstateError> { | ||||
| self.pack(parents, now) | self.pack(parents, now) | ||||
| } | } | ||||
| fn pack_v2( | fn pack_v2(&mut self, _now: Timestamp) -> Result<Vec<u8>, DirstateError> { | ||||
| &mut self, | |||||
| _parents: DirstateParents, | |||||
| _now: Timestamp, | |||||
| ) -> Result<Vec<u8>, DirstateError> { | |||||
| panic!( | panic!( | ||||
| "should have used dirstate_tree::DirstateMap to use the v2 format" | "should have used dirstate_tree::DirstateMap to use the v2 format" | ||||
| ) | ) | ||||
| } | } | ||||
| fn status<'a>( | fn status<'a>( | ||||
| &'a mut self, | &'a mut self, | ||||
| matcher: &'a (dyn Matcher + Sync), | matcher: &'a (dyn Matcher + Sync), | ||||
| use crate::errors::HgError; | use crate::errors::HgError; | ||||
| use crate::utils::hg_path::HgPath; | use crate::utils::hg_path::HgPath; | ||||
| use crate::DirstateEntry; | use crate::DirstateEntry; | ||||
| use crate::DirstateError; | use crate::DirstateError; | ||||
| use crate::DirstateParents; | use crate::DirstateParents; | ||||
| use crate::EntryState; | use crate::EntryState; | ||||
| use bytes_cast::unaligned::{I32Be, I64Be, U32Be}; | use bytes_cast::unaligned::{I32Be, I64Be, U32Be}; | ||||
| use bytes_cast::BytesCast; | use bytes_cast::BytesCast; | ||||
| use format_bytes::format_bytes; | |||||
| use std::borrow::Cow; | use std::borrow::Cow; | ||||
| use std::convert::TryFrom; | use std::convert::TryFrom; | ||||
| use std::time::{Duration, SystemTime, UNIX_EPOCH}; | use std::time::{Duration, SystemTime, UNIX_EPOCH}; | ||||
| /// Added at the start of `.hg/dirstate` when the "v2" format is used. | /// Added at the start of `.hg/dirstate` when the "v2" format is used. | ||||
| /// This a redundant sanity check more than an actual "magic number" since | /// This a redundant sanity check more than an actual "magic number" since | ||||
| /// `.hg/requires` already governs which format should be used. | /// `.hg/requires` already governs which format should be used. | ||||
| pub const V2_FORMAT_MARKER: &[u8; 12] = b"dirstate-v2\n"; | pub const V2_FORMAT_MARKER: &[u8; 12] = b"dirstate-v2\n"; | ||||
| /// Keep space for 256-bit hashes | |||||
| const STORED_NODE_ID_BYTES: usize = 32; | |||||
| /// … even though only 160 bits are used for now, with SHA-1 | |||||
| const USED_NODE_ID_BYTES: usize = 20; | |||||
| pub(super) const IGNORE_PATTERNS_HASH_LEN: usize = 20; | pub(super) const IGNORE_PATTERNS_HASH_LEN: usize = 20; | ||||
| pub(super) type IgnorePatternsHash = [u8; IGNORE_PATTERNS_HASH_LEN]; | pub(super) type IgnorePatternsHash = [u8; IGNORE_PATTERNS_HASH_LEN]; | ||||
| // Must match `HEADER` in `mercurial/dirstateutils/docket.py` | |||||
| #[derive(BytesCast)] | #[derive(BytesCast)] | ||||
| #[repr(C)] | #[repr(C)] | ||||
| struct Header { | struct DocketHeader { | ||||
| marker: [u8; V2_FORMAT_MARKER.len()], | marker: [u8; V2_FORMAT_MARKER.len()], | ||||
| parent_1: [u8; STORED_NODE_ID_BYTES], | |||||
| parent_2: [u8; STORED_NODE_ID_BYTES], | |||||
| data_size: Size, | |||||
| uuid_size: u8, | |||||
| } | |||||
| /// `dirstatemap.parents()` in `mercurial/dirstate.py` relies on this | pub struct Docket<'on_disk> { | ||||
| /// `parents` field being at this offset, immediately after `marker`. | header: &'on_disk DocketHeader, | ||||
| parents: DirstateParents, | uuid: &'on_disk [u8], | ||||
| } | |||||
| #[derive(BytesCast)] | |||||
| #[repr(C)] | |||||
| struct Header { | |||||
| root: ChildNodes, | root: ChildNodes, | ||||
| nodes_with_entry_count: Size, | nodes_with_entry_count: Size, | ||||
| nodes_with_copy_source_count: Size, | nodes_with_copy_source_count: Size, | ||||
| /// If non-zero, a hash of ignore files that were used for some previous | /// If non-zero, a hash of ignore files that were used for some previous | ||||
| /// run of the `status` algorithm. | /// run of the `status` algorithm. | ||||
| /// | /// | ||||
| /// We define: | /// We define: | ||||
| /// A `HgPath` of `len` bytes | /// A `HgPath` of `len` bytes | ||||
| type PathSlice = Slice; | type PathSlice = Slice; | ||||
| /// Either nothing if `start == 0`, or a `HgPath` of `len` bytes | /// Either nothing if `start == 0`, or a `HgPath` of `len` bytes | ||||
| type OptPathSlice = Slice; | type OptPathSlice = Slice; | ||||
| /// Make sure that size-affecting changes are made knowingly | /// Make sure that size-affecting changes are made knowingly | ||||
| fn _static_assert_size_of() { | fn _static_assert_size_of() { | ||||
| let _ = std::mem::transmute::<Header, [u8; 88]>; | let _ = std::mem::transmute::<DocketHeader, [u8; 81]>; | ||||
| let _ = std::mem::transmute::<Header, [u8; 36]>; | |||||
| let _ = std::mem::transmute::<Node, [u8; 49]>; | let _ = std::mem::transmute::<Node, [u8; 49]>; | ||||
| } | } | ||||
| /// Unexpected file format found in `.hg/dirstate` with the "v2" format. | /// Unexpected file format found in `.hg/dirstate` with the "v2" format. | ||||
| /// | /// | ||||
| /// This should only happen if Mercurial is buggy or a repository is corrupted. | /// This should only happen if Mercurial is buggy or a repository is corrupted. | ||||
| #[derive(Debug)] | #[derive(Debug)] | ||||
| pub struct DirstateV2ParseError; | pub struct DirstateV2ParseError; | ||||
| impl From<DirstateV2ParseError> for HgError { | impl From<DirstateV2ParseError> for HgError { | ||||
| fn from(_: DirstateV2ParseError) -> Self { | fn from(_: DirstateV2ParseError) -> Self { | ||||
| HgError::corrupted("dirstate-v2 parse error") | HgError::corrupted("dirstate-v2 parse error") | ||||
| } | } | ||||
| } | } | ||||
| impl From<DirstateV2ParseError> for crate::DirstateError { | impl From<DirstateV2ParseError> for crate::DirstateError { | ||||
| fn from(error: DirstateV2ParseError) -> Self { | fn from(error: DirstateV2ParseError) -> Self { | ||||
| HgError::from(error).into() | HgError::from(error).into() | ||||
| } | } | ||||
| } | } | ||||
| fn read_header(on_disk: &[u8]) -> Result<&Header, DirstateV2ParseError> { | impl<'on_disk> Docket<'on_disk> { | ||||
| let (header, _) = | pub fn parents(&self) -> DirstateParents { | ||||
| Header::from_bytes(on_disk).map_err(|_| DirstateV2ParseError)?; | use crate::Node; | ||||
| if header.marker == *V2_FORMAT_MARKER { | let p1 = Node::try_from(&self.header.parent_1[..USED_NODE_ID_BYTES]) | ||||
| Ok(header) | .unwrap() | ||||
| .clone(); | |||||
| let p2 = Node::try_from(&self.header.parent_2[..USED_NODE_ID_BYTES]) | |||||
| .unwrap() | |||||
| .clone(); | |||||
| DirstateParents { p1, p2 } | |||||
| } | |||||
| pub fn data_filename(&self) -> String { | |||||
| String::from_utf8(format_bytes!(b"dirstate.{}.d", self.uuid)).unwrap() | |||||
| } | |||||
| } | |||||
| pub fn read_docket( | |||||
| on_disk: &[u8], | |||||
| ) -> Result<Docket<'_>, DirstateV2ParseError> { | |||||
| let (header, uuid) = | |||||
| DocketHeader::from_bytes(on_disk).map_err(|_| DirstateV2ParseError)?; | |||||
| let uuid_size = header.uuid_size as usize; | |||||
| if header.marker == *V2_FORMAT_MARKER && uuid.len() == uuid_size { | |||||
| Ok(Docket { header, uuid }) | |||||
| } else { | } else { | ||||
| Err(DirstateV2ParseError) | Err(DirstateV2ParseError) | ||||
| } | } | ||||
| } | } | ||||
| pub(super) fn read<'on_disk>( | pub(super) fn read<'on_disk>( | ||||
| on_disk: &'on_disk [u8], | on_disk: &'on_disk [u8], | ||||
| ) -> Result< | ) -> Result<DirstateMap<'on_disk>, DirstateV2ParseError> { | ||||
| (DirstateMap<'on_disk>, Option<DirstateParents>), | |||||
| DirstateV2ParseError, | |||||
| > { | |||||
| if on_disk.is_empty() { | if on_disk.is_empty() { | ||||
| return Ok((DirstateMap::empty(on_disk), None)); | return Ok(DirstateMap::empty(on_disk)); | ||||
| } | } | ||||
| let header = read_header(on_disk)?; | let (header, _) = | ||||
| Header::from_bytes(on_disk).map_err(|_| DirstateV2ParseError)?; | |||||
| let dirstate_map = DirstateMap { | let dirstate_map = DirstateMap { | ||||
| on_disk, | on_disk, | ||||
| root: dirstate_map::ChildNodes::OnDisk(read_slice::<Node>( | root: dirstate_map::ChildNodes::OnDisk(read_slice::<Node>( | ||||
| on_disk, | on_disk, | ||||
| header.root, | header.root, | ||||
| )?), | )?), | ||||
| nodes_with_entry_count: header.nodes_with_entry_count.get(), | nodes_with_entry_count: header.nodes_with_entry_count.get(), | ||||
| nodes_with_copy_source_count: header | nodes_with_copy_source_count: header | ||||
| .nodes_with_copy_source_count | .nodes_with_copy_source_count | ||||
| .get(), | .get(), | ||||
| ignore_patterns_hash: header.ignore_patterns_hash, | ignore_patterns_hash: header.ignore_patterns_hash, | ||||
| }; | }; | ||||
| let parents = Some(header.parents.clone()); | Ok(dirstate_map) | ||||
| Ok((dirstate_map, parents)) | |||||
| } | } | ||||
| impl Node { | impl Node { | ||||
| pub(super) fn full_path<'on_disk>( | pub(super) fn full_path<'on_disk>( | ||||
| &self, | &self, | ||||
| on_disk: &'on_disk [u8], | on_disk: &'on_disk [u8], | ||||
| ) -> Result<&'on_disk HgPath, DirstateV2ParseError> { | ) -> Result<&'on_disk HgPath, DirstateV2ParseError> { | ||||
| read_hg_path(on_disk, self.full_path) | read_hg_path(on_disk, self.full_path) | ||||
| let len = usize::try_from(slice.len.get()).unwrap_or(std::usize::MAX); | let len = usize::try_from(slice.len.get()).unwrap_or(std::usize::MAX); | ||||
| on_disk | on_disk | ||||
| .get(start..) | .get(start..) | ||||
| .and_then(|bytes| T::slice_from_bytes(bytes, len).ok()) | .and_then(|bytes| T::slice_from_bytes(bytes, len).ok()) | ||||
| .map(|(slice, _rest)| slice) | .map(|(slice, _rest)| slice) | ||||
| .ok_or_else(|| DirstateV2ParseError) | .ok_or_else(|| DirstateV2ParseError) | ||||
| } | } | ||||
| pub(crate) fn parse_dirstate_parents( | |||||
| on_disk: &[u8], | |||||
| ) -> Result<&DirstateParents, HgError> { | |||||
| Ok(&read_header(on_disk)?.parents) | |||||
| } | |||||
| pub(crate) fn for_each_tracked_path<'on_disk>( | pub(crate) fn for_each_tracked_path<'on_disk>( | ||||
| on_disk: &'on_disk [u8], | on_disk: &'on_disk [u8], | ||||
| mut f: impl FnMut(&'on_disk HgPath), | mut f: impl FnMut(&'on_disk HgPath), | ||||
| ) -> Result<(), DirstateV2ParseError> { | ) -> Result<(), DirstateV2ParseError> { | ||||
| let header = read_header(on_disk)?; | let (header, _) = | ||||
| Header::from_bytes(on_disk).map_err(|_| DirstateV2ParseError)?; | |||||
| fn recur<'on_disk>( | fn recur<'on_disk>( | ||||
| on_disk: &'on_disk [u8], | on_disk: &'on_disk [u8], | ||||
| nodes: Slice, | nodes: Slice, | ||||
| f: &mut impl FnMut(&'on_disk HgPath), | f: &mut impl FnMut(&'on_disk HgPath), | ||||
| ) -> Result<(), DirstateV2ParseError> { | ) -> Result<(), DirstateV2ParseError> { | ||||
| for node in read_slice::<Node>(on_disk, nodes)? { | for node in read_slice::<Node>(on_disk, nodes)? { | ||||
| if let Some(state) = node.state()? { | if let Some(state) = node.state()? { | ||||
| if state.is_tracked() { | if state.is_tracked() { | ||||
| f(node.full_path(on_disk)?) | f(node.full_path(on_disk)?) | ||||
| } | } | ||||
| } | } | ||||
| recur(on_disk, node.children, f)? | recur(on_disk, node.children, f)? | ||||
| } | } | ||||
| Ok(()) | Ok(()) | ||||
| } | } | ||||
| recur(on_disk, header.root, &mut f) | recur(on_disk, header.root, &mut f) | ||||
| } | } | ||||
| pub(super) fn write( | pub(super) fn write( | ||||
| dirstate_map: &mut DirstateMap, | dirstate_map: &mut DirstateMap, | ||||
| parents: DirstateParents, | |||||
| ) -> Result<Vec<u8>, DirstateError> { | ) -> Result<Vec<u8>, DirstateError> { | ||||
| let header_len = std::mem::size_of::<Header>(); | let header_len = std::mem::size_of::<Header>(); | ||||
| // This ignores the space for paths, and for nodes without an entry. | // This ignores the space for paths, and for nodes without an entry. | ||||
| // TODO: better estimate? Skip the `Vec` and write to a file directly? | // TODO: better estimate? Skip the `Vec` and write to a file directly? | ||||
| let size_guess = header_len | let size_guess = header_len | ||||
| + std::mem::size_of::<Node>() | + std::mem::size_of::<Node>() | ||||
| * dirstate_map.nodes_with_entry_count as usize; | * dirstate_map.nodes_with_entry_count as usize; | ||||
| let mut out = Vec::with_capacity(size_guess); | let mut out = Vec::with_capacity(size_guess); | ||||
| // Keep space for the header. We’ll fill it out at the end when we know the | // Keep space for the header. We’ll fill it out at the end when we know the | ||||
| // actual offset for the root nodes. | // actual offset for the root nodes. | ||||
| out.resize(header_len, 0_u8); | out.resize(header_len, 0_u8); | ||||
| let root = | let root = | ||||
| write_nodes(dirstate_map, dirstate_map.root.as_ref(), &mut out)?; | write_nodes(dirstate_map, dirstate_map.root.as_ref(), &mut out)?; | ||||
| let header = Header { | let header = Header { | ||||
| marker: *V2_FORMAT_MARKER, | |||||
| parents: parents, | |||||
| root, | root, | ||||
| nodes_with_entry_count: dirstate_map.nodes_with_entry_count.into(), | nodes_with_entry_count: dirstate_map.nodes_with_entry_count.into(), | ||||
| nodes_with_copy_source_count: dirstate_map | nodes_with_copy_source_count: dirstate_map | ||||
| .nodes_with_copy_source_count | .nodes_with_copy_source_count | ||||
| .into(), | .into(), | ||||
| ignore_patterns_hash: dirstate_map.ignore_patterns_hash, | ignore_patterns_hash: dirstate_map.ignore_patterns_hash, | ||||
| }; | }; | ||||
| out[..header_len].copy_from_slice(header.as_bytes()); | out[..header_len].copy_from_slice(header.as_bytes()); | ||||
| // list_tracked_files.rs | // list_tracked_files.rs | ||||
| // | // | ||||
| // Copyright 2020 Antoine Cezar <antoine.cezar@octobus.net> | // Copyright 2020 Antoine Cezar <antoine.cezar@octobus.net> | ||||
| // | // | ||||
| // This software may be used and distributed according to the terms of the | // This software may be used and distributed according to the terms of the | ||||
| // GNU General Public License version 2 or any later version. | // GNU General Public License version 2 or any later version. | ||||
| use crate::dirstate::parsers::parse_dirstate_entries; | use crate::dirstate::parsers::parse_dirstate_entries; | ||||
| use crate::dirstate_tree::on_disk::for_each_tracked_path; | use crate::dirstate_tree::on_disk::{for_each_tracked_path, read_docket}; | ||||
| use crate::errors::HgError; | use crate::errors::HgError; | ||||
| use crate::repo::Repo; | use crate::repo::Repo; | ||||
| use crate::revlog::changelog::Changelog; | use crate::revlog::changelog::Changelog; | ||||
| use crate::revlog::manifest::{Manifest, ManifestEntry}; | use crate::revlog::manifest::{Manifest, ManifestEntry}; | ||||
| use crate::revlog::node::Node; | use crate::revlog::node::Node; | ||||
| use crate::revlog::revlog::RevlogError; | use crate::revlog::revlog::RevlogError; | ||||
| use crate::utils::hg_path::HgPath; | use crate::utils::hg_path::HgPath; | ||||
| use crate::DirstateError; | use crate::DirstateError; | ||||
| use rayon::prelude::*; | use rayon::prelude::*; | ||||
| /// List files under Mercurial control in the working directory | /// List files under Mercurial control in the working directory | ||||
| /// by reading the dirstate | /// by reading the dirstate | ||||
| pub struct Dirstate { | pub struct Dirstate { | ||||
| /// The `dirstate` content. | /// The `dirstate` content. | ||||
| content: Vec<u8>, | content: Vec<u8>, | ||||
| dirstate_v2: bool, | dirstate_v2: bool, | ||||
| } | } | ||||
| impl Dirstate { | impl Dirstate { | ||||
| pub fn new(repo: &Repo) -> Result<Self, HgError> { | pub fn new(repo: &Repo) -> Result<Self, HgError> { | ||||
| let mut content = repo.hg_vfs().read("dirstate")?; | |||||
| if repo.has_dirstate_v2() { | |||||
| let docket = read_docket(&content)?; | |||||
| content = repo.hg_vfs().read(docket.data_filename())?; | |||||
| } | |||||
| Ok(Self { | Ok(Self { | ||||
| content: repo.hg_vfs().read("dirstate")?, | content, | ||||
| dirstate_v2: repo.has_dirstate_v2(), | dirstate_v2: repo.has_dirstate_v2(), | ||||
| }) | }) | ||||
| } | } | ||||
| pub fn tracked_files(&self) -> Result<Vec<&HgPath>, DirstateError> { | pub fn tracked_files(&self) -> Result<Vec<&HgPath>, DirstateError> { | ||||
| let mut files = Vec::new(); | let mut files = Vec::new(); | ||||
| if !self.content.is_empty() { | if !self.content.is_empty() { | ||||
| if self.dirstate_v2 { | if self.dirstate_v2 { | ||||
| pub fn dirstate_parents( | pub fn dirstate_parents( | ||||
| &self, | &self, | ||||
| ) -> Result<crate::dirstate::DirstateParents, HgError> { | ) -> Result<crate::dirstate::DirstateParents, HgError> { | ||||
| let dirstate = self.hg_vfs().mmap_open("dirstate")?; | let dirstate = self.hg_vfs().mmap_open("dirstate")?; | ||||
| if dirstate.is_empty() { | if dirstate.is_empty() { | ||||
| return Ok(crate::dirstate::DirstateParents::NULL); | return Ok(crate::dirstate::DirstateParents::NULL); | ||||
| } | } | ||||
| let parents = if self.has_dirstate_v2() { | let parents = if self.has_dirstate_v2() { | ||||
| crate::dirstate_tree::on_disk::parse_dirstate_parents(&dirstate)? | crate::dirstate_tree::on_disk::read_docket(&dirstate)?.parents() | ||||
| } else { | } else { | ||||
| crate::dirstate::parsers::parse_dirstate_parents(&dirstate)? | crate::dirstate::parsers::parse_dirstate_parents(&dirstate)? | ||||
| .clone() | |||||
| }; | }; | ||||
| Ok(parents.clone()) | Ok(parents) | ||||
| } | } | ||||
| } | } | ||||
| impl Vfs<'_> { | impl Vfs<'_> { | ||||
| pub fn join(&self, relative_path: impl AsRef<Path>) -> PathBuf { | pub fn join(&self, relative_path: impl AsRef<Path>) -> PathBuf { | ||||
| self.base.join(relative_path) | self.base.join(relative_path) | ||||
| } | } | ||||
| // `RustDirstateMap`, rightfully so. | // `RustDirstateMap`, rightfully so. | ||||
| // All attributes also have to have a separate refcount data attribute for | // All attributes also have to have a separate refcount data attribute for | ||||
| // leaks, with all methods that go along for reference sharing. | // leaks, with all methods that go along for reference sharing. | ||||
| py_class!(pub class DirstateMap |py| { | py_class!(pub class DirstateMap |py| { | ||||
| @shared data inner: Box<dyn DirstateMapMethods + Send>; | @shared data inner: Box<dyn DirstateMapMethods + Send>; | ||||
| /// Returns a `(dirstate_map, parents)` tuple | /// Returns a `(dirstate_map, parents)` tuple | ||||
| @staticmethod | @staticmethod | ||||
| def new( | def new_v1( | ||||
| use_dirstate_tree: bool, | use_dirstate_tree: bool, | ||||
| use_dirstate_v2: bool, | |||||
| on_disk: PyBytes, | on_disk: PyBytes, | ||||
| ) -> PyResult<PyObject> { | ) -> PyResult<PyObject> { | ||||
| let dirstate_error = |e: DirstateError| { | let dirstate_error = |e: DirstateError| { | ||||
| PyErr::new::<exc::OSError, _>(py, format!("Dirstate error: {:?}", e)) | PyErr::new::<exc::OSError, _>(py, format!("Dirstate error: {:?}", e)) | ||||
| }; | }; | ||||
| let (inner, parents) = if use_dirstate_tree || use_dirstate_v2 { | let (inner, parents) = if use_dirstate_tree { | ||||
| let (map, parents) = | let (map, parents) = OwningDirstateMap::new_v1(py, on_disk) | ||||
| OwningDirstateMap::new(py, on_disk, use_dirstate_v2) | |||||
| .map_err(dirstate_error)?; | .map_err(dirstate_error)?; | ||||
| (Box::new(map) as _, parents) | (Box::new(map) as _, parents) | ||||
| } else { | } else { | ||||
| let bytes = on_disk.data(py); | let bytes = on_disk.data(py); | ||||
| let mut map = RustDirstateMap::default(); | let mut map = RustDirstateMap::default(); | ||||
| let parents = map.read(bytes).map_err(dirstate_error)?; | let parents = map.read(bytes).map_err(dirstate_error)?; | ||||
| (Box::new(map) as _, parents) | (Box::new(map) as _, parents) | ||||
| }; | }; | ||||
| let map = Self::create_instance(py, inner)?; | let map = Self::create_instance(py, inner)?; | ||||
| let parents = parents.map(|p| dirstate_parents_to_pytuple(py, &p)); | let parents = parents.map(|p| dirstate_parents_to_pytuple(py, &p)); | ||||
| Ok((map, parents).to_py_object(py).into_object()) | Ok((map, parents).to_py_object(py).into_object()) | ||||
| } | } | ||||
| /// Returns a DirstateMap | |||||
| @staticmethod | |||||
| def new_v2( | |||||
| on_disk: PyBytes, | |||||
| ) -> PyResult<PyObject> { | |||||
| let dirstate_error = |e: DirstateError| { | |||||
| PyErr::new::<exc::OSError, _>(py, format!("Dirstate error: {:?}", e)) | |||||
| }; | |||||
| let inner = OwningDirstateMap::new_v2(py, on_disk) | |||||
| .map_err(dirstate_error)?; | |||||
| let map = Self::create_instance(py, Box::new(inner))?; | |||||
| Ok(map.into_object()) | |||||
| } | |||||
| def clear(&self) -> PyResult<PyObject> { | def clear(&self) -> PyResult<PyObject> { | ||||
| self.inner(py).borrow_mut().clear(); | self.inner(py).borrow_mut().clear(); | ||||
| Ok(py.None()) | Ok(py.None()) | ||||
| } | } | ||||
| def get( | def get( | ||||
| &self, | &self, | ||||
| key: PyObject, | key: PyObject, | ||||
| Ok(self.inner(py).borrow_mut() | Ok(self.inner(py).borrow_mut() | ||||
| .has_dir(HgPath::new(d.data(py))) | .has_dir(HgPath::new(d.data(py))) | ||||
| .map_err(|e| { | .map_err(|e| { | ||||
| PyErr::new::<exc::ValueError, _>(py, e.to_string()) | PyErr::new::<exc::ValueError, _>(py, e.to_string()) | ||||
| })? | })? | ||||
| .to_py_object(py)) | .to_py_object(py)) | ||||
| } | } | ||||
| def write( | def write_v1( | ||||
| &self, | &self, | ||||
| use_dirstate_v2: bool, | |||||
| p1: PyObject, | p1: PyObject, | ||||
| p2: PyObject, | p2: PyObject, | ||||
| now: PyObject | now: PyObject | ||||
| ) -> PyResult<PyBytes> { | ) -> PyResult<PyBytes> { | ||||
| let now = Timestamp(now.extract(py)?); | let now = Timestamp(now.extract(py)?); | ||||
| let mut inner = self.inner(py).borrow_mut(); | |||||
| let parents = DirstateParents { | let parents = DirstateParents { | ||||
| p1: extract_node_id(py, &p1)?, | p1: extract_node_id(py, &p1)?, | ||||
| p2: extract_node_id(py, &p2)?, | p2: extract_node_id(py, &p2)?, | ||||
| }; | }; | ||||
| let result = inner.pack_v1(parents, now); | |||||
| match result { | |||||
| Ok(packed) => Ok(PyBytes::new(py, &packed)), | |||||
| Err(_) => Err(PyErr::new::<exc::OSError, _>( | |||||
| py, | |||||
| "Dirstate error".to_string(), | |||||
| )), | |||||
| } | |||||
| } | |||||
| def write_v2( | |||||
| &self, | |||||
| now: PyObject | |||||
| ) -> PyResult<PyBytes> { | |||||
| let now = Timestamp(now.extract(py)?); | |||||
| let mut inner = self.inner(py).borrow_mut(); | let mut inner = self.inner(py).borrow_mut(); | ||||
| let result = if use_dirstate_v2 { | let result = inner.pack_v2(now); | ||||
| inner.pack_v2(parents, now) | |||||
| } else { | |||||
| inner.pack_v1(parents, now) | |||||
| }; | |||||
| match result { | match result { | ||||
| Ok(packed) => Ok(PyBytes::new(py, &packed)), | Ok(packed) => Ok(PyBytes::new(py, &packed)), | ||||
| Err(_) => Err(PyErr::new::<exc::OSError, _>( | Err(_) => Err(PyErr::new::<exc::OSError, _>( | ||||
| py, | py, | ||||
| "Dirstate error".to_string(), | "Dirstate error".to_string(), | ||||
| )), | )), | ||||
| } | } | ||||
| } | } | ||||
| fn pack_v1( | fn pack_v1( | ||||
| &mut self, | &mut self, | ||||
| parents: DirstateParents, | parents: DirstateParents, | ||||
| now: Timestamp, | now: Timestamp, | ||||
| ) -> Result<Vec<u8>, DirstateError> { | ) -> Result<Vec<u8>, DirstateError> { | ||||
| self.get_mut().pack_v1(parents, now) | self.get_mut().pack_v1(parents, now) | ||||
| } | } | ||||
| fn pack_v2( | fn pack_v2(&mut self, now: Timestamp) -> Result<Vec<u8>, DirstateError> { | ||||
| &mut self, | self.get_mut().pack_v2(now) | ||||
| parents: DirstateParents, | |||||
| now: Timestamp, | |||||
| ) -> Result<Vec<u8>, DirstateError> { | |||||
| self.get_mut().pack_v2(parents, now) | |||||
| } | } | ||||
| fn status<'a>( | fn status<'a>( | ||||
| &'a mut self, | &'a mut self, | ||||
| matcher: &'a (dyn Matcher + Sync), | matcher: &'a (dyn Matcher + Sync), | ||||
| root_dir: PathBuf, | root_dir: PathBuf, | ||||
| ignore_files: Vec<PathBuf>, | ignore_files: Vec<PathBuf>, | ||||
| options: StatusOptions, | options: StatusOptions, | ||||
| /// language cannot represent a lifetime referencing a sibling field. | /// language cannot represent a lifetime referencing a sibling field. | ||||
| /// This is not quite a self-referencial struct (moving this struct is not | /// This is not quite a self-referencial struct (moving this struct is not | ||||
| /// a problem as it doesn’t change the address of the bytes buffer owned | /// a problem as it doesn’t change the address of the bytes buffer owned | ||||
| /// by `PyBytes`) but touches similar borrow-checker limitations. | /// by `PyBytes`) but touches similar borrow-checker limitations. | ||||
| ptr: *mut (), | ptr: *mut (), | ||||
| } | } | ||||
| impl OwningDirstateMap { | impl OwningDirstateMap { | ||||
| pub fn new( | pub fn new_v1( | ||||
| py: Python, | py: Python, | ||||
| on_disk: PyBytes, | on_disk: PyBytes, | ||||
| use_dirstate_v2: bool, | |||||
| ) -> Result<(Self, Option<DirstateParents>), DirstateError> { | ) -> Result<(Self, Option<DirstateParents>), DirstateError> { | ||||
| let bytes: &'_ [u8] = on_disk.data(py); | let bytes: &'_ [u8] = on_disk.data(py); | ||||
| let (map, parents) = if use_dirstate_v2 { | let (map, parents) = DirstateMap::new_v1(bytes)?; | ||||
| DirstateMap::new_v2(bytes)? | |||||
| } else { | |||||
| DirstateMap::new_v1(bytes)? | |||||
| }; | |||||
| // Like in `bytes` above, this `'_` lifetime parameter borrows from | // Like in `bytes` above, this `'_` lifetime parameter borrows from | ||||
| // the bytes buffer owned by `on_disk`. | // the bytes buffer owned by `on_disk`. | ||||
| let ptr: *mut DirstateMap<'_> = Box::into_raw(Box::new(map)); | let ptr: *mut DirstateMap<'_> = Box::into_raw(Box::new(map)); | ||||
| // Erase the pointed type entirely in order to erase the lifetime. | // Erase the pointed type entirely in order to erase the lifetime. | ||||
| let ptr: *mut () = ptr.cast(); | let ptr: *mut () = ptr.cast(); | ||||
| Ok((Self { on_disk, ptr }, parents)) | Ok((Self { on_disk, ptr }, parents)) | ||||
| } | } | ||||
| pub fn new_v2( | |||||
| py: Python, | |||||
| on_disk: PyBytes, | |||||
| ) -> Result<Self, DirstateError> { | |||||
| let bytes: &'_ [u8] = on_disk.data(py); | |||||
| let map = DirstateMap::new_v2(bytes)?; | |||||
| // Like in `bytes` above, this `'_` lifetime parameter borrows from | |||||
| // the bytes buffer owned by `on_disk`. | |||||
| let ptr: *mut DirstateMap<'_> = Box::into_raw(Box::new(map)); | |||||
| // Erase the pointed type entirely in order to erase the lifetime. | |||||
| let ptr: *mut () = ptr.cast(); | |||||
| Ok(Self { on_disk, ptr }) | |||||
| } | |||||
| pub fn get_mut<'a>(&'a mut self) -> &'a mut DirstateMap<'a> { | pub fn get_mut<'a>(&'a mut self) -> &'a mut DirstateMap<'a> { | ||||
| // SAFETY: We cast the type-erased pointer back to the same type it had | // SAFETY: We cast the type-erased pointer back to the same type it had | ||||
| // in `new`, except with a different lifetime parameter. This time we | // in `new`, except with a different lifetime parameter. This time we | ||||
| // connect the lifetime to that of `self`. This cast is valid because | // connect the lifetime to that of `self`. This cast is valid because | ||||
| // `self` owns the same `PyBytes` whose buffer `DirstateMap` | // `self` owns the same `PyBytes` whose buffer `DirstateMap` | ||||
| // references. That buffer has a stable memory address because the byte | // references. That buffer has a stable memory address because the byte | ||||
| // string value of a `PyBytes` is immutable. | // string value of a `PyBytes` is immutable. | ||||
| let ptr: *mut DirstateMap<'a> = self.ptr.cast(); | let ptr: *mut DirstateMap<'a> = self.ptr.cast(); | ||||
| use crate::ui::utf8_to_local; | use crate::ui::utf8_to_local; | ||||
| use crate::ui::UiError; | use crate::ui::UiError; | ||||
| use crate::NoRepoInCwdError; | use crate::NoRepoInCwdError; | ||||
| use format_bytes::format_bytes; | use format_bytes::format_bytes; | ||||
| use hg::config::{ConfigError, ConfigParseError, ConfigValueParseError}; | use hg::config::{ConfigError, ConfigParseError, ConfigValueParseError}; | ||||
| use hg::dirstate_tree::on_disk::DirstateV2ParseError; | |||||
| use hg::errors::HgError; | use hg::errors::HgError; | ||||
| use hg::exit_codes; | use hg::exit_codes; | ||||
| use hg::repo::RepoError; | use hg::repo::RepoError; | ||||
| use hg::revlog::revlog::RevlogError; | use hg::revlog::revlog::RevlogError; | ||||
| use hg::utils::files::get_bytes_from_path; | use hg::utils::files::get_bytes_from_path; | ||||
| use hg::{DirstateError, DirstateMapError, StatusError}; | use hg::{DirstateError, DirstateMapError, StatusError}; | ||||
| use std::convert::From; | use std::convert::From; | ||||
| impl From<DirstateError> for CommandError { | impl From<DirstateError> for CommandError { | ||||
| fn from(error: DirstateError) -> Self { | fn from(error: DirstateError) -> Self { | ||||
| match error { | match error { | ||||
| DirstateError::Common(error) => error.into(), | DirstateError::Common(error) => error.into(), | ||||
| DirstateError::Map(error) => error.into(), | DirstateError::Map(error) => error.into(), | ||||
| } | } | ||||
| } | } | ||||
| } | } | ||||
| impl From<DirstateV2ParseError> for CommandError { | |||||
| fn from(error: DirstateV2ParseError) -> Self { | |||||
| HgError::from(error).into() | |||||
| } | |||||
| } | |||||
| if py2exehacked: | if py2exehacked: | ||||
| cmdclass['py2exe'] = hgbuildpy2exe | cmdclass['py2exe'] = hgbuildpy2exe | ||||
| packages = [ | packages = [ | ||||
| 'mercurial', | 'mercurial', | ||||
| 'mercurial.cext', | 'mercurial.cext', | ||||
| 'mercurial.cffi', | 'mercurial.cffi', | ||||
| 'mercurial.defaultrc', | 'mercurial.defaultrc', | ||||
| 'mercurial.dirstateutils', | |||||
| 'mercurial.helptext', | 'mercurial.helptext', | ||||
| 'mercurial.helptext.internals', | 'mercurial.helptext.internals', | ||||
| 'mercurial.hgweb', | 'mercurial.hgweb', | ||||
| 'mercurial.interfaces', | 'mercurial.interfaces', | ||||
| 'mercurial.pure', | 'mercurial.pure', | ||||
| 'mercurial.templates', | 'mercurial.templates', | ||||
| 'mercurial.thirdparty', | 'mercurial.thirdparty', | ||||
| 'mercurial.thirdparty.attr', | 'mercurial.thirdparty.attr', | ||||
| # parsing 'fakenow' in YYYYmmddHHMM format makes comparison between | # parsing 'fakenow' in YYYYmmddHHMM format makes comparison between | ||||
| # 'fakenow' value and 'touch -t YYYYmmddHHMM' argument easy | # 'fakenow' value and 'touch -t YYYYmmddHHMM' argument easy | ||||
| fakenow = dateutil.parsedate(fakenow, [b'%Y%m%d%H%M'])[0] | fakenow = dateutil.parsedate(fakenow, [b'%Y%m%d%H%M'])[0] | ||||
| if rustmod is not None: | if rustmod is not None: | ||||
| # The Rust implementation does not use public parse/pack dirstate | # The Rust implementation does not use public parse/pack dirstate | ||||
| # to prevent conversion round-trips | # to prevent conversion round-trips | ||||
| orig_dirstatemap_write = dirstatemapmod.dirstatemap.write | orig_dirstatemap_write = dirstatemapmod.dirstatemap.write | ||||
| wrapper = lambda self, st, now: orig_dirstatemap_write( | wrapper = lambda self, tr, st, now: orig_dirstatemap_write( | ||||
| self, st, fakenow | self, tr, st, fakenow | ||||
| ) | ) | ||||
| dirstatemapmod.dirstatemap.write = wrapper | dirstatemapmod.dirstatemap.write = wrapper | ||||
| orig_dirstate_getfsnow = dirstate._getfsnow | orig_dirstate_getfsnow = dirstate._getfsnow | ||||
| wrapper = lambda *args: pack_dirstate(fakenow, orig_pack_dirstate, *args) | wrapper = lambda *args: pack_dirstate(fakenow, orig_pack_dirstate, *args) | ||||
| orig_module = parsers | orig_module = parsers | ||||
| orig_pack_dirstate = parsers.pack_dirstate | orig_pack_dirstate = parsers.pack_dirstate | ||||
| debugcomplete | debugcomplete | ||||
| debugconfig | debugconfig | ||||
| debugcreatestreamclonebundle | debugcreatestreamclonebundle | ||||
| debugdag | debugdag | ||||
| debugdata | debugdata | ||||
| debugdate | debugdate | ||||
| debugdeltachain | debugdeltachain | ||||
| debugdirstate | debugdirstate | ||||
| debugdirstateignorepatternshash | |||||
| debugdiscovery | debugdiscovery | ||||
| debugdownload | debugdownload | ||||
| debugextensions | debugextensions | ||||
| debugfileset | debugfileset | ||||
| debugformat | debugformat | ||||
| debugfsinfo | debugfsinfo | ||||
| debuggetbundle | debuggetbundle | ||||
| debugignore | debugignore | ||||
| debugcolor: style | debugcolor: style | ||||
| debugcommands: | debugcommands: | ||||
| debugcomplete: options | debugcomplete: options | ||||
| debugcreatestreamclonebundle: | debugcreatestreamclonebundle: | ||||
| debugdag: tags, branches, dots, spaces | debugdag: tags, branches, dots, spaces | ||||
| debugdata: changelog, manifest, dir | debugdata: changelog, manifest, dir | ||||
| debugdate: extended | debugdate: extended | ||||
| debugdeltachain: changelog, manifest, dir, template | debugdeltachain: changelog, manifest, dir, template | ||||
| debugdirstateignorepatternshash: | |||||
| debugdirstate: nodates, dates, datesort, dirs | debugdirstate: nodates, dates, datesort, dirs | ||||
| debugdiscovery: old, nonheads, rev, seed, local-as-revs, remote-as-revs, ssh, remotecmd, insecure, template | debugdiscovery: old, nonheads, rev, seed, local-as-revs, remote-as-revs, ssh, remotecmd, insecure, template | ||||
| debugdownload: output | debugdownload: output | ||||
| debugextensions: template | debugextensions: template | ||||
| debugfileset: rev, all-files, show-matcher, show-stage | debugfileset: rev, all-files, show-matcher, show-stage | ||||
| debugformat: template | debugformat: template | ||||
| debugfsinfo: | debugfsinfo: | ||||
| debuggetbundle: head, common, type | debuggetbundle: head, common, type | ||||
| dir1/file2 is ignored | dir1/file2 is ignored | ||||
| (ignore rule in $TESTTMP\ignorerepo\.hgignore, line 4: 'dir1/file*') | (ignore rule in $TESTTMP\ignorerepo\.hgignore, line 4: 'dir1/file*') | ||||
| $ hg up -qC . | $ hg up -qC . | ||||
| #endif | #endif | ||||
| #if dirstate-v2 | #if dirstate-v2 | ||||
| Check the hash of ignore patterns written in the dirstate at offset | Check the hash of ignore patterns written in the dirstate | ||||
| 12 + 20 + 20 + 4 + 4 + 4 + 4 = 68 | |||||
| $ hg status > /dev/null | $ hg status > /dev/null | ||||
| $ cat .hg/testhgignore .hg/testhgignorerel .hgignore dir2/.hgignore dir1/.hgignore dir1/.hgignoretwo | $TESTDIR/f --sha1 | $ cat .hg/testhgignore .hg/testhgignorerel .hgignore dir2/.hgignore dir1/.hgignore dir1/.hgignoretwo | $TESTDIR/f --sha1 | ||||
| sha1=6e315b60f15fb5dfa02be00f3e2c8f923051f5ff | sha1=6e315b60f15fb5dfa02be00f3e2c8f923051f5ff | ||||
| >>> import binascii; print(binascii.hexlify(open(".hg/dirstate", "rb").read()[68:][:20]).decode()) | $ hg debugdirstateignorepatternshash | ||||
| 6e315b60f15fb5dfa02be00f3e2c8f923051f5ff | 6e315b60f15fb5dfa02be00f3e2c8f923051f5ff | ||||
| $ echo rel > .hg/testhgignorerel | $ echo rel > .hg/testhgignorerel | ||||
| $ hg status > /dev/null | $ hg status > /dev/null | ||||
| $ cat .hg/testhgignore .hg/testhgignorerel .hgignore dir2/.hgignore dir1/.hgignore dir1/.hgignoretwo | $TESTDIR/f --sha1 | $ cat .hg/testhgignore .hg/testhgignorerel .hgignore dir2/.hgignore dir1/.hgignore dir1/.hgignoretwo | $TESTDIR/f --sha1 | ||||
| sha1=dea19cc7119213f24b6b582a4bae7b0cb063e34e | sha1=dea19cc7119213f24b6b582a4bae7b0cb063e34e | ||||
| >>> import binascii; print(binascii.hexlify(open(".hg/dirstate", "rb").read()[68:][:20]).decode()) | $ hg debugdirstateignorepatternshash | ||||
| dea19cc7119213f24b6b582a4bae7b0cb063e34e | dea19cc7119213f24b6b582a4bae7b0cb063e34e | ||||
| #endif | #endif | ||||
How does this work once we move to append-only (I'm not seeing it in the next changesets)?