( )⚙ D11633 dirstate-v2: Truncate directory mtimes to 31 bits of seconds

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11633

dirstate-v2: Truncate directory mtimes to 31 bits of seconds
ClosedPublic

Authored by SimonSapin on Oct 12 2021, 1:00 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG320de901896a: dirstate-v2: Truncate directory mtimes to 31 bits of seconds
Summary

… instead of 64 bits, while keeping the sub-second presision.
This brings the size of one timestamp from 12 bytes to 8 bytes.

31 bits is chosen instead of 32 because that’s already what happens for the
mtime of files and symlinks, because dirstate-v1 uses negative i32 values as
markers.

Later we’ll add sub-second precision for file/symlink mtimes, making their
dirstate-v2 representation the same as for directories.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

SimonSapin created this revision.Oct 12 2021, 1:00 PM
SimonSapin added a commit: rHG320de901896a: dirstate-v2: Truncate directory mtimes to 31 bits of seconds.Oct 12 2021, 2:03 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG320de901896a: dirstate-v2: Truncate directory mtimes to 31 bits of seconds (authored by SimonSapin). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 30754

mercurial/helptext/internals/dirstate-v2.txt

Otherwise, a 32-bit integer for expected modified time of the file Otherwise, a 32-bit integer for expected modified time of the file
(as in `stat_result.st_mtime`), (as in `stat_result.st_mtime`),
truncated to its 31 least-significant bits. truncated to its 31 least-significant bits.
Unlike in dirstate-v1, negative values are not used. Unlike in dirstate-v1, negative values are not used.
If an untracked node `HAS_MTIME` *unset*, this space is unused: If an untracked node `HAS_MTIME` *unset*, this space is unused:
* Offset 31: * Offset 31:
12 bytes set to zero 12 unused bytes, set to zero
If an untracked node `HAS_MTIME` *set*, If an untracked node `HAS_MTIME` *set*,
what follows is the modification time of a directory what follows is the modification time of a directory
represented similarly to the C `timespec` struct: represented similarly to the C `timespec` struct:
* Offset 31: * Offset 31:
4 unused bytes, set to zero
* Offset 35:
The number of seconds elapsed since the Unix epoch, The number of seconds elapsed since the Unix epoch,
as a signed (two’s complement) 64-bit integer. truncated to its lower 31 bits,
as a 32-bit integer.
* Offset 39: * Offset 39:
The number of nanoseconds elapsed since The sub-second number of nanoseconds elapsed since the Unix epoch,
the instant specified by the previous field alone,
as 32-bit integer. as 32-bit integer.
Always greater than or equal to zero, and strictly less than a billion. Always greater than or equal to zero, and strictly less than a billion.
Increasing this component makes the modification time
go forward in time regardless of the sign of the seconds component.
The presence of a directory modification time means that at some point, The presence of a directory modification time means that at some point,
this path in the working directory was observed: this path in the working directory was observed:
- To be a directory - To be a directory
- With the given modification time - With the given modification time
- That time was already strictly in the past when observed, - That time was already strictly in the past when observed,
meaning that later changes cannot happen in the same clock tick meaning that later changes cannot happen in the same clock tick

rust/hg-core/src/dirstate/entry.rs

use crate::dirstate_tree::on_disk::DirstateV2ParseError;
use crate::errors::HgError; use crate::errors::HgError;
use bitflags::bitflags; use bitflags::bitflags;
use std::convert::TryFrom; use std::convert::TryFrom;
use std::time::{SystemTime, UNIX_EPOCH}; use std::time::{SystemTime, UNIX_EPOCH};
#[derive(Copy, Clone, Debug, Eq, PartialEq)] #[derive(Copy, Clone, Debug, Eq, PartialEq)]
pub enum EntryState { pub enum EntryState {
Normal, Normal,
bitflags! { bitflags! {
pub(crate) struct Flags: u8 { pub(crate) struct Flags: u8 {
const WDIR_TRACKED = 1 << 0; const WDIR_TRACKED = 1 << 0;
const P1_TRACKED = 1 << 1; const P1_TRACKED = 1 << 1;
const P2_INFO = 1 << 2; const P2_INFO = 1 << 2;
} }
} }
#[derive(Copy, Clone, PartialEq)] /// A Unix timestamp with nanoseconds precision
pub struct Timestamp { #[derive(Copy, Clone)]
seconds: i64, pub struct TruncatedTimestamp {
truncated_seconds: u32,
/// In `0 .. 1_000_000_000`. /// Always in the `0 .. 1_000_000_000` range.
///
/// This timestamp is after `(seconds, 0)` by this many nanoseconds.
nanoseconds: u32, nanoseconds: u32,
} }
impl Timestamp { impl TruncatedTimestamp {
pub fn new(seconds: i64, nanoseconds: u32) -> Self { /// Constructs from a timestamp potentially outside of the supported range,
/// and truncate the seconds components to its lower 31 bits.
///
/// Panics if the nanoseconds components is not in the expected range.
pub fn new_truncate(seconds: i64, nanoseconds: u32) -> Self {
assert!(nanoseconds < NSEC_PER_SEC);
Self { Self {
seconds, truncated_seconds: seconds as u32 & RANGE_MASK_31BIT,
nanoseconds, nanoseconds,
} }
} }
pub fn seconds(&self) -> i64 { /// Construct from components. Returns an error if they are not in the
self.seconds /// expcted range.
pub fn from_already_truncated(
truncated_seconds: u32,
nanoseconds: u32,
) -> Result<Self, DirstateV2ParseError> {
if truncated_seconds & !RANGE_MASK_31BIT == 0
&& nanoseconds < NSEC_PER_SEC
{
Ok(Self {
truncated_seconds,
nanoseconds,
})
} else {
Err(DirstateV2ParseError)
}
}
/// The lower 31 bits of the number of seconds since the epoch.
pub fn truncated_seconds(&self) -> u32 {
self.truncated_seconds
} }
/// The sub-second component of this timestamp, in nanoseconds.
/// Always in the `0 .. 1_000_000_000` range.
///
/// This timestamp is after `(seconds, 0)` by this many nanoseconds.
pub fn nanoseconds(&self) -> u32 { pub fn nanoseconds(&self) -> u32 {
self.nanoseconds self.nanoseconds
} }
/// Returns whether two timestamps are equal modulo 2**31 seconds.
///
/// If this returns `true`, the original values converted from `SystemTime`
/// or given to `new_truncate` were very likely equal. A false positive is
/// possible if they were exactly a multiple of 2**31 seconds apart (around
/// 68 years). This is deemed very unlikely to happen by chance, especially
/// on filesystems that support sub-second precision.
///
/// If someone is manipulating the modification times of some files to
/// intentionally make `hg status` return incorrect results, not truncating
/// wouldn’t help much since they can set exactly the expected timestamp.
pub fn very_likely_equal(&self, other: &Self) -> bool {
self.truncated_seconds == other.truncated_seconds
&& self.nanoseconds == other.nanoseconds
}
} }
impl From<SystemTime> for Timestamp { impl From<SystemTime> for TruncatedTimestamp {
fn from(system_time: SystemTime) -> Self { fn from(system_time: SystemTime) -> Self {
// On Unix, `SystemTime` is a wrapper for the `timespec` C struct: // On Unix, `SystemTime` is a wrapper for the `timespec` C struct:
// https://www.gnu.org/software/libc/manual/html_node/Time-Types.html#index-struct-timespec // https://www.gnu.org/software/libc/manual/html_node/Time-Types.html#index-struct-timespec
// We want to effectively access its fields, but the Rust standard // We want to effectively access its fields, but the Rust standard
// library does not expose them. The best we can do is: // library does not expose them. The best we can do is:
let seconds; let seconds;
let nanoseconds; let nanoseconds;
match system_time.duration_since(UNIX_EPOCH) { match system_time.duration_since(UNIX_EPOCH) {
let negative_nanos = negative.subsec_nanos(); let negative_nanos = negative.subsec_nanos();
if negative_nanos == 0 { if negative_nanos == 0 {
seconds = -negative_secs; seconds = -negative_secs;
nanoseconds = 0; nanoseconds = 0;
} else { } else {
// For example if `system_time` was 4.3!seconds before // For example if `system_time` was 4.3!seconds before
// the Unix epoch we get a Duration that represents // the Unix epoch we get a Duration that represents
// `(-4, -0.3)` but we want `(-5, +0.7)`: // `(-4, -0.3)` but we want `(-5, +0.7)`:
const NSEC_PER_SEC: u32 = 1_000_000_000;
seconds = -1 - negative_secs; seconds = -1 - negative_secs;
nanoseconds = NSEC_PER_SEC - negative_nanos; nanoseconds = NSEC_PER_SEC - negative_nanos;
} }
} }
}; };
Self { Self::new_truncate(seconds, nanoseconds)
seconds,
nanoseconds,
}
} }
} }
pub const V1_RANGEMASK: i32 = 0x7FFFFFFF; const NSEC_PER_SEC: u32 = 1_000_000_000;
const RANGE_MASK_31BIT: u32 = 0x7FFF_FFFF;
pub const MTIME_UNSET: i32 = -1; pub const MTIME_UNSET: i32 = -1;
/// A `DirstateEntry` with a size of `-2` means that it was merged from the /// A `DirstateEntry` with a size of `-2` means that it was merged from the
/// other parent. This allows revert to pick the right status back during a /// other parent. This allows revert to pick the right status back during a
/// merge. /// merge.
pub const SIZE_FROM_OTHER_PARENT: i32 = -2; pub const SIZE_FROM_OTHER_PARENT: i32 = -2;
/// A special value used for internal representation of special case in /// A special value used for internal representation of special case in

rust/hg-core/src/dirstate_tree/dirstate_map.rs

use bytes_cast::BytesCast; use bytes_cast::BytesCast;
use micro_timer::timed; use micro_timer::timed;
use std::borrow::Cow; use std::borrow::Cow;
use std::convert::TryInto; use std::convert::TryInto;
use std::path::PathBuf; use std::path::PathBuf;
use super::on_disk; use super::on_disk;
use super::on_disk::DirstateV2ParseError; use super::on_disk::DirstateV2ParseError;
use super::owning::OwningDirstateMap; use super::owning::OwningDirstateMap;
use super::path_with_basename::WithBasename; use super::path_with_basename::WithBasename;
use crate::dirstate::parsers::pack_entry; use crate::dirstate::parsers::pack_entry;
use crate::dirstate::parsers::packed_entry_size; use crate::dirstate::parsers::packed_entry_size;
use crate::dirstate::parsers::parse_dirstate_entries; use crate::dirstate::parsers::parse_dirstate_entries;
use crate::dirstate::parsers::Timestamp; use crate::dirstate::parsers::Timestamp;
use crate::dirstate::CopyMapIter; use crate::dirstate::CopyMapIter;
use crate::dirstate::StateMapIter; use crate::dirstate::StateMapIter;
use crate::dirstate::TruncatedTimestamp;
use crate::dirstate::SIZE_FROM_OTHER_PARENT; use crate::dirstate::SIZE_FROM_OTHER_PARENT;
use crate::dirstate::SIZE_NON_NORMAL; use crate::dirstate::SIZE_NON_NORMAL;
use crate::matchers::Matcher; use crate::matchers::Matcher;
use crate::utils::hg_path::{HgPath, HgPathBuf}; use crate::utils::hg_path::{HgPath, HgPathBuf};
use crate::DirstateEntry; use crate::DirstateEntry;
use crate::DirstateError; use crate::DirstateError;
use crate::DirstateParents; use crate::DirstateParents;
use crate::DirstateStatus; use crate::DirstateStatus;
pub(super) fn state( pub(super) fn state(
&self, &self,
) -> Result<Option<EntryState>, DirstateV2ParseError> { ) -> Result<Option<EntryState>, DirstateV2ParseError> {
Ok(self.entry()?.map(|e| e.state())) Ok(self.entry()?.map(|e| e.state()))
} }
pub(super) fn cached_directory_mtime( pub(super) fn cached_directory_mtime(
&self, &self,
) -> Option<crate::dirstate::Timestamp> { ) -> Result<Option<TruncatedTimestamp>, DirstateV2ParseError> {
match self { match self {
NodeRef::InMemory(_path, node) => match node.data { NodeRef::InMemory(_path, node) => Ok(match node.data {
NodeData::CachedDirectory { mtime } => Some(mtime), NodeData::CachedDirectory { mtime } => Some(mtime),
_ => None, _ => None,
}, }),
NodeRef::OnDisk(node) => node.cached_directory_mtime(), NodeRef::OnDisk(node) => node.cached_directory_mtime(),
} }
} }
pub(super) fn descendants_with_entry_count(&self) -> u32 { pub(super) fn descendants_with_entry_count(&self) -> u32 {
match self { match self {
NodeRef::InMemory(_path, node) => { NodeRef::InMemory(_path, node) => {
node.descendants_with_entry_count node.descendants_with_entry_count
/// How many (non-inclusive) descendants of this node have an entry whose /// How many (non-inclusive) descendants of this node have an entry whose
/// state is "tracked". /// state is "tracked".
pub(super) tracked_descendants_count: u32, pub(super) tracked_descendants_count: u32,
} }
pub(super) enum NodeData { pub(super) enum NodeData {
Entry(DirstateEntry), Entry(DirstateEntry),
CachedDirectory { mtime: crate::dirstate::Timestamp }, CachedDirectory { mtime: TruncatedTimestamp },
None, None,
} }
impl Default for NodeData { impl Default for NodeData {
fn default() -> Self { fn default() -> Self {
NodeData::None NodeData::None
} }
} }
+ '_, + '_,
> { > {
let map = self.get_map(); let map = self.get_map();
Box::new(filter_map_results(map.iter_nodes(), move |node| { Box::new(filter_map_results(map.iter_nodes(), move |node| {
let debug_tuple = if let Some(entry) = node.entry()? { let debug_tuple = if let Some(entry) = node.entry()? {
entry.debug_tuple() entry.debug_tuple()
} else if !all { } else if !all {
return Ok(None); return Ok(None);
} else if let Some(mtime) = node.cached_directory_mtime() { } else if let Some(mtime) = node.cached_directory_mtime()? {
(b' ', 0, -1, mtime.seconds() as i32) (b' ', 0, -1, mtime.truncated_seconds() as i32)
} else { } else {
(b' ', 0, -1, -1) (b' ', 0, -1, -1)
}; };
Ok(Some((node.full_path(map.on_disk)?, debug_tuple))) Ok(Some((node.full_path(map.on_disk)?, debug_tuple)))
})) }))
} }
} }

rust/hg-core/src/dirstate_tree/on_disk.rs

//! The "version 2" disk representation of the dirstate //! The "version 2" disk representation of the dirstate
//! //!
//! See `mercurial/helptext/internals/dirstate-v2.txt` //! See `mercurial/helptext/internals/dirstate-v2.txt`
use crate::dirstate::Timestamp; use crate::dirstate::TruncatedTimestamp;
use crate::dirstate_tree::dirstate_map::{self, DirstateMap, NodeRef}; use crate::dirstate_tree::dirstate_map::{self, DirstateMap, NodeRef};
use crate::dirstate_tree::path_with_basename::WithBasename; use crate::dirstate_tree::path_with_basename::WithBasename;
use crate::errors::HgError; use crate::errors::HgError;
use crate::utils::hg_path::HgPath; use crate::utils::hg_path::HgPath;
use crate::DirstateEntry; use crate::DirstateEntry;
use crate::DirstateError; use crate::DirstateError;
use crate::DirstateParents; use crate::DirstateParents;
use bitflags::bitflags; use bitflags::bitflags;
use bytes_cast::unaligned::{I32Be, I64Be, U16Be, U32Be}; use bytes_cast::unaligned::{I32Be, U16Be, U32Be};
use bytes_cast::BytesCast; use bytes_cast::BytesCast;
use format_bytes::format_bytes; use format_bytes::format_bytes;
use std::borrow::Cow; use std::borrow::Cow;
use std::convert::{TryFrom, TryInto}; use std::convert::{TryFrom, TryInto};
/// Added at the start of `.hg/dirstate` when the "v2" format is used. /// Added at the start of `.hg/dirstate` when the "v2" format is used.
/// This a redundant sanity check more than an actual "magic number" since /// This a redundant sanity check more than an actual "magic number" since
/// `.hg/requires` already governs which format should be used. /// `.hg/requires` already governs which format should be used.
size: I32Be, size: I32Be,
mtime: I32Be, mtime: I32Be,
} }
/// Duration since the Unix epoch /// Duration since the Unix epoch
#[derive(BytesCast, Copy, Clone)] #[derive(BytesCast, Copy, Clone)]
#[repr(C)] #[repr(C)]
struct PackedTimestamp { struct PackedTimestamp {
seconds: I64Be, _padding: U32Be,
truncated_seconds: U32Be,
/// In `0 .. 1_000_000_000`.
///
/// This timestamp is after `(seconds, 0)` by this many nanoseconds.
nanoseconds: U32Be, nanoseconds: U32Be,
} }
/// Counted in bytes from the start of the file /// Counted in bytes from the start of the file
/// ///
/// NOTE: not supporting `.hg/dirstate` files larger than 4 GiB. /// NOTE: not supporting `.hg/dirstate` files larger than 4 GiB.
type Offset = U32Be; type Offset = U32Be;
) )
} }
pub(super) fn node_data( pub(super) fn node_data(
&self, &self,
) -> Result<dirstate_map::NodeData, DirstateV2ParseError> { ) -> Result<dirstate_map::NodeData, DirstateV2ParseError> {
if self.has_entry() { if self.has_entry() {
Ok(dirstate_map::NodeData::Entry(self.assume_entry())) Ok(dirstate_map::NodeData::Entry(self.assume_entry()))
} else if let Some(mtime) = self.cached_directory_mtime() { } else if let Some(mtime) = self.cached_directory_mtime()? {
Ok(dirstate_map::NodeData::CachedDirectory { mtime }) Ok(dirstate_map::NodeData::CachedDirectory { mtime })
} else { } else {
Ok(dirstate_map::NodeData::None) Ok(dirstate_map::NodeData::None)
} }
} }
pub(super) fn cached_directory_mtime(&self) -> Option<Timestamp> { pub(super) fn cached_directory_mtime(
&self,
) -> Result<Option<TruncatedTimestamp>, DirstateV2ParseError> {
Ok(
if self.flags.contains(Flags::HAS_MTIME) && !self.has_entry() { if self.flags.contains(Flags::HAS_MTIME) && !self.has_entry() {
Some(self.data.as_timestamp()) Some(self.data.as_timestamp()?)
} else { } else {
None None
} },
)
} }
fn assume_entry(&self) -> DirstateEntry { fn assume_entry(&self) -> DirstateEntry {
// TODO: convert through raw bits instead? // TODO: convert through raw bits instead?
let wdir_tracked = self.flags.contains(Flags::WDIR_TRACKED); let wdir_tracked = self.flags.contains(Flags::WDIR_TRACKED);
let p1_tracked = self.flags.contains(Flags::P1_TRACKED); let p1_tracked = self.flags.contains(Flags::P1_TRACKED);
let p2_info = self.flags.contains(Flags::P2_INFO); let p2_info = self.flags.contains(Flags::P2_INFO);
let mode_size = if self.flags.contains(Flags::HAS_MODE_AND_SIZE) { let mode_size = if self.flags.contains(Flags::HAS_MODE_AND_SIZE) {
let raw_entry = Entry { let raw_entry = Entry {
mode: mode.into(), mode: mode.into(),
size: size.into(), size: size.into(),
mtime: mtime.into(), mtime: mtime.into(),
}; };
(flags, raw_entry) (flags, raw_entry)
} }
fn from_timestamp(timestamp: Timestamp) -> Self { fn from_timestamp(timestamp: TruncatedTimestamp) -> Self {
let packed = PackedTimestamp { let packed = PackedTimestamp {
seconds: timestamp.seconds().into(), _padding: 0.into(),
truncated_seconds: timestamp.truncated_seconds().into(),
nanoseconds: timestamp.nanoseconds().into(), nanoseconds: timestamp.nanoseconds().into(),
}; };
// Safety: both types implement the `ByteCast` trait, so we could // Safety: both types implement the `ByteCast` trait, so we could
// safely use `as_bytes` and `from_bytes` to do this conversion. Using // safely use `as_bytes` and `from_bytes` to do this conversion. Using
// `transmute` instead makes the compiler check that the two types // `transmute` instead makes the compiler check that the two types
// have the same size, which eliminates the error case of // have the same size, which eliminates the error case of
// `from_bytes`. // `from_bytes`.
unsafe { std::mem::transmute::<PackedTimestamp, Entry>(packed) } unsafe { std::mem::transmute::<PackedTimestamp, Entry>(packed) }
} }
fn as_timestamp(self) -> Timestamp { fn as_timestamp(self) -> Result<TruncatedTimestamp, DirstateV2ParseError> {
// Safety: same as above in `from_timestamp` // Safety: same as above in `from_timestamp`
let packed = let packed =
unsafe { std::mem::transmute::<Entry, PackedTimestamp>(self) }; unsafe { std::mem::transmute::<Entry, PackedTimestamp>(self) };
Timestamp::new(packed.seconds.get(), packed.nanoseconds.get()) TruncatedTimestamp::from_already_truncated(
packed.truncated_seconds.get(),
packed.nanoseconds.get(),
)
} }
} }
fn read_hg_path( fn read_hg_path(
on_disk: &[u8], on_disk: &[u8],
slice: PathSlice, slice: PathSlice,
) -> Result<&HgPath, DirstateV2ParseError> { ) -> Result<&HgPath, DirstateV2ParseError> {
read_slice(on_disk, slice.start, slice.len.get()).map(HgPath::new) read_slice(on_disk, slice.start, slice.len.get()).map(HgPath::new)

rust/hg-core/src/dirstate_tree/status.rs

use crate::dirstate::entry::Timestamp; use crate::dirstate::entry::TruncatedTimestamp;
use crate::dirstate::status::IgnoreFnType; use crate::dirstate::status::IgnoreFnType;
use crate::dirstate_tree::dirstate_map::BorrowedPath; use crate::dirstate_tree::dirstate_map::BorrowedPath;
use crate::dirstate_tree::dirstate_map::ChildNodesRef; use crate::dirstate_tree::dirstate_map::ChildNodesRef;
use crate::dirstate_tree::dirstate_map::DirstateMap; use crate::dirstate_tree::dirstate_map::DirstateMap;
use crate::dirstate_tree::dirstate_map::NodeData; use crate::dirstate_tree::dirstate_map::NodeData;
use crate::dirstate_tree::dirstate_map::NodeRef; use crate::dirstate_tree::dirstate_map::NodeRef;
use crate::dirstate_tree::on_disk::DirstateV2ParseError; use crate::dirstate_tree::on_disk::DirstateV2ParseError;
use crate::matchers::get_ignore_function; use crate::matchers::get_ignore_function;
/// Bag of random things needed by various parts of the algorithm. Reduces the /// Bag of random things needed by various parts of the algorithm. Reduces the
/// number of parameters passed to functions. /// number of parameters passed to functions.
struct StatusCommon<'a, 'tree, 'on_disk: 'tree> { struct StatusCommon<'a, 'tree, 'on_disk: 'tree> {
dmap: &'tree DirstateMap<'on_disk>, dmap: &'tree DirstateMap<'on_disk>,
options: StatusOptions, options: StatusOptions,
matcher: &'a (dyn Matcher + Sync), matcher: &'a (dyn Matcher + Sync),
ignore_fn: IgnoreFnType<'a>, ignore_fn: IgnoreFnType<'a>,
outcome: Mutex<DirstateStatus<'on_disk>>, outcome: Mutex<DirstateStatus<'on_disk>>,
new_cachable_directories: Mutex<Vec<(Cow<'on_disk, HgPath>, Timestamp)>>, new_cachable_directories:
Mutex<Vec<(Cow<'on_disk, HgPath>, TruncatedTimestamp)>>,
outated_cached_directories: Mutex<Vec<Cow<'on_disk, HgPath>>>, outated_cached_directories: Mutex<Vec<Cow<'on_disk, HgPath>>>,
/// Whether ignore files like `.hgignore` have changed since the previous /// Whether ignore files like `.hgignore` have changed since the previous
/// time a `status()` call wrote their hash to the dirstate. `None` means /// time a `status()` call wrote their hash to the dirstate. `None` means
/// we don’t know as this run doesn’t list either ignored or uknown files /// we don’t know as this run doesn’t list either ignored or uknown files
/// and therefore isn’t reading `.hgignore`. /// and therefore isn’t reading `.hgignore`.
ignore_patterns_have_changed: Option<bool>, ignore_patterns_have_changed: Option<bool>,
.push((hg_path.to_owned().into(), BadMatch::OsError(errno))) .push((hg_path.to_owned().into(), BadMatch::OsError(errno)))
} }
fn check_for_outdated_directory_cache( fn check_for_outdated_directory_cache(
&self, &self,
dirstate_node: &NodeRef<'tree, 'on_disk>, dirstate_node: &NodeRef<'tree, 'on_disk>,
) -> Result<(), DirstateV2ParseError> { ) -> Result<(), DirstateV2ParseError> {
if self.ignore_patterns_have_changed == Some(true) if self.ignore_patterns_have_changed == Some(true)
&& dirstate_node.cached_directory_mtime().is_some() && dirstate_node.cached_directory_mtime()?.is_some()
{ {
self.outated_cached_directories.lock().unwrap().push( self.outated_cached_directories.lock().unwrap().push(
dirstate_node dirstate_node
.full_path_borrowed(self.dmap.on_disk)? .full_path_borrowed(self.dmap.on_disk)?
.detach_from_tree(), .detach_from_tree(),
) )
} }
Ok(()) Ok(())
} }
/// If this returns true, we can get accurate results by only using /// If this returns true, we can get accurate results by only using
/// `symlink_metadata` for child nodes that exist in the dirstate and don’t /// `symlink_metadata` for child nodes that exist in the dirstate and don’t
/// need to call `read_dir`. /// need to call `read_dir`.
fn can_skip_fs_readdir( fn can_skip_fs_readdir(
&self, &self,
directory_metadata: Option<&std::fs::Metadata>, directory_metadata: Option<&std::fs::Metadata>,
cached_directory_mtime: Option<Timestamp>, cached_directory_mtime: Option<TruncatedTimestamp>,
) -> bool { ) -> bool {
if !self.options.list_unknown && !self.options.list_ignored { if !self.options.list_unknown && !self.options.list_ignored {
// All states that we care about listing have corresponding // All states that we care about listing have corresponding
// dirstate entries. // dirstate entries.
// This happens for example with `hg status -mard`. // This happens for example with `hg status -mard`.
return true; return true;
} }
if !self.options.list_ignored if !self.options.list_ignored
&& self.ignore_patterns_have_changed == Some(false) && self.ignore_patterns_have_changed == Some(false)
{ {
if let Some(cached_mtime) = cached_directory_mtime { if let Some(cached_mtime) = cached_directory_mtime {
// The dirstate contains a cached mtime for this directory, set // The dirstate contains a cached mtime for this directory, set
// by a previous run of the `status` algorithm which found this // by a previous run of the `status` algorithm which found this
// directory eligible for `read_dir` caching. // directory eligible for `read_dir` caching.
if let Some(meta) = directory_metadata { if let Some(meta) = directory_metadata {
if let Ok(current_mtime) = meta.modified() { if let Ok(current_mtime) = meta.modified() {
let current_mtime = Timestamp::from(current_mtime); let truncated =
if current_mtime == cached_mtime { TruncatedTimestamp::from(current_mtime);
if truncated.very_likely_equal(&cached_mtime) {
// The mtime of that directory has not changed // The mtime of that directory has not changed
// since then, which means that the results of // since then, which means that the results of
// `read_dir` should also be unchanged. // `read_dir` should also be unchanged.
return true; return true;
} }
} }
} }
} }
} }
false false
} }
/// Returns whether all child entries of the filesystem directory have a /// Returns whether all child entries of the filesystem directory have a
/// corresponding dirstate node or are ignored. /// corresponding dirstate node or are ignored.
fn traverse_fs_directory_and_dirstate( fn traverse_fs_directory_and_dirstate(
&self, &self,
has_ignored_ancestor: bool, has_ignored_ancestor: bool,
dirstate_nodes: ChildNodesRef<'tree, 'on_disk>, dirstate_nodes: ChildNodesRef<'tree, 'on_disk>,
directory_hg_path: &BorrowedPath<'tree, 'on_disk>, directory_hg_path: &BorrowedPath<'tree, 'on_disk>,
directory_fs_path: &Path, directory_fs_path: &Path,
directory_metadata: Option<&std::fs::Metadata>, directory_metadata: Option<&std::fs::Metadata>,
cached_directory_mtime: Option<Timestamp>, cached_directory_mtime: Option<TruncatedTimestamp>,
is_at_repo_root: bool, is_at_repo_root: bool,
) -> Result<bool, DirstateV2ParseError> { ) -> Result<bool, DirstateV2ParseError> {
if self.can_skip_fs_readdir(directory_metadata, cached_directory_mtime) if self.can_skip_fs_readdir(directory_metadata, cached_directory_mtime)
{ {
dirstate_nodes dirstate_nodes
.par_iter() .par_iter()
.map(|dirstate_node| { .map(|dirstate_node| {
let fs_path = directory_fs_path.join(get_path_from_bytes( let fs_path = directory_fs_path.join(get_path_from_bytes(
let is_at_repo_root = false; let is_at_repo_root = false;
let children_all_have_dirstate_node_or_are_ignored = self let children_all_have_dirstate_node_or_are_ignored = self
.traverse_fs_directory_and_dirstate( .traverse_fs_directory_and_dirstate(
is_ignored, is_ignored,
dirstate_node.children(self.dmap.on_disk)?, dirstate_node.children(self.dmap.on_disk)?,
hg_path, hg_path,
fs_path, fs_path,
Some(fs_metadata), Some(fs_metadata),
dirstate_node.cached_directory_mtime(), dirstate_node.cached_directory_mtime()?,
is_at_repo_root, is_at_repo_root,
)?; )?;
self.maybe_save_directory_mtime( self.maybe_save_directory_mtime(
children_all_have_dirstate_node_or_are_ignored, children_all_have_dirstate_node_or_are_ignored,
fs_metadata, fs_metadata,
dirstate_node, dirstate_node,
)? )?
} else { } else {
// Having the same mtime again is not entirely impossible // Having the same mtime again is not entirely impossible
// since the system clock is not monotonous. It could jump // since the system clock is not monotonous. It could jump
// backward to some point before `directory_mtime`, then a // backward to some point before `directory_mtime`, then a
// directory change could potentially happen during exactly // directory change could potentially happen during exactly
// the wrong tick. // the wrong tick.
// //
// We deem this scenario (unlike the previous one) to be // We deem this scenario (unlike the previous one) to be
// unlikely enough in practice. // unlikely enough in practice.
let timestamp = directory_mtime.into(); let truncated = TruncatedTimestamp::from(directory_mtime);
let cached = dirstate_node.cached_directory_mtime(); let is_up_to_date = if let Some(cached) =
if cached != Some(timestamp) { dirstate_node.cached_directory_mtime()?
{
cached.very_likely_equal(&truncated)
} else {
false
};
if !is_up_to_date {
let hg_path = dirstate_node let hg_path = dirstate_node
.full_path_borrowed(self.dmap.on_disk)? .full_path_borrowed(self.dmap.on_disk)?
.detach_from_tree(); .detach_from_tree();
self.new_cachable_directories self.new_cachable_directories
.lock() .lock()
.unwrap() .unwrap()
.push((hg_path, timestamp)) .push((hg_path, truncated))
} }
} }
} }
} }
Ok(()) Ok(())
} }
/// A file with `EntryState::Normal` in the dirstate was found in the /// A file with `EntryState::Normal` in the dirstate was found in the