This is an archive of the discontinued Mercurial Phabricator instance.

Differential D10744

dirstate-tree: Remove DirstateMap::iter_node_data_mut
ClosedPublic

Authored by SimonSapin on May 19 2021, 12:36 PM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHG73f23e7610f8: dirstate-tree: Remove DirstateMap::iter_node_data_mut
Summary

In an upcoming changeset we want DirstateMap to be able to work directly
with nodes in their "on disk" representation, without always allocating
corresponding in-memory data structures. Nodes would have two possible
representations: one immutable "on disk" refering to the bytes buffer
of the contents of the .hg/dirstate file, and one mutable with HashMap
like the curren data structure.

These nodes would have copy-on-write semantics: when an immutable node
would need to be mutated, instead we allocate new mutable node for it and
its ancestors.

A mutable iterator of the entire tree would still be possible, but it would
become much more expensive since we’d need to allocate mutable nodes for
everything.

Instead, remove this iterator. It was only used to clear ambiguous mtimes
while serializing the DirstateMap. Instead clearing and serialization are
now two separate passes. Clearing first uses an immutable iterator to collect
the paths of nodes that need to be cleared, then accesses only those nodes
mutably.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

SimonSapin created this revision.May 19 2021, 12:36 PM
Alphare requested changes to this revision.May 20 2021, 10:03 AM
Alphare added a subscriber: Alphare.
Alphare added inline comments.
rust/hg-core/src/dirstate/parsers.rs
130

s/mtime_is_ambigous/mtime_is_ambiguous/

This revision now requires changes to proceed.May 20 2021, 10:03 AM
baymax updated this revision to Diff 28168.May 21 2021, 4:37 AM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

baymax updated this revision to Diff 28267.May 30 2021, 3:27 PM

✅ refresh by Heptapod after a successful CI run (🐙 💚)

Alphare accepted this revision.May 31 2021, 3:39 AM
This revision is now accepted and ready to land.May 31 2021, 3:39 AM
SimonSapin added a commit: rHG73f23e7610f8: dirstate-tree: Remove DirstateMap::iter_node_data_mut.May 31 2021, 3:55 AM
Closed by commit rHG73f23e7610f8: dirstate-tree: Remove DirstateMap::iter_node_data_mut (authored by SimonSapin). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 28279

rust/hg-core/src/dirstate/dirstate_map.rs

// dirstate_map.rs // dirstate_map.rs
// //
// Copyright 2019 Raphaël Gomès <rgomes@octobus.net> // Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
// //
// This software may be used and distributed according to the terms of the // This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version. // GNU General Public License version 2 or any later version.
use crate::dirstate::parsers::clear_ambiguous_mtime;
use crate::dirstate::parsers::Timestamp; use crate::dirstate::parsers::Timestamp;
use crate::{ use crate::{
dirstate::EntryState, dirstate::EntryState,
pack_dirstate, parse_dirstate, pack_dirstate, parse_dirstate,
utils::hg_path::{HgPath, HgPathBuf}, utils::hg_path::{HgPath, HgPathBuf},
CopyMap, DirsMultiset, DirstateEntry, DirstateError, DirstateMapError, CopyMap, DirsMultiset, DirstateEntry, DirstateError, DirstateMapError,
DirstateParents, StateMap, DirstateParents, StateMap,
}; };
pub fn clear_ambiguous_times( pub fn clear_ambiguous_times(
&mut self, &mut self,
filenames: Vec<HgPathBuf>, filenames: Vec<HgPathBuf>,
now: i32, now: i32,
) { ) {
for filename in filenames { for filename in filenames {
if let Some(entry) = self.state_map.get_mut(&filename) { if let Some(entry) = self.state_map.get_mut(&filename) {
if clear_ambiguous_mtime(entry, now) { if entry.clear_ambiguous_mtime(now) {
self.get_non_normal_other_parent_entries() self.get_non_normal_other_parent_entries()
.0 .0
.insert(filename.to_owned()); .insert(filename.to_owned());
} }
} }
} }
} }

rust/hg-core/src/dirstate/parsers.rs

packed.push(b'\0'); packed.push(b'\0');
packed.extend(source.as_bytes()); packed.extend(source.as_bytes());
} }
} }
/// Seconds since the Unix epoch /// Seconds since the Unix epoch
pub struct Timestamp(pub u64); pub struct Timestamp(pub u64);
pub fn clear_ambiguous_mtime( impl DirstateEntry {
entry: &mut DirstateEntry, pub fn mtime_is_ambiguous(&self, now: i32) -> bool {
AlphareUnsubmitted
Not Done

s/mtime_is_ambigous/mtime_is_ambiguous/

Alphare: `s/mtime_is_ambigous/mtime_is_ambiguous/`
mtime_now: i32, self.state == EntryState::Normal && self.mtime == now
) -> bool { }
let ambiguous =
entry.state == EntryState::Normal && entry.mtime == mtime_now; pub fn clear_ambiguous_mtime(&mut self, now: i32) -> bool {
let ambiguous = self.mtime_is_ambiguous(now);
if ambiguous { if ambiguous {
// The file was last modified "simultaneously" with the current // The file was last modified "simultaneously" with the current
// write to dirstate (i.e. within the same second for file- // write to dirstate (i.e. within the same second for file-
// systems with a granularity of 1 sec). This commonly happens // systems with a granularity of 1 sec). This commonly happens
// for at least a couple of files on 'update'. // for at least a couple of files on 'update'.
// The user could change the file without changing its size // The user could change the file without changing its size
// within the same second. Invalidate the file's mtime in // within the same second. Invalidate the file's mtime in
// dirstate, forcing future 'status' calls to compare the // dirstate, forcing future 'status' calls to compare the
// contents of the file if the size is the same. This prevents // contents of the file if the size is the same. This prevents
// mistakenly treating such files as clean. // mistakenly treating such files as clean.
entry.mtime = -1; self.clear_mtime()
} }
ambiguous ambiguous
} }
pub fn clear_mtime(&mut self) {
self.mtime = -1;
}
}
pub fn pack_dirstate( pub fn pack_dirstate(
state_map: &mut StateMap, state_map: &mut StateMap,
copy_map: &CopyMap, copy_map: &CopyMap,
parents: DirstateParents, parents: DirstateParents,
now: Timestamp, now: Timestamp,
) -> Result<Vec<u8>, HgError> { ) -> Result<Vec<u8>, HgError> {
// TODO move away from i32 before 2038. // TODO move away from i32 before 2038.
let now: i32 = now.0.try_into().expect("time overflow"); let now: i32 = now.0.try_into().expect("time overflow");
let expected_size: usize = state_map let expected_size: usize = state_map
.iter() .iter()
.map(|(filename, _)| { .map(|(filename, _)| {
packed_entry_size(filename, copy_map.get(filename).map(|p| &**p)) packed_entry_size(filename, copy_map.get(filename).map(|p| &**p))
}) })
.sum(); .sum();
let expected_size = expected_size + PARENT_SIZE * 2; let expected_size = expected_size + PARENT_SIZE * 2;
let mut packed = Vec::with_capacity(expected_size); let mut packed = Vec::with_capacity(expected_size);
packed.extend(parents.p1.as_bytes()); packed.extend(parents.p1.as_bytes());
packed.extend(parents.p2.as_bytes()); packed.extend(parents.p2.as_bytes());
for (filename, entry) in state_map.iter_mut() { for (filename, entry) in state_map.iter_mut() {
clear_ambiguous_mtime(entry, now); entry.clear_ambiguous_mtime(now);
pack_entry( pack_entry(
filename, filename,
entry, entry,
copy_map.get(filename).map(|p| &**p), copy_map.get(filename).map(|p| &**p),
&mut packed, &mut packed,
) )
} }

rust/hg-core/src/dirstate_tree/dirstate_map.rs

use bytes_cast::BytesCast; use bytes_cast::BytesCast;
use micro_timer::timed; use micro_timer::timed;
use std::borrow::Cow; use std::borrow::Cow;
use std::convert::TryInto; use std::convert::TryInto;
use std::path::PathBuf; use std::path::PathBuf;
use super::on_disk; use super::on_disk;
use super::path_with_basename::WithBasename; use super::path_with_basename::WithBasename;
use crate::dirstate::parsers::clear_ambiguous_mtime;
use crate::dirstate::parsers::pack_entry; use crate::dirstate::parsers::pack_entry;
use crate::dirstate::parsers::packed_entry_size; use crate::dirstate::parsers::packed_entry_size;
use crate::dirstate::parsers::parse_dirstate_entries; use crate::dirstate::parsers::parse_dirstate_entries;
use crate::dirstate::parsers::Timestamp; use crate::dirstate::parsers::Timestamp;
use crate::matchers::Matcher; use crate::matchers::Matcher;
use crate::utils::hg_path::{HgPath, HgPathBuf}; use crate::utils::hg_path::{HgPath, HgPathBuf};
use crate::CopyMapIter; use crate::CopyMapIter;
use crate::DirstateEntry; use crate::DirstateEntry;
let mut vec: Vec<_> = nodes.iter_mut().collect(); let mut vec: Vec<_> = nodes.iter_mut().collect();
// `sort_unstable_by_key` doesn’t allow keys borrowing from the value: // `sort_unstable_by_key` doesn’t allow keys borrowing from the value:
// https://github.com/rust-lang/rust/issues/34162 // https://github.com/rust-lang/rust/issues/34162
vec.sort_unstable_by(|(path1, _), (path2, _)| path1.cmp(path2)); vec.sort_unstable_by(|(path1, _), (path2, _)| path1.cmp(path2));
vec vec
} }
} }
/// `(full_path, entry, copy_source)`
type NodeDataMut<'tree, 'on_disk> = (
&'tree HgPath,
&'tree mut Option<DirstateEntry>,
&'tree mut Option<Cow<'on_disk, HgPath>>,
);
impl<'on_disk> DirstateMap<'on_disk> { impl<'on_disk> DirstateMap<'on_disk> {
pub(super) fn empty(on_disk: &'on_disk [u8]) -> Self { pub(super) fn empty(on_disk: &'on_disk [u8]) -> Self {
Self { Self {
on_disk, on_disk,
root: ChildNodes::default(), root: ChildNodes::default(),
nodes_with_entry_count: 0, nodes_with_entry_count: 0,
nodes_with_copy_source_count: 0, nodes_with_copy_source_count: 0,
} }
if node.entry.is_none() { if node.entry.is_none() {
self.nodes_with_entry_count += 1 self.nodes_with_entry_count += 1
} }
node.entry = Some(new_entry) node.entry = Some(new_entry)
} }
fn iter_nodes<'a>( fn iter_nodes<'a>(
&'a self, &'a self,
) -> impl Iterator<Item = (&'a HgPath, &'a Node)> + 'a { ) -> impl Iterator<Item = (&'a Cow<'on_disk, HgPath>, &'a Node)> + 'a {
// Depth first tree traversal. // Depth first tree traversal.
// //
// If we could afford internal iteration and recursion, // If we could afford internal iteration and recursion,
// this would look like: // this would look like:
// //
// ``` // ```
// fn traverse_children( // fn traverse_children(
// children: &ChildNodes, // children: &ChildNodes,
// call stack. Use an explicit stack instead: // call stack. Use an explicit stack instead:
let mut stack = Vec::new(); let mut stack = Vec::new();
let mut iter = self.root.iter(); let mut iter = self.root.iter();
std::iter::from_fn(move || { std::iter::from_fn(move || {
while let Some((key, child_node)) = iter.next() { while let Some((key, child_node)) = iter.next() {
// Pseudo-recursion // Pseudo-recursion
let new_iter = child_node.children.iter(); let new_iter = child_node.children.iter();
let old_iter = std::mem::replace(&mut iter, new_iter); let old_iter = std::mem::replace(&mut iter, new_iter);
let key = &**key.full_path(); let key = key.full_path();
stack.push((key, child_node, old_iter)); stack.push((key, child_node, old_iter));
} }
// Found the end of a `children.iter()` iterator. // Found the end of a `children.iter()` iterator.
if let Some((key, child_node, next_iter)) = stack.pop() { if let Some((key, child_node, next_iter)) = stack.pop() {
// "Return" from pseudo-recursion by restoring state from the // "Return" from pseudo-recursion by restoring state from the
// explicit stack // explicit stack
iter = next_iter; iter = next_iter;
Some((key, child_node)) Some((key, child_node))
} else { } else {
// Reached the bottom of the stack, we’re done // Reached the bottom of the stack, we’re done
None None
} }
}) })
} }
/// Mutable iterator for the `(entry, copy source)` of each node. fn clear_known_ambiguous_mtimes(&mut self, paths: &[impl AsRef<HgPath>]) {
/// for path in paths {
/// It would not be safe to yield mutable references to nodes themeselves if let Some(node) =
/// with `-> impl Iterator<Item = &mut Node>` since child nodes are Self::get_node_mut(&mut self.root, path.as_ref())
/// reachable from their ancestor nodes, potentially creating multiple {
/// `&mut` references to a given node. if let Some(entry) = node.entry.as_mut() {
fn iter_node_data_mut<'tree>( entry.clear_mtime();
&'tree mut self, }
) -> impl Iterator<Item = NodeDataMut<'tree, 'on_disk>> + 'tree {
// Explict stack for pseudo-recursion, see `iter_nodes` above.
let mut stack = Vec::new();
let mut iter = self.root.iter_mut();
std::iter::from_fn(move || {
while let Some((key, child_node)) = iter.next() {
// Pseudo-recursion
let data = (
&**key.full_path(),
&mut child_node.entry,
&mut child_node.copy_source,
);
let new_iter = child_node.children.iter_mut();
let old_iter = std::mem::replace(&mut iter, new_iter);
stack.push((data, old_iter));
} }
// Found the end of a `children.values_mut()` iterator.
if let Some((data, next_iter)) = stack.pop() {
// "Return" from pseudo-recursion by restoring state from the
// explicit stack
iter = next_iter;
Some(data)
} else {
// Reached the bottom of the stack, we’re done
None
} }
})
} }
} }
impl<'on_disk> super::dispatch::DirstateMapMethods for DirstateMap<'on_disk> { impl<'on_disk> super::dispatch::DirstateMapMethods for DirstateMap<'on_disk> {
fn clear(&mut self) { fn clear(&mut self) {
self.root.clear(); self.root.clear();
self.nodes_with_entry_count = 0; self.nodes_with_entry_count = 0;
self.nodes_with_copy_source_count = 0; self.nodes_with_copy_source_count = 0;
Ok(false) Ok(false)
} }
} }
fn clear_ambiguous_times(&mut self, filenames: Vec<HgPathBuf>, now: i32) { fn clear_ambiguous_times(&mut self, filenames: Vec<HgPathBuf>, now: i32) {
for filename in filenames { for filename in filenames {
if let Some(node) = Self::get_node_mut(&mut self.root, &filename) { if let Some(node) = Self::get_node_mut(&mut self.root, &filename) {
if let Some(entry) = node.entry.as_mut() { if let Some(entry) = node.entry.as_mut() {
clear_ambiguous_mtime(entry, now); entry.clear_ambiguous_mtime(now);
} }
} }
} }
} }
fn non_normal_entries_contains(&mut self, key: &HgPath) -> bool { fn non_normal_entries_contains(&mut self, key: &HgPath) -> bool {
self.get_node(key) self.get_node(key)
.and_then(|node| node.entry.as_ref()) .and_then(|node| node.entry.as_ref())
&mut self, &mut self,
) -> Box<dyn Iterator<Item = &HgPath> + '_> { ) -> Box<dyn Iterator<Item = &HgPath> + '_> {
Box::new(self.iter_nodes().filter_map(|(path, node)| { Box::new(self.iter_nodes().filter_map(|(path, node)| {
node.entry node.entry
.as_ref() .as_ref()
.filter(|entry| { .filter(|entry| {
entry.is_non_normal() || entry.is_from_other_parent() entry.is_non_normal() || entry.is_from_other_parent()
}) })
.map(|_| path) .map(|_| &**path)
})) }))
} }
fn set_non_normal_other_parent_entries(&mut self, _force: bool) { fn set_non_normal_other_parent_entries(&mut self, _force: bool) {
// Do nothing, this `DirstateMap` does not have a separate "non normal // Do nothing, this `DirstateMap` does not have a separate "non normal
// entries" and "from other parent" sets that need to be recomputed // entries" and "from other parent" sets that need to be recomputed
} }
fn iter_non_normal_paths( fn iter_non_normal_paths(
&mut self, &mut self,
) -> Box<dyn Iterator<Item = &HgPath> + Send + '_> { ) -> Box<dyn Iterator<Item = &HgPath> + Send + '_> {
self.iter_non_normal_paths_panic() self.iter_non_normal_paths_panic()
} }
fn iter_non_normal_paths_panic( fn iter_non_normal_paths_panic(
&self, &self,
) -> Box<dyn Iterator<Item = &HgPath> + Send + '_> { ) -> Box<dyn Iterator<Item = &HgPath> + Send + '_> {
Box::new(self.iter_nodes().filter_map(|(path, node)| { Box::new(self.iter_nodes().filter_map(|(path, node)| {
node.entry node.entry
.as_ref() .as_ref()
.filter(|entry| entry.is_non_normal()) .filter(|entry| entry.is_non_normal())
.map(|_| path) .map(|_| &**path)
})) }))
} }
fn iter_other_parent_paths( fn iter_other_parent_paths(
&mut self, &mut self,
) -> Box<dyn Iterator<Item = &HgPath> + Send + '_> { ) -> Box<dyn Iterator<Item = &HgPath> + Send + '_> {
Box::new(self.iter_nodes().filter_map(|(path, node)| { Box::new(self.iter_nodes().filter_map(|(path, node)| {
node.entry node.entry
.as_ref() .as_ref()
.filter(|entry| entry.is_from_other_parent()) .filter(|entry| entry.is_from_other_parent())
.map(|_| path) .map(|_| &**path)
})) }))
} }
fn has_tracked_dir( fn has_tracked_dir(
&mut self, &mut self,
directory: &HgPath, directory: &HgPath,
) -> Result<bool, DirstateMapError> { ) -> Result<bool, DirstateMapError> {
if let Some(node) = self.get_node(directory) { if let Some(node) = self.get_node(directory) {
} }
#[timed] #[timed]
fn pack_v1( fn pack_v1(
&mut self, &mut self,
parents: DirstateParents, parents: DirstateParents,
now: Timestamp, now: Timestamp,
) -> Result<Vec<u8>, DirstateError> { ) -> Result<Vec<u8>, DirstateError> {
let now: i32 = now.0.try_into().expect("time overflow");
let mut ambiguous_mtimes = Vec::new();
// Optizimation (to be measured?): pre-compute size to avoid `Vec` // Optizimation (to be measured?): pre-compute size to avoid `Vec`
// reallocations // reallocations
let mut size = parents.as_bytes().len(); let mut size = parents.as_bytes().len();
for (path, node) in self.iter_nodes() { for (path, node) in self.iter_nodes() {
if node.entry.is_some() { if let Some(entry) = &node.entry {
size += packed_entry_size( size += packed_entry_size(
path, path,
node.copy_source.as_ref().map(|p| &**p), node.copy_source.as_ref().map(|p| &**p),
) );
if entry.mtime_is_ambiguous(now) {
ambiguous_mtimes.push(path.clone())
}
} }
} }
self.clear_known_ambiguous_mtimes(&ambiguous_mtimes);
let mut packed = Vec::with_capacity(size); let mut packed = Vec::with_capacity(size);
packed.extend(parents.as_bytes()); packed.extend(parents.as_bytes());
let now: i32 = now.0.try_into().expect("time overflow"); for (path, node) in self.iter_nodes() {
for (path, opt_entry, copy_source) in self.iter_node_data_mut() { if let Some(entry) = &node.entry {
if let Some(entry) = opt_entry {
clear_ambiguous_mtime(entry, now);
pack_entry( pack_entry(
path, path,
entry, entry,
copy_source.as_ref().map(|p| &**p), node.copy_source.as_ref().map(|p| &**p),
&mut packed, &mut packed,
); );
} }
} }
Ok(packed) Ok(packed)
} }
#[timed] #[timed]
fn pack_v2( fn pack_v2(
&mut self, &mut self,
parents: DirstateParents, parents: DirstateParents,
now: Timestamp, now: Timestamp,
) -> Result<Vec<u8>, DirstateError> { ) -> Result<Vec<u8>, DirstateError> {
on_disk::write(self, parents, now) // TODO:!how do we want to handle this in 2038?
let now: i32 = now.0.try_into().expect("time overflow");
let mut paths = Vec::new();
for (path, node) in self.iter_nodes() {
if let Some(entry) = &node.entry {
if entry.mtime_is_ambiguous(now) {
paths.push(path.clone())
}
}
}
// Borrow of `self` ends here since we collect cloned paths
self.clear_known_ambiguous_mtimes(&paths);
on_disk::write(self, parents)
} }
fn set_all_dirs(&mut self) -> Result<(), DirstateMapError> { fn set_all_dirs(&mut self) -> Result<(), DirstateMapError> {
// Do nothing, this `DirstateMap` does not a separate `all_dirs` that // Do nothing, this `DirstateMap` does not a separate `all_dirs` that
// needs to be recomputed // needs to be recomputed
Ok(()) Ok(())
} }
fn copy_map_len(&self) -> usize { fn copy_map_len(&self) -> usize {
self.nodes_with_copy_source_count as usize self.nodes_with_copy_source_count as usize
} }
fn copy_map_iter(&self) -> CopyMapIter<'_> { fn copy_map_iter(&self) -> CopyMapIter<'_> {
Box::new(self.iter_nodes().filter_map(|(path, node)| { Box::new(self.iter_nodes().filter_map(|(path, node)| {
node.copy_source node.copy_source
.as_ref() .as_ref()
.map(|copy_source| (path, &**copy_source)) .map(|copy_source| (&**path, &**copy_source))
})) }))
} }
fn copy_map_contains_key(&self, key: &HgPath) -> bool { fn copy_map_contains_key(&self, key: &HgPath) -> bool {
if let Some(node) = self.get_node(key) { if let Some(node) = self.get_node(key) {
node.copy_source.is_some() node.copy_source.is_some()
} else { } else {
false false
} }
fn get(&self, key: &HgPath) -> Option<&DirstateEntry> { fn get(&self, key: &HgPath) -> Option<&DirstateEntry> {
self.get_node(key)?.entry.as_ref() self.get_node(key)?.entry.as_ref()
} }
fn iter(&self) -> StateMapIter<'_> { fn iter(&self) -> StateMapIter<'_> {
Box::new(self.iter_nodes().filter_map(|(path, node)| { Box::new(self.iter_nodes().filter_map(|(path, node)| {
node.entry.as_ref().map(|entry| (path, entry)) node.entry.as_ref().map(|entry| (&**path, entry))
})) }))
} }
} }

rust/hg-core/src/dirstate_tree/on_disk.rs

//! The "version 2" disk representation of the dirstate //! The "version 2" disk representation of the dirstate
//! //!
//! # File format //! # File format
//! //!
//! The file starts with a fixed-sized header, whose layout is defined by the //! The file starts with a fixed-sized header, whose layout is defined by the
//! `Header` struct. Its `root` field contains the slice (offset and length) to //! `Header` struct. Its `root` field contains the slice (offset and length) to
//! the nodes representing the files and directories at the root of the //! the nodes representing the files and directories at the root of the
//! repository. Each node is also fixed-size, defined by the `Node` struct. //! repository. Each node is also fixed-size, defined by the `Node` struct.
//! Nodes in turn contain slices to variable-size paths, and to their own child //! Nodes in turn contain slices to variable-size paths, and to their own child
//! nodes (if any) for nested files and directories. //! nodes (if any) for nested files and directories.
use crate::dirstate::parsers::clear_ambiguous_mtime;
use crate::dirstate::parsers::Timestamp;
use crate::dirstate_tree::dirstate_map::{self, DirstateMap}; use crate::dirstate_tree::dirstate_map::{self, DirstateMap};
use crate::dirstate_tree::path_with_basename::WithBasename; use crate::dirstate_tree::path_with_basename::WithBasename;
use crate::errors::HgError; use crate::errors::HgError;
use crate::utils::hg_path::HgPath; use crate::utils::hg_path::HgPath;
use crate::DirstateEntry; use crate::DirstateEntry;
use crate::DirstateError; use crate::DirstateError;
use crate::DirstateParents; use crate::DirstateParents;
use bytes_cast::unaligned::{I32Be, U32Be, U64Be}; use bytes_cast::unaligned::{I32Be, U32Be, U64Be};
.ok_or_else(|| { .ok_or_else(|| {
HgError::corrupted("dirstate v2 slice is out of bounds") HgError::corrupted("dirstate v2 slice is out of bounds")
}) })
} }
pub(super) fn write( pub(super) fn write(
dirstate_map: &mut DirstateMap, dirstate_map: &mut DirstateMap,
parents: DirstateParents, parents: DirstateParents,
now: Timestamp,
) -> Result<Vec<u8>, DirstateError> { ) -> Result<Vec<u8>, DirstateError> {
// TODO:!how do we want to handle this in 2038?
let now: i32 = now.0.try_into().expect("time overflow");
let header_len = std::mem::size_of::<Header>(); let header_len = std::mem::size_of::<Header>();
// This ignores the space for paths, and for nodes without an entry. // This ignores the space for paths, and for nodes without an entry.
// TODO: better estimate? Skip the `Vec` and write to a file directly? // TODO: better estimate? Skip the `Vec` and write to a file directly?
let size_guess = header_len let size_guess = header_len
+ std::mem::size_of::<Node>() + std::mem::size_of::<Node>()
* dirstate_map.nodes_with_entry_count as usize; * dirstate_map.nodes_with_entry_count as usize;
let mut out = Vec::with_capacity(size_guess); let mut out = Vec::with_capacity(size_guess);
// Keep space for the header. We’ll fill it out at the end when we know the // Keep space for the header. We’ll fill it out at the end when we know the
// actual offset for the root nodes. // actual offset for the root nodes.
out.resize(header_len, 0_u8); out.resize(header_len, 0_u8);
let root = write_nodes(&mut dirstate_map.root, now, &mut out)?; let root = write_nodes(&mut dirstate_map.root, &mut out)?;
let header = Header { let header = Header {
marker: *V2_FORMAT_MARKER, marker: *V2_FORMAT_MARKER,
parents: parents, parents: parents,
root, root,
nodes_with_entry_count: dirstate_map.nodes_with_entry_count.into(), nodes_with_entry_count: dirstate_map.nodes_with_entry_count.into(),
nodes_with_copy_source_count: dirstate_map nodes_with_copy_source_count: dirstate_map
.nodes_with_copy_source_count .nodes_with_copy_source_count
.into(), .into(),
}; };
out[..header_len].copy_from_slice(header.as_bytes()); out[..header_len].copy_from_slice(header.as_bytes());
Ok(out) Ok(out)
} }
/// Serialize the dirstate to the `v2` format after clearing ambigous `mtime`s.
fn write_nodes( fn write_nodes(
nodes: &mut dirstate_map::ChildNodes, nodes: &mut dirstate_map::ChildNodes,
now: i32,
out: &mut Vec<u8>, out: &mut Vec<u8>,
) -> Result<ChildNodes, DirstateError> { ) -> Result<ChildNodes, DirstateError> {
// `dirstate_map::ChildNodes` is a `HashMap` with undefined iteration // `dirstate_map::ChildNodes` is a `HashMap` with undefined iteration
// order. Sort to enable binary search in the written file. // order. Sort to enable binary search in the written file.
let nodes = dirstate_map::Node::sorted(nodes); let nodes = dirstate_map::Node::sorted(nodes);
// First accumulate serialized nodes in a `Vec` // First accumulate serialized nodes in a `Vec`
let mut on_disk_nodes = Vec::with_capacity(nodes.len()); let mut on_disk_nodes = Vec::with_capacity(nodes.len());
for (full_path, node) in nodes { for (full_path, node) in nodes {
on_disk_nodes.push(Node { on_disk_nodes.push(Node {
children: write_nodes(&mut node.children, now, out)?, children: write_nodes(&mut node.children, out)?,
tracked_descendants_count: node.tracked_descendants_count.into(), tracked_descendants_count: node.tracked_descendants_count.into(),
full_path: write_slice::<u8>( full_path: write_slice::<u8>(
full_path.full_path().as_bytes(), full_path.full_path().as_bytes(),
out, out,
), ),
base_name_start: u32::try_from(full_path.base_name_start()) base_name_start: u32::try_from(full_path.base_name_start())
// Could only panic for paths over 4 GiB // Could only panic for paths over 4 GiB
.expect("dirstate-v2 offset overflow") .expect("dirstate-v2 offset overflow")
.into(), .into(),
copy_source: if let Some(source) = &node.copy_source { copy_source: if let Some(source) = &node.copy_source {
write_slice::<u8>(source.as_bytes(), out) write_slice::<u8>(source.as_bytes(), out)
} else { } else {
Slice { Slice {
start: 0.into(), start: 0.into(),
len: 0.into(), len: 0.into(),
} }
}, },
entry: if let Some(entry) = &mut node.entry { entry: if let Some(entry) = &mut node.entry {
clear_ambiguous_mtime(entry, now);
OptEntry { OptEntry {
state: entry.state.into(), state: entry.state.into(),
mode: entry.mode.into(), mode: entry.mode.into(),
mtime: entry.mtime.into(), mtime: entry.mtime.into(),
size: entry.size.into(), size: entry.size.into(),
} }
} else { } else {
OptEntry { OptEntry {