This is an archive of the discontinued Mercurial Phabricator instance.

Differential D8663

hg-core: define a `dirstate_status` `Operation`
ClosedPublic

Authored by Alphare on Jun 25 2020, 5:03 AM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG98817e5daca7: hg-core: define a `dirstate_status` `Operation`
Summary

This is 3/3 in a series of patches to improve dirstate status' code.

Following in the footsteps of a46e36b82461, we move the main status
functionality to an Operation. This will most likely be subject to change in
the future (what function signature, what parameters, etc., but we will see
when rhg gets hg status support.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Alphare created this revision.Jun 25 2020, 5:03 AM
Alphare added a commit: rHG98817e5daca7: hg-core: define a `dirstate_status` `Operation`.Jul 15 2020, 12:13 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG98817e5daca7: hg-core: define a `dirstate_status` `Operation` (authored by Alphare). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 21914

rust/hg-core/src/dirstate/status.rs

// status.rs // status.rs
// //
// Copyright 2019 Raphaël Gomès <rgomes@octobus.net> // Copyright 2019 Raphaël Gomès <rgomes@octobus.net>
// //
// This software may be used and distributed according to the terms of the // This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version. // GNU General Public License version 2 or any later version.
//! Rust implementation of dirstate.status (dirstate.py). //! Rust implementation of dirstate.status (dirstate.py).
//! It is currently missing a lot of functionality compared to the Python one //! It is currently missing a lot of functionality compared to the Python one
//! and will only be triggered in narrow cases. //! and will only be triggered in narrow cases.
use crate::{ use crate::{
dirstate::SIZE_FROM_OTHER_PARENT, dirstate::SIZE_FROM_OTHER_PARENT,
filepatterns::PatternFileWarning, filepatterns::PatternFileWarning,
matchers::{get_ignore_function, Matcher, VisitChildrenSet}, matchers::{get_ignore_function, Matcher, VisitChildrenSet},
operations::Operation,
utils::{ utils::{
files::{find_dirs, HgMetadata}, files::{find_dirs, HgMetadata},
hg_path::{ hg_path::{
hg_path_to_path_buf, os_string_to_hg_path_buf, HgPath, HgPathBuf, hg_path_to_path_buf, os_string_to_hg_path_buf, HgPath, HgPathBuf,
HgPathError, HgPathError,
}, },
path_auditor::PathAuditor, path_auditor::PathAuditor,
}, },
type IoResult<T> = std::io::Result<T>; type IoResult<T> = std::io::Result<T>;
/// `Box<dyn Trait>` is syntactic sugar for `Box<dyn Trait, 'static>`, so add /// `Box<dyn Trait>` is syntactic sugar for `Box<dyn Trait, 'static>`, so add
/// an explicit lifetime here to not fight `'static` bounds "out of nowhere". /// an explicit lifetime here to not fight `'static` bounds "out of nowhere".
type IgnoreFnType<'a> = Box<dyn for<'r> Fn(&'r HgPath) -> bool + Sync + 'a>; type IgnoreFnType<'a> = Box<dyn for<'r> Fn(&'r HgPath) -> bool + Sync + 'a>;
/// We have a good mix of owned (from directory traversal) and borrowed (from /// We have a good mix of owned (from directory traversal) and borrowed (from
/// the dirstate/explicit) paths, this comes up a lot. /// the dirstate/explicit) paths, this comes up a lot.
type HgPathCow<'a> = Cow<'a, HgPath>; pub type HgPathCow<'a> = Cow<'a, HgPath>;
/// A path with its computed ``Dispatch`` information /// A path with its computed ``Dispatch`` information
type DispatchedPath<'a> = (HgPathCow<'a>, Dispatch); type DispatchedPath<'a> = (HgPathCow<'a>, Dispatch);
/// Dates and times that are outside the 31-bit signed range are compared /// Dates and times that are outside the 31-bit signed range are compared
/// modulo 2^31. This should prevent hg from behaving badly with very large /// modulo 2^31. This should prevent hg from behaving badly with very large
/// files or corrupt dates while still having a high probability of detecting /// files or corrupt dates while still having a high probability of detecting
/// changes. (issue2608) /// changes. (issue2608)
} }
} }
} }
/// Gives information about which files are changed in the working directory /// Gives information about which files are changed in the working directory
/// and how, compared to the revision we're based on /// and how, compared to the revision we're based on
pub struct Status<'a, M: Matcher + Sync> { pub struct Status<'a, M: Matcher + Sync> {
dmap: &'a DirstateMap, dmap: &'a DirstateMap,
matcher: &'a M, pub(crate) matcher: &'a M,
root_dir: PathBuf, root_dir: PathBuf,
options: StatusOptions, pub(crate) options: StatusOptions,
ignore_fn: IgnoreFnType<'a>, ignore_fn: IgnoreFnType<'a>,
} }
impl<'a, M> Status<'a, M> impl<'a, M> Status<'a, M>
where where
M: Matcher + Sync, M: Matcher + Sync,
{ {
pub fn new( pub fn new(
/// Checks all files that are in the dirstate but were not found during the /// Checks all files that are in the dirstate but were not found during the
/// working directory traversal. This means that the rest must /// working directory traversal. This means that the rest must
/// be either ignored, under a symlink or under a new nested repo. /// be either ignored, under a symlink or under a new nested repo.
/// ///
/// This takes a mutable reference to the results to account for the /// This takes a mutable reference to the results to account for the
/// `extend` in timings /// `extend` in timings
#[timed] #[timed]
fn handle_unknowns( pub fn handle_unknowns(
&self, &self,
results: &mut Vec<DispatchedPath<'a>>, results: &mut Vec<DispatchedPath<'a>>,
) -> IoResult<()> { ) -> IoResult<()> {
let to_visit: Vec<(&HgPath, &DirstateEntry)> = let to_visit: Vec<(&HgPath, &DirstateEntry)> =
if results.is_empty() && self.matcher.matches_everything() { if results.is_empty() && self.matcher.matches_everything() {
self.dmap.iter().map(|(f, e)| (f.deref(), e)).collect() self.dmap.iter().map(|(f, e)| (f.deref(), e)).collect()
} else { } else {
// Only convert to a hashmap if needed. // Only convert to a hashmap if needed.
Ok(()) Ok(())
} }
/// Add the files in the dirstate to the results. /// Add the files in the dirstate to the results.
/// ///
/// This takes a mutable reference to the results to account for the /// This takes a mutable reference to the results to account for the
/// `extend` in timings /// `extend` in timings
#[timed] #[timed]
fn extend_from_dmap(&self, results: &mut Vec<DispatchedPath<'a>>) { pub fn extend_from_dmap(&self, results: &mut Vec<DispatchedPath<'a>>) {
results.par_extend(self.dmap.par_iter().flat_map( results.par_extend(self.dmap.par_iter().flat_map(
move |(filename, entry)| { move |(filename, entry)| {
let filename: &HgPath = filename; let filename: &HgPath = filename;
let filename_as_path = hg_path_to_path_buf(filename)?; let filename_as_path = hg_path_to_path_buf(filename)?;
let meta = let meta =
self.root_dir.join(filename_as_path).symlink_metadata(); self.root_dir.join(filename_as_path).symlink_metadata();
match meta { match meta {
Err(e) => Err(e), Err(e) => Err(e),
} }
}, },
)); ));
} }
} }
#[timed] #[timed]
fn build_response<'a>( pub fn build_response<'a>(
results: impl IntoIterator<Item = DispatchedPath<'a>>, results: impl IntoIterator<Item = DispatchedPath<'a>>,
traversed: Vec<HgPathBuf>, traversed: Vec<HgPathBuf>,
) -> (Vec<HgPathCow<'a>>, DirstateStatus<'a>) { ) -> (Vec<HgPathCow<'a>>, DirstateStatus<'a>) {
let mut lookup = vec![]; let mut lookup = vec![];
let mut modified = vec![]; let mut modified = vec![];
let mut added = vec![]; let mut added = vec![];
let mut removed = vec![]; let mut removed = vec![];
let mut deleted = vec![]; let mut deleted = vec![];
matcher: &'a (impl Matcher + Sync), matcher: &'a (impl Matcher + Sync),
root_dir: PathBuf, root_dir: PathBuf,
ignore_files: Vec<PathBuf>, ignore_files: Vec<PathBuf>,
options: StatusOptions, options: StatusOptions,
) -> StatusResult<( ) -> StatusResult<(
(Vec<HgPathCow<'a>>, DirstateStatus<'a>), (Vec<HgPathCow<'a>>, DirstateStatus<'a>),
Vec<PatternFileWarning>, Vec<PatternFileWarning>,
)> { )> {
let (traversed_sender, traversed_receiver) = let (status, warnings) =
crossbeam::channel::unbounded();
let (st, warnings) =
Status::new(dmap, matcher, root_dir, ignore_files, options)?; Status::new(dmap, matcher, root_dir, ignore_files, options)?;
// Step 1: check the files explicitly mentioned by the user Ok((status.run()?, warnings))
let (work, mut results) = st.walk_explicit(traversed_sender.clone());
if !work.is_empty() {
// Hashmaps are quite a bit slower to build than vecs, so only build it
// if needed.
let old_results = results.iter().cloned().collect();
// Step 2: recursively check the working directory for changes if
// needed
for (dir, dispatch) in work {
match dispatch {
Dispatch::Directory { was_file } => {
if was_file {
results.push((dir.to_owned(), Dispatch::Removed));
}
if options.list_ignored
|| options.list_unknown && !st.dir_ignore(&dir)
{
st.traverse(
&dir,
&old_results,
&mut results,
traversed_sender.clone(),
)?;
}
}
_ => unreachable!("There can only be directories in `work`"),
}
}
}
if !matcher.is_exact() {
if options.list_unknown {
st.handle_unknowns(&mut results)?;
} else {
// TODO this is incorrect, see issue6335
// This requires a fix in both Python and Rust that can happen
// with other pending changes to `status`.
st.extend_from_dmap(&mut results);
}
}
drop(traversed_sender);
let traversed = traversed_receiver.into_iter().collect();
Ok((build_response(results, traversed), warnings))
} }

rust/hg-core/src/operations/dirstate_status.rs

  • This file was added.
// dirstate_status.rs
//
// Copyright 2019, Raphaël Gomès <rgomes@octobus.net>
//
// This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version.
use crate::dirstate::status::{build_response, Dispatch, HgPathCow, Status};
use crate::matchers::Matcher;
use crate::operations::Operation;
use crate::{DirstateStatus, StatusError};
/// A tuple of the paths that need to be checked in the filelog because it's
/// ambiguous whether they've changed, and the rest of the already dispatched
/// files.
pub type LookupAndStatus<'a> = (Vec<HgPathCow<'a>>, DirstateStatus<'a>);
impl<'a, M: Matcher + Sync> Operation<LookupAndStatus<'a>> for Status<'a, M> {
type Error = StatusError;
fn run(&self) -> Result<LookupAndStatus<'a>, Self::Error> {
let (traversed_sender, traversed_receiver) =
crossbeam::channel::unbounded();
// Step 1: check the files explicitly mentioned by the user
let (work, mut results) = self.walk_explicit(traversed_sender.clone());
if !work.is_empty() {
// Hashmaps are quite a bit slower to build than vecs, so only
// build it if needed.
let old_results = results.iter().cloned().collect();
// Step 2: recursively check the working directory for changes if
// needed
for (dir, dispatch) in work {
match dispatch {
Dispatch::Directory { was_file } => {
if was_file {
results.push((dir.to_owned(), Dispatch::Removed));
}
if self.options.list_ignored
|| self.options.list_unknown
&& !self.dir_ignore(&dir)
{
self.traverse(
&dir,
&old_results,
&mut results,
traversed_sender.clone(),
)?;
}
}
_ => {
unreachable!("There can only be directories in `work`")
}
}
}
}
if !self.matcher.is_exact() {
if self.options.list_unknown {
self.handle_unknowns(&mut results)?;
} else {
// TODO this is incorrect, see issue6335
// This requires a fix in both Python and Rust that can happen
// with other pending changes to `status`.
self.extend_from_dmap(&mut results);
}
}
drop(traversed_sender);
let traversed = traversed_receiver.into_iter().collect();
Ok(build_response(results, traversed))
}
}

rust/hg-core/src/operations/mod.rs

mod dirstate_status;
mod find_root; mod find_root;
pub use find_root::{FindRoot, FindRootError, FindRootErrorKind}; pub use find_root::{FindRoot, FindRootError, FindRootErrorKind};
/// An interface for high-level hg operations. /// An interface for high-level hg operations.
/// ///
/// A distinction is made between operation and commands. /// A distinction is made between operation and commands.
/// An operation is what can be done whereas a command is what is exposed by /// An operation is what can be done whereas a command is what is exposed by
/// the cli. A single command can use several operations to achieve its goal. /// the cli. A single command can use several operations to achieve its goal.
pub trait Operation<T> { pub trait Operation<T> {
type Error; type Error;
fn run(&self) -> Result<T, Self::Error>; fn run(&self) -> Result<T, Self::Error>;
} }