This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11899

rhg: Add support for `rhg status --copies`
ClosedPublic

Authored by SimonSapin on Dec 10 2021, 5:14 PM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHG473af5cbc209: rhg: Add support for `rhg status --copies`
Summary

Copy sources are collected during status() rather than after the fact like
in Python, because status() takes a &mut exclusive reference to the dirstate map
(in order to potentially mutate it for directory mtimes) and returns Cow<'_, HgPath>
that borrow the dirstate map.

Even though with Cow only some shared borrows remain, the still extend the same
lifetime of the initial &mut so the dirstate map cannot be borrowed again
to access copy sources after the fact:

https://doc.rust-lang.org/nomicon/lifetime-mismatch.html#limits-of-lifetimes

Additionally, collecting copy sources during the dirstate tree traversal that
status() already does avoids the cost of another traversal or other lookups
(though I haven’t benchmarked that cost).

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

SimonSapin created this revision.Dec 10 2021, 5:14 PM
Alphare accepted this revision.Dec 13 2021, 8:18 AM
Alphare added a subscriber: Alphare.
Alphare added inline comments.
rust/hg-core/src/dirstate_tree/status.rs
151

I swear I had this enum somewhere. Maybe it got removed with the old flat dirstate implementation?

This revision is now accepted and ready to land.Dec 13 2021, 8:18 AM
SimonSapin added a commit: rHG473af5cbc209: rhg: Add support for `rhg status --copies`.Dec 13 2021, 8:36 AM
Closed by commit rHG473af5cbc209: rhg: Add support for `rhg status --copies` (authored by SimonSapin). · Explain Why
This revision was automatically updated to reflect the committed changes.
SimonSapin added inline comments.Dec 13 2021, 8:40 AM
rust/hg-core/src/dirstate_tree/status.rs
151

Now that you say it that rings a bell, and yes it’s the likely explanation

Revision Contents
Changeset List

Diff 31448

rust/hg-core/src/dirstate/status.rs

#[derive(Debug, Copy, Clone)] #[derive(Debug, Copy, Clone)]
pub struct StatusOptions { pub struct StatusOptions {
/// Whether we are on a filesystem with UNIX-like exec flags /// Whether we are on a filesystem with UNIX-like exec flags
pub check_exec: bool, pub check_exec: bool,
pub list_clean: bool, pub list_clean: bool,
pub list_unknown: bool, pub list_unknown: bool,
pub list_ignored: bool, pub list_ignored: bool,
/// Whether to populate `StatusPath::copy_source`
pub list_copies: bool,
/// Whether to collect traversed dirs for applying a callback later. /// Whether to collect traversed dirs for applying a callback later.
/// Used by `hg purge` for example. /// Used by `hg purge` for example.
pub collect_traversed_dirs: bool, pub collect_traversed_dirs: bool,
} }
#[derive(Debug, Default)] #[derive(Default)]
pub struct DirstateStatus<'a> { pub struct DirstateStatus<'a> {
/// The current time at the start of the `status()` algorithm, as measured /// The current time at the start of the `status()` algorithm, as measured
/// and possibly truncated by the filesystem. /// and possibly truncated by the filesystem.
pub filesystem_time_at_status_start: Option<std::time::SystemTime>, pub filesystem_time_at_status_start: Option<std::time::SystemTime>,
/// Tracked files whose contents have changed since the parent revision /// Tracked files whose contents have changed since the parent revision
pub modified: Vec<HgPathCow<'a>>, pub modified: Vec<StatusPath<'a>>,
/// Newly-tracked files that were not present in the parent /// Newly-tracked files that were not present in the parent
pub added: Vec<HgPathCow<'a>>, pub added: Vec<StatusPath<'a>>,
/// Previously-tracked files that have been (re)moved with an hg command /// Previously-tracked files that have been (re)moved with an hg command
pub removed: Vec<HgPathCow<'a>>, pub removed: Vec<StatusPath<'a>>,
/// (Still) tracked files that are missing, (re)moved with an non-hg /// (Still) tracked files that are missing, (re)moved with an non-hg
/// command /// command
pub deleted: Vec<HgPathCow<'a>>, pub deleted: Vec<StatusPath<'a>>,
/// Tracked files that are up to date with the parent. /// Tracked files that are up to date with the parent.
/// Only pupulated if `StatusOptions::list_clean` is true. /// Only pupulated if `StatusOptions::list_clean` is true.
pub clean: Vec<HgPathCow<'a>>, pub clean: Vec<StatusPath<'a>>,
/// Files in the working directory that are ignored with `.hgignore`. /// Files in the working directory that are ignored with `.hgignore`.
/// Only pupulated if `StatusOptions::list_ignored` is true. /// Only pupulated if `StatusOptions::list_ignored` is true.
pub ignored: Vec<HgPathCow<'a>>, pub ignored: Vec<StatusPath<'a>>,
/// Files in the working directory that are neither tracked nor ignored. /// Files in the working directory that are neither tracked nor ignored.
/// Only pupulated if `StatusOptions::list_unknown` is true. /// Only pupulated if `StatusOptions::list_unknown` is true.
pub unknown: Vec<HgPathCow<'a>>, pub unknown: Vec<StatusPath<'a>>,
/// Was explicitly matched but cannot be found/accessed /// Was explicitly matched but cannot be found/accessed
pub bad: Vec<(HgPathCow<'a>, BadMatch)>, pub bad: Vec<(HgPathCow<'a>, BadMatch)>,
/// Either clean or modified, but we can’t tell from filesystem metadata /// Either clean or modified, but we can’t tell from filesystem metadata
/// alone. The file contents need to be read and compared with that in /// alone. The file contents need to be read and compared with that in
/// the parent. /// the parent.
pub unsure: Vec<HgPathCow<'a>>, pub unsure: Vec<StatusPath<'a>>,
/// Only filled if `collect_traversed_dirs` is `true` /// Only filled if `collect_traversed_dirs` is `true`
pub traversed: Vec<HgPathCow<'a>>, pub traversed: Vec<HgPathCow<'a>>,
/// Whether `status()` made changed to the `DirstateMap` that should be /// Whether `status()` made changed to the `DirstateMap` that should be
/// written back to disk /// written back to disk
pub dirty: bool, pub dirty: bool,
} }
#[derive(Clone, PartialEq, Eq, PartialOrd, Ord)]
pub struct StatusPath<'a> {
pub path: HgPathCow<'a>,
pub copy_source: Option<HgPathCow<'a>>,
}
#[derive(Debug, derive_more::From)] #[derive(Debug, derive_more::From)]
pub enum StatusError { pub enum StatusError {
/// Generic IO error /// Generic IO error
IO(std::io::Error), IO(std::io::Error),
/// An invalid path that cannot be represented in Mercurial was found /// An invalid path that cannot be represented in Mercurial was found
Path(HgPathError), Path(HgPathError),
/// An invalid "ignore" pattern was found /// An invalid "ignore" pattern was found
Pattern(PatternError), Pattern(PatternError),

rust/hg-core/src/dirstate_tree/dirstate_map.rs

) -> Result<Option<&'tree HgPath>, DirstateV2ParseError> { ) -> Result<Option<&'tree HgPath>, DirstateV2ParseError> {
match self { match self {
NodeRef::InMemory(_path, node) => { NodeRef::InMemory(_path, node) => {
Ok(node.copy_source.as_ref().map(|s| &**s)) Ok(node.copy_source.as_ref().map(|s| &**s))
} }
NodeRef::OnDisk(node) => node.copy_source(on_disk), NodeRef::OnDisk(node) => node.copy_source(on_disk),
} }
} }
/// Returns a `BorrowedPath`, which can be turned into a `Cow<'on_disk,
/// HgPath>` detached from `'tree`
pub(super) fn copy_source_borrowed(
&self,
on_disk: &'on_disk [u8],
) -> Result<Option<BorrowedPath<'tree, 'on_disk>>, DirstateV2ParseError>
{
Ok(match self {
NodeRef::InMemory(_path, node) => {
node.copy_source.as_ref().map(|source| match source {
Cow::Borrowed(on_disk) => BorrowedPath::OnDisk(on_disk),
Cow::Owned(in_memory) => BorrowedPath::InMemory(in_memory),
})
}
NodeRef::OnDisk(node) => node
.copy_source(on_disk)?
.map(|source| BorrowedPath::OnDisk(source)),
})
}
pub(super) fn entry( pub(super) fn entry(
&self, &self,
) -> Result<Option<DirstateEntry>, DirstateV2ParseError> { ) -> Result<Option<DirstateEntry>, DirstateV2ParseError> {
match self { match self {
NodeRef::InMemory(_path, node) => { NodeRef::InMemory(_path, node) => {
Ok(node.data.as_entry().copied()) Ok(node.data.as_entry().copied())
} }

rust/hg-core/src/dirstate_tree/status.rs

use crate::dirstate::entry::TruncatedTimestamp; use crate::dirstate::entry::TruncatedTimestamp;
use crate::dirstate::status::IgnoreFnType; use crate::dirstate::status::IgnoreFnType;
use crate::dirstate::status::StatusPath;
use crate::dirstate_tree::dirstate_map::BorrowedPath; use crate::dirstate_tree::dirstate_map::BorrowedPath;
use crate::dirstate_tree::dirstate_map::ChildNodesRef; use crate::dirstate_tree::dirstate_map::ChildNodesRef;
use crate::dirstate_tree::dirstate_map::DirstateMap; use crate::dirstate_tree::dirstate_map::DirstateMap;
use crate::dirstate_tree::dirstate_map::NodeData; use crate::dirstate_tree::dirstate_map::NodeData;
use crate::dirstate_tree::dirstate_map::NodeRef; use crate::dirstate_tree::dirstate_map::NodeRef;
use crate::dirstate_tree::on_disk::DirstateV2ParseError; use crate::dirstate_tree::on_disk::DirstateV2ParseError;
use crate::matchers::get_ignore_function; use crate::matchers::get_ignore_function;
use crate::matchers::Matcher; use crate::matchers::Matcher;
use crate::utils::files::get_bytes_from_os_string; use crate::utils::files::get_bytes_from_os_string;
use crate::utils::files::get_path_from_bytes; use crate::utils::files::get_path_from_bytes;
use crate::utils::hg_path::HgPath; use crate::utils::hg_path::HgPath;
use crate::BadMatch; use crate::BadMatch;
use crate::DirstateStatus; use crate::DirstateStatus;
use crate::EntryState; use crate::EntryState;
use crate::HgPathBuf; use crate::HgPathBuf;
use crate::HgPathCow;
use crate::PatternFileWarning; use crate::PatternFileWarning;
use crate::StatusError; use crate::StatusError;
use crate::StatusOptions; use crate::StatusOptions;
use micro_timer::timed; use micro_timer::timed;
use rayon::prelude::*; use rayon::prelude::*;
use sha1::{Digest, Sha1}; use sha1::{Digest, Sha1};
use std::borrow::Cow; use std::borrow::Cow;
use std::io; use std::io;
/// and therefore isn’t reading `.hgignore`. /// and therefore isn’t reading `.hgignore`.
ignore_patterns_have_changed: Option<bool>, ignore_patterns_have_changed: Option<bool>,
/// The current time at the start of the `status()` algorithm, as measured /// The current time at the start of the `status()` algorithm, as measured
/// and possibly truncated by the filesystem. /// and possibly truncated by the filesystem.
filesystem_time_at_status_start: Option<SystemTime>, filesystem_time_at_status_start: Option<SystemTime>,
} }
enum Outcome {
AlphareUnsubmitted
Not Done

I swear I had this enum somewhere. Maybe it got removed with the old flat dirstate implementation?

Alphare: I swear I had this enum somewhere. Maybe it got removed with the old flat dirstate…
SimonSapinAuthorUnsubmitted
Done

Now that you say it that rings a bell, and yes it’s the likely explanation

SimonSapin: Now that you say it that rings a bell, and yes it’s the likely explanation
Modified,
Added,
Removed,
Deleted,
Clean,
Ignored,
Unknown,
Unsure,
}
impl<'a, 'tree, 'on_disk> StatusCommon<'a, 'tree, 'on_disk> { impl<'a, 'tree, 'on_disk> StatusCommon<'a, 'tree, 'on_disk> {
fn push_outcome(
&self,
which: Outcome,
dirstate_node: &NodeRef<'tree, 'on_disk>,
) -> Result<(), DirstateV2ParseError> {
let path = dirstate_node
.full_path_borrowed(self.dmap.on_disk)?
.detach_from_tree();
let copy_source = if self.options.list_copies {
dirstate_node
.copy_source_borrowed(self.dmap.on_disk)?
.map(|source| source.detach_from_tree())
} else {
None
};
self.push_outcome_common(which, path, copy_source);
Ok(())
}
fn push_outcome_without_copy_source(
&self,
which: Outcome,
path: &BorrowedPath<'_, 'on_disk>,
) {
self.push_outcome_common(which, path.detach_from_tree(), None)
}
fn push_outcome_common(
&self,
which: Outcome,
path: HgPathCow<'on_disk>,
copy_source: Option<HgPathCow<'on_disk>>,
) {
let mut outcome = self.outcome.lock().unwrap();
let vec = match which {
Outcome::Modified => &mut outcome.modified,
Outcome::Added => &mut outcome.added,
Outcome::Removed => &mut outcome.removed,
Outcome::Deleted => &mut outcome.deleted,
Outcome::Clean => &mut outcome.clean,
Outcome::Ignored => &mut outcome.ignored,
Outcome::Unknown => &mut outcome.unknown,
Outcome::Unsure => &mut outcome.unsure,
};
vec.push(StatusPath { path, copy_source });
}
fn read_dir( fn read_dir(
&self, &self,
hg_path: &HgPath, hg_path: &HgPath,
fs_path: &Path, fs_path: &Path,
is_at_repo_root: bool, is_at_repo_root: bool,
) -> Result<Vec<DirEntry>, ()> { ) -> Result<Vec<DirEntry>, ()> {
DirEntry::read_dir(fs_path, is_at_repo_root) DirEntry::read_dir(fs_path, is_at_repo_root)
.map_err(|error| self.io_error(error, hg_path)) .map_err(|error| self.io_error(error, hg_path))
self.check_for_outdated_directory_cache(&dirstate_node)?; self.check_for_outdated_directory_cache(&dirstate_node)?;
let hg_path = &dirstate_node.full_path_borrowed(self.dmap.on_disk)?; let hg_path = &dirstate_node.full_path_borrowed(self.dmap.on_disk)?;
let file_type = fs_metadata.file_type(); let file_type = fs_metadata.file_type();
let file_or_symlink = file_type.is_file() || file_type.is_symlink(); let file_or_symlink = file_type.is_file() || file_type.is_symlink();
if !file_or_symlink { if !file_or_symlink {
// If we previously had a file here, it was removed (with // If we previously had a file here, it was removed (with
// `hg rm` or similar) or deleted before it could be // `hg rm` or similar) or deleted before it could be
// replaced by a directory or something else. // replaced by a directory or something else.
self.mark_removed_or_deleted_if_file( self.mark_removed_or_deleted_if_file(&dirstate_node)?;
&hg_path,
dirstate_node.state()?,
);
} }
if file_type.is_dir() { if file_type.is_dir() {
if self.options.collect_traversed_dirs { if self.options.collect_traversed_dirs {
self.outcome self.outcome
.lock() .lock()
.unwrap() .unwrap()
.traversed .traversed
.push(hg_path.detach_from_tree()) .push(hg_path.detach_from_tree())
children_all_have_dirstate_node_or_are_ignored, children_all_have_dirstate_node_or_are_ignored,
fs_metadata, fs_metadata,
dirstate_node, dirstate_node,
)? )?
} else { } else {
if file_or_symlink && self.matcher.matches(hg_path) { if file_or_symlink && self.matcher.matches(hg_path) {
if let Some(state) = dirstate_node.state()? { if let Some(state) = dirstate_node.state()? {
match state { match state {
EntryState::Added => self EntryState::Added => {
.outcome self.push_outcome(Outcome::Added, &dirstate_node)?
.lock() }
.unwrap()
.added
.push(hg_path.detach_from_tree()),
EntryState::Removed => self EntryState::Removed => self
.outcome .push_outcome(Outcome::Removed, &dirstate_node)?,
.lock()
.unwrap()
.removed
.push(hg_path.detach_from_tree()),
EntryState::Merged => self EntryState::Merged => self
.outcome .push_outcome(Outcome::Modified, &dirstate_node)?,
.lock()
.unwrap()
.modified
.push(hg_path.detach_from_tree()),
EntryState::Normal => self EntryState::Normal => self
.handle_normal_file(&dirstate_node, fs_metadata)?, .handle_normal_file(&dirstate_node, fs_metadata)?,
} }
} else { } else {
// `node.entry.is_none()` indicates a "directory" // `node.entry.is_none()` indicates a "directory"
// node, but the filesystem has a file // node, but the filesystem has a file
self.mark_unknown_or_ignored( self.mark_unknown_or_ignored(
has_ignored_ancestor, has_ignored_ancestor,
// Keep the low 31 bits // Keep the low 31 bits
fn truncate_u64(value: u64) -> i32 { fn truncate_u64(value: u64) -> i32 {
(value & 0x7FFF_FFFF) as i32 (value & 0x7FFF_FFFF) as i32
} }
let entry = dirstate_node let entry = dirstate_node
.entry()? .entry()?
.expect("handle_normal_file called with entry-less node"); .expect("handle_normal_file called with entry-less node");
let hg_path = &dirstate_node.full_path_borrowed(self.dmap.on_disk)?;
let mode_changed = let mode_changed =
|| self.options.check_exec && entry.mode_changed(fs_metadata); || self.options.check_exec && entry.mode_changed(fs_metadata);
let size = entry.size(); let size = entry.size();
let size_changed = size != truncate_u64(fs_metadata.len()); let size_changed = size != truncate_u64(fs_metadata.len());
if size >= 0 && size_changed && fs_metadata.file_type().is_symlink() { if size >= 0 && size_changed && fs_metadata.file_type().is_symlink() {
// issue6456: Size returned may be longer due to encryption // issue6456: Size returned may be longer due to encryption
// on EXT-4 fscrypt. TODO maybe only do it on EXT4? // on EXT-4 fscrypt. TODO maybe only do it on EXT4?
self.outcome self.push_outcome(Outcome::Unsure, dirstate_node)?
.lock()
.unwrap()
.unsure
.push(hg_path.detach_from_tree())
} else if dirstate_node.has_copy_source() } else if dirstate_node.has_copy_source()
|| entry.is_from_other_parent() || entry.is_from_other_parent()
|| (size >= 0 && (size_changed || mode_changed())) || (size >= 0 && (size_changed || mode_changed()))
{ {
self.outcome self.push_outcome(Outcome::Modified, dirstate_node)?
.lock()
.unwrap()
.modified
.push(hg_path.detach_from_tree())
} else { } else {
let mtime_looks_clean; let mtime_looks_clean;
if let Some(dirstate_mtime) = entry.truncated_mtime() { if let Some(dirstate_mtime) = entry.truncated_mtime() {
let fs_mtime = TruncatedTimestamp::for_mtime_of(fs_metadata) let fs_mtime = TruncatedTimestamp::for_mtime_of(fs_metadata)
.expect("OS/libc does not support mtime?"); .expect("OS/libc does not support mtime?");
// There might be a change in the future if for example the // There might be a change in the future if for example the
// internal clock become off while process run, but this is a // internal clock become off while process run, but this is a
// case where the issues the user would face // case where the issues the user would face
// would be a lot worse and there is nothing we // would be a lot worse and there is nothing we
// can really do. // can really do.
mtime_looks_clean = fs_mtime.likely_equal(dirstate_mtime) mtime_looks_clean = fs_mtime.likely_equal(dirstate_mtime)
} else { } else {
// No mtime in the dirstate entry // No mtime in the dirstate entry
mtime_looks_clean = false mtime_looks_clean = false
}; };
if !mtime_looks_clean { if !mtime_looks_clean {
self.outcome self.push_outcome(Outcome::Unsure, dirstate_node)?
.lock()
.unwrap()
.unsure
.push(hg_path.detach_from_tree())
} else if self.options.list_clean { } else if self.options.list_clean {
self.outcome self.push_outcome(Outcome::Clean, dirstate_node)?
.lock()
.unwrap()
.clean
.push(hg_path.detach_from_tree())
} }
} }
Ok(()) Ok(())
} }
/// A node in the dirstate tree has no corresponding filesystem entry /// A node in the dirstate tree has no corresponding filesystem entry
fn traverse_dirstate_only( fn traverse_dirstate_only(
&self, &self,
dirstate_node: NodeRef<'tree, 'on_disk>, dirstate_node: NodeRef<'tree, 'on_disk>,
) -> Result<(), DirstateV2ParseError> { ) -> Result<(), DirstateV2ParseError> {
self.check_for_outdated_directory_cache(&dirstate_node)?; self.check_for_outdated_directory_cache(&dirstate_node)?;
self.mark_removed_or_deleted_if_file( self.mark_removed_or_deleted_if_file(&dirstate_node)?;
&dirstate_node.full_path_borrowed(self.dmap.on_disk)?,
dirstate_node.state()?,
);
dirstate_node dirstate_node
.children(self.dmap.on_disk)? .children(self.dmap.on_disk)?
.par_iter() .par_iter()
.map(|child_node| self.traverse_dirstate_only(child_node)) .map(|child_node| self.traverse_dirstate_only(child_node))
.collect() .collect()
} }
/// A node in the dirstate tree has no corresponding *file* on the /// A node in the dirstate tree has no corresponding *file* on the
/// filesystem /// filesystem
/// ///
/// Does nothing on a "directory" node /// Does nothing on a "directory" node
fn mark_removed_or_deleted_if_file( fn mark_removed_or_deleted_if_file(
&self, &self,
hg_path: &BorrowedPath<'tree, 'on_disk>, dirstate_node: &NodeRef<'tree, 'on_disk>,
dirstate_node_state: Option<EntryState>, ) -> Result<(), DirstateV2ParseError> {
) { if let Some(state) = dirstate_node.state()? {
if let Some(state) = dirstate_node_state { let path = dirstate_node.full_path(self.dmap.on_disk)?;
if self.matcher.matches(hg_path) { if self.matcher.matches(path) {
if let EntryState::Removed = state { if let EntryState::Removed = state {
self.outcome self.push_outcome(Outcome::Removed, dirstate_node)?
.lock()
.unwrap()
.removed
.push(hg_path.detach_from_tree())
} else { } else {
self.outcome self.push_outcome(Outcome::Deleted, &dirstate_node)?
.lock()
.unwrap()
.deleted
.push(hg_path.detach_from_tree())
} }
} }
} }
Ok(())
} }
/// Something in the filesystem has no corresponding dirstate node /// Something in the filesystem has no corresponding dirstate node
/// ///
/// Returns whether that path is ignored /// Returns whether that path is ignored
fn traverse_fs_only( fn traverse_fs_only(
&self, &self,
has_ignored_ancestor: bool, has_ignored_ancestor: bool,
fn mark_unknown_or_ignored( fn mark_unknown_or_ignored(
&self, &self,
has_ignored_ancestor: bool, has_ignored_ancestor: bool,
hg_path: &BorrowedPath<'_, 'on_disk>, hg_path: &BorrowedPath<'_, 'on_disk>,
) -> bool { ) -> bool {
let is_ignored = has_ignored_ancestor || (self.ignore_fn)(&hg_path); let is_ignored = has_ignored_ancestor || (self.ignore_fn)(&hg_path);
if is_ignored { if is_ignored {
if self.options.list_ignored { if self.options.list_ignored {
self.outcome self.push_outcome_without_copy_source(
.lock() Outcome::Ignored,
.unwrap() hg_path,
.ignored )
.push(hg_path.detach_from_tree())
} }
} else { } else {
if self.options.list_unknown { if self.options.list_unknown {
self.outcome self.push_outcome_without_copy_source(
.lock() Outcome::Unknown,
.unwrap() hg_path,
.unknown )
.push(hg_path.detach_from_tree())
} }
} }
is_ignored is_ignored
} }
} }
struct DirEntry { struct DirEntry {
base_name: HgPathBuf, base_name: HgPathBuf,

rust/hg-cpython/src/dirstate/status.rs

//! `rustext.dirstate.status`. //! `rustext.dirstate.status`.
use crate::{dirstate::DirstateMap, exceptions::FallbackError}; use crate::{dirstate::DirstateMap, exceptions::FallbackError};
use cpython::exc::OSError; use cpython::exc::OSError;
use cpython::{ use cpython::{
exc::ValueError, ObjectProtocol, PyBytes, PyErr, PyList, PyObject, exc::ValueError, ObjectProtocol, PyBytes, PyErr, PyList, PyObject,
PyResult, PyTuple, Python, PythonObject, ToPyObject, PyResult, PyTuple, Python, PythonObject, ToPyObject,
}; };
use hg::dirstate::status::StatusPath;
use hg::{ use hg::{
matchers::{AlwaysMatcher, FileMatcher, IncludeMatcher}, matchers::{AlwaysMatcher, FileMatcher, IncludeMatcher},
parse_pattern_syntax, parse_pattern_syntax,
utils::{ utils::{
files::{get_bytes_from_path, get_path_from_bytes}, files::{get_bytes_from_path, get_path_from_bytes},
hg_path::{HgPath, HgPathBuf}, hg_path::{HgPath, HgPathBuf},
}, },
BadMatch, DirstateStatus, IgnorePattern, PatternFileWarning, StatusError, BadMatch, DirstateStatus, IgnorePattern, PatternFileWarning, StatusError,
StatusOptions, StatusOptions,
}; };
use std::borrow::Borrow; use std::borrow::Borrow;
fn collect_status_path_list(py: Python, paths: &[StatusPath<'_>]) -> PyList {
collect_pybytes_list(py, paths.iter().map(|item| &*item.path))
}
/// This will be useless once trait impls for collection are added to `PyBytes` /// This will be useless once trait impls for collection are added to `PyBytes`
/// upstream. /// upstream.
fn collect_pybytes_list( fn collect_pybytes_list(
py: Python, py: Python,
collection: &[impl AsRef<HgPath>], iter: impl Iterator<Item = impl AsRef<HgPath>>,
) -> PyList { ) -> PyList {
let list = PyList::new(py, &[]); let list = PyList::new(py, &[]);
for path in collection.iter() { for path in iter {
list.append( list.append(
py, py,
PyBytes::new(py, path.as_ref().as_bytes()).into_object(), PyBytes::new(py, path.as_ref().as_bytes()).into_object(),
) )
} }
list list
} }
let ignore_files: PyResult<Vec<_>> = ignore_files let ignore_files: PyResult<Vec<_>> = ignore_files
.iter(py) .iter(py)
.map(|b| { .map(|b| {
let file = b.extract::<PyBytes>(py)?; let file = b.extract::<PyBytes>(py)?;
Ok(get_path_from_bytes(file.data(py)).to_owned()) Ok(get_path_from_bytes(file.data(py)).to_owned())
}) })
.collect(); .collect();
let ignore_files = ignore_files?; let ignore_files = ignore_files?;
// The caller may call `copymap.items()` separately
let list_copies = false;
match matcher.get_type(py).name(py).borrow() { match matcher.get_type(py).name(py).borrow() {
"alwaysmatcher" => { "alwaysmatcher" => {
let matcher = AlwaysMatcher; let matcher = AlwaysMatcher;
let (status_res, warnings) = dmap let (status_res, warnings) = dmap
.status( .status(
&matcher, &matcher,
root_dir.to_path_buf(), root_dir.to_path_buf(),
ignore_files, ignore_files,
StatusOptions { StatusOptions {
check_exec, check_exec,
list_clean, list_clean,
list_ignored, list_ignored,
list_unknown, list_unknown,
list_copies,
collect_traversed_dirs, collect_traversed_dirs,
}, },
) )
.map_err(|e| handle_fallback(py, e))?; .map_err(|e| handle_fallback(py, e))?;
build_response(py, status_res, warnings) build_response(py, status_res, warnings)
} }
"exactmatcher" => { "exactmatcher" => {
let files = matcher.call_method( let files = matcher.call_method(
&matcher, &matcher,
root_dir.to_path_buf(), root_dir.to_path_buf(),
ignore_files, ignore_files,
StatusOptions { StatusOptions {
check_exec, check_exec,
list_clean, list_clean,
list_ignored, list_ignored,
list_unknown, list_unknown,
list_copies,
collect_traversed_dirs, collect_traversed_dirs,
}, },
) )
.map_err(|e| handle_fallback(py, e))?; .map_err(|e| handle_fallback(py, e))?;
build_response(py, status_res, warnings) build_response(py, status_res, warnings)
} }
"includematcher" => { "includematcher" => {
// Get the patterns from Python even though most of them are // Get the patterns from Python even though most of them are
&matcher, &matcher,
root_dir.to_path_buf(), root_dir.to_path_buf(),
ignore_files, ignore_files,
StatusOptions { StatusOptions {
check_exec, check_exec,
list_clean, list_clean,
list_ignored, list_ignored,
list_unknown, list_unknown,
list_copies,
collect_traversed_dirs, collect_traversed_dirs,
}, },
) )
.map_err(|e| handle_fallback(py, e))?; .map_err(|e| handle_fallback(py, e))?;
build_response(py, status_res, warnings) build_response(py, status_res, warnings)
} }
e => Err(PyErr::new::<ValueError, _>( e => Err(PyErr::new::<ValueError, _>(
py, py,
format!("Unsupported matcher {}", e), format!("Unsupported matcher {}", e),
)), )),
} }
} }
fn build_response( fn build_response(
py: Python, py: Python,
status_res: DirstateStatus, status_res: DirstateStatus,
warnings: Vec<PatternFileWarning>, warnings: Vec<PatternFileWarning>,
) -> PyResult<PyTuple> { ) -> PyResult<PyTuple> {
let modified = collect_pybytes_list(py, status_res.modified.as_ref()); let modified = collect_status_path_list(py, &status_res.modified);
let added = collect_pybytes_list(py, status_res.added.as_ref()); let added = collect_status_path_list(py, &status_res.added);
let removed = collect_pybytes_list(py, status_res.removed.as_ref()); let removed = collect_status_path_list(py, &status_res.removed);
let deleted = collect_pybytes_list(py, status_res.deleted.as_ref()); let deleted = collect_status_path_list(py, &status_res.deleted);
let clean = collect_pybytes_list(py, status_res.clean.as_ref()); let clean = collect_status_path_list(py, &status_res.clean);
let ignored = collect_pybytes_list(py, status_res.ignored.as_ref()); let ignored = collect_status_path_list(py, &status_res.ignored);
let unknown = collect_pybytes_list(py, status_res.unknown.as_ref()); let unknown = collect_status_path_list(py, &status_res.unknown);
let unsure = collect_pybytes_list(py, status_res.unsure.as_ref()); let unsure = collect_status_path_list(py, &status_res.unsure);
let bad = collect_bad_matches(py, status_res.bad.as_ref())?; let bad = collect_bad_matches(py, &status_res.bad)?;
let traversed = collect_pybytes_list(py, status_res.traversed.as_ref()); let traversed = collect_pybytes_list(py, status_res.traversed.iter());
let dirty = status_res.dirty.to_py_object(py); let dirty = status_res.dirty.to_py_object(py);
let py_warnings = PyList::new(py, &[]); let py_warnings = PyList::new(py, &[]);
for warning in warnings.iter() { for warning in warnings.iter() {
// We use duck-typing on the Python side for dispatch, good enough for // We use duck-typing on the Python side for dispatch, good enough for
// now. // now.
match warning { match warning {
PatternFileWarning::InvalidSyntax(file, syn) => { PatternFileWarning::InvalidSyntax(file, syn) => {
py_warnings.append( py_warnings.append(

rust/rhg/src/commands/status.rs

// status.rs // status.rs
// //
// Copyright 2020, Georges Racinet <georges.racinets@octobus.net> // Copyright 2020, Georges Racinet <georges.racinets@octobus.net>
// //
// This software may be used and distributed according to the terms of the // This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version. // GNU General Public License version 2 or any later version.
use crate::error::CommandError; use crate::error::CommandError;
use crate::ui::Ui; use crate::ui::Ui;
use crate::utils::path_utils::RelativizePaths; use crate::utils::path_utils::RelativizePaths;
use clap::{Arg, SubCommand}; use clap::{Arg, SubCommand};
use format_bytes::format_bytes; use format_bytes::format_bytes;
use hg; use hg;
use hg::config::Config; use hg::config::Config;
use hg::dirstate::has_exec_bit; use hg::dirstate::has_exec_bit;
use hg::dirstate::status::StatusPath;
use hg::dirstate::TruncatedTimestamp; use hg::dirstate::TruncatedTimestamp;
use hg::dirstate::RANGE_MASK_31BIT; use hg::dirstate::RANGE_MASK_31BIT;
use hg::errors::{HgError, IoResultExt}; use hg::errors::{HgError, IoResultExt};
use hg::lock::LockError; use hg::lock::LockError;
use hg::manifest::Manifest; use hg::manifest::Manifest;
use hg::matchers::AlwaysMatcher; use hg::matchers::AlwaysMatcher;
use hg::repo::Repo; use hg::repo::Repo;
use hg::utils::files::get_bytes_from_os_string; use hg::utils::files::get_bytes_from_os_string;
use hg::utils::files::get_path_from_bytes; use hg::utils::files::get_path_from_bytes;
use hg::utils::hg_path::{hg_path_to_path_buf, HgPath}; use hg::utils::hg_path::{hg_path_to_path_buf, HgPath};
use hg::{HgPathCow, StatusOptions}; use hg::StatusOptions;
use log::{info, warn}; use log::{info, warn};
use std::io; use std::io;
use std::path::PathBuf; use std::path::PathBuf;
pub const HELP_TEXT: &str = " pub const HELP_TEXT: &str = "
Show changed files in the working directory Show changed files in the working directory
This is a pure Rust version of `hg status`. This is a pure Rust version of `hg status`.
) )
.arg( .arg(
Arg::with_name("ignored") Arg::with_name("ignored")
.help("show only ignored files") .help("show only ignored files")
.short("-i") .short("-i")
.long("--ignored"), .long("--ignored"),
) )
.arg( .arg(
Arg::with_name("copies")
.help("show source of copied files (DEFAULT: ui.statuscopies)")
.short("-C")
.long("--copies"),
)
.arg(
Arg::with_name("no-status") Arg::with_name("no-status")
.help("hide status prefix") .help("hide status prefix")
.short("-n") .short("-n")
.long("--no-status"), .long("--no-status"),
) )
} }
/// Pure data type allowing the caller to specify file states to display /// Pure data type allowing the caller to specify file states to display
return Err(CommandError::unsupported( return Err(CommandError::unsupported(
"status.terse is not yet supported with rhg status", "status.terse is not yet supported with rhg status",
)); ));
} }
let ui = invocation.ui; let ui = invocation.ui;
let config = invocation.config; let config = invocation.config;
let args = invocation.subcommand_args; let args = invocation.subcommand_args;
let display_states = if args.is_present("all") { let all = args.is_present("all");
let display_states = if all {
// TODO when implementing `--quiet`: it excludes clean files // TODO when implementing `--quiet`: it excludes clean files
// from `--all` // from `--all`
ALL_DISPLAY_STATES ALL_DISPLAY_STATES
} else { } else {
let requested = DisplayStates { let requested = DisplayStates {
modified: args.is_present("modified"), modified: args.is_present("modified"),
added: args.is_present("added"), added: args.is_present("added"),
removed: args.is_present("removed"), removed: args.is_present("removed"),
clean: args.is_present("clean"), clean: args.is_present("clean"),
deleted: args.is_present("deleted"), deleted: args.is_present("deleted"),
unknown: args.is_present("unknown"), unknown: args.is_present("unknown"),
ignored: args.is_present("ignored"), ignored: args.is_present("ignored"),
}; };
if requested.is_empty() { if requested.is_empty() {
DEFAULT_DISPLAY_STATES DEFAULT_DISPLAY_STATES
} else { } else {
requested requested
} }
}; };
let no_status = args.is_present("no-status"); let no_status = args.is_present("no-status");
let list_copies = all
|| args.is_present("copies")
|| config.get_bool(b"ui", b"statuscopies")?;
let repo = invocation.repo?; let repo = invocation.repo?;
if repo.has_sparse() || repo.has_narrow() { if repo.has_sparse() || repo.has_narrow() {
return Err(CommandError::unsupported( return Err(CommandError::unsupported(
"rhg status is not supported for sparse checkouts or narrow clones yet" "rhg status is not supported for sparse checkouts or narrow clones yet"
)); ));
} }
let mut dmap = repo.dirstate_map_mut()?; let mut dmap = repo.dirstate_map_mut()?;
let options = StatusOptions { let options = StatusOptions {
// we're currently supporting file systems with exec flags only // we're currently supporting file systems with exec flags only
// anyway // anyway
check_exec: true, check_exec: true,
list_clean: display_states.clean, list_clean: display_states.clean,
list_unknown: display_states.unknown, list_unknown: display_states.unknown,
list_ignored: display_states.ignored, list_ignored: display_states.ignored,
list_copies,
collect_traversed_dirs: false, collect_traversed_dirs: false,
}; };
let (mut ds_status, pattern_warnings) = dmap.status( let (mut ds_status, pattern_warnings) = dmap.status(
&AlwaysMatcher, &AlwaysMatcher,
repo.working_directory_path().to_owned(), repo.working_directory_path().to_owned(),
ignore_files(repo, config), ignore_files(repo, config),
options, options,
)?; )?;
if !pattern_warnings.is_empty() { if !pattern_warnings.is_empty() {
warn!("Pattern warnings: {:?}", &pattern_warnings); warn!("Pattern warnings: {:?}", &pattern_warnings);
} }
if !ds_status.bad.is_empty() { if !ds_status.bad.is_empty() {
warn!("Bad matches {:?}", &(ds_status.bad)) warn!("Bad matches {:?}", &(ds_status.bad))
} }
if !ds_status.unsure.is_empty() { if !ds_status.unsure.is_empty() {
info!( info!(
"Files to be rechecked by retrieval from filelog: {:?}", "Files to be rechecked by retrieval from filelog: {:?}",
&ds_status.unsure ds_status.unsure.iter().map(|s| &s.path).collect::<Vec<_>>()
); );
} }
let mut fixup = Vec::new(); let mut fixup = Vec::new();
if !ds_status.unsure.is_empty() if !ds_status.unsure.is_empty()
&& (display_states.modified || display_states.clean) && (display_states.modified || display_states.clean)
{ {
let p1 = repo.dirstate_parents()?.p1; let p1 = repo.dirstate_parents()?.p1;
let manifest = repo.manifest_for_node(p1).map_err(|e| { let manifest = repo.manifest_for_node(p1).map_err(|e| {
CommandError::from((e, &*format!("{:x}", p1.short()))) CommandError::from((e, &*format!("{:x}", p1.short())))
})?; })?;
for to_check in ds_status.unsure { for to_check in ds_status.unsure {
if unsure_is_modified(repo, &manifest, &to_check)? { if unsure_is_modified(repo, &manifest, &to_check.path)? {
if display_states.modified { if display_states.modified {
ds_status.modified.push(to_check); ds_status.modified.push(to_check);
} }
} else { } else {
if display_states.clean { if display_states.clean {
ds_status.clean.push(to_check.clone()); ds_status.clean.push(to_check.clone());
} }
fixup.push(to_check.into_owned()) fixup.push(to_check.path.into_owned())
} }
} }
} }
let relative_paths = (!ui.plain()) let relative_paths = (!ui.plain())
&& config && config
.get_option(b"commands", b"status.relative")? .get_option(b"commands", b"status.relative")?
.unwrap_or(config.get_bool(b"ui", b"relative-paths")?); .unwrap_or(config.get_bool(b"ui", b"relative-paths")?);
let output = DisplayStatusPaths { let output = DisplayStatusPaths {
} }
impl DisplayStatusPaths<'_> { impl DisplayStatusPaths<'_> {
// Probably more elegant to use a Deref or Borrow trait rather than // Probably more elegant to use a Deref or Borrow trait rather than
// harcode HgPathBuf, but probably not really useful at this point // harcode HgPathBuf, but probably not really useful at this point
fn display( fn display(
&self, &self,
status_prefix: &[u8], status_prefix: &[u8],
mut paths: Vec<HgPathCow>, mut paths: Vec<StatusPath<'_>>,
) -> Result<(), CommandError> { ) -> Result<(), CommandError> {
paths.sort_unstable(); paths.sort_unstable();
for path in paths { for StatusPath { path, copy_source } in paths {
let relative; let relative;
let path = if let Some(relativize) = &self.relativize { let path = if let Some(relativize) = &self.relativize {
relative = relativize.relativize(&path); relative = relativize.relativize(&path);
&*relative &*relative
} else { } else {
path.as_bytes() path.as_bytes()
}; };
// TODO optim, probably lots of unneeded copies here, especially // TODO optim, probably lots of unneeded copies here, especially
// if out stream is buffered // if out stream is buffered
if self.no_status { if self.no_status {
self.ui.write_stdout(&format_bytes!(b"{}\n", path))? self.ui.write_stdout(&format_bytes!(b"{}\n", path))?
} else { } else {
self.ui.write_stdout(&format_bytes!( self.ui.write_stdout(&format_bytes!(
b"{} {}\n", b"{} {}\n",
status_prefix, status_prefix,
path path
))? ))?
} }
if let Some(source) = copy_source {
self.ui.write_stdout(&format_bytes!(
b" {}\n",
source.as_bytes()
))?
}
} }
Ok(()) Ok(())
} }
} }
/// Check if a file is modified by comparing actual repo store and file system. /// Check if a file is modified by comparing actual repo store and file system.
/// ///
/// This meant to be used for those that the dirstate cannot resolve, due /// This meant to be used for those that the dirstate cannot resolve, due

tests/test-rename-dir-merge.t

TODO: fix rhg bugs that make this test fail when status is enabled
$ unset RHG_STATUS
$ hg init t $ hg init t
$ cd t $ cd t
$ mkdir a $ mkdir a
$ echo foo > a/a $ echo foo > a/a
$ echo bar > a/b $ echo bar > a/b
$ hg ci -Am "0" $ hg ci -Am "0"
adding a/a adding a/a

tests/test-status.t

#testcases dirstate-v1 dirstate-v2 #testcases dirstate-v1 dirstate-v2
#if dirstate-v2 #if dirstate-v2
$ cat >> $HGRCPATH << EOF $ cat >> $HGRCPATH << EOF
> [format] > [format]
> exp-rc-dirstate-v2=1 > exp-rc-dirstate-v2=1
> [storage] > [storage]
> dirstate-v2.slow-path=allow > dirstate-v2.slow-path=allow
> EOF > EOF
#endif #endif
TODO: fix rhg bugs that make this test fail when status is enabled
$ unset RHG_STATUS
$ hg init repo1 $ hg init repo1
$ cd repo1 $ cd repo1
$ mkdir a b a/1 b/1 b/2 $ mkdir a b a/1 b/1 b/2
$ touch in_root a/in_a b/in_b a/1/in_a_1 b/1/in_b_1 b/2/in_b_2 $ touch in_root a/in_a b/in_b a/1/in_a_1 b/1/in_b_1 b/2/in_b_2
hg status in repo root: hg status in repo root:
$ hg status $ hg status
$ hg status $ hg status
A added A added
R removed R removed
! deleted ! deleted
? unknown ? unknown
hg status -n: hg status -n:
$ env RHG_STATUS=1 RHG_ON_UNSUPPORTED=abort hg status -n $ env RHG_ON_UNSUPPORTED=abort hg status -n
added added
removed removed
deleted deleted
unknown unknown
hg status modified added removed deleted unknown never-existed ignored: hg status modified added removed deleted unknown never-existed ignored:
$ hg status modified added removed deleted unknown never-existed ignored $ hg status modified added removed deleted unknown never-existed ignored