This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9897

rust: use HgError in RevlogError and Vfs
ClosedPublic

Authored by SimonSapin on Jan 27 2021, 4:13 PM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHG43d63979a75e: rust: use HgError in RevlogError and Vfs

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

SimonSapin created this revision.Jan 27 2021, 4:13 PM
Alphare added a subscriber: Alphare.Jan 29 2021, 8:51 AM

These changes start mixing user output with core logic, which didn't use to happen. Since we're not doing anything particular with those errors, I'm fine with that change (I makes the code a lot simpler and is a welcome change), but it's possible we'll have to re-introduce finer-grained errors if need be. The matter of i18n remains completely unsolved, but that's for another time.

rust/hg-core/src/repo.rs
73

I know this is makes for better errors, but the API is kind of clunky. Fortunately my solution would have been to encapsulate this... which is what the VFS is for.

Alphare accepted this revision.Jan 29 2021, 8:51 AM
SimonSapin added a comment.Jan 29 2021, 9:05 AM

mixing user output with core logic

How do you mean?

rust/hg-core/src/repo.rs
73

Why clunky? Because it makes callers handle variants of HgError that are never returned? We could introduce a IoError with context, as an independent type for just the corresponding variant of HgError.

Alphare added inline comments.Jan 29 2021, 12:29 PM
rust/hg-core/src/repo.rs
73

Oh no, simply because for_file(&path) kind of feels redundant, as if it should be in fs::read in the first place, but it's not a big deal, more of a remark.

SimonSapin added a commit: rHG43d63979a75e: rust: use HgError in RevlogError and Vfs.Feb 1 2021, 12:37 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG43d63979a75e: rust: use HgError in RevlogError and Vfs (authored by SimonSapin). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 25437

rust/hg-core/src/errors.rs

#[from] #[from]
File(std::path::PathBuf), File(std::path::PathBuf),
/// `std::env::current_dir` returned `std::io::Error` /// `std::env::current_dir` returned `std::io::Error`
CurrentDir, CurrentDir,
} }
impl HgError { impl HgError {
pub fn corrupted(explanation: impl Into<String>) -> Self { pub fn corrupted(explanation: impl Into<String>) -> Self {
// TODO: capture a backtrace here and keep it in the error value
// to aid debugging?
// https://doc.rust-lang.org/std/backtrace/struct.Backtrace.html
HgError::CorruptedRepository(explanation.into()) HgError::CorruptedRepository(explanation.into())
} }
} }
// TODO: use `DisplayBytes` instead to show non-Unicode filenames losslessly? // TODO: use `DisplayBytes` instead to show non-Unicode filenames losslessly?
impl fmt::Display for HgError { impl fmt::Display for HgError {
fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result { fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
match self { match self {

rust/hg-core/src/lib.rs

// Copyright 2018-2020 Georges Racinet <georges.racinet@octobus.net> // Copyright 2018-2020 Georges Racinet <georges.racinet@octobus.net>
// and Mercurial contributors // and Mercurial contributors
// //
// This software may be used and distributed according to the terms of the // This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version. // GNU General Public License version 2 or any later version.
mod ancestors; mod ancestors;
pub mod dagops; pub mod dagops;
pub mod errors; pub mod errors;
pub use ancestors::{AncestorsIterator, LazyAncestors, MissingAncestors}; pub use ancestors::{AncestorsIterator, LazyAncestors, MissingAncestors};
mod dirstate; mod dirstate;
pub mod discovery; pub mod discovery;
pub mod requirements; pub mod requirements;
pub mod testing; // unconditionally built, for use from integration tests pub mod testing; // unconditionally built, for use from integration tests

rust/hg-core/src/operations/cat.rs

repo: &Repo, repo: &Repo,
revset: &str, revset: &str,
files: &[HgPathBuf], files: &[HgPathBuf],
) -> Result<Vec<u8>, RevlogError> { ) -> Result<Vec<u8>, RevlogError> {
let rev = crate::revset::resolve_single(revset, repo)?; let rev = crate::revset::resolve_single(revset, repo)?;
let changelog = Changelog::open(repo)?; let changelog = Changelog::open(repo)?;
let manifest = Manifest::open(repo)?; let manifest = Manifest::open(repo)?;
let changelog_entry = changelog.get_rev(rev)?; let changelog_entry = changelog.get_rev(rev)?;
let manifest_node = Node::from_hex(&changelog_entry.manifest_node()?) let manifest_node =
.map_err(|_| RevlogError::Corrupted)?; Node::from_hex_for_repo(&changelog_entry.manifest_node()?)?;
let manifest_entry = manifest.get_node(manifest_node.into())?; let manifest_entry = manifest.get_node(manifest_node.into())?;
let mut bytes = vec![]; let mut bytes = vec![];
for (manifest_file, node_bytes) in manifest_entry.files_with_nodes() { for (manifest_file, node_bytes) in manifest_entry.files_with_nodes() {
for cat_file in files.iter() { for cat_file in files.iter() {
if cat_file.as_bytes() == manifest_file.as_bytes() { if cat_file.as_bytes() == manifest_file.as_bytes() {
let index_path = store_path(manifest_file, b".i"); let index_path = store_path(manifest_file, b".i");
let data_path = store_path(manifest_file, b".d"); let data_path = store_path(manifest_file, b".d");
let file_log = let file_log =
Revlog::open(repo, &index_path, Some(&data_path))?; Revlog::open(repo, &index_path, Some(&data_path))?;
let file_node = Node::from_hex(node_bytes) let file_node = Node::from_hex_for_repo(node_bytes)?;
.map_err(|_| RevlogError::Corrupted)?;
let file_rev = file_log.get_node_rev(file_node.into())?; let file_rev = file_log.get_node_rev(file_node.into())?;
let data = file_log.get_rev_data(file_rev)?; let data = file_log.get_rev_data(file_rev)?;
if data.starts_with(&METADATA_DELIMITER) { if data.starts_with(&METADATA_DELIMITER) {
let end_delimiter_position = data let end_delimiter_position = data
[METADATA_DELIMITER.len()..] [METADATA_DELIMITER.len()..]
.windows(METADATA_DELIMITER.len()) .windows(METADATA_DELIMITER.len())
.position(|bytes| bytes == METADATA_DELIMITER); .position(|bytes| bytes == METADATA_DELIMITER);
if let Some(position) = end_delimiter_position { if let Some(position) = end_delimiter_position {

rust/hg-core/src/operations/list_tracked_files.rs

// list_tracked_files.rs // list_tracked_files.rs
// //
// Copyright 2020 Antoine Cezar <antoine.cezar@octobus.net> // Copyright 2020 Antoine Cezar <antoine.cezar@octobus.net>
// //
// This software may be used and distributed according to the terms of the // This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version. // GNU General Public License version 2 or any later version.
use crate::dirstate::parsers::parse_dirstate; use crate::dirstate::parsers::parse_dirstate;
use crate::errors::{HgError, IoResultExt}; use crate::errors::HgError;
use crate::repo::Repo; use crate::repo::Repo;
use crate::revlog::changelog::Changelog; use crate::revlog::changelog::Changelog;
use crate::revlog::manifest::{Manifest, ManifestEntry}; use crate::revlog::manifest::{Manifest, ManifestEntry};
use crate::revlog::node::Node; use crate::revlog::node::Node;
use crate::revlog::revlog::RevlogError; use crate::revlog::revlog::RevlogError;
use crate::utils::hg_path::HgPath; use crate::utils::hg_path::HgPath;
use crate::EntryState; use crate::EntryState;
use rayon::prelude::*; use rayon::prelude::*;
/// List files under Mercurial control in the working directory /// List files under Mercurial control in the working directory
/// by reading the dirstate /// by reading the dirstate
pub struct Dirstate { pub struct Dirstate {
/// The `dirstate` content. /// The `dirstate` content.
content: Vec<u8>, content: Vec<u8>,
} }
impl Dirstate { impl Dirstate {
pub fn new(repo: &Repo) -> Result<Self, HgError> { pub fn new(repo: &Repo) -> Result<Self, HgError> {
let content = repo let content = repo.hg_vfs().read("dirstate")?;
.hg_vfs()
.read("dirstate")
// TODO: this will be more accurate when we use `HgError` in
// `Vfs::read`.
.for_file("dirstate".as_ref())?;
Ok(Self { content }) Ok(Self { content })
} }
pub fn tracked_files(&self) -> Result<Vec<&HgPath>, HgError> { pub fn tracked_files(&self) -> Result<Vec<&HgPath>, HgError> {
let (_, entries, _) = parse_dirstate(&self.content)?; let (_, entries, _) = parse_dirstate(&self.content)?;
let mut files: Vec<&HgPath> = entries let mut files: Vec<&HgPath> = entries
.into_iter() .into_iter()
.filter_map(|(path, entry)| match entry.state { .filter_map(|(path, entry)| match entry.state {
pub fn list_rev_tracked_files( pub fn list_rev_tracked_files(
repo: &Repo, repo: &Repo,
revset: &str, revset: &str,
) -> Result<FilesForRev, RevlogError> { ) -> Result<FilesForRev, RevlogError> {
let rev = crate::revset::resolve_single(revset, repo)?; let rev = crate::revset::resolve_single(revset, repo)?;
let changelog = Changelog::open(repo)?; let changelog = Changelog::open(repo)?;
let manifest = Manifest::open(repo)?; let manifest = Manifest::open(repo)?;
let changelog_entry = changelog.get_rev(rev)?; let changelog_entry = changelog.get_rev(rev)?;
let manifest_node = Node::from_hex(&changelog_entry.manifest_node()?) let manifest_node =
.map_err(|_| RevlogError::Corrupted)?; Node::from_hex_for_repo(&changelog_entry.manifest_node()?)?;
let manifest_entry = manifest.get_node(manifest_node.into())?; let manifest_entry = manifest.get_node(manifest_node.into())?;
Ok(FilesForRev(manifest_entry)) Ok(FilesForRev(manifest_entry))
} }
pub struct FilesForRev(ManifestEntry); pub struct FilesForRev(ManifestEntry);
impl FilesForRev { impl FilesForRev {
pub fn iter(&self) -> impl Iterator<Item = &HgPath> { pub fn iter(&self) -> impl Iterator<Item = &HgPath> {
self.0.files() self.0.files()
} }
} }

rust/hg-core/src/repo.rs

use crate::errors::HgError; use crate::errors::{HgError, IoResultExt};
use crate::operations::{find_root, FindRootError}; use crate::operations::{find_root, FindRootError};
use crate::requirements; use crate::requirements;
use memmap::{Mmap, MmapOptions}; use memmap::{Mmap, MmapOptions};
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
/// A repository on disk /// A repository on disk
pub struct Repo { pub struct Repo {
working_directory: PathBuf, working_directory: PathBuf,
} }
} }
} }
impl Vfs<'_> { impl Vfs<'_> {
pub(crate) fn read( pub(crate) fn read(
&self, &self,
relative_path: impl AsRef<Path>, relative_path: impl AsRef<Path>,
) -> std::io::Result<Vec<u8>> { ) -> Result<Vec<u8>, HgError> {
std::fs::read(self.base.join(relative_path)) let path = self.base.join(relative_path);
} std::fs::read(&path).for_file(&path)
AlphareUnsubmitted
Not Done

I know this is makes for better errors, but the API is kind of clunky. Fortunately my solution would have been to encapsulate this... which is what the VFS is for.

Alphare: I know this is makes for better errors, but the API is kind of clunky. Fortunately my solution…
SimonSapinAuthorUnsubmitted
Done

Why clunky? Because it makes callers handle variants of HgError that are never returned? We could introduce a IoError with context, as an independent type for just the corresponding variant of HgError.

SimonSapin: Why clunky? Because it makes callers handle variants of `HgError` that are never returned? We…
AlphareUnsubmitted
Not Done

Oh no, simply because for_file(&path) kind of feels redundant, as if it should be in fs::read in the first place, but it's not a big deal, more of a remark.

Alphare: Oh no, simply because `for_file(&path)` kind of feels redundant, as if it should be in `fs…
pub(crate) fn open(
&self,
relative_path: impl AsRef<Path>,
) -> std::io::Result<std::fs::File> {
std::fs::File::open(self.base.join(relative_path))
} }
pub(crate) fn mmap_open( pub(crate) fn mmap_open(
&self, &self,
relative_path: impl AsRef<Path>, relative_path: impl AsRef<Path>,
) -> std::io::Result<Mmap> { ) -> Result<Mmap, HgError> {
let file = self.open(relative_path)?; let path = self.base.join(relative_path);
let file = std::fs::File::open(&path).for_file(&path)?;
// TODO: what are the safety requirements here? // TODO: what are the safety requirements here?
let mmap = unsafe { MmapOptions::new().map(&file) }?; let mmap = unsafe { MmapOptions::new().map(&file) }.for_file(&path)?;
Ok(mmap) Ok(mmap)
} }
} }

rust/hg-core/src/requirements.rs

use crate::errors::{HgError, HgResultExt, IoResultExt}; use crate::errors::{HgError, HgResultExt};
use crate::repo::Repo; use crate::repo::Repo;
fn parse(bytes: &[u8]) -> Result<Vec<String>, HgError> { fn parse(bytes: &[u8]) -> Result<Vec<String>, HgError> {
// The Python code reading this file uses `str.splitlines` // The Python code reading this file uses `str.splitlines`
// which looks for a number of line separators (even including a couple of // which looks for a number of line separators (even including a couple of
// non-ASCII ones), but Python code writing it always uses `\n`. // non-ASCII ones), but Python code writing it always uses `\n`.
let lines = bytes.split(|&byte| byte == b'\n'); let lines = bytes.split(|&byte| byte == b'\n');
lines lines
.filter(|line| !line.is_empty()) .filter(|line| !line.is_empty())
.map(|line| { .map(|line| {
// Python uses Unicode `str.isalnum` but feature names are all // Python uses Unicode `str.isalnum` but feature names are all
// ASCII // ASCII
if line[0].is_ascii_alphanumeric() && line.is_ascii() { if line[0].is_ascii_alphanumeric() && line.is_ascii() {
Ok(String::from_utf8(line.into()).unwrap()) Ok(String::from_utf8(line.into()).unwrap())
} else { } else {
Err(HgError::corrupted("parse error in 'requires' file")) Err(HgError::corrupted("parse error in 'requires' file"))
} }
}) })
.collect() .collect()
} }
pub fn load(repo: &Repo) -> Result<Vec<String>, HgError> { pub fn load(repo: &Repo) -> Result<Vec<String>, HgError> {
if let Some(bytes) = repo if let Some(bytes) =
.hg_vfs() repo.hg_vfs().read("requires").io_not_found_as_none()?
.read("requires")
.for_file("requires".as_ref())
.io_not_found_as_none()?
{ {
parse(&bytes) parse(&bytes)
} else { } else {
// Treat a missing file the same as an empty file. // Treat a missing file the same as an empty file.
// From `mercurial/localrepo.py`: // From `mercurial/localrepo.py`:
// > requires file contains a newline-delimited list of // > requires file contains a newline-delimited list of
// > features/capabilities the opener (us) must have in order to use // > features/capabilities the opener (us) must have in order to use
// > the repository. This file was introduced in Mercurial 0.9.2, // > the repository. This file was introduced in Mercurial 0.9.2,

rust/hg-core/src/revlog/changelog.rs

use crate::errors::HgError;
use crate::repo::Repo; use crate::repo::Repo;
use crate::revlog::revlog::{Revlog, RevlogError}; use crate::revlog::revlog::{Revlog, RevlogError};
use crate::revlog::NodePrefix; use crate::revlog::NodePrefix;
use crate::revlog::Revision; use crate::revlog::Revision;
/// A specialized `Revlog` to work with `changelog` data format. /// A specialized `Revlog` to work with `changelog` data format.
pub struct Changelog { pub struct Changelog {
/// The generic `revlog` format. /// The generic `revlog` format.
self.bytes self.bytes
.split(|b| b == &b'\n') .split(|b| b == &b'\n')
.filter(|line| !line.is_empty()) .filter(|line| !line.is_empty())
} }
/// Return the node id of the `manifest` referenced by this `changelog` /// Return the node id of the `manifest` referenced by this `changelog`
/// entry. /// entry.
pub fn manifest_node(&self) -> Result<&[u8], RevlogError> { pub fn manifest_node(&self) -> Result<&[u8], RevlogError> {
self.lines().next().ok_or(RevlogError::Corrupted) self.lines()
.next()
.ok_or_else(|| HgError::corrupted("empty changelog entry").into())
} }
} }

rust/hg-core/src/revlog/index.rs

use std::convert::TryInto; use std::convert::TryInto;
use std::ops::Deref; use std::ops::Deref;
use byteorder::{BigEndian, ByteOrder}; use byteorder::{BigEndian, ByteOrder};
use crate::errors::HgError;
use crate::revlog::node::Node; use crate::revlog::node::Node;
use crate::revlog::revlog::RevlogError; use crate::revlog::revlog::RevlogError;
use crate::revlog::{Revision, NULL_REVISION}; use crate::revlog::{Revision, NULL_REVISION};
pub const INDEX_ENTRY_SIZE: usize = 64; pub const INDEX_ENTRY_SIZE: usize = 64;
/// A Revlog index /// A Revlog index
pub struct Index { pub struct Index {
} }
if offset == bytes.len() { if offset == bytes.len() {
Ok(Self { Ok(Self {
bytes, bytes,
offsets: Some(offsets), offsets: Some(offsets),
}) })
} else { } else {
Err(RevlogError::Corrupted) Err(HgError::corrupted("unexpected inline revlog length")
.into())
} }
} else { } else {
Ok(Self { Ok(Self {
bytes, bytes,
offsets: None, offsets: None,
}) })
} }
} }

rust/hg-core/src/revlog/node.rs

// Copyright 2019-2020 Georges Racinet <georges.racinet@octobus.net> // Copyright 2019-2020 Georges Racinet <georges.racinet@octobus.net>
// //
// This software may be used and distributed according to the terms of the // This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version. // GNU General Public License version 2 or any later version.
//! Definitions and utilities for Revision nodes //! Definitions and utilities for Revision nodes
//! //!
//! In Mercurial code base, it is customary to call "a node" the binary SHA //! In Mercurial code base, it is customary to call "a node" the binary SHA
//! of a revision. //! of a revision.
use crate::errors::HgError;
use bytes_cast::BytesCast; use bytes_cast::BytesCast;
use std::convert::{TryFrom, TryInto}; use std::convert::{TryFrom, TryInto};
use std::fmt; use std::fmt;
/// The length in bytes of a `Node` /// The length in bytes of a `Node`
/// ///
/// This constant is meant to ease refactors of this module, and /// This constant is meant to ease refactors of this module, and
/// are private so that calling code does not expect all nodes have /// are private so that calling code does not expect all nodes have
let prefix = NodePrefix::from_hex(hex)?; let prefix = NodePrefix::from_hex(hex)?;
if prefix.nybbles_len() == NODE_NYBBLES_LENGTH { if prefix.nybbles_len() == NODE_NYBBLES_LENGTH {
Ok(Self { data: prefix.data }) Ok(Self { data: prefix.data })
} else { } else {
Err(FromHexError) Err(FromHexError)
} }
} }
/// `from_hex`, but for input from an internal file of the repository such
/// as a changelog or manifest entry.
///
/// An error is treated as repository corruption.
pub fn from_hex_for_repo(hex: impl AsRef<[u8]>) -> Result<Node, HgError> {
Self::from_hex(hex.as_ref()).map_err(|FromHexError| {
HgError::CorruptedRepository(format!(
"Expected a full hexadecimal node ID, found {}",
String::from_utf8_lossy(hex.as_ref())
))
})
}
/// Provide access to binary data /// Provide access to binary data
/// ///
/// This is needed by FFI layers, for instance to return expected /// This is needed by FFI layers, for instance to return expected
/// binary values to Python. /// binary values to Python.
pub fn as_bytes(&self) -> &[u8] { pub fn as_bytes(&self) -> &[u8] {
&self.data &self.data
} }
} }

rust/hg-core/src/revlog/nodemap_docket.rs

use crate::errors::{HgError, HgResultExt};
use bytes_cast::{unaligned, BytesCast}; use bytes_cast::{unaligned, BytesCast};
use memmap::Mmap; use memmap::Mmap;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
use super::revlog::RevlogError; use super::revlog::RevlogError;
use crate::repo::Repo; use crate::repo::Repo;
use crate::utils::strip_suffix; use crate::utils::strip_suffix;
/// later hg, maybe that should be a requirement instead?), or /// later hg, maybe that should be a requirement instead?), or
/// * The docket file points to a missing (likely deleted) data file (this /// * The docket file points to a missing (likely deleted) data file (this
/// can happen in a rare race condition). /// can happen in a rare race condition).
pub fn read_from_file( pub fn read_from_file(
repo: &Repo, repo: &Repo,
index_path: &Path, index_path: &Path,
) -> Result<Option<(Self, Mmap)>, RevlogError> { ) -> Result<Option<(Self, Mmap)>, RevlogError> {
let docket_path = index_path.with_extension("n"); let docket_path = index_path.with_extension("n");
let docket_bytes = match repo.store_vfs().read(&docket_path) { let docket_bytes = if let Some(bytes) =
Err(e) if e.kind() == std::io::ErrorKind::NotFound => { repo.store_vfs().read(&docket_path).io_not_found_as_none()?
return Ok(None) {
} bytes
Err(e) => return Err(RevlogError::IoError(e)), } else {
Ok(bytes) => bytes, return Ok(None);
}; };
let input = if let Some((&ONDISK_VERSION, rest)) = let input = if let Some((&ONDISK_VERSION, rest)) =
docket_bytes.split_first() docket_bytes.split_first()
{ {
rest rest
} else { } else {
return Ok(None); return Ok(None);
}; };
let (header, rest) = DocketHeader::from_bytes(input)?; /// Treat any error as a parse error
fn parse<T, E>(result: Result<T, E>) -> Result<T, RevlogError> {
result.map_err(|_| {
HgError::corrupted("nodemap docket parse error").into()
})
}
let (header, rest) = parse(DocketHeader::from_bytes(input))?;
let uid_size = header.uid_size as usize; let uid_size = header.uid_size as usize;
// TODO: do we care about overflow for 4 GB+ nodemap files on 32-bit // TODO: do we care about overflow for 4 GB+ nodemap files on 32-bit
// systems? // systems?
let tip_node_size = header.tip_node_size.get() as usize; let tip_node_size = header.tip_node_size.get() as usize;
let data_length = header.data_length.get() as usize; let data_length = header.data_length.get() as usize;
let (uid, rest) = u8::slice_from_bytes(rest, uid_size)?; let (uid, rest) = parse(u8::slice_from_bytes(rest, uid_size))?;
let (_tip_node, _rest) = u8::slice_from_bytes(rest, tip_node_size)?; let (_tip_node, _rest) =
let uid = parse(u8::slice_from_bytes(rest, tip_node_size))?;
std::str::from_utf8(uid).map_err(|_| RevlogError::Corrupted)?; let uid = parse(std::str::from_utf8(uid))?;
let docket = NodeMapDocket { data_length }; let docket = NodeMapDocket { data_length };
let data_path = rawdata_path(&docket_path, uid); let data_path = rawdata_path(&docket_path, uid);
// TODO: use `std::fs::read` here when the `persistent-nodemap.mmap` // TODO: use `vfs.read()` here when the `persistent-nodemap.mmap`
// config is false? // config is false?
match repo.store_vfs().mmap_open(&data_path) { if let Some(mmap) = repo
Ok(mmap) => { .store_vfs()
.mmap_open(&data_path)
.io_not_found_as_none()?
{
if mmap.len() >= data_length { if mmap.len() >= data_length {
Ok(Some((docket, mmap))) Ok(Some((docket, mmap)))
} else { } else {
Err(RevlogError::Corrupted) Err(HgError::corrupted("persistent nodemap too short").into())
}
} }
Err(error) => {
if error.kind() == std::io::ErrorKind::NotFound {
Ok(None)
} else { } else {
Err(RevlogError::IoError(error)) Ok(None)
}
}
} }
} }
} }
fn rawdata_path(docket_path: &Path, uid: &str) -> PathBuf { fn rawdata_path(docket_path: &Path, uid: &str) -> PathBuf {
let docket_name = docket_path let docket_name = docket_path
.file_name() .file_name()
.expect("expected a base name") .expect("expected a base name")

rust/hg-core/src/revlog/revlog.rs

use std::borrow::Cow; use std::borrow::Cow;
use std::io::Read; use std::io::Read;
use std::ops::Deref; use std::ops::Deref;
use std::path::Path; use std::path::Path;
use byteorder::{BigEndian, ByteOrder}; use byteorder::{BigEndian, ByteOrder};
use crypto::digest::Digest; use crypto::digest::Digest;
use crypto::sha1::Sha1; use crypto::sha1::Sha1;
use flate2::read::ZlibDecoder; use flate2::read::ZlibDecoder;
use micro_timer::timed; use micro_timer::timed;
use zstd; use zstd;
use super::index::Index; use super::index::Index;
use super::node::{NodePrefix, NODE_BYTES_LENGTH, NULL_NODE}; use super::node::{NodePrefix, NODE_BYTES_LENGTH, NULL_NODE};
use super::nodemap; use super::nodemap;
use super::nodemap::NodeMap; use super::nodemap::{NodeMap, NodeMapError};
use super::nodemap_docket::NodeMapDocket; use super::nodemap_docket::NodeMapDocket;
use super::patch; use super::patch;
use crate::errors::HgError;
use crate::repo::Repo; use crate::repo::Repo;
use crate::revlog::Revision; use crate::revlog::Revision;
#[derive(derive_more::From)]
pub enum RevlogError { pub enum RevlogError {
IoError(std::io::Error),
UnsuportedVersion(u16),
InvalidRevision, InvalidRevision,
/// Found more than one entry whose ID match the requested prefix /// Found more than one entry whose ID match the requested prefix
AmbiguousPrefix, AmbiguousPrefix,
Corrupted, #[from]
UnknowDataFormat(u8), Other(HgError),
} }
impl From<bytes_cast::FromBytesError> for RevlogError { impl From<NodeMapError> for RevlogError {
fn from(_: bytes_cast::FromBytesError) -> Self { fn from(error: NodeMapError) -> Self {
RevlogError::Corrupted match error {
NodeMapError::MultipleResults => RevlogError::AmbiguousPrefix,
NodeMapError::RevisionNotInIndex(_) => RevlogError::corrupted(),
}
}
}
impl RevlogError {
fn corrupted() -> Self {
RevlogError::Other(HgError::corrupted("corrupted revlog"))
} }
} }
/// Read only implementation of revlog. /// Read only implementation of revlog.
pub struct Revlog { pub struct Revlog {
/// When index and data are not interleaved: bytes of the revlog index. /// When index and data are not interleaved: bytes of the revlog index.
/// When index and data are interleaved: bytes of the revlog index and /// When index and data are interleaved: bytes of the revlog index and
/// data. /// data.
/// interleaved. /// interleaved.
#[timed] #[timed]
pub fn open( pub fn open(
repo: &Repo, repo: &Repo,
index_path: impl AsRef<Path>, index_path: impl AsRef<Path>,
data_path: Option<&Path>, data_path: Option<&Path>,
) -> Result<Self, RevlogError> { ) -> Result<Self, RevlogError> {
let index_path = index_path.as_ref(); let index_path = index_path.as_ref();
let index_mmap = repo let index_mmap = repo.store_vfs().mmap_open(&index_path)?;
.store_vfs()
.mmap_open(&index_path)
.map_err(RevlogError::IoError)?;
let version = get_version(&index_mmap); let version = get_version(&index_mmap);
if version != 1 { if version != 1 {
return Err(RevlogError::UnsuportedVersion(version)); // A proper new version should have had a repo/store requirement.
return Err(RevlogError::corrupted());
} }
let index = Index::new(Box::new(index_mmap))?; let index = Index::new(Box::new(index_mmap))?;
let default_data_path = index_path.with_extension("d"); let default_data_path = index_path.with_extension("d");
// type annotation required // type annotation required
// won't recognize Mmap as Deref<Target = [u8]> // won't recognize Mmap as Deref<Target = [u8]>
let data_bytes: Option<Box<dyn Deref<Target = [u8]> + Send>> = let data_bytes: Option<Box<dyn Deref<Target = [u8]> + Send>> =
if index.is_inline() { if index.is_inline() {
None None
} else { } else {
let data_path = data_path.unwrap_or(&default_data_path); let data_path = data_path.unwrap_or(&default_data_path);
let data_mmap = repo let data_mmap = repo.store_vfs().mmap_open(data_path)?;
.store_vfs()
.mmap_open(data_path)
.map_err(RevlogError::IoError)?;
Some(Box::new(data_mmap)) Some(Box::new(data_mmap))
}; };
let nodemap = NodeMapDocket::read_from_file(repo, index_path)?.map( let nodemap = NodeMapDocket::read_from_file(repo, index_path)?.map(
|(docket, data)| { |(docket, data)| {
nodemap::NodeTree::load_bytes( nodemap::NodeTree::load_bytes(
Box::new(data), Box::new(data),
docket.data_length, docket.data_length,
/// Return the full data associated to a node. /// Return the full data associated to a node.
#[timed] #[timed]
pub fn get_node_rev( pub fn get_node_rev(
&self, &self,
node: NodePrefix, node: NodePrefix,
) -> Result<Revision, RevlogError> { ) -> Result<Revision, RevlogError> {
if let Some(nodemap) = &self.nodemap { if let Some(nodemap) = &self.nodemap {
return nodemap return nodemap
.find_bin(&self.index, node) .find_bin(&self.index, node)?
// TODO: propagate details of this error:
.map_err(|_| RevlogError::Corrupted)?
.ok_or(RevlogError::InvalidRevision); .ok_or(RevlogError::InvalidRevision);
} }
// Fallback to linear scan when a persistent nodemap is not present. // Fallback to linear scan when a persistent nodemap is not present.
// This happens when the persistent-nodemap experimental feature is not // This happens when the persistent-nodemap experimental feature is not
// enabled, or for small revlogs. // enabled, or for small revlogs.
// //
// TODO: consider building a non-persistent nodemap in memory to // TODO: consider building a non-persistent nodemap in memory to
// optimize these cases. // optimize these cases.
let mut found_by_prefix = None; let mut found_by_prefix = None;
for rev in (0..self.len() as Revision).rev() { for rev in (0..self.len() as Revision).rev() {
let index_entry = let index_entry =
self.index.get_entry(rev).ok_or(RevlogError::Corrupted)?; self.index.get_entry(rev).ok_or(HgError::corrupted(
"revlog references a revision not in the index",
))?;
if node == *index_entry.hash() { if node == *index_entry.hash() {
return Ok(rev); return Ok(rev);
} }
if node.is_prefix_of(index_entry.hash()) { if node.is_prefix_of(index_entry.hash()) {
if found_by_prefix.is_some() { if found_by_prefix.is_some() {
return Err(RevlogError::AmbiguousPrefix); return Err(RevlogError::AmbiguousPrefix);
} }
found_by_prefix = Some(rev) found_by_prefix = Some(rev)
/// snapshot to rebuild the final data. /// snapshot to rebuild the final data.
#[timed] #[timed]
pub fn get_rev_data(&self, rev: Revision) -> Result<Vec<u8>, RevlogError> { pub fn get_rev_data(&self, rev: Revision) -> Result<Vec<u8>, RevlogError> {
// Todo return -> Cow // Todo return -> Cow
let mut entry = self.get_entry(rev)?; let mut entry = self.get_entry(rev)?;
let mut delta_chain = vec![]; let mut delta_chain = vec![];
while let Some(base_rev) = entry.base_rev { while let Some(base_rev) = entry.base_rev {
delta_chain.push(entry); delta_chain.push(entry);
entry = entry = self
self.get_entry(base_rev).or(Err(RevlogError::Corrupted))?; .get_entry(base_rev)
.map_err(|_| RevlogError::corrupted())?;
} }
// TODO do not look twice in the index // TODO do not look twice in the index
let index_entry = self let index_entry = self
.index .index
.get_entry(rev) .get_entry(rev)
.ok_or(RevlogError::InvalidRevision)?; .ok_or(RevlogError::InvalidRevision)?;
let data: Vec<u8> = if delta_chain.is_empty() { let data: Vec<u8> = if delta_chain.is_empty() {
entry.data()?.into() entry.data()?.into()
} else { } else {
Revlog::build_data_from_deltas(entry, &delta_chain)? Revlog::build_data_from_deltas(entry, &delta_chain)?
}; };
if self.check_hash( if self.check_hash(
index_entry.p1(), index_entry.p1(),
index_entry.p2(), index_entry.p2(),
index_entry.hash().as_bytes(), index_entry.hash().as_bytes(),
&data, &data,
) { ) {
Ok(data) Ok(data)
} else { } else {
Err(RevlogError::Corrupted) Err(RevlogError::corrupted())
} }
} }
/// Check the hash of some given data against the recorded hash. /// Check the hash of some given data against the recorded hash.
pub fn check_hash( pub fn check_hash(
&self, &self,
p1: Revision, p1: Revision,
p2: Revision, p2: Revision,
// header. // header.
b'\0' => Ok(Cow::Borrowed(self.bytes)), b'\0' => Ok(Cow::Borrowed(self.bytes)),
// Raw revision data follows. // Raw revision data follows.
b'u' => Ok(Cow::Borrowed(&self.bytes[1..])), b'u' => Ok(Cow::Borrowed(&self.bytes[1..])),
// zlib (RFC 1950) data. // zlib (RFC 1950) data.
b'x' => Ok(Cow::Owned(self.uncompressed_zlib_data()?)), b'x' => Ok(Cow::Owned(self.uncompressed_zlib_data()?)),
// zstd data. // zstd data.
b'\x28' => Ok(Cow::Owned(self.uncompressed_zstd_data()?)), b'\x28' => Ok(Cow::Owned(self.uncompressed_zstd_data()?)),
format_type => Err(RevlogError::UnknowDataFormat(format_type)), // A proper new format should have had a repo/store requirement.
_format_type => Err(RevlogError::corrupted()),
} }
} }
fn uncompressed_zlib_data(&self) -> Result<Vec<u8>, RevlogError> { fn uncompressed_zlib_data(&self) -> Result<Vec<u8>, RevlogError> {
let mut decoder = ZlibDecoder::new(self.bytes); let mut decoder = ZlibDecoder::new(self.bytes);
if self.is_delta() { if self.is_delta() {
let mut buf = Vec::with_capacity(self.compressed_len); let mut buf = Vec::with_capacity(self.compressed_len);
decoder decoder
.read_to_end(&mut buf) .read_to_end(&mut buf)
.or(Err(RevlogError::Corrupted))?; .map_err(|_| RevlogError::corrupted())?;
Ok(buf) Ok(buf)
} else { } else {
let mut buf = vec![0; self.uncompressed_len]; let mut buf = vec![0; self.uncompressed_len];
decoder decoder
.read_exact(&mut buf) .read_exact(&mut buf)
.or(Err(RevlogError::Corrupted))?; .map_err(|_| RevlogError::corrupted())?;
Ok(buf) Ok(buf)
} }
} }
fn uncompressed_zstd_data(&self) -> Result<Vec<u8>, RevlogError> { fn uncompressed_zstd_data(&self) -> Result<Vec<u8>, RevlogError> {
if self.is_delta() { if self.is_delta() {
let mut buf = Vec::with_capacity(self.compressed_len); let mut buf = Vec::with_capacity(self.compressed_len);
zstd::stream::copy_decode(self.bytes, &mut buf) zstd::stream::copy_decode(self.bytes, &mut buf)
.or(Err(RevlogError::Corrupted))?; .map_err(|_| RevlogError::corrupted())?;
Ok(buf) Ok(buf)
} else { } else {
let mut buf = vec![0; self.uncompressed_len]; let mut buf = vec![0; self.uncompressed_len];
let len = zstd::block::decompress_to_buffer(self.bytes, &mut buf) let len = zstd::block::decompress_to_buffer(self.bytes, &mut buf)
.or(Err(RevlogError::Corrupted))?; .map_err(|_| RevlogError::corrupted())?;
if len != self.uncompressed_len { if len != self.uncompressed_len {
Err(RevlogError::Corrupted) Err(RevlogError::corrupted())
} else { } else {
Ok(buf) Ok(buf)
} }
} }
} }
/// Tell if the entry is a snapshot or a delta /// Tell if the entry is a snapshot or a delta
/// (influences on decompression). /// (influences on decompression).

rust/rhg/src/error.rs

} }
} }
} }
} }
impl From<(RevlogError, &str)> for CommandError { impl From<(RevlogError, &str)> for CommandError {
fn from((err, rev): (RevlogError, &str)) -> CommandError { fn from((err, rev): (RevlogError, &str)) -> CommandError {
match err { match err {
RevlogError::IoError(err) => CommandError::Abort(Some(
utf8_to_local(&format!("abort: {}\n", err)).into(),
)),
RevlogError::InvalidRevision => CommandError::Abort(Some( RevlogError::InvalidRevision => CommandError::Abort(Some(
utf8_to_local(&format!( utf8_to_local(&format!(
"abort: invalid revision identifier {}\n", "abort: invalid revision identifier {}\n",
rev rev
)) ))
.into(), .into(),
)), )),
RevlogError::AmbiguousPrefix => CommandError::Abort(Some( RevlogError::AmbiguousPrefix => CommandError::Abort(Some(
utf8_to_local(&format!( utf8_to_local(&format!(
"abort: ambiguous revision identifier {}\n", "abort: ambiguous revision identifier {}\n",
rev rev
)) ))
.into(), .into(),
)), )),
RevlogError::UnsuportedVersion(version) => { RevlogError::Other(err) => CommandError::Other(err),
CommandError::Abort(Some(
utf8_to_local(&format!(
"abort: unsupported revlog version {}\n",
version
))
.into(),
))
}
RevlogError::Corrupted => {
CommandError::Abort(Some("abort: corrupted revlog\n".into()))
}
RevlogError::UnknowDataFormat(format) => {
CommandError::Abort(Some(
utf8_to_local(&format!(
"abort: unknow revlog dataformat {:?}\n",
format
))
.into(),
))
}
} }
} }
} }