This is an archive of the discontinued Mercurial Phabricator instance.

Differential D9408

hg-core: add basic config module
ClosedPublic

Authored by Alphare on Nov 26 2020, 5:48 AM.

Details

Reviewers
marmoute
Group Reviewers
hg-reviewers
Commits
rHG95d6f31e88db: hg-core: add basic config module
rHGc5b9f088dcf4: hg-core: add basic config module
Summary

The config module exposes a Config struct, unused for now.

It only reads the config file local to the repository, but handles all valid
patterns and includes/unsets.
It is structured in layers instead of erasing by reverse order of precedence,
allowing us to transparently know more about the config for debugging purposes,
and potentially other things I haven't thought about yet.

This change also introduces format_bytes! to hg-core.

Diff Detail

Repository
rHG Mercurial
Branch
default
Lint
No Linters Available
Unit
No Unit Test Coverage

Event Timeline

Alphare created this revision.Nov 26 2020, 5:48 AM
pulkit added a subscriber: pulkit.Nov 26 2020, 7:43 AM

I am not sure if had a look at https://phab.mercurial-scm.org/D7575 or not.

This change also introduces format_bytes! to hg-core.

Kindly separate that into a different patch.

Alphare added a comment.Nov 26 2020, 8:20 AM
In D9408#141395, @pulkit wrote:

I am not sure if had a look at https://phab.mercurial-scm.org/D7575 or not.

I have, I'd even commented at the time. I decided against using this patch, because there was a lot of Facebook-specific work and differed too much from the Python implementation. This is a lot simpler and will grow according to future experimentation

This change also introduces format_bytes! to hg-core.

Kindly separate that into a different patch.

Just to make sure, do you want that I split the Cargo.toml change into a separate patch? This is the first patch that uses the macro in hg-core.

marmoute requested changes to this revision.Nov 26 2020, 9:02 AM
marmoute added a subscriber: marmoute.
In D9408#141443, @Alphare wrote:
In D9408#141395, @pulkit wrote:

This change also introduces format_bytes! to hg-core.

Kindly separate that into a different patch.

Just to make sure, do you want that I split the Cargo.toml change into a separate patch? This is the first patch that uses the macro in hg-core.

Looks like you found other users ;-)

Here is a group of questions and feedbacks.

rust/hg-core/src/config/config.rs
18–22

I don't think we should be talking about layer in the higher level API. Whats your motivation for it ?

46–52

note: I expect we will get more this kind of thing. Wehen we get them is will probably be worth avec a config.utils module somewhere.

92

Why is this called get_option ?

110–111

The layer should be a implementation details, why do we return it there ?

rust/hg-core/src/config/layer.rs
78

Do you mean the higher priority layer have index 0 and the lower have higher index ?

83–84

I liked the original comment better ;-)

137

I would expect this to be passed as reference instead of cloned everytime. What's the motivation to do it this way ?

This revision now requires changes to proceed.Nov 26 2020, 9:02 AM
Alphare marked 2 inline comments as done.Nov 26 2020, 10:12 AM
Alphare edited parent revisions, added: D9409: hg-core: add format-bytes dependency; removed: D9407: rhg: use `format_bytes!` for error messages.
Alphare updated this revision to Diff 23705.
Alphare added inline comments.Nov 26 2020, 10:12 AM
rust/hg-core/src/config/config.rs
18–22

Yes, that should have been moved to ConfigLayer

92

I suppose get_bool is a better name indeed.

110–111

This should be the private interface, I'll update with the real public one.

rust/hg-core/src/config/layer.rs
78

Yes. Maybe this is not clear?

83–84

Sorry ;)

137

This is due to a limitation in the HashMap's Entry API. I could work around it but I doubt very much the config is going to show up in our profiles. I'll be happily proven wrong when we start using it.

marmoute added inline comments.Nov 26 2020, 11:06 AM
rust/hg-core/src/config/layer.rs
78

Is seems like the wrong order to me. Because it will make adding more layer annoying. Why did you pick this order (especially since happening layers is already giving you the right order).

(Also, not sure it is a good idea to make conversatin as "done" yourself)

Alphare added inline comments.Nov 26 2020, 11:09 AM
rust/hg-core/src/config/layer.rs
78

Why did you pick this order

Because then layers are ordered by their importance, and it made sense to me. Maybe I'm not seeing the broader picture?

(Also, not sure it is a good idea to make conversatin as "done" yourself)

This was Phabricator's doing, sorry!

marmoute added inline comments.Nov 26 2020, 11:36 AM
rust/hg-core/src/config/layer.rs
78

You can apply the very same argument saying that the most important one have the highest index ;-)

Broader picture wise, The benefit of using layered config it make it easy to have temporary override. For example when adding the repository config, or the share config, or internal overrde inside a the code. So these things will keep getting new layers. It will be much simple to add (and drop) layer at the end of the vec than at the start of it.

Bonus point being that we can use an immutable Vec from im_rs and get a very low cost override.

Alphare updated this revision to Diff 23706.Nov 27 2020, 4:31 AM
Alphare updated this revision to Diff 23707.Nov 27 2020, 4:33 AM
Alphare added inline comments.Nov 27 2020, 4:34 AM
rust/hg-core/src/config/layer.rs
78

You can apply the very same argument saying that the most important one have the highest index ;-)

Precisely why I wasn't sure why you'd want it reversed, hehe

It will be much simple to add (and drop) layer at the end of the vec than at the start of it.

Sure, that makes sense, I'll reverse it then.

Bonus point being that we can use an immutable Vec from im_rs and get a very low cost override.

I'm not certain pushing a few bytes to a non-empty Vec will be very important in terms of performance. :)

Alphare updated this revision to Diff 23883.Nov 30 2020, 11:01 AM
Alphare updated this revision to Diff 23902.Dec 1 2020, 8:55 AM
marmoute added a comment.Dec 14 2020, 5:28 AM

I see comment mentionning feedback taken in account, but no code update. did you forgot to refresh that Diff ?

rust/hg-core/src/config/config.rs
44–50

Comment from the future: When the number of these grows it will probably make sense to move them in their own module. (but this is premature).

92

Was this not supposed to become "get_bool" ?

110–111

seems not updated either

rust/hg-core/src/config/layer.rs
78

If you include all default config this grow a bit further than "a few bytes", but I agree this is a details.

Alphare added a comment.Dec 14 2020, 5:36 AM
In D9408#144258, @marmoute wrote:

I see comment mentionning feedback taken in account, but no code update. did you forgot to refresh that Diff ?

... yeah it appears that somehow I forgot (or failed) to update this. So I just did and rebased for good measure.

pulkit added a comment.Dec 18 2020, 3:50 AM

Seems like this one is awaiting an update.

Alphare added a comment.Dec 18 2020, 3:51 AM
In D9408#145025, @pulkit wrote:

Seems like this one is awaiting an update.

Ok there is definitely something fishy. I remember clearly phabsending the update at least the second time. Thanks Pulkit, I'll see what happens on the third try.

Alphare updated this revision to Diff 24377.Dec 18 2020, 4:07 AM
Alphare added inline comments.Dec 18 2020, 4:13 AM
rust/hg-core/src/config/config.rs
92

Kept get_option and added get_bool because I feel like both are going to be useful.

marmoute accepted this revision.Dec 22 2020, 3:53 AM

Looks good beside the two small nits.

rust/hg-core/src/config/config.rs
92

But why does get_option returns a bool too then ?

117–118

I would probably move all the private method after the pub one for clarity.

Alphare added a commit: rHGc5b9f088dcf4: hg-core: add basic config module.Dec 28 2020, 7:46 AM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHGc5b9f088dcf4: hg-core: add basic config module (authored by Alphare). · Explain Why
This revision was automatically updated to reflect the committed changes.
pulkit added a comment.Dec 28 2020, 7:53 AM

@Alphare dropping this patch as it breaks the build. I had to prune it, so kindly hg touch to revive this.

Alphare updated this revision to Diff 24535.Dec 29 2020, 5:01 AM
Alphare updated this revision to Diff 24536.Dec 29 2020, 5:08 AM
Alphare added inline comments.Dec 29 2020, 5:08 AM
rust/hg-core/src/config/config.rs
92

It returns an Option<bool>, it may be useful to know whether the config item existed before being coerced to false.

117–118

Good point.

pulkit added a comment.Dec 30 2020, 12:46 PM

Queuing it again.

Revision Contents
Changeset List

Diff 24377

rust/hg-core/src/config.rs

  • This file was added.
// config.rs
//
// Copyright 2020
// Valentin Gatien-Baron,
// Raphaël Gomès <rgomes@octobus.net>
//
// This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version.
//! Mercurial config parsing and interfaces.
mod config;
mod layer;
pub use config::Config;

rust/hg-core/src/config/config.rs

  • This file was added.
// config.rs
//
// Copyright 2020
// Valentin Gatien-Baron,
// Raphaël Gomès <rgomes@octobus.net>
//
// This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version.
use super::layer;
use crate::config::layer::{ConfigError, ConfigLayer, ConfigValue};
use std::path::PathBuf;
use crate::operations::FindRoot;
use crate::utils::files::read_whole_file;
/// Holds the config values for the current repository
/// TODO update this docstring once we support more sources
pub struct Config {
layers: Vec<layer::ConfigLayer>,
}
marmouteUnsubmitted
Not Done

I don't think we should be talking about layer in the higher level API. Whats your motivation for it ?

marmoute: I don't think we should be talking about layer in the higher level API. Whats your motivation…
AlphareAuthorUnsubmitted
Done

Yes, that should have been moved to ConfigLayer

Alphare: Yes, that should have been moved to `ConfigLayer`
impl std::fmt::Debug for Config {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
for (index, layer) in self.layers.iter().rev().enumerate() {
write!(
f,
"==== Layer {} (trusted: {}) ====\n{:?}",
index, layer.trusted, layer
)?;
}
Ok(())
}
}
pub enum ConfigSource {
/// Absolute path to a config file
AbsPath(PathBuf),
/// Already parsed (from the CLI, env, Python resources, etc.)
Parsed(layer::ConfigLayer),
}
pub fn parse_bool(v: &[u8]) -> Option<bool> {
match v.to_ascii_lowercase().as_slice() {
b"1" | b"yes" | b"true" | b"on" | b"always" => Some(true),
b"0" | b"no" | b"false" | b"off" | b"never" => Some(false),
_ => None,
}
}
marmouteUnsubmitted
Not Done

Comment from the future: When the number of these grows it will probably make sense to move them in their own module. (but this is premature).

marmoute: Comment from the future: When the number of these grows it will probably make sense to move…
impl Config {
/// Loads in order, which means that the precedence is the same
marmouteUnsubmitted
Done

note: I expect we will get more this kind of thing. Wehen we get them is will probably be worth avec a config.utils module somewhere.

marmoute: note: I expect we will get more this kind of thing. Wehen we get them is will probably be worth…
/// as the order of `sources`.
pub fn load_from_explicit_sources(
sources: Vec<ConfigSource>,
) -> Result<Self, ConfigError> {
let mut layers = vec![];
for source in sources.into_iter() {
match source {
ConfigSource::Parsed(c) => layers.push(c),
ConfigSource::AbsPath(c) => {
// TODO check if it should be trusted
// mercurial/ui.py:427
let data = match read_whole_file(&c) {
Err(_) => continue, // same as the python code
Ok(data) => data,
};
layers.extend(ConfigLayer::parse(&c, &data)?)
}
}
}
Ok(Config { layers })
}
/// Loads the local config. In a future version, this will also load the
/// `$HOME/.hgrc` and more to mirror the Python implementation.
pub fn load() -> Result<Self, ConfigError> {
let root = FindRoot::new().run().unwrap();
Ok(Self::load_from_explicit_sources(vec![
ConfigSource::AbsPath(root.join(".hg/hgrc")),
])?)
}
/// Returns an `Err` if the first value found is not a valid boolean.
/// Otherwise, returns an `Ok(option)`, where `option` is the boolean if
/// found, or `None`.
pub fn get_option(
&self,
section: &[u8],
item: &[u8],
marmouteUnsubmitted
Not Done

Why is this called get_option ?

marmoute: Why is this called `get_option` ?
AlphareAuthorUnsubmitted
Done

I suppose get_bool is a better name indeed.

Alphare: I suppose `get_bool` is a better name indeed.
marmouteUnsubmitted
Not Done

Was this not supposed to become "get_bool" ?

marmoute: Was this not supposed to become "get_bool" ?
AlphareAuthorUnsubmitted
Done

Kept get_option and added get_bool because I feel like both are going to be useful.

Alphare: Kept `get_option` and added `get_bool` because I feel like both are going to be useful.
marmouteUnsubmitted
Not Done

But why does get_option returns a bool too then ?

marmoute: But why does get_option returns a `bool` too then ?
AlphareAuthorUnsubmitted
Done

It returns an Option<bool>, it may be useful to know whether the config item existed before being coerced to false.

Alphare: It returns an `Option<bool>`, it may be useful to know whether the config item existed before…
) -> Result<Option<bool>, ConfigError> {
match self.get_inner(&section, &item) {
Some((layer, v)) => match parse_bool(&v.bytes) {
Some(b) => Ok(Some(b)),
None => Err(ConfigError::Parse {
origin: layer.origin.to_owned(),
line: v.line,
bytes: v.bytes.to_owned(),
}),
},
None => Ok(None),
}
}
/// Returns the corresponding boolean in the config. Returns `Ok(false)`
/// if the value is not found, an `Err` if it's not a valid boolean.
pub fn get_bool(
&self,
section: &[u8],
marmouteUnsubmitted
Not Done

The layer should be a implementation details, why do we return it there ?

marmoute: The layer should be a implementation details, why do we return it there ?
AlphareAuthorUnsubmitted
Done

This should be the private interface, I'll update with the real public one.

Alphare: This should be the private interface, I'll update with the real public one.
marmouteUnsubmitted
Not Done

seems not updated either

marmoute: seems not updated either
item: &[u8],
) -> Result<bool, ConfigError> {
Ok(self.get_option(section, item)?.unwrap_or(false))
}
/// Returns the layer and the value of the first one found, or `None`.
fn get_inner(
marmouteUnsubmitted
Not Done

I would probably move all the private method after the pub one for clarity.

marmoute: I would probably move all the private method after the `pub` one for clarity.
AlphareAuthorUnsubmitted
Done

Good point.

Alphare: Good point.
&self,
section: &[u8],
item: &[u8],
) -> Option<(&ConfigLayer, &ConfigValue)> {
for layer in self.layers.iter().rev() {
if !layer.trusted {
continue;
}
if let Some(v) = layer.get(&section, &item) {
return Some((&layer, v));
}
}
None
}
/// Returns the raw value bytes of the first one found, or `None`.
pub fn get(&self, section: &[u8], item: &[u8]) -> Option<&[u8]> {
self.get_inner(section, item)
.map(|(_, value)| value.bytes.as_ref())
}
/// Get raw values bytes from all layers (even untrusted ones) in order
/// of precedence.
#[cfg(test)]
fn get_all(&self, section: &[u8], item: &[u8]) -> Vec<&[u8]> {
let mut res = vec![];
for layer in self.layers.iter().rev() {
if let Some(v) = layer.get(&section, &item) {
res.push(v.bytes.as_ref());
}
}
res
}
}
#[cfg(test)]
mod tests {
use super::*;
use pretty_assertions::assert_eq;
use std::fs::File;
use std::io::Write;
#[test]
fn test_include_layer_ordering() {
let tmpdir = tempfile::tempdir().unwrap();
let tmpdir_path = tmpdir.path();
let mut included_file =
File::create(&tmpdir_path.join("included.rc")).unwrap();
included_file.write_all(b"[section]\nitem=value1").unwrap();
let base_config_path = tmpdir_path.join("base.rc");
let mut config_file = File::create(&base_config_path).unwrap();
let data =
b"[section]\nitem=value0\n%include included.rc\nitem=value2";
config_file.write_all(data).unwrap();
let sources = vec![ConfigSource::AbsPath(base_config_path)];
let config = Config::load_from_explicit_sources(sources)
.expect("expected valid config");
dbg!(&config);
let (_, value) = config.get_inner(b"section", b"item").unwrap();
assert_eq!(
value,
&ConfigValue {
bytes: b"value2".to_vec(),
line: Some(4)
}
);
let value = config.get(b"section", b"item").unwrap();
assert_eq!(value, b"value2",);
assert_eq!(
config.get_all(b"section", b"item"),
[b"value2", b"value1", b"value0"]
);
}
}

rust/hg-core/src/config/layer.rs

  • This file was added.
// layer.rs
//
// Copyright 2020
// Valentin Gatien-Baron,
// Raphaël Gomès <rgomes@octobus.net>
//
// This software may be used and distributed according to the terms of the
// GNU General Public License version 2 or any later version.
use crate::utils::files::{
get_bytes_from_path, get_path_from_bytes, read_whole_file,
};
use format_bytes::format_bytes;
use lazy_static::lazy_static;
use regex::bytes::Regex;
use std::collections::HashMap;
use std::io;
use std::path::{Path, PathBuf};
lazy_static! {
static ref SECTION_RE: Regex = make_regex(r"^\[([^\[]+)\]");
static ref ITEM_RE: Regex = make_regex(r"^([^=\s][^=]*?)\s*=\s*((.*\S)?)");
/// Continuation whitespace
static ref CONT_RE: Regex = make_regex(r"^\s+(\S|\S.*\S)\s*$");
static ref EMPTY_RE: Regex = make_regex(r"^(;|#|\s*$)");
static ref COMMENT_RE: Regex = make_regex(r"^(;|#)");
/// A directive that allows for removing previous entries
static ref UNSET_RE: Regex = make_regex(r"^%unset\s+(\S+)");
/// A directive that allows for including other config files
static ref INCLUDE_RE: Regex = make_regex(r"^%include\s+(\S|\S.*\S)\s*$");
}
/// All config values separated by layers of precedence.
/// Each config source may be split in multiple layers if `%include` directives
/// are used.
/// TODO detail the general precedence
#[derive(Clone)]
pub struct ConfigLayer {
/// Mapping of the sections to their items
sections: HashMap<Vec<u8>, ConfigItem>,
/// All sections (and their items/values) in a layer share the same origin
pub origin: ConfigOrigin,
/// Whether this layer comes from a trusted user or group
pub trusted: bool,
}
impl ConfigLayer {
pub fn new(origin: ConfigOrigin) -> Self {
ConfigLayer {
sections: HashMap::new(),
trusted: true, // TODO check
origin,
}
}
/// Add an entry to the config, overwriting the old one if already present.
pub fn add(
&mut self,
section: Vec<u8>,
item: Vec<u8>,
value: Vec<u8>,
line: Option<usize>,
) {
self.sections
.entry(section)
.or_insert_with(|| HashMap::new())
.insert(item, ConfigValue { bytes: value, line });
}
/// Returns the config value in `<section>.<item>` if it exists
pub fn get(&self, section: &[u8], item: &[u8]) -> Option<&ConfigValue> {
Some(self.sections.get(section)?.get(item)?)
}
pub fn is_empty(&self) -> bool {
self.sections.is_empty()
}
marmouteUnsubmitted
Not Done

Do you mean the higher priority layer have index 0 and the lower have higher index ?

marmoute: Do you mean the higher priority layer have index 0 and the lower have higher index ?
AlphareAuthorUnsubmitted
Done

Yes. Maybe this is not clear?

Alphare: Yes. Maybe this is not clear?
marmouteUnsubmitted
Not Done

Is seems like the wrong order to me. Because it will make adding more layer annoying. Why did you pick this order (especially since happening layers is already giving you the right order).

(Also, not sure it is a good idea to make conversatin as "done" yourself)

marmoute: Is seems like the wrong order to me. Because it will make adding more layer annoying. Why did…
AlphareAuthorUnsubmitted
Not Done

Why did you pick this order

Because then layers are ordered by their importance, and it made sense to me. Maybe I'm not seeing the broader picture?

(Also, not sure it is a good idea to make conversatin as "done" yourself)

This was Phabricator's doing, sorry!

Alphare: > Why did you pick this order Because then layers are ordered by their importance, and it made…
marmouteUnsubmitted
Not Done

You can apply the very same argument saying that the most important one have the highest index ;-)

Broader picture wise, The benefit of using layered config it make it easy to have temporary override. For example when adding the repository config, or the share config, or internal overrde inside a the code. So these things will keep getting new layers. It will be much simple to add (and drop) layer at the end of the vec than at the start of it.

Bonus point being that we can use an immutable Vec from im_rs and get a very low cost override.

marmoute: You can apply the very same argument saying that the most important one have the highest index…
AlphareAuthorUnsubmitted
Done

You can apply the very same argument saying that the most important one have the highest index ;-)

Precisely why I wasn't sure why you'd want it reversed, hehe

It will be much simple to add (and drop) layer at the end of the vec than at the start of it.

Sure, that makes sense, I'll reverse it then.

Bonus point being that we can use an immutable Vec from im_rs and get a very low cost override.

I'm not certain pushing a few bytes to a non-empty Vec will be very important in terms of performance. :)

Alphare: > You can apply the very same argument saying that the most important one have the highest…
marmouteUnsubmitted
Not Done

If you include all default config this grow a bit further than "a few bytes", but I agree this is a details.

marmoute: If you include all default config this grow a bit further than "a few bytes", but I agree this…
/// Returns a `Vec` of layers in order of precedence (so, in read order),
/// recursively parsing the `%include` directives if any.
pub fn parse(src: &Path, data: &[u8]) -> Result<Vec<Self>, ConfigError> {
let mut layers = vec![];
// Discard byte order mark if any
marmouteUnsubmitted
Not Done

I liked the original comment better ;-)

marmoute: I liked the original comment better ;-)
AlphareAuthorUnsubmitted
Done

Sorry ;)

Alphare: Sorry ;)
let data = if data.starts_with(b"\xef\xbb\xbf") {
&data[3..]
} else {
data
};
// TODO check if it's trusted
let mut current_layer = Self::new(ConfigOrigin::File(src.to_owned()));
let mut lines_iter =
data.split(|b| *b == b'\n').enumerate().peekable();
let mut section = b"".to_vec();
while let Some((index, bytes)) = lines_iter.next() {
if let Some(m) = INCLUDE_RE.captures(&bytes) {
let filename_bytes = &m[1];
let filename_to_include = get_path_from_bytes(&filename_bytes);
match read_include(&src, &filename_to_include) {
(include_src, Ok(data)) => {
layers.push(current_layer);
layers.extend(Self::parse(&include_src, &data)?);
current_layer =
Self::new(ConfigOrigin::File(src.to_owned()));
}
(_, Err(e)) => {
return Err(ConfigError::IncludeError {
path: filename_to_include.to_owned(),
io_error: e,
})
}
}
} else if let Some(_) = EMPTY_RE.captures(&bytes) {
} else if let Some(m) = SECTION_RE.captures(&bytes) {
section = m[1].to_vec();
} else if let Some(m) = ITEM_RE.captures(&bytes) {
let item = m[1].to_vec();
let mut value = m[2].to_vec();
loop {
match lines_iter.peek() {
None => break,
Some((_, v)) => {
if let Some(_) = COMMENT_RE.captures(&v) {
} else if let Some(_) = CONT_RE.captures(&v) {
value.extend(b"\n");
value.extend(&m[1]);
} else {
break;
}
}
};
lines_iter.next();
}
current_layer.add(
marmouteUnsubmitted
Done

I would expect this to be passed as reference instead of cloned everytime. What's the motivation to do it this way ?

marmoute: I would expect this to be passed as reference instead of cloned everytime. What's the…
AlphareAuthorUnsubmitted
Done

This is due to a limitation in the HashMap's Entry API. I could work around it but I doubt very much the config is going to show up in our profiles. I'll be happily proven wrong when we start using it.

Alphare: This is due to a limitation in the `HashMap`'s `Entry` API. I could work around it but I doubt…
section.clone(),
item,
value,
Some(index + 1),
);
} else if let Some(m) = UNSET_RE.captures(&bytes) {
if let Some(map) = current_layer.sections.get_mut(&section) {
map.remove(&m[1]);
}
} else {
return Err(ConfigError::Parse {
origin: ConfigOrigin::File(src.to_owned()),
line: Some(index + 1),
bytes: bytes.to_owned(),
});
}
}
if !current_layer.is_empty() {
layers.push(current_layer);
}
Ok(layers)
}
}
impl std::fmt::Debug for ConfigLayer {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
let mut sections: Vec<_> = self.sections.iter().collect();
sections.sort_by(|e0, e1| e0.0.cmp(e1.0));
for (section, items) in sections.into_iter() {
let mut items: Vec<_> = items.into_iter().collect();
items.sort_by(|e0, e1| e0.0.cmp(e1.0));
for (item, config_entry) in items {
writeln!(
f,
"{}",
String::from_utf8_lossy(&format_bytes!(
b"{}.{}={} # {}",
section,
item,
&config_entry.bytes,
&self.origin.to_bytes(),
))
)?
}
}
Ok(())
}
}
/// Mapping of section item to value.
/// In the following:
/// ```text
/// [ui]
/// paginate=no
/// ```
/// "paginate" is the section item and "no" the value.
pub type ConfigItem = HashMap<Vec<u8>, ConfigValue>;
#[derive(Clone, Debug, PartialEq)]
pub struct ConfigValue {
/// The raw bytes of the value (be it from the CLI, env or from a file)
pub bytes: Vec<u8>,
/// Only present if the value comes from a file, 1-indexed.
pub line: Option<usize>,
}
#[derive(Clone, Debug)]
pub enum ConfigOrigin {
/// The value comes from a configuration file
File(PathBuf),
/// The value comes from the environment like `$PAGER` or `$EDITOR`
Environment(Vec<u8>),
/* TODO cli
* TODO defaults (configitems.py)
* TODO extensions
* TODO Python resources?
* Others? */
}
impl ConfigOrigin {
/// TODO use some kind of dedicated trait?
pub fn to_bytes(&self) -> Vec<u8> {
match self {
ConfigOrigin::File(p) => get_bytes_from_path(p),
ConfigOrigin::Environment(e) => e.to_owned(),
}
}
}
#[derive(Debug)]
pub enum ConfigError {
Parse {
origin: ConfigOrigin,
line: Option<usize>,
bytes: Vec<u8>,
},
/// Failed to include a sub config file
IncludeError {
path: PathBuf,
io_error: std::io::Error,
},
/// Any IO error that isn't expected
IO(std::io::Error),
}
impl From<std::io::Error> for ConfigError {
fn from(e: std::io::Error) -> Self {
Self::IO(e)
}
}
fn make_regex(pattern: &'static str) -> Regex {
Regex::new(pattern).expect("expected a valid regex")
}
/// Includes are relative to the file they're defined in, unless they're
/// absolute.
fn read_include(
old_src: &Path,
new_src: &Path,
) -> (PathBuf, io::Result<Vec<u8>>) {
if new_src.is_absolute() {
(new_src.to_path_buf(), read_whole_file(&new_src))
} else {
let dir = old_src.parent().unwrap();
let new_src = dir.join(&new_src);
(new_src.to_owned(), read_whole_file(&new_src))
}
}

rust/hg-core/src/lib.rs

CopyMap, CopyMapIter, DirstateEntry, DirstateParents, EntryState, CopyMap, CopyMapIter, DirstateEntry, DirstateParents, EntryState,
StateMap, StateMapIter, StateMap, StateMapIter,
}; };
pub mod copy_tracing; pub mod copy_tracing;
mod filepatterns; mod filepatterns;
pub mod matchers; pub mod matchers;
pub mod revlog; pub mod revlog;
pub use revlog::*; pub use revlog::*;
pub mod config;
pub mod operations; pub mod operations;
pub mod utils; pub mod utils;
// Remove this to see (potential) non-artificial compile failures. MacOS // Remove this to see (potential) non-artificial compile failures. MacOS
// *should* compile, but fail to compile tests for example as of 2020-03-06 // *should* compile, but fail to compile tests for example as of 2020-03-06
#[cfg(not(target_os = "linux"))] #[cfg(not(target_os = "linux"))]
compile_error!( compile_error!(
"`hg-core` has only been tested on Linux and will most \ "`hg-core` has only been tested on Linux and will most \

rust/hg-core/src/utils/files.rs

hg_path::{path_to_hg_path_buf, HgPath, HgPathBuf, HgPathError}, hg_path::{path_to_hg_path_buf, HgPath, HgPathBuf, HgPathError},
path_auditor::PathAuditor, path_auditor::PathAuditor,
replace_slice, replace_slice,
}; };
use lazy_static::lazy_static; use lazy_static::lazy_static;
use same_file::is_same_file; use same_file::is_same_file;
use std::borrow::{Cow, ToOwned}; use std::borrow::{Cow, ToOwned};
use std::fs::Metadata; use std::fs::Metadata;
use std::io::Read;
use std::iter::FusedIterator; use std::iter::FusedIterator;
use std::ops::Deref; use std::ops::Deref;
use std::path::{Path, PathBuf}; use std::path::{Path, PathBuf};
pub fn get_path_from_bytes(bytes: &[u8]) -> &Path { pub fn get_path_from_bytes(bytes: &[u8]) -> &Path {
let os_str; let os_str;
#[cfg(unix)] #[cfg(unix)]
{ {
need_sep = true need_sep = true
}; };
res.extend(c); res.extend(c);
} }
Cow::Owned(res) Cow::Owned(res)
} }
} }
/// Reads a file in one big chunk instead of doing multiple reads
pub fn read_whole_file(filepath: &Path) -> std::io::Result<Vec<u8>> {
let mut file = std::fs::File::open(filepath)?;
let size = file.metadata()?.len();
let mut res = vec![0; size as usize];
file.read_exact(&mut res)?;
Ok(res)
}
#[cfg(test)] #[cfg(test)]
mod tests { mod tests {
use super::*; use super::*;
use pretty_assertions::assert_eq; use pretty_assertions::assert_eq;
#[test] #[test]
fn find_dirs_some() { fn find_dirs_some() {
let mut dirs = super::find_dirs(HgPath::new(b"foo/bar/baz")); let mut dirs = super::find_dirs(HgPath::new(b"foo/bar/baz"));