( )⚙ D11700 dirstate-v2: add a new MTIME_SECOND_AMBIGUOUS flags

This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11700

dirstate-v2: add a new MTIME_SECOND_AMBIGUOUS flags
ClosedPublic

Authored by marmoute on Oct 19 2021, 6:03 PM.

Details

Reviewers
Alphare
Group Reviewers
hg-reviewers
Commits
rHG9205d9be8b41: dirstate-v2: add a new MTIME_SECOND_AMBIGUOUS flags
Summary

This flag will let us use more mtime value in the future. For now we have a
minimal handling of the flag at read time, but we will never put ourself in a
situation where we will needs to writes it.

See the flag documentation for details.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

marmoute created this revision.Oct 19 2021, 6:03 PM
Alphare accepted this revision.Oct 20 2021, 4:35 AM
This revision is now accepted and ready to land.Oct 20 2021, 4:35 AM
marmoute added a commit: rHG9205d9be8b41: dirstate-v2: add a new MTIME_SECOND_AMBIGUOUS flags.Oct 20 2021, 5:30 AM
Closed by commit rHG9205d9be8b41: dirstate-v2: add a new MTIME_SECOND_AMBIGUOUS flags (authored by marmoute). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 30944

mercurial/cext/parsers.c

&t->mtime_ns)) { &t->mtime_ns)) {
return NULL; return NULL;
} }
if (t->flags & dirstate_flag_expected_state_is_modified) { if (t->flags & dirstate_flag_expected_state_is_modified) {
t->flags &= ~(dirstate_flag_expected_state_is_modified | t->flags &= ~(dirstate_flag_expected_state_is_modified |
dirstate_flag_has_meaningful_data | dirstate_flag_has_meaningful_data |
dirstate_flag_has_file_mtime); dirstate_flag_has_file_mtime);
} }
if (t->flags & dirstate_flag_mtime_second_ambiguous) {
/* The current code is not able to do the more subtle comparison
* that the MTIME_SECOND_AMBIGUOUS requires. So we ignore the
* mtime */
t->flags &= ~(dirstate_flag_mtime_second_ambiguous |
dirstate_flag_has_meaningful_data |
dirstate_flag_has_file_mtime);
}
t->mode = 0; t->mode = 0;
if (t->flags & dirstate_flag_has_meaningful_data) { if (t->flags & dirstate_flag_has_meaningful_data) {
if (t->flags & dirstate_flag_mode_exec_perm) { if (t->flags & dirstate_flag_mode_exec_perm) {
t->mode = 0755; t->mode = 0755;
} else { } else {
t->mode = 0644; t->mode = 0644;
} }
if (t->flags & dirstate_flag_mode_is_symlink) { if (t->flags & dirstate_flag_mode_is_symlink) {

mercurial/cext/util.h

static const int dirstate_flag_mode_is_symlink = 1 << 7; static const int dirstate_flag_mode_is_symlink = 1 << 7;
static const int dirstate_flag_expected_state_is_modified = 1 << 8; static const int dirstate_flag_expected_state_is_modified = 1 << 8;
static const int dirstate_flag_all_unknown_recorded = 1 << 9; static const int dirstate_flag_all_unknown_recorded = 1 << 9;
static const int dirstate_flag_all_ignored_recorded = 1 << 10; static const int dirstate_flag_all_ignored_recorded = 1 << 10;
static const int dirstate_flag_fallback_exec = 1 << 11; static const int dirstate_flag_fallback_exec = 1 << 11;
static const int dirstate_flag_has_fallback_exec = 1 << 12; static const int dirstate_flag_has_fallback_exec = 1 << 12;
static const int dirstate_flag_fallback_symlink = 1 << 13; static const int dirstate_flag_fallback_symlink = 1 << 13;
static const int dirstate_flag_has_fallback_symlink = 1 << 14; static const int dirstate_flag_has_fallback_symlink = 1 << 14;
static const int dirstate_flag_mtime_second_ambiguous = 1 << 15;
extern PyTypeObject dirstateItemType; extern PyTypeObject dirstateItemType;
#define dirstate_tuple_check(op) (Py_TYPE(op) == &dirstateItemType) #define dirstate_tuple_check(op) (Py_TYPE(op) == &dirstateItemType)
#ifndef MIN #ifndef MIN
#define MIN(a, b) (((a) < (b)) ? (a) : (b)) #define MIN(a, b) (((a) < (b)) ? (a) : (b))
#endif #endif
/* VC9 doesn't include bool and lacks stdbool.h based on my searching */ /* VC9 doesn't include bool and lacks stdbool.h based on my searching */

mercurial/helptext/internals/dirstate-v2.txt

MODE_IS_SYMLINK = 1 << 7 MODE_IS_SYMLINK = 1 << 7
EXPECTED_STATE_IS_MODIFIED = 1 << 8 EXPECTED_STATE_IS_MODIFIED = 1 << 8
ALL_UNKNOWN_RECORDED = 1 << 9 ALL_UNKNOWN_RECORDED = 1 << 9
ALL_IGNORED_RECORDED = 1 << 10 ALL_IGNORED_RECORDED = 1 << 10
HAS_FALLBACK_EXEC = 1 << 11 HAS_FALLBACK_EXEC = 1 << 11
FALLBACK_EXEC = 1 << 12 FALLBACK_EXEC = 1 << 12
HAS_FALLBACK_SYMLINK = 1 << 13 HAS_FALLBACK_SYMLINK = 1 << 13
FALLBACK_SYMLINK = 1 << 14 FALLBACK_SYMLINK = 1 << 14
MTIME_SECOND_AMBIGUOUS = 1 << 15
The meaning of each bit is described below. The meaning of each bit is described below.
Other bits are unset. Other bits are unset.
They may be assigned meaning if the future, They may be assigned meaning if the future,
with the limitation that Mercurial versions that pre-date such meaning with the limitation that Mercurial versions that pre-date such meaning
will always reset those bits to unset when writing nodes. will always reset those bits to unset when writing nodes.
(A new node is written for any mutation in its subtree, (A new node is written for any mutation in its subtree,
system or operating system does not support that property, (e.g. system or operating system does not support that property, (e.g.
Windows). Windows).
`FALLBACK_SYMLINK` `FALLBACK_SYMLINK`
Should be ignored if `HAS_FALLBACK_SYMLINK` is unset. If set the file for Should be ignored if `HAS_FALLBACK_SYMLINK` is unset. If set the file for
this entry should be considered a symlink if that information cannot be this entry should be considered a symlink if that information cannot be
extracted from the file system. If unset it should be considered a normal extracted from the file system. If unset it should be considered a normal
file instead. file instead.
`MTIME_SECOND_AMBIGUOUS`
This flag is relevant only when `HAS_FILE_MTIME` is set. When set, the
`mtime` stored in the entry is only valid for comparison with timestamps
that have nanosecond information. If available timestamp does not carries
nanosecond information, the `mtime` should be ignored and no optimisation
can be applied.

mercurial/pure/parsers.py

DIRSTATE_V2_MODE_IS_SYMLINK = 1 << 7 DIRSTATE_V2_MODE_IS_SYMLINK = 1 << 7
DIRSTATE_V2_EXPECTED_STATE_IS_MODIFIED = 1 << 8 DIRSTATE_V2_EXPECTED_STATE_IS_MODIFIED = 1 << 8
DIRSTATE_V2_ALL_UNKNOWN_RECORDED = 1 << 9 DIRSTATE_V2_ALL_UNKNOWN_RECORDED = 1 << 9
DIRSTATE_V2_ALL_IGNORED_RECORDED = 1 << 10 DIRSTATE_V2_ALL_IGNORED_RECORDED = 1 << 10
DIRSTATE_V2_HAS_FALLBACK_EXEC = 1 << 11 DIRSTATE_V2_HAS_FALLBACK_EXEC = 1 << 11
DIRSTATE_V2_FALLBACK_EXEC = 1 << 12 DIRSTATE_V2_FALLBACK_EXEC = 1 << 12
DIRSTATE_V2_HAS_FALLBACK_SYMLINK = 1 << 13 DIRSTATE_V2_HAS_FALLBACK_SYMLINK = 1 << 13
DIRSTATE_V2_FALLBACK_SYMLINK = 1 << 14 DIRSTATE_V2_FALLBACK_SYMLINK = 1 << 14
DIRSTATE_V2_MTIME_SECOND_AMBIGUOUS = 1 << 15
@attr.s(slots=True, init=False) @attr.s(slots=True, init=False)
class DirstateItem(object): class DirstateItem(object):
"""represent a dirstate entry """represent a dirstate entry
It hold multiple attributes It hold multiple attributes
if has_meaningful_mtime: if has_meaningful_mtime:
self._mtime_s, self._mtime_ns = parentfiledata[2] self._mtime_s, self._mtime_ns = parentfiledata[2]
@classmethod @classmethod
def from_v2_data(cls, flags, size, mtime_s, mtime_ns): def from_v2_data(cls, flags, size, mtime_s, mtime_ns):
"""Build a new DirstateItem object from V2 data""" """Build a new DirstateItem object from V2 data"""
has_mode_size = bool(flags & DIRSTATE_V2_HAS_MODE_AND_SIZE) has_mode_size = bool(flags & DIRSTATE_V2_HAS_MODE_AND_SIZE)
has_meaningful_mtime = bool(flags & DIRSTATE_V2_HAS_FILE_MTIME) has_meaningful_mtime = bool(flags & DIRSTATE_V2_HAS_FILE_MTIME)
if flags & DIRSTATE_V2_MTIME_SECOND_AMBIGUOUS:
# The current code is not able to do the more subtle comparison that the
# MTIME_SECOND_AMBIGUOUS requires. So we ignore the mtime
has_meaningful_mtime = False
mode = None mode = None
if flags & +DIRSTATE_V2_EXPECTED_STATE_IS_MODIFIED: if flags & +DIRSTATE_V2_EXPECTED_STATE_IS_MODIFIED:
# we do not have support for this flag in the code yet, # we do not have support for this flag in the code yet,
# force a lookup for this file. # force a lookup for this file.
has_mode_size = False has_mode_size = False
has_meaningful_mtime = False has_meaningful_mtime = False

rust/hg-core/src/dirstate_tree/on_disk.rs

const MODE_IS_SYMLINK = 1 << 7; const MODE_IS_SYMLINK = 1 << 7;
const EXPECTED_STATE_IS_MODIFIED = 1 << 8; const EXPECTED_STATE_IS_MODIFIED = 1 << 8;
const ALL_UNKNOWN_RECORDED = 1 << 9; const ALL_UNKNOWN_RECORDED = 1 << 9;
const ALL_IGNORED_RECORDED = 1 << 10; const ALL_IGNORED_RECORDED = 1 << 10;
const HAS_FALLBACK_EXEC = 1 << 11; const HAS_FALLBACK_EXEC = 1 << 11;
const FALLBACK_EXEC = 1 << 12; const FALLBACK_EXEC = 1 << 12;
const HAS_FALLBACK_SYMLINK = 1 << 13; const HAS_FALLBACK_SYMLINK = 1 << 13;
const FALLBACK_SYMLINK = 1 << 14; const FALLBACK_SYMLINK = 1 << 14;
const MTIME_SECOND_AMBIGUOUS = 1 << 15;
} }
} }
/// Duration since the Unix epoch /// Duration since the Unix epoch
#[derive(BytesCast, Copy, Clone)] #[derive(BytesCast, Copy, Clone)]
#[repr(C)] #[repr(C)]
struct PackedTruncatedTimestamp { struct PackedTruncatedTimestamp {
truncated_seconds: U32Be, truncated_seconds: U32Be,
&& !self.flags().contains(Flags::EXPECTED_STATE_IS_MODIFIED) && !self.flags().contains(Flags::EXPECTED_STATE_IS_MODIFIED)
{ {
Some((self.synthesize_unix_mode(), self.size.into())) Some((self.synthesize_unix_mode(), self.size.into()))
} else { } else {
None None
}; };
let mtime = if self.flags().contains(Flags::HAS_FILE_MTIME) let mtime = if self.flags().contains(Flags::HAS_FILE_MTIME)
&& !self.flags().contains(Flags::EXPECTED_STATE_IS_MODIFIED) && !self.flags().contains(Flags::EXPECTED_STATE_IS_MODIFIED)
// The current code is not able to do the more subtle comparison that the
// MTIME_SECOND_AMBIGUOUS requires. So we ignore the mtime
&& !self.flags().contains(Flags::MTIME_SECOND_AMBIGUOUS)
{ {
// TODO: replace this by `self.mtime.try_into()?` to use // TODO: replace this by `self.mtime.try_into()?` to use
// sub-second precision from the file. // sub-second precision from the file.
// We don’t do this yet because other parts of the code // We don’t do this yet because other parts of the code
// always set it to zero. // always set it to zero.
let mtime = TruncatedTimestamp::from_already_truncated( let mtime = TruncatedTimestamp::from_already_truncated(
self.mtime.truncated_seconds.get(), self.mtime.truncated_seconds.get(),
0, 0,