This is an archive of the discontinued Mercurial Phabricator instance.

Differential D11747

internals: typo pass on the dirstate-v2 help file
ClosedPublic

Authored by Alphare on Nov 10 2021, 5:46 AM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG6248607381f2: internals: typo pass on the dirstate-v2 help file

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Alphare created this revision.Nov 10 2021, 5:46 AM

Pending CI refresh

Alphare added a commit: rHG6248607381f2: internals: typo pass on the dirstate-v2 help file.Nov 10 2021, 2:58 PM
This revision was not accepted when it landed; it landed in state Needs Review.
Closed by commit rHG6248607381f2: internals: typo pass on the dirstate-v2 help file (authored by Alphare). · Explain Why
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

Diff 31055

mercurial/helptext/internals/dirstate-v2.txt

The *dirstate* is what Mercurial uses internally to track The *dirstate* is what Mercurial uses internally to track
the state of files in the working directory, the state of files in the working directory,
such as set by commands like `hg add` and `hg rm`. such as set by commands like `hg add` and `hg rm`.
It also contains some cached data that help make `hg status` faster. It also contains some cached data that help make `hg status` faster.
The name refers both to `.hg/dirstate` on the filesystem The name refers both to `.hg/dirstate` on the filesystem
and the corresponding data structure in memory while a Mercurial process and the corresponding data structure in memory while a Mercurial process
is running. is running.
The original file format, retroactively dubbed `dirstate-v1`, The original file format, retroactively dubbed `dirstate-v1`,
is described at https://www.mercurial-scm.org/wiki/DirState. is described at https://www.mercurial-scm.org/wiki/DirState.
It is made of a flat sequence of unordered variable-size entries, It is made of a flat sequence of unordered variable-size entries,
so accessing any information in it requires parsing all of it. so accessing any information in it requires parsing all of it.
Similarly, saving changes requires rewriting the entire file. Similarly, saving changes requires rewriting the entire file.
The newer `dirsate-v2` file format is designed to fix these limitations The newer `dirstate-v2` file format is designed to fix these limitations
and make `hg status` faster. and make `hg status` faster.
User guide User guide
========== ==========
Compatibility Compatibility
------------- -------------
The file format is experimental and may still change. The file format is experimental and may still change.
Different versions of Mercurial may not be compatible with each other Different versions of Mercurial may not be compatible with each other
when working on a local repository that uses this format. when working on a local repository that uses this format.
When using an incompatible version with the experimental format, When using an incompatible version with the experimental format,
anything can happen including data corruption. anything can happen including data corruption.
Since the dirstate is entirely local and not relevant to the wire protocol, Since the dirstate is entirely local and not relevant to the wire protocol,
`dirstate-v2` does not affect compatibility with remote Mercurial versions. `dirstate-v2` does not affect compatibility with remote Mercurial versions.
When `share-safe` is enabled, different repositories sharing the same store When `share-safe` is enabled, different repositories sharing the same store
can use different dirstate formats. can use different dirstate formats.
Enabling `dirsate-v2` for new local repositories Enabling `dirstate-v2` for new local repositories
------------------------------------------------ ------------------------------------------------
When creating a new local repository such as with `hg init` or `hg clone`, When creating a new local repository such as with `hg init` or `hg clone`,
the `exp-rc-dirstate-v2` boolean in the `format` configuration section the `exp-rc-dirstate-v2` boolean in the `format` configuration section
controls whether to use this file format. controls whether to use this file format.
This is disabled by default as of this writing. This is disabled by default as of this writing.
To enable it for a single repository, run for example:: To enable it for a single repository, run for example::
$ hg init my-project --config format.exp-rc-dirstate-v2=1 $ hg init my-project --config format.exp-rc-dirstate-v2=1
Checking the format of an existing local repsitory Checking the format of an existing local repository
-------------------------------------------------- --------------------------------------------------
The `debugformat` commands prints information about The `debugformat` commands prints information about
which of multiple optional formats are used in the current repository, which of multiple optional formats are used in the current repository,
including `dirstate-v2`:: including `dirstate-v2`::
$ hg debugformat $ hg debugformat
format-variant repo format-variant repo
================================== ==================================
Requirements file Requirements file
----------------- -----------------
The `.hg/requires` file indicates which of various optional file formats The `.hg/requires` file indicates which of various optional file formats
are used by a given repository. are used by a given repository.
Mercurial aborts when seeing a requirement it does not know about, Mercurial aborts when seeing a requirement it does not know about,
which avoids older version accidentally messing up a respository which avoids older version accidentally messing up a repository
that uses a format that was introduced later. that uses a format that was introduced later.
For versions that do support a format, the presence or absence of For versions that do support a format, the presence or absence of
the corresponding requirement indicates whether to use that format. the corresponding requirement indicates whether to use that format.
When the file contains a `dirstate-v2` line, When the file contains a `dirstate-v2` line,
the `dirstate-v2` format is used. the `dirstate-v2` format is used.
With no such line `dirstate-v1` is used. With no such line `dirstate-v1` is used.
High level description High level description
---------------------- ----------------------
Whereas `dirstate-v1` uses a single `.hg/disrtate` file, Whereas `dirstate-v1` uses a single `.hg/dirstate` file,
in `dirstate-v2` that file is a "docket" file in `dirstate-v2` that file is a "docket" file
that only contains some metadata that only contains some metadata
and points to separate data file named `.hg/dirstate.{ID}`, and points to separate data file named `.hg/dirstate.{ID}`,
where `{ID}` is a random identifier. where `{ID}` is a random identifier.
This separation allows making data files append-only This separation allows making data files append-only
and therefore safer to memory-map. and therefore safer to memory-map.
Creating a new data file (occasionally to clean up unused data) Creating a new data file (occasionally to clean up unused data)
Its separation in this documentation from the rest of the docket Its separation in this documentation from the rest of the docket
reflects a detail of the current implementation. reflects a detail of the current implementation.
Since tree metadata is also made of fields at fixed offsets, those could Since tree metadata is also made of fields at fixed offsets, those could
be inlined here by adding 76 bytes to each offset. be inlined here by adding 76 bytes to each offset.
* Offset 120: * Offset 120:
The used size of the data file, as a 32-bit big-endian integer. The used size of the data file, as a 32-bit big-endian integer.
The actual size of the data file may be larger The actual size of the data file may be larger
(if another Mercurial processis in appending to it (if another Mercurial process is appending to it
but has not updated the docket yet). but has not updated the docket yet).
That extra data must be ignored. That extra data must be ignored.
* Offset 124: * Offset 124:
The length of the data file identifier, as a 8-bit integer. The length of the data file identifier, as a 8-bit integer.
* Offset 125: * Offset 125:
The data file identifier. The data file identifier.
The data file contains two types of data: paths and nodes. The data file contains two types of data: paths and nodes.
Paths and nodes can be organized in any order in the file, except that sibling Paths and nodes can be organized in any order in the file, except that sibling
nodes must be next to each other and sorted by their path. nodes must be next to each other and sorted by their path.
Contiguity lets the parent refer to them all Contiguity lets the parent refer to them all
by their count and a single pseudo-pointer, by their count and a single pseudo-pointer,
instead of storing one pseudo-pointer per child node. instead of storing one pseudo-pointer per child node.
Sorting allows using binary seach to find a child node with a given name Sorting allows using binary search to find a child node with a given name
in `O(log(n))` byte sequence comparisons. in `O(log(n))` byte sequence comparisons.
The current implemention writes paths and child node before a given node The current implementation writes paths and child node before a given node
for ease of figuring out the value of pseudo-pointers by the time the are to be for ease of figuring out the value of pseudo-pointers by the time the are to be
written, but this is not an obligation and readers must not rely on it. written, but this is not an obligation and readers must not rely on it.
A path is stored as a byte string anywhere in the file, without delimiter. A path is stored as a byte string anywhere in the file, without delimiter.
It is refered to by one or more node by a pseudo-pointer to its start, and its It is referred to by one or more node by a pseudo-pointer to its start, and its
length in bytes. Since there is no delimiter, length in bytes. Since there is no delimiter,
when a path is a substring of another the same bytes could be reused, when a path is a substring of another the same bytes could be reused,
although the implementation does not exploit this as of this writing. although the implementation does not exploit this as of this writing.
A node is stored on 43 bytes with components at fixed offsets. Paths and A node is stored on 43 bytes with components at fixed offsets. Paths and
child nodes relevant to a node are stored externally and referenced though child nodes relevant to a node are stored externally and referenced though
pseudo-pointers. pseudo-pointers.
When `mtime` is used, this is number of seconds since the Unix epoch When `mtime` is used, this is number of seconds since the Unix epoch
truncated to its lower 31 bits. truncated to its lower 31 bits.
* Offset 40: * Offset 40:
The nanoseconds component of an `mtime` field described below, The nanoseconds component of an `mtime` field described below,
as a 32-bit integer. as a 32-bit integer.
When `mtime` is used, When `mtime` is used,
this is the number of nanoseconds since `mtime.seconds`, this is the number of nanoseconds since `mtime.seconds`,
always stritctly less than one billion. always strictly less than one billion.
This may be zero if more precision is not available. This may be zero if more precision is not available.
(This can happen because of limitations in any of Mercurial, Python, (This can happen because of limitations in any of Mercurial, Python,
libc, the operating system, …) libc, the operating system, …)
When comparing two mtimes and either has this component set to zero, When comparing two mtimes and either has this component set to zero,
the sub-second precision of both should be ignored. the sub-second precision of both should be ignored.
False positives when checking mtime equality due to clock resolution False positives when checking mtime equality due to clock resolution
It can also be set for node where `DIRECTORY` is set. It can also be set for node where `DIRECTORY` is set.
See `DIRECTORY` documentation for details. See `DIRECTORY` documentation for details.
`DIRECTORY` `DIRECTORY`
When set, this entry will match a directory that exists or existed on the When set, this entry will match a directory that exists or existed on the
file system. file system.
* When `HAS_MTIME` is set a directory has been seen on the file system and * When `HAS_MTIME` is set a directory has been seen on the file system and
`mtime` matches its last modificiation time. However, `HAS_MTIME` not being set `mtime` matches its last modification time. However, `HAS_MTIME` not
does not indicate the lack of directory on the file system. being set does not indicate the lack of directory on the file system.
* When not tracked anywhere, this node does not represent an ignored or * When not tracked anywhere, this node does not represent an ignored or
unknown file on disk. unknown file on disk.
If `HAS_MTIME` is set If `HAS_MTIME` is set
and `mtime` matches the last modification time of the directory on disk, and `mtime` matches the last modification time of the directory on disk,
the directory is unchanged the directory is unchanged
and we can skip calling `std::fs::read_dir` again for this directory, and we can skip calling `std::fs::read_dir` again for this directory,
status) have been recorded and the `mtime` associated with status) have been recorded and the `mtime` associated with
`DIRECTORY` can be used for optimization even when "unknown" file `DIRECTORY` can be used for optimization even when "unknown" file
are listed. are listed.
Note that the amount recorded "unknown" children can still be zero if None Note that the amount recorded "unknown" children can still be zero if None
where present. where present.
Also note that having this flag unset does not imply that no "unknown" Also note that having this flag unset does not imply that no "unknown"
children have been recorded. Some might be present, but there is no garantee children have been recorded. Some might be present, but there is
that is will be all of them. no guarantee that is will be all of them.
`ALL_IGNORED_RECORDED` `ALL_IGNORED_RECORDED`
If set, all "ignored" children existing on disk (at the time of the last If set, all "ignored" children existing on disk (at the time of the last
status) have been recorded and the `mtime` associated with status) have been recorded and the `mtime` associated with
`DIRECTORY` can be used for optimization even when "ignored" file `DIRECTORY` can be used for optimization even when "ignored" file
are listed. are listed.
Note that the amount recorded "ignored" children can still be zero if None Note that the amount recorded "ignored" children can still be zero if None
where present. where present.
Also note that having this flag unset does not imply that no "ignored" Also note that having this flag unset does not imply that no "ignored"
children have been recorded. Some might be present, but there is no garantee children have been recorded. Some might be present, but there is
that is will be all of them. no guarantee that is will be all of them.
`HAS_FALLBACK_EXEC` `HAS_FALLBACK_EXEC`
If this flag is set, the entry carries "fallback" information for the If this flag is set, the entry carries "fallback" information for the
executable bit in the `FALLBACK_EXEC` flag. executable bit in the `FALLBACK_EXEC` flag.
Fallback information can be stored in the dirstate to keep track of Fallback information can be stored in the dirstate to keep track of
filesystem attribute tracked by Mercurial when the underlying file filesystem attribute tracked by Mercurial when the underlying file
system or operating system does not support that property, (e.g. system or operating system does not support that property, (e.g.
this entry should be considered a symlink if that information cannot be this entry should be considered a symlink if that information cannot be
extracted from the file system. If unset it should be considered a normal extracted from the file system. If unset it should be considered a normal
file instead. file instead.
`MTIME_SECOND_AMBIGUOUS` `MTIME_SECOND_AMBIGUOUS`
This flag is relevant only when `HAS_FILE_MTIME` is set. When set, the This flag is relevant only when `HAS_FILE_MTIME` is set. When set, the
`mtime` stored in the entry is only valid for comparison with timestamps `mtime` stored in the entry is only valid for comparison with timestamps
that have nanosecond information. If available timestamp does not carries that have nanosecond information. If available timestamp does not carries
nanosecond information, the `mtime` should be ignored and no optimisation nanosecond information, the `mtime` should be ignored and no optimization
can be applied. can be applied.