This is an archive of the discontinued Mercurial Phabricator instance.

Differential D3676

fuzzutil: make it possible to use absl when C++17 isn't supported
ClosedPublic

Authored by durin42 on May 30 2018, 5:28 PM.

Details

Reviewers
None
Group Reviewers
hg-reviewers
Commits
rHG36d55f90e2a3: fuzzutil: make it possible to use absl when C++17 isn't supported
Summary

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

durin42 created this revision.May 30 2018, 5:28 PM
indygreg added a subscriber: indygreg.May 30 2018, 8:54 PM

I'm assuming https://abseil.io/ is part of the standard environment in oss-fuzz. Otherwise, we may want to vendor it or provide a script to download it or something.

durin42 added a comment.May 30 2018, 8:57 PM
In D3676#57722, @indygreg wrote:

I'm assuming https://abseil.io/ is part of the standard environment in oss-fuzz. Otherwise, we may want to vendor it or provide a script to download it or something.

Actually, it's not needed on oss-fuzz because we have a C++17 compiler there. This just leaves the hooks in place so that it's easy-ish to build the fuzzers when you don't have a C++17 compiler (which I don't when building out of Piper, which gives me some insane options for running the fuzzer.)

I can live without this patch, but I'd rather we have it in place so that it's easy to drop a copy of abseil on a machine if needed when doing fuzzer work.

Closed by commit rHG36d55f90e2a3: fuzzutil: make it possible to use absl when C++17 isn't supported (authored by durin42). · Explain WhyMay 30 2018, 9:01 PM
This revision was automatically updated to reflect the committed changes.

Revision Contents
Changeset List

PathPackages
Mcontrib/fuzz/Makefile (13 lines)
Mcontrib/fuzz/fuzzutil.h (27 lines)
Mcontrib/fuzz/fuzzutil.cc (4 lines)

Diff 8937

contrib/fuzz/Makefile

fuzzutil.o: fuzzutil.cc fuzzutil.h fuzzutil.o: fuzzutil.cc fuzzutil.h
$$CXX $$CXXFLAGS -g -O1 -fsanitize=fuzzer-no-link,address \ $$CXX $$CXXFLAGS -g -O1 -fsanitize=fuzzer-no-link,address \
-std=c++17 \ -std=c++17 \
-I../../mercurial -c -o fuzzutil.o fuzzutil.cc -I../../mercurial -c -o fuzzutil.o fuzzutil.cc
fuzzutil-oss-fuzz.o: fuzzutil.cc fuzzutil.h
$$CXX $$CXXFLAGS -std=c++17 \
-I../../mercurial -c -o fuzzutil-oss-fuzz.o fuzzutil.cc
bdiff.o: ../../mercurial/bdiff.c bdiff.o: ../../mercurial/bdiff.c
$$CC $$CFLAGS -fsanitize=fuzzer-no-link,address -c -o bdiff.o \ $$CC $$CFLAGS -fsanitize=fuzzer-no-link,address -c -o bdiff.o \
../../mercurial/bdiff.c ../../mercurial/bdiff.c
bdiff: bdiff.cc bdiff.o fuzzutil.o bdiff: bdiff.cc bdiff.o fuzzutil.o
$$CXX $$CXXFLAGS -DHG_FUZZER_INCLUDE_MAIN=1 -g -O1 -fsanitize=fuzzer-no-link,address \ $$CXX $$CXXFLAGS -DHG_FUZZER_INCLUDE_MAIN=1 -g -O1 -fsanitize=fuzzer-no-link,address \
-std=c++17 \ -std=c++17 \
-I../../mercurial bdiff.cc bdiff.o fuzzutil.o -o bdiff -I../../mercurial bdiff.cc bdiff.o fuzzutil.o -o bdiff
bdiff-oss-fuzz.o: ../../mercurial/bdiff.c bdiff-oss-fuzz.o: ../../mercurial/bdiff.c
$$CC $$CFLAGS -c -o bdiff-oss-fuzz.o ../../mercurial/bdiff.c $$CC $$CFLAGS -c -o bdiff-oss-fuzz.o ../../mercurial/bdiff.c
bdiff_fuzzer: bdiff.cc bdiff-oss-fuzz.o fuzzutil.o bdiff_fuzzer: bdiff.cc bdiff-oss-fuzz.o fuzzutil-oss-fuzz.o
$$CXX $$CXXFLAGS -std=c++17 -I../../mercurial bdiff.cc \ $$CXX $$CXXFLAGS -std=c++17 -I../../mercurial bdiff.cc \
bdiff-oss-fuzz.o fuzzutil.o -lFuzzingEngine -o $$OUT/bdiff_fuzzer bdiff-oss-fuzz.o fuzzutil-oss-fuzz.o -lFuzzingEngine -o \
$$OUT/bdiff_fuzzer
x%.o: ../../mercurial/thirdparty/xdiff/x%.c ../../mercurial/thirdparty/xdiff/*.h x%.o: ../../mercurial/thirdparty/xdiff/x%.c ../../mercurial/thirdparty/xdiff/*.h
$$CC -g -O1 -fsanitize=fuzzer-no-link,address -c \ $$CC -g -O1 -fsanitize=fuzzer-no-link,address -c \
-o $@ \ -o $@ \
$< $<
xdiff: xdiff.cc xdiffi.o xprepare.o xutils.o fuzzutil.o xdiff: xdiff.cc xdiffi.o xprepare.o xutils.o fuzzutil.o
$$CXX $$CXXFLAGS -DHG_FUZZER_INCLUDE_MAIN=1 -g -O1 -fsanitize=fuzzer-no-link,address \ $$CXX $$CXXFLAGS -DHG_FUZZER_INCLUDE_MAIN=1 -g -O1 -fsanitize=fuzzer-no-link,address \
-I../../mercurial xdiff.cc \ -I../../mercurial xdiff.cc \
xdiffi.o xprepare.o xutils.o fuzzutil.o -o xdiff xdiffi.o xprepare.o xutils.o fuzzutil.o -o xdiff
fuzz-x%.o: ../../mercurial/thirdparty/xdiff/x%.c ../../mercurial/thirdparty/xdiff/*.h fuzz-x%.o: ../../mercurial/thirdparty/xdiff/x%.c ../../mercurial/thirdparty/xdiff/*.h
$$CC $$CFLAGS -c \ $$CC $$CFLAGS -c \
-o $@ \ -o $@ \
$< $<
xdiff_fuzzer: xdiff.cc fuzz-xdiffi.o fuzz-xprepare.o fuzz-xutils.o fuzzutil.o xdiff_fuzzer: xdiff.cc fuzz-xdiffi.o fuzz-xprepare.o fuzz-xutils.o fuzzutil-oss-fuzz.o
$$CXX $$CXXFLAGS -std=c++17 -I../../mercurial xdiff.cc \ $$CXX $$CXXFLAGS -std=c++17 -I../../mercurial xdiff.cc \
fuzz-xdiffi.o fuzz-xprepare.o fuzz-xutils.o fuzzutil.o \ fuzz-xdiffi.o fuzz-xprepare.o fuzz-xutils.o fuzzutil-oss-fuzz.o \
-lFuzzingEngine -o $$OUT/xdiff_fuzzer -lFuzzingEngine -o $$OUT/xdiff_fuzzer
all: bdiff xdiff all: bdiff xdiff
oss-fuzz: bdiff_fuzzer xdiff_fuzzer oss-fuzz: bdiff_fuzzer xdiff_fuzzer
.PHONY: all oss-fuzz .PHONY: all oss-fuzz

contrib/fuzz/fuzzutil.h

#ifndef CONTRIB_FUZZ_FUZZUTIL_H #ifndef CONTRIB_FUZZ_FUZZUTIL_H
#define CONTRIB_FUZZ_FUZZUTIL_H #define CONTRIB_FUZZ_FUZZUTIL_H
#include <iostream> #include <iostream>
#include <memory> #include <memory>
#include <optional>
#include <stdint.h> #include <stdint.h>
/* Try and use std::optional, but failing that assume we'll have a
* workable https://abseil.io/ install on the include path to get
* their backport of std::optional. */
#ifdef __has_include
#if __has_include(<optional>) && __cplusplus >= 201703L
#include <optional>
#define CONTRIB_FUZZ_HAVE_STD_OPTIONAL
#endif
#endif
#ifdef CONTRIB_FUZZ_HAVE_STD_OPTIONAL
namespace contrib
{
using std::nullopt;
using std::optional;
} /* namespace contrib */
#else
#include "third_party/absl/types/optional.h"
namespace contrib
{
using absl::nullopt;
using absl::optional;
} /* namespace contrib */
#endif
/* set DEBUG to 1 for a few debugging prints, or 2 for a lot */ /* set DEBUG to 1 for a few debugging prints, or 2 for a lot */
#define DEBUG 0 #define DEBUG 0
#define LOG(level) \ #define LOG(level) \
if (level <= DEBUG) \ if (level <= DEBUG) \
std::cout std::cout
struct two_inputs { struct two_inputs {
std::unique_ptr<char[]> right; std::unique_ptr<char[]> right;
size_t right_size; size_t right_size;
std::unique_ptr<char[]> left; std::unique_ptr<char[]> left;
size_t left_size; size_t left_size;
}; };
/* Split a non-zero-length input into two inputs. */ /* Split a non-zero-length input into two inputs. */
std::optional<two_inputs> SplitInputs(const uint8_t *Data, size_t Size); contrib::optional<two_inputs> SplitInputs(const uint8_t *Data, size_t Size);
#endif /* CONTRIB_FUZZ_FUZZUTIL_H */ #endif /* CONTRIB_FUZZ_FUZZUTIL_H */

contrib/fuzz/fuzzutil.cc

#include "fuzzutil.h" #include "fuzzutil.h"
#include <utility> #include <utility>
std::optional<two_inputs> SplitInputs(const uint8_t *Data, size_t Size) contrib::optional<two_inputs> SplitInputs(const uint8_t *Data, size_t Size)
{ {
if (!Size) { if (!Size) {
return std::nullopt; return contrib::nullopt;
} }
// figure out a random point in [0, Size] to split our input. // figure out a random point in [0, Size] to split our input.
size_t left_size = (Data[0] / 255.0) * (Size - 1); size_t left_size = (Data[0] / 255.0) * (Size - 1);
// Copy inputs to new allocations so if bdiff over-reads // Copy inputs to new allocations so if bdiff over-reads
// AddressSanitizer can detect it. // AddressSanitizer can detect it.
std::unique_ptr<char[]> left(new char[left_size]); std::unique_ptr<char[]> left(new char[left_size]);
memcpy(left.get(), Data + 1, left_size); memcpy(left.get(), Data + 1, left_size);