This is an archive of the discontinued Mercurial Phabricator instance.

fuzz: try and ensure fuzzer tests run against the right python-config
ClosedPublic

Authored by durin42 on Jan 13 2021, 3:44 PM.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

durin42 created this revision.Jan 13 2021, 3:44 PM
durin42 updated this revision to Diff 24822.Jan 13 2021, 6:41 PM
pulkit accepted this revision.Jan 14 2021, 8:07 AM
This revision is now accepted and ready to land.Jan 14 2021, 8:07 AM

This makes the test fail horribly for me. It starts like this:

--- /usr/local/google/home/martinvonz/hg/tests/test-fuzz-targets.t
+++ /usr/local/google/home/martinvonz/hg/tests/test-fuzz-targets.t.err
@@ -43,6 +43,1301 @@
 #if no-clang-libfuzzer clang-6.0
   $ CXX=clang++-6.0 havefuzz || exit 80
   $ $MAKE -s clean all CC=clang-6.0 CXX=clang++-6.0 PYTHON_CONFIG="$PYTHON_CONFIG"
+  clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
+  clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
+  clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
+  clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
+  clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
+  clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
+  clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
+  clang: warning: argument unused during compilation: '-specs=/usr/share/dpkg/no-pie-compile.specs' [-Wunused-command-line-argument]
+  /usr/bin/ld: pyutil.o: in function `contrib::initpy(char const*)':
+  /usr/local/google/home/martinvonz/hg/contrib/fuzz/pyutil.cc:53: undefined reference to `Py_SetPythonHome'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/pyutil.cc:57: undefined reference to `Py_InitializeEx'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/pyutil.cc:58: undefined reference to `PyImport_AddModule'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/pyutil.cc:59: undefined reference to `PyModule_GetDict'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/pyutil.cc:68: undefined reference to `PyDict_SetItemString'
+  /usr/bin/ld: /tmp/dirs-a2d89e.o: in function `LLVMFuzzerInitialize':
+  /usr/local/google/home/martinvonz/hg/contrib/fuzz/dirs.cc:17: undefined reference to `Py_CompileStringExFlags'
+  /usr/bin/ld: /tmp/dirs-a2d89e.o: in function `LLVMFuzzerTestOneInput':
+  /usr/local/google/home/martinvonz/hg/contrib/fuzz/dirs.cc:43: undefined reference to `PyBytes_FromStringAndSize'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/dirs.cc:44: undefined reference to `PyDict_New'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/dirs.cc:45: undefined reference to `PyDict_SetItemString'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/dirs.cc:46: undefined reference to `PyEval_EvalCode'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/dirs.cc:48: undefined reference to `PyErr_Print'
+  /usr/bin/ld: /tmp/dirs-a2d89e.o: in function `_Py_DECREF(char const*, int, _object*)':
+  /usr/include/python3.8/object.h:478: undefined reference to `_Py_Dealloc'
+  /usr/bin/ld: /usr/include/python3.8/object.h:478: undefined reference to `_Py_Dealloc'
+  /usr/bin/ld: /usr/include/python3.8/object.h:478: undefined reference to `_Py_Dealloc'
+  /usr/bin/ld: parsers-manifest.o: in function `manifest_module_init':
+  /usr/local/google/home/martinvonz/hg/contrib/fuzz/../../mercurial/cext/manifest.c:1004: undefined reference to `PyType_GenericNew'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/../../mercurial/cext/manifest.c:1005: undefined reference to `PyType_Ready'
+  /usr/bin/ld: /usr/local/google/home/martinvonz/hg/contrib/fuzz/../../mercurial/cext/manifest.c:1009: undefined reference to `PyModule_AddObject'
+  /usr/bin/ld: parsers-manifest.o: in function `_Py_DECREF':
+  /usr/include/python3.8/object.h:478: undefined reference to `_Py_Dealloc'
+  /usr/bin/ld: parsers-manifest.o: in function `lazymanifest_dealloc':
+  /usr/local/google/home/martinvonz/hg/contrib/fuzz/../../mercurial/cext/manifest.c:263: undefined reference to `PyObject_Free'
+  /usr/bin/ld: parsers-manifest.o: in function `lazymanifest_getkeysiter':
+  /usr/local/google/home/martinvonz/hg/contrib/fuzz/../../mercurial/cext/manifest.c:434: undefined reference to `_PyObject_New'
+  /usr/bin/ld: parsers-manifest.o: in function `lazymanifest_getkeysiter':
+  /usr/include/python3.8/object.h:(.text+0x102): undefined reference to `PyErr_NoMemory'
+  /usr/bin/ld: parsers-manifest.o: in function `_Py_DECREF':
+  /usr/include/python3.8/object.h:478: undefined reference to `_Py_Dealloc'
+  /usr/bin/ld: /usr/include/python3.8/object.h:(.text+0x113): undefined reference to `PyErr_NoMemory'
+  /usr/bin/ld: parsers-manifest.o: in function `lazymanifest_init':

There's probably a hundred lines more. Should we drop this for now or is my system unique? How can I help you debug it?

That’s super surprising to me, because it hasn’t passed for me _for months_ and our work systems should be basically identical.

We shouldn’t back this out IMO because we haven’t run the fuzzer against py2 in most of year, so this hasn’t been testing what’s in oss-fuzz in a while. But maybe we need to figure out a way to guard it more for now?