py3: use fsencode for vcr recording paths and strings for custom_patches args

Authored by Kwan.

Description

py3: use fsencode for vcr recording paths and strings for custom_patches args

This fixes phabricator.py's vcrcommand under py3

Differential Revision: https://phab.mercurial-scm.org/D6102