( )⚙ D6113 py3: convert to/from bytes/unicode for json.(dump|load)s in debugcallconduit

This is an archive of the discontinued Mercurial Phabricator instance.

py3: convert to/from bytes/unicode for json.(dump|load)s in debugcallconduit
ClosedPublic

Authored by Kwan on Mar 8 2019, 10:01 PM.

Diff Detail

Repository
rHG Mercurial
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Kwan created this revision.Mar 8 2019, 10:01 PM
yuja added a subscriber: yuja.Mar 9 2019, 8:30 PM
  • params = json.loads(ui.fin.read())
  • result = callconduit(repo, name, params)
  • s = json.dumps(result, sort_keys=True, indent=2, separators=(b',', b': '))
  • ui.write(b'%s\n' % s)

+ # json.(load|dump)s only returns/accepts unicode strings
+ params = pycompat.rapply(lambda x:
+ encoding.unitolocal(x) if isinstance(x, pycompat.unicode) else x,
+ json.loads(ui.fin.read())
+ )

Perhaps, the input data has to be converted to unicode. IIRC old Python 3
versions don't accept bytes.

+ result = pycompat.rapply(lambda x:
+ encoding.unifromlocal(x) if isinstance(x, bytes) else x,
+ callconduit(repo, name, params)
+ )
+ s = json.dumps(result, sort_keys=True, indent=2, separators=(u',', u': '))
+ ui.write(b'%s\n' % encoding.unitolocal(s))

Maybe we can use templatefilters.json() to dump internal (i.e. no unicode)
dict.

Kwan added a comment.Mar 14 2019, 11:18 AM
In D6113#88992, @yuja wrote:
-    params = json.loads(ui.fin.read())
-    result = callconduit(repo, name, params)
-    s = json.dumps(result, sort_keys=True, indent=2, separators=(b',', b': '))
-    ui.write(b'%s\n' % s)
+    # json.(load|dump)s only returns/accepts unicode strings
+    params = pycompat.rapply(lambda x:
+        encoding.unitolocal(x) if isinstance(x, pycompat.unicode) else x,
+        json.loads(ui.fin.read())
+    )

Perhaps, the input data has to be converted to unicode. IIRC old Python 3
versions don't accept bytes.

Ah, yeah you're right, it only started accepting bytes in 3.6.

In D6113#88992, @yuja wrote:
+    result = pycompat.rapply(lambda x:
+        encoding.unifromlocal(x) if isinstance(x, bytes) else x,
+        callconduit(repo, name, params)
+    )
+    s = json.dumps(result, sort_keys=True, indent=2, separators=(u',', u': '))
+    ui.write(b'%s\n' % encoding.unitolocal(s))

Maybe we can use templatefilters.json() to dump internal (i.e. no unicode)
dict.

Ah, we can indeed (thanks for the pointer), with one caveat: the formatting.
The existing output is nicely formatted for human readability, with each entry on a new line and indented appropriately. templatefilters.json() isn't capable of that.
Do we mind losing that? It's also tested for in test-phabricator.t. Compare before:

{
  "cursor": {
    "after": null,
    "before": null,
    "limit": 100,
    "order": null
  },
  "data": [],
  "maps": {},
  "query": {
    "queryKey": null
  }
}

After:

{"cursor": {"after": null, "before": null, "limit": 100, "order": null}, "data": [], "maps": {}, "query": {"queryKey": null}}
yuja added a comment.Mar 15 2019, 10:44 PM
Ah,  we can indeed (thanks for the pointer), with one caveat:  the formatting.
The existing output is nicely formatted for human readability, with each entry on a new line and indented appropriately.  `templatefilters.json()` isn't capable of that.
Do we mind losing that?  It's also tested for in test-phabricator.t. Compare before:

Good point. Let's not change the formatting as it is a debug command and
we'll probably want to read the output.

Kwan updated this revision to Diff 14523.Mar 16 2019, 7:54 AM

Make compatible with Py3.5, separate comments about json functions so they are more readable.

This revision was automatically updated to reflect the committed changes.