[Library] cJson.ahk (version 0.4.1 pre-release)

[kczx3]

I implemented it in V2, the test of 'movies.json', load 60~80ms and dump 80ms.

Is your v2 version available on GitHub?

[thqby]

I implemented it in V2, the test of 'movies.json', load 60~80ms and dump 80ms.

Is your v2 version available on GitHub?

Json is added to AHK_H. object dump can more faster because it takes some time for enumerators to copy strings.
https://github.com/thqby/AutoHotkey_H/blob/alpha/source/script_object.cpp#L4791

[burque505]

@thqby, just compiled your version on Win10, VS 2019 (MT). That was the most painless compiling ever of AHK_H. Thank you.
Regards,
burque505

[geek]

I implemented it in V2, the test of 'movies.json', load 60~80ms and dump 80ms.

Implemented your own JSON library, or did something relating to my code specifically?

Absolute time measurements are hard to get useful information out of--the interest comes from comparison, which can't be done directly cross-machine due to varying processor characteristics. Could you also share some benchmarks of mine or coco's libraries running on your PC to give a point of reference?

[thqby]

I implemented it in V2, the test of 'movies.json', load 60~80ms and dump 80ms.

Implemented your own JSON library, or did something relating to my code specifically?

Absolute time measurements are hard to get useful information out of--the interest comes from comparison, which can't be done directly cross-machine due to varying processor characteristics. Could you also share some benchmarks of mine or coco's libraries running on your PC to give a point of reference?

tested by cjson v0.2.2
load 120+ms and dump 60+ms, this may be related to the difference in object structure between the two.
I also added V8's double-convertion to solve the precision retention problem for floating point numbers.

[geek]

Not picking at you, just curious, was that with pre-initialization of cJson, including initialization, or including initialization but then averaged over multiple loads where the subsequent loads would not have re-initialized the library?

[kczx3]

I look forward to a AHK v2 compatible version of this as having a fast and accurate way to parse JSON into AHK objects/Maps is very necessary for getting WebView2 to work well with AHK.

[geek]

Version 0.3.0 has been released, adding 32-bit compatibility.

[guest3456]

good to see another project using Yunit

[2ndprotocol]

Hi. I get the following pop up when attempting to parse a file held in memory. It is quite large although I have tried with a short string as below

Code: Select all

testtext = {"2":[[1211328000000,188],[1629504000000,349,59800000],[1629590400000,351]]}
obj := cJson.Loads(testtext) 

image.png (28.33 KiB) Viewed 5040 times

Is this my lack of understanding or is it a bug?

[geek]

You are getting this warning because you have #Warn set in your code. It's warning you that the variable is being used before being assigned a value, which is irrelevant in this case because that DllCall is supposed to be setting that variable's initial value.

You can make the way warning go away by adding DecompressedSize := 0 on the line above that. There will be two places that cause that warning when you have #Warn set, so make sure to do that to both of them.

[geek]

cJson.ahk 0.4.0 has been released:

• Change the library interface to closer match Coco's JSON library
• Add preliminary support for pretty-printing when dumping

[20170201225639]

This is so insanely fast...

The largest json file on my drive is a 25M backup of an sqlite database and load and dump is 0.34 and 0.28 second respectively.

I noticed that cJson seems to escape a lot of unicode chars (not just the ones outside the Basic Multilingual Plane).
e.g
this:

Code: Select all

παράγραφος

becomes

Code: Select all

"\u03C0\u03B1\u03C1\u03AC\u03B3\u03C1\u03B1\u03C6\u03BF\u03C2"

But JSON standards don't seem to demand escaping ("the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)" https://tools.ietf.org/id/draft-ietf-json-rfc4627bis-09.html#rfc.section.8.1 , https://stackoverflow.com/questions/12271547/shouldnt-json-stringify-escape-unicode-characters). Would this behavior be configurable at some point? It could improve readability of json files for native speakers of languages other than english. I imagine it could also improve performance somewhat for unicode heavy cases.

[geek]

This is so insanely fast...

The largest json file on my drive is a 25M backup of an sqlite database and load and dump is 0.34 and 0.28 second respectively.

I noticed that cJson seems to escape a lot of unicode chars (not just the ones outside the Basic Multilingual Plane).
e.g
this:

Code: Select all

παράγραφος

becomes

Code: Select all

"\u03C0\u03B1\u03C1\u03AC\u03B3\u03C1\u03B1\u03C6\u03BF\u03C2"

But JSON standards don't seem to demand escaping ("the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)" https://tools.ietf.org/id/draft-ietf-json-rfc4627bis-09.html#rfc.section.8.1 , https://stackoverflow.com/questions/12271547/shouldnt-json-stringify-escape-unicode-characters). Would this behavior be configurable at some point? It could improve readability of json files for native speakers of languages other than english. I imagine it could also improve performance somewhat for unicode heavy cases.

They don't demand escaping, but they also permit escaping all character codes if so desired. I see your point, though I'm not sure the ideal way to implement it--Unicode encompasses a lot of characters, and whether a character is printable or non-printable isn't readily available information.

Would you be interested in:
1. Specifying custom character ranges as printable
2. Specifying individual characters as printable
3. Specifying that all unicode characters should be printed
4. I figure out what defaults JavaScript or Python use and use those
5. Something else entirely?

[20170201225639]

This is so insanely fast...

The largest json file on my drive is a 25M backup of an sqlite database and load and dump is 0.34 and 0.28 second respectively.

I noticed that cJson seems to escape a lot of unicode chars (not just the ones outside the Basic Multilingual Plane).
e.g
this:

Code: Select all

παράγραφος

becomes

Code: Select all

"\u03C0\u03B1\u03C1\u03AC\u03B3\u03C1\u03B1\u03C6\u03BF\u03C2"

But JSON standards don't seem to demand escaping ("the characters that must be escaped: quotation mark, reverse solidus, and the control characters (U+0000 through U+001F)" https://tools.ietf.org/id/draft-ietf-json-rfc4627bis-09.html#rfc.section.8.1 , https://stackoverflow.com/questions/12271547/shouldnt-json-stringify-escape-unicode-characters). Would this behavior be configurable at some point? It could improve readability of json files for native speakers of languages other than english. I imagine it could also improve performance somewhat for unicode heavy cases.

They don't demand escaping, but they also permit escaping all character codes if so desired. I see your point, though I'm not sure the ideal way to implement it--Unicode encompasses a lot of characters, and whether a character is printable or non-printable isn't readily available information.

Would you be interested in:
1. Specifying custom character ranges as printable
2. Specifying individual characters as printable
3. Specifying that all unicode characters should be printed
4. I figure out what defaults JavaScript or Python use and use those
5. Something else entirely?

I think, personally, I would be most interested in something like you option #3. Basically: an option to escape as little as possible while remaining valid JSON. My reasons:
1. It would be the simplest (I assume) to implement, and would save you the trouble of deciding on "an ideal way" to select a subset of chars (among those not-required-but-permitted-to-be-escaped) to be escaped
2. If I'm right that escaping is one source of overhead, it would expose a version of cJson that's maximally performant (which I take to be the whole point of going to c in the first place!). Which could make a practical difference for people like me, who frequently store/retrieve entire books in foreign languages in JSON. And I think the load/dump performance metrics on this could also be more interesting in itself.

[geek]

I think, personally, I would be most interested in something like you option #3. Basically: an option to escape as little as possible while remaining valid JSON.

I've added this functionality to the 0.4.1 release. You can enable it with JSON.EscapeUnicode := False. Please give it a try and let me know if I broke it, or if things are working as expected . Thanks!

[20170201225639]

I think, personally, I would be most interested in something like you option #3. Basically: an option to escape as little as possible while remaining valid JSON.

I've added this functionality to the 0.4.1 release. You can enable it with JSON.EscapeUnicode := False. Please give it a try and let me know if I broke it, or if things are working as expected . Thanks!

Oh, wow, thanks so much for this awesomeness, geek!

I will be sure to report any issue I encounter.

(And, jeez, adding this functionality involves a lot more moving parts than I imagined!

[geek]

(And, jeez, adding this functionality involves a lot more moving parts than I imagined!

Lucky for me, most of those changes are automatically generated by the MCL compile script. In terms of changes I wrote by hand for the update, those are found on this commit: https://github.com/G33kDude/cJson.ahk/commit/9b01c6e384e4855637cf17893b3c4feefb94afe7

[tuzi]

@geek

here is a bug.

the key should be char, but it shows CHAR.

Code: Select all

a=
(
{
	"char": {
		"*LPSTR": 0,
		"*PCHAR": 0,
		"*PSTR": 0,
		"CCHAR": 0,
		"CHAR": 0
	}
}
)

aa:=json.load(a)

for k, v in aa
	MsgBox, % k		; should be char, but it shows CHAR

ExitApp

#Include <json>

[geek]

This appears to be a bug with AHK, please file a report over in the bugs forum:

Code: Select all

js =
(
<script>
function test(Object) {
	var outer = Object();
	var inner = Object();
	inner["TEST"] = 1;
	outer["test"] = inner;
	return outer;
}
</script>
)

doc := ComObjCreate("htmlfile")
doc.Write(js)
doc.Close()

outer := doc.parentWindow.test(Func("Object"))

outer._NewEnum().Next(k1, inner)
inner._NewEnum().Next(k2, v2)
MsgBox, {"%k1%": {"%k2%": "%v2%"}} ; shows {"TEST": {"TEST": "1"}}

Edit: now being tracked at viewtopic.php?f=14&t=100622