JSON library, write by cpp

[thqby]

This library is compatible with AutoHotkey v2_L, is similar to geek's cJson.

Native.ahk
example

Code: Select all

#Include Native.ahk

js := Native.LoadModule('ahk-json\' (A_PtrSize * 8) 'bit\ahk-json.dll')['JSON']
obj1 := js.parse('[null,true,false,3123,"5345.432",{"ke":[42.4234]}]')
obj2 := JSON.parse('[null,true,false,3123,"5345.432",{"ke":[42.4234]}]')

str1 := JSON.stringify(obj1)
str2 := JSON.stringify(obj2)
MsgBox str1 '`n' str2 '`n' JSON.stringify(obj1, 4)

class JSON {
    static __New() {
        Native.LoadModule('ahk-json\' (A_PtrSize * 8) 'bit\ahk-json.dll', ['JSON'])
        this.DefineProp('true', {value: 1})
        this.DefineProp('false', {value: 0})
        this.DefineProp('null', {value: ''})
    }
    static parse(str) => 1
    static stringify(obj, space := 0) => 1
}

Updated at 2022/11/16
Performance testing, parse 32mb json file then dump it.
cpp-json
parse: 828ms stringify: 532ms

viewtopic.php?f=83&t=70235
HotkeyIt's Yaml and JSON parser
https://github.com/thqby/ahk2_lib/blob/master/JSON.ahk
Modified version
parse: 6172ms stringify: 9609ms

viewtopic.php?f=83&t=74799&hilit=json
TheArkive's JSON Serializer
parse: 11328ms stringify: 10766ms

[haichen]

Hi thqby, thank you for your library!
Could you please add Native.ahk?

[thqby]

@haichen
The link has been added. I sent it a few days ago.

[geek]

Interesting work, could you please provide the source of the DLL so I could check it out?

[thqby]

@geek
The source code has been uploaded.

[TheArkive]

@thqby

I've been hoping this would get done for AHK v2. Thank you! This is amazing.

[RaptorX]

Hello!

Sadly the function returns an "invalid arg type" when I pass this json file. Any reasons for this?

[thqby]

I have tested it and found no errors.

[RaptorX]

I tried a fileread and passing the string variable to your function and then got the error.

Can you confirm how did you perform your test?

[thqby]

image.png (134.97 KiB) Viewed 5651 times

[tranht17]

@RaptorX
I see your json data format is not correct.
Go here to validate your data first
https://jsoneditoronline.org

[RaptorX]

@RaptorX
I see your json data format is not correct.
Go here to validate your data first
https://jsoneditoronline.org

Interestingly the JXON library loads the file using jxon_load()

I will try with another JSON file.
I think The function should return an error explaining that the data is not formatted properly instead of saying "Invalid Arg".

[TheArkive]

I tried loading the large file and also managed to get an error.

Code: Select all

#Include Native.ahk

js := Native.LoadModule('ahk-json\' (A_PtrSize * 8) 'bit\ahk-json.dll')['JSON']

txt := FileRead(A_Desktop '\big_json_test.txt')
msgbox "txt len: " StrLen(txt) ; <-------------- returns "txt len: 9728341"
obj1 := js.parse(txt) ; <----------------------- error here "Invalid arg type"

txt2 := js.stringify(obj1)
msgbox txt2

FileAppend txt2, A_Desktop "\big_json_test_2.txt"

; ==============================================================
class JSON {
    static __New() {
        Native.LoadModule('ahk-json\' (A_PtrSize * 8) 'bit\ahk-json.dll', ['JSON'])
        this.DefineProp('true', {value: 1})
        this.DefineProp('false', {value: 0})
        this.DefineProp('null', {value: ''})
    }
    static parse(str) => 1
    static stringify(obj, space := 0) => 1
}

image.png (18.75 KiB) Viewed 5409 times

@thqby

I don't see any major difference between my code and the code/pic you posted showing no error. One thing i can say is that adding a CRLF to the end of the text had no effect.

I also tried using js obj and JSON obj. It made no difference.

It doesn't seem to be an issue with parsing at all. It seems to simply be an invalid type, as if a string is not being passed to the obj.parse() method.

Is there something super simple that we're missing?

[RaptorX]

Unfortunately I get the same error with this file.
I validated it with the site you sent and it was good.

EDIT:
Looking at the previous response, I think I know what is going on, I am trying load a 24mb file. Probably the function is truncating the string somehow.

Ill test a bit more.

This is what im trying:

Code: Select all

#Include <cppJSON\cppJSON>
#Include <JXON\_JXON>

data := FileRead("res\test.json")
DllCall("QueryPerformanceFrequency", "Int64*", &freq := 0)

OutputDebug "Performance check on a 24MB json file.`n`n"

OutputDebug "-- JXON`n"
DllCall("QueryPerformanceCounter", "Int64*", &CounterBefore := 0)
loadedData := Jxon_Load(&data)
DllCall("QueryPerformanceCounter", "Int64*", &CounterAfter := 0)
OutputDebug "-- JXON load time: " (CounterAfter - CounterBefore) / freq * 1000 " ms`n"

OutputDebug "-- c++ JSON`n"
DllCall("QueryPerformanceCounter", "Int64*", &CounterBefore := 0)
loadedData := JSON.parse(data)
DllCall("QueryPerformanceCounter", "Int64*", &CounterAfter := 0)
OutputDebug "-- c++ JSON load time: " (CounterAfter - CounterBefore) / freq * 1000 " ms`n"

and this is the result:

Performance check on a 24MB json file.

-- JXON
-- JXON load time: 51025.5818 ms
-- c++ JSON
D:\Cloud\RaptorX\OneDrive\Desktop\JSONTest\src\JSONTest.ahk (17) : ==> Invalid arg type.

So the other library reads the file correctly, it's this lib that has an issue.

[kczx3]

The file is either not valid JSON or not very useful. It repeats the same key of "event" in the results objects which makes no sense. "events" should probably just be an array of objects representing each event. That said, fixing that still seems to result in the "Invalid arg type" for me. I tried passing it ByRef also but that produced the error. If I instead pass StrPtr(txt) instead to js.parse(), then it works. Perhaps there is some limit where you have to pass the address of the string?

[TheArkive]

@kczx3

Thanks. I had not thought of the string pointer. It works for me too, but it drastically truncates the json that is read.

[vmech]

Only the most recent "event" key is saved after processing.

Spoiler

big_json_test_2.txt:{"result":{"count":"37859","event":{"category1":"Date unknown","date":"2012/12/31","description":"The Kars–Tbilisi–Baku railway across the Caucasus is scheduled to be completed sometime in 2012.ampamp''National Geographic'', August 2010, page 62.ampamp","granularity":"year","lang":"en"}}}

This fact confirms that the json downloaded from the @RaptorX link is incorrect.

PS. Any normal json (e.g. EGS manifest file) is processed normally - the results are expected.

[TheArkive]

@vmech

Thanks. That's a very helpful explanation.

[RaptorX]

After I noticed the JSON file was a "weird" file I began testing with this one instead.

I am having the same issue even though the file is validated with the online tool. Can you guys test with this new file?

I will check what @kczx3 suggested

EDIT:
Confirmed that passing the StrPtr() to the library worked. And the object returned was complete. I accessed loadedData[11300]['id'] which the original file shows 11,350 items, so I know the object was loaded without issues.

So The solution might be that the library always pass a pointer to a string to the dll function?

[RaptorX]

By the way just for fun this is what I was trying:

Performance check on a 24MB json file.

-- JXON
-- JXON load time: 49866.395300000004 ms

-- c++ JSON
-- c++ JSON load time: 482.48000000000002 ms

Thats an insane difference, which is why I have been waiting for a library like this )
Maybe if we fix the pointer thing it would be a great library for a lot of people that deal with data from the internet.