[Library] cJson.ahk (version 0.4.1 pre-release)

Post your working scripts, libraries and tools for AHK v1.1 and older
geek
Posts: 1068
Joined: 02 Oct 2013, 22:13
Location: GeekDude
Contact:

[Library] cJson.ahk (version 0.4.1 pre-release)

04 Jul 2021, 23:52

cJson.ahk

The first and only AutoHotkey JSON library to use embedded compiled C for high performance.


Compatibility

This library is compatible with AutoHotkey v1.1 U64 and U32.

Now that AHKv2 is out of Alpha, it's likely that its object structures will not change significantly again in the future. Compatibility with AHKv2 will require modification to both the AHK wrapper and the C implementation. Support is planned, but may not be implemented any time soon.


Using cJson

Converting an AHK Object to JSON:

Code: Select all

#Include <JSON>

; Create an object with every supported data type
obj := ["abc", 123, {"true": true, "false": false, "null": ""}, [JSON.true, JSON.false, JSON.null]]

; Convert to JSON
MsgBox, % JSON.Dump(obj) ; Expect: ["abc", 123, {"false": 0, "null": "", "true": 1}, [true, false, null]]
Converting JSON to an AHK Object:

Code: Select all

#Include <JSON>

; Create some JSON
str = ["abc", 123, {"true": 1, "false": 0, "null": ""}, [true, false, null]]
obj := JSON.Load(str)

MsgBox, % obj[1] ; abc
MsgBox, % obj[2] ; 123

MsgBox, % obj[3].true ; 1
MsgBox, % obj[3].false ; 0
MsgBox, % obj[3].null ; *nothing*

MsgBox, % obj[4, 1] ; 1
MsgBox, % obj[4, 2] ; 0
MsgBox, % obj[4, 3] == JSON.Null ; 1

Notes

Data Types

AutoHotkey does not provide types that uniquely identify all the possible values that may be encoded or decoded. To work around this problem, cJson provides magic objects that give you greater control over how things are encoded. By default, cJson will behave according to the following table:

Code: Select all

| Value         | Encodes as | Decodes as    |
|---------------|------------|---------------|
| `true`        | `1`        | `1` *         |
| `false`       | `0`        | `0` *         |
| `null`        | N/A        | `JSON.Null`   |
| `0.5` †       | `"0.5"`    | `0.500000`    |
| `0.5+0` †     | `0.500000` | N/A           |
| `JSON.True`   | `true`     | N/A           |
| `JSON.False`  | `false`    | N/A           |
| `JSON.Null`   | `null`     | N/A           |
* To avoid type data loss when decoding true and false, the class property JSON.BoolsAsInts can be set := true. Once set, boolean true and false will decode to JSON.True and JSON.False respectively.

† Pure floats, as generated by an expression, will encode as floats. Hybrid floats that contain a string buffer will encode as strings. Floats hard-coded into a script are saved by AHK as hybrid floats. To force encoding as a float, perform some redundant operation like adding zero.

Array Detection

AutoHotkey makes no internal distinction between indexed-sequential arrays and keyed objects. As a result, this distinction must be chosen heuristically by the cJson library. If an object contains only sequential integer keys starting at 1, it will be rendered as an array. Otherwise, it will be rendered as an object.


Roadmap
  • Add a pretty print mode for Dumps.
  • Add methods to extract values from the JSON blob without loading the full object into memory.
  • Add methods to replace values in the JSON blob without fully parsing and reformatting the blob.
  • Add a special class to force encoding of indexed arrays as objects.
  • Integrate with a future MCLib-hosted COM-based hash-table style object for even greater performance.
  • AutoHotkey v2 support.


Download cJson.ahk
geek
Posts: 1068
Joined: 02 Oct 2013, 22:13
Location: GeekDude
Contact:

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

04 Jul 2021, 23:53

First

For now, this is a pre-release. Don't be surprised if something or other is broken. Please let me know if it is though, so it can be fixed for the big 1.0.0!

Thanks everyone!

Edit: Also check out the WIP library "MCL" I'm working on with CloakerSmoker https://github.com/G33kDude/MCLib.ahk. It's a new MCode library that takes advantage of black magic to solve most of the problems with existing implementations. It powers the heart of cJson.ahk. It has not yet hit its first release.

Edit Edit: Check out my Discord https://geekdude.io/discord to get the latest on project updates, and to reach me directly

Edit Edit Edit: Oh, and the reason to use this is because it's fast. I don't have benchmarks up yet, but previous testing suggests that Loads is ~6x faster than Coco's in the average case, and on principle Dumps should be many times more than that faster
User avatar
elModo7
Posts: 221
Joined: 01 Sep 2017, 02:38
Location: Spain
Contact:

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

05 Jul 2021, 01:16

I will for sure be testing this one out in the upcoming days!
Thanks :bravo:
Tre4shunter
Posts: 139
Joined: 26 Jan 2016, 16:05

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

06 Jul 2021, 08:52

Awesome!

In my initial tests using this dataset:
https://raw.githubusercontent.com/prust/wikipedia-movie-data/master/movies.json

cJson parses in ~.15s on my system, as opposed to ~.75s using Cocos' parser. Not a huge dataset, either - about a 3.3kb file.

Excellent Work! :bravo: :dance: :clap:
william_ahk
Posts: 639
Joined: 03 Dec 2018, 20:02

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

06 Jul 2021, 23:25

I was just needing this, what a timely release. Thank you so much!
User avatar
Joe Glines
Posts: 772
Joined: 30 Sep 2013, 20:49
Location: Dallas
Contact:

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

07 Jul 2021, 08:12

Very cool @GeekDude! Thanks for creating it! perhaps we'll do a video showing some benchmark comparisons.
Sign-up for the 🅰️HK Newsletter

ImageImageImageImage:clap:
AHK Tutorials:Web Scraping | | Webservice APIs | AHK and Excel | Chrome | RegEx | Functions
Training: AHK Webinars Courses on AutoHotkey :ugeek:
YouTube

:thumbup: Quick Access Popup, the powerful Windows folders, apps and documents launcher!
AHK_user
Posts: 523
Joined: 04 Dec 2015, 14:52
Location: Belgium

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

23 Jul 2021, 12:17

Thanks, @GeekDude I just modified Chrome.ahk and started testing, seems to speed up the script greatly.

Very usefull. Great work!!!
:bravo: :bravo: :bravo:
AHK_user
Posts: 523
Joined: 04 Dec 2015, 14:52
Location: Belgium

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

26 Jul 2021, 12:41

To @GeekDude: Is it possible to add the following lines at the start of the _init() function?
I think this would prevent to answer the same question over and over :lol: .
I would also mention it in the comments on the top of the script.

Code: Select all

_init()
{
	if (A_PtrSize =4){
		MsgBox,16,cJson, Please run this script with 64-bit Autohotkey.`n32-bit is not supported by cJson.`nCurrent thread will exit.
		Exit
	}
	
	....
geek
Posts: 1068
Joined: 02 Oct 2013, 22:13
Location: GeekDude
Contact:

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

27 Jul 2021, 13:12

My bad, that check was in some earlier drafts of the machine code loader function and got lost through the months of development. I'll look at its inclusion in the next release if 32-bit is not yet supported by then.
burque505
Posts: 1747
Joined: 22 Jan 2017, 19:37

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

30 Jul 2021, 10:15

@AHK_user, would you mind sharing your modified version of Chrome.ahk?
Thanks very much.
Regards,
burque505
AHK_user
Posts: 523
Joined: 04 Dec 2015, 14:52
Location: Belgium

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

30 Jul 2021, 11:13

Here is the cJSon version of the script.
The LightJson class is not used, but I left it commented in case I would need it when some programs require 32 bit.
Maybe I add an If statement to use LightJson if 32bit is used

Code: Select all

; Chrome.ahk
; Created by GeekDude
; https://github.com/G33kDude/Chrome.ahk/blob/master/Chrome.ahk
; This version is modified with cJson class
; COMMENT: cJson only works for 64bit AHK!

class Chrome
{
	static DebugPort := 9222
	
	/*
		Escape a string in a manner suitable for command line parameters
	*/
	CliEscape(Param)
	{
		return """" RegExReplace(Param, "(\\*)""", "$1$1\""") """"
	}
	
	/*
		Finds instances of chrome in debug mode and the ports they're running
		on. If no instances are found, returns a false value. If one or more
		instances are found, returns an associative array where the keys are
		the ports, and the values are the full command line texts used to start
		the processes.
		
		One example of how this may be used would be to open chrome on a
		different port if an instance of chrome is already open on the port
		you wanted to used.
		
		```
		; If the wanted port is taken, use the largest taken port plus one
		DebugPort := 9222
		if (Chromes := Chrome.FindInstances()).HasKey(DebugPort)
			DebugPort := Chromes.MaxIndex() + 1
		ChromeInst := new Chrome(ProfilePath,,,, DebugPort)
		```
		
		Another use would be to scan for running instances and attach to one
		instead of starting a new instance.
		
		```
		if (Chromes := Chrome.FindInstances())
			ChromeInst := {"base": Chrome, "DebugPort": Chromes.MinIndex(), PID: Chromes[Chromes.MinIndex(), "PID"]}
		else
			ChromeInst := new Chrome(ProfilePath)
		```
	*/
	FindInstances()
	{
		Out := {}
		for Item in ComObjGet("winmgmts:").ExecQuery("SELECT * FROM Win32_Process WHERE Name = 'chrome.exe'")
			if RegExMatch(Item.CommandLine, "i)chrome.exe""?\s+--remote-debugging-port=(\d+)", Match)
				Out[Match1] := {cmd: Item.CommandLine, PID: Item.ProcessId}
		return Out.MaxIndex() ? Out : False
	}
	
	/*
		ProfilePath - Path to the user profile directory to use. Will use the standard if left blank.
		URLs        - The page or array of pages for Chrome to load when it opens
		Flags       - Additional flags for chrome when launching
		ChromePath  - Path to chrome.exe, will detect from start menu when left blank
		DebugPort   - What port should Chrome's remote debugging server run on
	*/
	__New(ProfilePath:="", URLs:="about:blank", Flags:="", ChromePath:="", DebugPort:="")
	{
		; Verify ProfilePath
		if (ProfilePath != "" && !InStr(FileExist(ProfilePath), "D"))
			throw Exception("The given ProfilePath does not exist")
		this.ProfilePath := ProfilePath
		
		; Verify ChromePath
		if (ChromePath == "")
			FileGetShortcut, %A_StartMenuCommon%\Programs\Google Chrome.lnk, ChromePath
		if (ChromePath == "")
			RegRead, ChromePath, HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\App Paths\chrome.exe
		if !FileExist(ChromePath)
			throw Exception("Chrome could not be found")
		this.ChromePath := ChromePath
		
		; Verify DebugPort
		if (DebugPort != "")
		{
			if DebugPort is not integer
				throw Exception("DebugPort must be a positive integer")
			else if (DebugPort <= 0)
				throw Exception("DebugPort must be a positive integer")
			this.DebugPort := DebugPort
		}
		
		; Escape the URL(s)
		for Index, URL in IsObject(URLs) ? URLs : [URLs]
			URLString .= " " this.CliEscape(URL)
		
		Run, % this.CliEscape(ChromePath)
		. " --remote-debugging-port=" this.DebugPort
		. (ProfilePath ? " --user-data-dir=" this.CliEscape(ProfilePath) : "")
		. (Flags ? " " Flags : "")
		. URLString
		,,, OutputVarPID
		this.PID := OutputVarPID
	}
	
	/*
		End Chrome by terminating the process.
	*/
	Kill()
	{
		Process, Close, % this.PID
	}
	
	/*
		Queries chrome for a list of pages that expose a debug interface.
		In addition to standard tabs, these include pages such as extension
		configuration pages.
	*/
	GetPageList()
	{
		http := ComObjCreate("WinHttp.WinHttpRequest.5.1")
		http.open("GET", "http://127.0.0.1:" this.DebugPort "/json")
		http.send()
		
		;return LightJson.Parse(http.responseText)
		return cJson.Loads(http.responseText)
	}
	
	/*
		Returns a connection to the debug interface of a page that matches the
		provided criteria. When multiple pages match the criteria, they appear
		ordered by how recently the pages were opened.
		
		Key        - The key from the page list to search for, such as "url" or "title"
		Value      - The value to search for in the provided key
		MatchMode  - What kind of search to use, such as "exact", "contains", "startswith", or "regex"
		Index      - If multiple pages match the given criteria, which one of them to return
		fnCallback - A function to be called whenever message is received from the page
	*/
	GetPageBy(Key, Value, MatchMode:="exact", Index:=1, fnCallback:="")
	{
		Count := 0
		for n, PageData in this.GetPageList()
		{
			if (((MatchMode = "exact" && PageData[Key] = Value) ; Case insensitive
				|| (MatchMode = "contains" && InStr(PageData[Key], Value))
				|| (MatchMode = "startswith" && InStr(PageData[Key], Value) == 1)
				|| (MatchMode = "regex" && PageData[Key] ~= Value))
				&& ++Count == Index)
				return new this.Page(PageData.webSocketDebuggerUrl, fnCallback)
		}
	}
	
	/*
		Shorthand for GetPageBy("url", Value, "startswith")
	*/
	GetPageByURL(Value, MatchMode:="startswith", Index:=1, fnCallback:="")
	{
		return this.GetPageBy("url", Value, MatchMode, Index, fnCallback)
	}
	
	/*
		Shorthand for GetPageBy("title", Value, "startswith")
	*/
	GetPageByTitle(Value, MatchMode:="startswith", Index:=1, fnCallback:="")
	{
		return this.GetPageBy("title", Value, MatchMode, Index, fnCallback)
	}
	
	/*
		Shorthand for GetPageBy("type", Type, "exact")
		
		The default type to search for is "page", which is the visible area of
		a normal Chrome tab.
	*/
	GetPage(Index:=1, Type:="page", fnCallback:="")
	{
		return this.GetPageBy("type", Type, "exact", Index, fnCallback)
	}
	
	/*
		Connects to the debug interface of a page given its WebSocket URL.
	*/
	class Page
	{
		Connected := False
		ID := 0
		Responses := []
		
		/*
			wsurl      - The desired page's WebSocket URL
			fnCallback - A function to be called whenever message is received
		*/
		__New(wsurl, fnCallback:="")
		{
			this.fnCallback := fnCallback
			this.BoundKeepAlive := this.Call.Bind(this, "Browser.getVersion",, False)
			
			; TODO: Throw exception on invalid objects
			if IsObject(wsurl)
				wsurl := wsurl.webSocketDebuggerUrl
			
			wsurl := StrReplace(wsurl, "localhost", "127.0.0.1")
			this.ws := {"base": this.WebSocket, "_Event": this.Event, "Parent": this}
			this.ws.__New(wsurl)
			
			while !this.Connected
				Sleep, 50
		}
		
		/*
			Calls the specified endpoint and provides it with the given
			parameters.
			
			DomainAndMethod - The endpoint domain and method name for the
			endpoint you would like to call. For example:
			PageInst.Call("Browser.close")
			PageInst.Call("Schema.getDomains")
			
			Params - An associative array of parameters to be provided to the
			endpoint. For example:
			PageInst.Call("Page.printToPDF", {"scale": 0.5 ; Numeric Value
			, "landscape": LightJson.true ; Boolean Value
			, "pageRanges: "1-5, 8, 11-13"}) ; String value
			PageInst.Call("Page.navigate", {"url": "https://autohotkey.com/"})
			
			WaitForResponse - Whether to block until a response is received from
			Chrome, which is necessary to receive a return value, or whether
			to continue on with the script without waiting for a response.
		*/
		Call(DomainAndMethod, Params:="", WaitForResponse:=True)
		{
			if !this.Connected
				throw Exception("Not connected to tab")
			
			; Use a temporary variable for ID in case more calls are made
			; before we receive a response.
			ID := this.ID += 1
			;~ this.ws.Send(LightJson.Stringify({"id": ID
			;~ , "params": Params ? Params : {}
			;~ , "method": DomainAndMethod}))
			this.ws.Send(cJson.Dumps({"id": ID
			, "params": Params ? Params : {}
			, "method": DomainAndMethod}))
			
			if !WaitForResponse
				return
			
			; Wait for the response
			this.responses[ID] := False
			while !this.responses[ID]{
				Sleep, 50
				
				; Added extra to prevent getting stuck
				; if (A_Index=200){
				;	throw Exception("Chrome not responding",,"")
				; }
			}
			
			; Get the response, check if it's an error
			response := this.responses.Delete(ID)
			if (response.error)
				throw Exception("Chrome indicated error in response",, cJson.Dumps(response.error))
			;~ throw Exception("Chrome indicated error in response",, LightJson.Stringify(response.error))
			
			return response.result
		}
		
		/*
			Run some JavaScript on the page. For example:
			
			PageInst.Evaluate("alert(""I can't believe it's not IE!"");")
			PageInst.Evaluate("document.getElementsByTagName('button')[0].click();")
		*/
		Evaluate(JS)
		{
			response := this.Call("Runtime.evaluate",
			( LTrim Join
			{
				"expression": JS,
				"objectGroup": "console",
				"includeCommandLineAPI": cJson.true,
				"silent": cJson.false,
				"returnByValue": cJson.false,
				"userGesture": cJson.true,
				"awaitPromise": cJson.false
			}
			))
			
			if (response.exceptionDetails)
				throw Exception(response.result.description,, cJson.Dumps(response.exceptionDetails))
			;~ throw Exception(response.result.description,, LightJson.Stringify(response.exceptionDetails))
			
			return response.result
		}
		
		/*
			Waits for the page's readyState to match the DesiredState.
			
			DesiredState - The state to wait for the page's ReadyState to match
			Interval     - How often it should check whether the state matches
		*/
		WaitForLoad(DesiredState:="complete", Interval:=100)
		{
			while this.Evaluate("document.readyState").value != DesiredState
				Sleep, Interval
		}
		
		/*
			Internal function triggered when the script receives a message on
			the WebSocket connected to the page.
		*/
		Event(EventName, Event)
		{
			; If it was called from the WebSocket adjust the class context
			if this.Parent
				this := this.Parent
			
			; TODO: Handle Error events
			if (EventName == "Open")
			{
				this.Connected := True
				BoundKeepAlive := this.BoundKeepAlive
				SetTimer, %BoundKeepAlive%, 15000
			}
			else if (EventName == "Message")
			{
				;~ data := LightJson.Parse(Event.data)
				data := cJson.Loads(Event.data)
				
				; Run the callback routine
				fnCallback := this.fnCallback
				if (newData := %fnCallback%(data))
					data := newData
				
				if this.responses.HasKey(data.ID)
					this.responses[data.ID] := data
			}
			else if (EventName == "Close")
			{
				this.Disconnect()
			}
			else if (EventName == "Error")
			{
				throw Exception("Websocket Error!")
			}
		}
		
		/*
			Disconnect from the page's debug interface, allowing the instance
			to be garbage collected.
			
			This method should always be called when you are finished with a
			page or else your script will leak memory.
		*/
		Disconnect()
		{
			if !this.Connected
				return
			
			this.Connected := False
			this.ws.Delete("Parent")
			this.ws.Disconnect()
			
			BoundKeepAlive := this.BoundKeepAlive
			SetTimer, %BoundKeepAlive%, Delete
			this.Delete("BoundKeepAlive")
		}
		
		class WebSocket
		{
			__New(WS_URL)
			{
				static wb
				
				; Create an IE instance
				Gui, +hWndhOld
				Gui, New, +hWndhWnd
				this.hWnd := hWnd
				Gui, Add, ActiveX, vWB, Shell.Explorer
				Gui, %hOld%: Default
				
				; Write an appropriate document
				WB.Navigate("about:<!DOCTYPE html><meta http-equiv='X-UA-Compatible'"
				. "content='IE=edge'><body></body>")
				while (WB.ReadyState < 4)
					sleep, 50
				this.document := WB.document
				
				; Add our handlers to the JavaScript namespace
				this.document.parentWindow.ahk_savews := this._SaveWS.Bind(this)
				this.document.parentWindow.ahk_event := this._Event.Bind(this)
				this.document.parentWindow.ahk_ws_url := WS_URL
				
				; Add some JavaScript to the page to open a socket
				Script := this.document.createElement("script")
				Script.text := "ws = new WebSocket(ahk_ws_url);`n"
				. "ws.onopen = function(event){ ahk_event('Open', event); };`n"
				. "ws.onclose = function(event){ ahk_event('Close', event); };`n"
				. "ws.onerror = function(event){ ahk_event('Error', event); };`n"
				. "ws.onmessage = function(event){ ahk_event('Message', event); };"
				this.document.body.appendChild(Script)
			}
			
			; Called by the JS in response to WS events
			_Event(EventName, Event)
			{
				this["On" EventName](Event)
			}
			
			; Sends data through the WebSocket
			Send(Data)
			{
				this.document.parentWindow.ws.send(Data)
			}
			
			; Closes the WebSocket connection
			Close(Code:=1000, Reason:="")
			{
				this.document.parentWindow.ws.close(Code, Reason)
			}
			
			; Closes and deletes the WebSocket, removing
			; references so the class can be garbage collected
			Disconnect()
			{
				if this.hWnd
				{
					this.Close()
					Gui, % this.hWnd ": Destroy"
					this.hWnd := False
				}
			}
		}
	}
}

class LightJson
{
	static JS := LightJson.GetJS(), true := {}, false := {}, null := {}
	
	Parse(json, _rec := false) {
		if !_rec
			obj := this.Parse(this.JS.eval("(" . json . ")"), true)
		else if !IsObject(json)
			obj := json
		else if this.JS.Object.prototype.toString.call(json) == "[object Array]" {
			obj := []
			Loop % json.length
				obj.Push( this.Parse(json[A_Index - 1], true) )
		}
		else {
			obj := {}
			keys := this.JS.Object.keys(json)
			Loop % keys.length {
				k := keys[A_Index - 1]
				obj[k] := this.Parse(json[k], true)
			}
		}
		Return obj
	}
	
	Stringify(obj, indent := "") {
		if indent|1 {
			for k, v in ["true", "false", "null"]
				if (obj = this[v])
					Return v
			
			if IsObject( obj ) {
				isArray := true
				for key in obj {
					if IsObject(key)
						throw Exception("Invalid key")
					if !( key = A_Index || isArray := false )
						break
				}
				for k, v in obj
					str .= ( A_Index = 1 ? "" : "," ) . ( isArray ? "" : """" . k . """:" ) . this.Stringify(v, true)
				
				Return str = "" ? "{}" : isArray ? "[" . str . "]" : "{" . str . "}"
			}
			else if !(obj*1 = "" || RegExMatch(obj, "^-?0|\s"))
				Return obj
			
			for k, v in [["\", "\\"], [A_Tab, "\t"], ["""", "\"""], ["/", "\/"], ["`n", "\n"], ["`r", "\r"], [Chr(12), "\f"], [Chr(8), "\b"]]
				obj := StrReplace( obj, v[1], v[2] )
			
			Return """" obj """"
		}
		sObj := this.Stringify(obj, true)
		Return this.JS.eval("JSON.stringify(" . sObj . ",'','" . indent . "')")
	}
	
	GetJS() {
		static Doc, JS
		if !Doc {
			Doc := ComObjCreate("htmlfile")
			Doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
			JS := Doc.parentWindow
			( Doc.documentMode < 9 && JS.execScript() )
		}
		Return JS
	}
}


#Include <cJson>

Code: Select all

;
; cJson.ahk 0.1.0
; Copyright (c) 2021 Philip Taylor (known also as GeekDude, G33kDude)
; https://github.com/G33kDude/cJson.ahk
;
; MIT License
;
; Permission is hereby granted, free of charge, to any person obtaining a copy
; of this software and associated documentation files (the "Software"), to deal
; in the Software without restriction, including without limitation the rights
; to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
; copies of the Software, and to permit persons to whom the Software is
; furnished to do so, subject to the following conditions:
;
; The above copyright notice and this permission notice shall be included in all
; copies or substantial portions of the Software.
;
; THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
; IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
; FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
; AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
; LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
; OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
; SOFTWARE.
;

class cJson
{
	static version := "0.1.0"
	
	_init()
	{
		
		if (A_PtrSize =4 && !A_IsCompiled){
			MsgBox,16,cJson, Please run this script with 64-bit Autohotkey.`n32-bit is not supported by cJson.`nCurrent thread will exit.
			Exit
		}
		
		if (this.lib)
			return
		
		this.lib := this.MCLib.FromString("V0;__mcode_e_dumps-0,__mcode_e_loads-3437,__size-7328;"
		. "trUAVUiJ5UiD7HAASIlNEEiJVRgATIlFIEyJTShASItFEEiLABBFADhIiwBIOcIPAIS0AAAAx0X8AQAUAOtHSIN9GAAAdC2LRfxImABIjRXlGwAA"
		. "RCgPtgQBYBgBYI1IAAJIi1UYSIkKAGZBD77QZokQBOsPADYgiwCNUAIBARCJEINF/AENBX6lAD8BPoTAdaUJA3pNIAE/SYnISICJweiqCwAAA3FC"
		. "GRBgxwAiAA5duAEBp+kuBgAAxkUk+wADU1AwAwdAIAkA0HVbABgBx0X0hQJoNYQQGItF9IBIAMHgBUgB0EiJTEXQgAuAAVAQgAuDDMABAA0AhZTA"
		. "iEUA+4NF9AGAfftoAHQTARljARQFLXwKsgNWLIIPCEG4W2EBMQZBuHuAAxBgRCKJj1/HRfCCYA4FCwIZhE3wiU3Ig33waAB+XhmMLA8sGRcgCY8L"
		. "D7aAOvABhMDAD4TYAQAAgSDGOnIwAUONgQE1K2dAAchTwA+NecMJRRIcVBLpZvZDUAkT6eLABMolOFXCJbDCq+zMq+zCm0baGfGr7MCrww8GwA/I"
		. "qykRM/YInkUsTBjrG2nUEm0D2xI6vnkBDw+AtkAYPAF1H4MQi8AsahAm4BnpfALCL2FDBQYPhZRAQsUFOYBFKHVpx0XoDC6y6AIm3hcfLg8u6AAu"
		. "TeMHnuAHBS7p9qoOMNWhDuSsDuSiBm2/DrAOVuSgDuMHLakOgKoOOJWhDuCsDuCiBv0Wvw5trw7goA7jB73gB6YOCsGnDkyLTShMoAViCYCLTThI"
		. "iUwkALUUTTABASBhNC76/1j/6dhDZGM0BdU5GqngCumtQmTcrBjcohA2PL8YsBjcoBjjB/wVu01kTRDUQZsfhwsM8AAMY0h9wa0PjNvgHkyuXQ1E"
		. "rn1frkrGSIPEcDxdw8TigCrH4jyTCAQNL5P04XYAFQ+3AGagg/gidWbZHVyVeK1WDpDDC0oOfEoOXF8OJy8HLwfmAukdLwfpCX0qBwgvBy8HLwcv"
		. "B+ICYtgA6aozRyoHljMBJAc+DC8HLwcvBy8H4gJmAKTpNy8H6SMqBwovBw8vBy8HLwfiAm4A6cTbcz4qB7AzASQHDS8HLweHLwcvB+ICcgDpUS8H"
		. "9Ok9KgcJLwcvBy8HLwch4gJ0AOne73vpyiMzASQHH3YNxwB+dr58/wf/B29v7wLgAnXvAnGkBg+3wLhJoEkxjeu6NMMEHs8EYABgAxIvUBkBCEUQ"
		. "YAITDIXAD3iF/Pu1SV8JX04yBJCdQEggR0jRUiNIZolATxCNBbEQEBOJRfAB1bgyD7dFEIPgUA9IicIABPDQoQ8YtgBmMFritWaJVIRF6BECZsHo"
		. "BBEEAYG3g338A37Ix0RF+LA7AOs8AwoiAQ8Ki1X4SGPSDzC3VFXo3w7wWG344aAE+AB5vsVTdwtgaVHHU8ZF/5Bm+EMCfRAQAHkIEAEBSPeCXaGO"
		. "EEi6Z2YDAAJIwA736kiJ0EgIwfgCsBhJwfg/mEwpwJENMQHgAqGvAAHASCnBSInKIInRi0X4sAiJVfD4jVEwkw6AYf8E9ASQSMH5P2ADSCnRswGh"
		. "IhAAdYCAff8YAHQShgRQBMdERbjQLQChDY8RjxFVn3BLjxGAETCFEcSAz9ZFWChIx3ECgTAwkwDr9hKRBEIHUAElsGjhKhEBEcMxIHTe+gAKdM4R"
		. "+gANdL76AAl0rmH6AHsPhY50Tl4GGNK5sHkA/3ESwI8IjwgPjwiPCI8I9AB9D4S6J6RINAFwM4SnIAFMjQxFuIM54BeNVbRIgIlUJCBNicEChAGw"
		. "Lk0Q6Kj+//9AhcB0HLj/AADp/FMNDw3fCt8K3wrfCvQASDp0CrMG5wy/BonOEACaMZkjC4tVQDAuCyz1/SILZASgYASLRSi0SGMQklXQDkUw+5Ce"
		. "wQBFwJ4BBsAvcZ5xBBpMcEe6kRngk0H/0h13D1u/CH8Pfw+LRfO0ACBIiwAPtwBmAIP4DXS+SItFEQfwCXSuCngsdRchBDxIjVACAiiJEEDp3/3/"
		. "/5AKTn0QdAq4/wAA6ccLDAAAD2IADChIi1UCwAMUMMcABQAAxAC4AAMA6ZYHMANKIFsPhd8BFEQYuQIEADP/0EiJRciM6xISIgdIIHTeCg8YCnTO"
		. "Cg+dfV0PhAYLB0SBCYXAD4T4lQBATAJnTQALRRgAawAwSIlUJCBNiYDBSYnISInCgAwAEOgG/P//hcAphYqxCoIjMABWY8gDA4aBVchIx0QkKAsB"
		. "gwEEIAEETItVEIBJiclJidC6AXVASInBQf/SACQcnQMkaAMkWXuTfKL+jXzyXQU31AnUJMB5wjADfzWARhCDfKMHDIMSIg/YhegDGDaAe/hAEMAA"
		. "x0AShRYADADpeIcNA2gKdQQlP80YXA+FJzvZGEQidYAqABuBB4lVIPhmxwAikAzpAZeLHUAXzg1c0Q3JAswNqi/ODS/RDZHNDWLODaoI0Q1ZzQ1m"
		. "zg0M8QaqIe0Gbu4GCvEG6etlqIP4cu4GDfEGse0GKnTuBgnxBnntBnUPDIU3lYehCAAAx0Uq9IJQ/yNT+OAGweCOBKF2AASsQS9+NuoBODl/JkQG"
		. "4gUEAwHQyIPoMAYI6ZADC6cGiEB+M+oBRn8jtAhSN6YI601KBmBMCGZNVghXRwgkYTUG8SGDAEX0AYN99AMPBI73IHtIg0X4AhTrLgMGBAYGEEiN"
		. "qkoEBgjDMUjAMU3gDvYSTSVAeHRAnsYsLw5jf2anIL8KBy10IT1oLg+EjtetazkPj8NgAnjHRfBgO+OJYQxjqsfmAIQCaAp1GWAFgRncfVgwdSKr"
		. "CQ1MjU0LMER+ckxCYus+JbqJANBIweACSAHQwEgBwEmJwOYJoCYUi1XgCQpgBkgPv9DATAHAgAzQx5lJTtIc6wt+ogU5bICpS7AwLg+FsvIWAxIA"
		. "ZoAP78DySA8qIRFwKPIPEeQlgHuwFOwBYRPrVYtV7InQAVALAdABwIlF7Hs/C6AGmAAusgSgBHAA0gFwAFXsZg8oyPIID17KswUQAPIPHFjBeAYp"
		. "DSsOOX6LEfoAZXQU+gBFD4UGfz88dF+LAIP4AUdUGw8OCA7GRetNIRg1MAEBbwXrQT13Ait1vha4A58jTw1DDVQaxsAHCMdF5PEk6y6LVVrkxhVB"
		. "DyHGFUSxF+RDDwYKBrLHReAxG8eERdzSBROLVeCoGwDgg0XcAYtF3AA7ReR85YB961gAdB+lGWASyTAbTabg8Bo2Gusd5AEIpB1aReABWSkcxBYZ"
		. "5C2LgEXwSJhID691K6brgTsyAgJ1vgTwuQSFoz+sT15/x0XYsgwQQ4tF2MAFjRWrAUACD7YEEGYPvtNxBsUCOcLVhmavIeJMFtiwDyMEaCQEhMB1"
		. "vqnzIABNVUMBFOMIHe0IqmbhCNTlCNSyBCHvCG3rCNf/N+II1OAIIwTeLxAC7whoR+MIjp1GbnUQecdF0OIIQItFWtCyBJi0BO8IB+II62JLvwiD"
		. "RdCwCPMDWBW3CKyzCAe/CADrBQHCBEiD7IBdw5ABAAAiVW5rbm93AG5fT2JqZWN0CF8AIoUAdHJ1ZQAAZmFsc2UAbhB1bGwAlgJWYWwAdWVfADAx"
		. "MjMANDU2Nzg5QUIQQ0RFRkIGAEdDAEM6ICh0ZG02ADQtMSkgMTAuGDMuMKMBAgA=")
		
		this.pfnPush := RegisterCallback(this._Push, "Fast CDecl",, &this)
		this.pfnGetObj := RegisterCallback(this._GetObj, "Fast CDecl",, &this)
	}
	
	_GetObj()
	{
		type := this, this := Object(A_EventInfo)
		return Object(Object())
	}
	
	_Push(typeObject, value, typeValue, key, typeKey)
	{
		pObject := this, this := Object(A_EventInfo)
		value := this._Value(value, typeValue)
		
		if (typeObject == 4)
			ObjPush(Object(pObject), value)
		else if (typeObject == 5)
			ObjRawSet(Object(pObject), this._Value(key, typeKey), value)
		
		return 0
	}
	
	Dumps(obj)
	{
		this._init()
		if (!IsObject(obj))
			throw Exception("Input must be object")
		size := 0
		DllCall(this.lib.dumps, "Ptr", &obj, "Ptr", 0, "Int*", size
		, "Ptr", &this.True, "Ptr", &this.False, "Ptr", &this.Null, "CDecl Ptr")
		VarSetCapacity(buf, size*2+2, 0)
		DllCall(this.lib.dumps, "Ptr", &obj, "Ptr*", &buf, "Int*", size
		, "Ptr", &this.True, "Ptr", &this.False, "Ptr", &this.Null, "CDecl Ptr")
		return StrGet(&buf, size, "UTF-16")
	}
	
	Loads(json)
	{
		this._init()
		
		VarSetCapacity(pJson, A_PtrSize)
		NumPut(&json, &pJson, 0, "Ptr")
		
		if (r := DllCall(this.lib.loads, "Ptr", this.pfnPush, "Ptr", this.pfnGetObj, "Ptr", &pJson, "Int64*", pResult
			, "Int*", resultType, "CDecl Int")) || ErrorLevel
		{
			throw Exception("Failed to parse JSON (" r "," ErrorLevel ")", -1
			, Format("Unexpected character at position {}: '{}'"
			, (NumGet(pJson)-&json)//2+1, Chr(NumGet(NumGet(pJson), "short"))))
		}
		
		return this._Value(pResult, resultType)
	}
	
	; type = 1: Integer, 2: Double, 3: String, 4: Array, 5: Object
	_Value(value, type)
	{
		switch (type)
		{
			case 1: return value+0
			case 2:
			VarSetCapacity(tmp, 8, 0)
			NumPut(value, &tmp, "Int64")
			return NumGet(&tmp, 0, "Double")
			case 3: return StrGet(value, "UTF-16")
			case 4, 5: return Object(value), ObjRelease(value)
			case 6: return value ? this.true : this.false
			case 7: return this.null
		}
		throw Exception("Rehydration error: " type)
	}
	
	True[]
	{
		get
		{
			static _ := {"value": true, "name": "true"}
			return _
		}
	}
	
	False[]
	{
		get
		{
			static _ := {"value": false, "name": "false"}
			return _
		}
	}
	
	Null[]
	{
		get
		{
			static _ := {"value": "", "name": "null"}
			return _
		}
	}
	
	;
	; MCLib.ahk redistributable pre-release
	; https://github.com/G33kDude/MCLib.ahk
	;
	; Copyright (c) 2021 G33kDude, CloakerSmoker (CC-BY-4.0)
	; https://creativecommons.org/licenses/by/4.0/
	;
	
	class MClib {
		class LZ {
			Decompress(pCData, cbCData, pData, cbData) {
				if (r := DllCall("ntdll\RtlDecompressBuffer"
					, "UShort",  0x102 ; USHORT CompressionFormat
					, "Ptr", pData     ; PUCHAR UncompressedBuffer
					, "UInt", cbData   ; ULONG  UncompressedBufferSize
					, "Ptr", pCData    ; PUCHAR CompressedBuffer
					, "UInt", cbCData  ; ULONG  CompressedBufferSize,
					, "UInt*", cbFinal ; PULONG FinalUncompressedSize
					, "UInt")) ; NTSTATUS
					throw Exception("Error calling RtlDecompressBuffer",, Format("0x{:08x}", r))
				
				return cbFinal
			}
		}
		
		Load(pCode, CodeSize, Symbols) {
			for SymbolName, SymbolOffset in Symbols {
				if (RegExMatch(SymbolName, "O)__mcode_i_(\w+?)_(\w+)", Match)) {
					DllName := Match[1]
					FunctionName := Match[2]
					
					hDll := DllCall("GetModuleHandle", "Str", DllName, "Ptr")
					
					if (ErrorLevel || A_LastError) {
						Throw "Could not load dll " DllName ", ErrorLevel " ErrorLevel ", LastError " Format("{:0x}", A_LastError)
					}
					
					pFunction := DllCall("GetProcAddress", "Ptr", hDll, "AStr", FunctionName, "Ptr")
					
					if (ErrorLevel || A_LastError) {
						Throw "Could not find function " FunctionName " from " DllName ".dll, ErrorLevel " ErrorLevel ", LastError " Format("{:0x}", A_LastError)
					}
					
					NumPut(pFunction, pCode + 0, SymbolOffset, "Ptr")
				}
			}
			
			if !DllCall("VirtualProtect", "Ptr", pCode, "Ptr", CodeSize, "UInt", 0x40, "UInt*", OldProtect, "UInt")
				Throw Exception("Failed to mark MCLib memory as executable")
			
			Exports := {}
			
			for SymbolName, SymbolOffset in Symbols {
				if (RegExMatch(SymbolName, "O)__mcode_e_(\w+)", Match)) {
					Exports[Match[1]] := pCode + SymbolOffset
				}
			}
			
			if (Exports.Count()) {
				return Exports
			}
			else {
				return pCode + Symbols["__main"]
			}
		}
		
		FromString(Code) {
			Parts := StrSplit(Code, ";")
			
			if (Parts[1] != "V0") {
				Throw "Unknown MClib packed code format"
			}
			
			Symbols := {}
			
			for k, SymbolEntry in StrSplit(Parts[2], ",") {
				SymbolEntry := StrSplit(SymbolEntry, "-")
				
				Symbols[SymbolEntry[1]] := SymbolEntry[2]
			}
			
			CodeBase64 := Parts[3]
			
			DecompressedSize := Symbols.__Size
			
			if !DllCall("Crypt32\CryptStringToBinary", "Str", CodeBase64, "UInt", 0, "UInt", 1
				, "UPtr", 0, "UInt*", CompressedSize, "Ptr", 0, "Ptr", 0, "UInt")
				throw Exception("Failed to parse MCLib b64 to binary")
			
			CompressedSize := VarSetCapacity(DecompressionBuffer, CompressedSize, 0)
			pDecompressionBuffer := &DecompressionBuffer
			
			if !DllCall("Crypt32\CryptStringToBinary", "Str", CodeBase64, "UInt", 0, "UInt", 1
				, "Ptr", pDecompressionBuffer, "UInt*", CompressedSize, "Ptr", 0, "Ptr", 0, "UInt")
				throw Exception("Failed to convert MCLib b64 to binary")
			
			if !(pBinary := DllCall("GlobalAlloc", "UInt", 0, "Ptr", DecompressedSize, "Ptr"))
				throw Exception("Failed to reserve MCLib memory")
			
			this.LZ.Decompress(pDecompressionBuffer, CompressedSize, pBinary, DecompressedSize)
			
			return this.Load(pBinary, DecompressedSize, Symbols)
		}
	}
}
burque505
Posts: 1747
Joined: 22 Jan 2017, 19:37

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

30 Jul 2021, 11:42

:+1: @AHK_user, that's very kind of you indeed.
Thank you!
Regards,
burque505

EDIT: In my initial testing it seems to me that

Code: Select all

		WaitForLoad(DesiredState:="complete", Interval:=100)
		{
			while this.Evaluate("document.readyState").value != DesiredState
				Sleep, Interval
		}
is quite a bit more reliable. Perhaps due to increased efficiency of Evaluate(JS) with your code (and cJson of course)?
AHK_user
Posts: 523
Joined: 04 Dec 2015, 14:52
Location: Belgium

Re: [Library] cJson.ahk (version 0.1.0 pre-release)

30 Jul 2021, 15:20

By my experience this mainly has impact on the speed, certainly if the return value is large. But I assume that the speed could also impact the reliability.
geek
Posts: 1068
Joined: 02 Oct 2013, 22:13
Location: GeekDude
Contact:

Re: [Library] cJson.ahk (version 0.2.0 pre-release)

31 Jul 2021, 23:29

Version 0.2.0 has been released. Performance of Dumps should be the same as before, but Loads is benchmarking much faster than the last release in my tests.

On my laptop using https://raw.githubusercontent.com/prust/wikipedia-movie-data/master/movies.json, calling ._init() ahead of time, using #NoEnv, SetbatchLines -1, and ListLines Off, over 25 calls:
v0.2.0 completes in 0.173 seconds
v0.1.0 completes in 0.771 seconds
Coco's completes in 2.165 seconds

Without #NoEnv, SetBatchLines -1, and ListLines Off:
v0.2.0 completes in 0.206 seconds
v0.1.0 untested
Coco's completes in 6.703 seconds


Note: MCL now supports 32-bit code generation. A 32-bit cJson may follow in the next few weeks/months.
magusneo
Posts: 45
Joined: 30 Sep 2013, 06:34

Re: [Library] cJson.ahk (version 0.2.0 pre-release)

01 Aug 2021, 06:31

GeekDude wrote:
31 Jul 2021, 23:29
Version 0.2.0 has been released. Performance of Dumps should be the same as before, but Loads is benchmarking much faster than the last release in my tests.

Note: MCL now supports 32-bit code generation. A 32-bit cJson may follow in the next few weeks/months.

:thumbup: :thumbup:
jj4156
Posts: 19
Joined: 17 Jun 2019, 07:03
Contact:

Re: [Library] cJson.ahk (version 0.2.0 pre-release)

01 Aug 2021, 08:36

Very cool script, impressive. But, why not load mcode at load-time by something like 'static __init := _init()' ? I think it will make libs easier to use.
geek
Posts: 1068
Joined: 02 Oct 2013, 22:13
Location: GeekDude
Contact:

Re: [Library] cJson.ahk (version 0.2.0 pre-release)

01 Aug 2021, 08:45

jj4156 wrote:
01 Aug 2021, 08:36
Very cool script, impressive. But, why not load mcode at load-time by something like 'static __init := _init()' ? I think it will make libs easier to use.
The library is designed to be nestable into other classes, for improved scope and dependency management. In AHK v1 you can't use that trick because A_ThisFunc is not populated in the context of static declarations, and when nested the name will change to add the class hierarchy as a prefix.
User avatar
kczx3
Posts: 1677
Joined: 06 Oct 2015, 21:39

Re: [Library] cJson.ahk (version 0.2.0 pre-release)

02 Aug 2021, 18:40

Also, static initializers like that are not possible in v2 to my knowledge.
User avatar
thqby
Posts: 560
Joined: 16 Apr 2021, 11:18
Contact:

Re: [Library] cJson.ahk (version 0.2.1 pre-release)

07 Aug 2021, 03:57

I implemented it in V2, the test of 'movies.json', load 60~80ms and dump 15~32ms.
Last edited by thqby on 15 Aug 2021, 19:37, edited 1 time in total.

Return to “Scripts and Functions (v1)”

Who is online

Users browsing this forum: Bing [Bot] and 106 guests