Is there a simple and reliable way to process JSON data? Topic is solved

Get help with using AutoHotkey (v1.1 and older) and its commands and hotkeys
hasantr
Posts: 933
Joined: 05 Apr 2016, 14:18
Location: İstanbul

Re: Is there a simple and reliable way to process JSON data?

16 Jan 2020, 14:33

These objects are not easy to use. I'll go with the Class method. Thanks for everything.
teadrinker
Posts: 4309
Joined: 29 Mar 2015, 09:41
Contact:

Re: Is there a simple and reliable way to process JSON data?

02 Mar 2020, 17:06

What is your Internet Explorer version number?
neogna2
Posts: 586
Joined: 15 Sep 2016, 15:44

Re: Is there a simple and reliable way to process JSON data?

07 Apr 2020, 04:26

@teadrinker: I want to parse a local .json file, read a few values and later on change a few values and then save (overwrite) the local .json file. The .json file has some values that are true / false and it is important that I write them back as true / false (not 1 / 0). Can this be done using your class ? Can you show a simple example that reads a value, changes a value and writes the changed JSON back to a file?

Sidenote: I have many times run into code snippets from you in these forums that seem to do what I want but makes me confused on exactly how they work (complex code and no code comments = hard to understand and learn from) and uncertain which is the latest version of the code and what it can do (no readme, no version numbers). It is a pity if your great code nuggets go unused because of such obstacles!
teadrinker wrote:
09 Jan 2020, 09:38
Hi
Last version:

Code: Select all

sJSON = {"Body.Date":"String."}
obj := JSON.Parse(sJSON)
for k, v in obj
   MsgBox, % "key: " . k . "`nvalue: " . v

MsgBox, % JSON.Stringify(obj,, "   ")

class JSON
{
   static JS := JSON._GetJScriptObject(), true := {}, false := {}, null := {}
   
   Parse(sJson, js := false)  {
      if jsObj := this.VerifyJson(sJson)
         Return js ? jsObj : this._CreateObject(jsObj)
   }
   
   Stringify(obj, js := false, indent := "") {
      if js
         Return this.JS.JSON.stringify(obj, "", indent)
      else {
         sObj := this._ObjToString(obj)
         Return this.JS.eval("JSON.stringify(" . sObj . ",'','" . indent . "')")
      }
   }
   
   GetKey(sJson, key, indent := "") {
      if !this.VerifyJson(sJson)
         Return
      
      try Return this.JS.eval("JSON.stringify((" . sJson . ")" . (SubStr(key, 1, 1) = "[" ? "" : ".") . key . ",'','" . indent . "')")
      catch
         MsgBox, Bad key:`n`n%key%
   }
   
   SetKey(sJson, key, value, indent := "") {
      if !this.VerifyJson(sJson)
         Return
      if !this.VerifyJson(value, true) {
         MsgBox, % "Bad value: " . value                      . "`n"
                 . "Must be a valid JSON string."             . "`n"
                 . "Enclose literal strings in quotes '' or """".`n"
                 . "As an empty string pass '' or """""
         Return
      }
      try {
         res := this.JS.eval( "var obj = (" . sJson . ");"
                            . "obj" . (SubStr(key, 1, 1) = "[" ? "" : ".") . key . "=" . value . ";"
                            . "JSON.stringify(obj,'','" . indent . "')" )
         this.JS.eval("obj = ''")
         Return res
      }
      catch
         MsgBox, Bad key:`n`n%key%
   }
   
   RemoveKey(sJson, key, indent := "") {
      if !this.VerifyJson(sJson)
         Return
      
      sign := SubStr(key, 1, 1) = "[" ? "" : "."
      try {
         if !RegExMatch(key, "(.*)\[(\d+)]$", match)
            res := this.JS.eval("var obj = (" . sJson . "); delete obj" . sign . key . "; JSON.stringify(obj,'','" . indent . "')")
         else
            res := this.JS.eval( "var obj = (" . sJson . ");" 
                               . "obj" . (match1 != "" ? sign . match1 : "") . ".splice(" . match2 . ", 1);"
                               . "JSON.stringify(obj,'','" . indent . "')" )
         this.JS.eval("obj = ''")
         Return res
      }
      catch
         MsgBox, Bad key:`n`n%key%
   }
   
   Enum(sJson, key := "", indent := "") {
      if !this.VerifyJson(sJson)
         Return
      
      conc := key ? (SubStr(key, 1, 1) = "[" ? "" : ".") . key : ""
      try {
         jsObj := this.JS.eval("(" sJson ")" . conc)
         res := jsObj.IsArray()
         if (res = "")
            Return
         obj := {}
         if (res = -1) {
            Loop % jsObj.length
               obj[A_Index - 1] := this.JS.eval("JSON.stringify((" sJson ")" . conc . "[" . (A_Index - 1) . "],'','" . indent . "')")
         }
         else if (res = 0) {
            keys := jsObj.GetKeys()
            Loop % keys.length
               k := keys[A_Index - 1], obj[k] := this.JS.eval("JSON.stringify((" sJson ")" . conc . "['" . k . "'],'','" . indent . "')")
         }
         Return obj
      }
      catch
         MsgBox, Bad key:`n`n%key%
   }
   
   VerifyJson(sJson, silent := false) {
      try jsObj := this.JS.eval("(" sJson ")")
      catch {
         if !silent
            MsgBox, Bad JSON string:`n`n%sJson%
         Return
      }
      Return IsObject(jsObj) ? jsObj : true
   }
   
   _ObjToString(obj) {
      if IsObject( obj ) {
         for k, v in ["true", "false", "null"]
            if (obj = this[v])
               Return v
            
         isArray := true
         for key in obj {
            if IsObject(key)
               throw Exception("Invalid key")
            if !( key = A_Index || isArray := false )
               break
         }
         for k, v in obj
            str .= ( A_Index = 1 ? "" : "," ) . ( isArray ? "" : """" . k . """:" ) . this._ObjToString(v)

         Return isArray ? "[" str "]" : "{" str "}"
      }
      else if !(obj*1 = "" || RegExMatch(obj, "\s"))
         Return obj
      
      for k, v in [["\", "\\"], [A_Tab, "\t"], ["""", "\"""], ["/", "\/"], ["`n", "\n"], ["`r", "\r"], [Chr(12), "\f"], [Chr(08), "\b"]]
         obj := StrReplace( obj, v[1], v[2] )

      Return """" obj """"
   }

   _GetJScriptObject() {
      static doc
      doc := ComObjCreate("htmlfile")
      doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
      JS := doc.parentWindow
      JSON._AddMethods(JS)
      Return JS
   }

   _AddMethods(ByRef JS) {
      JScript =
      (
         Object.prototype.GetKeys = function () {
            var keys = []
            for (var k in this)
               if (this.hasOwnProperty(k))
                  keys.push(k)
            return keys
         }
         Object.prototype.IsArray = function () {
            var toStandardString = {}.toString
            return toStandardString.call(this) == '[object Array]'
         }
      )
      JS.eval(JScript)
   }

   _CreateObject(jsObj) {
      res := jsObj.IsArray()
      if (res = "")
         Return jsObj
      
      else if (res = -1) {
         obj := []
         Loop % jsObj.length
            obj[A_Index] := this._CreateObject(jsObj[A_Index - 1])
      }
      else if (res = 0) {
         obj := {}
         keys := jsObj.GetKeys()
         Loop % keys.length
            k := keys[A_Index - 1], obj[k] := this._CreateObject(jsObj[k])
      }
      Return obj
   }
}
teadrinker
Posts: 4309
Joined: 29 Mar 2015, 09:41
Contact:

Re: Is there a simple and reliable way to process JSON data?

07 Apr 2020, 05:30

neogna2 wrote: The .json file has some values that are true / false and it is important that I write them back as true / false (not 1 / 0). Can this be done using your class ?
Of course.

Code: Select all

; suppose, you have read this json from the file
sJson = {"key1": true, "key2": [true, false]}

sJson := JSON.Setkey(sJson, "key1", "false", "  ") ; the last parameter is optional
MsgBox, % sJson
sJson := JSON.Setkey(sJson, "key2[0]", "'Hallo!'", "  ")
MsgBox, % sJson
; now you could write sJson to the file back
neogna2
Posts: 586
Joined: 15 Sep 2016, 15:44

Re: Is there a simple and reliable way to process JSON data?

07 Apr 2020, 07:12

Thank you. The true/false values are correctly carried over to the new file when using your JSON class. But decimal numbers with a zero, for example 790.0, get lost when reading, changing some value and writing back to file.

Code: Select all

sJson = {"x": 790.0, "y": 80.0}
MsgBox % sJson
sJson := JSON.Setkey(sJson, "y", "82.4")
MsgBox % sJson  ; expected 790.0 but the value is 790
Edit: This problem does not occur if the original JSON has quotation marks around the value 790.0 . But I want to change some values in a JSON file that does not have those quotation marks for decimal numbers and I want to if possible leave it that way, that is leave the output identical to the input except for the values I explicitly change with the Setkey method.
teadrinker
Posts: 4309
Joined: 29 Mar 2015, 09:41
Contact:

Re: Is there a simple and reliable way to process JSON data?

07 Apr 2020, 07:44

neogna2 wrote: I want to if possible leave it that way
I'm not sure if this is possible. See: Force float value when using JSON.stringify
neogna2
Posts: 586
Joined: 15 Sep 2016, 15:44

Re: Is there a simple and reliable way to process JSON data?

08 Apr 2020, 09:12

teadrinker wrote:
07 Apr 2020, 07:44
I'm not sure if this is possible. See: Force float value when using JSON.stringify
Ok, so a JSON/Javascript limitation rather than something specific to your code then. Luckily the 790.0 vs 790 issue didn't cause any problem in my use case after all so I can use your class.
hso
Posts: 7
Joined: 27 Apr 2020, 08:54

Re: Is there a simple and reliable way to process JSON data?

28 Apr 2020, 10:40

I can't seem to get the GetKey function to work as expected.

I wrote my own WebApi using standard stuff in C# and I have one Api Call that returns

Code: Select all

json = [{"NumberOfUnusedCodes":151}]
When I invoke GetKey(json, "NumberOfUnusedCodes") I get nothing. I'm fairly certain that if I remove the square brackets, it'll work, but this Api Get response is with the squared brackets. Any idea why this isn't working?
teadrinker
Posts: 4309
Joined: 29 Mar 2015, 09:41
Contact:

Re: Is there a simple and reliable way to process JSON data?

28 Apr 2020, 11:22

Code: Select all

sJson = [{"NumberOfUnusedCodes":151}]
MsgBox, % JSON.GetKey(sJson, "[0].NumberOfUnusedCodes")
hso
Posts: 7
Joined: 27 Apr 2020, 08:54

Re: Is there a simple and reliable way to process JSON data?

28 Apr 2020, 12:25

teadrinker wrote:
28 Apr 2020, 11:22

Code: Select all

sJson = [{"NumberOfUnusedCodes":151}]
MsgBox, % JSON.GetKey(sJson, "[0].NumberOfUnusedCodes")
Ahhh!! Now I'm starting to get the hang of how to use that key thing. I looked at your previous replies regarding GetKey, and I attempted "NumberOfUnusedCodes[0]" but it returned an error. This actually explained a whole bunch and I can now refactor my code to make it a lot more readable (and pretty).

I love your parser! :)
Last edited by hso on 28 Apr 2020, 13:32, edited 1 time in total.
hso
Posts: 7
Joined: 27 Apr 2020, 08:54

Re: Is there a simple and reliable way to process JSON data?

28 Apr 2020, 13:34

BoBo wrote:
28 Apr 2020, 13:21
@hso, just in case there's any use for it ... https www.mitec.cz /jsonv.html Broken Link for safety :shh: :)
I'm already using postman and I've wrapped my api in swagger, so it's actually fairly easy for me to test and get a nice visual. I just hadn't figured out how to handle that specific formatted bit of JSON.

But thanks anyway :)
User avatar
kczx3
Posts: 1640
Joined: 06 Oct 2015, 21:39

Re: Is there a simple and reliable way to process JSON data?

28 Apr 2020, 16:47

I’m not a fan of swagger at all
thomastthai
Posts: 18
Joined: 12 Mar 2020, 01:51

Re: Is there a simple and reliable way to process JSON data?

05 May 2020, 17:27

Thank you @teadrinker for your JSON class. It was helpful. I have a few questions or comments with regard to the included code below with comments.
  • When I added key33 using "key3[10]", SetKey() placed it at slot #11. That makes sense since an array starts at 0. When I added key34 using obj.key3[4], it placed it in slot 4. That also makes sense. However, the discrepancy can cause confusion. Perhaps, it could eliminate some confusion by aligning SetKey() to work similarly to the obj; i.e. using [10] with SetKey would place it in slot 10, not 11.
  • obj.key3[4] := "{'key34': 'value34'}" includes the double quotes in the value. I'm sure there is another way to assign a JSON object with curly brackets to the object key that I didn't understand.
    JSON-AHK.jpg
    JSON-AHK.jpg (14.17 KiB) Viewed 2345 times
  • AddKey(), that functions similar to SetKey() would be helpful. The only difference would be it would find an open slot to add it to without the user having to enter the slot. For example, with key33:

    Code: Select all

    sJSON := JSON.AddKey(sJSON, "key3", "{'key33': 'value33'}")
    , instead of

    Code: Select all

    sJSON := JSON.SetKey(sJSON, "key3[10]", "{'key33': 'value33'}")
    with the [10] was removed.
  • The value produced using GetKey() has double quotes around it, whereas the value retrieved from an obj (like in line 35), has no double-quotes.
  • Also, the reason "1", "2", ..., "11" are shown in the Stringify() output is because I added key35 around line #34.

Code: Select all

sJSON = 
(
	{
		"key1":	"value1", 
		"key2":	"value2", 
		"key3":	
			[
				{"key31":	"value31", "key31a": "value31a"}, 
				{"key32": "value32"}
			]
	}
)

; Does get added, but gets added to slot #11, instead of 10 since array starts at 0 count.
; This is different than key34 below using an obj. 
msgbox % A_LineNumber . ": " . sJSON := JSON.SetKey(sJSON, "key3[10]", "{'key33': 'value33'}") 
msgbox % A_LineNumber . ": " . sJSON := JSON.SetKey(sJSON, "key5", "'value5'") ; does get added
MsgBox % A_LineNumber . ": " . sJSON := JSON.SetKey(sJSON, "key6", "'✓'") ; does get added
MsgBox % A_LineNumber . ": " . sJSON := JSON.SetKey(sJSON, "key7", "null") ; does get added

msgbox % A_LineNumber . ": " . JSON.GetKey(sJSON, "key3") ; works: prints out the square brackets and everthing in between
msgbox % A_LineNumber . ": " . JSON.GetKey(sJSON, "key3[0].key31") ; works: prints "value31" <-- note the double quotes
msgbox % A_LineNumber . ": " . JSON.GetKey(sJSON, "key3[0].key31a") ; works: prints "value31a" <-- note the double quotes
msgbox % A_LineNumber . ": " . JSON.GetKey(sJSON, "key3[1]") ; works: prints {"key32": "value32"}
msgbox % A_LineNumber . ": " . JSON.GetKey(sJSON, "key3[1].key32") ; works: prints "value32" <-- note the double quotes
MsgBox % A_LineNumber . ": " . JSON.GetKey(sJSON, "key5") ; works: prints "value5" <-- note the double quotes

obj := JSON.Parse(sJSON)

; Gets added but has double quotes around the curly backets, instead of adding it as a JSON object.
; It gets added to slot #4 unlike key33
obj.key3[4] := "{'key34': 'value34'}" 
obj.key4 := "value4" ; works as expected
obj.key3.key35 := "value35" ; if this line is added, within key3, "1", "2", ..., "11" are added.

msgbox % A_LineNumber . ": " . obj.key3[1, "key31"] ; works: prints out value31 *without the double quotes*

for k, v in obj
   MsgBox, % "key: " . k . "`nvalue: " . v

MsgBox, % JSON.Stringify(obj,, "   ")

ExitApp

class JSON
{
   static JS := JSON._GetJScriptObject(), true := {}, false := {}, null := {}
   
   Parse(sJson, js := false)  {
      if jsObj := this.VerifyJson(sJson)
         Return js ? jsObj : this._CreateObject(jsObj)
   }
   
   Stringify(obj, js := false, indent := "") {
      if js
         Return this.JS.JSON.stringify(obj, "", indent)
      else {
         sObj := this._ObjToString(obj)
         Return this.JS.eval("JSON.stringify(" . sObj . ",'','" . indent . "')")
      }
   }
   
   GetKey(sJson, key, indent := "") {
      if !this.VerifyJson(sJson)
         Return
      
      try Return this.JS.eval("JSON.stringify((" . sJson . ")" . (SubStr(key, 1, 1) = "[" ? "" : ".") . key . ",'','" . indent . "')")
      catch
         MsgBox, Bad key:`n`n%key%
   }
   
   SetKey(sJson, key, value, indent := "") {
      if !this.VerifyJson(sJson)
         Return
      if !this.VerifyJson(value, true) {
         MsgBox, % "Bad value: " . value                      . "`n"
                 . "Must be a valid JSON string."             . "`n"
                 . "Enclose literal strings in quotes '' or """".`n"
                 . "As an empty string pass '' or """""
         Return
      }
      try {
         res := this.JS.eval( "var obj = (" . sJson . ");"
                            . "obj" . (SubStr(key, 1, 1) = "[" ? "" : ".") . key . "=" . value . ";"
                            . "JSON.stringify(obj,'','" . indent . "')" )
         this.JS.eval("obj = ''")
         Return res
      }
      catch
         MsgBox, Bad key:`n`n%key%
   }
   
   RemoveKey(sJson, key, indent := "") {
      if !this.VerifyJson(sJson)
         Return
      
      sign := SubStr(key, 1, 1) = "[" ? "" : "."
      try {
         if !RegExMatch(key, "(.*)\[(\d+)]$", match)
            res := this.JS.eval("var obj = (" . sJson . "); delete obj" . sign . key . "; JSON.stringify(obj,'','" . indent . "')")
         else
            res := this.JS.eval( "var obj = (" . sJson . ");" 
                               . "obj" . (match1 != "" ? sign . match1 : "") . ".splice(" . match2 . ", 1);"
                               . "JSON.stringify(obj,'','" . indent . "')" )
         this.JS.eval("obj = ''")
         Return res
      }
      catch
         MsgBox, Bad key:`n`n%key%
   }
   
   Enum(sJson, key := "", indent := "") {
      if !this.VerifyJson(sJson)
         Return
      
      conc := key ? (SubStr(key, 1, 1) = "[" ? "" : ".") . key : ""
      try {
         jsObj := this.JS.eval("(" sJson ")" . conc)
         res := jsObj.IsArray()
         if (res = "")
            Return
         obj := {}
         if (res = -1) {
            Loop % jsObj.length
               obj[A_Index - 1] := this.JS.eval("JSON.stringify((" sJson ")" . conc . "[" . (A_Index - 1) . "],'','" . indent . "')")
         }
         else if (res = 0) {
            keys := jsObj.GetKeys()
            Loop % keys.length
               k := keys[A_Index - 1], obj[k] := this.JS.eval("JSON.stringify((" sJson ")" . conc . "['" . k . "'],'','" . indent . "')")
         }
         Return obj
      }
      catch
         MsgBox, Bad key:`n`n%key%
   }
   
   VerifyJson(sJson, silent := false) {
      try jsObj := this.JS.eval("(" sJson ")")
      catch {
         if !silent
            MsgBox, Bad JSON string:`n`n%sJson%
         Return
      }
      Return IsObject(jsObj) ? jsObj : true
   }
   
   _ObjToString(obj) {
      if IsObject( obj ) {
         for k, v in ["true", "false", "null"]
            if (obj = this[v])
               Return v
            
         isArray := true
         for key in obj {
            if IsObject(key)
               throw Exception("Invalid key")
            if !( key = A_Index || isArray := false )
               break
         }
         for k, v in obj
            str .= ( A_Index = 1 ? "" : "," ) . ( isArray ? "" : """" . k . """:" ) . this._ObjToString(v)

         Return isArray ? "[" str "]" : "{" str "}"
      }
      else if !(obj*1 = "" || RegExMatch(obj, "\s"))
         Return obj
      
      for k, v in [["\", "\\"], [A_Tab, "\t"], ["""", "\"""], ["/", "\/"], ["`n", "\n"], ["`r", "\r"], [Chr(12), "\f"], [Chr(08), "\b"]]
         obj := StrReplace( obj, v[1], v[2] )

      Return """" obj """"
   }

   _GetJScriptObject() {
      static doc
      doc := ComObjCreate("htmlfile")
      doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
      JS := doc.parentWindow
      JSON._AddMethods(JS)
      Return JS
   }

   _AddMethods(ByRef JS) {
      JScript =
      (
         Object.prototype.GetKeys = function () {
            var keys = []
            for (var k in this)
               if (this.hasOwnProperty(k))
                  keys.push(k)
            return keys
         }
         Object.prototype.IsArray = function () {
            var toStandardString = {}.toString
            return toStandardString.call(this) == '[object Array]'
         }
      )
      JS.eval(JScript)
   }

   _CreateObject(jsObj) {
      res := jsObj.IsArray()
      if (res = "")
         Return jsObj
      
      else if (res = -1) {
         obj := []
         Loop % jsObj.length
            obj[A_Index] := this._CreateObject(jsObj[A_Index - 1])
      }
      else if (res = 0) {
         obj := {}
         keys := jsObj.GetKeys()
         Loop % keys.length
            k := keys[A_Index - 1], obj[k] := this._CreateObject(jsObj[k])
      }
      Return obj
   }
}
teadrinker
Posts: 4309
Joined: 29 Mar 2015, 09:41
Contact:

Re: Is there a simple and reliable way to process JSON data?

05 May 2020, 18:41

thomastthai wrote: Perhaps, it could eliminate some confusion by aligning SetKey() to work similarly to the obj; i.e. using [10] with SetKey would place it in slot 10, not 11.
Hi, thomastthai
I don't think it would be good to change the behavior of this class. Some people use it, and if it changes, their scripts will start work incorrectly. My idea was to copy JSON behavior in Javascript.
thomastthai wrote: obj.key3[4] := "{'key34': 'value34'}" includes the double quotes in the value.
It's right, since obj is not a json string in this case, it's an AHK object. To add an object as a value you shouldn't use double quotes.
thomastthai wrote: The value produced using GetKey() has double quotes around it, whereas the value retrieved from an obj (like in line 35), has no double-quotes.
If GetKey() retrieves a string, it will be enclosed in quotes, if an object, then not:

Code: Select all

sJson := "{'key1': 'string', 'key2': [1, 2, 3]}"
MsgBox, % JSON.GetKey(sJson, "key1") "`n" JSON.GetKey(sJson, "key2")
It's the way to show the difference between strings and objects.
thomastthai wrote: AddKey(), that functions similar to SetKey() would be helpful.
It makes sence, perhaps such method will be added.
Thanks for your reply! :)
thomastthai
Posts: 18
Joined: 12 Mar 2020, 01:51

Re: Is there a simple and reliable way to process JSON data?

06 May 2020, 11:34

Thank you @teadrinker for taking the time to respond!

With regard to not using the double quotes around the value and changing:

Code: Select all

obj.key3[4] := "{'key34': 'value34'}"
to

Code: Select all

obj.key3[4] := {'key34': 'value34'}
I would get an error about quote marks being required, which makes sense.
JSON-AHK error quote marks required.jpg
JSON-AHK error quote marks required.jpg (7.86 KiB) Viewed 2284 times
What I ended up doing that worked was:

Code: Select all

obj.key3[4] := {}
obj.key3[4].key34 := "value34" ; works
I'm still new to AHK. Is there a different way of doing that?

Return to “Ask for Help (v1)”

Who is online

Users browsing this forum: Google [Bot], marypoppins_1, Rohwedder, Spawnova and 150 guests