Page 1 of 5

Is there a simple and reliable way to process JSON data?

Posted: 22 Jun 2019, 07:18
by afe
Hello,

I want to split the JSON data, for example, split the following example into two parts "John" and "Mike".
I simply used StrSplite for processing, but found it tricky when choosing a separator. This method may fail if the data format to be processed is slightly changed.

In the following example, I can extract the data between "data": [ and ], and then split.
However, this example may be more difficult to split, and this has not been considered in cases where John or Mike data contains ],.

This method is prone to errors and is also handled for specific JSON data.
Is there any other simple and effective way to process JSON data?

Thanks.


Code: Select all

"data":[
{
  "Name": "John",
  "phone": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ]
},
{
  "Name": "Mike",
  "phone": [
    {
      "type": "home",
      "number": "212 555-5678"
    },
    {
      "type": "mobile",
      "number": "123 456-1234"
    }
  ]
}],something

Re: Is there a simple and reliable way to process JSON data?

Posted: 22 Jun 2019, 10:47
by teadrinker
The simplest way is using javascript to parse JSON:

Code: Select all

JSON =
(Join
{"data":[
{
  "Name": "John",
  "phone": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ]
},
{
  "Name": "Mike",
  "phone": [
    {
      "type": "home",
      "number": "212 555-5678"
    },
    {
      "type": "mobile",
      "number": "123 456-1234"
    }
  ]
}]
}
)
doc := ComObjCreate("htmlfile")
doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
JS := ObjBindMethod(doc.parentWindow, "eval")
JS.( "var obj = JSON.parse('" . JSON . "');"
   . "var json1 = JSON.stringify(obj.data[0]);"
   . "var json2 = JSON.stringify(obj.data[1])" )
                           
MsgBox, % JS.("json1") . "`n`n"
        . JS.("json2")

Re: Is there a simple and reliable way to process JSON data?

Posted: 22 Jun 2019, 11:31
by afe
teadrinker wrote:
22 Jun 2019, 10:47
The simplest way is using javascript to parse JSON:
Thank you very much!
Although the code may be longer than using RegExMatch and StrSplite, it is a reliable method.
If there is an entry such as "John" and all the items in the entry are parsed in the same way, will the speed be significantly different from using StrSplite and RegExMatch?

Re: Is there a simple and reliable way to process JSON data?

Posted: 22 Jun 2019, 12:04
by swagfag
just use cocos json lib

Re: Is there a simple and reliable way to process JSON data?

Posted: 22 Jun 2019, 12:22
by afe
swagfag wrote:
22 Jun 2019, 12:04
just use cocos json lib
A bit complicated, I don't know how to use it.

Re: Is there a simple and reliable way to process JSON data?

Posted: 22 Jun 2019, 12:42
by swagfag
besides having to #Include it(i trus u know how to do this), its literally a single function call u have to make: Obj := JSON.Load(theJsonString), then u access it as if it were a normal ahk object, u know? Obj.data[0].name etc

Re: Is there a simple and reliable way to process JSON data?

Posted: 22 Jun 2019, 14:52
by teadrinker
If you sure that json does not contain square brackets and braces as literals, you could using RegEx:

Code: Select all

json =
(
"data":[
{
  "Name": "John",
  "phone": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ]
},
{
  "Name": "Mike",
  "phone": [
    {
      "type": "home",
      "number": "212 555-5678"
    },
    {
      "type": "mobile",
      "number": "123 456-1234"
    }
  ]
}],something
)
RegExMatch(json, "x)""data"":\s*\K ( \[ (?: [^[\]]++ | (?-1) )* ] )", data)
MsgBox, % data
while RegExMatch(data, "xO)( \{ (?: [^{}]++ | (?-1) )* } )", person, person ? person.Pos + person.Len : 1)
   MsgBox, % person[0]

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 00:01
by afe
teadrinker wrote:
22 Jun 2019, 10:47
The simplest way is using javascript to parse JSON:
What if I want to get the value of "Name" for data[0]? I don't know much about JS.

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 00:02
by afe
swagfag wrote:
22 Jun 2019, 12:42
besides having to #Include it(i trus u know how to do this), its literally a single function call u have to make: Obj := JSON.Load(theJsonString), then u access it as if it were a normal ahk object, u know? Obj.data[0].name etc
Yes, but I have to learn more about its usage.

Re: Is there a simple and reliable way to process JSON data?  Topic is solved

Posted: 24 Jun 2019, 04:26
by teadrinker
afe wrote: What if I want to get the value of "Name" for data[0]? I don't know much about JS.
It's simple:

Code: Select all

JSON =
(Join
{"data":[
{
  "Name": "John",
  "phone": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ]
},
{
  "Name": "Mike",
  "phone": [
    {
      "type": "home",
      "number": "212 555-5678"
    },
    {
      "type": "mobile",
      "number": "123 456-1234"
    }
  ]
}]
}
)
doc := ComObjCreate("htmlfile")
doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
JS := ObjBindMethod(doc.parentWindow, "eval")
JS.("var obj = JSON.parse('" . JSON . "');")
                           
MsgBox, % JS.("obj.data[0].Name")
swagfag wrote: just use cocos json lib
I don't like using such libraries because it causes some issues. For example I need to change the value of "key3" here:

Code: Select all

sJson = {"key1": true, "key2": null, "key3": "test"}
ahkObj := JSON.Load(sJson)
ahkObj.key3 := "AHK"
MsgBox, % newJson := JSON.Dump(ahkObj)  ; {"key1":1,"key2":"","key3":"AHK"}
However:

Code: Select all

sJson = {"key1": true, "key2": null, "key3": "test"}

doc := ComObjCreate("htmlfile")
doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
JS := ObjBindMethod(doc.parentWindow, "eval")
JS.( "var obj = JSON.parse('" . sJson . "');"
   . "obj.key3 = 'AHK';" )
MsgBox, % JS.("JSON.stringify(obj)")    ; {"key1":true,"key2":null,"key3":"AHK"}

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 05:36
by swagfag
@afe if uve ever used an ahk associative array, u already know how to use it. theres nothing new to learn
@teadrinker i didnt know about this error. it looks like he has some things builtin to address that(JSon.Undefined) but havent been able to figure out what to do with them yet. i dont even know whether its at all possible for something like this to be fixed, since AHK doesnt have a dedicated null type
ur method produces correct results but is a pain in the ass to use

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 05:48
by teadrinker
@swagfag
What is this pain like? :) It's your choise: to get a wrong result or get a pain in the ass.

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 06:08
by afe
teadrinker wrote:
24 Jun 2019, 04:26
It's simple:
Great! This is a reliable and simple method. Thank you very much!
I should go to learn JavaScript. This can be very helpful when dealing with some web data.

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 06:21
by teadrinker
A simpler way:

Code: Select all

JSON =
(Join
{"data":[
{
  "Name": "John",
  "phone": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ]
},
{
  "Name": "Mike",
  "phone": [
    {
      "type": "home",
      "number": "212 555-5678"
    },
    {
      "type": "mobile",
      "number": "123 456-1234"
    }
  ]
}]
}
)
doc := ComObjCreate("htmlfile")
doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
MsgBox, % doc.parentWindow.eval("(" . JSON . ").data[0].Name")

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 06:26
by A_AhkUser
This syntax is more ahk-like (the only discordant factor is that one have to be careful when mixing both litteral dotted and between brackets property syntax while getting/setting an item):

Code: Select all

(doc:=ComObjCreate("htmlfile")).write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
parse := ObjBindMethod(doc.parentWindow.JSON, "parse"), stringify := ObjBindMethod(doc.parentWindow.JSON, "stringify")

JSON = {"key1": true, "key2": null, "key3": "test", "key4": [ 1, 2, 3 ]}

obj := parse.(JSON)
obj.key3 := "AHK"
MsgBox % stringify.(obj,, A_Tab)
MsgBox % json2 := stringify.(obj.key4.0)
var := 0
MsgBox % json2 := stringify.((obj.key4)[ var ]) ; see: https://www.autohotkey.com/docs/Objects.htm#Usage_Objects (> known limitation))

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 06:30
by swagfag
@teadrinker
the pain in the ass is that u have to juggle strings back and forth every time u want to do anything useful
can u rewrite this? i need an ahk array of all phone numbers

Code: Select all

AllPhoneNumbers := []
for each, Person in Obj.data
	for each, Phone in Person
		AllPhoneNumbers.Push(Phone.number)
yeah, think im gonna go with the simpler, faster, and admittedly occasionally wrong, method and a pain-free ass

@afe learning another language entirely is clearly simpler than applying what u already knew... bizarro ahk world

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 06:43
by A_AhkUser
You can write a enumerator:

Code: Select all

sJSON = {"key1": true, "key2": null, "key3": "test", "phoneNumber": [ "15635855", "225822224", "3363576347" ]}

obj := JSON.parse.(sJSON)
obj.key3 := "AHK"
MsgBox % JSON.stringify.(obj,, A_Tab)
MsgBox % json2 := JSON.stringify.(obj.phoneNumber.0)
var := 0
MsgBox % json2 := JSON.stringify.((obj.phoneNumber)[ var ]) ; see: https://www.autohotkey.com/docs/Objects.htm#Usage_Objects (> known limitation)

for each, person in new JSON(obj.phoneNumber)
	MsgBox % each "," person

Class JSON {
	i := -1
	static doc := ComObjCreate("htmlfile")
	__New(_collection) {
	static _ := JSON.doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
	return this, this.count := (this.keys:=JSON.doc.parentWindow.Object.keys(this.collection:=_collection).slice()).length
	}
	_NewEnum() {
	return this
	}
	next(ByRef _k:="", ByRef _v:="") {
	return (++this.i < this.count) ? (true, _k := (this.keys)[ this.i ], _v := (this.collection)[_k]) : (false, this.i:=-1)
	}
	static parse := ObjBindMethod(JSON.doc.parentWindow.JSON, "parse")
	static stringify := ObjBindMethod(JSON.doc.parentWindow.JSON, "stringify")
}

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 07:15
by swagfag
ah yes, enumerators are the fir- no, scratch that. AHK enumerators are the first thing that pops into one's head when they hear the word "simple"
i wanted to use it on
his data
do i have to write another enumerator for that?

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 07:31
by teadrinker
swagfag wrote: learning another language entirely is clearly simpler than applying what u already knew
Since JSON is a Javascript-based form, Javascript knowledge is sometimes required to work with it.
swagfag wrote: ah yes, enumerators are the fir- no, scratch that. AHK enumerators are the first thing that pops into one's head when they hear the word "simple"
Do you consider "cocos json lib" as simple?
swagfag wrote: do i have to write another enumerator for that?
I'm not sure that @A_AhkUser's class works properly, try mine:

Code: Select all

sJSON =
(Join
{"data":[
{
  "Name": "John",
  "phone": [
    {
      "type": "home",
      "number": "212 555-1234"
    },
    {
      "type": "mobile",
      "number": "123 456-7890"
    }
  ]
},
{
  "Name": "Mike",
  "phone": [
    {
      "type": "home",
      "number": "212 555-5678"
    },
    {
      "type": "mobile",
      "number": "123 456-1234"
    }
  ]
}]
}
)
Obj := JSON.Parse(sJSON)
MsgBox, % Obj.Data[1, "Name"]

AllPhoneNumbers := []
for each, Person in Obj.data
   for each, Phone in Person.phone
      AllPhoneNumbers.Push(Phone.number)
   
for k, v in AllPhoneNumbers
   MsgBox, % v

class JSON
{
   static JS := JSON._GetJScripObject()
   
   Parse(JsonString)  {
      try oJSON := this.JS.("(" JsonString ")")
      catch  {
         MsgBox, Wrong JsonString!
         Return
      }
      Return this._CreateObject(oJSON)
   }
   
   Stringify(obj) {
      sObj := this._ObjToString(obj)
      Return this.JS.("JSON.stringify(" . sObj . ")")
   }
   
   _ObjToString(obj) {
      if IsObject( obj )  {
         isArray := true
         for key in obj {
            if IsObject(key)
               throw Exception("Invalid key")
            if !( key = A_Index || isArray := false )
               break
         }
         for k, v in obj
            str .= ( A_Index = 1 ? "" : "," ) . ( isArray ? "" : k . ":" ) . this._ObjToString(v)

         Return isArray ? "[" str "]" : "{" str "}"
      }
      else if !(obj*1 = "" || RegExMatch(obj, "\s"))
         Return obj
      else
         Return """" obj """"
   }

   _GetJScripObject()  {
      static doc
      doc := ComObjCreate("htmlfile")
      doc.write("<meta http-equiv=""X-UA-Compatible"" content=""IE=9"">")
      JS := ObjBindMethod(doc.parentWindow, "eval")
      JSON._AddMethods(JS)
      Return JS
   }

   _AddMethods(ByRef JS)  {
      JScript =
      (
         Object.prototype.GetKeys = function () {
            var keys = []
            for (var k in this)
               if (this.hasOwnProperty(k))
                  keys.push(k)
            return keys
         }
         Object.prototype.IsArray = function () {
            var toStandardString = {}.toString
            return toStandardString.call(this) == '[object Array]'
         }
      )
      JS.(JScript)
   }

   _CreateObject(ObjJS)  {
      res := ObjJS.IsArray()
      if (res = "")
         Return ObjJS
      
      else if (res = -1)  {
         obj := []
         Loop % ObjJS.length
            obj[A_Index] := this._CreateObject(ObjJS[A_Index - 1])
      }
      else if (res = 0)  {
         obj := {}
         keys := ObjJS.GetKeys()
         Loop % keys.length
            k := keys[A_Index - 1], obj[k] := this._CreateObject(ObjJS[k])
      }
      Return obj
   }
}

Re: Is there a simple and reliable way to process JSON data?

Posted: 24 Jun 2019, 08:06
by swagfag
@teadrinker yep u right, json is a javascript based format. except for being aware of the fact that arrays are 0-based, zero javascript knowledge is required for u to start working with the object returned by JSON.Load. only ahk knowledge is required
i think the json lib is simple, yes. u only need to learn 3 commands(one of which u should know already anyway): #include, Json.Load(jsonString), Json.Dump(ahkObject). now u can do whatever, reassign, insert new keys, iterate with foreach - basic ahk stuff. versus:

Code: Select all

JS.( "var obj = JSON.parse('" . JSON . "');"
   . "var json1 = JSON.stringify(obj.data[0]);"
   . "var json2 = JSON.stringify(obj.data[1])" )
u want to iterate? oh, well, i hope u know what this does, or how to even write it in the first place:

Code: Select all

Object.prototype.GetKeys = function () {
            var keys = []
            for (var k in this)
               if (this.hasOwnProperty(k))
                  keys.push(k)
            return keys
         }
         Object.prototype.IsArray = function () {
            var toStandardString = {}.toString
            return toStandardString.call(this) == '[object Array]'
         }

i didnt know u had this lib. if u had shared it, i wouldnt have bothered recommending coco's.
but it looks like urs doesnt handle the null case either, so back to square 1? are the options only simple+wrong and reliable+pita?