XA_Load/Save - Saving/Loading Arrays in XML - AHK v1.1.33

Post your working scripts, libraries and tools
User avatar
TheArkive
Posts: 385
Joined: 05 Aug 2016, 08:06
GitHub: TheArkive

XA_Load/Save - Saving/Loading Arrays in XML - AHK v1.1.33

26 Feb 2020, 11:08

AHK v2 version can be found here.

Thanks to @trueski for posting this way back when, and thanks to @ahk7 for his update with XA_CleanInvalidChars().

This library converts linear and associative arrays into XML text, and takes XML text to convert into linear and associative arrays of any depth and nesting. Saving text to file is done with FileAppend and Loading text from file is done with FileRead. The user must do this in their code.

====================================================
Updates
====================================================

Code: Select all

; AHK v1
; Example ===================================================================================
; ===========================================================================================
; a := Object(), b := Object(), c := Object(), d := Object(), e := Object(), f := Object() ; Object() is more technically correct than {} but both will work.

; d["g"] := 1, d["h"] := 2, d["i"] := ["purple","pink","pippy red"]
; e["g"] := 1, e["h"] := 2, e["i"] := Object("1","test1","2","test2","3",["x","y","z"])
; f["g"] := 1, f["h"] := 2, f["i"] := [1,2,Object("a",1.0009,"b",2.0003,"c",3.0001)]

; a["test1"] := "test11", a["d"] := d
; b["test3"] := "test33", b["e"] := e
; c["test5"] := "test55", c["f"] := f

; myObj := Object()
; myObj["a"] := a, myObj["b"] := b, myObj["c"] := c, myObj["test7"] := "test77", myObj["test8"] := "test88"

; g := ["blue","green","red"], myObj["h"] := g ; add linear array for testing

; q := Chr(34)
; textData2 := XA_Save(myObj) ; ===> convert array to XML
; msgbox % "XML Breakdown:`r`n(Should match second breakdown.)`r`n`r`n" textData2


; ===========================================================================================
; Error Test - Duplicate Node ; =============================================================
; ===========================================================================================
; textData2 := StrReplace(textData2,"</Base>","<test8 type=" q "String" q ">test88</test8>`r`n</Base>") ; <--- test error, duplicate node
; ===========================================================================================
; ===========================================================================================


; newObj := XA_Load(textData2) ; ===> convert XML back to array

; textData3 := XA_Save(newObj) ; ===> break down array into 2D layout again, should be identical
; msgbox % "Second Breakdown:`r`n(should be identical to first breakdown)`r`n`r`n" textData3
; msgbox % Jxon_Dump(newObj,4)

; ExitApp
; ===========================================================================================
; End Example ; =============================================================================
; ===========================================================================================


; =============================================================
; XA_Save(Array,BaseName := "Base")		>>> dumps XML text from associative array var
; XA_Load(XMLText)						>>> converts XML text to associative array
;		Originally posted by trueski
;		in collaberation with user ahk7 contributing adding functions XA_XMLEncode() and XA_CleanInvalidChars()
;		https://autohotkey.com/board/topic/85461-ahk-l-saveload-arrays/
;		Modified by TheArkive
; - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
; Root node by default is called "Base".  It doesn't matter because it isn't
; used and is not part of the array after using XA_Load().
;
; User must choose how to handle output, ie.
;		1) Save XML to file after ... var := XA_Save()
;		2) Load XML text from file before converting with ... myArray := XA_Load()
; =============================================================
XA_Save(arr,BaseName := "Base") { ; XA_Save(Arr, BaseName := "Base")
	q := Chr(34)
	outVar := "<?xml version=" q "1.0" q " encoding=" q "UTF-8" q "?>`n<" BaseName " type=" q Type(arr) q ">`n" XA_ArrayToXML(arr) "`n</" BaseName ">"
	return OutVar
}
XA_Load(XMLText) { ; orig param was ... XA_Load(Path) ... Path = XML file location
	Local XMLObj, XMLRoot, Root1, Root2 ; XMLText
	
	XMLObj    := XA_LoadXML(XMLText)
	XMLObj    := XMLObj.selectSingleNode("/*")
	If (IsObject(XMLObj)) { ; check if settings are blank
		XMLRoot   := XMLObj.nodeName
		curType := ""
		Try curType := XMLObj.getAttribute("type")
		outputArray := XA_XMLToArray(XMLObj.childNodes,,curType)
		return outputArray
	} Else
		return "" ; if settings are blank return a blank Map()
}
XA_XMLToArray(nodes, NodeName:="", curType:="") {
	If (curType = "Array")
		Obj := []
	Else If (curType = "Map")
		Obj := Object()
	
	for node in nodes {
		if (node.nodeName != "#text") { ;NAME
			If (node.nodeName == "Invalid_Name" && node.getAttribute("ahk") == "True")
				NodeName := node.getAttribute("id")
			Else
				NodeName := node.nodeName
		} else { ;VALUE
			Obj := node.nodeValue
		}
		
		if node.hasChildNodes { 
			prevSib := ""
			Try prevSib := node.previousSibling.nodeName
			
			nextSib := ""
			Try  nextSib := node.nextSibling.nodeName
			
			nextType := ""
			Try nextType := node.getAttribute("type")
			
			If ((nextSib = node.nodeName || node.nodeName = prevSib) && node.nodeName != "Invalid_Name" && node.getAttribute("ahk") != "True") { ;Same node name was used for multiple nodes
				pN := "", cN := "", nN := ""
				Try pN := node.previousSibling.xml
				Try cN := node.xml
				Try nN := node.nextSibling.xml
				
				Throw "Duplicate node:`r`n`r`nPrev:`r`n" pN "`r`n`r`nCurrent:`r`n" cN "`r`n`r`nNext:`r`n" nN
				; If (!prevSib) { ;Create object - previous -> !node.previousSibling.nodeName
					; Obj[NodeName] := {}
					; ItemCount := 0
				; }
			  
				; ItemCount++
			  
				; If (node.getAttribute("id") != "") ;Use the supplied ID if available
					; Obj[NodeName][node.getAttribute("id")] := XA_XMLToArray(node.childNodes, node.getAttribute("id"))
				; Else ;Use ItemCount if no ID was provided
					; Obj[NodeName][ItemCount] := XA_XMLToArray(node.childNodes, ItemCount)
			} Else {
				If (curType = "Map") {
					Obj[NodeName] := XA_XMLToArray(node.childNodes, NodeName, nextType)
				} Else If (curType = "Array") {
					Obj.InsertAt(NodeName,XA_XMLToArray(node.childNodes, NodeName, nextType))
				}
			}
		}
	}
	
	return Obj
}
XA_LoadXML(ByRef data){
	o := ComObjCreate("MSXML2.DOMDocument.6.0")
	o.async := false
	o.LoadXML(data)
	return o
}
XA_ArrayToXML(theArray, tabCount:=1, NodeName:="") {
    Local tabSpace, extraTabSpace, tag, val, theXML, root, iTag, eTag, aXML, eVal, curType
	q := Chr(34)
	tabCount++
    tabSpace := "", extraTabSpace := "", curType := ""
	
	if (!IsObject(theArray)) {
		root := theArray
		theArray := %theArray%
    }
	
	While (A_Index < tabCount) {
		tabSpace .= "`t" 
		extraTabSpace := tabSpace . "`t"
    } 
     
	for tag, val in theArray {
        If (!IsObject(val)) { ; items/values
			iTag := "Invalid_Name"
			eTag := XA_XMLEncode(tag) ; xml encoded tag
			eVal := XA_XMLEncode(val)
			If (XA_InvalidTag(tag))
				theXML .= "`n" tabSpace "<" iTag " id=" q eTag q " ahk=" q "True" q " type=" q Type(val) q ">" eVal "</" iTag ">"
			Else
				theXML .= "`n" tabSpace "<" tag " type=" q Type(val) q ">" eVal "</" tag ">"
		} Else {
			iTag := "Invalid_Name"
			eTag := XA_XMLEncode(tag) ; xml encoded tag
			aXML := XA_ArrayToXML(val, tabCount, "")
			If (XA_InvalidTag(tag))
				theXML .= "`n" tabSpace "<" iTag " id=" q eTag q " ahk=" q "True" q " type=" q Type(val) q ">`n" aXML "`n" tabSpace "</" iTag ">"
			Else
				theXML .= "`n" tabSpace "<" tag " type=" q Type(val) q ">" "`n" aXML "`n" tabSpace "</" tag ">"
	    }
    } 
	
	theXML := SubStr(theXML, 2)
	Return theXML
} 
XA_InvalidTag(Tag) {
	q := Chr(34)
	Char1      := SubStr(Tag, 1, 1) 
	Chars3     := SubStr(Tag, 1, 3)
	StartChars := "~``[email protected]#$%^&*()_-+={[}]|\:;" q "'<,>.?/1234567890 `t`n`r"
	Chars := q "'<>=/ `t`n`r"
	
	Loop, Parse, StartChars
	{
		If (Char1 = A_LoopField)
		  Return 1
	}
	
	Loop, Parse, Chars
	{
		If (InStr(Tag, A_LoopField))
			Return 1
	}
	
	If (Chars3 = "xml")
		Return 1
	Else
		Return 0
}

Type(arr) {
	If (arr + 0 = arr) { ; is number
		If (arr = Format("{:d}",arr))
			return "Integer"
		Else
			return "Float"
	} If (arr.CaseSensitive != "" And arr.BackspaceIsUndo != "")
		return "InputHook"
	Else If (arr.AtEOF != "" And f.__Handle > 0)
		return "File"
	Else If (arr.IsBuiltIn != "" And arr.IsVariadic != "")
		return "Function"
	Else If (arr.__Class != "")
		return "Class"
	Else If (IsObject(arr)) {
		sequential := true
		For i, val in arr {
			
			If (i != A_Index) {
				sequential := false
				Break
			}
		}
		
		equal := false
		If (arr.Length() = arr.Count())
			equal := true
		
		If (sequential And equal)
			return "Array"
		Else If (!equal)
			return "Map"
		Else If (arr.Length() = 0 And arr.Count() > 0)
			return "Map"
		Else
			return ""
	} Else If (arr + 0 = "")
		return "String"
}


; XA_XMLEncode references:
; https://en.wikipedia.org/wiki/XML#Escaping
; https://en.wikipedia.org/wiki/List_of_XML_and_HTML_character_entity_references#Predefined_entities_in_XML
; added again as original source code posted at forum was lost due to forum upgrade 

XA_XMLEncode(Text) {
	Text := StrReplace(Text,"&","&amp;")
	Text := StrReplace(Text,"<","&lt;")
	Text := StrReplace(Text,">","&gt;")
	Text := StrReplace(Text,Chr(34),"&quot;")
	Text := StrReplace(Text,"'","&apos;")
	Return XA_CleanInvalidChars(Text)
}

XA_CleanInvalidChars(text, replace:="") {
	re := "[^\x09\x0A\x0D\x20-\x{D7FF}\x{E000}-\x{FFFD}\x{10000}-\x{10FFFF}]"
	Return RegExReplace(text, re, replace)
	
	/*
		Source: http://stackoverflow.com/questions/730133/invalid-characters-in-xml
		public static string CleanInvalidXmlChars(string text) 
		{ 
		// From xml spec valid chars: 
		// #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF]     
		// any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. 
		string re = @"[^\x09\x0A\x0D\x20-\uD7FF\uE000-\uFFFD\u10000-u10FFFF]"; 
		return Regex.Replace(text, re, ""); 
		}
	*/
}
XML uses a lot of overhead. If you need to save/load arrays/maps I recommend JSON format.

Suggested scripts:

[v2 only] Yaml and JSON parser written by HotKeyIt
YEJP - Yet Another Json Parser - AHK v1/2 written by me
Last edited by TheArkive on 17 Jul 2020, 01:02, edited 7 times in total.
User avatar
elModo7
Posts: 189
Joined: 01 Sep 2017, 02:38
GitHub: elModo7
Location: Spain
Contact:

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

26 Feb 2020, 15:38

Seems useful for some projects, I have been saving arrays into json for later use.
The one I really liked was the treeview one, man that saved so much time for a web rom indexer I did a few weeks ago.
Spoiler
I did an sqlite one, but wanted to have a listview for each system, instead of saving that on the bbdd or query again for the web, I had your treeview script keep all the info on disk and then load it from there.
It saved quite a few seconds and worked great!
:beer:
User avatar
TheArkive
Posts: 385
Joined: 05 Aug 2016, 08:06
GitHub: TheArkive

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

26 Feb 2020, 22:00

@elModo7 which library do you use for saving/loading arrays in JSON format? Sounds like something that will definitely come in handy...

I found this one initially.
Last edited by TheArkive on 15 Jul 2020, 03:07, edited 3 times in total.
User avatar
elModo7
Posts: 189
Joined: 01 Sep 2017, 02:38
GitHub: elModo7
Location: Spain
Contact:

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

27 Feb 2020, 06:18

TheArkive wrote:
26 Feb 2020, 22:00
@elModo7 which library do you use for saving/loading arrays in JSON format? Sounds like something that will definitely come in handy...

I found this one initially.
That one exactly, for saving it, I export the object to json, then I fileappend it.
For loading to object I then do fileread and json parse.

Been using it a lot for league of legends data for faster testing instead of querying the api over and over.
:beer:
ahk7
Posts: 318
Joined: 06 Nov 2013, 16:35

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

28 Feb 2020, 13:53

I added XA_CleanInvalidChars() to XA https://github.com/hi5/XA/blob/master/xa.ahk#L177 - I use XA often as it is just so easy to load/save settings.
User avatar
TheArkive
Posts: 385
Joined: 05 Aug 2016, 08:06
GitHub: TheArkive

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

18 Apr 2020, 01:27

thanks @ahk7 :-D ... not sure how I missed your reply, sorry about that.

The old forums actually replaced all the HTML encoded stuff with their text counterpart. So all I saw was:

StringReplace, Text, Text, &, &, All

... for the entire function *laughs*... I was wondering what the XML_Encode() function did. Now I know what it was supposed to be.

I'll update the OP soon. I noticed you had your version posted already a few years ago. If you want go ahead and take this code in the OP and I'll link the OP to your repo.
ahk7
Posts: 318
Joined: 06 Nov 2013, 16:35

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

18 Apr 2020, 05:02

I added links to the GH repo and forum post to this thread for a AHK v2 compatible version, having multiple versions can't hurt :)
songdg
Posts: 183
Joined: 04 Oct 2017, 20:04

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

13 Jul 2020, 04:46

Does not work on AHKv1? a problem occurs saying :call to nonexistent function of "return Map()".
User avatar
TheArkive
Posts: 385
Joined: 05 Aug 2016, 08:06
GitHub: TheArkive

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

13 Jul 2020, 12:58

@songdg
You'll need to check the comments and make the indicated changes for it to work on AHK v1. Map() is for AHK v2 only. Check comments. Should be just a few lines.

Let me know if you have issues, I'll help more.
songdg
Posts: 183
Joined: 04 Oct 2017, 20:04

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

13 Jul 2020, 23:45

TheArkive wrote:
13 Jul 2020, 12:58
@songdg
You'll need to check the comments and make the indicated changes for it to work on AHK v1. Map() is for AHK v2 only. Check comments. Should be just a few lines.

Let me know if you have issues, I'll help more.
Thank you very much!
songdg
Posts: 183
Joined: 04 Oct 2017, 20:04

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

14 Jul 2020, 04:27

another problem, MyArray:=[111111,222222,8888888], I use XA_Save(MyArray, "test.xml") to save the array, no error occurs but also can't find the file which I saved.And the use of BaseName := "Base"?
User avatar
TheArkive
Posts: 385
Joined: 05 Aug 2016, 08:06
GitHub: TheArkive

Re: XA_Load/Save - Convert Associative Array to XML and back (for saving/loading arrays) - AHKv1/2

14 Jul 2020, 05:50

@songdg
These functions don't save to file, they convert to text so that you can save to file. Try this first:

(ahk v1 right?)

var := XA_Save(MyArray)
MsgBox % var ; <--- just to test
FileAppend %var%, test.xml


edit: I'll run these functions again to see if AHK v1.1.33 and AHK v2 alphas have broken anything

EDIT: I found an error that doesn't convert a linear array back properly... trying to fix that now.
daywalker
Posts: 16
Joined: 18 Jun 2019, 01:37

Re: XA_Load/Save - Saving/Loading Arrays in XML - AHK v1.1.33 / AHK v2-a116

14 Jul 2020, 07:33

TheArkive wrote:
26 Feb 2020, 11:08

Check post #3 for AHK v1 version.
Just a proposal for AHK V1 & 2: Would it not make sense to have in AHK a #define/#ifdef-Compiler-Directives like in C ? So Code could be easier maintained?
User avatar
TheArkive
Posts: 385
Joined: 05 Aug 2016, 08:06
GitHub: TheArkive

Re: XA_Load/Save - Saving/Loading Arrays in XML - AHK v1.1.33

15 Jul 2020, 03:12

2020/07/15 Updates:
  • linear arrays are now properly supported - output XML also includes data types
  • now when a duplicate node is detected an error is thrown and the associated XML is displayed
  • created Type() function to try and detect as many types as possible (so far so good)
  • Known Issue: Empty arrays/maps can't be detected as a specific type in AHK v1, it's only detected as an object. Default for empty array is linear array.
@songdg
Updated, your array should load AND save now using these functions.
songdg
Posts: 183
Joined: 04 Oct 2017, 20:04

Re: XA_Load/Save - Saving/Loading Arrays in XML - AHK v1.1.33

16 Jul 2020, 23:44

TheArkive wrote:
15 Jul 2020, 03:12
2020/07/15 Updates:
  • linear arrays are now properly supported - output XML also includes data types
  • now when a duplicate node is detected an error is thrown and the associated XML is displayed
  • created Type() function to try and detect as many types as possible (so far so good)
  • Known Issue: Empty arrays/maps can't be detected as a specific type in AHK v1, it's only detected as an object. Default for empty array is linear array.
@songdg
Updated, your array should load AND save now using these functions.
Thanks! Something need to be change? I have a problem running this version saying "call to nonexistent function." on "DebugMsg("this shouldn't happen")".
User avatar
TheArkive
Posts: 385
Joined: 05 Aug 2016, 08:06
GitHub: TheArkive

Re: XA_Load/Save - Saving/Loading Arrays in XML - AHK v1.1.33

17 Jul 2020, 01:04

@songdg
Dang I'm having a heck of a time with this one.
Just comment out that line. I updated the OP too.
User avatar
elModo7
Posts: 189
Joined: 01 Sep 2017, 02:38
GitHub: elModo7
Location: Spain
Contact:

Re: XA_Load/Save - Saving/Loading Arrays in XML - AHK v1.1.33

17 Jul 2020, 11:27

Glad to see updates on this!
:beer:

Return to “Scripts and Functions”

Who is online

Users browsing this forum: Lorien and 22 guests