How can I compare 2 nested arrays effectively?

[fenchai]

I have 2 arrays, let's say this is the data of inventory, I am trying to compare yesterday's data with today's data.

This is the structure of the array (I can change the structure but I think this is a good starting point) so far:

Code: Select all

[
    {
        "car toy": 
            {
		nickname: "some nickname"
		height: "6 ft 4 inch"
		...
            }
    },
    {
        "yoshi toy": 
            {
		nickname: "dino toy"
		height: "2 ft 4 inch"
		...
            }
    },
    
    ...
]

• 1) First I need to separate any index that does not equal to the other:
  if 1st index's key of array1 was "car toy" but the 1st index's key from array2 was "yoshi" then it means "car toy" was eliminated and I need to extract "car toy" into a separate array to show on a listview later. Same thing if 1st index on array1 was "car toy" but 1st index on array2 was "dino toy" then it means "dino toy" was a new item added, now I need to separate "dino toy" into that separate array.

• 2) now after separating the recently added/discarded items, I need to compare "yoshi toy"'s data from array1 with array2 and see if their nickname, price, height changed.

NOTE: sometimes "yoshi toy" might have duplicates, I don't understand why the system does this, but if having duplicates complicate the code by a lot, then I can try to remove them before running the comparison code.

now I have no idea how to even begin, can AHK even get the height/nickname data without looping?

[Chunjee]

There are some deep object comparison functions, the one I am most familiar with is https://biga-ahk.github.io/biga.ahk/#/?id=isequal


Your example object was not immediately usable to me so I formatted it a little.

Code: Select all

A := new biga() ; requires https://www.npmjs.com/package/biga.ahk

car1 := {"car toy": {"nickname": "some nickname"
		, "height": "6 ft 4 inch" } }
car2 := {"car toy": {"nickname": "some nickname"
		, "height": "6 ft 4 inch" } }
dino := {"dino  toy": {"nickname": "some nickname"
		, "height": "9000 ft 1 inch" } }

msgbox, % A.isEqual(car1, car2)
; => true

msgbox, % A.isEqual(car1, dino1)
; => false

This will only return true on exact matches, for partial matches see https://biga-ahk.github.io/biga.ahk/#/?id=ismatch


Sorry I don't quite understand the questions marked 1) and 2); maybe delete the objects you don't want in the array.

[fenchai]

Sorry I don't quite understand the questions marked 1) and 2); maybe delete the objects you don't want in the array.

sorry for the spaguetti question.

I realized what I am trying to do is actually very complex but your lib is very good! I might be able to use it on this project.

I will try to find a way to compare both arrays, but as I said, if someone adds or deletes new items it gets a lot more difficult to compare them because their indexes will change. If "dino toy" was on index 4 but "dino toy 2.0" was added then the index shifts and it gets a lot more difficult to compare both arrays by using index loops, to compare it I think I would have to loop an entire array just to compare a single number, I think that would get too slow and less efficient (I was thinking on looping both arrays in a single loop but that might not be possible because the arrays will differ).

If there is any tips I will be glad to hear them.

[Chunjee]

Maybe you want to compare these arrays of inventory to see what sold? https://biga-ahk.github.io/biga.ahk/#/?id=difference should help:

subtract todays inventory from yesterday to see what disappeared (was sold)

subtract yesterdays inventory from today to see what was added (new inventory)

Code: Select all

A := new biga() ; requires https://www.npmjs.com/package/biga.ahk

yesterdayInv := [ {"car toy": {"nickname": "some nickname", "height": "6 ft 4 inch" } }, {"car toy": {"nickname": "some nickname", "height": "6 ft 4 inch" } }, {"dino  toy": {"nickname": "some nickname", "height": "9000 ft 1 inch" } }]
TodayInv := [ {"dino  toy": {"nickname": "some nickname", "height": "9000 ft 1 inch" } }, {"action fig": {"nickname": "some nickname", "height": "1 inch" } }]

whatSold := A.difference(yesterdayInv, TodayInv)
; => [{"car toy": {"height": "6 ft 4 inch", "nickname": "some nickname"}}, {"car toy": {"height": "6 ft 4 inch", "nickname": "some nickname"}}]

newArrivals := A.difference(TodayInv, yesterdayInv)
; => [{"action fig": {"height": "1 inch", "nickname": "some nickname"}}]

It sounded like maybe you wanted duplicates removed. try https://biga-ahk.github.io/biga.ahk/#/?id=uniq

[braunbaer]

Your problem becomes a lot easier if you use a unique article number, otherwise you may face problems when article data have been changed (suppose there was a typo that is corrected after a few days. Now it looks like an article has been deleted and another one is new.
Now if you first sort both arrays, you can easily check which articles are new and which articles have been deleted.

Code: Select all

ow I have no idea how to even begin, can AHK even get the height/nickname data without looping?

Why would you want to go through the array without looping. That's the normal way to proceed.

[Chunjee]

I will try to find a way to compare both arrays, but as I said, if someone adds or deletes new items it gets a lot more difficult to compare them because their indexes will change. If "dino toy" was on index 4 but "dino toy 2.0" was added then the index shifts and it gets a lot more difficult to compare both arrays by using index loops

If you are modifying the array, you should probably exit that loop and start again since the index will change. A way to do it without without that; I recommend it, is to create a new array with your modifications as you are looping. This is more of an immutable style.

At the end of the loop you can throw away or replace your old array if you are worried about memory efficiency.

[fenchai]

Your problem becomes a lot easier if you use a unique article number, otherwise you may face problems when article data have been changed (suppose there was a typo that is corrected after a few days. Now it looks like an article has been deleted and another one is new.
Now if you first sort both arrays, you can easily check which articles are new and which articles have been deleted.

That is the issue, the program I use to extract this data does not seem to use unique ids, this makes the entire process a lot more difficult than it should... I know.

I have to first get the db (which uses some weird -97 year db extension) then convert into excel file because sqlite won't support this, then I convert the entire excel file into ahk array and then I convert them into json object.

I have been thinking about using online json diff tools (which work very well) the only issue is that it being online takes a year to upload (the json file is 16mb) there are no offline tools (free at least).

yeah now you can understand my pain a little bit

[fenchai]

If you are modifying the array, you should probably exit that loop and start again since the index will change. A way to do it without without that; I recommend it, is to create a new array with your modifications as you are looping. This is more of an immutable style.

At the end of the loop you can throw away or replace your old array if you are worried about memory efficiency.

no, there won't be anyone modifying the arrays, I have to extract them everyday from an old software via this procedure:

I have to first get the db (which uses some weird -97 year db extension) then convert into excel file because sqlite won't support this, then I convert the entire excel file into ahk array and then I convert them into json object.

the above post explains a bit about how I create these arrays.

[Chunjee]

Maybe reword what you are trying to do once it's in ahk. Obviously we don't have your 97 db; if you can get that all into excel should be a hop-skip-and-jump to get it into ahk.

[braunbaer]

An idea: What about converting the file to a linear text file, one line per item tab-delimited fields, and use some standard text diff software to create a protocol of the changes?