Remove duplicates from a string Topic is solved

Get help with using AutoHotkey and its commands and hotkeys
john_c
Posts: 320
Joined: 05 May 2017, 13:19

Remove duplicates from a string

28 Aug 2019, 05:58

Is there an option to remove duplicates from a string?

The Sort is very easy to use, but I need to preserve the original order.

Code: Select all

String := "bbb aaa ccc aaa"
Sort, String, D U
MsgBox % String
gregster
Posts: 3295
Joined: 30 Sep 2013, 06:48

Re: Remove duplicates from a string  Topic is solved

28 Aug 2019, 06:22

Perhaps this (without RegEx, but using an associative array to keep track):

Code: Select all

String := "bbb aaa ccc aaa ccc ddd aaa bbb"
For i, value in StrSplit(String , A_space), uniques := {}, StringNew := ""
	if !uniques.HasKey(value)
		StringNew .= value . A_space , uniques[value] := true
MsgBox % StringNew
john_c
Posts: 320
Joined: 05 May 2017, 13:19

Re: Remove duplicates from a string

28 Aug 2019, 06:59

@gregster Thanks a lot, it works.
User avatar
jeeswg
Posts: 6842
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Remove duplicates from a string

28 Aug 2019, 10:53

Also, I posted some examples here:
jeeswg's objects tutorial - AutoHotkey Community
https://autohotkey.com/boards/viewtopic.php?f=7&t=29232

FREQUENCY COUNT (CASE INSENSITIVE) (SORT)
FREQUENCY COUNT (CASE INSENSITIVE) (MAINTAIN ORDER)
FREQUENCY COUNT (CASE SENSITIVE) (MAINTAIN ORDER/SORT)
REMOVE DUPLICATES (CASE INSENSITIVE) (MAINTAIN ORDER/SORT)
REMOVE DUPLICATES (CASE SENSITIVE) (MAINTAIN ORDER/SORT)
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
john_c
Posts: 320
Joined: 05 May 2017, 13:19

Re: Remove duplicates from a string

09 Nov 2019, 07:57

@gregster I know, this thread is from August (sorry for bumping), but could you explain the reason you use different types of syntax?

Code: Select all

if !uniques.HasKey(value)
uniques[value] := true
instead of

Code: Select all

if !uniques[value]
uniques[value] := true
? Is there some idea behind it? Something that improves the logic/readability/maintainability of the code?
swagfag
Posts: 3003
Joined: 11 Jan 2017, 17:59

Re: Remove duplicates from a string

09 Nov 2019, 09:47

one checks if a key exists in an associative array
the other fetches the value associated with the key in an associative array
the difference should be apparent
john_c
Posts: 320
Joined: 05 May 2017, 13:19

Re: Remove duplicates from a string

09 Nov 2019, 10:45

@swagfag Hmmm.

When we work with string variables, there is no difference (in AHK) between non-existing variables, variables that are assigned to False, and variables that are assigned to the empty string.

Code: Select all

; var1 doesn't exist
var2 := False
var3 := ""
MsgBox,,, % var1
MsgBox,,, % var2
MsgBox,,, % var3
Some people (including myself) trying to make this difference clear. So, instead of if (!Var2 && !Var3) we write if (!Var2 && var3 = "").

It seems that if !uniques.HasKey(value) and if !uniques[value] are related to the same issue - to make thing as explicit as possible?

* if !uniques.HasKey(value) - in plain English: "If value doesn't exist..."
* if !uniques[value] - In plain English: "If value is False..." (implicitly assuming that the value exists)

Right?
swagfag
Posts: 3003
Joined: 11 Jan 2017, 17:59

Re: Remove duplicates from a string

09 Nov 2019, 12:20

* if !uniques.HasKey(value) - in plainer English: "if the AssociativeArray/Object uniques does not contain a key, whose contents correspond to the value stored inside the variable value, ..."
* if !uniques[value] - In plainer English: "if the AssociativeArray/Object uniques either does not contain a key, whose contents correspond to the value stored inside the variable value, or the key, whose contents correspond to the value stored inside the variable value, is associated with a falsy value, ...."
guest3456
Posts: 2684
Joined: 09 Oct 2013, 10:31

Re: Remove duplicates from a string

09 Nov 2019, 21:01

this is typically better explained through code itself:

Code: Select all

assoc_array := {"one": 1, "zero": 0}

; both msgboxes are displayed
value := "two"
if !assoc_array.HasKey(value)
   MsgBox, key "%value%" doesn't exist
if !assoc_array[value]
   MsgBox, key "%value%" doesn't exist`nOR`nassoc_array["%value%"] is null/false/0/empty

; only the 2nd msgbox is displayed
value := "zero"
if !assoc_array.HasKey(value)
   MsgBox, key "%value%" doesn't exist
if !assoc_array[value]
   MsgBox, key "%value%" doesn't exist`nOR`nassoc_array["%value%"] is null/false/0/empty
in gregster's example it might not matter, but there is definitely a difference

gregster
Posts: 3295
Joined: 30 Sep 2013, 06:48

Re: Remove duplicates from a string

10 Nov 2019, 05:20

Yes, I just used true simply here as a dummy variable/value to be able to use an asscoc. array and the HasKey() method, but generally it certainly makes a difference if you look for the existence of a key or the for the value that belongs to a key.
john_c
Posts: 320
Joined: 05 May 2017, 13:19

Re: Remove duplicates from a string

10 Nov 2019, 11:29

@guest3456 Thanks a lot, excellent example. Also thanks swagfag and gregster!
Helgef
Posts: 4010
Joined: 17 Jul 2016, 01:02
Contact:

Re: Remove duplicates from a string

10 Nov 2019, 13:30

You can't do case sensitive selection with the associative array approach. Loop parse + instr,

Code: Select all

f(byref string, del := ",", casesense := false){ 
	local s := del
	loop parse, string, % del
		if !instr(s, del . a_loopfield . del, casesense)
			s .= a_loopfield . del
	return trim(s, del)
}
Cheers.
guest3456
Posts: 2684
Joined: 09 Oct 2013, 10:31

Re: Remove duplicates from a string

10 Nov 2019, 20:50

Helgef wrote:
10 Nov 2019, 13:30
You can't do case sensitive selection with the associative array approach. Loop parse + instr,

Code: Select all

f(byref string, del := ",", casesense := false){ 
	local s := del
	loop parse, string, % del
		if !instr(s, del . a_loopfield . del, casesense)
			s .= a_loopfield . del
	return trim(s, del)
}
Cheers.
why byref the string param, and why make the func assume global by declaring the first var as local?

swagfag
Posts: 3003
Joined: 11 Jan 2017, 17:59

Re: Remove duplicates from a string

10 Nov 2019, 22:56

byref to save urself having to copy the string into the arg once
the local usage eludes me. guard against someone having declared s superglobal? doesn't matter if its assume global, so long as u dont create any new variables
Helgef
Posts: 4010
Joined: 17 Jul 2016, 01:02
Contact:

Re: Remove duplicates from a string

11 Nov 2019, 01:05

thanks swagfag. Also, the use of local avoids warnings from #warn LocalSameAsGlobal.

Cheers.
guest3456
Posts: 2684
Joined: 09 Oct 2013, 10:31

Re: Remove duplicates from a string

11 Nov 2019, 01:10

swagfag wrote:
10 Nov 2019, 22:56
byref to save urself having to copy the string into the arg once
the local usage eludes me. guard against someone having declared s superglobal? doesn't matter if its assume global, so long as u dont create any new variables
interesting thanks. byref initially seemed like quite an extreme optimization simply to save one copy, but i suppose the haystack string could be quite large if we are trying to remove dupes. and yes assume global doesn't matter here, but as preference i'd rather force-local instead to achieve the goals of guarding against superglobal and #warn

john_c
Posts: 320
Joined: 05 May 2017, 13:19

Re: Remove duplicates from a string

11 Nov 2019, 07:54

I greatly appreciated Helgef for his case-sensitive implementation.

Also, here is a version by nnnik: https://www.autohotkey.com/boards/viewtopic.php?p=300598#p300598

(doesn't have case sensitivity)
User avatar
jeeswg
Posts: 6842
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Remove duplicates from a string

11 Nov 2019, 08:03

Helgef's script uses InStr, which can be really slow on large strings.
In the link that I posted above (3rd post), I have examples using a case-sensitive Scripting.Dictionary object and a case-insensitive AHK basic object.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA
john_c
Posts: 320
Joined: 05 May 2017, 13:19

Re: Remove duplicates from a string

11 Nov 2019, 09:13

@jeeswg Thanks a lot.

As I understand, you mean the examples from the jeeswg's Scripting.Dictionary mini-tutorial?
User avatar
jeeswg
Posts: 6842
Joined: 19 Dec 2016, 01:58
Location: UK

Re: Remove duplicates from a string

11 Nov 2019, 09:30

I have a code box on the thread that I linked to, and it contains the upper-case titles that I listed.

They come under GENERAL EXAMPLES.
homepage | tutorials | wish list | fun threads | donate
WARNING: copy your posts/messages before hitting Submit as you may lose them due to CAPTCHA

Return to “Ask For Help”

Who is online

Users browsing this forum: Google [Bot], MrGrayline, pythonker, QuestVBA, takayo97, ulraf and 200 guests