[v2] Custom for-loops: Enumerators, Iterators, Generators

[iseahound]

This is a guide that explains how to create a custom for loop.

Normally we use for loops without thinking about what goes on behind the scenes. In this case we have an array, and we're iterating over the array. But the for-loop is actually calling list._Enum(). So if you replace the line for letter in list with for letter in list.__Enum() it does the same thing.

Code: Select all

list := ['A', 'B', 'C']
for letter in list
   MsgBox letter

So how do we create an enumerator object? Well here's a simple function that generates 42 over and over. Note that &var must be passed as a VarRef.

Code: Select all

for i in constant
   MsgBox i

constant(&var) => (var := 42)

First, the for loop tries to call constant.__Enum(). It fails and assumes that our function is already an enumerator object, which is what we want. Then, it evaluates var as 42, and sets the letter i to var. Therefore i is now 42.
The for-loop then determines whether it should finish or keep going. This is determined by the return value of the enumerator object. Since the last line is var:=42, and 42 evaluates as True, the loop will continue forever.


How to write enumerators by dividing them into 3 sections.

Let's write a counting function.

single line

Code: Select all

nat() => (
   ; Constants
   start := 0,
   (&n) => (
      n := start, ; yield statement
      start += 1, ; do block
      True        ; continue?
   )
)
for n in nat()
   MsgBox n

fat arrow syntax

Code: Select all

nat() {
   ; Constants
   start := 0
   return (&n) => (
      n := start, ; yield statement
      start += 1, ; do block
      True        ; continue?
   )
}
for n in nat()
   MsgBox n

Code: Select all

nat() {
   ; Constants
   start := 0
   enumerate(&n) {
      n := start  ; yield statement
      start += 1  ; do block
      return True ; continue?
   }
   return enumerate
}
for n in nat()
   MsgBox n

The yield statement tells the for loop what the current value is. In this case, the first run yields 0, because start is set to 0. The yield statement is the only time that the VarRef n is set.
The do block is where things happen. In this case we are incrementing start. Why not increment n directly? It wouldn't work because then the function would only return 1 continuously by adding 1 to start.
The last line of the function should always be a boolean. It can be a condition that needs to be evaluated.


This counting function prints out 2, 3, 4.

single line

Code: Select all

str(s) => (
   pos := 1, 
   (&char) => (
      char := SubStr(s, pos, 1), ; yield statement
      pos++,                     ; do block
      pos-1 <= StrLen(s)         ; continue?
   )
)
for char in str("kangaroo")
   MsgBox char

fat arrow syntax

Code: Select all

range(start, stop) {
   ; Constants are start and stop.
   return (&n) => (
      n := start, ; yield statement
      start += 1, ; do block
      n < stop    ; continue?
   )
}
for n in range(2, 5)
   MsgBox n

Code: Select all

range(start, stop) {
   ; Constants are start and stop.
   enumerate(&n) {
      n := start      ; yield statement
      start += 1      ; do block
      return n < stop ; continue?
   }
   return enumerate
}
for n in range(2, 5)
   MsgBox n

The constants start and stop are determined by the user to be 2 and 5. It will start at 2 and stop before it reaches 5.
The yield statement yields start as the first value.
The do block increments start by 1.
The continue? expression evaluates whether n is currently less than 5. If it is, it will return false, ending the for-loop.


Enumerate the characters of a string.

single line

Code: Select all

str(s) => (
   pos := 1,
   (&char) => (
      char := SubStr(s, pos, 1), ; yield statement
      pos++,                     ; do block
      pos-1 <= StrLen(s)         ; continue?
   )
)
for char in str("kangaroo")
   MsgBox char

fat arrow syntax

Code: Select all

str(s) {
   pos := 1
   return (&char) => (
      char := SubStr(s, pos, 1), ; yield statement
      pos++,                     ; do block
      pos-1 <= StrLen(s)         ; continue?
   )
}
for char in str("kangaroo")
   MsgBox char

Code: Select all

str(s) {
   pos := 1
   enumerate(&char) {
      char := SubStr(s, pos, 1) ; yield statement
      pos++                     ; do block
      return pos-1 <= StrLen(s) ; continue?
   }
   return enumerate
}
for char in str("kangaroo")
   MsgBox char

The constant pos is the current position in the string. Strings in AHK start from 1 and end at the string length.
The yield statement yields the first letter of the string.
The do block increments the position by 1.
The continue? expression evaluates whether the position has reached the end of the string.
In this example, the do block and continue could be combined as pos++ <= StrLen(s).

Hopefully now enumerators won't be a mystery anymore, and you will be able to write your own.

[kczx3]

It may be somewhat easier for some to understand if you just returned a nested function instead of using a fat arrow and the comma operator. But all in all, I appreciated this post! Helpful tidbits!

[iseahound]

That's a good idea. The documentation prefers the fat arrow syntax + the comma operator, so I've kept both versions but am featuring nested functions more prominently as you suggested. Thanks!

[iseahound]

Part 2. Multiple variables in a for-loop

Here's some code that uses a multiple variables in a for-loop. The following snippet will display the pixel values, alpha, red, green, and blue of an image the user selects. The only difference is that enumerate now has more VarRef parameters.

Code: Select all

#Requires AutoHotkey v2.0-beta

for A, R, G, B in image(FileSelect())
   MsgBox 'Alpha:`t' A '`nRed:`t' R '`nGreen:`t' G '`nBlue:`t' B

image(file) {
   buf := GetImageAsBuffer(file)
   start := 0
   enumerate(&A, &R, &G, &B) {
      ; yield statements
      try {
         pixel := NumGet(buf, start, "uint")
         A := (pixel & 0xFF000000) >> 24
         R := (pixel & 0x00FF0000) >> 16
         G := (pixel & 0x0000FF00) >> 8
         B := (pixel & 0x000000FF)
      }

      ; do block
      start += 4

      ; continue?
      return start <= buf.size
   }
   return enumerate
}


GetImageAsBuffer(file) {
   pToken := gdiplusStartup()

   DllCall("gdiplus\GdipCreateBitmapFromFile", "wstr", file, "ptr*", &pBitmap:=0)
   DllCall("gdiplus\GdipGetImageWidth", "ptr", pBitmap, "uint*", &width:=0)
   DllCall("gdiplus\GdipGetImageHeight", "ptr", pBitmap, "uint*", &height:=0)

   pBits := Buffer(4 * width * height, 0)

   ; Create a Scan0 buffer pointing to pBits.
   Rect := Buffer(16, 0)             ; sizeof(Rect) = 16
      NumPut(  "uint",   width, Rect,  8) ; Width
      NumPut(  "uint",  height, Rect, 12) ; Height
   BitmapData := Buffer(16+2*A_PtrSize, 0)    ; sizeof(BitmapData) = 24, 32
      NumPut(  "uint",      width, BitmapData,  0) ; Width
      NumPut(  "uint",     height, BitmapData,  4) ; Height
      NumPut(   "int",  4 * width, BitmapData,  8) ; Stride
      NumPut(   "int",   0x26200A, BitmapData, 12) ; PixelFormat
      NumPut(   "ptr",  pBits.ptr, BitmapData, 16) ; Scan0

   ; Use LockBits to fill pBits with ARGB pixel data.
   DllCall("gdiplus\GdipBitmapLockBits"
            ,    "ptr", pBitmap
            ,    "ptr", Rect
            ,   "uint", 5            ; ImageLockMode.UserInputBuffer | ImageLockMode.ReadOnly
            ,    "int", 0x26200A     ; Format32bppArgb
            ,    "ptr", BitmapData)
   DllCall("gdiplus\GdipBitmapUnlockBits", "ptr", pBitmap, "ptr", BitmapData)

   gdiplusShutdown(pToken)

   return pBits
}

gdiplusStartup() {
   DllCall("LoadLibrary", "str", "gdiplus")
   si := Buffer(A_PtrSize = 8 ? 24 : 16, 0) ; sizeof(GdiplusStartupInput) = 16, 24
      , NumPut("uint", 0x1, si)
   DllCall("gdiplus\GdiplusStartup", "ptr*", pToken:=0, "ptr", si, "ptr", 0)
   return pToken
}

gdiplusShutdown(pToken) {
   DllCall("gdiplus\GdiplusShutdown", "ptr", pToken)
   DllCall("FreeLibrary", "ptr", DllCall("GetModuleHandle", "str", "gdiplus", "ptr"))
}

[FredOoo]

Code: Select all

class CE {
	__init(){
		this.data := [ 111, 222, 333 ]
	}
	__enum( * ){
		iKey := 0
		enum_get( &key, &value ){
			iKey++
			if iKey>this.data.length
				return false
			else {
				key   := iKey
				value := this.data[iKey]
				return true
			}
		}
		return enum_get
	}
}

E := CE()
for key, value in E
	msgBox key " → " value
	
;	1 → 111
;	2 → 222
;	3 → 333

To see it in real situation, have a look at class Monitors

[Helgef]

Thank you all for sharing

I have one too, which might be interesting to some, it is possible since VarRefs were introduced. An (array) enumerator which fetches as many values as output vars per iteration, with example (and documentation )

Code: Select all

multi_val(a){
	; a - array
	a := a.__enum(1)
	return e
	e(p*) {
		while a_index <= p.length {
			a & v
			if !isset(v)
				return false
			% p[a_index] % := v
		}
		return true
	}
}

for a,b,c in multi_val([1,2,3,4,5,6,7,8,9,10,11,12])
	msgbox '3 val:`n' a '`n' b  '`n' c

for a,b,c,d,e,f in multi_val([1,2,3,4,5,6,7,8,9,10,11,12])
	msgbox '6 val:`n' a '`n' b  '`n' c '`n' d '`n' e '`n' f

It is not for general purpose.

Cheers.

[FredOoo]

Thank you too for sharing.
Can you tell me more about the syntax a & v?
I see it returns an item in v but, what means &, where can I read about it in the doc?

[Helgef]

See Reference (&) . It is just f(&x), sorry about that, I was in that kind of mood. I'll consider cleaning up the code another day.

Cheers.

[FredOoo]

Thank. I get it now, this way:
 

Code: Select all

multi_val( array )
{
	next := array.__enum(1) ; retrieves the __enum from array for 1 more value
	return enum
	enum( p* ){
		while a_index<=p.length {
			next( &value )
			if !isSet( value )
				return false
			% p[a_index] % := value
		}
		return true
	}
}

¶ := '`n'
;for a,b in multi_val( [1,2, 3,4, 5,6, 7,8, 9,10, 11,12] )
;	msgbox "2 val:" ¶   a ¶   b

for a,b,c in multi_val( [1,2,3, 4,5,6, 7,8,9, 10,11,12] )
	msgbox "3 val:" ¶   a ¶   b ¶   c

[Helgef]

Very good, thank you. Also, you can ofc pass p[a_index] directly to next (which returns 0 when there are no more values) directly, helgef error.

Cheers.

[FredOoo]

I'm afraid I don't understand what you mean by

ofc pass p[a_index] directly to next

 
But look at this one, from Python-like range function above:
I'm not sure it's valid for educational use... but it works.
 

Code: Select all

	range( start, stop )=>( &n )=>( n:=start++ )<stop
	
	for n in range( 2, 5 )
		msgBox n

	;  2
	;  3
	;  4
	

 
This one looks easyer:
 

Code: Select all

	range( start, stop ){
		enum( &n ) {
			n := start++
			return n<stop
		}
		return enum
	}
	

[Helgef]

Example,

Code: Select all

enum( p* ){
	if !next(p.removeat(1))
		return false
	while a_index<=p.length
		if !next( p[a_index] ) 
			%p[a_index]% := ''	; no more values, set to blank string, can also break to leave output params uninitialised
	return true
}

p[x] already contains a varref, so just pass it to next.

Cheers.

[FredOoo]

Yes, it runs.
Now I love the slick enum loops.
Cheers.

[iseahound]

Feel free to request a part 3. This last part deals with sending data within loops, allowing one to create a trampoline or similar. This seems quite advanced, and I have not found any common use cases so far.

[FanaticGuru]

A Range with a Step including negative Steps.

Code: Select all

Range(Start, Stop, Step:=1) {
    Enumerate(&n) {
        n := Start, Start += Step
        Return Step > 0 ? n <= Stop : n >= Stop
    }
    Return Enumerate
}


For n in range(12, 2, -3)
    MsgBox n

Is there a way to condense this into a single line with fat arrows?

FG

[Descolada]

@FanaticGuru

Code: Select all

Range(Start, Stop, Step:=1) => (&n) => (n := Start, Start += Step, Step > 0 ? n <= Stop : n >= Stop)

For n in range(12, 2, -3)
    MsgBox n

[flyingDman]

@Descolada That's a keeper...

[FanaticGuru]

@FanaticGuru

Code: Select all

Range(Start, Stop, Step:=1) => (&n) => (n := Start, Start += Step, Step > 0 ? n <= Stop : n >= Stop)

For n in range(12, 2, -3)
    MsgBox n

I could have sworn I tried that exact same code right off. Basically, just separate each step of the function with commas. For whatever reason, I must have done something different then spent 10 minutes trying all different kinds of things, mostly doing with (). But thanks for getting me straightened out.

This is a nice little helper that I might end up using in a lot of scripts.

FG

[FanaticGuru]

Here is an expansion of Range specifically for an Array.

Code: Select all

ArrayRange(Arr, Start := 1, Stop := unset, Step := 1) {
    If !IsSet(Stop)
        Stop := Arr.Length
    if Start > Stop and Step = 1
        Step := -1
    Enumerate(&p1, &p2 := false) {
        If IsSet(p2) {
            p2 := Start, Start += Step
            Try p1 := Arr[p2]
            Return Step > 0 ? p2 <= Stop : p2 >= Stop
        }
        Else {
            p1 := Start, Start += Step
            Try p2 := Arr[p1]
            Return Step > 0 ? p1 <= Stop : p1 >= Stop
        }
    }
    Return Enumerate
}

; Array
Array1 := ["One", "Two", "Three"]
Array2 := ["Four", "Five", "Six", "Seven"]
For Index, Element in ArrayRange(Array1, 2, 2)
    MsgBox "Array1 - Two Parameters from 2 to 2`n" Index "`t" Element
For Element in ArrayRange(Array1, 2)
    MsgBox "Array1 - One Parameter from 2 to End`n" Element
For Element in ArrayRange(Array1, , 2)
    MsgBox "Array1 - One Parameter from Start to 2`n" Element
For Index, Element in ArrayRange(Array2, 3, 1)
    MsgBox "Array2 - Two Parameters from 3 to 1`n" Index "`t" Element
For Index, Element in ArrayRange(Array2, 4, 2, -2)
    MsgBox "Array2 - Two Parameters from 4 to 2 step -2`n" Index "`t" Element

FG

[iseahound]

Nice! I'm a fan of the closed interval myself, since AutoHotkey is 1-based.

using Range(Start, Stop, Step:=1) => (&n) => (n := Start, Start += Step, Step > 0 ? n <= Stop : n >= Stop)

with range(1, 3) outputs 1, 2, 3

whereas in Python it would output just 1, 2.