Facebook Icons change in size, causing ImageSearch to fail Topic is solved

[Slentzer]

edit: Turns out they were not actually changing in size but in the coloring and have a finite amount of version(2 to 4 in my case) that can be easily searched for in a loop.

I got ImageSearch working. That was hard for me. But apparently, depending on whatever parameters(any ideas?) FB icons change in size. They do change in position, hence why i bothered with ImageSearch in the first place.... but now that i found out that they ALSO change in size... i am completely lost how to proceed with my script. I am willing to learn and to look stuff up. But here i dont even know where to start.

I am testing on ONE laptop, with Win10, browser size/zoom is always at 90%, i made no changes in windows during my testing.
The settings wheel(where one changes resolution and latency) is different, from livestream to livestream. Does it have to do with the given uploader/creator/fb page?

There are also MORE than two version of the like button. The two are, the one on top of the comments and the one near the write comment field(bottom left). But of those two, there are multiple variations in sizes.

Here is my script. I have no problem to get it to work for a SPECIFIC livestream. I simply take screenshots, cut them out, the script works.
I test it in 10 other livestreams and in 9 out of 10 is does NOT work.

ANY AND ALL IDEAS ARE WELCOME. Everything that has the potential to get me to think of other solutions could lead to the solution.

Code: Select all

^!z::
sleep, % RndSleep(2000, 3000)
 Random, XRdmOffset, -40, -50
 Random, YRdmOffset, -40, -50
 Random, XRdmCenter, 776, 776
 Random, YRdmCenter, 713, 713
 Xsum := (XRdmCenter + XRdmOffset)
 Ysum := (YRdmCenter + YRdmOffset)
 MouseMove, %Xsum%, %Ysum%				;moves mouse over the video to make the settings wheel appear
sleep, % RndSleep(1000, 1500)
 CoordMode, Pixel, screen  				; Interprets the coordinates below as relative to the screen rather than the active window.
 ImageSearch, FoundX, FoundY, 0, 0, 1366, 768, *40 FBwheel2.bmp ;;;<--;;;
 if (ErrorLevel = 2)
 sleep, 3000
 else if (ErrorLevel = 1)
 sleep, 3000
 else
 {
 sleep, % RndSleep(400, 600)
 Random, XRdmOffset, 10, 20				;randomized the area that the mouse will move to and click
 Random, YRdmOffset, 10, 20
 Xsum := (FoundX + XRdmOffset)
 Ysum := (FoundY + YRdmOffset)			;Randomization finished
 MouseMove, %Xsum%, %Ysum%				;clicks the settings wheel
 sleep, % RndSleep(400, 600)
 MouseClick, left, %Xsum%, %Ysum%			;clicks the settings wheel
 sleep, % RndSleep(400, 600)
 }
sleep, % RndSleep(400, 600)
return

[Slentzer]

I found something in the AHK help:

*IconN: To use an icon group other than the first one in the file, specify *Icon followed immediately by the number of the group. For example, *Icon2 would load the default icon from the second icon group.

I read it before but instantly asumed it meant system based icons/icon groups, things one has on his drive. But maybe FB/Web pages have something similar? Icon groups that one could download and implement into the system/certain folders?

[boiler]

Code: Select all

 ImageSearch, FoundX, FoundY, 0, 0, 1366, 768, *40 FBwheel2.bmp
 if ErrorLevel
     ImageSearch, FoundX, FoundY, 0, 0, 1366, 768, *40 FBwheel3.bmp
 if ErrorLevel
     sleep, 3000
 else
 {

The if ErrorLevel result will be true when ErrorLevel is 1 or 2. No need to distinguish between 1 and 2 since you are doing the same on either result (and 2 shouldn’t happen once you have a valid command anyway).

You can do the same thing but with a loop for more than two images. Name them sequentially and use A_Index in referring to the names. Or if it’s just three, add another ImageSearch and if ErrorLevel. There’s no need to nest the if statements. It will fall through all of them once ErrorLevel is 0.

[Slentzer]

Thats a good start and certainly improved my code. But my main problem was the icon size changes themselfs.


1. I was an idiot.
1.2 I dont know where from, but i just ASUMED it was the size that was differrent.
1.3 I checked again, and the three wheel/settings icons i cut out from the screenshots are 18x18, 19x18 and 18x18. But the PIXELS within them are slightly differently colored. White in FBwheel1.bmp but grey or even black(which is the background colour) in FBwheel2 and 3.
2. Long story short changing *40 to *80 fixed it (ImageSearch, FoundX, FoundY, 0, 0, 1366, 768, *80 FBwheel%A_Index%.bmp)
3. I still need a loop and all three icons for it to work reliably. But three(or four or five) are much better than an infinite amount(which i initially asumed).

4. My current loop only breaks if i find the icon but continues looping(kinda, as it does not click the icon if i make it appear on the screen at a later point) if no icon was found. The entire script needs to be reloded for the script start hotkey to work again.

Code: Select all

^!a::
sleep, 500
CoordMode, Pixel, screen  ; Interprets the coordinates below as relative to the screen rather than the active window.
loop
 {
 sleep, 500
 ImageSearch, FoundX, FoundY, 0, 0, 1366, 768, *80 FBwheel%A_Index%.bmp
 if (ErrorLevel = 0)
  {
  sleep, % RndSleep(400, 600)
  Random, XRdmOffset, 10, 20
  Random, YRdmOffset, 10, 20
  Xsum := (FoundX + XRdmOffset)
  Ysum := (FoundY + YRdmOffset)
  MouseMove, %Xsum%, %Ysum%			;clicks wheel
  sleep, % RndSleep(400, 600)
  MouseClick, left, %Xsum%, %Ysum%		;clicks wheel
  sleep, % RndSleep(400, 600)
  break
  } 
 }
MsgBox, i am all outta loop 
sleep, 500
Return

[boiler]

Depending on the actual differences between those wheel images, you might be able to find all of them with the same reference image even without a large variation if it can be cropped to not include any background that might be changing and using the *TransN option in ImageSearch to have it ignore certain pixels that you would color over with a certain color. If you post the three images, I can tell you for sure. Even if you've solved it for this image by changing the variation to 80, it is a technique that could be very useful for other images.

[Slentzer]

I will welcome a better technique that will help me with other images, since the current solution only seems satisfying because i became SO desperate the last days

FBwheel4 is from 100% brwoser size screenshot (which i dont use), the other 3 are from 90%(which i do use), but still added for reference. If you dont need it, ignore it.

[boiler]

Well, by using *TransRed and coloring over a lot of the transition colors in a reference image based on FBWheel1.bmp, I was able to find #2 with a variation of 45 but still needed 80 to find #3. I doesn't seem like the gray colors at that much different in what I left in it, so I'm not sure what the issue is. Anyway, here's the script I used to have it cycle through the variations from 0 to 100 (or whatever you want to make the max) until it finds it:

Code: Select all

CoordMode, Pixel, Screen
Var := 0
MaxVar := 100
loop {
	Var := (A_Index - 1) * 5
	ImageSearch, FoundX, FoundY, 0, 0, 1366, 768, *TransRed *%Var% FBwheel_TransRed.bmp
} until !ErrorLevel || Var = MaxVar
MsgBox, % ErrorLevel ? ("Not found with variation up to " Var) : ("Found at " FoundX "," FoundY " with variation " Var)

[Slentzer]

Well, by using *TransRed and coloring over a lot of the transition colors in a reference image based on FBWheel1.bmp,

Can you upload the end result of your "coloring over a lot of transition" and say few words about it? What is the general idea of what you were trying to achieve?

I was able to find #2 with a variation of 45 but still needed 80 to find #3. I doesn't seem like the gray colors at that much different in what I left in it, so I'm not sure what the issue is.

I dont remeber when i found #2, but moving up in steps of 10, i found all 3 at *80 (without *TransRed, a feature i still dont understand the need/use of)

Anyway, here's the script I used to have it cycle through the variations from 0 to 100 (or whatever you want to make the max) until it finds it:

Code: Select all

CoordMode, Pixel, Screen
Var := 0
MaxVar := 100
loop {
	Var := (A_Index - 1) * 5
	ImageSearch, FoundX, FoundY, 0, 0, 1366, 768, *TransRed *%Var% FBwheel_TransRed.bmp
} until !ErrorLevel || Var = MaxVar
MsgBox, % ErrorLevel ? ("Not found with variation up to " Var) : ("Found at " FoundX "," FoundY " with variation " Var)

What is FBwheel_TransRed.bmp? Is it the edit/colored over version of one of my FBwheels?

[boiler]

Can you upload the end result of your "coloring over a lot of transition" and say few words about it? What is the general idea of what you were trying to achieve?

Here it is as a png file since the forum won't let me attach a bmp file. I prefer png files anyway (they're smaller files yet still lossless), so you can use it and just change the file name in the script to match it.

FBwheel_TransRed.png (328 Bytes) Viewed 589 times

The idea is to color over any pixels that you want the ImageSearch to ignore in a certain color. I picked pure red (0xFF0000), which is why the command includes *TransRed. I could have made it *TransFF0000 instead because it's the same. The reason you want to ignore certain pixels is that they're the ones that might vary from instance to instance, so it's just trying to match the remaining ones that aren't colored red. You can see that I colored a lot of it red. More than I initially intended to. But when I couldn't get it to find the #3 file with anything lower than a variation of 80, I tried coloring over even more pixels.

What is FBwheel_TransRed.bmp? Is it the edit/colored over version of one of my FBwheels?

Yes, I colored over the #1 image. That's the file that's attached above, although it's in a .png format as I mentioned. You could use it and just change the name to FBwheel_TransRed.png in the script.

[Slentzer]

Thats ALOT of difference between merely three icons that you had to color over.
How does it happen? Is it a thing that is caused by the nature of the given technology(dont know how/what web page icons are) or is it something that FB does on purpose?
The bearable results i achieved were with three differnet icons PLUS a variation of *80. That must mean, that the original icons are numerous and vastly different, right?
If it is something caused by the technology itself, then i can expect the same problems with other web pages in a possible future?

[boiler]

I imagine it’s caused by the mechanism that is used to draw a shape at whatever scale and position and still have it look smooth by using those various shades around the edges. It’s the same technique used for making the edges of letters look smooth, known as anti-aliasing or font smoothing. You don’t see it as much for shapes as you do for letters, but apparently it is being used here, perhaps because it’s a vector-based image that gets rendered differently in each instance based on the particular location, scale, background, etc.

[Slentzer]

I imagine it’s caused by the mechanism that is used to draw a shape at whatever scale and position and still have it look smooth by using those various shades around the edges. It’s the same technique used for making the edges of letters look smooth, known as anti-aliasing or font smoothing. You don’t see it as much for shapes as you do for letters, but apparently it is being used here, perhaps because it’s a vector-based image that gets rendered differently in each instance based on the particular location, scale, background, etc.

In the 50 instances that i tested it now, it worked out with just four icons. Some other icons ended up to need merely two versions. So all in all the hussle is minimal(yet suprising) and i dont need to learn tjhe whole Trans and colouring over stuff. THX for your help with the loop, which reduced some code/script in an already very "dirty" *.ahk file.

Thread is solved(as i see it).