Text Filter

[SundayProgrammer]

version star-three source code → viewtopic.php?p=411748#p411748 ← click this link

2021-7-25 added the udm feature mainly for non-alphabetic languages

---

version star-two-c source code → viewtopic.php?p=406242#p406242 ← click this link

2021-7-11 two adjustments are made, and a trivial feature is added

2021-7-7 added a feature to load the text in reverse order, by line

2021-6-30 made three tweaks

2021-6-20 added a "slim" gui

---

version star-one-b source code → viewtopic.php?p=404344#p404344 ← click this link

2021-6-13 added "3-tier search mechanism" feature

2021-6-10 fixed some missing for the new feature

2021-6-10 added "context peeking" feature

---

version star-zero-c source code → viewtopic.php?p=402780#p402780 ← click this link

2021-6-7 made three tweaks

2021-6-4 fixed a bug (existed since star-zero)

2021-6-2 made some adjustments

2021-6-1 packed almost everything into a class object

---

version planet-seven-h source code → viewtopic.php?p=393548#p393548 ← click this link

2021-5-5 added a treeview gui for "assorted matches"

2021-4-24 added an overall rightclick menu

2021-4-18 added a feature to glance committed levels and select one to fall back to

2021-4-13 added feature keys to change font size

2021-4-8 adjusted "non-word" character handling for "improvised shorthand"; fixed a new bug (regarding "matches highlighting" when only negation was entered)

2021-4-6 the tfHelp.rtf file is no longer needed

2021-4-3 made some adjustments

2021-4-2 made an adjustment (regarding "default...1000...long press...override...")

2021-4-1 added "matches highlighting" feature

---

version planet-six-a source code → viewtopic.php?p=387767#p387767 ← click this link

2021-3-21 expanded "improvised shorthand" feature

2021-3-15 added "improvised shorthand" feature

---

version planet-five-b source code → viewtopic.php?p=385997#p385997 ← click this link

2021-3-14 fixed a new bug; fixed an old bug; adjusted "select free target text" (a part of "find next match")

2021-3-9 made an adjustment

2021-3-7 added "input history" feature

---

version planet-four-b source code → viewtopic.php?p=380555#p380555 ← click this link

2021-2-10 fixed a new bug

2021-2-9 added a help page

2021-2-7 added "assorted matches" feature

---

version planet-three source code → viewtopic.php?p=377197#p377197 ← click this link

2021-1-21 added options for line numbering, and word wrap; added "select free target text" to find next match

---

version planet-two-a source code → viewtopic.php?p=376270#p376270 ← click this link

2021-1-19 fixed some new bugs

2021-1-16 added options for fayt, case insensitive, and whole word; added reverse direction for "find next match"; enabled mouse wheel for "find next match"

---

version planet-one source code → viewtopic.php?p=375412#p375412 ← click this link

2021-1-11 added "find next match" feature

---

version planet-zero (this post)

2021-1-7 (as below)

Code: Select all

; i call this: the text filter
; this is a handy little program which is
; quite useful to me. and i hope it makes
; sense to others as well.
;
; In order to grasp what it can achieve,
; try these:
;	Filter: oo{Enter}
;   Filter: the{Enter}
;	{Esc}
;	Filter: -w{Enter}
;	Filter: on{Enter}
;	{Esc}
;	{Esc}
;	{Esc}
;	Filter: oo -w on{Enter}
;	{Esc}
;	Filter: the oo{Enter}
;	{Esc}
;	Filter: as.*as{Enter}
;	Filter: [Aa]nd{Enter}
;	{Esc}
;	{Esc}
;	Filter: [Aa]nd -as.*as{Enter}
; If you've observed the changes in every
; step, you probably know how by now.
; Cheers.

gosub, init
theGui:
gui, destroy
gui, font, s18 bold, arial new
gui, add, text,, Filter:
w := a_screenwidth - 220
gui, add, edit, w%w% vNeedle%n% ym, % Needle%n%
gui, add, button, h38 default ym, Go
h := a_screenheight - 140, w := a_screenwidth - 50
gui, add, edit, readonly x20 h%h% w%w% hscroll vscroll, % Haystack%n%
gui, show
return
buttongo:
	gui, submit
	theNeedle := Needle%n%
	r := "m`a)^(?!"
	loop, parse, theNeedle, % a_space
		if substr(a_loopfield,1,1) = "-"
			r .= "(?!.*" substr(a_loopfield,2) ")"
		else
			r .= "(?=.*" a_loopfield ")"
	r .= ").*\R?"
	theHaystack := regexreplace(Haystack%n%,r)
	n += 1
	Haystack%n% := theHaystack, Needle%n% := theNeedle
	goto theGui
guicancel:
guiclose:
guiescape:
	n -= 1
	if n
		goto theGui
	else
		exitapp
init:
theHaystack=
(
The Road Not Taken
BY ROBERT FROST
Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
Then took the other, as just as fair,
And having perhaps the better claim,
Because it was grassy and wanted wear;
Though as for that the passing there
Had worn them really about the same,
And both that morning equally lay
In leaves no step had trodden black.
Oh, I kept the first for another day!
Yet knowing how way leads on to way,
I doubted if I should ever come back.
I shall be telling this with a sigh
Somewhere ages and ages hence:
Two roads diverged in a wood, and I—
I took the one less traveled by,
And that has made all the difference.
)
n = 1
Haystack%n% := theHaystack
return

[kunkel321]

This is really cool -- Thanks for sharing it!

[carno]

Excellent script and very useful.

[Deepak8811]

Please share a little background on the functionality and use case. I think it is filtering all lines with a character pattern. eg: If I type oo and press enter it will give me all lines with at least one word with double o.

[SundayProgrammer]

Please share a little background on the functionality and use case. I think it is filtering all lines with a character pattern. eg: If I type oo and press enter it will give me all lines with at least one word with double o.

i'm quite lazy in documentation, apparently ;p

anyway, i try to express it here real quick. let me put it this way, it kinda mimics a simpliest version of "google search" (though absolutely no AI at all, of course) (and strip down to only retain the most commonly use syntax elements) which find words/patterns that are wanted coexisting (but need not to concern their sequential order) and be arranged in the whole search term separated by space (just like what we all do in googleing, right?) and also can use the "minus" (or negation, i guess) sign to declare what must not existing in the result. that's it. (to me, these two google syntax elements are the most commonly use, and probably are the most useful ones to general public as well.)

by the way, since i like the idea of gradually narrowing down to the final result, so it can be just entering the search term one at a time without using the space (separator/delimiter) at all to construct the whole search term at once. that's why you see some consecutive {Esc} to go back levels in the samples i gave, coz one can search down a funnel a level by a level as one see fit.

lastly, it's a line based search engine, all about what you want (and/or don't want) existing in every line of your result.

one more thing, it can use regular expression in EACH search term separated by space.

still one more, it's a relatively short script, i consider its logic is simple (the key part is all in "buttongo" which consists just a few lines of code), and ran fairly fast.

[SundayProgrammer]

Please share a little background on the functionality and use case. I think it is filtering all lines with a character pattern. eg: If I type oo and press enter it will give me all lines with at least one word with double o.

as for "use case" goes, i mainly use it for reducing data entry workload myself, and also a quick lookup to specific information out of a huge day-to-day updating text file that i kept. since i put many aspects of personal data logged, thus copy and modify from the most similar past record is way better than all scratch, in at least two senses: faster, and consistent**.

besides, i also use it to explore certain huge file in order to grab an idea of what is it and/or what's in it for various purposes.

**side note: keep data this way not just fast and convenient, but also flexible. as for any new type of data structure, i just create it on the fly as the need arises, yet the data structure (and/or the way i put/express it for a particular type) consistency can easily be maintained. that's the way i go. not perfect, but good enough so far.

[SpeedMaster]

Very good script. Simple and convenient to use.
I liked the multi-level filtering
I've added 2 shortcuts that are essential to me:
one to open a file (ctrl+o) and another one to paste the clipboard (ctrl+v).

for those who might be interested, here are the shortcuts

Code: Select all

~^v::
Haystack1:=clipboard
n:=1
goto thegui
return

~^o::
filepath:=""
FileSelectFile, filepath
if !filepath
	return
n = 1
fileread, Haystack1, % filepath
goto thegui
return

I think it is filtering all lines with a character pattern. eg: If I type oo and press enter it will give me all lines with at least one word with double o.

yes, that's what it does and if you hit "-oo" it returns all the lines that don't contain "oo".

[SundayProgrammer]

Sometimes, we may hit a scenario that, even after filtering, the target pattern(s) are still not very easy to locate with bare eyes (may be the text is somewhat congested, or may be just because the lines are rather long.) so, this find next match feature may help a bit.

new samples enclosed.

Code: Select all

; try these:
;	Filter: w{Enter}
;	{Tab}
;	{f}
;	{f}
;	{f}
;	{f} /*as many times as you want, actually*/
;	{Esc}
;	{Tab}
;	{f}
;	{f}
;	{f}
;	{f} /*as many times as you want, actually*/
;	{Enter}
; Cheers.
gosub, init
theGui:
gui, destroy
gui, font, s18 bold, arial new
gui, add, text,, Filter:
w := a_screenwidth - 220
gui, add, edit, w%w% vNeedle%n% ym, % Needle%n%
gui, add, button, h38 default ym, Go
h := a_screenheight - 140, w := a_screenwidth - 50
gui, add, edit, readonly x20 h%h% w%w% hscroll vscroll
guicontrol,, Edit2, % Haystack%n%
strreplace(Haystack%n%, "`n",, blc)
gui, show,, The Text Filter - Level %n%   ( base %blc% lines )
return
buttongo:
	gui, submit
	theNeedle := Needle%n%
	r := "m`a)^(?!"
	loop, parse, theNeedle, % a_space
		if substr(a_loopfield,1,1) = "-"
			r .= "(?!.*" substr(a_loopfield,2) ")"
		else
			r .= "(?=.*" a_loopfield ")"
	r .= ").*\R?"
	theHaystack := regexreplace(Haystack%n%,r)
	n += 1
	Haystack%n% := theHaystack, Needle%n% := theNeedle
	goto theGui
guicancel:
guiescape:
	n -= 1
	if n
		goto theGui
	else
		exitapp
guiclose:
	exitapp
focusedcontrol() {
	controlgetfocus, foco
	return foco
}
#if winactive("The Text Filter") and focusedcontrol() = "Edit1"
	`::
	tab::controlfocus, Edit2, A
	~up::wheelup
	~down::wheeldown
	~left::wheelleft
	~right::wheelright
#if winactive("The Text Filter") and focusedcontrol() = "Edit2"
	~space::pgdn
	~+space::pgup
	~s::home
	~e::end
	~t::^home
	~b::^end
	`::
	~enter::
		controlget, cln, currentline,, Edit2, A
		hays := strsplit(Haystack%n%,"`n","`r")
		clipboard := hays[cln]
		msgbox,,, Has Put This Line In Clipboard, 3
		exitapp
	~f::
		if not Needle%n%
			controlgettext, Needle%n%, Edit1, A
		r =
		loop, parse, Needle%n%, % a_space
			if substr(a_loopfield, 1, 1) != "-"
				r .= a_loopfield "|"
		if r
		{	send, {right}
			if substr(r,0) = "|"
				r := substr(r,1,strlen(r)-1)
			controlgettext, hays, Edit2, A
			if regexmatch(hays,r)
			{	starthere:
				controlget, cln, currentline,, Edit2, A
				controlget, cco, currentcol,, Edit2, A
				cp := cln > 1 ? instr(hays,"`r`n",,,cln-1) + 1 + cco : cco
				rhays := substr(hays,cp) substr(hays,1,cp-1)
				dist := regexmatch(rhays,r) - 1
				if dist > 0
					if (dist > strlen(hays) - cp)
					{	send, ^{home}
						goto, starthere
					}else
					{	dhays := substr(rhays,1,dist)
						if instr(dhays,"`r`n")
						{	strreplace(dhays,"`r`n",,lc), dpos := strlen(dhays) - instr(dhays,"`r`n",,,lc) - 1
							send, {down %lc%}{home}{right %dpos%}
						}else send, {right %dist%}
					}
				else if (dist < 0)
					msgbox, Oops...
			}else msgbox, Target Not Found
		}
		return
#if
init:
theHaystack=
(
The Road Not Taken
BY ROBERT FROST
Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
Then took the other, as just as fair,
And having perhaps the better claim,
Because it was grassy and wanted wear;
Though as for that the passing there
Had worn them really about the same,
And both that morning equally lay
In leaves no step had trodden black.
Oh, I kept the first for another day!
Yet knowing how way leads on to way,
I doubted if I should ever come back.
I shall be telling this with a sigh
Somewhere ages and ages hence:
Two roads diverged in a wood, and I—
I took the one less traveled by,
And that has made all the difference.
)
n = 1
Haystack%n% := theHaystack
return

[burque505]

@SundayProgrammer, thank you very much for this script. I have made one tiny mod to the 'buttongo' label for my own use (case insensitive regex).

Code: Select all

	theNeedle := Needle%n%
	r := "mi`a)^(?!" ; <---- only modded line

Regards,
burque505

[kunkel321]

Now I'm wondering what {f} is... Do you just mean the f key? Also, If I press {Tab}, then the Go button is sellected. So pressing f doesn't do anything.

[SundayProgrammer]

@SundayProgrammer, thank you very much for this script. I have made one tiny mod to the 'buttongo' label for my own use (case insensitive regex).

Code: Select all

	theNeedle := Needle%n%
	r := "mi`a)^(?!" ; <---- only modded line

Regards,
burque505

it's a personal preference on the default case-sensitivity. anyhow, plz bear me to remind that, for consistency, every relevant regex (codes like regexreplace and regexmatch) statement needs to get the same adjustment. for instance, if you adopt the find next match feature, there are two regexmatch statements you may also want to touch. (to be specific, adjust for another "r" variable within "~f::").

by the way, bear me again to remind that, regardless of what default has been taken, one can always switch case-sensitivity on the spot by putting "(?i)" in front of where the switching is expected. for instance, try "(?i)a" vs "a" (or may be "(?i)A" vs "A") to see the difference.

[SundayProgrammer]

Now I'm wondering what {f} is... Do you just mean the f key? Also, If I press {Tab}, then the Go button is sellected. So pressing f doesn't do anything.

did you use the new script i gave? there i added a feature (that i called it "find next match") which enabled the "f" key to work, and in the way i mentioned. i also added some cosmetic changes there as well.

[kunkel321]

...did you use the new script i gave? ...

I had not! Pretty cool though!

[rommmcek]

Nice fuzzy search!
Here is my version:

Code: Select all

#NoEnv  ; Recommended for performance and compatibility with future AutoHotkey releases.
; #Warn  ; Enable warnings to assist with detecting common errors.
SendMode Input  ; Recommended for new scripts due to its superior speed and reliability.
SetWorkingDir %A_ScriptDir%  ; Ensures a consistent starting directory.

;https://www.autohotkey.com/boards/viewtopic.php?f=6&t=85394
; try these:
;	Filter: w
;	{Tab}
;	{f}
;	{f}
;	{f}
;	{f} /*as many times as you want, actually*/
;	{Tab}
;	{f}
;	{f}
;	{f}
;	{f} /*as many times as you want, actually*/
;	{Enter} ; not needed, except when new text was loaded or pasted to start a new filter with old needle
; Cheers.
SetBatchLines, -1
gosub, init
theGui:
gui, destroy
gui, -DPIScale
gui, font, s18, arial new
gui, add, text,, Filter:
w := a_screenwidth - 280
gui, add, edit, w%w% vNeedle gButtonGo ym, % Needle
;gui, add, button, h38 default ym, Go
Gui, font, s10
gui, add, Checkbox,  ym-2 vCase gButtonGo TabStop, Case s.
gui, add, Checkbox,  y+8 vWhole gButtonGo TabStop, Whole w.
Gui, font, s18
h := a_screenheight - 140, w := a_screenwidth - 50
gui, add, edit, readonly x20 h%h% w%w% hscroll vscroll vhStack +hwndhEdit
guicontrol,, Edit2, % Haystack
Gosub, SetWinTitle
gui, show,, The Text Filter ( base %blc% lines )
return

#if winactive("The Text Filter")
^s::
    GuiControlGet, stat,, Case
    if stat
        GuiControl,, Case, % False
    else GuiControl,, Case, % True
Return
^w::    
    GuiControlGet, stat,, Whole
    if stat
        GuiControl,, Whole, % False
    else GuiControl,, Whole, % True
Return
~Bs::
~Del::
buttongo:
    SetTimer, Go, -10
Return
Go:
	gui, submit, NoHide
	theNeedle := Needle, pos:= ""
	r := "m`a" (case? "": "i") ")^(?!"
	loop, parse, theNeedle, % a_space
		if (substr(a_loopfield,1,1) = "-")
			r .= "(?!.*" substr(a_loopfield,2) ")"
		else
			r .= "(?=.*" (whole? "\b": "") a_loopfield (whole? "\b": "") ")"
	r .= ").*\R?"
    
    theHaystack := regexreplace(theHay, r) ; seems better 
	lastHay:= Haystack := theHaystack, Needle := theNeedle
    guicontrol,, Edit2, % Haystack
    Gosub, SetWinTitle
Return
guicancel:
guiescape:
        MsgBox, 4,, Exit The Text Filter?
        ifMsgBox Yes
            exitapp
Return
guiclose:
	exitapp
focusedcontrol() {
	controlgetfocus, foco
	return foco
}
#if winactive("The Text Filter") and focusedcontrol() = "Button1"
    Left::
    Up:: controlfocus, Button2, A
#if winactive("The Text Filter") and focusedcontrol() = "Button2"
    ^Tab::
    Right::
    Down:: controlfocus, Button1, A
#if winactive("The Text Filter") and (focusedcontrol() = "Button1" || focusedcontrol() = "Button2")
    tab::controlfocus, Edit1, A
#if winactive("The Text Filter") and focusedcontrol() = "Edit1"
	`::
	tab::controlfocus, Edit2, A
	~up::wheelup
	~down::wheeldown
	;~left::wheelleft
	;~right::wheelright
  	~enter:: SetTimer, Go, -10
#if winactive("The Text Filter") and focusedcontrol() = "Edit2"
	~space::pgdn
	~+space::pgup
	~s::home
	~e::end
	~t::^home
	~b::^end

	~f::
		if not Needle
			controlgettext, Needle, Edit1, A
        Needle:= StrReplace(Needle, "^"), Needle:= StrReplace(Needle, "$") ; ... 
		r =
		loop, parse, Needle, % a_space
			if (substr(a_loopfield, 1, 1) != "-")
				r.= a_loopfield "|"
		if r {
			if (substr(r, 0) = "|")
				r:= substr(r, 1, strlen(r)-1)
			controlgettext, hays, Edit2, A
			if regexmatch(hays, (case? "": "i)") r) {
                starthere:
                dist := regexmatch(hays, (case? "": "i)") r, match, pos? pos: 1) - 1, pos:= dist+StrLen(match)
				if dist >= 0
					if (dist > strlen(hays)) {
                        pos:= ""
						goto, starthere
					} else {
                        SendMessage, 0xb1, dist, pos,, ahk_id %hEdit% ; 0xb1 - EM_SETSEL
                        SendMessage, 0xb7, 0, 0,, ahk_id %hEdit% ; 0xb7 - EM_SCROLLCARET
                        StrLen(match)>1? "": pos++
                    } 
                else {
                    pos:= 1
                    goto, starthere
                }
            } else msgbox, Target Not Found
		}
		return
#if
init:
theHay:= theHaystack:= "
(
The Road Not Taken
BY ROBERT FROST
Two roads diverged in a yellow wood,
And sorry I could not travel both
And be one traveler, long I stood
And looked down one as far as I could
To where it bent in the undergrowth;
Then took the other, as just as fair,
And having perhaps the better claim,
Because it was grassy and wanted wear;
Though as for that the passing there
Had worn them really about the same,
And both that morning equally lay
In leaves no step had trodden black.
Oh, I kept the first for another day!
Yet knowing how way leads on to way,
I doubted if I should ever come back.
I shall be telling this with a sigh
Somewhere ages and ages hence:
Two roads diverged in a wood, and I—
I took the one less traveled by,
And that has made all the difference.
)"
Haystack := theHaystack
return

#if winactive("The Text Filter") and focusedcontrol() = "Edit1"
~^v:: Return

#if winactive("The Text Filter") and focusedcontrol() = "Edit2"
~^v::
    KeyWait, Ctrl
    guicontrol,, Edit2, % theHay:= Haystack:= Clipboard
    Gosub, SetWinTitle
Return

#if winactive("The Text Filter")
~^o::
    GuiControl, Focus, hStack
    filepath:=""
    FileSelectFile, filepath
    if !filepath
        return
    fileread, Haystack, % filepath
    guicontrol,, Edit2, % theHay:= Haystack
    GuiControl, Focus, Needle
    Gosub, SetWinTitle
return

SetWinTitle:
    strreplace(RTrim(Haystack, "`r`n") "`n", "`n",, blc)
    WinSetTitle, The Text Filter,, The Text Filter ( output: %blc% lines )
Return

[Edit]: Added Hotkeys ^s, ^w for CheckBoxes, SetBatchLines, -1, inversed logic (more natural) for Case s., correction for output count and small changes.

[SundayProgrammer]

@rommmcek what a great enhancement, you made it so much better. i'm very glad to see that. THANK YOU

[garry]

@SundayProgrammer & @rommmcek works very fine ( just small adapted size to my 4K monitor )
also a short text search from user @teadrinker
;- read file to match input as you write and show it below
;- https://www.autohotkey.com/boards/viewtopic.php?f=76&t=84709

[SpeedMaster]

@rommmcek
very good improvements
if I understand well to make a multi-level search (based on the result) it is enough to separate the search terms by a space.

@garry
works well too, thanks for the reference.

[rommmcek]

There is no multi level anymore! Now it's real time! Every change of the needle searches across the whole haystack!

[garry]

have no idea how it works ... very good , useful , I like it . Also use teadrinker's script , added drag&drop function to add text-files ...

[SpeedMaster]

There is no multi level anymore! Now it's real time! Every change of the needle searches across the whole haystack!

by a "multi-level" search, I mean a step-by-step search like this

en <--- (step 1)

The Road Not Taken
To where it bent in the undergrowth;
Then took the other, as just as fair,
In leaves no step had trodden black.
Somewhere ages and ages hence:
And that has made all the difference.


en as <--- (step 2)

Then took the other, as just as fair,
And that has made all the difference.


en as oo <--- (step 3)

Then took the other, as just as fair,

SundayProgrammer has a different approach for each step/level there is a new window