Can you perform a math function on a RegExReplace backreference before replacing?

[gwarble]

Hey all,

I'm new to RegEx, trying to learn all the ins and outs... In this case, I'm extracting the feedrates from a G-code file, and I want to take the existing value (ie: F10.0 means a speed of ten inches per minute) and multiply it by some factor. This would be easy with normal variables, but I can't seem to do anything useful with the backreferences ($0, $1, etc) except treat it as a string. So in this example, instead of turning F10.0 into <F10.0> as shown, I want to turn it into F14.0

Code: Select all

FeedrateOverride:
 Override := 1.40
 string := "G0G90X1.2Y3.4F10.0M8"
 result := RegExReplace(string,"(?:F)([\-\d\.]+)", "<F$1>" ) ;"F" . ("$1")*Override doesn't work, Func($1) doesn't either
 MsgBox, %result%
Return

I hope that makes sense, and any help greatly appreciated, thanks
- Joel

[iPhilip]

Hi Joel,

I am not 100% sure this helps, but I would use RegexMatch instead of RegexReplace as follows:

Code: Select all

FeedrateOverride:
   Override := 1.40
   string := "G0G90X1.2Y3.4F10.0M8"
   if RegExMatch(string, "O)F([\-\d\.]+)", Match)
      MsgBox, % Format("F{:.1f}", Override * Match[1])  ; F14.0
Return

Cheers!

- iPhilip

[gwarble]

thanks, I'm experimenting with that now, and while this seems like a path to success, it sure seems like RegExReplace is super close and I was hoping I was just missing something...

The problem I'm running into now with a RegExMatch loop is when I go to replace the F10.0 with the newly computed F14.0 there's a possibility that the StringReplace will pick up one already multiplied (ie in the example here:)

Code: Select all

 Override := 140
 string := "G00G90X1.2Y3.4F14.00M8`nG01G90X2.2Y3.4F19.60`nG01G90X2.2Y3.4F14.00"
 pos = 1
 While pos := RegExMatch(string,"O)(?:F)([\-\d\.]+)", match, pos+1)
  StringReplace, string, string, % match.0, % "F" Round(match.1 * Override / 100, places)
 MsgBox, %string%

As you can see, first F14.00 is replaced with F19.60, then the second F19.60 tries to replace with with 27.44 but it accidently replaces the first again!... then the third accidently modifies the second etc... so then I'd have to replace in the string at a specific position of the match? otherwise I could do the regex match within a newline-parsing loop i guess, but some commands (not really F) will allow more than one on the same line so still not ideal...

If i could just do "F" ("$1"*1.4) in the RegExReplace it would be one line to replace all feedrates within the file

[gwarble]

looping line by line mostly solves that problem, and allows flexibility for other stuff that I don't know how to do yet with pure RegEx like skip lines with G84 in them and keep the feedrate unchanged, or skip F's within comments (F10.0):

Code: Select all

 Override := 140
 string := "G00G90X1.2Y3.4F10.00M8`nG01G90X2.2Y3.4F20.009`nG01G90X2.2Y3.4(F10.00)`nG84G92X1.2Y3.4F30."


newString =
Loop, Parse, string,  `n, `r
{
 pos = 1
 string_ := A_LoopField
 If !InStr(string_, "G84")
  While pos := RegExMatch(string_,"O)(?:F)([\-\d\.]+(?![^\(]*\)))", match, pos+1)
   StringReplace, string_, string_, % match.0, % "F" Round(match.1 * Override / 100, SigDig(match.1))
 Newstring .= string_ "`n"
}

 MsgBox, %NewString%


ExitApp

SigDig(n) {
 if (!RegExMatch(n, "^\d*\.(\d*?)0*$", m))
        return 2
    return StrLen(m1) = 0 ? 1 : StrLen(m1)
}

[iPhilip]

Now I better understand what you are trying to do. The code below should work. It takes advantage of the Limit and StartingPosition parameters in RegExReplace.

Code: Select all

 Override := 140
 string := "G00G90X1.2Y3.4F14.00M8`nG01G90X2.2Y3.4F19.60`nG01G90X2.2Y3.4F14.00"
 pos := 1
 places := 2
 While pos := RegExMatch(string,"O)F([\-\d\.]+)", match, pos+1)
   string := RegExReplace(string, match.1, Round(match.1 * Override / 100, places), , 1, pos)
 MsgBox, %string%

Note that the above won't work properly if the string begins with an F number.

[gwarble]

I see... so you're iterating through with RegExMatch like before, but using RegExReplace as a more advanced StringReplace where you can specify the position to ensure replacing the right instance is replaced. Makes sense, and can still benefit from looping by line rather than matching the entire string (which in the example is only 3 lines, but in practice will be thousands of lines, so a speed comparison will be interesting)

thanks for the help! i still am not understanding the reason that a "backreference" can't be used or transfered to a variable somehow... but this seems like a functional approach

[tmplinshi]

Code: Select all

string := "G00G90X1.2Y3.4F14.00M8`nG01G90X2.2Y3.4F19.60`nG01G90X2.2Y3.4F14.00"
MsgBox, % RegExReplaceF(string, "O)F([\d.-]+)", "Add")
MsgBox, % RegExReplaceF(string, "F\K[\d.-]+", "Add2")

Add(match) {
	return "F" Round(match.1 + 4, 2)
}

Add2(match) {
	return Round(match + 4, 2)
}

RegExReplaceF(ByRef Haystack, NeedleRegEx, FunctionName, ByRef OutputVarCount := "", Limit := -1, StartingPosition := 1) {
	VarSetCapacity(out, StrLen(Haystack) * (A_IsUnicode+1))
	OutputVarCount := 0
	while ( pos := RegExMatch(Haystack, NeedleRegEx, match, StartingPosition) )
	{
		out .= SubStr(Haystack, StartingPosition, pos-StartingPosition)
		out .= %FunctionName%(match)
		len := IsObject(match) ? match.Len : StrLen(match)
		StartingPosition := pos + len
		if (++OutputVarCount = Limit)
			break
	}
	return out . SubStr(Haystack, StartingPosition)
}

[gwarble]

very nice, thanks... modified a little for my specific use-case:
match decimal places to the original number, down to 1 place at the least
skip F commands within parenthesis (comments in g-code) with (?![^\(]*\)))

How would you recommend incorporating the logic "if there is a G84 on the same line as the F, don't Override it"

Code: Select all

string := "G00G90X1.2Y3.4F14.M8`nG01G90X2.2Y3.4F19.6005`nG01G90X2.2Y3.4F14.00(WAS F14.00)`nG0G91G28Z0`n"

MsgBox, % RegExReplaceF(string, "O)F([\-\d.]+(?![^\(]*\)))", "Override")

Override(match) {
	Return "F" Round(match.1 * 1.4, DecimalPlaces(match.1))
}

RegExReplaceF(ByRef Haystack, NeedleRegEx, FunctionName, ByRef OutputVarCount := "", Limit := -1, StartingPosition := 1) {
	VarSetCapacity(out, StrLen(Haystack)*2)
	OutputVarCount := 0
	while ( pos := RegExMatch(Haystack, NeedleRegEx, match, StartingPosition) )
	{
		out .= SubStr(Haystack, StartingPosition, pos-StartingPosition)
		out .= %FunctionName%(match)
		len := IsObject(match) ? match.Len : StrLen(match)
		StartingPosition := pos + len
		if (++OutputVarCount = Limit)
			break
	}
	return out . SubStr(Haystack, StartingPosition)
}

DecimalPlaces(n) {
 if (!RegExMatch(n, "^\d*\.(\d*)$", m))
        return 2
    return StrLen(m1) = 0 ? 1 : StrLen(m1)
}

[tmplinshi]

How would you recommend incorporating the logic "if there is a G84 on the same line as the F, don't Override it"

Code: Select all

string := "G00G90X1.2Y3.4F14.M8`nG01G90X2.2Y3.4F19.6005`nG84G90X2.2Y3.4F14.00(WAS F14.00)`nG0G91G28Z0`n"
MsgBox, % RegExReplaceF(string, "O)([^\r\n]*?)F([\-\d.]+(?![^\(]*\)))", "Override")

Override(match) {
	if InStr(match.1, "G84")
		return match.0
	else
		return match.1 "F" Round(match.2 * 1.4, DecimalPlaces(match.2))
}

RegExReplaceF(ByRef Haystack, NeedleRegEx, FunctionName, ByRef OutputVarCount := "", Limit := -1, StartingPosition := 1) {
	VarSetCapacity(out, StrLen(Haystack)*2)
	OutputVarCount := 0
	while ( pos := RegExMatch(Haystack, NeedleRegEx, match, StartingPosition) )
	{
		out .= SubStr(Haystack, StartingPosition, pos-StartingPosition)
		out .= %FunctionName%(match)
		len := IsObject(match) ? match.Len : StrLen(match)
		StartingPosition := pos + len
		if (++OutputVarCount = Limit)
			break
	}
	return out . SubStr(Haystack, StartingPosition)
}

DecimalPlaces(n) {
 if (!RegExMatch(n, "^\d*\.(\d*)$", m))
        return 2
    return StrLen(m1) = 0 ? 1 : StrLen(m1)
}

[gwarble]

awesome, thank you

one more thing if i still have your interest, how would you go about having a parameter passed to the Function Override...
RegExReplaceF(string, "O)(.*?)F([\-\d.]+(?![^\(]*\)))", "Override(140)")

to prevent needing a global variable like so:

Code: Select all

string := "G00G90X1.2Y3.4F14.M8`nG01G90X2.2Y3.4F19.6005`nG84G90X2.2Y3.4F14.00(WAS F14.00)`nG0G91G28Z0`n"

Override := 140 ;percent
MsgBox, % RegExReplaceF(string, "O)([^\r\n]*?)F([\-\d.]+(?![^\(]*\)))", "Override")

Override(match) {
	global Override
	if InStr(match.1, "G84")
		return match.0
	else
		return match.1 "F" Round(match.2 * Override/100, DecimalPlaces(match.2))
}

RegExReplaceF(ByRef Haystack, NeedleRegEx, FunctionName, ByRef OutputVarCount := "", Limit := -1, StartingPosition := 1) {
	VarSetCapacity(out, StrLen(Haystack)*2)
	OutputVarCount := 0
	while ( pos := RegExMatch(Haystack, NeedleRegEx, match, StartingPosition) )
	{
		out .= SubStr(Haystack, StartingPosition, pos-StartingPosition)
		out .= %FunctionName%(match)
		len := IsObject(match) ? match.Len : StrLen(match)
		StartingPosition := pos + len
		if (++OutputVarCount = Limit)
			break
	}
	return out . SubStr(Haystack, StartingPosition)
}

DecimalPlaces(n) {
 if (!RegExMatch(n, "^\d*\.(\d*)$", m))
        return 2
    return StrLen(m1) = 0 ? 1 : StrLen(m1)
}

thank you both for the help not only in solving my immediate problem but helping me understand RegEx better

[tmplinshi]

You can do it like this:

Code: Select all

MsgBox, % RegExReplaceF(string, "O)([^\r\n]*?)F([\-\d.]+(?![^\(]*\)))", Func("Override").Bind(140))

Override(percent, match) {
	...

Please note that I've changed VarSetCapacity(out, StrLen(Haystack)*2) to VarSetCapacity(out, StrLen(Haystack) * (A_IsUnicode+1)).

Your question actually is one of my to-do list that I'm lazy to do.

[gwarble]

what do you mean to-do? it seems like you already did it! this works exactly like I want:

Code: Select all

string := "G00G90X1.2Y3.4F14M8(WAS F14.)`nG01G90X2.2Y3.4F19.6005(WAS F19.6005)`nG84G90X2.2Y3.4F14.00(WAS F14.00)`nG0G91G28Z0`n"
MsgBox, % RegExReplaceF(string, "O)([^\r\n]*?)F([\-\d.]+(?![^\(]*\)))", Func("Override").Bind(140))

Override(percent, match) {
	if InStr(match.1, "G84") OR InStr(match.1, "G74")
		return match.0
	else
		return match.1 "F" Round(match.2 * percent/100, DecimalPlaces(match.2))
}

RegExReplaceF(ByRef Haystack, NeedleRegEx, FunctionName, ByRef OutputVarCount := "", Limit := -1, StartingPosition := 1) {
	VarSetCapacity(out, StrLen(Haystack) * (A_IsUnicode+1))
	OutputVarCount := 0
	while ( pos := RegExMatch(Haystack, NeedleRegEx, match, StartingPosition) )
	{
		out .= SubStr(Haystack, StartingPosition, pos-StartingPosition)
		out .= %FunctionName%(match)
		len := IsObject(match) ? match.Len : StrLen(match)
		StartingPosition := pos + len
		if (++OutputVarCount = Limit)
			break
	}
	return out . SubStr(Haystack, StartingPosition)
}

DecimalPlaces(n) {
 if (!RegExMatch(n, "^\d*\.(\d*)$", m))
        return 1
    return StrLen(m1) = 0 ? 1 : StrLen(m1)
}

[tmplinshi]

Here is another way using Regular Expression Callouts.

Code: Select all

RegExReplaceF(ByRef Haystack, NeedleRegEx, FunctionName, ByRef OutputVarCount := "", Limit := -1, StartingPosition := 1) {
	;VarSetCapacity(d, StrLen(Haystack) * (A_IsUnicode+1))
	global $__g_callout := {data: d, fn: FunctionName, startPos: StartingPosition}
	RegExReplace(Haystack, NeedleRegEx "(?CRegExReplaceF_Callout)",, OutputVarCount, Limit, StartingPosition)
	return $__g_callout.data . SubStr(Haystack, $__g_callout.startPos)
}

RegExReplaceF_Callout(Match, CalloutNumber, FoundPos, Haystack) {
	global $__g_callout
	o := $__g_callout, fn := o.fn
	o.data .= SubStr(Haystack, o.startPos, FoundPos-o.startPos)
	o.data .= %fn%(Match)
	len := IsObject(Match) ? Match.Len : StrLen(Match)
	o.startPos := FoundPos + len
}

[jeeswg]

These have been mentioned:
The RegExReplace StartPos parameter.
RegEx callouts.

One other trick is using unused characters (characters not in the haystack or before/after text).
For the first round of replacements, add the unused character.
For the second round of replacements, remove the unused character.

If your new string is longer than the old string.
In the first round of replacements, you could replace the old string with 'itself plus some padding characters' (using an unused character for the padding).
For the second round of 'replacements', you could use StrPut to 'replace' the old string with the new string.
And replace any padding characters for all strings at the end.
When using StrPut you might want to use something like A_IsUnicode?"UTF-16":"CP0".

[gwarble]

so far i like the workaround of using a loop with RegExMatch as it gives the flexibility, but it looks like you found the real answer (of how to perform a function on the match before replacement within RegExReplace) with the callout feature... i will experiment and do some time comparisons, thanks tmplinishi

jeeswg: thanks, not sure i see the benefit of adding fake characters and then removing them, unless the other options (match, callout) weren’t available. You do bring up a valid concern with changing the length of the replacement text that should be considered (pos)

[malcev]

It is interesting thing with callouts.
You cannot return number, You have to convert it to literal string.

Code: Select all

VarSetCapacity(r, 16)
MsgBox, % RegExReplace("1234", "\d(?C:Match)", r)

Match() {
   global r
   Random, r, 11111111, 99999999
   r .= ""
}

[tmplinshi]

@malcev Very nice, thanks! So RegExReplaceF can be improved to: Not recommend, see lexikos's reply below.

Code: Select all

RegExReplaceF(ByRef Haystack, NeedleRegEx, FunctionName, ByRef OutputVarCount := "", Limit := -1, StartingPosition := 1) {
	global __$rerf_r, __$rerf_fn := FunctionName
	VarSetCapacity(__$rerf_r, 20000)
	return RegExReplace(Haystack, NeedleRegEx "(?CRegExReplaceF_Callout)", __$rerf_r, OutputVarCount, Limit, StartingPosition)
	     , __$rerf_r := __$rerf_fn := ""
}

RegExReplaceF_Callout(Match) {
	global __$rerf_fn, __$rerf_r
	__$rerf_r := %__$rerf_fn%(Match) . ""
}

Setting VarSetCapacity(__$rerf_r, 20000) wasted 20KB memory temporary, it allows 10000 characters in the replacement, should be enough in most cases.

[gwarble]

nice job guys, very succinct and useful

I will post results of speed comparisons for my current use-case, which is always short replacements

Many thanks everyone

[ahk7]

Just for fun and only somewhat related my function from Feb 2011:

RegExReplaceInc() @ https://github.com/hi5/RegExReplaceInc/

Function to add incremental numbers or strings to your replacement text during a RegExReplace.
This function is inspired on the \i option from the "Find and Replace" of text editor TextPad. It allows you to add incremental numbers to your replacement text making it easier to create unique identifiers or labels. For example numbered anchor targets in a html file: name="anchor1", name="anchor2", name="anchor3" etc.

It also supports string incrementation where you can give a string as Start parameter and a delimiter for the increment parameter...

Examples https://github.com/hi5/RegExReplaceInc/#code-examples

[lexikos]

If you need the replacement text to vary depending on the matched text, you should use RegExMatch in a loop, not RegExReplace.

RegExReplace is not designed to allow the replacement string to change while the matching takes place. Setting the variable capacity to a sufficiently high number usually prevents the variable's memory from being reallocated (which would likely cause an access violation in RegExReplace, and at the very least would prevent it from using the new string), but this is still not supported.

The last version of RegExReplaceF has a significant flaw: if the function returns an empty string, the replacement variable is freed. For example, this returns "a" followed by garbage:

Code: Select all

MsgBox % RegExReplaceF("abcdef", "O).", "remove_every_2nd")
remove_every_2nd(m) {
    return (m.Pos & 1) ? m.0 : ""
}

To be safe, RegExReplace should take a copy of the replacement string before beginning, preventing any changes to the variable from affecting the function in any way (such as crashing or the replacement changing). But since callouts are very rare, the added safety doesn't seem worth the cost to performance.