AutoHotkey Community

It is currently May 27th, 2012, 4:54 am

All times are UTC [ DST ]




Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next
Author Message
PostPosted: April 7th, 2010, 4:11 am 
Offline

Joined: February 10th, 2007, 5:18 am
Posts: 92
I have created a polynomial root solver that will solve (theoretically) for any amount of roots. Polynomials behave in a manner where the highest power ultimately determines how many solutions exist in both the real and complex planes.

My solution is based on Lin-Bairstow method which can be found here

In order to allow virtually any amount of coefficients, all the coefficients are to be passed in as a single comma-deliminated parameter. The results will come out just the same

Code:
;******************************************************
;PolyRoots(A)
;Finds all real and complex roots of all polynomials
;------------------------------------------------------
;
;@A      Comma-deliminated coefficients starting with highest power
;
;Out     Comma-deliminated roots of the polynomials
;******************************************************

PolyRoots(A) {
   Loop Parse, A, `,
      n := A_Index-1

   of := false
   if Mod(n,2)=1 ; odd power, make even to avoid divergence
   {
      A .= ",0"
      n += 1
      of := true
   }
   
   Loop Parse, A, `,
   {
      i := A_Index-1
      If i = 0
         an := A_loopfield
      re%i% := im%i% := b%i% := d%i% := 0.0
      a%A_Index% := A_loopfield / an
   }
   a0 := 0.0,  b1 := d1 := 1.0,  b2 := a2 ; for degree 1 poly

   order := n,  n1 := n+1,  n2 := n+2

   tol := 1.e-10, LIMIT := 500
   q1 := q2 := r1 := r2 := r3 := 0.0
   i := j := k := cnt := 0         ; cnt = root counter

   While n > 1 {                   ; The main Lin-Bairstow loop
      p1 := q1 := -0.7            ; p(x) = x^2 + p1*x + q1
   Try:
      Loop %LIMIT% {
      i := 2, j := 1, k := 0
      Loop %n%  {
         b%i% := a%i% - p1 * b%j% - q1 * b%k%
         d%i% := b%i% - p1 * d%j% - q1 * d%k%
         i++, j++, k++
      }
      j  := n-1,  k := n-2
      r1 := d%j%**2 - (d%n% - b%n%) * d%k%
      If abs(r1) < 1.e-99
         break
      p2 := (b%n% * d%j% - b%n1% * d%k%) / r1
      q2 := (b%n1% * d%j% - (d%n% - b%n%) * b%n%) / r1
      p1 := p1 + p2
      q1 := q1 + q2

      If (Abs(p2) < tol && Abs(q2) < tol)
         GoTo Cont              ; Converged!
   }
   Random p1, -0.8, -0.6
   Random q1, -0.8, -0.6
   GoTo Try                    ; If not converged, random re-try

   Cont:
   r1 := p1*p1 - 4*q1           ; Continue with recording the found roots
   t := cnt + 1

   If (r1 < 0) {                ; imaginary roots
      re%t% := re%cnt% :=-p1/2
      im%t% :=-im%cnt% := sqrt(Abs(r1))/2
   }
   Else {                       ; real roots
      r2 := sqrt(r1)
      re%cnt% := (r2 - p1) / 2
      re%t%  := -(r2 + p1) / 2
      im%t% :=im%cnt% := 0
   }

   cnt += 2                     ; update root counter
   n -= 2, n1 -= 2, n2 -= 2     ; reduce polynomial order

   If (n > 1)
      Loop %n2%                 ; coefficients of the new polynomial
         a%A_Index% := b%A_Index%
   }

   if (n = 1)                      ; obtain last single real root
      re%cnt% := -b2, im%cnt% := 0

   Loop %order%                    ; assemble result
   {
      i := A_index-1, r := round(re%i%,9), j := round(im%i%,9)
      if (of && r < tol && j < tol)
         of := false
      else
         x .= r (j<0 ? "" : "+") j "i" (A_Index < order ? "," : "")
         
   }

   Return x
}


Quick example if anyone wants to see it in action

Quote:
x := PolyRoots("1,0,0,0,0,0,0,0,1") ; x^8+1
loop, parse, x, `,
msgbox, Ans %A_index%: %A_loopfield%


Last edited by maximo3491 on April 9th, 2010, 4:34 am, edited 9 times in total.

Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 7th, 2010, 5:46 am 
Offline

Joined: October 17th, 2009, 8:54 pm
Posts: 43
This is uber geeky stuff.. I love it..

What kind of applications do you use stuff like this on?


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 7th, 2010, 7:23 am 
Offline

Joined: February 10th, 2007, 5:18 am
Posts: 92
It has many possible applications, many of which are physics related.

For example, an object attracted by gravity can have its position in space be expressed in terms of quadratic equations. Even though those are easier to solve than higher powers, this function could easily find the answer for it. As you move to problems in physics with higher powers, it becomes much harder to use basic algebra to solve for x, even though this basic function will consistently work.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 7th, 2010, 1:56 pm 
Offline

Joined: March 20th, 2010, 9:49 am
Posts: 224
im dead


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 7th, 2010, 6:05 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
Nice!

However, numerical search for the solutions, as done in this function, has its own problems. It is usually slower than direct methods, when such methods exist (degree at most 4, or special type of polynomials), but more seriously, sometimes the internal iterations do not converge. Try:
Code:
Loop  {
   PolyRoots("1,0,0,0,0,-1",x)
   ToolTip %A_Index%
}
After several successful steps the ToolTip stops increasing, showing that the function is stuck in an infinite loop. This is due to the random initialization of an internal loop, which has a large chance to stuck.

This can be fixed, but the code gets longer and slower. One method found successful in the practice is to add a dummy root if necessary, to make the polynomial of even degree, where there is much less chance for divergence. You can also restart the internal search loop with other random starting points if there is no improvement for a few steps. In case of divergence you can also replace the Newton iteration with guarded optimization, where first bounds are established around the optimum (of different signs of the cost function), and if the iterated value gets outside, use bisection instead of Newton for a few steps. This later method is the fastest, but needs complicated code, and sometimes we cannot find a guard interval around the optimum.

I would use random restarts, and if it fails a couple of times, add a dummy root.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 7th, 2010, 11:06 pm 
Offline

Joined: February 10th, 2007, 5:18 am
Posts: 92
I read up a little on the Lin-Bairstow method, specifically the iteration with the random start. The thing is you can pick two arbitrary numbers and start with them, doesn't have to be random. So I changed the following...

Code:
      Random, rand, 0.0, 1.0
      alfa1 := rand - 0.5
      Random, rand, 0.0, 1.0
      beta1 := rand - 0.5


to

Code:
      alfa1 := 1
      beta1 := 1


I updated the PolyRoots functions. Try your test loop now, the tooltip doesn't hang anymore. I'm not sure what the floating points of many significant figures had to do with the eventual hang, but choosing any 2 arbitrary numbers works.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 8th, 2010, 1:09 am 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
No, setting α and β to 1 is not a good idea. In this case we can find some random 5th degree polynomial, which causes the iteration to diverge.
Code:
s = 1,0,0,0,0,
Random,,0
Loop {
  Random r, -1.e19, 0
  ToolTip %A_Index%  %r%
  PolyRoots(s r,X)
}
The 488th call makes the script to hang.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 8th, 2010, 1:57 am 
Offline

Joined: February 10th, 2007, 5:18 am
Posts: 92
Well, I'm not exactly sure why this is the case, I'm no expert on the inner workings for floating point precision, but the script hangs because beta2 is unable to fall below the tolerance.

If i make the tolerance just one hundred millionth larger (0.00000002 instead of 0.00000001) just for beta2. This script seems to work. I tested it using your script for 20k attempts.

So I changed the following and updated the first post with the new function.

Code:
if( Abs( alfa2 ) < TOLERANCE and Abs( beta2 ) < TOLERANCE )


to

Code:
if( Abs( alfa2 ) < TOLERANCE and Abs( beta2 ) < (TOLERANCE*2) )


I understand I shouldn't have to increase the tolerance since it is technically producing more inaccurate results, but I'm not sure why it hangs. I'm understand how the math works and trying to get floating point precision to agree with the math isn't exactly my strong suite, so thank you for pointing out all the flaws.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 8th, 2010, 2:19 am 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
Changing the tolerance might help here, but it does not solve the basic problem. It is not a floating point issue, but the limitation of the Newton iteration: depending on the function and the starting point it could diverge. In this case the divergence problem only pops up once in several hundred calls, so just restarting the iteration from another random point would work.

I have not checked if you need to reset the b and d vectors, or just use their last values at restart, but the script below contains the idea. I simplified your code a lot, and used shorter variable names so I could have the whole function in one page, which helps me at debugging.
Code:
s = 1,0,0,0,0,
Random,,0
Loop {
  Random r, -1.e19, 0
  ToolTip %A_Index%  %r%,180,0,1
  PolyRoots(s r)
}

PolyRoots(A) {
   Loop Parse, A, `,
   {
      i := A_Index-1
      If i = 0
         an := A_loopfield
      re%i% := im%i% := b%i% := d%i% := 0.0
      a%A_Index% := A_loopfield / an
   }
   a0 := 0.0,  b1 := d1 := 1.0,  b2 := a2 ; for degree 1 poly

   order := n := i,  n1 := n+1,  n2 := n+2

   tol := 1.e-8, LIMIT := 500
   q1 := q2 := r1 := r2 := r3 := 0.0
   i := j := k := cnt := 0         ; cnt = root counter

   While n > 1 {                   ; The main Lin-Bairstow loop
      p1 := q1 := 1                ; p(x) = x^2 + p1*x + q1
   Try:
      Loop %LIMIT% {
         i := 2, j := 1, k := 0
         Loop %n%  {
            b%i% := a%i% - p1 * b%j% - q1 * b%k%
            d%i% := b%i% - p1 * d%j% - q1 * d%k%
            i++, j++, k++
         }
         j  := n-1,  k := n-2
         r1 := d%j%**2 - (d%n% - b%n%) * d%k%
         p2 := (b%n% * d%j% - b%n1% * d%k%) / r1
         q2 := (b%n1% * d%j% - (d%n% - b%n%) * b%n%) / r1
         p1 := p1 + p2
         q1 := q1 + q2

         If (Abs(p2) < tol && Abs(q2) < tol)
            GoTo Cont              ; Converged!
      }
      ToolTip DIVERGED %A_Now%,0,0,2 ; For tests
      Random p1, -0.5, 0.5
      Random q1, -0.5, 0.5
      GoTo Try                     ; If not converged, random re-try

   Cont:
      r1 := p1*p1 - 4*q1           ; Continue with recording the found roots
      t := cnt + 1

      If (r1 < 0) {                ; imaginary roots
         re%t% := re%cnt% :=-p1/2
         im%t% :=-im%cnt% := sqrt(Abs(r1))/2
      }
      Else {                       ; real roots
         r2 := sqrt(r1)
         re%cnt% := (r2 - p1) / 2
         re%t%  := -(r2 + p1) / 2
         im%t% :=im%cnt% := 0
      }

      cnt += 2                     ; update root counter
      n -= 2, n1 -= 2, n2 -= 2     ; reduce polynomial order

      If (n > 1)
         Loop %n2%                 ; coefficients of the new polynomial
            a%A_Index% := b%A_Index%
   }

   if (n = 1)                      ; obtain last single real root
      re%cnt% := -b2, im%cnt% := 0

   Loop %order%                    ; assemble result
      i := A_index-1, x .= re%i% "+" im%i% "i" (A_Index < order ? "`n" : "")

   Return x
}
Two ToolTips give continuous status information. The one to the left shows the time-stamp of the last divergence found, the one to the right tells which iteration we are in, and what the input is.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 8th, 2010, 2:37 am 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
maximo3491 wrote:
the script hangs because beta2 is unable to fall below the tolerance
Interesting observation. After all, it could be a floating point precision problem, indeed. If I change the tolerance to 1.e-9, almost no Newton iteration converges, and the function stops working. Increasing the floating point precision does not make a difference:
Code:
SetFormat FloatFast, 0.17e
It is too bad that we cannot achieve higher precision.

If we reduce the range of the coefficients to ±1.e9, there is no convergence problem, even with a tolerance of 1.e-12. Maybe, we just have to warn the user to scale the polynomial, such that the coefficients won't get insanely large.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 8th, 2010, 2:45 am 
Offline

Joined: February 10th, 2007, 5:18 am
Posts: 92
Oh, I understand what you were saying. I was a little bit confused by your suggestions, mainly because I was unaware of the divergence that occurs in this specific cases of an odd degree with only 1 real root.

I'll take out the debugging tool tips and update the function with your simplified version.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 8th, 2010, 3:44 am 
Offline

Joined: February 10th, 2007, 5:18 am
Posts: 92
Both problems seem to be at work here. There a floating point precision problem as well as an inherit issue with divergence of odd power polynomials with only a single real root.

Unfortunately, the precision problem we cannot fix as its a limitation of programming. As such, it should be understood by all using this method of approximating roots that high precision for such large coefficients is very hard to achieve. In the end, results should moreover be applied to math homeworks rather than space exploration programs.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 8th, 2010, 3:48 am 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
One more point: with alfa = beta = 1 there are problems, e.g. with "1, 0, 0, 0,-1". 0 is equally funny, 0.1 seems to be better. I don't have time to debug the code, but it should work with 0 or 1.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 8th, 2010, 7:03 pm 
Offline

Joined: February 14th, 2005, 4:05 pm
Posts: 4710
Location: Boulder, CO
...the problem is that the variable r1 can become 0, which makes p2 and q2 empty (results of division by 0). Their absolute value is treated as 0, so the function accepts the wrong result as correct. This is why we don't see divergence at the tooltips, although there are a lot of them.

Here is the version of the function, which checks for this problems, too, and restarts the search. This seems to be robust enough if the coefficients remain below 1.e9. Just remove the tooltip lines if the code works as expected.
Code:
#SingleInstance Force
#NoEnv
SetBatchLines -1

SetFormat FloatFast, 0.17e ; maximum precision
CoordMode ToolTip, Screen

s = 1,0,0,0,0,
Random,,0
Loop {
  Random r, -1.e6, 0
  ToolTip %A_Index%  %r%,180,0,1
  PolyRoots(s r)
}

;------------------------------------------------------------------------------

PolyRoots(A) {
   Loop Parse, A, `,
   {
      i := A_Index-1
      If i = 0
         an := A_loopfield
      re%i% := im%i% := b%i% := d%i% := 0.0
      a%A_Index% := A_loopfield / an
   }
   a0 := 0.0,  b1 := d1 := 1.0,  b2 := a2 ; for degree 1 poly

   order := n := i,  n1 := n+1,  n2 := n+2

   tol := 1.e-12, LIMIT := 1000
   q1 := q2 := r1 := r2 := r3 := 0.0
   i := j := k := cnt := 0         ; cnt = root counter

   While n > 1 {                   ; The main Lin-Bairstow loop
      p1 := q1 := .2               ; Initial values for factor p(x) = x^2 + p1*x + q1
   Try:
      Loop %LIMIT% {
         i := 2, j := 1, k := 0
         Loop %n%  {
            b%i% := a%i% - p1 * b%j% - q1 * b%k%
            d%i% := b%i% - p1 * d%j% - q1 * d%k%
            i++, j++, k++
         }
         j  := n-1,  k := n-2
         r1 := d%j%**2 - (d%n% - b%n%) * d%k%
         If abs(r1) < 1.e-99
            break
         p2 := (b%n% * d%j% - b%n1% * d%k%) / r1
         q2 := (b%n1% * d%j% - (d%n% - b%n%) * b%n%) / r1
         p1 := p1 + p2
         q1 := q1 + q2
         If (Abs(p2) < tol && Abs(q2) < tol)
            GoTo Cont              ; Converged!
      }
      ToolTip DIVERGED %A_Now%,0,0,2 ; For tests
      Random p1, -0.5, 0.5
      Random q1, -0.5, 0.5
      GoTo Try                     ; If not converged, random re-try

   Cont:
      r1 := p1*p1 - 4*q1           ; Continue with recording the found roots
      t := cnt + 1

      If (r1 < 0) {                ; imaginary roots
         re%t% := re%cnt% :=-p1/2
         im%t% :=-im%cnt% := sqrt(Abs(r1))/2
      }
      Else {                       ; real roots
         r2 := sqrt(r1)
         re%cnt% := (r2 - p1) / 2
         re%t%  := -(r2 + p1) / 2
         im%t% :=im%cnt% := 0
      }

      cnt += 2                     ; update root counter
      n -= 2, n1 -= 2, n2 -= 2     ; reduce polynomial order

      If (n > 1)
         Loop %n2%                 ; coefficients of the new polynomial
            a%A_Index% := b%A_Index%
   }

   if (n = 1)                      ; obtain last single real root
      re%cnt% := -b2, im%cnt% := 0

   Loop %order%                    ; format result for display in proportional font
      i := A_index-1, j := round(im%i%,9), r := round(re%i%,9)
    , x .= (r<0 ? "" : "  ") r (j<0 ? "  " : " +") j "i" (A_Index < order ? "`n" : "")

   Return x
}
I also changed the format of the output for nicer look in MsgBox. You may want to replace multiple spaces with single ones, and "`n" with ",". There is no "+" in front of negative imaginary parts, either.


Report this post
Top
 Profile  
Reply with quote  
 Post subject:
PostPosted: April 9th, 2010, 4:32 am 
Offline

Joined: February 10th, 2007, 5:18 am
Posts: 92
Understanding that the actual problem of odd-powered polynomials suffering the chance of divergence (which ultimately causes longer processing time of the function call) is rooted in the basic logic of the method as well as the starting position. To address both issues, I conducted a test of the function as it is (divergences will be retried until successful) alongside an improved method. This improved method turns odd-powered polynomials into even-powered by multiplying the whole function by x (thus adding 0 as an additional solution to the problem, which I take out at the end).

This test consisted of generating 10000 random numbers, r, from -1e19 to 0 and using that random number in x^5+r=0. Then I retrieved all starting positions, s, from -1 to 1, inclusively and in intervals of .05. These 2,010,000 possible combinations of r and s were fed into both the original and improved functions. The results show average call time and the number of divergences for both methods. A picture is shown below.

Image

What I found was that, on most starting positions, the chance of divergence greatly decreased in the improved method, allowing for shorter function call time. In intervals where the rate of divergence was relatively equal between the two methods, this method comes out to be 3-5 ms longer per call than the original method.

In addition, I'd like to point out that a starting position of 0 always has divergence, even under the new method of turning the function to an even power. Also, the "retry interval" of -.5 to .5 where the function selects a new starting point is actually much more successful in the new method because the chance of divergence in the original method is about 50% on almost all starting positions within that range.

Lastly, this cannot be made out from the graph, but I found the expected call time given a random r and a random starting value. The old method took 19.77 ms per function call while the new method took only 13.37 mc.

I'm going to use the improved method even though it technically requires more computation and can take longer. Its just more accurate and for smaller coefficients would most likely be even faster than for larger coefficients. I'll be changing the "retry interval" to -.8 to -.6 and the starting position to -.7 just because it works faster in that range (not sure why exactly). Also, Laszlo, I'll add your addition of retrying when r1 becomes 0 but remove some of the formatting you did for the results so it's easier to parse them and not have to trim a bunch.

Initial post updated as such.


Report this post
Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 17 posts ]  Go to page 1, 2  Next

All times are UTC [ DST ]


Who is online

Users browsing this forum: Yahoo [Bot] and 13 guests


You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Powered by phpBB® Forum Software © phpBB Group