 |
AutoHotkey Community Let's help each other out
|
| View previous topic :: View next topic |
| Author |
Message |
Dippy46
Joined: 06 Jul 2004 Posts: 171 Location: Manchester, England.
|
Posted: Fri Jan 05, 2007 9:06 pm Post subject: |
|
|
@Skan,
The inabilty to read and write binary files is unfortunately a problem we all have to suffer, so its better to understand why we can't. As PhiLho says AHK cannot possible read binary files per se. This is not a limitation of AHK but the fundamental design of all string handling , the dreaded NULL terminator.
Back in the early days of MSBasic (when Bill was just another kid on the block), things were handled differently. Now let's recap
memory max 65535 just 2 bytes to hold segment + offset of string pointer
string max 256 just 1 byte to hold the length. Later 65535
That arrangement allowed one to make a string that contained all 256 bytes. You could manipulate string pointers using varptr, and bytes could be read using asc$ and written using chr$. So of course it begs the question: what nut introduced the NULL terminator. And no I don't know.
Below shows proof of concept that AHK DOES in fact handle binary files and strings just as well as text. Of course the subtlety here is we've made sure it doesn't contain any NULL's ie chr(0).
| Code: |
loop,1000
{
Random,ch,1,255 ; make sure no nulls
me := me chr(ch)
}
; Ahk string functions substr/strlen
chunk := substr(me,100,50)
msgbox % strlen(chunk) " Bytes`n`n" chunk "`n`nAll the bytes `n`n" me
; even
fileappend,% me,binary.txt
|
It doesn't mean there's no way out, but for the moment certainly, the easiest way is to treat the numbers as an array of 2byte hexA. I hope this helps a bit.
Regards
Dave. _________________ Simple ideas lie within reach, only of complex minds |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4710 Location: Boulder, CO
|
Posted: Fri Jan 05, 2007 9:23 pm Post subject: |
|
|
| These limitations are well documented in the Help, and solutions for reading and writing binary files (based on dll calls) were already posted a year and a half ago. There you can read: "The binary file I/O functions store/load the data to/from AHK variables, which may contain NULL characters, therefore, they must not be manipulated as ordinary strings. Not even assignments or StrLen is allowed. If these variables are used as function parameters, they must be defined as ByRef, otherwise the strings get truncated at the first NULL". |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4710 Location: Boulder, CO
|
Posted: Fri Jan 05, 2007 9:40 pm Post subject: |
|
|
You have to distinguish STRINGS form other (binary) data. Strings were introduced to handle text, mostly I/O messages. Texts do not contain certain characters, so it is natural to pick one as a delimiter. If you use strings for storing other data, you are in the state of sin.
There is no problem with NUL delimited strings, but only with their misuse. AHK is a string based language, where everything is a string: text, numbers, ID's, addresses, etc. Therefore, AHK cannot handle binary data in its native form. Several posts provided solutions (started almost two years ago) by coding binary data into strings of allowed characters: hex digits, base64, base85 or pebwa (invented by PhiLho). |
|
| Back to top |
|
 |
Dippy46
Joined: 06 Jul 2004 Posts: 171 Location: Manchester, England.
|
Posted: Fri Jan 05, 2007 9:41 pm Post subject: |
|
|
@Laszlo,
Oh yes Sir, I'm fully aware of all the references, possibly ad nauseum. The fact however is, that whilst we tell people that it's not possible, I thought it might be useful to people that don't have our knowledge, as to why it doesn't work. It also serves as a challenge, because anyone with the correct knowledge and determination, could quite easily write an add-on library that would provide all the necessary functionality.
Regards
Dave. _________________ Simple ideas lie within reach, only of complex minds |
|
| Back to top |
|
 |
PhiLho
Joined: 27 Dec 2005 Posts: 6836 Location: France (near Paris)
|
Posted: Sat Jan 06, 2007 9:55 am Post subject: |
|
|
| Skan wrote: | Actually, I was answering/supporting Laszlo
| Laszlo wrote: | http://www.autohotkey.com/forum/viewtopic.php?p=98142#98142
AHK could be optimized not to copy the result, but use the address (if any) of the variable or temporary storage, where the result ends up. I think Chris would give it a low priority, though, because saving the copy is not that significant and he can better spend his time on new features. |
| OK.
Well, such feature might come handy when arrays will be implemented: they are often large, and there should be some mechanism to return an array created in a function, without making it global.
Perhaps some special Return telling: push the content of this variable in a global scope (ie. don't erase/overwrite this memory when going out of function scope) and return a reference to it.
Something like:
| Code: | F(ByRef binData)
{
s := GetSize(binData)
; Process binData, creating otherData
Return ByRef otherData
}
G(ByRef bigData)
{
od := F(bigData)
}
| od is local to G, otherData is local to F but od gets its content without a copy of data.
Limiting this to Return values limits the use of references: they are powerful, but hard to understand (for newbies) and to manage, eg. when doing cross-references (I write about arrays here). _________________
vPhiLho := RegExReplace("Philippe Lhoste", "^(\w{3})\w*\s+\b(\w{3})\w*$", "$1$2") |
|
| Back to top |
|
 |
SKAN
Joined: 26 Dec 2005 Posts: 8688
|
Posted: Sat Jan 06, 2007 10:24 am Post subject: |
|
|
| PhiLho wrote: | | Perhaps some special Return telling: push the content of this variable in a global scope (ie. don't erase/overwrite this memory when going out of function scope) and return a reference to it. |
Exactly ..
| Code: | BinData := Hex2Bin( "FF00FF00")
; Actually, Hex2Bin() will be directly used as a parameter in WriteFile()
Hex2Bin( Data ) {
Local bData
....
DllCall("RtlFillMemory" ...
....
...
Return ByRef bData
} |
Would not this be nice ? Can you take this to Mr.Chris ?
Thanks for understanding PhiLho, though I sounded vague.
 _________________ URLGet - Internet Explorer based Downloader |
|
| Back to top |
|
 |
Laszlo
Joined: 14 Feb 2005 Posts: 4710 Location: Boulder, CO
|
Posted: Sat Jan 06, 2007 2:04 pm Post subject: |
|
|
| PhiLho wrote: | | Perhaps some special Return telling: push the content of this variable in a global scope (ie. don't erase/overwrite this memory when going out of function scope) and return a reference to it | My point was that this has to be done always. It does not cause any harm at short return values but saves a lot of time at copying large return values.
This does not solve all the problems with returning binary buffers, because an assignment x := f(y) still truncates the result at the first NUL. Either the assignment in AHK needs to be changed or a special function BinCopy(Byref TO, ByRef FROM, Len=0) has to be used. |
|
| Back to top |
|
 |
PhiLho
Joined: 27 Dec 2005 Posts: 6836 Location: France (near Paris)
|
Posted: Sun Jan 07, 2007 8:56 am Post subject: |
|
|
| Laszlo wrote: | | This does not solve all the problems with returning binary buffers, because an assignment x := f(y) still truncates the result at the first NUL. | It truncates because it makes a copy, the point of my proposal is precisely to avoid such copy.
Obviously a special Return mechanism should use a special assignment behind, otherwise it wouldn't work. Assignments of such value should take note it is a reference/pointer, not a simple value, and ajust the target variable accordingly.
I don't know the current behavior of AHK code, but a possible mechanism is perhaps:
Adjust memory of the variable on the left side of the assignment to the size of the returned string (which is still in the section of the local vars of the function);
Copy the string from the local var to the memory of the variable (that's where the truncate takes place).
The new mechanism can be to free the memory of the assigned variable and makes it point to the memory of the returned value which no longer belong to the list of local vars.
Or, if it clashes too much with current design, we can transform the Return ByRef to a Return BySize...
Ie. the new mechanism could be:
Adjust memory of the variable on the left side of the assignment to the size of the returned variable (that excludes probably using this to return expressions);
Copy the data from the local var to the memory of the variable with a memcpy instead of strpcy (or similar).
There is still a copy in the process, which can be expansive, but to avoid this we can still use ByRef parameters. _________________
vPhiLho := RegExReplace("Philippe Lhoste", "^(\w{3})\w*\s+\b(\w{3})\w*$", "$1$2") |
|
| Back to top |
|
 |
Chris Site Admin
Joined: 02 Mar 2004 Posts: 10716
|
Posted: Sun Jan 07, 2007 10:35 am Post subject: |
|
|
| Laszlo wrote: | | the copy induced by the Return from a function takes more than twice the time necessary to copy the data. This is the important observation here. | Yes, there are as many as two copies involved when a user-defined function returns a string: first, the function copies its return value to temporary memory, then the caller of the function copies that result into its variable. However, there are some optimizations that avoid one or both of these copies; but currently, the best optimizations apply only to built-in functions like SubStr() and RegExReplace().
| Laszlo wrote: | | This copy step is not performed if the result is returned in a ByRef parameter, f(x,y), where the return variable already contains the computed string. | Yes, there might be such an optimization. There are so many string/expression optimizations that I can no longer understand them all, and thus resort to comments in the code. I will continue to add more optimizations when the mood occurs, which happens a lot but has the drawback of taking time away from new features.
Originally, one of my main motivations for having ByRef was performance; so until something better comes along, ByRef can be used to "return" huge strings.
| PhiLho wrote: | | The new mechanism can be to free the memory of the assigned variable and makes it point to the memory of the returned value which no longer belong to the list of local vars. | Yes, I tried to allow a function that returns a non-static local to surrender that local to its caller (since it's about to be freed anyway). This would save at least one mem-copy. However, the complexity and code size created some problems; so I took it out for now. I'll probably try it again at some point.
| Laszlo wrote: | | If it is true, AHK could be optimized not to copy the result, but use the address (if any) of the variable or temporary storage, where the result ends up. | That's an interesting idea. It might be possible to directly assign to a variable the temporary storage that holds a return value (rather than making a copy). I've made a note to look into it.
| Laszlo wrote: | | In case of a function return, like x:=f(y) AHK could just do some pointer manipulation. The storage to be returned could be allocated in the caller's memory space, and there will be NO data copy necessary. | Yes, this seems to achieve the same improvement as the previous paragraph. Hopefully at least one of these approaches will be feasible.
The drawback to optimizations like these is that they tend to create recursion difficulties and interdependencies between sections of the code. This reduces modularity, maintainability, and (slightly) the overall performance of expressions (since extra checks are involved even in trivial expressions that have no big strings). So these factors must be weighed against the benefit.
| PhiLho wrote: | Well, such feature might come handy when arrays will be implemented: they are often large, and there should be some mechanism to return an array created in a function, without making it global.
Perhaps some special Return telling: push the content of this variable in a global scope (ie. don't erase/overwrite this memory when going out of function scope) and return a reference to it. Something like:... Return ByRef otherData | This is interesting. Although it might be made less important by future optimizations (such as the ones mentioned earlier), it's something to consider for the future.
Thanks for the great ideas. |
|
| Back to top |
|
 |
PhiLho
Joined: 27 Dec 2005 Posts: 6836 Location: France (near Paris)
|
Posted: Sun Jan 07, 2007 10:35 pm Post subject: |
|
|
I understand that Laszlo would prefer a generic solution, but in 90 to 99% of function uses, a Return by copy is OK, as most return values are small.
So if doing optimization is made at the risk of breaking something (recursion for example), I would prefer a special Return to operate on large values, ie. programmer will need to tell AHK: "This data is special, treat it in a special way".
If Chris can make it generic, well that's better, of course. _________________
vPhiLho := RegExReplace("Philippe Lhoste", "^(\w{3})\w*\s+\b(\w{3})\w*$", "$1$2") |
|
| Back to top |
|
 |
SKAN
Joined: 26 Dec 2005 Posts: 8688
|
Posted: Tue Jan 16, 2007 7:23 am Post subject: |
|
|
| In a PM, AngieX wrote: | HI - I really liked your Linear Gradient work. It is very very nice and professional. Can you please describe or even show me if you feel happy to oblige - to how I can start with a basic AHK script using this concept. The idea I have is a gradiant square like yours between any two color values. It should be a square I can define the (w)idth and (h)eight to. I would like to basically start with a play values like:
| Code: |
var amount = 50%
var width = 250px
var height = 250px
var start_color = 0208FF
var end_color = FFFFFF
{then draw 250x250 with 50% split gradient between start & end_color }
|
Can you please help me please with this? You would be the greatest. I have some ideas how to build this into some charting demographics and just getting my feet wet writing this cool script
Have wonderful day and thank you for being out there
Sincerely, Angie.
RE: How to Simulate a Linear Gradient ? - Part 1 & 2
http://www.autohotkey.com/forum/viewtopic.php?p=61081#61081 |
Dear Angie,
I am not much clear about your requirement, but try the following code and then post further queries:
| Code: | var_width = 250
var_height = 250
var_start_color = 0208FF
var_end_color = FFFFFF
;{then draw 250x250 with 50% split gradient between start & end_color }
GradientFile=Bg.bmp
IfNotExist, %GradientFile%
GradientFile:=CreateBMPGradient(GradientFile,var_start_color,var_end_color,1)
Gui, Margin, 0,0
Gui, Add, Picture, x0 y0 w%var_width% h%var_height%, %GradientFile%
Gui, Show, , AngieX
Return
GuiClose:
GuiEscape:
ExitApp
Return
#Include BitmapGradient.ahk |
The above code requires: BitmapGradient.ahk
My guess is that you are interested in creating some bar chart.. and that will be very interesting . Do not hesitate to post further queries.
Regards,  |
|
| Back to top |
|
 |
AngieX
Joined: 26 Nov 2006 Posts: 77
|
Posted: Tue Jan 16, 2007 7:54 am Post subject: |
|
|
I like how I am incorporated into the GUI I feel famous
Great looking code. May I ask, how this could be tweaked to do two things:
1) Have a var_input = %, so the 50% is flexible
2) To turn this into a callable function, so the display is updated when the function is called. I envision something like UpdateGraph(var_width, var_height, var_start_color, var_end_color, var_input). I don't know how piped in variables/functions perciely work under AHK (I can think JavaScript . This is really giving me quick-ramp up to its protocols so I apprecaite your descriptions so much. So I would envision this to be called like UpdateGraph(250,250,0208FF,FFFFFF,50%) <- I know that is wrong all-together, but u get the idea I hope
That will make it real flexible, simple and modular. Hopefully you think that is right software protocol.
Thanks for advancing my original idea!
-AngieX% |
|
| Back to top |
|
 |
SKAN
Joined: 26 Dec 2005 Posts: 8688
|
Posted: Tue Jan 16, 2007 10:22 am Post subject: |
|
|
| AngieX wrote: | | 1) Have a var_input = %, so the 50% is flexible |
No. That is not possible CreateBMPGradient()! That function has been hardcoded to create a 2 pixel bitmap. Each pixel will occupy half the control 50:50. If you want 33:66, then the function has to re-written to accomodate 3 pixels. Say for blue n black it would be blue+black+black, and effectively blue would occupy 33.33% and black 66.66%
Hope you get that right.
| Quote: | | 2) To turn this into a callable function, so the display is updated when the function is called. I envision something like UpdateGraph(var_width, var_height, var_start_color, var_end_color, var_input). |
Writing such a wrapper function id redundant, IMHO. Just assign a variable (like MyPic ) to the picture control and call the GuiControl command to change the picture ( or its dimension )
Try this:
| Code: | var_width = 250
var_height = 250
var_start_color = 0208FF
var_end_color = FFFFFF
GradientFile=Bg.bmp
IfNotExist, %GradientFile%
GradientFile:=CreateBMPGradient(GradientFile,var_start_color,var_end_color,1)
Gui, Margin, 0,0
Gui, Add, Picture, x0 y0 w%var_width% h%var_height% vMyPic, %GradientFile%
Gui, Show, , AngieX
Return
; Changing the background
+F2::GuiControl,,MyPic, % CreateBMPGradient("bg.bmp","000000","FF0000",1)
+F3::GuiControl,,MyPic, % CreateBMPGradient("bg.bmp","640067","DC6C00",1)
+F4::GuiControl,,MyPic, % CreateBMPGradient("bg.bmp","004080","808040",1)
+F5::GuiControl,,MyPic, % CreateBMPGradient("bg.bmp","804040","EDC47E",1)
+F6::GuiControl,,MyPic, % CreateBMPGradient("bg.bmp","000000","FF8000",1)
+F7::GuiControl,,MyPic, % CreateBMPGradient("bg.bmp","000000","00FF00",1)
+F8::GuiControl,,MyPic, % CreateBMPGradient("bg.bmp","000000","0000FF",1)
+F9::GuiControl,,MyPic, % CreateBMPGradient("bg.bmp","000000","DECADE",1)
GuiClose:
GuiEscape:
ExitApp
Return
#Include BitmapGradient.ahk |
Try Shift+functions to test the changing of background gradient.
 |
|
| Back to top |
|
 |
AngieX
Joined: 26 Nov 2006 Posts: 77
|
Posted: Tue Jan 23, 2007 11:12 pm Post subject: |
|
|
I am trying to cram this all in my head, but do follow your logic soundly!
two questions -
What does 1 signify?
% CreateBMPGradient("bg.bmp","000000","FF0000",1) < what is 1?
Can this be further explored, the idea to have a % input - to basically have a BMP assignment, for 1%, 2% to 100%. Just 1-100 to keep it simple. Each one has a mathamatatial computation to some BMP configuration, then its pumped into BMP.
So, 16% would flow like:
% CreateBMPGradient("bg.bmp","16%","000000","FF0000",1)
I am way way over my head actually writing it out, but maybe if I can see some ways you write, I can follow the rythum for improving this object. I have some other ideas to make this more interesting, like a decay? -- so if a 16% was pumped in, then a 42% three seconds later, it will kick back up, but not in a fast skippy way. Kinda like a very nice car stereo equalizer.... so it look real smooth, remain live and programmable. Do you see where I am going with this? I think it will be fun to mess around with. |
|
| Back to top |
|
 |
SKAN
Joined: 26 Dec 2005 Posts: 8688
|
Posted: Sun Jan 28, 2007 9:33 pm Post subject: |
|
|
| AngieX wrote: | What does 1 signify?
% CreateBMPGradient("bg.bmp","000000","FF0000",1) < what is 1? |
1 creates a vertical stack of two pixels whereas 0 creates a horizontal one.
The parameters have been explained in the original post.
| AngieX wrote: | | Can this be further explored, the idea to have a % input - to basically have a BMP assignment, for 1%, 2% to 100%. Just 1-100 to keep it simple. Each one has a mathamatatial computation to some BMP configuration, then its pumped into BMP. |
Well.. My function creates only a 2 pixel bitmap, which - when - stretched to abnormal limits - creates a gradient effect. It is a kind of simulation. For your requirements, You might have to create a full sized bitmap to achieve proper results which is beyond the scope of CreateBMPGradient() function.
I will give it a try when I have time and post again.
Regards,
PS: Sorry for the late reply. Offlate, I have not been frequenting the forum. |
|
| Back to top |
|
 |
|
|
You can post new topics in this forum You can reply to topics in this forum
|
Powered by phpBB © 2001, 2005 phpBB Group
|