View topic - TF: Text file & Variables/String Library v3.3 [lib] • AutoHotkey Community

View unanswered posts | View active topics

Board index » AutoHotkey » Scripts

All times are UTC [ DST ]

TF: Text file & Variables/String Library v3.3 [lib]

Page 5 of 13

[ 194 posts ]

Go to page Previous 1, 2, 3, 4, 5, 6, 7, 8 ... 13 Next

Print view | E-mail friend

Previous topic | Next topic

Author

Message

SoLong&Thx4AllTheFish

Post subject:

Posted: September 19th, 2009, 6:24 pm

Joined: May 27th, 2007, 9:41 am
Posts: 4999

@Tuncay: thanks for the suggestion and I'll compare it to the working solution I already made. But I'm scratching my head over something Majkinetor said a few posts above.

How to create a TF read function that creates a byref variable if the parameter is empty.

Code:

test=
(
1
2
3
)
FileDelete, tftest.txt
FileAppend, %test%, tftest.txt

TF("tftest.txt") ; this doesn't create the global T var but it should be possible somehow?
MsgBox % "empty:" t

TF("tftest.txt", t) ; this works
MsgBox % "not empty:" t

TF(TextFile, byref t="") { ; read contents of file in t and return it, create t if omitted in TF call
    FileRead, t, %TextFile%
    if IsOmitted(t) ; ????????? problem area
       return t                 
   }

IsOmitted(byref t){
   ;needs to "create" t somehow? with code from links below?
   Return
   }

I suspect the solution is here somewhere:

Passing parameters by name
http://www.autohotkey.com/forum/topic46176.html

A_ParamCount - to retrieve the number of PASSED parameters
http://www.autohotkey.com/forum/topic42502.html

_________________
AHK FAQ
TF : Text files & strings lib, TF Forum

Top

Tuncay

Post subject:

Posted: September 19th, 2009, 6:54 pm

Joined: November 7th, 2006, 9:47 pm
Posts: 1934
Location: Germany

This? Dont understand what you are trying todo ... Maybe something with getGlobal and setGlobal?

Code:

test=
(
1
2
3
)
FileDelete, tftest.txt
FileAppend, %test%, tftest.txt

TF("tftest.txt") ; this doesn't create the global T var but it should be possible somehow?
MsgBox % "empty:" t

TF("tftest.txt", t) ; this works
MsgBox % "not empty:" t

TF(TextFile, byref t="") { ; read contents of file in t and return it, create t if omitted in TF call
    FileRead, t, %TextFile%
    if (t!= "") ; ????????? problem area
    {
        setGlobal("t", t)
       return (getGlobal("t"))
    }
}

getGlobal(var)
{
    global
    var := %var%
    return var
}

setGlobal(var
         , content = "")
{
    global
    %var% := content
}

EDIT:
But the problem with global variable could be if the functions are used in other functions, where local variable is not known. Hmm I see, some static is needed here?

Top

SoLong&Thx4AllTheFish

Post subject:

Posted: September 19th, 2009, 8:23 pm

Joined: May 27th, 2007, 9:41 am
Posts: 4999

You did read Majkinetor post here:
http://www.autohotkey.com/forum/post-295680.html#295680

Quote:

Code:

TF(TextFile, ByRef t="") {
    FileRead, t, %TextFile%
    if IsOmitted(t)
             return t                 
}
TF_ReadLines(text, ....)
TF_ReadLines(TF("TextFile.txt") ... )

TF("TextFile.txt", t)
TF_ReadLines(t ... )

IsOmitted is some mechanism to detect if var is omited (or just at default value). While we wait for official way to have it, you can use some of the workarounds on the forum.

I think I'm a little (OK very) out of of my depth here. Yours seems to work, as I understand it is just to save yet another parameter to use each time you would use TF but still be able to do so if you wanted to.

Quote:

Tuncay: But the problem with global variable could be if the functions are used in other functions, where local variable is not known. Hmm I see, some static is needed here?

I don't think I understand that either :oops:

the idea is to have TF create a global the variable that you can then pass on the other TF functions so it has to be global and will always be known and modified by the TF. It either is a file and the output is a file OR it is a global var and the output is var. It's late and I'm rambling on a bit but I think I'm right :wink:

I now have all the bits & pieces to upgrade the lib to handle vars & strings not only files.

Unless someone sees a problem that is, I've used your example above in my test version and it does work like so

Code:

MsgBox % TF_CountLines(TF("mytestfile.txt"))
TF("mytestfile.txt")
MsgBox % TF_CountLines(t)
TF("mytestfile.txt", tffile)
MsgBox % TF_CountLines(tffile)

_________________
AHK FAQ
TF : Text files & strings lib, TF Forum

Top

Tuncay

Post subject:

Posted: September 19th, 2009, 9:56 pm

Joined: November 7th, 2006, 9:47 pm
Posts: 1934
Location: Germany

HugoV wrote:

I don't think I understand that either Embarassed the idea is to have TF create a global the variable that you can then pass on the other TF functions so it has to be global and will always be known and modified by the TF.

Oh ok, I was thinking it is the global variable would be created for the user. If it would be only internally used, then it should work fine. Just make sure it is a very unique identifier, for not overwrite any user global variable, i.e. beginning with TF_ and then following any unique randomly chosen characters. That variable you can use in all your functions.

Top

forkinpm

Post subject: Post on TF Library to Hugo V

Posted: October 28th, 2009, 8:36 pm

Joined: January 24th, 2009, 3:22 pm
Posts: 13
Location: Aldersbach, Germany

Hallo!
I am in the midst of trying to put together my ways to automate a translation model I have built to translate from English to German.
The model works in a manual environment and is described in simple and understandable steps.
I am preparing a document on my needs related to your library.
I would like to use the library and am trying to get some samples to understand how to use it.
The document I will have put together as a draft in a few days, when I have better understood the implications of the Library for a complete newcomer to AutoHotKey.
Should I publish the document on the forum when it is ready, or is there a possibility of sending it to you privately.
I should add that I am singularly impressed by what I have seen tthus far.
Regards, forkinpm.

_________________
I am building my platform based on a set of steps to take my English texts through segmenting into phrases, resequencing segments based on German rules, adding head-words and then translating with my dictionary.
Regards, forkinpm.

Top

SoLong&Thx4AllTheFish

Post subject: Re: Post on TF Library to Hugo V

Posted: October 28th, 2009, 10:24 pm

Joined: May 27th, 2007, 9:41 am
Posts: 4999

forkinpm wrote:

I would like to use the library and am trying to get some samples to understand how to use it.
... Should I publish the document on the forum when it is ready, or is there a possibility of sending it to you privately.

There are some basic examples here http://www.autohotkey.com/forum/viewtop ... 363#280363 (second post in this thread) and you can use the forums PM function (if you are signed you can click PM button to send me something in private if you wish)

_________________
AHK FAQ
TF : Text files & strings lib, TF Forum

Top

Tuncay

Post subject:

Posted: October 29th, 2009, 8:42 am

Joined: November 7th, 2006, 9:47 pm
Posts: 1934
Location: Germany

Here are regex based variants of TF_Tail():

Code:

string_getTail(ByRef _string)
{
   RegExMatch(_string, "S`a)(*ANYCRLF)(.*)$", string)
   Return string
}

file_getTail(_TextFile)
{
   FileRead, content, %_TextFile%
   RegExMatch(content, "S`a)(*ANYCRLF)(.*)$", content)
   Return content
}

I think, they are faster. The string variant works on variables, which is faster and usable in more cases. A set of ErrorLevel can be made easily by adding a "if" at RegExMatch.

I don`t like this version:

Code:

TF_Tail(TextFile, Lines = 1) ; has dependency on CountLines / ReadLines
   {
    Lines--
    Endline:=TF_CountLines(TextFile)
    StartLine:=EndLine - Lines
    OutPut:=TF_ReadLines(TextFile, Startline, EndLine)
    Return Output
   }

TF_CountLines(TextFile) ; was TotalLines in v1
   { 
    FileRead, Str, %TextFile%
    StringReplace, Str, Str, `n, `n, UseErrorLevel
    Return ErrorLevel+1
   }

TF_ReadLines(TextFile, StartLine = 1, EndLine = 0)
   {
    TF_MatchList:=_MakeMatchList_(TextFile, StartLine, EndLine) ; create MatchList
    FileRead, Str, %TextFile%
    Loop, Parse, Str, `n, `r
      {
       If A_Index in %TF_MatchList%
         OutPut .= A_LoopField "`n"
       Else if (A_Index => EndLine)
         Break
      }
    Return OutPut
   }

After all, it opens the file twice.

Top

SoLong&Thx4AllTheFish

Post subject:

Posted: October 29th, 2009, 9:22 am

Joined: May 27th, 2007, 9:41 am
Posts: 4999

There are already various tail solutions on the forum, some even with benchmarks, you should know :wink:

I already have done a complete re-write of the TF_Tail function, it now has some extra features (more on that later)

_________________
AHK FAQ
TF : Text files & strings lib, TF Forum

Top

Tuncay

Post subject:

Posted: October 29th, 2009, 9:28 am

Joined: November 7th, 2006, 9:47 pm
Posts: 1934
Location: Germany

Oh sorry then. I have the download-link from the new documentation page. Are the new rewritten source available currently?

Top

SoLong&Thx4AllTheFish

Post subject:

Posted: October 29th, 2009, 9:31 am

Joined: May 27th, 2007, 9:41 am
Posts: 4999

No it is not available yet, I'm aiming for a Halloween beta* release :wink:

And your input/feedback is appreciated, so no need to be sorry at all!

*Edit: beta as in it will surely make you scream at my horrid coding skills and all the bugs :wink:

_________________
AHK FAQ
TF : Text files & strings lib, TF Forum

Top

SoLong&Thx4AllTheFish

Post subject:

Posted: November 2nd, 2009, 9:43 pm

Joined: May 27th, 2007, 9:41 am
Posts: 4999

PROCEED WITH CARE:

For those with a taste for adventure, you can test a version of TF
that supports both files AND variables ...

EDIT 24 November 2009: link removed will work in final release in
the next few days

Before proceeding read the following carefully to understand how
to use it. Also note the examples included in the post below,
this will demonstrate how to use _TF with files and variables.

I only TESTED this and have not yet had an opportunity to use it
in day to day tasks/scripts, so your milage may vary.

Thanks to Majkinetor & Tuncay for the suggestions and code snippets,
any errors are mine :-)

Do note that there may be significant
changes in the final release based on the feedback so for your every
day tasks it is advised to use the regular TF.AHK as linked in the
first post of this thread.

During the testing phase you can keep on using the original tf.ahk
in your library because all functions have been renamed and use
a proceeding _. So TF_ReadLines is temporarily _TF_ReadLines

The new concept of variables (while keeping backward compatibility)
is probably best illustrated by some easy examples.

NEW: TF()

In order to save you the trouble of using a fileread to read a file
into a variable and proceed to use various TF functions there is the

TF() Function:

Quote:

TF(pathtofile, byref t="")
By default it read contents of file in a global variable t and returns it,
it creates variable t if a variable name is omitted

Examples:

Code:

TF("file.txt") ; will create globar var t
TF("file.txt", MyVar) ; will create globar var MyVar

I suspect the TF() will generate most feedback as people might not like
it that it generates global var t. It is quite possible I'm completely
missing the point here, so any feedback and or suggestions for improvements
are welcome.

Basic examples to illustrate the use of files & variables, more examples in the post below

Code:

#Include _Tf.ahk

TestFile= ; create variable
(join`r`n
1 Hi this
2 a test variable
3 to demonstrate
4 how to 
5 use this
6 new beta version
7 or alpha version of TF.AHK
)
FileDelete, _TestFile.txt
FileAppend, %TestFile%, _TestFile.txt ; create file
F=_TestFile.txt ; just a shorthand code for _TextFile.txt

;pass on file, read lines 5 to end of file:
MsgBox % "From File 1:`n" _TF_ReadLines("_TestFile.txt",5) ; same as before
MsgBox % "From File 2:`n" _TF_ReadLines(F,5)               ; same as before

;pass on variable, read lines 1 to 5
MsgBox % "From Variable 1:`n" _TF_ReadLines(TestFile,"1-5")     
MsgBox % "From Variable 2:`n" _TF_ReadLines("Hi`nthis`nis`na`ntest`nvariable`nfor`ntesting",1,3)

;Examples using TF(), uncomment both lines:

TF("_TestFile.txt") ; read file, create globar var t
t:=_TF_ReadLines(t,5) ; pass on global var t created by TF(), read lines 5 to end of file, asign result to t 
MsgBox % "TF(), example 1:`n" t

TF("_TestFile.txt", MyVar) ; read file, create globar var MyVar
MyVar:=_TF_ReadLines(MyVar,5) ; pass on global var MyVar created by TF(), read lines 5 to end of file, asign result to MyVar
MsgBox % "TF(), example 2:`n" MyVar

t:=_TF_ReadLines(TF("_TestFile.txt"),5) ; pass on global var t created by TF(), read lines 5 to end of file, asign result to t 
MsgBox % "TF(), example 3:`n" t

t:=_TF_ReadLines(TF(F),5) ; pass on global var t created by TF(), read lines 5 to end of file, asign result to t 
MsgBox % "TF(), example 4:`n" t

TF_MakeFile, TF_Merge, TF_Prepend, TF_Append, TF_SplitByLines/Text currently do not support variables and only work with FILES.

------------------------------------------------------------------
Note: this post will be removed once the testing is over and I'm
satisfied it works as advertised. At that point this temporary
release will be erased ....
------------------------------------------------------------------
Once this TF v3 is finished I can finally add more functions :-)

Edit Nov 4: I will merge some helper functions in the next few days and update the _TF test lib.

Edit Nov 6: Made some good progress, testing the new merged helper functions which also has made the library nearly 10k smaller :shock:

_________________
AHK FAQ
TF : Text files & strings lib, TF Forum

Last edited by SoLong&Thx4AllTheFish on November 24th, 2009, 3:33 pm, edited 4 times in total.

Top

SoLong&Thx4AllTheFish

Post subject:

Posted: November 2nd, 2009, 9:43 pm

Joined: May 27th, 2007, 9:41 am
Posts: 4999

edit: removed test code, no longer accurate with new lib comming up

_________________
AHK FAQ
TF : Text files & strings lib, TF Forum

Last edited by SoLong&Thx4AllTheFish on November 25th, 2009, 9:11 pm, edited 1 time in total.

Top

Tuncay

Post subject: About _TF-library

Posted: November 2nd, 2009, 10:39 pm

Joined: November 7th, 2006, 9:47 pm
Posts: 1934
Location: Germany

About _TF-library
I had a quick look in source file. Call of FileExist() should be avoided if possible, because any file operation takes very long. At my machine just to check if file exist take some time (not sure why, its not an old machine). That is not only to TF, it is with everything on my computer.

Suggestion:

Code:

2TF_FileOrVar(byref TextFile) ; HugoV, helper function to determine if VAR/TEXT or FILE is passed to TF
{
   OW=0 ; default setting: asume it is a file and create textfile_copy
    If (SubStr(TextFile,1,1)="!") ; first we check for "overwrite" 
   {
      TextFile:=SubStr(TextFile,2)
      OW=1 ; overwrite file (if it is a file)
   } 
   ; Call of FileExist() should be avoided if possible, because any file operation takes very long. At my machine 
   ; just to check if file exist take some time (not sure why, its not an old machine).
   StringReplace, TextFile, TextFile, `n, `n, UseErrorLevel
   If (ErrorLevel = 0 AND FileExist(TextFile)) ; now we can check if the file exists
   {
      If (OW=1) ; the variable started with a ! so we need to put it back because it is variable/text not a file
      {
         TextFile:= "!" . TextFile
      }
      OW=2 ; no file, so it is a var or Text passed on directly to TF
   } 
   Return OW ; returns OW used later to determine what (file or var) and how to return output (overwrite or copy)
}

untested!

Another suggestion is, global variable not to be set to "t". This is very unsure, I recommend to name it something no other script would use. It does not need to be easy to remember, because it is internally used only (I think).

Still, I had not time to test the TF-library.

As a sidenote, i have much trouble to read you source code because the indentation is very unfamiliar and irritating to me.

Enough criticism.

Keep your work, many users need this. Although I am not fully accepting and dont like all concepts, but this is secondary. Functionality have more priority to me.

Top

SoLong&Thx4AllTheFish

Post subject:

Posted: November 27th, 2009, 11:10 pm

Joined: May 27th, 2007, 9:41 am
Posts: 4999

V3 Complete overhaul of library, now accepts files & variables for input and output:

- Changed: New parameter for TF_Readlines & TF_Tail: trailing new line now optional
- Added: TF_Save, shorthand for filedelete+fileappend - HugoV
- Added: TF_GetData, helper function to determine if a file or a variable is passed on to function - HugoV
- Added: TF() To read a text file in global var, t by default - Credits various ...
- Added: TF_ReturnOutput has replaced: Overwrite, MakeCopy and the newly developed ReturnStr - HugoV
- Changed: Complete rewrite of TF_Tail, new options - borrowed bits from Tuncay (Thanks!)
- Changed: MakeMatchList: Removed TF_Countlines (one less fileread), Pass on "String" and not a TextFile
- Fix: TF_ConCat & TF_MakeFile didn't write output file :-(

same bug as splitby* earlier.

See first post for documentation and download link

@Tuncay:
- I've borrowed bits & pieces from your Tail & readLine functions. TF_Tail can now retrieve "3rd to last line" for example and also exclude empty lines
- With help I've solved the global var problem you also addressed
- Haven't looked at the if fileexist slownes problem you report, will do at some point. Thanks for your suggestions & tests

@All:

I've tried to test the v3 library extensively, but if you notice different behaviour compared to prior versions please report any bugs or strange behaviour. And suggestions are always welcome.

_________________
AHK FAQ
TF : Text files & strings lib, TF Forum

Last edited by SoLong&Thx4AllTheFish on November 27th, 2009, 11:30 pm, edited 1 time in total.

Top

Tuncay

Post subject:

Posted: November 27th, 2009, 11:28 pm

Joined: November 7th, 2006, 9:47 pm
Posts: 1934
Location: Germany

Good news. I am glad if you use any of my code in your project.

About slowdown with FileExist... like said, this applies not only to TF-library. This is a common problem (if we can speak about problem, in this case it isn`t). You should not focus to this, this is not important as the whole functionality.

Later if you release version 3 AND if I have more time (busy with my own libraries), then I will have more than one look into it. I look forward to see it. Testing intensively is a very good idea. Often I have made the mistake just to test superficial and found afterwards many BUGs. Good luck.

_________________
{1:"ahkstdlib", 2:"my libs", 3:"my apps", 4:"my license"}
--> Don't feed the troll! <--

Top

Page 5 of 13

[ 194 posts ]

Go to page Previous 1, 2, 3, 4, 5, 6, 7, 8 ... 13 Next

Board index » AutoHotkey » Scripts

All times are UTC [ DST ]

Who is online

Users browsing this forum: bekihito, Bing [Bot] and 9 guests

You can post new topics in this forum
You can reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum