• Welcome to Jose's Read Only Forum 2023.

Moving TCLib Into Mainstream

Started by Frederick J. Harris, April 02, 2016, 12:03:13 AM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

Frederick J. Harris

I'd tend to think you would want it to behave like the PowerBASIC Line Input # statement.  If you were reading a text file with Line Input # and outputting the results to the console, blank lines would be "" I assume, but printing them would add a CrLf I further assume.  So wouldn't you want that behavior as an end result? 

Frederick J. Harris

Just as an aside, that situation with CRT gets() has given me a fair amount of grief over the years in that strings end up with the \n at the end instead of its being removed. 

Frederick J. Harris

Also, TCLib seems to work OK with ODBC.  In 64 bit one can't connect to 32 bit Office/Access, but I tested with SQL Server and made the connection no sweat.  I'll test in a bit reading/writing data, but don't see that that will make any difference.

Also, can't seem to get the fprintf/fopen situation out of my head even though I said what I had was good enough.  Just now I tested an fopen / _wfopen, and I changed the fprintf's 1st parameter from HANDLE, which its not supposed to be, to a FILE*, which it is supposed to be, and it works perfectly except for one tiny weenie glitch.  WriteFile wants a "\r\n" for a Crlf whereas fprintf is satisfied with just \n.  Not sure what I want to do about it, if anything.  And for that matter, I could leave what I have alone, if it puts you out a lot in your use of what I provided.

James C. Fuller

Ok I got it.
Could be a lot cleaner.
Remember it was written with bc9Basic so there is a bit of not needed fluff that I didn't remove.
It probably should/could be wrapped in a class?



Frederick J. Harris

Nice Work Jim.  Just looked at it and don't understand it yet but I'll work on it tomorrow.  Actually, I never worked with memory mapped files.  I know you have been mentioning this line input issue in C++ for some time, so I know you've given it a lot of thought.  As I believe I've mentioned already, for text files I've always used CRT fgets, but of course that won't work with TCLib until fgets gets implemented through Win32!  Is there some reason you used memory mapped files to tackle the thing?  Since I don't know anything about them but do know and use the console and file Win32 functions a fair amount, I'm thinking if I were to tackle the issue as you have I'd have likely started there possibly with ReadFile() or something.  Just asking is all. 

I've always bounced around a lot between the CRT functions and Win32 functions for files.  Text files I've always found easiest to deal with using CRT.  Binary and Random Access files go either way but I've a leaning to just use Win32 for those. 

Was just looking at the docs for ReadFile(), and I didn't see anything there regarding any capacity to just read a 'line' up to a CR, LF or CrLf.  So I'm assuming one would have to read the whole file into memory and parse out the lines.  Never tried it.  Seems to me possible though.  With my String::Replace() I could replace each CrLf with a single delimiter such as space, tab, or comma or whatever.  Then use String::Parse() to break it into lines.  Just theorizing here, brainstorming, so to speak.  Don't know if it would work, but I've a sinking feeling it'll prey on my mind 'till I try it.  Tomorrow's another day!   

James C. Fuller

  Normally I'm an C++ STL guy using std
I used to be a minimalist in my younger days but have no real reason anymore.
You, and TCLib somehow awakened the tiny in me :)

As you mentioned ReadFile has no provisions for line input . I do have code that will read and write a buffer that I will post shortly.
I had vaguely remembered something about using MMF for line input and found Don's code along with some variations on Code project.
The best thing with MMF is you can process HUGE 4gb files. This is just asci text files but the text is converted into Unicode fstring's.
I am in the process of writing a class wrapper for it. Damn (good) classes are hard to write. I'm a coder not a designer and have a hard time sometimes with the big picture.

Also this is complete turn around for me ; finding and using Win32 functions instead of  the CRT to do a job. I'm usually doing exactly the opposite.

Having fun though!


Frederick J. Harris

Just occurred to me this morning when I woke up how one would write a Line Input / fgets() with ReadFile().  You would set up a 1024 or 2048 buffer or something, then set up ReadFile() to read one byte at a time, and you would of course write it to the buffer, but you would test for CrLf or end of line or whatever.  Several years back I used ReadFile() that way I think for reading GPS data from a COM port for our data collectors.

James C. Fuller

  First draft of an TCLib MMF input class.
Similar to you I have my own way of attacking c++, especially classes.
I found classes with VB5 and really liked the set, get properties.
When I started learning C++ I really missed them. Yes you can build getters and setters,
but it's not the same. I ran across a c++ template implementation several years ago that worked pretty good
but had quite a bit of overhead. I then discovered and or built (can't remember which) a set of macro's.
I have a similar set for PowerBASIC.
Not quite the same as the good old get/set of VB and PB but close.
The c++ forum regulars  whined loudly but .....

I rarely place methods in the class structure itself.
It looks fine in bc9Basic but... their placement in the c++ source is not usually right after the declarations.
I always preface private variables with m_ which makes it easy to do the props

These are my prop macros

#define PropGet(p,t) const t& p() const {return m_##p;}
#define PropSet(p,t) void p(const t& param_##p){m_##p=param_##p;}
#define PropGetSet(p,t) PropGet(p,t)PropSet(p,t)

When you access you do so without the m_

In the cAliInput class I have
    PropGet(iErr, int);
    PropGet(sErr, fstring);

You can see in main() how their accessed
        fsErr = cai->sErr();
        fsErr.Print( true);

We only have ProGet's here but to set you would use
  fsErr = "ERROR #72"

This is Bc9Basic translated c++ so it does have a bit of fluff.
My tcl implementation is not complete in the translator but not too bad.


James C. Fuller

  Maybe a candidate for addition to the TCLib?
It was written by frankie, one of the PellesC forum administrators. I'm not sure if/where on documentation.
I did not look at too it closely.


Frederick J. Harris

Ahhh yes!  The taste of ultimate and absolute success on this incredible three month journey into finally beating the C++ bloatware issue! 

Over the course of the past week or so I've tested a number of apps of ever increasing size and complexity with TCLib, and everything is succeeding.  ODBC simply works!  Have added fairly good file support with regard to text files which I'll post shortly.  And James Fuller has some nice code using memory mapped files which he has provided and which works with TCLib either directly through C++ or through BCX.   And just now I managed to crack the BIGGIE!  By that I mean the thing I felt that was likely beyond reach, and that is my ActiveX Grid Control. 

I guess it may be four or five years since I originally developed that in PowerBASIC and posted my work here.  With that work I was able to create a pretty fair grid control including all the self-registration code in about 49 k with PowerBASIC.  With UPX that compacted down to 22 k.  I've since incorporated that grid into all my mission critical work here where I live. 

I guess about three years ago, becomming interested in 64 bit development with C++, I re-coded the grid in that language but was not able to get the 32 bit or 64 bit binaries down to anything less than around 87 k I think.  Compacted it was comming in originally at something like 43 k, but then I added a few lines of high DPI aware code to it and blew through some compiler allocation limit and it ended up around 50 k.

Just now after about three days of struggle with TCLib trying to add dll support to it and get the grid to compile/link I finally succeeded and it works!  Its coming in 28 k and compacts with UPX down to 18,432 bytes!  So in 64 bit I'm actually seeing about 4 k smaller than my PowerBASIC 22 k dll.  Here are the lines of code in the *.cpp files...

Server.cpp     260 lines
WinCode.cpp   1365 lines
Grid.cpp      1128 lines
Registry.cpp   210 lines
Strings.cpp    165 lines

That comes to 3,128 lines.  Quite a few other files involved, i.e., *.def, *.idl, *.rc, *.h, etc., but the above files contain the meat of it.  The command line compiling I do with a text file I execute at the command prompt like so...

cl @Grid.txt

And that file looks like this...


/O1 /Os /GS- /GR- /FeFHGrid.dll /LD
/link /DLL /nodefaultlib:libcmt.lib


So with all that a 28k / 18 k binary seems really sweet to me.  I'm really glad I started out on the project of eliminating the C Runtime.  It worked out beyond my wildest hopes!  I'll get to posting the details and modifications as soon as I can for anyone who may wish to experiment with this.  I had to do quite a bit of ripping and tearing at things!

James C. Fuller

Looking forward to your latest.


Frederick J. Harris

Been struggling with the file support.  I see you have been too, and had asked Patrice what he does.  Yesterday I was working with my grid to make sure registration and unregistration was working correctly.  I really got scarred testing that.  I'm always leery of testing registry code.  Once I blew out an entire registry on an XP machine.  Amazingly enough it actually started.  It was completely useless though.  Anyway, that seems to be working fine.  Then I got embroiled again in the whole FILE* thing with my new fprintf/fgets code I haven't posted yet.  I thought I had it all working but now its giving me the fits.  The way I had it if UNICODE was defined, it was writing wide character strings to the file, i.e., creating UNICODE UCS-2 files.  And my fgets() was designed around the idea that all files were ansi, and the wide character version of fgets(), i.e., fgetws() would read an ansi file and convert/return a wide character string using the MultiByteToWideChar() function.  So I had a disjunction there.  If set up that way I wouldn't be able to read the files I just wrote, as my fgetws is looking to read an ansi file and convert that to wide.  So I've got to decide what to do and straighten out the mess.  And tomorrow my wife gets a knee replacement so my time will be a bit disjointed for awhile.

But the grid thing!  Like wow, it works!  Can't believe I managed it.  Don't know how I did it but somehow I did.  There were a couple nasty surprises that I though would finally do me in, but luck was with me I guess. 

Frederick J. Harris

I got my snafu worked out.  Lost a good hour over something stupid.  But it taught me a lesson which I immediately incorporated into the code.  For years and years and years I've been writing functions to determine the length in lines of text files using either CRT or WinApi.  So what happens is you read every byte to the end of the file and count lines.  Then the file pointer is at EOF.  Should the function call SetFilePointer() to the beginning of the file again before returning, or let the calling code do it?  Usually I let the calling code do it under the assumption that maybe the calling code wants the read pointer somewhere else other than at the beginning of the file.  Well, there you have my lost hour and a half!  I forgot to call my version or rewind() which calls WinApi SetFilePointer()!  So I put a call to rewind(), which I implemented in a function called Lof(File*) in fgets.cpp.

In the zip are all the files I modified (I think) from my last post here.  Try Demo25.cpp after rebuilding TCLib with my additions.  I also included Tlib.lib (latest version) if you want to use that.  Updated make file too.  Check out particularly my fprintf.cpp, fopen.cpp, crt_dll.cpp (allows creation of small dlls).  Changes had to be made too to stdio.h, string.h, stdlib.h.  Also had to add memcmp.cpp.  If you use == in QueryInterface it calls memcmp under the hood to check out IIDs. 

Demo25 is fun.  It reads the Demo25.cpp file using fgets and displays it in a GUI scrolling window.  I've attached it.  Let me know if what I've done seems reasonable with files.  I'm thinking it is, but not 100% sure.  Oh!  Your strchr() should be in there too, so if you make the lib be sure to have that present.

James C. Fuller

Good work Fred.
I'll  take a look first thing in the AM.
Yankees take precedent tonight :)


James C. Fuller

  I have just taken a very quick look.
This is what I am using for lof.
It could also be used as a file Exist function.
I almost always check if a file exists before trying to open it.
Is there a line input routine somewhere?
More later.

DWORD lof (const _TCHAR *FileName)
    HANDLE hFile;
    int FSize;
    if(_tcslen(FileName) == 0) return 0;
    hFile = FindFirstFile(FileName, &W32FD);
        FSize = W32FD.nFileSizeLow;
        return FSize;
    return 0;