• Welcome to Jose's Read Only Forum 2023.

An Unicode reflexion

Started by José Roca, April 17, 2011, 09:58:02 PM

Previous topic - Next topic

0 Members and 1 Guest are viewing this topic.

José Roca

The problem with the "old farts" is that they want to continue writing legacy code. Despite my advice, if they write a new program they will still use ansi strings. Hey! Adding a "W" is not so hard!

Patrice Terrier


As i see it myself, working with different languages:
I have no problem using my DLLs written with PB9 and ANSI, for example WinDev, while written itself in C++, is still using ANSI as default.

Converting all my code to UNICODE, is not a trivial task that means only changing everything to WSTRING and WSTRINGZ, and i will not do that until PB10 becomes more mature, and 64-bit.

Patrice Terrier
GDImage (advanced graphic addon)

José Roca

I'm talking of the calls to the Windows API functions. If you use ansi strings in your exported functions it's your business. When calling a "W" API function it is a matter of using an string literal, an unicode variable or use BYCOPY if it is an ansi string. PB implementation of unicode allows mixing ansi and unicode in an expresion, and conversion is automatic when you assign a variable of one type to another.

Theo Gottwald

As said Jose, I am a Fan  of the new Idea of FF 4 and CWindows. If this is going to be pure Unicode I'll be surely in the boat.

On the other side there is a lot of "old code", i also have libraries which i may need to change.

FF4 is not yet available, and when it's available and bug free we'll see more far.

I have however now understand your clear announcement, Jose.
It's logic, its the direction to go.

I'll also follow that path and from now think of "UNICODE".

Having said this, i know that i am one of those people learning new things with time not by revolution.

After reading you explanation, i have definitely decided that new projects i make will all be UNICODE.

José Roca

Oh, finally! Everybody has been talking of converting ansi code to ansi code! What for? If your code is already working as you want, why you have to convert it? I have legacy code such the TypeLib Browser that maybe I will convert to unicode one day or maybe not. As long as it is working fine, I don't need to convert it. I'm talking of new code that takes advantage of the new features. And if you start a new project, why to write it as ansi if you can do it as easily as unicode? And if you are going to write it in unicode, for what you need the ansi stuff? If you want to write it in ansi, you already have the headers for it.

Theo Gottwald

Ok, Jose, how do you actually organize the "old and new header chaos"?

You compile the old stuff - with the old headers.
Do you have an extra editor version with the proper settings - or how do you do it?

You do not want to change the editor settings each time - do you?

José Roca

You don't have to change any settings. CSED allows up to four sets of headers. To change the selection quickly, there is a drop down button in the bottom toolbar. Click the arrow of the button and an small menu allows you to select the ones that you want to use.

You set the paths of the headers with Menu-->Options-->Compiler and Paths dialogs.

José Roca

PB unicode implementation is excellent. You can have your exported functions in ansi and work with unicode internally without doing nothing special, e.g.



  s = strIn & " mixing ansi and unicode"



  strOut = Foo("We are")
  MSGBOX strOut


Theo Gottwald

QuoteYou don't have to change any settings. CSED allows up to four sets of headers.

That will still not help you in case you have backuped the project and want to compile it after 2 years - if you did not also backup the needed includes.

Yourdemonstration of the PB UNICODE support is interesting and as such it looks unspectacualr to work with UNICODE API's only.

Of course it makes sense in such cases to work internally with UNICODE also.

José Roca

I probably will stay with the current version and keep the "A" versions, but I wanted to demonstrate that they aren't really needed and that working with unicode with the new PB compilers is as easy as working with ansi and, therefore, new applications should be all unicode. There are advantages and no disadvantages. The problem is that, keeping the "A" versions, I have to duplicate the wrappers in some cases or add some overhead in the ones that deal with windows and controls calling the API function IsWindowUnicode.

Frederick J. Harris

I just tried an example like I said I would and it works OK Jose - just as you said it would.  I think this situation represents a worst case scenerio that a lot of PB'ers will likely face, i.e., use of an ansi based custom control which one wouldn't have the source code for to convert it to wide character strings.  In the example below (based on one I did for Mike Trader years ago where he wanted multi-color lines in a text box - I showed him how to do it with the SIGrid instead, which he had), I'm using your new 2.01 headers with %UNICODE defined.  The program puts an SIGrid control on its main form, and a button control.  For those who do not know, the SIGrid control was a custom grid control marketed and sold to PowerBASIC users several years ago.  Its a pretty much full featured grid custom control written in PowerBASIC Windows Version 7.  It sold for about a hundred bucks (slang for American Dollars).  Naturally, it uses the Ansi single byte character set.  Ansi strings need to be passed into it so that its internal creation/initialization code can create the grid columns, grid column captions, etc.  Like any custom control the control is created with a CreateWindowEx Api call.  Column label and width information is typically passed into the grid through the lpWindowName (Window Caption, i.e., 3rd param of CreateWindowEx) parameter of the CreateWindowEx() call.  That info is likely retrieved in the WM_CREATE handler of the grid, and it needs to receive Ansi strings.

Everything seems to work.  Note I included a button on the Form which when pressed writes some data to the grid which uses various colors.  I create the button with CreateWindowA in an otherwise fully unicode app.  Here's the code...

#Compile       Exe
%UNICODE       = 1
#Include Once  "windows.inc"
#Include       "Commctrl.inc"
#Include       "Sigrid.inc"
%IDC_BUTTON    = 1500  'Button Control ID
%IDC_GRID      = 1505  'Grid Control ID

Type WndEventArgs
 wParam As Long
 lParam As Long
 hWnd   As Dword
 hInst  As Dword
End Type

Declare Function FnPtr(wea As WndEventArgs) As Long

Type MessageHandler
 wMessage As Long
 dwFnPtr As Dword
End Type
Global MsgHdlr() As MessageHandler

Function fnWndProc_OnCreate(wea As WndEventArgs) As Long
 Local pCreateStruct As CREATESTRUCT Ptr
 Local hButton,hGrid,dwGrdStyle As Dword
 Local GrdSetup As siGridSetup
 Local strColSetup As String

 Call InitCommonControls()
 hButton=CreateWindowA("button","Put Data In Grid",%WS_CHILD Or %WS_VISIBLE,250,310,200,30,wea.hWnd,%IDC_BUTTON,wea.hInst,Byval 0)
 strColSetup= _
 "Tree #:50:^" & _               'Tree #
 "|Species:60:^" & _             'Species
 "|Dbh:40:^" & _                 'Dbh
 "|Logs:45:^" & _                'Logs
 "|% Cull:55:^" & _              '% Cull
 "|Grade:50:^:" & _              'Grade
 "|Volume:75:^" & _              'Volume
 "|Problem Description:305:^"    'Problem Description
 GrdSetup.RowCount        = 1                        : GrdSetup.ColCount         = 8
 GrdSetup.CurrentCol      = 1                        : GrdSetup.CurrentRow       = 1
 GrdSetup.FirstRow        = 1                        : GrdSetup.FirstCol         = 1
 GrdSetup.BackColor       = RGB(255,255,255)         : GrdSetup.FixedBackColor   = RGB(192,192,192)
 GrdSetup.DividerColor    = RGB(128,128,128)         : GrdSetup.ForeColor        = RGB(0,0,0)
 GrdSetup.FixedForeColor  = RGB(0,0,0)               : GrdSetup.WorkSpaceColor   = RGB(192,192,192)
 GrdSetup.SelBackColor    = RGB(0,0,128)             : GrdSetup.SelForeColor     = RGB(255,255,255)
 GrdSetup.GridFont        = 0                        : GrdSetup.RowHeight        = 20
 GrdSetup.FixedRowHeight  = 18                       : GrdSetup.ColDelimiter     = Asc("|")
 GrdSetup.CurRowBackColor =RGB(200,200,225)          : GrdSetup.CurRowForeColor  = RGB(0,0,200)
 hGrid=CreateWindowEx(0,"SIGrid",Bycopy strColSetup,dwGrdStyle,0,0,700,300,wea.hWnd,%IDC_GRID,wea.hInst,ByVal VarPtr(GrdSetup))
 Call siSetRowCount(hGrid,50,1)

End Function

Sub PutDataInGrid_OnClick(wea As WndEventArgs)
 Local CellAttribute As siCellAttributes
 Local hGridWnd As Dword
 Register i As Long

 Call siSetRowData(hGridWnd,1,"|300|12|10|0|3|56|No Comment",0)
 Call siSetRowData(hGridWnd,2,"|310|14|15|0|2|75|No Comment",0)
 Call siSetRowData(hGridWnd,3,"|320|16|20|0|1|175|No Comment",0)
 Call siSetRowData(hGridWnd,4,"|400|18|25|0|1|260|No Comment",0)
 Call siSetRowData(hGridWnd,5,"|305|12|10|15|3|56|305 Is An Invalid Species Code",0)
 Call siSetRowData(hGridWnd,6,"|402|24|35|3|1|523|'2' Suffix Here On Species Code Unnecessary!",0)
 Call siSetRowData(hGridWnd,7,"|590|32|45|0|1|1200|This Is One Beautiful Yellow poplar Tree!",0)
 CellAttribute.BackColor=&h000000FF      :  CellAttribute.ForeColor=&H00FFFFFF
 For i=1 To 7
   Call siSetCellAttributes(hGridWnd,5,i,CellAttribute)
 Next i
 CellAttribute.BackColor=RGB(255,255,0)  :  CellAttribute.ForeColor=RGB(0,0,0)
 For i=1 To 7
   Call siSetCellAttributes(hGridWnd,6,i,CellAttribute)
 Next i
 CellAttribute.BackColor=RGB(0,255,0)    :  CellAttribute.ForeColor=RGB(0,0,0)
 For i=1 To 7
   Call siSetCellAttributes(hGridWnd,7,i,CellAttribute)
 Next i
 Call siRefreshGrid(hGridWnd)
 Call siGoToCell(hGridWnd,1,1)
End Sub

Function fnWndProc_OnCommand(wea As WndEventArgs) As Long
 Select Case As Long Lowrd(wea.wParam)
     Call PutDataInGrid_OnClick(wea)
 End Select

End Function

Function fnWndProc_OnClose(wea As WndEventArgs) As Long
 Call PostQuitMessage(0)
 Call DestroyWindow(wea.hWnd)
End Function

Function fnWndProc(ByVal hWnd As Long,ByVal wMsg As Long,ByVal wParam As Long,ByVal lParam As Long) As Long
 Static wea As WndEventArgs
 Register iReturn As Long
 Register i As Long

 For i=0 To 2
   If wMsg=MsgHdlr(i).wMessage Then
      wea.hWnd=hWnd: wea.wParam=wParam: wea.lParam=lParam
      Call Dword MsgHdlr(i).dwFnPtr Using FnPtr(wea) To iReturn
      Exit Function
   End If
 Next i

End Function

Sub AttachMessageHandlers()
 ReDim MsgHdlr(2) As MessageHandler   'Associate Windows Message With Message Handlers
 MsgHdlr(0).wMessage=%WM_CREATE   :   MsgHdlr(0).dwFnPtr=CodePtr(fnWndProc_OnCreate)
 MsgHdlr(1).wMessage=%WM_COMMAND  :   MsgHdlr(1).dwFnPtr=CodePtr(fnWndProc_OnCommand)
 MsgHdlr(2).wMessage=%WM_CLOSE    :   MsgHdlr(2).dwFnPtr=CodePtr(fnWndProc_OnClose)
End Sub

Function WinMain(ByVal hIns As Long, ByVal hPrev As Long, ByVal lpCL As Asciiz Ptr, ByVal iShow As Long) As Long
 Local szAppName As WStringz*16
 Local wc As WndClassEx
 Local hWnd As Dword
 Local Msg As tagMsg

 szAppName="UnicodeTest"                         : Call AttachMessageHandlers()
 wc.lpszClassName=VarPtr(szAppName)              : wc.lpfnWndProc=CodePtr(fnWndProc)
 wc.cbClsExtra=0                                 : wc.cbWndExtra=0
 wc.style=%CS_HREDRAW Or %CS_VREDRAW             : wc.hInstance=hIns
 wc.cbSize=SizeOf(wc)                            : wc.hIcon=LoadIcon(%NULL, ByVal %IDI_APPLICATION)
 wc.hCursor=LoadCursor(%NULL, ByVal %IDC_ARROW)  : wc.hbrBackground=%COLOR_BTNFACE+1
 Call RegisterClassEx(wc)
 hWnd=CreateWindowEx(0,szAppName,"Unicode Test",%WS_OVERLAPPEDWINDOW,200,100,725,400,0,0,hIns,ByVal 0)
 Call ShowWindow(hWnd,iShow)
 While GetMessage(Msg,%NULL,0,0)
   TranslateMessage Msg
   DispatchMessage Msg

End Function

Note above in the CreateWindowEx() call that creates the SIGrid I used Bycopy strColSetup to pass the string info into the grid.  strColSetup is a regular ansi dynamic string.  I did that with the Bycopy because in this thread Jose intimated this as being a solution.  I have no clue whatsoever as to how this works or what it is doing.  I've never in my life before used the Bycopy parameter override.  My thoughs were to pass the starting address of the strColSetup dynamic string like so...

Byval Strptr(strColSetup)

but it didn't work.  When I did that it compiled fine and ran, but the grid failed to create correctly.  It didn't crash, but the area on the form where the grid should have been was blank.  Maybe someone can explain to me what kind of weird science is going on here as to why Bycopy works.  I have to admit I've never used bycopy and have a very poor understanding of what it does.  I read the help file on it and I understand what it is saying, but I fail to understand how to apply that knowledge to this present circumstance. I guess I only understand pushing a parameter's value or address on the stack.  I don't understand this copy business.  And why doesn't Byval Strptr(strColSetup) work?  This all I find quite mysterious, but I do accept that it indeed works.

Being as it does indeed work in what I'd consider to be a worst case scenerio, I have to admit Jose's idea of doing without ansi declares and capabilities seems to be a good idea.  Can anyone provide a piece of code or example where the ansi declares would be needed?


José Roca

I don't understand this copy business.  And why doesn't Byval Strptr(strColSetup) work?

You're creating the window with CreateWindowEx and the %UNICODE flag set. Therefore what will be called is CreateWindoeExW, and the parameter for the title is a WSTRINGZ, whereas strColSetup is a STRING. Using BYCOPY, the compiler creates a temporary string of the kind required by the parameter, so it makes an unicode copy of strColSetup. On the other hand, Windows checks if the target, the SiGrid control, is unicode or not by calling IsWindowUnicode. If it is not, then it converts what you have passed as the caption to ansi (the opposite is also true, i.e. if you call an unicode control from an ansi application, the caption parameter will be converted to unicode). Using BYVAL, you override parameter checking, and Windows gets an ansi string instead of unicode, and it will try to convert it to ansi assuming that it is unicode, and it will fail.

If both applications were unicode, then you will use unicode instead of ansi for strColSetup and no conversions will be needed. This is why using unicode with the Windows API is faster than using ansi.

Additional information from the documentation for IsWindowUnicode:

The character set of a window is determined by the use of the RegisterClass function. If the window class was registered with the ANSI version of RegisterClass (RegisterClassA), the character set of the window is ANSI. If the window class was registered with the Unicode version of RegisterClass (RegisterClassW), the character set of the window is Unicode.

The system does automatic two-way translation (Unicode to ANSI) for window messages. For example, if an ANSI window message is sent to a window that uses the Unicode character set, the system translates that message into a Unicode message before calling the window procedure. The system calls IsWindowUnicode to determine whether to translate the message.

José Roca

Being as it does indeed work in what I'd consider to be a worst case scenerio, I have to admit Jose's idea of doing without ansi declares and capabilities seems to be a good idea.  Can anyone provide a piece of code or example where the ansi declares would be needed?

All the "A" functions have a "W" counterpart, and there are some "W" functions that don't have an "A" counterpart. Besides, some "W" functions have added capabilities, such FindFirstFileW, that allows the use of paths up to 32,767 wide characters if you prepend the path with "\\?\".

If we remove the ansi stuff, we will still using CreateWindowEx and what will be called is CreateWindowExW, so we will be working with unicode transparently. The difference will be that instead of declaring strings to be used in the api calls as STRING or as ASCIIZ, we will use AS WSTRING or as WSTRINGZ. If the target is a parameter, and your variable is an STRING or ASCIIZ, then you can use BYCOPY and leave the conversions to PB and Windows, but this is more inefficient than using unicode natively.

But making your application unicode doesn't mean that you have to use unicode strings for everything. For data that is not being passed to a Windows API function, use wathever you wish.

My concern is that I write a lot of wrapper functions, and if the headers support both the "A" and "W" functions, then I have to duplicate the wrappers for no good reason, and most PBer's will use the more inefficient "A" versions because of the current pandemic of FUD (fear, uncertainty and doubt).

With the new compilers, DDT is fully unicode. Yet, most PBer's will use te ansi API functions against it, which can lead to unexpected results in some cases.

Frederick J. Harris

Using BYVAL, you override parameter checking, and Windows gets an ansi string instead of unicode

Byval Strptr(strColSetup) should have worked based on that idea then.  Would not the address within my process of an ansi string be passed into the grid anticipating an ansi string?

Actually, what I think is going on here is the special case of a Windows Api function call instead of a local function call within my application where I think my reasoning would have been sound.  I think a Local Procedure Call ( LPC ) is taking place with the Api functions where indeed this copying and conversion of string buffers is going on?

Yes, I see it works.  But you know me, I question things!

José Roca

Byval Strptr(strColSetup) should have worked based on that idea then.  Would not the address within my process of an ansi string be passed into the grid anticipating an ansi string?

It does not check if the data passed is ansi or unicode. If you are using CreateWindowExW, this function assumes that you have passed an unicode string, and if the target is an unicode window it will pass it without conversion, but if it is not an unicode window, it will attempt to convert it to ansi. If you want to use BYVAL STRPTR, then strColSetup must be unicode.