Hello Charles
i refer to your code in http://www.oxygenbasic.org/forum/index.php?topic=1575.15 (http://www.oxygenbasic.org/forum/index.php?topic=1575.15)
about the Saving and Loading a binary file, you were using numeric arrays.
so i have modified it to do string arrays but was not successful
' Save_LoadFile.o2bas
' http://www.oxygenbasic.org/forum/index.php?topic=1575.15
$ filename "Save_LoadFile.exe"
uses rtl64
' this program save array data into a random access file and then load
' the file data into another array
uses corewin
function LoadFile(string name, any*data, int e)
===============================================
sys f
f=fopen name,"r" 'open for reading
e=fread @data,1,e,f 'load data
fclose f 'close file
return e
end function
function SaveFile(string name, any*data, int e)
===============================================
sys f
f=fopen name,"w" 'open for writing
e=fwrite @data,1,e,f 'save data
fclose f 'close file
return e
end function
'TEST
=====
string v[100], w[100]
' Save the data from array V[ ] into the file myfil.bin
for i=1 to 100 : v[i]= str(i*1.5) : next
V[61] = "test string1"
V[71] = "test string2"
SaveFile "myfil.bin",v,bytesof v
' Load the data from myfil.bin to array W[ ]
LoadFile "myfil.bin",w,bytesof v
print w[51]
print w[61]
print w[69]
print w[71]
particularly the codes which i have changed from numeric arrays to string arrays
maybe the statement SaveFile "myfil.bin",v,bytesof v is incorrect as i'm using strings
what would be its replacement? Thanxx and appreciate your help
Hi Chris,
The problem is that LoadFile and SaveFile will only work for flat data. An array of strings is essentially an array of pointers - so you end up saving a bunch of pointers without the accompanying data. For random access in a file, you need a table of locations and lengths, followed by the string content of each element.
As this is a fairly common situation, let me see if I can come up with a generic solution...
I think I may have posted this before?
It takes data from a csv file and creates a binary file.
James
'=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*
' not needed with O2RadAsm
'% filename "cFileIO.exe"
'=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*=*
use rtl64
#autodim off
use console
use corewin
'------------------------------------------------------------------------------
'GetFields function from modified OxygenBasic\examples\DataProcessing\GetFields.o2bas app
include "GetFields.o2bas"
'------------------------------------------------------------------------------
Type AddressType
FirstName As zstring * 32
LastName As zstring * 32
Company As zstring * 64
Address As zstring * 64
City As zstring * 32
County As zstring * 32
State As zstring * 4
Zip As zstring * 12
Phone1 As zstring * 24
Phone2 As zstring * 24
Email As zstring * 64
Url As zstring * 64
End Type
'==============================================================================
Macro LineInput string (s,c)
s = nuls 1024
s = (char*)fgets(s,1024,c)
End Macro
'==============================================================================
Function main() As sys
string sfileIn = "Myus-500tab.csv"
string sfileOut = "Address50.bin"
string sLine
sys fp1,fp2,fields,Count
Dim As AddressType Address
int bytes,where
fp1 = fopen sfileIn,"r"
If not fp1 Then
printl "PROBLEM #1"
waitkey
exit function
End If
fp2 = fopen sfileOut,"w"
If not fp2 Then
printl "PROBLEM #2"
fclose fp1
waitkey
exit function
End If
Do
sLine = LineInput(fp1)
If Len(sLine) = 0 Then
fclose(fp1)
fclose(fp2)
printl "PROBLEM #3"
waitkey
exit function
End If
Count++
If Count > 50 Then
break
End If
fields = GetFields(sLine,chr(9))
With Address
.FirstName = arg(1)
.LastName = arg(2)
.Company = arg(3)
.Address = arg(4)
.City = arg(5)
.County = arg(6)
.State = arg(7)
.Zip = arg(8)
.Phone1 = arg(9)
.Phone2 = arg(10)
.Email = arg(11)
.Url = arg(12)
End With
fwrite(Address,sizeof(Address),1,fp2)
loop
fclose fp1
fclose fp2
printl "Is it done?"
Printl "Now lets read the first record from the binary file" cr "press any key"
waitkey
fp1 = fopen sfileOut,"r+"
If not fp1 Then
printl "PROBLEM #4"
waitkey
exit function
End If
fread Address,sizeof(Address),1,fp1
Printl "First Name -> " + Address.FirstName
Printl "Last Name -> " + Address.LastName
Printl "State -> " + Address.State
Printl ""
Printl "Now lets read the last record" + cr + "any key to continue"
waitkey
where = 49 * sizeof(Address)
fseek fp1,where,0
fread Address,sizeof(Address),1,fp1
Printl "First Name -> " + Address.FirstName
Printl "Last Name -> " + Address.LastName
Printl "State -> " + Address.State
fclose fp1
Printl cr "any key to continue"
waitkey
End Function
main
Thanxx a lot Charles and James
Yes James, you did send me a similar file name cFilieIO.o2bas together with its csv files
in this case, i was checking whether an alternative statement such as SaveFile can be use or not.
i think your random file method is a better one. i won't explore an alternative statement anymore.
This is what I had in mind. It converts an array of strings into a single string containing a table of lengths and offsets, followed by the raw strings' characters.
In this format, string arrays are easy to save and load, or to access strings directly from disk file:
'2018-04-30 T 21:19:28
function FlatTable(string s[], int n) as string
===============================================
'FORMAT:
'number of strings
'length of char data block
'for each string:
' length of string in bytes
' offset of first string character
'char data block
'
indexbase 1
string bt 'to hold the table
string bu 'to hold the char data
int *bd 'dynamic table
int lu 'length of char data buffer
int di=2 'index for the table elements
int offs=8+n*8 'base offset to the char data
int ii 'offset to the char data
int i 'iterator etc
bt=nuls 8+n*8 'create table buffer
@bd=strptr bt 'map table to its buffer
for i=1 to n
le=len s[i]
if ii+le+2>lu then
bu+=nuls le+0x1000 'expand buffer
lu=len bu
end if
mid bu,ii+1,s[i]
di++
bd[di]=le 'data length
di++
bd[di]=ii+offs 'char index (base 0)
ii+=le+2 'next position, allowing 2 null bytes
next
bd[1]=n 'count of elements
i=8+n*8 'length of table
bd[2]=ii 'length of data block
return left(bt,i)+left(bu,ii)
end function
'TESTS
======
string s[100]
for i=1 to 100 : s[i]="#" i*100 : next
string t=FlatTable(s[],100)
'DECODE AND EXTRACT STRINGS
int d at strptr t
redim string da(d)
int j=2
int le
for i=1 to d
j++ : le=d[j]
j++ : da[i]=mid t,d[j]+1,le
next
print da[99] '#9900
Thanxx a lot Charles
Beauty, when added the last lines as below, it just came out instantly
print " at index 99 : " +da[99] + chr(13,10) +
" at index 19 : " da[19] + chr(13,10) +
" at index 3 : " da[3]
Yup this is a much faster method than read and write into a disk file for each record
just use the entire string to do it in memory. so the modus operandi would be :
1. at the begining of the program, load the entire file into a string
2. do some data processing like update records etc
3. at the end of the program just save this single string into the disk file
i wonder how big would be file size that can be loaded into a single string. what would be the file size limit?
Megabyte files, no problem. With Gigabyte files, you would have to handle at low-level, to avoid large intermediate strings.