I would analyse the timings since to create a 250 MB serial file should not take long. In fact I ran the following program on my Windows system
0010 SERIAL "abcdef"
0020 OPEN PURGE (1)"abcdef"
0030 LET s=TMR(0)
0040 FOR i=1 TO 2500000
0050 PRINT (1)DIM(100) ! 100 characters
0060 NEXT i
0070 PRINT TMR(0)
And it took about 3.5 seconds to create the 250 MB file. ;D
Given this its likely either the reading of the EFF file or the computations on the data that is consuming the time.
A few things you can try to speed up processing:
- Try using an OPEN LOAD against the EFF file. This will eliminate any checking for records currently being updated. A standard READ always first checks to see if the record is locked by another process which can slow it down.
- If no other process needs the file at that time try an OPEN LOCK which also speeds up processing. If you are running on Linux and you need concurrent access you could try to make a copy at the OS level first and use OPEN LOCK on the copy.
- If the file does not have to be in any particular record sequence try read the file using the PHYSICAL read next as opposed to reading by key. This can be done by specifying KNO=* on your read.