Program Size limitation?

[James Zukowski]

We're still upgrading many of the program from our legacy system which uses explicit IOLISTs in the programs. As we update file definitions in Nomads, we also need to update some of these IOLISTs. To help, we've got a program that runs through the Nomads Data Dictionary and creates an IOLIST statement for the defined files, including non-normalized variants. These are loaded out to a line-numbered text file, then run through "*pg.cnv" to convert to a tokenized program.

Trying to run it today, I get an Error #19: Program size too large. The program itself is only about 74K. But is there a limit on the number of variables or the size of the variable table? From what I see, we've got about 3200 different variable names defined in the program (before it stopped accepting more). The program itself consists of !Comments and IOLISTs.

Any insights would be appreciated. Thank you!

[Mike King]

Are you including line numbers on the output program?  If so make sure the line number remain less than 65000.

There are a number of limits on program.  For example the total size of the variable name table on a program along with the line label table.  The variable name table consists of all variable names in the program plus 3 byte per name.  This table cannot exceed 32K so for your 3200 variables the average name length needs to be less than 7 bytes.

Also you will get an error if a line exceeds 32000 bytes.

And regarding your approach....

I don't quite understand what you are going to do with a single program containing all your IOLISTs.  Why are you not simply adding the IOLISTs to the actual file themselves?  The Nomads data dictionary does allow for non-normalized files (i.e. a variant file with multiple record formats) along with an Alternate IOLIST.

Alternatively, rather than having one LARGE program with all the IOLISTs and  you don't want to embed the IOLISTs into the files, why not put the IOLISTs (in text of CPL format) on a file referenced by file and record format.  You could then take a number of approaches to access the IOLISTs in your code from something like IOL=REC("Formats",KEY="File:Format") or use an Object/Function to open the files and set the IOLIST.

[James Zukowski]

Already accounting for line numbers and lengths. Yes, the total number of variables plus the additional 3 bytes brings the total to over 40K.
The intent was to have a single repository of IOLISTs for use in programs that don't open files with iol=*. An updated IOLIST could be retrieved and dropped into the program, if needed.
Thanks for the clarification.

[Loren Doornek]

Mike, is there any way to see what the current variable name table length is?  I have a few programs that are essentially several programs combined into a single programs, which makes for easier installation and maintenance.  The programs are fairly large, but that hasn't been a problem. The variable name table could be pushing the limit, though, since there are a lot of different variables stored in the program, and I'm notorious for using long descriptive variable names :-) 

I used the LV system command to list all of the variables in one program, and found 2700 variables with a total variable name length of 26104.  Using that, the variable name table should be (2700*3)+26104=34204.  That already exceeds 32k but I'm still able to run the program, so I'm guessing that my method of calculating the table length is wrong. 

I'd like to know what the length is so that I can start splitting up these programs, if necessary, before hitting any limits.

[Mike King]

Technically the variable name table size is the sum of the lengths of the variable names plus 1 byte.  So 100 variables all of which are 6 bytes long would be 100 * (6 + 1).  However global variables (due to the leading %) would take 1 more byte for the name length as would Integer variables (due to trailing %) so potentially its variable name length plus 2 if all your variables were global integers.  (Basically the leading/trailing % is considered part of the variable name).

Now as for finding out the current size of a variable name table, it is stored in the program file header.  If you subtract the start of the Variable name table (header bytes 105,4) from the start of the next table (bytes 101,4) you can find the current length.

[Loren Doornek]

Perfect!  Thanks, Mike!

I read the header into r$, and got the value.  Looks like I have room to add a few more really_long_variable_name$

-}?dec(mid(r$,101,4))-dec(mid(r$,105,4))
27173