Thursday, September 5, 2013

Externally described Data Structures

I have been racking my brain to try to remember when I first came across Externally Described Data Structures in RPG. I think it must have been when I transitioned from being a RPG II programmer on the IBM System/36 to a RPG III programmer on an AS400 (yes, I am that old). The others I was working with at the time were all former IBM System/38 programmers and were "fluent" in RPG III. They did not use Externally Described Data Structures. and I was the "newbie" learning RPG III from them, so neither did I.

It was while I was working on a project programming in Synon/2E, and later modifing Synon generated RPG code, did I find them used everywhere. Synon uses Externally Described DS for passing parameters, saving the values of fields in files, and for the Program Status DS (PSDS).

What do I use Externally described Data Structures for?

Every program I write I insert an Externally described DS for the Program Status DS as I find that it leads to a fixed naming convention of the PSDS fields, which makes it easier for myself and others to understand at a later date.

If I was to key the PSDS into the D-spec, Definition specifications, it would look something like:

DPgmDs           SDS                  qualified
D Proc_Name         *PROC
D Pgm_Status        *STATUS
D Prv_Status             16     20S 0
D Line_Nbr               21     28
D Routine           *ROUTINE
D Parms             *PARMS
D Excp_Type              40     42
D Excp_Num               43     46
D Pgm_Lib                81     90
D Excp_Data              91    170
D Excp_Id               171    174
D Date                  191    198
D Year                  199    200S 0
D Last_File             201    208
D File_Info             209    243
D Job_Name              244    253
D User                  254    263
D Job_Num               264    269S 0
D Job_Date              270    275S 0
D Run_Date              276    281S 0
D Run_Time              282    287S 0
D Crt_Date              288    293
D Crt_Time              294    299
D Cpl_Level             300    303
D Src_File              304    313
D Src_Lib               314    323
D Src_Mbr               324    333
D Proc_Pgm              334    343
D Proc_Mod              344    353

I created a file, called RPG4DS, that contains the subfields for the PSDS. This can be inserted as an Externally described DS into the source code like this:

D PgmDs         ESDS                  extname(RPG4DS) qualified

As this is an Externally described DS there has to be an 'E' in position 22, and the EXTNAME keyword is used for the name of the file.

The file, RPG4DS, is just a DDS file that looks like this:

A          R DUMMY
A            PROCNME       10A         COLHDG('Procedure' 'Name')
A            STSCDE         5S 0       COLHDG('Status' 'Code')
A            PRVSTSCDE      5S 0       COLHDG('Previous' 'Status' 'Code')
A            SRCLINNBR      8A         COLHDG('Source' 'Line' 'No,')
A            EXCPTSUBR      8A         COLHDG('Exception' 'Subroutine')
A            NBRPARMS       3S 0       COLHDG('No. of' 'Parameters')
A            EXCPTTYP       3A         COLHDG('Exception' 'Type')
A            EXCPTNBR       4A         COLHDG('Exception' 'No.')
A            RESERVED01     4A         COLHDG('Reserved')
A            MSGWRKAREA    30A         COLHDG('Message' 'Work' 'Area')
A            LIB           10A         COLHDG('Library')
A            RTVEXCPTDT    80A         COLHDG('Retrieved' 'Exception' +
                                              'Data')
A            EXCPTID        4A         COLHDG('Id. of' 'Exception' +
                                              'RNX9001')
A            UNUSED0001    16A         COLHDG('Unused')
A            DTEJOBENTR     8A         COLHDG('*DATE' 'Job entered' +
                                              'System')
A            CNTJOBENTR     2S 0       COLHDG('Century' 'Job entered' +
                                              'System')
A            LASTFILEOP     8A         COLHDG('File last' 'operation' +
                                              'Performed')
A            FILESTS       35A         COLHDG('Status' 'of' 'file')
A            JOBNME        10A         COLHDG('Job' 'Name')
A            USER          10A         COLHDG('User' 'Profile')
A            JOBNMBR        6S 0       COLHDG('Job' 'Number')
A            DTEJOBENT2     6S 0       COLHDG('Date job' 'Entered' +
                                              'System')
A            DTEPGMRUN      6S 0       COLHDG('Date' 'Program' 'Running')
A            TMEPGMRUN      6S 0       COLHDG('Time' 'Program' 'Running')
A            COMPILEDTE     6A         COLHDG('Compile' 'Date')
                                              'Data')
A            COMPILETME     6A         COLHDG('Compile' 'Time')
A            COMPILELVL     4A         COLHDG('Level' 'of' 'Compile')
A            SRCFILE       10A         COLHDG('Source' 'File' 'Name')
A            SRCLIB        10A         COLHDG('Source' 'Library')
A            SRCMBR        10A         COLHDG('Source' 'Member')
A            MODULEPGM     10A         COLHDG('Program' 'Containing' +
                                              'Module')
A            MODULEPROC    10A         COLHDG('Module' 'Containing' +
                                              'Procedure')
A            UNUSED0002    76A         COLHDG('Unused')

Another example of a good use of an Externally described DS is to save the values from a record in a file. This can be coded in just a few lines like this:

01 FORDHDRP    IF   E           K DISK
02 D Sav           E DS                  extname(ORDHDRP)
03 D                                       qualified
04 D New           E DS                  extname(ORDHDRP)
05  /free
06    read(e) ORDHDRP ;
07    Sav = New ;
08    read(e) ORDHDRP ;

On line 2 the data structure 'Sav' is defined as an Externally described DS of the file ORDHDRP. Notice that on line 3 the QUALIFIED keyword is used. This adds the name of the data structure to the front of the field name followed by a period (.), for example ORDNO becomes Sav.ORDNO .

On line 4 the data structure 'New' is defined in the same manner as 'Sav', but without the QUALIFIED, therefore, ORDNO is still ORDNO.

When the file is read, line 6, the 'New' DS is loaded with the values of the fields from the file.

Line 7 moves the entire contents of the 'New' DS to 'Sav'. Now all the subfields in 'Sav' have been updated with the values from 'New', and the file.

When the file is read again, line 8, the value of the subfields in the 'New' DS are the same as fields in the new record of the file. But the values in the 'Sav' DS are unchanged.

As this is only a brief introduction to Externally described Data Structures I am sure you can think of many other uses for them.

Warning:  If you convert a RPG III program using the CVTRPGSRC command and the source member contains the PSDS as an Externally described DS the conversion will complete successfully. But as the layout of the RPGIII and RPGLE/RPGIV PSDS are different you are going to have run-time issues. You need to change the new source member to use a RPGLE/RPG IV compatible PSDS.

You can learn more about this on the IBM web site:

 

This article was written for IBM i 7.1, and it should work with earlier releases too.

14 comments:

  1. How about using externalized DS for data area formatting for CLP?

    Obviate need for DCL?

    ReplyDelete
  2. Hi Simon - I was able to use externally described data structures to capture the definition of formats from a spool file, in order to write that information to a physical file, and then process that physical file to generate a PDF of the spool file with some additional characteristics that were missing from the spool file (logos, font changes, highlighting, color, etc.).

    ReplyDelete
  3. Why an externally described structure rather than a copy member, which is how I bring in the program data structure? An external structure does have language independence, which makes it useful if you want to bring in a data structure of system value in COBOL and RPG, and it is great for using with IO, and it is easy to bring in the same thing several times if you need to. But a copy can contain a number of things generally used together, and include useful constants like word names for the file statuses. And the field names can be over 10 bytes (yes, I know late releases can use aliases.)

    ReplyDelete
  4. I agree with Lynne about using a copy file instead for things like the PSDS, or any data structure that isn't related to an actual file in the application.

    A copy file has the same advantages of naming-consistency as an externally-described file.

    And as Lynne says, with a copy file you have additional ways to help programming consistency, like named constants, and prototypes for procedures to do some common PSDS-related tasks like say sendLockedRecordMsg.

    ReplyDelete
  5. Lynne, for PSDS a /COPY member is arguably the better idea because you can have an RPG III-style PSDS in QRPGSRC and an RPG IV-style PSDS in QRPGLESRC with the same name. When converting from III to IV, just change QRPGSRC to QRPGLESRC and off you go. With an external file, we'd have to have different names or play games with the library list.

    External data structures come into their own for things like passing an entire DB record between programs; some place where it's very convenient to move the buffer in one step, yet still be able to access the individual fields at will. Like the above example of saving a file's fields for later.

    ReplyDelete
  6. Simon, in your code, what if you used Prefix instead of Qualified? Are they equivilant?

    ReplyDelete
    Replies
    1. You can use the prefix for data structures too.

      I prefer the QUALIFIED as there is no mistaking which Data Structure subfield is being used.

      Delete
  7. Externally described data structures can be used in a myriad of ways and this was available when the System/38 was released externally to IBM Rochester, MN it was availiabel to RPG III and COBOL for sure. Being on the design/implementation team with the Lanuguage and Utilities team, I should know.
    You mentioned the following
    Warning: If you convert a RPG III program using the CVTRPGSRC command and the source member contains the PSDS as an Externally described DS the conversion will complete successfully. But as the layout of the RPGIII and RPGLE/RPGIV PSDS are different you are going to have run-time issues. You need to change the new source member to use a RPGLE/RPG IV compatible PSDS.

    I would suggest, when you transition for the System/38 PSDS to the newer one, remember to use the compiler statements along with the conditional compiler statements to compile based on your target release and your target system level, if you have not already done so. This does not take much change in the coding, it only requires you have planned this out ahead of time, so if you are supporting code to a larger base or customer set, you don't need to keep so many different copies per level.

    Happy times are when you have a plan and it works going forward and not just for the moment...

    ReplyDelete
  8. Externally described DS are a blessing especially when working with horizontally designed files containing arrays, such as sales by month. The RPG compiler builds the DS in perfect alignment with the record layout, so you can declare your arrays based on a pointer and set that pointer to the address of the first instance of the field (i.e. January sales). Then do your I/O operations using the DS.

    It's important to know that the native I/O buffers created by the compiler for each F-spec do NOT necessarily match the record layout, so even though the sales buckets may be contiguous in the layout, they may not be contiguous in memory. But the DS assures they are contiguous.

    ReplyDelete
  9. I am modifying very old code for a project where a couple of fields are being expanded. In my testing, I discovered CL programs where a 100 character field was initialized by hard coding. This field, however gets overlaid by a called program with a data structure in an RPGLE program. The data structure in the RPGLE program had been changed to reflect the expanded numbers, (from 4 P 0 to 6 S 0), but the CL program was not changed to initialize the field with the expanded numbers in mind.

    Getting tired of having to count and see which subfields were numeric vs. character, I had made a copybook of the data structure, and used the copybook in a procedure in a service program .

    I was thinking, however, that defining an external data structure would make the programs easier to maintain should there be another expansion. There would be no recalculating of positions, it would just be a matter of recompiling.

    ReplyDelete
  10. I aam extremely impressed with your writing skills and also with the
    layout on your weblog. Is this a paid theme or did you customizse it yourself?
    Eirher way keep up the excelleent quality writing, it's rare tto
    see a great blog like this one today.

    ReplyDelete
  11. Hi,
    It is very interesting to see such small synopsis which contains meaningful and understandable language. Almost all topics are short write-ups and wow, it has the code too.
    Coming to my query, I am declaring a data structure with extname keyword(DDS file) without specifying library. In this case, is the library *LIBL by default?
    2. When I introduce new fields in the DDS file, should the program be compiled again?
    3. Would like to know how extname resolves during compile and run time.

    ReplyDelete
    Replies
    1. The definition of the file in included when the program is compiled. If you change the file you will have to re-compile the program to included the changes.

      If you do not give the library name you are correct that *LIBL is the default.

      Delete
  12. Very helpful thanks

    ReplyDelete

To prevent "comment spam" all comments are moderated.
Learn about this website's comments policy here.

Some people have reported that they cannot post a comment using certain computers and browsers. If this is you feel free to use the Contact Form to send me the comment and I will post it for you, please include the title of the post so I know which one to post the comment to.