Wednesday, March 2, 2016

Defining files in subprocedures

rpg subprocedures

This is a subject I have mentioned in passing in other posts, but I have not addressed this subject in its own post. I had a discussion with some folks on Facebook who wanted to know how to define and use a file in a subprocedure. It got difficult referring to one part of one post, and another part of another post, etc. I thought it would be a good idea to write a post just about this subject.

Prior to IBM i 6.1 it was not possible to include a file in a subprocedure. You had to define it in the main part of the program to be able to use it in any subprocedure, see the example below. The problem is what when you do this when you use the file in one subprocedure the file pointer is left in the position where it was last used, when you exit the subprocedure and enter another the file pointer remains in that position and can cause unexpected results. This can cause problems as I will show later in this post.

My example programs are all going to be based on the same scenario. I have two subprocedures, First and Second, that both read a file, TESTFILE (I know I get no marks for original or exciting names). The subprocedure First reads the file twice, and Second only once. They both return an indicator to main part of the program that is end of file indicator for the file they both read.

First comes the pre-IBM i 6.1, where the file has to be defined in the main part of the program:

01  H dftactgrp(*no)
02  FTESTFILE  IF   E             DISK
03  D First           PR              N
04  D Second          PR              N

05  D FirstEof        S               N
06  D SecondEof       S               N
     /free
07    dou (FirstEof and SecondEof) ;
08      if not(FirstEof) ;
09        FirstEof = First() ;
10      endif ;

11      SecondEof = Second() ;
12    enddo ;

13    *inlr = *on ;
     /end-free
14  P First           B
15  D First           PI              N
     /free
16    read TESTFILER ;
17    if (%eof) ;
18      return *on ;
19    endif ;
20    dsply ('First No.1: F001 = ' + F001) ;

21    read TESTFILER ;
22    if (%eof) ;
23      return *on ;
24    endif ;
25    dsply ('First No.2: F001 = ' + F001) ;

26    return *off ;
     /end-free
27  P                 E
     *-----------------
28  P Second          B
29  D Second          PI              N
     /free
30    read TESTFILER ;
31    if (%eof) ;
32      return *on ;
33    endif ;
34    dsply ('Second: F001 = ' + F001) ;

35    return *off ;
     /end-free
36  P                 E

I am only going to briefly explain this code as I want to concentrate on the newer code, which will be shown later in this post.

Line 1: If the program contains subprocedures I need to use the H-spec, to Control Options in free format definitions code, to not to use the Default Activation Group.

Line 2: This is the definition of TESTFILE.

Lines 3 and 4: Definitions for my two procedures, that both return an indicator value.

Lines 5 and 6: Definitions for the variables that will contain the values returned from the subprocedures.

Lines 7 – 12: This Do-loop calls each of the subprocedures until they return the end of file indicator from TESTFILE.

The first subprocedure, First is very simple. It starts on line 14 and end on line 27.

Line 15: This is the procedure interface. The only parameter is the indicator value that will be returned to the calling program.

Line 16: The file, TESTFILE, is read using the record format name.

Line 17 – 19: If end of file was encountered by the read operation a value of "*ON" is returned.

Line 20: If the end of file was not encountered then the value of the field in the file is displayed.

The same logic that is used in lines 16 – 20 is repeated for a second read.

Line 26: If the end of file was not encountered then the value of "*OFF" is returned to the calling program.

The second subprocedure, Second, uses the same logic seen in lines 16 – 20 to perform a single read.

What happens when I run this program?

DSPLY  First No.1: F001 = 1
DSPLY  First No.2: F001 = 2
DSPLY  Second: F001 = 3
DSPLY  First No.1: F001 = 4
DSPLY  First No.2: F001 = 5
DSPLY  Second: F001 = 6

As TESTFILE is defined in the main part of the program it means that the file pointer is also defined in the main part too. Everything defined in the main part of the program is available to all the program's subprocedures. Therefore, when the file is read as the file pointer is common to both the subprocedures and the records are read one after another, regardless of which subprocedure does the read.

In IBM i 6.1 and later it has been possible to define the files within the subprocedures. The code I am giving below is in fully free RPG, and there is an example with fixed definitions later in this post here.

01  **free
02  ctl-opt dftactgrp(*no) ;

03  dcl-pr First ind ;
04  end-pr ;
05  dcl-pr Second ind ;
06  end-pr ;

07  dcl-s FirstEof ind ;
08  dcl-s SecondEof ind ;

09  dou (FirstEof and SecondEof) ;
10    if not(FirstEof) ;
11      FirstEof = First() ;
12    endif ;

13    SecondEof = Second() ;
14  enddo ;

15  *inlr = *on ;

16  dcl-proc First ;
17    dcl-pi *n ind ;
18    end-pi ;

19    dcl-f TESTFILE ;
20    dcl-ds F1 likerec(TESTFILER) ;

21    read TESTFILER F1 ;
22    if (%eof) ;
23      return *on ;
24    endif ;
25    dsply ('First No.1: F1.F001 = ' + F1.F001) ;

26    read TESTFILER F1 ;
27    if (%eof) ;
28      return *on ;
29    endif ;
30    dsply ('First No.2: F1.F001 = ' + F1.F001) ;

31    return *off ;
32  end-proc ;

33  dcl-proc Second ;
34    dcl-pi *n ind ;
35    end-pi ;

36    dcl-f TESTFILE ;
37    dcl-ds F2 likerec(TESTFILER) ;

38    read TESTFILER F2 ;
39    if (%eof) ;
40      return *on ;
41    endif ;
42    dsply ('Second: F2.F001 = ' + F2.F001) ;

43    return *off ;
44  end-proc ;

So what is different from the previous code, apart from this one being in fully free RPG?

Line 1: This needs to be given in the source member of any full free RPG code. If your IBM i does not have the PTFs for fully free then remove this line and start the rest of this code in the eighth column of your code.

Line 2: This Control Option is the equivalent of the H-spec from the previous code.

Lines 3 – 6: These are the free format of the two subprocedures, which both return an indicator value. The same as the previous code.

Lines 7 – 8: Definition of our two variables used in the main body of the program, which are identical to those from the previous example.

Lines 9 – 15: The free format "calculation specifications" are identical in both examples of code.

The first lines of the First procedure, 16 – 18, are just the free format definition equivalents of what was given in the first example.

Line 19: Now we come to the file definition. As I have not stated how this file is to be used the compiler assumes it is to be used for input only.

Subprocedures do not have Input or Output specifications. "What is the big deal?" I can hear some say, "Neither does the main part of the program when I placed the file definition there?" Even though I do not have to code the Input specifications in the first example, the compiler creates them for me from the definition of the file. As subprocedures do not have Input specifications I have to use a data structure with all my file I/O operations.

Line 20: This is where I have defined my file data structure, F1, using the LIKEREC keyword. The LIKEREC must contain the file's record format name. If I had renamed the record format when defining the file I would need to give the renamed record format. There is an optional second parameter, this is used to define which field are to be defined in the data structure as its subfields. There are three possibilities:

  1. LIKEREC(record_format:*input) - only define the input capable fields in the record format, this is the default, therefore, if the second parameter is not given input is assumed
  2. LIKEREC(record_format:*output) - only define the output capable fields in the record format
  3. LIKEREC(record_format:*all) - define all the fields in the record format

In this example I do not have to give the type of fields as the default is input capable, as all fields in physical files are input and output capable. The LIKEREC data structure automatically qualifies the names of the subfields, and if I had used QUALIFIED keyword.

Lines 21 and 26: The reads are different. As there is no input or output specifications in a subprocedure I must read the file "into" a data structure, the name of the data structure follows the file name in the read operation. In this case it is F1, as this was the data structure I defined using the LIKEREC.

The rest of subprocedure First is the same as the previous example, until the end of procedure line on line 32.

The subprocedure Second is the same as the First except it performs only one read, and I decided to all the file structure a different name for no other reason than I can.

When I run this program this is what I see:

DSPLY  First No.1: F1.F001 = 1
DSPLY  First No.2: F1.F001 = 2
DSPLY  Second: F2.F001 = 1
DSPLY  First No.1: F1.F001 = 1
DSPLY  First No.2: F1.F001 = 2
DSPLY  Second: F2.F001 = 1

The first thing to notice is that the file pointer in the subprocedures act independently, which is why we read the first two records in First and the first record in Second. When the subprocedure ends the file is closed, therefore, when I call the subprocedures a second time the file is opened and I read the same records as before. With this code the end of file is never reached in either subprocedure.

So what about other file I/O types? In my next example I want to change the value of the third records key field.

33  dcl-proc Second ;
34    dcl-pi *n ind ;
35    end-pi ;

36    dcl-f TESTFILE keyed usage(*update) ;
37    dcl-ds F2 likerec(TESTFILER) ;

38    chain ('3') TESTFILER F2 ;
39    if not(%found) ;
40      return *on ;
41    endif ;

42    F2.F001 = '3X' ;
43    update TESTFILER F2 ;

44    return *off ;
45  end-proc ;

Line 36: I can only use a chain on a keyed file, so I have added the KEYED keyword to the definition. As I am going to be updating the file I need to define the usage as update, which I have done with the USAGE keyword.

Line 38: I am using a key field list to chain to TESTFILE and the retrieved fields are moved to subfields in the F2 data structure. If you are unfamiliar with key field lists you can learn about them in the post How to replace Key Lists.

Lines 39 – 41: If the record is not found by the chain the value of "*ON" is returned to the calling program.

Line 42: As the values of the fields are in the F2 data structure I need to change the value of the data structure subfield with the value I desire.

Line 43: I update the file with the values held in the F2 data structure.

So what about writing to a file defined in a subprocedure:

33  dcl-proc Second ;
34    dcl-pi *n ind ;
35    end-pi ;

36    dcl-f TESTFILE keyed usage(*output) ;
37    dcl-ds F2 likerec(TESTFILER) ;

38    F2.F001 = '11' ;
39    write TESTFILER F2 ;
                    
40    return *off ;
41  end-proc ;

Line 36: This time my file is defined for USAGE(*OUTPUT), as I am going to be writing to the file.

Line 38: I am putting the value for what will be my file's key field in the data structure.

Line 39: Then I write using the values from the subfields in the data structure F2 to write to the fields in the file.

 

You can learn about defining other types of files in subprocedures in Part 2: More about defining files in subprocedures.

 

You can learn more about the LIKEREC keyword in RPG from the IBM website here.

 

This article was written for IBM i 7.2, and should work for earlier releases too.

 

Fixed format definition code for subprocedures with files defined within them


01  H dftactgrp(*no)
02  D First           PR              N
03  D Second          PR              N

04  D FirstEof        S               N
05  D SecondEof       S               N
     /free
06    dou (FirstEof and SecondEof) ;
07      if not(FirstEof) ;
08        FirstEof = First() ;
09      endif ;

10      SecondEof = Second() ;
11    enddo ;

12    *inlr = *on ;
     /end-free
13  P First           B
14  FTESTFILE  IF   E             DISK
15  D First           PI              N
16  D F1              DS                  likerec(TESTFILER)
     /free
17    read TESTFILER F1 ;
18    if (%eof) ;
19      return *on ;
20    endif ;
21    dsply ('First No.1: F1.F001 = ' + F1.F001) ;

22    read TESTFILER F1 ;
23    if (%eof) ;
24      return *on ;
25    endif ;
26    dsply ('First No.2: F1.F001 = ' + F1.F001) ;

27    return *off ;
28   /end-free
29  P                 E
     *-----------------
31  P Second          B
32  FTESTFILE  IF   E             DISK
33  D Second          PI              N
34  D F2              DS                  likerec(TESTFILER)
     /free
35    read TESTFILER F2 ;
36    if (%eof) ;
37      return *on ;
38    endif ;
39    dsply ('Second: F2.F001 = ' + F2.F001) ;

40    return *off ;
     /end-free
41  P                 E

Return

8 comments:

  1. This has to affect performance with all the opening and closing of data file paths. Can you talk about performance and if/how that affects your decision to use subprocedure file I/O

    ReplyDelete
  2. Yet another good Simon's post ... but it got me curious ... and I want try with SQL cursor instead of setll-read ... it's not the same ... Here's my example code (http://goo.gl/Z4p2NY)

    ReplyDelete
    Replies
    1. I have for sometime been using SQL statements in subprocedures for file I/O.

      I would do a blocked fetch to load a data structure array in one subprocedure. And then in a second procedure "read" the data structure.

      Delete
  3. To keep the sub-procedure files open between calls, try using STATIC on the F-spec.

    http://www.gateway400.org/documents/Gateway400/Handouts/RPG_6.1_7.1_OA.pdf

    Chris RInger

    ReplyDelete
    Replies
    1. That just feels like dangerous ground.

      Delete
  4. Joachim GuetzlaffMarch 5, 2016 at 2:43 AM

    A good while ago I wrote a simple testing program to measure the cost of file access for
    a) globally defined files
    b) locally defined files (in procedures/functions)
    c) SQL single record access
    So it was a) vs b) vs c). The fastest was option a), the slowest and most costly option b).
    Procedure-level based file-specs only make sense if you specify STATIC and close the file with a special proc-call, or you don't have a lot of I/O (don't call the proc very often). In all other cases, this programming technique is not advisable.

    ReplyDelete
  5. Hi Joachim

    IMO this programming technique is advisable for all procedures in service programs. There it is good practice to close the files before leaving the procedure.

    Regards
    Jan

    ReplyDelete
  6. > There it is good practice to close the files
    > before leaving the procedure.

    Yeah, I gotta disagree. Have you seen any performance benchmarks of this technique? Very very slow. Opening and closing files on every call is 30x slower than leaving them open for subsequent calls. In other words, that's 3000% slower. Keep files open. I close them on a final call.

    If ( %Parms = 0 ) ;
    Close(e) *ALL ; // Global F-Specs
    Close(e) PARTMAST ; // Sub-Proc F-Spec
    Return ;
    EndIf ;

    Chris Ringer

    ReplyDelete

To prevent "comment spam" all comments are moderated.
Learn about this website's comments policy here.

Some people have reported that they cannot post a comment using certain computers and browsers. If this is you feel free to use the Contact Form to send me the comment and I will post it for you, please include the title of the post so I know which one to post the comment to.