Wednesday, November 12, 2014

Getting a list of files in IFS directory

directory folder list for ifs

Update

What I have described here has been made redundant by the introduction of the SQL Table Function IFS_OBJECT_STATISTICS.

Click on this link to read about IFS_OBJECT_STATISTICS.

 


 

I needed to make a job that would display a list of files in an IFS directory, allow a user to select one, copy contents of the file in the IFS directory to a file in the IBM i, and process the data. Always hearing the mantra KISS, Keep It Simple Simon, in my mind I searched for alternatives.

The person who taught me this mantra always made last the 'S' interchangeable between 'Simon' and 'Stupid', which left me thinking if I was being stupid? She always refused to elaborate.

I could use the 'Display Object links' DSPLNK, command. Alas, the output from this either display or print. If I output to print I could then copy the spool file to a physical file, and… that is unnecessarily complicated for something that should be simple.

There are the C APIs OPENDIR, STRUCT, and CLOSEDIR.

There is the 'Retrieve Directory Information', RTVINF, command. That looked promising. But when I tried it I found that its output could not be placed in QTEMP. The command produces two files, QAEZD0001D and QAEZD0001O. The one end with the 'O' (O the letter not zero) contains the information, but when I view the file the output is not in the CCSID I could read. I would need to use CAST function in SQL to make it appear readable to me.

  RTVDIRINF DIR('/myfolder/') INFLIB(MYLIB)

  select cast(qezobjnam as char(100) ccsid 37) from qaezd0001o

What I really wanted was just to use the Unix command ls, which lists the directory contents. Fortunately, on the IBM i we can execute many Unix commands using Qshell. There are two CL commands to start Qshell, STRQSH and QSH, which are identical. I just prefer to use QSH as it is less characters to type, KISS. If I could direct the output of LS to a file I would have exactly what I want.

Rather than show parts of the CL program, this is small enough that I am going to show the whole thing and then describe what the various parts do.

01  PGM      PARM(&PATH &ERROR)

02  DCL      VAR(&PATH) TYPE(*CHAR) LEN(50)
03  DCL      VAR(&ERROR) TYPE(*CHAR) LEN(10)

04  CHGVAR   VAR(&ERROR) VALUE(' ')

05  CD       DIR(&PATH)
06  MONMSG   MSGID(CPFA09C) EXEC(DO)
07    CHGVAR VAR(&ERROR) VALUE('NOT_AUTH')
08    RETURN
09  ENDDO

10  DLTF     FILE(QTEMP/DIRLIST)
11  MONMSG   MSGID(CPF0000)
 
12  OVRDBF   FILE(STDOUT) TOFILE(QTEMP/DIRLIST) +
               OVRSCOPE(*CALLLVL)

13  QSH      CMD('ls -lt *.*')

14  DLTOVR   FILE(STDOUT) LVL(*)

15  ENDPGM

Line 1-4 should be familiar to everyone. Line 1 marks the start of the program and that there are two parameters passed to the program, &PATH which is the IFS directory and &ERROR which will return a code to the calling program if an error is encountered. Those variables are declared in lines 2 and 3. On line 4 &ERROR is blanked, i.e. no error was encountered.

I decided to change the directory, using the CD command. This way if the user is not authorized to the directory then this can be captured by a MONMSG, line 6, and an "error" returned to the calling program, lines 7 and 8.

Line 10 deletes the file is going to contain the output from the LS.

When you use Qshell its output is written to your display, which is known as the STDOUT. As I want the output to be written to a file I can override STDOUT to a file, line 12.

Line 13 is where I execute the ls command in Qshell, QSH. Notice that ls is followed by -lt, these are arguments used by the command, like parameters, to describe the type of output I want. Do remember Qshell is Unix-like, therefore, it is case sensitive. Lower case 'l' is not the same as upper case 'L'. 'l' means I want a long listing format, and 't' sorted by file modification time.

Line 14 deletes the override of the STDOUT, and the program ends on line 16.

The file produced is a source file. All source files have the same three fields:

             Data        Field
  Field      Type       Length
  SRCSEQ     ZONED        6  2
  SRCDAT     ZONED        6  0
  SRCDTA     CHAR          254

The data we are interested in is in the SRCDTA field:

....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+....8
SRCDTA                                                                          
-rwxrwxrwx  1 MYLIB  0                     4 Oct  3 09:54 test.txt             

Start Length
File permissions 2 9
Link count (file) 12 2
File owner 15 8
File size 42 4
File date stamp 47 12
File name 60 end

If there are no files in the directory the SRCDTA field looks like:


....+....1....+....2....+....3....+....4....+....5....+....6....+....7....+....8....+....9
SRCDTA
ls: 001-2113 Error found getting information for object *.*. No such path or directory.

For testing I created an RPGLE to extract the data I want from SRCDTA:

  if (%subst(SRCDTA:5:9) = '001-2113') ;
    Error = 'No files' ;
  else ;
    User = %subst(SRCDTA:15:8) ;
    DateTime = %subst(SRCDTA:47:12) ;
    FileName = %subst(SRCDTA:60:100) ;
  endif ;

With this information now I can write my program to present the user with a subfile that will allow them to select the file they want to process.

 

You can learn more about these on the IBM website:

And not from IBM:

 

This article was written for IBM i 7.2, and it should work with earlier releases too.

22 comments:

  1. Not a QSH or directory expert myself, I'm not sure if a change via the CD command persists for the current thread or job. If it does, some "good citizen" cleanup might be useful in this program, to reset the current directory back to what it was before the program was called. I have not tested the program changes shown below, but using RTVCURDIR illustrates the concept.


    01 PGM PARM(&PATH &ERROR)

    02 DCL VAR(&PATH) TYPE(*CHAR) LEN(50)
    DCL VAR(&PREVPATH) TYPE(*CHAR) LEN(9999)
    DCL VAR(&PPATHLEN) TYPE(*DEC) LEN(7 0) VALUE(0.0)
    03 DCL VAR(&ERROR) TYPE(*CHAR) LEN(10)

    MONMSG MSGID(CPF0000) EXEC(GOTO UNEXPERR)

    04 CHGVAR VAR(&ERROR) VALUE(' ')

    RTVCURDIR RTNDIR(&PREVPATH) DIRNAMLEN(&PPATHLEN)

    05 CD DIR(&PATH)
    06 MONMSG MSGID(CPFA09C) EXEC(DO)
    07 CHGVAR VAR(&ERROR) VALUE('NOT_AUTH')
    08 RETURN
    09 ENDDO

    10 DLTF FILE(QTEMP/DIRLIST)
    11 MONMSG MSGID(CPF0000)

    12 OVRDBF FILE(STDOUT) TOFILE(QTEMP/DIRLIST) +
    OVRSCOPE(*CALLLVL)

    13 QSH CMD('ls -lt *.*')

    IF (&PPATHLEN *GT 0) THEN( CD DIR(&PREVPATH) )

    14 DLTOVR FILE(STDOUT) LVL(*)

    UNEXPERR:
    CHGVAR VAR(&ERROR) VALUE('UNEXPECTED')
    IF (&PPATHLEN *GT 0) THEN( CD DIR(&PREVPATH) )


    15 ENDPGM

    ReplyDelete
  2. Note also that LS command results vary depending on the length of the Owner Name.

    ReplyDelete
  3. Thank you! I coded as QSH CMD('ls -A *.*') since I only needed the file name. I too had the issue that the name didn't always start at position 60 with (-l) because of the length of the user id. By removing (-l), the field SRCDTA will only consist of the file name.

    ReplyDelete
  4. Thank you, this got me started on a search an IFS folder utility using the grep command.

    ReplyDelete
  5. Very useful for finding files that need to be deleted after x days.

    BTW: there's a broken link:
    Retrieve Directory Information CL command RTVDIRINF

    The requested resource is not found: /support/knowledgecenter/api/content/ssw_ibm_i_71/cl/rtvdirinf.htm

    ReplyDelete
    Replies
    1. Thank you for the information about the bad link. IBM keeps changing the KnowledgeCenter and does not redirect the old URLs to the new. I will change the link on this post to the today's correct URL.

      Delete
    2. Simon, how can I read the output file created by ls in cl using rcvf? Should I create the file for dclf? Any trick to use any existing source file without creating the brand new file? I need to list ifs directory file names and use them in my cl in a loop.

      Delete
    3. If you have used the search function you would have found this post about Read, write, and update a file in CL.

      Delete
    4. Simon, thank you for the reply. I saw the post. I just wanted to make sure that I have to create similar to the source file (srcseq, srcdat, srcdata) table which I did and I overriding it to the qtenmp/dirlist. I need to read this file in cl by part of the file name from ifs in order to copy it to my as400 file. I cannot use sql so I am using rcvf.
      Also I am using "QSH CMD('ls -A *.*') " to get just the file name. Do you know id ls command sorts files alphabetically by file name in ifs and it is a default?

      Delete
  6. If you modify line 13 to

    QSH CMD('ls -lt *.* 2>/dev/null')

    you can eliminate the spool files created by call qsh.

    ReplyDelete
  7. This worked great, but now it does not show the latest files in the directory ... not sure why.
    No ownership or authority differences between files that show and files that do not.
    Anyone have any idea what is going on?

    ReplyDelete
  8. We have used this program and it works great, most of the time. Occasionally we get an error saying that the file DIRLIST in QTEMP is not found. Our current work around is to cancel the program, sign off and back on and try the program again. Please let me know if there are any code changes which can correct this issue. Thanks

    ReplyDelete
  9. This use of QSH is very interesting and useful. Thanks Simon.

    ReplyDelete
  10. Unfortunately (to me) for maaany elements in the dir, it answers (example):
    > ls -l tmp/QACX*
    > qsh: 001-0085 Too many [etc. and no list at all]

    while (example):
    > ls -l tmp/QACXABT*
    -rwxrwxrwx 1 QTMHHTP1 0 0 Aug 17 2016 tmp/QACXABT7JT_wbsm
    -rwxrwxrwx 1 QTMHHTP1 0 0 Jun 8 2016 tmp/QACXABT85R_wbsm
    -rwxrwxrwx 1 QTMHHTP1 0 0 Nov 24 2016 tmp/QACXABTDR2_wbsm
    -rwxrwxrwx 1 QTMHHTP1 0 0 Apr 11 2017 tmp/QACXABTFHH_wbsm
    -rwxrwxrwx 1 QTMHHTP1 0 0 Feb 27 2017 tmp/QACXABTG03_wbsm
    -rwxrwxrwx 1 QTMHHTP1 0 0 Mar 24 2017 tmp/QACXABTW0F_wbsm


    ReplyDelete
  11. Instead of performing -
    select cast(qezobjnam as char(100) ccsid 37) from qaezd0001o

    isnt it simple to perform -
    select char(qezobjnam) from qaezd0001o

    Any difference between the statements? I get same output only.

    ReplyDelete
  12. "visually" you might, but if you do a DSPFFD against QAEZD0001O, you'll find that QEZOBJNAM is a graphic character type at a CCSID of 1200 -- which sometimes has incompatibilities based on my experience (E1), and the CAST reduces the length and uses a EBCDIC CCSID. If it works without, then simpler is better. Otherwise, it's a useful tool when required. Have a nice evening....

    ReplyDelete
  13. qsh cmd('ls /mydir/* >> /QSYS.LIB/MYLIB.LIB/MYFILE.FILE/MYFILE.MBR')

    ReplyDelete
  14. I'd recommend avoiding the QSH ls command; I ended up using the API's instead. See this SQL oriented one from Scott:
    https://www.scottklement.com/udtf/ifsDirArticle.html
    -- the code download link is at the bottom.
    -- if you want it driven by CL, then execute the SQL using RUNSQL.

    ReplyDelete
  15. My list shows files with two separate time stamp formats. Any idea?
    Feb 6 11:03
    Nov 21 2019

    ReplyDelete
  16. Your posts are the most useful thing on the internet for AS/400 developers. Keep up the good work.

    ReplyDelete
  17. I used find $PWD , then you have path and name file on one line

    ReplyDelete

To prevent "comment spam" all comments are moderated.
Learn about this website's comments policy here.

Some people have reported that they cannot post a comment using certain computers and browsers. If this is you feel free to use the Contact Form to send me the comment and I will post it for you, please include the title of the post so I know which one to post the comment to.