Tuesday, November 26, 2013

Zipping files in the IFS

zip file ifs

The information that went into creating this post was provided by Al Bernard. Al is an experienced IBM i (AS400) developer and colleague. In a recent conversation he mentioned to me something he had done that I have not.

He was asked to create a download file that would be emailed to a group of people. The file was so large that the Administrator of the email servers asked if the file could be zipped, to reduce its size, and then sent. My present employer uses a third party tool to send email from the IBM i, and I will not be discussing that here. What I will be discussing is the way he copied the file to the IFS and zipped it. Then I will describe how to do the reverse, taking the zipped file and putting the data into a DDS file.

Before you attempt this for yourself you will need a folder in the IFS you can use. If you do not have one you can use either create it for yourself or ask your IBM i Administrator to do so for you. In this post I will use the name MyFolder for my folder.

The extracted data has been output to a file that I will call EXTRACT and it is in my library, MYLIB.

First I needed to get the data from the file EXTRACT into the IFS as a CSV (Comma Seperated Values) file. This I do using the Copy To Import File, CPYTOIMPF, command, see below.

   CPYTOIMPF  FROMFILE(MYLIB/EXTRACT) +
                TOSTMF('MyFolder/extract.csv') +
                MBROPT(*REPLACE) +
                STMFCCSID(*STDASCII) +
                RCDDLM(*CRLF)

The first two parameters of the command define where the file is, MYLIB/EXTRACT, and where you want it to go, into the folder MyFolder as extract.csv .

MBROPT(*REPLACE) replaces any file that is already in the MyFolder with the name extract.csv. It will also replace a file called EXTRACT.CSV or ExTrAcT.cSv too.

The STMFCCSSID is how the characters will be converted from EBCDIC to ASCII. In my experience I have found the best format is *STDASCII.

Finally we have to give what character is going to delimit the end of record. I always choose *CRLF, carriage return and line feed.

When this has completed you can copy the file extract.csv from the folder MyFolder to your PC and open it to make sure the data transferred is now in a PC readable form.

To zip the file I use the Java Jar command, which is run in the QSHELL environment using the STRQSH command.

   STRQSH     CMD('Jar cfM MyFolder/extract.zip +
                MyFolder/extract.csv')

The format of the Jar command is as follows: Jar  options  zipped-file-name  file-to-be-zipped

Each letter in the options is for something. There can be one or many options depending on what you want to do. Notice that the options are both upper and lower case. It is important to use the correct character as c does not do the same as C.

Option Description
c Create new archive (zip file)
C Change to the specified directory and include following file
f Specify archive file name
i Generate index for specified Jar file
m Include manifest information in manifest file
M Do not create manifest file
t List table of contents for archive
u Update existing archive
v Generate verbose ouput on standard output
x Extract named file or all files from archive
0 (zero) Store only, no zipping

In my example I used the options cfM, which translates to:

  • I want to create a new archive file (c)
  • I am specifying the zip file name (f)
  • I do not want a manifest file (M)

You can see if the Jar was success by looking at the job's joblog. If you see the following then you know it was successful.

     1200 - STRQSH CMD('Jar cfM MyFolder/extract.zip MyFolder/extract.csv')
   Command ended normally with exit status 0.                            

Having zipped the file, how do I unzip it and put it back into a DDS file?

First I have to extract the file from the zip file.

   STRQSH     CMD('Jar xf MyFolder/extract.zip +
                   MyFolder/extract.csv')       

I use the Jar command again this time with the options xf, which means:

  • I want to extract the named file from the zip file (x)
  • I need to specify the zip file name (f)

To copy the data from the IFS back into a DDS file I use the Copy From Import File, CPYFRMIMPF, which does the opposite of its sister command CPYTOIMPF.

   CPYFRMIMPF FROMSTMF('MyFolder/extract.csv') +   
                TOFILE(MYLIB/EXTRACT) +
                MBROPT(*REPLACE) +
                RCDDLM(*CRLF) +
                RPLNULLVAL(*FLDDFT)

The only different parameter is the RPLNULLVAL which replaces any null values with the default for the field type in the DDS file.

When this command has completed the data has been writen back to the DDS file.

I can put all of this together in a CL program, see below.

01 PGM

02 CPYTOIMPF  FROMFILE(MYLIB/EXTRACT) +
03              TOSTMF('MyFOlder/extract.csv') +
04              MBROPT(*REPLACE) STMFCCSID(*STDASCII) +
05              RCDDLM(*CRLF)

06 STRQSH     CMD('Jar cfM MyFolder/extract.zip +
07                 MyFolder/extract.csv')

08 DEL        OBJLNK('MyFolder/extract.csv')

09 STRQSH     CMD('Jar xf MyFolder/extract.zip +
10                 MyFolder/extract.csv')

11 CPYFRMIMPF FROMSTMF('MyFolder/extract.csv') +
12              TOFILE(MYLIB/EXTRACT) MBROPT(*REPLACE) +
13              RCDDLM(*CRLF) RPLNULLVAL(*FLDDFT)

14 ENDPGM

Line 2 copies the data from the DDS file to the CSV file in the IFS folder.

Line 6 zips it.

Line 8 deletes the CSV file in the IFS folder.

Line 9 unzips the file.

Line 11 copies it back into the DDS file.

You can learn more about these commands on the IBM website:

 

This article was written for IBM i 7.1, and it should work with earlier releases too.

17 comments:

  1. Hi, Interesting...I always wondered how to do this. Knew there had to be a way! Thanks for sharing. :-)
    A practical use would be eliminating typing numbers from paper reports into spreadsheets (I suspect there is still some of that happening).

    ReplyDelete
  2. Sharing always great!
    Hope to see more coming

    ReplyDelete
  3. In V7R1M0, IBM added API's QzipZip and QzipUnzip

    ReplyDelete
    Replies
    1. Djurre: Thank you for that info.

      I'll investigate those & they could become a new post.

      Delete
    2. Djurre, with V7R1 IBM invented the wheel. QSHELL based ZIP capabilities have been in System i since at least 10 years. For instance, they were already available with release V5R2. IBM just forgot to tell this to its customers. I myself learned this from a PC Press magazine issue years ago and what I did was simply to build a user interface to these system capabilities.

      Delete
    3. Giovanni,
      You are right. I have been using the ajar stuff for quite a while too.
      Wrote a set of (un-)zip-handlers in RPG using the Qsh API's (QzshSystem etc.)

      Delete
    4. Hi,

      I am new to iSeires (just 4 yrs). I would be interested to know the Zip ratio for this...

      Delete
    5. I tried qzipunzip but want to generate the zip contents of a directory that generates the zip I can not open. It fails. If I make a zip of the directory if it works but the content gives I get an error when opening.

      Delete
    6. I tried qzipunzip but want to generate the zip contents of a directory that generates the zip I can not open. It fails. If I make a zip of the directory if it works but the content gives I get an error when opening.

      Delete
  4. Amitava Sutradhar, MBADecember 12, 2013 at 12:33 PM

    Zipping Single File:
    STRQSH CMD('iconv -f 37 -t 819 /MyFoledr/MyFile.EXT > /MyFoledr/MyFile.ZIP && setccsid 819 /MyFoledr/MyFile.ZIP')

    Zipping Multiple Files:
    STRQSH CMD('iconv -f 37 -t 819 /MyFoledr/*.* > /MyFoledr/MyZipFile.ZIP && setccsid 819 /MyFoledr/MyZipFile.ZIP')

    ReplyDelete
    Replies
    1. iconv does not compress - it only changes the CCSID of the data. If one were to send MyZipFile.ZIP to a PC and try to UNZIP it, one could not do it, because iconv does not do the same thing as ZIP.

      Delete
  5. Copy to import only copy the records not a header. How to copy with header?
    Kindly suggest mw

    ReplyDelete
  6. Jar and ajar are invaluable especially in the WebSphere on IBM i world. Also, in Navigator for i you can download multiple IFS files/folders as a zip file by default by right clicking and choosing Download. That feature is just bad ass.

    ReplyDelete
  7. command: strqsh cmd('jar cfM myFile.zip *.*')
    error: qsh: 001-0078 Process ended by signal 5.

    Have anyone encountered this error? Please suggest solution.

    ReplyDelete
    Replies
    1. per new2qsh :
      strqsh cmd('jar cfM /Public/test_xml.zip /Public/*.xml') = zippa tutti i files con estensione xml che sono nella cartella Public
      oppure:
      strqsh cmd('jar cfM /Public/test_xml.zip /Public/ ')
      zippa tutti i file che sono nella cartella Public

      Delete
    2. Translation:
      strqsh cmd ('jar cfM /Public/test_xml.zip /Public/*.xml') = zipping all files with the xml extension that are in the Public folder
      or:
      strqsh cmd ('jar cfM /Public/test_xml.zip / Public /')
      zipping all the files that are in the Public folder

      Delete
    3. @new2qsh
      My guess is that *.* select more than 16.383 objects and you then reach the limit mentioned here https://www.ibm.com/support/pages/argument-number-limits-ls-rm-and-other-qshell-commands-qsh-001-0078-process-ended-signal-5

      Delete

To prevent "comment spam" all comments are moderated.
Learn about this website's comments policy here.

Some people have reported that they cannot post a comment using certain computers and browsers. If this is you feel free to use the Contact Form to send me the comment and I will post it for you, please include the title of the post so I know which one to post the comment to.