The MVS user's survival guide to UNIX--or useful things to know about using Unix or PC SAS to do SAS and Fortran computations that you used to do on MVS.

• Unix Environment and Commands
• Text Editors
• Submitting Jobs
• File Formats on MVS and Unix
• SAS Datasets
• UTCC's document on creating portable SAS files
• Downloading Files From MVS to Unix

I have spent entirely too much time figuring out how to do our computer work on the UNIX system or on the PC. In most ways UNIX is a pain in the butt. The biggest problem is that most of the experienced users log onto unix with an X-windows terminal, or a DEC, or anything else that naturally emulates a vt100 or vt200.  This means that there isn't much documentation or help for those of us who use PCs and IBM terminals. I am working on getting correct setups so our keys work like we would like them to. The fact is, however, that you will have to learn some new keystrokes and techniques.  These problems notwithstanding, I hope that this document will help you to do your work on UNIX or the PC with as little disruption as possible.

Unix environment:

Unix is a lot like a the old MSDOS on a PC. It has files, directories, and subdirectories. Instead of using a backward slash--'\' to separate directories like in a PC, unix uses a forward slash--'/'. The commands are very terse, and Unix doesn't give you much help when you screw up. Following are some useful commands:

(See CAS's documentation also.  They have improved it substantially recently!)

Since rm removes a file very easily, sometimes you accidentally remove a file you really want.  If you do this the best way to get it back is to pray.  Pray hard!   Because you aren't going to get it back any other way unless it is on one of the (weekly?  daily?) tape backups that CAS does.  And it will cost you $50 per hour to get it back.  Instead, I recommend the following procedure:

1.  create a directory called /deleted

2.  create another directory called /bin

3.  using pico, create a file in your /bin directory called rmsave with the following lines
echo "do you want to remove file $1? (y or n) \c"
read response
if [ $response = "y" ]
then
/bin/mv $1 ~/deleted/$1
fi

4.  enter the command

chmod 744 rmsave

5.  using pico, edit your .kshrc file (which is in your root directory) to include the following 2 lines

alias rm='~/bin/rmsave'
alias reallyrm='/bin/rm -i'

and save the .kshrc file

All of this will do the following.  If you type rm file it will copy the file to your /deleted directory.  If you deleted a file in error, just go look for it in the /deleted directory, and copy it back where you want it.  Eventually, your /deleted directory will fill up with stuff you have deleted.  Then you can use the command reallyrm file to really delete a file.

Text Editors

The first thing you need to know about in Unix is how to enter text into a file. 'pico' is the best editor so far on Unix. That isn't saying much, though. 'the' is better, and as soon as all of the keys are set up so it works like xedit on unix, it would be a good one to use. I'll keep you posted.

Submitting jobs--batch? foreground? background? interactive?

In the past on MVS we had a 2-step process for running jobs. First we would edit a text file in CMS or TSO that had JCL and program statements in it. Then we would submit this job (rje) it to MVS. Once the job was done, we would view the output using STAT.

On Unix you can still run batch jobs. If you do this the output of the job and any logs from it will be written to your workspace so you can view it using any text editor. You have two choices with batch jobs: foreground or background. Jobs in the foreground should execute faster, but they keep your unix work space tied up so you can't do other work on unix at the same time. Background jobs allow you to work on unix at the same time, but run slower. For example: suppose you have a text file called test.sas that has a SAS program in it. To get all of the paths setup in your unix account you type

setup sas

(Note: you only type 'setup sas' when you first login. You could put it in your .profile file so that it is automatically executed I think)

Then in order to run the sas job in the foreground type

sas test.sas

The log for the sas job will be placed in a file called test.log, and the output from the program will be in test.lst

If you want to run the job in the background, type

sas test.sas &

The & symbol causes the job to run in the background.

Note: If you screwed up and submitted a long job to run in the foreground and you really wanted it to run in the background, then type

^z

bg

where ^ is the control key. This should return control of your workspace to you and force the job into the background. If for some strange reason you wanted to force a background job into the foreground you would just type

fg

and the background job would take control of your workspace until it completed, or until you put it back in the background.

If you want to run sas in interactive mode from a telnet connection (which is the way we usually log into unix) then enter

sas -fsd ascii

This tells sas to go into interactive mode, but not give you all the great graphics and the use of a mouse, etc. It is a royal pain in the butt to run SAS this way because the terminals aren't set up so the insert, function, delete keys work correctly. I don't like to run SAS this way.

Since you can start all sorts of jobs in the background, it is important that you know what jobs you have running there. I believe that if you start a number of jobs (called processes in unix-speak) they will all run simultaneously, and if you and a lot of other people start a number of jobs they will all run simultaneously (and slowly). To see what jobs you have running type

ps

If you have a job running you will get a response that looks like

PID TTY TIME CMD

24978 pts/3 0:00 sas
24673 pts/3 0:01 csh

The first number is the process id. I don't know what the second entry is, but the third entry is the command that is running. csh, or ksh, or bash are the operating systems that you might use and it will always show up as one of your processes.

If you logoff and your job was running in the background you might not be able to see it when you log back in just using ps. Rather, you need to enter

ps -ef | grep userid

this searches among all of the active processes for ones owned by userid (you, provided you enter your userid) and tells you their process ids.  Your userid is the name you use to log in to unix.

Killing a job

If you have started a job that is taking too long or you think it is in a loop or something, you can kill it. If it is in the foreground you can type

^c to cancel a command—if that doesn't work type ^z. If it is in the background type

kill nnnnn

where nnnn is the process id for the command you saw when you typed ps. If you start a bunch of jobs and leave them all running simultaneously for a long time someone will probably call you and complain because it has made their jobs run slowly.

Sometimes jobs can't be killed because of how they are setup. If for some reason you can't kill a job with plain old kill, then use

kill -9 nnnnn

This terminates with prejudice.

A digression on various types of file formats on MVS and Unix.

Formatted or ascii datasets are basically strings of ascii symbols. These files may be fixed length or variable length. On unix an end of record character appears at the end of every record, and this end of record character is hex 0a. You might create one of these files using a fortran write statement

write(22,999) i,x,y,z
999 format(1x,i4,1x,f10.6,1x,f10.8,1x,f8.4)

In this case, each record would be 1+4+1+10+1+10+8 = 35 bytes long. There would be an additional byte of 0a at the end of each record--but you don't have to worry about that because the system takes care of that. Unix and MVS are similar in this respect I believe.

Unformatted datasets are another matter entirely. On MVS the statement

write(22) i,x,y,z

in conjunction with a jcl statement like DCB=(RECFM=FB,LRECL=16,BLKSIZE=1600) creates a binary file that is 16 bytes long. An advantage of an unformatted write is that you don't need to worry about how big your numbers are, and you won't loose any precision. On MVS you can read in these unformatted datasets using fortran or SAS. In fortran the appropriate read statement would simply be

read(22) i,x,y,z

and in SAS on MVS you would input the data with the statement

input i ib4. (x y z) (rb4.);

However, things don't work this way on Unix. For some odd reason our implementation of unix insists on padding output written with an unformatted sequential write statement at the beginning and ending of each record.

Suppose you wrote data in fortran on our unix system using the following statements:

open(33, file='testout.bin',form='UNFORMATTED',access='SEQUENTIAL')

write(33) i,x,y,z

There are 4x4 = 16 bytes of data in this write statement--a 4-byte integer data item and three 4-byte real data items; call these data1, data2, data3, and data4. However, the sequence of bytes that would be physically present on the unix machine would be

pad1 data1 data2 data3 data4 pad2

where pad1 and pad2 are each 4-byte integers that are some sort of "byte count,' and the real*4 data elements are in IEEE format. You don't need to know exactly what the pad bytes are used for, but you do need to know they are there if you ever hope to use SAS to read this dataset. Basically, the rb4. input statement in SAS expects to see a stream of data bytes without the pads. To read this record you need to tell SAS that it is 16 + 4 + 4 = 24 bytes long, and you need to throw away the first 4 bytes by reading them into a dummy variable. Doesn't this seem dumb?  To make things more complicated, this is NOT the case with unformatted direct access fortran files!  Below summarizes the fortran write statements and corresponding SAS input statements to read the various types of files on Unix:

On Unix, if you write in fortran, you read in sas using the following commands:

Fortran Sequential Formatted File (an ASCII file)

In fortran:
      open(unit=10,access='sequential',form='formatted',file='testfile')
      write(10,99) a,b,c
In SAS:
filename test 'testfile';
data f1;
infile test;
input a b c;

Fortran Sequential Unformatted File

In fortran:
      open(unit=10,access='sequential',form=unformatted',file='testfile')
      write(10) a,b,c
In SAS:
filename test 'testfile';
data f1;
* lrecl = length of data + 8 bytes--4 bytes at front, 4 at end;
* but you don't have to read the last pad;
infile test recfm=f lrecl=20;
input pad1 ib4. (a b c) (ieee4.);

Fortran Direct Unformatted File

In fortran:
      open(unit=10,access='direct',form='unformatted',file='testfile')
      write(10,rec=i) a,b,c

In SAS:
filename test 'testfile';
data f1;
* direct access files don't have the pad bytes;
infile test recfm=f,lrecl=12;
input (a b c) (ieee4.);

Important note:  You HAVE TO HAVE the recfm=f option on the infile statement!   According to the sas manual, you can also use the float4. format for the input instead of the ieee4.  What you CANNOT use is rb4.  According to the sas manual, if you are inputting double precision numbers, you can use rb8. or ieee8.  I haven't tried it.

SAS Information:

Datasets, Libraries, and members.

SAS treats dataset libraries as directories in unix and on the PC. For example, suppose you had a dataset on MVS called J61771.BETAS.SASDATA and there were 3 SAS datasets in this library called OLS, SCHOLES, DIMSON for betas calculated the 3 different ways. In MVS you might have referenced this library with a DD statement in your JCL as

//GO.INBETAS DD DSN=J61771.BETAS.SASDATA,UNIT=ONLINE,DISP=SHR

and in your SAS program you would have referenced, say the ols betas with the statement:

DATA F1;
SET INBETAS.OLS;

SAS on the micro or unix won't allow you to have these multiple levels of filenames. The library is set up as a directory, and the individual datasets in the library are files in the directory. On the PC you might create the directory:

C:\SAS\BETAS

and have the following files in it:

C:\SAS\BETAS\OLS
C:\SAS\BETAS\SCHOLES
C:\SAS\BETAS\DIMSON

The statements

libname inbetas 'c:\sas\betas';
data f1;
set inbetas.ols;

would correspond to the DD statement and access the SAS dataset named OLS.

Ordinary Files:

Often you create a SAS dataset by inputting data from a file you have created in fortran. Again, in MVS you will have referenced this dataset with a DD statement:

//GO.NMSTRADE DD DSN=J61771.VOLWANS.NMSTRADE,UNIT=ONLINE,DISP=SHR

and then you might have input the data with the code:

DATA F1;
INFILE NMSTRADE;
INPUT ICAL VOL PRICE SPVOL SHVOL ;

provided the data is ascii or formatted fortran output.

Suppose that the dataset c:\sas\data\nmstrade.vol is on your hard drive, then the following statements would do the same thing:

filename nmstrade 'c:\sas\data\nmstrade.vol';
data f1;
infile nmstrade;
input ical vol price spvol shvol ;

Reading unformatted fortran output

On MVS if the data happen to be unformatted fortran output (created in something like a WRITE(21) statement) then you might have read it using

DATA F1;
INFILE NMSTRADE;
INPUT ICAL IB4. VOL IB4. PRICE RB4. SPVOL RB4. SHVOL RB4.;

provided that the fortran job that created the data also ran on MVS. Unfortunately its not so simple on Unix.

The corresponding UNIX SAS commands would be:

filename nmstrade '~\sas\data\nmstrade.vol';
data f1;
infile nmstrade lrecl=....whatever is the lrecl on the file + 8 bytes for the pad characters;
input ipad ib4. ical ib4. vol ib4. (price spvol shvol) (ieee4.);

note that you need to use ieee4. to read unformatted rb4. fortran output on Unix.

I think, but am not sure, that you can omit the 8 bytes pad, and the ipad ib4. read if you are doing this with PC-based unformatted fortran output on PC SAS.

Note: If you are going to use unformated inputs like this the dataset must have been created on the platform on which you are reading the data. If you are on unix, then you must have created the dataset using fortran on unix. If you want to read an unformatted dataset (one created in fortran using a statement like WRITE(21)) that you created on MVS and then downloaded to unix or your pc, then see the section on SAS CONNECT. You will need to make sure you download it as a binary file, you will need to know the lrecl for the dataset, and you will need to use the s370fib4. and the s370frb4. formats rather than ib4. and rb4. with the infile statement. Of course if your datasets are formatted output from fortran, then don't worry about this.

Downloading files from MVS to Unix or PC SAS.

One of the problems with moving from one platform to another is that datasets and files are not always compatible across platforms. Here are instructions on how to download files so you can use them on whatever platform you have. The instructions are different for regular old ASCII files, unformated files (like those written with the fortran statement like WRITE(21)), and SAS datasets.

ASCII files

These are the simplest. You can simply ftp them to the machine you want to use. You can also use SAS CONNECT to transfer the files. See the section on SAS CONNECT. To ftp to your PC use WS_FTP and click on the ASCII button. To ftp files from MVS into your unix account

cd directory_you_want_to_receive_files
ftp utkmvs1

follow the directions entering your userid, which is p….. and your password.

cd 'j61771'

this will give you access to files in your project code. Be sure to include the quotes and use your own project code—not mine.

ls

this will list all of the files you have. Use it if you want to make sure you are in the correct directory

ascii

get filename.ext

This will fetch your file to your directory. You don't need the j….. prefix for the file.

Note: If your file has a record length longer than 256, I think, you will need to specify the lrecl with the command

quote site lrecl=nnn

To exit ftp type

quit

Binary files

Use either SAS CONNECT with the binary option or use ftp, making sure that you specify that the transfer is a binary transfer, not an ascii or text transfer. If you are using WS_FTP then click on the binary button before you do the transfer. If you are using ftp to transfer files to your unix account, follow the same directions as for ascii files, except issue the command

binary

instead of the ascii command. Again, if your file has a record length longer than 256, you will need to explicitly set the record length with the

quote site lrecl=nnnn

comand.

SAS Datasets

As far as I can tell, there are only two convenient ways to move a SAS dataset from MVS to unix or to the pc. The first is to use SAS CONNECT, as I will describe below. The second is to create what is called a transport dataset that is in a very standard format, ftp it to where you want it, and then read it as a transport dataset. Supposedly the advantage of creating a transport dataset is all of the datasets in the library are transported at once. I couldn't figure out how to download all of the members of a library at once using SAS CONNECT. I've never done the transport thing, but I will describe how SAS says it should be done:

Creating a transport dataset (I haven't done this yet--but Tom Boehm has and he says it works.)--also you can now go to the UTCC documentation that tells you how they suggest creating portable datasets.

Summary: In MVS run a SAS job that copies the dataset library you are interested in to a transport form, ftp it where you want it, and read it in transport form.

For example suppose you want to transport the MVS SAS dataset library SOMEDATA.SAS to either the pc or unix. Run the job on MVS:

//PHILJOB JOB ,DAVES,GROUP=…. etc
….
// EXEC SAS
//OLD DD DSN=J61771.SOMEDATA.SAS,UNIT=ONLINE,DISP=SHR
//TRAN DD DSN=J61771.SOMEDATA.TRAN,UNIT=ONLINE,
// SPACE=(TRK,(10,10),RLSE),DCB=(RECFM=FB,LRECL=80,BLKSIZE=8000),
// DISP=(NEW,CATLG)
//SYSIN DD *
libname tran xport;
proc copy in=old out=tran memtype=data;
run;

It seems that you need the libname statement to specify that you are creating a transport dataset. You also must specify the lrecl and blksize exactly as in the example. This is supposed to copy all of the individual datasets in the library SOMEDATA.SAS to the transport file. Now you just ftp this dataset where ever you want it—unix, pc, whatever. Be sure that you issue commands to make the dataset have an lrecl of 80 and a blksize of 8000 and make it a binary transfer. On mvs you issue the ftp commands

binary
quote site lrecl=80
quote site blksize=8000

I don't know if it works correctly or not if you don't issue the quote site commands.

Once you have downloaded this dataset library, suppose you have saved it in the file c:\sas\data\somedata.tra and you want to actually create a dataset library from this transport file called c:\sas\data\newdata (remember—dataset libraries are subdirectories in unix and on the PC, and the datasets themselves are files in the subdirectory) then the following program will read the dataset library:

libname tran xport 'c:\sas\data\somedata.tra';
libname new 'c:\sas\data\newdata';
proc copy in=tran out=new;
run;

SAS CONNECT

You can use SAS CONNECT to download files from MVS to Unix or your PC without having to go through the ftp step above, or having to create a transport dataset in a separate step. Basically SAS connects up to MVS through a telnet connection and runs SAS on both your computer and the mainframe simultaneously. If you were a glutton for punishment you could even figure out which parts of a SAS job you wanted to run on which machine, and have the two parts interact. But I'm not going to tell you how to do that.

Preliminaries

In order to use SAS CONNECT you need a script for logging into MVS. This script is called tcptso.scr and it resides somewhere on unix, but I couldn't find it when I was looking for it. I found a copy on CMS in the C1 directory. I found an incorrect version of the script in the usr/sas611/misc/connect/ subdirectory. Hopefully CAS will have put the correct .scr files there by now. If you can't find a correct one there, I have one on my unix account and on my PC. This file needs to be in the directory that you start SAS from on unix, and it needs to be in the c:\sas directory on your PC.

In order to use SAS CONNECT on your PC, SAS needs to be able to find a file called services in the directory where tcpman.exe resides. On my computer this is the c:\lan directory. Make sure you have the services file there, and if you don't, I have a copy you can put there. Next, you need to make sure that there is an entry that looks like

telnet 23/tcp

as one of the lines in this file. Use notepad to edit the file and make sure the line is there.

Using SAS CONNECT to download a file or a SAS dataset

Interactive SAS—PC or unix

If you are in interactive SAS click on locals, signon, and enter tcptso.scr, utkmvs1, and tcp in the various boxes. SAS will begin communicating with MVS. Next MVS will ask you for your account, password, and project code. Enter these.

Downloading a SAS dataset

Suppose you have a SAS dataset library called J61771.SOMEDATA.SAS and it has 3 members: OLS, SCHOLES, and DIMSON. Enter the following commands to save these in a PC subdirectory called c:\sas\data\somedata

libname remote 'j61771.somedata.sas';

click on locals, remote submit to submit this command to MVS. Next, enter

libname local 'c:\sas\data\somedata';

click on locals, submit to submit this command to your machine.

proc download data=remote.ols out=local.ols;
proc download data=remote.scholes out=local.scholes;
proc download data=remote.dimson out=local.dimson;
run;

click on locals, remote submit to submit this command to MVS.

This should create the following files

c:\sas\data\somedata\ols, c:\sas\data\somedata\scholes, and c:\sas\data\somedata\dimson.

You could now access them with the SAS program

libname somedata 'c:\sas\data\somedata';
data f1;
set somedata.ols;
…..

There is a way to do this in batch mode on unix, but I haven't tried it yet. All of these things work on unix and on the PC in interactive SAS if you can stand the interface. Just remember unix uses forward slashes '/' rather than '\'.

Downloading an ascii or binary dataset

You can use SAS CONNECT to download ascii or binary datasets too. Functionally this is the same as using ftp on these datasets. Suppose you have an ascii dataset called J61771.SOMEDATA.DAT and another binary dataset called J61771.SOMEDATA.BIN on MVS and you would like to download them to the files c:\sas\data\somedata.dat and c:\sas\data\somedata.bin

First establish a SAS CONNECT data connection like above. Then enter

proc download infile = ' j61771.somedata.dat' outfile = 'c:\sas\data\somedata.dat';
proc download infile = ' j61771.somedata.bin' outfile = 'c:\sas\data\somedata.bin' binary;
run;

click on locals, remote submit, to submit this to MVS. Note the statement binary at the end of the binary download statement.

Important note. This download will not translate an MVS binary file into a UNIX binary file. For example, suppose you created the file J61771.SOMEDATA.BIN on MVS using the following fortran write statement

WRITE(21) PERMNO,NSHARES,RET,SIZE,PRICE

with the DCB statement giving LRECL=20.

In MVS SAS, you would have input this with the following statements

DATA F1;
INFILE IN1;
INPUT PERMNO IB4. NSHARES IB4. RET RB4. SIZE RB4. PRICE RB4.;

If you tried these statements on the downloaded version of somedata.bin you would get gibberish because MVS, UNIX, and the PC all use different machine language representations of numbers.

So suppose you used either ftp in binary mode or SAS CONNECT in binary mode to download this file to c:\sas\data\somedata.bin You would input this file on the PC or in unix with the statements

filename some 'c:\sas\data\somedata.bin';
data f1;
infile some recfm=f lrecl=20;
input permno s370fib4. nshares s370fib4. ret s370frb4. size s370frb4. price s370frb4.;

The s370fib4. and the s370frb4. formats are for reading IBM 370 unformatted output on some other machine. Note you have to know the lrecl that the unformatted file was created with when you use it! (When you did this on MVS before, the MVS catalog knew the lrecl of the file so you didn't have to state it explicitly).