A trick to read directly GZIPPED data files

We can save large amount of disk space by directly reading compressed (Gzipped) files in our test beam analysis programs. This is achieved by a simple Unix trick.

mkfifo /tmp/PIPE

Notice that on Linux, the /tmp/ is probably necessary if your home directory is NFS mounted. /tmp/ is a local disk, avoiding this kind of problem.

This created a named pipe (or a fifo). This pseudo file is read by the program in the usual way (cfopen etc) but in ZEBRA C Exchange mode (LI). 

call cfopen(lundei,0,lrecl,'r   ',0,'/tmp/PIPE',iostat)
if(lundei.eq.0.and.iostat.eq.0) iostat = 7777
iquest(1) = lundei
infori = 'LI'
call fzfile(lunin,lrecl,infori)

Then the data are sent into the pipe. In order to allow C-Zebra to read the file, you need to compile and link cfget.c (modifications to the Cern lib version). This is done with the cfget.c subprogram (Thanks to Francesco Ragusa).

Compile the subprogram first:

cc -c -I/cern/pro/include -o cfget cfget.c

Then issue the command 

zcat the_compressed_file.gz > /tmp/PIPE
gzcat the_compressed_file.gz  > /tmp/PIPE
gzip -d -c  the_compressed_file.gz > /tmp/PIPE

The first or secong form could be available on your machine. They should 'cat' a gzipped file on stdout. The latest form should be working on any system.

This command can even be included in a system call inside the analysis program.

call system('gzip -d -c the_compressed_file.gz > /tmp/PIPE & 2> /dev/null')

In case of problem, please send me an email.

MTh