Software useful for synchrotron users (ESRF, ...)
Contents:
- unspec: parser of spec files
- Useful spec macros:
stepscan, step2scan, where, pic, cen, cenmass
- Readers and workers for ESRF Data File image data(.edf and .ehf files):
- applications:
gnuplot (sources in C),
zimg (sources in C),
ImageJ plugin (sources in Java)
- source codes:
pmedf for Octave/Matlab (sources in m)
February 2017
unspec: parser of spec files
This program is being written (updated, fixed, extended, ...) since our first
experiment at the ESRF in 1993 (?) until nowadays. The text below is a copy from
my freeware page.
Unspec decomposes (splits) data files created by the program spec
(by Certified Software;
hm, they have no such a command-line drive convertor)
and it extracts scans from these data files.
I think that unspec
has more options than the specex
program by ESRF; moreover, unspec
comes with the source code thus
it is available for any platform.
Further, unspec
supports also PSD/MCA scans inside or outside the
spec file.
You may know that spec is a program to driver diffractometers
and other staff at beamlines, used e.g. in the
ESRF.
[Version 18. 1. 2017] [What is new: select scan types for output; support for multiple MCA's during measurement.]
spec macros
The following are my favourite macros, developed over time measuring at the ESRF.
You can "install" them yourself via e.g. qdo stepscan.mac, but it is a good
idea that these macros become loaded automatically on spec startup
(ask your beamline local contact).
File stepscan.mac contains these commands:
File dtp_cen.mac contains these commands:
- where: draw a line in the plot window where the last scanning motor is now
- pic: move to peak of last motor scans
- cen: move to centre of last motor scan
- cenmass: move to centre of mass of last motor scan
File stepscan.mac was written by me, file dtp_cen.mac by Daniel Luebbert
and updated by me. You can use them as you like, but don't change their names please.
Readers and workers for ESRF Data Files (EDF, EHF)
EDF/EHF files mean ESRF Data (Header) Files; currently they are used at the
ESRF for storing and dealing with data from
two-dimensional detectors (cameras). They contain a text header (multiple of 512
bytes, usually 1024 bytes), followed by the binary data (EDF) or with the binary
data in another file (EHF).
The official documentation
for this format from the ESRF.
The following summarizes my contributions to the EDF reading/analyzing/writing
open-source software.
Applications: gnuplot
Gnuplot 4.2
and higher contains two new features which enable reading and displaying edf files:
- New plotting style plot ... with image.
- Generic binary data file reader.
See gnuplot 4.2 image demos for a screenshot.
What are the advantages of using gnuplot for displaying edf files? Axes and annotations:
you can use all the other well-known gnuplot commands to annotate your graph or to
print it to many output formats (postscript, png, epslatex, fig, ...).
What are the disadvantages? Gnuplot is not a typical bitmap-image drawing program, and thus
it is a bit slower in displaying very large images on screen.
You can draw edf files with this command:
plot "data.edf" binary filetype=edf with image
plot "data.edf" binary filetype=auto with image
or
set datafile binary filetype=auto
plot "data.edf" binary with image
It is useful to have a script file plotedf.gp in your path which contains
commands
set autoscale fix
set tics out
set grid front lt 6
set datafile binary filetype=auto
plot "$0" binary with image
then you can easily
call "plotedf.gp" "data.edf"
Hint: you can also put line
set datafile binary filetype=auto
into your $HOME/.gnuplot (gnuplot start-up file).
You can have a look into
breaders.c
how I coded gnuplot reader of .edf files in C.
Applications: zimg
Program zimg by Johannes Zellner generates bitmap images
(gif, png, jpeg, ppm, pgm) from arbitrary formatted two-dimensional ascii or binary data.
It can also read EDF/EHF files.
I have added a possibility to use for generating movies from a series of edf files.
See what man zimg says:
zimg -P -o- --scale=0.25 --gray *.edf | convert -delay 100 - movie.mng
read all edf files, scale them down and convert them into a gray scale movie
(mng is an animated png). Note that not using --crange makes color scaling
(z-range) individually autoscaled in each image instead of using the same
z-range for all images as in the previous example.
Some other similar commands are:
zimg -P -o- --scale=0.25 --gray *.edf | animate
zimg -P -o- --scale=0.25 --crange=100,2000 *.edf | convert -delay 333 - movie.gif
Hints for *.edf and *.ehf files associations
Often you use some kind of a file commander to browse your filesystem.
Here are some hints for file associations of *.edf and *.ehf by means
of zimg:
Midnight Commander (any OS): Go
to menu Edit extension file
or to file $HOME/.mc/bindings
and add there:
# edf, ehf
regex/\.e[dh]f$
Open=zimg --gray %f | xv -
View=zimg %f | xv -
Note that you can add --cbox
and --statistics options,
if you like to see more information on your screen.
File Commander (OS/2, Windows):
Go to menu File associations
or to file fc2.ini / fcw.ini and add there:
*.{edf,ehf}: (zimg -o c:/tmp/zimg$.png !) && (start
C:\Apps\Graphics\IrfanView\i_view32.exe c:\\tmp\\zimg$.png)
Under OS/2, use path to PMView instead of Irfan View.
Under file managers which honor MIME types (e.g. Konqueror), I recommend
to add new MIME type image/x-edf
associated to *.edf and *.ehf files with commands like in the
Midnight Commander above.
Applications: ImageJ plug-in
ImageJ is an open-source application for image data
analysis. Being written in Java, it is portable to numerous platforms. It is very efficiently
written and it is very fast.
Plug-in ESRF Reader for reading EDF files was originally written by Olof Svensson, then
recoded by me. Download the ESRF_reader
from here.
(The source code is in Java, of course.)
How to use it: copy the class file into ImageJ's plugin directory. Then you can read
EDF files when going into menus Plugins / Input-Output / ESRF Reader.
Hmm, I wonder the plugin is not on ESRF web page?
Note: there is an
ImageJ plugins repository, maybe
this plugin could be contributed there?
Source codes: pmedf package for GNU Octave and Matlab
Note: GNU Octave is an open source GPL variant to
the commercial Matlab. They are cca 99 % compatible;
mainly (if you miss something, then get an
Octave-forge package.
Package pmedf (version: February 28, 2017)
is a package for dealing with ESRF Data and Header Files .edf / .ehf (and .edf.gz/.ehf.gz).
Note: here is the previous version (version: August 11, 2010).
Advantages of pmedf:
- Compatible to both GNU Octave and Matlab.
- Contains m-code as well as faster C++ oct/mex implementation for fast reading and
direct gzipped (.edf.gz) reading.
- Reads properly EDF files according to the ESRF EDF/EHF Data Structure
Documentation; most notably:
(1) header is a multiple of 512 B;
(2) regular expression for a header line is
"\nkey [ ]*= [ ]*value[ ]*;\n".
- Includes several useful parsing and edf header dealing routines.
- Includes documentation, usage and example.
- Reuses given header when writing a new file.
- Literate programming style :-)
- Support by the author.
- Open source: GNU GPL license.
Features:
- Reading edf files: pmedf_read.m (header or header + data), pmedfread.m (only data).
- Writing: pmedf_write.m.
- Parsing information from header: pmedf_findInHeader.m, pmedf_findMnePos.m,
pmedf_findPos.m.
- Changing header: pmedf_putInHeader.m, pmedf_removeInHeader.m,
pmedf_emptyHeader.m.
- Split ascii header to a structure: pmedf_splitHeader.m, pmedf_splitMnePos.m.
- Some probably useful demo-like routines: pmedf_average4series.m, pmedf_average4list.m,
pmedf_diff_series.m.
- Conversion of a edf file series into volume VTK file: edfs2vtk.m.
- Get list of files using wildcards wildcard2filelist.m (mainly for
Matlab, because Octave has built-in function glob).
See here some examples.
Last change: January 2017