Checks if there is data loss between the boundaries of subsequent IDI files produced from a single MeasurementSet. Reports how much observing time (in seconds) was lost across all boundaries.
The script does not have a
--help command line option. In fact it has
no options at all.
Run as follows:
> check-multipart-fits.py '/path/to/exp/*.IDI*' '/path/to/other/*.IDI*'
and wait. The
''s are necessary in order to let the script expand the
wildcards in stead of the shell.
Note that hierarchical wildcards are supported too to dig through directories of directories (etc) of FITS-IDI files:
> check-multipart-fits.py '/path/to/*/*/*.IDI*'
JIVE limits the IDI fits files to <= 2 GB per IDI file in order to not upset
32-bit software. If a MeasurementSet is larger than this,
break it up into chunks of at most 2 GB / piece, naming the files
> tConvert input.ms OUTPUT.IDI ... > ls OUTPUT* OUTPUT.IDI1 OUTPUT.IDI2 ... OUTPUT.IDIn
Due to a bug/erroneous assumption made inside tConvert, an unfixed version
could trigger data loss between
This tool checks the time stamps at start of
m+1 with the last time stamp
m and sums up all the differences across all
There is a caveat that if a scan boundary/gap would be such that all data from
the last scan before the gap is in chunks <=
m and data for the first scan
after the gap is in chunk
m+1, that would count as a loss of
gap duration seconds.
However the chances of that are minimal and such occurrences have not yet been observed checking the whole EVN archive (approx. 111 TB of FITS files at the time of writing this).