I have seen this bug in a few cases, all of them replicated by the following steps:
Opening antab_editor for a given observation, loading multiple antab files.
For a given telescope, flag a few outlier Tsys measurements. In that sense that there is only one or two columns for a given time stamp without values. The rest of IFs are kept.
Select "fill empty values" > "Interpolate".
The empty time stamps / IFs do not get filled.
I guess it may be due to the way that antab_editor checks for empty values, so it may not get realized about these empty cells in particular.
You can test this issue with the experiment EM144B stored at jop83:/jop83_0/pipe/in/marcote/em144b. You can directly load the current *.editor files, where the one for Ir contain a few of these empty cells (flagged from existing data in the original antabfs file) and the interpolation seems to not work to fill them.
I have seen this bug in a few cases, all of them replicated by the following steps:
- Opening `antab_editor` for a given observation, loading multiple `antab` files.
- For a given telescope, flag a few outlier Tsys measurements. In that sense that there is only one or two columns for a given time stamp without values. The rest of IFs are kept.
- Select "fill empty values" > "Interpolate".
- The empty time stamps / IFs do not get filled.
I guess it may be due to the way that `antab_editor` checks for empty values, so it may not get realized about these empty cells in particular.
You can test this issue with the experiment `EM144B` stored at `jop83:/jop83_0/pipe/in/marcote/em144b`. You can directly load the current `*.editor` files, where the one for `Ir` contain a few of these empty cells (flagged from existing data in the original `antabfs` file) and the interpolation seems to not work to fill them.
Thanks for the report. The problem turned out to be double timestamp values. This caused the interpolation algorithm to return "Not a number" values, which is exactly what we use to represent non existing values (the internal representation is done using pandas).
I added a warning message indicating the double timestamps, when this problem occurs. This will stop the interpolation, so it will not fix the problem, since I'm not sure which of the 2 timestamps to use. I can imagine that if this occurs frequently a button of some sort to remove double values mgiht be useful (maybe just use the last of the duplicate timestamps?).
Thanks for the report. The problem turned out to be double timestamp values. This caused the interpolation algorithm to return "Not a number" values, which is exactly what we use to represent non existing values (the internal representation is done using pandas).
I added a warning message indicating the double timestamps, when this problem occurs. This will stop the interpolation, so it will not fix the problem, since I'm not sure which of the 2 timestamps to use. I can imagine that if this occurs frequently a button of some sort to remove double values mgiht be useful (maybe just use the last of the duplicate timestamps?).
Ok! Sorry I actually did not noticed that the timestamps were exactly the same. I imagine that there would belong to TPI measurements taken in a fraction of a second or a few seconds (so intrinsically in two different time stamps). But then during the creation of the ANTAB file the writen timestamps were the same as they are typically rounded to 0.01 min (~6 s).
The warning will be fine. Thanks!
Ok! Sorry I actually did not noticed that the timestamps were exactly the same. I imagine that there would belong to TPI measurements taken in a fraction of a second or a few seconds (so intrinsically in two different time stamps). But then during the creation of the ANTAB file the writen timestamps were the same as they are typically rounded to 0.01 min (~6 s).
The warning will be fine. Thanks!
I have seen this bug in a few cases, all of them replicated by the following steps:
antab_editorfor a given observation, loading multipleantabfiles.I guess it may be due to the way that
antab_editorchecks for empty values, so it may not get realized about these empty cells in particular.You can test this issue with the experiment
EM144Bstored atjop83:/jop83_0/pipe/in/marcote/em144b. You can directly load the current*.editorfiles, where the one forIrcontain a few of these empty cells (flagged from existing data in the originalantabfsfile) and the interpolation seems to not work to fill them.Thanks for the report. The problem turned out to be double timestamp values. This caused the interpolation algorithm to return "Not a number" values, which is exactly what we use to represent non existing values (the internal representation is done using pandas).
I added a warning message indicating the double timestamps, when this problem occurs. This will stop the interpolation, so it will not fix the problem, since I'm not sure which of the 2 timestamps to use. I can imagine that if this occurs frequently a button of some sort to remove double values mgiht be useful (maybe just use the last of the duplicate timestamps?).
Ok! Sorry I actually did not noticed that the timestamps were exactly the same. I imagine that there would belong to TPI measurements taken in a fraction of a second or a few seconds (so intrinsically in two different time stamps). But then during the creation of the ANTAB file the writen timestamps were the same as they are typically rounded to 0.01 min (~6 s).
The warning will be fine. Thanks!