Automatic Pipeline Data Reduction
Selecting a Recipe
Use the ORAC Observing Tool (OT) to select your data reduction recipe. This will set the recipe name in the image header so that the pipeline knows how to manipulate the data. Be careful to select a recipe appropriate for the acquisition sequence e.g. sets of 4 offsets for the Nod4 recipe etc. Also, for the pipeline to work, ensure that you take any calibration frames required by the recipe before you take the science frames. Please refer to ORACDR Recipes to see what each recipe requires.
Select a recipe using , the DR component editor. A screen will be displayed as shown in the following image. Use the scroll bar (when running in the OT) to see the entire list. If you wish to change the recipe, highlight the one you want and click on the object ” set ” button. You do not normally need to change the other settings.
ORACDR must be run from a UNIX machine. At the summit this will usually be Kauwa. To specify which instrument you are using type
Data is expected to have the current UT date as a prefix, and be in the UT-appropriate summit data directories. To specify other dates and directories see the following section.
To set ORACDR into a loop where it recognises that new data has come in, reduces it according to the recipe name in the header, and displays the images, type
oracdr -loop flag &
The ampersand is only to allow you use of the xterm after issuing the command. Note loop mode is different for IRCAM/TUFTI pre- and post-ORAC. There is now a flag identifying new data. New windows will come up on your screen, an ORACDR status window, and a GAIA display.
Warning messages are informational only. If there is a fatal error such as lack of a suitable calibration frame ORACDR will exit, with an error message (such data can be subsequently reduced using the explicit commands shown below). If no data comes in for an hour ORACDR will also exit. To restart ORACDR in loop mode from for example observation number 204, type
oracdr -loop flag -from 204 &
To abort ORACDR type cntrl-c in the xterm, or close the status window.
Selective Data Reduction and Other Useful Features
It is often useful even at the telescope to run ORACDR in a non-automated, explicit, way.
- To run the DR on certain frames only
oracdr -list 1:8 &
At the telescope for example this will reduce the array tests (observations 1 to 8) and allow you to delay starting ORACDR in loop mode until you’re ready to observe.
- To override the recipe given in the header
oracdr JITTER9_SELF_FLAT -list 10:18,20:28 &
will tell ORACDR to use JITTER9_SELF_FLAT to reduce observation numbers 10 to 18 and 20 to 28, instead of whatever recipe was in the header.
- To override the default calibration frame
oracdr -list 21:25 -cal dark=dark_9 &
will allow you to use a different dark from the default (taken closest in time) avoiding for example using a dark with a persistence stain. Pointing to a specific flat can also be useful.
- To not have the image display windows come up
oracdr -list 21:25 -nodisplay &
This can simplify your desktop if you’re doing something routine, or re-reducing data that you do not need to see again.
- To restart loop mode reductions
oracdr -loop flag -from 204 &
Note loop mode is different for IRCAM/TUFTI pre- and post-ORAC, there is now a flag identifying new data.
Reducing Data in a Non-Standard Data Directory
To point ORACDR to non-standard directory areas use for example
oracdr_ircam 20000101 setenv ORAC_DATA_IN /local/ukirt/tempdata/skl/20000101/raw/ setenv ORAC_DATA_OUT /local/ukirt/tempdata/skl/20000101/reduced/ oracdr -list 1:999 ....
To point ORACDR to data directories in e.g. /export/data/skl/ instead of /ukirtdata/ type
setenv ORAC_DATA_ROOT /export/data/skl/
Help can be obtained by typing
oracdr_ircam oracdr -h
More detailed help is available on individual recipes by typing for example
oracdr_ircam oracman NOD4_SELF_FLAT_NO_MASK
The data reduction recipes can be cpu-intensive. Use the sub-array where possible. The _NO_MASK options are faster and the _BASIC faster still. If you are using the full array and need your reduced mosaics coming in in almost real-time you will need to use the _BASIC recipes. It will help if you use a different machine for things like Netscape, email, etc.