Spectroscopy with UIST Observing single page
UIST – Instrument Run Up and Down
The Telescope Systems Specialist (TSS) will run the instrument up and down.
The observer takes the data using the OCS and runs the data-reduction pipeline (orac-dr).
The TSS runs UIST from the “cassControl” gui. (Note – cassControlEng gives a similar Gui, though data are then saved to engineering directories.)
From the above window open the uist_oper screen. From here, datum the wheels and turn the black-body on. (Use the uist-ccs console to reboot the CCS [mechanisms – grism, filter wheels, etc.] if necessary.)
To run-up UIST the TSS and observer simply run through the steps on the left of the window:
- START [1] – this starts the low level software AND launches a GAIA quick-look display on Ohi. The log in the right-half of the window should say Starting camera 5 and then (after 10 secs or so) wfacq5: drama:Running filesave:Running camera:Running rtai:Running. If you DON’T get a GAIA display – don’t go any further – since you won’t get one later in the run-up sequence.
- OCS_UP [2] – this runs the “Query Tool” (QT) – used for selecting which MSBs to observe, the “Queue monitor” – used for lining up observations to be executed, and the “Sequence Console”, which will actually run (execute) the observations. The observer should run this on Ohi.
- ADD INST [3] – this will activate the Sequence Console on Ohi (which will have come up blank).
- ENABLE [4] – this will enable the array. Note that the array says “On” on the sequence console.
Once the above four steps have been executed successfully, the observer can run the UIST Array tests and take data.
The run-down sequence is displayed in red (steps [6] to [9]).
- DISABLE [6] – to disable the array; it will say “Off” in a red box on the sequence console.
- REM INST [7] – this kills the Sequence Console.
- OCS_DOWN [8] – to be done by the observer. This should kill the QT and Queue Monitor on Ohi, though you may have to close the GAIA quick look manually.
- STOP [9] – this will finally run down the low-level software. The log in the right of the window should say: wfacq5: drama:Stopped filesave:Stopped camera:Stopped rtai:Stopped.
IMPORTANT: In you are unsure about whether the array is enabled or disabled, check the LEDs on the controller in the dome. Most of the green LEDs should be OFF.
Once the software has been run down, set UIST to dark from the uist_oper screen, switch off the black-body and arc lamps, and put the calibration unit in the beam.
ENGINEERING: Instead of running cassControlEng, “ocs_up -simTel -eng” can be run from the command line, together with “uistMenu”.
NUKE: use this to kill drama and rtai processes if you are having problems running up.
Spectroscopy: Preparing a Programme
Preparing an Observing Programme: the UKIRT-OT
FIRST TIME USERS: Please read the General Introduction to the OMP before reading the notes below (which deal only with spectroscopy).
Your complete observing programme can be prepared either in Hilo or before you arrive in Hawaii from your home institute (provided you have access to the ukirt-ot). From any Unix or Linux box in Hilo (or at the summit) type ukirtot to run-up the observing tool (the OT).
A small window will appear (containing a photo of UKIRT) in addition to the copyright notice window; you may use the former to open existing programmes, create new programmes or access the database. If you’re new to ORAC, close the copyright box and read on…
TIP: Prepare ONE observation (one MSB), save this to the database, and ask your support scientist to check it over. You can then simply use copies of this MSB for your other targets. Changing one MSB is a lot easier than having to change a dozen or more!
The UIST Template Library
Start with a “Template MSB” from the template library (available template observations are described on a separate page; a table of DR recipes is also available). DON’T try and write your MSBs from scratch, and DON’T make huge changes to the template MSB without discussing these with your support scientist (obviously changing slits, grisms and using different offsets on sky are ok). Major structural changes are probably not necessary!
With this important point in mind, open the Template Library by selecting this option from the menu under “File” (top-left corner of the small “UKIRT” window). At the same time, create a new programme by selecting this option from the same (File) menu. After a few moments, two Programme windows – like the one shown in Fig.1 – will appear.
In the template library, click on the button to the left of the UIST “folder” icon and open the folder labeled Spectroscopy. There you’ll find the available sequences for spectroscopy (plus some useful notes!). Examine those that may be of use to you by clicking on the button to the left of the icon (blue/pink square); the observations (calibration, standard and source) should be displayed (e.g. Fig.2). Click on any observation to unfold this as a “flow chart”; the elements within this observation are described below.
A Typical Spectroscopy Observation
In a nut-shell, a spectroscopy observation should comprise a flat, an arc and a sequence of “object” and “sky” exposures on a standard star, followed by a similar sequence of object/sky frames on the target itself. The example below contains all of these components.
Point sources may be “nodded” up and down the slit so that the source is observed even in the “sky” frames. Subtraction of the sky frames from the object frames will remove OH line and thermal background emissions, giving positive and negative spectra which may be extracted separately and combined to give a spectrum of the source. An arc spectrum may be used to accurately wavelength-calibrate the data, and a similarly-reduced standard star spectrum can be used to divide out atmospheric absorption bands and flux-calibrate the source spectrum.
For an extended source, nodding to blank sky will probably be necessary. In this case only half of the data contain spectra from the target, though subtraction of the “sky” spectral images from the “source” spectral images will again yield an image largely free from OH sky lines. Sections of this spectral image may then be extracted and calibrated to give reduced data at different locations along the slit/across the extended target.
Moderately extended sources may be slid up and down the slit; here’s an example of +30arcsec and -30arcsec offsets relative to the centre of the array.
Flexible Scheduling and Minimum Schedulable Blocks:
All UKIRT observing are flexibly-scheduled. Consequently, observations must be grouped within “Minimum Schedulable Blocks”, or MSBs. An MSB represents the minimum amount of data that needs to be obtained for an observation to be useful. You or indeed any other observer will then be equipped to properly observe one or more of your targets, simply by executing everything in the MSB. For spectroscopy, an MSB usually includes a flat, arc, standard star observation and the target observation; in Fig.2a below the “opened” MSB is represented by the blue+pink square.
Flats, arcs, the UIST component and inheritance
In Fig.2a we show an example MSB for a point source. The programme currently only contains one MSB; many may be needed if multiple targets are to be observed. The “Point Source” MSB, now labeled “HK Spectrum of HH1”, has been opened; it contains a Calibration (flat and arc) observation , a Bright (standard) star observation and a Science target observation. UIST must be set up in exactly the same way for the flat and arc as for the standard and target observations (i.e. same slit width, grism, etc.). This is achieved by placing the UIST component (the broken blue square) above the three observations. The observations then “inherit” the UIST component parameters; the slit width, position angle, grism, etc. Only the exposure time is changed in each observation, as described below. The flat and arc have default exposure times, set by clicking “Use defaults” in the flat and arc observations (see Fig.2b).
The three observations in Fig.2a also inherit the standard star coordinates (from the Target component), although these coordinates are subsequently “overwritten” by the coordinates of the science star, HH1, in the third observation. (Note that, in Fig.2d, the science target observation contains a second Target component.) With this arrangement, the flat and arc will be observed (first) at the location of the standard star.
Fig.2a – the UIST component
Fig.2b – Flats and Arcs
Fig.2c – Imaging acquisition
Fig.2d – Flushing the array
Fig.2e – a longer expos time on target
Fig.2f – Repeats and Offsets
The Components of a Spectroscopy Observation
To understand the layout of a typical spectroscopy observation, consider the MSB in Figs.2a-2f. Each observation needs THREE components (the “broken” blue squares), which specify the Target information (target and guide star coordinates), the UIST instrument configuration and the Data Reduction (DR) Recipe. All three observations will inherit the UIST configuration from the components above them; the flat/arc and standard star observations will inherit the standard coordinates from above; the data reduction recipe is specified inside each observation.
- The Target information component is used to enter the source coordinates. It may also be used to display a Digitised Sky Survey image of the target field, the instrument aperture size, offset positions on the sky and various guide-star catalogues (see this ORAC-OMP Guide for a comprehensive description of this tool).
- The UIST instrument component is used to select grism, slit width, exposure time, position angle, etc. In Fig.2a the UIST component is highlighted, so that the UIST configuration is displayed on the right half of the window: in this case, UIST has been set for HK spectroscopy with an east-west, 4-pixel-wide slit. One 3sec exposure will be taken (with the default NDSTARE 1024×1024 readout) at each object and sky position (defined by repeats and offsets; see below).
- The DRRecipe component allows you to select the recipe appropriate to your mode of observation, so that the DR can reduce the data on-line. An observation copied from the template library should already have the DR recipe set correctly. All object files obtained as part of this observation will be flagged with this recipe. In our example, the recipe STANDARD_STAR is used to reduce the standard star observations (Fig.2c), and FAINT_POINT_SOURCE is used to reduce the spectra of the science target, HH1 (Fig.2d).
Fig.2c shows the observation of the standard star. Recall that this inherits the UIST component and the standard star coordinates from above, so it only contains the DR recipe component. Below this DRRecipe component there is a “running man” icon or “iterator” labeled Sequence. Embedded “within” this Sequence iterator is the Spec/IFU Target Acquisition observation (an “eyeball”) and the actual spectroscopy Exposures (the Observe eye symbol), as well as a note and two nested iterators (more running-man symbols), labeled Repeat and Offset. These iterators are stacked much like “embedded do-loops” in a computer programme. With the setup in Figs.2c an object-sky-sky-object “quad”, defined by the offset iterator, will be repeated two times (specified by the repeat iterator). The offsets up and down the slit are set in the offset iterator by “p” and “q” parameters, q being along the slit and p being perpendicular to it, regardless of the slit position angle (Fig.2f). For point sources we recommend a 12 arcsecond slide up and down the slit, and an east-west slit position angle. The offsets can be changed (if a larger nod is required, say) by clicking on the offset iterator symbol. And of course, if only one quad is needed on the star, the repeat iterator can also be set to one.
The observation of the science target, HH1 (Figs. 2d-2f) is much the same as the observation of the standard described above EXCEPT for the optional “flush-array” Darks and the UIST Spec/IFU iterator used to change the exposure time for the science target. The HH1 observation also contains a target component which provides the coords of HH1 and a guide star.
The use of the darks to flush the array is described below (these are less important now that coadds have been implemented with imaging-acquisition). The UIST spec/ifu iterator is used to override the short exposure time used on the standard star (and defined in the “blue” UIST component inherited by this observation of HH1). In this case, 240secs will be used per exposure on HH1 (Fig.2e). Finally, the offsets and repeats (Fig.2f) are again used to move between the target and sky positions, and to repeat this “quad” of exposures to build up signal-to-noise. For HH1, 5 repeats with 240sec exposures would give a total of 20mins on the target. All of these data will be grouped together into a reduced spectral image by the DR, so that the observer knows just how great his or her final data will be …
IMPORTANT: If you change the wavelength (or anything for that matter) in the UIST component, you must click on “Use default” in the FLAT, ARC and the flush-array DARK. This ensures that these observations pick up the changes made in the UIST component. Remember, though, to set the flush-array dark exposure time back to a few seconds (and 1 co-add) – you don’t want to be taking lengthy darks to flush the array.
Imaging acquisition
For both the bright standard and faint science target the source will be “acquired”, or put down the slit, in imaging acquisition mode. The TSS will do this at the telescope. However, the standard and science target observations must already include the “Target Acquisition” eyeball (e.g. Fig.2c and 2e). By clicking on this icon in the OT you can enter either 9-10th mag for the standard star, or an appropriate magnitude for your fainter science target. For the bright standard the shortest possible exposure time must be used (9-10th gives the minimum 1sec full-array readout). For a faint science target 20secs or more may be needed; either enter this exposure time directly or select a fainter source magnitude. Source acquisition is discussed further on a separate page. Beware of latency, however (see below) – for acquisition of faint targets use short exposure times and a few coadds (e.g. 4 x 5sec for the HK grism) rather than one long exposure (1 x 20sec).
Image Latency
UIST suffers from image latency, i.e. residual signal (like dark current) at less than 1%. Because imaging acquisition involves taking images, often with long-ish exposures through a very broad spectral blocking filter, this can leave some residual sky signal on subsequent frames. Likewise, if a bright star is observed in acquisition, there may be a residual (weak) image of the star in the next frame or two. This latent signal gets weaker with time, and it should to some extent “subtract off” when skies are subtracted from object frames.
The problem can be avoided by using short exposures and a few coadds for imaging acquisition, rather than one long exposure. The penalty is readout overheads, specifically 1-2 seconds per coadd. We recommend using three or four 5sec exposures for faint targets (one 1sec exposure for a bright target). However, even with these short exposures, residual signal from imaging acquisition could still introduce additional noise to the first few frames taken directly after imaging acquisition. Consequently, it may be a good idea to “flush” the array, by taking a few short darks after imaging acquisition, and before taking a first long (perhaps two or three-hundred second) spectrum of the science target. The optional flush darks are available for this purpose. They are potentially useful for very faint targets and/or long spectroscopy exposure times and/or short wavelengths, although their usefulness is limited – the residual signal fades with time, not the number of read-outs.
Saving and Storing your handy-work…
When preparing MSBs, keep saving the file to disk: click on “File – Save As” at the top-left corner of the programme window. Once the programme is complete, save it to disk one last time. If you already have a project ID (e.g. u/03a/99) and password you may then store it to the telescope site (Database – store to telescope site). The programme can then be retrieved from the database at the summit and your observation executed.
HOT TIP: Set up one MSB – for just one flat/arc, standard and science target, say – then send this to your Support Scientist. He or she will check it over. In most cases, you can then simply copy this MSB “n” times and just change the coordinates of the standards and science targets.
The above discussion and example is of course meant only as a brief guide. A more comprehensive guide to the OMP, the OT and flexible scheduling in general is available here. A UKIRT Support Scientist is assigned to each project (Visitor or Queue-scheduled) to assist with the preparation of OT observations.
Choosing the best exposure time
In order to be background limited in the non-thermal regime (<2.3 microns), the sky noise must be greater than the array read noise (discussed earlier). Times needed to reach background-limited performance are listed in the section on sky counts. Narrower slits will obviously require longer, though beyond about 2.2 microns, the background increases markedly as the thermal background from the sky and telescope begin to increase, and the background-limited exposure time drops rapidly. However, for non-thermal spectroscopy of faint sources, long exposure times will be needed: 240sec exposure times are recommended for all sources fainter than about 9th magnitude, or 120sec if the OH sky lines are not being subtracted off too well (because of cirrus, say). At the end of the day, the longest “possible” exposure time should be used, given the caveats outlined below.
As an example, during commissioning the same star, Roque 25, was observed 4 times with the HK grism. On each occasion, the same total time (16 minutes) was spent on the target. However, this sixteen minute period was made up of individual exposures with different exposure times. The white spectrum in the figure below represents the co-addition of 48 10sec exposures; the individual exposures were not background limited and the source is barely detected. The blue spectrum, on the other hand, comprises just 2 240sec exposures. This time, the H and K-bands are easily discernible and a decent detection was attained (the red and green plots represent 16 30sec exposures and 4 120sec exposures).
As already noted, in photometric conditions even longer exposures may be preferable, particularly in the J-band. The plot below is the (flat-fielded) difference of two 600 second exposures on a standard star. Note how well the sky lines subtract off, even after 20 minutes.
The drawbacks to using long exposures include variations in the sky background and OH line intensities, OH line saturation (at H and K), and the increasing possibility of spikes on individual or small groups of detector pixels. If the critical wavelengths are well clear of OH lines (assuming sufficiently high spectral resolution is being used to allow observations between the OH lines), then the ~5-10 minute variation timescales of the OH lines may not cause concern (i.e. imperfect sky subtraction may not be a problem). But if a desired spectral feature is close to an OH line, and particularly if conditions are not photometric, then these variations can become problematic. In addition, long exposure times can lead to a lot of wasted telescope time if a fault should occur or a bad observation be loaded and executed.
For spectra obtained while nodding along the slit, subtraction of the negative spectrum from the positive spectrum will remove most of the sky and OH fluctuations because both vary slowly across the rows of the array. When observing faint and compact sources it is always advisable to nod a small number of rows along the slit, so that the cancellation of sky and OH residuals is as accurate as possible (you are also more likely to keep the source on the slit!). Remaining residuals can be removed with polyfitting techniques, using blank sky rows adjacent to the rows of interest (although doing this will increase the noise in the final spectrum a little).
Finally, note that when observing in between the OH lines at moderate spectral resolution (with the short-/long- grisms) it may be impossible to reach background limited performance. The lower-resolution HK grism, on the other hand, is background limited at “reasonable” exposure times at all wavelengths because an OH line is present in almost every resolution element. Background limited exposure times are listed here.
To check whether a particular emission line falls on or near a strong OH sky line, consult the Tables of OH Spectra compiled by Tom Geballe and Tom Kerr from CGS4 observations.
Spectroscopy: Template Observations
UIST observing programmes will be set up using the ORAC-OMP version of the OT. Your starting point will be the template sequences in the UIST library (under UKIRT Template Library). Available sequences are described below.
| MSB | Contents | Description | DR Recipes Used |
|---|---|---|---|
| Point Source nod along slit | Flat/Arc, Bright-(standard)-star Target-observation | Observe a flat and arc, then standard (slid +6″ then -6″ along slit). Acquire target in imaging mode, then observe by again sliding +6″ then -6″ along the slit. Default point-source spectroscopy mode. | STANDARD_STAR POINT_SOURCE |
| Faint Point Source nod along slit | Flat/Arc, Bright-(standard)-star Target-observation | Observe a flat and arc, then standard (slid +6″ then -6″ along slit). Acquire target in imaging mode, then observe by again sliding +6″ then -6″ along the slit. The DR assumes standard and target spectra are on the same row | STANDARD_STAR FAINT_POINT_SOURCE |
| Extended Source nod 60″ along slit | Flat/Arc, Bright-(standard)-star Target-observation | Observe a flat and arc, then standard (slid +6″ then -6″ along slit). Acquire target in imaging mode, then observe by sliding source +60″ then -60″ along slit. | STANDARD_STAR EXTENDED_SOURCE |
| Extended Source nod 24″ along slit | Flat/Arc, Bright-(standard)-star Target-observation | Observe a flat and arc, then standard (slid +6″ then -6″ along slit). Acquire target in imaging mode, then observe by sliding source +24″ then -24″ along slit. | STANDARD_STAR EXTENDED_SOURCE |
| Extended Source nod to sky | Flat/Arc, Bright-(standard)-star Target-observation | Observe a flat and arc, then standard (slid +6″ then -6″ along slit). Acquire target in imaging mode, then observe by nodding between source and blank sky. | STANDARD_STAR EXTENDED_SOURCE |
Checklist for Preparing OT Programmes
Below we list a few things to check after you’ve prepared and validated your programme. This list isn’t complete (and we’ll add pointers as they occurs to us), though it may help avoid loss of telescope time…
- After changing the wavelength (or anything for that matter) in the UIST component, did you click on “Use default” in the FLAT and ARC? This ensures that these observations pick up the changes made in the UIST component.
- Did you select Guide Stars for your targets. If there’s no guide-star specified, the telescope will assume you’re guiding on your target. If the guide star is dodgey, add a “guide2” and a note to the observer describing its availability (see below).
- Imaging Acquisition: with faint targets use short exposure times and a few coadds rather than one long exposure, to avoid latency in subsequent spectroscopy/IFU frames. 3coadds x 5secs works well.
- Maximum exposure time: is currently 240seconds with UIST spectroscopy (NDSTARE readout).
- Do ALL MSBs contain a flat, arc and a standard star in addition to the science target? ALL MSBs must contain a complete, calibratable set of observations for the science target. MSBs can contain more than one science target, though they should still be short (~1 hour at most). If you don’t want calibrations taken with every target, make this extremely clear in an observer note, and flag the flat/arc and standard star observations as optional in the OT (they then appear “green”). NOTE: calibrations don’t have to have the same position angle as standard star and/or target spectra, though obviously the same grism and slit should be used.
- Are your MSBs well-documented? Notes, flagged with “show to observer”, are EXTREMELY helpful when the observer is unfamiliar with a given project. A URL for a finding chart can be very useful too. Imaging acquisition is the BIGGEST PROBLEM faced by observers, particularly if coordinates are poor or targets are in confused/busy regions. Will the observer be able to identify your target easily during acquisition? Are the coordinates good enough so that s/he can assume that the blob nearest the nominal field centre is the target? Or will s/he need some guidance…?
Spectroscopy: Data Reduction
Introduction
Pipeline DR software is provided at the telescope to allow observers to assess the quality of their data in real time. The data reduction is actually quite sophisticated and may, in many cases, yield publishable results! Data reduction “Recipes” are provided to deal with the different methods of observing in the near-IR. If you do not find one specific to your needs, then please contact your support scientist – given sufficient notice, we may be able to provide a new recipe for your observing run.
Many of the available DR recipes, or methods of observing/reducing the data, expect a flat and an arc spectrum. Indeed, the pipeline software may stop if a flat or an arc hasn’t been observed with the same instrument setup just prior to the target observations. Likewise, many recipes insist on a standard star observation before a target observation, since this will always be required for proper data reduction, be it pipeline DR or offline reduction with Starlink or IRAF software. Note, however, that recipe versions are available which do not require a flat, an arc or a standard; these have the suffix _NOFLAT, _NOARC and _NOSTD (and _NOFLAT_NOSTD!). A full list of available recipes is given here.
To flat-field or not to flat-field…
This may indeed be the question… Strictly speaking, the pixel-to-pixel response of the array, and the wavelength-dependent transmission of the telescope and UIST optics, will be corrected for when an extracted target spectrum is divided by an extracted standard star spectrum, provided that the two stars are observed on exactly the same rows of the array. However, since this is usually not the case, it is always prudent to flat-field all data with an internal black-body lamp image. The pipeline DR will do this (unless _NOFLAT is specified in the recipe); the additional noise introduced by the flat-field division should be minimal, given the bright lamp signal. The DR recipe “REDUCE_FLAT” normalises the flat-field image by fitting a blackbody function to all rows in the dispersion direction (assuming a specific temperature for the BB lamp). Any remaining deviation in the normalised BB “image” will be due to instrumental transmission effects.
An arc, an arc…
My kingdom for an arc! An argon lamp is currently being used for wavelength calibration. Note, however, that lamp spectra may not be necessary for moderate-resolution spectroscopy, particularly in the H and K bands, where there are many atmospheric OH lines. If you choose not to observe arc spectra, make sure you check that suitably bright sky lines are available for calibration (a dark frame may also be useful for “subtracting” bad pixels off of the raw calibration frame). However, given the time needed to take an arc, again, skipping this calibration is NOT recommended
Running the pipeline
The template observing sequences available in the UKIRT-OT contain recipes appropriate to the associated observing mode. You should not normally need to change the recipe; if you do be careful as many have specific requirements in terms of flat fields and standards, which must be acquired before a target observation is obtained and reduced on-line. Tables of available DR recipes – and links to detailed descriptions of them – are available.
1. To run ORAC-DR at the telescope type the following:
oracdr_uist oracdr -loop flag
2. To run ORAC-DR at your home institute type the following:
oracdr_uist 20071225 setenv ORAC_DATA_IN /home/cdavis/my/raw/data/ setenv ORAC_DATA_OUT /home/cdavis/my/reduced/data/ oracdr -list 1:100 &
When running ORAC-DR at your home institution, the pipeline needs to know when the data were taken (since files are labelled with the UT data), where the raw data are, and where reduced data are to be written. Obviously these directories need to exist on your machine, and the raw data need to be in the specified directory. In the above example, frames 1 to 100 from 20071225 will be reduced.
Note that the “array test” observations taken at the start of the night MUST be reduced BEFORE you try and reduce a block of your own data. Use the -list option to specify the data range. For example, if frames 1-19 were darks taken to measure the readnoise and set up a bad pixel mask (in the log the DR recipe will be set to MEASURE_READNOISE and DARK_AND_BPM), and frames 38,39,40-43 were the flat, arc and ABBA-style “quad” of observations on your standard star, you could type the following:
oracdr -list 1:19,38:43 &
Having reduced the array tests and flat/arc/standard star calibrations, you can then reduce your target observations, e.g.
oracdr -from 44 &
When running ORAC-DR, several windows will open as they are needed; an ORAC text display, GAIA windows and kapview 1-D spectral plotting windows. If you are at the telescope the pipeline will reduce the data as they are stored to disk, using the recipe name in the image header.
The DR recipe name stored in each file header will be used by the pipeline. However, this can be overridden if, for example, you decide you do not want to ratio the target spectra by a standard star, e.g.:
oracdr POINT_SOURCE_NOSTD -list 31:38
where 31 to 38 were the eight observations of the target.
Help on this and other ORAC-DR topics is available by typing
oracdr -help
To exit (or abort) ORACDR click on EXIT in the text log window, or type ctrl-c in the xterm. The command oracdr_nuke can be used to kill all DR-related processes, should you be having problems.
Extracted spectra may also be examined using the splat. In the reduced data directory (cd $ORAC_DATA_OUT) type “splat &”.
Finally, note that there are tips on how to reduce and analyse Spectroscopy data in this Starlink Cookbook
Read Noise and Variance
Finally, if you re-reduce your data on another machine (down in Hilo or back at your home institute) using ORAC-DR you will probably have to reduce the READ_NOISE dark exposures taken at the start of the night first. This sequence of array characterisation observations gives a measure of the readnoise as a general health check at the start of each night of UIST observations. However, this information is also used by the DR to estimate whether the data is background-limited or not, and to assign to the data and maintain a variance array component at each stage in the reduction.
Briefly, the variance mapped across the array is derived from the readnoise variance (usually about 40e- or about 3 counts) and the Poisson variance (based on the signal from the sky and source – so the variance will be higher along the spectrum of a bright star); this then gets propagated through the DR when images are subtracted/co-added/divided-by-standard, etc. The variance is used to weight the data during optimal extraction of source spectra (Figaro’s optextract routine) so that a cleaner spectrum can be obtained.
So, each data file will consist of two components, the actual data (counts from the source and sky) and the variance calculated for each pixel across the array. You can create an image of the variance across a spectral image using the starlink-kappa commands:
> creobj type=ndf dims=0 object=gu20030101_999_var > copobj gu20030101_999.variance gu20030101_999_var.data_array > gaiadisp gu20030101_999_var
Step 1 creates a blank image called gu20030101_999_var.sdf, step 2 copies the variance information from the reduced group image gu20030101_999.sdf to the data array in gu20030101_999_var.sdf, and step three displays gu20030101_999_var.sdf in the gaia window.
Note for IRAF Users: If your data have been converted to fits using the convert package, then the two array components in each NDF (.sdf) will be converted to FITS extensions. Because pipeline-reduced spectral images contain both the data and the variance array, it will be necessary to read in each data file by specifying the data array specifically.
In other words, if you get an error like ERROR: FXF: must specify which FITS extension , try
> imhead u20020101_00999_wce.fit[0] or > imhead u20020101_00999_wce.fit[1]
The former will be the data, the latter the variance.
Spectroscopy: Data Reduction Recipes
The following tables describe the UIST spectroscopy ORACDR recipes; click on the recipe name for more information. The Requirements listed below are frames that should be obtained prior to use of the recipe, although data can always be re-reduced when the appropriate frames have been taken. Flats and standard stars must be obtained through the same grism and slit.
| Recipe Name | Requirements | Comments |
|---|---|---|
| REDUCE_BIAS | None | Reduce a Bias observation |
| REDUCE_ARC | None | Stores as an arc |
| REDUCE_FLAT | Dark | Stores as a flat |
| Recipe Name | Requirements | Comments |
|---|---|---|
| POINT_SOURCE | Flat, Arc, Standard star | Default point source reduction |
| POINT_SOURCE_NOSTD | Flat, Arc | For sources without a standard |
| POINT_SOURCE_NOFLAT | Arc, Standard star | For sources without a flat |
| POINT_SOURCE_NOARC | Flat, Standard star | For sources without an arc |
| POINT_SOURCE_NOFLAT_NOSTD | None | Sources with no flat or standard |
| FAINT_POINT_SOURCE | Flat, Arc, Standard | For faint targets (weak continuum) |
There are versions of these recipes which reduce a standard star sequence; these data are then filed for use with the target observations (see e.g. STANDARD_STAR)
| Recipe Name | Requirements | Comments |
|---|---|---|
| EXTENDED_SOURCE | Flat field, Arc, standard star | Default extended source reduction |
| EXTENDED_SOURCE_NOSTD | Flat field, Arc | For sources without a standard |
| EXTENDED_SOURCE_NOFLAT | Standard star, Arc | For sources without a flat |
| EXTENDED_SOURCE_NOARC | Standard star, Flat | For sources without an arc |
| EXTENDED_SOURCE_NOFLAT_NOSTD | None | Sources with no flat or standard |
| Recipe Name | Requirements | Comments |
|---|---|---|
| REDUCE_SINGLE_FRAME | Flat field | Reduces a single frame |
| REDUCE_SINGLE_FRAME_NOFLAT | None | Reduces a single frame without a flat |
| Recipe Name | Requirements | Comments |
|---|---|---|
| POINT_SOURCE_POL | Flat field, Arc | Reduces Spec-Pol quads |
| Recipe Name | Requirements | Comments |
|---|---|---|
| QUICK_LOOK | None | Display raw data file (and convert from HDS to NDF) |
| NIGHT_LOG | None | Generate an ascii log |
| EMISSIVITY | flat | Calculates telescope emissivity |
Spectroscopy: Data Format and Location
Wavelength scale
The astute observer may notice that the dispersion axis on the raw frames is flipped, i.e. wavelength INCREASES to the LEFT. To deal with this, the pipeline DR updates the CDELT1 header early in the reduction process. The DISPERN header (the dispersion in um/pixel) remains positive, but the CDELT1 header (the wavelength co-ordinate increment, also in um/pixel) becomes negative, although it retains the same absolute value as DISPERN.
Note that, at present, orac-dr does not accurately wavelength calibrate UIST long-slit spectroscopy data (IFU data are properly calibrated). Only an estimated wavelength scale is attached to the data.
Slit orientation on the sky
A slit position angle anywhere between 90 degrees and -90 degrees may be selected in the OT. All angles are measured east-of-north. An angle of -90 degrees is recommended for point sources; this will put EAST to the TOP of the array. In the table below we list the orientation of the slit on the sky for specific position angles.
| Posn. Angle (E of N) | Top of slit is to the… |
| -90 degrees | East |
| -45 degrees | South-East |
| 0 degrees | South |
| +45 degrees | South-West |
| +90 degrees | West |
Raw files
Raw, unreduced, data files are in /ukirtdata/raw/uist/YYYYMMDD (where YYYYMMDD is the numeric UT date) or $ORAC_DATA_IN. The files are stored as starlink HDS containers (a file with multiple data arrays). Each file is equivalent to 1 observation, and as such contains a header component and 1 or more (actually NINT) integrations. Each integration is stored as an NDF component (single data array) of the HDS file. The raw filenames are uYYYYMMDD_NNNNN.sdf where NNNNN is the observation number, padded with leading zeros when necessary. Note: you may have a hard time working with this data format unless you have access to starlink and ORAC-DR at your home institute (see below).
Reduced single frame files
Reduced data files are in /ukirtdata/reduced/uist/YYYYMMDD (where YYYYMMDD is the numeric UT date) or $ORAC_DATA_OUT. The filename structure is: (PREFIX)(UTDATE)_(FRAME NUMBER)_[EXTENSION].sdf, where (THIS) is always there and [THIS] is optional. (PREFIX) is the letter u if the file contains data from a single observation. It is gu if the file contains data from a number of observations – i.e. a group (see below). _(EXTENSION) is used by individual primitives for their output files; think of a primitive as a single step within a recipe. The pipeline keeps track of passing these files between primitives; useful ones are left on the disk at the end so you can look at intermediate data products if you wish.
For example, u20000410_00123_ff.sdf would be data from a single observation, number 123, that has had all the reduction steps up to and including flat fielding applied to it (see, e.g., the table below). Reduced files can have either HDS (multiple data arrays) or NDF (single data array) format as appropriate. Extensions for single frames are listed in the table below.
| Extension | Description |
| _mraw | A Modifiable copy of the raw data |
| _bp | Bad Pixel Mask has been applied |
| _rnv | Read Noise Variance added |
| _sbf | Subtracted Bias Frame (only for STARE frames) |
| _pov | Includes Poisson Variance |
| _bgl | How Background Limited the integration is |
| _m | Masked |
| _c | Coadded |
| _ff | Flat Field applied |
| _nf | Data is a Normalised Flat Field |
| _reo | Wavelength flipped so increases to right |
| _wce | Wavelength Calibrated by Estimation, equivalent of the old CGS4 ro* file |
| _ss | Sky Subtracted |
Reduced group files
The pipeline adds the individual frames into a “group” file as they are processed. The group number is usually the frame number of the first frame in the group. Extensions for group frames are listed in the table below. Note that some of these data are 2-D spectral images, and some are extracted 1-D spectra; the former will be displayed in gaia and the latter in a Kapview plotting window.
File Extensions for group files
| Extension | Description |
| No extension | This is simply the difference between all the main and offset beam images. These are the equivalents of the old CGS4 rg* files. |
| _srsvv | Residual sky subtracted from the above file |
| _ypr | The spectral image collapsed along the x (dispersion) axis |
| _oep | The opt-extract profiles |
| _oer | The opt-extract profiling residuals |
| _oes | The opt-extracted spectra |
| _ccs | Cross-Correlated and Shifted, spectrally aligned to beam 1 |
| _ccf | The Cross-Correlation Functions from forming the _ccs frames |
| _sp | Extracted Spectrum – the coaddition of all the beams |
| _snr | S/N plot of the extracted spectrum |
| _nsp (for the ratioing star only) | _sp normalized to 1-sec exposure |
| _std (for the ratioing star only) | _nsp divided by a blackbody curve with a temperature same as that of the ratioing star |
| _aws | Aligned with Standard – spectrally aligned with the standard star |
| _scf | Standard cross-correlation function from forming the _aws frames |
| _dbs | Divided by a Standard star (including the standard star black body model) |
| _fc | Flux Calibrated |
So which data products should I work with?
Once you have transferred the raw data to your home institute, you can run ORAC-DR locally to produce reduced and semi-reduced data products, and to get a first-look at the data.
For post-reduction with Starlink software or IRAF you probably only need the individual _wce spectral images; from these you can do your own “object-sky” differencing and subsequent coaddition of the data to produce a “group” spectral image (that contains all of the data on any one target). Alternatively, if the weather was stable and all data are usable, you may be able to simply work with the reduced group image produced by ORAC-DR – this is the group file in the above table with no extension. Either way, the arc spectra (also stored as _wce files) may then be used to accurately calibrate the dispersion axis of the 2-D image or of extracted spectra. An extracted spectrum from the “group” spectral image of your standard star can then be used to flux-calibrate and correct for telluric absorption lines.
Arguably, the “most useful” data products are highlighted in red, though this is really up to you…
Finally, note that each time you run ORAC-DR a “hidden” log file is written called .oracdr-number.log. If you type ls -a you’ll see these. Have a look at one of them; it’ll list everything the DR did during a particular session. This could be useful for following DR steps or identifying some of the more obscure files.
NOTE: At present, long-slit spectroscopy data only have an ESTIMATED wavelength scale (IFU data are the exception; these have a proper wavelength calibration).
Converting to fits
Non-Starlink users may convert their data to fits format with the Starlink routines:
> convert
> ndf2fits "*" "*"
or, to get .fits as the file extension (instead of .fit)
> ndf2fits "*" "*|.fit|.fits|"
IRAF users having problems reading pipeline data into iraf should check the notes on multi-component data arrays at the bottom of the previous page.
Spectroscopy: Target Acquisition
Acquiring Spectroscopy Sources in Imaging Mode
Raw or sky-subtracted Movie-mode images are used to position the source on the slit or IFU (these images are not saved to disk). Note that movie-mode images will be taken with the same position angle as your spectroscopy or IFU observations.
Before you get to the telescope
- In the OT, prepare standard star and target sequences which include the Target Acquisition “Eyeball” – see the programme preparation for spectroscopy web pages for details. Remember to select the shortest possible exposure time in the acquisition eyeball for the standard (1 second), though longer for the science target (20 or 30 seconds, perhaps, made up of a few coadds with 5sec exposures). The acquisition mode will be set automatically, depending on which grism is in use (i.e. with the same spectral blocking filter, but with the grism and slit removed from the beam).
When observing
- After having taken the flat and arc, load up the standard star sequence, run the sequence, slew to the target and configure the instrument for source acquisition but DO NOT continue after the green “Break for Acquisition” line (note that only the slit and grism wheels are reset, so this process should be quick).
- Having reached the break, run Movie and use the Gaia tool “View – Pick Object” to centroid on the object and measure its precise position (magnification of 2 or 3 usually works best). The telescope operator will move this target onto the correct pixel on the array, and probably ask you to check the position a second time.
- Once correctly positioned, Movie should be stopped and dismissed. Once UIST is “idle”, check that the telescope is guiding, and then just continue on down the sequence. The sequence will configure UIST for spectroscopy and pause. If you’re happy with the setup, hit continue again and start taking your spectroscopy data.
- Repeat the above process for the fainter science targets.
Note that you can run Movie with or without Sky-Subtraction; with sky subtraction, the first movie frame will be subtracted from all subsequent frames. The operator will offset the telescope a few arcseconds before you start movie, then after the first exposure move back onto the target. Pick Object in Gaia is used in the same way to centre the source on the slit/IFU.
Imaging Acquisition with Faint Sources
In the table below we list very approximate limiting magnitudes for source acquisition in imaging mode. Sky-subtraction was used with Movie in each case. The magnitudes are of course subject to seeing and transparency. Note also that, although longer exposure times may be used at shorter wavelengths, at thermal wavelengths this time is limited by saturation on the sky. Moreover, since coadds are possible with imaging acquisition, we recommend using, e.g., 5 x 6sec rather than 1x30sec , to avoid latency issues (discussed here).
| Grism | Blocking filter | Point source mag. | Exposure time used for imaging acquisition |
|---|---|---|---|
| IJ | J-spectral | J~16.5 | 5x6sec |
| JH | J-spectral | J~16.5 | 5x6sec |
| HK | B2 (cgs4) | K~17.0 | 5x6sec |
| KL | B2 (cgs4) | K~17.0 | 5x6sec |
| short_J | J-spectral | J~17.0 | 5x6sec |
| long_J | J-spectral | J~17.0 | 5x6sec |
| short_H | H-spectral | H~17.8 | 5x6sec |
| long_H | H-spectral | H~17.8 | 5x6sec |
| short_K | B2 (cgs4) | K~17.0 | 5x6sec |
| long_K | B2 (cgs4) | K~17.0 | 5x6sec |
| short_L | 3.4nbL | L~8.9 | 2x5sec |
| long_L | BrA | L~8.5 | 1x2sec |
| M | BrA | L~8.5 | 1x2sec |
Acquisition modes that use the same blocking filters should have the same limiting magnitude (these use the same exposure time).
Orientation during imaging acquisition
Because the raw frames from UIST on the Movie display are orientated with N-left and E-up, and because non-zero position angles are often used for long-slit and IFU spectroscopy, imaging acquisition can be confusing. However, clicking on the “re-orientation button” in Gaia may help (note that the acquisition process will still work with the frame re-orientated)…
The diagrams below show the orientation for a number of different position angles, with and without the array reorientated. As a general rule of thumb, with the images “flipped”, N is up and E is left for a position angle of -90 degrees.
