Introduction

This document defines the UKIRT operational model for WFCAM observing periods. Some information is included on how PATT, UH, and Japanese programs are incorporated, but the main subject is UKIDSS completion and monitoring. 

Statements in italics identify software or operational areas where assumptions may be made, which would benefit from further discussion.

Definitions

UKIDSS comprises five major surveys, some of which themselves have significant sub-surveys that are well-separated on the sky. We employ the following definitions of the various components of the UKIDSS survey.

1. Survey – One entire UKIDSS survey, e.g. the LAS. 
2. Project – One part of a survey program (for example, one DXS field or one RA slice of the LAS). Defined mainly for operational and scheduling convenience as described later, and roughly corresponding to a PATT programme in the way it is treated by the database and supported by UKIRT scientists

Timescales

Monitoring of progress is done on three different timescales. 

1. Day-to-Day, including adjustments made in the course of an observing night. The summit observer is responsible for in-night adjustments (such as flagging Observations and MSBs as bad); the UKIRT scheduler and support astronomers are responsible for adjustments to the queue made on the basis of the night report from the previous night.
2. Short-term, defined roughly as the time taken to obtain data, ship to CASU, receive back data quality information (as required to implement repeats of given MSBs) and for the data to arrive in the science archive. Of order three weeks to one month.
3. Long-term, defined as equal to or longer than the time for significant progress to be made on a given survey (greater than one month). 

OMP Database Structure

Surveys are divided into projects consistent with a monitoring timescale of order one WFCAM block. Each project is given an individual “project ID” – for example, DXS sub-fields entitled u/ukidss/dxs1, u/ukidss/dxs2 etc. Bigger surveys such as the LAS are divided up into N-S strips, each of which is effectively a project and which is also allocated a separate project ID (u/ukidss/las1, 2, etc.). Uploading projects into the OMP database is done on a flexible timescale; all projects for a 6-month period may be uploaded, but the local survey manager may keep most of them disabled until they become relevant. In part, this is to keep the OMP database size manageable and database query time to a reasonable level; in part, to allow for easy monitoring of completion on the short-term timescale. Table 1 shows typical parameters of the Projects.

Survey	Subsurvey Component	Example Component ID	Upload or Enable/Disable Frequency
LAS	One N-S Strip	U/ukidss/las1	Monthly, approximately two hours of RA range
GPS	One Dec/RA Strip	U/ukidss/gps1	Monthly, as for LAS
DXS	One DXS field (or its two-year plan subset), with a repeat count	U/ukidss/dxs1	One or two fields at a time
GCS	One cluster	U/ukidss/gcs1	One or more clusters at a time, RA depending
UDS	UDS, with a repeat count	U/ukidss/uds	Whole survey always present in the OMP database count

Each survey group produces its MSBs in advance, and the uploads will be done from the UK, on agreed timescales. Uploads will be the responsibility of the survey head or their delegate.

Observing Blocks

The semester is divided into blocks, with UKIDSS time interspersed with approximately one-week blocks of PATT time. PATT time is typically staffed by one program’s PI per block. In the event that no UKIDSS MSBs appear in a given set of observing conditions during a UKIDSS run, the summit observer fails over to PATT-sponsored programs (and vice versa).

UH and Japanese time is similarly blocked and staffed by UH and Japanese observers. In these cases, there is no requirement to undertake either PATT or UKIDSS observing.

Selection of MSBs at the Summit

The UKIRT Query Tool is used to select MSBs to execute. Therefore no MSB is ever commenced in inappropriate conditions (i.e., not matched to the MSB’s embedded requirements on photometric quality and seeing). 

UKIDSS may specify lower seeing limits for surveys if required, and at the summit, it is also possible to employ an hour-angle limit. The individual Survey Heads set internal priorities according to their needs (e.g., top-down priorities in the case of a need for continuity). Surveys operate partly manually, partly automatic, with some input from the local survey manager and program support astronomers. For example, given the large number of MSBs in some of the surveys, occasional direct intervention on the part of the survey manager is needed to instruct summit observers to prefer some particular survey at a given time. Such forcing is catered for in the Query Tool and OMP database.

The summit pipeline continually produces indexed information on observing conditions, including delivered seeing and photometric quality. This is used to set MSB constraints in the query tool and effectively filter the database for appropriate MSBs. 

Quality Control Information and MSB Repeats

Wherever possible, MSBs determined to have produced substandard data are repeated within the WFCAM block in which they were obtained. On the shorter timescales, repeat requirements are determined in two ways: 

(i) Observer feedback entries and summit-pipeline DQ information are used to schedule repeats quickly. This is possible on the day-to-day schedule and can be accommodated entirely in-house, requiring no interaction with UKIDSS, WFCAU, or CASU. In this case, the relevant OMP database feedback entries and good/bad flags are set such that CASU will know in advance of processing that the data being compromised and not to be propagated into the science archive. 

(ii) DQ feedback from CASU will be used to schedule repeats of MSBs on the short-term timescale. In this case, it has been agreed that there will need to be a feedback mechanism between CASU and the UKIRT scheduler, which will probably need to be no more complex than a weekly email unambiguously identifying the affected MSBs. The affected MSBs will then be flagged as available once more. 

This remains in planning. A unique MSB instance identifier (the MSB transaction ID) is being developed. Similarly, we will need a small utility to reenable a range of MSBs using an ID list sent from CASU. 

Survey Monitoring

On the short-term timescale, an overview of progress is the UKIRT survey manager’s responsibility, with the Head of Operations standing in for absence or illness. Duties include ensuring that the expected progress rate is maintained in each of the surveys (and its components, if relevant). Particular requirements and day-to-day monitoring of the data quality from individual surveys are the responsibility of the associated support scientist (see Annex 1). The main tools in this are queries to the OMP database.

On the long-term timescale, the surveys are monitored by UKIDSS survey heads and their delegates, using the science archive facilities. 

Observer Instructions

Various issues relating to observer execution of UKIDSS projects are covered in the UKIDSS Observer Notes pages.

ANNEX 1: Survey Support

Each survey is supported by a primary and secondary UKIRT support scientist, provisionally as follows:

GPS – C Davis, M Rawlings

GCS – M Rawlings, W Varricatt

DXS – T Kerr, A Adamson

LAS – W Varricatt, C Davis

UDS – A Adamson, T Kerr

The primary support scientist has the following responsibilities for their survey: 

1. Supporting UKIDSS in generating the MSBs
2. MSB vetting (in as much as this is possible)
3. Monitoring pipeline output and DQ parameters the day after a night’s observing. It is not expected that this item requires much more than checking that data were taken in appropriate conditions (i.e., the pipeline data quality output was adhered to by the observer). In practice, the support scientist for each UKIDSS run takes more of a role in this.
4. Feedback to the PI (UKIDSS survey chair or their delegate)
5. Detailed interaction with the UKIRT survey manager where necessary
6. Training visiting UKIDSS observers
7. Observing. With its current level of support, UKIRT accounts for some 30% of the survey observing.

The secondary support scientist stands in for the primary in the event of absence or illness, etc.

ANNEX 2: Communications Between JAC and UKIDSS

All regular communication between JAC and UKIDSS must be done through the OMP feedback system. For each survey, UKIDSS nominates “P.I.s and Co-I.s” to whom each nights’ worth of feedback entries are automatically emailed. These include the CSV, the Survey head, and the UKIDSS Survey Scientist. 

For information, the figure below shows an example of a project feedback page – in this case, the Co-Is are identified as not receiving every email the program generates. There are two support astronomers – primary and secondary – and this is also the case for the UKIDSS surveys. For each UKIDSS survey, there is one such feedback page.

Notes: 

1. In this example, only the PI and support scientists receive project email (the PI can alter this at any time);
2. Any number of Co-Is can be defined; 
3. Data can be “retrieved” via the night links at the bottom of the page (though, of course, this cannot be used for the WFCAM data rates, the “eavesdrop” link may grant access to reduced-data “thumbnails” as there are for the other UKIRT instruments).

Nightly DQC parameters are also plotted on the night report and eavesdropping pages.

On the short-term timescale, plots are generated showing observation completion of sub-survey components. These are needed to decide whether to carry over unobserved parts of a sub-survey component into the next block, etc. This information will be made available on a separate UKIDSS-related web page, maintained by the UKIRT survey manager and Associate Director.

ANNEX 3: Communications Between JAC and UK Data Centres

Data are shipped to CASU on Ultrium LTO-II tapes, using a recognized express carrier. The data format has been separately agreed upon between JAC and CASU.

The JAC makes available all quality control and feedback information resulting from each night of observing to CASU. CASU is free to specify the database fields they wish to copy. 

CASU will feed back repeat observation requests resulting from analysis of full nights and longer-term trends in the form of a weekly email. 

It is not expected that there will be any direct feedback mechanism from the WFAU science archive and the Hawaii operation; on the long-term timescale, UKIDSS will use the science archive to make decisions about the global properties of the surveys and their requirements.