WORKING WITH WFCAM DATA AND THE WFCAM SCIENCE ARCHIVE

Lead Author: Chris Davis
WFCAM data are reduced by the Cambridge Astronomical Survey Unit (CASU) and distributed by the WFCAM Science Archive (WSA).
This cookbook describes:
- Data access (see also the Accessing UKIDSS or Accessing Flexed Data links on the sidebar),
- The processes involved in generating complete tiles from the “pawprints” (or “Multiframes”) in the WSA,
- How to make use of the Multi-Extension FITs catalogues supplied with each Multiframe, and
- How to get started with SQL searches of the source tables hosted by WSA (i.e. how to get photometry data without downloading the actual images!). This includes a very brief introduction to SQL and the Starlink table viewer/editor TOPCAT.
Commands are mostly from the Starlink software collection, which is available from http://starlink.eao.hawaii.edu/starlink.
You may need to update your Starlink installation for some of the commands described below to work properly!
- For notes on KAPPA, type showme sun95 or go to http://docs.eao.hawaii.edu/star/sun95.htx/sun95.html
- For notes on CCDPack, type showme sun139 or go to http://docs.eao.hawaii.edu/star/sun139.htx/sun139.html
- For notes on GAIA, type showme sun214 or go to http://docs.eao.hawaii.edu/star/sun214.htx/sun214.html
For details of the reduction process leading up to the pawprint/Multiframe files themselves, see the very useful technical notes section at CASU: http://casu.ast.cam.ac.uk/surveys-projects/wfcam/technical
The WFCAM Science Archive also host a Frequently asked Questions webpage.
Finally, if you are still having problems (having read the notes below), please contact your UKIRT support astronomer (if you have one) or the WSA helpdesk (wsa-support@roe.ac.uk).
1. Access to Survey and non-Survey Data from WSA
Registering
UKIDSS data are made available via “Data Releases”. Immediate access is available to all ESO astronomers, though they do need to be registered with the archive. Public data – observations in the World Data release – are available to everyone. Users do not need a username/password to access these data (see the notes below on “having a quick go…”) and therefore they do not need to register.
PI-led projects (PATT, Service, UH, etc.) must be REGISTERED, by the PI, with WSA. THIS DOES NOT HAPPEN AUTOMATICALLY! Projects may be registered at any time, though the end of the semester (or shortly after the last data have been acquired) is a good time to do this.
To register your PATT, Service, Japanese or UH programme, click on the nonSurvey link on the WSA sidebar and fill in the registration box. IMPORTANT: If you observe the same field in multiple filters, select the filters in the last row of the form. If you don’t, a merged source catalogue won’t be created (see below). When your data have been ingested in the archive you will be sent an email informing you that data are available. Login at WSA and try the “Archive Listing” link on the WSA sidebar to see what’s available… The WSA “Start here” link is a good starting-point for first-time users!
File-naming convention
The WSA archive in Edinburgh supplies fully-reduced images in RICE-compressed Multi-Extension Fits (MEF) files. Data in one filter for a 0.8×0.8 degree tile would appear as four “stack” or “leavstack” files in the database, one for each of the four offsets on sky needed to cover the tile. Microstepped observations result in leavstack data; if the observations were not microstepped, you’ll just get stack data.
The MEF is probably of the form: w{date}_{num}_sf_st.fit
For nonSurvey data, initially only Flat File access may be available. In this case images are fully reduced and photometry (FITS) catalogues are made available – one for each MEF (see below). However, if a region has been observed in multiple filters, then merged tables (Source tables), where multi-band photometry for each source are collated and presented in the same row, are only made available when “full” data access is attained. Moreover, the FITS catalogues available with “flat file access” don’t include columns of magnitudes, although integrated fluxes (aperture photometry) are provided for each source, along with instrumental magnitudes (zeropoints) and other calibration coefficients.
A full description of the data products available in the archive is given at the WSA website.
Data processing steps are described in a series of papers (note in particular Hambly et al. (2007) and Irwin et al. (2008)). See also the technical notes at CASU: http://casu.ast.cam.ac.uk/surveys-projects/wfcam/technical
The WFCAM Archive – having a quick go…
Some of the UKIDSS data taken for the five surveys are now world-accessible from the WSA archive in Edinburgh. This means that you don’t need to register or login – you can go straight to the data querying tools listed on the WSA side-bar:
The regions covered by the latest release are shown on the surveys page. Note that the date given for each data release pertains to ESO releases; world data releases will be approximately 18 months later. For example, DR1 (and DR1plus) had their world data release in January 2008.
- Archive Listing on the WSA side-bar is perhaps the quickest way to access large volumes of data. If you enter a range in RA/DEC and select a filter you should get a list of observations (the frame type should be set to “stack” – see above). Try for example just changing “filter” to H2 and leaving everything else as default. Submit should give a list of observations; if you click on “view” for one of the observations you will get 4 GIF images showing the reduced data. The FITS Img and Cat files listed in the same table are Multi-Extension Fits (MEF) images and Multi-Extension Fits catalogues – basically four images or four tables per file, again as described earlier.
One thing to note is that the RA and Dec listed in the “archive listing” table are actually located midway between the four images. The WFCAM arrays are separated on sky (as described ) so you don’t actually get data at the coordinate listed in the table! - Get Image is another useful tool, which allows you to get a small snap-shot image around a target coordinate. This time, data are returned for the coordinate you enter. For example, if you enter 5:35:27,-5:4:00 as coords and submit you’ll get a list of reduced images (one per filter) which you can view as GIFs or download as FITs images. Setting X-size and Y-size to 10 arcmin gives a nice view of Haro 5a/6a (or there-abouts).
- Region is used in a similar way to “Get image”, although this time it returns a source table, with multi-band photometry listed in various apertures. Again, if you enter 5:35:27,-5:4:00 (and select “UKIDSS Galactic Clusters Survey” and “Source table/merged catalogue”) in the Programme/Survey and Table pull-down menus, you’ll get a table centred on Haro 5a/6a which can be saved in various formats: HTML – the default – simply displays the table in your browser.
For your favourite source you can set the search radius to 0.05 (3 arcsec), resubmit the query, and get photometry for just that one source. If you click on “view” in the resulting table, you even get a snap-shot image in each of the available filters.
The real power behind the archive is the ability to make SQL queries of the vast catalogues hosted by WSA to look for sources with specific colours, etc. To get you started, see the notes later on in this page…
2. Image Post-Processing
The notes in this section describe one way of combining adjacent WFCAM images into a large mosaic. These employ starlink software routines, although TERAPIX is also popular with WFCAM users. Below we assume that you have already downloaded the MEF image files from WSA.
Extracting from RICE MEFs
A RICE compressed MEF file can be opened directly in Gaia. A pop-up will allow you to choose which image to display.
To extract the four images from the MEF file use fits2ndf and ndfcopy, for example:
> fits2ndf in=”w${date}_${num}_sf_st.fit” out=”temp” container=true > ndfcopy in=’temp.HDU_1′ out=’tempw’ reset > ndfcopy in=’temp.HDU_2′ out=’tempx’ reset > ndfcopy in=’temp.HDU_3′ out=’tempy’ reset > ndfcopy in=’temp.HDU_4′ out=’tempz’ reset |
Alternatively, use imcopy to convert the rice-compressed MEF to an uncompressed MEF; fits2ndf can then be used to extract the individual images, e.g. fits2ndf in=’temp.fit[1]’ out=’w101′ reset
If the four MEF files (obtained for the tile in one filter) are called
w20050101_101_sf_st.fit
w20050101_102_sf_st.fit
w20050101_103_sf_st.fit
w20050101_104_sf_st.fit
then repeating the above steps on the four MEFs would give
w101.sdf, x101.sdf, y101.sdf, z101.sdf
w102.sdf, x102.sdf, y102.sdf, z102.sdf
w103.sdf, x103.sdf, y103.sdf, z103.sdf
w104.sdf, x104.sdf, y104.sdf, z104.sdf
The layout of the four cameras on sky is as follows:
Y | Z |
X | W |
Depending on the order of the 4 offsets needed to make the tile, the layout of the 16 frames on sky might then be something like:
y103 | y104 | z103 | z104 |
y102 | y101 | z102 | z101 |
x103 | x104 | w103 | w104 |
x102 | x101 | w102 | w101 |
If you display each extracted image in GAIA they should have the correct WCS coordinates. The KAPPA commands flip and rotate can be used to correctly orient the images, though this may not be necessary (obviously they can also be reoriented in GAIA).
Creating a mosaic covering a full tile
Note that the 16 extracted files are large, especially if the images were taken with microstepping. Each 2×2 microstepped image is 4k x 4k pixels, or 131 Mb. Consequently, compressing (binning) the data may be desirable before further processing to create the tile mosaic. Use KAPPA/COMPADD, e.g.
> compadd IN – NDF to be compressed /@temp_cube2/ > w101 COMPRESS – Compression factors > 2,2 OUT – NDF after being compressed > w101_bin |
where “compress” is the number of pixels binned/added in each dimension.
To construct a mosaic from the 16 binned NDFs use the WCS info in each file. Registration on stars with, e.g. CCDPACK/PAIRNDF is not necessary. Instead, use KAPPA/WCSALIGN to align the images in the tile, then CCDPACK/MAKEMOS to create the mosaic. e.g.:
> wcsalign “in=*_bin ref=z101_bin lbnd=! acc=0.2 method=bilinear params=[0,2] out=*_wcs reset” > makemos in=’*_wcs’ out=dr21_mosaic method=broad zero reset |
Alternatively, try wcsmosaic, which can be used in place of wcsalign and makemos (type kappa then kaphelp wcsmos for further details).
If you find that the final mosaic is missing frames, repeat the wcsalign and makemos steps using a different reference frame with WCSALIGN. Note – some methods in wcsalign (sincsinc) were leaving bad pixels adjacent to stars. Bi-linear seems to work ok.
Note, even with 2×2 binning of micro-stepped data, a reduced tile will be almost 500 Mbytes in size!
Flat-fielding
With early archival data, additional flat-fielding and sky-subtraction was needed to create a smooth tile. This may be due to gradients across the full tile field (from the Moon, OH emission etc).
Users may find the KAPPA command surfit and the CCDPACK routines makeflat and flatcor useful. For example, a script might include the lines:
foreach d ( w x y z ) foreach n ( 101 102 103 104 ) # stats ${d}${n}_bin clip=3 > /dev/null set s = `parget mean stats` set lo = `calc exp=”($s)-($s)/(5)”` set hi = `calc exp=”($s)+($s)/(5)”` # echo “Mean background level in ${d}${n}_bin is : ” ${s} echo “Low and High values for surface fit are : ” ${lo} ${hi} # surfit “in=${d}${n}_bin out=${d}${n}_sf fittype=spline knots=4 estimator=mode wlim=0.9 thrlo=${lo} thrhi=${hi} reset” end end |
The above will fit a coarse surface to the background in each of the 16 frames. This works well even in crowded fields. The four surface fits in each camera are then used separately to create a flat and flat-field the four images in each corner of the tile:
foreach d ( w x y z ) makeflat “in=${d}*_sf out=flat_${d} method=broad clean=f reset” # creates a flat for the four images in each corner of the tile # foreach n ( ${a[1]} ${a[2]} ${a[3]} ${a[4]} ) flatcor in=${d}${n}_bin out=${d}${n}_ff flat=flat_${d} reset # flat fields data from each camera/corner of the tile end end |
An alternative – or additional – step might be to simply subtract the surface fit from each image, effectively sky-subtracting (rather than flat-fielding) the data.
Finally – a colour image!
To show off your data, create a colour image from tiles created from observations in three filters using KAPPA/COLCOMP. Again use WCSALIGN to register the three images first.
> wcsalign “in=Jmos,Hmos,Kmos ref=Jmos lbnd=! out=*_tran reset” > colcomp inr=Jmos_tran ing=Hmos_tran inb=Kmos_tran rhigh=300 rlow=30 ghigh=300 glow=30 bhigh=300 blow=30 out=\! ppm=colour-pic.ppm percentiles=\! > xv colour-pic.ppm & |
It’s a good idea to establish the high and low values for the three channels before attempting this, since again it can be a slow process.

Colour images direct from the WSA
The WSA now produces colour images from archived pixel data (though note that it does not mosaic together images to form large tile-sized mosaics, so colour image sizes are limited). Use the ColourImage link on the side-bar to create colour pictures from GPS, GCS or LAS data, or from your own project images. You must first login to access these data, and you may want to do some research first; if the field hasn’t been observed in multiple filters the web-page will obviously not be able to perform the task, and you’ll simply get “No multiframes found in the area requested”.
As an example, login as a UKIDSS user and try the following coords: 20:38:35 (2000) 42:37:30 with the survey set to GPS and filters K_1, H and J (multiple-epochs in a particular filter will be listed as K_1, K_2, etc.). These should give a nice colour picture of the massive star forming region W75N.
3. Catalogues
File naming convention
WSA supply one catalogue file with each stack or leavstack frame. For example:
w20060702_00204_sf_st.fit (the MEF containing 4 images) w20060702_00204_sf_st_cat.fits (the MEF containing 4 catalogues)
Like the images, the catalogues are stored in multi-extension FITS files as FITS binary tables; the catalogue fits file contains four tables, one for each array.
Inspection & Concatenation
For a quick look, you should be able to display the MEF image file and the associated MEF catalogue in GAIA.
Open the MEF catalogue first, e.g.:
> gaiadisp w20060702_00204_sf_st_cat.fits & |
In the pop-up window, highlight and “open” the table you’re interested in (this may take a few moments). Now, from the main GAIA file pull-down menu open the image MEF fits file (in this case w20060702_00204_sf_st.fit), and select the associated image from the list of four. Finally, “plot” (or “search” and “plot”) in the table pop-up window should mark the sources on the image.
Alternatively, you can extract the four images from the MEF as described above, and similarly extract the four table files using the STILTS tcopy command:
> stilts tcopy w20060702_00204_sf_st_cat.fits#1 w204cat.fits > stilts tcopy w20060702_00204_sf_st_cat.fits#2 x204cat.fits > stilts tcopy w20060702_00204_sf_st_cat.fits#3 y204cat.fits > stilts tcopy w20060702_00204_sf_st_cat.fits#4 z204cat.fits |
NB: if you put the above into a c-shell script, remember to put ” ” around the #1, #2, etc. in your script; otherwise c-shell will think everything after the # is a comment!
Open the image in GAIA; then open the catalogue file with “Data Servers – Local catalogues – Load from file”. The sources should automatically be marked on the image, and the catalogue displayed.
Individual tables in a catalogue can also be opened in TOPCAT (note that if you omit the #3 below, only the first of the four tables in the multi-extension fits catalogue file will be opened by TOPCAT):
> topcat w20060702_00204_sf_st_cat.fits#3 & |
If you have combined the 16 images to create a WFCAM tile, you may wish to do this to the 16 tables in the four MEF fits catalogues that cover the same tile. This can be done with a simple c-shell script using STILTS and the tcopy and tcatn commands. For example, write the four tables from each MEF catalogue to separate files as described above, then concatenate all sixteen tables with, e.g.
> stilts tcatn nin=16 \ in1=w1cat.fits in2=w2cat.fits in3=w3cat.fits in4=w4cat.fits \ in5=x1cat.fits in6=x2cat.fits in7=x3cat.fits in8=x4cat.fits \ in9=y1cat.fits in10=y2cat.fits in11=y3cat.fits in12=y4cat.fits \ in13=z1cat.fits in14=z2cat.fits in15=z3cat.fits in16=z4cat.fits \ out=combined.fits |
You should now be able to display a full tile image in GAIA, and over-plot (with “Data Servers – Local catalogues – Load from file”) a full tile catalogue. However, note that the above does not deal with sources in the overlap regions, which will appear in multiple MEF tables. Also, zero-points will change from camera to camera; you may thus be better combining only tables for each camera, i.e. for each corner of the tile.
Either way, you should be able to open the combined catalogue in TOPCAT for further processing (see below).
Photometry Calibration with MEF Catalogues
As noted above, Flat-File access provides reduced MEF images and MEF catalogues; a Full data release also provides merged tables, i.e. JHK mags for the same source in one table, that are hosted by WSA and searchable using SQL (see below). However, MEF catalogues don’t include source magnitudes, only fluxes in counts measured in various apertures. The image file headers do include zeropoints (and the number of standards used to get these), e.g.:
- MAGZPT = 23.98 / Photometric ZP (mags) at airmass of 1
- NIGHTZPT= 23.0 / Average photometric ZP (mags) for night
- NUMZPT = 1317 / Number of standards used
When working with the MEF catalogues, users should use MAGZPT to convert aperture photometry values into magnitudes. This can be done in TOPCAT by opening the MEF catalogue and creating a new column (discussed further below) using the appropriate math:
Mag = ZP - 2.5*log10(flux/exptime) - extin*(airmass-1) - apcor - percorr
For the integrated flux of point sources, Aper_flux_3 is recommended. Note that flux values must be scaled by the exposure time; the data should also be corrected for extinction using the airmass of the observations; an aperture correction (apcor) and sky calibration (percorr) may also be applied, although the latter is often very small or zero.
The parameters needed to convert aperture fluxes to magnitudes are stored in each FITS catalogue header. To view these parameters (MAGZPT, EXP_TIME, EXTINCT, AMSTART/AMEND, APCOR* and PERCOR), open the catalogue in Gaia: in the main Gaia window click on “View – Fits header”. The resulting pop-up will contain five tabs; those labelled 2,3,4,5 contain the headers for each of the four cameras. Note that these can be listed alphabetically…
For a more detailed description of these parameters see this document from CASU, or consult the notes on catalogue generation. The WFCAM photometry system is described here. The archive also provide a nice cookbook on working with WFCAM flat files and catalogues.
Excluding saturated sources
The simplest way to flag sources as saturated is to make use of the “Peak_height” column and “Sky_level” column information in the MEF catalogue, and compare these with the saturation value SATURATE in the catalogue header. Objects may be flagged as being saturated if they reach 90% of the saturation level with respect to the sky; i.e. if peak height > 0.9*(saturate-skylevel).
TOPCAT is an interactive graphical viewer and editor for tabular data. Both TOPCAT and STILTS require a Starlink installation (although strictly speaking catalogue analysis alone doesn’t require the full Starlink suite – see the TOPCAT webpage below). Non-Starlink tools include HEASARC’s fv, and ESO’s Midas package seems to handle the FITS MEFS also.
Further info on TOPCAT (including software download) is available at http://www.starlink.ac.uk/topcat/
Further info on STILTS is available here: http://www.starlink.ac.uk/stilts/
4. A SESSION WITH THE WSA ARCHIVE – SQL and TOPCAT
Note that if only Flat-File access is available, SQL querying of your (PATT, UH, Service or Japanese) data is not possible. This is because there is no release database. SQL queries for PI-led projects can only be executed when “full” data access is available.
The WFCAM Science Archive (WSA) can be used to download tables of photometry, either from one of the UKIDSS surveys, or from a specific PATT or UKIRT/Service programme. Here we provide an example session based on data taken in a service programme; a session querying UKIDSS data would be very similar, except that obviously the user must login as a UKIDSS user, and search a different source catalogue, probably using somewhat different parameters (see below). Login is not necessary when querying public data from a World Data release.
The WFCAM Science Archive (WSA) – Downloading Source/Photometry Tables
- Registering: Necessary for non-survey data (PATT, service, UH or Japanese programmes) – see above.
- Accessing data: When non-survey data become available in the archive, WSA will email you with a password. (UKIDSS data releases are announced on the UKIRT and UKIDSS homepages.) Return to the WSA website and login: for PATT or service data, the Community is nonSurvey; for UKIDSS data the Community will probably be derived from your email address, e.g. jach.hawaii.edu.
- SQL query: Go straight to the Freeform SQL interface (link on the sidebar). First-time users should check out the “notes and tips” link, or the “Cookbook”. SQL examples are also given in the appendix of Phil Lucas’ GPS survey paper (MNRAS, 2008). An example query is given here: select sourceID, ra, dec, mergedClass, pStar, jAperMag3, jAperMag3Err, hAperMag3, hAperMag3Err, kAperMag3, kAperMag3Err, h2AperMag3, h2AperMag3Err, jppErrBits, hppErrBits, kppErrBits from userv1635Source where pstar >0.990 /* Exclude artifacts */ and mergedClass !=0 /* Exclude multiple detections of same source */ and (PriOrSec=0 or PriOrSec=framesetID) This query searches the userv1635Source source catalogue created for service programme u/serv/1635. The source catalogue associated with the GPS, for example, is called gpsSource. Hopefully, most of the above is self-explanatory (note: JHK and narrow-band H2 data were obtained for this particular project). pStar and mergedClass define the likelihood that a source in the catalogue is a star; the text within “/*” and “*/” is a comment; PriOrSec is used to exclude duplicate sources in overlap regions between frames; the ErrBits parameters are used to identify cross-talk artifacts, bad pixels and/or saturated (or non-linear/near-saturation) sources. (pNoise and pSaturated are two other useful parameters.)
- Schema Browser, or “what on earth are hppErrBits?” To establish which additional parameters are available for query and/or download, and to get a description of all catalogue entries, click on Schema Browser on the side bar. Once there, select WSA nonSurvey from the new sidebar and click on your project.
- Submitting a query: When you are happy with the SQL query, select VOTable file as the Data Format and Submit. WSA may initially claim “zero rows returned”; be patient as it continues the search. You should eventually see the first 30 lines of the table displayed in your browser.
- Downloading a data table: Finally, save the table to your local disk as VOTable ascii. You may have to right-click on the link to avoid your browser displaying the text in the browser window (rather than saving to a file). The VOTable ascii table should be readable in TOPCAT (see below) and GAIA.
VOTable Photometry Tables and TOPCAT
Below we give a few tips on displaying WSA photometry in TOPCAT, Starlink’s very nifty interactive graphical viewer and editor for tabular data.
- Getting started: Having installed TOPCAT (available here), run TOPCAT by simply typing topcat on the command line, and Open the VOTable ascii file. The file name will be displayed top-left in the main TOPCAT window.
- Displaying table contents: Click on Display Table Cell Data (4th icon – main window, or just double-click on the table in the “Table List”) to see the contents of the table.
- Creating new/deleting obsolete columns: Right-click on a column in the table browser to define a “new synthetic column”. Fill in the “name” (the column heading) and “expression” used to define the new data (the rest are optional). For example, a column listing H-K/J-H values could be defined as hAperMag3-kApermag3/jApermag3-hAperMag3. Note that column headings should not include mathematical characters (these will mess up subset selection – described below), so instead of H-K/J-H, the column heading should be something like HmKoverJmH.
- Choosing subsets: TOPCAT allows you to select a portion of the tabulated data using a parameter string. Different criteria can be used to select different “subsets” of the data. Multiple subsets can then be displayed on the same line or scatter plot. For example, blue and red stars could be selected and plotted with different symbols on the same Colour-Colour or Colour-Magnitude diagram. To select a subset of the opened table, click on Display Row subset (7th icon – main window); click on the green “+” in the new “Row subsets” window and enter a subset “name” and “expression”. E.g. a subset called All Stars in L1448 could have an expression: equals(region,”L1448″) && pstar > 0.995 Or you could select a subset based on RA or Dec range. Note that if RA and Dec are strings (e.g. 4:20:30.5 -5:30:20), then this should work: radiansToDegrees(dmsToRadians(Dec)) >= -5.5 A second, tighter subset could then be selected by again clicking on the “+” button and entering: equals(region,”L1448″) && pStar > 0.995 && pNoise < 0.003 && jAperMag3Err<0.1 && hAperMag3Err<0.1 && kAperMag3Err<0.1 && jppErrBits == 0 && hppErrBits == 0 && kppErrBits == 0 The above criteria (all typed in on one long line – stretch the window and/or scale the column widths!) should select only data with decent point-source photometry. Limiting the maximum and minimum magnitudes of sources (i.e. targets too faint to be believed, or too bright to be in the linear regime of the array) may be another way to do this, e.g.: equals(region,”L1448″) && pStar > 0.995 && pNoise < 0.003 && jAperMag3Err<0.1 && hAperMag3Err<0.1 && kAperMag3Err<0.1 && kAperMag3>10.5 && jAperMag3>10.5 && jAperMag3<21.0 Sources within a given RA and DEC range can also be selected, e.g.: equals(region,”L1448″) && pStar > 0.995 && pNoise < 0.003 && jAperMag3Err<0.1 && hAperMag3Err<0.1 && kAperMag3Err<0.1 && kAperMag3>10.5 && jAperMag3>10.5 && jAperMag3<21.0 && ra < 309.855 && ra > 309.655 && dec < 42.43 && dec > 42.255 Note that RA is tabulated in decimal degrees (360 degs = 24 hrs in RA). Finally, a subset of just the red sources, with H-K/J-H > 0.7 could be selected: equals(region,”L1448″) && pStar > 0.995 && pNoise < 0.003 && jAperMag3Err<0.1 && hAperMag3Err<0.1 && kAperMag3Err<0.1 && kAperMag3>10.5 && jAperMag3>10.5 && jAperMag3<21.0 && HmKoverJmH > 0.7 TIP: to edit a parameter in an existing subset selection expression, double-click on the expression in the “Row subsets” window; you should then be able to edit it. Rather than type in longer and longer subset strings, a subset can be saved to a new, smaller table. The user may then simply work with this smaller table (see below). Alternatively, it is possible (I think) to select a subset of a subset: if you highlight a subset sample in the “Row Subsets” window, then define a new subset, the next subset will in fact be a “sub-sub-set”, that is, a subset of the highlighted subset!
- Plotting data (Click on the scatter plot below to see an example session): Any of the above subsets can be plotted on the same axes. With the original table highlighted in the “Table List” in the main TOPCAT window, click on the Scatter plot icon in the main TOPCAT window. In the Scatter Plot window, the X and Y-axes to be plotted can be selected bottom-left, and the “Row Subsets” to be displayed can be chosen bottom-tight. The symbol size/colour can be altered (by clicking on the symbol key bottom-right), as can the labelling of the key. The axes scales and labels can be set from the Scatter-Plot window panel (third icon), and GIF and EPS versions of plotted figures saved to disk. Lastly, if a second table is opened in the main TOPCAT window, these new data can be included on the same plot. In the Scatter-Plot window, click on the add a new dataset icon (just below the graph) to include these new data. In the example below, a subset of “all stars” are plotted in blue; a “red stars” subset is over-plotted in red, and data from a second and third table – that define the Main Sequence dwarf and giant branches – have been drawn as black lines.

- Saving subsets as new (smaller) tables: Once you’ve gone to the trouble of creating new columns, deleting obsolete columns, and establishing which sources are, e.g. red, you can save just this subset of data to a new table. In the main TOPCAT window, click on Save Table (second icon). Select the “Row subset” to be saved – so in our example this would be the “Red Stars” – the “order” in which rows are to be sorted, and set the output format to “VOTable table data”, “FITS” or “ASCII”.
- Over-plotting sources onto a WFCAM image: New tables can be re-opened in TOPCAT at a later date. Alternatively, open the FITS or ASCII files in GAIA. GAIA should easily handle the RA and DEC parameters, which are both stored in decimal degrees (you may need GAIA version 4.0 or above). The FITS table can be opened under “Data servers -> Local catalogues”. The ASCII table must be imported as a text file: “Image analysis – > position -> Import plain text”. (The ascii file is delimited with spaces; I had more luck with this file when over-plotting the positions onto a WFCAM image than with the FITS file…)
Miscellaneous WSA Tips
Photometry for just a few targets: can be obtained using the “Region” link. Log on to your PATT project, or as a UKIDSS user (described above), and go straight to “Region”. Enter the RA,Dec of your target and – for individual sources – set the Search radius to 0.03 (2 arcsec). Select the catalogue to be queried and submit.
IMPORTANT – If you suspect that a source is only detected in one or two bands, select the “detection table” rather than the “merged table”. This will return separate rows for each filter. Filters are identified by number: 3 is J, 4 is H, 5 is K, 6 is H2, etc. If for example a target was only detected in K, only one row will be returned. You can also examine the data (a gif cut-out) by clicking on View in the returned table.
5. SCRIPTS
Many of the above commands are combined into runnable scripts, available here. (please note – these are not maintained, so they may need a bit of tweaking – depends when I last used them!)
These are:
1. Extract images from MEFs
2. Bin images
3. Create mosaics
Here, the first two scripts mosaic all frames in a look-up file, mos_list.txt. The basic version does no fitting (sky-subtraction) – just the mosaicking. The third is the “standard” script for reducing a tile of 16 frames; the “corner” version just forms a mosaic from one camera (so one corner of a tile).
4. Clean images (after patching in GAIA, perhaps)
5. Register, scale and subtract images
6. Concatenate MEF tables
7. Miscellaneous
- surface_fit_wfcam.prg (fits and removes a coarse surface to background regions)
6. MORE INFORMATION
- CASU have produced a web page describing how to get sensible colour-magnitude and colour-colour diagrams from the archive: casu.ast.cam.ac.uk/surveys-projects/wfcam/science/2005a-science-data-examples