The imaging catalogs

Introduction

The calibrated object lists reports positions, fluxes, and shapes of all objects detected at >5 sigma on the survey images. Here we describe the basics of obtaining and searching the catalogs, and the metadata associated with each field and run. This page describes how to use the imaging catalogs. For more detail on how the catalogs were created, see the Algorithms pages.

To understand the imaging data, you first need an understanding of the basic imaging data information. Below we refer to the algorithms pages for details on the calibration and the photometric parameters. In general, you will obtain the best results only after carefully reading that documentation. In particular, you need to look at the object flags in the object lists to obtain meaningful results.

photoObj table and flat files

Calibrated object lists are stored both as flat files and as the photoObj table CAS. These two sets of data are nearly identical, with only some minor naming convention differences due to historical artifacts, and a small number of columns that exist in one but not the other. This data has photometric and astrometric calibrations applied, and contains enough information to select unique objects and to perform quality cuts.

For many purposes the best source of the object lists is the photoObj table in CAS, which allows for SQL searches across the entire dataset. The primary key of this table is objID, which can be used to join with the external catalog tables (e.g. TwoMass, TwoMassXSC, First, ROSAT, RC3, and USNOB), the proper motions tables (properMotions), and the spectroscopic catalogs (specObjAll, sppParams, sppLines, etc).

For example, the follow query will return some basic magnitudes of a handful of stars:

  SELECT top 50 run, camcol, field, obj, ra, dec, psfMag_u, psfMag_g, psfMag_r
  FROM photoObj
  WHERE type = 6

In the flat files on SAS, the photoObj*.fits files directly correspond to the photoObj table in CAS, and contain all the detected objects. There is one file per field. The files are binary fits tables with one row per object. The total data set is about 3.7 TB, so downloading the entire set will not be practical for most users; for many purposes the sweeps files will be more sensible. The external catalogs are stored in separate files, in a parallel set of directories.

The photoop IDL package distributed with the SDSS software contains code to read in the photoObj files, and is designed to read in the external catalogs together with the photoObj files as desired. For example, to read in the objects in run 745, camcol 4, field 300, and the matches to USNO-B and the 2MASS point source catalog:

  pobj= photoobj_read(745, 4, 300, rerun=301, extcats=['2MASS', 'USNOB'])

In both the flat files and in CAS, detections corresponding to the same source on the sky are included in the table. One must use the techniques described in the resolve documentation to select unique objects across the survey (or within a single run). Note that the photoPrimary table in CAS is a "view" of photoObj which only includes unique objects.

Additionally, some objects have been flagged by the photometric software as untrustworthy (e.g. including saturated pixels, or close to a field edge, or badly deblended). Please see the discussion of these flags object flags for a full description of the diagnostic flags.

A number of estimators of the flux, the classification, and shape parameters of the objects are included in the files, described in the magnitude and classification algorithms pages. The algorithms pages also describe the ubercalibration and other details of the photometric pipeline.

Notes

In DR6 and previous, the flat files corresponding to photoObj files were known as "tsObj" files; these are now obsolete. In DR7, these flat files had a different name, "drObj". Because of the changing format, we additionally changed the name.
The primary header of photoObj files contains some parameters applying to the entire field, such as the seeing (but also see the PSF parameters in the photoField file below).
The reported magnitudes are calibrated but not Galactic extinction corrected. The Galactic extinction for each object as derived from Schlegel, Finkbeiner, and Davis (1998) dust maps is reported as extinction.

Datasweep Files

The calibObj, AKA "datasweep", files contain a subset of the data in the photoObj files. The datasweeps are designed for those who want to work with essentially every well measured object, but only need the most commonly used parameters. This use case is practical with the datasweep files which total about 300GB of data.

Only objects with well determined flux in at least one band are included. For galaxies, the limit is set at an extinction corrected model magnitude less than [21.0,22.0,22.0,20.5,20.1] in u,g,r,i,z respectively. For stars, the limit is set at an extinction-corrected PSF magnitude less than [22.5,22.5,22.5,22.0,21.5] in u,g,r,i,z. In addition, only a subset of the measured parameters are included, such as positions, magnitudes, shapes, sizes, processing flags, etc.

Where there is overlap between multiple images, objects are matched and statistics about the ensemble flux distribution are stored. For model, cmodel, psf flux types, there are additional fields for the "clean" averaged flux as well as inverse variance in the mean, overall variance, and chi squared of the flux. There are additionally the variance in MJD and maximum difference in MJD over all observations. The "clean" detections are defined as objects in images marked photometric, inverse flux variance greater than zero, and all of the same type classification in the data sweeps (e.g. star or galaxy).

As opposed to the photoObj files, there is one calibObj file per camcol. There is also a separate file for each of the primary types: "star", "gal", and "sky". Each of these files contains a FITS binary table. See the calibObj description for full details.

The calibObj files can be read with any FITS binary table reader. The photoop IDL package contains code to simplify reading these files by run,camcol, and type. If the environment variable $PHOTO_SWEEP is set to the base directory holding these files, one can use the sweep_readobj function to read the data. E.g. to read run=756, camcol=2, rerun=301 for galaxies, one can use the sweep_readobj function:

IDL> data=sweep_readobj(756, 2, rerun=301, type='gal')

See the software page to get photoop and other officially supported software.

Further cuts should be made on these data before using them for science. Most users will only want the SURVEY PRIMARY objects. See the resolve documentation for how to select primary objects. Of the primary objects, most users will want to remove saturated and other problematic detections. See the object flags for a full description of the diagnostic flags.

Field and run metadata

For each run, and each field within each run, we provide metadata with information on the location of each run, the observing conditions, quality flags, and other relevant information. This are stored in the CAS as the Run and Field tables, and in the flat files as the photoField and photoRun files.

The following information is included for each field:

The following information is included for each run:

date of observation
stripe and strip designation
starting and ending field numbers

Uncalibrated files: fpAtlas, fpObjc, and others

Our release also contains all of the processed, but uncalibrated files for reference, in a directory referred to as PHOTO_REDUX (click here to browse the actual files). For example, corresponding to each photoObj file there is an uncalibrated set of data in an fpObjc file. Included in the same directory are other intermediate outputs of the pipeline, explained by the data model.

Most of these files are not for general consumption, but the directories do contain the fpAtlas files, which contain the individual atlas images for each object. See the images page for instructions on how to read these files.

Inside the PHOTO_REDUX directory for DR8 we also include the reductions of the Apache Wheel scans in the rerun 157 directories, described in the page on calibration. These scans are fast binned scans that are lower signal-to-noise and lower resolution than standard SDSS camera imaging. These reductions are essential to the survey calibration, and we include them here for that reason. However, we do not recommend their general use.