Imaging caveats

There are several small caveats to watch out for in SDSS imaging data. Some affect only a few objects or a few data columns, while some have wider impacts. Some caveats in DR8 have been fixed with Data Release 9; those are listed on this page to allow for easier comparison between releases.

This page contains a list of known caveats in SDSS Data Release 9 imaging data.

List of caveats

Not a caveat anymore! Astrometry errors corrected in DR9

This caveat is appropriate for users of the DR8 catalog, but irrelevant for DR9 and subsequent releases. The DR9 astrometry and proper motions are superior to both DR8 and DR7.

Because of several errors, the absolute astrometry in DR8 was degraded relative to that in DR7, and degraded substantially northward of a declination of around 40 deg. Note that the proper motions are considerably less degraded. See the full details on the DR8 astrometry algorithms page. Systematic errors introduced in DR8 southward of 40 deg declination are typically smaller than or comparable to the 45 mas systematic errors that characterize the SDSS astrometry for brighter stars. However, northward of 40 deg there is a systematic offset of around 250 mas in the declination direction.

For very precise astrometry it is necessary to use the DR9 version of the catalog or, if it is necessary to use DR8, two special CAS tables included in that release: astromDR9 and properMotionsDR9, described on the DR8 astrometry algorithms page. These tables contain the values for positions and proper motions that are distributed in the DR9 release. (Note that there are some much smaller differences between DR8 and DR9 because of astrometry, in the sizes and position angles of objects for example; these are tiny and are not included in these tables).

SDSS Object IDs have not changed as a result as a the improved astrometry, although the IAU identifiers (SDSSJ...) may have changed.

Overestimation of sky levels near bright galaxies

Differences between the true and measured sizes and magnitudes of large galaxies, based on simulated images. The measured sizes are based on the better of the exponential and de Vaucouleurs models. The measured magnitudes are the cmodel magnitudes. The galaxies are measured by photo to be smaller and less bright than they truly are.

A number of investigators have shown that the sky subtraction algorithm used by the photometric pipeline causes it to systematically underestimate the brightness of large galaxies (Blanton et al. 2005, Lauer et al. 2007, Bernardi et al. 2007, and Lisker et al. 2007, among others). We quantified this by adding 1300 fake galaxies at random positions to SDSS imaging frames, reducing them with both the old (DR7) and new (DR8) versions of photo, and comparing the results with the truth. The galaxies, which have Sérsic radial profiles with a range of inclinations and Sérsic indices, follow the observed correlation between apparent magnitude and angular size seen for real galaxies. However, we biased the sample somewhat to larger and brighter objects, as this is the regime in which the sky subtraction errors are likely to be worst; in addition, the sample is approximately size-limited at r₅₀ ~ 5 arcsec.

The results are shown at right, where we plot the difference between measured and true magnitude in the r band for the simulated galaxies for the DR7 (red) and DR8 (blue) versions of the pipeline. Results in the other bands are similar. The new sky subtraction algorithm helps, but is not a panacea. The principal trend is with galaxy size, because that couples most directly to the sky measurement. The bias at the largest sizes (of well over a magnitude) is reduced in the new code by only about 0.25 magnitudes. The improvement as a function of half-light radius is also subtle at best, and is visible only for galaxies with r₅₀ > 30 arcsec. Some of the problem may be due not to sky subtraction, but rather to the deblender systematically assigning some of the light in the outer parts of galaxies to superposed fainter stars and galaxies.

Background galaxy counts around bright galaxies, for several different magnitude bins, in both DR7 and DR8. Except in the case of the brightest foreground galaxies, the DR8 counts are always closer to flat with distance (which is the desired result).

A related problem reported by Mandelbaum et al. (2005) was an observed suppression in the number density of faint galaxies around bright galaxies, due to the sky misestimation caused by the latter. We have reinvestigated this problem for the new photometric pipeline. The results are shown at right, the number counts of faint galaxies around bright galaxies as a function of bright galaxy magnitude. In three of the panels, all potential background galaxies are included, and both DR7 (black) and DR8 (red) results are shown. In the bottom right panel, faint galaxies with a possible physical association with the bright galaxies are excluded, and only DR8 is shown.

In all cases, the background galaxy counts are perturbed by the presence of the foreground galaxies; in particular, as the bottom right panel shows, the effect seems to be a net removal of sources. For DR8, the faint galaxy counts are substantially less affected by the bright galaxies than for DR7, particularly for foreground galaxies with r > 15. For the brightest set of foreground galaxies (very rare on the sky), the sense of the change is ambiguous, since it may be caused in an improvement in the deblending of the brightest galaxies.

Changes in OBJID and primary objects in DR8

There are two major imaging bookkeeping changes between DR7 and subsequent releases.

First, for DR8 and later releases the photometry was reduced using a new version of the photo pipeline, under rerun "301". The objID identifier depends on both the rerun value and the object number within each field (called id in flat files or obj in CAS). Both of these values can change under a new rerun. Thus, all of the objID values of every object changes from DR7 to DR8.

Second, the resolve algorithm changed between DR7 and DR8. This algorithm determines which fields are used as the "primary" observation of any given region of sky and which catalog entries are the "primary" detections of the given source. Thus, some fields (and objects) which were considered primary in DR7 are not primary in DR8 (and vice-versa). Thus, the set of runs that contribute to the photoPrimary table in CAS are different, even in areas where no new imaging was taken between DR7 and DR8. Note that one aspect of the new resolve algorithm is that there are no "special" runs or areas of the survey; SEGUE runs and other imaging outside the "Legacy" survey area is all treated the same way.

For users who need to match between DR7 and DR8 (or subsequent), we recommend using the photoObjDR7 and photoPrimaryDR7 tables in CAS> The photoObjDR7 stores a positional match between the DR7 and DR8 data within each run (that is, it only allows a match between the two data releases for two catalog entries within the same run). The photoPrimaryDR7 table matches between the "primary" photometry for each data release (so can contain a match between an object in run X of DR7 but run Y of DR8).

DR8 field timeouts more common

About 1% of fields in DR8 timed out, whereas that rate was about 0.1% in DR7. These cases are usually due to large galaxies or bright stars; on fields containing such objects, the new sky-subtraction techniques cause larger deblends and processing times. Many of the timed-out fields in DR8 are in the Galactic Plane; the rate at Galactic latitudes greater than 15 deg is about 0.5%.

One can identify a timed-out field in the field table or photoField files with the photoStatus flag, which is set to 3 for timed-out fields. See the image quality documentation for a more complete description.

Not a caveat anymore! Early DR8 version missing bits in (flags, flags_u, flags_g, flags_r, flags_i, flags_z), now fixed in DR8 and DR9

Until June 2011, the CAS photoObj tables (and all related views), the flags variables were missing the last 32 bits of information. This included flags, flags_u, flags_g, flags_r, flags_i, and flags_z. This error occurred due to an error in preparing the inputs for the database load.

This problem has been repaired in the current version of the DR8 database.

Bad CCD columns

Some chips have bad CCD columns which get interpolated over by the photometric pipeline, leading to noticeably correlated noise. The bad columns for each run are currently available in fpM*.fits. These files can be found on the Science Archive Server in the objcs subdirectory of each run/rerun directory (e.g., http://data.sdss3.org/sas/dr9/boss/photo/redux/301/1740/objcs/1/. They can be read with the read_mask command; see our description of reading atlas images for access to that software.

Very red objects

The u filter has a natural red leak around 7100 Å which is supposed to be blocked by an interference coating. However, under the vacuum in the camera, the wavelength cutoff of the interference coating has shifted redward (see the discussion in the EDR paper), allowing some of this red leak through. The extent of this contamination is different for each camera column. It is not completely clear if the effect is deterministic; there is some evidence that it is variable from one run to another with very similar conditions in a given camera column. Roughly speaking, however, this is a 0.02 magnitude effect in the u magnitudes for mid-K stars (and galaxies of similar color), increasing to 0.06 magnitude for M0 stars (r-i ~ 0.5), 0.2 magnitude at r-i ~ 1.2, and 0.3 magnitude at r-i = 1.5. There is a large dispersion in the red leak for the redder stars, caused by three effects:

The differences in the detailed red leak response from column to column, beating with the complex red spectra of these objects.
The almost certain time variability of the red leak.
The red-leak images on the u chips are out of focus and are not centered at the same place as the u image because of lateral color in the optics and differential refraction - this means that the fraction of the red-leak flux recovered by the PSF fitting depends on the amount of centroid displacement.

To make matters even more complicated, this is a detector effect. This means that it is not the real i and z which drive the excess, but the instrumental colors (i.e., including the effects of atmospheric extinction), so the leak is worse at high airmass, when the true ultraviolet flux is heavily absorbed but the infrared flux is relatively unaffected. Given these complications, we cannot recommend a specific correction to the u-band magnitudes of red stars, and warn the user of these data about over-interpreting results on colors involving the u band for stars later than K.

u-band sky

There is a slight and only recently recognized downward bias in the determination of the sky level in the photometry, at the level of roughly 0.1 DN per pixel. This is apparent if one compares large-aperture and PSF photometry of faint stars; the bias is of order 29 mag arcsec^-2 in r. This, together with scattered light problems in the u band, can cause of order 10% errors in the u band Petrosian fluxes of large galaxies. As implied, this effect does perturb galaxy colors; that is the sky subtraction problem described above has slightly worse effects on the u band than on other bands.

Missing high proper-motion stars

A comparison of SDSS catalogs has shown that high proper motion stars from Sebastien Lepine's database (SUPERBLINK) are not registered as high proper motion stars in the DR6. For those stars, the ProperMotions table lists pm=0.0. The reason their motion is not registered in DR6 is because of the incompleteness of the USNO-B catalog, from which the DR6 proper motions are derived. Areas where the incompleteness is particularly severe include regions where there are bad SERC-I or POSS-II N plates (open squares, list from J. Munn).

If one tries to select off nearby stars with e.g. a pm<0.75 mas/yr proper motion cutoff, then the sample will be contaminated with these "pm=0" high proper motion stars. The plot shows the stars with pm>100 mas/yr, which are relatively rare, but I suspect that a similar fraction of 10 mas/yr < pm < 100 mas/yr stars will be similarly unregistered in the DR6, which can add up a lot of foreground contaminants.

Top panel: high proper motion stars from the Superblink survey that are missing in DR6. Bottom panel: High proper motion stars recovered by SDSS.

At the moment, the only mitigation strategy is to avoid the regions where contamination will be most severe.

Incomplete and/or inaccurate photometry at low galactic latitudes

DR8 includes a fair amount of imaging at low Galactic latitude |b| < 25 degrees, and as such, there are highly crowded fields, and regions of high extinction. These data were processed with the standard SDSS photo pipelines. Since these pipelines were not designed to work in such crowded regions, the quality of the photometry in these areas is not guaranteed to be accurate to the SDSS quoted limits of 2% in color and r magnitude, nor is each and every crowded frame fully deblended; i.e., many fields are incompletely cataloged. Some fields, in particular many at |b| < 5 degrees, time out completely and have no cataloged objects whatsoever. Using the image quality flags is often useful to identify such cases.

Missing survey geometry

The survey geometries, including Sectors and Footprints, will be created shortly. We are also developing a new Geometry viewer, which will allow interactive inspection of the different survey boundaries.

SkyServer returns "response buffer limit exceeded"

A SQL Search query on SkyServer might, on very rare occasions, return the following error:

SQL returned the following error message: 006~ASP 0251~Response Buffer Limit Exceeded~Execution of the ASP page caused the Response Buffer to exceed its configured limit. Your SQL command was:

This message is due to a default behavior of Microsoft Internet Information Services (IIS) version 6.0 and higher - documented in the Microsoft Knowledge Base - in which responses to actions of Active Server Pages (.asp) are limited to 4 MB file size.

This error is extremely rare, but there is a simple workaround: simply run the query in CasJobs instead.

Missing outlines in some runs

Some fields in DR9 imaging are missing object outlines. Thus, in these fields, the Outlines option in the SkyServer Navigate tool does not work. The table below shows run-camcol combinations that do not have outlines, or have partial outlines only.

Run	No outlines	Partial outlines
1462	3-6	2
2326	3-6	2
6794	2-6	None