2014年1月31日星期五

Deal with Catalogs in astronomy

Did you have try to run Sextractor? Did you save spectra data with a catalog file of several columns? How did you save your astronomical data? It could be a catalog. Now I introduce my way to deal with these files.

(1)  Command lines. Using small program+awk
My most work is using the object catalogs from Sextractor. I must search the data from a particular row in a particular column. So I made a program search.py to search the one column. To print out the results, I utilize the "awk" like this.
search [-i] catalog colx coly x y [err] | awk '{printf "%f\n",$1}' 


(2) Using TOPCAT or STILTS
Here is a ppt for quick review for these two softwares.
If you deal with large catalog and want to try some plots to show the relation, please try the two! I just begin to use them. It seems good when you want to have a quick view.
Let us enjoy the CATALOG.








2014年1月29日星期三

Sextractor【0】Something You want to [MUST] know about SEXTRACTOR

FIRST, Sextractor is a quite powerful tool. If you want to make detections, please use it !  If  you have never used it, please have a try. As a graduated  student with the major of Glaxies and AGN, it is necessary to learn this software. Although I spent a lot of time in this software, I found it worth. Here I will give you my experience.
[More information will be added in the future!]


Simple Example:
Dual model
  sex -c configfile detection.fits,detection.fits
You may need several files which you can find in the configure file.

Tips:
Use the check images, especially the segmentation image and the aperture image which will be useful!
Set the gain to CCDGAIN * exposure_time, if  you use images in units of counts/s


















Some Formula:

Error:
    FLUX_ERR = sqrt( (A*sig^2) + F/g)
where A is the object area, sig is the background RMS (? I think it is estimated from the local background! After a test, I find that the error are the total flux from the MAP_RMS image.) , F is the total flux, and g is the gain (You can see why you need to set the gain to gain*exptime when you use unit of counts/s. Because you need conserve the unit!). The following can be find in ~ Page 36 of the User's Mnual.










Correcting for correlate noise when using MultiDrizzle images
Regardless of how you run SExtractor, the errors will be incorrect if the images have been MultiDrizzle'd. For details on this, see Casertano et al. 2000. If you have run MultiDrizzle with pixfrac p and scale s, then you should calculate a correction factor
sqrt(F_A) = (s/p)(1-(s/3p)) if s<p
sqrt(F_A) = 1 - p/(3s)      if s>p
NOTE: Here, s is the multiplicative factor applied to the original image pixel scale. For ACS, if your output drizzled image has 0.03" pixels, then s=0.6. For instance, for HAGGLES, p=0.9 and s=0.6, so √FA=0.51852. This correction factor needs to be applied to the background noise estimate generated by SExtractor (output to the screen at runtime as the RMS), and therefore to the flux and magnitude errors that are output.
Find the explanation in the handbook or from Casertano 2000
       

FLUX_ERR_CORRECT = sqrt( ((A*sig^2)/F_A) + FLUX/GAIN )
MAG_ERR_CORRECT = FLUX_ERR_CORRECT/FLUX * 2.5/ln(10)






IMPORTANT NOTE REGARDING WEIGHT AND RMS MAPS
Although the mathematical relationship between a weight map and an rms map is simply
      weight = 1/(rms^2)    
(check the sextractor book for the explanation)
________________________________________________________________
However, I found 
      rms = 1/sqrt(weight) * std(weight)   ---- Wrong!
Sometimes someone chooses weight/exptime other then weight, then
      rms = 1/sqrt(weight/exptime) * std(weight/exptime)  ---- Wrong! 
Here the std(weight) is the standard deviation for the whole weight image. This means if you choose a larger grid to store the image and set the edges of weight images to value 0, then the rms values will be underestimated. This will affect the detection threshold you set in the configure file. You may need to set a higher detection threshold.   ---- Wrong!
________________________________________________________________
Weight images is calculated from rms images in Multidrizzle. You can find this in the handbook (Check here)

Where D and B are the counts per pixel (in DN) due to the dark current and background, respectively, averaged over the entire image. t is the exposure time in seconds, g is the gain of the detector (users should be aware of the units of their image and use the appropriate gain value), and  is the readnoise in DN/pixel. The quantity f represents the inverse flat field, corresponding to the way in which the HST pipeline flats are defined.

However, notice that the weight maps are different in Multidrizzle and Sextractor. So you 'd better use MAP_RMS instead. I still do not figure out how to convert these between these two softwares.  
SExtractor does NOT treat these two map types by just converting one to the other using this relation! Instead, it does the following:
  • MAP_RMS: The image is assumed to be in units of absolute standard deviations, in ADU/pixel [which is what we have if we take 1/sqrt(weightmap) ]
  • MAP_WEIGHT: Assumed to be in units of RELATIVE weights. Data are CONVERTED to variance, using var=1/weight AND scaling to internal, low-resolution absolute variance map built from science image. This is not the case in MultiDrizzle data, and in fact is a misuse of the term "weight map". Be careful if you use MAP_WEIGHT for Sextractor, and you should set gain correctly if you really want.
Want to know more about weight image? 
Check here (the basic explanation from http://www.astromatic.net/) and  here (the detail explanation from DrizzlePac).


Parameter


BACKPHOTO_THICK

    The exact limits of the background "rectangular annulus" are defined as follow.
First of all, a box is defined by the limits in x and y of the isophotal boundaries (at the detection level) of the object. This box is then grown symmetrically on both axes by 50%. This sets the inner boundaries of the background annulus. Simply add BACKPHOTO_THICK more pixels on both sides of each axis and you obtain the outer limits of the background annulus.
EXAMPLE:
    My object's center is defined at x=0, y=0, and its isophotal detection boundaries are defined by x=20 and y=20 (giving a total diameter of 40 pixels for my object). The radius of my object is increased by 50% giving an inner radius of the rectangular annulus of 30 pix.
    Then, the if BACKPHOTO_THICK is, say, 40 pix, the outer radius of the annulus is 70 pix (as measured from the center of the object) making the total area defined by this annulus: (70*70)-(30*30)=4000 pixels sqaured.
(From Here)

BACK_FILTERSIZE
BACK_FILTERSIZE is used before BACK_SIZE is. It is used in the smoothing that goes on "to get rid of the deviations resulting from bright or extended objects."

BACK_SIZEBACK_SIZE contributes to the background map by using "a bi-cubic-spline interpolation over all the areas of size BACK_SIZE." SExtrator for Dummies recommends setting BACK_SIZE larger than the average size of the objects in your image. (From Here)

DETECT_THRESH 1.5
will set the detection threshold at 1.5σ above the LOCAL background.
the standard deviation quoted here is that of the unFILTERed image, at the pixel scale.

DETECT MINAREA [1-5]
In most cases it is therefore recommended to keep DETECT MINAREA at a small value, typically 1 to 5 pixels, and let DETECT THRESH and the filter define SExtractor’s sensitivity.


Details about the route and the relatio with these parameters:
BACK_FILTERSIZE   BACK_SIZE  BACKPHOTO_THICK  RMS MAPS

(1) The program substract the filtered smoothed backgound from the whole image based on the BACK_FILTERSIZE   BACK_SIZE parameter. The background and be checked with CATALOG TYPE BACKGROUND. 
(2) Source detection based on the RMS MAPS and the DETECT_THRESH.
(3) Substract the local background based on the parameters  BACKPHOTO_THICK, Calculate the flux and magnitude. The errors are also calculated in the same region based on the RMS MAPS.
FWHM_IMAGE

   I made a simple test and compared SE measurments with those given by IRAF's imexam task.
FWHM's of extended and stellar objects are overestimated by SE relative to those computed using imexam by factor of ~20%.
   An 'imexam' does iterative gaussian profile fitting. How about SE? Which 'object pixels' are used to calculated FWHM and how it is done?

There are currently 2 ways to measure the FWHM of a source in SExtractor. Both are rather imperfect.
     FWHM_IMAGE derives the FWHM from the isophotal area of the object at half maximum.
     FLUX_RADIUS estimates the radius of the circle centered on the barycenter that encloses about half of the total flux. For a Gaussian profile, this is equal to 1/2 FWHM. But with most images on astronomical images it will be slightly higher.








Making a Flag Image
You can make a complex flag image that uses different values to denote different exposure levels. The below example just sets a binary flag value for areas that are very underexposed (flag=1 if bad).
cl> median ACSJ160449+432050_F814W_wht.fits med_foo.fits 7 7 boundary=nearest
Examine the image, and find the data value below which you wish to flag (say, 500).
cl> stsdas
cl> toolbox
cl> imgtools
cl> imcalc med_foo.fits flag_foo.fits "if im1 .lt. 500. then 1 else 0"
cl> boxcar flag_foo.fits box_foo.fits 7 7  boundary=nearest
cl imcalc box_foo.fits ACSJ160449+432050_F814W_flag.fits "if im1 .gt. 0.2 then 1 else 0" pixtype=int
It is important to set the output pixel type to "int" in the last step - SExtractor expects flag images to contain only integers.






Other sites for the discussion about Sextractor:

Useful Things that I Always Forget

2014年1月28日星期二

Tips about aperture photometry

About the Aperture Photometry Tool (APT) 


About the Optimum Aperture Radius for a Gaussian Profile

We derive the optimum aperture size for photometry of sources whose Point Response Function
(PRF) profiles can be approximated as Gaussian. By optimum, we mean the aperture size (radius
Rmax) which maximizes the signal-to-noise ratio. If you’re after the answer right away, here it is:

       Rmax ~1.5852σ  = 0.6731 FWHM,

where σ and FWHM (Full Width Half Maximum) characterize the Gaussian profile’s width

About An Introduction to Astronomical Photometry Using CCDs




2014年1月19日星期日

tasks in IRAF

The NOAO Packages
IRAF Version 2.10

(This list does not reflect any tasks added by the various patches to V2.10.)

  • noao:
    
             artdata - Artificial data generation package   [up]
          astrometry - Astrometry package
             astutil - Astronomical utilities package    [up]
            digiphot - Digital stellar photometry package   [up]
               focas - Faint object classification and analysis package
               imred - Image reductions package    [up]
             mtlocal - Magtape i/o for special NOAO format tapes  [up]
           nobsolete - Obsolete tasks to be phased out in a future release [up]
              nproto - Prototype (temporary, contributed) tasks  [up]
         observatory - Examine and define observatory parameters  [up]
            onedspec - One dimensional spectral red & analysis package  [up]
                  rv - Radial velocity analysis package   [up]
            surfphot - Galaxy isophotal analysis package
            twodspec - Two dimensional spectral red & analysis package  [up]
    
  • noao.artdata:
    
             gallist - Make an artificial galaxies list
            mk1dspec - Make/add artificial 1D spectra
            mk2dspec - Make/add artificial 2D spectra using 1D spectra templates
           mkechelle - Make artificial 1D and 2D echelle spectra
          mkexamples - Make artificial data examples
            mkheader - Append/replace header parameters
             mknoise - Make/add noise and cosmic rays to 1D/2D images
           mkobjects - Make/add artificial stars and galaxies to 2D images
           mkpattern - Make/add patterns to images
            starlist - Make an artificial star list
    
  • noao.astutil:
    
             airmass - Compute the airmass at a given elevation above the horizon
            asttimes - Compute UT, Julian day, epoch, and siderial time
             ccdtime - Compute time required to observe star of given magnitude
            galactic - Convert ra, dec to galactic coordinates
            gratings - Compute and print grating parameters
                 pdm - Find periods in light curves by Phase Dispersion Minimization
             precess - Precess a list of astronomical coordinates
           rvcorrect - Compute radial velocity corrections
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
    
  • noao.digiphot:
    
              apphot - Aperture Photometry Package
             daophot - Dao Crowded-Field Photometry Package
             photcal - Photometric Calibration Package
              ptools - Photometry Tools Package
    
  • noao.digiphot.apphot:
    
              aptest - Run basic tests on the apphot package tasks
              center - Compute accurate centers for a list of objects
          centerpars - Edit the centering parameters
             daofind - Find stars in an image using the DAO algorithm
            datapars - Edit the data dependent parameters
              fitpsf - Model the stellar psf with an analytic function
              fitsky - Compute sky values in a list of annular or circular regions
          fitskypars - Edit the sky fitting parameters
                phot - Measure magnitudes for a list of stars
            photpars - Edit the photometry parameters
            polymark - Create polygon lists for polyphot
            polyphot - Measure magnitudes inside a list of polygonal regions
            polypars - Edit the polyphot parameters
               qphot - Measure quick magnitudes for a list of stars
             radprof - Compute the stellar radial profile of a list of stars
               wphot - Measure magnitudes for a list of stars with weighting
             lintran - Linearly transform a coordinate list
            pexamine - Interactively examine or edit an apphot output file
              txdump - Dump select fields from an apphot output file
    
  • noao.digiphot.daophot:
    
             addstar - Add artificial stars to an image using the computed psf
             allstar - Group and fit psf to multiple stars simultaneously
          centerpars - Edit the centering algorithm parameters
             daofind - Find stars in an image using the DAO algorithm
             daopars - Edit the daophot algorithms parameter set
             daotest - Run basic tests on the daophot package tasks
            datapars - Edit the data dependent parameters
          fitskypars - Edit the sky fitting algorithm parameters
               group - Group stars based on positional overlap and signal/noise
               nstar - Fit the psf to groups of stars simultaneously
                peak - Fit the psf to single stars
                phot - Compute sky values and initial magnitudes for a list of stars
            photpars - Edit the photometry parameters
                 psf - Fit the point spread function
              seepsf - Compute an image of the point spread function
             substar - Subtract the fitted stars from the original image
             pappend - Concatenate a list of daophot databases
            pconvert - Convert a text database to a tables database
               pdump - Print selected fields from a list of daophot databases
           grpselect - Select groups of a specified size from a daophot database
            pexamine - Interactively examine and edit a daophot database
           prenumber - Renumber stars in a daophot database
             pselect - Select records from a daophot database
               psort - Sort a daophot database
    
  • noao.digiphot.ptools:
    
             istable - Is a file a table or text database file ?
             pappend - Concatenate a list of apphot/daophot databases
            pconvert - Convert from an apphot/daophot text to tables database
               pdump - Print selected columns of a list of  daophot/apphot databases
           prenumber - Renumber a list of apphot/daophot databases
            pexamine - Interactively examine and edit an apphot/daophot database
             pselect - Select records from a list of apphot/daophot databases
               psort - Sort a list of apphot/daophot databases
              pttest - Run basic tests on the ptools package tasks
            tbappend - Concatenate a list of apphot/daophot tables databases
              tbdump - Print selected columns of a list of tables databases
          tbrenumber - Renumber a list of  apphot/daophot tables databases
            tbselect - Select records from a list of apphot/daophot tables databases
              tbsort - Sort a list of apphot/daophot tables databases
            txappend - Concatenate a list of apphot/daophot text databases
              txdump - Print selected columns of a list of apphot/daophot text databases
          txrenumber - Renumber a list of apphot/daophot text databases
            txselect - Select records from a list of apphot/daophot text databases
              txsort - Sort a list of apphot/daophot text databases
    
  • noao.imred:
    
               argus - CTIO ARGUS reduction package
                bias - General bias subtraction tools
              ccdred - Generic CCD reductions
            ctioslit - CTIO spectrophotometric reduction package
                dtoi - Density to Intensity reductions for photographic plates
             echelle - Echelle spectra reductions (slit and FOE)
             generic - Generic image reductions tools
               hydra - KPNO HYDRA (and NESSIE) reduction package
                iids - KPNO IIDS spectral reductions
               irred - KPNO IR camera reductions
                 irs - KPNO IRS spectral reductions
           kpnocoude - KPNO coude reduction package (slit and 3 fiber)
            kpnoslit - KPNO low/moderate dispersion slits (Goldcam, RCspec, Whitecam)
             specred - Generic slit and fiber spectral reduction package
                vtel - Solar vacuum telescope image reductions
    
  • noao.imred.argus:
    
               apall - Extract 1D spectra (all parameters in one task)
           apdefault - Set the default aperture parameters
              apedit - Edit apertures interactively
              apfind - Automatically find spectra and define apertures
          aprecenter - Recenter apertures
            apresize - Resize apertures
               apsum - Extract 1D spectra
             aptrace - Trace positions of spectra
               bplot - Batch plots of spectra
           continuum - Fit the continuum in spectra
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
            identify - Identify features in spectrum for dispersion solution
            msresp1d - Create 1D response spectra from flat field and sky spectra
          refspectra - Assign wavelength reference spectra to other spectra
          reidentify - Automatically identify features in spectra
          sapertures - Set or change aperture header information
              sarith - Spectrum arithmetic
            scombine - Combine spectra having different wavelength ranges
               scopy - Select and copy apertures in different spectral formats
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
               slist - List spectrum header parameters
            specplot - Stack and plot multiple spectra
               splot - Preliminary spectral plot/analysis
             doargus - Process ARGUS spectra
               demos - Demonstrations and tests
    
  • noao.imred.bias:
    
             colbias - Fit and subtract an average column bias
            linebias - Fit and subtract an average line bias
    
  • noao.imred.ccdred:
    
         badpiximage - Create a bad pixel mask image from a bad pixel file
           ccdgroups - Group CCD images into image lists
            ccdhedit - CCD image header editor
       ccdinstrument - Review and edit instrument translation files
             ccdlist - List CCD processing information
             ccdproc - Process CCD images
             ccdtest - CCD test and demonstration package
             combine - Combine CCD images
          cosmicrays - Detect and replace cosmic rays
         darkcombine - Combine and process dark count images
         flatcombine - Combine and process flat field images
         mkfringecor - Make fringe correction images from sky images
          mkillumcor - Make flat field illumination correction images
         mkillumflat - Make illumination corrected flat fields
            mkskycor - Make sky illumination correction images
           mkskyflat - Make sky corrected flat field images
       setinstrument - Set instrument parameters
         zerocombine - Combine and process zero level images
    

    ADDITIONAL HELP TOPICS

    
         ccdgeometry - Discussion of CCD coordinate/geometry keywords
            ccdtypes - Description of the CCD image types
          flatfields - Discussion of CCD flat field calibrations
               guide - Introductory guide to using the CCDRED package
         instruments - Instrument specific data files
             package - CCD image reduction package
             subsets - Description of CCD subsets
    
  • noao.imred.ccdred.ccdtest:
    
              artobs - Create an artificial CCD observation
                demo - Run a demonstration of the CCD reduction package
             mkimage - Make or modify an image with simple values
          subsection - Create an artificial subsection CCD observation
    
  • noao.imred.ctioslit:
    
               apall - Extract 1D spectra (all parameters in one task)
           apdefault - Set the default aperture parameters
              apedit - Edit apertures interactively
              apfind - Automatically find spectra and define apertures
          aprecenter - Recenter apertures
            apresize - Resize apertures
               apsum - Extract 1D spectra
             aptrace - Trace positions of spectra
               bplot - Batch plot of spectra with SPLOT
           calibrate - Apply extinction and flux calibrations to spectra
           continuum - Fit and normalize the continuum of multispec spectra
            deredden - Apply interstellar extinction corrections
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
            identify - Identify arc lines and determine a dispersion function
          refspectra - Assign reference spectra to object spectra
          reidentify - Reidentify arc lines and determine new dispersion functions
              sarith - Spectrum arithmetic
            scombine - Combine spectra
               scopy - Copy spectra including aperture selection and format changes
            sensfunc - Create sensitivity function
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
               slist - List spectral header elements
            specplot - Stack and plot multiple spectra
               splot - Plot and analysis spectra
            standard - Identify standard stars to be used in sensitivity calc
              doslit - Process CTIO slit spectra
               demos - Demonstrations and tests
    
  • noao.imred.dtoi:
    
             dematch - Match a list of density values to exposure values
               hdfit - Fit a curve to density, log exposure values
             hdshift - Align related HD curves
               hdtoi - Apply DTOI transformation to density image
            selftest - Self test program to check DTOI transformation
            spotlist - Generate a list of calibration spot values
    
  • noao.imred.echelle:
    
               apall - Extract 1D spectra (all parameters in one task)
           apdefault - Set the default aperture parameters and apidtable
              apedit - Edit apertures interactively
              apfind - Automatically find spectra and define apertures
               apfit - Fit 2D spectra and output the fit, difference, or ratio
           apflatten - Remove overall spectral and profile shapes from flat fields
              apmask - Create an IRAF pixel list mask of the apertures
         apnormalize - Normalize 2D apertures by 1D functions
          aprecenter - Recenter apertures
            apresize - Resize apertures
           apscatter - Fit and subtract scattered light
               apsum - Extract 1D spectra
             aptrace - Trace positions of spectra
               bplot - Batch plots of spectra
           calibrate - Apply extinction and flux calibrations to spectra
           continuum - Fit the continuum in spectra
            deredden - Apply interstellar extinction corrections
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
          ecidentify - Identify features in spectrum for dispersion solution
        ecreidentify - Automatically reidentify features in spectra
          refspectra - Assign wavelength reference spectra to other spectra
              sarith - Spectrum arithmetic
            scombine - Combine spectra
               scopy - Select and copy apertures in different spectral formats
            sensfunc - Compute sensitivity function
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
               slist - List spectrum header parameters
            specplot - Stack and plot multiple spectra
               splot - Preliminary spectral plot/analysis
            standard - Identify standard stars to be used in sensitivity calc
            doecslit - Process Echelle slit spectra
               dofoe - Process Fiber Optic Echelle (FOE) spectra
               demos - Demonstrations and tests
    
  • noao.imred.generic:
    
          background - Fit and subtract a line or column background
          cosmicrays - Detect and replace cosmic rays
             darksub - Scale and subtract a dark count image
              flat1d - Make flat field by fitting a 1D func. to the lines or columns
             flatten - Flatten images using a flat field
           normalize - Normalize images
            normflat - Create a flat field by normalizing and replacing low values
    
  • noao.imred.hydra:
    
               apall - Extract 1D spectra (all parameters in one task)
           apdefault - Set the default aperture parameters
              apedit - Edit apertures interactively
              apfind - Automatically find spectra and define apertures
          aprecenter - Recenter apertures
            apresize - Resize apertures
           apscatter - Fit and remove scattered light
               apsum - Extract 1D spectra
             aptrace - Trace positions of spectra
               bplot - Batch plots of spectra
           continuum - Fit the continuum in spectra
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
            identify - Identify features in spectrum for dispersion solution
            msresp1d - Create 1D response spectra from flat field and sky spectra
          refspectra - Assign wavelength reference spectra to other spectra
          reidentify - Automatically identify features in spectra
          sapertures - Set or change aperture header information
              sarith - Spectrum arithmetic
            scombine - Combine spectra having different wavelength ranges
               scopy - Select and copy apertures in different spectral formats
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
               slist - List spectrum header parameters
            specplot - Stack and plot multiple spectra
               splot - Preliminary spectral plot/analysis
             dohydra - Process HYDRA spectra
               demos - Demonstrations and tests
    
  • noao.imred.iids:
    
             addsets - Add subsets of strings of spectra
            batchred - Batch processing of IIDS/IRS spectra
               bplot - Batch plots of spectra
             bswitch - Beam-switch strings of spectra to make obj-sky pairs
           calibrate - Apply sensitivity correction to spectra
               coefs - Extract mtn reduced ceofficients from henear scans
             coincor - Correct spectra for detector count rates
           continuum - Fit the continuum in spectra
            deredden - Apply interstellar extinction corrections
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
             extinct - Use BSWITCH for extinction correction
             flatdiv - Divide spectra by flat field
             flatfit - Sum and normalize flat field spectra
            identify - Identify features in spectrum for dispersion solution
              lcalib - List calibration file data
              mkspec - Generate an artificial spectrum
               names - Generate a list of image names from a string
            powercor - Apply power law correction to mountain reduced spectra
             process - A task generated by BATCHRED
          refspectra - Assign reference spectra to object spectra
          reidentify - Automatically identify features in spectra
            scombine - Combine spectra having different wavelength ranges
            sensfunc - Create sensitivity function
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
             sinterp - Interpolate a table of x,y pairs to create a spectrum
             slist1d - List spectral header elements
            specplot - Stack and plot multiple spectra
               splot - Preliminary spectral plot/analysis
            standard - Identify standard stars to be used in sensitivity calc
             subsets - Substract pairs in strings of spectra
                sums - Generate sums of object and sky spectra by aperture
    
  • noao.imred.irred:
    
              center - Compute accurate centers for a list of objects
          centerpars - Edit the centering parameters
            datapars - Edit the data dependent parameters
             flatten - Flatten images using a flat field
             iralign - Align the image produced by irmosaic
           irmatch1d - Align and intensity match the image produced by irmosaic (1D)
           irmatch2d - Align and intensity match the image produced by irmosaic (2D)
            irmosaic - Mosaic an ordered list of images onto a grid
             mosproc - Prepare images for quick look mosaicing
              txdump - Select fields from the center task output text file
    
  • noao.imred.irs:
    
             addsets - Add subsets of strings of spectra
            batchred - Batch processing of IIDS/IRS spectra
               bplot - Batch plots of spectra
             bswitch - Beam-switch strings of spectra to make obj-sky pairs
           calibrate - Apply sensitivity correction to spectra
               coefs - Extract mtn reduced ceofficients from henear scans
           continuum - Fit the continuum in spectra
            deredden - Apply interstellar extinction corrections
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
             extinct - Use BSWITCH for extinction correction
             flatdiv - Divide spectra by flat field
             flatfit - Sum and normalize flat field spectra
            identify - Identify features in spectrum for dispersion solution
              lcalib - List calibration file data
              mkspec - Generate an artificial spectrum
               names - Generate a list of image names from a string
             process - A task generated by BATCHRED
          refspectra - Assign reference spectra to object spectra
          reidentify - Automatically identify features in spectra
            scombine - Combine spectra having different wavelength ranges
            sensfunc - Create sensitivity function
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
             sinterp - Interpolate a table of x,y pairs to create a spectrum
             slist1d - List spectral header elements
            specplot - Stack and plot multiple spectra
               splot - Preliminary spectral plot/analysis
            standard - Identify standard stars to be used in sensitivity calc
             subsets - Substract pairs in strings of spectra
                sums - Generate sums of object and sky spectra by aperture
    
  • noao.imred.kpnocoude:
    
               apall - Extract 1D spectra (all parameters in one task)
           apdefault - Set the default aperture parameters
              apedit - Edit apertures interactively
              apfind - Automatically find spectra and define apertures
          aprecenter - Recenter apertures
            apresize - Resize apertures
               apsum - Extract 1D spectra
             aptrace - Trace positions of spectra
               bplot - Batch plot of spectra with SPLOT
           calibrate - Apply extinction and flux calibrations to spectra
           continuum - Fit and normalize the continuum of multispec spectra
            deredden - Apply interstellar extinction corrections
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
            identify - Identify arc lines and determine a dispersion function
            msresp1d - Create fiber response spectra from flat field and sky spectra
          refspectra - Assign reference spectra to observations
          reidentify - Reidentify arc lines and determine new dispersion functions
          sapertures - Set or change aperture header information
              sarith - Spectrum arithmetic
            scombine - Combine spectra
               scopy - Copy spectra including aperture selection and format changes
            sensfunc - Create sensitivity function
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
               slist - List spectrum headers
            specplot - Stack and plot multiple spectra
               splot - Plot and analyze spectra
            standard - Identify standard stars to be used in sensitivity calc
            do3fiber - Process KPNO coude three fiber spectra
              doslit - Process KPNO coude slit spectra
               demos - Demonstrations and tests
    
  • noao.imred.kpnoslit:
    
               apall - Extract 1D spectra (all parameters in one task)
           apdefault - Set the default aperture parameters
              apedit - Edit apertures interactively
              apfind - Automatically find spectra and define apertures
          aprecenter - Recenter apertures
            apresize - Resize apertures
               apsum - Extract 1D spectra
             aptrace - Trace positions of spectra
               bplot - Batch plot of spectra with SPLOT
           calibrate - Apply extinction and flux calibrations to spectra
           continuum - Fit and normalize the continuum of multispec spectra
            deredden - Apply interstellar extinction corrections
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
            identify - Identify arc lines and determine a dispersion function
          refspectra - Assign reference spectra to observations
          reidentify - Reidentify arc lines and determine new dispersion functions
              sarith - Spectrum arithmetic
            scombine - Combine spectra
               scopy - Copy spectra including aperture selection and format changes
            sensfunc - Create sensitivity function
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
               slist - List spectrum headers
            specplot - Stack and plot multiple spectra
               splot - Plot and analyze spectra
            standard - Identify standard stars to be used in sensitivity calc
              doslit - Process slit spectra
               demos - Demonstrations and tests
    
  • noao.imred.specred:
    
               apall - Extract 1D spectra (all parameters in one task)
           apdefault - Set the default aperture parameters and apidtable
              apedit - Edit apertures interactively
              apfind - Automatically find spectra and define apertures
               apfit - Fit 2D spectra and output the fit, difference, or ratio
           apflatten - Remove overall spectral and profile shapes from flat fields
              apmask - Create and IRAF pixel list mask of the apertures
         apnormalize - Normalize 2D apertures by 1D functions
          aprecenter - Recenter apertures
            apresize - Resize apertures
           apscatter - Fit and subtract scattered light
               apsum - Extract 1D spectra
             aptrace - Trace positions of spectra
               bplot - Batch plot of spectra with SPLOT
           calibrate - Extinction and flux calibrate spectra
           continuum - Fit the continuum in spectra
            deredden - Apply interstellar extinction correction
             dispcor - Dispersion correct spectra
              dopcor - Doppler correct spectra
            fitprofs - Fit gaussian profiles
            identify - Identify features in spectrum for dispersion solution
            msresp1d - Create 1D response spectra from flat field and sky spectra
          refspectra - Assign wavelength reference spectra to other spectra
          reidentify - Automatically reidentify features in spectra
          sapertures - Set or change aperture header information
              sarith - Spectrum arithmetic
            scombine - Combine spectra
               scopy - Select and copy apertures in different spectral formats
            sensfunc - Compute instrumental sensitivity from standard stars
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
                sfit - Fit spectra and output fit, ratio, or difference
              skysub - Sky subtract extracted multispec spectra
               slist - List spectrum header parameters
            specplot - Scale, stack, and plot multiple spectra
               splot - Preliminary spectral plot/analysis
            standard - Tabuluate standard star counts and fluxes
            dofibers - Process fiber spectra
              doslit - Process slit spectra
    
  • noao.imred.vtel:
    
            destreak - Destreak He 10830 grams.
           destreak5 - First pass processing CL script for 10830 grams.
            dicoplot - Make dicomed plots of carrington maps.
            fitslogr - Make a log of certain header parameters from a FITS tape.
             getsqib - Extract the squibby brightness image from a full disk scan.
          makehelium - Cl script for processing destreaked 10830 grams(second pass).
          makeimages - Cl script for processing magnetograms into projected maps
               merge - Merge daily grams into a Carrington map.
            mrotlogr - Log some header parameters from a FITS rotation map tape.
               mscan - Read all sector scans on a tape and put them into images.
             pimtext - Put text directly into images using a pixel font.
             putsqib - Merge a squibby brightness image into a full disk image.
            quickfit - Fit an ellipse to the solar limb.
              readvt - Read a full disk tape and produce an IRAF image.
                rmap - Map a full disk image into a 180 by 180 flat image.
             syndico - Make dicomed print of daily grams 18 cm across.
               tcopy - Tape to tape copy routine.
                trim - Set all pixels outside the limb to 0.0 (use sqib for limb).
              unwrap - Remove effects of data wraparound on continuum scans.
             vtblink - Blink daily grams on the IIS to check for registration.
           vtexamine - Examine a vacuum telescope tape, print headers and profile.
           writetape - Cl script to write 5 full disk grams to tape.
             writevt - Write an IRAF image to tape in vacuum telescope format.
    
  • noao.mtlocal:
    
              ldumpf - List the permanent files on a Cyber DUMPF tape
                r2df - Convert a CTIO 2-d frutti image into an IRAF image
             rcamera - Convert a CAMERA image into an IRAF image
              rdumpf - Convert IPPS rasters from a DUMPF tape to IRAF images
            ridsfile - Convert IDSFILES from a DUMPF tape to IRAF images
             ridsmtn - Convert mountain format IDS/IRS data to IRAF images
             ridsout - Convert a text file in IDSOUT format to IRAF images
                rpds - Convert a PDS image into an IRAF image
              rrcopy - Convert IPPS rasters from an RCOPY tape to IRAF images
            widstape - Convert ONEDSPEC spectra to IDSOUT text format
    
  • noao.nproto:
    
            binpairs - Bin pairs of (x,y) points in log separation
            findgain - Estimate the gain and readnoise of a CCD
          findthresh - Estimate a CCD's sky noise from the gain and readnoise
             iralign - Align the mosaiced image produced by irmosaic
           irmatch1d - Align and intensity match image produced by irmosaic (1D)
           irmatch2d - Align and intensity match image produced by irmosaic (2D)
            irmosaic - Mosaic an ordered list of images onto a grid
              linpol - Calculate polarization frames and Stoke's parameters
             slitpic - Generate IRAF image of aperture slit mask
    
  • noao.onedspec:
    
               bplot - Batch plots of spectra
           calibrate - Apply extinction and flux calibrations to spectra
           continuum - Fit the continuum in spectra
            deredden - Apply interstellar extinction correction
            dispaxis - Dispersion axis parameters for 2D images
             dispcor - Dispersion correct spectra
              dopcor - Apply doppler corrections
            fitprofs - Fit gaussian profiles
            identify - Identify features in spectrum for dispersion solution
              lcalib - List calibration file data
              mkspec - Generate an artificial spectrum
               names - Generate a list of image names from a string
              ndprep - Make neutral density filter calibration image
          refspectra - Assign wavelength reference spectra to other spectra
          reidentify - Automatically identify features in spectra
          sapertures - Set or change aperture header information
              sarith - Spectrum arithmetic
            scombine - Combine spectra having different wavelength ranges
               scopy - Select and copy apertures in different spectral formats
            sensfunc - Create sensitivity function
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
                sfit - Fit spectra and output fit, ratio, or difference
             sinterp - Interpolate a table of x,y pairs to create a spectrum
               slist - List spectrum header parameters
            specplot - Stack and plot multiple spectra
               splot - Preliminary spectral plot/analysis
            standard - Identify standard stars to be used in sensitivity calc
    

    ADDITIONAL HELP TOPICS

    
             package - Discussion and overview of package including sections on: spectral formats,
                                    dispersion coordinates, and units
    
  • noao.rv:
    
          continpars - Edit continuum subtraction parameters
            filtpars - Edit the filter function parameters
               fxcor - Radial velocities via Fourier cross correlation
            keywpars - Translate the image header keywords used in RV package
           rvcorrect - Compute radial velocity corrections
    
  • noao.twodspec:
    
           apextract - Aperture Extraction Package
            longslit - Longslit Package
    
  • noao.twodspec.apextract:
    
               apall - Extract 1D spectra (all parameters in one task)
           apdefault - Set the default aperture parameters and apidtable
             apdemos - Various tutorial demonstrations
              apedit - Edit apertures interactively
              apfind - Automatically find spectra and define apertures
               apfit - Fit 2D spectra and output the fit, difference, or ratio
           apflatten - Remove overall spectral and profile shapes from flat fields
              apmask - Create and IRAF pixel list mask of the apertures
         apnormalize - Normalize 2D apertures by 1D functions
          aprecenter - Recenter apertures
            apresize - Resize apertures
           apscatter - Fit and subtract scattered light
               apsum - Extract 1D spectra
             aptrace - Trace positions of spectra
    

    ADDITIONAL HELP TOPICS

    
        apbackground - Background subtraction algorithms
          approfiles - Profile determination algorithms
          apvariance - Extractions, variance weighting, cleaning, and noise model
             package - Package parameters and general description of package
    
  • noao.twodspec.longslit:
    
          background - Fit and subtract a line or column background
          extinction - Apply atmospheric extinction corrections to images
           fitcoords - Fit user coordinates to image coordinates
           fluxcalib - Apply flux calibration to images
            identify - Identify features
        illumination - Determine illumination calibration
          reidentify - Reidentify features
            response - Determine response calibration
          setairmass - Compute effective airmass and middle UT for an exposure
               setjd - Compute and set Julian dates in images
           transform - Transform longslit images to user coordinates
    
Many of the tasks in the NOAO packages are visible in several different packages. Although these are the same tasks and provide the same functionality, the default parameters are often set differently to account for different instruments or uses.

How to measure the limiting magnitudes

Limiting Magnitudes
The limiting magnitudes of the images are measured in three ways:
  • Number count histogram
  • 5-sigma point source detection
  • Adding fake objects
The first method is quite simple, indeed crude. The magnitudes of the objects are sorted into a histogram. The peak value of the histogram, where the number counts start to turn over, is a rough measure of the limiting magnitude of the image.

The second method is also simple. The estimated magnitude error of each source is plotted against its magnitude. In this case, the SExtractor MAG_AUTOor Kron-style magnitude is plotted. At the faint magnitudes typical of MegaCam images, the sky noise dominates the magnitude error. This means that extended objects (which have more sky in their larger Kron apertures) will be noisier for a given magnitude than compact sources. Turning this around, this means that, for a given fixed magnitude error, a point source will be fainter than an extended source. A 5-sigma detection corresponds to a S/N of 5 or, equivalently, a magnitude error of 0.198 magnitudes. Thus, to find the 5-sigma point source detection limit, one finds the faintest source whose magnitude error is 0.198 magnitudes or less. It must be a point source, therefore, its magnitude is the 5-sigma point source detection limit. A more refined approach would be to isolate the point sources, by using the half-light radius for example. In practice, the quick and dirty method gives answers that are correct to within 0.3 magnitudes or so, which is accurate enough for many purposes. This is the value that is given in the image headers for the MAGLIM keyword. This is also the value that is used in theimage search page.
The figure at the right illustrates these methods. The top panel shows the number count histogram. The number counts peak at 26 in magnitude as shown by the vertical redline.
The bottom panel shows magnitude error plotted against magnitude. The horizontal red line lies at 0.198 magnitudes. The verticalred line intersects the horizontal line at the locus of the faintest object with a magnitude error less than 0.198 magnitudes. The magnitude limit by this method is 26.6 magnitudes.
Similar plots to the figure at right are provided for each image in the MegaPipe stacks on their respective webpages.
Limiting magnitude by number counts and sigma

The final way the limiting magnitudes of the images are tested is by adding fake galaxies to the images and then trying to recover them using the same parameters used to generate the real image catalogues. The fake galaxies used were taken from the images themselves, rather than adding completely artificial galaxies. A set of 40 bright, isolated galaxies are selected out of the field and assembled into a master list. Postage stamps of these galaxies are cut out of the field. The galaxies are faded in both surface brightness and magnitude through a combination of scaling the pixel values and resampling the images.
To test the recovery rate at a given magnitude and surface brightness, galaxy postage stamps are selected from the master list, faded as described above to the magnitude and surface brightness in question, and then added to the image at random locations. SExtractor is then run on new image. The fraction of fake galaxies found gives the recovery rate at that magnitude and surface brightness, An illustration of adding the galaxies is shown at the right. The same galaxy has been added multiple times to the image. The galaxy has been faded to various magnitudes and surface brightnesses. The red boxes contain the galaxy. The boxes are labeled by mag/surface brightness. Note the galaxy ati=23, μi=25 accidentally ended up near a bright galaxy and is only partially visible. Normally of course, the galaxies are not placed in such a regular grid.Example of added galaxies
To test the false-positive rate, the original image is multiplied by -1; the noise peaks became noise troughs and vice-versa. SExtractor is run, using the same detection criteria. Since there are no real negative galaxies, all the objects thus detected are spurious.
The magnitude/surface brightness plot at the right shows the results of such simulations. The black points are real objects. The bottom edge of the black points is the locus of point-like objects. Thegreen points show the false-positive detections. The rednumbers show the percent of artificial galaxies that were recovered at that magnitude/surface brightness. The blue contour lines show the 70% and 50% completeness levels.Limiting magnitude and surface brightness
Deriving a single limiting magnitude from such a plot is somewhat difficult. The cleaner cut in the false positives seems to be in surface brightness. Extended objects become harder to detect at brighter magnitudes whereas stellar objects are detectable a magnitude or so fainter.
Note that this plot is of limited usefulness in crowded fields. In this case, an object may be missed even if it is relatively bright because it lies on top of another object. However, the objects in crowded fields are almost always stellar. This suggests the use of the DAOphot package rather than using the SExtractor catalogs provided as part of MegaPipe.
Similar graphs to the above plot are provided for each group on their respective pages.


From: http://www4.cadc-ccda.hia-iha.nrc-cnrc.gc.ca/megapipe/docs/photometry.html