astro.filerecipes package

Submodules

astro.filerecipes.AstromCorr module


This recipe is used to derive the astrometric correction for a mosaic of detectors. First the linear terms of the entire mosaic are determined, then optionally each detector’s offset from the reference pixel can be determined. In both steps, the headers of the original files are updated as necessary.

The recipe always takes as input a list of RawScienceFrames. Optionally, applying the changes to the headers can be skipped, but if determination of the CRPIXn adjustments is requested, the change to the mosaic MUST be applied. The detection threshhold for extraction can also be specified.


Mandatory inputs :

-i : list of input RawScienceFrames

Configuration parameter (optional) :

-nc : don’t commit determination to headers (mandatory for the
mosaic if adjusting the CRPIXn values)
-ac : determine the adjustment to the CRPIXn offsets for the
individual detectors
-dt : SExtractor DETECT_THRESH (usually between 100 and 150, but
below 50 is sometimes useful

Example of use :

awe AstromCorr.py -i raw*.fits -ac -dt 100

class astro.filerecipes.AstromCorr.AstromCorr(filenames=[], commit=True, adjust_crpix=False, detect_thresh=120, iterations=10)

Bases: object

check_preconditions()
cleanup()
correct_chips()
correct_mosaic()
derive_catalogs()
execute()
make_rawframes()
exception astro.filerecipes.AstromCorr.AstromCorrError(message)

Bases: common.log.Error.Error

astro.filerecipes.AstromCorr.main()
astro.filerecipes.AstromCorr.parse_command_line()
astro.filerecipes.AstromCorr.usage()

astro.filerecipes.Astrometry_pickles module


CATEGORY : data processing, astrometric pipeline.

PURPOSE : derive the astrometric calibration for a given frame.


This task is used to derive the astrometric calibration for a given frame.

The recipe takes as input a list of file names. The task is executed by calling its ‘execute’ method.


class astro.filerecipes.Astrometry_pickles.AstrometricParametersMaker(args)

Bases: object

check_preconditions()
instantiate_reduceds()
load_pickles()
make_astromparams()
save_pickle(astrom_params)
astro.filerecipes.Astrometry_pickles.main()

astro.filerecipes.BPZSourceCollectionTask module

class astro.filerecipes.BPZSourceCollectionTask.BPZSourceCollectionTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : create Bayesian photometric redshifts for (KIDS)
SourceCollections
execute()
get_process_time()

Return the processing time of this task

make_sourcecollection()
parser = Recognized options for the astro.filerecipes.BPZSourceCollectionTask OptionsParser are: "s" or "sc" (default: ) "n" or "name" (default: ) "u" or "du" (default: 0.0) "g" or "dg" (default: 0.0) "r" or "dr" (default: 0.0) "i" or "di" (default: 0.0)
target

alias of astro.main.sourcecollection.SourceCollection.SourceCollection

exception astro.filerecipes.BPZSourceCollectionTask.BPZSourceCollectionTaskError(message)

Bases: common.log.Error.Error

astro.filerecipes.BPZSourceCollectionTask.main()
astro.filerecipes.BPZSourceCollectionTask.test()

astro.filerecipes.Bias module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid master bias (Calfile 541).

FULFILLS : requirement 541 (Category I)


This recipe is used to derive a valid master bias (Calfile 541) for one particular chip.

The recipe always takes as input a list of raw bias frames, and a value for the read noise of the chip under consideration. Optionally, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


Mandatory inputs :

-i : list of input raw bias frames -r : input read noise value

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Bias.py -i raw1.fits raw2.fits -r 1.0 -oc 2

astro.filerecipes.Bias.configure_process_parameters(masterbias, overscan)
astro.filerecipes.Bias.instantiate_objects(raw_bias_list, read_noise_value)
astro.filerecipes.Bias.main()
astro.filerecipes.Bias.make_masterbias(raw_bias_frames, read_noise, overscan=6)

This routine derives Calfile 541.

Inputs:
raw_bias_frames – list of raw bias frames read_noise – a read noise calibration overscan – overscan correction mode
astro.filerecipes.Bias.make_using_pickles(raw_bias_list, overscan)
astro.filerecipes.Bias.parse_command_line()
astro.filerecipes.Bias.usage()

astro.filerecipes.Bias_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid master bias (Calfile 541).

FULFILLS : requirement 541 (Category I)


This recipe is used to derive a valid master bias (Calfile 541) for one or more chips.

The recipe always takes as input a list of raw bias frames, and a value for the read noise of the chip under consideration. Optionally, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


class astro.filerecipes.Bias_pickles.BiasFrameMaker(args)

Bases: object

configure_process_parameters(masterbias)
load_pickles()
make_biasframe(raw_bias_frames, chip_id)

This routine derives Calfile 541.

Inputs:
raw_bias_frames – list of raw bias frames read_noise – a read noise calibration overscan – overscan correction mode
make_biasframes()
save_as_pickle()
astro.filerecipes.Bias_pickles.main()

astro.filerecipes.Coadd module


CATEGORY : data processing, image pipeline.

PURPOSE : coadding calibrated, regridded science images

FULFILLS : sequence 636


Requires Swarp.

This recipe is used to derive coadded images, for any number of chips. A catalog and measure for the seeing are also derived. The recipe always takes as input a list of regridded science frames, and optionally target. Regridded individual weight frames (Seqfile 633) associated with the regridded science images are assumed to have filenames equal to that of the regridded science image, but with the added suffix ‘weight’.


Mandatory inputs:

-i : a list of input regridded science frames
Optional inputs:
-t : a target filename for the coadded image

Example of use:

awe Coadd.py -i raw1.regridded.fits raw2.regridded.fits
raw3.regridded.fits -t CDF4_B_small.fits

astro.filerecipes.Coadd.instantiate_objects(regridded_list)
astro.filerecipes.Coadd.main()
astro.filerecipes.Coadd.make_coadd(regridded_frames, target_name=None)

Coadd regridded frames.

astro.filerecipes.Coadd.parse_command_line()
astro.filerecipes.Coadd.usage()

astro.filerecipes.Cold_Pixels module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid cold pixel map (Calfile 535).

FULFILLS : requirement 535 (Category I)


This recipe is used to derive a valid cold pixel map (Calfile 535) for one particular chip and filter.

The recipe always takes as input either a master dome flat frame, or a master twilight flat frame (Calfile 542/Calfile543, respectively). Optionally, two process parameters can be set : a lower flagging threshold, and a higher flagging threshold.


Mandatory inputs :

-i : input flat (Calfile 542/ Calfile 543) or -dp : input domeflat pickle file

Configuration parameters (optional) :

-low : lower flagging threshold (default : 0.96) -hig : higher flagging threshold (default : 1.04)
Switch :
-use_pickles (switch between single filename of Pickle files)

Example of use :

awe Cold_Pixels.py -i masterdomeflat.fits -low 0.50 -hig 3.0

astro.filerecipes.Cold_Pixels.configure_process_parameters(coldpixels, low_thresh, high_thresh)
astro.filerecipes.Cold_Pixels.construct_filename(flat)

This routine constructs a file name for the output Calfile based on the attributes of the input flat frame.

astro.filerecipes.Cold_Pixels.instantiate_objects(flat_name)
astro.filerecipes.Cold_Pixels.main()
astro.filerecipes.Cold_Pixels.make_coldpixels(flat, low_thresh=0.94, high_thresh=1.06)

This routine derives Calfile 535.

Inputs :
flat – a master dome flat/master twilight flat (Calfile 542/543) low_thresh – lower level flagging threshold high_thresh – higher level flagging threshold
astro.filerecipes.Cold_Pixels.make_normal(flat_name, low_thresh, high_thresh)
astro.filerecipes.Cold_Pixels.make_using_pickles(domeflat_pickle, low_thresh, high_thresh)
astro.filerecipes.Cold_Pixels.parse_command_line()
astro.filerecipes.Cold_Pixels.usage()

astro.filerecipes.Cold_Pixels_Plot module

astro.filerecipes.Cold_Pixels_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid cold pixel map (Calfile 535).

FULFILLS : requirement 535 (Category I)


This recipe is used to derive a valid cold pixel map (Calfile 535) for one particular chip and filter.

The recipe always takes as input either a master dome flat frame, or a master twilight flat frame (Calfile 542/Calfile543, respectively). Optionally, two process parameters can be set : a lower flagging threshold, and a higher flagging threshold.


class astro.filerecipes.Cold_Pixels_pickles.ColdPixelMapMaker(args)

Bases: object

configure_process_parameters(coldpixels)
load_pickles()
make_coldpixelmap(flat)

This routine derives Calfile 535.

Inputs:
flat – a master dome flat/master twilight flat (Calfile 542/543) low_thresh – lower level flagging threshold high_thresh – higher level flagging threshold
make_coldpixelmaps()
save_pickle()
astro.filerecipes.Cold_Pixels_pickles.main()

astro.filerecipes.Dark_Current module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid measurement of the dark count rate
(Calfile 531) and the particle event rate (Calfile 532).

FULFILLS : requirements 531/532 (Category I)


This recipe is used to derive a valid measurement of the dark count rate and the particle event rate (Calfile 531/Calfile 532) for one particular chip.

The recipe takes as input exactly three raw dark frames, and a master bias. Optionally, a hot pixel map and an overscan correction mode can be set. The default is to apply per-row overscan X correction.


Mandatory inputs :

-i : list of input raw dark frames (exactly three frames) -b : master bias (Calfile 541)

Optional parameters :

-h : hot pixel map (Calfile 522)

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Dark_Current.py -i dark1.fits dark2.fits dark3.fits
-b cal541.fits -h hotpixels.fits -oc 2

astro.filerecipes.Dark_Current.configure_process_parameters(darkcurrent, overscan)
astro.filerecipes.Dark_Current.instantiate_objects(raw_dark_list, bias_name, hot_name=None)
astro.filerecipes.Dark_Current.main()
astro.filerecipes.Dark_Current.make_darkcurrent(raw_dark_frames, bias, hot=None, overscan=6)

This routine derives Calfile 531/Calfile 532.

Inputs :
raw_dark_frames – list of raw dark frames (exactly three frames) bias – a master bias (Calfile 541) overscan – overscan correction mode
astro.filerecipes.Dark_Current.make_normal(raw_dark_list, bias_name, hot_name, overscan)
astro.filerecipes.Dark_Current.make_using_pickles(raw_dark_list, overscan)
astro.filerecipes.Dark_Current.parse_command_line()
astro.filerecipes.Dark_Current.usage()
astro.filerecipes.Dark_Current.write_to_disk(darkcurrent)

astro.filerecipes.Dark_Current_Plot module

astro.filerecipes.Dark_Current_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid measurement of the dark count rate
(Calfile 531) and the particle event rate (Calfile 532).

FULFILLS : requirements 531/532 (Category I)


This recipe is used to derive a valid measurement of the dark count rate and the particle event rate (Calfile 531/Calfile 532) for one particular chip.

The recipe takes as input exactly three raw dark frames, and a master bias. Optionally, a hot pixel map and an overscan correction mode can be set. The default is to apply per-row overscan X correction.


class astro.filerecipes.Dark_Current_pickles.DarkCurrentMaker(args)

Bases: object

configure_process_parameters(darkcurrent)
load_pickles()
make_darkcurrent(raw_dark_frames, chip_id)

This routine derives Calfile 531/Calfile 532.

Inputs :
raw_dark_frames – list of raw dark frames (exactly three frames) bias – a master bias (Calfile 541) overscan – overscan correction mode
make_darkcurrents()
save_pickle()
astro.filerecipes.Dark_Current_pickles.main()

astro.filerecipes.Dome_Flat module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid domeflat frame (Calfile 542).

FULFILLS : requirement 542 (Category I)


This recipe is used to derive a valid domeflat frame (Calfile 542) for one particular chip and filter.

The recipe always takes as input a list of raw domeflat frames, a master bias and a measurement of the gain of the chip under consideration. In addition, a hot pixel map (Calfile 522) and a cold pixel map (Calfile 535) can be provided. Optionally, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


Mandatory inputs :

-i : list of input raw domeflat frames -b : master bias (Calfile 541) -g : gain (range 0.1-10.0)

Optional inputs :

-c : cold pixel map (Calfile 535) -h : hot pixel map (Calfile 522)

Switch :

-use_pickles : search the CWD for Python pickle files to serve as
input

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Dome_Flat.py -i raw1.fits raw2.fits raw3.fits -b bias.fits -g 3.0
-c cal535.fits.gz -h cal522.fits.gz -oc 2

astro.filerecipes.Dome_Flat.configure_process_parameters(domeflat, overscan)
astro.filerecipes.Dome_Flat.instantiate_objects(raw_domeflat_list, bias_name, gain_value, cold_name, hot_name)
astro.filerecipes.Dome_Flat.main()
astro.filerecipes.Dome_Flat.make_domeflat(raw_domeflat_frames, bias, gain, cold=None, hot=None, overscan=6)

This routine derives Calfile 542.

Inputs :
raw_domeflat_frames – a list of raw domeflat frames bias – a master bias (Calfile 541) gain – a gain calibration (Calfile 523) cold – a cold pixel map (Calfile 535) hot – a hot pixel map (Calfile 522) overscan – overscan correction mode
astro.filerecipes.Dome_Flat.make_normal(raw_domeflat_list, bias_name, gain_value, cold_name, hot_name, overscan)
astro.filerecipes.Dome_Flat.make_using_pickles(raw_domeflat_list, overscan)
astro.filerecipes.Dome_Flat.parse_command_line()
astro.filerecipes.Dome_Flat.usage()

astro.filerecipes.Dome_Flat_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid domeflat frame (Calfile 542).

FULFILLS : requirement 542 (Category I)


This recipe is used to derive a valid domeflat frame (Calfile 542) for one particular chip and filter.

The recipe always takes as input a list of raw domeflat frames, a master bias and a measurement of the gain of the chip under consideration. In addition, a hot pixel map (Calfile 522) and a cold pixel map (Calfile 535) can be provided. Optionally, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


class astro.filerecipes.Dome_Flat_pickles.DomeFlatMaker(args)

Bases: object

configure_process_parameters(domeflat)
load_pickles(filter_id)
make_domeflat(raw_domeflat_frames)

This routine derives Calfile 542.

Inputs :
raw_domeflat_frames – a list of raw domeflat frames bias – a master bias (Calfile 541) gain – a gain calibration (Calfile 523) cold – a cold pixel map (Calfile 535) hot – a hot pixel map (Calfile 522) overscan – overscan correction mode
make_domeflats()
save_pickle(filter_id)
astro.filerecipes.Dome_Flat_pickles.main()

astro.filerecipes.Focus module

Example focus file

r336362_1.fits 493.03 2151.41 3.37 r336362_2.fits 958.64 1619.82 3.45 r336362_3.fits 1333.67 2181.74 2.85 r336362_4.fits 1258.37 2583.8 3.78

class astro.filerecipes.Focus.CCD(instrument_id, chip_id)

Bases: object

chip_to_mosaic(x, y)
plot_regions(panel)

Plot trim, pre- and overscan regions on the specified panel

plot_trim_region(panel)
region_to_mosaic(region)

region in [[x0,x1], [y0,y1]] format

set_cd_matrix()
set_crpix()
set_crpix_mosaic()
set_dummy_frame()

Set a dummy frame to find its regions.

set_pc_matrix()
set_pixelsize()
set_regions()

Populate the regions dictionary. Keys are “trim”, and optionally “prscx”, “ovscx”, “prscy” and “ovscy”

class astro.filerecipes.Focus.CCDParameters

Bases: object

BBOX_COLOR = 'c'
OVSC_COLOR = 'b'
PRSC_COLOR = 'm'
TRIM_COLOR = 'k'
class astro.filerecipes.Focus.FocusTilt(filenames=[], instrument='OCAM', show=False)

Bases: object

calculate_raw_tilts()

Calculate the raw tilts (with contributions from both detector plane tilt and rotator plane tilt).

check_preconditions()
disentangle_tilts()

Disentangle the tilt of the detector plane and that of the rotator plane from the two input tilts at rotator 0 and 180 degrees.

Consider two measurements of a sum of normal vectors:

n1 = nr + nd n2 = nr + nd’

where the reference coordinate system is defined by the detector plane and:

n1 = normal vector of focal plane w.r.t. detector plane n2 = normal vector of focal plane w.r.t. detector plane after

180 degree rotation of focal plane w.r.t detector plane

nr = normal vector of focal plane w.r.t. rotator plane nd = normal vector of rotator plane w.r.t. detector plane nd’ = normal vector nd rotated 180 degrees w.r.t. rotator plane

and:

n1 = [a1,b1,c1] ; n2 = [a2,b2,c2] nr = [ar,br,cr] ; nr’ = [-ar,-br,cr] ; nd = [ad,bd,cd]

For a unit normal vector:

a**2 + b**2 + c**2 = 1 or c = (1 - a**2 - b**2)**0.5

Solving for nd from the initial measurements:

nd = (n1 + n2 + nr + nr’) / 2

ad = (a1 + a2) / 2 bd = (b1 + b2) / 2 cd = (1 - ad**2 - bd**2)**0.5

where cd comes from the definition of a unit normal vector. Similarly, the components of nr are obtained via:

nr - nr’ = n1 - n2

ar = (a1 - a2) / 2 br = (b1 - b2) / 2 cr = (1 - ar**2 - br**2)**0.5

where cr comes from the definition of a unit normal vector.

load_data()
make()
show_tilt()
class astro.filerecipes.Focus.MIDASFile(filename)

Bases: object

read()
class astro.filerecipes.Focus.Mosaic(instrument_id, center=True, prescan=True, overscan=True)

Bases: object

A Mosaic contains one or more CCDs

chip_to_mosaic(chip_id, xpos, ypos)
create(show=True, filename='')
plot_mosaic_positions(xpos, ypos)
set_ccds()
class astro.filerecipes.Focus.Tilt(instrument_id, data, invert=False)

Bases: object

make_fit()

Use common.math.lfit to do a least squares fit

set_angles_from_coeffs_rtp()

Consider the cartesian representation of a plane:

ax + by + cz + d = 0

Divide by c to get: (a/c)*x + (b/c)*y + z + (d/c) = 0

This is a plane which has a normal vector: N = {a/c,b/c,1}.

We perform a least-squares fit to: z = - (a/c)*x - (b/c)*y - (d/c)

The resulting fit coefficients we call A, B, D: A = -a/c B = -b/c D = -d/c

Normal vector N for our plane is then N = {-A,-B,1}

The normal vector is calculated as above in cartesian coordinates, additionally it is converted to spherical coordinates.

set_mosaic_positions_and_offsets()

What is being fitted are the coefficients a1, a2, a3 in the following equation for a plane:

z = a1*x + a2*y + a3

where x,y are the positions in the mosaic plane (“positions”) and z are the offsets of the secondary mirror (“offsets”), all in units of mm.

show()

3D plot showing x (mm), y (mm) and focal offset (mm).

class astro.filerecipes.Focus.TiltParameters

Bases: object

M2_step = 1
astro.filerecipes.Focus.cartesian_to_spherical(x, y, z)

Returns tuple of r, theta, phi where r = radius theta = inclination in degrees from Z phi = position angle in degrees from +X to +Y

astro.filerecipes.Focus.main()
astro.filerecipes.Focus.parse_command_line()
astro.filerecipes.Focus.poly_x_y(x, i)

This function can be used as input for lfit

x is a list of (x,y) tuples

astro.filerecipes.Focus.usage()

astro.filerecipes.Fringe_Flat module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid fringe frame (Calfile 545).

FULFILLS : requirement 545 (Category I)


This recipe is used to derive a valid fringe frame (Calfile 545) for one particular chip and filter.

The recipe always takes as input a list of raw science frames, a master bias (Calfile 541), and a master flat (Calfile 546). In addition to these, a hot pixel map (Calfile 522) and a cold pixel map (Calfile 535) can also be provided. As an option, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


Mandatory inputs :

-i : list of input raw science frames -b : master bias (Calfile 541) -f : master flat (Calfile 546)

Optional inputs :

-c : cold pixel map (Calfile 535) -h : hot pixel map (Calfile 522)

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Fringe_Flat.py -i raw1.fits raw2.fits raw3.fits -b cal541.fits
-f cal546.fits -c cal535.fits.gz -h cal522.fits.gz -oc 2

astro.filerecipes.Fringe_Flat.configure_process_parameters(fringeframe, overscan)
astro.filerecipes.Fringe_Flat.construct_filename(raw_science)
astro.filerecipes.Fringe_Flat.instantiate_objects(raw_science_list, bias_name, flat_name, cold_name, hot_name)
astro.filerecipes.Fringe_Flat.main()
astro.filerecipes.Fringe_Flat.make_fringeframe(raw_science_frames, bias, flat, cold=None, hot=None, overscan=6)

This routine derives Calfile 545.

Inputs :
raw_science_frames – a list of raw science frames bias – a master bias (Calfile 541) flat – a master flat (Calfile 546) cold – a cold pixel map (Calfile 535) hot – a hot pixel map (Calfile 522) overscan – overscan correction mode
astro.filerecipes.Fringe_Flat.parse_command_line()
astro.filerecipes.Fringe_Flat.usage()

astro.filerecipes.Fringe_Flat_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid fringe frame (Calfile 545).

FULFILLS : requirement 545 (Category I)


This recipe is used to derive a valid fringe frame (Calfile 545) for one particular chip and filter.

The recipe always takes as input a list of raw science frames, a master bias (Calfile 541), and a master flat (Calfile 546). In addition to these, a hot pixel map (Calfile 522) and a cold pixel map (Calfile 535) can also be provided. As an option, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


Mandatory inputs :

-i : list of input raw science frames -b : master bias (Calfile 541) -f : master flat (Calfile 546)

Optional inputs :

-c : cold pixel map (Calfile 535) -h : hot pixel map (Calfile 522)

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Fringe_Flat.py -i raw1.fits raw2.fits raw3.fits -b cal541.fits
-f cal546.fits -c cal535.fits.gz -h cal522.fits.gz -oc 2

class astro.filerecipes.Fringe_Flat_pickles.FringeFrameMaker

Bases: object

configure_process_parameters(fringeframe)
load_pickles()
make_fringeframe(chipd_id)

This routine derives Calfile 545.

Inputs :
raw_science_frames – a list of raw science frames bias – a master bias (Calfile 541) flat – a master flat (Calfile 546) cold – a cold pixel map (Calfile 535) hot – a hot pixel map (Calfile 522) overscan – overscan correction mode
make_fringeframes()
save_pickle()
astro.filerecipes.Fringe_Flat_pickles.main()

astro.filerecipes.GAaPSourceCollectionTask module

Recipe to create SourceCollections containing results of running the Gaussian aperture and PSF (GaaP) code.

See Konrad Kuijken, Astronomy and Astrophysics, Volume 482, Issue 3, 2008, pp.1053-1067

http://adsabs.harvard.edu/abs/2008A%26A…482.1053K http://arxiv.org/abs/astro-ph/0610606

class astro.filerecipes.GAaPSourceCollectionTask.GAaPSourceCollectionTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : run Gaussian aperture and PSF (GaaP) code and store resulting
flux values in a SourceCollection
cleanup_gaap_tempfiles()
execute()
get_process_time()

Return the processing time of this task

make_gaap(sc, gauss)

Find/make a GAaP catalog for a SourceCollection.

See GAaPAttributeCalculator.py for details.

make_sourcecollection()
parser = Recognized options for the astro.filerecipes.GAaPSourceCollectionTask OptionsParser are: "s" or "sc" (default: ) "g" or "gauss" (default: ) "k" or "kermap" (default: ) "c" or "coadd" (default: ) "w" or "weight" (default: ) "n" or "name" (default: )
set_config()

Set instrument specific default config properties

target

alias of astro.main.sourcecollection.SourceCollection.SourceCollection

exception astro.filerecipes.GAaPSourceCollectionTask.GAaPSourceCollectionTaskError(message)

Bases: common.log.Error.Error

astro.filerecipes.GAaPSourceCollectionTask.main()
astro.filerecipes.GAaPSourceCollectionTask.test()

astro.filerecipes.Gain module

astro.filerecipes.Gain_Plot module

astro.filerecipes.Gain_pickles module

astro.filerecipes.Hot_Pixels module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid hot pixel map (Calfile 522).

FULFILLS : requirement 522 (Category I)


This recipe is used to derive a valid hot pixel map (Calfile 521) for one particular chip.

The recipe always takes as input a master bias (Calfile 541). Optionally, two process parameters can be set : the maximum number of iterations used in deriving the read noise, and the rejection threshold in sigma. The defaults for these parameters are 5 and 5.0, respectively.


Mandatory inputs :

-i : input master bias (Calfile 541)

Configuration parameters (optional) :

-max : maximum iterations (default : 5) -rej : rejection threshold (default : 5.0 sigma)

Example of use :

awe Hot_Pixels.py -i cal541.fits -max 10 -rej 3.0

astro.filerecipes.Hot_Pixels.configure_process_parameters(hotpixels, rej_thresh, max_iter)
astro.filerecipes.Hot_Pixels.instantiate_objects(bias_name)
astro.filerecipes.Hot_Pixels.main()
astro.filerecipes.Hot_Pixels.make_hotpixels(bias, rej_thresh=5.0, max_iter=5)

This routine derives Calfile 522.

Inputs :
bias – a master bias (Calfile 541) rej_thresh – rejection threshold in sigma max_iter – maximum iterations
astro.filerecipes.Hot_Pixels.make_normal(bias_name, rej_thresh, max_iter)
astro.filerecipes.Hot_Pixels.make_using_pickles(rej_thresh, max_iter)
astro.filerecipes.Hot_Pixels.parse_command_line()
astro.filerecipes.Hot_Pixels.parse_command_line_pickle()
astro.filerecipes.Hot_Pixels.usage()

astro.filerecipes.Hot_Pixels_Plot module

astro.filerecipes.Hot_Pixels_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid hot pixel map (Calfile 522).

FULFILLS : requirement 522 (Category I)


This recipe is used to derive a valid hot pixel map (Calfile 521) for one particular chip.

The recipe always takes as input a master bias (Calfile 541). Optionally, two process parameters can be set : the maximum number of iterations used in deriving the read noise, and the rejection threshold in sigma. The defaults for these parameters are 5 and 5.0, respectively.


class astro.filerecipes.Hot_Pixels_pickles.HotPixelMapMaker(args)

Bases: object

configure_process_parameters(hotpixels)
load_pickles()
make_hotpixelmap(bias)

This routine derives Calfile 522.

Inputs :
bias – a master bias (Calfile 541)
make_hotpixelmaps()
save_pickle()
astro.filerecipes.Hot_Pixels_pickles.main()

astro.filerecipes.Illumination_Correction module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid illumination correction frame (Calfile
548).

FULFILLS : requirement 548 (Category II)


This recipe is used to derive an illumination correction frame for one particular chip and filter.

The recipe always takes as input a fit of the illumination correction over the full focal plane for a given filter. A chip name should also be specified.


Mandatory inputs :

-i : input illumination correction fit -c : chip name

Example of use :

awe Illumination_Correction.py -i illum.fits -c ccd50

astro.filerecipes.Illumination_Correction.construct_filename(illumcorr, chip)
astro.filerecipes.Illumination_Correction.instantiate_objects(illum_name, chip_name)
astro.filerecipes.Illumination_Correction.main()
astro.filerecipes.Illumination_Correction.make_illuminationcorrectionframe(illumcorr, chip)

This routine derives Calfile 548.

Inputs :
illumcorr – the illumination correction fit chip – the chip for which to make the frame
astro.filerecipes.Illumination_Correction.parse_command_line()
astro.filerecipes.Illumination_Correction.usage()

astro.filerecipes.Illumination_Correction_Verify module


CATEGORY : diagnostic tools, health check.

PURPOSE : providing a quick check on the presence of illumination
variations.

FULFILLS : (initial) verification requirement 548 (Category III)


This recipe is used to quickly check the presence and magnitude of illumination variations across the focal plane of the instrument for one particular filter.

The recipe always takes as input a list of de-biased, flatfielded and astrometrically calibrated science frames of one image. The recipe also needs a standard star catalog as input. Optionally, a table of color-transformation coefficients can also be provided.

To every separate input frame, a fit is made to the zeropoints of the standard stars found on that frame. The output of the recipe consists of a file containing the fit parameters found for every input frame, as well as a set of illumination correction frames that can be inspected visually.

Note that this quick-and-dirty recipe only produces sensible results if the input frames are uniformly and densely covered with standard stars.


Mandatory inputs :

-i : list of input frames -s : standard star catalog

Additional input if color-transformation coefficients are required :

-t : transformation table (Calfile 565/565C)

Example of use :

awe Illumination_Correction_Verify.py -i sci01.fits sci02.fits sci03.fits -s cal569.cat -t cal565.cat

astro.filerecipes.Illumination_Correction_Verify.construct_filename(illummap, chip)
astro.filerecipes.Illumination_Correction_Verify.derive_catalogs(sciframes, refcat, transform)

This routine derives the catalogs from the input frames that will be used in the fit to the illumination variation.

astro.filerecipes.Illumination_Correction_Verify.do_illuminationcorrection_check(sciframes, refcat, transform)

This routine makes a quick-and-dirty determination of the illumination variation.

Inputs :
sciframes – the input science frames refcat – the standard star catalog (Calfile 569(E)) transform – transformation table (Calfile 565/565C)
astro.filerecipes.Illumination_Correction_Verify.instantiate_objects(sciframe_list, refcat_name, transform_name)
astro.filerecipes.Illumination_Correction_Verify.main()
astro.filerecipes.Illumination_Correction_Verify.parse_command_line()
astro.filerecipes.Illumination_Correction_Verify.usage()

astro.filerecipes.Illumination_Correction_Verify_pickles module


CATEGORY : diagnostic tools, health check.

PURPOSE : providing a quick check on the presence of illumination
variations.

FULFILLS : (initial) verification requirement 548 (Category III)


This recipe is used to quickly check the presence and magnitude of illumination variations across the focal plane of the instrument for one particular filter.

The recipe always takes as input a list of de-biased, flatfielded and astrometrically calibrated science frames of one image. The recipe also needs a standard star catalog (cal569/569E) as input. Optionally, a table of color-transformation coefficients (cal565/565C) can also be provided.

To every separate input frame, a fit is made to the zeropoints of the standard stars found on that frame. The output of the recipe consists of a file containing the fit parameters found for every input frame, as well as a set of illumination correction frames that can be inspected visually.

Note that this quick-and-dirty recipe only produces sensible results if the input frames are uniformly and densely covered with standard stars.


class astro.filerecipes.Illumination_Correction_Verify_pickles.IlluminationCorrectionMaker(args)

Bases: object

derive_catalogs()

This routine derives the catalogs from the input frames that will be used in the fit to the illumination variation.

load_pickles()
make_illuminationcorrection()

This routine makes a quick-and-dirty determination of the illumination variation.

Inputs :
sciframes – the input science frames refcat – the standard star catalog (Calfile 569(E)) transform – transformation table (Calfile 565/565C)
save_pickle()
astro.filerecipes.Illumination_Correction_Verify_pickles.main()

astro.filerecipes.Illumination_Correction_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid illumination correction frame (Calfile
548).

FULFILLS : requirement 548 (Category II)


This recipe is used to derive an illumination correction frame for one particular chip and filter.

The recipe always takes as input a fit of the illumination correction over the full focal plane for a given filter. A chip name should also be specified.


Mandatory inputs :

-i : input illumination correction fit -c : chip name

Example of use :

awe Illumination_Correction.py -i illum.fits -c ccd50

astro.filerecipes.Illumination_Correction_pickles.construct_filename(illumcorr, chip)
astro.filerecipes.Illumination_Correction_pickles.instantiate_objects(illum_name, chip_name)
astro.filerecipes.Illumination_Correction_pickles.main()
astro.filerecipes.Illumination_Correction_pickles.make_illuminationcorrectionframe(illumcorr, chip)

This routine derives Calfile 548.

Inputs :
illumcorr – the illumination correction fit chip – the chip for which to make the frame
astro.filerecipes.Illumination_Correction_pickles.parse_command_line()
astro.filerecipes.Illumination_Correction_pickles.usage()

astro.filerecipes.Linearity_Plot module

astro.filerecipes.Master_Flat module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid masterflat frame (Calfile 546).

FULFILLS : requirement 546 (Category I)


This recipe is used to derive a valid masterflat frame (Calfile 546) for one particular chip and filter. The recipe always takes as input a master domeflat frame (Calfile 542), and a master twilightflat frame (Calfile 543). Optionally, a NightSkyFlatFrame (Calfile 544), a hot pixel map and a cold pixel map can be provided.

Mandatory inputs :

-d : master domeflat (Calfile 542) -t : master twilightflat (Calfile 543)

Optional inputs :

-n : nightsky flat (Calfile 544) -c : cold pixel map (Calfile 535) -h : hot pixel map (Calfile 522)

Example of use :

awe Master_Flat.py -d cal542.fits -t cal543.fits -n cal544.fits
-c cal535.fits.gz -h cal522.fits.gz

astro.filerecipes.Master_Flat.configure_process_parameters(masterflat)
astro.filerecipes.Master_Flat.instantiate_objects(domeflat_name, twilightflat_name, nightskyflat_name, cold_name, hot_name)
astro.filerecipes.Master_Flat.main()
astro.filerecipes.Master_Flat.make_masterflat(domeflat, twilightflat, nightskyflat=None, cold=None, hot=None)

This routine derives Calfile 546.

Inputs :
domeflat – a master domeflat (Calfile 542) twilightflat – a master twilightflat (Calfile 543) nightskyflat – (optional) a nightskyflat (Calfile 544) cold – (optional) a cold pixel map (Calfile 535) hot – (optional) a hot pixel map (Calfile 522)
astro.filerecipes.Master_Flat.make_normal(domeflat_name, twilightflat_name, nightskyflat_name, cold_name, hot_name)
astro.filerecipes.Master_Flat.make_using_pickles(domeflat_pickle, twilightflat_pickle, nightskyflat_pickle)
astro.filerecipes.Master_Flat.parse_command_line()
astro.filerecipes.Master_Flat.usage()

astro.filerecipes.Master_Flat_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid masterflat frame (Calfile 546).

FULFILLS : requirement 546 (Category I)


This recipe is used to derive a valid masterflat frame (Calfile 546) for one particular chip and filter. The recipe always takes as input a master domeflat frame (Calfile 542), and a master twilightflat frame (Calfile 543). Optionally, a NightSkyFlatFrame (Calfile 544), a hot pixel map and a cold pixel map can be provided.

class astro.filerecipes.Master_Flat_pickles.MasterFlatFrameMaker(args)

Bases: object

configure_process_parameters(masterflat)
load_pickles()
make_masterflat(chip)

This routine derives Calfile 546.

Inputs :
domeflat – a master domeflat (Calfile 542) twilightflat – a master twilightflat (Calfile 543) nightskyflat – (optional) a nightskyflat (Calfile 544) cold – (optional) a cold pixel map (Calfile 535) hot – (optional) a hot pixel map (Calfile 522)
make_masterflats()
save_pickle(filter_id)
astro.filerecipes.Master_Flat_pickles.main()

astro.filerecipes.Monitoring module

astro.filerecipes.Nightsky_Flat module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid nightskyflat frame (Calfile 544).

FULFILLS : requirement 544 (Category I)


This recipe is used to derive a valid nightskyflat frame (Calfile 544) for one particular chip and filter.

The recipe always takes as input a list of raw science frames, a master bias (Calfile 541), and a master flat (Calfile 546). In addition to these, a hot pixel map (Calfile 522) and a cold pixel map (Calfile 535) can also be provided. As an option, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


Mandatory inputs :

-i : list of input raw science frames -b : master bias (Calfile 541) -f : master flat (Calfile 546)

Optional inputs :

-c : cold pixel map (Calfile 535) -h : hot pixel map (Calfile 522)

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Nightsky_Flat.py -i raw1.fits raw2.fits raw3.fits -b cal541.fits
-f cal546.fits -c cal535.fits.gz -h cal522.fits.gz -oc 2

astro.filerecipes.Nightsky_Flat.configure_process_parameters(nightskyflat, overscan)
astro.filerecipes.Nightsky_Flat.construct_filename(raw_science)
astro.filerecipes.Nightsky_Flat.get_argument(cmndline, key, conversion=<class 'str'>)
astro.filerecipes.Nightsky_Flat.get_object(cmndline, key, obj)
astro.filerecipes.Nightsky_Flat.main()
astro.filerecipes.Nightsky_Flat.make_nightskyflat(raw_science_frames, bias, flat, cold=None, hot=None, overscan=6)

This routine derives Calfile 544.

Inputs :
raw_science_frames – a list of raw science frames bias – a master bias (Calfile 541) flat – a master flat (Calfile 546) cold – a cold pixel map (Calfile 535) hot – a hot pixel map (Calfile 522) overscan – overscan correction mode
astro.filerecipes.Nightsky_Flat.parse_command_line(cmndline)
astro.filerecipes.Nightsky_Flat.usage()

astro.filerecipes.Nightsky_Flat_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid nightskyflat frame (Calfile 544).

FULFILLS : requirement 544 (Category I)


This recipe is used to derive a valid nightskyflat frame (Calfile 544) for one particular chip and filter.

The recipe always takes as input a list of raw science frames, a master bias (Calfile 541), and a master flat (Calfile 546). In addition to these, a hot pixel map (Calfile 522) and a cold pixel map (Calfile 535) can also be provided. As an option, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


Mandatory inputs :

-i : list of input raw science frames -b : master bias (Calfile 541) -f : master flat (Calfile 546)

Optional inputs :

-c : cold pixel map (Calfile 535) -h : hot pixel map (Calfile 522)

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Nightsky_Flat.py -i raw1.fits raw2.fits raw3.fits -b cal541.fits
-f cal546.fits -c cal535.fits.gz -h cal522.fits.gz -oc 2

class astro.filerecipes.Nightsky_Flat_pickles.NightSkyFlatMaker(args)

Bases: object

configure_process_parameters(nightskyflat)
load_pickles()
make_nightskyflat(chip_id)

This routine derives Calfile 544.

Inputs :
raw_science_frames – a list of raw science frames bias – a master bias (Calfile 541) flat – a master flat (Calfile 546) cold – a cold pixel map (Calfile 535) hot – a hot pixel map (Calfile 522) overscan – overscan correction mode
make_nightskyflats()
save_pickle()
astro.filerecipes.Nightsky_Flat_pickles.main()

astro.filerecipes.PSF_Anisotropy module


CATEGORY : Health check

PURPOSE : Detect PSF Anisotropy

FULFILLS : Requirement 554 (Category III)


This filerecipe makes 5 plots of Sextractor parameters: 1) Bars of length ELLIPTICITY (times a scale factor) and angle w.r.t.

x-axis Theta.
  1. MAG_ISO vs. FLUX_RADIUS (half-light radius)
  2. FWHM_IMAGE vs. THETA (position angle)
  3. FWHM_IMAGE vs. Xpos-CRPIX1
  4. FWHM_IMAGE vs. Ypos-CRPIX2

Mandatory inputs :

-i : list of input split raw or processed science images

Optional inputs :

-w : list of input split weight images (must match science images)

Configuration parameters (min - [default] - max) :

-d : sextractor detection threshold range: 0.0 - [10.0] - 100.0
-f : maximum FLAG of sources range: 0 - [0] - 255
-m : maximum MAX_VAL of sources range: 0 - [64000] - 70000
-r : minimum FLUX_RADIUS of sources range: 0.0 - [0.0] - 50.0
-rm : maximum FLUX_RADIUS of sources
range: 0.0 - [50.0] - 50.0

Switches :

-in : interactive mode (if omitted a postscript file is created)

Example of use :

awe PSF_Anisotropy.py -i image12345_?.fits [-s] [-d 5.0] [-f 1]
[-m 50000] [-r 2.0] [-rm 5.0] [-in]

class astro.filerecipes.PSF_Anisotropy.PSF_Anisotropy(filenames=[], weights=[], as_table=False, interactive=False, reuse_catalogs=False, outfile='', fwhm_world=False, ellipticity=False, max_seeing=0.0)

Bases: object

This is the file-based interface to create PSFAnisotropyPlots

check_preconditions()

Make sure all the files exist that are needed.

make()

Run everything.

make_catalogs()
plot()
set_data()
class astro.filerecipes.PSF_Anisotropy.PSF_AnisotropyCatalog

Bases: object

This is a container for data that can be used by PSFAnisotropyPlot, where the data originates from a Catalog object

Multiple instances of this class are expected to update a single dictionary using the update_data_dictionary() method.

Complements PSF_AnisotropySourceList.

NEEDED_COLUMNS = ['Xpos', 'Ypos', 'A', 'B', 'Theta', 'THETA_WORLD', 'FWHM_WORLD', 'FWHM_IMAGE', 'ELLIPTICITY', 'FLUX_RADIUS', 'MaxVal', 'MAG_ISO', 'Flag']
check()
copy_process_parameters(process_params)

Copy the attributes of an outside process_params object to the process_params object of this object

make_from_file(filename, weight=None)
make_from_frame(frame)
set_data()

Extract data from the catalog.

update_data_dictionary(dictionary={})

Copies keys and values from self.data to the specified dictionary. If the key exists in the specified dictionary the two (asserted) lists are concatenated / extended.

class astro.filerecipes.PSF_Anisotropy.PSF_AnisotropyCatalogParameters

Bases: object

DETECT_THRESH = 10.0
REUSE_CATALOGS = False
SEX_PARAMS = ['ELLIPTICITY', 'THETA_WORLD', 'FWHM_WORLD']
exception astro.filerecipes.PSF_Anisotropy.PSF_AnisotropyError(message)

Bases: common.log.Error.Error

class astro.filerecipes.PSF_Anisotropy.PSF_AnisotropyParameters

Bases: object

DETECT_THRESH = 10.0
FILENAME = 'cal554.eps'
FLAG_MAX = 0
FWHM_SCALEFACTOR = 5.0
MAX_FLUX_RADIUS = 50.0
MAX_VAL = 64000
MIN_FLUX_RADIUS = 0
SEX_PARAMS = ['ELLIPTICITY', 'THETA_WORLD', 'FWHM_WORLD']
exception astro.filerecipes.PSF_Anisotropy.ParserError(message)

Bases: common.log.Error.Error

astro.filerecipes.PSF_Anisotropy.adjust_coordinates(frame, xdata, ydata)

Rotate and shift coordinates in (xdata, ydata) using CD matrix and CRPIXn values from header and return the adjusted arrays.

astro.filerecipes.PSF_Anisotropy.get_mult_args(cmndline, opt, opts=[], conversion=<class 'str'>)

Command line parser. Returns arguments in case of multiple arguments for a option opt.

astro.filerecipes.PSF_Anisotropy.get_single_arg(cmndline, opt, opts, conversion=<class 'str'>)

Command line parser. Returns argument in case of single argument for option opt.

astro.filerecipes.PSF_Anisotropy.main()
astro.filerecipes.PSF_Anisotropy.make_psfanisotropy(filenames, weights=[], as_table=False, interactive=False, detect_thresh=None, flag_max=None, flux_max=None, min_flux_radius=None, max_flux_radius=None, use_boxes=None)

Make PSF Anisotropy catalog and plot

astro.filerecipes.PSF_Anisotropy.parse_command_line(cmndline)
astro.filerecipes.PSF_Anisotropy.usage()

astro.filerecipes.Particle_Event_Rate_Plot module

astro.filerecipes.PhotCal_Extract_Resulttable module


CATEGORY : data pre-processing.

PURPOSE : derive a set of photometric result tables for further
processing in the photometric part of the pipeline.

This recipe is used to derive photometric result tables which are used for further processing in the photometric part of the pipeline.

The recipe always takes a list of frames from a standard field as input that are at least de-biased and flatfielded. The astrometric solutions associated with these frames should also be supplied, as well a their weights. Besides these, a standard star catalog is also required.

If one wants to derive photometry result tables for photometric bands that are not supported by the standard star catalog, an additional input is required. This input consists of a transformation table (Calfile 565/565C) containing the color-transformation coefficients.


Mandatory inputs :

-i : list of (at least) de-biased and flatfielded frames from a
photometric standard field.
-a : list of the astrometric solution files associated with the input
frames. The n-th entry in the list should belong to the n-th entry of the list of frames.
-w : list of the weights associated with the input frames. The n-th
entry in the list should belong to the n-th entry of the list of frames.

-s : standard star catalog (Calfile 569/569E).

Additional input if color-transformation coefficients are required :

-t : transformation table (Calfile 565/565C).

Example of use :

awe PhotCal_Extract_Resulttable -i frame1 frame2 frame3 frame4
-a astrom1 astrom2 astrom3 astrom4 -w weight1 weight2 weight3 weight4 -s cal569.cat

[-t cal565.cat]


astro.filerecipes.PhotCal_Extract_Resulttable.instantiate_objects(frame_list, astrom_list, weight_list, refcat_name, transform_name)
astro.filerecipes.PhotCal_Extract_Resulttable.main()
astro.filerecipes.PhotCal_Extract_Resulttable.make_catalogs(frames, astroms, weights, refcat, transform)

This routine derives the photometric result tables that are used for further processing in the photometric part of the pipeline..

Inputs:
frames – the reduced science frames from which the photometry
result tables are to be derived
astroms – the astrometric solutions associated with the input
frames weights

– the weights associated with the input frames

refcat – standard star catalog (Calfile 569/569E, mandatory) transform – transformation table (Calfile 565/565C)

astro.filerecipes.PhotCal_Extract_Resulttable.parse_command_line()
astro.filerecipes.PhotCal_Extract_Resulttable.usage()

astro.filerecipes.PhotCal_Extract_Resulttable_pickles module


CATEGORY : data pre-processing.

PURPOSE : derive a set of photometric result tables for further
processing in the photometric part of the pipeline.

This recipe is used to derive photometric result tables which are used for further processing in the photometric part of the pipeline.

The recipe always takes a list of frames from a standard field as input that are at least de-biased and flatfielded. The astrometric solutions associated with these frames should also be supplied, as well a their weights. Besides these, a standard star catalog is also required.

If one wants to derive photometry result tables for photometric bands that are not supported by the standard star catalog, an additional input is required. This input consists of a transformation table (Calfile 565/565C) containing the color-transformation coefficients.


Mandatory inputs :

-i : list of (at least) de-biased and flatfielded frames from a
photometric standard field.
-a : list of the astrometric solution files associated with the input
frames. The n-th entry in the list should belong to the n-th entry of the list of frames.
-w : list of the weights associated with the input frames. The n-th
entry in the list should belong to the n-th entry of the list of frames.

-s : standard star catalog (Calfile 569/569E).

Additional input if color-transformation coefficients are required :

-t : transformation table (Calfile 565/565C).

Example of use :

awe PhotCal_Extract_Resulttable -i frame1 frame2 frame3 frame4
-a astrom1 astrom2 astrom3 astrom4 -w weight1 weight2 weight3 weight4 -s cal569.cat

[-t cal565.cat]


class astro.filerecipes.PhotCal_Extract_Resulttable_pickles.PhotSrcCatalogMaker(args)

Bases: object

get_astrom_params(reduced)

Find the astrometric parameters pickle for the reducedframe

make_photcat(reduced)

This routine derives the photometric result tables that are used for further processing in the photometric part of the pipeline..

Inputs:
frames – the reduced science frames from which the
photometry result tables are to be derived
astroms – the astrometric solutions associated with the
input frames weights

– the weights associated with the input frames

refcat – standard star catalog (Calfile 569/569E) transform – transformation table (Calfile 565/565C)

make_photcats()
save_pickle(photcat)
astro.filerecipes.PhotCal_Extract_Resulttable_pickles.main()

astro.filerecipes.PhotCal_Extract_Zeropoint module


CATEGORY : data processing, photometric pipeline.

PURPOSE : generating a set of valid Calfiles 563/564 which are used by
the image pipeline in the photometric calibration.

FULFILLS : requirements 563/564 (Category I)


This recipe is used to derive Calfiles 563/564 which are used for the photometric calibration of science data.

The recipe always takes at least one list of photometric result tables derived from a standard field as input, and a standard extinction curve (Calfile 564E). Besides these, either a monitoring report (Calfile 562) must be supplied, or a second list of photometric result tables from an additional standard field.

In case a second list of tables from an additional standard field is given, this second list can be shorter than the first obligatory list. However, all the chips present in this second list should also be present in the first list. In both lists of photometric result tables, all chips should be present only once.

The recipe must be run for every photometric band/filter separately.

The dataflow of the recipe consists of two steps. In the first step, a value for the atmospheric extintion is derived. In the second step, this value for the extinction is used in deriving the zeropoints.

Depending on the inputs, the atmospheric extinction will be derived in one of two ways. In case only ONE list of input tables is provided and a monitoring report (Calfile 562) is supplied, the atmospheric extinction is simply derived by taking a value from the input standard extinction curve and applying the shift to this value as recorded in the monitoring report. If TWO lists of input tables are provided, the atmospheric extinction is derived by combining the tables in these two lists (the airmass of the two lists of tables should differ). For this case, the standard extinction curve is also a required input, but here it is used for verification purposes.


Mandatory inputs modus operandi 1 (one list of tables/monitoring report) :

-e : list of tables derived from (equatorial) standard field -m : monitoring report (Calfile 562) -c : standard extinction curve (Calfile 564E)

Mandatory inputs modus operandi 2 (two lists of tables) :

-e : list of tables derived from (equatorial) standard field -p : list of tables derived from additional standard field -c : standard extinction curve (Calfile 564E)

Example of use (modus operandi 1):

awe PhotCal_Extract_Zeropoint -e table1 table2 table3 table4 table5
-m cal562.dat -c cal564E.dat

Example of use (modus operandi 2):

awe PhotCal_Extract_Zeropoint -e table1 table2 table3 table4 table5
-p table6 table7 table8 table9 table10 -c cal564E.dat
NOTE : the input tables should all have been derived from frames that
have been observed in only one photometric band, and every chip should be present only once in either input list.

astro.filerecipes.PhotCal_Extract_Zeropoint.construct_filename(table)
astro.filerecipes.PhotCal_Extract_Zeropoint.get_subset(equat, polar)

Based on the chip names present in the polar list of tables, a subset is distilled from the equat list of tables.

astro.filerecipes.PhotCal_Extract_Zeropoint.instantiate_objects(equat_list, polar_list, report_name, extcurve_name)
astro.filerecipes.PhotCal_Extract_Zeropoint.main()
astro.filerecipes.PhotCal_Extract_Zeropoint.make_photometrics(equat, polar, report, extcurve)

This routine derives the Calfiles 563/564.

Inputs :
equat – list of tables derived from (equatorial) standard field
(mandatory)

polar – list of tables derived from additional standard field report – monitoring report (Calfile 562) extcurve – standard extinction curve (Calfile 564E, mandatory)

Note that depending on processing mode different combinations of these inputs are required.

astro.filerecipes.PhotCal_Extract_Zeropoint.parse_command_line()
astro.filerecipes.PhotCal_Extract_Zeropoint.usage()

astro.filerecipes.PhotCal_Extract_Zeropoint_pickles module


CATEGORY : data processing, photometric pipeline.

PURPOSE : generating a set of valid Calfiles 563/564 which are used by
the image pipeline in the photometric calibration.

FULFILLS : requirements 563/564 (Category I)


This recipe is used to derive Calfiles 563/564 which are used for the photometric calibration of science data.

The recipe always takes at least one list of photometric result tables derived from a standard field as input, and a standard extinction curve (Calfile 564E). Besides these, either a monitoring report (Calfile 562) must be supplied, or a second list of photometric result tables from an additional standard field.

In case a second list of tables from an additional standard field is given, this second list can be shorter than the first obligatory list. However, all the chips present in this second list should also be present in the first list. In both lists of photometric result tables, all chips should be present only once.

The recipe must be run for every photometric band/filter separately.

The dataflow of the recipe consists of two steps. In the first step, a value for the atmospheric extintion is derived. In the second step, this value for the extinction is used in deriving the zeropoints.

Depending on the inputs, the atmospheric extinction will be derived in one of two ways. In case only ONE list of input tables is provided and a monitoring report (Calfile 562) is supplied, the atmospheric extinction is simply derived by taking a value from the input standard extinction curve and applying the shift to this value as recorded in the monitoring report. If TWO lists of input tables are provided, the atmospheric extinction is derived by combining the tables in these two lists (the airmass of the two lists of tables should differ). For this case, the standard extinction curve is also a required input, but here it is used for verification purposes.


Mandatory inputs modus operandi 1 (one list of tables/monitoring report) :

-e : list of tables derived from (equatorial) standard field -m : monitoring report (Calfile 562) -c : standard extinction curve (Calfile 564E)

Mandatory inputs modus operandi 2 (two lists of tables) :

-e : list of tables derived from (equatorial) standard field -p : list of tables derived from additional standard field -c : standard extinction curve (Calfile 564E)

Example of use (modus operandi 1):

awe PhotCal_Extract_Zeropoint -e table1 table2 table3 table4 table5
-m cal562.dat -c cal564E.dat

Example of use (modus operandi 2):

awe PhotCal_Extract_Zeropoint -e table1 table2 table3 table4 table5
-p table6 table7 table8 table9 table10 -c cal564E.dat
NOTE : the input tables should all have been derived from frames that
have been observed in only one photometric band, and every chip should be present only once in either input list.

class astro.filerecipes.PhotCal_Extract_Zeropoint_pickles.PhotometricParametersMaker(args)

Bases: object

filter_equatorial()

Based on the chip names present in the polar list of tables, a subset is distilled from the equat list of tables.

get_atmospheric_extinction()
load_pickles()
make_atmospheric_extinction_curve()

Create the AtmosphericExtinction based on ExtinctionReport

make_atmospheric_extinction_frames()

Create the AtmosphericExtinction based on polar fields

make_photometric_parameters()

This routine derives the Calfiles 563/564.

Inputs :
equat – list of tables derived from (equatorial) standard
field (mandatory)

polar – list of tables derived from additional standard field report – monitoring report (Calfile 562) extcurve – standard extinction curve (Calfile 564E, mandatory)

Note that depending on processing mode different combinations of these inputs are required.

save_pickle()
astro.filerecipes.PhotCal_Extract_Zeropoint_pickles.main()

astro.filerecipes.PhotCal_Monitoring module


CATEGORY : data processing, photometric pipeline.

PURPOSE : generating a valid Calfile 562 (extinction/monitoring report).

FULFILLS : requirement 562 (Category I)


This recipe is used to derive Calfile 562 (extinction/monitoring report).

The recipe always takes as input a list of photometric result tables of standard fields observed at various moments through the night, as well as a list of Calfiles 563 which contain the zeropoint of the various chip/filter combinations that go into the processing. Besides these, a standard extinction curve (Calfile 564E) is also required.

The recipe assumes that the input tables were extracted from frames that have been observed through at least two different filters/photometric bands. Note that every chip/photometric band/filter combination that occurs in the list of input tables should also be present in the list of Calfiles 563.


Mandatory inputs :

-i : list of input photometric result tables -z : list of zeropoints (Calfiles 563) -c : standard extinction curve (Calfile 564E)

Example of use :

awe PhotCal_Monitoring.py -i table1.1 table2.1 table1.2 table2.2
-z cal563_1.1.dat cal563_2.1.dat -c cal564E.cat

astro.filerecipes.PhotCal_Monitoring.OBJECTS_table_definition()
astro.filerecipes.PhotCal_Monitoring.check_preconditions(photcats, photoms)
astro.filerecipes.PhotCal_Monitoring.create_record(mag_id, date_obs, atmos, extinct_diff)
astro.filerecipes.PhotCal_Monitoring.derive_extinctions(photcats, photoms)
astro.filerecipes.PhotCal_Monitoring.echo_results_of_fit(records, fit_results)
astro.filerecipes.PhotCal_Monitoring.fit_curve_to_extinctions(atmos_dict, extcurve)
astro.filerecipes.PhotCal_Monitoring.get_central_date(dates)
astro.filerecipes.PhotCal_Monitoring.get_filename(instrument_name)
astro.filerecipes.PhotCal_Monitoring.get_record_list(records)
astro.filerecipes.PhotCal_Monitoring.get_shift_for_the_night(record_list)
astro.filerecipes.PhotCal_Monitoring.group_objects_by_filter(object_list)
astro.filerecipes.PhotCal_Monitoring.group_photcats_by_observing_date(photcat_list)
astro.filerecipes.PhotCal_Monitoring.instantiate_objects(table_list, photom_list, extcurve_name)
astro.filerecipes.PhotCal_Monitoring.main()
astro.filerecipes.PhotCal_Monitoring.make_monitoringreport(photcats, photoms, extcurve)

This routine derives Calfile 562.

Inputs :
photcats – list of photometry result tables photoms – list of Calfiles 563 containing zeropoints extinct – standard extinction curve (Calfile 564E)
astro.filerecipes.PhotCal_Monitoring.parse_command_line()
astro.filerecipes.PhotCal_Monitoring.save_results_to_file(records, instrument_name, shift, shift_err)
astro.filerecipes.PhotCal_Monitoring.update_records(records, fit_results)
astro.filerecipes.PhotCal_Monitoring.usage()

astro.filerecipes.PhotCal_Monitoring_Sky module


CATEGORY : diagnostic tools, health check.

PURPOSE : providing a quick check on the spectrum of the atmosphere.

FULFILLS : requirement 562 Monitoring_Sky (Category III)


This recipe is used to obtain a quick check on the spectrum of the skybackground.

The recipe always takes as input four lists of raw science frames, one list for every key band or quadrant of the composite filter. The appropriate masterbias frames and masterflat frames must also be supplied. Besides these, for every key band there should also be given a table with skybrightness values as a function of days after new moon, and the number of days after new moon. Optionally, an overscan correction mode can be set used in the processing of the input raw science frames. The default is to apply no overscan correction.

To correct for band-to-band differences in zeropoint, it is possible to optionally supply the recipe with the overall zeropoints of the detector block for the various bands.

In case composite filter data is used, the recipe accepts only ONE full image at once.


Mandatory inputs :

-iu : list of raw science frames (Sloan u band) -ig : list of raw science frames (Sloan g band) -ir : list of raw science frames (Sloan r band) -ii : list of raw science frames (Sloan i band) -fu : list of masterflat frames (Sloan u band) -fg : list of masterflat frames (Sloan g band) -fr : list of masterflat frames (Sloan r band) -fi : list of masterflat frames (Sloan i band) -b : list of masterbias frames (Calfile 541) -s : list of skybrightness tables (Calfile 562S) -lp : number of days after new moon (integer)

Optional inputs :

-zu : global zeropoint for Sloan u (default : 0.00) -zg : global zeropoint for Sloan g (default : 0.00) -zr : global zeropoint for Sloan r (default : 0.00) -zi : global zeropoint for Sloan i (default : 0.00)

Configuration parameter (optional) :

-oc : overscan correction (range 0-6, default : 0)

Example of use :

awe PhotCal_Monitoring_Sky.py -iu raw1 -ig raw2 -ir raw3 -ii raw4
-fu flat1 -fg flat2 -fr flat3 -fi flat4 -s table_u table_g table_r table_i -b bias1 -lp 12 -oc 2

astro.filerecipes.PhotCal_Monitoring_Sky.derive_color_and_compare(reduced_band1, reduced_band2, table_dict, zp_dict, lunar_phase)
astro.filerecipes.PhotCal_Monitoring_Sky.instantiate_frame_objects(file_list, cls)
astro.filerecipes.PhotCal_Monitoring_Sky.instantiate_table_objects(table_list)
astro.filerecipes.PhotCal_Monitoring_Sky.main()
astro.filerecipes.PhotCal_Monitoring_Sky.make_skyreport(raw_u_dict, raw_g_dict, raw_r_dict, raw_i_dict, flat_u_dict, flat_g_dict, flat_r_dict, flat_i_dict, bias_dict, table_dict, zp_dict, lunar_phase, overscan)
astro.filerecipes.PhotCal_Monitoring_Sky.parse_command_line()
astro.filerecipes.PhotCal_Monitoring_Sky.parse_lists(param, param_list, cmndline)
astro.filerecipes.PhotCal_Monitoring_Sky.parse_single_item(param, param_list, cmndline)
astro.filerecipes.PhotCal_Monitoring_Sky.process_rawframes(raw_dict, flat_dict, bias_dict, overscan)

This function processes the input raw science frames. It returns a list of de-biased and flatfielded frames.

astro.filerecipes.PhotCal_Monitoring_Sky.usage()

astro.filerecipes.PhotCal_Monitoring_Sky_pickles module


CATEGORY : diagnostic tools, health check.

PURPOSE : providing a quick check on the spectrum of the atmosphere.

FULFILLS : requirement 562 Monitoring_Sky (Category III)


This recipe is used to obtain a quick check on the spectrum of the skybackground.

The recipe always takes as input four lists of raw science frames, one list for every key band or quadrant of the composite filter. The appropriate masterbias frames and masterflat frames must also be supplied. Besides these, for every key band there should also be given a table with skybrightness values as a function of days after new moon, and the number of days after new moon. Optionally, an overscan correction mode can be set used in the processing of the input raw science frames. The default is to apply no overscan correction.

To correct for band-to-band differences in zeropoint, it is possible to optionally supply the recipe with the overall zeropoints of the detector block for the various bands.

In case composite filter data is used, the recipe accepts only ONE full image at once.


class astro.filerecipes.PhotCal_Monitoring_Sky_pickles.SkyReportMaker(args)

Bases: object

derive_color_and_compare()

Derive colors u-g, g-r, r-i

load_pickles()
load_skybrightness()
make_skyreport()
process_rawframes()

This function processes the input raw science frames. It returns a list of de-biased and flatfielded frames.

astro.filerecipes.PhotCal_Monitoring_Sky_pickles.main()

astro.filerecipes.PhotCal_Monitoring_pickles module


CATEGORY : data processing, photometric pipeline.

PURPOSE : generating a valid Calfile 562 (extinction/monitoring report).

FULFILLS : requirement 562 (Category I)


This recipe is used to derive Calfile 562 (extinction/monitoring report).

The recipe always takes as input a list of photometric result tables of standard fields observed at various moments through the night, as well as a list of Calfiles 563 which contain the zeropoint of the various chip/filter combinations that go into the processing. Besides these, a standard extinction curve (Calfile 564E) is also required.

The recipe assumes that the input tables were extracted from frames that have been observed through at least two different filters/photometric bands. Note that every chip/photometric band/filter combination that occurs in the list of input tables should also be present in the list of Calfiles 563.


Mandatory inputs :

-i : list of input photometric result tables -z : list of zeropoints (Calfiles 563) -c : standard extinction curve (Calfile 564E)

Example of use :

awe PhotCal_Monitoring.py -i table1.1 table2.1 table1.2 table2.2
-z cal563_1.1.dat cal563_2.1.dat -c cal564E.cat

class astro.filerecipes.PhotCal_Monitoring_pickles.PhotSrcCatalogMaker(rawfitsdata)

Bases: object

make_astrom(reduced)
make_photcat(reduced, astrom)
make_photcats()

Pick either 1, 2, 3 or 4 CCDs from each quadrant, to minimize reduction time.

Special CCDs:
  • Corners: 1, 13, 20, 32
  • Central CCDs: 8,12,21,25 (most affected by cross)
  • 2 away from corners, but on edge: 3, 15, 18, 30
  • Avoiding crosstalk CCDs: 26, 28
Set is then: 1,3,13,15,18,20,30,32 (could perhaps skip flatfield) or
1,3,13,15,18,29,26,28 (avoiding crosstalk)
make_raw(extension)
make_reduced(raw)
pipeline_to_photcat(extension)
class astro.filerecipes.PhotCal_Monitoring_pickles.PhotometricExtinctionReportMaker(photcats)

Bases: object

OBJECTS_table_definition()
check_preconditions()
create_record(mag_id, wavelength, date_obs, atmos, extinct_diff)
derive_extinctions()
echo_results_of_fit(records, fit_results)
fit_curve_to_extinctions(atmos_dict)
get_filename(instrument_name)
get_record_list(records)
get_shift_for_the_night(record_list)
make_monitoringreport()

This routine derives Calfile 562.

Inputs :
photcats – list of photometry result tables photoms – list of Calfiles 563 containing zeropoints extinct – standard extinction curve (Calfile 564E)
make_simple()
save_results_to_file(records, shift, shift_err)
update_records(records, fit_results)
class astro.filerecipes.PhotCal_Monitoring_pickles.SkyColor(photcats)

Bases: object

Class to calculate the color of the sky based on a list of PhotSrcCatalogs, at least 1 for every quadrant of the composite filter

calculate()
derive_color_and_compare()

Derive colors u-g, g-r, r-i

get_zeropoint(photcats)
load_skybrightness()
astro.filerecipes.PhotCal_Monitoring_pickles.get_central_date(dates)
astro.filerecipes.PhotCal_Monitoring_pickles.group_objects_by_filter(object_list)
astro.filerecipes.PhotCal_Monitoring_pickles.group_photcats_by_observing_date(photcat_list)
astro.filerecipes.PhotCal_Monitoring_pickles.main()

astro.filerecipes.QuickCheckPlot module

class astro.filerecipes.QuickCheckPlot.QuickCheckPlot(data1={}, data2={}, filename='', statistics=True)

Bases: astro.plot.OCAMMosaicPlot.OCAMMosaicPlot

Plot an overview of QuickCheckFrame values in a representation of the OmegaCAM mosaic.

The plot takes as input a list of QuickCheckFrame objects

check_preconditions()
plot_data()
title = 'Quick detector responsivity check'
xlabel = 'DATE-OBS'
ylabel = 'Exposure level [ADU]'
ylim = (25000, 40000)

astro.filerecipes.Quick_Check module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a quick check on the detector responsivity
(Calfile 547)

FULFILLS : requirement 547 (Category I)


This recipe is used to generate a quick check on the detector responsivity for one particular chip (Calfile 547).

The recipe takes as input exactly one raw domeflat, and a master bias. Optionally, a previous QuickCheckFrame can be specified for comparison, and an overscan correction mode can be set.


Mandatory inputs :

-i : input raw domeflat -b : master bias (Calfile 541)

Optional inputs :

-p : previous quick check frame

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Quick_Check.py -i raw.fits -b cal541.fits -p quick.fits -oc 2

astro.filerecipes.Quick_Check.configure_process_parameters(quickcheck, overscan)
astro.filerecipes.Quick_Check.construct_filename(raw_dome)
astro.filerecipes.Quick_Check.instantiate_objects(raw_domeflat_name, bias_name, previous_name=None)
astro.filerecipes.Quick_Check.main()
astro.filerecipes.Quick_Check.make_quickcheck(raw_domeflat, bias, previous=None, overscan=6)

This routine derives Calfile 547.

Inputs:
raw_domeflat – a raw domeflat frame bias – a master bias (Calfile 541) previous – optional previous quick check frame for comparison overscan – overscan correction mode
astro.filerecipes.Quick_Check.parse_command_line()
astro.filerecipes.Quick_Check.usage()

astro.filerecipes.Quick_Check_simple module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a quick check on the detector responsivity
(Calfile 547)

FULFILLS : requirement 547 (Category I)


This recipe is used to generate a quick check on the detector responsivity for one particular chip (Calfile 547).

The recipe takes as input one or more unsplit raw domeflat filenames.


class astro.filerecipes.Quick_Check_simple.QuickCheckMaker(filenames, basedir='./', end_date=None, timespan=14, outfile='')

Bases: object

add_new_data()
calculate_new_points(rawfitsdata)
instantiate_rawfitsdatas()
load_table()
make_quickcheck()

This routine derives Calfile 547.

plot()
rawfitsdata_present_in_table(rawfitsdata)
save_table()
update_data(new_data)
astro.filerecipes.Quick_Check_simple.main()

astro.filerecipes.RawBiasFramePlot module

astro.filerecipes.RawDomeFlatFramePlot module

astro.filerecipes.Read_Noise module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid read noise calibration (Calfile 521).

FULFILLS : requirement 521 (Category I)


This recipe is used to derive a valid calibration of the read noise (Calfile 521) for one particular chip.

The recipe always takes as input a list of raw bias frames. Optionally, two process parameters can be set: the maximum number of iterations used in deriving the read noise, and the rejection threshold in sigma. The defaults for these parameters are 5 and 5.0, respectively.


Mandatory inputs :

-i : list of input raw bias frames

Configuration parameters (optional) :

-max : maximum iterations (default : 5) -rej : rejection threshold (default : 5.0 sigma)

Optional :

-use_pickles : Python pickles are used for more automated processing

Example of use :

awe Read_Noise.py -i raw1.fits raw2.fits -max 10 -rej 3.0

astro.filerecipes.Read_Noise.check_grouped_dict(dictionary)
astro.filerecipes.Read_Noise.configure_process_parameters(readnoise, rej_thresh, max_iter)
astro.filerecipes.Read_Noise.instantiate_objects(raw_bias_list)
astro.filerecipes.Read_Noise.main()
astro.filerecipes.Read_Noise.make_normal(raw_bias_list, rej_thresh, max_iter)
astro.filerecipes.Read_Noise.make_readnoise(raw_bias_frames, rej_thresh=5.0, max_iter=5)

This routine derives Calfile 521.

Inputs :
raw_bias_frames – list of raw bias frames rej_thresh – rejection threshold in sigma max_iter – maximum iterations
astro.filerecipes.Read_Noise.make_using_pickles(raw_bias_list, rej_thresh, max_iter)
astro.filerecipes.Read_Noise.parse_command_line()
astro.filerecipes.Read_Noise.usage()

astro.filerecipes.Read_Noise_Plot module

astro.filerecipes.Read_Noise_Plot_Electrons module

astro.filerecipes.Read_Noise_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid read noise calibration (Calfile 521).

FULFILLS : requirement 521 (Category I)


This recipe is used to derive a valid calibration of the read noise (Calfile 521) for one particular chip.

The recipe always takes as input a list of raw bias frames. Optionally, two process parameters can be set: the maximum number of iterations used in deriving the read noise, and the rejection threshold in sigma. The defaults for these parameters are 5 and 5.0, respectively.


class astro.filerecipes.Read_Noise_pickles.ReadNoiseMaker(args)

Bases: object

check_grouped_dict(dictionary)
configure_process_parameters(readnoise)
make_readnoise(raw_bias_frames)

This routine derives Calfile 521.

Inputs:
raw_bias_frames – list of raw bias frames
make_readnoises()
save_pickle()
astro.filerecipes.Read_Noise_pickles.main()

astro.filerecipes.Reduce module


CATEGORY : data processing, image pipeline

PURPOSE : process science data for a single chip

FULFILLS : requirements 632, 633, 634, 635 (Category I)


This recipe is used to preprocess a single science observation. The recipe assumes that the data is for a single CCD.

The following operations are performed:

  1. Trim, overscan-correct, de-bias, flatfield and de-fringe the data to produce a reduced science image
  2. Produce maps of saturated pixels, satellite tracks and cosmic rays and produce a weight map from a flatfield and the bad pixel maps
  3. Extract a source catalog and perform astrometric calibration
  4. Apply photometric calibration, including illumination correction
  5. Regrid the science image to a new grid.

Mandatory inputs :

-i : raw science frame
-c : cold pixelmap (Calfile 535)
-h : hot pixelmap (Calfile 522)
-b : a master bias frame (Calfile 541)
-f : flat field (Calfile 546)
-p : photometric parameters file (Calfile 563/Calfile 564)
-g : gain (range 0.1-10.0) (Calfile 523)
-ra : the right ascension of the new grid center (decimal degrees)

-dec : the declination of the new grid center (decimal degrees) -ps : the pixel scale of the output grid (arcsec)

Optional inputs :

-fr : fringe map (Calfile 545) -il : Illumination correction frame (Calfile 548)

Configuration parameters (optional) :

-sl : low saturation cutoff (default 50.0)
-sh : high saturation cutoff (default 50000.0)
-oc : overscan correction (range 0-8, default : 6)
-ft : de-fringing image thresh (not used if no fringes, default 5.0)
-fl : de-fringing low cutoff (not used if no fringes, default 1.5)
-fh : de-fringing high cutoff (not used if no fringes, default 5.0)
-ct : cosmic-ray detection threshold (default 5.0)

-sit : satellite detection image threshold (default 5.0) -sht : satellite detection hough threshold (default 1000.0) -dt : detection threshold for astrometry

Example of use :

awe Reduce.py -i raw.fits -h hot.fits -c cold.fits -b bias.fits
-f flat.fits -p cal563.cat -g 2.0 -ra 186.376 -dec -13.4043 -ps 0.238

astro.filerecipes.Reduce.get_argument(cmndline, key, conversion=<class 'str'>)
astro.filerecipes.Reduce.get_gridtarget(ra, dec, pixelscale)
astro.filerecipes.Reduce.get_object(cmndline, key, obj)
astro.filerecipes.Reduce.main()
astro.filerecipes.Reduce.make_all(raw, hot, cold, bias, flat, photom, gainlin, ra, dec, pixelscale, fringe=None, illumination=None, saturation_threshold_low=None, saturation_threshold_high=None, overscan_correction=None, image_threshold=None, fringe_threshold_low=None, fringe_threshold_high=None, cosmic_detection_threshold=None, satellite_detection_threshold=None, satellite_hough_threshold=None, detection_threshold=None)

Make a fully calibrated ReducedScienceFrame, derive an astrometric solution, and make a RegriddedFrame. The weights are also produced.

Mandatory inputs :

raw – A RawScienceFrame object hot – A HotPixelMap object cold – A ColdPixelMap object bias – A BiasFrame object flat – A MasterFlatFrame object photom – A PhotometricParameters object gainlin – A GainLinearity object ra – Right ascension of grid target dec – Declination of grid target pixelscale – Pixel scale of grid target

Optional inputs :

fringe – A FringeFrame object illumination – An IlluminationCorrectionFrame

Configuration parameters :

saturation_threshold_low saturation_threshold_high overscan_correction image_threshold fringe_threshold_low fringe_threshold_high cosmic_detection_threshold satellite_detection_threshold satellite_hough_threshold detection_threshold
astro.filerecipes.Reduce.make_astrom(reduced)

This routine derives an astrometric calibration for the input reduced science frame.

Mandatory inputs :
reduced – reduced science frame
astro.filerecipes.Reduce.make_reduced_science_frame(raw, hot, cold, bias, flat, fringe=None, illumination=None, overscan_correction=None, image_threshold=None, fringe_threshold_low=None, fringe_threshold_high=None)

Make a ReducedScienceFrame object.

Mandatory inputs :

raw – A RawScienceFrame object bias – A BiasFrame object flat – A MasterFlatFrame object

Optional inputs :

fringe – A FringeFrame object illumination – As IlluminationCorrectionFrame hot – A HotPixelMap object

(not used if fringe is None, else optional)
cold – A ColdPixelMap object
(not used if fringe is None, else optional)

Configuration parameters :

overscan_correction – overscan correction method image_threshold – imstat threshold for fringe-scaling

(not used if fringe is None)
fringe_threshold_low – imstat threshold for fringe-scaling
(not used if fringe is None)
fringe_threshold_high – imstat threshold for fringe-scaling
(not used if fringe is None)
astro.filerecipes.Reduce.make_regridded(reduced, astrom, photom, gainlin, ra, dec, pixelscale)

Regrid the reduced science frame to a predefined grid.

Mandatory inputs :
reduced – reduced science frame astrom – the astrometric calibration of the reduced science frame photom – the photometric calibration gainline – the gain
astro.filerecipes.Reduce.parse_command_line(cmndline)
astro.filerecipes.Reduce.usage()

astro.filerecipes.Reduce_pickles module


CATEGORY : data processing, image pipeline

PURPOSE : process science data for a single chip

FULFILLS : requirements 632, 633, 634, 635 (Category I)


This recipe is used to preprocess a single science observation. The recipe assumes that the data is for a single CCD.

The following operations are performed:

  1. Trim, overscan-correct, de-bias, flatfield and de-fringe the data to produce a reduced science image
  2. Produce maps of saturated pixels, satellite tracks and cosmic rays and produce a weight map from a flatfield and the bad pixel maps
  3. Extract a source catalog and perform astrometric calibration
  4. Apply photometric calibration, including illumination correction
  5. Regrid the science image to a new grid.

class astro.filerecipes.Reduce_pickles.ReducedScienceFrameMaker(args)

Bases: object

Make a fully calibrated ReducedScienceFrame, derive an astrometric solution, and make a RegriddedFrame. The weights are also produced.

Mandatory inputs :

raw – A RawScienceFrame object hot – A HotPixelMap object cold – A ColdPixelMap object bias – A BiasFrame object flat – A MasterFlatFrame object photom – A PhotometricParameters object gainlin – A GainLinearity object ra – Right ascension of grid target dec – Declination of grid target pixelscale – Pixel scale of grid target

Optional inputs :

fringe – A FringeFrame object illumination – An IlluminationCorrectionFrame

Configuration parameters :

saturation_threshold_low saturation_threshold_high overscan_correction image_threshold fringe_threshold_low fringe_threshold_high cosmic_detection_threshold satellite_detection_threshold satellite_hough_threshold detection_threshold
instantiate_raw_frames()
load_pickles(filter_id)
make_reduced(raw)
make_reduceds()
save_pickle(reduced)
astro.filerecipes.Reduce_pickles.main()

astro.filerecipes.Shutter_Delay module


CATEGORY : diagnostic tools, health check.

PURPOSE : verify the timing of the shutter.

FULFILLS : requirement 561 (Category III)


This recipe is used to verify the timing of the shutter, and to monitor any position-dependent delay of the shutter motion in both directions.

The recipe always takes as input two lists of two raw domeflat frames each, and a master bias. Optionally, a hot pixel map, and the overscan correction mode can be set. The default is to apply overscan correction 6.


Mandatory inputs :

-u : list of two input raw domeflat frames (shutter up) -d : list of two input raw domeflat frames (shutter down) -b : master bias (Calfile 541)

Optional input :

-h : hot pixel map (Calfile 522)

Configuration parameters (optional) :

-oc : overscan correction (integer: 0 - [6] - 10) -c : collapse direction (integer: 0 (y-direction, default) or 1)

Example of use :

awe Shutter_Delay.py -u flat1.fits flat2.fits -d flat3.fits
flat4.fits -h hotpixels.fits -b bias.fits -oc 2 -c 0

astro.filerecipes.Shutter_Delay.construct_filename(bias, direction)
astro.filerecipes.Shutter_Delay.instantiate_objects(raw_domeflat_list, bias_name, hot_name=None)
astro.filerecipes.Shutter_Delay.main()
astro.filerecipes.Shutter_Delay.make_shutterdelay(raw_domeflat_frames, bias, hot=None, overscan=6, collapse_direction=1)
astro.filerecipes.Shutter_Delay.make_using_pickles(raw_domeflat_list, overscan, collapse_direction)
astro.filerecipes.Shutter_Delay.parse_command_line()
astro.filerecipes.Shutter_Delay.usage()

astro.filerecipes.Shutter_Delay_Plot module

astro.filerecipes.Shutter_Delay_simple module


CATEGORY : diagnostic tools, health check.

PURPOSE : verify the timing of the shutter.

FULFILLS : requirement 561 (Category III)


This recipe is used to verify the timing of the shutter, and to monitor any position-dependent delay of the shutter motion in both directions.

The recipe takes as input the filenames of the 4 images produced by the template.


class astro.filerecipes.Shutter_Delay_simple.ShutterCheckMaker(filenames, interactive=True)

Bases: object

get_shutter_direction(rawfitsdata)
group_rawfitsdata_by_shutter_direction()
make_rawframe(rawfitsdata, extension)
make_shuttercheck(raw_domeflats, shutter_direction=1)
make_shutterchecks()
astro.filerecipes.Shutter_Delay_simple.main()

astro.filerecipes.Split module


CATEGORY : data processing, image pipeline.

PURPOSE : extracting single extension FITS files from unsplit data.

FULFILLS : requirement 631 (Category I)


This recipe is used to extract separate FITS frames from input multi- extension FITS images.

The recipe always takes as input a list of unsplit images of one and the same type (eg, RawBias, RawScience). It returns a list of separate frames extracted from these unsplit images.


class astro.filerecipes.Split.RawFitsDataMaker(args)

Bases: object

make_rawfitsdata(raw_image_name)

Makes a single RawFitsData object (i.e. an object for unsplit, completely raw data.

make_rawfitsdatas()
make_rawframes(rawfitsdata, force=False)

This routine derives Seqfiles 631.

Inputs :
rawfitsdata – a RawFitsData object image_type – type of image (eg bias, dome)
save_pickle(rawframes=[])
astro.filerecipes.Split.main()

astro.filerecipes.Twilight_Flat module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid twilightflat frame (Calfile 543).

FULFILLS : requirement 543 (Category I)


This recipe is used to derive a valid twilightflat frame (Calfile 543) for one particular chip and filter.

The recipe always takes as input a list of raw twilightflat frames, a master bias, and a measurement of the gain of the chip under consideration. In addition, a hot pixel map (Calfile 522) and a cold pixel map (Calfile 535) can be provided. Optionally, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


Mandatory inputs :

-i : list of input raw twilightflat frames -b : master bias (Calfile 541) -g : gain (range 0.1-10.0)

Optional inputs :

-c : cold pixel map (Calfile 535) -h : hot pixel map (Calfile 522)

Configuration parameter (optional) :

-oc : overscan correction (range 0-8, default : 6)

Example of use :

awe Twilight_Flat.py -i raw1.fits raw2.fits raw3.fits -b bias.fits
-g 3.0 -c cal535.fits.gz -h cal523.fits.gz -oc 2

astro.filerecipes.Twilight_Flat.configure_process_parameters(twilightflat, overscan)
astro.filerecipes.Twilight_Flat.instantiate_objects(raw_twilightflat_list, bias_name, gain_value, cold_name, hot_name)
astro.filerecipes.Twilight_Flat.main()
astro.filerecipes.Twilight_Flat.make_normal(raw_twilightflat_list, bias_name, gain_value, cold_name, hot_name, overscan)
astro.filerecipes.Twilight_Flat.make_twilightflat(raw_twilightflat_frames, bias, gain, cold=None, hot=None, overscan=6)

This routine derives Calfile 543.

Inputs :
raw_twilightflat_frames – a list of raw twilightflat frames bias – a master bias (Calfile 541) gain – a gain calibration (Calfile 523) cold – a cold pixel map (Calfile 535) hot – a hot pixel map (Calfile 522) overscan – overscan correction mode
astro.filerecipes.Twilight_Flat.make_using_pickles(raw_twilightflat_list, overscan)
astro.filerecipes.Twilight_Flat.parse_command_line()
astro.filerecipes.Twilight_Flat.usage()

astro.filerecipes.Twilight_Flat_pickles module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a valid twilightflat frame (Calfile 542).

FULFILLS : requirement 542 (Category I)


This recipe is used to derive a valid twilightflat frame (Calfile 542) for one particular chip and filter.

The recipe always takes as input a list of raw twilightflat frames, a master bias and a measurement of the gain of the chip under consideration. In addition, a hot pixel map (Calfile 522) and a cold pixel map (Calfile 535) can be provided. Optionally, an overscan correction mode can be set. The default is to apply per-row overscan X correction.


class astro.filerecipes.Twilight_Flat_pickles.TwilightFlatMaker(args)

Bases: object

configure_process_parameters(twilightflat)
load_pickles(filter_id)
make_twilightflat(raw_twilightflat_frames)

This routine derives Calfile 542.

Inputs :
raw_twilightflat_frames – a list of raw twilightflat frames bias – a master bias (Calfile 541) gain – a gain calibration (Calfile 523) cold – a cold pixel map (Calfile 535) hot – a hot pixel map (Calfile 522) overscan – overscan correction mode
make_twilightflats()
save_pickle(filter_id)
astro.filerecipes.Twilight_Flat_pickles.main()

astro.filerecipes.aweimports module

automatic imports for the interpreter

astro.filerecipes.utilities module

astro.filerecipes.utilities.instantiate_grouped_by_ccd(filenames, cls)
astro.filerecipes.utilities.instantiate_grouped_by_ccd_filter(filenames, cls)

Module contents

Data Reduction Recipes

These are the Data Reduction Recipes as described in the OmegaCAM Data Flow System users and programmers manual, available from the documents section of the OmegaCAM website:

http://www.astro.rug.nl/~omegacam/documents/

Each Recipe takes filenames as arguments and processes these files accordingly.

To find the requirement in the above-mentioned document the table below lists the requirements which lead to a Recipe.

Name of the Recipe | Requirement in the OmegaCAM documentation


Read_Noise | 5.2.1 CCD read noise - doit Hot_Pixels | 5.2.2 Hot pixels Gain | 5.2.3 CCD gain

5.3.3 CCD Linearity
Dark_Current | 5.3.1 CCD Dark Current - doit
5.3.2 CCD Particle Event Rate

Cold_Pixels | 5.3.5 CCD Cold Pixels Bias | 5.4.1 Bias - doit Dome_Flat | 5.4.2 Flat-field - dome key bands + user bands - doit Twilight_Flat | 5.4.3 Flat-field - twilight Nightsky_Flat | 5.4.4 Flat-field - night sky Fringe_Flat | 5.4.5 Flat-field - fringing Master_Flat | 5.4.6 Flat-field - master flat and weight map Quick_Check | 5.4.7 Quick detector responsivity check - doit Illumination_Correction | 5.4.8 Illumination correction PSF_Anisotropy | 5.5.4 PSF Anisotropy PhotCal_Extract_Zeropoint | 5.6.3 Photometric Calibration - zeropoint key bands - doit

5.6.4 Photometric Calibration - zeropoint user bands

PhotCal_Monitoring | 5.6.2 Photometric Calibration - monitoring Split | 6.3.1 Ingest Reduce | 6.3.1 - 6.3.6 Image pipeline Coadd | 6.3.6 Image pipeline