astro.recipes package

Submodules

astro.recipes.ApertureCorrectionTask module

Recipe to create ApertureCorrection SourceCollections.

class astro.recipes.ApertureCorrectionTask.ApertureCorrectionTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : calucalte apertur corrections

ac = None
clean_up_before_pickle()
commit = False
execute()
get_name()
get_process_result()
get_process_time()

Return the processing time of this task.

make()

Find/make ApertureCorrections for a SourceCollection.

name = None
parser = Recognized options for the astro.recipes.ApertureCorrectionTask OptionsParser are: "S" or "scid" (default: 0) "n" or "name" (default: ) "C" or "commit" (default: False)
process_result = None
process_time = 7200
query_database()
sc = None
scid = 0
target

alias of astro.main.ApertureCorrectionAttributeCalculator.AttributeCalculator

astro.recipes.ApertureCorrectionTask.main()

Call ApertureCorrectionTask from the command-line.

astro.recipes.AssociateList module

task to make AssociateLists

class astro.recipes.AssociateList.AssociateListTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data analysis, sourcelists.

PURPOSE : associate SourceLists and/or AssociateLists

This task is used to associate two or more SourceLists and/or AssociateLists.

Note that when N SourceLists are associated, this is done by creating N-1 AssociateLists, where only the last created AssociateList contains the associations of all sources.

NOTE: CURRENTLY SUPPORT FOR ASSOCIATING MORE THAN TWO SOURCE/ASSOCIATE
LISTS IS LACKING.

Associate sourcelists

awe> task = AssociateListTask(ids=[(1101345, ‘s’), (123120, ‘s’)]) awe> task.execute()

Possible keywords :

ids : list of tuples in form (SLID/ALID identifier, type), example:
[(1101345, ‘s’), (123120, ‘a’)]
atype : integer (associatelisttype), values: 1-3, for “chain”,
“master”, resp. “matched” associations. Default 1.

name : string, name given to the AssociateList pars : parameters (dictionary), example:

{‘AssociateList.process_params.SEARCH_DISTANCE’: 6.0})
check_format_ids()
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_input_list(ident, ltype)
get_process_result()
get_process_time()

return the processing time of this task

make_associatelist()
parser = Recognized options for the astro.recipes.AssociateList OptionsParser are: "i" or "instrument" (default: ) "ids" or "ids" (default: []) "atype" or "atype" (default: 1) "n" or "name" (default: ) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_for_associatelist(alid)
query_for_sourcelist(slid)
target

alias of astro.main.AssociateList.AssociateList

exception astro.recipes.AssociateList.AssociateListTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Astrometry module

class astro.recipes.Astrometry.AstrometricParametersTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, astrometric pipeline.

PURPOSE : derive the astrometric calibration for a given frame.

This task is used to derive the astrometric calibration for a given frame.

The task assumes that a datasever is active and that it has been instantiated with a valid value for the ‘date’, ‘instrument’, ‘chip’, ‘filter’, and ‘object’ parameters. Alternatively, a list of file names can be given. The task is executed by calling its ‘execute’ method.

Example 1 of use from the awe prompt :

awe> task = AstrometricParametersTask(instrument=’OMEGACAM’,
chip=’ESO_CCD_#65’, filter=’OCAM_i_SDSS’, date=‘2015-03-01’, object=’KIDS_186.5_-2.5’)

awe> task.execute()

Example 2 of use from the awe prompt :

awe> task = AstrometricParametersTask(instrument=’WFI’,
red_filenames=[‘Sci-AWUSER-#843-ccd50.fits’])

awe> task.execute()

Example 3 of use from the awe prompt :

awe> task = AstrometricParametersTask(instrument=’WFI’,
raw_filenames=[‘WFI.2000-04-28T01:01:01_7.fits’,
‘WFI.2000-04-28T05:05:05_3.fits’])

awe> task.execute()

See the section below for an overview of the possible constructor keywords.

Two sets of query keywords can be provided in the constructor of the task :

  1. ONE of two possible lists of filenames.
  2. a set of global query parameters (like date, filter and chip).

The first of these sets has precedence over the second.

The first set of possible keywords is ONE of these two lists of filenames :

raw_filenames (filenames of the raw science frames, list of strings) red_filenames (filenames of the reduced science frames, list of strings)

Depending on which of these is given, a different query is send to the database.

The second set of possible keywords is:
date (string) instrument (string) chip (string) filter (string) template (string) object (string or using wildcards like M33_?_?, M33_V*)
Finally, the following parameters can also be set in the constructor :

commit (boolean False or True. Default False) inspect (integer 0 or 1. Default 0) pars (dictionary of configuration parameters, example:

{‘AstrometricParameters.process_params.MIN_NREF’: 16}})

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

return the processing time of this task

make_astrometric_parameters()
parser = Recognized options for the astro.recipes.Astrometry OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: None) "rfd" or "rfd_filenames" (default: []) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "p" or "pars" (default: {}) "L" or "ldac" (default: True) "S" or "scamp" (default: False) "H" or "header" (default: False) "C" or "commit" (default: False) "I" or "inspect" (default: 0)
query_database()
set_config(astrom_params, reduced)

Set instrument specific default config properties

set_dependencies(reducedframes)
target

alias of astro.main.AstrometricParameters.AstrometricParameters

exception astro.recipes.Astrometry.AstrometricParametersTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Astrometry.main()
astro.recipes.Astrometry.parse_command_line()
astro.recipes.Astrometry.usage()

astro.recipes.BDSM module

class astro.recipes.BDSM.BDSMTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, source extraction.

PURPOSE : Calling BDSM software and storing the resulting
sourcelist and sources, gaussian list and guassians shapelet list and shapelets parameters and statistics

into the database The residual file(s) will be stored on the dataserver.

Example of use from the awe prompt :

awe> query = RadioFrame.instrument.name == ‘WSRT’ awe> filename = query[0].filename awe> task = BDSMTask(filenames=[filename]) awe> task.execute()

Mandatory keywords :

filenames - The filenames of RadioFrame’s (or Cubes) to run BDSM on. input_cls - The input class (default=RadioFrame).

Optional keywords :

commit - Switch for committing the targets. pars - The process parameters dictionary, see the class

BDSM_GaussianParameters for possible parameters. This can be a single dictionary that will be used for all filenames or a list of dicitonaries for each filename.
check_preconditions()

check if mandatory parameters are set

execute()

execute the task

get_process_time()

return the processing time of this task

parser = Recognized options for the astro.recipes.BDSM OptionsParser are: "F" or "filenames" (default: []) "cls" or "input_cls" (default: <class 'astro.main.RadioCube.RadioFrame'>) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
run_bdsm()

Running BDSM consists of the following (sub) tasks 1. run bdsm on filename 2. read sources and commit SourceList 3. read gaussians and commit BDSM_GaussianList, BDSM_GaussianParameters and BDSM_Gaussian(s) 4. store residual and image files 5. read shapelets and commit BDSM_Shapelet(s) and BDSM_ShapeletParameters 6. read and store RadioFrame statistics (RadioStat)

exception astro.recipes.BDSM.BDSMTaskError

Bases: Exception

astro.recipes.BPZSourceCollectionTask module

Task for creating BPZ photometric redshifts.

class astro.recipes.BPZSourceCollectionTask.BPZSourceCollectionTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : create Bayesian photometric redshifts for (KIDS)
SourceCollections
ac = None
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

commit = False
execute()
gaap_multi = None
gaia = False
get_name()

Get the name of the to-be-made BPZ SourceCollection.

TODO: Something more sensible, and DR3 compatible.

get_process_result()
get_process_time()

Return the processing time of this task.

make_sourcecollection()
name = ''
parser = Recognized options for the astro.recipes.BPZSourceCollectionTask OptionsParser are: "scid" or "scid" (default: 0) "g" or "gaia" (default: 0) "n" or "name" (default: ) "C" or "commit" (default: 0)
process_result = None
process_time = 10800
query_database()
scid = 0
target

alias of astro.main.BPZAttributeCalculator.AttributeCalculator

astro.recipes.BackgroundFrameTask module

class astro.recipes.BackgroundFrameTask.BackgroundFrameTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a background frame for one or more specific raw
science frames

Example of use from the awe prompt :

awe> task = BackgroundFrameTask(instrument=’OMEGACAM’,
raw_filenames=[‘OMEGACAM.2013-11-20T10:10:10.123_1.fits’])

awe> task.execute()

Mandatory keywords :

instrument (string)

or

raw_filenames (list of strings)

Optional keywords :

time_from (string in yyyy-mm-ddThh:mm:ss format) time_to (string in yyyy-mm-ddThh:mm:ss format) overscan (integer: 0-8. Default 6) commit (integer: 0 or 1. Default 0) pars (dictionary of configuration parameters)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

return the processing time of this task

make_backgroundframe()
parser = Recognized options for the astro.recipes.BackgroundFrameTask OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "tpl" or "template" (default: None) "o" or "object" (default: ) "tf" or "time_from" (default: ) "tt" or "time_to" (default: ) "a" or "after" (default: 6) "b" or "before" (default: 6) "ta" or "time_after" (default: 3600) "tb" or "time_before" (default: 3600) "oc" or "overscan" (default: 6) "raw" or "raw_filenames" (default: []) "rfds" or "rfd_filenames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
query_raw_from_raw_filenames(raw_filenames)

Find the RawScienceFrame for which we want to create a BackgroundFrame

query_raw_from_rawfitsdata(rfd_filename, chip)

Find the RawScienceFrame for which we want to create a BackgroundFrame

set_config()

Set instrument specific default config properties

target

alias of astro.main.BackgroundFrame.BackgroundFrame

exception astro.recipes.BackgroundFrameTask.BackgroundFrameTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Bias module

class astro.recipes.Bias.BiasTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a masterbias for a specific date and chip.

This task is used to create BiasFrame objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = BiasTask(instrument=’OMEGACAM’, date=‘2015-02-12’,
chip=’ESO_CCD_#65’)

awe> task.execute()

Subimage processing :

awe> task = BiasTask(instrument=’OMEGACAM’, date=‘2015-02-12’,
chip=’ESO_CCD_#65’, subimage=(1,1,512,512))

awe> task.execute()

Mandatory keywords :

instrument (string), chip (string), date (string)
or
instrument (string), chip (string), template (string)
or

raw_filenames (list of strings)

Optional keywords :

overscan (integer: 0-13. Default 6) commit (boolean: False or True. Default False) pars (dictionary of configuration parameters, example:

{‘BiasFrame.process_params.SIGMA_CLIP’: 3.5})

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

return the processing time of this task

make_masterbias()
parser = Recognized options for the astro.recipes.Bias OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "oc" or "overscan" (default: 6) "raw" or "raw_filenames" (default: []) "path" or "path_names" (default: []) "cv" or "check_validity" (default: True) "cq" or "check_quality" (default: True) "sub" or "subimage" (default: ()) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties and set user specified values

target

alias of astro.main.BiasFrame.BiasFrame

exception astro.recipes.Bias.BiasTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Coadd module

class astro.recipes.Coadd.CoaddTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : coadding regridded frames

This task is used to coadd a set of regridded frames.

The task assumes that a dataserver is active, and that it has been instantiated with valid values for the ‘date’, ‘instrument’, ‘chip’, ‘filter’, and ‘object’ parameters. Alternatively, a list of file names can be given. The task is executed by calling its ‘execute’ method’.

Example 1 of use from the awe prompt :

awe> task = CoaddTask(instrument=’OMEGACAM’, filter=’OCAM_i_SDSS’,
date=‘2015-03-01’, object=’KIDS_186.5_-2.5’)

awe> coadd.execute()

Example 2 of use form the awe prompt :

awe> coadd = CoaddTask(instrument=’WFC’,
raw_filenames=[‘r336395_1.fits’,
‘r336395_2.fits’])

awe> coadd.execute()

Example 3 of use from the awe prompt :

awe> coadd = CoaddTask(instrument=’WFI’,
reg_filenames=[‘Sci-#842-Regr-ccd50-Regr.fits’,
‘Sci-#842-Regr-ccd51-Regr.fits’])

awe> coadd.execute()

See the section below for an overview of the possible constructor keywords.

Two sets of query keywords can be provided in the constructor of the task:

  1. ONE of three possible lists of filenames.
  2. a set of global query parameters (like date, filter and chip).

The first of these sets has precedence over the second.

The first set of possible keywords is ONE of these three lists of filenames:

raw_filenames (filenames of the raw science frames, list of strings) red_filenames (filenames of the reduced science frames, list of

strings)

reg_filenames (filenames of the regridded frames, list of strings)

Depending on which of these is given, a different query is send to the database.

The second set of possible keywords is:

date (string) instrument (string) chip (string) filter (string) object (string or using wildcards like M33_?_?, M33_V*)

Finally, it is possible to specify whether the process result should be committed or not by setting the following parameter in the constructor :

commit (boolean False or True. Default False) pars (dictionary of configuration parameters, example:

{‘CoaddedRegriddedFrame.process_params.MAXIMUM_PSF_DIFFERENCE’: 0.25})

Finally, if the length of the OBJECT name of the output CoaddedRegriddedFrame constructed from a concatenation of the OBJECT names of all the RegriddedFrames exceeds 297 characters, it can no longer be stored in the database. In this case, the name will be intelligently compressed (see CoaddedRegriddedFrame.derive_objectname) unless a replacement name is given in the parameter coadd_name.

coadd_name (string)

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

coadd_regridded_frames()
execute()
get_process_result()
get_process_time()

Return the processing time of this task

parser = Recognized options for the astro.recipes.Coadd OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: ) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "reg" or "reg_filenames" (default: []) "name" or "coadd_name" (default: ) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
query_database_for_manual_masks()
set_config()

Set instrument specific default config properties

store_manual_mask_comment()
target

alias of astro.main.RegriddedFrame.CoaddedRegriddedFrame

exception astro.recipes.Coadd.CoaddTaskError(message)

Bases: common.log.Error.Error

astro.recipes.ColdPixels module

class astro.recipes.ColdPixels.ColdPixelsTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a coldpixelmap for a specific instrument, date, chip
and filter.
The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = ColdPixelsTask(instrument=’OMEGACAM’, date=‘2015-02-16’,
chip=’ESO_CCD_#65’, filter=’OCAM_g_SDSS’)

awe> task.execute()

Mandatory keywords :

instrument (string), date (string), chip (string), filter (string)
or

instrument (string), template (string), chip (string)

Optional keywords :

commit : (boolean: False or True. Default False) pars : (dictionary of configuration parameters, example:

{‘ColdPixelMap.process_params.THRESHOLD_LOW’: 0.95})

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Return the processing time of this task

make_coldpixelmap()
parser = Recognized options for the astro.recipes.ColdPixels OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "twi" or "twilight" (default: False) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.ColdPixelMap.ColdPixelMap

exception astro.recipes.ColdPixels.ColdPixelsTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Convolve module

class astro.recipes.Convolve.ConvolveTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

execute()
make_all()
parser = Recognized options for the astro.recipes.Convolve OptionsParser are: "reg" or "reg_filenames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
target

alias of astro.main.ConvolvedFrame.ConvolvedFrame

exception astro.recipes.Convolve.ConvolveTaskError

Bases: Exception

astro.recipes.CrossTalkCorrectedFrameTask module

class astro.recipes.CrossTalkCorrectedFrameTask.CrossTalkCorrectedFrameTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, ingestion.

PURPOSE : making CrossTalkCorrectedFrames from RawFitsData objects

This task is used to create CrossTalkCorrectedFrame objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = CrossTalkCorrectedFrameTask(
filenames=[‘OMEGACAM.2014-11-31T00:00:00.123.fits’], chip_names=[‘ESO_CCD_#95’, ‘ESO_CCD_#96’])

awe> task.execute()

Mandatory keywords :

filenames : list of RawFitsData filenames chip_names : list of CCDs for which CrossTalkCorrectedFrames must be

created

Optional keywords :

commit : 0 or 1, commit results
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Return the estimated processing time of this task

make_crosstalkcorrectedframes()

Make the CrossTalk object (measure the cross-talk)

parser = Recognized options for the astro.recipes.CrossTalkCorrectedFrameTask OptionsParser are: "f" or "filenames" (default: []) "cn" or "chip_names" (default: []) "C" or "commit" (default: 0)
query_database()

Query the database for RawFitsData objects with filenames as given

target

alias of astro.main.CrossTalkCorrectedFrame.CrossTalkCorrectedRawScienceFrame

exception astro.recipes.CrossTalkCorrectedFrameTask.CrossTalkCorrectedFrameTaskError(message)

Bases: common.log.Error.Error

astro.recipes.CrossTalkTask module

class astro.recipes.CrossTalkTask.CrossTalkTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, ingestion.

PURPOSE : making CrossTalk(-Coefficients) from RawFitsData objects

This task is used to create CrossTalk objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = CrossTalkTask(filenames=[‘OMEGACAM.2014-11-31T00:00:00.123.fits’], chip_names=[‘ESO_CCD_#95’, ‘ESO_CCD_#96’]) awe> task.execute()

Mandatory keywords :

filenames : list of RawFitsData filenames chip_names : list of CCDs for which the CrossTalk is measured

Optional keywords :

commit : 0 or 1, commit results
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Return the processing time of this task

make_crosstalks()

Make the CrossTalk object (measure the cross-talk)

parser = Recognized options for the astro.recipes.CrossTalkTask OptionsParser are: "f" or "filenames" (default: []) "cn" or "chip_names" (default: []) "C" or "commit" (default: 0)
query_database()

Query the database for RawFitsData objects with filenames as given.

target

alias of astro.main.CrossTalk.CrossTalk

exception astro.recipes.CrossTalkTask.CrossTalkTaskError(message)

Bases: common.log.Error.Error

astro.recipes.DarkCurrent module

class astro.recipes.DarkCurrent.DarkCurrentTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : deriving the dark current and particle event rate of a chip.

This task is used to derive the dark current of a chip and the event rate of cosmic particles on the chip.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = DarkCurrentTask(instrument=’OMEGACAM’, chip=’ESO_CCD_#65’,
template=‘2015-02-10T08:03:50’)

awe> task.execute()

Mandatory keywords :

instrument (string), chip (string), time_from (string, yyyy-mm-ddThh:mm:ss) time_to (string, yyyy-mm-ddThh:mm:ss)

or

instrument (string), chip (string), template (string)

Optional keywords :

overscan (integer : 0-13. Default : 6) commit (boolean : False or True. Default : False) pars (dictionary of configuration parameters, example:

{‘DarkCurrent.process_params.REJECTION_THRESHOLD’: 6.0})

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Return the processing time of this task

make_darkcurrent()
parser = Recognized options for the astro.recipes.DarkCurrent OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "tpl" or "template" (default: None) "dt1" or "time_from" (default: ) "dt2" or "time_to" (default: ) "oc" or "overscan" (default: 6) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.DarkCurrent.DarkCurrent

exception astro.recipes.DarkCurrent.DarkCurrentTaskError(message)

Bases: common.log.Error.Error

astro.recipes.DomeFlat module

class astro.recipes.DomeFlat.DomeFlatTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a master dome flat for a specific date, chip and
filter.
This task is used to create a master dome flat. The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Mandatory keywords :

instrument (string), filter (string), chip (string), date (string)
or
instrument (string), filter (string), chip (string), template (string)
or

raw_filenames (list of strings)

Optional keywords:

overscan (integer: 0-13. Default 6) commit (boolean: False or True. Default False) pars (dictionary of configuration parameters, example:

{‘DomeFlatFrame.process_params.SIGMA_CLIP’: 4.0})

The type of the values assigned to the keywords is given between parentheses.

Example of use from the awe prompt :

awe> task = DomeFlatTask(instrument=’OMEGACAM’, date=‘2015-02-16’,
chip=’ESO_CCD_#65’, filter=’OCAM_g_SDSS’)

awe> task.execute()

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Return the processing time of this task

make_domeflat()
parser = Recognized options for the astro.recipes.DomeFlat OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "oc" or "overscan" (default: 6) "raw" or "raw_filenames" (default: []) "tpl" or "template" (default: None) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.DomeFlatFrame.DomeFlatFrame

exception astro.recipes.DomeFlat.DomeFlatTaskError(message)

Bases: common.log.Error.Error

astro.recipes.ESOIDTask module

Recipe to create ESOID identifiers for sources for ESO releases.

class astro.recipes.ESOIDTask.ESOIDTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : Data processing, image pipeline.

PURPOSE : Create ESO-IDs.

ac = None
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_name()
get_process_result()
get_process_time()

Return the processing time of this task.

make()

Find/make a ESOID catalog for a SourceCollection.

name = ''
parser = Recognized options for the astro.recipes.ESOIDTask OptionsParser are: "S" or "scid" (default: 0) "n" or "name" (default: ) "C" or "commit" (default: False)
process_result = None
process_time = 7200
query_database()
sc = None
target

alias of astro.main.ESOIDAttributeCalculator.AttributeCalculator

astro.recipes.ESOIDTask.main()

Run ESOIDTask from the command line.

astro.recipes.EuclidDataValidation module

Methods for Euclid EXT data validation

class astro.recipes.EuclidDataValidation.StackedImageValidationTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

Validation of stacked KiDS images for ingestion into EAS

This class provides data product validation for stacked KiDS images produced by the LIS (Astro-WISE) for ingestion into the EAS.

Four types of validation can be performed and switched on or off: - format validation of the data product; this verifies that the

data product conforms to the Euclid data model and writes the results in a log file
  • basic image validation; this extracts a source catalog from the image and writes a log file with the median PSF size and ellipticity, as well as the limiting magnitude
  • sanity checks on the astrometry and photometry; this associates the source catalog to the reference catalog and produces a matched catalog as well as a log file containing astrometric and photometric offsets and inspection figures
  • PSF model validation; TBD

The input FITS file should be present in the working directory.

Output is optional, depending on the validation modes used.

Format validation: - log file with validation results

Image validation: - a SExtractor-based catalog extracted from the image - log file with mean PSF size, mean PSF ellipticity, limiting mag

Photometric and astrometric inspection: - source catalog associated with reference catalog - log file with the statistics of the astrometric and photometric

comparisons
  • inspect figure showing astrometric and photometric comparisons with reference catalog

PSF model validation: - log file with validation results

filename (fn) : str
Filename of the stacked image file
format (F) : int
Perform format validation
validate (V) : int
Perform basic image validation
inspect (I) : int
Perform photometric and astrometric inspection
psfmodel (P) : int
Perform PSF model validation
execute()

Execute data validation for a Euclid EXT stacked image

make_associated_catalog()

Match sources between reference catalog and stacked image SExtractor catalog

make_sextractor_catalog()

Run SExtractor on stacked image

parser = Recognized options for the astro.recipes.EuclidDataValidation OptionsParser are: "fn" or "filename" (default: ) "F" or "format" (default: 1) "V" or "validate" (default: 1) "I" or "inspect" (default: 1) "P" or "psfmodel" (default: 0)
query_database()

Query AW database for reference catalog

validate_format()

Perform format validation

validate_image()

Perform basic image validation

validate_photometry_astrometry()

Perform photometric and astrometric validation

validate_psfmodel()

Perform PSF model validation

exception astro.recipes.EuclidDataValidation.StackedImageValidationTaskError(message)

Bases: common.log.Error.Error

astro.recipes.EuclidDataValidation.plot_statistics_from_logs(plotname='StackedImageValidation_statistics.png')

Plots statistics from StackedImageValidationTask log files in working directory.

plotname : str, optional
name of output figure (default = StackedImageValidation_statistics.png)

astro.recipes.EuclidExport module

class astro.recipes.EuclidExport.EUCLIDCanonicalName

Bases: object

Every Euclid file name should be composed of six fields separated with an underscore as follows (maximum size of 128 characters):

MMM_CCC_TTTTTTTTTTTTTTT_<instanceID>_<CreationDate>_<Release>

where

MMM: Mission ID. EUC for all MER products. CCC: Processing function acronym. MER for all MER products. TTTTTTTTTTTTTTT: Data product naming. See discussion bellow. <instanceID>: Instance ID that differentiates products of the same type (e.g. central coordinates). <CreationDate>: Data product creation date: YYYYMMDDTHHmmss.sZ (e.g., 20161019T103022.5Z) <Release>: Data release version: 00.00 (e.g., 00.01)
Examples:
filename = ‘EUC_MER_REBINNED-PSF-DESG__20161019T103022.5Z_00.01’ filename = ‘EUC_MER_BGSUB-STACK-DESG__20161019T103022.5Z_00.01’ filename = ‘EUC_MER_MOSAIC-VIS-FLAG__20161019T103022.5Z_00.01’ filename = ‘EUC_MER_BGSUB-MOSAIC-DESG-RMS__20161019T103022.5Z_00.01’
creation_date = None
data_product = 'MASTER-BIAS'
filter_name = ''
instance_id = ''
mission_id = 'KID'
processing_function = 'EXT'
release = '00.01'
to_string()
class astro.recipes.EuclidExport.EuclidExportTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data export PURPOSE : exporting KiDS LIS objects as data products with all

recursive metadata into an XML file and converting the pixel data to a Euclid-compatible form

This task is used to create an XML file per data product and converted pixel data files. The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

wd or workdir : str
Directory containing all working files, including inputs and outputs.
ld or logdir : str
Directory containing all log files.
si or singles : str
Pathname containing the list of output DpdExtSingleImages.
st or stack : str
Pathname containing the output DpdExtStackedImages.

TBW

clean_up_before_pickle()

This method is called before the Job is pickled on the cluster side, remove at least all the database objects.

execute()
make_data_products()
parser = Recognized options for the astro.recipes.EuclidExport OptionsParser are: "wd" or "workdir" (default: .) "ld" or "logdir" (default: .) "d" or "detrendeds" (default: ./detrendeds.json) "f" or "frames" (default: []) "C" or "commit" (default: False)
save_data_files()
save_data_products()
save_logs()
set_dependencies(dependencies)
set_up_workspace()

Change the current working directory to the workdir and set up any new directories.

target

alias of astro.main.ReducedScienceFrame.ReducedScienceFrame

exception astro.recipes.EuclidExport.EuclidExportTaskError(message)

Bases: common.log.Error.Error

astro.recipes.EuclidExport.combineSignalRMSFlagBackMaps(frame)

Combine a signal map, a RMS map, flag map, and background into one MEF.

frame : BaseFrame
object containing a WeightFrame in weight attribute
str
filename of a multi-extension FITS
astro.recipes.EuclidExport.convertPixelMapsToFlagMap(frame, filename='flagMap.fits')

Combine the set of PixelMaps in a WeightFrame into a flagMap.

frame : BaseFrame
object containing a WeightFrame in weight attribute
filename : str
name of the file to write the flagMap into
BitMask
a high-level object containing the created flagMap
astro.recipes.EuclidExport.convertWeightsToRMS(frame, filename='RMS.fits')

Convert a SExtractor output inverse variance map to RMS map.

The inverse variance map is converted in-place to an RMS map.

frame : BaseFrame
object containing a WeightFrame in weight attribute
astro.recipes.EuclidExport.dumpListToJson(pathnames, filename)
class astro.recipes.EuclidExport.extAstromSolution(reduced)

Bases: object

make()
queryDatabase()
writeXml()
class astro.recipes.EuclidExport.extBackgroundMap(reduced)

Bases: object

make()
make_image()
set_canonical_name()

Filename for Background image

writeXml()
class astro.recipes.EuclidExport.extBiasFrame(bias)

Bases: object

get_canonical_name()
make()
update_filename()

Give the data product a Euclid compliant name

writeXml()
class astro.recipes.EuclidExport.extDetrendedFrame(reduced)

Bases: object

convertData()
make()
set_spatial_footprint()
writeXml()
class astro.recipes.EuclidExport.extMasterFlatFrame(flat)

Bases: object

get_canonical_name()
make()
update_filename()

Give the data product a Euclid compliant name

writeXml()
class astro.recipes.EuclidExport.extPhotomSolution(reduced)

Bases: object

make()
queryDatabase()
writeXml()
class astro.recipes.EuclidExport.extPsfModel(reduced)

Bases: object

make()
make_image()
set_canonical_name()
writeXml()
class astro.recipes.EuclidExport.extRawScienceFrame(raw)

Bases: object

get_canonical_name()

The Euclid compliant name

make()
update_filename()

Give the data product a Euclid compliant name

writeXml()
astro.recipes.EuclidExport.getDataProduct(frame)

Create all parts of the the data product MER needs for a single image: combined signal, RMS, and flag maps, background map, and PSF model.

frame : BaseFrame
object containing the signal map from which to create the data product
(str, str, str)
a tuple of three filenames (combined, back, and psf) of the output FITS images
astro.recipes.EuclidExport.get_canonical_name_background(frame)

Filename for Background image

astro.recipes.EuclidExport.get_canonical_name_combined(frame)

Filename for the combined image

astro.recipes.EuclidExport.get_canonical_name_flag(frame)

Filename for Flag / Mask image

astro.recipes.EuclidExport.get_canonical_name_rms(frame)

Filename for RMS image

astro.recipes.EuclidExport.get_canonical_name_signal(frame)

Filename for the Signal (=Science?) image

astro.recipes.EuclidExport.import_data_to_eas_dss(filenames=[], username='', password='', commit=False, verbose=False)

Import referenced data files into the Euclid dataserver (EAS-DSS).

filenames : list
Names of files to be imported.
username : str
Username credential for import server.
password : str
Password credential for import server.
commit : bool
Save the objects.
verbose : bool
Give extra timing information.
astro.recipes.EuclidExport.import_xml_to_eas_dps(xmlNames=[], tgzNames=[], importServer='eas-dps-mis.test.euclid.astro.rug.nl', username='', password='', project='EUCLID', privileges=2, commit=False, verbose=False)

Import objects in XML form into the Euclid database (EAS-DPS).

xmlNames : list
Names of XML files to be imported.
tgzNames : list
Names of tar-gzipped archives containing a hierarchy of XML files to be imported.
importServer : str
URL of EAS-DPS import server.
username : str
Username credential for import server.
password : str
Password credential for import server.
project : str
Project in which to import the objects.
privileges : int
Privileges at which to import the objects.
commit : bool
Save the objects.
verbose : bool
Give extra timing information.
list
Filenames of data files associated with the XML objects.
astro.recipes.EuclidExport.load_map(pixelMap)

Use DARMA to load the pixelmap property of a PixelMap.

pixelMap : PixelMap
object containg the pixelmap to be loaded
astro.recipes.EuclidExport.main()

Ingest data products from XML files into the EAS-DPS and referenced data files into the EAS-DSS.

astro.recipes.EuclidExport.makeBackgroundMap(frame)

Create a SExtractor background image for the input frame.

frame : BaseFrame
object containing signal map for which to determine the background
str
filename of the output FITS image
astro.recipes.EuclidExport.makePsfModel(frame)

Create a PSFEx PSF model for the input frame.

frame : BaseFrame
object containing the signal map from which to create a PSF model
str
filename of the output FITS image
astro.recipes.EuclidExport.round_date_to_microseconds(date, digits_to_show=1)

astro.recipes.EuclidExportSimple module

class astro.recipes.EuclidExportSimple.EuclidExportRawScienceFrameTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

execute()
import_xml_to_eas()
make_data_products()
parser = Recognized options for the astro.recipes.EuclidExportSimple OptionsParser are: "un" or "username" (default: ) "pw" or "password" (default: ) "db" or "database" (default: test) "pro" or "project" (default: EUCLID) "pri" or "privileges" (default: 2) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "b" or "bias_filenames" (default: []) "m" or "flat_filenames" (default: []) "cf" or "coadd_filename" (default: ) "f" or "frames" (default: []) "C" or "commit" (default: False) "V" or "verbose" (default: 0)
query_database()
target

alias of astro.main.ReducedScienceFrame.ReducedScienceFrame

exception astro.recipes.EuclidExportSimple.EuclidExportRawScienceFrameTaskError(message)

Bases: common.log.Error.Error

astro.recipes.EuclidExportSimple.import_data_to_eas_dss(filenames=[], username='', password='', commit=False, verbose=False)

Import referenced data files into the Euclid dataserver (EAS-DSS).

filenames : list
Names of files to be imported.
username : str
Username credential for import server.
password : str
Password credential for import server.
commit : bool
Save the objects.
verbose : bool
Give extra timing information.
astro.recipes.EuclidExportSimple.main()

Ingest data products from XML files into the EAS-DPS and referenced data files into the EAS-DSS.

astro.recipes.EuclidImport module

Import from XML files the metadata of Euclid DPS data products into the KiDS Legacy Information System (LIS).

class astro.recipes.EuclidImport.EuclidImportTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data import PURPOSE : importing science frame objects with all recursive

metadata from an XML file or archive of XML files based on a datamodel stored in an XSD file

This task is used to create LIS DataObjects from a set of XML files. The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

wd or workdir : str
Directory containing all working files, including inputs and outputs.
m or mdb : str
Pathname relative to workdir containing the Mission DataBase.
r or raws : str
Pathname relative to workdir containing the list of input DpdRawScienceFrames.
b or biases : str
Pathname relative to workdir containing the list of input DpdBiasFrames.
fl or flats : str
Pathname relative to workdir containing the list of input DpdMasterFlatFrames.
h or hots : str
Pathname relative to workdir containing the list of input DpdHotPixelMaps.
c or colds : str
Pathname relative to workdir containing the list of input DpdColdPixelMaps.
i or illums : str
Pathname relative to workdir containing the list of input DpdIlluminationCorrectionFrames.
fr or fringes : str
Pathname relative to workdir containing the list of input DpdFringeFrames.
p or pars : str
Pathname relative to workdir containing the processing parameters.
si or singles : str
Pathname relative to workdir containing the list of output DpdExtSingleImages.
st or stack : str
Pathname relative to workdir containing the output DpdExtStackedImages.

TBW

check_mission_database()

Parse and check the LIS parameters stored in the Mision DataBase. Not yet implemented.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

create_targets()

Create TargetProcessor dictionaries containing dependencies

execute()

Assembly of Task steps.

get_process_result()

This method is used to retrieve the result from executing the task. This result should be a list of instances of one of the many sub-types of DBObject.

import_pars()

Parse the process parameters from XML input. Not yet implemented.

import_raws()

Parse the source data from the XML inputs.

parser = Recognized options for the astro.recipes.EuclidImport OptionsParser are: "wd" or "workdir" (default: .) "m" or "mdb" (default: ) "r" or "raws" (default: ) "b" or "biases" (default: ) "fl" or "flats" (default: ) "h" or "hots" (default: ) "c" or "colds" (default: ) "i" or "illums" (default: ) "fr" or "fringes" (default: ) "d" or "detrendeds" (default: ) "C" or "commit" (default: False)
set_up_workspace()

Change the current working directory to the workdir and set up any new directories.

target

alias of astro.main.AstrometricParameters.AstrometricParameters

update_kwargs()

Determine which extra options need to passed to the next task and update the kwargs accordingly.

exception astro.recipes.EuclidImport.EuclidImportTaskError(message)

Bases: common.log.Error.Error

astro.recipes.EuclidImport.getObjectFromXml(filename, cls)

Parse the XML in filename and return a DataObject in the LIS, ingesting if necessary.

NB: All DataObjects originate in the LIS. Only modified versions
will need to be ingested. Currently, this function does not do this.
filename : str
name of XML file to parse
cls : subclass of DataObject
the class of the instance to instantiate from/ingest into the database

an instance of a DataObject subclass

astro.recipes.EuclidImport.getOidFromXml(filename)

Parse the XML in filename and return the extObjectId of the DataObject in the LIS.

filename : str
Name of XML file to parse.
oracleoidtype
extObjectId stored in XML file cast to an oracleoidtype.
astro.recipes.EuclidImport.loadListFromJson(filename)

astro.recipes.EuclidLISTest module

Test task for EudlicLIS

class astro.recipes.EuclidLISTest.EuclidLISTestTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()

This method is used to retrieve the result from executing the task. This result should be a list of instances of one of the many sub-types of DBObject.

make_test_output()
parser = Recognized options for the astro.recipes.EuclidLISTest OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "wd" or "workDir" (default: ) "ld" or "logDir" (default: ) "ip" or "inputs" (default: []) "op" or "outputs" (default: [])
query_database()
retrieve_file()
target = None
exception astro.recipes.EuclidLISTest.EuclidLISTestTaskError(message)

Bases: common.log.Error.Error

astro.recipes.ExportCoaddForESOTask module

Recipe to export a KiDS coadd (+ weight frame, mask and (KiDS-CAT) single-band SourceList) to the format required for ESO Phase 3

class astro.recipes.ExportCoaddForESOTask.ExportCoaddForESOTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : create ESO-compliant FITS files for a KiDS DR coadd, its
weight frame, and corresponding mask and KiDS-CAT SourceList
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

create_sourcelist()
execute()
find_mask()
find_sourcelist()
get_process_result()
get_process_time()

Return the processing time of this task

make_delivery_headers()
make_output_files()
parser = Recognized options for the astro.recipes.ExportCoaddForESOTask OptionsParser are: "v" or "version" (default: 0) "c" or "coaddname" (default: ) "C" or "commit" (default: 0) "f" or "ftpfolder" (default: ) "o" or "overwrite" (default: 0) "d" or "delete" (default: 1)
query_database()
set_config()

Set instrument specific default config properties

target = None
upload_output_files()
exception astro.recipes.ExportCoaddForESOTask.ExportCoaddForESOTaskError(message)

Bases: common.log.Error.Error

class astro.recipes.ExportCoaddForESOTask.KiDSHeader

Bases: object

addcard(item, value, comment)
astro.recipes.ExportCoaddForESOTask.deg2dms(degree)
astro.recipes.ExportCoaddForESOTask.getDATEItems(rawfitsdatas, name)
astro.recipes.ExportCoaddForESOTask.getDOUBLEItems(rawfitsdatas, name)
astro.recipes.ExportCoaddForESOTask.getINTEGERItems(rawfitsdatas, name)
astro.recipes.ExportCoaddForESOTask.getSTRINGItems(rawfitsdatas, name)
astro.recipes.ExportCoaddForESOTask.getTUNIT(column)

astro.recipes.ExtinctionSourceCollectionTask module

Recipe to create SourceCollections containing results of running the ExtinctionAttributeCalculator code.

class astro.recipes.ExtinctionSourceCollectionTask.ExtinctionSourceCollectionTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : Data processing, image pipeline.

PURPOSE : Run Extinction code and store resulting
flag values in a SourceCollection.
ac = None
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_name()
get_process_result()
get_process_time()

Return the processing time of this task.

make()

Find/make a Extinction catalog for a SourceCollection.

name = ''
parser = Recognized options for the astro.recipes.ExtinctionSourceCollectionTask OptionsParser are: "S" or "scid" (default: 0) "n" or "name" (default: ) "C" or "commit" (default: False)
process_result = None
process_time = 7200
query_database()
sc = None
target

alias of astro.main.ExtinctionAttributeCalculator.AttributeCalculator

astro.recipes.ExtinctionSourceCollectionTask.main()

Run ExtinctionSourceCollectionTask from the command line.

astro.recipes.FringeFlat module

class astro.recipes.FringeFlat.FringeFlatTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a fringe frame for a specific date, chip and filter

This task is used to create a fringe map.

The task assumes that a dataserver is active, and that it has been instantiated with valid values for the ‘instrument’, ‘date’, ‘chip’, ‘filter’ and (optionally) ‘object’, ‘time_from’, ‘time_to’, and ‘overscan’ parameters. The task is executed by calling its ‘execute’ method.

Example of use from the awe prompt :

awe> task = FringeFlatTask(instrument=’OMEGACAM’, chip=’ESO_CCD_#65’,
template=‘2015-02-13T08:28:41’)

awe> task.execute()

Mandatory keywords :

instrument (string), date (string), chip (string), filter (string)

or

raw_filenames (list of strings)

Optional keywords :

time_from (string in yyyy-mm-ddThh:mm:ss format) time_to (string in yyyy-mm-ddThh:mm:ss format) overscan (integer: 0-13. Default 6) commit (boolean: False or True. Default False) pars (dictionary of configuration parameters)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Estimate the processing time of this task

make_fringemap()
parser = Recognized options for the astro.recipes.FringeFlat OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "o" or "object" (default: ) "dt1" or "time_from" (default: ) "dt2" or "time_to" (default: ) "oc" or "overscan" (default: 6) "raw" or "raw_filenames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.FringeFrame.FringeFrame

exception astro.recipes.FringeFlat.FringeFlatTaskError(message)

Bases: common.log.Error.Error

astro.recipes.GAstrom module

class astro.recipes.GAstrom.GAstromTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, astrometric pipeline.

PURPOSE : derive an improved astrometric calibration for a given
frame from a global astrometrical solution.

This task is used to derive an improved astrometric calibration from a given frame from a global astrometrical solution.

The task assumes that a datasever is active and that it has been instantiated with a valid value for the ‘date’, ‘instrument’, ‘chip’, ‘filter’, and ‘object’ parameters, Alternatively, a list of file names can be given, or a list of names of sourcelists. The task is executed by calling its ‘execute’ method.

Example 1 of use from the awe prompt:

awe> task = GAstromTask(instrument=’WFI’, date=‘2003-02-12’, … chip=’ccd50’, filter=’#842’, … object=’CDF4_B_?’) awe> task.execute()

Example 2 of use from the awe prompt:

awe> task = GAstromTask(red_filenames=[‘Sci-AWUSER-#843-ccd50-1.fits’, … ‘Sci-AWUSER-#843-ccd50-2.fits, … …]) awe> task.execute()

Example 3 of use from the awe prompt:

awe> task = GAstomTask(raw_filenames=[
‘WFI.2000-04-28T01:01:01_7.fits’,

… ‘WFI.2000-04-28T05:05:05_3.fits’, …]) awe> task.execute()

Example 4 of use from the awe prompt:

awe> task = GAstromTask(names=[
‘GAS-Sci-AWUSER-#843-ccd50-1.slist,

… ‘GAS-Sci-AWUSER-#843-ccd50-2.slist, …])

See the section below for an overview of the possible constructor keywords.

Three sets of query keywords can be provided in the constructor of the task:

  1. a list of sourcelist names derived for the purpose of making a GAS.
  2. ONE of two possible lists of filenames of frames.
  3. a set of global query parameters (like date, filter and chip).

The first of these sets has precedence over the second, the second has precedence over the last.

The first set of possible keywords is:

names (names of sourcelists, list of strings) slids (SLIDs of sourcelists, list of ints/longs)

The second set of possible keywords is ONE of these two lists of filenames of frames:

raw_filenames (filenames of the raw science frames, list of strings) red_filenames (filenames of the reduced science frames, list of

strings)

Depending on which of these is given, a different query is send to the database.

The final set of possible keywords is:
date (string) instrument (string) chip (string) filter (string) object (string or using wildcards like M33_?_?, M33_V*)
Finally, the following parameters can also be set in the constructor:
pars (dictionary of configuration parameters) commit (integer 0 or 1. Default 0) inspect (integer 0 or 1. Default 0) check_existing (boolean True or False. Default False)

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

check_if_solution_exists()

Check if there is already a GAstrometric present

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Return the processing time of this task

make_global_astrometric_solution()
parser = Recognized options for the astro.recipes.GAstrom OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: ) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "ce" or "check_existing" (default: False) "N" or "names" (default: []) "S" or "slids" (default: []) "globalphot" or "globalphot" (default: False) "p" or "pars" (default: {}) "I" or "inspect" (default: False) "C" or "commit" (default: False) "l" or "ldac" (default: False)
query_database()
target

alias of astro.main.AssociateList.AssociateList

exception astro.recipes.GAstrom.GAstromTaskError(message)

Bases: common.log.Error.Error

astro.recipes.GAstrom.main()
astro.recipes.GAstrom.parse_commandline()
astro.recipes.GAstrom.usage()

astro.recipes.GAstromSourceList module

class astro.recipes.GAstromSourceList.GAstromSourceListTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data analysis, sourcelists.

PURPOSE : making sourcelists from reduced science frames, using the most
recent astrometric solution as input

This task is used to create sourcelists from reduced science frames and astrometric parameters.

The task assumes that a dataserver is active, and that it has been instantiated with valid combinations of the parameters ‘date’, ‘instrument’, ‘chip’, ‘filter’, and ‘object’, Alternatively, a list of filenames can be given. The task is executed by calling its ‘execute’ method’.

Example 1 of use from the awe prompt :

awe> task = GAstromSourceListTask(date=‘2000-04-28’, chip=’ccd50’,
instrument=’WFI’, filter=’#842’, object=’M31_?’)

awe> task.execute()

Example 2 of use from the awe prompt :

awe> task = GAstromSourceListTask(red_filenames=[‘Sci-#842-ccd50-Red.fits’]) awe> task.execute()

See the section below for an overview of the possible constructor keywords.

Two sets of query keywords can be provided in the constructor of the task:

  1. ONE of two possible lists of filenames.
  2. a set of global query parameters (like date, filter and chip).

The first of these sets has precedence over the second.

The first set of possible keywords is ONE of these two lists of filenames:

raw_filenames (filenames of the raw science frames, list of strings) red_filenames (filenames of the reduced science frames, list of

strings)

Depending on which of these is given, a different query is send to the database.

The second set of possible keywords is:
date (string) instrument (string) chip (string) filter (string) object (string or using wildcards like M33_?_?, M33_V*)

Finally, the following parameters can also be set in the constructor:

pars (dictionary of configuration parameters) commit (integer 0 or 1. Default 0) check_existing (boolean True or False. Default False) use_ldac (boolean True or False. Default False)

The commit switch commits all results to the database/dataserver recursively, check_existing forces the use of existing GAstromSourceList objects, and use_ldac forces the use of LDAC tools to create the catalog to be ingested as a SourceList.

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

return the processing time of this task

make_sourcelist()
parser = Recognized options for the astro.recipes.GAstromSourceList OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: ) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "ce" or "check_existing" (default: False) "ul" or "use_ldac" (default: False) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
set_dependencies(astromparams)
target

alias of astro.main.SourceList.SourceList

exception astro.recipes.GAstromSourceList.GAstromSourceListTaskError(message)

Bases: common.log.Error.Error

astro.recipes.GAstromSourceList.main()
astro.recipes.GAstromSourceList.parse_command_line()
astro.recipes.GAstromSourceList.usage()

astro.recipes.GPhotom module

Experimental Global Photometry recipe

class astro.recipes.GPhotom.GPhotomTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, photometric pipeline.

PURPOSE : derive an improved photometric calibration for a given
frame from a global photometric solution.

This task is used to derive an improved photometric calibration from a global photometrical solution. The GAstrometric framework (including AssociateList class) is used to assemble the input source pairings.

The task assumes that a datasever is active and that it has been instantiated with a valid value for the ‘date’, ‘instrument’, ‘filter’, and ‘object’ parameters, Alternatively, a list of file names can be given, or a list of names of sourcelists. The task is executed by calling its ‘execute’ method.

Example 1 of use from the awe prompt:

awe> task = GPhotomTask(instrument=’OMEGACAM’, date=‘2013-02-12’, … filter=’OCAM_r_SDSS’, object=’KIDS*’) awe> task.execute()

Example 2 of use from the awe prompt:

awe> task = GPhotomTask(red_filenames=[‘Sci-AWUSER-#843–1.fits’, … ‘Sci-AWUSER-#843–2.fits, … …]) awe> task.execute()

Example 3 of use from the awe prompt:

awe> task = GPhotomTask(raw_filenames=[
‘OMEGACAM.2012-04-28T01:01:01_7.fits’,

… ‘OMEGACAM.2012-04-28T05:05:05_3.fits’, …]) awe> task.execute()

Example 4 of use from the awe prompt:

awe> task = GPhotomTask(names=[
‘GAS-Sci-AWUSER-#843–1.slist,

… ‘GAS-Sci-AWUSER-#843–2.slist, …])

See the section below for an overview of the possible constructor keywords.

Three sets of query keywords can be provided in the constructor of the task:

  1. a list of sourcelist names derived for the purpose of making a GPS.
  2. ONE of two possible lists of filenames of frames.
  3. a set of global query parameters (like date and filter).

The first of these sets has precedence over the second, the second has precedence over the last.

The first set of possible keywords is:

names (names of sourcelists, list of strings) slids (SLIDs of sourcelists, list of ints)

The second set of possible keywords is ONE of these two lists of filenames of frames:

raw_filenames (filenames of the raw science frames, list of strings) red_filenames (filenames of the reduced science frames, list of

strings)

Depending on which of these is given, a different query is send to the database.

The final set of possible keywords is:
date (string) instrument (string) chip (string) filter (string) object (string or using wildcards like M33_?_?, M33_V*)
Finally, the following parameters can also be set in the constructor:
pars (dictionary of configuration parameters) commit (integer 0 or 1. Default 0) inspect (integer 0 or 1. Default 0) check_existing (boolean True or False. Default False)

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

check_if_solution_exists()

Check if there is already a GPhotometric present

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

return the processing time of this task

make_global_photometric_solution()
parser = Recognized options for the astro.recipes.GPhotom OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: ) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "ce" or "check_existing" (default: False) "N" or "names" (default: []) "S" or "slids" (default: []) "r" or "refsl" (default: None) "p" or "pars" (default: {}) "I" or "inspect" (default: 0) "C" or "commit" (default: 0) "s" or "scamp" (default: 0)
query_database()
target

alias of astro.main.AssociateList.AssociateList

exception astro.recipes.GPhotom.GPhotomTaskError(message)

Bases: common.log.Error.Error

astro.recipes.GPhotom.main()
astro.recipes.GPhotom.parse_commandline()
astro.recipes.GPhotom.usage()

astro.recipes.GaAPSourceCollectionTask module

Recipe to create SourceCollections containing results of running the

Gaussian aperture and PSF (GaaP) code.

See Konrad Kuijken, Astronomy and Astrophysics, Volume 482, Issue 3, 2008, pp.1053-1067

http://adsabs.harvard.edu/abs/2008A%26A…482.1053K http://arxiv.org/abs/astro-ph/0610606

class astro.recipes.GaAPSourceCollectionTask.GaAPSourceCollectionTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : run Gaussian aperture and PSF (GaaP) code and store resulting
flux values in a SourceCollection
ac = None
clean_up_before_pickle()
commit = False
execute()
gauss = None
get_gaap(sc, gauss)

Find/make a GAaP catalog for a SourceCollection.

See GAaPAttributeCalculator.py for details.

get_process_result()
get_process_time()

Return the processing time of this task.

make_sourcecollection()
name = None
parser = Recognized options for the astro.recipes.GaAPSourceCollectionTask OptionsParser are: "S" or "scid" (default: 0) "s" or "slid" (default: 0) "g" or "gauss" (default: ) "n" or "name" (default: ) "a" or "apertures" (default: [0.7, 2.0]) "C" or "commit" (default: False)
process_result = None
process_time = 7200
scid = 0
slid = 0
target

alias of astro.main.sourcecollection.AttributeCalculator.AttributeCalculator

exception astro.recipes.GaAPSourceCollectionTask.GaAPSourceCollectionTaskError(message)

Bases: common.log.Error.Error

Error for GaAPSourceCollectionTask.

astro.recipes.GaAPSourceCollectionTask.main()

Call GaAPSourceCollectionTask from the command-line.

astro.recipes.Gain module

class astro.recipes.Gain.GainTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : deriving the gain and checking the linearity of a chip.

This task is used to derive the gain of a chip and check its linearity.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Mandatory keywords :

instrument (string), chip (string), date (string), filter (string)
or
instrument (string), chip (string), template (string)
or

instrument (string), chip (string), time_from (string), time_to (string)

Optional keywords :

overscan (integer: 0-13. Default 6) commit (boolean: False or True. Default False) pars (dictionary of configuration parameters)

The type of the values assigned to the keywords is given between parentheses.

Example of use from the awe prompt :

awe> task = GainTask(instrument=’OMEGACAM’, chip=’ESO_CCD_#65’,
template=‘2015-02-14T14:37:35’, commit=False)

awe> task.execute()

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Estimate the processing time of this task

make_gainlinearity()
parser = Recognized options for the astro.recipes.Gain OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "oc" or "overscan" (default: 6) "dt1" or "time_from" (default: ) "dt2" or "time_to" (default: ) "tpl" or "template" (default: None) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.GainLinearity.GainLinearity

exception astro.recipes.Gain.GainTaskError(message)

Bases: common.log.Error.Error

astro.recipes.GalFit module

class astro.recipes.GalFit.GalFitTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

Example of use from the awe prompt:

awe> task = GalFitTask(slid=401441, sids=[10]) awe> task.execute()

awe> dpu.run(‘GalFit’, i=’WFC’, slid=61239, sids=range(1, 100), m=[‘sersic’, ‘sky’])

Optional keywords:

slid (integer: SourceList identifier) sids (list: list of Source identifiers for which to create

GalFitModels)
gflid (integer: GalFitList identifier; this model will be part of
the (previously created) GalFitList with this identifier)

models (list of dictionaries: describe model components and their initial values) constaints (list of dictionaries) commit (integer: 0 or 1. Default 0) pars (dictionary of configuration parameters, example:

{‘DomeFlatFrame.process_params.SIGMA_CLIP’: 4.0})
execute()
get_process_time()

Return the processing time of this task

make_galfitmodels()
parser = Recognized options for the astro.recipes.GalFit OptionsParser are: "slid" or "slid" (default: 0) "sids" or "sids" (default: []) "gflid" or "gflid" (default: 0) "m" or "models" (default: []) "c" or "constraints" (default: []) "t" or "process_time" (default: 0) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
target

alias of astro.main.GalFitModel.GalFitModel

exception astro.recipes.GalFit.GalFitTaskError(message)

Bases: common.log.Error.Error

astro.recipes.GalPhot module

class astro.recipes.GalPhot.GalPhotTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

Example of use at the awe prompt:

awe> task = GalPhotTask(slid=40441, sids=[10]) awe> task.execute()

awe> dpu.run(‘GalPhot’, slid=61239, sids=range(1, 100))

Optional keywords:

slid (integer: SourceList identifier) sids (list: list of Source identifiers for which to create GalPhotModels) gpid_in (integer: GalPhotModel identifier, this model is used as an initial estimate for the newly created models) gplid (integer: GalPhotList identifier; this model will be part of the (previously created) GalPhotList with this identifier) mask_files (list: list of filenames of ASCII files containing masked areas in pixel position of original frame, one file for each SID) pars (dictionary: contains process parameters) residual (integer: 0 or 1, 1: also commit the GalPhotResidual image when commit=1) commit (integer: 0 or 1, 1: commit the results to the database)
execute()
get_process_time()

Return the processing time of this task.

make_galphotmodels()
parser = Recognized options for the astro.recipes.GalPhot OptionsParser are: "slid" or "slid" (default: 0) "sids" or "sids" (default: []) "gplid" or "gplid" (default: 0) "gin" or "gpid_in" (default: 0) "m" or "mask_files" (default: []) "r" or "residual" (default: 0) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
target

alias of astro.main.GalPhotModel.GalPhotModel

exception astro.recipes.GalPhot.GalPhotTaskError(message)

Bases: common.log.Error.Error

astro.recipes.GaussianizedFrameTask module

class astro.recipes.GaussianizedFrameTask.GaussianizedFrameTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : create a GaussianizedFrame

Two main inputs are used:
  • All RegriddedFrames for a single exposure (32 for KiDS)
    or
  • A CoaddedRegriddedFrame

Two versions of the Gaussianization code can be used and indicated with the v / code_version parameter:

  • 1: pre-2017 version, including tweaking step (default)
  • 2: 2017 version

The 2017 version of the code is faster and does not create a PSF map

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_pixelscale()
get_process_result()
get_process_time()

return the processing time of this task

get_regridded_frames_for_template(regrid)
get_target_gauss_stdev()

Returns the standard deviation of the Gaussian PSF, in pixel units

get_target_psf_size()
make_gaussianized_frame()
parser = Recognized options for the astro.recipes.GaussianizedFrameTask OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: ) "rfd" or "rawfitsdata" (default: ) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "reg" or "reg_filenames" (default: []) "do" or "dateobs" (default: ) "cf" or "coadd_filename" (default: ) "name" or "coadd_name" (default: ) "s" or "size" (default: 0.0) "p" or "pars" (default: {}) "v" or "code_version" (default: 1) "C" or "commit" (default: 0)
query_database()
set_config()

Set instrument specific default config properties

set_detector_block_extent()
target

alias of astro.main.GaussianizedFrame.GaussianizedFrame

exception astro.recipes.GaussianizedFrameTask.GaussianizedFrameTaskError(message)

Bases: common.log.Error.Error

astro.recipes.GaussianizedFrameTask.main()

astro.recipes.HotPixels module

class astro.recipes.HotPixels.HotPixelsTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a hotpixelmap for a specific date and chip.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = HotPixelsTask(instrument=’OMEGACAM’, date=‘2015-02-12’,
chip=’ESO_CCD_#65’)

awe> task.execute()

Mandatory keywords :

instrument (string), chip (string), date (string)
or

instrument (string), chip (string), template (string)

Optional keywords :

commit (boolean: False or True. Default False) pars (dictionary of configuration parameters, example:

{‘HotPixelMap.process_params.REJECTION_THRESHOLD’: 6.0})

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Estimate the processing time of this task

make_hotpixelmap()
parser = Recognized options for the astro.recipes.HotPixels OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.HotPixelMap.HotPixelMap

exception astro.recipes.HotPixels.HotPixelsTaskError(message)

Bases: common.log.Error.Error

astro.recipes.IlluminationCorrection module

class astro.recipes.IlluminationCorrection.IlluminationTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, photometric pipeline.

PURPOSE : generating a valid Calfile 548 (Illumination Correction
Frame).

This task is used to derive a Calfile 548 to be used in the image pipeline.

The task assumes that a dataserver is active, and that it has been instantiated with a valid value for the ‘date’, ‘chip’, ‘instrument’ and ‘filter’ parameters, and (optionally) the ‘commit’ boolean switch. It also assumes that an illumination correction map is present in the database for the requested filter and instrument. The task is executed by calling its ‘execute’ method’.

By default the processing results are not committed to the database or stored on the fileserver, unless the ‘commit’ switch was set during instantiation.

Example of use from the awe prompt :

awe> task = IlluminationTask(date=‘2003-02-12’, instrument=’WFI’, … chip=’ccd50’, filter=’#843’, commit=1) awe> task.execute()

Mandatory keywords :

date (string, yyyy-mm-dd), chip (string), instrument (string), filter (string)

Optional keyword :

commit (integer : 0 or 1. Default : 0)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Return the processing time of this task

make_illuminationcorrectionframe()
parser = Recognized options for the astro.recipes.IlluminationCorrection OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "f" or "filter" (default: ) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.IlluminationCorrectionFrame.IlluminationCorrectionFrame

exception astro.recipes.IlluminationCorrection.IlluminationTaskError(message)

Bases: common.log.Error.Error

astro.recipes.IlluminationCorrectionVerify module

class astro.recipes.IlluminationCorrectionVerify.IlluminationVerifyTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, photometric pipeline.

PURPOSE : generating a valid Calfile 548F (Illumination Correction
Fitparameters).

This task is used to derive a Calfile 548F.

The task assumes that a dataserver is active, and that it has been instantiated with a valid value for the ‘raw’ parameter (without extension), and (optionally) the ‘commit’ or ‘inspect’ boolean switches. The task is executed by calling its ‘execute’ method’. The necessary photometric catalogs should already be present in the database.

By default the processing results are not committed to the database, unless the ‘commit’ switch was set during instantiation.

Example of use from the awe prompt :

awe> task = IlluminationVerifyTask(raw=’r336604’, commit=1) awe> task.execute()

Mandatory keyword :

raw (string)

Optional keywords :

pars (dictionary of configuration parameters) commit (integer : 0 or 1. Default : 0) inspect (integer : 0 or 1, Default : 0)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_distinct_chips(photcat_query)
get_process_time()

Return the processing time of this task

make_illuminationcorrection()
parser = Recognized options for the astro.recipes.IlluminationCorrectionVerify OptionsParser are: "i" or "instrument" (default: ) "raw" or "raw" (default: ) "p" or "pars" (default: {}) "I" or "inspect" (default: 0) "C" or "commit" (default: 0)
query_database()
set_default_config()

Set instrument specific default config properties

target

alias of astro.main.IlluminationCorrection.IlluminationCorrection

exception astro.recipes.IlluminationCorrectionVerify.IlluminationVerifyTaskError(message)

Bases: common.log.Error.Error

astro.recipes.KidsCatMatchedTask module

Task to create Matched KidsCat catalogs.

class astro.recipes.KidsCatMatchedTask.KidsCatMatchedTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data analysis, sourcelists.

PURPOSE : making KiDS-CAT matched sourcelists from the 4
CoaddedRegriddedFrames of a single tile.

This task is used to create matched aperture KiDS-CAT catalogs from coadds. The task is executed by calling its ‘execute’ method. The r-band coadds are used as detection coadds.

Single band KidsCat catalogs must exist in order to copy the SExtractor configuration from, in particular SEEING_FWHM.

Required data objects are CoaddedRegriddedFrames and Pullecenella masks (with or without manually masked regions) for a single tile.

Make sourcelists:

awe> task = KidsCatMatchedTask(
coadd_filenames=[‘coadd_u.fits’, ‘coadd_g.fits’, …], mask_filenames=[‘mask_u.fits’, ‘mask_g.fits’, …]

) awe> task.execute()

Mandatory keywords:

coadd_filename: filename of CoaddedRegriddedFrame (string) mask_filename: filename of PixelMap mask (string)
Optional keywords:
commit: False or True. Default False (boolean)
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

coadd_filenames = None
coadds = None
commit = False
execute()
force_creation = False
get_process_result()
get_process_time()

Return the processing time of this task.

kidscatmulti = None
make()
mask_filename = None
masks = None
overlap = None
parser = Recognized options for the astro.recipes.KidsCatMatchedTask OptionsParser are: "cf" or "coadd_filenames" (default: []) "m" or "mask_filenames" (default: []) "fc" or "force_creation" (default: False) "C" or "commit" (default: False)
process_result = None
query_database()

Query for required dependencies: - coadd - weight frame - mask

TODO: - Allow finding of coadd automatically on OBJECT. - Allow finding of mask automatically.

target

alias of astro.main.sourcecollection.ConcatenateAttributes.ConcatenateAttributes

astro.recipes.KidsCatTask module

Task to create KidsCat catalogs.

class astro.recipes.KidsCatTask.KidsCatTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data analysis, sourcelists.

PURPOSE : making KiDS-CAT sourcelist from a CoaddedRegriddedFrame

This task is used to create KiDS-CAT sourcelists from coadds. The task is executed by calling its ‘execute’ method.

Required data objects are a CoaddedRegriddedFrame, its weight frame, and a Pulecenella mask (with or without manually masked regions).

Make sourcelists for a CoaddedRegriddedFrame:

awe> task = KidsCatTask(coadd_filename=’coadd1.fits’, mask_filename=’mask1.fits’) awe> task.execute()

Mandatory keywords:

coadd_filename: filename of CoaddedRegriddedFrame (string) mask_filename: filename of PixelMap mask (string)
Optional keywords:
commit: False or True. Default False (boolean)
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

coadd = None
coadd_filename = ''
commit = False
execute()
force_creation = False
get_process_result()
get_process_time()

Return the processing time of this task.

ingest_kidscat()
kidscat = None
mask = None
mask_filename = ''
parser = Recognized options for the KidsCatTask OptionsParser are: "cf" or "coadd_filename" (default: ) "m" or "mask_filename" (default: ) "fc" or "force_creation" (default: False) "C" or "commit" (default: False)
process_result = None
query_database()

Query for required dependencies: - coadd - weight frame - mask

TODO: - Allow finding of coadd automatically on OBJECT. - Allow finding of mask automatically.

run_kidscat()
target

alias of astro.main.sourcecollection.ConcatenateAttributes.ConcatenateAttributes

weight = None
exception astro.recipes.KidsCatTask.KidsCatTaskError(message)

Bases: common.log.Error.Error

astro.recipes.MDia module

class astro.recipes.MDia.MDiaTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

execute()
make_all()
make_rerun(scale=1)
parser = Recognized options for the astro.recipes.MDia OptionsParser are: "ref" or "ref_filename" (default: ) "reg" or "reg_filenames" (default: []) "lc" or "lc_filename" (default: ) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
target

alias of astro.main.LightCurve.LightCurve

exception astro.recipes.MDia.MDiaTaskError

Bases: Exception

astro.recipes.ManMask module

class astro.recipes.ManMask.ManMaskTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : creating a ManualBitMask for T-bar reflections and
manually discovered satellite tracks and their regions

This task is used to create ManualBitMask objects based on manually created regions of residual satellite tracks as well as regions affected by T-bar reflections/vignetting in subsets of OmegaCAM detector rows.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt:

awe> task = ManMaskTask(filename=’coadd.fits’, coordinates=’ends.txt’, tbars=1) awe> task.execute()

Alternative example:

awe> ends = list(numpy.loadtxt(‘ends.txt’)) awe> task = ManMaskTask(filename=’coadd.fits’, endpoints=ends, tbars=1) awe> task.execute()

See the section below for an overview of the possible constructor keywords.

One set of query keywords can be provided in the constructor of the task:

filename: string
filename of the CoaddedRegriddedFrame containing satellite tracks
coordinates: string
filename of ascii file containing satellite track end points in coordinates in the thumbnail in pixel units
endpoints: list
Array containing the endpoints of manually marked satellite tracks (shape = n, 2)
tbars: integer

integer indicating which rows of OmegaCAM detectors require T-bar masking (0 = no T-bar masking (default); 1 = rows 1 and 3;

2 = rows 2 and 4; 3 = all rows)
tbarwidth: integer
width of T-bar area to mask in (RegriddedFrame) pixels (default = 250)
pars: dictionary
dictionary containing configuration parameters
commit: boolean
switch to store and commit results
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()

Return the list of made bitmasks

get_process_time()

Return the processing time of this task.

make_manmasks()

Set dependencies and configurations and make the bitmasks

make_satmasks()

Set dependencies and configurations and make the bitmasks

make_tbarmasks()

Set dependencies and configurations and make the bitmasks

parser = Recognized options for the astro.recipes.ManMask OptionsParser are: "f" or "filename" (default: ) "c" or "coordinates" (default: ) "e" or "endpoints" (default: []) "t" or "tbars" (default: 0) "w" or "tbarwidth" (default: 250) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()

Find the input CoaddedRegriddedFrame.

target

alias of astro.main.ManualBitMask.ManualBitMask

exception astro.recipes.ManMask.ManMaskTaskError(message)

Bases: common.log.Error.Error

astro.recipes.ManualMask module

class astro.recipes.ManualMask.ManualMaskTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : creating an automatic bitmask of diffraction spikes and
reflections based on the geometry of an optical system

This task is used to create ManualBitMask objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Examples of use from the awe prompt :

awe> task = ManualMaskTask(regnames=[‘manmask1.reg’],
sizes=[(2048,4096)])

awe> task.execute()

awe> task = ManualMaskTask(regnames=[‘manmask1.reg’],
filenames=[baseframe1.filename])

awe> task.execute()

See the section below for an overview of the possible constructor keywords.

One set of query keywords can be provided in the constructor of the task:

a list of region file names (with image sizes or BaseFrame filenames)

The set of possible keywords is a list of regnames and image sizes:

regnames (filenames of manually created SkyCat region files, list
of strings)

sizes (list of integer tuples denoting X- and Y-size of output)

or:

filenames (filenames of associated BaseFrames, list of strings)

In both cases, the parameters can be updated with a dictionary from a Pars object:

pars (Pars.get() -> dict)

Finally, it is possible to specify whether the process result should be committed or not by setting the following parameter in the constructor:

commit (integer 0 or 1. Default 0)

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()

Return the list of made bitmasks

get_process_time()

Return the processing time of this task.

make_bitmasks()

Set dependencies and configurations and make the bitmasks

parser = Recognized options for the astro.recipes.ManualMask OptionsParser are: "r" or "regnames" (default: []) "s" or "sizes" (default: []) "f" or "filenames" (default: []) "flg" or "flags" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()

Find the input CoaddedRegriddedFrames.

target

alias of astro.main.ManualBitMask.ManualBitMask

exception astro.recipes.ManualMask.ManualMaskTaskError(message)

Bases: common.log.Error.Error

astro.recipes.MasterFlat module

class astro.recipes.MasterFlat.MasterFlatTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a masterflat frame for a specific date, chip
and filter

This task is used to create a master flat.

The task assumes that a dataserver is active, and that it has been instantiated with valid values for the ‘instrument’, ‘date’, ‘chip’ and ‘filter’ parameters. The optional ‘combine’ parameter can be given to:

  1. combine dome and twilight (default)
  2. only dome flats
  3. only twilight flats

If only a dome or twilight flat is used the master flat will be a direct copy of it. Additionally the parameter ‘nightsky’ can be set to 1 to query for and use the night sky flat. The task is executed by calling its ‘execute’ method.

Example of use from the awe prompt :

awe> task = MasterFlatTask(instrument=’OMEGACAM’, chip=’ESO_CCD_#65’,
filter=’OCAM_r_SDSS’, date=‘2015-02-12’, combine=1)

awe> task.execute()

Mandatory keywords :

instrument(string), date (string), chip (string), filter (string)

Optional keywords :

combine (integer: 1-3. Default 1) nightsky (integer: 0 or 1. Default 0) commit (boolean: False or True. Default False) pars (dictionary of configuration parameters)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

return the processing time of this task

make_masterflat()
parser = Recognized options for the astro.recipes.MasterFlat OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "dtpl" or "dtemplate" (default: ) "ttpl" or "ttemplate" (default: ) "ct" or "combine" (default: 1) "ns" or "nightsky" (default: 0) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_default_config()

Set instrument specific default config properties

target

alias of astro.main.MasterFlatFrame.MasterFlatFrame

exception astro.recipes.MasterFlat.MasterFlatTaskError(message)

Bases: common.log.Error.Error

astro.recipes.NightSkyFlat module

class astro.recipes.NightSkyFlat.NightSkyFlatTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a night-sky flat for a specific date, chip and filter

This task is used to create a night-sky flat.

The task assumes that a dataserver is active, and that it has been instantiated with valid values for the ‘instrument’, ‘date’, ‘chip’, ‘filter’ and (optionally) ‘overscan’ parameters. The task is executed by calling its ‘execute’ method.

Example of use from the awe prompt :

awe> task = NightSkyFlatTask(instrument=’OMEGACAM’, chip=’ESO_CCD_#65’,
filter=’OCAM_g_SDSS’, date=‘2015-02-15’)

awe> task.execute()

Mandatory keywords :

instrument (string), date (string), chip (string), filter (string)

or

raw_filenames (list of strings)

Optional keyword :

overscan (integer: 0-13. Default 6) commit (boolean: False or True. Default False) pars (dictionary of configuration parameters)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Return the processing time of this task

make_nightskyflat()
parser = Recognized options for the astro.recipes.NightSkyFlat OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "oc" or "overscan" (default: 6) "raw" or "raw_filenames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.NightSkyFlatFrame.NightSkyFlatFrame

exception astro.recipes.NightSkyFlat.NightSkyFlatTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Null module

class astro.recipes.Null.NullClass

Bases: object

A fake persistent object.

PROCESS_TIME = 1
class astro.recipes.Null.NullTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : processing, testing

PURPOSE : run a job for testing purposes

The task must be instantiated with some set of keyword arguments, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = NullTask(f=[‘’], s=10) awe> task.execute()

awe> task = NullTask(i=’OMEGACAM’, n=32, d=‘1990-01-01’) awe> task.execute()

Arguments must be one of the following two sets:

filenames: a list of any strings (e.g., [‘’, ‘’, ‘’])

or:

number: the number of processes [int]

instrument: the name of a valid instrument [str]

and one of:

date: any valid date [str]

template: any valid date [str] or [datetime]

Optionally, a sleep period can be specified:

sleep: number of seconds for each sub-task to run [int]
clean_up_before_pickle()

Prevent pickle errors.

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()

Return the created classes. Usually this is a list.

get_process_time()

Return the processing time of this task

make_null()

Make the target class. This includes instantiating it.

This assumes the dependencies are already set, but includes retrieving them.

Most Tasks have a ‘commit’ option. The created class is committed and stored if this flag is set.

null = None
parser = Recognized options for the astro.recipes.Null OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "n" or "number" (default: 0) "f" or "filenames" (default: []) "s" or "sleep" (default: 0) "C" or "commit" (default: False)
query_database()

Search the database for objects to be used as dependencies.

Search the database for objects to be used as dependencies of the target class.

The dependencies might already be set, e.g. by set_dependencies. In that case, return early from this function.

set_dependencies(dependencies)

Sets the dependencies explicitly.

This function is used for compound tasks to set the process_result of one task as the dependencies of another.

target

alias of NullClass

exception astro.recipes.Null.NullTaskError(message)

Bases: common.log.Error.Error

astro.recipes.PSFModelTask module

Task to create PSFModels

class astro.recipes.PSFModelTask.PSFModelTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

Make a PSFModel

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Return the processing time of this task

make_psfmodel()
parser = Recognized options for the astro.recipes.PSFModelTask OptionsParser are: "cf" or "coadd_filename" (default: ) "rf" or "red_filename" (default: ) "rfd" or "rfd_filenames" (default: []) "reg" or "reg_filename" (default: ) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()
target

alias of astro.main.PSFModel.PSFModel

exception astro.recipes.PSFModelTask.PSFModelTaskError(message)

Bases: common.log.Error.Error

astro.recipes.PSF_Anisotropy module


CATEGORY : Health check

PURPOSE : Detect PSF Anisotropy

FULFILLS : Requirement 554 (Category III)


This recipe makes 5 plots of Sextractor parameters: 1) Bars of length ELLIPTICITY (times a scale factor) and angle w.r.t.

x-axis Theta.
  1. MAG_ISO vs. FLUX_RADIUS (half-light radius)
  2. FWHM_IMAGE vs. THETA (position angle)
  3. FWHM_IMAGE vs. Xpos-CRPIX1
  4. FWHM_IMAGE vs. Ypos-CRPIX2

class astro.recipes.PSF_Anisotropy.PSF_Anisotropy(frames=[], sourcelists=[], interactive=False, force_new_catalogs=False)

Bases: object

get_data()
make()

Run everything.

plot()
class astro.recipes.PSF_Anisotropy.PSF_AnisotropyParameters

Bases: object

ELLIPTICITY = True
ELLIPTICITY_SCALEFACTOR = 5000.0
FILENAME = 'cal554.eps'
FLAG_MAX = 0
FLUX_RADIUS_FACTOR = 1.3
FWHM_SCALEFACTOR = 5.0
LEGEND_ELLIPTICITY = 0.1
LEGEND_FWHM = 10.0
MAX_FLUX_RADIUS = 50.0
MAX_SEEING = 0.0
MAX_VAL = 64000
MIN_FLUX_RADIUS = 1.0
SEX_PARAMS = []
USE_FWHM_WORLD = True
WBOX = 200.0
class astro.recipes.PSF_Anisotropy.PSF_AnisotropySourceList

Bases: object

This is a container for data that can be used by PSFAnisotropyPlot, where the data originates from a SourceList (rather than a Catalog on disk).

Multiple instances of this class are expected to update a single dictionary using the update_data_dictionary() method.

Complements PSF_AnisotropyCatalog.

NEEDED_COLUMNS = ['Xpos', 'Ypos', 'A', 'B', 'POSANG', 'FWHM_IMAGE', 'FWHM_WORLD', 'FLUX_RADIUS', 'ELLIPTICITY', 'MaxVal', 'MAG_ISO', 'Flag']
classmethod check(sourcelist)

Checks whether the specified SourceList can provide the necessary data.

Returns True or False

make()
set_data()

Populates self.data (a dictionary).

update_data_dictionary(dictionary={})

Updates the specified dictionary with data extracted from the sourcelist.

astro.recipes.PhotCalExtractResulttable module

class astro.recipes.PhotCalExtractResulttable.PhotcatTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data pre-processing, photometric pipeline.

PURPOSE : derive photometric result tables used for further
processing in the photometric pipeline.

This task is used to derive photometric result tables which are used for further processing in the photometric pipeline.

The task assumes that a dataserver is active, and that it has been instantiated with valid values for the ‘instrument’, the ‘raw_filenames’ OR ‘red_filenames’ parameters, and (optionally) the ‘commit’ and ‘transform’ boolean switches. The task also assumes that a standard star catalog is present in the database. The task is executed by calling its ‘execute’ method.

When needed, a transformation table containing color terms can also be used in the processing. The ‘transform’ boolean switch must be set for this. Note, that a proper transformation table should be present in the database for this to work.

By default, the processing result is not committed to the database, unless the ‘commit’ switch was set during instantiation of the task. The catalog produced can be inspected on screen by setting the ‘inspect’ switch.

Example of use from the awe prompt :

awe> task = PhotcatTask(instrument=’WFI’, raw_filenames=[‘wfi60747_1.fits’]) awe> task.execute()

Mandatory keywords :

  1. instrument (string),

  2. raw_filenames (filenames of raw science frames, list of strings),

    OR

    red_filenames (filenames of reduced science frames, list of strings) In case both are specified, the raw_filenames take precedence.

    OR

    template (string), chip (string)

Optional keywords :

transform (integer : 0 or 1, default : 0) pars (dictionary of configuration parameters) commit (integer : 0 or 1, default : 0) inspect (integer : 0 or 1, default : 0)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

return the processing time of this task

make_resulttables()
parser = Recognized options for the astro.recipes.PhotCalExtractResulttable OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "tpl" or "template" (default: None) "t" or "transform" (default: 0) "p" or "pars" (default: {}) "I" or "inspect" (default: 0) "C" or "commit" (default: 0)
query_database()
set_config(photcat, astrom_params)

Set instrument specific default config properties

set_dependencies(astromparams)
target

alias of astro.main.PhotSrcCatalog.PhotSrcCatalog

exception astro.recipes.PhotCalExtractResulttable.PhotcatTaskError(message)

Bases: common.log.Error.Error

astro.recipes.PhotCalExtractZeropoint module

class astro.recipes.PhotCalExtractZeropoint.PhotomTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, photometric pipeline.

PURPOSE : generating valid Calfiles 563 which are used by the
image pipeline for the photometric calibration.

This task is used to derive Calfiles 563 which are used for the photometric calibration of science data.

The task assumes that a dataserver is active, and that it has been instantiated with a valid value for the ‘instrument’ parameter and either the ‘raw_filenames’ or the ‘red_filenames’ parameter. Optionally, the ‘inspect’ and ‘commit’ boolean switches may be set. The task assumes that a standard extinction curve is present in the database. The task is executed by calling its ‘execute’ method’.

The atmospheric extinction is derived from a standard extinction curve combined with a shift of the curve contained within an extinction report. If no extinction report is present in the database, a default extinction coefficient is used.

By default, the processing results are not committed to the database, unless the ‘commit’ switch was set during instantiation of the task. The result produced can be inspected on screen by setting the ‘inspect’ switch.

Example of use from the awe prompt :

awe> task = PhotomTask(instrument=’WFC’, raw_filenames=[‘r336603_4.fits’]) awe> task.execute()

Mandatory keywords :

  1. instrument (string)

  2. raw_filenames (filenames of raw standard star frames, list of strings)

    OR

    red_filenames (filenames of reduced standard star frames, list of strings) In case both are specified, the raw_filenames take precedence.

    OR

    template (string), chip (string)

Optional keywords :

pars (dictionary of configuration parameters) commit (integer : 0 or 1. Default : 0) inspect (integer : 0 or 1. Default : 0)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

return the processing time of this task

make_zeropoint()
parser = Recognized options for the astro.recipes.PhotCalExtractZeropoint OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "tpl" or "template" (default: None) "p" or "pars" (default: {}) "C" or "commit" (default: 0) "I" or "inspect" (default: 0)
query_database()
set_config(photom, photcat)

Set instrument specific default config properties

set_dependencies(photcats)
target

alias of astro.main.PhotometricParameters.PhotometricParameters

exception astro.recipes.PhotCalExtractZeropoint.PhotomTaskError(message)

Bases: common.log.Error.Error

astro.recipes.PhotCalMonitoring module

class astro.recipes.PhotCalMonitoring.MonitoringTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, photometric pipeline.

PURPOSE : generating a valid Calfile 562 (extinction report).

This task is used to derive a Calfile 562 which is used for the evaluation of the photometric quality of the night.

The task assumes that a dataserver is active, and that it has been instantiated with a valid value for the ‘instrument’, ‘raw_filenames’ and ‘date’ parameters, and (optionally) the ‘commit’ and/or ‘inspect’ boolean switches. It also assumes that a standard extinction curve is present in the database. The task is executed by calling its ‘execute’ method’.

The input list of raw filenames and the date are used to retrieve the correct photometric result tables and the zeropoints for the night from the database.

By default the processing results are not committed to the database, unless the ‘commit’ switch was set during instantiation of the task.

Example of use from the awe prompt :

awe> task = MonitoringTask(instrument=’WFC’, date=‘2003-02-12’,
commit=1, raw_filenames=[‘r336601_4.fits’], inspect=1)

awe> task.execute()

Mandatory keywords :

raw_filenames (list of strings) instrument (string) date (string, yyyy-mm-dd)

Optional keywords :

pars (dictionary of configuration parameters) commit (integer : 0 or 1. Default : 0) inspect (integer : 0 or 1. Default : 0)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_distinct_chips(photcats)
get_distinct_filters(photcats)
get_process_time()

Return the processing time of this task.

make_extinctionreport()
midnight(date_obs)
parser = Recognized options for the astro.recipes.PhotCalMonitoring OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "p" or "pars" (default: {}) "I" or "inspect" (default: 0) "C" or "commit" (default: 0)
query_database()
query_for_extinction_coefficients(photoms)
query_for_extinction_curve()
query_for_photcats()
query_for_zeropoints(photcats)
set_config()

Set instrument specific default config properties

target

alias of astro.main.PhotometricExtinctionReport.PhotometricExtinctionReport

exception astro.recipes.PhotCalMonitoring.MonitoringTaskError(message)

Bases: common.log.Error.Error

astro.recipes.PhotRedCatalog module

task to make Photometric redshifts

class astro.recipes.PhotRedCatalog.PhotRedCatalogTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data analysis, sourcelists.

PURPOSE : create photometric redshifts from Sourcelists

This task is used to create photometric redshifts from a master sourcelist containing the detections and several additional sourcelists with photometry in additional passbands

Create photometric redshifts

awe> task = PhotRedCatalogTask( master_sl=1122110, sls=[1122110,1122111,1122112,1122113], config=’WFI_all_filters’, min_num_sources=5, mag=’MAG_APER1’) awe> task.execute()

Required keywords :

master_sl : Sourcelist with object detections and photometry sls : list of additional sourcelists (including the master sourcelist) config : PhotRedConfig object to be used for the run

Optional keywords :

All additional keywords from the PhotRedCatalog class are supported
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()
make_photredcatalog()
parser = Recognized options for the astro.recipes.PhotRedCatalog OptionsParser are: "msl" or "master_sl" (default: ) "sls" or "sourcelists" (default: []) "config" or "config" (default: ) "min" or "min_num_sources" (default: None) "ext" or "extinc" (default: None) "merr" or "model_error" (default: None) "name" or "name" (default: None) "datpz1_name" or "datpz1_name" (default: None) "datstar_name" or "datstar_name" (default: None) "mag" or "mag" (default: None) "flux" or "flux" (default: None) "fluxerr" or "fluxerr" (default: None)
query_config()
query_master_sourcelist()
query_sourcelists()
target

alias of astro.main.PhotRedCatalog.PhotRedCatalog

exception astro.recipes.PhotRedCatalog.PhotRedCatalogTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Pullecenella module

class astro.recipes.Pullecenella.PullecenellaTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : creating an automatic bitmask of diffraction spikes and
reflections based on the geometry of an optical system

This task is used to create AutomaticBitMask objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Examples of use from the awe prompt :

awe> task = PullecenellaTask(instrument=’OMEGACAM’,
template=‘2013-01-01 00:00:00’)

awe> task.execute()

awe> task = PullecenellaTask(filenames=[‘Sci-AWUSER-Coadd-Sci.fits’]) awe> task.execute()

See the section below for an overview of the possible constructor keywords.

Two sets of query keywords can be provided in the constructor of the task:

  1. a list of filenames (with optional manual mask filenames).
  2. a set of global query parameters (like instrument and template).

The first of these sets has precedence over the second.

The first set of possible keywords is a list of filenames and possibly masknames:

filenames (filenames of CoaddedRegriddedFrames, list of strings) masknames (filenames of ManualBitMask objects, list of strings)

The query sent to the database is such that all frames with matching filenames are returned. For example:

query = (CoaddedRegriddedFrame.filename == filenames[0]) for filename in filenames[1:]:

query |= (CoaddedRegriddedFrame.filename == filename)

Note that if the masknames are included, they should be matched to the filenames.

The second set of possible keywords is:

instrument (string) filter (string) template (string) object (string or using wildcards like KIDS*_?)

In both cases, the parameters can be updated with a dictionary from a Pars object:

pars (Pars.get() -> dict)

Finally, it is possible to specify whether the process result should be committed or not by setting the following parameter in the constructor:

commit (integer 0 or 1. Default 0)

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()

Return the list of made bitmasks

get_process_time()

Return the processing time of this task.

make_bitmasks()

Set dependencies and configurations and make the bitmasks

parser = Recognized options for the astro.recipes.Pullecenella OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: ) "fn" or "filenames" (default: []) "m" or "masknames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()

Find the input CoaddedRegriddedFrames.

set_config(bitmask, coadd)

Set instrument specific default config properties

target

alias of astro.main.AutomaticBitMask.AutomaticBitMask

exception astro.recipes.Pullecenella.PullecenellaTaskError(message)

Bases: common.log.Error.Error

astro.recipes.QuickCheck module

class astro.recipes.QuickCheck.QuickCheckTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : health check

PURPOSE : making a quick check frame(s) for a specific date and
chip.

This task is used to create QuickCheckFrames.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = QuickCheckTask(instrument=’WFI’, date=‘2003-02-12’,
chip=’ccd50’)

awe> task.execute()

Mandatory keywords :

instrument (string), date (string), chip (string)
or

instrument (string), template (string), chip (string)

Optional keywords :

overscan (integer: 0-13. Default 6) commit (integer: 0 or 1. Default 0)

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Return the processing time of this task

make_quickcheck()
parser = Recognized options for the astro.recipes.QuickCheck OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "tpl" or "template" (default: None) "oc" or "overscan" (default: 6) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
set_config(quickcheck)

Set instrument specific default config properties

target

alias of astro.main.QuickCheckFrame.QuickCheckFrame

exception astro.recipes.QuickCheck.QuickCheckTaskError(message)

Bases: common.log.Error.Error

astro.recipes.QuickCheckSimple module


CATEGORY : data processing, calibration pipeline.

PURPOSE : generating a quick check on the detector responsivity
(Calfile 547)

FULFILLS : requirement 547 (Category I)


This recipe is used to generate a quick check on the detector responsivity for a set of RawFitsData (Calfile 547).

class astro.recipes.QuickCheckSimple.QuickCheckMaker(end_date=None, timespan=14, outfile='')

Bases: object

make_quickcheck()

This routine derives Calfile 547.

plot()
query_database()

astro.recipes.RawFitsDataInspectTask module

class astro.recipes.RawFitsDataInspectTask.RawFitsDataInspectTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

return the processing time of this task

make_inspect_figures()
parser = Recognized options for the astro.recipes.RawFitsDataInspectTask OptionsParser are: "f" or "filenames" (default: []) "C" or "commit" (default: 0)
query_database()

Query the database for RawFitsData objects with filenames as given.

target

alias of astro.main.InspectFigure.InspectFigure

exception astro.recipes.RawFitsDataInspectTask.RawFitsDataInspectTaskError(message)

Bases: common.log.Error.Error

astro.recipes.RawFrameTask module

class astro.recipes.RawFrameTask.RawFrameTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, ingestion.

PURPOSE : making RawFrames from RawFitsData objects

This task is used to create RawFrame objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = RawFrameIngestTask(filenames=[‘WFI.2005-12-31T00:00:00.123.fits’], image_type=’flat’) awe> task.execute()

Mandatory keywords :

filenames : list of RawFitsData filenames image_type : type of the RawFrames to be made from the given RawFitsData objects (“dome”, “twilight”, “bias”, etc)

Optional keywords :

commit : 0 or 1, commit results force : 0 or 1, force ingestion of data that is on the dataserver, but

not in the database where it is visible to this user (i.e. previously ingested)
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Return the processing time of this task

make_rawframes()

Make the reduced frames from the RawFitsData

parser = Recognized options for the astro.recipes.RawFrameTask OptionsParser are: "f" or "filenames" (default: []) "t" or "image_type" (default: ) "C" or "commit" (default: 0) "F" or "force" (default: 0)
query_database()

Query the database for RawFitsData objects with filenames as given.

target

alias of astro.main.RawFrame.RawFrame

exception astro.recipes.RawFrameTask.RawFrameTaskError(message)

Bases: common.log.Error.Error

astro.recipes.ReadNoise module

class astro.recipes.ReadNoise.ReadNoiseTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, quality control, calibration pipeline.

PURPOSE : determining the read noise for a specific chip.

This task is used to create ReadNoise objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = ReadNoiseTask(instrument=’OMEGACAM’, date=‘2015-02-12’,
chip=’ESO_CCD_#65’)

awe> task.execute()

Mandatory keywords :

instrument(string), date (string), chip (string)
or
instrument(string), template (string), chip (string)
or

raw_filenames (list of strings)

Optional keywords :

commit (boolean: False or True. Default False) pars (dictionary of configuration parameters, example:

{‘ReadNoise.process_params.REJECTION_THRESHOLD’: 4.0})

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Estimate the processing time of this task

make_readnoise()
parser = Recognized options for the astro.recipes.ReadNoise OptionsParser are: "i" or "instrument" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "raw" or "raw_filenames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: False) "s" or "slice" (default: (0, 2))
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.ReadNoise.ReadNoise

exception astro.recipes.ReadNoise.ReadNoiseTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Reduce module

class astro.recipes.Reduce.ReduceTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : making reduced science frames (flat-fielded, de-biased)

This task is used to create ReducedScienceFrame’s.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example 1 of use from the awe prompt:

awe> task = ReduceTask(instrument=’OMEGACAM’, chip=’ESO_CCD_#65’,
filter=’OCAM_i_SDSS’, date=‘2015-03-01’, object=’KIDS_186.5_-2.5’)

awe> task.execute()

Example 2 of use from the awe prompt:

awe> task = ReduceTask(instrument=’WFI’,
raw_filenames=[‘wfi60757_1.fits’])

awe> task.execute()

See the section below for an overview of the possible constructor keywords.

Two sets of query keywords can be provided in the constructor of the task:

  1. one list of filenames.
  2. a set of global query parameters (like date, filter and chip).

The first of these sets has precedence over the second.

The first set of possible keywords is one list of filenames:

raw_filenames (filenames of the raw science frames, list of strings)

The second set of possible keywords is:

date (string) instrument (string) chip (string) filter (string) template (string) object (string or using wildcards like M33_?_?, M33_V*)

Finally, the following parameters can also be set in the constructor:

force_defringe (integer 0 or 1. Default 0. Use this only for example
for WFI R filter, which is not automatically defringed)

overscan (integer 0-13. Default 6) pars (dictionary of configuration parameters, example:

{‘ReducedScienceFrame.process_params.FRINGE_THRESHOLD_HIGH’: 6.0})

commit (boolean: False or True. Default False)

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Return the processing time of this task

make_frames()
parser = Recognized options for the astro.recipes.Reduce OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: None) "oc" or "overscan" (default: 6) "rfd" or "rfd_filenames" (default: []) "raw" or "raw_filenames" (default: []) "e" or "exptime" (default: -1) "fd" or "force_defringe" (default: 0) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config(reduced, raw)

Set instrument specific default config properties

target

alias of astro.main.ReducedScienceFrame.ReducedScienceFrame

exception astro.recipes.Reduce.ReduceTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Reference module

class astro.recipes.Reference.ReferenceTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

execute()
get_process_time()

return the processing time of this task

make_all()
parser = Recognized options for the astro.recipes.Reference OptionsParser are: "aslid" or "aslid" (default: 0) "slid" or "slid" (default: 0) "o" or "object" (default: ) "reg" or "reg_filenames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
target

alias of astro.main.ReferenceFrame.ReferenceFrame

exception astro.recipes.Reference.ReferenceTaskError

Bases: Exception

astro.recipes.Regrid module

class astro.recipes.Regrid.RegridTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : creating regridded frames

This task is used to create RegriddedFrame objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Examples of use from the awe prompt :

awe> task = RegridTask(instrument=’OMEGACAM’, chip=’ESO_CCD_#65’,
filter=’OCAM_i_SDSS’, date=‘2015-03-01’, object=’KIDS_186.5_-2.5’)

awe> task.execute()

awe> task = RegridTask(instrument=’OMEGACAM’,
red_filenames=[‘Sci-AWUSER-Red—Sci.fits’])

awe> task.execute()

awe> task = RegridTask(instrument=’WFI’, raw_filenames=[ ‘WFI.2000-04-28T01:01:01_7.fits’,
‘WFI.2000-04-28T05:05:05_3.fits’])

awe> task.execute()

See the section below for an overview of the possible constructor keywords.

Two sets of query keywords can be provided in the constructor of the task:

  1. ONE of two possible lists of filenames.
  2. a set of global query parameters (like date, filter and chip).

The first of these sets has precedence over the second.

The first set of possible keywords is ONE of these two lists of filenames:

raw_filenames (filenames of the raw science frames, list of strings) red_filenames (filenames of the reduced science frames, list of

strings)

Depending on which of these is given, a different query is send to the database. Using raw_filenames causes RegridTask to use the most recent ReducedScienceFrame which exists for the RawScienceFrame with given raw_filename. When no ReducedScienceFrame exists no RegriddedFrame will be made. Using red_filenames causes RegridTask to use the ReducedScienceFrame which has as filename the given red_filename.

The second set of possible keywords is:

date (string) instrument (string) chip (string) filter (string) template (string) object (string or using wildcards like M33_?_?, M33_V*) grid_ra (float) grid_dec (float) grid_pixelscale (float) force_single_gridtarget (boolean) local_only (boolean) global_only (boolean) pars (Pars.get() -> dict)

Finally, it is possible to specify whether the process result should be committed or not by setting the following parameter in the constructor:

commit (boolean False or True. Default False)

For all the keywords documented here, the type of the values assigned to it is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_grid_target(astrom_params)
get_process_result()
get_process_time()

Return the processing time of this task

make_regridded_frames()
no_filenames_in_query()
parser = Recognized options for the astro.recipes.Regrid OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "o" or "object" (default: ) "tpl" or "template" (default: ) "rfd" or "rfd_filenames" (default: []) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "gra" or "grid_ra" (default: None) "gdec" or "grid_dec" (default: None) "gps" or "grid_pixelscale" (default: 0.0) "fsg" or "force_single_gridtarget" (default: False) "l" or "local_only" (default: False) "g" or "global_only" (default: False) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config(regridded_frame, astrom_params)

Set instrument specific default config properties

set_dependencies(astromparams)
set_grid_target()
target

alias of astro.main.RegriddedFrame.RegriddedFrame

exception astro.recipes.Regrid.RegridTaskError(message)

Bases: common.log.Error.Error

astro.recipes.SatMask module

class astro.recipes.SatMask.SatMaskTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing, image pipeline.

PURPOSE : creating a ManualBitMask based on manually discovered
satellite tracks and their regions

This task is used to create ManualBitMask objects based on manually created regions of residual satellite tracks.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt:

awe> task = SatMaskTask(filename=’coadd.fits’, coordinates=’ends.txt’) awe> task.execute()

Alternative example:

awe> ends = list(numpy.loadtxt(‘ends.txt’)) awe> task = SatMaskTask(filename=’coadd.fits’, endpoints=ends) awe> task.execute()

See the section below for an overview of the possible constructor keywords.

One set of query keywords can be provided in the constructor of the task:

filename: string
filename of the CoaddedRegriddedFrame containing satellite tracks
coordinates: string
filename of ascii file containing satellite track end points in coordinates in the thumbnail in pixel units
endpoints: list
Array containing the endpoints of manually marked satellite tracks (shape = n, 2)
pars: dictionary
dictionary containing configuration parameters
commit: boolean
switch to store and commit results
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()

Return the list of made bitmasks

get_process_time()

Return the processing time of this task.

make_satmasks()

Set dependencies and configurations and make the bitmasks

parser = Recognized options for the astro.recipes.SatMask OptionsParser are: "f" or "filename" (default: ) "c" or "coordinates" (default: ) "e" or "endpoints" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()

Find the input CoaddedRegriddedFrame.

target

alias of astro.main.ManualBitMask.ManualBitMask

exception astro.recipes.SatMask.SatMaskTaskError(message)

Bases: common.log.Error.Error

astro.recipes.Scamp module

Task to run Scamp

class astro.recipes.Scamp.ScampTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data analysis, sourcelists.

PURPOSE : run Scamp

This task is used to create sourcelists using Scamp. The task is executed by calling its ‘execute’ method’.

Make sourcelists for two frames (example 1):

awe> task = ScampTask(slids=[123214, 1232155], commit=True) awe> task.execute()

Mandatory keywords:

slids (list of integers)

Optional keywords:

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
export_sourcelists()
get_process_time()

Returns the estimated processing time (in seconds) of this task

make_associatelist_from_scamp_output()
parser = Recognized options for the astro.recipes.Scamp OptionsParser are: "n" or "name" (default: ) "s" or "slids" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
run_scamp()
set_scampconf()
target

alias of astro.main.AssociateList.AssociateList

exception astro.recipes.Scamp.ScampTaskError(message)

Bases: common.log.Error.Error

Errors for ScampTask.

astro.recipes.Scamp.query_sourcelist(slid)

astro.recipes.ShutterCheck module

class astro.recipes.ShutterCheck.ShutterCheckTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, quality control, calibration pipeline.

PURPOSE : determining the shutter homogeneity

This task is used to create ShutterCheck objects.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = ShutterCheckTask(instrument=’WFI’, date=‘2003-02-12’,
chip=’ccd50’)

awe> task.execute()

Mandatory keywords :

instrument(string), date (string), chip (string)
or
instrument(string), template (string), chip (string)
or

raw_filenames (list of strings)

Optional keywords :

commit (integer: 0 or 1. Default 0) pars (dictionary of configuration parameters, example:

{‘ShutterCheck.process_params.REJECTION_THRESHOLD’: 4.0})

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Return the processing time of this task

get_shutter_direction(rawframe)
make_shuttercheck()
parser = Recognized options for the astro.recipes.ShutterCheck OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "tpl" or "template" (default: None) "raw" or "raw_filenames" (default: []) "p" or "pars" (default: {}) "oc" or "overscan" (default: 6) "C" or "commit" (default: 0)
query_database()
set_config(shuttercheck)

Set instrument specific default config properties

target

alias of astro.main.ShutterCheckFrame.ShutterCheckFrame

exception astro.recipes.ShutterCheck.ShutterCheckTaskError(message)

Bases: common.log.Error.Error

astro.recipes.SourceList module

Task to make SourceLists.

class astro.recipes.SourceList.SourceListTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data analysis, sourcelists.

PURPOSE : making sourcelists from frames

This task is used to create sourcelists from frames. The task is executed by calling its ‘execute’ method’.

Make sourcelists for two frames (example 1):

awe> task = SourceListTask(filenames=[‘coadd1.fits’, ‘coadd2.fits’]) awe> task.execute()

Make a sourcelist using Sextractor double image mode (example 2):

awe> task = SourceListTask(filenames=[‘regridded_B.fits’],
detection_filenames=[‘regridded_V.fits’])

awe> task.execute()

Mandatory keywords:

filenames (list of strings)

Optional keywords:

commit (boolean: False or True. Default False) detection_filenames (List of filenames of detection images used in

Sextractor double image mode. If defined should be equal in length to filenames.)
overlap (When making multi-band SourceLists, the minimum overlap
region for all bands must be calculated. This dictionary contains filename, (xmin,xmax,ymin,ymax) items.)
name (String, name given to all (!) SourceLists derived from
“filenames”)

sexparam (List of strings, the output parameters of Sextractor) pars (Configuration dictionary),

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

detection_filenames = None
detection_frames = None
detection_slids = None
detection_sourcelists = None
execute()
filenames = None
frames = None
get_process_time()

Return the processing time of this task

make_sourcelists()
name = None
overlap = None
parser = Recognized options for the astro.recipes.SourceList OptionsParser are: "n" or "name" (default: ) "f" or "filenames" (default: []) "df" or "detection_filenames" (default: []) "ds" or "detection_slids" (default: []) "ov" or "overlap" (default: {}) "sp" or "sexparam" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_default_config(sourcelist, frame)

Set instrument specific default config properties

sexparam = None
target

alias of astro.main.SourceList.SourceList

exception astro.recipes.SourceList.SourceListTaskError(message)

Bases: common.log.Error.Error

Errors for SourceListTask.

astro.recipes.SourceList.query_science(filename)
astro.recipes.SourceList.query_sourcelist(slid)

astro.recipes.SubimageSourceList module

task to make SourceLists with sub-images

Make sourcelists for two frames (example 1):

awe> task = SourceListTaskSub(filenames=[‘coadd1.fits’, ‘coadd2.fits’],
sub_m=(1,1,128,128))

awe> task.execute()

Make a sourcelist using Sextractor double image mode (example 2):

awe> task = SourceListTask(filenames=[‘regridded_B.fits’],
detection_filenames=[‘regridded_V.fits’], sub_m=(1,1,128,128), sub_d=(1,1,128,128))

awe> task.execute()

class astro.recipes.SubimageSourceList.SourceListTask(*args, **kwargs)

Bases: object

execute()
get_process_time()

return the processing time of this task

get_processed_SLIDS()
get_processed_data()
classmethod get_subimage(obj, coord)
make_sourcelists()
parser = Recognized options for the astro.recipes.SubimageSourceList OptionsParser are: "n" or "name" (default: ) "f" or "filenames" (default: []) "df" or "detection_filenames" (default: []) "sp" or "sexparam" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
set_default_config(sourcelist, frame)

Set instrument specific default config properties

target

alias of astro.main.SourceList.SourceList

exception astro.recipes.SubimageSourceList.SourceListTaskError(message)

Bases: common.log.Error.Error

astro.recipes.SubimageSourceList.create_astrom_params_for_subimage(frame, astrom_params, subimage)
astro.recipes.SubimageSourceList.set_additional_dependencies(sourcelist)

set additional dependencies of the SourceList

astro.recipes.TargetProcessor module

Target Processor Task definition

The Target Processor Task can be run using the dpu or lpu with:

dpu.run(‘Target’, …arguments…) lpu.run(‘Target’, …arguments…)

The main argument to give is the target class to process:

dpu.run(‘Target’, target_cls=<cls> …)

The other arguments determine what to process, for example :

dpu.run(‘Target’, target_cls=<cls>, template=<template_start>)

There are multiple ways to specify the targets to process.

Besides arguments to specify the target(s) there are arguments to control the Target Processor. For example do not commit the result(s):

dpu.run(‘Target’, commit=0, …)

All options are described below in the Task.

Examples :

# OmegaCAM, specify by template start dpu.run(‘Target’, target_cls=RegriddedFrame, instrument=’OMEGACAM’,

filter=’OCAM_g_SDSS’, template=‘2017-02-16T16:52:24.133325’, chip=’ESO_CCD_#65’, depth=1, commit=False)
class astro.recipes.TargetProcessor.TargetProcessorTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : data processing

PURPOSE : Running the Target Processor from the dpu or lpu interface

Depending on the class to process, different options are used to select the input.

Global mandatory options :

target_cls - the persistent class to process instrument - the instrument name of the data

Mandatory options for calibration_classes:

chip, filter, template

Mandatory options for science_classes:

raw_filenames

or:

red_filenames

or combination of (these are given to RawScienceFrame.select):

date, chip, object, filter, template, exptime

Mandatory options for coadd_classes:

dependencies - This dictionary specifies the input RegriddedFrames as
{CoaddedRegriddedFrame.regridded_frames’:
(RegriddedFrame, [object_id1, object_id2, …])}

where each object_id specifies a ReigrddedFrame {‘CoaddedRegriddedFrame.regridded_frames’:

(RegriddedFrame, [(date1,chip1,filter1), (date2,chip2,filter2), …])}

where each (date,chip,filter) specifies a ReigrddedFrame

chip, date - These are mandatory options for the Pipeline framework,
but ignored by the Target processor, set to any dummy value

Optional keywords :

commit - Switch for committing the targets.

dependencies - The fixed dependencies dictionary, specify as:
{‘what.to.fix’: (class, [specifier, …]), ….} see coadd_classes above for an example or common.main.OnTheFly or astro.main.OnTheFly
depth - The calibration depth (default is 1), travesring the number of calibration objects in the tree.
If the value is: 1; it is only checked if a calibration dependency exists 2; it is checked if the calibration dependency can be remade, if the calibration dependencies of the calibration dependency exist 3; it is checked if the calibration dependencies of a calibration dependency can be remade 4; … etc … -1; infinite ! WARNING; setting the depth to a higher value then 1 (or -1) can cause the processing of many objects !

GAS - Set to use the Global AStrometry, the GAS solutions must be already be present

pars - The process parameters dictionary, specify as:

{‘parameter.specifier’: parameter_value, ….} So for example to specify the process parameter MAXIMUM_PSF_DIFFERENCE and the swarp configuration BACK_FILTERSIZE for a RegriddedFrame : {‘RegriddedFrame.process_params.MAXIMUM_PSF_DIFFERENCE’: 0.35,

‘RegriddedFrame.swarpconf.BACK_FILTERSIZE’: 5}
sdepth - The science depth, traversing the number of sience objects in the tree
Default is to process all available science dependencies in the tree (value -1). WARNING; there is probably no use case to change this !

Example instantiating the Task directly :

awe> filename = RawScienceFrame.select_all()[0].filename awe> task = TargetProcessorTask(raw_filenames=[filename]) awe> task.execute()
calibration_classes = (<class 'astro.main.ColdPixelMap.ColdPixelMap'>, <class 'astro.main.DarkCurrent.DarkCurrent'>, <class 'astro.main.DomeFlatFrame.DomeFlatFrame'>, <class 'astro.main.GainLinearity.GainLinearity'>, <class 'astro.main.HotPixelMap.HotPixelMap'>, <class 'astro.main.PhotometricParameters.PhotometricParameters'>, <class 'astro.main.PhotSrcCatalog.PhotSrcCatalog'>, <class 'astro.main.TwilightFlatFrame.TwilightFlatFrame'>)
check_options()

check if mandatory parameters are set

clean_up_before_pickle()

This method is called before the Job is pickled on the cluster side, remove at least all the database objects.

coadd_classes = (<class 'astro.main.RegriddedFrame.CoaddedRegriddedFrame'>,)
date_format = '%Y-%m-%d %H:%M:%S'
default_processing_time = 1799
dpu_conn = None
execute()

execute the task, compute node side

get_process_result()

This method is used to retrieve the result from executing the task. This result should be a list of instances of one of the many sub-types of DBObject.

get_process_time()

return the processing time of this task

get_targets()

get the targets to make, specified as dictionaries

make_targets(targets)

make the targets found

parser = Recognized options for the astro.recipes.TargetProcessor OptionsParser are: "cls" or "target_cls" (default: None) "c" or "chip" (default: ) "d" or "date" (default: ) "e" or "exptime" (default: -1) "f" or "filter" (default: ) "i" or "instrument" (default: ) "o" or "object" (default: ) "raw" or "raw_filenames" (default: []) "red" or "red_filenames" (default: []) "tpl" or "template" (default: ) "C" or "commit" (default: 0) "D" or "depth" (default: 1) "deps" or "dependencies" (default: {}) "G" or "GAS" (default: None) "p" or "pars" (default: {}) "S" or "sdepth" (default: -1) "gra" or "grid_ra" (default: None) "gdec" or "grid_dec" (default: None) "gps" or "grid_pixelscale" (default: 0.0)
run_targetprocessor(job_dicts)

Run the Target Processor Task

science_classes = (<class 'astro.main.AstrometricParameters.AstrometricParameters'>, <class 'astro.main.ReducedScienceFrame.ReducedScienceFrame'>, <class 'astro.main.RegriddedFrame.RegriddedFrame'>)
set_dependencies(targets)
update_options(*args, **kwargs)

Overload of Task.update_options

This is required when TargetProcessor is run as part of the EuclidLIS Pipeline.

exception astro.recipes.TargetProcessor.TargetProcessorTaskError

Bases: Exception

astro.recipes.TinyTim module

class astro.recipes.TinyTim.TinyTimTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data analysis.

PURPOSE : making a TinyTim PSF image.

This task is used to create a TinyTimImage.

Example of use from the awe prompt :

awe> task = TinyTimTask(pars={‘TinyTimImage.tinytimconf.x’: 200,
‘TinyTimImage.tinytimconf.y’: 250})

awe> task.execute()

Mandatory keywords :

Optional keywords :

pars (configuration dictionary) commit (integer: 0 or 1. Default 0)

The type of the values assigned to the keywords is given between parentheses.

execute()
get_process_time()

return the processing time of this task

make_tinytimimage()
parser = Recognized options for the astro.recipes.TinyTim OptionsParser are: "p" or "pars" (default: {}) "C" or "commit" (default: 0)
query_database()
if not self.instrument:
raise TinyTimTaskError, ‘Instrument not specified!’
if not self.filter:
raise TinyTimTaskError, ‘Filter not specified!’
if not self.chip:
raise TinyTimTaskError, ‘Chip not specified!’

query = (Instrument.name == self.instrument) if not len(query):

raise TinyTimTaskError, ‘Query failed to find instrument with name %s’ % self.instrument

instrument = query[0] query = (Filter.name == self.filter) if not len(query):

raise TinyTimTaskError, ‘Query failed to find filter with name %s’ % self.filter

filter = query[0] query = (Chip.name == self.chip) if not len(query):

raise TinyTimTaskError, ‘Query failed to find chip with name %s’ % self.chip

chip = query[0] return instrument, filter, chip

target

alias of astro.main.TinyTimImage.TinyTimImage

exception astro.recipes.TinyTim.TinyTimTaskError(message)

Bases: common.log.Error.Error

astro.recipes.TwilightFlat module

class astro.recipes.TwilightFlat.TwilightFlatTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data processing, calibration pipeline.

PURPOSE : making a master twilight flat for a specific date, chip and
filter.

This task is used to create a master twilight flat.

The task must be instantiated with the appropriate keyword arguments described below, and is executed using its ‘execute’ method.

Example of use from the awe prompt :

awe> task = TwilightFlatTask(instrument=’OMEGACAM’, date=‘2015-02-12’,
chip=’ESO_CCD_#65’, filter=’OCAM_g_SDSS’)

awe> task.execute()

Mandatory keywords :

instrument (string), date (string, yyyy-mm-dd), chip (string), filter (string)

or
instrument (string), chip (string), template (string)
or

raw_filenames (list of strings)

Optional keywords :

overscan (integer: 0-13. Default 6) commit (boolean: False or True. Default False) pars (dictionary of configuration parameters, example:

{‘TwilightFlatFrame.process_params.SIGMA_CLIP’: 4.0})

The type of the values assigned to the keywords is given between parentheses.

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Return the processing time of this task

make_twilightflat()
parser = Recognized options for the astro.recipes.TwilightFlat OptionsParser are: "i" or "instrument" (default: ) "f" or "filter" (default: ) "c" or "chip" (default: ) "d" or "date" (default: ) "tpl" or "template" (default: None) "oc" or "overscan" (default: 6) "raw" or "raw_filenames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: False)
query_database()
set_config()

Set instrument specific default config properties

target

alias of astro.main.TwilightFlatFrame.TwilightFlatFrame

exception astro.recipes.TwilightFlat.TwilightFlatTaskError(message)

Bases: common.log.Error.Error

astro.recipes.TwoDPhotSourceCollectionTask module

Recipe to create SourceCollections containing results of running the 2DPHOT Flag code.

class astro.recipes.TwoDPhotSourceCollectionTask.TwoDPhotSourceCollectionTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

CATEGORY : Data processing, image pipeline.

PURPOSE : Run 2DPHOT code and store resulting
flag values in a SourceCollection.
clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_result()
get_process_time()

Return the processing time of this task

make_twodphot()

Find/make a 2DPHOT catalog for a SourceCollection.

parser = Recognized options for the astro.recipes.TwoDPhotSourceCollectionTask OptionsParser are: "S" or "scid" (default: 0) "n" or "name" (default: ) "C" or "commit" (default: False)
set_config()

Set instrument specific default config properties

target

alias of astro.main.sourcecollection.SourceCollection.SourceCollection

exception astro.recipes.TwoDPhotSourceCollectionTask.TwoDPhotSourceCollectionTaskError(message)

Bases: common.log.Error.Error

astro.recipes.TwoDPhotSourceCollectionTask.main()

astro.recipes.Variability module

class astro.recipes.Variability.VariabilityTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task, astro.recipes.mods.Task.ImagePipelineMixin

clean_up_before_pickle()

This method is intended to prevent both pickle errors and triggering problems with the Oracle object cache leading to erroneous query results in the AWE prompt.

execute()
get_process_time()

Return the processing time of this task

make_var_pipe()
parser = Recognized options for the astro.recipes.Variability OptionsParser are: "i" or "instrument" (default: ) "d" or "date" (default: ) "c" or "chip" (default: ) "tpl" or "template" (default: None) "f" or "filter" (default: ) "o" or "object" (default: ) "C" or "commit" (default: 0) "p" or "pars" (default: {}) "reg" or "reg_filenames" (default: [])
query_database()
retrieve_data()
target

alias of astro.main.VariabilityFrame.VariabilityFrame

exception astro.recipes.Variability.VariabilityTaskError

Bases: Exception

astro.recipes.aweimports module

automatic imports for the interpreter

astro.recipes.pyBDSM module

exception astro.recipes.pyBDSM.BDSMTaskError

Bases: Exception

class astro.recipes.pyBDSM.pyBDSMTask(*args, **kwargs)

Bases: astro.recipes.mods.Task.Task

Recipe for running pyBDSM

Example of use from the awe prompt :

check_preconditions()

check if mandatory parameters are set

execute()

execute the task

get_process_time()

return the processing time of this task

parser = Recognized options for the astro.recipes.pyBDSM OptionsParser are: "F" or "filenames" (default: []) "p" or "pars" (default: {}) "C" or "commit" (default: 0)
run_py_bdsm()

run pyBDSM

Module contents

This directory contains the recipes that are used in a fully functional database environment. It also contains the tasks that are sent to the cluster.