HOW-TO Photometric pipeline(3): extinction and zeropoint

This chapter describes how the extinction and zeropoint are derived for a given night. The two steps, for the time being, should be done in tandem. The results are only committed to the database at the very last processing step.

Deriving the atmospheric extinction

The photometric pipeline supports several ways of obtaining the atmospheric extinction:

  1. using two observations of standard fields at two different airmasses
  2. using a default value for the extinction coefficient stored in the database
  3. using one single observation of a standard field and a set of already known zeropoints
  4. using a combination of a standard extinction curve and an extinction report

The first two of these methods are the most flexible, while the fourth method derives from the OmegaCAM photometric calibration plan. It should be pointed out that, contrary to other parts of the photometric pipeline, the first and third way of deriving the atmospheric extinction use input data from whole images. All the extinctions used in the photometric pipeline are in units of mag/am.

Using two observations of standard fields

The ‘classical’ case of the two standard field observations taken at two different airmasses is dealt with by the AtmosphericExtinctionFrames class. Its dependencies consist of two lists of PhotSrcCatalog objects derived earlier from the standard field data, one for every observation. Instances of this class are used as follows:

 1. awe> from astro.main.AtmosphericExtinctionFrames import \
    ...                  AtmosphericExtinctionFrames
 2. awe> from astro.main.PhotSrcCatalog import PhotSrcCatalog
 3. awe> import datetime
 4. awe> date_obs_first = datetime.datetime(2003,2,12,20,15)
 5. awe> date_obs_second = datetime.datetime(2003,2,12,20,20)
 6. awe> photcats_1 = (PhotSrcCatalog.date_obs == date_obs_first) &\
    ...               (PhotSrcCatalog.filter.name == '191') &\
    ...               (PhotSrcCatalog.is_valid == 1)
 7. awe> photcats_2 = (PhotSrcCatalog.date_obs == date_obs_second) &\
    ...               (PhotSrcCatalog.filter.name == '191') &\
    ...               (PhotSrcCatalog.is_valid == 1)
 8. awe> extinct = AtmosphericExtinctionFrames()
 9. awe> extinct.polar = list(photcats_1)
10. awe> extinct.equat = list(photcats_2)
11. awe> extinct.make()

In lines (1)-(2) the relevant classes are imported, and in steps (6) and (7) the necessary dependencies are retrieved from the database. In lines (8)-(11), an AtmosphericExtinctionFrames object is instantiated, its dependencies are set, and the make method is called.

Using a default value from the database

If no suitable data is present to derive the extinction from, it is (or at least should be) possible to retrieve a ready-made extinction coefficient from the database. This particular extinction is covered by the AtmospericExtinctionCoefficient class. Such an extinction is retrieved from the database as follows:

awe> from astro.main.AtmosphericExtinction import AtmosphericExtinctionCoefficient
awe> extinct = AtmosphericExtinctionCoefficient.get('191', 'A5382-1-7')

which will give you the value for the INT/WFC B filter and chip A5382-1-7:

awe> extinct.value
0.22

This extinction can readily be used.

Inserting an extinction coefficient into the system

To insert an extinction coefficient for a certain filter/instrument combination into the database, a simple tool is available. This tool is located in the ./awe/astro/toolbox/photometry/ directory and is called ingest_extinction. To get information about how to use the tool just type:

awe $AWEPIPE/astro/toolbox/photometry/ingest_extinction.py

and a doc-string will appear on screen. The tool only accepts input for filters and instruments that are actually present in the database. If a certain filter and/or instrument is not present, the tool refuses to comply.

Using one observation and known zeropoints

This particular way of deriving the extinction is covered by the AtmosphericExtinctionZeropoint class. For instances of this class to work, two dependencies should be set: polar and photoms. The first dependency expects a list of PhotSrcCatalog objects derived from the single observation of the standard field, whereas the second dependency likes to see a list of PhotometricParameters objects (see Making the zeropoint from the awe-prompt for more on this class). An example of its use:

 1. awe> from astro.main.AtmosphericExtinctionZeropoint import AtmosphericExtinctionZeropoint
 2. awe> from astro.main.PhotSrcCatalog import PhotSrcCatalog
 3. awe> from astro.main.PhotometricParameters import PhotometricParameters
 4. awe> import datetime
 5. awe> date_obs = datetime.datetime(2003,2,12,20,15)
 6. awe> photcats = (PhotSrcCatalog.date_obs == date_obs) &\
    ...             (PhotSrcCatalog.filter.name == '191') &\
    ...             (PhotSrcCatalog.is_valid == 1)
 7. awe> photoms = (PhotometricParameters.timestamp_start <= date_obs) &\
    ...            (PhotometricParameters.timestamp_end > date_obs) &\
    ...            (PhotometricParameters.filter.name == '191') &\
    ...            (PhotometricParameters.is_valid == 1)
 8. awe> extinct = AtmosphericExtinctionZeropoint()
 9. awe> extinct.polar = list(photcats)
10. awe> extinct.photoms = list(photoms)
11. awe> extinct.make()

In lines (1)-(3) the relevant classes are imported, and in steps (6) and (7) the necessary dependencies are retrieved from the database. In lines (8)-(11), an AtmosphericExtinctionZeropoint object is instantiated, its dependencies are set, and the make method is called. In the context of deriving a zeropoint, this particular method of deriving an atmospheric extinction seems somewhat out of place. Its primary use lies in the derivation of a nightly extinction report. It is mentioned here for completeness.

Using an extinctioncurve and an extinction report

The final, and likely most inflexible, way of deriving the extinction combines the results from a nightly monitoring/extinction report with a standard extinction curve. The class that deals with this situation is the AtmosphericExtinctionCurve class. Its use is as follows:

 1. awe> from astro.main.AtmosphericExtinctionCurve import AtmosphericExtinctionCurve
 2. awe> from astro.main.PhotometricExtinctionReport import PhotometricExtinctionReport
 3. awe> from astro.main.PhotExtinctionCurve import PhotExtinctionCurve
 4. awe> from astro.main.Filter import Filter
 5. awe> import datetime
 6. awe> date = datetime.datetime(2003,2,12)
 7. awe> report = (PhotometricExtinctionReport.timestamp_start <= date) &\
    ...           (PhotometricExtinctionReport.timestamp_end > date) &\
    ...           (PhotometricExtinctionReport.is_valid == 1)
 8. awe> extcurve = PhotExtinctionCurve.get()
 9. awe> filter = (Filter.name == '191')[0]
10. awe> extinct = AtmosphericExtinctionCurve()
11. awe> extinct.report = report[0]
12. awe> extinct.extcurve = extcurve
13. awe> extinct.filter = filter
14. awe> extinct.make()

In lines (1)-(4) the relevant classes are imported, and in steps (7)-(9) the necessary dependencies are retrieved from the database (note the Filter object). In lines (10)-(14) an AtmosphericExtinctionCurve object is instantiated, its dependencies are set and the make method is called.

Notes on the monitoring/extinction report

The monitoring/extinction report is defined in the OmegaCAM photometric calibration plan. This report is derived from a set of data with a very unique feature: every single observation in the data set is observed in FOUR photometric bands SIMULTANEOUSLY. In the photometric pipeline, this particular, dedicated report is represented by the PhotometricExtinctionReport class. The make method of the PhotometricExtinctionReport class is very strict with respect to the DATE_OBS of the original standard field observations that go into making it: unless the instrument from which the data to be reduced happens to have the capability to observe FOUR different photometric bands simultanesouly, using this class will be very difficult.

Making the zeropoint from the awe-prompt

The zeropoint for the night is derived in the final processing step of the photometric pipeline. It is this end result that is used by the image pipeline. The class that represents the final result is PhotometricParameters. Instances of this class combine the zeropoint and the atmospheric extinction for the night.

Using a pre-cooked recipe

Deriving the zeropoint for the night using a pre-cooked recipe is done as follows:

awe> from astro.recipes.PhotCalExtractZeropoint import PhotomTask
awe> task = PhotomTask(instrument='WFI', raw_filenames=['r336603_3.fits'])
awe> task.execute()

To get detailed and full information about the use of the task, type:

awe> help(PhotomTask)

which will, for example, show the parameters that can be passed to the constructor of the task.

The PhotomTask object will, by default, search the database for a PhotometricExtionctionReport object to fulfill the dependency of the atmospheric extinction. If the database does not return a result, the task will try to retrieve an AtmosphericExtinctionCoefficient object from the database instead. If both queries fail, the PhotomTask will raise an error.

Using the basic building blocks

Besides using the pre-cooked recipe, it is also possible to use the basic building blocks of the photometric pipeline themselves. This gives much more control over the processing down to the smallest of details. The following code gives an example of what the use of the basic building blocks would look like:

1. awe> from astro.main.PhotometricParameters import PhotometricParameters
2. awe> from astro.main.PhotSrcCatalog import PhotSrcCatalog
3. awe> photcat = (PhotSrcCatalog.frame.filename == 'my_standards.fits')[0]
4. awe> photom = PhotometricParameters()
5. awe> photom.photcat = photcat
6. awe> photom.extinct = extinct
7. awe> photom.make()
8. awe> photom.commit()

In lines (1)-(2) the relevant classes are imported, and in step (3) the necessary dependency is retrieved from the database. In lines (4)-(7), a PhotometricParameters object is instantiated, its dependencies are set, and the make method is called. In step (8), the PhotometricParameters object is committed to the database. Please note that in step (6) the atmospheric extinction is the one derived previously. The result of calling the make method is a zeropoint that is valid for an airmass of 0.00 (the response of your instrument), and an exposure time of 1 second.

Configuring the photometric CalFile

Any PhotometricParameters object has a knob that allows the user to configure the behaviour of its make method. To get information about the available configuration options, just type:

awe> photom.process_params.info()

which will show the available configurable parameters, their meaning, and their default setting.

Inspecting the results of making the zeropoint

Just as for photometric catalogs, the PhotometricParameters object has an inspect method that allows the user to see and validate the results of deriving the zeropoint. The method is called thus:

awe> photom.inspect()

which will result in an output to screen that looks like the one shown in Figure [photom_inspect_plot]. The inspect plot shows the magnitudes of the individual standard stars as known to the standard star catalog on the x-axis, and their associated (extinction corrected) zeropoints on the y-axis. The blue dot-dashed lines enclose the sources that have been used in deriving the zeropoint, and the thick, blue, dashed line shows the location of the finally derived zeropoint within the distribution of individual zeropoints. The inspect method of course also functions on PhotometricParameters objects retrieved from the database.

the *inspect* method of a PhotometricParameters object

The result of invoking the inspect method of a PhotometricParameters object.

Inserting a photometric CalFile into the system

To insert a photometric CalFile for a given filter/chip/instrument combination into the database in the case that no photometric calibration can be done, a simple tool is available. The tool is located in the ./awe/astro/toolbox/photometry/ directory and is called ingest_photometrics. To get information about how to use the tool just type:

awe $AWEPIPE/astro/toolbox/photometry/ingest_photometrics.py

and a doc-string will appear on screen. The tool only accepts input for chips, filters and instruments that are actually present in the database. If a certain chip, filter and/or instrument are not present, the tool refuses to comply.