Data Mining Exercises

Investigating Twilight Behavior from RawTwilightFlatFrames

1. Find all OmegaCAM raw twilight flat frames for the chip with name ESO_CCD_#92 and the filter with OCAM_g_SDSS, which have image statistics such that the median is smaller than 5000 ADU. Only select those that have their quality_flags set to zero. Assign the query to a Python variable.

   How many raw twilight flat frames satisfy the above criteria?

   awe> qinstr  = RawTwilightFlatFrame.instrument.name == 'OMEGACAM'
   awe> qfilter = RawTwilightFlatFrame.filter.name   == 'OCAM_g_SDSS'
   awe> qchip   = RawTwilightFlatFrame.chip.name     == 'ESO_CCD_#92'
   awe> qim     = RawTwilightFlatFrame.imstat.median >  1000
   awe> qqf     = RawTwilightFlatFrame.quality_flags == 0
   awe> query  = qinstr & qfilter & qchip & qim & qqf
   awe> print(len(query))
   2314
   

2. Using the result from the previous question, assign the MJD_OBS (the modified Julian date) of each RawTwilightFlatFrame to variable t and the median of the image statistics divided by the EXPTIME of each RawTwilightFlatFrame to variable y. Make a scatter plot of t versus y.

   awe> t = [rtf.MJD_OBS for rtf in query]
   awe> y = [rtf.imstat.median/rtf.EXPTIME for rtf in query]
   or, quicker,
   awe> result = [(rtf.MJD_OBS, rtf.imstat.median/rtf.EXPTIME) for rtf in query]
   awe> t, y = zip(*result)
   awe> pylab.scatter(t, y)
   

3. Since there seems to be a recurring pattern in the previous plot, we’ll have a closer look. Take the fractional part of the t variable, which contains the modified Julian date for each RawTwilightFlatFrame and assign it to the variable tfrac. Clear the figure and make a scatter plot of tfrac versus y.

   awe> tfrac = [k - math.floor(k) for k in t]
   awe> pylab.clf()
   awe> pylab.scatter(tfrac, y)
   

4. Determine the time difference between Greenwich and the place where the RawTwilightFlatFrames were observed - Paranal. Does this correspond to the longitude of Paranal?

Bias level for OmegaCAM

1. Display the bias level as a function of time for chip ESO_CCD_#96 and the first year of observations of the OmegaCAM camera (raw biases are represented in Astro-WISE by the class RawBiasFrame).

   awe> query = (RawBiasFrame.instrument.name == 'OMEGACAM') &\
                (RawBiasFrame.DATE_OBS > datetime.datetime(2011,8,1)) &\
                (RawBiasFrame.DATE_OBS < datetime.datetime(2012,8,1)) &\
                (RawBiasFrame.chip.name == 'ESO_CCD_#96') &\
                (RawBiasFrame.quality_flags == 0) &\
                (RawBiasFrame.is_valid > 0)
   awe> res = [(b.DATE_OBS, b.imstat.median) for b in query]
   awe> dat, val = zip(*res)
   awe> pylab.clf()
   awe> pylab.scatter(dat, val)