# Data Mining Exercises¶

## Investigating Twilight Behavior from RawTwilightFlatFrames¶

1. Find all OmegaCAM raw twilight flat frames for the chip with name ESO_CCD_#92 and the filter with OCAM_g_SDSS, which have image statistics such that the median is smaller than 5000 ADU. Only select those that have their quality_flags set to zero. Assign the query to a Python variable.

How many raw twilight flat frames satisfy the above criteria?

awe> qinstr  = RawTwilightFlatFrame.instrument.name == 'OMEGACAM'
awe> qfilter = RawTwilightFlatFrame.filter.name   == 'OCAM_g_SDSS'
awe> qchip   = RawTwilightFlatFrame.chip.name     == 'ESO_CCD_#92'
awe> qim     = RawTwilightFlatFrame.imstat.median >  1000
awe> qqf     = RawTwilightFlatFrame.quality_flags == 0
awe> query  = qinstr & qfilter & qchip & qim & qqf
awe> print(len(query))
2314

2. Using the result from the previous question, assign the MJD_OBS (the modified Julian date) of each RawTwilightFlatFrame to variable t and the median of the image statistics divided by the EXPTIME of each RawTwilightFlatFrame to variable y. Make a scatter plot of t versus y.

awe> t = [rtf.MJD_OBS for rtf in query]
awe> y = [rtf.imstat.median/rtf.EXPTIME for rtf in query]
or, quicker,
awe> result = [(rtf.MJD_OBS, rtf.imstat.median/rtf.EXPTIME) for rtf in query]
awe> t, y = zip(*result)
awe> pylab.scatter(t, y)

3. Since there seems to be a recurring pattern in the previous plot, we’ll have a closer look. Take the fractional part of the t variable, which contains the modified Julian date for each RawTwilightFlatFrame and assign it to the variable tfrac. Clear the figure and make a scatter plot of tfrac versus y.

awe> tfrac = [k - math.floor(k) for k in t]
awe> pylab.clf()
awe> pylab.scatter(tfrac, y)

4. Determine the time difference between Greenwich and the place where the RawTwilightFlatFrames were observed - Paranal. Does this correspond to the longitude of Paranal?

## Bias level for OmegaCAM¶

1. Display the bias level as a function of time for chip ESO_CCD_#96 and the first year of observations of the OmegaCAM camera (raw biases are represented in Astro-WISE by the class RawBiasFrame).

awe> query = (RawBiasFrame.instrument.name == 'OMEGACAM') &\
(RawBiasFrame.DATE_OBS > datetime.datetime(2011,8,1)) &\
(RawBiasFrame.DATE_OBS < datetime.datetime(2012,8,1)) &\
(RawBiasFrame.chip.name == 'ESO_CCD_#96') &\
(RawBiasFrame.quality_flags == 0) &\
(RawBiasFrame.is_valid > 0)
awe> res = [(b.DATE_OBS, b.imstat.median) for b in query]
awe> dat, val = zip(*res)
awe> pylab.clf()
awe> pylab.scatter(dat, val)