astro.util package

Submodules

astro.util.BinaryFitsTable module

Write a SourceList as binary Fits table.

Write the Fits file per incoming SourceList row. This is usefull for web serivces that do not need to have the whole SourceList in memory before writing out the binary Fits table to the client.

Example :

# this can also be a socket for example, as long as it has a write method f = open(‘test.fits’) sl = (SourceList.SLID==414761)[0] from astro.util.BinaryFitsTable import write_binary_fits_table write_binary_fits_table(f, sl)

WJ Vriend

astro.util.BinaryFitsTable.test()

method uses a ‘fake’ sourcelist for testing write_binary_fits_table

class astro.util.BinaryFitsTable.test_sourcelist

Bases: object

“fake” SourceList for testing purposes

get_attributes_on_sources()
number_of_sources = 4
sources = [{'col2': 312, 'col4': 312.0, 'col1': 'NGC1', 'col3': 312}, {'col2': 334, 'col4': 334.0, 'col1': 'NGC2', 'col3': 334}, {'col2': 308, 'col4': 308.0, 'col1': 'NGC3', 'col3': 308}, {'col2': 317, 'col4': 317.0, 'col1': 'NGC4', 'col3': 317}]
astro.util.BinaryFitsTable.translate_attr(attr_type)

tranlsate the attribute type

astro.util.BinaryFitsTable.write_binary_fits_table(f, sl)

write SourceList as binary fits table

astro.util.BinaryFitsTable.write_primary(f)

write the primary fits hdu

astro.util.CalibratePhotometryToGaia module

Use Gaia DR2 and Stellar Locus Regression to perform photometric homogenization.

Authors: Jelte de Jong (jdejong@astro.rug.nl)
Koen Kuijken (kuijken@strw.leidenuniv.nl)

Usage: the starting point is a ConcatenateAttributes that combines GAaP photometry in ugri (the output from the GAaPSourceCollecctionTasks), extinctions (the output from the ExtinctionSourceCollectionTask) as well as the columns RA, DEC, Flag_r, CLASS_STAR_r, FWHM_IMAGE_r, and IMAFLAGS_ISO_r. Returned is a dictionary with the magnitude offsets per filter from the SLR analysis, the total magnitude offsets per filter, and the number of stars the latter are based on. By default an inspection plot is also generated.

Example:

awe> from astro.util.CalibratePhotometryToGaia import CalibratePhotometryToGaia awe> calibrate = CalibratePhotometryToGaia(concatenate_attr) awe> offsets = calibrate.calculate_zpt_offsets()

class astro.util.CalibratePhotometryToGaia.CalibratePhotometryToGaia(concatenate_attributes, make_inspect_figure=True)

Bases: object

Use Gaia DR2 and Stellar Locus Regression to perform photometric homogenization.

Authors: Jelte de Jong (jdejong@astro.rug.nl)
Koen Kuijken (kuijken@strw.leidenuniv.nl)

Usage: the starting point is a ConcatenateAttributes that combines GAaP photometry in ugri (the output from the GAaPSourceCollecctionTasks), extinctions (the output from the ExtinctionSourceCollectionTask) as well as the columns RA, DEC, Flag_r, CLASS_STAR_r, FWHM_IMAGE_r, and IMAFLAGS_ISO_r. Returned is a dictionary with the magnitude offsets per filter from the SLR analysis, the total magnitude offsets per filter, and the number of stars the latter are based on. By default an inspection plot is also generated.

Example:

awe> from astro.util.CalibratePhotometryToGaia import CalibratePhotometryToGaia awe> calibrate = CalibratePhotometryToGaia(concatenate_attr) awe> offsets = calibrate.calculate_zpt_offsets()

calculate_zpt_offsets()
derive_offset_r()
get_SLR_offsets()
load_data()
astro.util.CalibratePhotometryToGaia.get_gaia_sources(minra, maxra, mindec, maxdec)
astro.util.CalibratePhotometryToGaia.gisdss_from_gikids(gi_kids)
astro.util.CalibratePhotometryToGaia.rkids_min_rsdss(gi_kids)
astro.util.CalibratePhotometryToGaia.stellar_locus_rG_gi_SDSS(gi_sdss)

astro.util.CanonicalName module

CanonicalName

Implements a filename convention

class astro.util.CanonicalName.CanonicalName

Bases: object

Implements a filename convention

chip_name = '-'
date = '-'
extension = 'fits'
file_basetype = '-'
file_subtype = '-'
filter_name = '-'
get_uid()
instrument_name = '-'
mode = '-'
owner = '-'
process_level = '-'
project = '-'
set_owner()

Read the user from the environment and set the property owner

strategy = '-'
to_string()

Return the contents of the CanonicalName object as a string.

astro.util.CanonicalName.replace_reserved(word)

Replace any occurance of a reserved character (e.g., ‘/’ for a filename) with a non-reserved character (e.g., ‘_’ or URL encoded equivalent).

astro.util.CatalogAssociation module

Astrometrically associate 2 or more FITS catalogs.

class astro.util.CatalogAssociation.CatalogAssociation(filenames=[], catalogs=[], table='LDAC_OBJECTS', alpha='ALPHA_SKY', delta='DELTA_SKY', radius=5.0)

Bases: object

Astrometrically associate 2 or more catalogs stored in FITS tables resulting in a data dictionary with keys of the table column names and values of matched arrays. In addition to the original columns will be added the coordinates of the associate and the delta between them.

The type of association is a matched association. This means that all sources are matched against all other sources in all the input catalogs.

check_preconditions()
make()
make_association()
read_catalogs()

astro.util.CosmologicalDistance module

A class to calculate cosmological distances The procedure by Hoggs 2000 and Peebles 1993 are followed http://arxiv.org/abs/astro-ph/9905116

class astro.util.CosmologicalDistance.CosmologicalDistance(omega_m=0.3, omega_l=0.7, h0=0.7)

Bases: object

A class to calculate cosmological distances The procedure by Hoggs 2000 and Peebles 1993 are followed http://arxiv.org/abs/astro-ph/9905116

The cosmological parameters of interest are h0 ~0.7 = dimensionless constant: hubble parameter / (100 (km/s)/Mpc) omega_m ~ 0.3 = matter content of the universe omega_l ~ 0.7 dark energy omega_k = 1 - omega_m - omega_l = curvature

Other parameters: redshift = the redshift of the source(s) for which distance is

being measured
angular_diameter_distance(redshift)

Returns the angular diameter distance: the ratio of an object’s physical transverse size to its angular size. Hogg 2000, chapter 6, equation 18.

angular_separation(ra0, dec0, ra1, dec1)

Calculates the angular separation between two points using the haversine formula. Input and output in radians.

c = 299792.458
comoving_distance(redshift)

Returns the comoving distance of an object of given redshift Hogg 2000, chapter 5, equation 15.

comoving_volume(redshift)

Returns the comoving volume within radius redshift for the entire sky in Mpc^3.

comoving_volume_element(redshift)

Returns the comoving volume element in Mpc^3. Hogg 2000, chapter 9, equation 28.

comoving_volume_slice(redshiftmin, redshiftmax, solidangle)

Returns the comoving volume in Mpc^3of a slice of angulare size ‘solidangle’ (in sterradian) between redshifts ‘redshiftmin’ and ‘redshiftmax’.

ee(redshift)

Hogg 2000, equation 14. Peebles 1993, pp 310-321.

eei(redshift)

The inverse of ee, used for integration

eeii(redshift)

The integration of the inverse of ee.

flat = 5e-05
h0(h0=nan)

read or set the Hubble Parameter in units of (100 km/s/Mpc)

hubble_distance()

Returns the Hubble distance. Hogg 2000 eqn. 4

inverse_comoving_distance(comoving_distance)

Returns the redshift from the comoving distance.

luminosity_distance(redshift)

Returns the luminosity distance. Hogg 2000, chapter 7, equation 21.

omega_k = 0.0
omega_l = 0.7
omega_m = 0.3
proper_motion_distance(redshift)

Returns the propermotion distance at distance redshift. Hogg 2000, chapter 5, equation 16.

set_omega_k()

You cannot set the curvature of the universe directly.

set_omega_l(omega_l)

Set the vacuum/dark energy/lambda content of the universe.

set_omega_m(omega_m)

Set the matter content of the universe (dark and baryonic).

transverse_comoving_distance(redshift, arc)

Returns the comoving distance between two points at a distance ‘redshift’ seperated by an angular separation arc Hogg 2000, chapter 5.

astro.util.CosmologicalDistance.integrate_simple(func, a, b, tolerance=1e-06)

Integrate func from a to b with tolerance tolerance.

Closed Simpson’s rule for
int_a^b f(x) dx

Divide [a,b] iteratively into h, h/2, h/4, h/8, … step sizes; and, for each step size, evaluate f(x) at a+h, a+3h, a+5h, a+7h, .., b-3h, b-h, noting that other points have already been sampled.

At each iteration step, data are sampled only where necessary so that the total data is represented by adding sampled points from all previous steps:

step 1: h a—————b step 2: h/2 a——-^——-b step 3: h/4 a—^——-^—b step 4: h/8 a-^—^—^—^-b total: a-^-^-^-^-^-^-^-b

So, for step size of h/n, there are n intervals, and the data are sampled at the boundaries including the 2 end points.

If old = Trapezoid formula for an old step size 2h, then Trapezoid formula for the new step size h is obtained by

new = old/2 + h{f(a+h) + f(a+3h) + f(a+5h) +…+ f(b-3h)
  • f(b-h)}
Also, Simpson formula for the new step size h is given by
simpson = (4 new - old)/3
astro.util.CosmologicalDistance.newton(func, funcd, x, tolerance=1e-06)

f(x)=func(x), f’(x)=funcd(x)

Ubiquitous Newton-Raphson algorithm for solving
f(x) = 0
where a root is repeatedly estimated by
x = x - f(x)/f’(x)

until |dx|/(1+|x|) < tolerance is achieved. This termination condition is a compromise between

|dx| < tolerance, if x is small |dx|/|x| < tolerance, if x is large
astro.util.CosmologicalDistance.quad(func, a, b, tolerance=1e-06)

Integrate func from a to b with tolerance tolerance.

Closed Simpson’s rule for
int_a^b f(x) dx

Divide [a,b] iteratively into h, h/2, h/4, h/8, … step sizes; and, for each step size, evaluate f(x) at a+h, a+3h, a+5h, a+7h, .., b-3h, b-h, noting that other points have already been sampled.

At each iteration step, data are sampled only where necessary so that the total data is represented by adding sampled points from all previous steps:

step 1: h a—————b step 2: h/2 a——-^——-b step 3: h/4 a—^——-^—b step 4: h/8 a-^—^—^—^-b total: a-^-^-^-^-^-^-^-b

So, for step size of h/n, there are n intervals, and the data are sampled at the boundaries including the 2 end points.

If old = Trapezoid formula for an old step size 2h, then Trapezoid formula for the new step size h is obtained by

new = old/2 + h{f(a+h) + f(a+3h) + f(a+5h) +…+ f(b-3h)
  • f(b-h)}
Also, Simpson formula for the new step size h is given by
simpson = (4 new - old)/3

astro.util.ExportDict module

class astro.util.ExportDict.ExportAction(eid='', name='')

Bases: object

class astro.util.ExportDict.ExportObject

Bases: object

Class used to export objects through SAMP.

get_property(name, nr=None)
sampdict()

Returns a dictionary to be used over SAMP.

txt(d=0)
class astro.util.ExportDict.ExportProperty(name='', eclass='', description='', value='', readonly=True)

Bases: object

Class used to export properties through SAMP.

sampdict()

astro.util.GlobalPhotometry module

astro.util.GlobalPhotometry.apply_global(ext, chips_diffs, obj, filter, template_start, q_slids, time_creation, commit=0)

calculate (and optionally commit) the PhotometricParameters First: we search for ZP observations (PhotometricParameters derived from SA fields excluding SA110 and SA98) valid for the night of the OB, if these are not found a forever is used. After calculating a averaged ZP, the globalphotometry is applied for each ReducedScienceFrame within an OB, and PhotometricParameters valid for each ReducedScienceFrame are committed if needed

astro.util.GlobalPhotometry.calculate_quality(star_dict, sl_dither_dict, rs, zp_offsets, Object)
astro.util.GlobalPhotometry.deltaMag(filter)
astro.util.GlobalPhotometry.det_magnitude_range(mags_info, filter, Object, solution='good')

a magnitude range is determined, which is used to select objects to calculate the global photometry (extinction + off-sets). We calculate, in magnitude bins, the scatter of magnitude differences of similar detected objects in the OB. For bright saturated objects, this results in a large standard deviation (SD); for fainter magnitudes, this SD has will drop. The ‘drop point’ defines the brighter magnitude limit (ll). The fainter magnitude limit is determined by looking at increase of SD. When this is larger than 0.025 magnitude than the SD of the ll, this defines the fainter magnitude limit (ul). If (ul - ll) is less than some filter dependent value, the ul is increased to satisfy that (ul - ll) is equal to this filter dependent value. When no ul is found, the ul and ll are fixed to some filter dependent values.

astro.util.GlobalPhotometry.do_calc(filename, Object, star_dict, template_start)

the procedure is the main engine of the global photometry calculations, it first reads the matched association catalog. After many filtering steps that select the proper objects, two dictionaries are made with keywords related to the separate dithers and chips. Next: the dithers dictionary is used to calculate the extinction between dithers. These extinctions are subsequently applied to the magnitudes present in the chips dictionary. The latter dictionary is used to calculate the global photometry, i.e. offsets between chips.

astro.util.GlobalPhotometry.do_moments(data_dict, obs, weights=None)

this is the fitting algorithm of Maddox et al. 1990

astro.util.GlobalPhotometry.get_default_ul_ll(filter, deltamag)
astro.util.GlobalPhotometry.get_forever_off_sets(filter, julian_date)

as of 26-06-2015 forevers for gri will be a weighted average over 1) calculated global photometry zp offsets + their stdvs and 2) calculated weighted zeropoints zp offsets + stdvs. Data before 2012-06-03 (~julian_date=56081),when ccd82 was malfuntioning, uses the ‘old’ solutions

astro.util.GlobalPhotometry.get_forever_zp_template_start(filter)
astro.util.GlobalPhotometry.make(slids, commit=0, do_global='yes', do_calculate_quality=False)
astro.util.GlobalPhotometry.sigma_clip(D, cl=3, kmax=10)

astro.util.HeaderExtractor module

Extract a primary header and the first extension header (if it exists) from a FITS file. The FITS file can be (un)compressed on disk or from the data-server.

class astro.util.HeaderExtractor.HeaderExtractor(filename, bytes=17280, raw=False)

Bases: object

Extract a header(s) from a FITS file with support for multiple formats.

check_preconditions()

Does the FITS file exists?

execute()
get_header()
is_compressed = False
exception astro.util.HeaderExtractor.HeaderExtractorError

Bases: Exception

astro.util.HeaderExtractor.extract_headers(string, raw=False)

Extracts the primary header and the first extended header (if it exists) from a string extracted from a fits file object. Returns a list of two darma.header.header instances, or a list of one darma.header.header instance and None if there is no extension.

string: string containing raw header information (i.e., the begining of
a FITS file)

raw: return the raw strings, not the header instances

astro.util.HeaderExtractor.get_string(filename, bytes)

Open a FITS file from a (un)compressed file or URL and return the first bytes bytes and a status flag indicating if it is a FITS file. In the case it is not or if there is an error, an appropriate message is returned. The return format is: (some_string, True|False)

filename: filename string
bytes: the maximum number of bytes to read from the file
astro.util.HeaderExtractor.is_compressed(pri, ext)

Find out if a FITS file is compressed using primary and extension headers.

astro.util.Image module


Make multi-extension FITS (MEF) images from single-extension FITS (SEF) images. It takes as input a list of filenames, and the associated files must exist on disk.


Mandatory inputs :

-o : filename of output image -i : filename list of input images

Configuration parameter (optional) :

-t : type of imput images: raw or trimmed -c : clobber rather than rename the output file if it exists

Example of use :

awe Image.py -o output.fits -i input*.fits -t raw -c

class astro.util.Image.Image(filename, filenames=[], frames=[], retrieve=True, frame_type='trimmed', clobber=False, memmap=True)

Bases: object

A class used to create multi-extension FITS (MEF) from a set of single- extension FITS (SEF). It can take as input, Frame objects of any kind, or simply a list of filenames. In either case, the associated files must exist on disk.

To create a MEF from Frame objects:

> MEF = Image(‘out_file.fits’) > MEF.frames = list_of_instantiated_frames > MEF.make()

or from filenames:

> MEF = Image(‘out_file.fits’) > MEF.filenames = list_of_filenames > MEF.make()

If both frames and filenames exists, frames take precedence. filenames or frames can also be specified as an argument in the constructor.

check_preconditions()
combine_separate_frames()

Take the separate SEF files and combine them into one MEF file.

copy_attributes()
copy_keys_to_header(primary)

Copy keys and comments from an example header to the primary header using a list provided by the header translator.

filenames = []
frame_types = ['raw', 'trimmed']
frames = []
instrument_name = None
make()
remove_keys_from_header(ext)

Remove unnecessary keys from extension headers using a list provided by the header translator.

An implicit assumption by the system is that keywords that need translating upon ingestion occur only in the primary header. Currently, Eclipse’s header.merge() method does not overwrite existing keywords, so the extension should have them removed so the translated values can be copied back properly.

retrieve_frames()

Retrieve frames if desired and available.

class astro.util.Image.RawData(filename, filenames=[], frames=[], frame_type='raw', clobber=False, memmap=1)

Bases: astro.util.Image.Image

A class used to create multi-extension FITS (MEF) from a set of single- extension FITS (SEF). It can take as input, Frame objects of any kind, or simply a list of filenames. In either case, the associated files must exist on disk. The new MEF is RawFitsData to be ingested.

To create a MEF from Frame objects:

> RD = RawData(‘out_file.fits’) > RD.frames = list_of_instantiated_frames > RD.make()

or from filenames:

> RD = RawData(‘out_file.fits’) > RD.filenames = list_of_filenames > RD.make()

If both frames and filenames exists, frames take precedence. filenames or frames can also be specified as an argument in the constructor.

This class uses Image as its base class.

copy_attributes()
copy_keys_to_header(primary)

Copy keys and comments from an example header to the primary header using a list provided by the header translator.

remove_keys_from_header(ext)

Remove unnecessary keys from extension headers using a list provided by the header translator.

An implicit assumption by the system is that keywords that need translating upon ingestion occur only in the primary header. Currently, Eclipse’s header.merge() method does not overwrite existing keywords, so the extension should have them removed so the translated values can be copied back properly.

astro.util.Image.array_size(data)

Return the number of array elements in an array data.

astro.util.Image.check_astrometry_in_header(ext, trans)
astro.util.Image.check_chip_attributes_of_inputs(chip_list)

This little routine checks the properties of the chip attributes of the frames contained in the input list. It checks whether every chip is included only once in a list.

astro.util.Image.check_uniqueness_of_instrument(instrument_names)

This small routine checks whether all the input frames are from the same instrument. The “input_frames” parameter must be a list, and all the items included in the list must have a valid “instrument” attribute.

astro.util.Image.create_extension(file_name, extname='IMAGE', memmap=1)

This function opens a FITS file via PyFits, checks the data for validity (if the FITS file is FITS, if the data exists, if the data has more than one value), and returns a PyFits ImageHDU constructed from the data and header of the FITS file if valid.

astro.util.Image.is_homogeneous(data)

Test if all values in the array are the same, taking into account non-finite numbers.

astro.util.Image.main()
astro.util.Image.parse_command_line()
astro.util.Image.update_crpix(ext, trans)
astro.util.Image.usage()

astro.util.LVFrame module

class astro.util.LVFrame.LVFrame(**kw)

Bases: astro.main.BaseFrame.BaseFrame

DATE

UTC date the original data file was saved [None]

EPOCH

Epoch of the mean equator [year]

EQUINOX

Equinox of the coordinate system used to express the WCS mapping [year]

NAXIS1

Length of data in axis 1 [pixel]

NAXIS2

Length of data in axis 2 [pixel]

OBJECT

Object name [None]

RADECSYS

System of the RA/Dec coordinates [None]

SKY

Sky background [mag / arcsec**2]

astrom

Information about the astrometry [None]

correctEPOCH()
creation_date

Date this object was created [None]

filename

The name of the associated file [None]

filter

Information about the filter [None]

globalname

The name used to store and retrieve file to and from Storage [None]

imstat

Image statistics for the frame [None]

initialize_chip_attribute()
initialize_filter_attribute()
initialize_imstat_attribute()
initialize_instrument_attribute()
instrument

Information about the instrument [None]

is_valid

Manual/external flag to disqualify bad data (SuperFlag) [None]

make()
make_astrom()
object_id

The object identifier

The object identifier is an attribute shared by all persistent instances. It is the prime key, by which object identity is established

process_status

A flag indicating the processing status [None]

quality_flags

Automatic/internal quality flag [None]

astro.util.LVRecipe module

astro.util.LVRecipe.main()
astro.util.LVRecipe.make_LVFrames(catalog)
astro.util.LVRecipe.make_valid_filenames(basename)
astro.util.LVRecipe.parse_command_line()
astro.util.LVRecipe.usage()

astro.util.LVtoBaseFrame module

astro.util.LVtoBaseFrame.LVsourcelist(catdir)
astro.util.LVtoBaseFrame.convert_filename(inputname)
astro.util.LVtoBaseFrame.main()
astro.util.LVtoBaseFrame.stop_pgm(f_log)

astro.util.MakePointings module

class astro.util.MakePointings.MakePointings(ins='VIRCAM', object='', filter='', naxis=(2048, 2048), crval1=(33, 40), crval2=(-10, -2), dist=0.05, check_validity=True, check_quality=True, project='VIKING', privileges=2, dates=(None, None), creator=0)

Bases: object

Primarily developed for VIKING data, this utility groups sets of data by:

tile: OBJECT, filter dither: OBJECT, filter, DATE_OBS jitter: OBJECT, filter, DATE_OBS, a spatial index

A tile is all dithers, all chips for a given observation. A jitter is all jitters, all chips that are observed on a given night, assuming that they belong together. A jitter is one or more exposures in a dither that within some spatial limit.

Running the make() method after instantiation queries the database and contructs a dictionary for all three groupings whose keys are tuples of the listing above, and whose values are lists of ReducedScienceFrame filenames. If necessary, this utility can be expanded to go to RawScienceFrames as well.

Their is also the ability to expand PhotometricParameters timestamps given the restriction of the VIKING data. This method is likely unique to VIKING data and is thus not recommended for general use.

Finally, there is the ability to discover and process via the DPU all new regrids/coadds that are found. This functionality is also optimized for VIKING data and should be used with care for other instruments.

expand_phot_timestamps()

For VIKING data, locate the sets of jitters for each chip and expand PhotometricParameters timestamps to cover each set. These Photometric parameters are only defined for the combined jitter set for a chip, known as a stack, and needs to be expanded to cover the indivual exposures taht went into the stack.

find_coadds_to_process()

Discover if the regrids for a given tile exist and if all have the same grid_target, and if a coadd has been made from them. If they exists properly and no coadd exists, mark them for processing by adding an entry in the coadds attribute.

find_regrids_to_process()

Discover if the regrids for a given tile exist and if all have the same grid_target. If they do not, mark them for processing by adding an entry in the regrids attribute.

make()

Run query_database(), parse_dithers(), and parse_jitters() to populate the tiles, dithers, and jitters attributes.

parse_dithers()

Create dictionaries with lists of ReducedScienceFrame filenames describing the tiles and dithers found in the query results.

parse_jitters()

Create dictionaries with lists of ReducedScienceFrame filenames describing the jitters found in the query results.

process_found_coadds(wait=False, nthreads=0)

Send the lists of ReducedScienceFrames found to need regridding to the DPU for processing.

wait: wait between dpu jobs until the previous one is
finished
nthreads: specify the NTHREADS value for SWarp while
processing the coadds
process_found_regrids(wait=False, subtype=2, force=False)

Send the lists of ReducedScienceFrames found to need regridding to the DPU for processing.

wait: wait between dpu jobs until the previous one is
finished
subtype: specify the BACKGROUND_SUBTRACTION_TYPE for the
regrids

force: force (re)processing of all regrids in a list

query_database()

Find all the ReducedScienceFrames that meet the specified criteria.

astro.util.MakePointings.query_coadded(instrument='', object='', filter='', crval1=(0, 0), crval2=(0, 0), check_validity=True, check_quality=True, project='', privileges=0, dates=(None, None), creator=0)

Return a dictionary of a list for the filenames of the RegriddedFrames for a CoaddedRegriddedFrame matching the specified criteria.

astro.util.MakePointings.query_reduced(instrument='', object='', filter='', naxis=(0, 0), crval1=(0, 0), crval2=(0, 0), check_validity=True, check_quality=True, project='', privileges=0, dates=(None, None))

Return a dictionary of linked lists for some of the persistent properties of the ReducedScienceFrames matching the specified criteria.

astro.util.MakePointings.query_regridded(instrument='', object='', filter='', naxis=(0, 0), crval1=(0, 0), crval2=(0, 0), check_validity=True, check_quality=True, project='', privileges=0, dates=(None, None), creator=0)

Return a dictionary of linked lists for some of the persistent properties of the RegriddedFrames matching the specified criteria.

astro.util.NameResolver module

Resolve names to coordinates using Simbad web service

Usage :

from astro.util.NameResolver import resolve result = resolve(‘M51’)

result is a dictionary with all info from Simbad ra and dec should always be present

Throws exception when Simbad does not return data

ra and dec will be None when the object is unknown

exception astro.util.NameResolver.SimbadError

Bases: Exception

class astro.util.NameResolver.SimbadSoapXmlHandler

Bases: xml.sax.handler.ContentHandler

class for parsing (Soap) xml from simbad

characters(content)
class astro.util.NameResolver.SimbadXmlHandler(result)

Bases: xml.sax.handler.ContentHandler

class for parsing xml from simbad

characters(content)

a set of character data in the xml

endElement(name)

end of element in xml

startElement(name, attrs)

element is started in xml

astro.util.NameResolver.resolve(search_term, force_encoding='')

resolve the search_term to coordinates (optional) use force_encoding to force the use of an encoding (example UTF-8)

astro.util.OldDataFinder module

Find old data at a specified privilege level.

class astro.util.OldDataFinder.OldDataFinder(privileges=1, project='ALL')

Bases: object

Find instances of ReducedScienceFrame, AstrometricParameters, RegriddedFrames, and CoaddedRegriddedFrames that reside at a specific prvilege level. List the type of object and give a summary by instrument of how many exist and how much disk space the data files take up in terabytes (1TB = 2**40 bytes)

A recursive delete tool is also available. It takes only objects, so they must be instantiated first from the object_id.

Four data dictionaries are created: reds, aps, regs, coadds. They all have the format:

{instruent.name : [(object_id, filename, NAXIS1, NAXIS2), …], …}

except for aps. It has the format:

{instruent.name : [(object_id, residuals), …], …}
find()
find_objects()
reset_project()
set_project()
astro.util.OldDataFinder.delete_obj(obj, verbose=True, commit=False, delete_config=False, recursive=True)

astro.util.OverlapImage module


Overlapping images can be used to check anomolies and similarities between any two images.


class astro.util.OverlapImage.OverlapImage(filename1, filename2=None, xshift=0, yshift=0, xstride=1, ystride=1, rotate=0, viewer='skycat')

Bases: object

Compare two images by subtracting them, including an optional offset and/or sub-sample in X and/or Y of one of the images, an optional rotation of one of the images, then display the results as a FITS image.

check_preconditions()
display_overlap()

Write the result image array to a FITS file and display with a viewer then delete the image.

load_arrays()

Use PyFits to load arrays from FITS images.

make()
overlap_images()

Pad the smaller of the two image arrays, offset it, then subtract. The second array is also optionally rotated and sub-sampled (i.e., with strides).

exception astro.util.OverlapImage.OverlapImageError

Bases: Exception

astro.util.Pars module

class astro.util.Pars.Pars(identifier, instrument='', filter='', chip='', project='')

Bases: object

get()

Returns a dictionary containing configuration parameters which can be specified when running tasks locally or on the distributed processing unit.

show()

Show the parameters that can be configured and their current values

astro.util.Pars.main()

astro.util.PlateSystem module

PlateSystem — 7 June 2006 KIDS with eye on second decimal in field —- plots produce contours of fieldsize_ra and fieldsize_dec — 1 June 2006 KIDS adaptations — for KIDS different overlap in RA and DEC 1 june 2006 — SMOOTHING DISABLED- not needed FOR SUCH SMALL OVERLAP — overlap defined at BOTTOM of fields -> more fields/ more overlap Define a platesystem, a collection fields (“plates”) of a constant size covering the entire sky. Edwin Valentijn - —- 7 March 2004 —- version with smooth grid - more advanced than PAE

class astro.util.PlateSystem.PlateSystem(fieldsizera=1.0127, fieldsizedec=1.02451, overlapra=1.186, overlapdec=4, efflimit=0.96)

Bases: object

Define a plate system

The plate system is defined by a set of fields covering the sky. Each field has a size in ra and dec, a centre and a requested amount of overlap with neighbouring fields in both ra and dec and a minimun allowed overlap in ra. The fields are arranged in strips perpendicular to the meredian. This means that with increasing declination there are less fields per strip. Script optimizes for continuity form one dec strip to the next; therfor the positions in various dec stips are dependent on each other.

get_field_centre(rain, decin)

get nearest ra fieldcenters, needs dec too

get_numberra_centers(log)

log - print log to screen

get_ra(decin)

get seq of ra’s for a single decin

nearest_dec(decin)

get nearest fieldcenter dec for any dec

plot_it(rastart=-1.0, raend=360.0, decstart=0.0, decend=90.0, title=' ')

plots the grid in range can be used as survey preparation tool selects fields within rectangle on sky returns ra,dec of selected fields in self autoscaling implemented

print_grid()

print grid properties in file

astro.util.PlotTiles module

Plot on the sky in schematic form the completeness of processed, valid data tiles. Plot one or a group of markers (here, N squares, one for each dither position, of the size of the OMEGACAM mosaic as projected in RA/Dec space) for a given ObservingBlock. Show only the level to which processing has created a valid, good quality full set of frames. For the case of CoaddedRegriddedFrames, the group of markers is retained for simplicity and for informative purposes (e.g., to show how many dither points went into the coadd), but can be replaced with an appropriately sized single marker. For clarity, one plot is made per filter.

Additions:
  • Within each set of markers can be set various flags (e.g., a green ‘X’ indicating is_valid=2, a red ‘X’ indicating the data is unqualified, etc.).
  • Somehow add links to DBView, CalTS, Process, and Quality to each set of markers (Quality currently has no front-end for ObservingBlocks or RawFitsData, use DBView for linking).
  • Add overview plot. This may take the form of one colored square per valid coadd per filter, all on one plot.
  • Plot KIDS-N patches.
class astro.util.PlotTiles.KIDSTiles(tiles=['KIDS*'], ext='', qualified=True, all_levels=False, coadd_only=False, project='KIDS', privileges=1, verbose=False)

Bases: object

Plot KIDS tiles with completion of processing level.

check_preconditions()

Check that the project and privileges are allowed.

compile_processed()

Compile and reorder the data acquired from the queries.

make()

Run all queries, compile and plot results.

plot_tiles()

Abstract base class method

query_database()

Run SQL queries to acquire the data to be plotted.

class astro.util.PlotTiles.KIDS_N(tiles=['KIDS*'], ext='', qualified=True, all_levels=False, coadd_only=False, project='KIDS', privileges=1, verbose=False)

Bases: astro.util.PlotTiles.KIDSTiles

Plot tiles in the KIDS-N patch with completion of processing level.

> KIDS_N().make()
plot_tiles()

Plot the compiled data, one set of squares per OB, one plot per filter.

ToDo: Add combined filter plot.

class astro.util.PlotTiles.KIDS_S(tiles=['KIDS*'], ext='', qualified=True, all_levels=False, coadd_only=False, project='KIDS', privileges=1, verbose=False)

Bases: astro.util.PlotTiles.KIDSTiles

Plot tiles in the KIDS-S patch with completion of processing level.

> KIDS_S().make()
plot_tiles()

Plot the compiled data, one set of squares per OB, one plot per filter.

ToDo: Add combined filter plot.

astro.util.Polygon module

class astro.util.Polygon.Point(x, y)

Bases: object

inside(polygon)

# http://www.ariel.com.au/a/python-point-int-poly.html

# determine if a point is inside a given polygon or not # Polygon is a list of (x,y) pairs.

class astro.util.Polygon.Polygon(vertices=None, edges=None)

Bases: object

check_self_intersection()
check_unique_vertices()
edges
find_union_edges(other)

Finds the edges that form the union between self and other.

First loops over all edges of ‘self’, then over all edges of ‘other’ and adds those parts that are outside the other polygon. Both polygons have to be oriented in the same way.

orientation()

http://stackoverflow.com/questions/1165647/how-to-determine-if-a-list-of-polygon-points-are-in-clockwise-order

overlapping_bounding_boxes(other)
reorient()
shuffle(r=0.1)
union(other)
vertices
volume_bounding_box()
xmax
xmean
xmin
xs()
ymax
ymean
ymin
ys()
class astro.util.Polygon.Segment(start, end)

Bases: object

extend_end(factor=1.0)
extend_start(factor=1.0)
intersection(other)

# http://stackoverflow.com/questions/4543506/algorithm-for-intersection-of-2-lines

intersects(other)

#http://stackoverflow.com/questions/3838329/how-can-i-check-if-two-segments-intersect

length()
xs()
ys()
astro.util.Polygon.lazyproperty(fn)

astro.util.RatioImage module


Inspect the ratio of two full mosaics


class astro.util.RatioImage.RatioImage(numerator, denominator)

Bases: object

Make the ratio of two RawFitsData

check_preconditions()
make()
make_ratio_image()
make_rawframes()

astro.util.SetRelations module

A SetRelations object describes a specific logical relation between d sets as a Boolean matrix of size 2^d.

class astro.util.SetRelations.SetRelations(relation='t', labels=None)

Bases: object

A SetRelations object describes a specific logical relation between d sets as a Boolean matrix of size 2^d.

Every relation has a unique number that is created by flattening the matrix and interpreting it as a binary number after prefixing a 1 to it.

The ordering of the dimensions of the matrix represents the ordering of the sets. Dimensions are labeled with uppercase letters alphabetically.

add(label=None, equalto=None)

Add a new set.

label : str
Label of the new set.
equalto : str
Optional, label of an existing set that is equal to the new set.
add_alias(label, label1)

Add an alias to one of the sets.

label : str
Original label.
label1 : str
New label.
add_crosssection(label, label1, label2)

Add a set that is a cross section of two other sets.

label : str
Label of the new set.
label1 : str
Label of the first set of the cross section.
label2 : str
Label of the second set of the cross section.
add_disjoint(label)

Add a new set that is disjoint with all existing sets.

label : str
Label of the new set.
add_empty(label)

Add an empty set.

label : str
Label of the new set.
add_equal(label=None, equalto=None)

Add a set that is equal to another set.

This function should only be called internally. Use the add_element function with an equalto argument to add an element that is equal to another element manually.

label : str
Label of the set.
equalto : str
Label of the set that the new set is equal to.
add_union(label, label1, label2)

Add a set that is a union of two other sets.

label : str
Label of the new set.
label1 : str
Label of the first set of the union.
label2 : str
Label of the second set of the union.
all_labels()

Get labels of all sets.

list of str
Labels of all sets.
static bin_from_num(n)

Convert denary integer n to binary list.

n : int

list of int
Binary digits of n.
combine(other)

Combine this SetRelations with another.

other : SetRelations
The other SetRelations to combine.
set of SetRelations
SetRelations that are consistent with self and other.
combine_raw(other)

Combines two relations in most simplistic form.

All sets from the other relation are added one by one and the corresponding SetRelations are filtered.

other : SetRelations
The other SetRelations to combine.
set of SetRelations
SetRelations that are consistent with self and other.
copy()

Creates a new copy of this relation.

SetRelations
A copy of self.
dimension_from_label(label)

Get the dimension that corresponds to the given label.

label : str
Set label.
int
Dimension corresponding to the label.
dimension_matrix()

Get the dimension of the matrix.

int
Dimension of the matrix.
find_equals(label)

Find sets that are equal to the set with the given label.

label : str
Label of the set to find equals from
list of str
List of labels of sets that are equal to set with given label.
flatten()

Split all dimensions that correspond to multiple sets.

from_number(number)

Create the matrix from a number.

number : int
The number corresponding to the matrix.
from_symbol(symbol)

Create the matrix from a symbol.

symbol : str
The symbol.
isequal(label1, label2)

Determine whether two sets are equal.

Much faster than using relation().

label1 : str or int
Label of the first set to check for equality.
label2 : str or int
Label of the second set to check for equality.
msum(dim)

Sums the matrix to remove a dimension and the corresponding sets.

This is a separate function for profiling reasons because it is slow. Use remove() to remove a set/dimension properly.

dim : int
The dimension to sum over.
numpy.ndarray
The summed array.
new_label()

Returns a new label that is not yet used.

It is preferred to set labels explicitly.

str
A new, unused label.
number()

Get the number that corresponds to this relation.

This is only sensible if the order of the sets is known.

int
The number corresponding to this relation.
number_from_symbol(symbol)

Get the number from a symbol.

symbol : str
The symbol.
int
The number corresponding to the symbol.
order_labels()

Sort the labels of the sets.

The order of the labels is arbitrary. Relations can only be compared if the order of their relations is the same.

relabel(old, new)

Relabel a set.

old : str
Old label of the set.
new : str
New label of the set.
relation(relation)

Determines the relation between a group of sets.

relation : list of str
Labels of sets to find the relation between. TODO: Rename
SetRelations
The relation between the labels.
remove(label=None)

Remove a set identified by label or dimension.

All sets matching the dimension are removed when a dimension is given.

label : str or int
Label of the set or dimension.
split(label)

Split the dimension corresponding to label if it is shared with other sets.

label : str
The label of the set.
symbol()

Get the number that corresponds to this relation.

str
Symbol corresponding to this relation.
class astro.util.SetRelations.SetRelationsSet(infinite_points=False)

Bases: object

A list of SetRelations that meet the proposed criteria.

add(label=None, equalto=None)

Add a new set.

label : str
Label of the new set.
equalto : str
Optional, label of an existing set that is equal to the new set.
add_alias(label, label1)

Add an alias to one of the sets.

label : str
Original label.
label1 : str
New label.
add_crosssection(label, label1, label2)

Add a set that is a cross section of two other sets.

label : str
Label of the new set.
label1 : str
Label of the first set of the cross section.
label2 : str
Label of the second set of the cross section.
add_disjoint(label)

Add a new set that is disjoint with all existing sets.

label : str
Label of the new set.
add_empty(label)

Add an empty set.

label : str
Label of the new set.
add_equal(label=None, equalto=None)

Add a set that is equal to another set.

label : str
Label of the set.
equalto : str
Label of the set that the new set is equal to.
add_inbetween(label, label1, label2)

Add a set that is larger or equal than label1 but smaller or equal then label2.

label : str
Label of the new set.
label1 : str
Label of the set that is a subset of the new set.
label2 : str
Label of the set that is a superset of the new set.
add_subset(label, label1)

Add a new set that is a subset of an existing set.

label : str
Label of the new set.
label1 : str
Label of the set that is a superset of the new set.
add_super(label, label1)

Add a superset of an existing set.

label : str
Label of the new set.
label1 : str
Label of the existing set that is a subset of the new set.
add_union(label, label1, label2)

Add a set that is a union of two other sets.

label : str
Label of the new set.
label1 : str
Label of the first set of the union.
label2 : str
Label of the second set of the union.
all_labels()

Get labels of all sets.

list of str
Labels of all sets.
assure_same_labels()

Assure that all subrelations have the same labels.

bool
Whether all subrelations have the same labels.
combine(other)

Ensure this SetRelationsSet is consistent with another one.

Creates .relations that is consistent with both self and other.

other : SetRelationsSet
The other SetRelationsSet to be compatible with.
combine_equal()

Find and combine two sets that are equal in all subrelations.

bool
Whether two previously uncombined equal sets where found.
combine_equals()

Combine all sets that are equal.

copy()

Make a copy of this SetRelationsSet.

SetRelationsSet
Copy of this SetRelationsSet.
debug = 0
ensure_points_left()

Remove all relations that do not have points left.

equal_identifier(label, set_labels=None, equal_elements=None)

Given an identifier to the label such that every label with the same identifier represents the same set.

filter(sets, allowed)

Filter all set relations that are not allowed.

sets : list of str or list of int
List of labels of sets to filter on.
allowed : list
List of allowed symbols. [TODO: also numbers]

TODO MT M: [SETRELATIONS] Implement more ways of defining the allowed relations.

filterequal(label1, label2)

Ensure that two sets are equal.

label1 : str
Label of the first set.
label2 : str
Label of the second set.
find_equals(label1)

Find sets that are equal to the set with the given label.

label1 : str
Label of the set to find equals from. TODO: Rename
list of str
List of labels of sets that are equal to set with given label.
find_supers(label1, strict=False)

Find sets that are equal to a given set or a superset.

label1 : str
Label of the set to find supersets of.
strict : bool
Whether the found sets should be strict supersets (not equal).
list of str
Labels of sets that are equal to the given set or a superset.
get_equal_sets()

Find equal sets.

list of list of str
List of lists of labels of sets that are equal.
isequal(label1, label2)

Determine whether two sets are equal.

label1 : str or int
Label of the first set to check for equality.
label2 : str or int
Label of the second set to check for equality.
relabel(old, new)

Relabel a set.

old : str
Old label of the set.
new : str
New label of the set.
relation(labels=None)

Determines the relation between a group of sets.

labels : list of str
Labels of sets to find the relation between.
set of SetRelations
The possible relations between the labels.
relation_holds(labels=None, relations=None)

Test whether a relation holds for the elements.

labels : list of str
Labels of sets to find the relation between.
relations : list of SetRelations.
Relation between sets indicated by labels.
bool
Whether the relation holds.
remove(label)

Remove a set.

label : str
Label of the set.
split(name)

Split the dimension corresponding to label if it is shared with other sets.

name : str
The label of the set.
class astro.util.SetRelations.TestSetRelations(methodName='runTest')

Bases: unittest.case.TestCase

Test SetRelations.

static test_Ccontiguous()

Do not give this error in Python 3: ValueError: ndarray is not C-contiguous

static test_adding_disjoint()
static test_adding_empty()
static test_adding_inbetween()
static test_combine()
static test_crosssection()
static test_crosssection_new()
static test_crosssection_triple()
static test_numberlabels()
static test_order()
static test_sdss_relations()
static test_transitivity()
static test_transitivity2()
static test_union_new()

astro.util.StellarLocusRegression module

Stellar Locus Regression analysis to perform photometric homogenization. Authors: Ewout Helmich (helmich@astro.rug.nl) for implementation and Gijs Verdoes Kleijn (verdoes@astro.rug.nl) for algorithm. Usage: the starting point must be GaaP photometry stored as a ConcatenateAttributes. Further instruction remain to be written…

class astro.util.StellarLocusRegression.StellarLocusRegression(concatenate_attributes)

Bases: object

calculate_offsets()

Calculate photometric offets given the 4-band photometry stored in self.data.

check_data()
get_data()

Distill data to go from stellar locus to photometric offsets (i.e., run calculate_offsets). Input can be self.concatenate_attributes, a SourceCollection which contains the GaaP photometry. Input can be self.fitscatalog_name, the filename of a fits file, which contains the GaaP photometry. Output is stored in self.data

XXX TODO Move this outside this class

get_statistics_principal_colors(p2, locus_stars)

Compute statistics on stellar locus via principal colors.

Returns sampled median, error on sampled median and sampled standard deviation.

inspect()

Plot principal colors of stars, given self.tilename and self.principal_colors.

FIXME: add plot FWHM_IMAGE_r vs MAG_r (color indicating selected stars). FIXME: make this interactive with topcat & samp.

select_locus_stars(p1, p2, p1limits, p2limits)

Identify the stars on stellar locus in a list of principal colors values. These can be used to compute principal color stats.

astro.util.StellarLocusRegression.main()

astro.util.TableConverter module

TableConverter class.

class astro.util.TableConverter.TableConverter

Bases: object

TableConverter holds tabular data, which can be stored in various forms: - a numpy dictionary (internal format) - votable (ascii, binary or inline-fits) - fits (binary) - SourceList

It is mainly used by SourceCollections and SAMP.

add_attribute(attr, format, null=None, ucd=None, length=1, data=None)

Add an attribute to the TableConverter. ‘format’ should be a format that exists in .type_defaults.

append_rows(tc, append_zeros=True)

Appends the rows of tc to this TableConverter.

apply_mask(mask)

Applies mask to all the attributes in data.

array_from_string_list(data, format=None)

TODO: Describe this function.

attribute_order = None

A list with the order of the attributes.

attributes = None

A dictionary of the attributes of the data, contains information about the format of the data etc.

attributes_from_aw_types(aw_types)

Uses result from SourceList.sources.get_attributes() to create the attributes.

check()

Checks self consistency of the data.

compare(tc, printok=False, attributes=None, nulls=True, printbad=True, ucds=True, check_attribute_order=True)

Compares this TableConverter with another one. If ‘attributes’ is specified, only those attributes are checked. If nulls is False, null values are not checked.

copy(attributes=None)

Create a copy of this TableConverter. If ‘attributes’ is a list of attributes, then only these attributes are copied.

copy_attribute(old, new)

Copy an attribute.

create_from_datadict(data)

Create a Tableconverter from dictionary of numpy arrays.

create_mask_from_query(query, debug=0)

Creates a mask from a query using a parser. ‘query’ should be an SQL WHERE clause using:

AND, OR, BETWEEN, +-*/, ()

with all attributes in double quotes. Examples: ‘ “DEC” < 10 + 1 AND “RA” BETWEEN 93.0*2 AND 187 ‘ ‘ “MAG_ISO” < 17 ‘

data = None

The data, a dictionary of numpy arrays. The keys are the attribute names.

find_tablename_columns()

Searches for a SOURCELIST*SOURCES(**%02i) that can be used to store the data of this TableConverter in.

Returns the tablename and the column names that should be used in order of self.attribute_names. The USER_%i_%s columns are used in case no SOURCELIST*SOURCES with all the necesseray columns exist.

fits_columns()

fits_columns makes columns for in a fits file

fix_attributes()

If self.attributes is incomplete, it is fixed here.

format_from_string_list(data, threshold=1.0, nulls=None)

Returns what is most likely the format of data

get_row(*rowids)

Returns a dictionary with as keys attribute names and as values the values of the attributes of the source that correspond to the rowids. The rowids should correspond to the .row_identifiers if set, otherwise it should be a zero-based index.

get_row_id(*rowids)

Convert values of the .row_identifiers to a row number.

get_votable_cell_size(votable_, field)

returs the integer arraysize of field in votable if it is not given, 1 is assumed if the cell is more than 1D (there is an ‘x’ in arrysize), it is not supported if the arraysize is flexible (‘*’), than we determine the actual maximum size a flexible array with no content (size=0) will be set to size=1

get_votable_footers()

Create VOTable footers.

get_votable_headers(tablename='', resourcename='')

Get the headers of a VOTable.

get_votable_row(row)

Create a VOTable row.

join_column(tc, key)

Add every column from tc2 to this tc, matching on key.

TODO MT L [TABLECONVERTER]: Use self.row_identifiers as default key.

len()

Get the length of the table.

load_associatelist_raw(associatelist)

Retrieves the data of an AssociateList.

load_associatelist_worg(associatelist)

Retrieves the data of an AssociateList including the SLIDorg/SIDorg columns.

load_aw_query(query_string, aw_types=None)

Load data from an AW SELECT query.

load_aw_query_many(query_string)

Experimental code to load a query in parts. Used for Samp.broadcastSourceCollection_in_parts.

load_catalog(catalog)

Loads data from a PhotSrcCatalog or a PhotRefCatalog.

load_csv(filename, names=None, formats=None, striprows=True, stripcells=True, commentchars='#%;', skiplines=0)

Load a CSV file.

load_delimited_text(filename, delimiter=None, names=None, formats=None, striprows=True, stripcells=True, commentchars='#%;', skiplines=0)

Reads a file with ascii values. Every cell is delimited by ‘delimiter’, e.g. ‘,’, if not specified, any whitespace. striprows denote whether the rows have to be stripped before parsed (default) stripcells denote whether the cells have to be stripped before parsed (default) commentchars denote characters that skip a line if it starts with this characters skiplines are the number of lines that are skipped before parsing names is a list of column names, if None it tries to parse the names from the first line if that does not start

with a number

formats is a list of formats that the cells ought to have, if none it is determined automatically

load_fits(filename, extnum=1)

Load a fits file.

load_pickle(filename)

Load table from pickle file.

load_sourcelist(sl, attr_list=None, query_string='', area=None, nulls=None)

Load data fa SourceList.

load_votable(location)

Loads an VOTable either from a file or from an url.

load_votable_file(filename)

Load a VOTable.

load_votable_string(votable_string)

Load a VOTable from a string.

load_votable_url(url)

Load a VOTable from an url.

nancompare(d1, d2)

compares wether numpy arrays d1 and d2 are the same, with regards for nans

number_of_rows()

Returns the number of sources (rows) in this TableConverter.

number_of_sources()

Alias to get number_of_rows().

remove_attribute(attr)

Remove an attribute from the TableConverter.

rename_attribute(old, new)

Rename an attribute.

row_identifiers = None

A list of attributes whose combination that uniquely identifies a row.

TODO MT M [TABLECONVERTER]: Use this property within this class as
well. Currently it is used in the SourceCollection classes for catalogs that are not yet stored in the database and thus do not yet have a SLID and/or SID. E.g. in join_column().
save_csv(filename, header=None, clobber=True)

Writes the data as a comma-separated-value file header is printed before the data, if no header is specified it defaults to # followed by the column names

save_fits(filename, clobber=True)

Save a fits file.

save_fits_sourcelist(filename, clobber=True)

Save a SourceList.

save_pickle(filename)

Save table as pickle file.

save_sourcelist(filename='temp.fits', commit=False, tablename_store=None, name=None, debug=0, boundingbox=None)

Save the table as a SourceList.

save_text(filename, format, header='', clobber=True)

Write the data as a txt file.

format is a formatting string for each individual row
e.g. “%f,%f,%f
header is printed before the data
save_votable(filename, tablename='', resourcename='')

Save table as a VOTable.

save_votable_fits(filename, directory=None, clobber=True)

Save table as a FITS VOTable.

save_votable_xml(filename, tablename='', resourcename='')

Save table as XML VOTable.

set_number_of_rows(nos)

Sets the number of rows, only applicable when there are no attributes.

set_number_of_sources(nos)

Alias for set_number_of_rows()

sort(attributes)

Sort the data on one or more attributes.

TODO MT L [TABLECONVERTER]: Use self.row_identifiers as default.

table_parameters = None

A dictionary with some parameters of the table.

typ = 'int32'
type_defaults = {'awe_bool': {'null': nan, 'fits': 'L', 'oracle': 'SMALLINT', 'votable': 'boolean', 'numpy': 'bool', 'awe': 'bool'}, 'awe_float': {'null': nan, 'fits': 'D', 'oracle': 'BINARY_DOUBLE', 'votable': 'double', 'numpy': 'float64', 'awe': 'float'}, 'awe_int': {'null': -2147483648, 'fits': 'J', 'oracle': 'SMALLINT', 'votable': 'int', 'numpy': 'int32', 'awe': 'int'}, 'awe_long': {'null': -9223372036854775808, 'fits': 'K', 'oracle': 'NUMBER', 'votable': 'long', 'numpy': 'int64', 'awe': 'long'}, 'awe_single_float': {'null': nan, 'fits': 'E', 'oracle': 'BINARY_FLOAT', 'votable': 'float', 'numpy': 'float32', 'awe': 'single_float'}, 'awe_str': {'null': '', 'fits': 'A', 'oracle': 'VARCHAR', 'votable': 'char', 'numpy': 'str', 'awe': 'str'}, 'bool': {'null': nan, 'fits': 'L', 'oracle': 'SMALLINT', 'votable': 'boolean', 'numpy': 'bool', 'awe': 'bool'}, 'fits_A': {'null': '', 'fits': 'A', 'oracle': 'VARCHAR', 'votable': 'char', 'numpy': 'str', 'awe': 'str'}, 'fits_B': {'null': 255, 'fits': 'B', 'oracle': 'SMALLINT', 'votable': 'unsignedByte', 'numpy': 'uint8', 'awe': 'int'}, 'fits_D': {'null': nan, 'fits': 'D', 'oracle': 'BINARY_DOUBLE', 'votable': 'double', 'numpy': 'float64', 'awe': 'float'}, 'fits_E': {'null': nan, 'fits': 'E', 'oracle': 'BINARY_FLOAT', 'votable': 'float', 'numpy': 'float32', 'awe': 'single_float'}, 'fits_I': {'null': -32768, 'fits': 'I', 'oracle': 'SMALLINT', 'votable': 'short', 'numpy': 'int16', 'awe': 'int'}, 'fits_J': {'null': -2147483648, 'fits': 'J', 'oracle': 'SMALLINT', 'votable': 'int', 'numpy': 'int32', 'awe': 'int'}, 'fits_K': {'null': -9223372036854775808, 'fits': 'K', 'oracle': 'NUMBER', 'votable': 'long', 'numpy': 'int64', 'awe': 'long'}, 'fits_L': {'null': nan, 'fits': 'L', 'oracle': 'SMALLINT', 'votable': 'boolean', 'numpy': 'bool', 'awe': 'bool'}, 'float32': {'null': nan, 'fits': 'E', 'oracle': 'BINARY_FLOAT', 'votable': 'float', 'numpy': 'float32', 'awe': 'single_float'}, 'float64': {'null': nan, 'fits': 'D', 'oracle': 'BINARY_DOUBLE', 'votable': 'double', 'numpy': 'float64', 'awe': 'float'}, 'int16': {'null': -32768, 'fits': 'I', 'oracle': 'SMALLINT', 'votable': 'short', 'numpy': 'int16', 'awe': 'int'}, 'int32': {'null': -2147483648, 'fits': 'J', 'oracle': 'SMALLINT', 'votable': 'int', 'numpy': 'int32', 'awe': 'int'}, 'int64': {'null': -9223372036854775808, 'fits': 'K', 'oracle': 'NUMBER', 'votable': 'long', 'numpy': 'int64', 'awe': 'long'}, 'numpy_bool': {'null': nan, 'fits': 'L', 'oracle': 'SMALLINT', 'votable': 'boolean', 'numpy': 'bool', 'awe': 'bool'}, 'numpy_float32': {'null': nan, 'fits': 'E', 'oracle': 'BINARY_FLOAT', 'votable': 'float', 'numpy': 'float32', 'awe': 'single_float'}, 'numpy_float64': {'null': nan, 'fits': 'D', 'oracle': 'BINARY_DOUBLE', 'votable': 'double', 'numpy': 'float64', 'awe': 'float'}, 'numpy_int16': {'null': -32768, 'fits': 'I', 'oracle': 'SMALLINT', 'votable': 'short', 'numpy': 'int16', 'awe': 'int'}, 'numpy_int32': {'null': -2147483648, 'fits': 'J', 'oracle': 'SMALLINT', 'votable': 'int', 'numpy': 'int32', 'awe': 'int'}, 'numpy_int64': {'null': -9223372036854775808, 'fits': 'K', 'oracle': 'NUMBER', 'votable': 'long', 'numpy': 'int64', 'awe': 'long'}, 'numpy_str': {'null': '', 'fits': 'A', 'oracle': 'VARCHAR', 'votable': 'char', 'numpy': 'str', 'awe': 'str'}, 'numpy_uint8': {'null': 255, 'fits': 'B', 'oracle': 'SMALLINT', 'votable': 'unsignedByte', 'numpy': 'uint8', 'awe': 'int'}, 'oracle_BINARY_DOUBLE': {'null': nan, 'fits': 'D', 'oracle': 'BINARY_DOUBLE', 'votable': 'double', 'numpy': 'float64', 'awe': 'float'}, 'oracle_BINARY_SINGLE': {'null': nan, 'fits': 'E', 'oracle': 'BINARY_FLOAT', 'votable': 'float', 'numpy': 'float32', 'awe': 'single_float'}, 'oracle_NUMBER': {'null': -9223372036854775808, 'fits': 'K', 'oracle': 'NUMBER', 'votable': 'long', 'numpy': 'int64', 'awe': 'long'}, 'oracle_SMALLINT': {'null': -2147483648, 'fits': 'J', 'oracle': 'SMALLINT', 'votable': 'int', 'numpy': 'int32', 'awe': 'int'}, 'str': {'null': '', 'fits': 'A', 'oracle': 'VARCHAR', 'votable': 'char', 'numpy': 'str', 'awe': 'str'}, 'uint8': {'null': 255, 'fits': 'B', 'oracle': 'SMALLINT', 'votable': 'unsignedByte', 'numpy': 'uint8', 'awe': 'int'}, 'votable_boolean': {'null': nan, 'fits': 'L', 'oracle': 'SMALLINT', 'votable': 'boolean', 'numpy': 'bool', 'awe': 'bool'}, 'votable_char': {'null': '', 'fits': 'A', 'oracle': 'VARCHAR', 'votable': 'char', 'numpy': 'str', 'awe': 'str'}, 'votable_double': {'null': nan, 'fits': 'D', 'oracle': 'BINARY_DOUBLE', 'votable': 'double', 'numpy': 'float64', 'awe': 'float'}, 'votable_float': {'null': nan, 'fits': 'E', 'oracle': 'BINARY_FLOAT', 'votable': 'float', 'numpy': 'float32', 'awe': 'single_float'}, 'votable_int': {'null': -2147483648, 'fits': 'J', 'oracle': 'SMALLINT', 'votable': 'int', 'numpy': 'int32', 'awe': 'int'}, 'votable_long': {'null': -9223372036854775808, 'fits': 'K', 'oracle': 'NUMBER', 'votable': 'long', 'numpy': 'int64', 'awe': 'long'}, 'votable_short': {'null': -32768, 'fits': 'I', 'oracle': 'SMALLINT', 'votable': 'short', 'numpy': 'int16', 'awe': 'int'}, 'votable_unsignedByte': {'null': 255, 'fits': 'B', 'oracle': 'SMALLINT', 'votable': 'unsignedByte', 'numpy': 'uint8', 'awe': 'int'}}

astro.util.UCD module

UCD.py map AWE-table-attributes to VO Unified Code Descriptors (UCD)

class astro.util.UCD.UcdXmlHandler

Bases: xml.sax.handler.ContentHandler

class to handle xml returned from web services (http://cdsweb.u-strasbg.fr/cdsws/ucdClient.gml)

characters(content)
endDocument()
endElement(name)
startDocument()
startElement(name, attrs)
astro.util.UCD.do_service(service_path)

call web service and extract result

class astro.util.UCD.ucd

Bases: dict

the ucd class is a dictionary setting the relation between column names in AW and ucds

assign(descr)
assign_path = '/axis/services/UCD?method=assign&ucd=%s'
base_url = 'cdsws.u-strasbg.fr'
explain(ucd)
explain_path = '/axis/services/UCD?method=explain&ucd=%s'
find(ucd)

find column names with same ucd

find_types(ucd)

find all types which have an attribute of ucd type

translate(ucd)
translate_path = '/axis/services/UCD?method=translate&ucd=%s'
validate(ucd)
validate_path = '/axis/services/UCD?method=validate&ucd=%s'

astro.util.Unit module

UNIT.py map AWE-table-attributes to units

********* OBSOLETE ********* Use the [] in the doc string of persistent attributes to define units ! ********* OBSOLETE ********* This class will be removed !!!! ********* OBSOLETE *********

astro.util.Unit.Unit = {}

see sites :

http://vizier.u-strasbg.fr/viz-bin/Unit?%3f

http://vizier.u-strasbg.fr/doc/catstd-3.2.htx

class astro.util.Unit.unit

Bases: dict

the unit class is a dictionary defining the unit of columns in AW

astro.util.Unit.units = {}

‘RA’ : ‘deg’, ‘DEC’ : ‘deg’, ‘A’ : ‘pixel’, ‘B’ : ‘pixel’, ‘BackGr’ : ‘count’, ‘Corr’ : ‘pixel**2’, ‘DEC’ : ‘deg’, ‘ERRA_IMAGE’ : ‘pixel’, ‘ERRB_IMAGE’ : ‘pixel’, ‘ERRTHETA_IMAGE’ : ‘deg’, ‘ERRX2_IMAGE’ : ‘pixel**2’, ‘ERRXY_IMAGE’ : ‘pixel**2’, ‘ERRY2_IMAGE’ : ‘pixel**2’, ‘FLUXERR_APER’ : ‘count’, ‘FLUXERR_AUTO’ : ‘count’, ‘FLUXERR_ISO’ : ‘count’, ‘FLUXERR_ISOCOR’ : ‘count’, ‘FLUX_APER’ : ‘count’, ‘FLUX_AUTO’ : ‘count’, ‘FLUX_ISO’ : ‘count’, ‘FLUX_ISOCOR’ : ‘count’, ‘FLUX_RADIUS’ : ‘pixel’, ‘FWHM_IMAGE’ : ‘pixel’, ‘Level’ : ‘count’, ‘MAGERR_APER’ : ‘mag’, ‘MAGERR_ISO’ : ‘mag’, ‘MAGERR_ISOCOR’ : ‘mag’, ‘MAG_APER’ : ‘mag’, ‘MAG_AUTO’ : ‘mag’, ‘MAG_ISO’ : ‘mag’, ‘MAG_ISOCOR’ : ‘mag’, ‘MU_MAX’ : ‘mag * arcsec**(-2)’, ‘MU_THRESHOLD’ : ‘mag * arcsec**(-2)’, ‘NPIX’ : ‘pixel**2’, ‘POSANG’ : ‘deg’, ‘RA’ : ‘deg’, ‘XM2’ : ‘pixel**2’, ‘Xpos’ : ‘pixel’, ‘YM2’ : ‘pixel**2’, ‘Ypos’ : ‘pixel’, ‘CRVAL1’ : ‘deg’, ‘CRVAL2’ : ‘deg’

astro.util.XSD module

Functionality to transform xsds

This module contains function to help transform xsds and can be used as a script to generate an xml template from a set of xsds for a given xsd type.

class astro.util.XSD.XSDType(xsd, item)

Bases: object

Map xml xsd type to a Python class

fmt()
fulltype()
isatomic()
isvalid()
astro.util.XSD.capitalize(s)
astro.util.XSD.expand_complextype(xsd, item, outelem, prefix, XSD)

Expand a complexType

astro.util.XSD.expand_sequence(xsd, item, outelem, prefix)

Expand the xs:sequence part of an xml element

astro.util.XSD.prettyxml(elem)
astro.util.XSD.read_all_xsds(dictionarydir=None)

Parse all xsd files inside a directory tree

astro.util.XSD.with_appinfo(elem, name, attr)

These are exceptions if the appinfo tag is present.

astro.util.XSD.without_appinfo(elem, name)

These are exceptions if the appinfo tag is missing.

astro.util.XsdMapping module

Map Python objects via XML schemas (XSDs) to XML file(s)

AW Python classes are translated into XML using an XML schema (stored in an XSD file). Objects in the form of XML files can be converted back to non-AW Python classes. It is possible to support (non-)AW Python to XML and XML to (non-)AW Python, but not all combinations may be necessary.

astro.util.XsdMapping.EucStrToDatetime(input)

Convert datetime strings in Euclid format to datetime objects, passing through datetime objects

input: Euclid time string or datetime object
class astro.util.XsdMapping.Node(typeName, properties)

Bases: object

A (parsed) XML node

Each node has a TypeName (e.g., schema, annotation, etc.) and Properties (e.g., name, id, etc.) stored in a dictionary, character data (CData) and subnodes (Children) with the same structure (recursive instances of Node objects)

Based on: http://code.activestate.com/recipes/149368/

addChild(child)

Add a reference to a child node

child: child node
addChildren(children)

Add references to a list child nodes

children: list of child nodes
copy()

Return a copy of self

getAllChildren(typeName='')

Return a list of all child nodes that match the given type name recursively (just getChildren() with recursive=True)

typeName: type name of (a) child node(s)
NOTE: When typeName is an empty string, return ALL child nodes
recursively.
getCData()

Get the character data

getChild(typeName)

Get the first child of this node with the given type name if it exists

typeName: type name of a child node
getChildLike(typeName)

Get the first child of this node with the a type name similar to the given type name if it exists

typeName: type name of a child node
getChildren(typeName='', recursive=False)

Return a list of child nodes that match the given type name

typeName: type name of (a) child node(s)

recursive: return child nodes recursively

NOTE: When typeName is an empty string, return ALL children of
this node or all (child) nodes recursively if recursive is set.
getChildrenLike(typeName='', recursive=False)

Return a list of child nodes that are similar to the given type name

typeName: type name of (a) child node(s)

recursive: return child nodes recursively

NOTE: When typeName is an empty string, return ALL children of
this node or all (child) nodes recursively if recursive is set.
getProperty(propertyName)

Get a property value

propertyName: name of property
hasChild(typeName)

Test if this node has a child by the given type name

typeName: string to search for
hasChildLike(typeName)

Test if this node has a child with a type name similar to the given type name (a matching substring of any child type name)

typeName: substring to search for
NOTE: This method only returns the existence one or more
children whose type name matched the given substring typeName. It does not give the number of matches.
setProperty(propertyName, propertyValue, clobber=True)

Set a property value

propertyName: name of property
propertyValue: value of property to set
clobber: overwrite value if it already exists
toString(level=0, indentChars=' ')

Return node and children as a string

class astro.util.XsdMapping.Xml2Object(encoding='utf-8', stripData=True)

Bases: object

XML to Object converter

Based on: http://code.activestate.com/recipes/149368/

characterData(data)

Expat character data event handler

data: character data string
endNode(typeName)

Expat end node (element) event handler

typeName: (placeholder) name of XML node type
parse(filename='', xml='')

Create an Expat parser

filename: name of file containing XML
xml: XML string
startNode(typeName, properties)

Expat start node (element) event handler

typeName: name of XML node type

properties: dictionary of property {name: value} pairs

class astro.util.XsdMapping.XsdAnnotation(annotationNode, strip=True)

Bases: object

XML schema annotation

class astro.util.XsdMapping.XsdAttribute(attributeNode, parentSchema=None, imports={})

Bases: astro.util.XsdMapping.XsdObject

XML schema attribute

Datatype

Mapping of XML type to Python datatype

PropertyName

Mapping of XML id to Python property name

Units

Units of the attribute

info()
updateNode(typeNode, obj, depth, level, header='', hierarchical=True, verbose=0)

Update a Node object with the AW class instance obj

typeNode: Node instance to update
obj: instance of a supported AW class
depth: descend only so many levels of data lineage
(0 is unlimited)

level: current level of depth

hierarchical: use a hierarchy of files instead of only one
file

verbose: adjust verbosity of output [0, 1, 2, 4]

class astro.util.XsdMapping.XsdElement(elementNode, parentSchema=None, imports={})

Bases: astro.util.XsdMapping.XsdObject

XML schema element

Datatype

Mapping of XML type to Python datatype

PropertyName

Mapping of XML id to Python property name

Unit

Units of the element

info()
updateNode(typeNode, obj, depth, level, header='', hierarchical=True, verbose=0)

Update a Node object with the AW class instance obj

typeNode: Node instance to update
obj: instance of a supported AW class
depth: descend only so many levels of data lineage
(0 is unlimited)

level: current level of depth

hierarchical: use a hierarchy of files instead of only one
file

verbose: adjust verbosity of output [0, 1, 2, 4]

class astro.util.XsdMapping.XsdImport(filepath, importNode, namespacePrefix)

Bases: astro.util.XsdMapping.XsdObject

XML import

Name

Import name (last section of namespace)

Schema
info()
exception astro.util.XsdMapping.XsdMappingError

Bases: Exception

class astro.util.XsdMapping.XsdObject

Bases: object

Abstract base class for XML Schema Definition entities

Properties = []
getAppinfo(source='')
getDocumentation(source='')
info()
class astro.util.XsdMapping.XsdSchema(pathname, namespacePrefix='')

Bases: astro.util.XsdMapping.XsdObject

XML schema (a package)

Filename = ''
Filepath = './'
Name

Schema name (last section of namespace)

Pathname
info()
readXml(filename, depth=0, verbose=0)

Return an AW class instance instantiated from the given XML file.

filename: name of input XML file
depth: descend only so many levels of data lineage (0 is
unlimited, see NOTE)

verbose: adjust verbosity of output [0, 1, 2, 4]

NOTE: The given XML file is either monolithic or part of a
hierarchy. If the latter, it should be the root XML file specifying child XML files in the ‘fileid’ attribute. Depth is only valid when the object does not exist in the database.
writeXml(obj, depth=0, hierarchical=True, verbose=0, archive='')

Return the filename for the AW class instance XML derived from the given object

obj: instance of a supported AW class (will be
tested)
depth: descend only so many levels of data lineage
(0 is unlimited)
hierarchical: use a hierarchy of files instead of only one
file (see NOTE)

verbose: adjust verbosity of output [0, 1, 2, 4] archive: compress XML file(s) with specific compression

(gzip or bzip, see NOTE)
NOTE: When archive is given, the XML file(s) is(are) compressed
in the following way: [g|b]zipped single XML file if hierarchical is False, and tar, [g|b]zipped archive of all XML files when hierarchical is True. In either case, the returned filename is that of the compressed file.
class astro.util.XsdMapping.XsdType(typeNode, parentSchema=None, imports={})

Bases: astro.util.XsdMapping.XsdObject

XML schema type (simple or complex)

ClassName

Mapping of XML id to Python class name

getAttribute(name='', id='')

Return an attribute by name or id, if it exists

DEPRECATED

getElement(name='', id='')

Return an element by name or id, if it exists

DEPRECATED

info()
readNode(typeNode, depth=0, verbose=0)

Read a Node object and return the corresponding AW class instance.

typeNode: (parent) Node instance to update
depth: descend only so many levels of data lineage
(0 is unlimited, see NOTE)

verbose: adjust verbosity of output [0, 1, 2, 4]

NOTE: Depth is only valid when the object does not exist in
the database.
writeNode(typeNode, obj, depth=0, level=0, header='', hierarchical=True, verbose=0)

Update a Node object with the AW class instance obj

typeNode: (parent) Node instance to update
obj: instance of a supported AW class
depth: descend only so many levels of data lineage
(0 is unlimited)

level: current level of depth

hierarchical: use a hierarchy of files instead of only one
file

verbose: adjust verbosity of output [0, 1, 2, 4]

astro.util.XsdMapping.basExtObjectId(type, node, typeName, value)

Handler for types bas:extObjectId

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.basObjectName(type, node, typeName, value)

Handler for types bas:objectName

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.dssDataContainer(type, node, typeName, value)

Handler for types dss:dataContainer

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.dtdNameRestriction(type, node, typeName, value)

Handler for types dtd:nameRestriction

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.eraseChildProperties(node, exceptions=[])
astro.util.XsdMapping.extAssociateListDict(type, node, typeName, value)

Handler for types ext:associateListDict

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.extSourceListDict(type, node, typeName, value)

Handler for types ext:sourceListDict

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.fitFitsFile(type, node, typeName, value)

Handler for types fit:fitsFile

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.imgMagnitudeID(type, node, typeName, value)

Handler for types img:magnitudeID

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.impProjectionType(type, node, typeName, value)

Handler for types imp:projectionType

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.insFilterName(type, node, typeName, value)

Handler for types ins:filterName

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.myJoin(iterable=[], string=':', compress=True)

Add functionality to the str.join() method

iterable: an iterable object containing strings to be joined
string: string used to join the strings in iterable together

compress: toggle to remove null entries (i.e., ‘’) in iterable

astro.util.XsdMapping.sgsCuration(type, node, typeName, value)

Handler for types sgs:curation

type: XsdType of attribute/element node: Node to be updated

value: value of attribute/element

astro.util.XsdMapping.splitPrefixAttribute(string='')

Find prefix of an attribute name (string before first ‘:’ character) and return a tuple of (prefix, suffix), where suffix is everything until the next ‘:’ or the end of the string. Suffix is truncated at the first ‘=’ character.

string: string with one embedded ‘:’ character
astro.util.XsdMapping.splitPrefixElement(string='')

Find prefix of an element name (string before first ‘:’ character) and return a tuple of (prefix, suffix1, …). The length of the returned tuple depends on the hierarchical depth (i.e., how many :’s)

string: string with zero or more embedded ‘:’ characters
NOTE: If no prefix is found (i.e., no colon-separation), the first
element of the tuple is blank (null string), anf the second is the input string.

astro.util.angsep module

astro.util.angsep.angsep(ra1deg, dec1deg, ra2deg, dec2deg)

Determine separation in degrees between two celestial objects arguments are RA and Dec in decimal degrees.

astro.util.bwsm module

Biweight location and scale estimator.

Implemented from bwsm subroutine by Naples KiDS-CAT (La Barbera et al.).

astro.util.bwsm.bin_mat(nx, num_per_bin_min, num_per_bin, x, y, plot=None)

Do binned trend, in each bin calculating average of edges, median, and standard deviation.

Implemented from bin_mat subroutine by Naples KiDS-CAT (La Barbera et al.).

Argument plot is set by KidsCatCompl to run additional steps for that plot.

Loop through x, which has length nx, and make bins with num_per_bin or at least num_per_bin_min.

Keep in the xb list the average of the edges of the bin. Keep in the yb list the numpy medians of the bins. Keep in the eyb list the numpy standard deviation of the median. Count the number of sources and store in nb.

Return xb, yb, nb, eyb.

Should be called by child classes.

astro.util.bwsm.bwsm(xx, num_sigma)

Calculates the biweight location and scale estimator.

Implemented from bwsm subroutine by Naples KiDS-CAT (La Barbera et al.).

The argument xx is the list of numbers. The argument num_sigma is a float to indicate the number of standard deviations from the location that we want to keep.

Return list of floats with biweight location and scale.

Should be called by method bin_mat.

astro.util.bwsm.bwsm_loop(n, x, num_sigma, par_2, t1, s1)

Loop used by method bwsm.

Implemented from bwsm subroutine by Naples KiDS-CAT (La Barbera et al.).

Return list of floats with biweight location and scale.

Should be called by method bwsm.

astro.util.bwsm.chi(diff, num_sigma)

Calculates part of the biweight scale.

Implemented from chi subroutine by Naples KiDS-CAT (La Barbera et al.).

The argument diff is the value to compare. The argument num_sigma is a float to indicate the number of standard deviations from the MAD that we want to keep.

Return float.

Should be called by method bwsm_loop.

astro.util.bwsm.integrate_par2(num_sigma)

Calculates par2 through integration.

astro.util.bwsm.psi(diff)

Calculates part of the biweight location.

Implemented from psi subroutine by Naples KiDS-CAT (La Barbera et al.).

The argument diff is the value to compare.

Return float.

Should be called by method bwsm_loop.

astro.util.bwsm.really_integrate_par2(num_sigma)

Calculates par2 of bwsm.

Values calculated by bwsm.f and this function: num_sigma/CC par_2 1 0.258029384450943 0.2579972183544455 2.5 0.488780198167117 0.48878747011089507 0 0.000000000000000 0.0

astro.util.bwsm.std_fast(x)

The std() function of bwsm in KiDSCAT calculates something else than the standard deviation. What KiDSCAT calculates is the same as the standard deviation for a Gaussian distribution. Which could be justified since the validity of using this std assumes an almost Gaussian distribution:

astro.util.epoco module

astro.util.epoco.epoco(a1, d1, t1=1950.0, t2=2000.0)

epoco transforms equatorial coordinates from one epoch to another epoch.

input: a1 R.A. of input coordinate
d1 Dec. of input coordinate t1 Epoch of input coordinates [1950.0] t2 Epoch of output coordiantes [2000.0]

output: tuple of two containing R.A. and Dec. at epoch t2.

astro.util.extinction module

astro.util.extinction.Aband(inp_filter)
astro.util.extinction.Arenou(lonvec, latvec, dist=None)
astro.util.extinction.Rband(inp_filter)
exception astro.util.extinction.SFDError

Bases: Exception

astro.util.extinction.bounds(xinp, xmin, xmax)
astro.util.extinction.djs_angpos(xval)
astro.util.extinction.extinction(longvec, latvec, source='SFD', interp=False, dist=None)
extinction returns an extinction in desired coordinates in Ebv
input:

longvec - input longtitude in galactic coordinates latvec - input latitude in galactic coordinates source - type of the source for an extinction

SFD : Schlegel, D., Finkbeiner, D., & Davis, M., ApJ, 1998, 500, 525 Arenou: Arenou F., Grenon M., Gomez A., Astron. Astrophys. 258, 104, 1992
output:
Ebv in mag
astro.util.extinction.fxread(data, x=-1, y=-1, noscale=True, N1=0, N2=0, BSCALE=1, BZERO=0)
astro.util.extinction.p2i(inp)
astro.util.extinction.wcs_coord2pix(lonvec, latvec, ctype1_coord, ctype2_coord, ctype1_proj, ctype2_proj, dradeg, sqtwo, nsgp, scale, crpix1, crpix2, crval1, crval2, cd1_1, cd1_2, cd2_1, cd2_2, lonpole, fractional=False)
astro.util.extinction.wcs_getval(lonvec, latvec, interp=True)

astro.util.extra_parameters module

astro.util.extra_parameters.get_extra_parameters(params, name=None)
astro.util.extra_parameters.put_extra_parameters(object, params={})
astro.util.extra_parameters.set_extra_parameter(object, key, val, skiplist=[])
astro.util.extra_parameters.set_extra_parameters(object, skiplist=[])

astro.util.htmsafe module

Safe usage of the htm module.

The bare htm module might cause the awe-prompt to hang or segfault. This wrapper module ensures that an exception is raised instead of passing dangerous values to the htm module itself.

Improvements w.r.t. to the normal htm module: - Prevents get_depth() from segfaulting by providing native Python

implementation.
  • Prevents NaN valued arguments from hanging/aborting/crashing the awe-prompt through a wrapper.

Possible improvements to add: - Prevent segfaults when passing invalid values (e.g. a dec > 90).

Possible improvements to this module itself: - Maintain docstring etc. through proper decorator? (This information

is not available in the htm module anyway.)
  • Put this functionality directly in the SWIG wrapper?

Tests for this module can be found in astro/test/functional/htmsafe_test.py

astro.util.htmsafe.area(*args, **kwargs)
astro.util.htmsafe.get_circle_indices(*args, **kwargs)
astro.util.htmsafe.get_circle_indices_dummy(*args, **kwargs)
astro.util.htmsafe.get_depth(trixel)
astro.util.htmsafe.get_distance(*args, **kwargs)
astro.util.htmsafe.get_edges_dec(*args, **kwargs)
astro.util.htmsafe.get_edges_ra(*args, **kwargs)
astro.util.htmsafe.get_nearest_indices(*args, **kwargs)
astro.util.htmsafe.get_ranges_3(*args, **kwargs)
astro.util.htmsafe.get_ranges_4(*args, **kwargs)
astro.util.htmsafe.get_ranges_4_test(*args, **kwargs)
astro.util.htmsafe.get_ranges_circle(*args, **kwargs)
astro.util.htmsafe.get_ranges_ellipse(*args, **kwargs)
astro.util.htmsafe.get_shortest_side(*args, **kwargs)
astro.util.htmsafe.htm_get_edges(*args, **kwargs)
astro.util.htmsafe.lookup(*args, **kwargs)
astro.util.htmsafe.lookup_(level, ra, dec)
astro.util.htmsafe.nan_checker(func)
astro.util.htmsafe.ok(*args, **kwargs)

astro.util.idllib module

exception astro.util.idllib.IDLLibError

Bases: Exception

astro.util.idllib.bprecess(ra, dec, mu_radec=None, parallax=None, rad_vel=None, epoch=None)
astro.util.idllib.glactc(ra, dec, year, degree=False, fk4=False, SuperGalactic=False, eqtogal=True)
astro.util.idllib.jprecess(ra, dec, mu_radec=None, parallax=None, rad_vel=None, epoch=None)
astro.util.idllib.precess(ra, dec, equinox1, equinox2, fk4=False, radian=False)
astro.util.idllib.premat(equinox1, equinox2, fk4=False)
astro.util.idllib.total(a, b)

astro.util.match module

astro.util.match.matchids(id1, id2)

Match two sets of ids. Returns:

ibest – array of indices of i1 that match i2; -1 if no match
astro.util.match.matchidsorted(ids, targetid)

Find id matches, return index in i1 that matches targetid; -1 if no match.

astro.util.match.matchjoin(si1, si2, ibest, sep=[], dict1={}, dict2={})

Keep only elements that match in both catalogs. Arguments:

si1 – object with data arrays as attributes (e.g. si1.mag_auto, si1.id) si2 – object with data arrays as attributes ibest – indices of si1 that match si2 (in order of si2)
Keyword Arguments:
sep – array of separations, will be added as an attribute to second object dict1 – dictionary of attributes for si1 (e.g. {‘mag_auto’:0,’id’:1} ) dict2 – dictionary of attributes for si2

The objects si1 and si2 would normally be sextractor objects returned by sextutils.sextractor(). In that case, the column names are stored in si1._d and si2._d. If the objects are not sextractor objects, the user can provide separate dictionaries whose keys are the object attributes that correspond to the numpy data arrays.

Returns:
s1, s2 – object arrays that include only the matching objects contained
in both s1 and s2.
astro.util.match.matchpos(ra1, dec1, ra2, dec2, tol)

Match two sets of ra,dec within a tolerance. Longer catalog should go first Arguments:

ra1 - Right Ascension decimal degrees (numpy array) dec1 - Declination decimal degrees (numpy array) ra2 - Right Ascension decimal degrees (numpy array) dec2 - Declination decimal degrees (numpy array) tol - Matching tolerance in decimal degrees.
Returns:
ibest - indices of the best matches within tol; -1 if no match within tol sep - separations (defaults to tol if no match within tol)
astro.util.match.matchsorted(ra, dec, ra1, dec1, tol)

Find closest ra,dec within tol to a target in an ra-sorted list of ra,dec. Arguments:

ra - Right Ascension decimal degrees (numpy sorted in ascending order) dec - Declination decimal degrees (numpy array) ra1 - RA to match (scalar, decimal degrees) ra1 - Dec to match (scalar, decimal degrees) tol - Matching tolerance in decimal degrees.
Returns:
ibest - index of the best match within tol; -1 if no match within tol sep - separation (defaults to tol if no match within tol)

astro.util.projections module

class astro.util.projections.car(cdelt1=None, cdelt2=None, crpix1=None, crpix2=None, crota2=0.0, crval1=None, crval2=None, CD=None, PV=None, ctype1=None, ctype2=None, astrom=None)

Bases: astro.util.projections.flt

proj_type = 'CAR'
astro.util.projections.degrad(x)

Converts radians -> degrees

astro.util.projections.distance(ra1, dec1, ra2, dec2)

Calculates distance in degrees

class astro.util.projections.flt(cdelt1=None, cdelt2=None, crpix1=None, crpix2=None, crota2=0.0, crval1=None, crval2=None, CD=None, PV=None, ctype1=None, ctype2=None, astrom=None)

Bases: astro.util.projections.projections

gg_to_ss(x, y)
proj_type = 'FLT'
ss_to_gg(a, d)
class astro.util.projections.gnomonic(cdelt1=None, cdelt2=None, crpix1=None, crpix2=None, crota2=0.0, crval1=None, crval2=None, CD=None, PV=None, ctype1=None, ctype2=None, astrom=None)

Bases: astro.util.projections.projections

do_PV(l, m)
do_inverse_PV(l, m)
gg_to_ss(x, y)
proj_type = 'TAN'
ss_to_gg(a, d)
class astro.util.projections.ncp(cdelt1=None, cdelt2=None, crpix1=None, crpix2=None, crota2=0.0, crval1=None, crval2=None, CD=None, PV=None, ctype1=None, ctype2=None, astrom=None)

Bases: astro.util.projections.projections

gg_to_ss(x, y)
proj_type = 'NCP'
ss_to_gg(a, d)
astro.util.projections.posang(ra1, dec1, ra2, dec2)
class astro.util.projections.projections(cdelt1=None, cdelt2=None, crpix1=None, crpix2=None, crota2=0.0, crval1=None, crval2=None, CD=None, PV=None, ctype1=None, ctype2=None, astrom=None)

Bases: object

cdelt1 = 0.0
cdelt2 = 0.0
cosd0 = 1.0
cosrho = 1.0
crota2 = 0.0
crpix1 = 0.0
crpix2 = 0.0
crval1 = 0.0
crval2 = 0.0
gg_to_ss(x, y)
gs_to_sg(x, d)
lm_to_xy(l, m)
proj_type = '???'
sg_to_gs(a, y)
sind0 = 0.0
sinrho = 0.0
ss_to_gg(a, d)
xy_to_lm(x, y)
astro.util.projections.raddeg(x)

Converts degrees -> radians

class astro.util.projections.sin(cdelt1=None, cdelt2=None, crpix1=None, crpix2=None, crota2=0.0, crval1=None, crval2=None, CD=None, PV=None, ctype1=None, ctype2=None, astrom=None)

Bases: astro.util.projections.projections

gg_to_ss(x, y)
proj_type = 'SIN'
ss_to_gg(a, d)
class astro.util.projections.zpn(cdelt1=None, cdelt2=None, crpix1=None, crpix2=None, crota2=0.0, crval1=None, crval2=None, CD=None, PV=None, ctype1=None, ctype2=None, astrom=None)

Bases: astro.util.projections.projections

proj_type = 'ZPN'

astro.util.skyco module

astro.util.skyco.arc2deg(xxx_todo_changeme, xxx_todo_changeme1)

convert the ra and dec coordinates in (hour,minute,second) and (degrees,minutes,seconds) to degrees

astro.util.skyco.deg2arc(ra, dec)

convert degrees (ra,dec) to (hour,minute,second) and (degrees,minutes,seconds)

astro.util.skyco.deg2arc_dec(dec)

convert dec in degrees to degrees,minutes,seconds

astro.util.skyco.deg2arc_ra(ra)

convert ra in degrees to hour,minute,second

astro.util.skyco.degrad(x)
astro.util.skyco.dist_on_sky(p1, p2)

Calculate the distance in arcseconds between two positions on the sky; p1 and p2. These are tuples containing the equatorial coordinates in decimal degrees of the two positions.

astro.util.skyco.raddeg(x)
astro.util.skyco.skyco(x, y)

astro.util.utilities module

astro.util.utilities.group_objects_by_chip(objects)
astro.util.utilities.group_objects_by_date_obs(objects)
astro.util.utilities.group_objects_by_filter(objects)
astro.util.utilities.group_objects_by_mag_id(objects)
astro.util.utilities.group_unique_objects_by_chip(objects)
astro.util.utilities.group_unique_objects_by_filter(objects)

astro.util.votable module

Class to handle VOTable Created: 2005-05-31 by Shui Hung Kwok, shkwok at computer.org Edited: 2007-02-12 by Noel Winstanley, noel.winstanley at manchester.ac.uk

  • added ability to parse strings as votable-xml

See http://www.ivoa.net/Documents/latest/VOT.html .

class astro.util.votable.VONode(tagname=('', ''))

Bases: object

Class representing an XML node of a VOTable

addAttribute(attr)
addAttributes(attrs)
addNode(node)
buildName(tname)

Returns a name with namespace as prefix or just name if no namespace Note that the prefix is the real namespace and not the abbreviation used in the original XML

getAttribute(name)

Returns attribute by name or ‘’ if not found

getAttributes()

Returns all attributes.

getContent()

Returns all strings of the node.

getNamePart()
getNode(path)

Returns a node for a given path. Path is of the form /tag1/tag2/tag3. Path can include array index, like /tag1/tag2[3]/tag4.

getNodeList()

Returns a list of nodes that are of type VONode

getNodesByName(look4)

Returns a list of nodes whose tagname = look4

getNodesByPath(path)

Returns an array of VONodes for a given path. Path is of the form /tag1/tag2/tag3. Path can include array index, like /tag1/tag2[3]/tag4.

printAllNodes(func=<bound method Myfile.write of <common.config.startup.Myfile object>>, prefix='')

Recursive method to visit all nodes of the tree and calls the provided function to output the content.

class astro.util.votable.VOTable(source=None, vonode=<class 'astro.util.votable.VONode'>)

Bases: astro.util.votable.VOXML

Implementation of VOTable

append(vot)

Appends votable vot to the end of this VOTable. No tests to see if fields are the same. vot must have the same fields.

getColumnIdx(val)

Returns the column index for the given name Will return any attribute value matching val.

getData(row)

row is a VONode <TR> parent of a list of <TD>. Returns a list of values.

getDataRows()

Returns a list of VONodes representing rows of the table. Use getData () to extract data from each row. for x in getDataRows ():

data = getData (x) #data = [values …]
getDataType()
getFields()

Returns a list of VONode representing all the fields

getFieldsAttrs()

Returns a list of maps that contains attributes. Returned list looks like this: [{},{},…]

getFits()
getParams()

Returns a list of VONode representing all PARAMS

output2CSV(write=<bound method Myfile.write of <common.config.startup.Myfile object>>)

Outputs the content in CSV format with fields names as column heads.

class astro.util.votable.VOXML(source=None, vonode=<class 'astro.util.votable.VONode'>)

Bases: object

Class to implement XML parser

getContent(path)

Returns the content of a node. Only strings are returned.

getNode(path)

Returns a VONode of the given path.

parse(source)

Invokes XML parser and stores VOXML in self.root as VONode.

parseText(text)

Invokes XML parser on inlined xml text

printAllNodes(func=<bound method Myfile.write of <common.config.startup.Myfile object>>)

Output entire content as XML. func is the output method, defined as:

func (outString)
class astro.util.votable.VOXMLHandler(vonode=<class 'astro.util.votable.VONode'>)

Bases: xml.sax.handler.ContentHandler

Class implementing callbacks for the SAX parser.

characters(chunk)
endElementNS(urn_name, qname)
parse(source)

Main entry point. Source can be URL or file name.

parseText(t)

Alternate entry point. Text is some xml to be parsed

startElementNS(urn_name, qname, attrs)

astro.util.xsdsupport module

class astro.util.xsdsupport.ImgType(Category, FirstType, SecondType, ThirdType, Technique)

Bases: object

astro.util.xsdsupport.asduos(alist)
astro.util.xsdsupport.astrom_format(self, format)
astro.util.xsdsupport.baseframe_format(self, format)
astro.util.xsdsupport.debug(tpl, obj=None, datetimefmt=None)
astro.util.xsdsupport.filter_format(self, format)
astro.util.xsdsupport.fixlayout(elem)
astro.util.xsdsupport.oracleoidtype_format(self, format)
astro.util.xsdsupport.substitute(root, pattern, listattribute)
astro.util.xsdsupport.typed_list_format(self, format)

Module contents

util