astro.test.omegacen package

Submodules

astro.test.omegacen.ACTooFewSources module

AttributeCalculator that calculates data for too few sources.

astro.test.omegacen.ACTooFewSources.ACTooFewSources

alias of astro.test.omegacen.ACTooFewSources.AttributeCalculator

class astro.test.omegacen.ACTooFewSources.AttributeCalculator(sourcelist_data=None)

Bases: astro.main.sourcecollection.AttributeCalculator.AttributeCalculator

AttributeCalculator that calculates data for too few sources.

SCID

SourceCollection identifier [None]

acd_attributes = [{'null': None, 'length': 1, 'format': 'float32', 'name': 'Z_B', 'ucd': ''}]
acd_input_attribute_names = ['RA', 'DEC']
acd_parameters = []
all_data_stored

Flag to indicate whether all data has been stored in sourcelist_data, 0 means no (or unknown), 1 means yes

attribute_columns

Column names of the attributes in the sourcelist_data

attribute_names

Names of the attributes corresponding to the attribute_columns

calculate_attributes_vector(RA, DEC)
creation_date

Date this object was created [None]

definition

The Definition of this AttributeCalculator instance.

get_attribute_names(cache=False)
get_attribute_names_nc(cache=False)

Returns the names of the attributes.

get_attributes(cache=False)
get_attributes_full(cache=False)
get_attributes_full_nc(cache=False)

Returns a TableConverter like attribute list with extra keys.

Should only be used by SourceCollection functions. Other classes and awe-prompt users should use get_attributes().

This function should be overloaded by the derived classes. The version in this base class is essentially the variant of the External.

get_attributes_nc(cache=False)

Returns a list of dictionaries with meta data about the attributes.

Keys of the dictionaries:
  • name: The name (and identifier) of the attribute.
  • format: A format string that can be used by numpy.dtype()
  • ucd: Unified Content Descriptor (not always filled properly)
  • null: A null value, usually numpy.nan (not always present)
  • length: The length for multi length cells, only used for strings.

The dictionaries have the same structure as those used in the TableConverter class.

TODO LT L: [NOCAT] Use ‘ucd’ and ‘null’ properly.

is_valid

Manual/external flag to disqualify bad data (SuperFlag) [None]

make(optimize=True)

New make() function that works with old ACDs for which the original make() function does not accept the optimize keyword.

E.g. ACD 100031 for calculating comoving distances.

make_nc(optimize=True)

This is a virtual make() method for an AttributeCalculator instance.

The AttributeCalculatorDefinition should provide either: - an entire make() function - a calculate_attributes() or calculate_attributes_vector() function.

The default make() of the AttributeCalculator is a wrapper around the calculate_attributes_vector() function.

name

Name of the SourceCollection [None]

object_id

The object identifier

The object identifier is an attribute shared by all persistent instances. It is the prime key, by which object identity is established

parent_collection

Parent SourceCollections [None].

process_parameters

Process parameters

process_status

A flag indicating the processing status [None]

quality_flags

Automatic/internal quality flag [None]

sourcelist_data

Optional SourceList containing the data described by this SourceCollection

sourcelist_sources

Optional SourceList containing the sources described by this SourceCollection

astro.test.omegacen.AttributeCalculator_test module

Functionality tests of the AttributeCalculator class.

class astro.test.omegacen.AttributeCalculator_test.AttributeCalculatorTest(methodName='runTest')

Bases: unittest.case.TestCase

Tests the AttributeCalculator classes.

setUp()
test_acd_equality()

Test equality of AttributeCalculatorDefinitions.

test_coveragereport()

This tests whether a proper coverage report is being generated. scac is an AttributeCalculator for which the calculation is stored in a file on the dataserver that is fetched with the AC. This can cause problems with the generation of the coverage html file.

test_incomplete_sources()

Tests whether an exception is raised when the make method of an AttributeCalculator returns an incorrect number of sources for some of the attributes.

astro.test.omegacen.CatalogClasses_test module

These are tests in order to unify the functionality of different catalog classes.

In particular, the goal is to unify the interface of the (non sextractor related part of the) SourceList and the SourceCollection classes with the goal of integrating them. For details see the wiki: http://wiki.astro-wise.org/people:hb:catalogclasses

class astro.test.omegacen.CatalogClasses_test.testSourceCollection(methodName='runTest')

Bases: unittest.case.TestCase

setUp()
tearDown()
test_sources1()

astro.test.omegacen.InlineNone_test module

Test an inline object with None values.

class astro.test.omegacen.InlineNone_test.TestInlineImstat(methodName='runTest')

Bases: unittest.case.TestCase

Test an inline object with None values.

setUp()

Need to use an existing RegriddedFrame in project KIDS because it is unclear how to create these objects.

tearDown()
test_inlineimstat()

astro.test.omegacen.InspectFigure_test module

Functionality tests of the InspectFigure class.

class astro.test.omegacen.InspectFigure_test.testInspectFigure(methodName='runTest')

Bases: unittest.case.TestCase

test_retrieveextension()

This tests whether the extension is set correctly.

TODO: - Rewrite this test such that it will work without a specific

object in the database.
test_selectbyextension()

This tests whether it is possible to select on the extension.

TODO: - Rewrite this test such that it will work without a specific

objects in the database.

astro.test.omegacen.KidsCatFigures_test module

Tests the KidsCat statistics and figures.

class astro.test.omegacen.KidsCatFigures_test.TestKidsCatFigures(methodName='runTest')

Bases: unittest.case.TestCase

Tests the figures and the statistics.

setUp()
tearDown()
test_kcstats()

Test creating the statistics from comment and dictionary gives the same result.

test_statsandfigures()

Test creating the stats from the figures gives the correct result.

astro.test.omegacen.KidsCatProcessing_test module

Processing tests for the KidsCat classes.

class astro.test.omegacen.KidsCatProcessing_test.TestKidsCatProcessing(methodName='runTest')

Bases: unittest.case.TestCase

Testing Processing of KidsCat.

setUp()
skipped_test_full_matched_dr3()

A matched test for a full DR3 tile.

skipped_test_kidscat_dr3()

A KidsCat test on a full DR3 tile.

tearDown()
class astro.test.omegacen.KidsCatProcessing_test.TestKidsCatProcessingSmall(methodName='runTest')

Bases: unittest.case.TestCase

Small Testing Processing of KidsCat.

disabled_test_kidscat_small()

A small KidsCat test.

setUp()
tearDown()
test_full_matched_small()

A matched test for a small tile.

astro.test.omegacen.KidsCatProcessing_test.full_matched_test_dr3()

Do a full test with DR3 Data.

# TODO: Force the creation of the catalog to compare agaist can.

astro.test.omegacen.KidsCatProcessing_test.full_matched_test_small()

Do a full test with a small dataset. These coadds are created of the middle 4 CCDs of a larger coadd.

TODO: Add test of ingesting the data here?

astro.test.omegacen.KidsCatProcessing_test.kidscat_test_dr3()

Do a full coadd->figures KidsCat testrun.

astro.test.omegacen.KidsCatProcessing_test.kidscat_test_small()

Do a small end to end test.

TODO: the full_kc shouldn’t do anything anymore, integrate this test elsewhere
e.g. full_matched_test_dr3() Or better, recreate the single-band kidscat test here.
astro.test.omegacen.KidsCatProcessing_test.round_to_n(x, n)

Rounds to n significant digits.

>>> round_to_n(16512653884241.266, 6)
16512700000000.0

astro.test.omegacen.KidsCatTask_test module

Processing tests for the KidsCat classes.

class astro.test.omegacen.KidsCatTask_test.TestKidsCatTask(methodName='runTest')

Bases: unittest.case.TestCase

Testing Processing of KidsCat.

setUp()
tearDown()
test_kidscat_small()

astro.test.omegacen.KidsCatWFI_test module

Processing tests for the KidsCat classes using WFI data.

class astro.test.omegacen.KidsCatWFI_test.TestKidsCatProcessingWFI(methodName='runTest')

Bases: unittest.case.TestCase

Small Testing Processing of KidsCat.

setUp()
test_wfi()

A matched test for a small tile.

astro.test.omegacen.KidsCatWFI_test.kidscat_wfi_test()

Do a small end to end test.

TODO: Do some actual tests on the data.

astro.test.omegacen.KidsCat_test module

Unit tests for the KidsCat classes.

class astro.test.omegacen.KidsCat_test.TestKidsCatHelpers(methodName='runTest')

Bases: unittest.case.TestCase

Test KidsCat helper functions.

setUp()
test_bwsm()

Tests the bwsm code.

Also implicitly tests par2, chi() and psi() for num_sigma=1.0.

test_par2()

Tests the calculation of par2 of bwsm.

Also implicitly tests chi().

test_std_fast()

Tests the std() of bwsm.

test_twodphot_bit1()
test_twodphot_bit2()

astro.test.omegacen.OnTheFly_test module

Tests to determine whether OnTheFly functions properly.

Currently works by searching for objects that exist in the database and is therefore in the ‘omegacen’ directory.

class astro.test.omegacen.OnTheFly_test.testOnTheFlyExist(methodName='runTest')

Bases: unittest.case.TestCase

setUp()
tearDown()
test_exist1()

Test 1 whether the exist method returns the right object.

test_exist3()

Test 3 whether the exist method returns the right object.

test_exist4()

Test 4 whether the exist method returns the right object.

class astro.test.omegacen.OnTheFly_test.testOnTheFlyGetOnTheFlyDependencies(methodName='runTest')

Bases: unittest.case.TestCase

setUp()
tearDown()
test_get_onthefly_dependencies_al()

SourceCollections cannot be used directly in the AssociateList, but have to be converted to SourceLists first. However, this looses data lineage, so the AssociateList should return the SourceCollections as well as the SourceLists.

test_get_onthefly_dependencies_cl()

Test whether the AssociateList that is used to create a CombinedList is returned as a dependency of the resulting SourceList.

Ultimately the CombinedList should be a true persistent class, for example derived from SourceCollection, and the used AssociateList should be a true dependency. E.g. see http://wiki.astro-wise.org/people:hb:catalogclasses

test_get_onthefly_dependencies_slsc()

Test whether a SourceList that correspond to a SourceCollection returns that SourceCollection as a dependency. Technically the SourceCollection would not be a dependency of the SourceList, but conceptually this makes sense in a way because the SourceCollection has to be recreated in order to recreate the SourceList and if one of the dependencies of this SourceCollection is outdated, then so is the SourceList.

astro.test.omegacen.RelabelSources_test module

Tests to see whether the RelabelSources works correctly.

class astro.test.omegacen.RelabelSources_test.skipped_testRelabelSourcesPython(methodName='runTest')

Bases: unittest.case.TestCase

setUp()
tearDown()
test_rspython()
class astro.test.omegacen.RelabelSources_test.testRelabelSourcesSQL(methodName='runTest')

Bases: unittest.case.TestCase

setUp()
tearDown()
test_rssql()

astro.test.omegacen.SAMPQDV_test module

Tests to determine wether the query driven visualization over SAMP works properly.

class astro.test.omegacen.SAMPQDV_test.testSAMPQDV(methodName='runTest')

Bases: unittest.case.TestCase

setUp()
skipped_test_duplicate_derive()

Derive a catalog over SAMP twice. The result should be the same both times.

tearDown()

astro.test.omegacen.SourceCollectionPropagation_test module

These are tests to see whether some attributes of SourceLists can be propagated into SourceCollections derived from it.

The following properties might be relevant for SourecCollections as well: - filter - the bounding box: ulRA - OBJECT

Not for all SourceCollections these are sensible. However, they should be available when they are meaningful.

The goals are: - Have this information ‘retrievable’ for any SC where they make sense and

can be derived.
  • Have this information be propagated to the sourcelist_data of the SC.
  • (Ultimately) have this information available in the SC itself.
class astro.test.omegacen.SourceCollectionPropagation_test.testSourceCollection(methodName='runTest')

Bases: unittest.case.TestCase

Todo:
  • Add more tests for bounding box propagation. E.g. for RelabelSources etc.
  • Implement the required get_bounding_box functionality.
  • Add tests to store propagated attributes in sourcelist_data.
setUp()
tearDown()
test_propagation1()
test_propagation2()
test_propagation3()
test_propagation4()
test_propagation5()
test_propagation6()
test_propagation_storing()

A continuation of test_propagation1 where the information is stored as sourcelist_data.

astro.test.omegacen.SourceCollectionQueries_test module

Functionality tests.

These are more like integration tests, but for now they suffice.

exception astro.test.omegacen.SourceCollectionQueries_test.TimeoutError

Bases: Exception

astro.test.omegacen.SourceCollectionQueries_test.find_public_sourcelist_for_table(tableid)

Searching for an SL to use for queryA above.

class astro.test.omegacen.SourceCollectionQueries_test.testSourceCollection(methodName='runTest')

Bases: unittest.case.TestCase

disabledtest_query30()

Q30 Selects the AIDs that correspond to 1129 sources.

This query used to work, but it does not anymore. This should be fixed, but for now it is disabled because it slows the unittests down a lot.

queryA(slid, table_id=None, sid=0)

Test to fetch sources from SourceLists that use different SOURCELIST*SOURCES tables.

queryB(sl, sid)
setUp()
tearDown()
test_query31()

Q31 Selects the RAs that correspond to the 1129 sources.

test_queryA0()
test_queryA1()
test_queryA10()
test_queryA11()
test_queryA2()
test_queryA3()
test_queryA5()
test_queryA6()
test_queryA7()
test_queryA8()
test_queryA9()
astro.test.omegacen.SourceCollectionQueries_test.timeout(seconds=10, error_message='Timeout')

astro.test.omegacen.SourceCollection_loaddata_test module

Tests to determine whether the data of specific SourceCollections of interest can be loaded into Python.

These tests should, to some extend, represent common access patterns, at least with respect to KiDS.

class astro.test.omegacen.SourceCollection_loaddata_test.skipped_testSourceCollectionLoadData(methodName='runTest')

Bases: unittest.case.TestCase

setUp()
tearDown()
test_gaaptile1()

Relatively small 4 band GaAP tile. The problems with loading this dataset resulted in the ‘ROWNUM > 0’ clause to be added to get_query_self in SourceCollection.

test_gaaptile2()

Relatively large 4 band GaAP tile.

test_sdssselection1()

Selection of large SDSS catalog. The ROWNUM>0 clause in SourceCollection causes the query of this FilterSources to take long (at least 30 minutes). Therefore, the ROWNUM>0 has been removed for large catalogs (more than a million sources).

astro.test.omegacen.SourceCollection_test module

Functionality tests.

These are more like integration tests, but for now they suffice.

class astro.test.omegacen.SourceCollection_test.TestSourceCollection(methodName='runTest')

Bases: unittest.case.TestCase

Test SourceCollections.

setUp()
tearDown()
test_appendnulls()

This test will trigger expanding TableConverters in a way that fails when the null values are not properly handled. No error = success.

test_datasupercreation()
test_demo4()
test_demo4a()
test_demo4python()
test_demo5()
test_demo5d()
test_doublederive()

This test pulls the same catalog twice. The second time the SourceCollectionTree will try to reuse SourceCollections it created during the first run. Therefore the second run will go into branches that are skipped in the first part of the test.

test_howto1()
test_partialingest()
test_query_parse()
test_query_self()
test_setquery()

Test the set_query function of FilterSources.

TODO: Handle the error cases in a nicer way.

astro.test.omegacen.SourceList_test module

Tests for SourceList.

class astro.test.omegacen.SourceList_test.testSourceList(methodName='runTest')

Bases: unittest.case.TestCase

SourceList tests.

setUp()
tearDown()
test_get_data_dict_from_sql()

Test to fetch sources from a SourceList.

test_set_flag_frame()

astro.test.omegacen.Two2DPhotAttributeCalculator_test module

Tests the TwoDPhotAttributeCalculator.

class astro.test.omegacen.Two2DPhotAttributeCalculator_test.TwoDPhotAttributeCalculatorTest(methodName='runTest')

Bases: unittest.case.TestCase

Tests the 2DPhotAttributeCalculator.

setUp()
tearDown()
test2DPhotAC()

astro.test.omegacen.aw_sourcelistsources_test module

Checks whether the columns of the SOURCELIST*SOURCES[**0?] tables match the OBJECTLIST_TYPE_ATTRS table.

class astro.test.omegacen.aw_sourcelistsources_test.test_aw_sourcelistsources(methodName='runTest')

Bases: unittest.case.TestCase

check_bad_columns(tableid)
check_bad_types(tableid)
setUp()
tearDown()
test_columns_0()
test_columns_1()
test_columns_10()
test_columns_11()
test_columns_12()
test_columns_2()
test_columns_3()
test_columns_4()
test_columns_5()
test_columns_6()
test_columns_7()
test_columns_8()
test_columns_9()
test_types_0()
test_types_1()
test_types_10()
test_types_11()
test_types_12()
test_types_2()
test_types_3()
test_types_4()
test_types_5()
test_types_6()
test_types_7()
test_types_8()
test_types_9()

astro.test.omegacen.sourceprogenitors_test module

Tests to see wether we can find the proper progenitors of a source.

class astro.test.omegacen.sourceprogenitors_test.test_aw_sourcelistsources(methodName='runTest')

Bases: unittest.case.TestCase

setUp()
tearDown()
test_progenitors()

Module contents