astro.toolbox.ingest package

Submodules

astro.toolbox.ingest.Ingest module

Ingestor for raw and pre-reduced optical and near-IR pixel data.

class astro.toolbox.ingest.Ingest.IngestController(input_images, image_type, weight_images=[], convert=1, memmap=1, commit=0, force=0, force_invalid_template=0, force_single_template=0, rawfitsdata_only=0)

Bases: object

check_preconditions()

Make sure all required preconditions are met.

convert_rawdata()

Convert the data to proper MEF if necessary.

execute()

Execute the ingestion.

ingest_rawfitsdata()

Create the RawFitsData to ingest from.

NOTE: In the case of pre-reduced data, RawFitsData is created only
for convenience. Because ingested pre-reduced frames have no prior processing history, the RawFitsData is not retained.
ingest_rawframes()

Ingest the reduced frames from the RawFitsData.

exception astro.toolbox.ingest.Ingest.IngestControllerError

Bases: Exception

astro.toolbox.ingest.Ingest.main()
astro.toolbox.ingest.Ingest.parse_command_line()
astro.toolbox.ingest.Ingest.split_lines(word_list)
astro.toolbox.ingest.Ingest.usage()

astro.toolbox.ingest.IngestCASUCatalogs module

class astro.toolbox.ingest.IngestCASUCatalogs.CASUCatalogIngestor(filenames, force=False, commit=False)

Bases: object

execute()
exists_in_database(catalog)

Check if a Catalog object exists in the database with the same filename.

Returns True or False

get_frame_for_catalog(catalog)

There should be an already ingested frame to connect the catalog to.

Get it. Return it.

ingest_catalog(filename)
astro.toolbox.ingest.IngestCASUCatalogs.main()

astro.toolbox.ingest.IngestCASUTiles module

class astro.toolbox.ingest.IngestCASUTiles.CASUTileIngestor(filenames, commit=False)

Bases: object

create_comments_reduced(frame, casuname='')
create_comments_weight(frame, weight)
execute()
exists_in_database(redframe)

Check if a ReducedScienceFrame object exists in the database for the same DATE-OBS, chip name and CASUVERS.

Returns True or False

exists_in_database_quick(filename)

Check if a file ReducedScienceFrame object exists in the database for the same DATE-OBS, chip name and CASUVERS.

Returns True or False

get_weight_for_reduced(reduced)
ingest_tile_as_reduced(filename)
update_chip_id(filename)
astro.toolbox.ingest.IngestCASUTiles.main()

astro.toolbox.ingest.IngestCatalogs module

class astro.toolbox.ingest.IngestCatalogs.IngestCatalogsTask(filenames='', store=0, **kw)

Bases: astro.recipes.mods.Task.Task

CATEGORY : data ingestion
check_instrument(rawfitsdata)
check_preconditions()
execute()
make_catalog(filename)
exception astro.toolbox.ingest.IngestCatalogs.IngestCatalogsTaskError

Bases: Exception

astro.toolbox.ingest.IngestCatalogs.main()
astro.toolbox.ingest.IngestCatalogs.parse_command_line()
astro.toolbox.ingest.IngestCatalogs.usage()

astro.toolbox.ingest.IngestLofar module

class astro.toolbox.ingest.IngestLofar.IngestLofarTask(filenames=[], commit=False, verbose=False)

Bases: astro.recipes.mods.Task.Task

CATEGORY : Lofar data ingestion

This recipe is used to ingest Lofar (..) This is experimental, untill now it is only been used to ingest generated datacubes, test data from Niruj.

images into Astro-WISE. The (3D) radio images are stored as RadioCubes.

The task assumes that a dataserver and database is active, and that it has been instantiated with valid values for ‘filenames’. The task is executed by calling its ‘execute’ method’.

Example of use from the awe prompt :

awe> task = IngestLofarTask(filenames=[‘LOFAR120.01.FITS’]) awe> task.execute()

Mandatory keywords :

filenames (list)

The type of the values assigned to the keywords is given between parentheses.

INSTRUMENT = 'LOFAR'
check_header(header)

check the header items : - existance - valid values

check_preconditions()
execute()
expected_values = {'INSTRUME': 'LOFAR'}
ingest(filename)
astro.toolbox.ingest.IngestLofar.main()
astro.toolbox.ingest.IngestLofar.parse_command_line()
astro.toolbox.ingest.IngestLofar.usage()

astro.toolbox.ingest.IngestOCAM module

astro.toolbox.ingest.IngestWENSS module

class astro.toolbox.ingest.IngestWENSS.IngestWENSSTask(filenames=[], commit=0)

Bases: astro.recipes.mods.Task.Task

CATEGORY : WENSS data ingestion

This recipe is used to ingest WENSS (Westerbork Northern Sky Survey) images into Astro-WISE. The radio images are stored as RadioCubes.

The task assumes that a dataserver and database is active, and that it has been instantiated with valid values for ‘filenames’. The task is executed by calling its ‘execute’ method’.

Example of use from the awe prompt :

awe> task = IngestWENSSTask(filenames=[‘WN55354H’]) awe> task.execute()

Mandatory keywords :

filenames (list)

The type of the values assigned to the keywords is given between parentheses.

The catalogues WENSS and WISH have also been ingested in the Astro-Wise system, see http://www.aoc.nrao.edu/aips/nvss.html for details of the catalogues. The catalogues can be found with their SLID :

WENSS : SLID 80891 WISH : SLID 80901

For details contact wjvriend@astro.rug.nl

FILTER = 'RadioP'
INSTRUMENT = 'WSRT'
check_header(header)
check_preconditions()
convertDateMap(date)

convrt dd/mm/yyyy to datetime

convertDateObs(fractional_years)

convert the date-obs header item from (format yyyy.yyy) to a datetime object

execute()
expected_values = {'INSTRUME': 'WSRT', 'NAXIS': '3'}
ingest(filename)
astro.toolbox.ingest.IngestWENSS.main()
astro.toolbox.ingest.IngestWENSS.parse_command_line()
astro.toolbox.ingest.IngestWENSS.usage()

Module contents

Tools for ingestion