[Issues] global astrometry problem
Kor G. Begeman
kgb at astro.rug.nl
Tue Feb 20 14:21:28 CET 2007
Johannes,
I'm sorry to report back so late but indeed you are right! This is a
feature which should NOT happen if we have a federated database but
happens if you are on a stand-alone database doing GAstrometry in
parallel. The thing is that once the USNO sourcelist exists and has
number of sources > 0 the software can put a lock on the sourcelist.
When the number of sources is 0 or the sourcelist does not exist at all
the lock obviously does not work. In a federated database you will have
no problems because the Sourcelist already exists (and at the moment is
completely filled with all of the USNO-A2).
Regards,
Kor
Johannes Koppenhoefer wrote:
> Dear Kor,
>
> sorry, of course SourceLists are stored in the database.
> Concerning your question, the number_of_sources is different (23088,10623,...).
> By doing further tests with GAstrom I have found out, that the problem does not occure if you first run one single GAstromTask and wait until a Sourcelist with name 'USNO-4.0' is created and stored in the database. Any further GAstromTask works fine because it will not create another Sourcelist with name 'USNO-4.0' anymore. So the problem only exists if you have no Sourcelist with name 'USNO-4.0' and you submit several GAstrom job exactly the same time.
>
> Bug or a feature?
>
> Cheers,
> Johannes
>
>
>
> "Kor G. Begeman" <kgb at astro.rug.nl> schrieb am 21.12.2006 09:45:42:
>> Johannes,
>>
>> Johannes Koppenhoefer wrote:
>>> Hi all,
>>>
>>> I am currently having problems while running GAstromTask on the dpu:
>>>
>>> In some cases (unfortunately I cannot tell in which exactly) a SourceList 'ldac.cat' with name 'USNO-A2.0' is created and stored on the data server.
>> The Sourcelist is stored in the database, not on a data server. It is
>> (part of) the USNO-A2.0 reference catalog which covers your data.
>>
>>> After running several GAstrom jobs this has happened multiple times and a number of SourceLists with identical names and identical filenames have been created, commited and stored.
>> Could you please find out how many records each reference catalog has.
>> In awe do:
>>
>> awe> s = SourceList.name == 'USNO-A2.0'
>> awe> for l in s: print l.number_of_sources
>>
>>> After a while, running another GAstrom on the dpu I get an error: "GAstromTask ERROR: Multiple Catalogues with identical names!"
>>> Obviously caused by the previously created SourceLists 'ldac.cat' with all of them having the same name 'USNO-A2.0'.
>>
>>> The get_logs() output is the following:
>>>
>>> [apus] 11:53:00 - ++++++++++++++++++++++++++++++++++++++++++++
>>> [apus] 11:53:00 - job returned could not be unpickled!
>>> [apus] 11:53:00 - ++++++++++++++++++++++++++++++++++++++++++++
>>> [apus] 11:53:00 - Starting GAstromTask
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Querying database for instances of class ReducedScienceFrame...
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.6701261-27a11652ca9d50bd4144ecc425fcea0911aa85a8.fits.slist, SLID = 2513, number of sources = 5149
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.6097162-4ef04fcb7fc85d89a4f48f5c5275610ae386696c.fits.slist, SLID = 2376, number of sources = 4470
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.5281988-a1c10fe250033125c9e249e2d156617d0ecec056.fits.slist, SLID = 2397, number of sources = 4752
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.5593435-1135be0f889d2552a903769d8b10dd55a8dd9f76.fits.slist, SLID = 2418, number of sources = 4640
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.6653329-7b9af92f7d381546032d78a8fe65f98565bdb680.fits.slist, SLID = 2437, number of sources = 4687
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.5302463-50afeb344cd67fe486fc733a59ab563bb8576d6a.fits.slist, SLID = 2454, number of sources = 4529
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.6643428-aef799dd1c675b9edfa0449022e28cc73d7373a1.fits.slist, SLID = 2473, number of sources = 4554
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.5767374-2ed594c36b23ce320de87cd60eb7340678b6e6d4.fits.slist, SLID = 2476, number of sources = 4385
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.5048954-61e0be050680de959ecc62980be6af456a28a6c7.fits.slist, SLID = 2493, number of sources = 4267
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.5614838-5a1e24c5acbbbdd4a76652847e96f6374a670ca0.fits.slist, SLID = 2384, number of sources = 4382
>>> [apus] 11:53:00 - [ocam-n01] 11:51:42 - Using SourceList GAS-Sci-JKOPPENH-WFI-------#844-ccd54-Red---Sci-54068.6605039-30cadbf04667afc9eb86009dccabac9ad1fdeb6f.fits.slist, SLID = 2396, number of sources = 4031
>>> [apus] 11:53:00 - Error executing 'execute'
>>> [apus] 11:53:00 - Exception: Multiple Catalogues with identical names!
>>> [apus] 11:53:00 - File "/usr/local/opt/astrowise/opipe/current/astro/net/dpu_runner.py", line 174, in main
>>> [apus] 11:53:00 - job.execute()
>>> [apus] 11:53:00 - File "/home/apus/koppenh/opipe/DBRecipes/GAstrom.py", line 139, in execute
>>> [apus] 11:53:00 - File "/home/apus/koppenh/opipe/DBRecipes/GAstrom.py", line 176, in make_global_astrometric_solution
>>> [apus] 11:53:00 - File "/home/apus/koppenh/opipe/astro/main/GAstrometric.py", line 549, in make
>>> [apus] 11:53:00 - File "/home/apus/koppenh/opipe/astro/main/GAstrometric.py", line 174, in make_associate
>>> [apus] 11:53:00 - File "/home/apus/koppenh/opipe/astro/main/SourceList.py", line 1819, in ImportSourceList
>>> [apus] 11:53:00 - Error pickling job
>>> [apus] 11:53:00 - Exception: can't pickle function objects
>>> [apus] 11:53:00 - File "/usr/local/opt/astrowise/opipe/current/astro/net/dpu_runner.py", line 191, in main
>>> [apus] 11:53:00 - pjob = pickle.dumps(job)
>>> [apus] 11:53:00 - File "/usr/lib/python2.3/copy_reg.py", line 69, in _reduce_ex
>>> [apus] 11:53:00 - raise TypeError, "can't pickle %s objects" % base.__name__
>>> [apus] 11:53:00 -
>>>
>>> Cheers,
>>> Johannes
>>
>> Cheers,
>>
>>
>> Kor
>>> P.S.: I used the latest CVS version from this morning.
>>> _______________________________________________________________________
>>> Viren-Scan für Ihren PC! Jetzt für jeden. Sofort, online und kostenlos.
>>> Gleich testen! http://www.pc-sicherheit.web.de/freescan/?mc=022222
>>>
>>>
>>> _______________________________________________
>>> Issues mailing list
>>> Issues at astro-wise.org
>>> http://listman.astro-wise.org/mailman/listinfo/issues
>> --
>>
>> Dr. K.G. Begeman
>> OmegaCEN
>> Kapteyn Institute
>> University of Groningen
>> Postbus 800 NL-9700 AV Groningen
>> Landleven 12 NL-9747 AD Groningen
>> The Netherlands
>> Telephone +31-(0)50-3632454/4073
>> Telefax +31-(0)50-3636100
>> e-Mail kgb at astro.rug.nl
>> WWW http://www.astro.rug.nl/~kgb
>> _______________________________________________
>> Issues mailing list
>> Issues at astro-wise.org
>> http://listman.astro-wise.org/mailman/listinfo/issues
>>
> _______________________________________________________________________
> Viren-Scan für Ihren PC! Jetzt für jeden. Sofort, online und kostenlos.
> Gleich testen! http://www.pc-sicherheit.web.de/freescan/?mc=022222
>
>
> _______________________________________________
> Issues mailing list
> Issues at astro-wise.org
> http://listman.astro-wise.org/mailman/listinfo/issues
--
Dr. K.G. Begeman
OmegaCEN
Kapteyn Institute
University of Groningen
Postbus 800 NL-9700 AV Groningen
Landleven 12 NL-9747 AD Groningen
The Netherlands
Telephone +31-(0)50-3632454/4073
Telefax +31-(0)50-3636100
e-Mail kgb at astro.rug.nl
WWW http://www.astro.rug.nl/~kgb
More information about the Issues
mailing list