[WP3] Re: Database Evolution

[Danny R. Boxhoorn]

Hoi Jan,

> Hi Danny,
> 
> here are some 'impressions' from the database evolution, that I tried out
> here using our old database.
> 
> Most problems so far came from a wrong order of upgrades, leading to
> "cannot alter existing object blablabla" when importing, what I  found so
> far:
> 
> ReadNoise needs evolved RawDomeFlat
> 
> HotPixelMap, needs evolved BiasFrame
> 
> CosmicMap seems to need evolved GainLinearity and DomeFlatFrame,
> 
> (here I found it a bit confusing, that one class is modified at different
> places, so one has to track down all the changes)
> 
> ...


Ok, I think this is due to not being completely clear (or being completely
unclear [) about what the `docs/DBCHANGES' contains.
The `docs/dbchanges' contains a list of changes to the database schema
between a certain CVS version of a persistent class and another one.
The list contains the changes exactly in the order in which I performed them.
This means you also have to apply these changes in the same order.
Normally you would have done within a few days from when the changes had first
been made. But because of the high number of changes of the last months it
made sense to spend your time on more useful things.
The procedure for each change is then to apply the changes as they
appear between SQL BEGIN and SQL END, AWE BEGIN and AWE END and SHELL BEGIN
and SHELL END. Under some circumstances you have to iterate, mainly because
the database tells you that you cannot change an evolved type.
After that you have to move to the version in CVS that is given in the
`FROM 1.a TO 1.b' line.

As an example, taking the first class in DBCHANGES

   CLASS astro.main.FilterTransmissionCurve

After dropping the items mentioned in the SQL section (if only other schema
evolution was that simple ...) you would do

   cvs update -r 1.5 astro/main/FilterTransmissionCurve.py

Now, assuming that you've done exactly that, you should not have had
problems with ReadNoise and the addition of the Lamps to the RawDomeFlat
later on.

However, the following error is one of those that I haven't yet seen
while evolving types, so your tedious work is paying off.

> Currently something seems to have gone wrong somewhere, I now get:
> 
> ORA-04045: errors during recompilation/revalidation of
> AWOPER.RawDomeFlatFrame$
> ORA-22345: recompile type AWOPER.RawDomeFlatFrame$ before attempting this
> 
> (I suspect the then-not-yet imported Lamp class to have caused the havoc)

Have you tried recompiling AWOPER.RawDomeFlatFrame$ as the error message
suggests?
(ALTER TYPE AWOPER."AWOPER.RawDomeFlatFrame$" COMPILE or recompiling via
oemapp will do the trick).

In addition, what version of Oracle are you running?
Have you applied the latest patches?


> P.S. Roberto Saglia asked me about the demo that was announced at the last
> teleconference, but I did not yet find it in the CVS.

Good! Interestingly enough the demo is completely described in the Manual.tex
that you find in opipe/docs. The demo is therefore a complete session (raw
data, making calibration files reducing science data, regridding and coadding
science data, making a source list) in the astro-wise system. To be able to run
it you need a particular subset of the CdF data. I'm sure that Fedor, Mike or
Roberto Silvotti can help you with that. Otherwise, any other reasonable set of
observations will do. You'd only have to change the dates and filters
here and there. You don't have to look for particular demo code in case
you were doing that.

Thanks for trying to evolve your schema,

                                                   Danny