Queued Service Observing with MegaPrime, WIRCam, and ESPaDOnS:

Semester 2008A Report

09/03/2008

TABLE OF CONTENT

A - Introduction
B - General Comments
C - Global Statistics, Program Completeness, and Overheads
D - Agency Time Accounting
E - Conclusions


A - Introduction

The Queued Service Observing (QSO) Project is part of a larger ensemble of software components defining the New Observing Process (NOP) which includes NEO (acquisition software), Elixir (data analysis for MegaCam), `I`iwi (data analysis for WIRCam), Upena (data analysis for ESPaDOnS) and DADS (data archiving and distribution). The semester 2008A was the first semester in the history of QSO during which three instruments were offered in that mode: MegaPrime, WIRCam, and ESPaDOnS.

The main development for semester 2008A was of course the switch of ESPaDOnS to QSO mode after over 3 years of classical use. Thanks to careful planning and preparations, the transition went very smoothly. In addition, many new functionalities and improvements were incorporated to increase the quality and value of the data, such as automated telescope focus sequences, telescope focus model, automated Image Quality measurement, quick-look data analysis with reports of S/N.


B - General Comments

MegaPrime

The 2008A semester for MegaPrime started with seriously bad weather, but we were able to catch up and ended up with excellent completion rates.

  1. Technically, the entire chain of operation, QSO --> NEO --> TCS, is efficient and robust. The time lost to the NOP chain is negligible. The system is quite reliable and efficient.
  2. The QSO concept is sound. The possibility of preparing several queues covering a wide range of possible sky conditions (absorption and Image Quality) in advance of an observing night is essential for imaging in the visible, where seeing can change quite a lot, and very quickly. Thanks to the flexibility allowed by the QSO mode, a large fraction of the exposures were taken within constraints. The ensemble of QSO tools allows also the quick preparation of queues during an observing night for adaptation to variable conditions, or in case of unexpected overheads. The QSO mode also makes possible time constrained programs such as the CFHTLS. As usual, for 2008A, the global validation rate (validated/observed) for MegaPrime was excellent (92%).
  3. QSO is well adapted for time constrained programs.The Phase 2 Tool allows the PIs to specify time constraints, even very restrictive ones. Time constrained programs complicate the planning and scheduling but CFHT provides tools for both PIs and Queue Coordinators.
  4. Very variable seeing and non-photometric nights represent the worse sky conditions for the QSO mode with MegaPrime. Snapshot programs and regular programs requesting mediocre conditions (1" to 1.2") are used to cover those conditions. Fields requesting photometric sky conditions but originally done during non-photometric conditions are calibrated on perfect nights. SkyProbe and real-time measurements of the transparency are used to decide what observations should be undertaken.

WIRCam

The 2008A semester for WIRCam went very well, with some periods of bad weather, but not affecting observations as badly as for MegaPrime.

  1. Technically, the entire chain of operation, QSO --> NEO --> TCS, is efficient and robust. The time lost to the NOP chain is very small. Certain operational modes specific to WIRCam, like nodding (target-sky-target...) and chip-to-chip dithering, have higher, unavoidable, overheads but some of them are charged directly to PIs during Phase 2.
  2. The QSO concept is sound. As with MegaPrime, the possibility of preparing several queues covering a wide range of possible sky conditions (absorption and Image Quality) in advance of an observing night results in a very large fraction of the observations done within the specifications. For WIRCam, the sky background is more of a factor although its global variation through the night on Mauna Kea is fairly well known. Seeing is of course another important parameter but variations during the night in the near-IR are generally not as brutal as in the visible.
  3. QSO is well adapted for time constrained programs.The Phase 2 Tool allows the PIs to specify time constraints. We can handle those easily if the weather is cooperative although the introduction of time constrained observations on a large-scale adds up definitive complexity in the scheduling process.
  4. Non-photometric nights represent the worse sky conditions for the QSO mode with WIRCam. An important difficulty on near-IR astronomy is the removal of the sky background. Non-photometric conditions make that operation a more difficult one. Nodding for instance cannot be done. The availability of SkyProbe and real-time measurements of the transparency is extremely valuable and regularly used do decide what observations should be undertaken. Also, the real-time analysis through `I`iwi provides a direct estimate of the extinction through the 2MASS catalog, helping even more the observing process.

ESPaDOnS

The 2008A semester with ESPaDOnS was a great success. The only major issue was a software bug which caused the instrument to be incorrectly configured for the 4th exposure of each polarimetric set of 4 (spectroscopic observations were not affected). We re-observed all targets affected (as much as we could), at no charge to the PIs, and reduced the incomplete sets of data with only 2 exposures (again, at no "cost" to the PIs). CFHT also implemented improvements to the guiding algorithm, and to the telescope focussing procedure. For the first 2-3 nights of an ESPaDOnS run, Service Observers perform a 2-step focus sequence using exposures from the ESPaDOnS guider. The data gathered during those sequences are used to update a telescope focus model which is then used for the rest of the run, without having to refocus on each target. An automatic Image Quality measurement routine has also been developed to get a measurement of the seeing on each target.

  1. Technically, the entire chain of operation, QSO --> NEO --> TCS, is efficient and robust. The time lost to the NOP chain is small, but there is some room for improvements. Most of the overheads come from acquiring (finding) the targets, performing full telescope focus sequences for the first 1-3 nights of a run, measuring the Image Quality, and initiating the guiding.
  2. The QSO concept is sound. For ESPaDOnS, and in contrast to our imagers, seeing and extinction are much less of an issue and do not factor much in the preparation of the queues. Queue Coordinators usually prepare one or at most 2 queues per night. The advantage of the QSO mode comes from the ability to schedule observations exactly when they are needed.
  3. QSO is well adapted for time constrained programs. ESPaDOnS observations are characterized by a high demand for time constrained observations and monitoring requirements: about half of the programs had special requests for timing. The queues are usually prepared by taking first into account programs with strict time windows. It was not unusual to juggle a program with a target to observe every night, a second program with a target to observe 2-3 times a night with 1-hr gaps between Observing Groups, a third program with a target to observe every 2 or 3 or 4 nights, and 2 programs to execute within a certain time window from one another. There are also programs that necessitates continuous blocks of 4 to 8 hours during the same night. The PH2 tool allows PIs to specify all sorts of time constraints, and add any comment to help the QSO Team select appropriate programs.
  4. ESPaDOnS can deal with non-photometric nights and bad Image Quality Except for very faint targets (fainter than about 13) which can be difficult to find and center with cloudy conditions or highly degraded seeing, most observations can be carried under a very wide range of sky conditions. Extinction and bad seeing reduce the amount of flux getting into the instrument, but the Service Observers compensate by repeating Observing Groups (at no cost to the PIs) to recover some of the lost flux.


C - Global Statistics, Program Completeness, and Overheads

(1) Global Statistics

The following table presents some general numbers regarding the queue observations for 2008A (C, F, H, L agency for MegaPrime, and T, D-time, excluding snapshot programs unless noted otherwise). Note: to convert from number of hours to number of nights, a factor of 9.5 hours per night was used.

Parameter MegaPrime WIRCam ESPaDOnS
Total number of Nights79 50 41
Hours/Nights lost to weather~234hrs or ~25nights (31%) ~61hrs or ~6.5nights (13%) ~42hrs or ~4.4nights (11%)
Hours/Nights lost to (engineering + technical) problems~12hrs or ~1.3nights (1.6%) ~12.5hrs or ~1.3nights (2.5%) ~27.5hrs or ~2.9nights (7%)
QSO Programs Requested38 (A/B/C) + 5 Snapshots 27 (A/B/C) + 3 Snapshots 25 (A/B/C) + 0 Snapshots
QSO Programs Started35 (A/B/C) + 4 Snapshots 24 (A/B/C) + 0 Snapshot 24 (A/B/C)
QSO Programs Completed25 (A/B/C) + 1 Snapshot 22 (A/B/C) + 0 Snapshot 14 (A/B/C)
Total I-time requested (hr.) (A+B+C)455 294 297
Total I-time validated (hr.) (A+B+C)390 (86%) 287 (98%) 248 (84%)
Completion A+B Programs92% 97% 82%
Queue Validation Efficiency A+B programs92% 95% 86%

Notes concerning MegaPrime

Notes concerning WIRCam

Notes concerning ESPaDOnS

This was the first semester with ESPaDOnS in QSO mode, and significant differences were found in terms of queue preparation, handling of priorities (grades/ranks), and particular scheduling difficulties.

(2) Program Completeness

MegaPrime

The figure below presents the completion level for all MegaPrime programs in 2008A, according to their grade:

Remarks:

WIRCam

The figure below presents the completion level for all WIRCam programs in 2008A, according to their grade:

Remarks:

ESPaDOnS

The figure below presents the completion level for all ESPaDOnS programs in 2008A, according to their grade:

Remarks:

(3) Overheads

MegaPrime

The following table presents the main operational overheads (that is, other than readout time of the mosaic) with MegaPrime. These numbers have not changed, and are given as a reference. Overheads are highly variable during a given night depending on the conditions, complexity of science programs, etc. The table below shows that overheads take a maximum of ~35min per night. A short summer night lasts about 9 hours with MegaPrime, so overheads take less than 10%. The number originally expected was 10-15%.

Event Events/night Overhead Total overhead per night
Filter Change ~12 / night 90s /change 1115 seconds
Focus Sequence ~ 0 / night 0 seconds
Dome Rotation > 45deg 5 ? 120s < 600 seconds
Guide Star Acquisition 20 - 30 ? 20s / acq < 600 seconds

Notes:

WIRCam

The following table presents the main operational overheads (that is, other than readout time of the mosaic) with WIRCam. Overheads are highly variable during a given night depending on the conditions, complexity of science programs, etc. The table below shows that overheads take a maximum of ~25min per night. A short summer night lasts about 9.5 hours with WIRCam, so overheads take around 5%, which is quite low.

Event Events/night Overhead Total overhead per night
Filter Change 15 / night 15s /change 225 seconds
Focus Sequence 2 / night 65s / seq 130 seconds
Dome Rotation > 45deg 5 ? 120s < 600 seconds
Acquisition 36 / night 12.5s / acq 455 seconds

ESPaDOnS

The following table presents the main operational overheads (that is, other than readout time of the mosaic) with ESPaDOnS. Overheads during a given night depend mostly on the number of targets and the need to change Observing Mode or not. The table below shows that overheads can take up to 1hr per night, most of that coming from the acquisition stage (pointing the telescope, finding the target, centering the target, starting the guiding). A short summer night lasts about 9.5 hours with ESPaDOnS, so overheads take around 10%.

Event Events/night Overhead Total overhead per night
Readout Mode Change 3 0 0
Obs Mode Change 1 to 3
(< 1 on average)
7-8min / change 4-5min / night
on average
Telescope Focus Sequence ~ 1-1.5 / night
on average
120-180s / focus seq. 160 - 240 seconds
Image Quality Measurement ~ 6 - 12 / night
on average
30 - 60s / measurement 180 - 600 seconds
Dome Rotation > 45deg 5 ? 120s < 600 seconds
Star Acquisition
(pointing, finding)
12 / night
on average
1-5 min / acq 10 - 30 min

Notes:



D - Agency Time Accounting

(1) Global Accounting

Balancing of the telescope time between the different Agencies is another constraint in the selection of the programs used to build the queues. After this first semester with ESPaDOnS in QSO mode, it was found that this is much more difficult to do with ESPaDOnS than with the other 2 instruments, because a lot of ESPaDOnS programs request strict time constraints or have narrow windows of execution, which takes precedence over Agency Balancing. The queues are basically done according to the requested time constraints and try to follow the programs' ranks as much as possible. The agency balance is whatever it ends up being at the end of the semester. This being said, the agency balance for ESPaDOnS was quite acceptable for 2008A.

MegaPrime

The figure below presents the Agency time accounting for 2008A.The top panel presents the relative fraction allocated by the different agencies (programs A + B), according to the total I-time allocated from the Phase 2 database. The bottom panel represents the fraction of observations validated (programs A+B+C) for the different Agencies, that is, [Total I-Time Validated for a given Agency]/[Total I-Time Validated]. As showed in the plots, CFHTLS is the agency lagging behind, with less validated I-Time than allocated. The significant fraction of time lost to bad weather (~30%) contributed to making the agency balance difficult to obtain.

WIRCam

The figure below presents the Agency time accounting for 2008A.The top panel presents the relative fraction allocated by the different agencies (programs A + B), according to the total I-time allocated from the Phase 2 database. The bottom panel represents the fraction of observations validated (programs A+B+C) for the different Agencies, that is, [Total I-Time Validated for a given Agency]/[Total I-Time Validated]. As showed in the plots, the Taiwanese agency is well ahead while CNRS is slightly behind.

ESPaDOnS

The figure below presents the Agency time accounting for 2008A.The top panel presents the relative fraction allocated by the different agencies (programs A + B), according to the total I-time allocated from the Phase 2 database. The bottom panel represents the fraction of observations validated (programs A+B+C) for the different Agencies, that is, [Total I-Time Validated for a given Agency]/[Total I-Time Validated]. As showed in the plots, the balance is good, with the French agency a little ahead, the Taiwanese ahead, and Canada behind (mostly because of the 30-hr program that could not be started).

(2) CFHTLS Accounting

CFHTLS occupies a large fraction of the I-time allocated for QSO for MegaPrime. The following figures show the time accounting for the different CFHTLS components for 2008A (L01 and L04 are the Deep Survey, L02 and L05 are the Wide Survey, L03 is the Very Wide Survey, and L99 is the photometric grid for the Wide Survey):

Fraction Requested
Fraction Validated

After grouping together the programs for each component of the CFHTLS, the numbers confirm what is seen in the plots: the photometric grid is well behind with respect to the surveys themselves. The global fractions are presented in the following table:

Survey Programs Fraction requested Fraction validated
Deep Synoptic L01 + L04 46.5% + 3.3% =
49.8%
48.5% + 4.3% =
52.8%
Wide Synoptic L02 + L05 28.8% + 9.9% =
38.7%
32.1% + 10.1 =
42.4%
Very Wide Synoptic L03 1.6% 1.8%
Photometric Grid
for the Wide
L99 9.9% 3.3%


E - Conclusions

MegaPrime

The 11th semester with MegaPrime in QSO mode was overall very good. Bad weather again hit MegaPrime the hardest, but overheads have been decreased, the completion of A and B programs and agency balance are very good.

WIRCam

The 6th semester with WIRCam in QSO mode was excellent. The programs completion and efficiency are the highest of the 3 instruments.

ESPaDOnS

The first semester with ESPaDOnS in QSO mode, after 3 years of Classical use, went very well. The transition went very smoothly and a minimum of time was lost to engineering. New functionalities, such as automated telescope focus sequences, telescope focus model, and automated Image Quality measurement, were successfully developed and implemented, insuring the best data quality possible. Preparing queues with ESPaDOnS turned out to be completely different than preparing queues for our two imagers, because (1) ESPaDOnS does not care as much about Image Quality and amount of extinction, and (2) many programs have requests for time constraints.