Downloading Files

Updated June 2023

General Information

Your personal science data and associated metadata files can be downloaded directly from CFHT. This is performed through the use of a unique Data Access Key (DAK), a randomly-generated alphanumeric string assigned to your observing program. Your data access key is listed in the "Exposures" section of the Kealahou K2 interface for your program. If your program does not use Kealahou K2, CFHT would have provided you this key via email directly or as part of a URL for 'quicklook' data access. If you have multiple programs, or a single program split across multiple semesters or Program IDs, you will receive a separate data access key for each one.

We recommend that you keep your key(s) secret, as anyone with access to the key will be able to download the science data. Note that any co-I who have access to a program's Kealahou K2 interface also have access to the data access key. Should you choose to share your data access key with any other colleagues or collaborators, CFHT is not responsible for verifying their identities, nor will CFHT re-generate new data access keys to block access to anyone who was previously provided a data access key by you.

CFHT does not keep data available for download for indefinitely. Therefore, once available to you, we recommend that you download your processed and raw data (if desired) as soon as possible. Note that in addition to being initially available from CFHT, your data is permanently archived at the Canadian Astronomy Data Centre ↗. If you need to (re)download your data more than a year after it was observed, we recommend you do so through CADC.

Please note: For some instruments, files are compressed using the RICE algorithm in the CFITSIO library. Files compressed this way usually have a .fz suffix. Several decompression utilities exist for this format, including imcopy which is distributed with CFITSIO ↗.

Downloading Options

There are three options for accessing and downloading your data: through the CFHT Direct Data Distribution Tool web application, or directly via Wget or rsync commands. Which option you use is solely your preference. If using the Wget or rsync commands, you are free to automate their use within your own scripts and programs, but note that you are solely responsible for their operation. Should CFHT detect improper use of these commands or privileges, such as rapid, repeated, or incessant download requests, we reserve the right to revoke your access privileges.

CFHT Direct Data Distribution Tool

To access and download your files through the Direct Data Distribution Tool, navigate to the following URL: https://share.cfht.hawaii.edu/fileshare.

Once at the Tool, input your data access key into the proper field, and select whether you would like to view the HTML webpage associated with your program, or simply show a list of available files.

Note that only ESPaDOnS, MegaCam, and WIRCam have HTML webpages associated with each program. If you are accessing data from a SPIRou or SITELLE observing program, only the file tree will be displayed, which you will have to navigate manually.

If you choose to show a list of available files, these will appear within the Tool; you will not be directed to the HTML page for your program (should one exist). Additionally, the Tool will generate the proper Wget and rsync commands that can be used to download your data. Details on these commands are below.

Wget

To download files using Wget, you must have it installed on your system. Wget is available here ↗. You are responsible for the installation and maintenance of the software on your system.

Once installed, you can download your data through the terminal/command line of your system. To do this you will need to use your data access key (DAK), and the following URL: https://share.cfht.hawaii.edu/. Your data access key must be appended to the URL when supplied to Wget. An example of the usage is below.

This example shows the syntax needed to retrieve FITS files from your CFHT data repository using Wget. The command can be run repeatedly to retrieve new files.

In the example, [DAK] is your data access key. [area] is a particular subdirectory, such as "quicklook". [area] is not required.

wget -r -nc -c -A.fits https://share.cfht.hawaii.edu/[DAK]/[area]

Useful options shown above include:

rsync

If you prefer not to, or are unable to, install Wget, you can download your data files using rsync. rysnc is available on most modern operating systems.

To use rsync, you will need to use your data access key (DAK), and the following URL: share.cfht.hawaii.edu/. Your data access key must be appended to the URL when supplied to rsync. An example of the usage is below.

In the example, [DAK] is your data access key. [location] is the location on your system where you want the files to be copied to.

rsync -va --relative share.cfht.hawaii.edu/::[DAK]/ ./[location]/

Useful options shown above include:

Note that unlike for Wget, you do not need to specify any "continue" option to resume download of partial files. rysnc will automatically resume these without specification.

Need Help?

If you require assistance with downloading your data, contact the CFHT QSO Team.