sdod0060 p guide to sdo data analysis sdoguide

Guide to SDO Data Analysis

edited by

Marc DeRosa & Greg Slater

February 19, 2013

Contents

1 Introduction 3

1.1 Synopsis of the SDO Mission. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2 About this Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 3

1.3 Data Products from AIA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

1.4 Data Products from EVE. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

1.5 Data Products from HMI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .4

2 How to Browse SDO Data 10

2.1 “The Sun Now” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . 10

2.2 “The Sun Today” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 10

2.3 SolarMonitor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 10

2.4 The Helioviewer Project. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

3 How to Find SDO Data 17

3.1 Heliophysics Events Knowledgebase. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

3.2 iSolSearch. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . . . 17

4 How to Get AIA and HMI Data 234.1 How Data are Organized at the JSOC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

4.2 ThelookdataTool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . 23

4.2.1 Getting Dataseries Names. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4.2.2 Selecting Records. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24

4.2.3 Exporting Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 27

4.2.4 JSOC Image Timing Details. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

4.3 The Cutout Service. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 29

4.4 The Virtual Solar Observatory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

4.5 Synoptic Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . . . . 31

4.6 Uncompressing the Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31

4.6.1 Compiling and Installingimcopy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5 How to Use SolarSoft with SDO Data 41

5.1 Installing or Upgrading SolarSoft. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.1.1 Doing a Clean Install. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

5.1.2 Updating Packages for an Existing Installation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

1

Guide to SDO Data Analysisversion dated February 19, 2013

5.1.3 Using a Proxy Server with SSW. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

5.2 Querying the HER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. . . 43

5.2.1 Basic Query Syntax. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

5.2.2 Adding Filters to Queries. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

5.3 Retrieving Data from the JSOC. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.3.1 Getting Dataseries Names. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

5.3.2 Selecting Records. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49

5.3.3 Exporting Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 52

5.3.4 Exporting Specific Segments. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

5.4 Requesting a Cutout Using SSW. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

5.5 Using the SSW Interface to the VSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5.5.1 Querying the VSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5.5.2 Downloading Data from the VSO. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

5.6 Uncompressing the Image Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5.6.1 Usingread_sdo.pro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

5.6.2 Usingread_sdo.pro with a Shared Library. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

6 How to Process AIA Data 656.1 Converting AIA Data to Level 1.5. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

6.2 Co-Aligning HMI Data with AIA Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

6.3 Despiking/Respiking AIA Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

6.4 Point Spread Function Deconvolution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

6.5 Getting AIA Filter Response Data. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

6.6 Miscellaneous AIA Tasks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76

6.6.1 Using AIA Standard Scaling and Colors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

6.6.2 Creating AIA Tri-Color Images. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

6.6.3 Plotting an AIA Light Curve. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

6.6.4 Interfacing withplot_map.pro (Dom Zarro’s Mapping Software). . . . . . . . . . . . . . . . . . . . . . . 78

6.6.5 Checking theQUALITY Keyword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

7 Frequently Asked Questions 81

8 Lists of Useful Links 84

9 Version History of this Guide 86

10 Contributors to this Guide 89

PDF version of this guide

2

https://www.lmsal.com/sdodocs/doc?cmd=dcur&proj_num=SDOD0060&file_type=pdf


1 Introduction

1.1 Synopsis of the SDO Mission

The Solar Dynamics Observatory(SDO) is the first mission to be launched under NASA’sLiving With a Star(LWS) program. This

program involves a diverse set of research topics that aim toprovide a greater understanding how the activity and variability of the Sun

affect life on Earth. The purpose of SDO is to understand how the Sun’s magnetic field is generated and structured, and how this stored

magnetic energy is converted and released into the heliosphere and geospace in the form of solar wind, energetic particles, and variations

in the solar irradiance.

SDO waslaunchedaboard an Atlas V rocket on 11 February 2010 from Kennedy Space Center in Florida. It is now located in a

nearly geosynchronous orbitthat allows continuous contact with its dedicated ground stations in New Mexico. The observatory has been

operating nominally since launch, without any major problems.

The observatory contains three instruments (presented in alphabetical order):

• TheAtmospheric Imaging Assembly(AIA) is an array of 4 telescopes that together provide full-disk images of the solar corona

at 1′′ resolution (4096×4096-pixel images) in 10 UV and EUV wavelengths every 10 seconds;

• TheExtreme ultraviolet Variability Explorer(EVE) measures the EUV spectral irradiance with unprecedented spectral resolution,

temporal cadence, accuracy, and precision; and

• The Helioseismic and Magnetic Imager(HMI) provides full-disk, high-cadence Doppler, intensity, and magnetic images at 1′′

resolution (4096×4096-pixel images) of the solar photosphere, allowing studies of the sources and evolution of activity within the

solar interior.

Together, this suite of instruments enables the monitoringof the solar interior, chromosphere, and corona with high spatial and contin-

uous temporal coverage, as described in theSDO overview paper(Pesnell et al. 2012; appearing in the SDO issue ofSolar Physics). Low-

level processing and calibration for data from the two imaging instruments (AIA and HMI) occurs at theJoint Science Operations Center

(JSOC), located on the campus of Stanford University. EVE data are processed at the Laboratory for Atmospheric and SpacePhysics

(LASP) at the University of Colorado. Brief descriptions ofthe data products produced by each of the three SDO instruments are given

below in Sections1.3, 1.4, and1.5.

1.2 About this Guide

The massive amount of image data produced by SDO (of order several terabytes per day) requires new techniques for processing,

browsing, retrieving, and analyzing these data. Indeed, significant samples of full-disk, full-resolution images from either AIA or HMI

can easily exceed hundreds of gigabytes. Consequently, remote users will not be able to download very much SDO image databefore

it is realized that either one’s internet bandwidth is not sufficient to download the amount of data requested in a reasonable amount of

time, or one’s available on-site computing power and storage are not sufficient to interact or process the data once it resides locally (or

both).

The purpose of this guide is to demonstrate the ways in which users can more readily progress through the sequence of browsing,

finding, downloading, and analyzing SDO data. To that end, the guide contains sections describing:

• How to browse data: Various “Sun-in-time” pages, summary images and movies, and popular event and feature listings (Sec-

tion 2)

• How to find data: Heliophysics Event Knowledgebase, a searchable databasethat is useful for finding events and features of

interest during (and prior to) the SDO era for which the datesare not known (Section3)

• How to get data: Multiple interfaces to the data centers through which SDO data can be downloaded (Section4)

3

http://sdo.gsfc.nasa.gov

http://lws.gsfc.nasa.gov

http://www.youtube.com/embed/AIBr3wlL4X8

http://www.wolframalpha.com/input/?i=sdo+spacecraft

http://aia.lmsal.com

http://lasp.colorado.edu/eve

http://hmi.stanford.edu

http://www.springerlink.com/content/w687736278532677/

http://jsoc.stanford.edu


• How to find and get data using SolarSoft: Demonstration of SolarSoft IDL commands useful for findingand downloading data

(Section5)

• How to perform common AIA data-processing tasks: Demonstration of how to manipulate AIA data for scientific research

(Section6)

• Frequently Asked Questionslist (Section7), list of useful links (Section8), and theversion history of this guide (Section9)

The guide is a living document, and materials will be added asneeded. The online version of this guide can be found either at

http://www.lmsal.com/sdouserguide.html, or by navigating to theSDO Documentation(SDOdocs) webpage where it is the first link in

the list. Alternatively, aPDF versionof this guide is also available. Both versions of the guide contain numerous links in order to make

finding SDO-related information easier. In the PDF version,such links are active, with external links colored blue, andlinks to other

items in the guide colored red.

If you have questions, problems, or other feedback related to the analysis of SDO data or other related topics that do not seem to be

covered in this guide, please send an e-mail to the address listed onthis webpage.

1.3 Data Products from AIA

AIA obtains full-Sun images in multiple EUV and UV passbands, as summarized in Figure1 and described in more detail on the

AIA Instrument webpage. These images are used to generate higher-level data products, including browse-quality data and feature and

event entries in the Heliophysics Events Knowledgebase (see Section3.1), all of which in turn support the science requirements of AIA.

A summary of this process appears in the flowchart presented in Figure2. For more details about AIA, refer to theAIA instrument paper

(Lemen et al. 2012; appearing in the SDO issue ofSolar Physics).

The different data-processing levels for AIA are summarized as follows (more details are available fromthis documenton SDOdocs):

• Data atLevel 0are images that have been constructed from the raw telemetrystream.

• Data atLevel 1.0are images that have been converted from Level 0, with processing including bad-pixel removal, despiking and

flat-fielding. All higher-level products for AIA are based onLevel 1.0 data.

• Data atLevel 1.5are images that have been adjusted to a common 0.6′′ plate scale, and that share common centers and rotation

angles, but arenot exposure-time corrected.

1.4 Data Products from EVE

EVE produces a multitude of spectral and photometer data products that support theEVE primary science objectives, as discussed in the

EVE instrument paper(Woods et al. 2012; appearing in the SDO issue ofSolar Physics). EVE data products are summarized in Figure3

and available through theEVE data access webpage.

1.5 Data Products from HMI

HMI produces a multitude of data products that support the HMI science investigation. The four main observables are: Dopplergrams

(maps of line-of-sight photospheric velocity), continuumimages (images of the solar photosphere near the 6173Å absorption line of

Fe I), and both line-of-sight and vector magnetograms (mapsof the photospheric magnetic field).

The images of Doppler velocity comprise the input for both global and local helioseismology analyses. Time series of these images

are used to infer a mean rotation and sound speed in the solar interior. Such time series are also used to sense the presenceof active

regions either before they have rotated onto the Earth-facing half of the sun or before they have fully emerged onto the surface. The

magnetic data products include line-of-sight and vector magnetic maps, synoptic maps, the locations of surface activity, and related

space-weather indicators. Parameters and data cutouts associated with HMI Active Region Patches (HARPs), such as geometric extent,

total flux, etc., have also been catalogued. Please refer tothis webpagefor additional information regarding HMI magnetic data, in

4

http://www.lmsal.com/sdouserguide.html

https://www.lmsal.com/sdodocs


http://www.lmsal.com/hek/her?cmd=contact

http://aia.lmsal.com/public/instrument.htm

http://www.springerlink.com/content/g71494170q64jpt1/


http://lasp.colorado.edu/home/eve/science/

http://www.springerlink.com/content/m3811106121h2575/

http://lasp.colorado.edu/home/eve/data/data-access

http://hmi.stanford.edu/magnetic/


particularthis webpagefor detailed examples of HARPs, and lastlythis webpagefor a summary of the vector magnetic field processing

scheme.

The different data-processing levels for HMI are summarized as follows, with more details availablefrom this source:

• Data atLevel 0are images that have been constructed from the raw telemetrystream.

• Data atLevel 1.0are images that have been converted from Level 0, with processing including bad-pixel removal, flat-fielding,

and quality assessment checks, but otherwise not having undergone any irreversible data alterations.

• Data atLevel 1.5are images of the physical observables (Dopplergrams, magnetograms, and continuum images), which were

constructed from the individual Level 1.0 filtergrams.

• Data atLevel 2 have been irrevocably filtered, time-sequence-merged, Fourier-transformed or otherwise changed from Level 1.5

data in a way that is irreversible. Level 2 products include intermediate products for later production of mission science data

products, such as helioseismic inferences of solar subsurface flows.

For more details and specifications, see theHMI instrument paper(Scherrer et al. 2012; appearing in the SDO issue ofSolar Physics),

theHMI instrument overviewwebpage at Stanford, or the accompanying analysis pipelineflowchart in Figure4.

5

http://jsoc.stanford.edu/jsocwiki/HARPDataSeries

http://jsoc.stanford.edu/jsocwiki/VectorMagneticField

http://jsoc.stanford.edu/jsocwiki/Processing

http://www.springerlink.com/content/n0k3765711577156/

http://hmi.stanford.edu/Description/hmi-overview/hmi-overview.html


UV

17194

304

Fe XVIII

He II

335

131193

211

Fe XIV

Fe XII / XXIV

+Y

+Z

Fe XVI

12Fe IX

34

Looking at the AIA from the Sun

4500 White Light

1700 UV Cont.

http://aia.lmsal.com/

AIA Telescopes, Wavelengths, and Thermal Response

Fe VIII / XXI

1600 C IV + Cont.

tmo

sp

hericImaging Assem

bl yA

IX

XIIXIV XVI

XVIII

XX

L M S A L

NA

S A S

AO

Figure 1: A summary of the passbands of AIA, showing the filterresponses of each of the EUV channels, and their arrangementon the

instrument as viewed from the sun (source: “Summary of AIA telescopes, wavelength channels, and thermal responses” at SDOdocs).

See also Section6.5.

6



Figure 2: AIA science analysis pipeline, illustrating how the main filtergram data, along with data from EVE and HMI produce the data

products that support the science goals of AIA. Note that AIALevel 1.5 data arenotcurrently being corrected to a standardized exposure

time (as is incorrectly indicated in the figure).

7


Figure 3: Table of EVE data products (sourced here), showing the range of wavelengths covered, their resolution, and the different levels

of processing applied.

8

http://lasp.colorado.edu/eve/data/data_product_summary.htm


Figure 4: HMI science analysis pipeline flowchart (sourced here), showing how the four main observables are used to create the many

data products that support the science objectives of the HMIinstrument.

9

http://hmi.stanford.edu/Requirements/Science_Pipeline.jpg


2 How to Browse SDO Data

In this section, we briefly illustrate a few of the online resources that allow users to browse SDO data: “The Sun Now” (Section 2.1),

“The Sun Today” (Section2.2), SolarMonitor (Section2.3), and Helioviewer and JHelioviewer (Section2.4).

2.1 “The Sun Now”

TheSDO mission webpagehosted by NASA’s Goddard Space Flight Center contains a wealth of information about SDO. The first entry

under the “Data” tab is “The Sun Now”, which contains thumbnail images, links to full-size and downsampled images, movies, and

composite images from AIA and HMI. Also present are a soft X-ray image from the pinhole camera on EVE as well as a heliographic

representation of the corona assembled from the two STEREO spacecraft and AIA. The other entries under the “Data” tab provide

additional functionality, including templates that enable users to generate a customized image sequence or movie. Thetop portion of

“The Sun Now” webpage is shown in Figure5. Users may also find useful the array ofself-updating 48-hr movies.

2.2 “The Sun Today”

“The Sun Today”, located athttp://sdowww.lmsal.com/suntoday , is a webpage summarizing images, movies, events, and

features that occurred on the Sun through any given day in theSDO mission. The page keys off the date indicated at the top ofthe page,

which can be changed to any date of interest either by using the plus and minus buttons in the upper-right corner of the webpage, or by

entering a URL with thesuntoday_date parameter set to the date of interest. For example, the summary page for 1 August 2010 is

located atthis link (http://sdowww.lmsal.com/index.html?suntoday_date=20 10-08-01 ), as illustrated in Figure6.

The page layout (currently) includes three panels that contain: a series of AIA and HMI images for the day of interest, theresults of an

iSolSearch query (see Section3.2) for that day, and a plot of AIA light curves for that day.

The image panel contains a row of four preview images, under which are lists labeling various wavelength channels from AIA and

HMI. In each column of labels, the label of the channel that isunderlined corresponds to the preview image atop the column. Mousing

over any of the other wavelength channel labels changes the preview picture, enabling users to compare images from up to four channels

at once.

Clicking on the links next to each wavelength channel will open a new browser window containing a larger image from that channel.

For example, the “1K” link will open a 1024×1024-pixel image, the “4K” link will open a 4096×4096-pixel image, and the “PFSS”

link will open a 4096×4096-pixel image with fieldlines from a potential-field source-surface model overlaid. Links to summary movies

of four of the wavelength channels appear above the row of preview images. There is also a link to an interactive fieldline viewer that

allows the user to manipulate a rendering of pre-computed PFSS fieldlines from within WebGL-enabled web browsers.

To put these images in some context, the results of an iSolSearch query and a series of selected AIA light curves for the date of

interest are displayed in the panels underneath the image preview panel. For more information on iSolSearch, as well as instructions

describing how to use the full search tool, please see Section 3.2.

2.3 SolarMonitor

The SolarMonitorwebsite provides a straightforward way to browse a variety of solar imagery, including magnetograms, continuum

images, Hα images, and images in ultraviolet and X-ray passbands. The front page for 3 August 2010 is shown in Figure7 below.

Zoomed-in summary images for each NOAA-identified active region are also available (search for these first by date, and then use the

links along the left-hand sidebar or underneath the image display to view the active-region summaries). Links along theright-hand

sidebar will take the user to external websites that provideother ways to monitor the sun, space weather, and the heliosphere.

2.4 The Helioviewer Project

The helioviewer.orgdata browser is a user-friendly way to align and display multiple images from a selection of recent solar and

heliospheric observatories, as shown in Figure8. The tool consists of a web-based interface that enables users to interactively pan and

10


http://sdo.gsfc.nasa.gov/data/SDO_Self_Updating_6.htm

http://sdowww.lmsal.com/suntoday

http://sdowww.lmsal.com/index.html?suntoday_date=2010-08-01

http://www.solarmonitor.org

http://www.helioviewer.org


Figure 5: The top portion of the “The Sun Now” webpage showingthe array of AIA image channels.

11


Figure 6: The top two sections of “The Sun Today” webpage for 1August 2010, showing the assortment of preview images as well as

the results of an iSolSearch query for that day.

12


Figure 7: The SolarMonitor.org front page for 3 August 2010.

13


zoom an image, overlay images from different observatorieswith the proper registration, and indicate the location of features and events

of interest. The interface downloads image and event data asynchronously (usingAJAX), which allows the user to continue browsing

while the tool downloads additional data. For more information, please visit the website and/or refer to theHelioviewer.org User Guide.

JHeliovieweris a Java/OpenGL-based application to browse time series ofsolar images. JHelioviewer overlays image series from

SOHO and SDO in space and time and makes use of the JPEG2000 standard to stream large datasets interactively in real-time.Users

can, for example, zoom and pan full-size SDO movies, apply basic image processing methods (including feature tracking)and overlay

event markers from the Heliophysics Event Knowledgebase (see Section3.1). JHelioviewer is open-source and has a plugin architecture

that allows users to expand its functionality. More information can be found by visiting the website, and in particular reading the

JHelioviewer Handbook.

14

http://en.wikipedia.org/wiki/Ajax_%28programming%29

http://helioviewer.org/wiki/index.php?title=Helioviewer.org_User_Guide

http://jhelioviewer.org

http://helioviewer.nascom.nasa.gov/wiki/index.php?title=JHelioviewer_Handbook


Figure 8: The helioviewer.org display showing an eruption visible in LASCO C2 and C3 images, as well as a (mostly) co-temporal and

co-aligned image from the AIA 171Å channel. The three imagesare all taken within 15 minutes of 2 August 2010, 00h UT.

15


Figure 9: A screen shot of the JHelioviewer application illustrating a zoomed-in portion of an AIA image with icons associated with

Heliophysics Events Knowledgebase overlaid on the image.

16


3 How to Find SDO Data

In this section, we will demonstrate how to use theHeliophysics Events Knowledgebase(HEK). The HEK is essentially a list of ob-

servational sequences combined with a catalog of features and events that have occurred on the sun. It is useful in situations when the

user knowsthat a particular type of event occurred, but who cannot rememberexactlywhen. It is also useful in situations when the user

wants to know whether a particular sequence of observation has occurred.

The HEK allows users to report new features and events, and tocontribute information on existing ones. Consequently, the HEK

is useful in an environment where browsing large amounts of data is cumbersome (as will be the case during the SDO era), andallows

users to focus on establishing connections between similarevents and features. More information on the HEK is available in this article

(Hurlburt et al. 2012; appearing in the SDO issue ofSolar Physics).

3.1 Heliophysics Events Knowledgebase

The HEK is comprised of two major components: the Heliophysics Events Registry (HER) and the Heliophysics Coverage Registry

(HCR). The HER indicates what features and events have been identified, and the HCR identifies which data sequences are available.

There is an interplay between the HER and HCR (see Figure10), as searches for entries in the HER also reveal related observations

cataloged in the HCR. We now discuss each one in sequence.

As illustrated in the schematic in Figure11, the HER accepts data from a multitude of providers, and various Feature Recognition

Methods (FRMs) operate on these data streams and report events and features of interest. FRMs encompass both automated feature-

finding algorithms as well as human annotators. A consortiumtasked with developing many of the FRMs that supply events tothe HER

is described inthis article(Martens et al. 2012; appearing in the SDO issue ofSolar Physics).

Users access the HER by web portals such as the iSolSearch tool (see Section3.2), or can use the specifications given by the

HEK Application Programming Interface(API). Additionally, for users familiar with SolarSoft, there exist SSWIDL routines for query-

ing with the HER (see Section5.2).

Each feature or event entry in the HEK contains information about its data source (e.g., observatory, instrument, wavelength), its

spatiotemporal location, and the FRM (automated or human) used to identify the item. Thefull list of event classestabulates the required

and optional attributes for each class of event, and it is these attributes than can be used to query the HEK.

Because the name of the FRM is an attribute for each event or feature stored within the HEK, multiple FRMs can generate events

belonging to the same event class. There are many cases wherethe same physical feature has been identified by different FRMs,

but no choice is made as to which one is “better” or “more useful” as such qualitative assessments often depend on their intended

use within a research program. Using coronal mass ejection (CME) events as an example, multiple catalogs of CMEs based ondif-

ferent observatories and/or event specifications (such as,for example, theLASCO (CDAW) CME Catalog, the SEEDS Catalog, the

LASCO ARTEMIS Catalog, theSTEREO COR1 Catalog, and theCactus CME Catalog) are already in use by the community, and it is

likely that the research community has benefited from havingmore than one independently generated catalogs of the same event. The

same benefits can be expected by being able to query events from multiple FRMs seeking to identify each event class in the HEK.

The HCR is a tool that tracks data requests. Users can, for example, view recentor view populardata sequences from AIA, or

view the recent eventsfrom SDO reported by human observers. Users can make more advanced searches using theHCR search form.

As data (especially, the starting and ending dates, instrument, and wavelength) are entered into the HCR search form, the number of

observational sequences that satisfy the query is updated at the bottom of the page.

Clicking on the “Show Search Result Details” button revealsmore details about each observation, including pointing information

and a brief description of the science goal of the observation. These observations should be readily available via the “Get All Data” link

on the summary page for each observation.

3.2 iSolSearch

The easiest way to interact with the HEK is probably by usingiSolSearch, which enables users to easily query the HEK for a listing

of events between two points in time. In this section, the basic usage of iSolSearch is summarized. Alternatively, thereis a brief

17

http://www.lmsal.com/hek

http://www.springerlink.com/content/g244ng5t72866u67/

http://www.springerlink.com/content/t27t647m2hx2701h/

http://vso.stanford.edu/hekwiki/ApplicationProgrammingInterface

http://www.lmsal.com/hek/VOEvent_Spec.html

http://cdaw.gsfc.nasa.gov/CME_list

http://spaceweather.gmu.edu/seeds

http://www.oamp.fr/lasco/

http://cor1.gsfc.nasa.gov/catalog

http://sidc.oma.be/cactus/catalog.php

http://www.lmsal.com/hek/hcr?cmd=view-recent-events&instrument=aia

http://www.lmsal.com/hek/hcr?cmd=view-popular-events&instrument=aia

http://www.lmsal.com/hek/her

http://www.lmsal.com/hek/hcr?cmd=search-events

http://www.lmsal.com/isolsearch


Figure 10: A schematic illustration of the operation of the HEK, showing the interrelationship between the HER and HCR.

YouTube video(with audio) that also demonstrates the basic usage of the iSolSearch tool. Users are encouraged to ask questions in the

YouTube comments section.

To query the HEK and learn what happened on the sun during a particular time frame, simply enter two dates in the search panel,

choose a selection of event classes, and press the “Search” button underneath the event-class listing. Any events in theHEK that satisfy

this query will appear as icons on the disk view panel, as wellas in a listing in the results panel to the right of the disk view. The results

from a typical search are illustrated in Figure12.

Mousing over a particular icon will on the disk view will cause it to be highlighted in yellow in the event listing. Likewise, mousing

over an event in the event listing will enlarge the icon in thedisk view. When clicking on an event (either its icon in the disk view, or its

corresponding entry in the event listing), the tool will display more detailed information about this event in the gray information panel,

as shown in Figure13. There, the user will find a more detailed description of the event, a summary image or movie (if available), and a

link to the underlying XML code (in the “VOEvent XML” link) associated with this event. Users can also add a reference or comment

regarding this event, and rate this event, by clicking on thesmall icons at the top of the information panel.

Observational sequences that overlap the selected event inboth space and time, as found from a HCR search based on the event

properties, are listed under the “Observations in the neighborhood” heading in the information panel. Clicking on one of these related

observational sequences will reveal a summary page as well as a link to related data sequences that already exist (thus allowing users

to avoid longer wait times when requesting data). If none of the HCR observational sequences satisfy the needs of the user, clicking on

18

http://www.youtube.com/watch?v=7RfwXaJvBAo


Figure 11: A schematic illustration of the operation of the HER.

19


the “Request data” link leads to the data-request form for the cutout service (see Section4.3), where data customized by the user can be

requested.

To customize the event display, the disk view panel has clickable buttons that zoom in and out in the upper-left corner, and clickable

buttons that switch between an orthographic and (Carrington) latitude-longitude projections of the sun in the upper-right corner. Zooming

in and out can also be done using the mouse scroll wheel when the cursor is located within main portion of the disk view panel. In the

lower-left corner of the disk view panel, clicking on the arrows will rerun the existing query after adding or subtracting one day (single

arrows) or one week (double arrows) to or from the start and end dates of the existing query. Clicking on the key icon in the lower-right

corner will take the user to the aforementionedlist of event classes, where the icons in the disk view panel are defined in terms of the

corresponding event classes.

More complex queries can be constructed by clicking on the “Filters” tab in the search panel. Users can add multiple filters that

screen for, or screen out, events that satisfy a set of user-defined conditions. As an example, suppose one wants to find allflares having

GOES Class M2 or greater that occurred in the year 2010. Underthe “Search” tab, set the start and end dates to2010-01-01 and

2010-12-31 , respectively, and clear all of the event-class checkboxesexcept for “Flare”. Then, under the “Filters” tab, adding a

constraint likeFL_GOESCls >= M2.0 is achieved by choosingFL_GOESCls (flare GOES class) from the “Attribute” drop-down

menu, setting the operator (“Op”) to>=, choosingM2.0 from the “Value” drop-down menu, and then pressing the “Add”button. After

pressing the “Add” button, the constraint should appear in the white box. After going back to the “Search” tab, press “Search” button to

query the HEK. At the time of this writing,this query returned 18 events.

After the query has produced a listing of results within iSolSearch, the query itself and the associated list of results are available

by clicking on the “export” link at the top of the panel showing the event listing. In the menu that appears, the “Query link” will open

a window containing a URL of the query and the “SolarSoft call” link will open a window containing the same query formattedas

an SSWIDL command using the SolarSoft API (thus allowing theevent listing to be exported to SolarSoft). The listing of results is

available by clicking on the “Table(CSV)” link. In this way,queries and their results can be saved for later use, or published in journal

articles so as to be repeatable by other researchers. Furthermore, users can even subscribe to the query (allowing the listing of results to

be updated, and the user notified) by clicking on the “RSS” link in order to set up anRSS feed.

20


https://www.lmsal.com/isolsearch?hek_query=https://www.lmsal.com/hek/her?cosec=2&&cmd=search&type=column&event_type=fl&event_region=all&event_coordsys=helioprojective&x1=-5000&x2=5000&y1=-5000&y2=5000&result_limit=200&event_starttime=2010-01-01T00:00:00&event_endtime=2010-12-31T23:59:59&param0=FL_GOESCls&op0=>=&value0=M2.0

http://en.wikipedia.org/wiki/RSS


Figure 12: The iSolSearch tool, showing the results of aquery for 1 August 2010. The tool displays an assortment events that happened

on this day, including several filament eruptions, coronal holes, active regions, flares, and even a coronal dimming event. The various

events are displayed as icons (based on event class) in the disk view, and as a list in the results panel to the right of the disk display.

21

https://www.lmsal.com/isolsearch?hek_query=https://www.lmsal.com/hek/her?cosec=2&&cmd=search&type=column&event_type=ar,ce,cc,cd,ch,cj,cr,cw,ef,er,fi,fe,fa,fl,lp,os,sg,sp,ss,pg,ot,nr&event_region=all&event_coordsys=helioprojective&x1=-5000&x2=5000&y1=-5000&y2=5000&result_limit=200&event_starttime=2010-08-01T00:00:00&event_endtime=2010-08-01T05:59:59


Figure 13: The iSolSearch tool, showing the gray information panel that provides more details about the filament eruption event from

the results of the query shown in Figure12. In the right-most panel, more details about this event are shown, including a description,

links to a summary frame and movie, the starting and ending time, observatory, and FRM. Additional links to related resources also

appear, including a link to the cutout request form that willallow users to download the associated data underlying thisevent from AIA.

The icons at the top of the information panel permit users to add references or comments and rate the particular event being displayed.

22


4 How to Get AIA and HMI Data

Low-level processing and calibration for all HMI and AIA observables occurs at theJoint Science Operations Center(JSOC), located

at Stanford University. After such “pipeline” processing has taken place, the JSOC is responsible for cataloging and archiving the data

products and serving it to all users. In this section, the basic elements of the JSOC are briefly described (Section4.1), followed by

descriptions of various ways to retrieve data (Sections4.2, 4.3, and4.4). As the data are typically exported in a compressed format,the

final section (Section4.6) illustrates the various methods of uncompressing and reading in the data. Additionally, for users familiar with

SolarSoft, there exist SSWIDL alternatives for retrievingdata from the JSOC (see Sections5.3, 5.4, and5.5).

4.1 How Data are Organized at the JSOC

To better comprehend what is happening when retrieving data, it is useful to have a general understanding of how the data are cataloged

in the JSOC. The system employed by the JSOC is called theData Record Management System(DRMS), and was created by software

developers at Stanford to manage the large volume of data andmetadata produced by HMI and AIA. In the DRMS, data are organized

into dataseries, which is the name given to a sequence of related or similar items. These items are calledrecords, which in turn are

comprised ofkeywords, their values, and the associated datasegments.

Most dataseries of interest currently published by the JSOCare listed in Table1. While most dataseries are essentially sequences of

images in time, such ashmi.V_45s (HMI Dopplergrams with a 45s cadence),aia.lev1 (AIA Level 1.0 data), as well as historical

data such asmdi.fd_M_96m_lev18 (full-disk MDI Level 1.8 magnetograms with a 96m cadence), this is not always necessarily the

case. Some dataseries contain “derived” data products suchas synoptic or diachronic charts, or cubes of remapped Dopplergrams for

use in helioseismology.

Several points about the DRMS cataloging system are worth mentioning:

• A key aspect of the way the DRMS catalogs data is thatall records of the same dataseries have the same set of allowed keywords.

Of course, for different records these keywords will take ondifferent values. Together, the list of records and list of keywords (and

their values) form a database that is easily searched by DRMS.

• The keywords and their values are stored separately from thedata they describe. This allows the metadata, i.e., the values of the

keywords, to be accessed efficiently (and updated) without having to deal with large image files.

• In some dataseries, such ashmi.V_45s , records contain exactly one image segment (such as the Dopplergram image in this

example). In others, records contain multiple segments of related data. It is important to note that,regardless of how many

segments a record contains, all segments of a record share many, if not most, of the same keyword values. As an example, most of

the records in theaia.lev1 dataseries contain a data segment (the image) and another segment containing a record of pixels that

were changed during the despiking process. Another exampleof dataseries that contain records with multiple segments are the

various HMI vector magnetogram dataseries, for which the HMI Level 1.0 filtergram data are processed to yield various magnetic

field parameters, including|Btotal|, inclination and azimuth angles, filling factors, and uncertainties. Each of these quantities will

be stored in a different image segment, and yet because thesesegments will be part of the same record, they will share the same

set of keyword data.

For more detailed information on the JSOC and DRMS, we refer the reader to theJSOC home page, and in particular theJSOC wiki.

4.2 Thelookdata Tool

The lookdatatool is an online Javascript front end to the DRMS at Stanford, andcan be used to browse names of dataseries, to get

dataseries keyword lists, to examine metadata for sets of records, and to generate export requests. Thelookdatatool is comprised of

several tabs that, when progressed from left to right, allowthe user to locate and export data from the JSOC. Throughout this process,

there are several steps; bold labels in the subsections below correspond to bold labels in the tool. The tool itself provides some additional

help, available by clicking on the yellow or blue question marks that are visible from within the tool.

23


http://jsoc.stanford.edu/netdrms/


http://jsoc.stanford.edu/jsocwiki

http://jsoc.stanford.edu/ajax/lookdata.html


JSOC Dataseries Name Description

aia.lev1 AIA Level 1.0 data

aia.lev1_euv_12s AIA Level 1.0 data for the 7 EUV channels of AIA

aia.lev1_uv_24s AIA Level 1.0 data for the 1600Å and 1700Å channels of AIA

aia.lev1_vis_1h AIA Level 1.0 data for the 4500Å channel of AIA

hmi.B_ * HMI Level 1.5 full-disk disambiguated vector magnetograms(12m cadence)

hmi.Ic_ * HMI Level 1.5 continuum intensity data (various cadences)

hmi.Ld_ * HMI Level 1.5 line depth data (various cadences)

hmi.Lw_ * HMI Level 1.5 line width data (various cadences)

hmi.M_ * HMI Level 1.5 line-of-sight magnetograms (various cadences)

hmi.ME_ * HMI Level 1.5 full-disk Milne-Eddington inversions (12m cadence)

hmi.Mharp_ * ; hmi.sharp_ * HMI active region patch (HARP) cutouts (12m cadence)

hmi.S_720s HMI Stokes parameters (IQUV) data (12m cadence)

hmi.Synoptic * HMI synoptic charts of the radial and line-of-sight photospheric field

hmi.V_ * HMI Level 1.5 Dopplergrams (various cadences)

hmi.fsV * HMI Dopplergram data associated with far-side analyses

hmi.rdV * HMI Dopplergram data associated with ring-diagram analyses

mdi.fd_M * MDI full-disk line-of-sight magnetograms (various cadences)

mdi.fd_V * MDI full-disk Dopplergrams

Table 1: Alphabetical listing of the published dataseries of note available through the JSOC. The asterisk* in the dataseries names is

used here as a wildcard character. The three indentedaia.lev1_ * series are subsets of the mainaia.lev1 series.

4.2.1 Getting Dataseries Names

The first step is to determine the name of the dataseries containing the data you wish to examine (and possibly export). To search for

available dataseries, enter a search string in the “Seriesname filter” field (Step 1) under the “Series Select” tab on thelookdatawebpage,

as shown in Figure14. Pressing the “Fetch seriesname list” button will return a list of all dataseries names containing that search string

in the larger white box to the right of the “Series Select” field. In this white box (Step 2), click on the desired dataseries name from the

list, and thelookdatatool will switch to the “RecordSet Select” tab.

TECHNICAL NOTE: EnteringNOTbefore the search string in the “Seriesname filter” field in Step 1 will return the list of dataseries

names that exclude that search string.

TECHNICAL NOTE: Prepending the caret (circumflex) characterˆ to the search string in the “Seriesname filter” field in Step 1

will return only those dataseries names that start with the search string.

4.2.2 Selecting Records

The next step is to filter the full set of data records in the chosen dataseries down to a more manageable number. This is doneunder the

“RecordSet Select” tab, shown in Figure15, where a DRMS query can be entered into the white box (Step 3). You will notice that the

query box is already pre-populated with the dataseries nameselected in Step 2. Typically, users will want to refine the query by selecting

a particular range of dates and times, and/or by filtering based on values of particular keywords.

Filtering by date and time is achieved by adding to the dataseries name a date or dates enclosed in square brackets. For informational

purposes, near the top of the “RecordSet Select” tab, the dates of the earliest and most recent records in the dataseries are displayed.

The following examples demonstrate this syntax (here thehmi.M_45s dataseries is used, but the syntax is generally applicable to most

other dataseries):

24


Figure 14: The “Series select” tab from thelookdatatool, showing Steps 1 and 2. Here, in Step 1 the user has searched for dataseries

names containing the search stringhmi , and has selected thehmi.M_45s dataseries in Step 2.

• hmi.M_45s[2011.01.01_01:30_TAI-2011.01.01_02:00_TAI ] — This query requests all HMI magnetograms for

1 January 2011 between 01:30 and 02:00 TAI. It will return 41 records (endpoints are inclusive).

• hmi.M_45s[2011.01.01_01:30_TAI/30m] — This query is almost identical to the above query, except here the forward-

slash modifier is used to indicate the time range (the/30m means “over the next 30 minutes”). It will return 40 records (exclusive

of the final endpoint).

• hmi.M_45s[2011.01.01_01:30_TAI/30m@3m] — This query is similar to the above queries, except that now the cadence

is reduced from 45 seconds to 3 minutes (the@3mmeans “once per 3 minutes”). It will return 10 records.

Queries can be further refined by using keyword logic. A list of the full set of keywords for the selected dataseries can be found in

the “Series Content” tab, located between the “Series Select” and “RecordSet Select” tabs. More detailed information about HMI and

AIA keywords can be found by referencing theJSOC keywords for metadatadocument (sourced from the “All About JSOC Names”

page from theJSOC wiki) and theAIA/SDO FITS keyword list(sourced from the “AIA Keywords” entry onSDOdocs). The following

examples demonstrate queries using keyword logic (using theaia.lev1 dataseries):

• aia.lev1[2011.01.01_01:30/30m] — As above, this query requests all AIA image records for 1 January 2011 for the

half hour starting at 01:30. It will return 1200 records (oneimage in each of 8 wavelength channels every 12 seconds).

• aia.lev1[2011.01.01_01:30/30m][? WAVELNTH=171 ?] — This query now subselects the data from the above

query to include only those image records for which theWAVELNTHkeyword equals171 (i.e., images from the 171Å channel of

AIA). It will return 150 records (one image every 12 seconds).

25


http://jsoc.stanford.edu/doc/keywords/JSOC_Keywords_for_metadata.pdf



https://www.lmsal.com/sdodocs/


• aia.lev1[2011.01.01_01:30/30m][? WAVELNTH=171 or WAVEL NTH=304 ?] — This query returns those im-

age records for which theWAVELNTHkeyword equals either171 or 304 . It returns 300 records.

Subselecting theaia.lev1 dataseries using keyword logic is (admittedly) a bit cumbersome. Consequently, three additional

dataseries for AIA data are available:aia.lev1_euv_12s ,aia.lev1_uv_24s , andaia.lev1_vis_1h . These three dataseries

can be thought of as sub-series of the all-encompassingaia.lev1 dataseries, but with the data grouped into EUV-, UV-, and visible-

channel subsets. Unlike theaia.lev1 dataseries, however, these dataseries allow users to querythe JSOC and subselect by wavelength

channel without the use of the explicit keyword logic demonstrated above, and in fact are typically faster when doing so.Additionally,

the specification of a cadence using the “@” character (as shown in the preceding HMI queries) is now more user-friendly when searching

for AIA data. Such features are demonstrated in the examplesbelow (using theaia.lev1_euv_12s dataseries):

• aia.lev1_euv_12s[2011.01.01_01:30/30m][171] — This query selects all AIA image records for the 171Å channel

for 1 January 2011 for the half hour starting at 01:30. It returns 150 records (one image every 12 seconds).

• aia.lev1_euv_12s[2011.01.01_01:30/30m][171,304] — This query selects all AIA image records for the 171Å

and 304Å channels for 1 January 2011 for the half hour starting at 01:30. It returns 300 records (two images every 12 seconds).

• aia.lev1_euv_12s[2011.01.01_01:30/30m@3m][171,304] — This query is similar to the above query, except that

the cadence is reduced from 12 seconds to 3 minutes. It returns 20 records.

• aia.lev1_euv_12s[2011.01.01/10d@1h][171,304] — This query selects AIA image records for the 171Å and

304Å channels for the 10 days beginning 1 January 2011, sampled once per hour. It returns 480 records.

TECHNICAL NOTE: The elements in brackets following the dataseries name (and that do not involve keyword logic) in the queries

shown here correspond to searches ofprime keysfor the dataseries. For nearly all dataseries of interest, eitherT_RECor T_OBS

will be a prime key, and this is what enables quick date and time searching without the use of more complicated keyword logic. For

the three AIA sub-series (and unlikeaia.lev1 ), WAVELNTHis also a prime key, which is what enables the more straightforward

wavelength subselection shown above. One aspect of prime keys is that no two records displayed in a set of records have the

same set of prime key values (i.e., each record in the set is uniquely specified by the values of its prime keys). The prime keys

associated with a selected dataseries are listed near the top of the “RecordSet Select” tab (see Figure15).After entering a query, press the “GetRecordCount” button,and the tool will return the record count. It is often prudentto press

the “GetRecordCount” button after modifications to the query are made, in order to update the record count. Additionally, entering a

numbern in the (optional) “Record Limit” field will limit the number of records displayed, with positive numbers limiting the display

to the firstn records and negative numbers limiting the display to the last n records.

The user is now able to select which keywords to inspect (Step 4), which segments are desired (Step 5), and which links are desired

(Step 6). Selecting** ALL** in these boxes will return all of each of these, but to speed things up users may want to select only a subset

of keywords, segments, and/or links. Users can select individual elements in these lists by holding down the COMMAND or ALT key

while clicking the left mouse button, and ranges of keywordscan be selected by holding down the SHIFT key while clicking.Note that

the “Select Links” box will often be empty.

Proceed to the next step by clicking on the “Fetch Keyword Values for RecordSet”, at which time thelookdatatool will switch to

the “Values Display” tab, as shown in the example in Figure16. The table shown in the “Values Display” tab allows users to inspect the

keyword values in the filtered set of records (Step 7). Refining the selected set of records is achieved by going back to the “RecordSet

Select” tab (by clicking on the tab itself), adjusting the query, and repeating Steps 3 through 6.

It should be noted that selecting keywords, segments, and links in Steps 4–6 is entirely optional. If no selections are made, the

default behavior is for all keywords, segments, and links tobe displayed in the “Values Display” tab once the “Fetch Keyword Values for

RecordSet” button is pressed. Furthermore, users can choose to skip the “Values Display” tab altogether, and instead proceed directly to

the data-export process, as described in the next subsection, after entering a valid query in Step 3.

26


TECHNICAL NOTE: Near the top of the “RecordSet Select” tab, the dates of the earliest and most recent records in the dataseries

are displayed. However, in several dataseries (one of whichis aia.lev1 ) the first record is listed as being in the year

4712 BC. This (nonsense) date is used to indicate records that have blank values ofT_REC. The workaround for determining

the date of the actual first record of a particular dataseriesinvolves setting the date range of the query to be something like

[2010-01-01/365d] (or whatever date range the first record is thought to be located in, for the dataseries of interest) and

placing a+1 in the “Record Limit” field below the query. Pressing the “GetRecordCount” button should yield one valid record (if

not, try increasing the record limit), and pressing the “Fetch Keyword Values for RecordSet” button will then display the prime

keys for this record (one of which is usuallyT_RECor T_OBS).

TECHNICAL NOTE: An explanation of JSOC image timing details (i.e., why HMI images are in TAI and why AIA images are

not) is given in Section4.2.4.

4.2.3 Exporting Data

When the user wishes to export the selected set of records, clicking on the “Export Data” tab will bring upStep 8. After double-

checking that the record set query is correct in Step 8, clicking on the “Export” button will open a new webpage containingthe

“JSOC Export Data” form, the top section of which is shown in Figure17.

On the export form, the query and record count should alreadybe pre-populated in the “RecordSet” and “Record Count” fields,

respectively. The “Method” drop-down menu gives the user the option of choosing various export methods. For most remoteusers, we

recommend the “url-tar” method. After selecting “url-tar”, several new fields will appear, of which “Protocol” should be set to “FITS”,

and “Compression” set to “compress Rice”. Also of note is the“Filename Format” field allowing users to specify the formatof the

filenames. For example, if it is desired that the wavelength of the image be included in the filename, then one can add{WAVELNTH}to

this field.

After pressing the “Submit Export Request” button, the request will be given a “RequestID” tag. Users should make note ofthe ID

tag that appears in the “RequestID” field below the button. Entering the ID tag in the darker-blue area at the bottom of the form will

provide either an estimated time to completion or (if completed) a link to the archive containing the data along with a list of files that

comprise the archive. A completed export using the suggested “url-tar”/“FITS”/“compress Rice” selections is illustrated in Figure18.

TECHNICAL NOTE: In Step 8 and on the export form, if there is an argument in thequery contained in curly braces, this indicates

a listing of segments. No such argument present correspondsto the default setting of{ ** ALL** } (indicating all segments). If

a different set of segments is desired, going back to the “RecordSet Select” tab, selecting the segments of interest fromthe list in

Step 5, and pressing the “GetRecordCount” button will update the query shown in Step 8. (One will need to press the “Export”

button again to get the new query transferred to the export form.) For example, if one wants the list of despiked pixels fora set of

records belonging to theaia.lev1 dataseries, the query will look likeaia.lev1[2011.01.01_01:30/30m]{spikes} ,

since these data are contained in thespikes segment of theaia.lev1 dataseries.

TECHNICAL NOTE: Users who already know which records they want and who can construct the relevant query can save time by

bypassing thelookdatatool altogether and entering the query directly onto a blankexport form (using the link above).

4.2.4 JSOC Image Timing Details

This section gives an overview of some of the technical details regarding the various timing conventions used by the JSOC. In particular,

the differences between theT_OBS, T_REC, DATE__OBS, andDATE-OBSkeywords will be discussed below, and for additional

information users may want to consult theJSOC keywords for metadatadocument (sourced from the “All About JSOC Names” page

from theJSOC wiki) and theAIA/SDO FITS keyword list(sourced from the “AIA Keywords” entry onSDOdocs).

For temporal co-alignments between AIA and HMI, the time contained in theT_OBSkeyword should be used. This is because

theT_OBSkeyword in AIA and HMI image headers indicates the time of thecenter of the respective observation interval. For HMI

Level 1.5 observables, which are constructed from a sequence of HMI Level 1.0 filtergrams,T_OBSreflects the peak of the weighting

function applied to the constituent images comprising the sequence. For AIA,T_OBSis the center of the time for which the camera

27

http://jsoc.stanford.edu/ajax/exportdata.html






Figure 15: The “RecordSet Select” tab from thelookdatatool, showing Steps 3 through 6. Here, the user has entered the query

hmi.M_45s[2011-01-01_01:30_TAI/30m@3m] , and the tool has indicated that 10 records match this query.

28



shutter is open. In both cases, the most probable time with which each observation can be associated will be contained in theT_OBS

keyword.

The FITS standard requires the presence of theDATE-OBSkeyword, indicating the beginning of the observation interval, in Coor-

dinated Universal Time (UTC). Consequently,DATE-OBSwill be different thanT_OBS(since AIA and HMI observations occur over a

finite interval). For AIA and HMI Level 1.0 images, the time contained inDATE-OBSis equal toT_OBS−(0.5×EXPTIME), where the

EXPTIMEkeyword is the duration over which the camera shutter was open for that image, as one might expect.

For HMI observables, the time contained in theT_RECkeyword is offset fromT_OBSby the light travel time between 1 AU and the

location of SDO in its orbit. This offset is primarily due to the ellipticity of the Earth’s orbit around the sun, and thus varies on an annual

basis by at most±8 seconds. However, this effect, if unaccounted for, introduces measurable errors in the helioseismology results. For

AIA images,T_RECis the same asT_OBSafter rounding to the nearest second. When querying the JSOC, users should be aware that

in many cases (including queries of all dataseries listed inTable1) it is theT_RECkeyword, and notT_OBSor DATE-OBS, that is used

to determine whether records fall into the time range specified in the query.

The times contained inT_OBSandT_RECare tagged in either International Atomic Time (TAI) for HMI, or in UTC for AIA. The

difference between TAI and UTC is due to leap seconds that have been added to UTC over the years (due to the gradual slowing down

of the Earth’s rotation rate); such leap seconds have not been added to TAI. At the time of this writing, UTC lags TAI by 35 seconds.

When specifying a JSOC query, a suffix added to the time indicates which time format is meant. Thus, the_TAI suffixes added to the

query times in the examples shown above in Section4.2.2indicate a time in the TAI format. Alternatively, suffixes of_UTC, _UTor _Z,

or omitting the suffix altogether, imply UTC.

To put these timing details in perspective, a quick calculation indicates that, in the 35 seconds of time representing the offset between

TAI and UTC, a point at the equator of the sun will have rotatedthrough about 1/6 of one pixel (assuming an image having the canonical

0.6′′ AIA plate scale). As a result, many users will not need to worry about such small time differences, especially in light of the fact

that inaccuracies in the values in the various instrument pointing keywords are likely at least as great as 1/6 of one pixel (especially for

AIA data from the EUV channels).

TECHNICAL NOTE: The JSOC keyword naming convention does not allow for hyphens, and consequently theDATE-OBSkey-

word is represented asDATE__OBS(with two underscores) within the JSOC. However, upon export to FITS files, this keyword

gets converted toDATE-OBSso as to be compliant with the FITS standard. Additionally, note that the keywordDATE_OBS(with

one underscore) that was more common during the SOHO era is not used by the JSOC.

4.3 The Cutout Service

Thecutout serviceenables users to request sequences of images from the various AIA wavelength channels as well as of HMI line-of-

sight magnetograms and continuum intensity images. Users can access the cutout service directly using the link to the data-request form

above. The cutout service is also employed when using the “Request data” link associated with events returned in iSolSearch queries

(see Section3.2), in which case the data-request form will be pre-populatedwith the spatiotemporal bounds of the event so that data

associated with HER events is easily retrievable by users. As an alternative to this online form, there is also a SolarSoft interface to the

cutout service that allows additional customization (see Section5.4).

As shown in Figure19, the online form has a straightforward layout, containing input fields for specifying the spatiotemporal extent

of the cutout, as well as the wavelength channels desired. The spatial extent can be specified either by drawing a boundingbox on the

sample image or typing in coordinates directly into the appropriate parameter fields. As an alternative to specifying a subfield, users can

instead request full-disk data at various spatial resolutions. Clicking on the “full-disk” link (underneath the sample image) will bring up

radio buttons that allow the resulting image size to be specified, enabling the user to specify full-resolution images (4096×4096 pixels)

or images binned down by a factor of 2, 4, or 8.

In the “Service” drop-down menu (underneath the sample image), users can select either “SSW Service” or, if available, “JSOC

Service”. Selecting “JSOC Service” causes the form to construct a query as if the user had filled out the“JSOC Export Data” form

described in Section4.2. Upon successful submission, the user will be provided with“RequestID” tag(s) that can be used to monitor

the status of the JSOC export request. Alternatively, the “SSW Service” option causes SSW routines to be used to process the cutout.

29

http://www.lmsal.com/get_aia_data



Upon submission of a cutout request using the “SSW Service” option, the user should receive an acknowledgement containing the SSW

request ID that can be used to monitor the status of the SSW cutout request.

Underneath the “Service” drop-down menu is a checkbox labeled “Tracking”. Checking this box will cause the field of view of the

cutout to follow the mean rotation rate of the latitude of thecenter of the cutout. The reference time of the cutout (to which the cutout

width, height, and center apply) is the same as the referencetime of the image displayed, which is usually around the 23h mark of the

start date. (The sample image displayed is the same as on the archived version of “The Sun Today” corresponding to the start date.) In

the last four fields of the request form, users should input their name, e-mail address, a brief description of the data request, and the

desired cadence (image frequency).

Once the dataset is ready, an email will be sent with instructions describing how to download the data. If the “SSW Service” option

was chosen, the e-mail will contain a link to the service-request output page. There, the user can view a summary movie using the

“JobWWW” link, or download the data by going into SSWIDL and using the command in the “Get_Data” field to download the data

to the currently active directory. If, instead, the “JSOC Service” option was chosen, the e-mail will contain a link to the (temporary)

location online at the JSOC from which the data can be downloaded. The data are presented as a set of links to individual files, and as

mentioned earlier, browser add-ons (such asfor Firefox) come in handy for downloading such lists of linksin a few mouse-clicks.

If an error occurs in the request or if the request exceeds ourcurrently tested capacity, it will stay in the queue and we shall try to

process it in due time, or otherwise we may contact you describing how to reformulate your request. Users who have questions about

their cutout request should send an e-mail containing the Request ID(s) to the address listed onthis webpage.

TECHNICAL NOTE: The online form will adjust the spatial bounds upward so that the x and y dimensions of the cutout are both

multiples of 64 pixels.

4.4 The Virtual Solar Observatory

The Virtual Solar Observatory (VSO) has been described as “one-stop shopping for (nearly) all of your solar physics needs”. It allows

users to query a multitude of data providers (e.g., SDO, Hinode, and other observatories) using standardized interfaces: either via the

web or via SolarSoft. Here, in this section, we will demonstrate how to retrieve SDO data using the web-based interface, whereas

the SolarSoft interface is demonstrated for SDO data in Section 5.5. Online help at any step is available either by clicking on the

yellow-question-mark information icons visible on the various the web forms, or by perusing theVSO FAQ webpageor theVSO blog.

To use the web interface, navigate to theVSO entry page, shown in Figure20, where the user is presented with a list of attributes

that will frame the subsequent VSO query. Note that “Time” isalways selected. (If the date and time of interest are not already known,

then the resources described in Section3 of this guide will be of use.) For AIA or HMI, choosing the “Instrument / Source / Provider”

and then “Instrument Only” options is a simple way to narrow down the source options. Choosing “Observable” (for HMI) or “Spectral

Range” (for AIA) may also be of use when selecting SDO data. Pressing the “Generate VSO Search Form” button at the bottom ofthe

entry page will display the search form.

On the search form, shown in Figure21, the user enters a start and end time for the query, and checksoff one or more instruments.

Pressing the “Search” button will, after a few moments, return the results of the query. At this point, the user selects which records they

wish to retrieve by marking the check-boxes in the left-mostcolumn of the table. The “CheckBox Tools” section on the left-hand sidebar

may come in handy here, as it enables users to check oruncheck large swathes of records. The columns can be sorted by checking

off the boxes next to the column headings, and then clicking on the “Sort”/”Rearrange” buttons above the table. The results page with

some boxes checked is shown in Figure22. Pressing the “Request Data” button at the bottom of the left-hand sidebar, and selecting

“URL-FILE_Rice” (for Rice-compressed FITS files), will bring up a set of links to the data, as shown in Figure23.

At this stage, the image data can be downloaded individuallyby clicking on each of the URLs. To download large numbers of

images, one will probably want to either use a browser add-on(such asDownThemAll! for Firefox), or alternatively export the list of

URLs to a text file by pressing the “Export Links” button at thebottom of the left-hand sidebar and then make use of a utilitylike curl to

download the data. For example, suppose the list of URLs wereexported from the VSO and saved in a file namedvso_export.csv ,

then one can download all of the URLs in the list to the currentworking directory by entering

30

http://sdowww.lmsal.com/suntoday

https://addons.mozilla.org/en-US/firefox/addon/downthemall/


http://docs.virtualsolar.org/wiki/VsoFAQ

http://vso.nso.edu/blogs

http://sdac.virtualsolar.org/cgi/search

https://addons.mozilla.org/en-US/firefox/addon/downthemall/


xargs -n1 curl -O --remote-header-name < vso_export.csv

at the Unix command prompt. Note that one needscurl version 7.21.2 or later for the--remote-header-name option (which

properly handles the content-disposition headers used by the VSO, which in turn provides sensible filenames for the downloaded data)

to work properly.

TECHNICAL NOTE: The command above may seem a bit unwieldy for downloading a list of URLs contained in a file,

but becausecurl doesn’t have a clean option for specifying an input file from which the URLs are read (à la the

-i option for wget ), the workaround is to invokecurl via xargs . One could alternatively use a command sim-

ilar to “curl -O --remote-header-name -K vso_export.csv ” (i.e., without usingxargs ), but this requires

editing vso_export.csv to enablecurl to interpret each URL as if it were a command-line option of the form

--url="http://..." . In contrast, using thexargs command withcurl as shown above enables the use of a

vso_export.csv -type file exported from the VSO without modification.

4.5 Synoptic Data

Synoptic data streams are made on a regular basis from the JSOC pipeline. For HMI, the synoptic stream that exists for line-of-sight

magnetogram data is locatedhere. This stream contains HMI rebinned 1024×1024-pixel line-of-sight magnetograms sampled once per

hour. Additionally, there are images and movies of both continuum and magnetogram image series locatedhere.

For AIA, there is a set of synoptic data locatedhere. These synoptic data consist of compressed FITS files containing rebinned

(1024×1024-pixel) Level 1.5 images in each of the AIA wavelength channels, sampled every two minutes. There is a mission-long

stream as well as a near-real time stream. There is also a stream for full-resolution AIA images in the JPEG 2000 format, which allows

hierarchical uncompression of these data, locatedhere.

4.6 Uncompressing the Data

Data from SDO are typically downloaded as Rice-compressed FITS files, in order to reduce the bandwidth needed for the download. It

is anticipated that users will prefer to save disk space and store the data in the compressed format, after realizing thatthe time needed to

read in compressed files and perform the subsequent uncompression is often about the same as to the time needed to read in the (larger)

uncompressed files. Of course, the precise tradeoff dependson the compression ratio, disk I/O speeds, CPU processing speeds, etc., and

so (as they say)your mileage may vary.

Rice-compressed files can be uncompressed (and, in many cases, viewed) using a number of freely-available image-manipulation

software programs, includingSAOImage DS9, and theimcopy utility and fpack packagefrom theCFITSIO subroutine library. Also,

theread_sdo.pro IDL routine, available via theONTOLOGYSolarSoft package (see Section5), provides a way to uncompress and

read in Rice-compressed header data and images from the IDL command line. Additional information onread_sdo.pro can be

found in Section5.6.

4.6.1 Compiling and Installing imcopy

Becauseimcopy is a useful utility to have around, especially for users who are not SolarSoft-inclined, we here demonstrate how to

compile and install it. First, download theCFITSIO subroutine libraryand expand the archive, which should automatically create the

directorycfitsio that contains all of the CFITSIO source code.

On Unix systems, one creates the makefile by entering “./configure ” at the shell prompt. Optionally, one may include a

prefix argument that specifies the target installation directory.For example, entering “./configure --prefix=/usr/local ”

will cause the CFITSIO library to end up in/usr/local/lib/ and include files to end up in/usr/local/include/ . If no

prefix argument is given, the libraries and include files are created in the current working directory. If theconfigure script exits

without encountering any problems, it will state somethingsimilar to, “Congratulations, Makefile update was successful” and one can

31

http://jsoc.stanford.edu/data/hmi/fits

http://jsoc.stanford.edu/data/hmi

http://jsoc.stanford.edu/data/aia/synoptic

http://jsoc.stanford.edu/data/aia/images

http://hea-www.cfa.harvard.edu/RD/ds9/site/Home.html

http://heasarc.gsfc.nasa.gov/fitsio/fpack

http://heasarc.gsfc.nasa.gov/fitsio



proceed to the next step. If theconfigure script encounters a problem, it should indicate what went wrong. Common problems

usually involve compilers or include files either not installed or not in the current path, and the user will need to rectify these problems

before compilation can occur.

After creating the makefile, one then creates the CFITSIO library libcfitsio.a by entering “make” at the Unix prompt. If all

goes well, several dozen objects will get created and storedin the library. At this stage, one can optionally install theCFITSIO library

in the installation path specified byprefix above by entering “make install ”. Depending on the installation path, one may need

superuser permission to do this.

Lastly, to compile the stand-alone utilityimcopy , one needs simply to type “make imcopy ”. Users may also want to also enter

“make fpack ” and “make funpack ” if using these routines to compress or uncompress FITS filesis anticipated. At this stage, the

executableimcopy should be present, and can be moved into the active path.

Using imcopy to uncompress a Rice-compressed FITS file, such as those downloaded from the JSOC or from the VSO, can be

achieved by entering either

imcopy image_compressed.fits image_uncompressed.fits

or

imcopy ’image_compressed[1]’ image_uncompressed.fits

on the Unix command line. (The first command usually generates a FITS file with an IMAGE extension, which might not be what was

desired.) Theimcopy utility is extremely versatile, as it recognizes the extensive CFITSIO extended filename syntax. In particular,

one can not only deal with file compression, but also extract cutouts, resample data, download files from remote servers, and convert

between various image formats via a single command. Readersmay refer to theseadditional examplesfrom the CFITSIO documentation

for additional ideas.

32

http://heasarc.nasa.gov/docs/software/fitsio/filters.html

http://heasarc.nasa.gov/docs/software/fitsio/cexamples.html#imcopy


Figure 16: The “Values Display” tab from thelookdatatool, showing a listing of all keywords and their values for the 10records

matching thehmi.M_45s[2011-01-01_01:30_TAI/30m@3m] query illustrated in Figure15.

33



Figure 17: The top section of the“JSOC Export Data” form, illustrating how to export the 10 records matching the

hmi.M_45s[2011-01-01_01:30_TAI/30m@3m] query illustrated in Figure15. After pressing the ”Submit Export Request”

button and waiting for the export to process, the lower section will appear as in Figure18.

34



Figure 18: The lower section of“JSOC Export Data” form, illustrating the completed export requested in Figure17.

35



Figure 19: The cutout service request form, showing a request for data from the AIA 171Å and 304Å channels and HMI line-of-sight

magnetograms covering a localized region of the solar disk.

36


Figure 20: TheVSO web interfaceentry page, with the “Instrument / Source / Provider” and “Instrument Only” options checked.

Pressing the “Generate VSO Search Form” button brings up thequery form shown in Figure21.

37



Figure 21: The VSO search form, with the user about to requestone hour’s worth of AIA data from 1 August 2010. Pressing “Search”

yields the results page shown in Figure22.

38


Figure 22: The results of the VSO query from Figure21, showing the top portion of the listing of 2336 records. One minute’s worth of

AIA data have been selected for export. Pressing the “Request Data” button at the bottom of the left-hand sidebar resultsin the download

page shown in Figure23.

39


Figure 23: The VSO download page corresponding to the set of records checked off in Figure22. The list of links can be exported to a

file by clicking on the “Export Links” button on the left-handsidebar.

40


5 How to Use SolarSoft with SDO Data

TheSolarSoftWare(SSW) system, or “SolarSoft” for short, is a software distribution system that originally stemmed from the need to

coordinate solar mission data analysis across several institutions, going back as far as theYohkoh mission(which launched in 1991). It is

comprised of integrated software libraries, databases, and system utilities that provide a coordinated data analysisenvironment for solar

physics research. It is largely hardware- and site-independent, with mirroring capabilities to keep installations atvarious sites consistent.

Although most of the software distributed via SSW is in Exelis Visual Information Solutions’Interactive Data Language(IDL) — the

lingua francaof the solar physics community — SSW can easily accommodate software in other languages.

In the following sections, we will describe how to install orupgrade SSW for use with SDO data (Section5.1), and the how to use

SSW to interface with the HEK (Section5.2) and to get HMI or AIA image data from the JSOC (Section5.3). Also demonstrated are

the SSW interfaces to the cutout service (Section5.4) and the VSO (Section5.5). Once data have been received, users will need to know

how to uncompress these data within SSW (Section5.6). Additional items related to AIA image processing, while almost exclusively

implemented in SSW, are found in the next section (Section6). Information regarding access to EVE data via SSW is availablehere.

This section contains several examples of IDL code. To guideusers, many of these examples are given in pairs of code blocks: the

first block contains code that users can copy and paste into their IDL session, and the second block shows the output that should result.

5.1 Installing or Upgrading SolarSoft

5.1.1 Doing a Clean Install

To do a clean install of SSW, follow these steps:

• Navigate to the mainSSW installation webpage. This is the webpage containing instructions for installing SSW on different

operating systems.

• Next, click on the “SSW INSTALLATION FORM” link. The installation form allows the user to select the top-level path in which

the software will reside, along with which packages are to bedownloaded. For SDO-related software, selectAIA , HMI, EVE, and

ONTOLOGY. TheONTOLOGYpackage includes software that interacts with the JSOC, andthe others contain software specific to

the respective SDO instruments. If you think that you will beusing the SSW interface to the VSO (see Section5.5), then theVSO

package should also be selected.

• After completing the installation form, click on the “Generate Installation Script” button near the bottom of the page.The website

will then give you an indication of how much disk space your SSW installation will occupy, and provide you with a link to your

personalized installation script or batch file.

• Next, save the installation script or batch file to disk by right-clicking on the link underneath the installation table.Run the script

or batch file on your machine. On Windows systems, the batch file will be contained in an archive, which needs to be unzipped.

Running the script or batch file will cause the SSW packages tobe downloaded.

At this point, SSW and its constituent packages (should) have been downloaded to your system. On Unix and Unix-like systems, a

few more steps are required for IDL to have access to this software, as described on theSSW setup webpage. First, it is necessary to

define theSSWenvironment variable so that it points to the top-level pathof the SSW tree:

setenv SSW [whatever you specified on the form]

SSW also needs to know which packages are available. This is achieved by defining theSSW_INSTRenvironment variable so that it

includes the packages that were downloaded:

setenv SSW_INSTR "ontology aia hmi eve vso [etc.]"

Then, after sourcing the setup file,

41

http://www.lmsal.com/solarsoft

http://en.wikipedia.org/wiki/Yohkoh

http://www.exelisvis.com/ProductsServices/IDL.aspx

http://www.lmsal.com/solarsoft/ssw/sdo/eve/doc/

http://www.lmsal.com/solarsoft/ssw_install_howto.html

http://www.lmsal.com/solarsoft/ssw_setup.html


source $SSW/gen/setup/setup.ssw [/loud]

one simply typessswidl (an alias that was defined in the setup file) at the Unix prompt to start up IDL. IDL combined with the

SolarSoft libraries and databases is frequently referred to as “SSWIDL”.

TECHNICAL NOTE: Thesetenv commands above apply only to the C shell (csh) or related shells. If using Bourne or bash, then

the proper syntax for setting an environment variable is “export VARIABLE=value ”.

TECHNICAL NOTE: After the setup filesetup.ssw is sourced, an alias namedsetssw will be defined that allows users to

load specific libraries and instrument packages. This feature is useful when the SSW startup process is found to be undesireably

lengthy, a situation often caused by a large number of packages being loaded when SSWIDL is starting up. In these situations,

users can reduce the startup time by loading a smaller numberof packages. For example, if one only needs theaia andchianti

packages, then entering “setssw aia chianti ” will cause SSWIDL to only load those two packages upon startup until

eithersetssw is invoked again or theSSW_INSTRenvironment variable is changed.

5.1.2 Updating Packages for an Existing Installation

If you already have an existing SSW installation and you simply want to add a new package or update an existing package, then at a

SSWIDL prompt issue a command of the form:

ssw_upgrade,/spawn,/loud,/package1 [,/package2] [,/...]

For example, adding or updating theAIA andONTOLOGYpackages is achieved by entering the following (note that some systems also

require the/PASSIVE_FTP flag):

ssw_upgrade,/spawn,/loud,/aia,/ontology [,/passive_ftp]

Afterward, be sure to add any new packages to your list of instruments in theSSW_INSTRenvironment variable. More information

regarding SSW upgrades is available on theSSW upgrade webpage.

5.1.3 Using a Proxy Server with SSW

A proxy server is a server located between two networks that acts as an intermediary for all network traffic routed betweenthem. Often,

a proxy server is inserted between a local network and a larger network (such as the internet) to provide anonymity, increased security,

or performance enhancements. Because some of the functionality of SSW involves downloading information from remote databases or

otherwise interacting with the internet, there are times that SSW meeds to be proxy-aware in order to work properly. In some cases,

there may be global system settings that automatically route outgoing traffic generated by SSW to the correct proxy server (obviating

the need for the user to do any additional configuration), in which case the information in this subsection may be superfluous.

If additional configuring is required, however, the following may be useful. One way to make SSW proxy-aware is to set the

environment variablesHTTP_PROXYand NO_PROXYprior to (or during) SSW startup. TheHTTP_PROXYenvironment variable

should contain the name or IP address of the proxy server, andthe NO_PROXYenvironment variable should list domains that do not

require routing through the proxy.

For example, a user on theexample.edu network may need to use a proxy server atproxy.example.edu:80 in order to

access the internet. In this case, for SSW to work the user will need to ensure that the environment variableHTTP_PROXYis set to

proxy.example.edu:80 , either by setting it prior to entering SSW or by inserting a line such as

set_logenv,’http_proxy’,’http://proxy.example.edu:80’

42

http://www.lmsal.com/solarsoft/ssw_upgrades.html


in the$SSW/site/setup/IDL_STARTUP file located in one’s SSW installation. For domains that do not require routing through

the proxy server for access (domains on the local network typically fall into this category), these domains should be given as a comma-

delimited list in theNO_PROXYenvironment variable. As before, inserting

set_logenv,’no_proxy’,’.example.edu’

in $SSW/site/setup/IDL_STARTUP or setting this environment variable prior to SSW startup will cause SSW to bypass the proxy

server in order to access any URL ending in.example.edu . Note that the setup file at$SSW/site/setup/IDL_STARTUP in

one’s SSW installation is not overwritten during SSW upgrades; thus, one has the ability to edit this file and not have these changes

undone when SSW is upgraded.

5.2 Querying the HER

5.2.1 Basic Query Syntax

Querying the HER (see Section3.1) using SSW is achieved by using the IDL code available through theONTOLOGYpackage. This

package contains basic routines that interface with the HEKusing theHEK API.

For SSWIDL queries to the HER, SolarSoft provides two routines: ssw_her_make_query.pro , which generates a string of

parameters in used in the subsequent CGI query, andssw_her_query.pro , which issues an input verbatim query to the HEK CGI

interface and translates the output to an IDL structure vector of event matches. In essence, these routines provide the translation between

the SSW-style index metadata structures and the XML-style<paramater>=<value> syntax used within the HEK.

The following example demonstrates how to run the query using ssw_her_make_query.pro :

; copy and paste these commands into your SSWIDL session:

t0=’2010-01-01’ & t1=’2010-12-31’

query = ssw_her_make_query(t0,t1,/fl)

print,query

IDL> t0=’2010-01-01’ & t1=’2010-12-31’

IDL> query=ssw_her_make_query(t0,t1,/fl)

IDL> print,query

event_starttime=2010-01-01T00:00:00&event_endtime=2010-12-31T00:00:00&event_coordsys=helioprojective&

Here, the/fl switch is set to indicate that we are querying for HER entrieshaving the flare event class, since “fl ” is the two-letter

code for “flare” (as shown in thefull list of event classes).

To execute the query, thessw_her_query.pro routine is used:


t0=’2010-01-01’ & t1=’2010-12-31’

query=ssw_her_make_query(t0,t1,/fl)

events=ssw_her_query(query)

help,events

help,events,/structures

help,events.fl

help,events.fl,/structures

IDL> t0=’2010-01-01’ & t1=’2010-12-31’

IDL> query=ssw_her_make_query(t0,t1,/fl)

43

http://www.lmsal.com/hek/api.html



IDL> events=ssw_her_query(query)

IDL> help,events

EVENTS STRUCT = -> <Anonymous> Array[1]

IDL> help,events,/structures

** Structure <1c4dc0c8>, 1 tags, length=182400, data length=182400, refs=1:

FL STRUCT -> <Anonymous> Array[100]

IDL> help,events.fl

<Expression> STRUCT = -> <Anonymous> Array[100]

IDL> help,events.fl,/structures

** Structure <2838808>, 114 tags, length=1824, data length=1824, refs=2:

EVENT_PIXELUNIT STRING ’’

BOUND_CCSTARTC2 STRING ’’

EVENT_DESCRIPTION

STRING ’’

EVENT_NPIXELS STRING ’’

FRM_HUMANFLAG STRING ’false’

REVISION STRING ’1’

EVENT_EXPIRES STRING ’’

KB_ARCHIVID STRING ’ivo://helio-informatics.org/FL_SECstandard_20100613_150757_201001011202

[...deletia...]

The output of thessw_her_query.pro routine is an array of event structures (one mer match), in which are stored the metadata

describing each event. The 100 matches reflect the current default match limit. This can be changed by setting theRESULT_LIMIT

keyword in the call tossw_her_query.pro .

TECHNICAL NOTE: In IDL Version 8.0, the “help,events ” and “help,events.fl ” IDL commands in the above examples

do not work as shown here, and instead behave as if the/STRUCTURESswitch has been set. This is apparently a bug and has

been fixed in IDL Version 8.0.1, according tothis newsgroup discussion(Google login required).

5.2.2 Adding Filters to Queries

Users can add filters that screen for, or screen out, events that satisfy a set of user-defined conditions by defining theSEARCH_ARRAY

keyword in the call tossw_her_make_query.pro . Using the same example as in the iSolSearch example of Section 3.2, suppose

one wants to find all flares having GOES Class M2 or greater thatoccurred in the year 2010. The resulting query would filter the results

based on theFL_GOESCls (flare GOES class) attribute, and would be constructed and executed as follows:


t0=’2010-01-01’ & t1=’2010-12-31’

query=ssw_her_make_query(t0,t1,/fl,search_array=[’FL_GOESCls>=M2.0’])



IDL> t0=’2010-01-01’ & t1=’2010-12-31’

IDL> query=ssw_her_make_query(t0,t1,/fl,search_array=[’FL_GOESCls>=M2.0’])



** Structure <1c48a7c8>, 1 tags, length=32832, data length=32832, refs=1:

FL STRUCT -> <Anonymous> Array[18]

44

http://groups.google.com/group/comp.lang.idl-pvwave/browse_thread/thread/c87aaa6a12684a61


Again, at the time of this writing, the query returned 18 events. Valid relational operators for theSEARCH_ARRAYkeyword include=,

>=, <=, <>, >, <, andlike .

Thelike operator searches for the occurrence of substrings, and is useful for searching through the event descriptions (or any other

attribute defined as a string) for specific words or phrases. The following example demonstrates how to search for eruption events whose

descriptions contain the word “bubble” (note that search strings such as “bubble” are case insensitive):


t0=’2010-01-01’ & t1=’2012-12-31’

query=ssw_her_make_query(t0,t1,/er,search_array=[’EVENT_DESCRIPTIONlikeBUBBLE’])



IDL> t0=’2010-01-01’ & t1=’2012-12-31’

IDL> query=ssw_her_make_query(t0,t1,/er,search_array=[’EVENT_DESCRIPTIONlikeBUBBLE’])



** Structure <1aea54d8>, 1 tags, length=99552, data length=99552, refs=1:

ER STRUCT -> <Anonymous> Array[51]

This query returned 51 events wherein the event descriptioncontained the string “bubble”. The HER is not intended to be complete (not

all eruptions where a bubble is evident will be return), but it will show results that were noteworthy enough for (to use the above query

as an example) a human annotator to indicate that a bubble wasassociated with an eruption.

TECHNICAL NOTE: Although one cannot presently generate queries using filters containing thelike operator from

within the iSolSearch GUI (see Section3.2), it is still possible to generate a valid URL that will perform such queries

and display the results in iSolSearch. To do this, first go into SSWIDL and generate the list of parameters and val-

ues corresponding to the query. These will be returned by thessw_her_make_query.pro routine in the standard

HTTP_GET syntax. Then, take these query parameters and append them to the end of a blank HER search URL, which begins

https://www.lmsal.com/isolsearch?hek_query=https:// www.lmsal.com/hek/her?cosec=2&cmd=search&type=c

In the case of the above example where the user is searching for the substring “bubble” in eruption event descriptions, open

this link in a new browser window to see the complete URL and the resultsof this query in iSolSearch. The list of events

satisfying this query that are displayed in the iSolSearch GUI should match the event listing returned in SSWIDL exampleabove.Within the SSWIDL environment, thestruct4event.pro routine is available for determining the current list of required and

optional attributes for each event class (basically by returning a blank event entry). For example, one can view the attributes associated

with flare events as follows:


flaretags=struct4event(’FL’)

help,flaretags.required,/structures

help,flaretags.optional,/structures

IDL> flaretags=struct4event(’FL’)

IDL> help,flaretags.required,/structures

** Structure <480d208>, 31 tags, length=392, data length=388, refs=2:

EVENT_TYPE STRING ’FL: Flare’

KB_ARCHIVDATE STRING ’Reserved for KB archivist: KB entry date’

KB_ARCHIVID STRING ’Reserved for KB archivist: KB entry identifier’

KB_ARCHIVIST STRING ’Reserved for KB archivist: KB entry made by’

45

https://www.lmsal.com/isolsearch?hek_query=https://www.lmsal.com/hek/her?cosec=2&cmd=search&type=column&event_starttime=2010-01-01T00:00:00&event_endtime=2012-12-31T00:00:00&event_coordsys=helioprojective&x1=-5000&x2=5000&y1=-5000&y2=5000&event_type=ER&cosec=2&type=column&cmd=search&param0=EVENT_DESCRIPTION&op0=like&value0=BUBBLE


KB_ARCHIVURL STRING ’Reserved for KB archivist: URL to suppl. info.’

EVENT_COORDSYS STRING ’UTC-HPC-TOPO’

EVENT_COORDUNIT STRING ’blank’

EVENT_ENDTIME STRING ’1492-10-12 00:00:00’

EVENT_STARTTIME STRING ’1492-10-12 00:00:00’

[...deletia...]

IDL> help,flaretags.optional,/structures

** Structure <4802a08>, 55 tags, length=632, data length=586, refs=2:

EVENT_PROBABILITY

FLOAT Inf

EVENT_EXPIRES STRING ’1492-10-12 00:00:00’

EVENT_COORD3 FLOAT Inf

EVENT_MAPURL STRING ’blank’

EVENT_MASKURL STRING ’blank’

EVENT_CLIPPEDSPATIAL

STRING ’blank’

EVENT_CLIPPEDTEMPORAL

STRING ’blank’

EVENT_TESTFLAG STRING ’blank’

[...deletia...]

5.3 Retrieving Data from the JSOC

The SSWIDL routines that interface with the JSOC (see Section 4.1) all have the formssw_jsoc * .pro , and are located in the

ONTOLOGYpackage in the SSW installation. These routines have many more parameters and keywords than are outlined here, and

we urge users to look at the source-code preamble for more detailed lists of options. The source code can be viewed either by using

thexdoc.pro routine from within SSWIDL, by using theSSW code searchwebpage to search for a routine of interest, or by directly

viewing the source code once the local path is known (say, after using the SSWIDL routinewhich.pro , which returns the path of any

routine in a user’s IDL path). Additionally, readers shouldbe familiar with terms describing the JSOC data management system, such

asdataseries, record, andsegment, as described in Section4.1.

There are three basic steps that enable users to download data from the JSOC directly into SSWIDL, as discussed in the following

subsections. The steps illustrated here are directly analogous to the steps needed to use thelookdatatool demonstrated in Section4.2.

5.3.1 Getting Dataseries Names

The first step is to search for the JSOC dataseries of interestusing theFILTER keyword ofssw_jsoc.pro :

result=ssw_jsoc(filter=’[filter string]’)

This function returns a structure, with each element in the structure corresponding to a different dataseries. All of the dataseries names

contain the substring provided in theFILTER keyword on input. TheNAMEStag of this structure contains the dataseries name, the list

of prime keys, and a one-line description of each dataseries. For example, to find all JSOC dataseries containing the substring aia , enter

the following commands into SSWIDL:


result=ssw_jsoc(filter=’aia’)

46

http://www.lmsal.com/codesearch


help,result,/structures

help,result.names(1),/structures

IDL> help,result,/structures

** Structure <39259a8>, 3 tags, length=352, data length=340, refs=1:

STATUS INT 0

NAMES STRUCT -> <Anonymous> Array[7]

N INT 7

IDL> help,result.names(1),/structures


NAME STRING ’aia.lev1’

PRIMEKEYS STRING ’T_REC, FSN’

NOTE STRING ’AIA Level 1’

This result should mimic the output displayed in Step 2 of thelookdatatool (see Section4.2.1).

TECHNICAL NOTE: If a proxy server is being used, then SSW needs to be proxy-aware in order forssw_jsoc.pro to work

properly; otherwise,ssw_jsoc.pro will be unable to query the JSOC and will eventually time out.To configure SSW to work

with a proxy server, see Section5.1.3.

TECHNICAL NOTE: When searching for valid dataseries usingssw_jsoc.pro , prepending the caret (circumflex) characterˆ

to the search substring in theFILTER keyword will return only those dataseries whose names that start with the search string.When a dataseries name is known, thessw_jsoc.pro routine can be used (with the/SERIES_STRUCT switch set) to fetch

information that defines the dataseries, such as the list of applicable keyword and segment names:


result=ssw_jsoc(ds=’aia.lev1’,/series_struct)


help,result.keywords

print,transpose(result.keywords.name)

help,result.segments

print,transpose(result.segments.name)

IDL> result=ssw_jsoc(ds=’aia.lev1’,/series_struct)


** Structure <50af208>, 12 tags, length=9456, data length=9444, refs=1:

NOTE STRING ’AIA Level 1’

RETENTION INT 10000

UNITSIZE INT 1

ARCHIVE INT 0

TAPEGROUP INT 10000

PRIMEKEYS STRING Array[2]

PRIMEKEYSINFO STRUCT -> <Anonymous> Array[2]

DBINDEX STRING Array[2]

KEYWORDS STRUCT -> <Anonymous> Array[1]

SEGMENTS STRUCT -> <Anonymous> Array[3]

INTERVAL STRUCT -> <Anonymous> Array[1]

47


STATUS LONG 0

IDL> help,result.keywords

** Structure <58539d8>, 3 tags, length=9024, data length=9024, refs=2:

NAME STRING Array[188]

TYPE STRING Array[188]

NOTE STRING Array[188]

IDL> print,transpose(result.keywords.name)

cparms_sg000

image_lev1_bzero

image_lev1_bscale

cparms_sg001

bad_pixel_bzero

bad_pixel_bscale

cparms_sg002

spikes_bzero

spikes_bscale

BLD_VERS

LVL_NUM

T_REC

[...deletia...]

IDL> help,result.segments

** Structure <5866e28>, 5 tags, length=80, data length=80, refs=2:

NAME STRING ’image_lev1’

UNITS STRING ’dn’

PROTOCOL STRING ’fits’

DIMS STRING ’4096x4096’

NOTE STRING ’lev1 image fits file’

IDL> print,transpose(result.segments.name)

image_lev1

bad_pixel

spikes

As shown above, theKEYWORDSandSEGMENTStags of the structure returned byssw_jsoc.pro (when the/SERIES_STRUCT

switch is set) are themselves structures that contain information about the keywords and segments belonging to the dataseries specified

by theDSkeyword. These sub-structures are defined with tags namedNAME, TYPE, andNOTE, which contain the name, datatype, and a

brief description of the keyword or segment, respectively.This allows for an alphabetized list of the keywords applicable to, for example,

theaia.lev1 dataseries, to be generated by the following commands:


result=ssw_jsoc(ds=’aia.lev1’,/series_struct)

print,transpose(result.keywords.name(sort(strupcase(result.keywords.name))))

IDL> result=ssw_jsoc(ds=’aia.lev1’,/series_struct)

IDL> print,transpose(result.keywords.name(sort(strupcase(result.keywords.name))))

ACS_CGT

ACS_ECLP

48


ACS_MODE

ACS_SAFE

ACS_SUNP

AECDELAY

AECMODE

AECTYPE

AGT1SVY

AGT1SVZ

AGT2SVY

AGT2SVZ

[...deletia...]

Information about the keywords and segments of a dataseriesretrieved as shown above should match the information displayed

in the “Series Content” tab of thelookdata tool. For further information about HMI and AIA keywords, users can refer to ei-

ther theJSOC keywords for metadatadocument (sourced from the “All About JSOC Names” page on theJSOC wiki) and/or the

AIA/SDO FITS keyword list(sourced from the “AIA Keywords” entry onSDOdocs).

5.3.2 Selecting Records

Once the dataseries name is known, the next step is to get a listing of available records from the JSOC. This is achieved using the

ssw_jsoc_time2data.pro routine:

ssw_jsoc_time2data,t0,t1,index,ds=’[dataseries name]’

If images from the specified dataseries exist in the date/time range specified by theT0 andT1 arguments, the procedure will return (in

theINDEX variable) a listing of these image headers. The structure tags mimic the list of keywords shown by thelookdatatool in Step 4

(see Section4.2.2). For example:


ssw_jsoc_time2data,’1-jan-2011 01:00’,’1-jan-2011 01:30’,index,ds=’aia.lev1’

help,index

help,index,/structures

IDL> ssw_jsoc_time2data,’1-jan-2011 01:00’,’1-jan-2011 01:30’,index,ds=’aia.lev1’

-----------------------------

| /SERIES_STRUCT, ds=aia.lev1 |

-----------------------------

----------------------------------------------------------------

| ssw_jsoc_time2query output: aia.lev1[2011-01-01T01:00:00Z/30m] |

----------------------------------------------------------------

-------------------------------

| selecting segment> image_lev1 |

-------------------------------

-------------------------------------------------------------

| /rs_list, ds=aia.lev1[2011-01-01T01:00:00Z/30m]{image_lev1} |

-------------------------------------------------------------

----------------------

49






| HEX handler QUALLEV0 |

----------------------

---------------------

| HEX handler QUALITY |

---------------------

IDL> help,index

INDEX STRUCT = -> <Anonymous> Array[1200]

IDL> help,index(0)

** Structure <18a2b808>, 178 tags, length=1120, data length=1063, refs=1:

BLD_VERS STRING ’V5R12X’

LVL_NUM FLOAT 1.00000

T_REC STRING ’2011-01-01T01:00:01Z’

T_REC_STEP DOUBLE 1.0000000

T_REC_EPOCH STRING ’1977.01.01_00:00:00_TAI’

T_REC_ROUND LONG 1

ORIGIN STRING ’SDO\/JSOC-SDP’

DATE STRING ’2011-07-02T14:05:51Z’

TELESCOP STRING ’SDO/AIA’

INSTRUME STRING ’AIA_3’

DATE__OBS STRING ’2011-01-01T01:00:00.34Z’

T_OBS STRING ’2011-01-01T01:00:01.34Z’

[...deletia...]

The INDEX structure having 1,200 elements indicates that for the 30-minute time period of interest in this example, there exist 1,200

records.

TECHNICAL NOTE: If a proxy server is being used, then SSW needs to be proxy-aware in order for

ssw_jsoc_time2data.pro to work properly; otherwise,ssw_jsoc_time2data.pro will be unable to query the JSOC

and will eventually time out. To configure SSW to work with a proxy server, see Section5.1.3.For AIA data queries, a common activity is to select only particular wavelength channels. To do this, specify the channel(s) of

interest in theWAVESkeyword, which will be then checked against the value of theWAVELNTHkeyword for each record:


ssw_jsoc_time2data,’1-jan-2011 01:00’,’1-jan-2011 01:30’,index,$

ds=’aia.lev1_euv_12s’,waves=[171,304]

help,index

IDL> ssw_jsoc_time2data,’1-jan-2011 01:00’,’1-jan-2011 01:30’,index,$

IDL> ds=’aia.lev1_euv_12s’,waves=[171,304]

-----------------------------


-----------------------------

----------------------------------------------------------------

| ssw_jsoc_time2query output: aia.lev1[2011-01-01T01:00:00Z/30m] |

----------------------------------------------------------------

-------------------------------


50


-------------------------------

-------------------------------------------------------------

| /rs_list, ds=aia.lev1[2011-01-01T01:00:00Z/30m]{image_lev1} |

-------------------------------------------------------------

----------------------


----------------------

---------------------


---------------------

IDL> help,index


As a result of screening out all images except for those from either the 171Å or 304Å channels of AIA, the 1,200 records of the earlier

example have now been reduced to 300.

As mentioned in Section4.2.2, there exist dataseries that contain subsets of the records(grouped by wavelength) from theaia.lev1

dataseries, namedaia.lev1_euv_12s , aia.lev1_uv_24s , andaia.lev1_vis_1h . The advantages of using these series

with ssw_jsoc_time2data.pro are that subselecting by wavelength channel is typically faster, and that the use of theCADENCE

keyword is now more straightforward when searching for AIA data. For example, one can reduce the cadence of the above query to 3

minutes as follows:


ssw_jsoc_time2data,’1-jan-2011 01:00’,’1-jan-2011 01:30’,index,$

ds=’aia.lev1_euv_12s’,waves=[171,304],cadence=’3m’

help,index

IDL> ssw_jsoc_time2data,’1-jan-2011 01:00’,’1-jan-2011 01:30’,index,$

IDL> ds=’aia.lev1_euv_12s’,waves=[171,304],cadence=’3m’

-----------------------------


-----------------------------

---------------------------------------------------------------------------

| ssw_jsoc_time2query output: aia.lev1_euv_12s[2011-01-01T01:00:00Z/30m@3m] |

---------------------------------------------------------------------------

-------------------------------


-------------------------------

------------------------------------------------------------------------

| /rs_list, ds=aia.lev1_euv_12s[2011-01-01T01:00:00Z/30m@3m]{image_lev1} |

------------------------------------------------------------------------

----------------------


----------------------

---------------------


---------------------

51


IDL> help,index


As a result of reducing the cadence, there are now only 20 records found that satisfy the query.

TECHNICAL NOTE: As mentioned in Section4.2.4, JSOC queries can be in either TAI or UTC time formats. By default, the

ssw_jsoc_time2data.pro routine will use the time format from the series definition. Users can force TAI or UTC by setting

either the/TAI or /UTC switches, respectively, in the call tossw_jsoc_time2data.pro . Additionally, theDATE-OBS

keyword discussed in Section4.2.4appears asDATE_OBSin index structures returned byssw_jsoc_time2data.pro .

TECHNICAL NOTE: In IDL Version 8.0, the “help,index ” IDL commands in the above examples do not work as shown here,

and instead behave as if the/STRUCTURESswitch has been set. This is apparently a bug and has been fixedin IDL Version

8.0.1, according tothis newsgroup discussion(Google login required).

5.3.3 Exporting Data

Downloading the image data (in addition to the header data) is achieved using thessw_jsoc_time2data.pro procedure but with

an additional argument:

ssw_jsoc_time2data,t0,t1,index,data,ds=’[dataseries name]’

On output, the newly addedDATAargument will contain the image data. What is actually beingdownloaded are Rice-compressed

images, butssw_jsoc_time2data.pro automatically uncompresses the data usingread_sdo.pro (as mentioned in Section4.6)

prior to exiting the procedure. There will be a one-to-one correspondence between the images in theDATAarray and the header

information in theINDEX array.

The MAX_FILES keyword limits the number of files that are downloaded.It is often prudent to use theMAX_FILES keyword

unless you know how much data you are requesting.Each full-resolution, full-disk AIA or HMI image is approximately 12MB when

compressed, expanding to 64MB when uncompressed.

Here is an example illustrating how one would download a sample of 4 AIA images from one minute of mission time:


t0=’1-jan-2011 01:00’

t1=’1-jan-2011 01:01’

ssw_jsoc_time2data,t0,t1,index,data,ds=’aia.lev1’,waves=[171,304],max_files=4

IDL> t0=’1-jan-2011 01:00’

IDL> t1=’1-jan-2011 01:01’

IDL> ssw_jsoc_time2data,t0,t1,index,data,ds=’aia.lev1’,waves=[171,304],max_files=4

-----------------------------


-----------------------------

----------------------------------------------------------------

| ssw_jsoc_time2query output: aia.lev1[2011-01-01T01:00:00Z/60s] |

----------------------------------------------------------------

-------------------------------


-------------------------------

-------------------------------------------------------------

52



| /rs_list, ds=aia.lev1[2011-01-01T01:00:00Z/60s]{image_lev1} |

-------------------------------------------------------------

----------------------


----------------------

---------------------


---------------------

FILTER>> ssnew=where(gt_tagval(index(ss),/wavelnth) eq 171 OR gt_tagval(index(ss),/wavelnth) eq

------------------------------------------------------------

| /export, ds=aia.lev1[2011-01-01T01:00:00Z/60s]{image_lev1} |

------------------------------------------------------------

-------------------

| segment subselect |

-------------------

--------------------------

| Throtteling to MAX_FILES |

--------------------------

-----------------------------------------------------------------------------

| Getting> http://jsoc.stanford.edu//SUM103/D190077569/S00000/image_lev1.fits |

-----------------------------------------------------------------------------

-----------------------------------------------------------------------------


-----------------------------------------------------------------------------

-----------------------------------------------------------------------------


-----------------------------------------------------------------------------

-----------------------------------------------------------------------------


-----------------------------------------------------------------------------

----------------------------------

| Tile compressed data handling... |

----------------------------------

-------------------------

| Uncompressing to> $HOME |

-------------------------

---------------------------------------------------------------------------------------

| $HOME//SUM103/D190077569/S00000/image_lev1.fits -> $HOME/AIA20110101_010036_0171.fits |

---------------------------------------------------------------------------------------

---------------------------------------------

| struct2fitshead - using procedure: sxaddpar |

---------------------------------------------

---------------------------------------------------------------------------------------


---------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------

53



---------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------


---------------------------------------------------------------------------------------

After downloading and uncompressing all of the requested data, the standardized AIA color tables can be loaded by using the

aia_lct.pro routine (see Section6.6.1), and a (downsampled) image can be displayed on screen. In the following example, the

appropriate color table is determined from the setting of theWAVELNTHkeyword from the image header:


aia_lct,wave=index(0).wavelnth,/load

window,0,xsiz=512,ysiz=512

tvscl,rebin(data(*,*,0),[512,512])

5.3.4 Exporting Specific Segments

For dataseries with multiple segments, the default ofssw_jsoc_time2data.pro is to export the first segment. (In the example in

the previous subsection, theimage_lev1 segment happens to be the first segment.) To download any of the other segments, simply

specify the segment name in theSEGMENTkeyword. Be aware that only one segment name can be specified at a time, and that a series

of data arrays with varying dimensions cannot be done in one pass. The following examples demonstrate how to download specific

segments usingssw_jsoc_time2data.pro .

As a first example, the records of thehmi.S_720s dataseries contains 24 image segments (the four Stokes I, Q,U, and V parameters

at each of six wavelengths). Because each of these segments are 4096×4096-pixel images, data from the same segment of multiple

records can be downloaded in one pass. For example, retrieving one of the segments (in this example, theQ3segment) for over an hour

of time is achieved as follows:


t0=’1-jan-2011 01:00’

t1=’1-jan-2011 02:00’

result=ssw_jsoc(ds=’hmi.S_720s’,/series_struct)

print,result.segments.name

ssw_jsoc_time2data,t0,t1,index,data,ds=’hmi.S_720s’,segment=’Q3’

IDL> t0=’1-jan-2011 01:00’

IDL> t1=’1-jan-2011 02:00’

IDL> result=ssw_jsoc(ds=’hmi.S_720s’,/series_struct)

IDL> print,result.segments.name

I0 Q0 U0 V0 I1 Q1 U1 V1 I2 Q2 U2 V2 I3 Q3 U3 V3 I4 Q4 U4 V4 I5 Q5 U5 V5

IDL> ssw_jsoc_time2data,t0,t1,index,data,ds=’hmi.S_720s’,segment=’Q3’

-------------------------------

| /SERIES_STRUCT, ds=hmi.S_720s |

-------------------------------

------------------------------------------------------------------------

| ssw_jsoc_time2query output: hmi.S_720s[2011.01.01_01h:00m:00s_TAI/60m] |

------------------------------------------------------------------------

-----------------------

54


| selecting segment> Q3 |

-----------------------

[...deletia...]

This can be done in one pass because allQ3segments are 4096×4096-pixel images.

In contrast, retrieving multiple segments from the same record (even if they are all dimensioned alike) must be done via multiple

calls tossw_jsoc_time2data.pro :


t0=’1-jan-2011 01:00’

t1=’1-jan-2011 01:01’

result=ssw_jsoc(ds=’hmi.S_720s’,/series_struct)

print,result.segments.name

allq=where(strpos(result.segments.name,’Q’) eq 0) ; get desired segment indices

ssw_jsoc_time2data,t0,t1,index,data,ds=’hmi.S_720s’,$

segment=result.segments(allq(0)).name ; do first one

for i=1,5 do begin $

ssw_jsoc_time2data,t0,t1,itemp,dtemp,ds=’hmi.S_720s’,$

segment=result.segments(allq(i)).name & $ ; do the others

data=[[[data]],[[dtemp]]] & endfor ; concatenate

help,index,data

IDL> t0=’1-jan-2011 01:00’

IDL> t1=’1-jan-2011 01:01’

IDL> result=ssw_jsoc(ds=’hmi.S_720s’,/series_struct)

IDL> print,result.segments.name

I0 Q0 U0 V0 I1 Q1 U1 V1 I2 Q2 U2 V2 I3 Q3 U3 V3 I4 Q4 U4 V4 I5 Q5 U5 V5

IDL> allq=where(strpos(result.segments.name,’Q’) eq 0) ; get desired segment indices

IDL> ssw_jsoc_time2data,t0,t1,index,data,ds=’hmi.S_720s’,$

IDL> segment=result.segments(allq(0)).name ; do first one

-------------------------------


-------------------------------

------------------------------------------------------------------------

| ssw_jsoc_time2query output: hmi.S_720s[2011.01.01_01h:00m:00s_TAI/60s] |

------------------------------------------------------------------------

-----------------------


-----------------------

[...deletia...]

IDL> for i=1,5 do begin $

IDL> ssw_jsoc_time2data,t0,t1,itemp,dtemp,ds=’hmi.S_720s’,$

IDL> segment=result.segments(allq(i)).name & $ ; do the others

IDL> data=[[[data]],[[dtemp]]] & endfor ; concatenate

-------------------------------


-------------------------------

55


------------------------------------------------------------------------

| ssw_jsoc_time2query output: hmi.S_720s[2011.01.01_01h:00m:00s_TAI/60s] |

------------------------------------------------------------------------

-----------------------


-----------------------

[...deletia...]

IDL> help,index,data

INDEX STRUCT = -> MS_034130194003 Array[1]

DATA FLOAT = Array[4096, 4096, 6]

Here, theQ0–Q5 segments from the same record are being read one at a time, andconcatenated into a singleDATAarray. Recall that

because each of these segments are all part of the same record, the INDEX structure (which contains the header information for this

record) will be the same for each segment, and thus there is noneed to concatenate these.

Retrieving segments from a set of records that are not dimensioned alike must also be done via multiple calls. In the following

example, we use thespikes segment from theaia.lev1 dataseries, which contains the list of pixels that have beendespiked. (For

more information on despiking, see Section6.3.) Because the number of despiked pixels varies from image toimage, the amount of data

stored in thespikes segment varies from record to record. One way to store this data within IDL is by using pointers, which enables

the dynamic allocation of arrays:


t0=’1-jan-2011 01:00’

t1=’1-jan-2011 01:01’

ssw_jsoc_time2data,t0,t1,index,ds=’aia.lev1’,waves=[171] ; get 5 records

data=ptrarr(n_elements(index)) ; set up a pointer array

for i=0,n_elements(index)-1 do begin & $

ssw_jsoc_time2data,index(i).t_obs,index(i).t_obs,ds=’aia.lev1’, $

segment=’spikes’,itemp,dtemp & $ ; get spikes segment

data(i)=ptr_new(dtemp) & endfor

for i=0,n_elements(index)-1 do help,*data(i) ; look at the data

IDL> t0=’1-jan-2011 01:00’

IDL> t1=’1-jan-2011 01:01’

IDL> ssw_jsoc_time2data,t0,t1,index,ds=’aia.lev1’,waves=[171] ; get 5 records

-----------------------------


-----------------------------

----------------------------------------------------------------


----------------------------------------------------------------

-------------------------------


-------------------------------

-------------------------------------------------------------

| /rs_list, ds=aia.lev1[2011-01-01T01:00:00Z/60s]{image_lev1} |

-------------------------------------------------------------

56


FILTER>> ssnew=where(gt_tagval(index(ss),/wavelnth) eq 171,sscnt)

FILTER>> ssnew=where(strmatch(gt_tagval(index(ss),/img_type),’LIGHT’,fold_case=fold_case) eq 1,sscnt)

IDL> data=ptrarr(n_elements(index)) ; set up a pointer array

IDL> for i=0,n_elements(index)-1 do begin & $

IDL> ssw_jsoc_time2data,index(i).t_obs,index(i).t_obs,ds=’aia.lev1’, $

IDL> segment=’spikes’,itemp,dtemp & $ ; get spikes segment

IDL> data(i)=ptr_new(dtemp) & endfor

-----------------------------


-----------------------------

---------------------------------------------------------------


---------------------------------------------------------------

---------------------------

| selecting segment> spikes |

---------------------------

[...deletia...]

IDL> for i=0,n_elements(index)-1 do help,*data(i) ; look at the data

<PtrHeapVar44> LONG = Array[4995, 3]





5.4 Requesting a Cutout Using SSW

The online cutout service described in Section4.3 can also be accessed from SSWIDL using thessw_cutout_service.pro

procedure:

ssw_cutout_service,t0,t1,ref_helio=’[centroid coordinates]’,fovx=[width],fovy=[height],$

waves=[wavelengths],instr=’[instrument]’,cadence=[cadence string],$

max_frames=[maximum frames],email=’[email protected]’

To use this routine, the user should specify position (REF_HELIO is the latitude and longitude of the center of the cutout), the field

of view (FOVXandFOVYare the sizes of the field of view, with arcseconds as the default unit), and the wavelength channels of interest

(via theWAVESkeyword). TheMAX_FRAMESkeyword should be set to the desired number of frames in the time series; setting this

keyword to a value that is lower than that implied by full cadence will result in the time series being subsampled in time. Region-tracking

is automatically turned on, unless the/NOTRACKkeyword is set. A more comprehensive list of keywords and their default values can

be found atthis webpagewhich lists the API for the cutout service.

As an example, suppose a user wants a tracked cutout from the 171Å and 304Å channels of AIA, sampled every four minutes over

the course of a two-hour time span. The request might look similar to this sequence:

t0=’01-jan-2011 01:00’

t1=’01-jan-2011 03:00’

ssw_cutout_service,t0,t1,fovx=256,fovy=256,ref_helio=’N35W20’,$

waves=[171,304],max_frames=30, instrument=’aia’,email=’[email protected]’

57

http://www.lmsal.com/solarsoft/ssw_service/ssw_service_track_fov_api.html


In this example, setting theMAX_FRAMESkeyword to 30 in combination with the two-hour time window asspecified in theT0 andT1

variables results in the desired four-minute image cadence.

As another example,ssw_cutout_service.pro can be used to spatially downsample full-disk images to a more manageable

size via use of the/FULL_DISK switch (which supersedes theREF_HELIO, FOVX, andFOVYkeywords), with a query similar to the

following:

t0=’01-jan-2011 01:00’

t1=’01-jan-2011 03:00’

ssw_cutout_service,t0,t1,fovx=256,fovy=256,/full_disk,$

waves=[171,304],max_frames=30, instrument=’aia’,email=’[email protected]’

In both cases, when a cutout request is finished, an e-mail containing a link to the data will be sent to the e-mail address provided in the

EMAIL keyword.

5.5 Using the SSW Interface to the VSO

Queries to the VSO can either be submitted online (see Section 4.4) or through SSWIDL. Using SSWIDL, the basic sequence of

query→select→download that users undergo when using the online portal is implemented using IDL routines in theVSOpackage from

SSW, as is described in the following subsections.

5.5.1 Querying the VSO

To run a VSO query, use thevso_search.pro routine. The routine has several keywords that may be usefulfor constructing queries,

includingPROVIDER, INSTRUMENT, PHYSOBS, andWAVE, that match the checkboxes on the online VSO entry page (see Figure20

of Section4.4). For example, in SSWIDL a query similar to that of Figure21 for one hour’s worth of AIA data can be achieved as

follows:


result=vso_search(’2011-01-01 1:00’,’2011-01-01 1:59’,inst=’aia’)

help,result


IDL> result=vso_search(’2011-01-01 1:00’,’2011-01-01 1:59’,inst=’aia’)

Records Returned : JSOC : 2362/2362


IDL> help,result

RESULT STRUCT = -> VSORECORD Array[2362]


** Structure VSORECORD, 13 tags, length=256, data length=252:

TIME STRUCT -> VSOTIME Array[1]

EXTENT STRUCT -> VSOEXTENT Array[1]

WAVE STRUCT -> VSOWAVE Array[1]

DETECTOR STRING ’’

INSTRUMENT STRING ’AIA’

SOURCE STRING ’SDO’

PROVIDER STRING ’JSOC’

INFO STRING ’AIA level 1, 4096x4096 [2.000 exposure] [100.00 percentd]’

PHYSOBS STRING ’intensity’

58


FILEID STRING ’aia__lev1:171:1072918835’

SIZE FLOAT 66200.0

EXPTIME FLOAT 1.99961

DARK LONG 0

ECLIPSE LONG 0

PERCENTD FLOAT 100.000

URL STRING ’’

GETINFO STRING ’’

The above search returns 2362 records corresponding to images from all of the AIA channels. Some details about each record are

stored in the array of structures returned by thevso_search.pro routine, with each element in the array corresponding to a valid

record. To narrow this list down by wavelength channel, use theWAVEkeyword:


result=vso_search(’2011-01-01 1:00’,’2011-01-01 1:59’,inst=’aia’,wave=171)

help,result

IDL> result=vso_search(’2011-01-01 1:00’,’2011-01-01 1:59’,inst=’aia’,wave=171)



IDL> help,result


This query results in a more manageable 296 records. Additionally, this query can be further refined by specifying a sampling cadence

(in seconds) in theSAMPLEkeyword:


result=vso_search(’2011-01-01 1:00’,’2011-01-01 1:59’,inst=’aia’,wave=171,sample=60)

help,result

IDL> result=vso_search(’2011-01-01 1:00’,’2011-01-01 1:59’,inst=’aia’,wave=171,sample=60)



IDL> help,result


The above query returns 60 AIA records from the 171Å channel from the requested hour.

TECHNICAL NOTE: SDO data stored on the various VSO servers may be subject to expiration. When this happens, the result

of vso_search.pro will be an empty string. In these cases, users will have to request the data directly from the JSOC, as

demonstrated in Sections4.2and5.3.

TECHNICAL NOTE: In IDL Version 8.0, the “help,result ” IDL commands in the above examples do not work as shown here,

and instead behave as if the/STRUCTURESswitch has been set. This is apparently a bug and has been fixedin IDL Version

8.0.1, according tothis newsgroup discussion(Google login required).

59



5.5.2 Downloading Data from the VSO

To retrieve records, one simply passes an array of structures returned fromvso_search.pro to thevso_get.pro routine, as

demonstrated in this two-image example:



log=vso_get(result,out_dir=’data’,filenames=fnames,/rice)

print,transpose(fnames)

IDL> result=vso_search(’2011-01-01 1:00’,’2011-01-01 1:01’,inst=’aia’,wave=171,sample=60)



IDL> log=vso_get(result,out_dir=’data’,filenames=fnames,/rice)

% VSO_GET: This will download 2 file(s)

2 : http://vso.tuc.noao.edu/cgi-bin/drms_test/drms_export.cgi?series=aia__lev1;compress=rice;record=1

% RDWRT_BUFF: Please wait. Downloading...

% File: /cgi-bin/drms_test/drms_export.cgi?series=aia__lev1;compress=rice;record=171_1072918895-10729

% Size: 12343680 bytes

% From: vso.tuc.noao.edu

% To: data

% HTTP::COPY: 12343680 bytes of 12343680 total bytes copied in 3.24 seconds

% HTTP::COPY: Wrote 12343680 bytes to file data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits

1 : http://vso.tuc.noao.edu/cgi-bin/drms_test/drms_export.cgi?series=aia__lev1;compress=rice;record=1

% RDWRT_BUFF: Please wait. Downloading...

% File: /cgi-bin/drms_test/drms_export.cgi?series=aia__lev1;compress=rice;record=171_1072918835-10729

% Size: 12343680 bytes

% From: vso.tuc.noao.edu

% To: data

% HTTP::COPY: 12343680 bytes of 12343680 total bytes copied in 3.62 seconds

% HTTP::COPY: Wrote 12343680 bytes to file data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits

Downloading completed

IDL> print,transpose(fnames)

data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits


In this example, theOUT_DIRkeyword is used to specify an (already existing) output directory, theFILENAMESkeyword contains the

list of files downloaded, and the/RICE switch is used to indicate that the data are to be downloaded in a Rice-compressed format (which,

compared with data in an uncompressed format, significantlyreduces the download time). Unlike thessw_jsoc_time2data.pro

routine described in the previous section, thevso_get.pro does not load the data into an IDL variable; it instead only downloads

the requested FITS files to the specified directory. To uncompress Rice-compressed files, one of the utility programs mentioned in

Section4.6or theread_sdo.pro routine (see Section5.6below) must be used.

TECHNICAL NOTE: If a proxy server is being used, then SSW needs to be proxy-aware in order forvso_get.pro to work

properly; otherwise,vso_get.pro will appear to hang. To configure SSW to work with a proxy server, see Section5.1.3.

When SSW is properly configured to use a proxy server, the output to thevso_get.pro routine from the above example will

be slightly different than what is shown.

60


5.6 Uncompressing the Image Data

5.6.1 Usingread_sdo.pro

The SSWIDL routineread_sdo.pro can be used to uncompress and read both the header information and the data into IDL variables:

read_sdo,fnames,index,data,/nodata,/noshell

The input argumentFNAMEScontains a list of filenames to be uncompressed and read into IDL. On output, the index (header) informa-

tion and image data will appear in the variablesINDEX andDATA, respectively. Either setting the/NODATAkeyword or omitting the

DATAargument will result in only the header information being read in. Setting/NOSHELLmay result in a slight speedup (if it works)

when the underlyingimcopy utility (from theCFITSIO subroutine library) is called during the uncompression step.

The examples below assume that one has first downloaded from the VSO the small (two-image) AIA Level 1.0 dataset demonstrated

in Section5.5.2:




After doing so, one can useread_sdo.pro to uncompress the data:


read_sdo,fnames,index,data

help,index,data

IDL> read_sdo,fnames,index,data

------------------------

| Uncompressing to> /tmp |

------------------------

-------------------------------------------------------------------------------------------------

| data/aia.lev1.171A_2011-01-01T08_00_00.34Z.image_lev1.fits -> /tmp/AIA20110101_080000_0171.fits

-------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------


INDEX STRUCT = -> MS_033956771002 Array[2]

DATA INT = Array[4096, 4096, 2]

At this point, the data have been uncompressed and are now stored in theDATAvariable. One can load the standardized AIA color tables

via theaia_lct.pro routine (see Section6.6.1) prior to displaying an image on screen. In the following example, the first image

in the DATAarray is displayed using the appropriate color table, whichis determined from theWAVELNTHkeyword from the image

header:


aia_lct,wave=index(0).wavelnth,/load

window,0,xsiz=512,ysiz=512

tvscl,rebin(data(*,*,0),[512,512])

61



The above example shows how to uncompress and load full-resolution AIA images into IDL, and in order to be able to display the

image onscreen, the IDL functionREBIN was used to downsize the image. However, the downsizing stepcan also be performed within

read_sdo.pro by setting theOUTSIZEkeyword equal to the desired output image size:


read_sdo,fnames,index,data,outsize=[700,700]

help,index,data

IDL> read_sdo,fnames,index,data,outsize=[700,700]

------------------------


------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------



DATA INT = Array[700, 700, 2]

The dimensions in theOUTSIZE keyword need not be the same. Users should also take note of the fact that, when specifying the

OUTSIZEkeyword, the downsizing is performed inread_sdo.pro by using IDL’sCONGRIDfunction, which has a different behavior

than theREBIN function used in the earlier example of this section: by default, when shrinking an array,CONGRIDuses nearest-

neighbor resamplingwhile REBIN averages multiple points, and the result is generallynot the same.

The read_sdo.pro routine accepts additional argumentsLLX, LLY, NX, andNY that result in a cutout being returned. The

cutout is specified by providing its lower-left corner in pixel coordinates inLLX andLLY, along with its dimensions in theNXandNY

arguments, as follows:


read_sdo,fnames,index,data,1024,2048,1200,600

help,index,data

IDL> read_sdo,fnames,index,data,1024,2048,1200,600

------------------------


------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------



DATA INT = Array[1200, 600, 2]

62


Here, a rectangular 1200×600-pixel cutout starting at pixel location (1024,2048) isreturned byread_sdo.pro . Please note that

specifying both the cutout information and theOUTSIZEkeyword is likely to give unexpected results, and is not recommended.

TECHNICAL NOTE: The FITS world coordinate system standard uses the convention that the lower-left pixel of an image is num-

bered 1 (and not 0, as in IDL). Because HMI and AIA data conformto this standard, the user is urged to take care in determining

the values passed toread_sdo.pro in theLLX andLLY arguments when specifying a cutout. For more information onworld

coordinates, seeWells et al. (1981)or Greisen & Calabretta (2002)or, for solar-specific information,Thompson (2006).

5.6.2 Usingread_sdo.pro with a Shared Library

The default method discussed above in Section5.6.1works by spawning theimcopy utility from within SSWIDL. Theimcopy utility

is part of theCFITSIO subroutine library, and is used to read in a compressed FITS file and write out an uncompressed FITS file to

a temporary disk location, only to be read in byread_sdo.pro moments later. This process contains an extra I/O sequence,and

consequently often results in a noticeable performance reduction, especially when large volumes of data are being processed.

To mitigate this problem, theread_sdo.pro procedure also accepts a/USE_SHARED_LIB switch that, when set, will cause

read_sdo.pro to attempt to use an external, pre-compiled shared library to perform the uncompression and return the result directly

to IDL in memory. The shared library is accessed using IDL’sCALL_EXTERNALinterface, and so needs to be compatible with the

machine architecture and the version (32-bit or 64-bit) of IDL being used. Unfortunately, pre-compiled libraries do not exist for all

platforms. Ifread_sdo.pro is called from a platform for which there is no library available, read_sdo.pro will default to using

imcopy to perform the uncompression as in Section5.6.1, although we encourage users familiar with compiling C codeto attempt to

create the shared library (see below).

In this example using the/USE_SHARED_LIBswitch, assume that one has first downloaded from the VSO the same small (two-

image) AIA Level 1.0 dataset demonstrated above in Section5.6.1:




Then, uncompress the data using the shared library as follows:


read_sdo,fnames,index,data,/use_shared_lib

help,index,data

IDL> read_sdo,fnames,index,data,/use_shared_lib

--------------------------------------------------------------------

| reading data/aia.lev1.171A_2011-01-01T01_00_00.34Z.image_lev1.fits |

--------------------------------------------------------------------

--------------------------------------------------------------------

| reading data/aia.lev1.171A_2011-01-01T01_01_00.34Z.image_lev1.fits |

--------------------------------------------------------------------



DATA INT = Array[4096, 4096, 2]

If a pre-compiled shared library is not available for the machine architecture on which you are running SSWIDL, you will see the

following warning message when callingread_sdo.pro with the/USE_SHARED_LIBswitch:

63

http://adsabs.harvard.edu/abs/1981A%26AS...44..363W

http://adsabs.harvard.edu/abs/2002A%26A...395.1061G

http://adsabs.harvard.edu/abs/2006A%26A...449..791T



--------------------------------------------------------------------------------

| fitsio/shared object request but not available for this OS/ARCH - using imcopy |

--------------------------------------------------------------------------------

When the/USE_SHARED_LIB switch is set, optional I/O keywords (such asOUTDIR, PARENT_OUT, ONLY_UNCOMPRESS, or

COMP_DELETE) that can be passed toread_sdo.pro are not relevant and thus will be ignored. However, the imagemanipulation

capabilities ofread_sdo.pro , such as those invoked by setting theOUTSIZE keyword or by providing theLLX, LLY, NX, and

NY arguments (as demonstrated above in Section5.6.1), should still work with the/USE_SHARED_LIB switch. If both the cutout

information and theOUTSIZE keyword are specified,and /USE_SHARED_LIB is set, thenread_sdo.pro will first extract the

cutout and then resize the image. (However, as stated in Section 5.6.1above, if/USE_SHARED_LIB is not set, then specifying both a

cutout andOUTSIZEmay give unexpected results, and is not recommended.)

TECHNICAL NOTE: The FITS world coordinate system standard uses the convention that the lower-left pixel of an image is num-

bered 1 (and not 0, as in IDL). Because HMI and AIA data conformto this standard, the user is urged to take care in determining

the values passed toread_sdo.pro in theLLX andLLY arguments when specifying a cutout. For more information onworld

coordinates, seeWells et al. (1981)or Greisen & Calabretta (2002)or, for solar-specific information,Thompson (2006).At present, shared libraries exist for only a few operating systems. If the value of the IDL variableOSARC = !version.os

+’_’+ !version.arch is not any ofdarwin_x86_64 , linux_x86 , or linux_x86_64 , then there is currently no pre-

compiled shared library compatible withread_sdo.pro . However, if this is the case, we encourage users to attempt to create a

working shared library and contribute such libraries for broader distribution by SSW. Libraries for machine architectures not listed here

can be contributed by sending the library in an e-mail attachment to the address listed onthis webpage.

To create the shared library, a C compiler is needed (in the example belowgcc has been chosen). The first step is to compile the

CFITSIO subroutine library, and take note of the location oflibcfitsio.a (information in Section4.6.1may be useful here). Next,

compile the source code for the shared library. To do this, locate the two files namedget_info.c andget_data.c , available

in the $SSW_ONTOLOGY/binaries/sources directory (or Windows equivalent) of your SSWIDL distribution. Also locate the

header fileidl_export.h that is distributed with IDL (usually found in$IDL_DIR/external/include ). The compilation step

is typically done using commands like

gcc -I/usr/include -I$IDL_DIR/external/include -c get_info.c get_data.c

to create the compiled objectsget_info.o andget_data.o . If this step fails, read through the error output to see whatfiles or

objects are missing, and ensure that the compiler knows the correct locations of the various include files needed for compilation. Once

the objects have been created, issue a command like

gcc -shared -o fitsio.so get_info.o get_data.o libcfitsio.a

to create a shared library namedfitsio.so . Note that Mac OS X uses-bundle instead of-shared to indicate to the compiler

and linker that a shared library is being created. Once thefitsio.so shared library has been created, users can callread_sdo.pro

with the /USE_SHARED_LIB keyword by providing its path via theSHARED_LIB_PATHkeyword. It is likely that the compiler

may require additional switches to be set before a usable shared library can be created, and as a result some consultationwith compiler

manuals may be necessary. Be forewarned that usingread_sdo.pro with incompatible shared library files may give incorrect or

unexpected results at best, and may cause IDL to hang or crashat worst.

64







6 How to Process AIA Data

In the following sections, we will describe the more common data-processing tasks related to the preparation of AIA datafor scientific

use, including information outlining how to convert AIA data from Level 1.0 to Level 1.5 (Section6.1), how to align HMI Level 1.5 data

with AIA Level 1.5 images (Section6.2), how to find out which pixels have been affected during the despiking process (Section6.3), how

to PSF-deconvolve AIA Level 1.0 images (Section6.4), how to determine the filter-response profiles for the AIA telescopes (Section6.5),

as well as some miscellaneous tasks (Section6.6).

These tasks by and large require IDL, along with an up-to-date SSW installation with theAIA package (see Section5 on how to use

SSW and install or update theAIA package). In some cases, theONTOLOGYpackage also needs to be installed. As with Section5, this

section contains several examples of SSWIDL code. To guide users, many of these examples are given in pairs of code blocks: the first

block contains code that users can copy and paste into their IDL session, and the second block shows the output that shouldresult.

6.1 Converting AIA Data to Level 1.5

AIA data at Level 1.0 have already been flat-fielded, and processed to remove bad pixels and spikes. However, AIA data at Level 1.0

are not co-aligned and likely have slightly different platescales and roll angles across the various AIA wavelength channels, making

multi-wavelength analyses problematic. Furthermore, thevalues of the keywordsCDELT1, CRPIX1, CDELT2, CRPIX2, andCROTA2

that specify the orientation and scale for AIA Level 1.0 datamay be out of date, as these values are likely to have been determined from

the pointing information in effect at the time of the creation of the archived image file. Because the pointing information may undergo

modest changes over time, and is occasionally subject to major revisions (such as in September 2012 when the more accurate telescope

pointings acquired during the transit of Venus in June 2012 were implemented), one shouldnot assume that the image headers contain

the most accurate pointing information.

To remedy these issues, the data should be converted to Level1.5, which can be done using theaia_prep.pro routine available

through SSW. Converting AIA data to Level 1.5 entails co-aligning images from all of the AIA channels, rescaling the images to a

common 0.6′′ plate scale, and rotating the images so that the solar east-west and north-south axes are aligned with the horizontal and

vertical axes of the image, respectively. Theaia_prep.pro routine will seek the currently best-known pointing information before

performing these manipulations. Please note that data at Level 1.5 arenot exposure-time corrected.

In its most basic usage, theaia_prep.pro routine is called as follows:

aia_prep,fnames,indices,out_index,out_data

The requiredFNAMESandINDICES arguments respectively contain either the index and data arrays from an already-loaded dataset

(such as returned fromread_sdo.pro ) or a list of filenames (as a string array) and their corresponding index structures. In the latter

case,aia_prep.pro will use read_sdo.pro to read the data into SSWIDL prior to performing the conversion to Level 1.5. Also

in the latter case, setting theINDICES argument to a value of-1 indicates that all files inFNAMESare to be processed. In both cases,

the resulting Level 1.5 index structures and data are returned to IDL in theOUT_INDEXandOUT_DATAarguments.

The resulting Level 1.5 index structures and image data can be written to disk as follows:

aia_prep,fnames,indices,out_index,out_data,outdir=[output directory],/do_write_fits

Here, the/DO_WRITE_FITS switch indicates that the resulting Level 1.5 data are to be written to disk in the output directory specified

by theOUTDIRkeyword. TheOUT_INDEXandOUT_DATAarguments, which are optional, can be omitted in this case.

For full-disk images, the data will be translated so that suncenter is located at the center of each image. For cutouts, specifying the

/CUTOUTswitch is needed, which indicates toaia_prep.pro that the image-center point should be taken from a referenceframe

(instead of using sun-center as the alignment point). This reference frame is assumed to be the first frame in the image sequence, but can

also be set to a different frame by specifying theINDEX_REFkeyword.

After runningaia_prep.pro , the keywordsCDELT1, CRPIX1, CDELT2, andCRPIX2 should all be updated to reflect the new

plate scale and image centering. Other keywords affected includeLVL_NUM, R_SUN, CROTA1, andCROTA2, as well as the statistical

65


keywords (e.g.,DATAMIN, DATAMAX, DATAMEDN, DATAMEAN, DATARMS, DATASKEW, DATAKURT, etc.). Processing data using

aia_prep.pro should preserve any binning or cropping that has been done prior to export.

To determine whether AIA data are at Level 1.0 or Level 1.5, users can check theLVL_NUMkeyword in the image headers. Much

of what is served, including AIA data requested directly from the JSOC (as described in Sections4.2 and5.3) and from the VSO (as

described in Sections4.4 and5.5), are at Level 1.0. AIA data exported using the SSW cutout service (as described in Sections4.3

and5.4), can be at either Level 1.0 or Level 1.5. If theaia_prep.pro routine detects via theLVL_NUMkeyword that the data have

already been processed to Level 1.5, it will simply return the data unchanged.

As an example of how to useaia_prep.pro , first download the small (two-image) AIA Level 1.0 dataset used as an example in

Section5.5.2:




Converting these Level 1.0 data to Level 1.5 is performed as follows:


print,transpose(fnames)

aia_prep,fnames,-1,out_index,out_data

help,out_index,out_data

IDL> print,transpose(fnames)



IDL> aia_prep,fnames,-1,out_index,out_data

Running AIA_PREP.PRO V4.13

------------------------


------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------

-------------------------------------------

| Removing uncompressed versions on request |

-------------------------------------------

----------------------------------------

| /MIXED_COMP set but already homogenous |

----------------------------------------

------------------------


------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------

-------------------------------------------


-------------------------------------------

66


----------------------------------------

| /MIXED_COMP set but already homogenous |

----------------------------------------

------------------------


------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------

-------------------------------------------


-------------------------------------------

IDL> help,out_index,out_data

OUT_INDEX STRUCT = -> MS_033967876001 Array[2]

OUT_DATA INT = Array[4096, 4096, 2]

TECHNICAL NOTE: If it is necessary to revert AIA data to their state prior to the despiking process, as discussed in Section6.3,

and/or to perform a PSF-deconvolution, as in Section6.4, these should be performed prior to converting AIA data to Level 1.5.

These operations are not common, but may be necessary in somecases.

TECHNICAL NOTE: The FITS world coordinate system keywords that specify theorientation and scale for SDO images are

CDELT1, CRPIX1, CDELT2, CRPIX2, andCROTA2. TheCRPIX1 andCRPIX2 keywords specify the x and y offset in pixels

from the lower-left pixel of the image to sun center, theCDELT1andCDELT2keywords specify the plate scale in arcseconds

per pixel, and theCROTA2keyword specifies the image roll in degrees relative to solarnorth (and is positive is the solar north

pole is rotated clockwise with respect to the image). The FITS world coordinate system standard specifies (among many other

things) that theCRPIX1 andCRPIX2 keywords use the convention that the lower-left pixel is numbered 1 (and not 0, as in

IDL). Consequently, a full-resolution 4096×4096-pixel AIA Level 1.5 image will have these keywords set to 2048.5, because a

4096-pixel array has pixels numbered from 1 to 4096, for which the image center lies in between pixels 2048 and 2049 (hence

2048.5). For more information on world coordinates, seeWells et al. (1981)or Greisen & Calabretta (2002)or, for solar-specific

information,Thompson (2006).

TECHNICAL NOTE: The default behavior foraia_prep.pro is to use the best-available values of the plate scale, pointing,

and roll angle for each image. These values are not necessarily those given in the image headers, though they are usually fairly

close. The best-avilable pointing parameters are derived from the master pointing table (MPT) in effect at the time of the call to

aia_prep.pro , which is based upon “pointing solutions” calculated on a weekly basis and stored in a JSOC series. In cases

where, for example, a user wishes to reproduce the results ofan earlier study, it may be necessary to use a different pointing

solution than the current default, because the pointing-dependent keyword values used in the original analysis may be different

than those associated with the current MPT. In these cases, acall toaia_prep.pro may specify either a particular MPT record

(via theMPT_RECkeyword) or a particular reference date (via theT_REFkeyword) to force the use of an earlier pointing solution.

One can alternatively force the use of the pointing-relatedkeyword values in the header by setting the switch/USE_HDR_PNT,

though this is not recommended. Instances where some awareness of MPT matters may be useful include the analysis of near-

real-time data (where the pointing is based on extrpolated MPT records and thus is not final) or after major MPT updates (such

as in September 2012 when the calibration data acquired during the transit of Venus in June 2012 was used to better define the

telescope plate scale).

67





6.2 Co-Aligning HMI Data with AIA Data

Because HMI images have a different plate scale than AIA images (HMI images are slightly more “zoomed-in” than AIA images), the

ability to overlay data from HMI onto data from AIA requires the extra step of remapping data from one instrument so that ithas the same

plate scale as the other. Any roll angle differences must also be taken into account. To facilitate such alignment, theaia_prep.pro

routine discussed in Section6.1can be used to process HMI Level 1.5 data so that it is co-aligned with AIA Level 1.5 data.

To demonstrate this, the following example will download a concurrent AIA image and HMI line-of-sight magnetogram fromthe

VSO, and demonstrate how to useaia_prep.pro to regrid them so that they have the same plate scale and orientation.


result1=vso_search(’2011-01-01 1:00’,’2011-01-01 1:01’,inst=’aia’,wave=171,sample=60)

result2=vso_search(’2011-01-01 1:00’,’2011-01-01 1:01’,inst=’hmi’,physobs=’los_magnetic_field’)

log=vso_get(result1(0),out_dir=’data’,filenames=fnames1,/rice)

read_sdo,fnames1,index_171,data_171

log=vso_get(result2(0),out_dir=’data’,filenames=fnames2,/rice)

read_sdo,fnames2,index_Blos,data_Blos

aia_prep,index_171,data_171,out_index_171,out_data_171

aia_prep,index_Blos,data_Blos,out_index_Blos,out_data_Blos

IDL> result1=vso_search(’2011-01-01 1:00’,’2011-01-01 1:01’,inst=’aia’,wave=171,sample=60)



IDL> result2=vso_search(’2011-01-01 1:00’,’2011-01-01 1:01’,inst=’hmi’,physobs=’los_magnetic_field’


IDL> log=vso_get(result1(0),out_dir=’data’,filenames=fnames1,/rice)

[...deletia...]

IDL> read_sdo,fnames1,index_171,data_171

------------------------


------------------------

-------------------------------------------------------------------------------------------------


-------------------------------------------------------------------------------------------------

IDL> log=vso_get(result2(0),out_dir=’data’,filenames=fnames2,/rice)

[...deletia...]

IDL> read_sdo,fnames2,index_Blos,data_Blos

------------------------


------------------------

----------------------------------------------------------------------------------------------

| data/hmi.m_45s.2011.01.01_01_01_30_TAI.magnetogram.fits -> /tmp/HMI20110101_010025_6173.fits |

----------------------------------------------------------------------------------------------

IDL> aia_prep,index_171,data_171,out_index_171,out_data_171


IDL> aia_prep,index_Blos,data_Blos,out_index_Blos,out_data_Blos


68


Figure 24: Sample HMI magnetogram images before and afteraia_prep.pro . Note that a roll angle of approximately 180 degrees

has been accounted for.

Comparing the images stored in the variablesDATA_171 and DATA_BLOS, which contain the data before being run through

aia_prep.pro with those contained inOUT_DATA_171andOUT_DATA_BLOS, will reveal that the latter set of images have been

rescaled and derotated to a common plate scale. The following set of commands should open two IDL graphics windows containing

images similar to those seen in Figures24and25:


window,0 & !p.multi=[0,2,1]

plot_image,data_blos<300>(-300),title=’BEFORE AIA_PREP’

plot_image,out_data_blos<300>(-300),title=’AFTER AIA_PREP’

window,1 & !p.multi=[0,2,1]

plot_image,alog(data_171>10),title=’BEFORE AIA_PREP’

plot_image,alog(out_data_171>10),title=’AFTER AIA_PREP’

6.3 Despiking/Respiking AIA Data

High-energy particles (e.g., cosmic rays) interacting with the detectors on AIA result in data values in the image data that are abnormally

high. The sites at which such high data values occur are usually spatially localized, and consequently are referred to asspikesin the

69


Figure 25: Sample AIA 171Å-channel images before and afteraia_prep.pro . The right-hand image is only slightly different than

the left-hand image, but is now co-aligned with the right-hand magnetogram image in Figure24.

data. The standard processing scheme used to produce AIA Level 1.0 datadespikes(removes the spikes from) each image using a fast

algorithm that searches for single pixels (or small clumps of pixels) that appear to have usually high data numbers relative to a selection

of neighboring pixels. The algorithm flags such pixels, and determines a replacement value based on the values of those neighboring

pixels that are deemed valid.

The current despiking algorithm is generally very good for many, if not most, scientific applications, but it is not completely fool-

proof. For example, flare kernels can often be erroneously flagged (since they too fit the general profile of a spike), and so some users

may want torespike(undo the despiking of) their data and apply their own despiking algorithm. In this section, we will describe how

to get the listing of pixels that have been affected by the despiking algorithm, and how to restore these pixels with theiroriginal data

numbers.

Respiking the data is performed using theaia_respike.pro routine available through SSW:

aia_respike,in_index,in_data,out_index,out_data

Here, theIN_INDEX and IN_DATA arguments contain the input Level 1.0 index structure and data array. TheOUT_INDEXand

OUT_DATAwill contain the resulting respiked index and data. At present, theaia_respike.pro routine will operate on only one

image record at a time. As an example, let’s assume that one first downloads and uncompresses the first image in the AIA Level1.0

dataset used in Section5.5.2to demonstrate the SSW interface to the VSO:

70




log=vso_get(result(0),out_dir=’data’,filenames=fnames,/rice)


Respiking the image is performed as follows:


print,index.nspikes ; NSPIKES keyword contains number of spikes

aia_respike,index,data,out_index,out_data

print,total(data ne out_data)

IDL> print,index.nspikes ; NSPIKES keyword contains number of spikes

4995

IDL> aia_respike,index,data,out_index,out_data

----------------------


----------------------

---------------------


---------------------

------------------------------------------------------------------------

| Copying> http://jsoc.stanford.edu/SUM100/D121871420/S00000/spikes.fits |

------------------------------------------------------------------------

MRDFITS: Image array (4995,3) Type=Int*4

IDL> print,total(data ne out_data)

4995.00

As is evident from this example, the number of despiked pixels is given by theNSPIKES keyword in the image header. The last

statement confirms that theaia_respike.pro routine changed the correct number of pixels (4995 for this particular image) that

were despiked during the original Level 1.0 processing.

For each image record, the data in thespikes segment is aNSPIKES×3 array containing the pixel index, prior data value, and new

data value for each spike. Whenaia_respike.pro is called, the routine exports thespikes segment from the JSOC for the input

record, and then uses this information to reconstruct the image prior to despiking. Users who have already downloaded a set of image

records and wish to know which pixels in each image have been despiked should simply export (download) the associatedspikes

segment from the JSOC for each record (either by using the methods discussed in Sections4.2 or 5.3, or by using the helper routine

aia_jsoc_getcaldata.pro distributed via SSW), and then view the contents of these segments by usingread_sdo.pro or

any of the other compressed-image readers listed in Section4.6.

TECHNICAL NOTE: Usingread_sdo.pro to read in multiplespikes segments in one call is not feasible at this time, due to

the fact thatread_sdo.pro cannot handle multiple files containing data arrays that have different dimensions. Users will need

to read in multiplespikes segments one at a time in this case.

6.4 Point Spread Function Deconvolution

Extreme ultraviolet radiation entering the AIA telescopesis diffracted and scattered by various components in the light path, including

the wire meshes that support both the entrance and focal-plane filters, and the primary and secondary mirrors. The diffraction and

71


scattering caused by these components distributes incoming light from a point source across the image plane in a patterncalled the

point spread function(PSF). By considering each of the rays incident on the imaging system as a collection of point sources, the images

recorded by AIA can be described mathematically as a convolution of the PSF with the image formed by the incoming radiation at the

source (object) plane. Consequently, the act ofremovingthe effect of the PSF from AIA images requiresdeconvolvingthe PSF from

these images.

Note that deconvolution is a non-standard step in processing the AIA data. Besides being time-consuming, it can create artifacts

by amplifying noise occurring on small spatial scales. Therefore, unless users are specifically interested in removingstray light due to

diffraction and scattering by the telescope optics and/or sharpening the image, this step should probably be skipped. If the deconvolution

of AIA images is desired, it should occur after respiking (asdescribed in Section6.3) but prior to other image manipulations, such as

converting AIA images from Level 1.0 to Level 1.5 (as described in Section6.1) or otherwise calibrating the data.

Before performing a deconvolution, one needs to calculate amodel PSF, which depends on the wavelength channel. This is achieved

by using theaia_calc_psf.pro function in SSW:

psf=aia_calc_psf(wavelength)

The WAVELENGTHargument specifies the AIA channel of interest, and needs to be a string, with the routine accepting a value

of either ’94’ , ’131’ , ’171’ , ’193’ , ’211’ , ’304’ , or ’335’ . Computing the PSF takes a bit of time, but only needs

to be done once per wavelength channel. The PSFs that are assumed to apply to each of the AIA channels are described in the

AIA PSF and Deconvolution Notesand in theAIA calibration paper(Boerner et al. 2012; appearing in the SDO issue ofSolar Physics).

Once a PSF is calculated, the deconvolution step is performed by theaia_deconvolve_richardsonlucy.pro routine,

which (as the name implies) uses a Richardson-Lucy iterative procedure for recovering the source image:

image_deconvolved=aia_deconvolve_richardsonlucy(image,psf,/verbose)

This function takes as input theIMAGEargument, containing the (floating-point) AIA image to be deconvolved, and thePSFargument,

containing the model PSF computed usingaia_calc_psf.pro . On output, theIMAGE_DECONVOLVEDvariable will contain the

resulting deconvolved image. The/VERBOSEswitch causes theaia_deconvolve_richardsonlucy.pro routine to optionally

display informational updates as the deconvultion progresses.

For example, after using SSWIDL to download and read the firstimage from the example AIA Level 1.0 dataset in Section5.5.2via



log=vso_get(result(0),out_dir=’data’,filenames=fnames,/rice)


one can deconvolve this image as follows:


psf=aia_calc_psf(strcompress(index.wavelnth,/remove))

image_deconv=aia_deconvolve_richardsonlucy(float(data),psf,/verbose)

IDL> psf=aia_calc_psf(strcompress(index.wavelnth,/remove))

wx= 4.50000wy= 4.50000

First pass wide angle -100





72

http://www.lmsal.com/sdodocs/doc?cmd=dcur&proj_num=SDOD0076&file_type=pdf

http://www.springerlink.com/content/n2281w1r49g26h31/


[...deletia...]

IDL> image_deconv=aia_deconvolve_richardsonlucy(float(data),psf,/verbose)

Richardson-Lucy deconvolution (ver:v2011-Jul-13) of a 4096x4096 image and a 4096x4096 PSF.

Performing 2D iteration 1 of 25





[...deletia...]

At this stage, theIMAGE_DECONVvariable contains the deconvolved image. Theaia_deconvolve_richardsonlucy.pro

routine only accepts one image at a time, but the same PSF can be used for multiple images from the same AIA wavelength channel.

TECHNICAL NOTE: As detailed in theAIA PSF and Deconvolution Notes, the PSF calculated byaia_calc_psf.pro in-

cludes a component to model the large-scale diffraction pattern caused by the entrance and focal-plane filter meshes, and a

Gaussian component to model the core of the PSF. By default, aPSF based upon on-orbit data is generated, and this PSF is

the recommended model for removing the diffraction patternand other large-scale stray light effects from AIA images, while

simultaneously limiting the sharpening of noise and other image artifacts. However, theaia_calc_psf.pro routine also ac-

cepts a/USE_PREFLIGHTCOREswitch that, if set, produces a PSF for which the Gaussian core is based solely upon pre-flight

measurements of the optical system. (The diffraction pattern component is based upon the same analysis as for the default PSF.)

The core of this pre-flight model is broader than that of the (default) on-orbit model, and while using this PSF produces a more

aggressive deconvolution of AIA images, it may also result in an excessive amount of sharpening of noise artifacts.

6.5 Getting AIA Filter Response Data

The filter responses for the EUV channels can be loaded using theaia_get_response.pro procedure. In the most recent version of

the package (dated 13 February 2012; see Section5.1.2for instructions on how to update theAIA package), the following modifications

were made:

• The emissivity model has been updated to use CHIANTI version7 to cover a broader wavelength range, and to fix a bug in the

calculation of the continuum emissivity.

• There are new options to apply corrections to the wavelengthor temperature response for time-dependence and cross-calibration

with EVE.

• An empirical fix for missing emission lines in the AIA 94Å and 131Å channels is now available.

• The wavelength response (effective area) has not changed.

• The default behavior of theget_aia_response.pro routine has changed. The routine now requires one to specifyeither

/DN or /PHOT to explicitly include or exclude the DN-per-photon correction.

To return the response curves as a function of wavelength in units of photons cm2:


result=aia_get_response(/phot)


IDL> result=aia_get_response(/phot)


73

http://www.lmsal.com/sdodocs/doc?cmd=dcur&proj_num=SDOD0076&file_type=pdf


** Structure <2850c08>, 14 tags, length=352536, data length=352504, refs=1:

NAME STRING ’AIA’

DATE STRING ’20100804_232510’

CHANNELS STRING Array[7]

WAVE FLOAT Array[4001]

ALL FLOAT Array[7, 4001]

PLATESCALE FLOAT 8.46158e-12

UNITS STRING ’cm2’

A94 STRUCT -> <Anonymous> Array[1]







CORRECTIONS STRUCT -> <Anonymous> Array[1]

When the/DN switch is set, the effective area function also includes thedata-number-to-photon conversion:


result=aia_get_response(/dn)


IDL> result=aia_get_response(/dn)




DATE STRING ’20100804_232510’


WAVE FLOAT Array[4001]

ALL FLOAT Array[7, 4001]

PLATESCALE FLOAT 8.46158e-12

UNITS STRING ’cm^2 DN phot^-1’









The response curves can also be computed as a function of (log) temperature, as shown in Figure1, by setting the/TEMP switch:


result=aia_get_response(/phot,/temp)


74


IDL> result=aia_get_response(/phot,/temp)

-------------------------------------------------------

| Generating temperature response function from |

| /ssw/sdo/aia/response/aia_preflight_all_fullinst.genx |

| /ssw/sdo/aia/response/aia_preflight_fullemiss.genx |

-------------------------------------------------------


** Structure <19fa008>, 12 tags, length=14944, data length=14912, refs=1:


LOGTE FLOAT Array[101]


UNITS STRING ’phot cm^5 s^-1 pix^-1’

ALL DOUBLE Array[101, 7]









There are a number of additional options that can be specified:

• /UV : Returns response for the AIA 1600Å 1700Å and 4500Å channelsinstead of the EUV channels.

• /ALL : Includes functions for both thin and thick focal plane filters

• /FULL : Includes full detail on the components (mirrors, filters, etc.). Note that the resulting structure is formatted differently than

the simple effective-area structure.

• /NOBLEND: Ignores potential crosstalk between two EUV channels sharing telescope (applies to the AIA 131Åand 335Å channels

and to the AIA 94Å and 304Å channels).

• VERSION: Allows the user to specify a version. Currently only two versions are available. SettingVERSION=1uses the preflight

effective area, and settingVERSION=2is the current release. There is no difference in the wavelength response functions.

There are also some optional corrections to the effective area that can be applied by specifying the following keywords:

• /EVENORM: Normalizes the response to agree with observations from EVE on 1 May 2010.

• TIMEDEPEND_DATE: Applies a time-dependent correction to account for on-orbit degradation. This keyword can be set either

as a switch, in which case the current date is used, or as a string specifying the time for which the correction is desired.

More examples usingaia_get_response.pro can be found in the text files included in the SSW telescope response tree, located

in $SSW/sdo/aia/response/ , or onlinehere.

75

http://sohowww.nascom.nasa.gov/solarsoft/sdo/aia/response/


6.6 Miscellaneous AIA Tasks

6.6.1 Using AIA Standard Scaling and Colors

The images displayed on many data-browsing webpages, such as those mentioned in Section2, are often processed using a standardized

color table and image scaling. The contrast levels of these images were chosen to bring out much of the detail in the image data, while

the distinct color schemes allow fast identification of the AIA wavelength channel while browsing.

The standardized scaling can be applied using theaia_colors.pro function from theAIA package from SSWIDL:

result=aia_intscale(image,wave=171,exptime=exptime)

After completion, theRESULTvariable will contain the scaled image for the particular wavelength channel specified in theWAVE

keyword (here the 171Å channel is used as an example). If exposure-time normalization is needed, one may provide the exposure time

of the image in theEXPTIMEkeyword. Note that this routine is not parallelized.

The standardized AIA color tables can be accessed via theaia_lct.pro routine is also distributed via SSWIDL. Again using the

171Å channel as an example, the color table can be loaded intovariables as follows:

aia_lct,wave=171,red,green,blue

and/or made active using the/LOAD switch:

aia_lct,wave=171,/load

For bothaia_intscale.pro andaia_lct.pro , valid arguments for theWAVEkeyword are the integers94 , 131 , 171 , 193 ,

211 , 304 , 335 , 1600 , 1700 , or 4500 .

As an example, first download an image from the AIA 171Å channel and applyaia_prep.pro :


result=vso_search(’2011-01-01 1:00:00’,’2011-01-01 1:00:11’,inst=’aia’,wave=171)



Next, apply the standardized scaling, load in the color table for this image, and display this image on screen:


scaled_im=aia_intscale(out_data,wave=out_index.wavelnth,exptime=out_index.exptime)

aia_lct,wave=out_index.wavelnth,/load

window,0 & plot_image,scaled_im

6.6.2 Creating AIA Tri-Color Images

Tri-color images, such as those provided on “The Sun Now” and “The Sun Today” browse webpages (see Sections2.1and2.2, respec-

tively), have proved popular with research scientists and the general public alike. These types of images are created byslotting 8-bit

grayscale images from three separate AIA wavelength channels into the red, green, and blue planes of a 24-bit (true-color) image.

To make tri-color images, one first co-aligns images from thedifferent AIA channels, usually usingaia_prep.pro (see Sec-

tion 6.1) and then passes these data to theaia_colors.pro routine available through SSWIDL. The folling example illustrates how

to make the EUV triplet from the 171Å, 193Å, and 211Å channelsof AIA. First, download a co-temporal set of images from these

channels and run them throughaia_prep.pro :

76



result171=vso_search(’2011-01-01 1:00:00’,’2011-01-01 1:00:11’,inst=’aia’,wave=171)



log=vso_get([result171,result193,result211],out_dir=’data’,filenames=fnames,/rice)


Then, issue the call toaia_colors.pro to create the tri-color image:

aia_colors,out_index,out_data,data_euv=data_euv,/do_jpeg

After being provided an input index and data array, the routine will output a true-color image to the IDL variable specified in the

DATA_EUVvariable and/or to disk in the form of a JPEG image if the/DO_JPEGswitch is set.

There are three varieties of tri-color images using images from different combinations of AIA channels. The example above

is the EUV triplet comprised of the 211Å, 193Å, and 171Å channels and is output in theDATA_EUVkeyword. Additionally, a

high-temperature triplet comprised of the 94Å, 335Å, and 193Å channels can be output in theDATA_HI_T keyword and a chro-

mosphere/corona triplet comprised of the 304Å, 211Å, and 171Å channels is output in theDATA_LOHIkeyword.

In fact, if one provides a co-temporal set of images from all AIA wavelength channels, theaia_colors.pro routine can output

all three varieties of tri-color images as well as a data cubeof images (via theDATA_CUBE_OUTkeyword) to which the standardized

scaling from theaia_intscale.pro function discussed in Section6.6.1above has been applied:


result=vso_search(’2011-01-01 1:00:00’,’2011-01-01 1:00:11’,inst=’aia’)



aia_colors,out_index,out_data,data_cube_out=data_cube_out,$

data_euv=data_euv,data_hi_t=data_hi_t,data_lohi=data_lohi,/do_jpeg

The end result here is the downloading of eight AIA images, and processing them to create a 4096×4096×8-pixel data cube in

the DATA_CUBE_OUTvariable and the three varieties of tri-color images, both contained in theDATA_EUV, DATA_HI_T, and

DATA_LOHIvariables and written to disk as JPEG images.

6.6.3 Plotting an AIA Light Curve

Because the image headers are segregated from the images in the JSOC, any quantity that can be derived directly from the keywords can

be computed quickly. For example, to plot an exposure-time-corrected AIA light curve as a function of time, one can use data contained

in theDATAMEANandT_OBSkeywords from the image headers:


ssw_jsoc_time2data,’2011-02-20’,’2011-03-06’,index,ds=’aia.lev1_euv_12s’,$

waves=’335’,key=’t_obs,wavelnth,datamean,exptime’,cadence=’1h’

utplot,index.t_obs,index.datamean/index.exptime,psym=1

In this example, a light curve from the 335Å channel for two weeks in early 2011 is plotted using the SSW procedureutplot.pro .

(Incidentally, the jump in the light curve occurring late inthe day of 24 February 2011 is a result of a bakeout of AIA Telescope #1 on

this day, an event listed on theSDO Joint Science Operations Calendar.)

77

http://aia.lmsal.com/public/SDOcalendar.html


6.6.4 Interfacing with plot_map.pro (Dom Zarro’s Mapping Software)

Many solar physicists make use of themapping softwaredeveloped by Dom Zarro and written in IDL that enables quick and easy

registration and alignment of images from different sources. The software, which is included in every SolarSoft installation, works by

converting images to more generic map structures that contain both the image data and their associated coordinate and scaling properties.

Once in the map structures, data can then be plotted, rotated, stretched, and resized. The documentation linked to abovecontains several

such examples.

HMI and AIA data read into SSWIDL viaread_sdo.pro can be readily converted to the map structure format used by the

mapping software. To demonstrate this, first acquire the two-image AIA Level 1.0 dataset used in Section5.5.2and convert them to

Level 1.5 (as in Section6.1):





The following two commands convert the two images to a map structure, and then plots the first one:

index2map,out_index,out_data,map

plot_map,map,/log,drange=[1e2,1e4]

At this point, one should see an IDL graphics window open withthe first image. Axes showing the pixel coordinates in arcseconds are

plotted, and the data are scaled to a range of 100 and 10,000 and plotted using logarithmic scaling.

Note that theplot_map.pro will display images using the current roll angle defined in the map structure. This comes into play

when working with HMI observables such as continuum intensity and line-of-sight magnetogram data, which often have a default roll

angle of about 180 degrees. For example, if one were to createand plot the map structure from the HMI continuum intensity image

shown in the example below, it would be plotted upside-down:


result=vso_search(’2011-01-01 1:00’,’2011-01-01 1:01’,inst=’hmi’,physobs=’intensity’)



index2map,index,data,map

plot_map,map

To plot the map with solar north at the top of the image (that is, with a roll angle of 0 degrees), therot_map.pro routine can be used:


rotmap=rot_map(map,roll=0) ; derotated map

plot_map,rotmap

6.6.5 Checking theQUALITY Keyword

On occasion, AIA image headers and data are affected by operational items (e.g., off-pointing or defocusing operationsassociated with

various calibration maneuvers) or are missing significant data (e.g., partial images during eclipse seasons). Such operational items

affecting the regular AIA data stream are usually listed in the SDO Joint Science Operations Calendar, and in addition theQUALITY

keyword will be set to a non-zero value. As a result, it is worth checking either the calendar and/or theQUALITY keyword before

performing any kind of analysis using AIA Level 1.0 images.

78

http://hesperia.gsfc.nasa.gov/rhessidatacenter/complementary_data/maps



TheQUALITY keyword is a 32-bit integer that encodes various flags in a bitwise fashion. The meanings of the individual bits are

listed in Table2. Of particular interest are bits 8–11 (indicating missing image data), bits 12–15 (indicating pointing issues), and bit 18

(indicating that a calibration maneuver is taking place).

The following example shows how to examine theQUALITY keyword from the headers of three images (one is eclipsed, one is

nominal, and one was taken during a calibration maneuver) inorder to determine whether the image can be used for analysis:


ssw_jsoc_time2data,’2011-01-20 15:30’,’2011-01-20 15:31’,in1,ds=’aia.lev1’,key=’quality’



for i=0,31 do if ishft(in1(0).quality,-i) then print,i



IDL> ssw_jsoc_time2data,’2011-01-20 15:30’,’2011-01-20 15:31’,in1,ds=’aia.lev1’,key=’quality’

[...deletia...]


[...deletia...]


[...deletia...]

IDL> for i=0,31 do if ishft(in1(0).quality,-i) then print,i

17

18



12

13

14

17

21

Here, the last three IDL command-line statements loop through each of the bits comprising theQUALITY keyword for the first image

in each of the three datasets, and print out which of these bits are set. By taking advantage of the ability to download onlythe header

information from the JSOC (as shown here), one can examine the QUALITY keyword and screen out images that are likely undesirable

without first downloading the image.

In the above example, the first image (from 20 January 2011) was taken during an AIA calibration maneuver, and as a result bit 18

is set. Bit 17 is also set for this image. No output occurred for the second image (from 20 February 2011), indicating thatQUALITY

is zero and that the image is presumably clean. The third image (from 20 March 2011) was taken during the spring eclipse season for

2011, resulting in bits 12–14 being set. Additionally, bits17 and 21 are set for this image.

79


QUALITY bit(s) Meaning, if set

0 Flatfield data are not available

1 Orbit data are not available

2 Ancillary science data are not available

3 Master pointing data are not available

4 Limb-fit data are not available (generally not applicable to AIA images)

5–7 (empty)

8 Value ofMISSVALSkeyword is nonzero

9 Value ofMISSVALSkeyword is more than 1% of the value of theTOTVALSkeyword



12 Spacecraft is not in science pointing mode

(coincides withACS_MODEkeyword set to a value other thanSCIENCE)

13 Spacecraft eclipse flag is set

14 Spacecraft sun presence flag is not set

15 Spacecraft safemode flag is set

16 Dark image flag is set (coincides withIMG_TYPEkeyword set toDARK)

17 Image Stabilization System (ISS) loop is open

18 Calibration image

19 (empty)

20 Focus is out of range

21 Register flag is set

22–29 (empty)

30 Quicklook image

31 Image is not available

Table 2: Listing of the meaning of the bits encoded by theQUALITY keyword for AIA Level 1.0 data. This table is reproduced from

Appendix 3 of theAIA/SDO FITS keyword list.

80



7 Frequently Asked Questions

• Questions, comments, and feedback:

– Is there someone I can contact with problems or issues related to finding, getting, or analyzing SDO data?If, after looking through the Table of Contents as well as this list of frequent questions, your issue does not seem to be

addressed, please send an e-mail to the address listed onthis webpage.

• Browsing and finding data:

– How do I browse through the data?

You can easily browse and download summary images and moviesat any of the places mentioned in Section2.

– How do I get a more detailed look instead of just a “browse”?One option is Helioviewer (see Section2.4), which enables users to overlay images in multiple layers from multiple data

sources. Another option is to query the HCR (see Section3.1) to see whether data exists for the feature or event of interest.

– How do I find data for a specific feature or event, but I haven’tdetermined (or don’t know) the dates/times?The HEK (see Section3.1) contains a fully cataloged and readily searchable database of many different events and features,

such as active regions, filament eruptions, and coronal holes seen in SDO data. The database can be queried using iSolSearch

(see Section3.2), and the resulting event descriptions contain links to theassociated data.

• Accessing the data:

– How do I get science-grade data for HMI and/or AIA?There are several options. Keep in mind that full-disk, full-resolution data adds up quickly. (If you would be satisfied with

a subfield, or are willing to subsample the data either in space or time, see also the next two questions.) Once you have

determined that your data volume constitutes a manageable size, HMI and AIA data are available through the JSOC (see

Sections4.2or 5.3), the cutout service (see Sections4.3or 5.4), or the VSO (see Sections4.4or 5.5).

– Do I really have to learn about the inner workings of the JSOCin order to get AIA or HMI data?No, but it helps. Refer to the brief description in Section4.1 for an accessible primer. More detailed information on the

JSOC can be found by visiting the links found in that section.

– How do I request a subfield (localized patch) of the full-resolution image data?

For AIA data, the easiest way to do this is to use the cutout service (see Section4.3). If your desired subfield is related to

that of a HEK event (say, as found using iSolSearch [see Section3.2]), be aware that each HEK event entry has a link to the

cutout service, as well as links to related observations that may point to relevant datasets that have already been created and

thus are readily available.

– How do I subsample the data in either space or time?

One option for subsampling in time is to use the VSO, which provides a great deal of flexibility in accessing solar data

at various temporal cadences (see Sections4.4 or 5.5). Another option is to use the online cutout service form (see Sec-

tion 4.3), which allows users to readily subsample in both space and/or time. A third option for subsampling in time is to

specify a cadence when querying the JSOC, either using thelookdatatool (see Section4.2.2) or using the SolarSoft routine

ssw_jsoc_time2data.pro (see Section5.3.2).

81



– I’ve heard that not all AIA data are available all of the time. How can I find out what exists and what doesn’t?At the time of this writing, AIA data from 13 May 2010 onward have been processed to Level 1.0, are online, and are

available for export from the JSOC directly (see Sections4.2or 5.3) or for processing by the cutout service (see Sections4.3

or 5.4). The exception is that there is usually about a 4-day lag forthe most recent AIA data to be processed in the pipeline

to Level 1.0. Changes to the data-processing status will be reflected on theAIA Data webpage. The caveat is that, due to

limitations in storage resources, it is possible that some AIA Level 1.0 data may be taken offline at some point in the future.

(Regarding the VSO, it is likely that the data online at the caches located at the various VSO locations will be different than

what is online at the JSOC.)

– What about EVE data?

EVE data are available from theEVE data access webpage. Access to EVE data via SolarSoft is currently under develop-

ment.

• Analyzing the data:

– Do I need to prepare the data in any way before analyzing it (ala eit_prep.pro or trace_prep.pro )?

For all published AIA Level 1.0 data, the standard and best-available gain (flat-fielding), filter, vignette, and bad-pixel/cosmic-

ray corrections will have already been applied. Removing stray light, scattering and diffraction patterns from AIA Level 1.0

images is described in Section6.4. If you have AIA Level 1.5 data, the images will additionallyhave already been centered,

rescaled to a standard 0.6′′ plate scale, and rotated so that solar north is up in the image. If you have AIA Level 1.0 data,

then theaia_prep.pro routine will need to be run to make these adjustments (see Section 6.1). For HMI Level 1.5

data based on filtergrams from the Doppler camera (includingDopplergrams, continuum intensity images, and line-of-sight

magnetograms), such “always-needed” corrections will have already been applied. However, note that many HMI Level 1.5

images have a roll angle (contained in theCROTA2keyword) of near 180 degrees (and so may appear upside-down).

– How do I know what data-processing level my data are at?TheLVL_NUMkeyword in the image header contains the data-processing level.

– Why are my HMI images upside-down?HMI images typically have a roll angle (contained in theCROTA2keyword) of near 180 degrees, and it is likely that this roll

angle has not been accounted for.

– How can I co-align AIA data with HMI data?

Co-aligning data from HMI and AIA can be done using theaia_prep.pro procedure (see Section6.2), which aligns,

derotates, and rescales images from both instruments to a common 0.6′′ plate scale. Co-alignment can also be performed

using the mapping software mentioned in Section6.6.4.

– Where can I get AIA filter response data?

For a summary plot, see Figure1. For the actual numbers, see Section6.5.

– Are the AIA color tables that are used on the browsing webpages available?Yes. See Section6.6.1.

– How do I make the tri-color images available from the browsing webpages?For some SSWIDL commands, see Section6.6.2.

– Where can I get a list of HMI or AIA keywords?Each instrument team has produced a document listing the various keywords. Because many keywords are shared between

HMI and AIA, there is probably much overlap between theJSOC keywords for metadatadocument (sourced from the “All

About JSOC Names” page from theJSOC wiki) and theAIA/SDO FITS keyword list(sourced from the “AIA Keywords”

entry onSDOdocs).

82

http://aia.lmsal.com/public/data.htm

http://lasp.colorado.edu/eve/data/data_access.htm






• Operations and pipelines:

– Is there ops information somewhere?

Yes, there is anSDO Joint Science Operations Calendarthat lists instrument calibration maneuvers and other known issues

that affect the standard operation of the various instruments on SDO.

– When are the eclipse seasons?

The eclipse seasons occur annually at about the time of the vernal and autumnal equinoxes. For specific dates, consult the

SDO Joint Science Operations Calendar.

– What is the status of the AIA and HMI processing pipeline?

Check theJSOC processing status page.

83



http://jsoc.stanford.edu/data/jsoc_proc_status.html


8 Lists of Useful Links

• Observatory and instrument links:

– SDO homepage at Goddard

∗ SDO Joint Science Operations Calendar

∗ JSOC processing status page

– AIA homepage at LMSAL

∗ SDOdocs(HEK and AIA documentation, mostly)

– EVE homepage at CU/LASP

– HMI homepage at Stanford

∗ HMI magnetic data splash page

• Data links:

– Sungate

– VSO entry page and search form

– JSOC data portaland thelookdatatool

– EVE science data access

– NOAA SWPC data center

• SolarSoft links:

– SSW installation webpage

– SSW upgrade webpage

– SSW codesearch form(for fast doc-header searches of entire SolarSoft library)

– SSW cutout service API

• Journal articles appearing in theSDO volume ofSolar Physics(volume 275, Jan. 2012), all of which are available for download

for free from the publisher via Open Access:

– Fisher (2012)(preface to the dedicated volume)

– Pesnell et al. (2012)(SDO overview)

– Lemen et al. (2012)(AIA instrument paper)

– Boerner et al. (2012)(AIA calibration paper)

– Hurlburt et al. (2012)(HEK paper)

– Martens et al. (2012)(AIA feature-finding consortium paper)

– Woods et al. (2012)(EVE instrument paper)

– Hock et al. (2012)(EVE/MEGS calibration paper)

– Didkovsky et al. (2012)(EVE/ESP calibration paper)

– Scherrer et al. (2012)(HMI instrument paper)

– Schou et al. (2012)(HMI design paper)

– Wachter et al. (2012)(HMI calibration paper #1)

84



http://jsoc.stanford.edu/data/jsoc_proc_status.html

http://aia.lmsal.com

https://www.lmsal.com/sdodocs

http://lasp.colorado.edu/eve

http://hmi.stanford.edu

http://hmi.stanford.edu/magnetic/

http://www.lmsal.com/sungate


http://jsoc.stanford.edu/jsocwiki/JsocDataAccess


http://lasp.colorado.edu/eve/data_access

http://www.swpc.noaa.gov/Data

http://www.lmsal.com/solarsoft/ssw_install_howto.html

http://www.lmsal.com/solarsoft/ssw_upgrades.html

http://www.lmsal.com/codesearch

http://www.lmsal.com/solarsoft/ssw_service/ssw_service_track_fov_api.html

http://www.springerlink.com/content/0038-0938/275/1-2/

http://www.springerlink.com/content/u182k5234l085875/

http://www.springerlink.com/content/w687736278532677/

http://www.springerlink.com/content/g71494170q64jpt1/

http://www.springerlink.com/content/n2281w1r49g26h31/

http://www.springerlink.com/content/g244ng5t72866u67/

http://www.springerlink.com/content/t27t647m2hx2701h/

http://www.springerlink.com/content/m3811106121h2575/

http://www.springerlink.com/content/43426m337478q708/

http://www.springerlink.com/content/r4v7170v6374751g/

http://www.springerlink.com/content/n0k3765711577156/

http://www.springerlink.com/content/r302377589530242/

http://www.springerlink.com/content/m6462hpw1411lmx8/


– Couvidat et al. (2012)(HMI calibration paper #2)

– Schou et al. (2012)(HMI calibration paper #3)

– Couvidat et al. (2012)(HMI time-distance pipeline paper #1)

– Zhao et al. (2012)(HMI time-distance pipeline paper #2)

– Drobnes et al. (2012)(SDO E/PO paper)

85

http://www.springerlink.com/content/44242qku011076l8/

http://www.springerlink.com/content/8872723624112062/

http://www.springerlink.com/content/b7u7276866856411/

http://www.springerlink.com/content/h147jt8177452104/

http://www.springerlink.com/content/x206675626r05k11/


9 Version History of this Guide

Below are links to earlier versions of the online version of this guide, along with brief descriptions of major changes made from the

previous version.

• This version:

– Included in Section1.5some links to the magnetic field data products for HMI.

– Updated Table1 (list of JSOC dataseries) of Section4.1 to include thehmi.B_ * , hmi.ME_ * andhmi.[Ms]harp_ *dataseries for HMI.

– Removed mention of the leap second added to UTC on 30 June 2012in Section4.2.4.

– Added Section4.5containing brief descriptions of and links to the HMI and AIAsynoptic data streams.

– Corrected an error in theimcopy syntax presented in Section4.6.1.

– Split Section5.2 into two subsections, and added an example of how to search for substrings in event descriptions when

performing a HER query via SolarSoft in Section5.2.2.

– Removed from the examples in Sections5.3.3and5.3.4the /UNCOMP_DELETEand/COMP_DELETEswitches, as these

are now specified by default and are thus no longer needed.

– Added to Section5.6.1(formerly Section5.6) information on how to useread_sdo.pro to downsample or extract cutouts

from HMI or AIA images.

– Added Section5.6.2, containing information on how to callread_sdo.pro using a external library to uncompress AIA

and HMI data. The use of the external library often reduces the time needed to read in full-resolution AIA or HMI images

(but is not yet available on all computing platforms), compared with the more I/O-intensive default process described in

Section5.6.1.

– Changed the title of Section6.1 to better reflect its contents, edited the text to be more clear regarding the treatment of

cutouts, and added a technical note regarding the ability ofaia_prep.pro to make use of historical master pointing

tables.

– Expanded Section6.6.1 to include information on how to apply the standardized scaling to AIA images (such as those

available from the various data-browsing websites).

– Added Section6.6.2that illustrates how to create tri-color images from AIA channel triplets, and added a corresponding

question in the FAQ (Section7).

• Version dated 5 March 2012:

– Added navigational options to HTML version of the guide, namely links to sub-subsections in the sidebar and previ-

ous/up/next links in the headers and footers on each node.

– Updated Table1 (list of JSOC dataseries) of Section4.1to include the slottedaia.lev1_ * dataseries for AIA.

– Added examples to Section4.2.2demonstrating how to construct JSOC queries using the slottedaia.lev1_ * dataseries

for AIA.

– Updated the description of the cutout service in Section4.3as well as Figure19to match a more recent version of the online

form.

– Added text to Section4.4discussing how to export a list of links from the VSO and download them usingcurl .

– Added Section4.6.1providing some pointers on how one might compile and installthe imcopy utility.

– Added a technical note to Section5.1.1that discusses uses of thesetssw alias.

– Added Section5.1.3describing how to configure SolarSoft so that it works with a proxy server.

86

https://www.lmsal.com/sdodocs/doc/rep/sdod168/fid234/zip/entry/index.html


– Added an example to Section5.3.2to illustrate the use of theCADENCEkeyword withssw_jsoc_time2data.pro .

– Added Section6.4discussing how to perform a point spread function deconvolution on AIA images.

– Updated Section6.5to be consistent with the updated version of the AIA calibration routines in SSW.

– Replaced the light-curve example in Section6.6.3with a new one that uses theaia.lev1_euv_12s dataseries instead of

the deprecatedaia_test.synoptic2 dataseries.

– Added Section6.6.4describing how to use HMI and AIA image data with Dom Zarro’s mapping software.

– Updated the output of the SSWIDL examples throughout Sections5 and6 to reflect more recent versions of the software.

Also changed thevso_get.pro examples so that theFILENAMESkeyword is employed to determine the list of down-

loaded files.

– Added questions to the FAQ (Section7) addressing the HMI roll angle and the co-alignment of AIA and HMI data, and

updated the answers to the questions about temporal subselection of data and AIA data availability.

– Added to Section8 links to the articles appearing in the recently published (and freely available) volume ofSolar Physics

dedicated to SDO.

• Version dated 5 July 2011:

– Added mention of the AIA feature-finding consortium to Sections3.1and8.

– Added Section4.2.4, detailing the differences between some of the various timing keywords found in AIA and HMI image

headers.

– In Sections5 and6, updated the outputs to several of the examples that were outof date.

• Version dated 20 May 2011:

– Added Section6.2, describing how to co-align HMI data with AIA data.

– Added a contact e-mail in Section1.2and in the FAQ (Section7).

• Version dated 28 April 2011:

– Added Section6.6.5, containing information on the AIAQUALITY keyword

– Made minor changes to the formatting of the FAQ list in Section 7.

– Added links to the SDO Joint Science Operations Calendar in Sections7 and8.

• Version dated 31 March 2011:

– Updated Fig.1 with a new version of the figure, which corrected a minor typo in the y-axis label.

– Updated Sections4.3and5.4to provide more accurate information about the cutout service.

– Added Section5.6to provide a more detailed description of how to useread_sdo.pro .

– Added more details to Section6.1 regarding the particular keywords affected when converting AIA data from Level 1.0 to

Level 1.5.

• Version dated 26 January 2011:

– Added Table1 (list of dataseries published by the JSOC) to Section4.1.

– Replaced references to the deprecatedhmi_test. * andaia_test. * dataseries in Sections4.1and4.2, in Figures14–

18, and in the SSWIDL examples in Section5.3, with references to the final production dataserieshmi. * andaia. * ,

respectively.

87







– Added text to Section5.3describing how to get keyword and segment informational data.

– Added text to Section4.2, and a new subsection in5.3, describing how to export specific segments of JSOC dataseries.

– Updated the demo in Section5.5to use a more recent dataset, and added information describing how to useread_sdo.pro

to read Rice-compressed data into IDL.

– Created Section6 on AIA data processing, which includes sections discussingaia_prep.pro (Section6.1) as well as

aia_respike.pro (Section6.3), and moved the “Miscellaneous SSW Items” section from Section 5 to this new section,

where they comprise Sections6.5and6.6.

– Revised the “Do I need to prepare the data ...” question in theFAQ (Section7) to reflect the existence ofaia_prep.pro .

Also made minor editions to some of the other questions.

– Added Section9 (this section).

– Moved the link to the PDF version of the guide so that it appears (more prominently) after the link to the Contents section in

the sidebar.

• Version dated 10 December 2010:

– Partitioned Section5.3into three subsections that directly correspond to the three subsections of Section4.2.

– Made minor editions to the text in Sections4 and5.

• Version dated 30 November 2010:

– Earliest version of the guide in its current form.

88




10 Contributors to this Guide

• Editors: Marc DeRosa, Greg Slater

• Browse webpages: Dean Pesnell and Kevin Addison (“The Sun Now”), Greg Slater and Linus Chang (“The Sun Today”), Peter

Gallagher (SolarMonitor), Jack Ireland and Daniel Mueller(Helioviewer and jHelioviewer)

• AIA instrumental insight: Paul Boerner, Dick Shine, Mark Weber, Yingna Su, Paolo Grigis

• JSOC information and demos: Phil Scherrer, John Serafin, Yang Liu, Keh-Cheng Chu, Rock Bush, Jesper Schou

• HEK information and demos: Mark Cheung, Neal Hurlburt, Linus Chang, Ryan Timmons, Ankur Somani, Scott Green

• SSW information and demos: Sam Freeland

• VSO information and demos: Joe Hourclé

• Sage advice: Karel Schrijver, Barbara Thompson, Karin Muglach, and many others...

89

sdod0060 p guide to sdo data analysis sdoguide

Documents