cplinear model

I have added Pat Broos' cplinear model to the additional models page. This is a non-physical model developed within the AE package to handle background in the low count case.

Yaqoob & Murphy toroidal reprocessor model

I've added a link to the additional models page for the Yaqoob & Murphy model for X-ray obscuration and reflection by a toroidal reprocessor, valid in the Compton-thick regime.

new local models

I have added two new local model pages to the collection. Maurice Leutenegger has supplied his models for hot star winds and David Ballantyne has provided table model files for the reflection of a blackbody from a constant density ionized accretion disk.

xspec patches from May

12.6.0i	When running an XSPEC script, if a call to a user-defined Tcl procedure immediately follows a call to an XSPEC multi-line command (such as model, fakeit, or editmod), the procedure is not executed if its definition is in a nested script called from the current script file. Our thanks to Paul Nulsen for pointing this out. Report added on May 11, 2010.
12.6.0j	A crash occurs when a delcomp operation is performed AND one or more parameters are linked to multiple parameters. Our thanks to Jeremy Sanders for pointing this out. Report added on May 19, 2010.
12.6.0k	The tclout modval option should output a model's flux array even when no spectra are loaded. (In this case XSPEC will ignore the spectrum number parameter.) Our thanks to Veronica Biffi for pointing this out. Report added on May 26, 2010.

xspec patches from April

A few problems in 12.6 have been identified and patched. Information and patches are available on the xspec bugs page.

12.6.0c	The setplot rebin option does not work properly when used in energy/wavelength mode AND channel energies are listed in descending order in the response EBOUNDS extension. Our thanks to Roderick Johnstone for pointing this out. Report added on Apr 01, 2010.
12.6.0d	The chi-square does not change when running steppar on the redshift z parameter of a table model. Our thanks to Jeremy Sanders for pointing this out. Report added on Apr 08, 2010.
12.6.0e	This improves the initpackage local models tool so that it now recognizes *.f03 file extensions for Fortran 2003 code, and it also now allows *.f90 extensions. Our thanks to Pablo Cassatella for bringing this issue to our attention. Report added on Apr 15, 2010.
12.6.0f	For better compatibility with previous XSPEC releases, when multi-panel plots are performed interactively with iplot, the first panel should be designated PLT's currently active window rather than the last panel. Our thanks to Maria Caballero-Garcia for pointing this out. Report added on Apr 21, 2010.
12.6.0g	This increases an internal buffer for handling PLT command strings. A larger size is now necessary when producing multi-panel plots which happen to contain a large number of plot vectors. Our thanks to Lijun Gou for pointing this out. Report added on Apr 30, 2010.
12.6.0h	A couple of modifications to the behavior of steppar when not starting from a best fit: The delta fit statistic column will now show values relative to the original fit statistic, not the value it has after steppar performs an initial renormalization. Also when steppar detects a new best fit, it will now prompt the user only when the improvement exceeds the current setting of the fit's critical delta parameter. Our thanks to Jeremy Sanders for this suggestion. Report added on Apr 30, 2010.

xselect bug in new release

Aargh ! There is an error in xselect in the new HEAsoft release. It has been fixed this morning but anyone who has already downloaded the distribution will need to replace the xselect.mdb by one available on the bugs page.

First two patches for xspec 12.6

We received two xspec bug reports whose fixes were too late to include in the latest release. Patches are in the usual place.

12.6.0a	XSPEC issues a fatal error if the swind1 and zxipcf models are both used within the same session. Our thanks to Matteo Cerruti for pointing this out. Report added on Mar 29, 2010.
12.6.0b	When applying fractional systematic errors to multiple channels grouped into a single bin, XSPEC should multiply by the square of the sum of the binned channels rather than the sum of the squares of the individual channels. Our thanks to Mike Nowak for pointing this out. Report added on Mar 31, 2010.

xspec 12.6 released

HEAsoft 6.9 is out including xspec v12.6. The release notes are below.

One point to note is that there have been internal changes to the models using the Magzdiarz & Zdziarski Compton reflection code. If there are local models out there which use internals from the xspec models they may break. Let us know and we will fix them.

New features in v12.6.0:

The main improvements in version 12.6.0 are to XSPEC's plotting capabilities:

 - Multi-panel plotting is now supported for all combinations except
   contour plots.  For example, "plot data model resid ratio" will
   produce a 4-panel plot on a single page. Up to 6 panels can be
   plotted in this manner.

 - There are many choices for axis units.  These can be selected
   using the "setplot energy" and "setplot wave" commands. For
   example, "setplot energy GeV" uses GeV on the x- and y-axes.
   "setplot wave" also has a new "perhz" option for displaying
   the Y-axis in 1/Hz units.
  
 - The "setplot" command has a new "redshift " option for shifting
   displayed energies to the source frame.
  
Other new features:

 - New models :
      ireflect is a convolution model based on the pexriv code.
      sirf is a multiblackbody self-irradiated funnel

 - The normalizations on all power-law models (ie. powerlaw,
   bknpow, bkn2pow, cutoffpl)can be changed to a flux over an
   energy range by setting POW_EMIN and POW_EMAX keywords in "xset".
   The powerlaw model then becomes equivalent to the pegpwlw model.

 - The Compton reflection models (b/p)exr(a/i)v and (i)reflect have
   been restructured to use adaptive Gauss-Kronrod quadrature for
   the Greens' function integrals. The precision to which the
   integrals are calculated can be set allowing a trade-off between
   speed and precision.
  
 - The wrapper functions additiveTable and multiplicativeTable
   give external C++ models access to XSPEC's table model interpolation
   routines (equivalent to the xsatbl and xsmtbl functions for Fortran
   models).

-  The display of link expressions has been simplified to show only the
   parameter numbers and not the extraneous component information.
   Also "show model" will now only display the model components and
   not the individual parameters.  The parameters can be seen with
   "show par". 

Additional enhancements previously released as patches to 12.5.1:

 - Added the solar abundance data set of Asplund, Grevesse, and Sauval
   (2006) to the list of available tables accessed with the "abund"
   command.
  
 - New "tclout nchan" option for returning the number of channels in a
   spectrum.
  
 - The "save" command now stores relative rather than absolute paths to
   allow easier porting to other machines.
  
 - The recorn model component has been converted from a mixing to a
   multiplicative type.  This allows a model to define multiple recorn
   components.

 - A warning message is now issued if a user attempts to load a
   response for a source n when there are still slots to fill for
   source n-1.  This is intended to catch cases where a user mistakenly
   reverses the source and spectrum number input to the "response"
   command.
  
All bug fixes to v12.5.1 released as patches a - o are included in
v12.6.0.  In addition the following problems have been corrected:

 - It was possible for the addition of a systematic model error to
   actually decrease the overall variance, when it was applied to a
   zero-variance bin that was artificially increased by XSPEC for chi-
   square fitting.
  
 - Bug in "plot ratio" when using "setplot wave" with Hz units.  Y-axis
   model values < 10^-20 were not displayed in plot.
  
 - The comptt model no longer stops and prompts the user when it fails
   during its incomplete gamma calculation.
  
 - The powerlaw model has been modified to avoid a numerical
   instability that could occur if the index were within 10^-12 to
   10^-15 of 1.0.

Plotting updates

The plotting in xspec has been restructured for greater flexibility. The main changes are as follows:

1. Up to six plots can be put on a single page. eg "plot data resid ratio model"

2. There are options for the units on both x- and y-axis. "setplot energy" can now be used to plot the x-axis in keV, MeV, GeV, or Hz. "setplot wave" as the unit options angstrom, cm, micron, or nm. y-axis units are mostly modified in the obvious way depending on the x-axis choice. For setplot energy options any energies in the y-axis unit will be the same as chosen for the y-axis. The exception is for the choice of Hz when emodel/eufspec is in Jy and eemodel/eeufspec in ergs/cm^2/s. For setplot wave options energies in the y-axis unit are in ergs. An additional choice is available for setplot wave : the command "setplot wave perhz" will put y-axis units in /Hz. In this case the emodel/eufspec and eemodel/eeufspec units are the same as for setplot energy hz. The option can be turned off by "setplot wave perhz off".

3. "setplot redshift" can be used to shift energies in plots to the source frame assuming the redshift given. Note that this is not connected in any way to redshift parameters in the model and should only be used for illustrative purposes.

These changes are available in the Goddard development version and will be included in the next HEAsoft release.

xspec power-law models

Added option to peg all the power-law models (ie powerlaw, bknpow, bkn2pow, cutoffpl) in the same way that the pegged power-law operates. If POW_EMIN and POW_EMAX are set (using xset) then all these models will have their normalizations adjusted to be the flux in (POW_EMIN,POW_EMAX) in units of 1e-12 erg/cm²/s or, if POW_EMIN=POW_EMAX, the flux density at POW_EMIN in units of micro-Jy.

A couple of things to watch out for are : POW_EMIN and POW_EMAX should lie within the energy range being calculated; the xset command doesn't automatically update the model so either POW_EMIN or POW_EMAX should be set before defining the model or a parameter value should be changed immediately after the xset.

These changes will be included in the next heasoft release.

xspec bugs update

Not much activity in xspec bug reporting and fixing. The only recent patch is

12.5.1o Small fix to calculation of plot emodel/eemodel and eufspec/eeufspec when in setplot wave mode. This correction is only noticeable for very wide bins (bin widths approximately the size as the bin energy).

xselect and Einstein IPC data

Rich Mushotzky reported that xselect doesn't work on Einstein IPC event files. This was due to a bug in reading data from the MDB which only appears to be triggered in this specific case. I've checked in a fix. A workaround is to run extractor on its own and specify the xfkey, yfkey, xhkey, yhkey, and phamax hidden parameters.

updated Webspec for Chandra Cycle 12

Replaced the Cycle 11 responses and arfs with those for Cycle 12 and bumped the Chandra mission options to the top of the menu.

Snow Leopard

I've upgraded my MacBook Pro from Tiger all the way to Snow Leopard so here are a few observations.

1. Heasoft builds and runs successfully using gcc from Xcode (4.2.1) and gfortran from the R project Mac installer (http://r.research.att.com/tools/ : 4.2.3).

2. Heasoft builds and runs successfully using gcc from Xcode (4.2.1) and gfortran from the fink-installed gcc44 package (4.4.2).

3. Heasoft builds but xspec crashes using gcc and gfortran from the fink-installed gcc44 package. The problem appears to be conflicts in exception handling perhaps involving the system X11 library.

4. The iStat Menus application can be used to monitor usage of individual cores. I find that when running compute-intensive processes (such as xspec) both cores on my laptop get hammered. This implies that Apple, at least, are performing some parallelization in the compiler without specific user directives. This may depend on optimization settings.

replaced broken power law model code with C++

Rationalized the broken power law model code. Rewritten in C++ and calls powerLaw to evaluate individual sections of the model.

cflux model

One thing to note about the cflux model which is not clear in the documentation is that the model component(s) to which cflux is applied must integrate to a non-zero flux. For instance, cflux*pow where the pow norm is zero will generate NaNs from a divide-by-zero.

obscure extractor bug

Under some circumstances ROTANG elements in the output region extension could have junk values but only for region types where the rotation angle is irrelevant. This is fixed in extractor v5.14.

updated xselect XMM script

Updated the script run by save spectrum in the XMM case since I had assumed that CCDNR was the first DS keyword which appears not to always be the case.

WebSpec diagonal response

In answer to a request from Andy Lawrence I've added a unit diagonal response option to WebSpec. It appears at the bottom of the Mission/Instrument menu.

bug in addascaspec

The addascaspec perl script in HEAsoft v6.7 does not run. A fixed version is available through the HEAsoft bugs page. Note that addascaspec can be used for spectra from missions other than ASCA.

definition of cemekl model

Paul Nulsen points out that the help for the cemekl (and cevmkl) model is misleading. The actual differential emission measure equation is dEM = (T/T_{max})^{alpha-1} dT/T_{max}. The documentation will be changed to match this.

patches for xspec v12.5.1

Patches 12.5.1a - k are available at the usual place. The most important are 12.5.1c and e which add the Solar abundances from Asplund et al. (2009) to the options and 12.5.1i which modifies the way the recorn model works (following comments from Rick Rothschild).

max size of GTI arrays in extractor

At Lorella's request I increased the allowed GTI array size to 200,000. This should really be a dynamic array but that would require an extensive rewrite.

Solar abundances in xspec

Jeremy Sanders provided Solar abundances from Asplund, Grevesse & Sauval (2005) which I have added to abundances.dat in the develop version of xspec.

Update 9/8/09: Martin Asplund supplied values from his 2009 ARAA paper so I used them instead for the aspl abundance option.

light curves with extractor

Alex requested a way of better controlling the start time of a light curve created using extractor. I've added a new parameter lcstart which sets the time of the first bin (in spacecraft time units). This is checked into the development version as extractor v5.12.

cleansis

We have seen a few cases of observations of bright sources using Suzaku where running cleansis creates a hole in the image. Part of the problem is running with too high a log probability parameter but there is also a subtle error when iterating. Pixels which are already identified as bad and have their counts set to zero are still included when calculating the local background. This artificially lowers the estimated background hence increases the probability that a pixel will be flagged as bad.

I changed the algorithm so that pixels flagged as bad in previous iterations are ignored when calculating the local background. This change has been checked into the development version.

New release

HEAsoft 6.7 is now available. This includes XSPEC v12.5.1. The biggest change is in gain fitting which has been reworked as the preliminary step towards allowing the creation of response models. Response models will be created analogously to spectral models and will be used to either replace or augment response matrices.

This release is also the first which does not include XSPEC v11. This version has in practice not been supported for some time and it is getting increasingly difficult to ensure that it builds and runs on all systems.

XSPEC bug fixes update

12.5.0af	The zredden multiplicative model component is calling the same function as redden, and therefore its redshift parameter has no effect. Our thanks to Brian Refsdal for pointing this out. Report added on May 05, 2009.
12.5.0ag	When in setplot wave mode and the user's Xspec.init file entry for WAVE_PLOT_UNITS is set to angstrom, plot efficiency and plot icounts don't display due to a missing plot label error. Our thanks to Maurice Leutenegger for pointing this out. Report added on May 08, 2009.
12.5.0ah	Fix to sqrt(2) error in the definition of the thermal broadening used in APEC models. Our thanks to Irina Zhuravleva for pointing this out. Report added on May 21, 2009.
12.5.0ai	This adds a new C-callable function to provide access to the version string for users linking the XSPEC models library into their own programs. The function is declared in src/XSUtil/FunctionUtils/xsFortran.h, and has the prototype: int xs_getVersion(char* buffer, int buffSize); Report added on May 22, 2009.
12.5.0aj	The model command fails to parse expressions that are explicitly of the form: M(M1(A1)+M2(A2)+M3(A3)). Our thanks to Jeremy Sanders for pointing this out. Report added on May 22, 2009.
12.5.0ak	When soft limits are in use during fitting, the pegged parameter test should be comparing the actual value against the hard limits, not the adjusted value. Otherwise the parameter may in some cases peg too soon. Report added on June 08, 2009.
12.5.0al	The same sqrt(2) fix mentioned in patch 12.5.0ah is needed when thermal broadening is turned on for v2.0 NEI models. Our thanks to Richard Sturm for pointing this out. Report added on June 08, 2009.

xselect cleansis

Fixed a bug spotted by Ed Cackett that the iterate_clean parameter is not recognized. Just needed to add it to the xselect.key file.

Note that there are issues turning up using the probabilistic sisclean tool for bright Suzaku sources. There is a tendency for the core of the PSF to be removed. Suggested fixes are either to change the critical probability or turn the iteration off.

self-irradiated funnel

Pavel Abolmasov sent in his self-irradiated funnel model (Abolmasov et al. 2009) and I added an entry to the new models web page.

addascaspec

The perl script addascapec, which is actually useful for other missions in addition to ASCA, had a couple of problems which I've fixed. The default errmeth should have been POISS-0 instead of POISS-1 and this parameter was not being used when summing background spectra.

Lexicon of handy statistical methods

Andrew Gelman's statistical lexicon.

bug in thermal broadening

Irina Zhuravleva at MPE pointed out that the thermal broadening option in the xspec apec model gives a sigma which is a factor of sqrt(2) too large. Fortunately, I don't think has any science implications since there have been no observations capable of measuring this broadening.

update on 6/8/09: Richard Sturm points out there is the same error in the NEI code.

panda/epanda/bpanda regions

I've added support to cfitsio region filtering for the panda/epanda/bpanda regions produced by ds9. Note that this assumes only one azimuthal region and one annulus although the region specification itself allows multiple for both. Made corresponding changes in extractor though note that the FITS region extension written in this case is not covered by the standard which does not include these shapes.

Spectral file format standard document

I updated the PHA file standard document to try to make it clearer. I fixed an error in the type II example which had HDUCLAS3 and HDUCLAS4 inverted.

XSPEC bug fixes update

Here are the bug fixes from the last month. They can be found in the usual place.

12.5.0w	A new tclout option has been added to make it easier to retrieve the fit parameters' sigma values. The syntax for this option is: tclout sigma [<>:]n where n is the parameter number. If it is not a variable parameter or if the fit was unable to calculate its sigma, a value of -1.0 is returned. Report added on Mar 06, 2009.
12.5.0x	The model.dat entry for the vequil model is missing the parameter for Ar abundance, causing the vequil parameters which follow it to be misinterpreted in the code. (This bug does not affect the equil model.) Report added on Mar 06, 2009.
12.5.0y	The kerrconv convolution model code is still using the xspec11-only LMODDIR environment variable for locating the kerrtable.dat model data file. This should be modified for usage in xspec12, where LMODDIR doesn't exist. Report added on Mar 11, 2009.
12.5.0z	This removes an ambiguous reference build error specific to gcc-4.1.1 on the Solaris-2.9 platform. Our thanks to Dacheng Lin for pointing this out. Report added on Mar 26, 2009.
12.5.0aa	The diskir model goes to infinity when its rirr parameter = 1.0, which is also the default value of rirr's lower limit. Our thanks to Brian Refsdal for pointing this out. Report added on Mar 26, 2009.
12.5.0ab	The nsmax model function is only able to find its auxiliary files when it is run directly from XSPEC's model data directory (heasoft-6.x/spectral/modelData). Our thanks to Stephen Doe for pointing this out. Report added on Mar 31, 2009.
12.5.0ac	Fixes to a couple of obscure cases of gain parameter usage: If a gain parameter belongs to a response temporarily replaced by a dummy response, or a gain parameter is indirectly removed through the data command erasing its associated spectrum and response, the gain shift may still be applied to a response occupying its former spectrum and source number. This patch also improves the show response output. Report added on Apr 06, 2009.
12.5.0ad	The nthcomp model has an uninitialized variable for the case of input energies less than kT_bb/10^4, which may cause a crash to occur. Our thanks to Gulab Dewangan for pointing this out. Report added on Apr 14, 2009.
12.5.0ae	For those linking the models library into their own programs, the swind1 model crashes when outputing a warning message as it indirectly accesses a variable intended for use only in XSPEC. Our thanks to Stephen Doe for pointing this out. Report added on Apr 14, 2009.

Notes on using SWIG to import C++ in Python

As a test case using my heasp library which is being converted from C to C++. Created heasp.i file to define interface. Basically a concatenation of the heasp.h, PHA.h and PHAII.h files with the following at the top. Adding additional classes will just require cat'ing their *.h file to the end and adding the appropriate #include at the top. Note the %include of "std_string.i" which is required to make the string arguments work. Similar %include specifications will be required if other STL classes are arguments for methods.

%module heasp

%include "std_string.i"

%{
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include
#include

#include

#include "heasp.h"
#include "PHA.h"
#include "PHAII.h"
%}

The interface file is then converted to C++ using

swig -python -c++ -classic heasp.i

which creates heasp_wrap.cxx and heasp.py. These must then be combined with the *.o files to build a shareable library called _heasp.so (note that the _ prefix and .so suffix are required). Compiling heasp_wrap.cxx will require include files from python. On my mac the relevant flag is
-I/Library/Frameworks/Python.framework/Versions/Current/include/python2.5
and on the lab Linux network
-I/usr1/local/include/python2.5

On the mac the command to make the shareable library is
ld -bundle -flat_namespace -undefined suppress -o _heasp.so ${OFILES} heasp_wrap.o -L${HEADAS}/lib -lCCfits_2.1 -lcfitsio_3.12
and on Linux
$(CXX) -shared -o _heasp.so ${OFILES} heasp_wrap.o -L${HEADAS}/lib -lCCfits_2.1 -lcfi
tsio_3.13

The module is loaded into python by
>>> import heasp
On the mac this generates an undefined symbol ___eprintf. This may require an update of Xcode to fix. On linux (running python2.5) there are no problems. Example use is

>>> import heasp
>>> spectrum = heasp.PHA()
>>> spectrum.read('file1.pha',1,1)
>>> spectrum.disp()
>>> spectrum.write('test.pha')

chkarf

Updated and much simplified chkarf so it actually lists the correct keywords as mandatory. This tool really should just be a perl script - the current program is overkill.

using openMP in gcc

I'm testing out parallelization using openMP on an 8-core Linux box. To build xspec to support openMP options requires the hmakerc to be edited to add -fopenmp to CFLAGS, FFLAGS, and CXXFLAGS and -lgomp to be added to F77LIBS4C (I added it immediately before -lgfortran).

First test is in sumape.f to parallelize over the individual elements when interpolating the continuum and pseudo-continuum...

C$OMP PARALLEL PRIVATE(ien,limdown,limup,ihigh,energy)

C$OMP DO SCHEDULE(DYNAMIC)

DO ielt = 1, nmelt

...


ENDDO

C$OMP END DO

C$OMP END PARALLEL

The variables defined as private are to avoid threads overwriting variables being set and used by other threads.

Timing tests show that parallelization doesn't win enough in this case. Including the parallel directives slows a newpar down from 0.02 seconds to 0.4 seconds due to the overhead.

Computing with GPUs

There is web site all about using GPUs for general purpose computing. One particularly useful looking article goes into some of the options and issues of coding for GPUs.

problem with extractor5.09

In the course of Suzaku pipeline testing Lorella spotted a problem I introduced during the attempted speed-up of the fixwmp routine. Pixels with non-zero counts show up even within the part of the region which is supposed to be set to -1.

The particular case of Suzaku demonstrated a conceptual issue here. Those pixels showing up within the supposed excluded region are real events. What is happening is that the region is defined in sky coordinates but the WMAP is written in detector coordinates. I calculate a conversion to determine which WMAP pixels are within the image region however it is a time-averaged solution. In the pre-5.09 version the satellite attitude wobble results in some events being incorrectly excluded from the WMAP. It was these events that the 5.09 version actually included however they are potentially confusing to users.

The fix is checked in as v5.10.

xselect and Chandra grating data

Jane Turner reported that the current version of xselect fails to read Chandra HETG evt2 data. This is due to a bug in xsel_mdb.f which I think only matters for Chandra grating files.

XSPEC units for setplot wave

In v12.5 we switched the y-axis units under setplot wave to per Hz instead of per Angstrom. Since this has not been universally popular we have added an initialization option to decide between the two cases. This is available as patch 12.5.0t. That and other recent patches can be obtained from the usual place and are summarized below

12.5.0r	Fix needed for all etable model calculations. Our thanks to Roderick Johnstone for pointing this out. Report added on Feb 12, 2009.
12.5.0s	When the both the absori and pexriv models are loaded during the same XSPEC session, whichever model is loaded second will not work properly. Note: this bug originated in version 12.5.0. Report added on Feb 13, 2009.
12.5.0t	This adds an option for choosing Y-axis units when plotting in setplot wave mode. The selection is made by the WAVE_PLOT_UNITS entry, which has been added to the new version of the Xspec.init file. The units may now be specified in Hz (the default for version 12.5.0), or angstroms as they were prior to 12.5.0. This applies to plots of data, counts, and all variants of model and ufspec plots. Report added on Feb 19, 2009.
12.5.0u	Minor code changes mostly to clean up the messages reported when running XSPEC under the valgrind software tool. Report added on Feb 19, 2009.
12.5.0v	Multiplicative model components which are created with the mdefine command should not be multiplied by the energy bin widths. Our thanks to Roderick Johnstone for pointing this out. Report added on Feb 19, 2009.

non-integer COUNTS column

Steve Snowden ran into a silent problem because he made spectrum files with non-integer COUNTS columns. This is in violation of the standard and many ftools will simply silently truncate the float value to the nearest integer. I've added checks in the low-level library routines to write warnings in this case.

Update on 5/7/09. It turns out that the RXTE s/w writes PHA files with a real COUNTS column although the values are integers. So, I've improved the checking so that no warnings are given if the values are integers even if the column is defined as real. If the values really are non-integer then a warning is written and the values truncated to integers.

Gelman on statistics

I'm going to start linking to interesting blog posts by Andrew Gelman because they often have important points to remember. Today he talks about a couple of reviews of "The Black Swan" and includes the following note on Bayesian inference :

From a philosophical point of view, I think the most important point of confusion about Bayesian inference is the idea that it's about computing the probability that a model is true. In all the areas I've ever worked on, the model is never true. But what you can do is find out that certain important aspects of the data are highly unlikely to be captured by the fitted model, which can facilitate a "model shift" moment. This sort of falsification is why I believe Popper's philosophy of science to be a good fit to Bayesian data analysis.

Recent XSPEC bug fixes

The following are all available from the usual place.

12.5.0l	This slightly modifies the state of newly untied parameters. When a parameter is untied, its frozen/unfrozen setting should retain the setting of the parameter(s) to which it was linked, rather than be restored to its original value. Our thanks to Phil Evans and Delphine Porquet for pointing this out. Report added on Jan 14, 2009.
12.5.0m	When steppar is executed, it should not automatically run a fit at the end to place the parameters in their best-fit state. Instead it should restore the parameters to their pre-steppar values, regardless of the fit state. Our thanks to Tim Kallman for pointing this out. Report added on Jan 14, 2009.
12.5.0n	Bug fix for a C-to-Fortran boolean conversion error, affecting the default setting for the thermal broadening flag of the bapec and bvapec models. Unless overridden by the APECTHERMAL setting, thermal broadening should be ON by default for bapec and bvapec. This bug has only been observed on Solaris platforms. Report added on Jan 15, 2009.
12.5.0o	When a delcomp operation is performed on the unnamed (default) model and no spectra are loaded, the model calculation is not updated. Our thanks to Laura Brenneman for pointing this out. Report added on Jan 21, 2009.
12.5.0p	This relaxes the OGIP requirement that spectral data files must contain a RESPFILE keyword, primarily to allow the loading of certain RXTE/PCA files. Our thanks to David Smith for pointing this out. Report added on Jan 30, 2009.
12.5.0q	A new option has been added to the tclout command: tclout gain. See the online tclout manual entry for proper syntax and usage. Prior to this, it was not possible to retrieve the values of gain parameters from tclout. Our thanks to Frank Haberl for pointing this out. Report added on Feb 03, 2009.

xselect and region filters

I've added clarification to the on-line help for xselect about region filters. It is dangerous to make region filters in Image coordinates - the correct options are WCS or Physical. If the xselect image binning is not 1 then regions in Image coordinates will not be handled correctly.

Chandra CY11 webspec updated

Updated webspec with the final ARFs for Chandra Cycle 11.

Word-cloud of the 171 Astro2010 NOI titles

recent xspec patches

12.5.0j	Parameter links both to and from a model which is not associated with any spectra (shown as "Active/Off" in the model display), should not be removed when the model's state gets updated (such as when using the dummyrsp command). Our thanks to Laura Brenneman for pointing this out. Report added on Dec 24, 2008.
12.5.0k	The dummyrsp command may cause a crash when entering a very large number of energy bins. Our thanks to Joel Frioriksson for pointing this out. Report added on Dec 31, 2008.

webspec for Chandra

I've updated webspec to use the Chandra CY11 rmfs and arfs released by the CUC. Note that in mid January they will be releasing new arfs with corrected contamination corrections and effective areas.

xspec patch for energies extend

12.5.0i The energies extend option does not work properly for the case of a data group containing multiple responses whose energy bins differ. Our thanks to Joel Frioriksson for pointing this out. Report added on Dec 15, 2008.

webspec updated to include pile-up

At the request of Andrea Prestwich at the CUC I've added the pile-up model to WebSpec. In the simple interface there is a check box to added it while in the advanced interface the user is expected to select pileup as the option for the first model component.

One tweak could be to set the default pile-up parameters appropriately for each instrument.

cfitsio region filtering bug

The recent cfitsio version region filtering fails when the first region is an exclude. This was an error I introduced when extending to multiple components as allowed in the FITS REGION extension. The fix should be available next week. Meanwhile a work-around is to include an initial region covering the entire field.

Update 12/15/08: There is also a memory-handling error in the code which reads a FITS REGION. This bug is only triggered if the extension contains a large number (>100) of exclude regions.

extractor bug for detector coordinate regions

Dave Henley reported problems with the new extractor when using detector coordinates for the region. This is because I wasn't correctly handling the case when the WCS type is blank. This is corrected in v5.09 which should be released as part of HEAsoft patch update next week.

xspec 12.5 patches

Another set of recent xspec patches. Two of these are minor mistakes in the 12.5 release, the others cover relatively obscure problems.

12.5.0e	In the FNINIT function (for users linking the XSPEC models library into their own programs), the default model data directory name needs to be updated from "modelIonData" to "modelData" to coincide with v12.5.0 restructuring. Our thanks to John Houck for pointing this out. Report added on Dec 05, 2008.
12.5.0f	Patch to improve the behavior of an exceptional case of Levenberg-Marquardt fitting, where a parameter becomes pegged due to a zero second-derivative matrix diagonal element AND the user chooses to exit before convergence. This prevents an attempt at a covariance calculation on the parameter, which otherwise might lead to a segmentation fault. Our thanks to Stefano Bianchi and Jeremy Sanders for pointing this out. Report added on Dec 05, 2008.
12.5.0g	Fix to the swind1 model function. Now that it is included as part of XSPEC's built-in models library, it should no longer look to the $LMODDIR symbol for finding the model data directory. Our thanks to Delphine Porquet for pointing this out. Report added on Dec 09, 2008.
12.5.0h	When doing a 2-panel plot of plot counts/lcounts chisq and the current fit statistic is cstat, the Y-axis scaling of the chisq panel is wrong. Our thanks to Fill Humphrey for pointing this out. Report added on Dec 09, 2008.

extractor bug for annular regions

Phil Evans spotted a problem in the new release when using an annulus region type. The region filtering is performed correctly however the WMAP created is too small in the case when the WMAP and image coordinates are the same. This is actually an error I introduced in the cfitsio code (which is however not used internally in cfitsio so doesn't change its standard use). A work-around is to use the elliptannulus region type instead of annulus. I also have a fix within the extractor code to get around this.

This bug is important for Swift analysis.

xselect and ASCA

The switch to wild cards in xselect.mdb was a little too enthusiastic in the case of the ASCA SIS - the catcol list still needs to be specified for individual detectors. The current version causes an error when running make obscat for ASCA SIS. I have checked in a fixed version.

script to split *.rsp files into *.rmf and *.arf

I've written a little perl script split_rsp_to_rmf_arf which takes an input response matrix file and splits it into a response matrix file with unit total response at each energy and an arf which gives the total response at each energy. This can be used to mock up in v12 xspec the behaviour of v11 xspec with /b models. An example is given on the wiki section on backgrounds.

three more patch updates for xspec 12.5

Three minor patches to the new release are available on the bugs page.

12.5.0b	When the editing commands addcomp, delcomp, or editmod have been used on a model that already contains linked parameters, the link expressions are not properly updated in the output to a save model file. Our thanks to Laura Brenneman for pointing this out. Report added on Dec 03, 2008.
12.5.0c	This removes the warnings issued if datasets contain CHANTYPE strings other than PI or PHA. It will still issue warnings if the CHANTYPE differs between the spectral data and RMF files. Report added on Dec 03, 2008.
12.5.0d	Additional C/Fortran function wrappers provided for the following recently added C++ models: cflux, partcov, simpl, and spexpcut. These are required for users who link the XSPEC models library into an external Fortran or C program. Our thanks to John Houck for pointing this out. Report added on Dec 03, 2008.

First xspec v12.5 patch released.

Li Xin-Li noted that his kerrbb model has an error in turning on/off self-irradiation and limb darkening. The model is fine if either both are on or both are off but if one is on and the other off then the wrong combination was calculated. Fixed as 12.5.0.

A couple of extractor fixes

A couple of bug fixes that were too late to get into the recent release.

1. When using a GTI file for time filtering the fall-back position of looking in the first table extension for the table data was not working. This should not be a problem if the gtinam parameter was set correctly.

2. The fixwmp routine was unnecessarily slow with the new method of region filtering. It is still slow but not as slow as it was.

These fixes are in v5.07 and will appear on the HEAsoft bugs page.

minor tweak to extractor region extension output

Switched the region extension written by extractor back to the XMM convention of REG001##. This works for Chandra because CIAO looks for the HDUNAME='REGION' keyword which is also included.

xspec 12.5

HEAsoft 6.6 which includes xspec v12.5 is now available.

updated xselect Chandra response matrix script

The xselect script xsl_chandra_acis_makeresp made some unwarranted assumptions about DSVAL1 holding the list of selected chips. Generalized this and fixed bug in one of the calls to mkacisrsp where I'd left off an wmap parameter specification. This change will make it into the release coming up this week.

writeFITS.tcl

Jörn Wilms pointed out it would be really useful in xspec to be able to write a bunch of information including best-fit parameters and errors to a single row in a FITS file. I've added a script writeFITS.tcl to do this. It either creates a new file or appends to an existing one. Obviously, this is not a general solution for all the information a user might want to save however it can easily be modified to save additional information.

Band model for GRB spectra

Dick Willingale pointed out that the grbm model has a hard-coded, artificial lower limit of 1 keV on the third (tem) parameter. There is no good reason for this so I have removed it for the upcoming release.

simftest.tcl

Moved the ftest.tcl in manager to simftest.tcl and added it to TclIndex so that it is automatically loaded in xspec and available as the command simftest.

Suzaku AO4 on WebSpec

I've updated WebSpec with Suzaku AO4 files. The HXD background files supplied had a total exposure of 3e6 seconds. The recommended procedure is to have a background file with the same exposure as the source file. This assumes that statistical errors on the background are Poisson and the systematic error can be represented by changing the total normalization (using corfile). I modified the standard background files to change them from COUNTS to RATE so an input background file could be created for a given exposure time just by changing the EXPOSURE keyword.

Note that fakeit with a background file creates a simulated source spectrum with the exposure requested and a simulated background spectrum with the same exposure as that of the input background file. A useful enhancement for fakeit would be to control the exposure of the simulated background file - one possibility would be to allow commands such as
XSPEC12> fakeit inbackground.pha 50000

UPDATE 11/04/08: Fixed an error on the energy ranges for the PIN. The input background spectrum is only valid up to 75 keV. Also, use an background exposure time of 10 x that of the source for the PIN as recommended by the HXD team.

latest xspec bug fixes

Three patches have been put out in the last couple of weeks. Two of these deal with the case when parameters differ by very large amounts. This is not recommended - xspec will work best if all the parameters are scaled to O(1).

12.4.0ap	In a Levenberg-Marquardt fit, the zero threshold for the diagonals of the second-derivatives matrix needs to be reduced. This is relevant for cases where fit parameters may differ by enormously large (~10^20) magnitudes. Our thanks to Roderick Johnstone for pointing this out. Report added on Oct 21, 2008.
12.4.0aq	The tclout model command no longer appends a newline character to the model name. This allows its output value to be used more easily as an argument to another XSPEC command. Our thanks to Jeremy Sanders for pointing this out. Report added on Oct 23, 2008.
12.4.0ar	This adds automatic normalization to the covariance calculation performed at the end of a Levenberg-Marquardt fit, which makes it more robust when dealing with large differences in magnitude among the fit parameters. Report added on Oct 29, 2008.

Redshifts in models

Here is a possible generic way to handle redshifts in model components. Allow any component to have a "z" prepended to its name eg pow-> zpow. The model parser strips off the z while setting a has-redshift flag. Any model component with the has-redshift flag set gets a redshift parameter added (after all the other parameters and before any normalization). When evaluating the model the function is passed a redshifted energy array if has-redshift is set. The output flux array could also be corrected for the (1+z) normalization factors although we would have to be careful to make this consistent with the flux and luminosity commands.

The only type of model where this would not work is one which uses the redshift for something in addition to the energy shift - I think the only such case is the cooling flow which needs the luminosity distance to convert the Mdot parameter to an amount of flux to add.

Partial covering convolution model

The new release will include a partial covering convolution model (partcov) which can be used to turn any absorption model into a partially covered version. This depends on the new model parsing that takes into account the non-associative nature of convolution models. So, if P is the partial covering model, M the multiplicative absorption model and A the additive model then (PM)A turns M into a partial covering absorption. However PMA or P(MA) applies the partial covering the product of M and A which does not give the desired result.

CHANTYPE problems

The Suzaku HXD PIN files use a CHANTYPE of PI_PIN. Strictly speaking this is a violation of the standard however it seems too limiting to just allow CHANTYPE the values of PHA or PI. xspec currently generates a spurious error message about mismatched CHANTYPE when reading the HXD spectrum and response. I've put a temporary workaround in RealResponse.cxx which suppresses the spurious warning however we need a more general solution.

update to Jeremy's spex python script

Jeremy Sanders pointed out that files created by his python script using SPEX have no pseudo-continuum causing a seg fault in sumape.f. I've checked in a fix to sumape and Jeremy has also supplied an updated version of the script which avoids the problem (by creating a zero pseudo-continuum).

recorn model

I've worked out how to implement the v11 recornrm model in v12 and allow the correction file norm to be a free parameter. This is done through a new model recorn which has parameters the spectrum number and the correction norm. It is implemented as a mixing model though it doesn't actually do anything to the model spectrum - just changes the correction norm stored in the dataset object. It has to be done as a mixing model to ensure that the XSFunctions library stays free of symbol references to xspec internals - mixing models are not included in XSFunctions but have their own library. This model will be included in the release planned for early November.

Fixed bug in new extractor

Pre-release testing on the Swift pipeline showed up an embarrassing bug in extractor v5 - the BACKSCAL keyword was not written correctly. This is now fixed in v5.02.

There was also a problem writing out the ROTBOX shape in the REGION extension. Further, Swift software does not recognize ROTBOX at present so I changed this to BOX even when there is a rotation angle. This may produce problems with other missions. These changes are v5.03.

And another bug this time when writing the REGION extension in the case when the input event file already has one and further region filtering is performed by extractor. v5.04.

More minor fixes. Removed the reference to number of events rejected because they fall outside the region (which extractor can no longer no). Eliminated duplication of identical regions in the same component being written to the REGION extension. v5.05.

new Comptonization model

Jack Steiner, Ramesh Narayan, Jeff McClintock (CfA) and Ken Ebisawa (ISAS) have supplied a new comptonization convolution model for xspec. This is available under the additional models website and will be included in the next xspec release.

xselect bug in save time

Roderick Johnstone reported that the command "save time cursor" ignores the final argument and always then prompts for it. Triggering this bug depended on which commands had been used earlier. Fixed xsel_save.f.

Fortran compiler notes

I've been working on building heasoft distributions using both gfortran and g95. This under Tiger with Xcode 2.4.1 (which is gcc 4.0.1).

g95 is the current snapshot binary from www.g95.org which is v0.92. It is installed under /usr/local. To build I had to edit the hmakerc files after configure to remove -fno-f2c from the FFLAGS macro.

gfortran is from "fink install gcc42" and is v4.2.2. To build requires editing hmakerc to change -O2 to -O in the FOPT macro and to add at the beginning of the F77LIBS4C the directories -L/usr/lib -L/usr/lib/i686-apple-darwin8/4.0.1 and -L/sw/lib/gcc4.2/lib/gcc/i686-apple-darwin8/4.2.2. Note that it is vital they be in this order and at the beginning of the macro or else the loader will link in gcc v4.2.2 libraries instead of gcc v4.0.1.

compbb documentation

Ken Ebisawa pointed out that the v12 description of the compbb model did not include a change made to the v11 help. I've corrected this and updated the on-line page.

gfortran on the mac

To get gfortran using fink get the package gcc42 or gcc43.

plot delchi

Jeremy Sanders points out that the label for delchi plots is wrong or at least misleading. Fixed this by changing the entry in plotLabels.dat. Note that the label is now hardwired for the chi-squared statistic however the delchi plot doesn't make any sense for other statistics.

Jeremy also noted that the y-axis label is a long way to the left and this looks strange. The reason for the position is that with LAB ROT we need to leave space to write numbers of the form
A x 10^(-B). Curiously, we are defaulting to rotated labels for 2-panel plots but unrotated labels for 1-panel plots. I've no idea why. Note that the easy way to move around the position of the y-axis label is
PLT> lab pos y 3.0

log files in xspec

Fixed bug that was adding a "#" to the next line instead of the current line of log file output. This is a partial fix to issue 1024. Input when setting parameter values (in model or newpar) is still not echoed to the log file - problem is in ModelContainer::setParameterFromPrompt.

Update: had to back this change out because it doesn't work with tcerr, which is not buffered and hence passes one character at a time to TclIO::write. Craig worked out how to fix xsLog.cxx to fix the problem.

Changing NOMAD password

Since I now have to change this every 60 days these notes are to help me remember what to change so that my Thunderbird e-mail keeps working.

1. Change NOMAD password on the webmail.nasa.gov site
2. Using Firefox Preferences remove saved password for webmail.nasa.gov
3. Using Thunderbird Preferences remove saved password for smtp://gsfc...
4. Click on NOMAD inbox and when prompted enter new password. Check the Password Manager box.
5. Restart Thunderbird.
6. Send an e-mail and when prompted enter new password. Check the Password Manager box.

Also need to fix CNE wireless connection. Open Internet Connect, go to 802.1X then "staff CNE wireless" in the configuration menu. Enter the new password in the appropriate box.

xspec priorities

I've created a Wiki page with a list of xspec priority enhancements. These are divided into large and small projects. At the moment they are not priority-ordered but I will add that along with additional information on the large projects.

xspec issue fixes

Updated documentation to match recent enhancements and additional models. Note that the description of the bvapec model is now included along with the bapec model, not in a separate file.

Fixed a plotting bug. The green line marking 0 or 1 in the res, ratio, chi etc. plots did not go all the way across the plot if a rescale was done so that only part of the x-range of the data was included. Note that a rescale to start at an x-value below the initial data will not draw the green line for the x values below the start of the data.

XMM ao8 webspec files

I've replaced the XMM rmfs and arfs in WebSpec with the appropriate ones for AO8.

super-exponential cutoff absorption

At the request of Jean Ballet added a super-exponential cutoff absorption model to the additional models page. This is useful for fitting gamma-ray observations of pulsars (eg Nel & de Jager 1995).

Also added model to the development version so it will be included in the next release.

energies command and table models

Roderick Johnstone pointed out that recent bug fixes have broken use of the energies command for table models. This is fixed in patch 12.4.0an.

added new models to development version

Added the following contributed additional models to the development version so they become standard in the next release : diskir, cflux, nsmax, zxipcf, swind1, kerrdisk, kerrconv, nthcomp.

latest xspec patches

12.4.0ak	The cutoffpl model can show a distortion for very low energies (E/B approx. less than 10^-3). The incomplete gamma function calculation which this is based on needs to be improved in this region. Our thanks to Joern Wilms, Moritz Boeck, and Manfred Hanke for pointing this out. Report added on Aug 28, 2008.
12.4.0al	This adds the Fortran wrapper function xgtcht(console,log) for those wishing to retrieve XSPEC's console and log chatter level settings from inside their own Fortran local models. The function is equivalent to the xgtcht contained in the xanlib library used by XSPEC11. Report added on Aug 28, 2008.
12.4.0am	Enhancement patch required for reading data sets and responses in the new format used by GLAST GBM files. The modifications include the removal of the SPEC_NUM column for type-II data files, and the storing of multiple response matrix extensions into one response file. XSPEC will now distinguish among multiple response extensions by appending a curly-bracket specifier to the response file name: myResponse.rsp{n} where n is the EXTVER keyword value of the extension. Report added on Aug 28, 2008.

parallelizing xspec

Craig has been experimenting with openMP which is included in gcc v4.2 and above. Since openMP uses a shared memory model the trick is to find places where multiple threads can be used which do not trash each others' memory. A valid case is the loop over responses in Model::fold. This speeds up the code if there are multiple datasets in the same datagroup. It involves changes to Model.cxx as well as MultiResponse.cxx and RealResponse.cxx.

Test was with a dual-processor machine running Solaris. Note that the -xopenmp compiler flag required either no optimization or -O3.

calculating fluxes and errors

Prompted by a question from Jeremy Sanders I have developed a new way of calculating fluxes and errors for models and parts of models. The trick is to provide a new convolution model called cflux whose first two parameters are the minimum and maximum energy over which the flux is calculated. The final parameter is the flux. The component is placed in front of the model component(s) whose flux is being calculated and one of the additive components has its normalization frozen. The standard fit and error command on the flux parameter will then provide the best fit flux and confidence region. This model is available through the extra models web page.

xselect and XMM-Newton spectra

The current version of xselect/extractor writes spectra with region extensions having the EXTNAME keyword set to REGION. This is required by CIAO. However, SAS requires EXTNAME to be REG00101 so added a command to xsl_xmm_epic_makeresp to change EXTNAME before calling the response and arf generation tools.

genrsp and LO_THRES

Allyn Tennant pointed out that genrsp writes a keyword LO_THRESH which should actually be LO_THRES. I fixed this as v2.04 by adding the new keyword while leaving the old just in case there is other s/w that expects it.

LISA xspec stuff current status

strain2xspectime.pl creates input file correctly for time series data and this is fit using the mbht model. However, residuals show high frequency correlations presumably because noise is being
applied in frequency space in synthlisa. The strain2xspec.pl and mbh model which work in frequency space are not working correctly - the model doesn't match the data when it should. All current changes are checked into the repository.

I'm not clear how and why the postBuffer is used - need to check this with Sean or Ira.

Larger question is how should the noise be generated ? Is it correct to do this in the frequency domain ? A better detector model may be required.

xspec patches

Some recent patches from the bugs page :

12.4.0ae	When using the gsmooth or swind1 models, certain combinations of parameters can cause an invalid array access. On 64-bit Linux systems this will cause a crash. Our thanks to Curtis McCully for pointing this out. Report added on Jul 08, 2008.
12.4.0af	When settings have changed through either the xsect or cosmo commands, the model calculations should be immediately updated. (This patch also improves the cosmo command-line parser.) Our thanks to Stefano Bianchi for pointing this out. Report added on Jul 08, 2008.
12.4.0ag	The kerrd and nsa additive models will cause XSPEC to crash in certain cases, the latter primarily on Intel/Mac platforms. Report added on Jul 14, 2008.

bug in genrsp

Allyn Tennant spotted a bug in genrsp when making a response for a grating. I'd missed a case when the channel boundaries are in decreasing energy order and the result was the center bin of the gaussian response was chopped out. This is fixed in v2.03.

extractor and filenames with "+"

Kim Page at Leicester reported that the HEAsoft 6.5 version of extractor dies when using a time filter if the filename contains a "+". It was interpreting this as an HDU number. Fixed this and checked in a change which will be included in extractor v5.0.

xspec model library

At the request mainly of the ISIS folks I've managed to make a stripped-down HEAsoft 6.5 distribution which only includes the Xspec model library files to build only libXSFunctions, libXSUtil and libXS. I assume that the user has a cfitsio library already. Some minor hacks of the top level and Xspec configure files were necessary to make sure that hmake didn't try to build any heacore libraries. Also, moved some include files into Xspec/src/include from directories that otherwise aren't needed.