WebSpec - security

I recently installed a new version of WebSpec which fixes security holes and adds some new features.

Since the security fixes might be helpful for others supporting websites I will make a few comments about them. WebSpec operates mainly using Perl CGI scripts which accept information from the user entered in various boxes, process it to create an XSPEC script, run the script, and process the output to generate HTML for the next page. Any instance where information is accepted from the user then either passed to a program or echo'ed back in HTML is a potential serious security risk. The safest course is to check that all input contains only the characters which are expected.

Perl has a paranoid TAINT mode invoked using -T. In this mode Perl will generate an error if any user input is passed to a potentially dangerous command (eg system) without having first been run through a regex which limits the acceptable characters. To illustrate how this works lets look at some WebSpec code. A typical WebSpec Perl script reads from STDIN or the environment variable QUERY_STRING looking for units of form "name = value". Thus

if ($ENV{'REQUEST_METHOD'} eq "POST") {
   read (STDIN, $_, $ENV{'CONTENT_LENGTH'});
} elsif ($ENV{'REQUEST_METHOD'} eq "GET") {
   $_ = $ENV{'QUERY_STRING'};
}


s/^\s*//g;                                  # remove leading whitespace
foreach $_ (split (/&/)) {              # & delimits name/value pairs
   s/\+/ /g;                               # .. which have '=' inside.
   s/\%(..)/sprintf("%c",hex($1)-'0')/geo;
   ($name, $value) = split (/=/, $_, 2);
   $entries{$name} .= $value;
}

This somewhat obscure piece of code puts all the information into an associate array called entries.

Now consider the variable "backsys" which is the systematic error on the background normalization. This must be a positive decimal number so we process it as follows:

($backsys = $entries{"backsys"}) =~ /([0-9\.]+)/;
$backsys = $1;

The term on the right hand side of the first line is a regex expression which says extract a string containing only the characters 0,1,2,..9 and "." and place the result in $1. The square brackets contain the list of allowed characters and the "+" indicates that multiple instances of each character are allowed.

A slightly more complicated case is that for a filename:

($rsp = $entries{"rsp"}) =~ /(\w{1}[\w\_\.]+)/;
$rsp = $1;

Here "\w" means any alphanumeric character so I allow a filename to consist of alphanumeric characters as well as "_" and ".". I also require the first character in the name to be alphanumeric. The importance of these restrictions is to eliminate the possibility of the user giving a filename starting with multiple "../" which means they could get anywhere in the directory tree.

Every member of the entries array is treated in a similar fashion. This is relatively little inconvenience and is worth doing even if you don't think any particular entry is a security threat. It may save you down the line when the script is changed to use the entry in a different fashion.

I'm not aware that other programming languages have anything exactly like the Perl TAINT mode however they can have features which help restrict what might be dangerous code eg Python has RExec.

A couple of useful PHP functions

I've recently added to my home page a count of the number of papers which include the string "xspec". This accesses the new ADSlabs full text search. The html instruction is



where the getXSPECpapers.php file contains

1 - 20 of (.+?)%';
preg_match($regex,$data,$match);
echo $match[1];
?>

This can be modified for any other string by changing the q=xspec section of the URL.

I've also added to my CV page so that it updates the number of citations to each of my papers. The HTML to do this is eg:



where the bibcode argument is the standard ADS bibcode. The getCitations.php file contains

(.+?) abstracts.%';
preg_match($regex,$data,$match);
echo $match[1];
?>

New HEAsoft release: eqpair and related models

The new XSPEC (v12.7.1) has updated eqpair, eqtherm, compth, compps and ntee models. These now all use the same Compton reflection code as reflect and ireflct. This may change results slightly for ionized reflectors because the new code uses the input spectrum to calculate ionization fractions while the old code assumed a power-law.

Note however that these models are still not using a realistic physical model for the ionized reflector.

Release of HEAsoft 6.12/Xspec 12.7.1/PyXspec 1.0

The latest release of HEAsoft (6.12) is available from the usual place. This includes a new release of XSPEC, with the changes being primarily new and modified models, and our first full release of PyXspec (v1.0).

New HEAsoft release: PyXspec non-backwards compatibility

The next HEAsoft release should occur before the end of the week. This is the first of a few notes on some of the more important changes.

This is the release of v1.0 of PyXspec. Our thanks to everyone who has provided feedback on the pre-v1 releases, the comments have been most helpful. Based on this feedback and some further thought by ourselves, we have made a couple of changes which are not backwards compatible. We hope this will not be too inconvenient and believe that the new syntax is clearer. These changes are:

(1) When using multiple data groups, the Model objects assigned to the higher-numbered groups now all have their parameters indexed from 1 to nPar. For example, with a 3 parameter model applied to 2 data groups, you would now access the first parameter in the 2nd model object with "mod2(1)" rather than "mod2(4)".

(2) The Model.setPars() function (introduced with patch 12.7.0f) used the p# keyword argument syntax to set non-consecutive parameters. This has been replaced by the use of Python dictionaries. For example, m.setPars(p2=.3, p4=1.1) should now be m.setPars({2:.3, 4:1.1}).

rfxconv model

Chris Done's convolution model which combines the reflionx table model with the Magdziarz & Zdziarski angle-dependent Compton reflection code is now available from the extra models webpage. This model is described in Kolehmainen, Done & Diaz Trigo (2011).

coplrefl table model file

There is a new table model available from the extra models page. This table model by David Ballantyne and collaborators is intended mainly for observations of X-ray pulsars and describes the reflection of a power-law (with variable high-energy cutoff) from a constant density ionized accretion disk.

Trapping a crash when plotting and a new tclout option

12.7.0t	Fix for a crash which may occur when attempting to plot data for a spectrum which contains no noticed channels. Our thanks to Eduardo Ojero Pascual for pointing this out. Report added on Jan 03, 2012.
12.7.0u	This adds a new tclout option, tclout rerror, which is necessary for retrieving the results of an rerror command run on gain parameters. The syntax for this is: tclout rerror [< source number>:]< gain par number> Our thanks to Matteo Guainazzi for bringing this to our attention. Report added on Jan 18, 2012.