Ivan Debono

A notebook of science, thoughts and reason

Month: April 2018

The APC cluster (3): Montepython with Euclid likelihoods

The latest public release of Montepython includes Euclid likelihoods for the redshift survey (euclid_pk) and the cosmic shear survey (euclid_lensing).

The __init__.py  file needs to be edited because of Python syntax issues. If you try to use it as provided, you will get two errors messages.

File "/home/[APClogin]/montepython/montepython/likelihoods/euclid_pk/__init__.py", line 224, in loglkl
k_sigma = np.zeros(2.*self.nbin+1, 'float64')
TypeError: 'float' object cannot be interpreted as an index

The problem here is the decimal point after the 2, which makes it a float, when it is being used to create an index, which must be an integer.

Correction:
k_sigma = np.zeros(2*self.nbin+1, 'float64')

The second error is caused once again by an unnecessary decimal point in the index:
File "/home/[APClogin]/montepython/montepython/likelihoods/euclid_pk/__init__.py", line 330, in integrand
return self.k_fid[:]**2/(2.*pi)**2*((self.tilde_P_th[:,index_z,index_mu] - self.tilde_P_fid[:,index_z,index_mu])**2/((2./self.V_survey[index_z])*(self.tilde_P_th[:,index_z,index_mu] + self.P_shot[index_z])**2 + (self.alpha[:,2.*index_z+1,index_mu]*self.tilde_P_th[:,index_z,index_mu])**2
*self.k_fid[:]**3/2./pi**2
*self.nbin*log(self.kmax/self.kmin)))
IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indice

Correction:
return self.k_fid[:]**2/(2.*pi)**2*((self.tilde_P_th[:,index_z,index_mu] - self.tilde_P_fid[:,index_z,index_mu])**2/((2/self.V_survey[index_z])*
(self.tilde_P_th[:,index_z,index_mu] + self.P_shot[index_z])**2 + (self.alpha[:,2*index_z+1,index_mu]*self.tilde_P_th[:,index_z,index_mu])**2
*self.k_fid[:]**3/2./pi**2*
self.nbin*log(self.kmax/self.kmin)))

The APC cluster (2): Using Montepython

The official documentation is here http://monte-python.readthedocs.io/en/latest but it glosses over some important details. You may find more information here: http://www.iac.es/congreso/cosmo2017/media/montepython.pdf

Installing Montepython

Installing Montepython is quite straightforward if you follow the installation guide. Just make sure that that your version of Python is 2.7. There are some syntax changes in Python 3 which prevent the code from installing.

Running Montepython

Running Montepython on your local machine is easy if you follow the official documentation. For the code to be any use, however, you need to output chains with thousands of points. And that means running it on the APC cluster.

Here are some helpful tips.

The graphical backend

Montepython and the CLASS Python wrapper use Matplotlib. You need to log in with  the -Y option for both apcssh and apcclm:
$ ssh -Y APClogin@apcssh.in2p3.fr
followed by
$ ssh -Y apcclm

When you run Montepython on the cluster using a script, you will need to set this environment variable in the script itself (see below).

External programs within CLASS

If you modify CLASS by calling an external program (let’s call it PowerSpectrumExtension.py) to calculate some quantity, remember to make it executable by running
chmod +x PowerSpectrumExtension.py

Job submission

You need to write a script that gets the job done. This is described here https://www.apc.univ-paris7.fr/FACeWiki/pmwiki.php?n=Apc-cluster.Scheduler.

When you run jobs on a cluster, you are sharing resources with the other users. If you ask for resources (memory, number of nodes) that are unavailable, or ask for too much, your job will be sent to the back of the queue, or aborted.

Here’s an example of a message for an aborted run:
There are not enough slots available in the system to satisfy the 4 slots
that were requested by the application:
Montepython.py

Either request fewer slots for your application, or make more slots available
for use.

You also need to set the right environment variables for the required libraries

This is an example of a script which ran succesfully on the APC cluster:

#!/bin/bash
#PBS -N JOBNAME
#PBS -o $PBS_JOBID.out
#PBS -e $PBS_JOBID.err
#PBS -q furious
#PBS -m bea
#PBS -M name.surname@apc.univ-paris7.fr
#PBS -l nodes=1:ppn=32,mem=64GB,walltime=200:00:00
export SCRATCH="/scratch/$USER.$PBS_JOBID"
export PATH=/usr/local/openmpi/bin:$PATH
export OMP_NUM_THREADS=8
export LD_LIBRARY_PATH=/usr/local/openmpi/lib/:/usr/local/openmpi/lib/openmpi/:$LD_LIBRARY_PATH
set -e
cd ~/montepython
/usr/local/openmpi/bin/mpirun -np 4 env MPLBACKEND=Agg montepython/Montepython.py run -p input/lcdm.param -o chains/planck/lcdm -N 20000 --silent

The –silent command suppresses Montepython’s screen output (which you don’t need when you submit a cluster job).

Here are some good resources explaingin qsub settings:

 https://hpcc.usc.edu/support/documentation/running-a-job-on-the-hpcc-cluster-using-pbs

http://www.arc.ox.ac.uk/content/pbs

Analysing the chains

Once the run has terminated, output the plots and information by running:

cd montepython
env MPLBACKEND=Agg montepython/Montepython.py info [path]/[to]/[chains]/*.txt --want-covmat

The option –want-covmat outputs the covariance matrix.

Make sure to include env MPLBACKEND=AGG or you will get the usual matplotlib display problems.

© 2018 Ivan Debono

Theme by Anders NorenUp ↑