CRAB3

crab is used to create and submit CMSSW jobs, distributing them to computing centers all over the world. The reference TWiki pages are:

The status of the submitted jobs can be monitored at:

The status of Asynchronous Stage Out (ASO) can be monitored at:

To find the site names for whitelisting or blacklisting, check the CMS CRIC web portal. If you know which dataset you want to analyze, you can also find the site names of the sites that host the dataset using DAS. Note that a user can only access datasets on Tier2 sites, not Tier1 sites. For quicker DAS queries, use the command-line tool dasgoclient.

The CRAB3 output files are stored in a directory with this structure: outLFNDirBase/inputPrimaryDataset/outputDatasetTag/timestamp/; if doing MC event generation, the directory structure is outLFNDirBase/outputPrimaryDataset/outputDatasetTag/timestamp/ where outputPrimaryDataset is specified by the user.

The source codes responsible for the CRAB server and client and other services can be found at:

If you have questions regarding CRAB, you can ask in the Hypernews forum:


Monte Carlo sample production

McM is a webpage used for CERN CMS Monte Carlo Request Management. It is documented at:

It provides the exact commands used to generate official MC samples. To get the commands:

  1. Query CMS DAS to get the McM ‘prepid’.
    • e.g. dataset dataset=/TTJets_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIISpring16DR80-PUSpring16RAWAODSIM_80X_mcRun2_asymptotic_2016_v3-v2/RAWAODSIM gives McM prepid: TOP-RunIISpring16DR80-00030
  2. Edit this URL https://cms-pdmv.cern.ch/mcm/public/restapi/requests?prepid=PREPID by replacing PREPID with the McM prepid.

When the DAS web interface is not working, one can also do the search from the command line

  • e.g. das_client.py --query="mcm dataset=/TTJets_TuneCUETP8M1_13TeV-amcatnloFXFX-pythia8/RunIISpring16DR80-PUSpring16RAWAODSIM_80X_mcRun2_asymptotic_2016_v3-v2/RAWAODSIM"

A quicker way is simply editing the URL:

There is also a list of all the official campaigns:


Event generation with cmsDriver

cmsDriver.py is a tool in CMSSW that is used to create workflows for sample generation. Its syntax and usage are explained in the following TWiki pages:

The implementation of the script is in $CMSSW_RELEASE_BASE/src/Configuration/Applications/python/ConfigBuilder.py.


Install Xilinx PetaLinux 2015.4 on Ubuntu 14.04

The user guide for Xilinx PetaLinux 2015.4 installation is UG1144. It is best used together with Xilinx Vivado 2015.4 version (see my last post about how to install Vivado). More info about PetaLinux embedded OS can be found on Xilinx Products page, and on the Xilinx Wiki site.

Firstly, go to the Xilinx Downloads page to obtain the installer. Select version 2015.4 on the left sidebar. Choose “PetaLinux 2015.4 Installer”. It is a single-file executable that is 1.68 GB large. Note: you have to be a registered user to download it.

Before you proceed, make sure all the prerequisites are satisfied:

sudo apt-get install tofrodos iproute gawk gcc git-core make net-tools \
    libncurses5-dev tftpd zlib1g-dev libssl-dev flex bison libselinux1 \
    lib32z1 lib32ncurses5 lib32bz2-1.0 lib32stdc++6 \
    zip

Change /bin/sh to bash

sudo dpkg-reconfigure dash
# --> Select <No>

Now install PetaLinux into the /opt/PetaLinux directory. The installer is a shell script that runs in the terminal.

chmod +x petalinux-v2015.4-final-installer-dec.run
sudo ./petalinux-v2015.4-final-installer-dec.run /opt/PetaLinux

Press ENTER to see the licenses, q to quit reading the licenses, and y + ENTER to accept the licenses. The installation should last for about 15-30 mins.

Every time you want to use PetaLinux tools, remember to source the “settings” script to have the right environment variables:

source /opt/PetaLinux/petalinux-v2015.4-final/settings.sh

The following is a super simple walkthrough of how to use PetaLinux tools.

  1. To create a PetaLinux project using Zynq for instance:

    petalinux-create -t project -n MYPROJECT --template zynq
    
  2. Get hardware description from Vivado. The hardware description is usually exported by Vivado into e.g. MYVIVADOPROJECT.sdk of a Vivado project.

    cd MYPROJECT
    petalinux-config --get-hw-description=MYVIVADOPROJECT.sdk
    
  3. Make necessary changes to rootfs and device tree (using petalinux -c rootfs or directly editing files under subsystems/linux/configs/). Then, build the embedded OS for Zynq:

    petalinux-build
    
  4. Make the SD card image:

    cd images/linux
    petalinux-package --boot --fsbl zynq_fsbl.elf --fpga system_wrapper.bit --uboot
    
  5. Copy BOOT.BIN and image.ub in that directory to the SD card and use the SD card to boot the Zynq hardware.


Install Xilinx Vivado 2015.4 on Ubuntu 14.04

The user guide for Xilinx Vivado 2015.4 installation is UG973. More info about Vivado Design Suite can be found on Xilinx Products page.

Firstly, go to the Xilinx Downloads page to obtain the installer. Select version 2015.4 on the left sidebar. I use the “Single File Download” option and choose “Vivado HLx 2015.4: Full Installer For Linux Single File Download Image Including SDK”. It is a tarball that is 8.68 GB large. Note: you have to be a registered user to download it.

Now, untar the tarball, cd into the extracted directory, and execute the GUI installer.

tar -zxvf Xilinx_Vivado_SDK_Lin_2015.4_1118_2.tar.gz
cd Xilinx_Vivado_SDK_Lin_2015.4_1118_2
sudo ./xsetup

Agree to the terms and choose Vivado HL WebPACK Edition. Next, tick also Xilinx Software Development Kit (SDK) on the next page, since it’s free and very useful.

At the end of the installation, the license manager will ask for a license. The “Obtain a license” button in the license manager didn’t work for me, so I just go to Xilinx Licensing site directly and get a WebPACK license and install it.

Done that, install the JTAG cable drivers that are needed for many purposes e.g. programming the hardware. Note: by default, Vivado is installed into the /opt/Xilinx/Vivado/2015.4 directory.

cd /opt/Xilinx/Vivado/2015.4/data/xicom/cable_drivers/lin64/install_script/install_drivers
sudo ./install_drivers

Now, change the ownership of the .Xilinx directory so that you may use Vivado without superuser privilege:

sudo chown -hR $USER:$USER $HOME/.Xilinx/

Every time you want to fire up Vivado, remember to source the “settings” scripts to have the right environment variables:

source /opt/Xilinx/Vivado/2015.4/settings64.sh
source /opt/Xilinx/SDK/2015.4/settings64.sh

Finally, fire up Vivado:

vivado

or SDK:

xsdk

(alternatively, you can call SDK from within Vivado.)

Done! =)