Csvkit is a suite of command-line tools for converting to and working with CSV, the king of tabular file formats.

• https://csvkit.readthedocs.io/en/1.0.3/

• Module Name: csvkit (see the modules page for more information)
• example : /usr/local/apps/csvkit/csvkit_tutorial/ne_1033.data.xlsx

Allocate an interactive session and run the program. Sample session:

[user@biowulf]$ sinteractive
salloc.exe: Pending job allocation 46116226
salloc.exe: job 46116226 queued and waiting for resources
salloc.exe: job 46116226 has been allocated resources
salloc.exe: Granted job allocation 46116226
salloc.exe: Waiting for resource configuration
salloc.exe: Nodes cn3144 are ready for job

[user@cn3144 ~]$ module load csvkit
[user@cn3144 ~]$ cd /data/$USER/dir
[user@cn3144 ~]$ curl -L -O https://github.com/onyxfish/csvkit/raw/master/examples/realdata/ne_1033_data.xlsx
[user@cn3144 ~]$ in2csv /data/$USER/dir/ne_1033_data.xlsx

[user@cn3144 ~]$ exit
salloc.exe: Relinquishing job allocation 46116226
[user@biowulf ~]$

Create a batch input file (e.g. batch.sh). For example:

#!/bin/bash
set -e
module load csvkit
in2csv exampleInfile

Submit this job using the Slurm sbatch command.

sbatch [--mem=#] batch.sh

Create a swarmfile (e.g. job.swarm). For example:

cd dir1; in2csv infile
cd dir2; in2csv infile
cd dir3; in2csv infile

Submit this job using the swarm command.

swarm -f job.swarm [-g #] --module csvkit