Pegasus 4.2 is a major release which contains several improvements on
data management capabilities, a new web based monitoring dashboard, job
submission interfaces via CREAM CE, new replica catalog backends and
support for PMC only workflows and IO forwarding for PMC clustered jobs.
The data management improvements include a new simpler site catalog
schema to describe the site layouts, and enables data to be transferred
to and from staging sites using different protocols. A driving force
behind this change was Open Science Grid, in which it is common for the
compute sites to have Squid caches available to the jobs. For example,
Pegasus workflows can now be configured to stage data into a staging
site using SRM or GridFTP, and stage data out over HTTP. This allows the
compute jobs to automatically use the Squid caching mechanism provided
by the sites, when pulling in data to the worker nodes over HTTP.
Also, with this release we include a beta version of a web based
monitoring dashboard (built on flask) that users can use to monitor and
debug their running workflows. The dashboard provides workflow overview,
graphs and job status/outputs.
Job submissions to the CREAM job management system has been implemented
New simpler replica catalog backends are included that allow the user
to specify the input directory where the input files reside instead of
specifying a replica catalog file that contains the mappings.
There is prototypical support for setting up Pegasus to generate the
executable workflow as a PMC task workflow instead of a Condor DAGMan
workflow. This is useful for environments, where Condor cannot be
deployed such as Blue Waters. I/O forwarding in PMC enables each task in
a PMC job to write data to an arbitrary number of shared files in a safe
way. This is useful for clustered jobs that contain lots of tasks and
each task only writes out a few KB of output data.
A complete list of new features is documented in the release notes at
The new version of Pegasus can be downloaded from the Pegasus Website or the apt/yum repositories