Registering analytic workflows

Application or processor

A containerized (docker) application that takes in some form of input and generates specific output. An example of such an application is one that may take some files as input, do some analysis on them and produce a report of the results in a desired format, such as a csv or pdf file. These applications can be written in any programming language, as long as they can be containerized. Our initial target users code analytics pipelines in either Python or R.

Workflow

A collection of applications or processors that are executed in a specified order in order to compete a specific task. One application may for instance prepare data (pre-processor), another application perform analysis on the prepared data (analytic processor) and output some data, and a third application post process the output data (post-processor). The Pennsieve platform currently supports a three-processor workflow.

Registration of an application

1. From within the Analysis View select the Applications tab
2. Click on Create Application
3. Complete the form, inputting all the required fields such as Name, Description, Application Type, Resources (CPU, Memory), Compute Node, Source Type, Source URL. The application type is one of three options: Preprocessor, Processor and PostProcessor. A Preprocessor is the application that prepares the data, the Processor is the application that runs analysis on the data and the Postprocessor is the application that performs post-processing on the data. Pennsieve provides a default preprocessor and postprocessor for users. The Compute Node dropdown lists the compute that you would like your application to be registered on. The Parameters are optional and these represent the parameters that your application will take as ENV variables.
4. Click Create Application to submit the form
5. Once the form submission is complete the application will appear under Applications tab and the status will be updated to show the various states as it is registered.