Operations: Modeler

From ILWIS
Jump to: navigation, search
Main Page Arrow.png Operations Arrow.png
Modeler
Modeler concept

In ILWIS, Modeler is designed to automate and execute processes where data are passed from one operation to another for actions, according to a set of connections defined in the workflow. The ultimate goal is to provide users with an easier way to orchestrate or describe complex processing of data in a visual form and consequently to improve efficiency in term of automation, flexibility and analysis. Theoretically in ILWIS, workflow is part of the ILWIS Modeler where users are allowed to create, edit and manage complex models. The ILWIS Modeler consists of four components (Figure 1, e) providing users with necessary tools required to construct, test and execute a workflow:

  1. Conceptual view
  2. App wizard
  3. Specialist editor
  4. Workflow builder

The Conceptual view provides an overview of the model including e.g., descriptive definition, justification for the employed procedures, description for input and output parameters, supporting arguments for the output accuracy, etc. The App wizard , allow users to view the inputs and outputs of the model and the flow of the producers. The available tools allow for manipulating the model inputs and outputs but not the procedures. It guides users to run the workflow with different inputs and outputs, somehow similar to Run form of the workflow builder. The Specialist editor provides details on the procedures and operations as its employed by a specialist using SMCE decision tree that includes internal relationship between model components and flow of the data between operations. The available tools supports manipulation and editing of the procedures, operations and the inputs data. The Workflow builder , is the interface allows user to develop and construct a workflow based on the conceptual design explained in the conceptual view. Basically, the workflow is designed to simplify and shorten sequence of procedures, to be executed multiple times with same or different input data and to be shared with others. However, users need to be aware, that the model or the workflow cannot perform operations or tasks that are not supported by ILWIS package. It can be recommended that users, for the first time, run each of the steps manually, before executing the model entirely.

Workflow builder
Figure 1, Workflow builder panel

The Workflow builder is used to create workflows able to perform sequence of operations on maps (raster, vector) and tables. A simple workflow such as example given below, may includes only one or two operations and input data and generating one outputs while a complex workflow may contains chain of operations, several inputs and a complex flow of data between operations. A workflow can be created in ILWIS via Create tab from ILWIS main window. The available tools in workflow builder panel allows user to construct a workflow and manipulate its content at any stage. A workflow builder (Figure 1) consists of the following:

  • Workflow tool bar (Figure 1, a); allows you to save the model, save the workflow, remove a selected flow or operation, run the workflow in one go or in step mode (step by step), generate a break, insert a new condition, and minimized or maximized the workflow operations.
  • Workflow panel (Figure 1, b); allows user to construct a workflow by dragging operations from the catalog and dropping them in the workflow panel. The connection between operations can also be established here by dragging an arrow (representing inputs/outputs data)from one operation to an other.
  • Workflow action tabs(Figure 1, c); allow user to perform various actions. An action form will be opened once a tab is selected.
  • Workflow action forms (Figure 1, d); are available once an action tab is selected. The current action tabs are: Available workflow, Run form, Select operations, Metadata, Step form and Script form.
The first tab available workflow allows user to open multiple workflows. The - and + buttons can be used to add or remove a workflow to or from the list.
The Run form allows users to enter data including inputs, outputs and a format for the outputs data. The contents of the Run form in term of the number of inputs and outputs varies depends on the complexity of workflow.
The Select operation form allows user to view the parameters required for a selected operation and therefore, its contents is varied depends on the selected operation. When no operation is selected, its content is empty and when, e.g., Binarymathraster operation is selected it shows the operation form where you can enter the inputs and select an operator. The button Set fixed parameters is used to define an input or an operator as a fixed parameters. Once an input or operator is set as fixed, its display color in the workflow will change to green as it is illustrated in Figure 1.
The Metadata form allows users to label the input parameters of an operation. To add labels you need to select an operation first and then open the Metadata form where you can add labels to the input parameters. As you can see in Figure 1, name of the inputs and output files are used as labels to replace the default texts on each operation.
The Step form allows users to view the outputs of every steps of the workflow when it is executed in step mode. This make possible to control and validate the output of the workflow at each step after it being implemented.
The Script form allows user to view the Python syntax used to build the workflow. The Python syntax generated by defaults and provides the possibility to implement the workflow from ILWI-Python console or directly via Python editor. Note that the generated syntax required a bit of editing when it is executed from ILWI-Python console or Python editor.
Workflow example

The following example, illustrated in Figure 1, is an attempt to provide users with instructions on how to build or develop and execute a workflow. The data used in this example, are the Red (subb3.mpr) and Near Infrared (subb4.mpr) bands of Landsat TM and the objective is to calculate NDVI (Normalized Difference Vegetation Index). The required operation is Binary math raster with 3 operators including add, subtract and divide. The NDVI can be calculated using the following equation:

NDVI =

where NIR is the near infrared band and R is the red band of the Landsat TM. The output results of the Workflow can also be obtained via ILWIS-Python editor. The required Python syntax is:


name of the output raster coverage = ilwis.Engine.do(name of the operation, name of an input raster coverage, name of an input raster coverage, operator)


In this exercise, the above Python syntax need to be used 3 times; for add, subtract and divide operators. In the example bellow, the first input raster coverage is TM band 4(subb4.mpr), and the seconded input raster coverage is the TM band 3 (subb3.mpr) and the 3 operators are:

  • subtract

imsub = ilwis.Engine.do("binarymathraster",“file:///C:/my director/subb4.mpr",
“file:///C:/my director/subb3.mpr", "subtract")
  • add

imadd = ilwis.Engine.do('binarymathraster',“file:///C:/my director/subb4.mpr",
“file:///C:/my director/subb3.mpr", "add")
  • divide

NDVI = ilwis.Engine.do('binarymathraster', imsub, imadd", "divide")


In general, the URL used in the Python syntax can be replaced with variable name created in Python. For example: you can open the TM images as variables and give them names as following:


NIRtm = ilwis.FeatureCoverage("file:///C:/my director/subb4.mpr")
Rtm = ilwis.FeatureCoverage("file:///C:/my director/subb3.mpr")


Once the variables are created, they can be used directly in Python syntax without quotation mark and the path as following:

  • subtract

imasub = ilwis.Engine.do('binarymathraster', NIRtm, Rtm, "subtract")

  • add

imadd = ilwis.Engine.do('binarymathraster', NIRtm, Rtm, "add")

  • divide

NDVI = ilwis.Engine.do('binarymathraster', imsub, imadd", "divide")

The same results can be directly obtained using Python operators:


NDVI = (NIRtm - Rtm) / (NIRtm + Rtm)


To create a Workflow from ILWIS main window:

Mar.jpg
  • Click the Create tab on toolbar tabs


Consequently, the workflow object creation form is opened (Figure 2). The form allows user to enter a name for the workflow and also, optionally, addition information such as a description and one or more keywords. The available keywords are listed in a dropdown list. The buttons Delete and + allow you to delete a keyword or assign one to the workflow.

Mar.jpg
  • Click Create & Open button to create the workflow, once the form is completed.


Note, that the created workflow is not saved automatically and has to be saved manually. Even though, the workflow is saved in the active catalog, but it can be only viewed from the system operation catalog.

Figure 2, The workflow object creation form


Once the Create & Open button is clicked, the workflow builder panel will be displayed where users can add operations, set parameters required for the operations including input(s), output(s), memory location, data format and define the flow of the data (Figure 1).
To successfully execute a workflow, you need to complete the forms available bellow the workflow panel with proper inputs and outputs data required for each operation and the flow sequence. The operations and inputs data must be dragged from the catalog and dropped in their corresponding fields in the form or directly typed in.


Workflow builder action tabs form

Available workflow: Displays the name of the active workflow. It allows to have multiple workflows, but displays only one at the time in the panel.
Run form: Allows user to inter inputs, outputs and a format for the output data to be saved. Use drag and drop to enter inputs data in their corresponding field of the form. The default output format is Memory ; if you select it, the output will be temporally saved in the internal catalog and will be removed when you exit the program. If you select other format, the output will be saved in the active catalog.
Select operation: Once an operation is selected in the workflow panel, the inputs and outputs required for the selected operation can be visualized and be set as fixed parameters when it is necessary.
Metadata: The Metadeta form is used to label the input and output parameters for a selected operation.
Step form In case the workflow is executed by predefined steps, this form can be used to view each step and validate the workflow results.
Script form Shows Python syntax required for this workflow. Simply, users can use them (with a bit of modification) to generate the same results by ILWIS-Python editor.