wei.types.workflow_types

Types related to workflows

Classes

WorkflowStatus(value[, names, module, ...])

Status for a workflow run

pydantic model wei.types.workflow_types.Workflow

Grand container that pulls all info of a workflow together

Show JSON schema

{
   "title": "Workflow",
   "description": "Grand container that pulls all info of a workflow together",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "type": "string"
      },
      "metadata": {
         "$ref": "#/$defs/Metadata"
      },
      "parameters": {
         "anyOf": [
            {
               "items": {
                  "$ref": "#/$defs/WorkflowParameter"
               },
               "type": "array"
            },
            {
               "type": "null"
            }
         ],
         "default": [],
         "title": "Parameters"
      },
      "flowdef": {
         "items": {
            "$ref": "#/$defs/Step"
         },
         "title": "Flowdef",
         "type": "array"
      },
      "modules": {
         "default": [],
         "items": {
            "anyOf": [
               {
                  "type": "string"
               },
               {
                  "$ref": "#/$defs/SimpleModule"
               }
            ]
         },
         "title": "Modules",
         "type": "array"
      }
   },
   "$defs": {
      "Metadata": {
         "additionalProperties": true,
         "description": "Metadata container",
         "properties": {
            "author": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Author"
            },
            "description": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Description"
            },
            "version": {
               "anyOf": [
                  {
                     "type": "number"
                  },
                  {
                     "type": "string"
                  }
               ],
               "default": "",
               "title": "Version"
            }
         },
         "title": "Metadata",
         "type": "object"
      },
      "SimpleModule": {
         "description": "Simple module for use in the workflow file (does not need as much info)",
         "properties": {
            "name": {
               "title": "Name",
               "type": "string"
            }
         },
         "required": [
            "name"
         ],
         "title": "SimpleModule",
         "type": "object"
      },
      "Step": {
         "description": "Container for a single step",
         "properties": {
            "name": {
               "title": "Name",
               "type": "string"
            },
            "module": {
               "title": "Module",
               "type": "string"
            },
            "action": {
               "title": "Action",
               "type": "string"
            },
            "args": {
               "default": {},
               "title": "Args",
               "type": "object"
            },
            "files": {
               "additionalProperties": {
                  "anyOf": [
                     {
                        "type": "string"
                     },
                     {
                        "format": "path",
                        "type": "string"
                     }
                  ]
               },
               "default": {},
               "title": "Files",
               "type": "object"
            },
            "checks": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Checks"
            },
            "locations": {
               "default": {},
               "title": "Locations",
               "type": "object"
            },
            "requirements": {
               "default": {},
               "title": "Requirements",
               "type": "object"
            },
            "dependencies": {
               "default": [],
               "items": {
                  "type": "string"
               },
               "title": "Dependencies",
               "type": "array"
            },
            "priority": {
               "anyOf": [
                  {
                     "type": "integer"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Priority"
            },
            "comment": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Comment"
            },
            "data_labels": {
               "anyOf": [
                  {
                     "additionalProperties": {
                        "type": "string"
                     },
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Data Labels"
            },
            "id": {
               "title": "Id",
               "type": "string"
            },
            "start_time": {
               "anyOf": [
                  {
                     "format": "date-time",
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Start Time"
            },
            "end_time": {
               "anyOf": [
                  {
                     "format": "date-time",
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "End Time"
            },
            "duration": {
               "anyOf": [
                  {
                     "format": "duration",
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Duration"
            },
            "result": {
               "anyOf": [
                  {
                     "$ref": "#/$defs/StepResponse"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null
            }
         },
         "required": [
            "name",
            "module",
            "action"
         ],
         "title": "Step",
         "type": "object"
      },
      "StepResponse": {
         "description": "Standard Response returned by module interfaces\nin response to action requests",
         "properties": {
            "status": {
               "$ref": "#/$defs/StepStatus",
               "default": "succeeded"
            },
            "error": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Error"
            },
            "data": {
               "anyOf": [
                  {
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Data"
            },
            "files": {
               "anyOf": [
                  {
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Files"
            }
         },
         "title": "StepResponse",
         "type": "object"
      },
      "StepStatus": {
         "description": "Status for a step of a workflow",
         "enum": [
            "idle",
            "not_ready",
            "running",
            "succeeded",
            "failed",
            "cancelled"
         ],
         "title": "StepStatus",
         "type": "string"
      },
      "WorkflowParameter": {
         "description": "container for a workflow parameter",
         "properties": {
            "name": {
               "title": "Name",
               "type": "string"
            },
            "default": {
               "anyOf": [
                  {},
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Default"
            }
         },
         "required": [
            "name"
         ],
         "title": "WorkflowParameter",
         "type": "object"
      }
   },
   "required": [
      "name",
      "flowdef"
   ]
}

Config:

use_enum_values: bool = True

Fields:

flowdef (List[wei.types.step_types.Step])
metadata (wei.types.base_types.Metadata)
modules (List[str | wei.types.module_types.SimpleModule])
name (str)
parameters (List[wei.types.workflow_types.WorkflowParameter] | None)

Validators:

ensure_data_label_uniqueness » flowdef
validate_modules » modules

field flowdef: List[Step] [Required]

User Submitted Steps of the flow

Validated by:

ensure_data_label_uniqueness
validate_ulids

field metadata: Metadata [Optional]

Information about the flow

Validated by:

validate_ulids

field modules: List[str | SimpleModule] = []

DEPRECATED: List of modules needed for the workflow. These are no longer validated or required, but the key remains to support legacy workflows.

Validated by:

field name: str [Required]

Name of the workflow

Validated by:

validate_ulids

field parameters: List[WorkflowParameter] | None = []

Inputs to the workflow

Validated by:

validate_ulids

validator ensure_data_label_uniqueness » flowdef: Ensure that the names of the arguments and files are unique

validator validate_modules » modules: Converts SimpleModule objects to strings

pydantic model wei.types.workflow_types.WorkflowParameter

container for a workflow parameter

Show JSON schema

{
   "title": "WorkflowParameter",
   "description": "container for a workflow parameter",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "type": "string"
      },
      "default": {
         "anyOf": [
            {},
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Default"
      }
   },
   "required": [
      "name"
   ]
}

Config:

use_enum_values: bool = True

Fields:

default (Any | None)
name (str)

Validators:

field default: Any | None = None

the default value of the parameter

Validated by:

validate_ulids

field name: str [Required]

the name of the parameter

Validated by:

validate_ulids

pydantic model wei.types.workflow_types.WorkflowRun

Container for a workflow run

Show JSON schema

{
   "title": "WorkflowRun",
   "description": "Container for a workflow run",
   "type": "object",
   "properties": {
      "name": {
         "title": "Name",
         "type": "string"
      },
      "metadata": {
         "$ref": "#/$defs/Metadata"
      },
      "parameters": {
         "anyOf": [
            {
               "items": {
                  "$ref": "#/$defs/WorkflowParameter"
               },
               "type": "array"
            },
            {
               "type": "null"
            }
         ],
         "default": [],
         "title": "Parameters"
      },
      "flowdef": {
         "items": {
            "$ref": "#/$defs/Step"
         },
         "title": "Flowdef",
         "type": "array"
      },
      "modules": {
         "default": [],
         "items": {
            "anyOf": [
               {
                  "type": "string"
               },
               {
                  "$ref": "#/$defs/SimpleModule"
               }
            ]
         },
         "title": "Modules",
         "type": "array"
      },
      "label": {
         "anyOf": [
            {
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Label"
      },
      "run_id": {
         "title": "Run Id",
         "type": "string"
      },
      "payload": {
         "default": {},
         "title": "Payload",
         "type": "object"
      },
      "status": {
         "$ref": "#/$defs/WorkflowStatus",
         "default": "new"
      },
      "steps": {
         "default": [],
         "items": {
            "$ref": "#/$defs/Step"
         },
         "title": "Steps",
         "type": "array"
      },
      "experiment_id": {
         "title": "Experiment Id",
         "type": "string"
      },
      "step_index": {
         "default": 0,
         "title": "Step Index",
         "type": "integer"
      },
      "simulate": {
         "default": false,
         "title": "Simulate",
         "type": "boolean"
      },
      "start_time": {
         "anyOf": [
            {
               "format": "date-time",
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Start Time"
      },
      "end_time": {
         "anyOf": [
            {
               "format": "date-time",
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "End Time"
      },
      "duration": {
         "anyOf": [
            {
               "format": "duration",
               "type": "string"
            },
            {
               "type": "null"
            }
         ],
         "default": null,
         "title": "Duration"
      }
   },
   "$defs": {
      "Metadata": {
         "additionalProperties": true,
         "description": "Metadata container",
         "properties": {
            "author": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Author"
            },
            "description": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Description"
            },
            "version": {
               "anyOf": [
                  {
                     "type": "number"
                  },
                  {
                     "type": "string"
                  }
               ],
               "default": "",
               "title": "Version"
            }
         },
         "title": "Metadata",
         "type": "object"
      },
      "SimpleModule": {
         "description": "Simple module for use in the workflow file (does not need as much info)",
         "properties": {
            "name": {
               "title": "Name",
               "type": "string"
            }
         },
         "required": [
            "name"
         ],
         "title": "SimpleModule",
         "type": "object"
      },
      "Step": {
         "description": "Container for a single step",
         "properties": {
            "name": {
               "title": "Name",
               "type": "string"
            },
            "module": {
               "title": "Module",
               "type": "string"
            },
            "action": {
               "title": "Action",
               "type": "string"
            },
            "args": {
               "default": {},
               "title": "Args",
               "type": "object"
            },
            "files": {
               "additionalProperties": {
                  "anyOf": [
                     {
                        "type": "string"
                     },
                     {
                        "format": "path",
                        "type": "string"
                     }
                  ]
               },
               "default": {},
               "title": "Files",
               "type": "object"
            },
            "checks": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Checks"
            },
            "locations": {
               "default": {},
               "title": "Locations",
               "type": "object"
            },
            "requirements": {
               "default": {},
               "title": "Requirements",
               "type": "object"
            },
            "dependencies": {
               "default": [],
               "items": {
                  "type": "string"
               },
               "title": "Dependencies",
               "type": "array"
            },
            "priority": {
               "anyOf": [
                  {
                     "type": "integer"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Priority"
            },
            "comment": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Comment"
            },
            "data_labels": {
               "anyOf": [
                  {
                     "additionalProperties": {
                        "type": "string"
                     },
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Data Labels"
            },
            "id": {
               "title": "Id",
               "type": "string"
            },
            "start_time": {
               "anyOf": [
                  {
                     "format": "date-time",
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Start Time"
            },
            "end_time": {
               "anyOf": [
                  {
                     "format": "date-time",
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "End Time"
            },
            "duration": {
               "anyOf": [
                  {
                     "format": "duration",
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Duration"
            },
            "result": {
               "anyOf": [
                  {
                     "$ref": "#/$defs/StepResponse"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null
            }
         },
         "required": [
            "name",
            "module",
            "action"
         ],
         "title": "Step",
         "type": "object"
      },
      "StepResponse": {
         "description": "Standard Response returned by module interfaces\nin response to action requests",
         "properties": {
            "status": {
               "$ref": "#/$defs/StepStatus",
               "default": "succeeded"
            },
            "error": {
               "anyOf": [
                  {
                     "type": "string"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Error"
            },
            "data": {
               "anyOf": [
                  {
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Data"
            },
            "files": {
               "anyOf": [
                  {
                     "type": "object"
                  },
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Files"
            }
         },
         "title": "StepResponse",
         "type": "object"
      },
      "StepStatus": {
         "description": "Status for a step of a workflow",
         "enum": [
            "idle",
            "not_ready",
            "running",
            "succeeded",
            "failed",
            "cancelled"
         ],
         "title": "StepStatus",
         "type": "string"
      },
      "WorkflowParameter": {
         "description": "container for a workflow parameter",
         "properties": {
            "name": {
               "title": "Name",
               "type": "string"
            },
            "default": {
               "anyOf": [
                  {},
                  {
                     "type": "null"
                  }
               ],
               "default": null,
               "title": "Default"
            }
         },
         "required": [
            "name"
         ],
         "title": "WorkflowParameter",
         "type": "object"
      },
      "WorkflowStatus": {
         "description": "Status for a workflow run",
         "enum": [
            "new",
            "queued",
            "running",
            "in_progress",
            "paused",
            "completed",
            "failed",
            "unknown",
            "cancelled"
         ],
         "title": "WorkflowStatus",
         "type": "string"
      }
   },
   "required": [
      "name",
      "flowdef",
      "experiment_id"
   ]
}

field duration: timedelta | None = None

Duration of the workflow’s run

Validated by:

validate_ulids

field end_time: datetime | None = None

Time the workflow finished running

Validated by:

validate_ulids

field experiment_id: str [Required]

ID of the experiment this workflow is a part of

Validated by:

validate_ulids

field flowdef: List[Step] [Required]

User Submitted Steps of the flow

Validated by:

ensure_data_label_uniqueness
validate_ulids

field label: str | None = None

Label for the workflow run

Validated by:

validate_ulids

field metadata: Metadata [Optional]

Information about the flow

Validated by:

validate_ulids

field modules: List[str | SimpleModule] = []

DEPRECATED: List of modules needed for the workflow. These are no longer validated or required, but the key remains to support legacy workflows.

Validated by:

field name: str [Required]

Name of the workflow

Validated by:

validate_ulids

field parameters: List[WorkflowParameter] | None = []

Inputs to the workflow

Validated by:

validate_ulids

field payload: Dict[str, Any] = {}

input information for a given workflow run

Validated by:

validate_ulids

field run_id: str [Optional]

ID of the workflow run

Validated by:

validate_ulids

field simulate: bool = False

Whether or not this workflow is being simulated

Validated by:

validate_ulids

field start_time: datetime | None = None

Time the workflow started running

Validated by:

validate_ulids

field status: WorkflowStatus = WorkflowStatus.NEW

current status of the workflow

Validated by:

validate_ulids

field step_index: int = 0

Index of the current step

Validated by:

validate_ulids

field steps: List[Step] = []

WEI Processed Steps of the flow

Validated by:

validate_ulids

get_all_datapoint_ids_by_label(label: str) → List[str]: Return the IDs of all datapoints with the given label in a workflow run

get_datapoint_id_by_label(label: str) → str: Return the ID of the first datapoint with the given label in a workflow run

get_step_by_id(id: str) → Step: Return the step object indexed by its id

get_step_by_name(name: str) → Step: Return the step object by its name

class wei.types.workflow_types.WorkflowStatus(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)

Status for a workflow run

CANCELLED = 'cancelled': Workflow run has been cancelled

COMPLETED = 'completed': Workflow run has completed

FAILED = 'failed': Workflow run has failed

IN_PROGRESS = 'in_progress': Workflow run has started, but is not actively running a step

NEW = 'new': Newly created workflow run, hasn’t been queued yet

PAUSED = 'paused': Workflow run is paused

QUEUED = 'queued': Workflow run is queued, hasn’t started yet

RUNNING = 'running': Workflow is currently running a step

UNKNOWN = 'unknown': Workflow run status is unknown

property is_active: bool: Whether or not the workflow run is active