Experiments

An Experiment, at least as we use the term here, is a collection of related Workflow Runs, data, and Events. Every time a Workflow is run, that run is associated with an Experiment, along with any data, files, or results that are generated by that run. In addition, users can log Events to the Experiment from their own application logic. This allows the user to easily track and manage the results of their Workflow runs, and submit their own results and actions to create a complete picture of an experiment.

../../_images/ColorPicker.jpg

A diagram of the logical structure of our “Color Picker” benchmarking experiment.

Experiment Applications

The ad_sdl.wei python package provides the wei.experiment_client.ExperimentClient class to :

  • Easily create or attach to an existing Experiment

  • Submit Workflows to be run as part of an Experiment

  • Log Events related to an Experiment

  • Get results and data from an Experiment

While all this can be done using the wei_server’s REST API, the ExperimentClient provides a convenient wrapper.

We refer to any program or application using the ExperimentClient or the REST API as an Experiment Application. These experiment applications are the primary way to interact with the WEI system, and allow WEI user’s to leverage the capabilities of a WEI Workcell to run their Workflows, while giving them the flexibility to integrate their own custom code and logic. This is especially useful for dynamic or closed-loop AI-driven experiments, where the same Workflow may be run multiple times with different parameters, or where the results of one Workflow run may be used to inform the parameters of the next run.

../../_images/WEI_Diagram.png

A diagram of how a WEI experiment application interacts with a workcell.

Next Steps

To learn how to write your own Experiment Application, consult How-To: Experiments.