Could our model be deployed in a different context?

This page is a fallback for search engines and cases when javascript fails or is disabled.
Please view this card in the library, where you can also find the rest of the plot4ai cards.

Safety Category
Design PhaseInput PhaseModel PhaseOutput Phase
Could our model be deployed in a different context?

Are you testing the product in a real environment before releasing it? If the model is tested with one set of data and then is deployed in a different environment receiving other types of inputs there is less guarantee that it is going to work as planned. This is also the case in reinforcement learning with the so called wrong objective function where slight changes in the environment often require a full retrain of the model.

If you answered Yes then you are at risk

If you are not sure, then you might be at risk too

Recommendations

  • Use different data for testing and training. Make sure diversity is reflected in the data. Specify your training approach and statistical method. Explore the different environments and contexts and make sure your model is trained with the expected different data sources. This also applies to reinforcement learning.
  • Are you considering enough aspects in the environment? Did you forget any environmental variable that could be harmful? Could limited sampling due to high costs be an issue? Document this risk and look for support in your organisation. The organisation is accountable and responsible for the mitigation or acceptance of this risk. And hopefully you get extra budget assigned.
  • Consider applying techniques such as cultural effective challenge; this is a technique for creating an environment where technology developers can actively participate in questioning the AI process. This better translates the social context into the design process by involving more people and can prevent issues associated with target leakage where the AI system trains on data that prepares it for an alternative job other than the one it was initially intended to complete.

Interesting resources/references