Registry of medically relevant artificial intelligence models

Interested, who to contact:

@Vladimir Makarov

The innovation team: projectenquiries@pistoiaalliance.org

The Problem:

Artificial intelligence and machine learning models are used more and more often in development of pharmaceuticals and as software components in medical devices (ref 6).
In many cases the models, and the training and testing datasets, are kept as trade secrets, and only the summary performance figures are published. See notes below.
This practice makes the science underlying these products irreproducible, and subjects the users of such medications and devices to the risk of lack of fitness for use of the underlying AI models.
Continuous improvement of such AI models by anyone other than the initial inventor is impossible.
On the other hand, inventors of AI models may object to disclosure of the model particulars that constitute their IP.
The public has the right to access, evaluate, and assess mathematical models that are used to make decisions about public health. Lack of such access is increasing risks to pubic health.

The Proposal:

Create a databank that would house AI and ML models that are medically relevant.
The data would include the model code and all datasets needed to train and validate the model.
The full model code and data would only be avaialble after an embargo period.

This databank would support a feature allowing third-party users to validate the models on the original and on third-party data by taking data to model or model to data, in a secure way, and without breaking the prior generations or versions of models.

Ideally, the proposed databank would be aligned with the National Library of Medicine in the US and its national equivalents elsewhere, as well as the major publishers, and would be accepted as the default destination for publication of any medical AI/ML models by industry.

There are already existing software tools, such as ML model versioning systems (e.g. MLflow.org), electronic notebooks (e.g. Jupyter) and trusted software depositories (ref. 13), that offer some parts of the proposed functionality; but none of them is able to do it all.

The model catalogue would likely have the following interested parties:

Academics could deposit their models and access other's models
Pharma will be interested in looking at other peoples models
Regulators may also become involved

Others have recently made comments and suggested best practices that support this idea:

Bias and the prospect of societal harm increasingly plague artificial-intelligence research (ref 1). The points made in this paper are not industry-specific, and these concerns apply to the medical field.
Best practices in Ai in medicine (refs 2 and 3) have been recently proposed.
Calls are made to exclude the 'black box' from making of important decisions (ref 4) and for transparency of research-use only models (ref. 14).

Public criticisms against some of the high-profile Ai models in medicine are being published: COVID-19 patient decline predictor is not sufficiently tested, yet is pushed into the clinic (ref 5); IBM Watson (refs 8 and 11) is blamed for making faulty recommendations; Google makes extremely far-fetched claims about its cancer diagnostic Ai tools (refs. 9 and 10), but explicitly refuses to release the code; predictive models for hospital readmission risk (reviewed in ref. 12) are typically held as trade secrets.

Adherance of the medical Ai models to the international standards in software development (e.g. ref 7) is not known.

Trusted digital repositories (ref. 13, 15) are now emerging in various fields of science, although many lack features that are described in this proposal and are seen as essential in the highly sensitive medical application.

Intent to run studies aiming to improve reproducibility in computational biology has been declared by others recently (ref. 16).

References: