Minimum Information About an AI/ML Model

Interested, who to contact?

@Vladimir Makarov

The innovation team: projectenquiries@pistoiaalliance.org

The Problem:

In many cases (ref 1) clinically relevant AI/ML models are reported with insufficient detail to be reproducible or without sufficient evidence of calibration. This makes deployment and comparison of AI algorithmic solutions hard.

The Proposal:

Develop a standard for reporting of biomedically-relevant AI/ML models. In other fields, similar standards evolved over time. For example, in the analysis of gene expression, the MIAME standard (Minimum Information About a Microarray Experiment) exists. Others in the past made suggestions for AI/ML reporting specifications, see for instance refs. 2,3,4,5,6, and also declared intent to conduct studies in this field (ref. 7), but to the best of our knowledge, there is no accepted standard at this time.

This is a sister proposal to the https://pistoiaalliance.atlassian.net/wiki/spaces/IC/pages/1877311553 and https://pistoiaalliance.atlassian.net/wiki/spaces/IC/pages/1877180462

Community of Interest:

First Name

Last Name

Organization

First Name

Last Name

Organization

Atul

Butte

UCSF

Beau

Norgeot

UCSF

Fotis

Psomopoulos

ELIXIR EU

Jennifer

Harrow

ELIXIR EU

John

Apathy

BMS

Silvio

Tosatto

ELIXIR EU

Ton

van Daelen

BIOVIA

The first call for the CoI was on October 14th, 2020. The main action that the team agreed on is to organize a webinar panel on this topic. We have some members volunteering to take part in that panel (Atul Butte) and may find additional participants by following up with the SPIRIT AI and CONSORT Ai Working Groups.

References:

  1. https://www.sciencedirect.com/science/article/abs/pii/S0895435618310813

  2. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5238707/

  3. https://tvst.arvojournals.org/article.aspx?articleid=2761237

  4. https://arxiv.org/pdf/1810.03993.pdf

  5. https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(19)30037-6/fulltext

  6. https://www.acpjournals.org/doi/pdf/10.7326/M14-0697

  7. https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1007881