Risk Category | Risk | Cause/trigger | Implication | Chances | Mitigation |
|---|---|---|---|---|---|
Scope | Assay examples insufficient to answer business questions | Planning failure | High - may lead others to think the project is useless | Medium | Focus on high-value business questions during elaboration. Iterative assay selection during implementation of the pilot. |
Scope | List of business questions may be too broad for the pilot project | Planning failure | Medium | High | Prioritize business questions deeper. Define a threshold (number or per cent) of business questions that could be answered for the project to be called an overall success. |
Vendor | Providers not able or not willing to take on the pilot project due to lack of technology, resourcing, or small scope of the pilot | Market | High - may not be able to execute the pilot if this event occurs | Low. At least one provider has expressed interest. | Explore licensing of the tool from CDD or another vendor and driving the annotation work with individual contractors. This in turn may have quality and timing implications. |
Vendor | Market may be too small with only a few providers, and the RFI may result in only a single qualified provider. This may result in unfavorable pricing and terms and conditions of contract | Market | Medium | Medium | Same as above |
Vendor | If the cost per annotated assay in the pilot is too high, sponsors may refuse to support the full-scale project | Market | High | Low. Preliminary studies show reasonable costs for this kind of work. | Accept the risk. Finding a typical cost/assay is one of the objectives of the pilot project. |
Timing | If the pilot is too long, sponsors may lose interest and refuse to support the full-scale project | Planning failure | High | Low | Accept the risk. So far no indication that sponsors desire to cram planning in favor of a deadline. |
Quality | An oversight of some important terms on our part may result in useless annotation | Planning failure | Medium, because not all annotations would be equally affected | High | Review the ontologies prior to start. Run annotations in an agile, iterative manner, and add terms if it is determined that the annotation is incomplete (does not have full value) with the already available terms |
Quality | Annotators may be insufficiently qualified, and we may not be aware of this, resulting in errors | Planning failure | High | Medium | Create a process of training of annotators: first X of the annotations by a new worker are to be reviewed by an already qualified annotator or supervisor. Create a QC process or require that vendor have an appropriate QC process. (Details TBD). Create an audit process, where formal qualifications of the annotators (such as academic credentials, work experience) could be reviewed by the project team members, and workers who do not meet the job qualifications are not allowed to annotate assays. OR require that supplier implement a qualifications review process for workers. |
Quality | Annotators may make mistakes due to time pressure and/or desire to earn more | Bad process | Medium, because not all annotations would be equally affected | Medium | Audit trail that includes record of the identity of the annotator Hide annotator identity from QC Implement a payment scheme that rewards annotations that pass QC (in general or on the first try) Independent double data entry |
General
Content
Integrations