Risk Log

Risk Category	Risk	Cause/trigger	Implication	Chances	Mitigation	Materialized?

Risk Category	Risk	Cause/trigger	Implication	Chances	Mitigation	Materialized?
Scope	Assay examples insufficient to answer business questions	Planning failure	High - may lead others to think the project is useless	Medium	Focus on high-value business questions during elaboration. Iterative assay selection during implementation of the pilot.	No
Scope	List of business questions may be too broad for the pilot project	Planning failure	Medium	High	Prioritize business questions deeper. Define a threshold (number or per cent) of business questions that could be answered for the project to be called an overall success.	No
Vendor	Providers not able or not willing to take on the pilot project due to lack of technology, resourcing, or small scope of the pilot	Market	High - may not be able to execute the pilot if this event occurs	Low. At least one provider has expressed interest.	Explore licensing of the tool from CDD or another vendor and driving the annotation work with individual contractors. This in turn may have quality and timing implications.	No
Vendor	Market may be too small with only a few providers, and the RFI may result in only a single qualified provider. This may result in unfavorable pricing and terms and conditions of contract	Market	Medium	Medium	Same as above	No
Vendor	If the cost per annotated assay in the pilot is too high, sponsors may refuse to support the full-scale project	Market	High	Low. Preliminary studies show reasonable costs for this kind of work.	Accept the risk. Finding a typical cost/assay is one of the objectives of the pilot project.	No
Timing	If the pilot is too long, sponsors may lose interest and refuse to support the full-scale project	Planning failure	High	Low	Accept the risk. So far no indication that sponsors desire to cram planning in favor of a deadline.	No
Quality	An oversight of some important terms on our part may result in useless annotation	Planning failure	Medium, because not all annotations would be equally affected	High	Review the ontologies prior to start. Run annotations in an agile, iterative manner, and add terms if it is determined that the annotation is incomplete (does not have full value) with the already available terms	No
Quality	Annotators may be insufficiently qualified, and we may not be aware of this, resulting in errors	Planning failure	High	Medium	Create a process of training of annotators: first X of the annotations by a new worker are to be reviewed by an already qualified annotator or supervisor. Create a QC process or require that vendor have an appropriate QC process. (Details TBD). Create an audit process, where formal qualifications of the annotators (such as academic credentials, work experience) could be reviewed by the project team members, and workers who do not meet the job qualifications are not allowed to annotate assays. OR require that supplier implement a qualifications review process for workers.	No for Pilot Open for Phase 1
Quality	Annotators may make mistakes due to time pressure and/or desire to earn more	Bad process	Medium, because not all annotations would be equally affected	Medium	Audit trail that includes record of the identity of the annotator Hide annotator identity from QC Implement a payment scheme that rewards annotations that pass QC (in general or on the first try) Independent double data entry	No for Pilot Open for Phase 1
Vendor	Vendors may be late with either RFP responses or with the project delivery	Market	Low	Medium	Accept the risk. In the very worst case we’d experience a delay.	No