Page Comparison

Document Information

...

We have defined the detailed requirements for an Ontologies Mapping tool and used this to conduct an initial survey of existing mapping tools. We concluded from this exercise that some of these are likely to substantially meet our requirements. Therefore, the process for a formal request for information (RFI) has been devised by the project team to allow us to evaluate existing mapping tools against our requirements.

It is expected that the participating tool providers will benefit from meaningful engagement with the Life Science industry. This year the Pistoia Alliance (http://www.pistoiaalliance.org) reached more than 100 member organisations which span all sizes of company such as large biopharmaceutical companies, product and service vendors including publishers, start-up small businesses and major academic institutions. This will give participants highly valuable visibility and an understanding of the ontology mapping requirements of Pistoia member organisations. It will also give the tool providers a unique opportunity to demonstrate how well their product meets our requirements and it will generate valuable feedback on aspects which would benefit from further development.

The Pistoia Alliance will not publicly recommend or rank or otherwise confer some kind of status or stratification on any third party solutions. The RFI results from participating tool providers will remain confidential to each provider and Pistoia Alliance members and they are in no way an endorsement or recommendation of anything.  Further details are described in the RFI terms and conditions.

2. Team and Governance

...

We expect sufficient detail about all aspects of our requirements to enable us to evaluate how well they are satisfied. We have provided a template sheet which lists our requirements for the tool provider to summarise existing capabilities and evidence examples:- 

Link to Capabilities Template sheet wiki page

Although the emphasis in our requirements is mapping between ontologies, please describe any additional functionality. In particular, how this impacts on the tool interface design, performance and technical requirements to run.

...

Any additional data sources used by participating tools to assist with these mapping tasks should be described by the tool provider. Where appropriate, the tool provider will demonstrate mapping performance with and without additional data sources.

Our process for evaluating the quality of the mappings produced by the tool provider is described in the next section.

Evaluation of mapping quality

The quality of mappings will be evaluated by the project team against available reference mappings and though manual inspection. To facilitate our evaluation of mappings the tool provider must include three specific areas of disease or phenotype:-

...

Evaluation of mappings will make use of reference mappings (e.g. NCBO, UMLS, MeSH) and it will include the following aspects:-

• manual assessment to compare with reference mappings
• inspection of perfect matches. Are they all found? Does it miss any out correctly? What is missed?
• evaluation of similar matches based on synonyms
• evaluation based on axiom similarity (same parents, came class descriptions)

The mapping tool should generate measures of quality which will include 1) confidence for the match and 2) level of similarity for the match and any additional measures. Similarity of match will be expected to range from equivalent to close similarity to broadly similar. These measures of quality should be recorded in the mapping output which should be in at least one of the standard formats defined in the requirements document. 

If additional data has been used to make the match, please provide access through links to such data. Performance of the tool with and without additional data sources should be shown if practical.

3.4 RFI Process

3.4.1 RFI Schedule

All times are GMT.

3.4.2 RFI Related Questions / Clarifications 

All questions should be sent in writing to ian.harrow@pistoiaalliance.org and martin.romacker@pistoiaalliance.org. Answers to questions will be circulated to all confirmed respondents.

Tool provider respondents may request a Q&A teleconference; if a teleconference is held, all respondents will be invited.

...

Respondents may use their own descriptive format for detailing their submission to this RFI. This will include example mappings including the two tasks described in section 3.3. The mappings must be in one of the format standards listed in the requirements document. 

Submissions should be emailed to ian.harrow@pistoiaalliance.org with the subject title “Ontologies Mapping Tool RFI response by <respondent organisation>”

...

• Any estimate submitted does not constitute a proposal for the work and will not form part of any future RFP process. The project will not expect any respondent to be constrained by the estimate they submitted in any future bid against all or part of these requirements.
• This RFI is the intellectual property of the Pistoia Alliance and will be openly available to other parties to view.
• The Pistoia Alliance will not be liable for any costs incurred in preparation or submission of a response to this RFI.
• The Pistoia Alliance will accept no liability for any loss caused by response to this RFI.
• The Pistoia Alliance reserves the right to extend timelines if circumstances require.
• The Pistoia Alliance will not publicly recommend or rank or otherwise confer some kind of status or stratification on any third party solutions.

Submissions to the RFI by each participating tool provider will be evaluated in relation to our requirements and these results will remain strictly confidential to the project team and shared only with the original tool provider because feedback should remain confidential. This means that none of the response data or summaries will enter the public domain. It permits the project team to make use of the information to determine whether to run a full RFP to develop alternative solutions, or whether they can then make their own decisions as to which existing tools to use themselves in their own organisations but without making any endorsement or comparison externally.

In the case of an existing solution being found which substantially satisfies our requirements, the public output would be the publication of the tool requirements and an indication that existing tools can be found that meet most of them, but without naming those tools. This could involve a third-party specialist website that is likely to be read by the majority of companies active in ontology mapping. Project team and Pistoia members could share internally a summary grid of tool features against our requirements, but without making any recommendations beyond what readers can infer themselves from looking at the grid. 

4. Appendices

4.1 Glossary

*The scope of this RFI is ontologies mapping in the same or closely related data domains