...
Description | Mitigation | Responsible Party |
Failure to identify business questions, or picking too many or too few | Draft appropriate business questions - DONE | Lee Harland, John Wise, Bruce Press, Peter Revill |
BioCypher could be the right KG. It contains Open Targets - DONE | The Hyve Open Targets/EBI: Sebastian Lobentanzer Ellen McDonagh | |
Technology research, feature and cost analysis, and selection | Jon Stevens, Etzard Stolte, Helena Deus; Brian Evarts; Wouter Franke, Matthijs van der Zee | |
Perhaps The Hyve team has a ready answer | ||
Failure to download a large volume of data (all of the PubMed as a maximum) for the prompt-tuning of the LLM | TBD | |
Failure to perform KG generation from text by an LLM |
| |
Failure to perform local KG comparison with calculation of a score |
| |
Failure to generate a proper query for a KG database system by an LLM | Technology research.
| The Hyve Open Targets/EBI: Sebastian Lobentanzer Ellen McDonagh |
Failure to build a prototypical target discovery pipeline on the limited budget in case of mounting technical difficulties | Schedule the project in phases. Aim to answer known unknowns and to establish risk mitigation strategies early in this phase (“project elaboration”) | |
Is there need for an architecture that would be broadly applicable beyond the narrow POC? |
| Need to assemble a new sub-team |
Project Stakeholders
Sponsors:
...
2024.02.07 Recording (Passcode: L58@v7Dg) Slides | Slides from the talk by Sebastian Lobentanzer
References
https://www.sciencedirect.com/science/article/pii/S1359644613001542
Open LLM Leaderboard: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard
Chatbot Arena: https://chat.lmsys.org/?arena
Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning
Knowledge-Consistent Dialogue Generation with Language Models and Knowledge Graphs
BioChatter Benchmark Results: https://biochatter.org/benchmark-results/#biochatter-query-generation
MBET Benchmark (embeddings) https://huggingface.co/spaces/mteb/leaderboard
Lora-Land and Lorax: https://predibase.com/lora-land