...

• 2024.01.10 Recording (Passcode: P#69H5dm) Slides

• 2024.01.24 Recording (Passcode: t!O5T38a) Slides

• 2024.02.07 Recording (Passcode: L58@v7Dg) Slides | Slides from the talk by Sebastian Lobentanzer

• 2024.02.21 Recording (Passcode: B59W3wT+) Slides

• 2024.03.13 Recording (Passcode: f8B#zunH) Slides

• 2024.03.20 Recording (Passcode: LZ!jZT4z) Slides | Architecture diagram in Draw.io | Architecture diagram PNG file

• 2024.04.03 Recording (Passcode: mSH4#u2%) Slides

• 2024.04.17 Recording (Passcode: Yn2!5qJK) Slides | Slides from the talk by Jon Stevens

• 2024.05.01 Recording (Passcode: 54MvxsP#) Slides

• 2024.05.15 Recording (Passcode: rU#y91m@) Slides

• 2024.05.29 Recording (Passcode: c3df=mWx) Slides

• 2024.07.09 Recording (Passcode: LY=QRI9H) Slides

• 2024.07.24 Recording (Passcode: G36*B=Qv) Slides

• 2024.08.07 Recording (Passcode: %.1&ukfM) Slides | Includes a talk by Peter Dorr: SPARQL query code generation with LLMs

• 2024.09.04 Recording (Passcode: t3?B*?CX) Slides | Includes a talk by Oleg Stroganov on agents controlling the actions of LLMs

Lessons Learned

• The highest risk item is generation of the structured query (Cyphrer or SPARQL) from a plain English request. Some publications estimate success rate of about 48% on the first attempt.

• The structure of the database used for queries matters. LLMs can easier produce meaningful structured queries for databases with flat, simple structure.

• Practically useful system requires filtering or secondary mining of output in addition to natural language narration.

...

1. https://www.sciencedirect.com/science/article/pii/S1359644613001542

2. https://www.nature.com/articles/s41573-020-0087-3

3. https://www.epam.com/about/newsroom/press-releases/2023/epam-launches-dial-a-unified-generative-ai-orchestration-platform

4. https://epam-rail.com/open-source

5. Open LLM Leaderboard: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard

6. Chatbot Arena: https://chat.lmsys.org/?arena

7. Reasoning on Graphs: Faithful and Interpretable Large Language Model Reasoning

   https://arxiv.org/abs/2310.01061

8. Knowledge-Consistent Dialogue Generation with Language Models and Knowledge Graphs

   https://openreview.net/forum?id=WhWlYzUTJfP&source=post_page-----97a4cf96eb69--------------------------------

9. BioChatter Benchmark Results: https://biochatter.org/benchmark-results/#biochatter-query-generation

10. MBET Benchmark (embeddings) https://huggingface.co/spaces/mteb/leaderboard

11. Lora-Land and Lorax: https://predibase.com/lora-land

12. A Benchmark to Understand the Role of Knowledge Graphs on Large Language Model's Accuracy for Question Answering on Enterprise SQL Databases. Summary: queries over a KG with GPT 4 are much more accurate than queries over a SQL database with GPT 4. https://arxiv.org/abs/2311.07509

13. https://towardsdatascience.com/evaluating-llms-in-cypher-statement-generation-c570884089b3

14. https://medium.com/neo4j/enhancing-the-accuracy-of-rag-applications-with-knowledge-graphs-ad5e2ffab663

15. linkedlifedata.com

16. Kazu - Biomedical NLP Framework: https://github.com/AstraZeneca/KAZU

17. https://github.com/f/awesome-chatgpt-prompts/tree/main