Skip to content

Latest commit

 

History

History

Lab3-GenAI

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
assets
assets
 
 
Q&A-in-the-wild.md
Q&A-in-the-wild.md
 
 
README.md
README.md
 
 
lab3-genAI-nlp-explore.ipynb
lab3-genAI-nlp-explore.ipynb
 
 

README.md

Lab 3: Generative AI and Large Language Models

This lab is part of our journey through computational modeling techniques and AI in biomedical and clinical applications. It is designed to give you a comprehensive understanding of how generative AI is transforming society in general and healthcare in particular and the role it will play in the future of medicine.
update: 2024-02-03


If you have a subscription to ChatGPT Plus, you can also try out the the Medical AI Assistant (UiBmed - ELMED219 & BMED365)
GPT and see if you can get it to answer some of your questions. See also Q&A-in-the-wild

Slides

Learning motivations - watch these

(in the order of duration ...)

  • Foundation Models: An Explainer for Non-Experts by Stanford HAI [link] (2:08 min)

    • see also Stanford Center for Research on Foundation Models code
    • and get informed and be inspired by Azeem Azhar’s 2020 conversation with the pioneering AI scientist Fei-Fei Li, professor of computer science at Stanford University and the founding co-director of Stanford’s Human-Centered AI Institute [link] (37:46 min)

  • Introducing GPT-4 by OpenAI [link] (3:12 min)

  • What ChatGPT is and what it's not: A three minutes guide by Richard Van Noorden, Features Editor, Nature [link] (3:51 min)

  • Generative AI from scratch (Graphics in 5 Minutes) by Steve Seitz, UWash GoogleScholar

    • Large Language Models from scratch [link] (8:25 min)
    • Large Language Models: Part 2 [link] (7:19 min)
    • Text to Image in 5 minutes: Parti, Dall-E 2, Imagen [link] (6:00 min)
    • Text to Image: Part 2 -- how image diffusion works in 5 minutes [link] (6:13 min)
    • Reinforcement Learning from scratch [link] (8:25 min)
    • Reinforcement Learning: AlphaGo [link] (8:14 min)
    • Reinforecment Learning: ChatGPT and RLHF [link] (6:31 min)

  • The Exciting, Perilous Journey Toward AGI, TED talk by Ilya Sutskever (OpenAI) [link] (12:25 min)

  • Can AI Catch What Doctors Miss?, TED talk by Eric Topol [link] (14:06 min)

  • Introduction to large language models by John Ewald Google Cloud Tech [link] (15:45 min)

  • Large Language Models for Health 101 by Nigam Shah, Stanford HAI [link] (16:44 min)

    • see also his "A framework for shaping the future of AI in health care" [link]

  • GTP-4 - Complete Beginners Guide by [link] (19:12 min)

  • Introduction to Generative AI by Gwendolyn Stripling Google Cloud Tech [link] (22:07 min)

  • New APPLE AI by TheAIGRID (Apples New Multimodal AI BEATS GPT-4 Vision) NOT an Apple production [link] (22:24 min)

    • see also Ferret: "An End-to-End MLLM that Accept Any-Form Referring and Ground Anything in Response" [paper] [code] license

  • Geoffrey Hinton: Large Language Models in Medicine. They Understand and Have Empathy by Eric Topol, Ground Truth (highly recommended podcast, with transcript) [link] (36:33 min)

  • Embeddings: What they are and why they matter by Simon Willison [link] (38:37 min)

  • Intro to Large Language Models by Andrej Karpathy [link] (59:47 min)

  • Large Language Models and The End of Programming, CS50 Tech Talk with Matt Welsh [link] (66:55 min)

    • CS50 is Harvard University's introduction to the intellectual enterprises of computer science and the art of programming

  • Large Language Models (LLMs) Concenpts, DataCamp interactive course, Beginner (+ Understanding Machine Learning), 15 videos, 50 exercises, [link] (~120 min)

Readings:

(in the order of most recent ...)

  • Karthikesalingam A et al. (Google Research, 12 Jan 2024) AMIE: A research AI system for diagnostic medical reasoning and conversations [link] [arXiv]

  • Oniani D et al. Adopting and expanding ethical principles for generative artificial intelligence from military to healthcare (perspective article published 2 Dec 2023). npj Digital Medicine 2023;6:225. Addresses the ethical dilemmas and challenges posed by the integration of generative AI into healthcare practice, compared with genAI in military use. CC-BY-4.0 [link] [pdf]

  • Toma A et al. Generative AI could revolutionize health care — but not if control is ceded to big tech (comment published 30 Nov 2023). Nature 2023;624:36-38. [link]

  • Clusman J et al. The future landscape of large language models in medicine (perspective published 10 Oct 2023). Communications Medicine 2023;3:141 [link]

  • Thirunavukarasu AJ et al. Large language models in medicine (review article published 17 Jul 2023) Nature Medicine 2023;29:1930–1940. [link]

  • Moore M et al. Foundation models for generalist medical artificial intelligence (perspective article published 12 Apr 2023) Nature 2023;616:259–265. A seminal paper on foundation models in medicine (GMAI). [link]

Repos:

  • AI-in-Health/MedLLMsPracticalGuide: A curated list of practical guide resources of Medical LLMs [link]
    (provides a very comprehensive and updated overview of the field)

  • S. Raschka: LLMs from scratch [https://github.com/rasbt/LLMs-from-scratch] how LLMs work from the inside out ...

    • See also his book Build a Large Language Model (From Scratch) Manning Early Accesss Program [link] ... how LLMs work under the hood, tearing the lid off the Generative AI black box (in progresses from Dec 2023 - final publication in early 2025)
    • The Ahead of AI blogpost: Understanding and Coding Self-Attention, Multi-Head Attention, Cross-Attention, and Causal-Attention in LLMs (published 14 Jan 2024) [link] ... Since self-attention is now everywhere, it's important to understand how it works.

  • Large Language Model Course by Maxime Labonne [link] A frequently updated and "deep" course to get into Large Language Models (LLMs) with roadmaps and Colab notebooks.

OpenAI: Prompt engineering guide

This guide (https://platform.openai.com/docs/guides/prompt-engineering) is highly recommended and shares strategies and tactics for getting better results from large language models like GPT-4. The methods (Six strategies, each with a set of tactics) described in this guide can sometimes be combined for greater effect. Experimentation is encouraged to find the methods that work best for your intentions.

The Six strategies for getting better results:

  • Write clear instructions [link]
  • Provide reference text [link]
  • Split complex tasks into simpler subtasks [link]
  • Give the model time to "think" [link]
  • Use external tools [link]
  • Test changes systematically [link]

Other OpenAI resources

Other sources of inspiration: