Large Language Model

Aka: Large Language Model, Generative AI, Agentic AI, Chat-GPT

advertisement

II. Precautions

  1. Non-Medical
    1. Not for Medical Care
    2. The author uses several software packages and programming languages to develop FPnotebook content
    3. For convenience, a dozen pages in FPNotebook are dedicated to quick notes on content creation
  2. Quality and Safe Usage
    1. Generative AI is imperfect and requires human oversight, evaluation and iterative improvement
    2. Remember: LLMs generate answers by stringing together the statistically most likely next word
      1. The answer may not be correct, complete, legal, ethical or responsible
      2. Always operate with guard rails and oversight
  3. Integrating AI as a learning resource in medicine
    1. Conditions
      1. At work or school, limit AI tools to officially approved agents
      2. Maintaining patient privacy (e.g. HIPAA) and patient safety are paramount
      3. Always verify AI output with reputable sources
    2. Journal clubs
      1. Compare the accuracy with standard journal articles and medical texts
      2. Critique the AI responses
      3. Review the output of an AI tool analyzing the quality of a journal article
      4. Review accuracy of AI extracted statistics from an article (e.g. NNT, Likelihood Ratio)
    3. Practice patient interactions
      1. Motivational Interviewing with a Chat-Bot
      2. Difficult conversation practice (e.g. Breaking Bad News, Difficult Clinical Encounter)
      3. Mock patient interviews (e.g. guess the medical condition)
    4. Documentation review
      1. Have an AI agent review a de-identified SOAP encounter note for clarity and substance
    5. Other learning strategies
      1. Use AI tools to generate a flash card system (chat gives question, user gives answer)
      2. Log effective prompts for later reuse
    6. References
      1. (2026, Jan) Presc Insights, No. 420112
      2. Rincón (2025) BMC Med Educ 25(1):526 +PMID: 40221725 [PubMed]

III. Types: LLM Models

  1. General
    1. Text data passed to LLMs is tokenized
      1. Words are broken into common segments (~4 chars each)
      2. Numbers are broken into groups of 3 numbers each
    2. Large frontier models typically require payment in the form of price per million tokens
      1. Cost is based on the combination of input tokens, compute tokens and output tokens
  2. Large Frontier Models (run in the cloud, 100s of billions to trillions or parameters)
    1. OpenAI GPT
    2. Anthropic Claude
    3. Google Gemini
    4. Deepseek
    5. Grok
    6. Openai/gpt-oss-120b
  3. Smaller Models (may be run locally with Ollama, model variants with<10-20 Billion parameters)
    1. Meta LLama
    2. Mistral
    3. Microsoft Phi-3
    4. Alibaba Qwen
    5. Google Gemma
    6. Deepseek
    7. Openai/gpt-oss-20b
  4. Resources
    1. Vellum Leaderboard
      1. https://www.vellum.ai/llm-leaderboard

IV. Types: LLM Usage Tiers

  1. Level 1: Transformers (multi-shot prompting)
    1. Chat with a model via Model Specific Web Interface (e.g. Chat GPT)
    2. Chat with a model via Consolidator (e.g. Perplexity)
    3. Chat with a model via API (e.g. python in Jupyter Notebook)
      1. Create json with an array of dictionary objects (contains entire history of chat)
        1. role: system, user, assistant, tool, developer
        2. content: message
  2. Level 2: Agents (one-shot prompting)
    1. Give a model extensive system instructions to be used with every query
    2. Give the agent access to a personal or business knowledge base in various forms
      1. Data may be vectorized for rapid access and context (Retrieval-Augmented Generation or RAG)
    3. Agent masters domain and task specific queries
  3. Level 3: Agentic AI Workflows (Virtual Assistants)
    1. Agents are used in various combinations with one another to form an overall workflow
      1. Agents may be used in serial or parallel, with data passed along the chain of agents
      2. Workflow may include if-then branching logic
      3. Workflows may be autonomous, but typically involve human oversight
    2. Interfaces
      1. Web Interfaces (e.g. Cassidy, TypingMind)
      2. Coded Interfaces (e.g. python)

V. Techniques: Prompting Pearls

  1. Prompting instructions to ensure quality Generative AI answers
    1. Stop and Think deeply about the question
    2. Think Step by Step
    3. If you do not know the answer, say so (do NOT hallucinate)
  2. Recursive use of AI Transformers and Agents
    1. Prompt: "What questions should I answer so that you can generate the best answer to my question?"
    2. Prompt: "I asked another model the question X, and they responded with answer Y. Evaluate their answer."

VI. Technique: System Instructions for Agent Development

  1. Have AI chat generate the system instructions for you
    1. Give it the agents general tasks to reach target output (ultimate goal for the agent)
    2. Ask the AI chat to ask you 5-10 questions related to the agent development it needs to start
    3. Answer the questions, and any follow-up questions
    4. Have the AI chat generate a draft of the system instructions into fields as shown below
  2. System instruction fields (in Markdown for ChatGPT or XML for Claude)
    1. Role
      1. Assistant primary function, expertise and personality
    2. Context
      1. Background information (business info, common scenarios, company values)
    3. Instructions
      1. Step-by-step guides and protocols
      2. Address general and specific scenarios
    4. Criteria
      1. Benchmarks on which to grade the quality of the output
      2. Identify strict rules (Do's and Don'ts)
    5. Examples
      1. Sample scenarios (at least 2-3) with their expected outputs
  3. References
    1. Tyler Fisk, Maven Course: Agentic AI Foundations

VII. Resources: Useful Commercial AI Medical Applications

  1. Open Evidence
    1. https://www.openevidence.com/
  2. PM Cardio (Queen of Hearts)
    1. https://www.powerfulmedical.com/pmcardio-stemi/

VIII. Resources: Useful General AI Tools

  1. Online Search (run multiple LLMs)
    1. Typing Mind
      1. https://www.typingmind.com/
    2. Perplexity
      1. https://www.perplexity.ai/
  2. Frontier LLM
    1. Chat-GPT
      1. https://chatgpt.com/
    2. Google Gemini
      1. https://gemini.google.com/app
    3. Anthropic Claude
      1. https://claude.ai/
  3. Local LLM Use
    1. Ollama (LLM)
      1. https://ollama.com/
    2. Pinokio (image and video generation)
      1. https://pinokio.co/
    3. Hugging Face
      1. https://huggingface.co/

IX. Resources: Learning

  1. AI Engineer Core Track: LLM Engineering, RAG, QLoRA, Agents
    1. https://www.udemy.com/course/llm-engineering-master-ai-and-large-language-models/
  2. AI Engineer Agentic Track: The Complete Agent & MCP Course
    1. https://www.udemy.com/course/the-complete-agentic-ai-engineering-course/

Images: Related links to external sites (from Bing)

These images are a random sampling from a Bing search on the term "Large Language Model." Click on the image (or right click) to open the source website in a new browser window. Search Bing for all related images

Related Topics in Medical Informatics

advertisement