Building Secure and Responsible LLM Applications Training Course

Overview of LLM Architecture and Attack Surface

• Understanding how LLMs are built, deployed, and accessed via APIs
• Key components within LLM application stacks (e.g., prompts, agents, memory, APIs)
• Identifying where and how security issues emerge in real-world scenarios

Prompt Injection and Jailbreak Attacks

• Definition and dangers of prompt injection
• Direct and indirect prompt injection scenarios
• Techniques used to bypass safety filters (jailbreaking)
• Strategies for detection and mitigation

Data Leakage and Privacy Risks

• Accidental exposure of data through model responses
• Risks of PII leaks and misuse of model memory
• Designing privacy-conscious prompts and retrieval-augmented generation (RAG) strategies

LLM Output Filtering and Guarding

• Using Guardrails AI for content filtering and validation
• Defining output schemas and constraints
• Monitoring and logging unsafe outputs

Human-in-the-Loop and Workflow Approaches

• Determining appropriate points for human oversight
• Implementing approval queues, scoring thresholds, and fallback mechanisms
• Calibrating trust and leveraging explainability

Secure LLM App Design Patterns

• Applying least privilege and sandboxing for API calls and agents
• Implementing rate limiting, throttling, and abuse detection
• Ensuring robust chaining with LangChain and maintaining prompt isolation

Compliance, Logging, and Governance

• Ensuring auditability of LLM outputs
• Maintaining traceability and version control for prompts
• Aligning operations with internal security policies and regulatory requirements

Summary and Next Steps