Get in Touch

Course Outline

Introduction to Open-Source LLMs

  • Definition and importance of open-weight models.
  • Overview of LLaMA, Mistral, Qwen, and other community-driven models.
  • Use cases for private, on-premise, and secure deployments.

Environment Setup and Tools

  • Installing and configuring Transformers, Datasets, and PEFT libraries.
  • Selecting appropriate hardware for fine-tuning tasks.
  • Loading pre-trained models from Hugging Face or other repositories.

Data Preparation and Preprocessing

  • Dataset formats: instruction tuning, chat data, and text-only inputs.
  • Tokenization techniques and sequence management.
  • Creating custom datasets and data loaders.

Fine-Tuning Techniques

  • Comparing standard full fine-tuning with parameter-efficient methods.
  • Applying LoRA and QLoRA for efficient fine-tuning.
  • Utilizing the Trainer API for rapid experimentation.

Model Evaluation and Optimization

  • Assessing fine-tuned models using generation and accuracy metrics.
  • Managing overfitting, generalization, and validation sets.
  • Performance tuning tips and logging strategies.

Deployment and Private Use

  • Saving and loading models for inference purposes.
  • Deploying fine-tuned models within secure enterprise environments.
  • Comparing on-premise and cloud deployment strategies.

Case Studies and Use Cases

  • Examples of enterprise adoption of LLaMA, Mistral, and Qwen.
  • Handling multilingual and domain-specific fine-tuning challenges.
  • Discussion on trade-offs between open and closed models.

Summary and Next Steps

Requirements

  • Fundamental understanding of large language models (LLMs) and their architecture.
  • Proficiency in Python and PyTorch.
  • Basic familiarity with the Hugging Face ecosystem.

Audience

  • Machine Learning practitioners.
  • AI developers.
 14 Hours

Number of participants


Price per participant

Upcoming Courses

Related Categories