Skip to main content
Ctrl+K
NeMo-AutoModel - Home NeMo-AutoModel - Home

NeMo-AutoModel

  • GitHub
NeMo-AutoModel - Home NeMo-AutoModel - Home

NeMo-AutoModel

  • GitHub

Table of Contents

Get Started

  • Introduction to the NeMo Automodel Repository
  • Install NeMo Automodel
  • Run on Your Local Workstation

Announcements

  • Challenges in Enabling PyTorch Native Pipeline Parallelism for Hugging Face Transformer Models
  • Google Gemma 3n: Efficient Multimodal Fine-tuning Made Simple
  • Fine-tune Hugging Face Models Instantly with Day-0 Support with NVIDIA NeMo AutoModel

E2E Examples

  • Supervised Fine-Tuning (SFT) with NeMo Automodel
  • Parameter-Efficient Fine-Tuning (PEFT) with NeMo Automodel
  • Pretraining using Megatron-Core Datasets with NeMo Automodel
  • NanoGPT-style Pre-Training with NeMo Automodel
  • Fine-Tune Gemma 3 and Gemma 3n

Model Coverage

  • Large Language Models (LLMs)
  • Vision Language Models (VLMs)

Datasets

  • Dataset Overview: LLM and VLM Datasets in NeMo Automodel
  • Integrate Your Own Text Dataset
  • Use the ColumnMappedTextInstructionDataset
  • Integrate Your Own Multi-Modal Dataset

Development

  • Checkpointing in NeMo AutoModel
  • 🚀 Gradient (Activation) Checkpointing in NeMo-AutoModel
  • Pipeline Parallelism with AutoPipeline
  • Knowledge Distillation with NeMo-AutoModel
  • FP8 Training in NeMo Automodel
  • API Reference
    • nemo_automodel
      • nemo_automodel.shared
        • nemo_automodel.shared.import_utils
        • nemo_automodel.shared.utils
      • nemo_automodel.components
        • nemo_automodel.components._transformers
        • nemo_automodel.components.loggers
        • nemo_automodel.components.moe
        • nemo_automodel.components.config
        • nemo_automodel.components.distributed
        • nemo_automodel.components._peft
        • nemo_automodel.components.optim
        • nemo_automodel.components.checkpoint
        • nemo_automodel.components.quantization
        • nemo_automodel.components.utils
        • nemo_automodel.components.datasets
        • nemo_automodel.components.attention
        • nemo_automodel.components.launcher
        • nemo_automodel.components.models
        • nemo_automodel.components.training
        • nemo_automodel.components.loss

Performance

  • NeMo AutoModel Performance Summary
  • API Reference
  • nemo_automodel
  • nemo_automodel.components
  • nemo_automodel.components.datasets

nemo_automodel.components.datasets#

Subpackages#

  • nemo_automodel.components.datasets.vlm
    • nemo_automodel.components.datasets.vlm.datasets
    • nemo_automodel.components.datasets.vlm.utils
    • nemo_automodel.components.datasets.vlm.collate_fns
  • nemo_automodel.components.datasets.llm
    • nemo_automodel.components.datasets.llm.megatron
      • nemo_automodel.components.datasets.llm.megatron.megatron_utils
      • nemo_automodel.components.datasets.llm.megatron.builder
      • nemo_automodel.components.datasets.llm.megatron.sampler
      • nemo_automodel.components.datasets.llm.megatron.indexed_dataset
      • nemo_automodel.components.datasets.llm.megatron.helpers
      • nemo_automodel.components.datasets.llm.megatron.gpt_dataset
    • nemo_automodel.components.datasets.llm.mock_iterable_dataset
    • nemo_automodel.components.datasets.llm.squad
    • nemo_automodel.components.datasets.llm.mock_packed
    • nemo_automodel.components.datasets.llm.formatting_utils
    • nemo_automodel.components.datasets.llm.nanogpt_dataset
    • nemo_automodel.components.datasets.llm.packed_sequence
    • nemo_automodel.components.datasets.llm.hellaswag
    • nemo_automodel.components.datasets.llm.mock
    • nemo_automodel.components.datasets.llm.megatron_dataset
    • nemo_automodel.components.datasets.llm.column_mapped_text_instruction_dataset

Submodules#

  • nemo_automodel.components.datasets.utils

previous

nemo_automodel.components.utils.sig_utils

next

nemo_automodel.components.datasets.vlm

On this page
  • Subpackages
  • Submodules
NVIDIA NVIDIA
Privacy Policy | Manage My Privacy | Do Not Sell or Share My Data | Terms of Service | Accessibility | Corporate Policies | Product Security | Contact

Copyright © 2025, NVIDIA Corporation.