Picture for Yanghua Peng

Yanghua Peng

ByteDance

OVERLORD: Ultimate Scaling of DataLoader for Multi-Source Large Foundation Model Training

Add code
Apr 14, 2025
Viaarxiv icon

Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model

Add code
Apr 11, 2025
Viaarxiv icon

Goku: Flow Based Video Generative Foundation Models

Add code
Feb 10, 2025
Viaarxiv icon

HybridFlow: A Flexible and Efficient RLHF Framework

Add code
Sep 28, 2024
Figure 1 for HybridFlow: A Flexible and Efficient RLHF Framework
Figure 2 for HybridFlow: A Flexible and Efficient RLHF Framework
Figure 3 for HybridFlow: A Flexible and Efficient RLHF Framework
Figure 4 for HybridFlow: A Flexible and Efficient RLHF Framework
Viaarxiv icon

Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation

Add code
Aug 07, 2024
Figure 1 for Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
Figure 2 for Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
Figure 3 for Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
Figure 4 for Optimus: Accelerating Large-Scale Multi-Modal LLM Training by Bubble Exploitation
Viaarxiv icon

ByteCheckpoint: A Unified Checkpointing System for LLM Development

Add code
Jul 29, 2024
Figure 1 for ByteCheckpoint: A Unified Checkpointing System for LLM Development
Figure 2 for ByteCheckpoint: A Unified Checkpointing System for LLM Development
Figure 3 for ByteCheckpoint: A Unified Checkpointing System for LLM Development
Figure 4 for ByteCheckpoint: A Unified Checkpointing System for LLM Development
Viaarxiv icon

QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices

Add code
Jul 02, 2024
Figure 1 for QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
Figure 2 for QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
Figure 3 for QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
Figure 4 for QSync: Quantization-Minimized Synchronous Distributed Training Across Hybrid Devices
Viaarxiv icon

LLM-PQ: Serving LLM on Heterogeneous Clusters with Phase-Aware Partition and Adaptive Quantization

Add code
Mar 02, 2024
Viaarxiv icon

MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs

Add code
Feb 23, 2024
Figure 1 for MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Figure 2 for MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Figure 3 for MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Figure 4 for MegaScale: Scaling Large Language Model Training to More Than 10,000 GPUs
Viaarxiv icon

CDMPP: A Device-Model Agnostic Framework for Latency Prediction of Tensor Programs

Add code
Nov 17, 2023
Viaarxiv icon