Picture for Arthur Douillard

Arthur Douillard

Communication-Efficient Language Model Training Scales Reliably and Robustly: Scaling Laws for DiLoCo

Add code
Mar 12, 2025
Viaarxiv icon

Eager Updates For Overlapped Communication and Computation in DiLoCo

Add code
Feb 18, 2025
Viaarxiv icon

Streaming DiLoCo with overlapping communication: Towards a Distributed Free Lunch

Add code
Jan 30, 2025
Viaarxiv icon

WARP: On the Benefits of Weight Averaged Rewarded Policies

Add code
Jun 24, 2024
Figure 1 for WARP: On the Benefits of Weight Averaged Rewarded Policies
Figure 2 for WARP: On the Benefits of Weight Averaged Rewarded Policies
Figure 3 for WARP: On the Benefits of Weight Averaged Rewarded Policies
Figure 4 for WARP: On the Benefits of Weight Averaged Rewarded Policies
Viaarxiv icon

A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI

Add code
Apr 23, 2024
Figure 1 for A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI
Figure 2 for A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI
Figure 3 for A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI
Figure 4 for A Mechanism-Based Approach to Mitigating Harms from Persuasive Generative AI
Viaarxiv icon

DiPaCo: Distributed Path Composition

Add code
Mar 15, 2024
Viaarxiv icon

Asynchronous Local-SGD Training for Language Modeling

Add code
Jan 17, 2024
Figure 1 for Asynchronous Local-SGD Training for Language Modeling
Figure 2 for Asynchronous Local-SGD Training for Language Modeling
Figure 3 for Asynchronous Local-SGD Training for Language Modeling
Figure 4 for Asynchronous Local-SGD Training for Language Modeling
Viaarxiv icon

DiLoCo: Distributed Low-Communication Training of Language Models

Add code
Nov 14, 2023
Figure 1 for DiLoCo: Distributed Low-Communication Training of Language Models
Figure 2 for DiLoCo: Distributed Low-Communication Training of Language Models
Figure 3 for DiLoCo: Distributed Low-Communication Training of Language Models
Figure 4 for DiLoCo: Distributed Low-Communication Training of Language Models
Viaarxiv icon

Towards Compute-Optimal Transfer Learning

Add code
Apr 25, 2023
Figure 1 for Towards Compute-Optimal Transfer Learning
Figure 2 for Towards Compute-Optimal Transfer Learning
Figure 3 for Towards Compute-Optimal Transfer Learning
Figure 4 for Towards Compute-Optimal Transfer Learning
Viaarxiv icon

CoMFormer: Continual Learning in Semantic and Panoptic Segmentation

Add code
Nov 25, 2022
Viaarxiv icon