Picture for Soufiane Hayou

Soufiane Hayou

Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory

Add code
Oct 11, 2024
Figure 1 for Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory
Figure 2 for Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory
Figure 3 for Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory
Figure 4 for Maximizing the Potential of Synthetic Data: Insights from Random Matrix Theory
Viaarxiv icon

Visualising Feature Learning in Deep Neural Networks by Diagonalizing the Forward Feature Map

Add code
Oct 05, 2024
Figure 1 for Visualising Feature Learning in Deep Neural Networks by Diagonalizing the Forward Feature Map
Figure 2 for Visualising Feature Learning in Deep Neural Networks by Diagonalizing the Forward Feature Map
Figure 3 for Visualising Feature Learning in Deep Neural Networks by Diagonalizing the Forward Feature Map
Figure 4 for Visualising Feature Learning in Deep Neural Networks by Diagonalizing the Forward Feature Map
Viaarxiv icon

The Impact of Initialization on LoRA Finetuning Dynamics

Add code
Jun 12, 2024
Figure 1 for The Impact of Initialization on LoRA Finetuning Dynamics
Figure 2 for The Impact of Initialization on LoRA Finetuning Dynamics
Figure 3 for The Impact of Initialization on LoRA Finetuning Dynamics
Figure 4 for The Impact of Initialization on LoRA Finetuning Dynamics
Viaarxiv icon

How Bad is Training on Synthetic Data? A Statistical Analysis of Language Model Collapse

Add code
Apr 07, 2024
Viaarxiv icon

LoRA+: Efficient Low Rank Adaptation of Large Models

Add code
Feb 19, 2024
Viaarxiv icon

Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks

Add code
Oct 12, 2023
Figure 1 for Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
Figure 2 for Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
Figure 3 for Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
Figure 4 for Tensor Programs VI: Feature Learning in Infinite-Depth Neural Networks
Viaarxiv icon

Commutative Width and Depth Scaling in Deep Neural Networks

Add code
Oct 02, 2023
Viaarxiv icon

Leave-one-out Distinguishability in Machine Learning

Add code
Sep 29, 2023
Figure 1 for Leave-one-out Distinguishability in Machine Learning
Figure 2 for Leave-one-out Distinguishability in Machine Learning
Figure 3 for Leave-one-out Distinguishability in Machine Learning
Figure 4 for Leave-one-out Distinguishability in Machine Learning
Viaarxiv icon

On the Connection Between Riemann Hypothesis and a Special Class of Neural Networks

Add code
Sep 17, 2023
Viaarxiv icon

Data pruning and neural scaling laws: fundamental limitations of score-based algorithms

Add code
Feb 14, 2023
Viaarxiv icon