Picture for Ryo Karakida

Ryo Karakida

Recurrent Self-Attention Dynamics: An Energy-Agnostic Perspective from Jacobians

Add code
May 26, 2025
Viaarxiv icon

Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation

Add code
Nov 04, 2024
Viaarxiv icon

Optimal Layer Selection for Latent Data Augmentation

Add code
Aug 24, 2024
Figure 1 for Optimal Layer Selection for Latent Data Augmentation
Figure 2 for Optimal Layer Selection for Latent Data Augmentation
Figure 3 for Optimal Layer Selection for Latent Data Augmentation
Figure 4 for Optimal Layer Selection for Latent Data Augmentation
Viaarxiv icon

Hierarchical Associative Memory, Parallelized MLP-Mixer, and Symmetry Breaking

Add code
Jun 18, 2024
Viaarxiv icon

Self-attention Networks Localize When QK-eigenspectrum Concentrates

Add code
Feb 03, 2024
Figure 1 for Self-attention Networks Localize When QK-eigenspectrum Concentrates
Figure 2 for Self-attention Networks Localize When QK-eigenspectrum Concentrates
Figure 3 for Self-attention Networks Localize When QK-eigenspectrum Concentrates
Figure 4 for Self-attention Networks Localize When QK-eigenspectrum Concentrates
Viaarxiv icon

On the Parameterization of Second-Order Optimization Effective Towards the Infinite Width

Add code
Dec 19, 2023
Viaarxiv icon

MLP-Mixer as a Wide and Sparse MLP

Add code
Jun 02, 2023
Viaarxiv icon

Attention in a family of Boltzmann machines emerging from modern Hopfield networks

Add code
Dec 09, 2022
Viaarxiv icon

Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias

Add code
Oct 06, 2022
Figure 1 for Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Figure 2 for Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Figure 3 for Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Figure 4 for Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Viaarxiv icon

Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel

Add code
Feb 10, 2022
Figure 1 for Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
Figure 2 for Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
Figure 3 for Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
Figure 4 for Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel
Viaarxiv icon