Picture for William Merrill

William Merrill

Michael Pokorny

RWKV-7 "Goose" with Expressive Dynamic State Evolution

Add code
Mar 18, 2025
Viaarxiv icon

A Little Depth Goes a Long Way: The Expressive Power of Log-Depth Transformers

Add code
Mar 05, 2025
Viaarxiv icon

Between Circuits and Chomsky: Pre-pretraining on Formal Languages Imparts Linguistic Biases

Add code
Feb 26, 2025
Viaarxiv icon

Humanity's Last Exam

Add code
Jan 24, 2025
Viaarxiv icon

2 OLMo 2 Furious

Add code
Dec 31, 2024
Figure 1 for 2 OLMo 2 Furious
Figure 2 for 2 OLMo 2 Furious
Figure 3 for 2 OLMo 2 Furious
Figure 4 for 2 OLMo 2 Furious
Viaarxiv icon

Evaluating $n$-Gram Novelty of Language Models Using Rusty-DAWG

Add code
Jun 18, 2024
Viaarxiv icon

Let's Think Dot by Dot: Hidden Computation in Transformer Language Models

Add code
Apr 24, 2024
Viaarxiv icon

The Illusion of State in State-Space Models

Add code
Apr 12, 2024
Viaarxiv icon

Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment

Add code
Feb 29, 2024
Figure 1 for Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment
Figure 2 for Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment
Figure 3 for Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment
Figure 4 for Can You Learn Semantics Through Next-Word Prediction? The Case of Entailment
Viaarxiv icon

OLMo: Accelerating the Science of Language Models

Add code
Feb 07, 2024
Figure 1 for OLMo: Accelerating the Science of Language Models
Figure 2 for OLMo: Accelerating the Science of Language Models
Figure 3 for OLMo: Accelerating the Science of Language Models
Figure 4 for OLMo: Accelerating the Science of Language Models
Viaarxiv icon