Picture for Mostafa Elhoushi

Mostafa Elhoushi

Hardware Scaling Trends and Diminishing Returns in Large-Scale Distributed Training

Add code
Nov 20, 2024
Viaarxiv icon

PETAH: Parameter Efficient Task Adaptation for Hybrid Transformers in a resource-limited Context

Add code
Oct 23, 2024
Viaarxiv icon

Brevity is the soul of wit: Pruning long files for code generation

Add code
Jun 29, 2024
Figure 1 for Brevity is the soul of wit: Pruning long files for code generation
Figure 2 for Brevity is the soul of wit: Pruning long files for code generation
Figure 3 for Brevity is the soul of wit: Pruning long files for code generation
Figure 4 for Brevity is the soul of wit: Pruning long files for code generation
Viaarxiv icon

LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding

Add code
Apr 29, 2024
Viaarxiv icon

CHAI: Clustered Head Attention for Efficient LLM Inference

Add code
Mar 12, 2024
Figure 1 for CHAI: Clustered Head Attention for Efficient LLM Inference
Figure 2 for CHAI: Clustered Head Attention for Efficient LLM Inference
Figure 3 for CHAI: Clustered Head Attention for Efficient LLM Inference
Figure 4 for CHAI: Clustered Head Attention for Efficient LLM Inference
Viaarxiv icon

Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks

Add code
Mar 07, 2024
Figure 1 for Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Figure 2 for Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Figure 3 for Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Figure 4 for Evaluation of LLMs on Syntax-Aware Code Fill-in-the-Middle Tasks
Viaarxiv icon

AST-T5: Structure-Aware Pretraining for Code Generation and Understanding

Add code
Jan 05, 2024
Figure 1 for AST-T5: Structure-Aware Pretraining for Code Generation and Understanding
Figure 2 for AST-T5: Structure-Aware Pretraining for Code Generation and Understanding
Figure 3 for AST-T5: Structure-Aware Pretraining for Code Generation and Understanding
Figure 4 for AST-T5: Structure-Aware Pretraining for Code Generation and Understanding
Viaarxiv icon

Decoding Data Quality via Synthetic Corruptions: Embedding-guided Pruning of Code Data

Add code
Dec 05, 2023
Viaarxiv icon

SIEVE: Multimodal Dataset Pruning Using Image Captioning Models

Add code
Oct 03, 2023
Viaarxiv icon

Large Language Models for Compiler Optimization

Add code
Sep 11, 2023
Figure 1 for Large Language Models for Compiler Optimization
Figure 2 for Large Language Models for Compiler Optimization
Figure 3 for Large Language Models for Compiler Optimization
Figure 4 for Large Language Models for Compiler Optimization
Viaarxiv icon