Hi, I’m

Ivan Ilin

Machine Learning ResearcherPhD Candidate

I work on efficient optimization and compression methods for large language models, including pruning, sparse fine-tuning, quantization, and pipeline parallelism.

Open to machine learning research internships

ivan.ilin@kaust.edu.sa

View CV

StatusPhD Candidate

AffiliationKAUST · Computer Science

Google Scholar

153Citations

5h-index

4i10

GitHub

622Stars

449Followers

26Repos

Research profiles

Google Scholar ORCID GitHub

Social media

YouTube X LinkedIn

View research Explore projects

Research areas

Making large models more efficient

My research focuses on making large models more efficient through compression, adaptation, and distributed training.

Research overview

LLM Pruning

Methods for removing redundant model parameters while preserving model quality.

Sparse Fine-Tuning

Training a small, carefully selected subset of model parameters efficiently.

Quantization

Reducing model precision and memory requirements while controlling quality degradation.

Pipeline Parallelism

Optimization and convergence analysis for training models across pipeline stages with delayed updates.

Featured projects

Here are some of my favorite projects, from research implementations to open-source tools.

QK-Wanda

We couple query and key pruning by extending Wanda scores with opposite-projection activation energies and sharing the pruning budget across Q and K.

First Theory for PipeDream

Randomized PipeDream captures PipeDream's stale-weight behavior in an analyzable block-SGD model, revealing how pipeline depth affects convergence.

Pruning LLMs with Thanos

A block-wise algorithm for pruning large language models using second-order information and coordinated weight compensation.

Super-Tuning for LLMs

We introduce Super, which selects a sparse trainable support using activation-aware pruning scores, and Supra, a matched-budget sparse-plus-LoRA adapter.

See all my projects