Pipeline Parallelism Theory
A theoretical study of convergence and delayed updates in randomized PipeDream-style pipeline parallelism.
Hi, I’m
Ivan Ilin
Machine Learning ResearcherPhD Candidate
I work on efficient optimization and compression methods for large language models, including pruning, sparse fine-tuning, quantization, and pipeline parallelism.
ivan.ilin@kaust.edu.sa
View CV
StatusPhD Candidate
AffiliationKAUST · Computer Science
Google ScholarUpdated Aug 3, 2026
129Citations
4h-index
4i10
GitHubUpdated Aug 3, 2026
620Stars
448Followers
24Repos
Research areas
Making large models more efficient
Four connected directions across model compression, adaptation, and distributed training.
01
LLM Pruning
Methods for removing redundant model parameters while preserving model quality.
02Sparse Fine-Tuning
Training a small, carefully selected subset of model parameters efficiently.
03Quantization
Reducing model precision and memory requirements while controlling quality degradation.
04Pipeline Parallelism
Optimization and convergence analysis for training models across pipeline stages with delayed updates.
Selected work
Featured projects
Research implementations and open-source tools built around practical constraints.
Sparse Fine-Tuning
Methods for adapting language models by updating only a small, carefully selected subset of parameters.
Thanos
A method for pruning large language models using second-order information and coordinated weight compensation.
VoiceCut
An open-source, local-first tool for automatically editing recorded speech.
Publications
Selected publications
Peer-reviewed papers and recent preprints in optimization and efficient machine learning.
Writing
Latest notes
Project updates and technical writing from ongoing work.
Introducing VoiceCut
Why I built an open-source, local-first tool for cleaning retake-heavy narration.