Reproducing Diffusion-LM
Reproduced a diffusion text model from Li et al 2022. PyTorch, transformers, diffusion models.
Hi. I'm looking to contribute to the alignment effort as a manager or research engineer.
Reproduced a diffusion text model from Li et al 2022. PyTorch, transformers, diffusion models.
Reproduced mech interp technique for automatically identifying channels that detect visual concepts. PyTorch, convolutional networks.
The original deconvolutional visualization technique from Zeiler and Fergus, 2013. PyTorch, computer vision.
Built transformer from scratch (43M parameters) achieving 3 perplexity on WikiText. PyTorch, attention mechanisms, training optimization.
Co-authored research on activation steering in LLMs. Created contrastive datasets, ran benchmarks on Gemma 2 and Llama models. Python, HuggingFace, GPU computing.
Extended open-source activation steering library to support Gemma 2 and Llama 3. Python, model architectures, layer-wise interventions.
Co-authored research paper on LLM alignment stability. Designed multi-armed bandit experiments, built evaluation framework. Python, OpenAI API, Jupyter.