Piotr Zulawski
  • Home
  • Cycling
  • Machine Learning
  • Game Development
  • Other
  • Bookshelf
  1. Machine Learning
  • Machine Learning
  • Hearing Things
    • Where Do AED Hallucinations Come From?
    • AED Hallucinations - Causal Intervention
    • AED Hallucinations - The Decoder Knows Too
  • GPT Adventures
    • GPT Adventures
    • GPT Adventures — Part 1: Baseline Implementation
    • Part 2 - Profiling
    • Phase 3
    • GPT Adventures — Part 4: Distribute It
  • A Neural Net From Scratch
  • Corneal Endothelium Image Processing
  • Image Generation with GANs
  • Better GANs - Gradient Penalisation

On this page

  • Machine Learning
    • Hearing Things
    • GPT Adventures
    • Miscellaneous ML Side-Quests

Machine Learning

A collection of ML project write-ups.

Hearing Things

Interpretability experiments on an AED speech recognition model — linear probing and causal activation steering to understand the mechanistic basis of non-speech hallucinations.

AED Hallucination Probe

Hearing Things — Part 1

Linear probing: does the encoder know it’s looking at non-speech?

AED Causal Intervention

Hearing Things — Part 2

Causal intervention: does the decoder read the signal?

AED Decoder Probe

Hearing Things — Part 3

Decoder probe: the model knows it’s hallucinating at both ends

GPT Adventures

A GPT-2 Small implemented from scratch in PyTorch, then made progressively faster — profiling, optimisation, and distributed training across multiple GPUs.

GPT Adventures — Overview

Training a GPT-2 Small from scratch

GPT Adventures Part 1

GPT Adventures — Part 1

Baseline implementation

GPT Adventures Part 2

GPT Adventures — Part 2

Profile & find bottlenecks

GPT Adventures Part 3

GPT Adventures — Part 3

Make it faster, one change at a time

GPT Adventures Part 4

GPT Adventures — Part 4

Distribute it

Miscellaneous ML Side-Quests

Image Generation with GANs

Image Generation with GANs

More convnet fun & basics of GANs

Corneal Endothelium

Corneal Endothelium Image Processing

Reproducing old ML papers & experimenting with convnets

Neural Net From Scratch

A Neural Net From Scratch

Understanding the bread & butter of ML

Better GANs

Better GANs — Gradient Penalisation

Improving GANs by addressing vanishing gradients & mode collapse

Bayesian Optimisation

Bayesian Optimisation For Multi-Objective Problems

A collaboration between Reaction Engines & SecondMind on tricky, real world optimisation problems.

See the blog post and the paper by the excellent SecondMind colleagues.