Ali Taha Technical notes and personal essays by Ali Taha. https://aliestaha.com en 22580: From GPT2 to Kimi3, Explained A visual walkthrough of the model lineage from GPT2 to Kimi3. https://x.com/waterloo_intern/status/2081762065392541951?s=20 https://x.com/waterloo_intern/status/2081762065392541951?s=20 Mon, 27 Jul 2026 00:00:00 -0700 some notes on writing the fastest video kernel in the world Notes on building and optimizing the fastest video kernel in the world. https://x.com/waterloo_intern/status/2070643039668974060?s=20 https://x.com/waterloo_intern/status/2070643039668974060?s=20 Fri, 26 Jun 2026 00:00:00 -0700 waterloo intern = head of infra A technical account of taking ownership of production infrastructure as an intern. https://x.com/waterloo_intern/status/2057977885840036104?s=20 https://x.com/waterloo_intern/status/2057977885840036104?s=20 Sat, 23 May 2026 00:00:00 -0700 TurboQuant Notes on TurboQuant and practical low-bit model optimization. https://x.com/AliesTaha/status/2037272772305707405 https://x.com/AliesTaha/status/2037272772305707405 Thu, 26 Mar 2026 00:00:00 -0700 Quantization-aware distillation for Qwen 2512 A QAD pipeline that quantizes all 60 transformer blocks to FP4 with imperceptible quality loss. https://x.com/AliesTaha/status/2030074784894308770 https://x.com/AliesTaha/status/2030074784894308770 Sat, 07 Mar 2026 00:00:00 -0800 Optimizing FLUX.2 on B200 Profiling, kernel fusion, autotuning, and selective quantization for state-of-the-art image generation throughput. https://x.com/AliesTaha/status/2024493443905683859 https://x.com/AliesTaha/status/2024493443905683859 Thu, 19 Feb 2026 00:00:00 -0800 Four bits A 1.6× speedup with no perceptible quality loss through 4-bit quantization of FLUX.2 [dev]. https://www.baseten.co/blog/four-bits/ https://www.baseten.co/blog/four-bits/ Wed, 18 Feb 2026 00:00:00 -0800 Elements — a reflection A short note on faith. https://aliestaha.com/writing/elements-a-reflection/ https://aliestaha.com/writing/elements-a-reflection/ Fri, 10 Oct 2025 00:00:00 -0700 How to find your purpose An unfinished note on finding direction. https://aliestaha.com/writing/how-to-find-your-purpose/ https://aliestaha.com/writing/how-to-find-your-purpose/ Fri, 03 Oct 2025 00:00:00 -0700 Matrix multiplication on Blackwell, part 4: breaking SOTA The final optimizations that push a matmul kernel past cuBLAS to 103% of the comparison baseline on Blackwell. https://www.modular.com/blog/matrix-multiplication-on-blackwell-part-4---breaking-sota https://www.modular.com/blog/matrix-multiplication-on-blackwell-part-4---breaking-sota Fri, 19 Sep 2025 00:00:00 -0700 Matrix multiplication on Blackwell, part 3: the optimizations behind 85% of SOTA Register allocation, memory coalescing, and pipelining strategies for higher matmul performance. https://www.modular.com/blog/matrix-multiplication-on-nvidias-blackwell-part-3-the-optimizations-behind-85-of-sota-performance https://www.modular.com/blog/matrix-multiplication-on-nvidias-blackwell-part-3-the-optimizations-behind-85-of-sota-performance Fri, 12 Sep 2025 00:00:00 -0700 Matrix multiplication on Blackwell, part 2: using the hardware Using tcgen05 instructions and tensor memory to build faster matrix multiplication kernels. https://www.modular.com/blog/matrix-multiplication-on-nvidias-blackwell-part-2-using-hardware-features-to-optimize-matmul https://www.modular.com/blog/matrix-multiplication-on-nvidias-blackwell-part-2-using-hardware-features-to-optimize-matmul Fri, 05 Sep 2025 00:00:00 -0700 Matrix multiplication on Blackwell, part 1: introduction GPU programming fundamentals and the first step toward building a matmul kernel that can beat cuBLAS. https://www.modular.com/blog/matrix-multiplication-on-nvidias-blackwell-part-1-introduction https://www.modular.com/blog/matrix-multiplication-on-nvidias-blackwell-part-1-introduction Thu, 28 Aug 2025 00:00:00 -0700