A Beginner's look into Artificial Intelligence

Photo of The Matrix. Cite:  https://www.reddit.com/r/MovieDetails/comments/joik3h/to_create_the_code_for_the_matrix_1999_a/ Today, I looked into why ANEs aren't used heavily on Macs for LLMs. I also got through a few more minutes of Baseten's video, which I'm almost done with, and I'm nearly finished with chapter 1 of Basten's inference engineering book. ANEs on Macs are typically only good for fixed-size matrices, so there's a potential speedup for linear attention on Macs. This also explains their lack of use in Mac-based engines. People have shown that they're slower for certain models, though I forgot whether those models were linear. I learned that expert parallelism divides the experts in each layer across GPUs. This means GPU:0 could contain experts 1 and 2 for several layers. I also relearned tensor parallelism. It splits a model's weights for each layer across multiple GPUs, then concatenates the outputs through an all-gather to get the final ou...

Image of Kimi Linear Attention (different than the attention described here). Cite:  https://arxiv.org/abs/2510.26692 A Quick Note,  This is an explainer series to my daily X posts on "becoming a kernel engineer." I'll usually post daily. You can find them here:  https://x.com/dj_sidequests . I'll keep these "blog posts" mostly unrefined. That is, I will be publishing my notes here on my blog. Feel free to message me asking to clarify something! The Problem + Brief Intro 1. Useful background intuition:         1) Information reduction. You reduced a 1million nxn, and n >> d, dxd matrix causes information collapse by compressing that much information. We must solve this issue somehow.         2) Orthogonality: the vectors that make up our matrix are perpendicular (90deg separation). We need to look at our key values. And, if 0 => then orthogonal (else not orthogonal).         3) One way to think about thi...

Engram architecture from Deepseek's paper. Image Link: https://github.com/deepseek-ai/Engram/blob/main/Engram_paper.pdf (page 3) Intro I’ve had a wack idea recently: could a small “hashed-memory module” add useful info to an existing computer vision model? This idea was inspired after DeepSeek’s Engram work (paper: https://arxiv.org/abs/2601.07372 ). Idea Explained TLDR; Engram uses hashed token sequences to look up learned embeddings to combine the important learned tokens with the model’s hidden states.  What I want to do with a learned “look-up table” is see if I can add useful corrections to the final layer of a network to bump up accuracy? I want to first experiment with the last layer of a network to keep my experiment small and controlled at first. Adding corrections to all hidden layers would be useful later on to prevent full hallucinations in tasks such as segmentation (e.g. a model forgot to segment out a whole portion of an object because it did not recognize it was a c...