Building a Tiny Tensor Library in Raw C
The Problem
Machine learning libraries like TensorFlow and PyTorch make neural networks feel like magic. But if you want to actually understand how they work under the hood? Good luck wading through thousands of lines of abstracted C++ and Python. This repo is for anyone who wants to strip it all down and build a neural network framework from scratch in C—raw, unvarnished, no-nonsense C. No numpy, no hand-holding, just math and pointers.
What This Does
This project creates a minimal tensor library in C, with abstractions for tensor operations, autograd, and enough functionality to train basic neural networks. The key file is tensor.h, which includes the core tensor structure and operations like matrixmultiplication. The repo also has tensorcu.h for CUDA support if you want to offload operations to the GPU (assuming you're feeling adventurous).
Tests are in test.c (for CPU functionality) and testcuda.cu (for GPU). There's also a script, createmnist_csv.py, to preprocess the MNIST dataset into a CSV format for training. The README.md does a pretty solid job of walking you through the basics of neural networks and how the library works—great for anyone starting from scratch.
Real-World Use
Want to see how backprop works without wading through someone else's framework? This is your playground. Here's a snippet to whet your appetite:
You can use the provided files to build, train, and test a neural net to classify MNIST digits. It's not going to beat GPT-4, but that's not the point. This is about learning.
The Bottom Line
If you're just here to "get stuff done," run back to your pytorch warm blanket. But if you're the kind of person who likes writing their own linked list for fun, this repo is your jam. It's rough, it's raw, and that's the point. Just don’t expect hand-holding or production-ready code—it’s a learning tool, not a silver bullet.