Neural Networks you can try to implement from scratch (for beginners)
I was reading a tweet talking about how useful it is to implement neural networks from scratch. How it allowed for a greater understanding of the topic. The author said he found it more useful than other people explaining the concept to him.
While I disagree with the author’s opinion that it stops the need for explanations. It certainly does help the understanding of one’s model.
I recommend giving it a go. In the blog post, I will suggest which models you should try to implement from scratch using NumPy or your favourite library. Also, I will link to some accompanying resources.
Simple Feedforward Network
This is the most famous example because it’s so simple. But allows you to learn so much. I heard about this idea from Andrew Trask. It also helped me think about implementing networks from scratch in general.
In the Feedforward network, you will be using NumPy. As you won't need Pytorch or TensorFlow. To do the heavy-lifting for complex calculations.
You can simply create a Numpy Array for training and testing data. You can also create a nonlinear function using Numpy. Then work out the error rate between the layer’s guess and real data.
Resource for this task: https://iamtrask.github.io/2015/07/12/basic-python-network/
Follow this tutorial. It does a much better job of explaining how to do this in NumPy. With code examples to follow.
Feedforward Network with Gradient Descent
This is an extension of the network above. In this network, we allow the model to optimise its weights. This can also be done in NumPy.
Resource for this task: https://iamtrask.github.io/2015/07/27/python-network-part2/
A follow-on from the previous article.
Pytorch version of Perceptrons and Multi-layered Perceptrons.
Here will go up a level by using a library. Examples I'm using will be done in Pytorch. But you can use whatever library you prefer. When implementing these networks, you learn how much a library does the work for you.
Recourses for the task:
https://medium.com/@tomgrek/building-your-first-neural-net-from-scratch-with-pytorch-56b0e9c84d54
https://becominghuman.ai/pytorch-from-first-principles-part-ii-d37529c57a62
K Means Clustering
Yes, this does not count as a network. But a traditional machine learning algorithm is still very useful. As this is non deep learning algorithm it should be easier to understand. This can be done just using NumPy or Pandas depending on the implementation.
Recourse for this task:
https://www.machinelearningplus.com/predictive-modeling/k-means-clustering/
https://gdcoder.com/implementation-of-k-means-from-scratch-in-python-9-lines/
There are quite a few choices to choose from. So pick whatever implementation helps you understand the concepts better.
These networks or models should be simple enough that you won't get lost trying to implement them. But still, help learn a few stuff along the way.
-
If you found this post useful, then check out my mailing list where I write more stuff like this.