Preface
Back in October 2018, to improve my understanding of the building blocks of Machine Learning, I decided to try and implement the core algorithms in shaders using the GLSL language and Derivative’s TouchDesigner as a platform.
That was not a novel idea. Back then, I knew of at least one person on a similar quest: my friend Tim Gerritsen, a shader enthusiast and avid TouchDesigner programmer. He had told me about his willingness to try some Deep Learning experiments with shaders on an overly geeky afternoon in Berlin, where we coincided for the TouchDesigner summit in 2018.
Tim and I had been on and off independently researching things concerning Machine Learning and often met on virtual channels to exchange notes. After trying the first GLSL implementation on my own, I decided to contact him and tell him of my experiments, as I was curious to know his. I was glad to get an excited reply from him with some pointers.
We thus began very organically to collaborate, developing in parallel sibling platforms and adopting what we found to be the best features of each other for our own. It was only natural that we decided to simply combine our efforts in developing a single unified platform, which we have called: TDNeuron.
The tutorials
The content here started as personal notes, when I was developing my framework, more like a development diary than anything else. After working on TDNeuron, Tim and I decided that besides providing the TouchDesigner community with a native platform for Deep Learning, we should guide those who were embarking on the journey and wanted to see the inner workings of it all.
Therefore, I decided to re-write my notes as a series of tutorials. These can serve as insight to implement Deep Learning using shaders in the GLSL language, using Derivative’s TouchDesigner.
The principles outlined here are more or less the same we have applied to create our platform. You may use it as an accompaniment and as an “under the hood” introduction to TDNeuron. Naturally, some things here are simplified so that it is more didactic and comprehensible.
Be aware that this guide was not written by an expert. I am an autodidact toying around with concepts, so please feel free to correct/improve/clarify things that in this article may need polish! You need to know a fair amount of TouchDesigner to follow along, so you will need to get comfortable with it if you haven’t already. Nonetheless, the main bulk of the implementation is in GLSL, which should be easily portable to other platforms if you want to try that.
Without further ado let’s melt our brains a little bit, shall we?