Re: Neural Network basics - Artificial Intelligence using AutoHotkey!
Posted: 04 Jan 2019, 20:08
Wow, Gio, thanks for all that information. What I'm realising is that this is such a complex subject that I'll need to do more learning - I haven't even really studied this thread yet.
I understand the principle of adjusting the weights, and also the use of the gradient (although my calculus is a bit rusty these days); my problem is following the complex formula by which all the data gets weighted and combined and crunched, so it shows when I actually try to construct code to do something new! I didn't do matrix stuff in school, so I'm trying to catch up, but I've been a hobby programmer for 30 years so I'm used to complex data structures. I think it's just about getting familiar enough until it clicks and you kind of see it. Maybe.
I did a little experiment in the last couple of days translating your code to deal with a more complex problem, but it's highlighting how little I know. It's not giving any meaningful results, either because the problem needs a different approach, or because I've stripped the second dimension out and this is when I need to use it. Please don't feel obliged to keep schooling me through this, though, I'm just sharing the journey. I've also switched language, because I find AHK a struggle and I can make quicker progress in BASIC.
The idea was to find a simple mathematical function that I could plug in instead of the three bits and an output bit. I used the highest common factor (HCF) of two integers, which is pretty easy to calculate, and then, if successful, I'd hope the net to "infer" a correct answer to a new pair of integers. Obviously this is different because it accesses the HCF result as an integer >=1, not just a binary as in the original. Anyway, my final results are all approaching 0.99999999, which would be nice if that represented anything.
I'm thinking now about other ways to approach the question, for instance, having just two input "neurons" representing the integers, and normalizing these with A / Range so they're both floats between 0 and 1 - this is a little more like the inputs in the handwritten integers example 3blue1brown uses here, where the brightness of pixels are normalized first. That would obviously be the way to go if the whole sherbang requires 0-1 data. But it might also require Range number of output neurons, as that uses 10 for the different numerals.
As I say, don't go to any trouble over this, but if you want to chip in with suggestions, that's very kind of you. I like messing about with stuff I don't understand until I either understand it or get bored and give up! I'll go back to reading for a while. Cheers.
I understand the principle of adjusting the weights, and also the use of the gradient (although my calculus is a bit rusty these days); my problem is following the complex formula by which all the data gets weighted and combined and crunched, so it shows when I actually try to construct code to do something new! I didn't do matrix stuff in school, so I'm trying to catch up, but I've been a hobby programmer for 30 years so I'm used to complex data structures. I think it's just about getting familiar enough until it clicks and you kind of see it. Maybe.
I did a little experiment in the last couple of days translating your code to deal with a more complex problem, but it's highlighting how little I know. It's not giving any meaningful results, either because the problem needs a different approach, or because I've stripped the second dimension out and this is when I need to use it. Please don't feel obliged to keep schooling me through this, though, I'm just sharing the journey. I've also switched language, because I find AHK a struggle and I can make quicker progress in BASIC.
The idea was to find a simple mathematical function that I could plug in instead of the three bits and an output bit. I used the highest common factor (HCF) of two integers, which is pretty easy to calculate, and then, if successful, I'd hope the net to "infer" a correct answer to a new pair of integers. Obviously this is different because it accesses the HCF result as an integer >=1, not just a binary as in the original. Anyway, my final results are all approaching 0.99999999, which would be nice if that represented anything.
I'm thinking now about other ways to approach the question, for instance, having just two input "neurons" representing the integers, and normalizing these with A / Range so they're both floats between 0 and 1 - this is a little more like the inputs in the handwritten integers example 3blue1brown uses here, where the brightness of pixels are normalized first. That would obviously be the way to go if the whole sherbang requires 0-1 data. But it might also require Range number of output neurons, as that uses 10 for the different numerals.
As I say, don't go to any trouble over this, but if you want to chip in with suggestions, that's very kind of you. I like messing about with stuff I don't understand until I either understand it or get bored and give up! I'll go back to reading for a while. Cheers.