Implemented a vanilla RNN from scratch using numpy to bettter understand the vanishing gradients problem.
Trained a vailla char-RNN having 128 hidden layers for 20 epochs (~2 hours on Google Colab GPU) with basic sampling:
RYORD:
No, this thou doue to hust by.
CIAINIUS:
You should in shechoronier,
Woat have the child o's waited rites to go.
BRUTUS:
It your eyfy And sends the grour.
MENENIUS:
There a veor that
To be me I compend lodest Rome of to boutar that a suspose as sones war show sprits of us, the like place open,
To have whom him.
MENENIUS:
Of the wing late.
Sfrother with yonatian.
'Til, who worl you do whan and you. Tentend. But cumest fill hole so us a propity.
I come him them's not there.
Brarcity. are sich
Though he deld, the will.
COPINIUS:
Your vengear it pitionce toble seated standfectoor'd
Wife thinks had to such our to.
One you much in
The what abolish and my lifestn
Tuntlemed,
Hath for sust helply such of thus I gade that flon encus:
Be not firn to hum,
Your worther's raultwer
When's worthy out
what you do't them gaver, ald
Had that etans
It them, if this put dustarude.