Echo State Networks are easy-to-train recurrent neural networks, a variant of Reservoir Computing. In some sense, these networks show how far you can get with nothing but a good weight initialisation.
This ESN implementation is relatively simple and self-contained, though it offers tricks like noise injection and teacher forcing (feedback connections), plus a zoo of dubious little hyperparameters.
However! If your aims are practical and your gradients automatic, consider using a fully trained network.