Project Overview
NeuralMIDI-Streamer is a real-time generative music system that bridges the gap between Deep Learning and live audio synthesis. Unlike static AI generation tools, this system functions as a responsive virtual instrument, using LSTM (Long Short-Term Memory) neural networks to generate polyphonic MIDI streams on the fly.
Trained on datasets of classical composers (e.g., Bach, Beethoven), the model predicts musical events in real-time, which are transmitted via OSC (Open Sound Control) to a dedicated audio engine built in PureData. The system features a custom state-machine to handle complex voice allocation and allows the performer to modulate generation parameters—such as „Temperature“ (creativity) and „Density“ (sparsity)—live during performance.
Technical Features
- LSTM Architecture: Implementation of a recurrent neural network capable of learning and predicting polyphonic chord structures and time-based dependencies.
- Hybrid Pipeline: A robust communication bridge between Python (TensorFlow inference) and PureData (hosting VST plugins via
vstplugin~). - Live Modulation: Real-time control over stochastic parameters (entropy/density) and „Hot-Swapping“ capabilities to switch composer models (e.g., morphing from Bach to Beethoven) without audio interruption.
- Smart Voice Allocation: A custom dispatcher logic ensures note-off pairing and velocity smoothing to prevent „stuck notes“ and ensure organic phrasing.
Tech Stack
- AI/ML: Python, TensorFlow, NumPy
- Audio Engine: PureData, VST Plugins (Spitfire LABS)
- Protocols: Open Sound Control (OSC)