Radio2Speech: High Quality Speech Recovery from Radio Frequency Signals

Considering the microphone is easily affected by noise and soundproof materials, the radio frequency (RF) signal is a promising candidate to recover audio as it is immune to noise and can traverse many soundproof objects. In this paper, we introduce Radio2Speech, a system that uses RF signals to recover speech with high quality from the loudspeaker. Radio2Speech can recover speech comparable to the quality of a microphone, advancing from recovering only single tone music or incomprehensible speech in existing approaches. We use Radio UNet to accurately recover speech in time-frequency domain from the RF signal with limited frequency band. Also, we incorporate the neural vocoder to synthesize the speech waveform from the estimated time-frequency representation without using the contaminated phase. Quantitative and qualitative evaluations show that in quiet, noisy and soundproof scenarios, Radio2Speech achieves state-of-the-art performance and is on par with a microphone that works in quiet scenarios.

Millimeter wave (mmWave) radar transmits RF signals to the vibrating diaphragm of loudspeaker, while the reflected signals contain vibration characteristics associated with speech. Radio2Speech parses reflected RF signals to recover speech. In quiet scenarios, Radio2Speech can recover high quality speech like a microphone. Moreover, since RF signals are not affected by the noise, our system is still able to recover high quality speech in noisy scenarios, while microphone almost fails. The existence of soundproof materials can make the microphone deaf, but RF signals can traverse occlusions. Thus, the speech recovered by Radio2Speech remains intelligible in soundproof scenarios. As shown in the pipeline of Radio2Speech, the input RF signal is upsampled using the cubic spline interpolation, and then it is transformed into Mel spectrogram. The Radio UNet is proposed to recover the Mel spectrogram of the speech signal. Finally, the neural vocoder, Parallel WaveGAN, is employed to reconstruct the natural speech waveform from the estimated Mel spectrogram.