DDSP-Based Neural Vehicle Sound Synthesis from Driving Signals

Minsuk Choi1, Dabin Kim1, Daehun Song2, Juhan Nam1

1 Graduate School of Culture Technology, KAIST, South Korea

2 Hyundai Motor Group, South Korea

Supporting Webpage for the 6th AES International Conference on Automotive Audio, 2026.

Framework overview

Abstract

This paper presents a DDSP-based neural vehicle sound synthesis framework conditioned on driving signals collected from the CAN bus of an internal combustion engine (ICE) vehicle, and demonstrates the feasibility of realistic and coherent vehicle sound synthesis within this framework. We investigate three design choices for synthesis configuration: the definition of the fundamental frequency (F0), the configuration of driving signal inputs, and the conditioning representation. Specifically, we compare crank-based and firing-based F0 definitions, multiple driving signal combinations constructed from engine RPM, gear level, accelerator pedal position, vehicle speed, and longitudinal acceleration, and two conditioning representations: direct and encoded conditioning. The framework is evaluated using objective and subjective measures together with qualitative spectrogram observation. The results show that the crank-based F0 provides more accurate synthesis than the firing-based F0 in the present four-cylinder four-stroke vehicle setting. Richer driving signal configurations generally improve synthesis quality, while the contribution of individual signals depends on their redundancy and complementarity. Encoded conditioning yields better objective performance, especially when the available driving signals are limited, whereas direct conditioning achieves the best perceptual results under full driving signal configuration and offers practical advantages in simplicity and efficiency. These findings provide practical guidelines for DDSP-based neural vehicle sound synthesis and highlight the potential of driving-signal-conditioned neural audio synthesis for automotive audio applications such as vehicle sound design and driving simulation.

Demo A: F0 Definition

Compares synthesized vehicle sounds using crank-based and firing-based F0 definitions under direct and encoded conditioning.

Demo B: Driving Signal Configuration

Compares synthesized vehicle sounds across driving signal configurations: (RPM), (RPM, Gear Level, Pedal Position) and (RPM, Gear Level, Pedal Position, Speed, Acceleration), under direct and encoded conditioning.

Demo A: F0 Definition

Compares synthesized vehicle sounds using crank-based and firing-based F0 definitions under direct and encoded conditioning.

Candidate Description
Ground Truth The reference audio.
Direct
Firing-based		 Synthesized audio using direct conditioning with the firing-based F0 definition.
Direct
Crank-based		 Synthesized audio using direct conditioning with the crank-based F0 definition.
Encoded
Firing-based		 Synthesized audio using encoded conditioning with the firing-based F0 definition.
Encoded
Crank-based		 Synthesized audio using encoded conditioning with the crank-based F0 definition.

Sample 01

Reference
Spectrogram for F0 Definition sample 024 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 024 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 024 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 024 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 024 - Encoded Crank-based

Encoded
Crank-based

Sample 02

Reference
Spectrogram for F0 Definition sample 031 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 031 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 031 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 031 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 031 - Encoded Crank-based

Encoded
Crank-based

Sample 03

Reference
Spectrogram for F0 Definition sample 052 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 052 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 052 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 052 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 052 - Encoded Crank-based

Encoded
Crank-based

Sample 04

Reference
Spectrogram for F0 Definition sample 080 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 080 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 080 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 080 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 080 - Encoded Crank-based

Encoded
Crank-based

Sample 05

Reference
Spectrogram for F0 Definition sample 095 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 095 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 095 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 095 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 095 - Encoded Crank-based

Encoded
Crank-based

Sample 06

Reference
Spectrogram for F0 Definition sample 099 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 099 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 099 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 099 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 099 - Encoded Crank-based

Encoded
Crank-based

Sample 07

Reference
Spectrogram for F0 Definition sample 104 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 104 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 104 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 104 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 104 - Encoded Crank-based

Encoded
Crank-based

Sample 08

Reference
Spectrogram for F0 Definition sample 112 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 112 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 112 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 112 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 112 - Encoded Crank-based

Encoded
Crank-based

Sample 09

Reference
Spectrogram for F0 Definition sample 172 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 172 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 172 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 172 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 172 - Encoded Crank-based

Encoded
Crank-based

Sample 10

Reference
Spectrogram for F0 Definition sample 179 - Ground Truth

Ground Truth

Direct
Spectrogram for F0 Definition sample 179 - Direct Firing-based

Direct
Firing-based

Spectrogram for F0 Definition sample 179 - Direct Crank-based

Direct
Crank-based

Encoded
Spectrogram for F0 Definition sample 179 - Encoded Firing-based

Encoded
Firing-based

Spectrogram for F0 Definition sample 179 - Encoded Crank-based

Encoded
Crank-based

Demo B: Driving Signal Configuration

Compares synthesized vehicle sounds across driving signal configurations: (RPM), (RPM, Gear Level, Pedal Position) and (RPM, Gear Level, Pedal Position, Speed, Acceleration), under direct and encoded conditioning.

Candidate Description
Ground Truth The reference audio.
Direct
(RPM)		 Synthesized audio using direct conditioning with (RPM).
Direct
(RPM, Gear Level, Pedal Position)		 Synthesized audio using direct conditioning with (RPM, Gear Level, Pedal Position).
Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)		 Synthesized audio using direct conditioning with (RPM, Gear Level, Pedal Position, Speed, Acceleration).
Encoded
(RPM)		 Synthesized audio using encoded conditioning with (RPM).
Encoded
(RPM, Gear Level, Pedal Position)		 Synthesized audio using encoded conditioning with (RPM, Gear Level, Pedal Position).
Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)		 Synthesized audio using encoded conditioning with (RPM, Gear Level, Pedal Position, Speed, Acceleration).

Sample 01

Reference
Spectrogram for Driving Signal Configuration sample 024 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 024 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 024 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 024 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 024 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 024 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 024 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 02

Reference
Spectrogram for Driving Signal Configuration sample 031 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 031 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 031 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 031 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 031 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 031 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 031 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 03

Reference
Spectrogram for Driving Signal Configuration sample 052 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 052 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 052 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 052 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 052 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 052 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 052 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 04

Reference
Spectrogram for Driving Signal Configuration sample 080 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 080 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 080 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 080 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 080 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 080 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 080 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 05

Reference
Spectrogram for Driving Signal Configuration sample 095 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 095 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 095 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 095 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 095 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 095 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 095 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 06

Reference
Spectrogram for Driving Signal Configuration sample 099 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 099 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 099 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 099 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 099 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 099 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 099 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 07

Reference
Spectrogram for Driving Signal Configuration sample 104 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 104 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 104 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 104 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 104 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 104 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 104 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 08

Reference
Spectrogram for Driving Signal Configuration sample 112 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 112 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 112 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 112 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 112 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 112 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 112 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 09

Reference
Spectrogram for Driving Signal Configuration sample 172 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 172 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 172 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 172 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 172 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 172 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 172 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Sample 10

Reference
Spectrogram for Driving Signal Configuration sample 179 - Ground Truth

Ground Truth

Direct
Spectrogram for Driving Signal Configuration sample 179 - Direct (RPM)

Direct
(RPM)

Spectrogram for Driving Signal Configuration sample 179 - Direct (RPM, Gear Level, Pedal Position)

Direct
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 179 - Direct (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Direct
(RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
Spectrogram for Driving Signal Configuration sample 179 - Encoded (RPM)

Encoded
(RPM)

Spectrogram for Driving Signal Configuration sample 179 - Encoded (RPM, Gear Level, Pedal Position)

Encoded
(RPM, Gear Level, Pedal Position)

Spectrogram for Driving Signal Configuration sample 179 - Encoded (RPM, Gear Level, Pedal Position, Speed, Acceleration)

Encoded
(RPM, Gear Level, Pedal Position, Speed, Acceleration)