How to make a rhythm game in Unity

1. Sync to the audio clock, not to frame time

The most common reason a homemade rhythm game drifts out of sync is moving notes with Time.deltaTime. Frame time and audio playback run on different clocks, and small errors add up over a three-minute song. Use the audio system's own clock, AudioSettings.dspTime, and derive the song position from it every frame:

double dspStart;
public AudioSource music;
public float offset; // calibration, in seconds

void StartSong() {
    dspStart = AudioSettings.dspTime + 0.1; // schedule slightly ahead
    music.PlayScheduled(dspStart);
}

void Update() {
    float songPosition =
        (float)(AudioSettings.dspTime - dspStart - offset) * music.pitch;
    // place every note from songPosition, never by accumulating deltaTime
}

Multiplying by music.pitch keeps notes aligned if you add a speed modifier, and the offset value is what you expose to players as audio calibration.

2. Store the chart as timed note data

A chart (also called a beatmap or sequence) is a list of notes, each with a time, a lane and a length. Store it in a ScriptableObject so designers can edit it in the Inspector:

[System.Serializable]
public class ChartNote {
    public float time;    // seconds from song start
    public int lane;      // which track
    public float length;  // 0 for a tap, > 0 for a hold
}

[CreateAssetMenu]
public class Chart : ScriptableObject {
    public AudioClip clip;
    public int bpm;
    public List<ChartNote> notes = new();
}

Storing times in seconds keeps playback simple. Also storing the BPM lets you convert to beats when you build an editor that snaps to a grid.

3. Position notes from the song time

Each note's position is a pure function of the song time, so a hitch in frame rate never shifts it:

float distance = (note.time - songPosition) * scrollSpeed;
noteTransform.localPosition = hitLine + Vector3.up * distance;

Spawn each note a fixed number of seconds before its time and despawn it after it passes the line. Because a song can have thousands of notes, recycle note objects with a pool instead of calling Instantiate and Destroy during play, which causes garbage-collection stutters at exactly the wrong moment.

4. Judge hits with timing windows

When the player presses a lane, find the nearest unhit note in that lane and measure how far off they were. Grade the hit by comparing that difference to a set of windows:

float diff = Mathf.Abs(note.time - songPosition);
if (diff <= 0.05f) Judge("Perfect");
else if (diff <= 0.10f) Judge("Great");
else if (diff <= 0.15f) Judge("Good");
// outside every window: ignore the press, or count a miss

Notes that pass the line without being hit count as misses and break the combo. Hold notes need a second check: the player must keep pressing until time + length.

5. Handle touch and keyboard input

Map keys (for example D F J K) to lanes on desktop, and on mobile map touch positions to lanes, tracking each finger separately so players can hold one note and tap another. If you use Unity's Input System package, read input in the same frame you compute songPosition so the judgement uses consistent time.

6. Score, combo and feedback

Keep score, current combo, max combo and miss count in one stats component that raises events, and let UI and effects subscribe to them. Hit effects (a flash, a particle burst, a sound) are what make a rhythm game feel responsive, so pool them as well.

Making charts: the part that takes the longest

Once the engine works, most of the time goes into charting songs. Typing note times into an Inspector does not scale. You need either an in-editor sequence editor, a way to record notes by tapping along to the song, or an import path from a tool you already use, such as a DAW exporting MIDI. See turning MIDI files into rhythm game charts for that workflow.

The shortcut: start from a working template

Everything above is already built in Rhythm Game Starter, a Unity rhythm game template with full C# source:

• SongManager drives note positions from AudioSettings.dspTime, with pause, delay and speed-modifier support.
• Charts live in SongItem assets, created by importing a MIDI file, drawn in the Sequence Editor (Tools → Rhythm Game → Sequence Editor), or recorded by tapping along with the Note Recorder.
• Pooled tap, long-press and swipe notes, plus pooled hit effects.
• StatsSystem with configurable hit levels (Perfect, Good and so on), combo, score and miss events.
• Touch, keyboard and Input System handlers, and demo scenes for vertical 4-track and horizontal 6-track layouts.

You supply the music, and the template covers the engine, so you can spend your time on charts and game feel.